HN Showcase
Back to browse
GitHub Repository

If it's on the screen, it's an API. Control any desktop app via REST + MCP. Rust.

113 starsRust

OculOS – Any desktop app as a JSON API via OS accessibility tree

by stif1337·Mar 3, 2026·5 points·0 comments
Visit ProjectView on HN

AI Analysis

●●●BangerZero to OneWizardryBig Brain

Accessibility-tree-to-API bridge lets Claude autonomously control any desktop app without screenshots or pixel coords.

Strengths
  • •Wire-protocol approach (accessibility tree) means zero code injection or browser-specific hacks required.
  • •MCP integration enables AI agents to control real applications out-of-the-box; Spotify demo proves it works.
  • •Cross-platform single binary with built-in dashboard, REST API, and WebSocket events — genuinely comprehensive.
Weaknesses
  • •Relies on OS accessibility APIs; fragile for apps that don't expose proper semantic trees (custom frameworks).
  • •No clear rate limiting, authentication, or isolation for multi-user or untrusted code execution scenarios.
Target Audience

AI agents (Claude, GPT, Gemini), automation engineers, RPA developers

Similar To

Playwright · Selenium · UiPath RPA

Post Description

Single Rust binary (~3 MB) that reads the OS accessibility tree and gives every UI element a REST endpoint. Click buttons, type text, toggle checkboxes — all via JSON.

Works as an MCP server too, so Claude/Cursor/Windsurf can control any desktop app out of the box.

Windows + Linux + macOS. MIT licensed.

Similar Projects