Back home

agent-desktop

x 0.80
Visit source

Intro

agent-desktop is an open-source command-line tool from Vercel Labs, publicly launched on October 31, 2026 (Beijing time), that gives any AI agent reliable computer use on the desktop. It is the sibling of Vercel's agent-browser: where agent-browser drives web pages, agent-desktop drives native applications on macOS, Windows and Linux.

Instead of guessing from screenshots, it reads each app's real UI structure through the operating system's accessibility tree, so element references stay stable and actions are safe to retry. It is written in Rust and installed with npm i -g agent-desktop.

Explain

What makes it different

  • Accessibility tree, not pixels: agents see a structured list of buttons, fields and menus with stable refs, which is cheaper in tokens and far more reliable than screenshot-based clicking.
  • Drill-down: an agent can inspect just part of a window to save tokens.
  • Parallel & background: it runs headless by default, so agents can work in the background while you keep using the machine, and many agents can operate one app at once.
  • Cross-platform: macOS (needs Accessibility permission — agent-desktop permissions --guide), Windows, and Linux (needs at-spi2-core).
  • Cloud option: you can start a Linux desktop in a Vercel Sandbox, run agent-desktop inside it and get JSON back.
  • E2E testing: Vercel pitches it for end-to-end testing of desktop apps with agents as well as general automation.

Context

The GitHub repo vercel-labs/agent-desktop existed earlier in 2026 (about 1.8k stars at launch), but the public "Introducing agent-desktop" launch by Chris Tate and Guillermo Rauch happened on Oct 30–31, 2026. Paired with agent-browser, it gives an agent computer use across both the web and native desktop apps.

Usage

Getting started

  1. Install: npm i -g agent-desktop.
  2. Grant permissions: on macOS run agent-desktop permissions --guide and allow Accessibility for your terminal; on Linux install at-spi2-core.
  3. Let your coding agent (Claude Code, Codex, Cursor, etc.) call the CLI: list windows, read an app's accessibility tree, then click / type using the returned refs.
  4. Combine with agent-browser when a task spans both web pages and native apps.
  5. For isolation, run it inside a Vercel Sandbox Linux desktop instead of your own machine.

Repo and docs: github.com/vercel-labs/agent-desktop · agent-browser.dev/desktop.

Tip: avoid logging sensitive accounts into a desktop that an agent controls.