Cursor Computer Use: the agent steps out of the browser and drives any desktop app, including the iOS Simulator

Cursor Computer Use: the agent steps out of the browser and drives any desktop app, including the iOS Simulator

🔎 The coding agent gains eyes and hands

Coding agents had an acknowledged blind spot: they write code they never see running. Unit tests? Yes. CI? Yes. But clicking the button they just generated, navigating through the flow they just implemented? Never. On October 23, 2026, Cursor closed that gap.

Its new feature, simply called Computer Use, allows its agents to click and type in any desktop application. The showcase: the iOS Simulator. The agent develops an iOS feature, launches the Simulator, navigates the flow, inspects the rendering, fixes the code. One single task, zero human intervention.

Why now? Because computer-use is becoming the next battleground. OpenAI is pushing Codex Computer Use on Windows, browser-use is extending the concept all the way to video, and the open source world is publishing models that rival closed solutions. Cursor is striking back where it is strongest: integration into the developer workflow.

The Essentials

• Cursor launches Computer Use on October 23, 2026: its coding agents click and type in desktop applications, including the iOS Simulator, to test what they've just built (Runtimewire, AICoder).
• On macOS, the feature relies on a dedicated app called "Cursor Computer Use" (bundle co.anysphere.cursor-computer-use) and requires Accessibility and Screen Capture permissions (Cursor Docs).
• Triggered via @-mention in the agent prompt, or selected automatically when the task requires it; on the CLI side: agent worker --computer-use start.
• What we don't know yet: supported platforms, the full permissions model, benchmarks — nothing has been published (AICoder).
• The context: on OSWorld-Verified, the open source OpenCUA-72B reaches 45.0%, ahead of Claude 4 Sonnet (41.5%) and OpenAI CUA (31.4%) (OpenCUA, 2025).

Recommended tools

What Cursor Actually Announced — and What It's Still Not Saying

On October 23, 2026, Cursor officially rolled out Computer Use: its coding agents can now operate desktop applications — clicking, typing, navigating — well beyond the browser that had been their only visual playground.

The announcement came via two posts on X on October 23 (explainx), with a blessing from the official @cursor_ai account (AICoder). Runtimewire describes the scope without ambiguity: the agent can click and type in desktop applications to test what it has just built.

The feature didn't come out of nowhere. A community changelog from September already mentioned Computer Use on Mac in version 3.20.21 (AICoder). Cursor was letting it mature in silence before making it official — a classic strategy for AnySphere.

But let's keep a cool head about what's missing. Supported platforms, full permissions model, benchmarks: none of that was published at launch. The best practice is to monitor the official changelog rather than X threads, which ignite faster than they inform.

A telling detail flagged by AGTP: on macOS, everything runs through a small helper application called Cursor Computer Use, driven by the CLI. This architectural choice says a lot about the ambition — and about the guardrails. More on that below.

This offensive fits into a broader strategy: at a time when the official spec for Cursor's plugins arrives on GitHub, the company is clearly building a platform, not just a text editor. Enough to consolidate its first place in our comparison of the best AI coding tools.

The flagship use case: the iOS Simulator in a closed loop

The scenario that justifies the announcement can be summed up in one sentence: the agent develops an iOS feature, then verifies it itself in the Simulator — without touching your keyboard.

explainx describes the complete loop: build, launch the Simulator, type, read the screen, fix, repeat. An end-to-end iteration, in a single agent task.

Concretely: you ask for a settings screen with a notifications toggle. The agent writes the SwiftUI, compiles, launches the Simulator, navigates to the screen, notices the toggle is spilling out of its frame, reopens the code, recompiles. Before, you were the tester. Now, you're the reviewer.

Why this is a big deal: coding agents had tests, linters, pipelines — but no eyes. UI regressions, the ones users see first, stayed outside the loop. Computer Use brings them inside.

As for triggering, the agent is invoked via @-mention in the prompt — the famous @Computer Use — or selected automatically when the task calls for it (Runtimewire). The mention becomes a verb in the developer's vocabulary, like @Files two years ago.

My take: this autonomous verification loop is exactly the dividing line between an autocompleter and an agent. It's also the criterion that structures our roundup of the best autonomous AI agents — and Cursor just scored a serious point.

Under the hood: a dedicated app, two permissions, one CLI

On macOS, Computer Use doesn't hack anything: it relies on a dedicated application and on Apple's standard permission mechanisms. It's clean, and that's a good sign.

The application is called "Cursor Computer Use", bundle co.anysphere.cursor-computer-use (Cursor Docs). It is this app — and neither Cursor nor Terminal — that must be granted the Accessibility and Screen Capture permissions.

This separation isn't cosmetic. It's least privilege: the only surface that sees your screen is isolated, identifiable, and revocable independently of the editor itself. More on this in the privacy section.

Triggering goes through the Cursor CLI:

agent worker --computer-use start

The --share-desktop option enables session sharing — in other words, watching what the agent does on screen instead of taking its word for it.

Non-negotiable constraint: the worker must be signed and run within a macOS graphical (GUI) session. A headless server is out of the picture, for obvious reasons — no screen, no computer use.

The philosophy is explicitly opt-in: these capabilities are only activated on explicit action, never by the server. For a step-by-step installation guide on the machine side, Flavio Copes has published a guide dedicated to the feature.

Activate and test: the express checklist

To test it at home, you need three ingredients: an up-to-date Cursor, a macOS graphical session, and two permissions granted to the right application. Nothing more.

1. Update Cursor and check the version number in the official changelog — the feature was already present in 3.20.21 on Mac back in September.
2. Launch the worker via agent worker --computer-use start from the Cursor CLI.
3. In Settings → Privacy & Security, grant the Accessibility and Screen Recording permissions to "Cursor Computer Use" — not to Terminal, not to Cursor (Cursor Docs).
4. In your agent task, @-mention Computer Use, or phrase your request so that the agent selects it on its own when the task calls for it (Runtimewire).
5. Keep an eye on things via --share-desktop: you see exactly what the agent sees and does, in real time.

Watch out for pitfall number one: an SSH session or an environment without a graphical interface will not work. The macOS GUI session is a requirement, not an option.

Ten minutes of setup to eliminate hours of manual testing: the ratio is unbeatable, provided you accept the permissions pact that comes with it. That's the real issue.

OpenAI, browser-use, OpenCUA: the computer-use race is accelerating

Cursor isn't inventing the computer-use category: it's industrializing it for developers, on ground already contested by three camps with very different strategies.

OpenAI is fielding its CUA on one side — publicly benchmarked — and pushing Codex Computer Use on Windows, capable of controlling a PC remotely (our analysis). Cursor picks macOS, OpenAI bets on Windows: the coincidence is no coincidence, it's platform tactics.

browser-use, which has become the open-source standard for browser control, is pushing the logic further: with video-use, a code agent can now edit videos (our breakdown). The frontier of computer use clearly no longer stops at clicking inside a web page.

On the open-source side, OpenCUA — a NeurIPS 2025 spotlight — is releasing a complete stack: data, pipeline, and models from 7B to 72B. Our analysis of OpenGCUA, the open stack of the computer-use agent, shows it cuts the number of steps by 53% on OSWorld-v2 — fewer steps, fewer screenshots, fewer tokens.

My take: the real battle is no longer the raw model, but integration. Whoever controls the developer's workflow controls the entry point of computer-use. Cursor starts with the editor already installed on millions of machines — an advantage no benchmark can measure.

What the numbers say: OSWorld measures the road ahead

On OSWorld-Verified (100 steps), the best public score is 45.0%: no one yet succeeds at the majority of tasks end to end. Computer use remains difficult terrain, despite the spectacular demos.

Source: OpenCUA, benchmark published on the occasion of the NeurIPS 2025 spotlight.

Two readings of this table. First, open source leads: OpenCUA-72B moves ahead of Claude 4 Sonnet and clearly ahead of OpenAI CUA. Second, humility: even the leader fails on more than one task in two. Launch demos always flatten reality.

Where does Cursor stand? Nowhere, for now: no published benchmark for Computer Use at launch. Impossible to place it honestly on this grid. Judge on your own tasks, not on the GIFs.

And remember that the number of steps is a cost metric: each step represents a screenshot, an inference, and an action. Work on step reduction — that famous −53% on OSWorld-v2 — therefore translates into direct savings on the token bill.

Privacy: when the agent reads your entire screen

The price of magic is visibility: to click and type accurately, the agent must see — potentially everything displayed at any given moment on your machine.

Screenshot plus Accessibility access means a complete read of the interface: windows, messages, notifications, and those two-factor authentication codes that always pop up at the worst possible moment. explainx explicitly lists the risks right next to the confirmed capabilities.

The safeguards on Cursor's side are real but minimal: explicit opt-in, never server-triggered activation, permissions isolated to the dedicated app, and --share-desktop for auditing. Better than nothing, but not a complete answer.

The minimal hygiene I recommend: a dedicated user session for testing, revoking permissions when you're not using the feature, and no agent loops during your sensitive operations — payments, system administration, private messaging.

If the cloud makes you uneasy, there is an alternative: open source AI agents running locally with Ollama keep your screens on your machine. Less polished, more austere — but your pixels never leave.

My take: this permissions/privacy friction will be the real blocker to enterprise adoption, far more than the technology itself. An agent that reads your screen is THE question every CIO will ask — and "it's opt-in" may not be a sufficient answer.

What This Changes for Your Workflow (and What Remains to Be Proven)

For iOS and macOS devs, the gain is immediate: the code-compile-test loop closes without human intervention. For other platforms, caution is still warranted.

The concrete gains: automated UI smoke tests, accelerated design iterations, detection of visual regressions before human review. Manual UI testing—that hole in a developer's day—finally has a credible replacement.

The limitations, acknowledged: unannounced platforms, missing benchmarks, and a permission model detailed only for macOS. Treat Computer Use as a very promising beta, not as a guaranteed revolution.

The cost, too: every iteration goes through screenshots analyzed by a vision model. Agent loops consume noticeably more than chat, and quality is capped by the underlying model. The current agentic benchmarks—GPT-5.5 (98.2), Claude Opus 4.7 (94.3) on our June 2025 grid—set that ceiling, and our guide to LLMs for agents details how to choose based on your tasks.

My prediction: in six months, "the agent tested in the Simulator" will be a checkbox ticked by default on iOS tasks. The question will no longer be "is it possible?" but "what permissions do we grant it?". Start preparing your answers now.

❌ Common Mistakes

Mistake 1: Granting permissions to the wrong application

Many people will go and check Accessibility and Screen Capture for Cursor or for Terminal. The documentation is explicit: it's the dedicated "Cursor Computer Use" app that requires them (Cursor Docs). Without that, nothing works — and you needlessly widen the access surface of your tools.

Mistake 2: Trying to run the agent on a headless server

The worker must be signed and run within a macOS GUI session. A screenless Linux VPS is not a driver's seat for Computer Use. Keep the server for hosting what the agent tests — not for driving it.

Mistake 3: Believing the agent is watching your screen at all times

No: the feature is opt-in and is never activated by the server. If you don't trigger it, nothing happens. And --share-desktop lets you verify for yourself what is being seen and done.

Mistake 4: Extrapolating the iOS demo to the entire ecosystem

The iOS Simulator is the showcase, not the guarantee. Supported platforms and benchmarks are not published. Test on your own stack before rewriting your QA processes around the feature.

❓ Frequently Asked Questions

Is Cursor Computer Use available on Windows?

Not as of October 24, 2026. The documentation describes a macOS flow — Accessibility and Screen Capture permissions, signed GUI session — and no official platform list has been published (AICoder, October 24, 2026). OpenAI, for its part, chose Windows for its Codex Computer Use: the trajectories should converge, though no date has been announced on Cursor's side.

Can the agent read my messages or my passwords?

Technically, screen capture and Accessibility access offer a view of everything displayed, 2FA codes included. Cursor's answer is opt-in: no server-side activation, permissions carried by the dedicated app and revocable at any time. On a session containing sensitive data, caution is nonetheless still in order.

How much does Cursor Computer Use cost?

No dedicated pricing has been published at launch. Usage falls under your existing Cursor plans: the Pro subscription is listed at $20/month (October 2026, check cursor.com). One caveat, though: screen-capture loops consume significantly more tokens than a standard chat or autocomplete session.

Which AI model powers Computer Use?

Cursor hasn't specified. Coding agents typically rely on models built for agentic work — GPT-5.3 Codex and Claude Sonnet 4.6 are among the standouts in our June 2025 ranking. On a vision-action loop, that model choice matters even more than for plain code completion.

Is there an open source alternative?

Yes. The OpenCUA stack ships models ranging from 7B to 72B, with an OpenCUA-72B scoring 45.0% on OSWorld-Verified, ahead of Claude 4 Sonnet (41.5%) and OpenAI CUA (31.4%). For fully local agents on your machine, our Ollama guide walks you through the setup. Open source has never been this close to the state of the art.

✅ Conclusion

By giving its agents eyes and hands, Cursor transforms manual interface testing into an automated routine — provided you accept the permissions pact that comes with it. Benchmarks and platforms will be judged in the changelog; the closed loop, for its part, is already a reality. To go further, our guide on creating an AI agent shows you how to build your own agent loops, from prompt to verification.