OpenGCUA: the open stack for computer-use agents that blasts through OSWorld-v2 with 53% fewer steps

OpenGCUA: the open stack for computer-use agents that slashes 53% of steps on OSWorld-v2

🔎 Computer use becomes an open problem — and the numbers sting

Computer use has become the front line of agentic AI — and it remains massively closed. Today's most effective systems drive computers behind proprietary harnesses: nobody on the outside can inspect when they take a screenshot, how they chain their actions, or why. On October 15, 2026, Alibaba's Tongyi-MAI team releases OpenGCUA (arXiv 2610.22200), presented as the first complete open stack for a "generalist computer-use agent". The paper hits the number one spot among trending papers on Hugging Face right out of the gate (AlphaXiv).

The GitHub repo leans into the provocation: "GPT-6 made computer use the frontier. We make it open." It's not an empty slogan. On OSWorld-v2, the open stack lifts the average partial score from 45.3 to 64.9 — a +19.6-point gain — and cuts the number of steps per task by 53.0%. The detail that makes the result solid: these gains hold across the eight frontier models tested, from GPT-6 Astra to DeepSeek-V4.1-Flash (GitHub).

What remains is to sort out what has actually been released, what these numbers measure — and what they don't — and what it changes for anyone building agents that take control of a computer. That's the purpose of this article.

The Essentials

• OpenGCUA (arXiv 2610.22200, October 15, 2026): first open release of a complete stack for a generalist computer-use agent. The model decides for itself what to observe, how to act, and how to organize its work.
• +19.6 points average partial score on OSWorld-v2 (45.3 → 64.9) and -53.0% steps per task vs batched GUI-only execution, measured across 8 frontier models.
• GPT-6 Astra drops from 102 to 34 steps per task; Claude Opus 5 gains 4.4 points (65.7 → 70.1) despite an already high baseline.
• Code execution — a Python program combining PyAutoGUI, Playwright, shell, and app APIs — added on top of equivalent GUI+CLI access brings +6.1 points and -14.8% steps.
• Available now: harness, prompts, evaluation trajectories for the 8 models, real macOS harness (preliminary), Cross-Device SDK (preliminary). The dedicated GCUA model itself, however, is not yet part of the release.

Recommended Tools

What exactly is a GCUA?

A GCUA (generalist computer-use agent) is not defined by its components, but by who makes the decisions. In a GCUA, it's the model itself that chooses how to observe, how to act, and how to organize its work, step after step (project page).

The distinction is tectonic. In most current frameworks, these choices are hard-coded into the harness: a screenshot after every action, a fixed action space, external orchestration in Python. The model executes, it doesn't decide. OpenGCUA transfers these three decisions to the model — hence the expression "harness-to-model decision transfer" that structures the entire paper.

And the word "generalist" matters as much as the rest: this is not an agent trained for a specific workflow, but a system that has to cope with any application, any OS, any task. This is exactly the bet the entire industry is making — Google launches Gemini Agent with the same universal-agent ambition, but closed.

Decision 1: what to observe

Screenshots only when the model needs them, instead of a systematic screenshot after every action. On the two models tested, this on-demand observation pushes the cache hit rate up by +10.5 and +14.4 points, with no drop in score. In other words: fewer redundant images in the context, more cache reuse, same level of performance.

Decision 2: how to act

A single Python program per step, combining the accessibility tree, application APIs, shell commands, and GUI actions — with loops, branches, and internal checks. The agent writes code that clicks, rather than clicking a thousand times hoping it eventually works.

Decision 3: how to organize

The model maintains a todo list and states its intention before each program. No more agent that improvises its plan action by action and gets lost after twenty clicks. The plan becomes explicit, verifiable, and above all correctable along the way.

OSWorld-v2 numbers: +19.6 points and -53% steps, across 8 frontier models

Yes, the gains hold across all eight frontier models evaluated — which is precisely what makes the result hard to dismiss with a wave of the hand. On the OSWorld-v2 offline set, the average partial score rises from 45.3 to 64.9, and the number of steps per task drops by 53.0% on average, compared to a batched GUI-only execution (arXiv).

Two important readings from this table. First, GPT-6 Astra — already very efficient in GUI mode — goes from 102 to 34 steps per task: even the best models benefit massively from a better harness. Second, the gap between the best and worst model shrinks by more than half. The stack levels upward, and benefits most the models that started with the worst tooling.

Another efficiency signal, less highlighted: output tokens drop by 27.2% on six of the eight models. Fewer steps, fewer generated tokens, higher score. This winning trio is rare in the agent literature, where you usually gain on one axis by degrading the other two.

Keep one nuance in mind: this is an average partial score on a benchmark, not a success rate in real-world conditions. We'll come back to this in the common pitfalls section.

Why GUI-only agents waste half of their steps

Because the harness forces them to loop screenshot → one action → screenshot, even when the task calls for a loop or a bulk operation. Clicking is the slowest API call an agent can afford. Reid Marlow's analysis of GUI-only agents makes this diagnosis — OpenGCUA's numbers confirm it across eight models.

The paper's ablations quantify precisely what code execution brings:

(Averages computed over 3 models for these ablations.)

The number to remember is the second row. Even with equal access — GUI and CLI available in both cases — simply letting the model write a program with loops, branches, and checks adds 6.1 points and removes 14.8% of the steps. This isn't a side effect, it's a paradigm shift in action.

The same dynamic is playing out elsewhere in the ecosystem: Browser Use releases Video Use and is betting on a code agent to edit videos rather than a sequence of recorded clicks. Code is the natural language of repetitive tasks — including when the target is a graphical interface.

Code doesn't kill the click, it frames it

An important clarification to avoid a misreading: OpenGCUA doesn't remove GUI actions. PyAutoGUI remains in the generated program, alongside Playwright and the shell. The paper's thesis is more nuanced: the model must be able to pick the right tool for each subtask — an API when one exists, a click when nothing else works — and compose it all into a single controlled program.

Harness-to-model decision transfer: the quiet innovation that matters

OpenGCUA's real contribution isn't a new model — it's a shift in responsibility. Observation and orchestration decisions, previously hardcoded in the harness, now move to the model's side (Hugging Face).

Why it matters: a harness that decides on the model's behalf is optimized for an average model. A model that decides for itself when to observe and how to organize adapts to its own strengths and to the task at hand. What's more, the published trajectories of the 8 models make it possible to verify this claim trajectory by trajectory, not just on average.

The flip side is reasoning cost. On four real macOS tasks, Qwen3.8-Max's selected trajectories consume 19.4× more reasoning tokens than those of GPT-6 Astra. In other words: giving the model more decision-making freedom comes at a high token cost for the models that are least efficient at reasoning. The repo also lists efficiency optimization as upcoming work — that's probably where the next iteration will play out.

What is actually open — and what's still missing

Almost everything, except the model. Concretely, the version released on October 15, 2026 contains:

• a ready-to-use OSWorld integration example;
• the complete evaluation trajectories of the 8 models (GPT-6 Astra, Claude Opus 5, Gemini 3.8 Flash, GLM-5.3-Flash, Qwen3.8-Max, Qwen3.8-Flash, Kimi K3, DeepSeek-V4.1-Flash);
• a real, preliminary macOS harness, in the form of a menu-bar app;
• a preliminary Cross-Device SDK (computers, browsers, phones);
• an interactive online lab on the project page.

Announced for later: the dedicated GCUA model, efficiency optimization, and a benchmark on real-world environments. Alibaba is also releasing MAI-Gym, the associated training environment (AICoder, October 17, 2026).

My take: the trajectories of the 8 frontier models may be the most underrated release of the bunch. This is high-quality imitation data, exactly the fuel needed to distill a future GCUA model — and rare analysis material for anyone studying how these agents actually make their decisions. Few closed teams publish this level of detail about their own systems.

How to test OpenGCUA today

Start with the GitHub repo and the interactive lab — reading trajectories costs nothing and tells you more than any demo video.

git clone https://github.com/Tongyi-MAI/OpenGCUA.git
cd OpenGCUA

The harness relies on models called via API — the 8 tested cover most of the landscape, and our guide to the best LLMs for AI agents helps you choose based on your budget and latency needs. The macOS harness and the Cross-Device SDK are explicitly preliminary: test in an isolated environment, not on your production machine. A VPS does the job perfectly well for that (Hostinger).

If you want to compare with a fully local stack, our guide on open source AI agents with Ollama covers the API-free alternative — along with the limitations that implies for computer use, where frontier models keep a clear lead. And if you're just getting started, learning to create an AI agent remains the best entry point before tackling a harness of this size.

What this changes for the agent ecosystem

OpenGCUA establishes a public baseline. From now on, any closed player claiming computer use performance will have to justify their harness choices, not just their model — since we now know that a better harness is worth up to 19.6 points, regardless of the model. That's new pressure on the entire sector.

The big players are making their moves in parallel: OpenAI Codex Computer Use arrives on Windows to control the PC remotely, while Google pushes its universal agent toward the full desktop. All closed, all without published trajectories. The comparison with OpenGCUA will be inevitable — and this time, it can happen on measurable ground.

On the open source side, the long-term trend is confirmed. ByteDance's DeerFlow is already pushing open agents on long tasks; OpenGCUA adds the "control a computer" building block with credible numbers and a multi-device SDK. To place all this in the broader landscape, our selection of the best autonomous AI agents remains the right entry point.

The thing to watch going forward: the release of the dedicated GCUA model. If Tongyi-MAI trains a model natively designed for this harness, on MAI-Gym data, we'll have — for the first time — a coherent end-to-end computer-use stack. And an open one.

❌ Common Mistakes

Mistake 1: reading "+19.6 points" as an absolute success rate

The metric is an average partial score on the OSWorld-v2 offline set, not a binary success rate in real-world conditions. A benchmark score of 64.9 does not predict how the agent will behave on your everyday tasks. Solution: read the metric definitions in the paper before citing the number.

Mistake 2: recycling figures from unsourced analysts

Part of the coverage — notably on dev.to — cites analyst projections with no verifiable source. The paper and the repo are more than enough: stick to the figures in the abstract, which can be verified line by line, and demand a source whenever a percentage circulates without a reference.

Mistake 3: believing the open harness is enough to match a frontier agent

The same harness has Qwen3.8-Max consuming 19.4× more reasoning tokens than GPT-6 Astra on the macOS tasks: model capability remains the decisive factor. The harness raises the floor, but it doesn't erase the gaps between models. And the dedicated GCUA model hasn't been released yet.

Mistake 4: testing the macOS harness on your main machine

The released version is explicitly preliminary, and an agent that drives a computer has access to that entire computer. Use a dedicated VM or VPS, and treat every trajectory as unaudited code — because that's exactly what it is.

❓ Frequently Asked Questions

Is OpenGCUA really open?

The code, the prompts, the evaluation trajectories of the 8 models, the preliminary macOS harness, and the Cross-Device SDK are published on GitHub and on the project page. The dedicated GCUA model is not yet — it's the main missing piece of the stack, slated for a later release.

Which models were tested?

Eight frontier models: GPT-6 Astra, Claude Opus 5, Gemini 3.8 Flash, GLM-5.3-Flash, Qwen3.8-Max, Qwen3.8-Flash, Kimi K3, and DeepSeek-V4.1-Flash. All of them gain points with the full stack, with the gap between the best and the worst cut by more than half.

How does it differ from the computer use agents of the big labs?

The frontier implementations remain closed: no access to the harness, the prompts, or the trajectories. OpenGCUA publishes everything, making it possible to reproduce, audit, and improve. The fact that Claude Opus 5 still gains 4.4 points with this harness shows that even the best closed systems leave points on the table.

Do you need powerful hardware to use it?

Not for the harness itself, which orchestrates models called via API. Local compute becomes relevant if you're aiming for fully self-hosting the models — our guide on Ollama details this path, along with its trade-offs on complex agentic tasks.

Does it work on Windows and mobile?

The harness that's actually published is macOS, in a preliminary version. The Cross-Device SDK targets computers, browsers, and phones, but full Windows coverage is still to come. In the meantime, closed solutions dominate this space — keep an eye on repo updates.

✅ Conclusion

OpenGCUA moves computer use from the era of closed demos to the era of reproducible science: +19.6 points and -53% steps across 8 frontier models, with the harness, prompts, and trajectories openly available. Clone the repo, read the trajectories before the tweets — and keep an eye on the upcoming GCUA model, because that's probably where the next chapter will play out.