AMD Helios + Cerebras : disaggregated inference defies Nvidia-Groq with 5x tokens/sec/watt on models >1T parameters

AMD Helios + Cerebras: disaggregated inference challenges Nvidia-Groq with 5x tokens/sec/watt on models >1T parameters

🔎 Why the GPU just got split in two

The AI industry has just crossed a point of no return. The monolithic GPU, the centerpiece that made Nvidia's fortune, is being dismantled into two specialized pieces.

AMD and Cerebras announced at AMD Advancing AI 2026 a disaggregated inference solution that separates prefill from decode. The result: 5x more tokens per second per watt than a Cerebras-only configuration, according to internal benchmarks published on August 12, 2026.

This is not just a paper partnership. OpenAI, Meta, Oracle, HPE, and the US Department of Energy are already listed as announced customers. Mass production is scheduled for Q2 2027, with availability on Cerebras Cloud as early as H2 2026.

The timing is not coincidental. It has been less than two months since Nvidia snapped up Groq for $20 billion, and since Groq pivoted to neocloud with its LPX platform. AMD and Cerebras are taking the exact opposite approach to this strategy: no acquisition, no exclusive proprietary cloud, but an open architecture based on market standards.

The Essentials

• Disaggregated architecture: prefill (compute-intensive processing of the prompt) is handled by the AMD Helios racks (72 MI455X, 31 TB HBM4, 2.9 exaFLOPS FP4), and decode (token-by-token generation) by the Cerebras WSE (wafer-scale engine) chips.
• Performance: 5x more tokens/sec/watt than a Cerebras-only system, targeting models with over 1 billion parameters like GPT-5.5, Claude Opus 4.7, or Gemini 3.1 Pro.
• Price: a Helios rack is estimated between 5 and 5.5 million dollars (August 2026, check on mlq.ai).
• Availability: Cerebras Cloud in H2 2026, mass production in Q2 2027.
• Strategic stake: direct response to Nvidia's acquisition of Groq and the Groq LPX platform, with an open approach compatible with SGLang, vLLM, and Dynamo.

Recommended Tools

Disaggregated architecture: why split the GPU in two

LLM inference follows two fundamentally different phases. Prefill computes the representations of the entire prompt at once — it's a massively parallel, memory-bound compute problem. Decode generates tokens one by one — it's a sequential, compute-bound problem with minimal memory accesses.

A GPU does both, but it is optimal for neither phase. It wastes compute cycles during decode and underutilizes its memory bandwidth during prefill.

The disaggregated solution assigns each phase to the most suitable hardware. This is exactly what Cerebras has been theorizing about for months on its blog dedicated to the subject: the GPU is being split in two.

AMD brings prefill with Helios. Cerebras brings decode with the WSE. The 5x gain in tokens/sec/watt comes from this radical specialization: each chip does what it does best, without compromise.

For models over 1 trillion parameters — GPT-5.5, Claude Opus 4.7 Adaptive, Gemini 3 Pro Deep Think — this difference is not marginal. It's the difference between profitable inference and inference that burns millions with no economic justification.

AMD Helios: the prefill monster in numbers

The Helios rack is not just a simple assembly of GPUs. It is an infrastructure specifically designed for the prefill of massive models, with specifications that surpass anything that existed on the market in August 2026.

According to AMD's official press and the detailed analysis by Spheron:

• 72 MI455X GPUs per rack, CDNA 4 architecture
• 31 TB of HBM4 total — enough to load a 1T+ parameter model in FP4 without pagination
• 2.9 exaFLOPS in FP4 per rack — FP4 precision is now the standard for large model inference
• Ultra-fast interconnect to minimize latency between GPUs during distributed prefill

The price estimated by MLQ.ai is between 5 and 5.5 million dollars per rack (August 2026). It's expensive, but the calculation is simple: if this rack does prefill 5x more efficiently than a monolithic equivalent, the 3-year TCO leans heavily in its favor.

The bottleneck is no longer pure compute. As highlighted by Wing.vc in its analysis of inference unbundling, the bottleneck has shifted to the interconnect — the ability to make prefill chips communicate with decode chips without adding prohibitive latency.

This is precisely where the partnership makes sense. AMD and Cerebras worked on the communication protocol between Helios and the WSE, although the exact technical details of this interconnect remain under NDA.

Cerebras WSE: the decode king that finally finds its role

Cerebras' wafer-scale engine has always been a technical marvel without a clear market. A chip the size of an entire wafer, with 46 billion transistors, but designed for a use case (sequential decode) that the market did not yet separate from prefill.

The game changes radically with this partnership. The WSE becomes the decode component of a disaggregated pipeline, and that is exactly what it was designed for.

The decode of an LLM is fundamentally sequential: each token depends on the previous one. A traditional GPU executes this task suboptimally because it has to manage thread warps, shared memory, complex schedulers — all of this just to generate one token at a time.

The WSE, with its massive grid of simple cores, excels in this scenario. Each core can be assigned to a portion of the decoding computation without the overhead of a GPU. The result: drastically reduced per-token latency, especially on massive models.

The concrete use case is immediate. When running GPT-5.6 Sol on Cerebras at 750 tokens/second, as we already analyzed, we see the raw power of the WSE on decode. But without a good prefill partner, the system remains unbalanced.

AMD Helios fills this gap. The WSE is no longer a technical anomaly looking for a use case — it is the heart of decode in the most efficient architecture on the market.

The competitive context: Nvidia-Groq, the giant and its acquisition

To understand the importance of this AMD-Cerebras partnership, we need to look at what just happened at Nvidia.

The $20 billion acquisition of Groq changed the game. Groq raised 650 million dollars and pivots to neocloud to ensure its survival after Nvidia scooped up its soul. The Groq LPX platform, based on LPUs (Language Processing Units), was until now Cerebras' main challenger in the fast inference space.

But with the acquisition, Groq's LPU becomes an internal asset for Nvidia. And Nvidia has no interest in seeing disaggregated inference thrive outside its ecosystem — because that challenges the monolithic GPU model that generates its huge margins.

The AMD-Cerebras partnership is a direct response. It essentially says: you can buy Nvidia's monolithic GPU (which does both phases poorly), or you can assemble a disaggregated solution with the best prefill hardware (AMD) and the best decode hardware (Cerebras), for 5x less energy consumption per generated token.

Add to this that OpenAI is working on its own inference chip, Jalapeño with Broadcom, which promises -50% on serving costs. Everyone wants to break free from Nvidia dependency for inference. AMD-Cerebras is the most aggressive and immediately available option.

Announced customers: why OpenAI, Meta and Oracle are interested

The list of customers announced by AMD in its official press release is telling: OpenAI, Meta, Oracle, HPE and the Department of Energy.

OpenAI is the most significant. The leader in agentic LLMs with GPT-5.5 (agentic score of 98.2) has a massive inference cost problem. Each request on GPT-5.5 costs a fraction of a cent, but multiplied by hundreds of millions of users, the GPU bill skyrockets. A 5x reduction in energy consumption per token is not a luxury, it is an economic survival necessity.

Meta has its own model, and the open source one with Llama. But the inference of massive open source models on proprietary infrastructures is a cost nightmare. The disaggregated approach allows Meta to deploy its models on an infrastructure where each component is optimized — a philosophy that matches their engineering culture.

Oracle has massively invested in AI infrastructure with its OCI AI Infrastructure. Being able to offer Helios+WSE clusters to its cloud customers is a powerful differentiator against AWS (which already has a Cerebras partnership) and Azure (which remains dominated by Nvidia).

The US Department of Energy uses massive models for scientific simulation and data analysis. The low latency of Cerebras decode coupled with the prefill power of Helios is exactly what they need for workloads where every millisecond counts.

The software ecosystem: SGLang, vLLM and Dynamo already compatible

Revolutionary hardware architecture is useless without the software to leverage it. And this is where the AMD-Cerebras partnership takes on an additional dimension.

The most popular inference frameworks already support disaggregation. SGLang, vLLM and Dynamo (Anthropic's framework) can separate prefill and decode in their execution pipeline. This means a customer deploying Helios+WSE does not need to rewrite their software stack — they configure both backends and the framework handles the routing.

This is a considerable advantage over the Groq LPX approach, which requires deeper adaptation to the LPU format. And it is an overwhelming advantage over custom chips like OpenAI's Jalapeño, which will require a dedicated stack.

Optimizing the interconnect between prefill and decode remains the main software challenge. Frameworks must be able to transfer hidden states (KV cache) from the prefill backend (Helios) to the decode backend (WSE) with minimal latency. This is a non-trivial engineering problem, but the open-source community is moving fast.

The case of DeepSeek's DeepEP is instructive here. The DeepEP library optimizes GPU-GPU communication for distributed models, and the same principles apply to prefill-decode communication in a disaggregated architecture.

Benchmark and performance: what 5x tokens/sec/watt really means

The figure of 5x more tokens/sec/watt is the core of the announcement. But it needs to be put into context.

According to Cerebras' official press release and the analysis by Tom's Hardware, this benchmark compares a standalone Cerebras WSE system (which struggles with prefill because it wasn't designed for it) with a disaggregated Helios+WSE system (where each component is specialized).

The 5x gain is therefore not a comparison with an equivalent Nvidia system — it is an internal comparison showing what specialization brings. The comparison with Nvidia remains to be done by independent third parties.

However, the logic is sound. On a model like GPT-5.5 (1T+ parameters), an Nvidia H200 GPU must devote a significant fraction of its power to prefill, then switch to decode. During prefill, the decode cores are underutilized. During decode, memory bandwidth is underutilized. The waste is structural.

With Helios+WSE, prefill runs at full speed on the 72 MI455X while the WSE is ready to seamlessly chain into decode. No transition, no waste, no compromises. Energy efficiency soars.

It should be noted that these figures come from internal benchmarks. AI history is full of optimized internal benchmarks that do not survive contact with real workloads. METR recently highlighted the trap of benchmark gaming in model evaluation. Caution is warranted until independent third parties (MLPerf, for example) publish their own measurements.

Comparison table: Helios+WSE vs inference alternatives

This table reveals a clear dynamic: for models under 500 billion parameters, the monolithic GPU remains sufficient. But beyond 1 trillion — where the best agentic models now live, such as GPT-5.5 (98.2), Gemini 3 Pro Deep Think (95.4), and Claude Opus 4.7 Adaptive (94.3) — the disaggregated architecture becomes essential.

AWS, Cerebras and the inference cloud war

It should not be forgotten that Cerebras already has a major partnership with AWS. Cerebras instances on EC2 have been available since late 2025, and AWS was the first hyperscaler to adopt the WSE at scale.

The AMD-Cerebras partnership does not call into question the agreement with AWS. On the contrary, it broadens it. AWS will be able to offer instances combining AMD Helios GPUs for prefill and Cerebras WSEs for decode, all integrated into its proprietary network infrastructure (EFA, UltraCluster).

The real question is: will AWS offer this solution before Oracle, which is also a named customer? The AI inference cloud war is shifting from the race for the number of GPUs (which was won by AWS and Azure) to the race for per-token efficiency (which is still open).

If Oracle OCI can offer Helios+WSE clusters before AWS, it could attract the customers most sensitive to inference costs — exactly those running 1T+ parameter models in production.

Impact on developers: what actually changes

For a developer running models via APIs, the short answer is: not much in the short term. Disaggregated architecture is an infrastructure change, not an API change.

But in the medium term, the impact is real. If the cost per token of a model like GPT-5.5 or Claude Opus 4.7 drops by 3 to 5x thanks to disaggregated inference, this will be reflected in API prices. Providers that adopt Helios+WSE will be able to offer more aggressive pricing.

For teams doing self-hosting — and this is a fast-growing segment with models like Kimi K2.6 (88.1 agentic, self-host) or GLM-5 Reasoning (82, self-host) — the architecture opens up new possibilities. Deploying a Helios cluster for prefill and a WSE for decode makes it possible to achieve hyperscaler-level performance without depending on a single hardware provider.

It is from this perspective that the best models on LM Studio and the best Ollama models like Llama 4, Qwen3 and Gemma 3 will take on a new dimension. When disaggregated inference becomes accessible beyond hyperscalers, open source models of 100-500B parameters will be able to compete with proprietary models of 1T+ in terms of latency perceived by the user.

The bottleneck has shifted: from compute to interconnect

This is perhaps the most important point of this entire announcement, and the least commented on.

For years, the AI inference bottleneck has been raw compute power. More FLOPS = more tokens. GPUs became compute monsters, HBM exploded in capacity, and models followed suit.

But with disaggregated architecture, compute is no longer the problem. The problem is getting the prefill backend to communicate with the decode backend. Every millisecond lost in transferring the KV cache between Helios and the WSE is a millisecond where the client is waiting.

As Wing.vc notes in its analysis, the AI inference battlefield is shifting from FLOPS to interconnect. The companies that will win are those that solve the problem of state transfer between specialized components.

It's a fascinating paradox: by separating the GPU to better specialize each phase, we create a new communication problem that didn't exist in the monolithic architecture. The net gain is positive (5x on tokens/sec/watt), but the remaining room for improvement is massive — and it lies in the interconnect, not in compute.

Availability schedule and current limitations

The schedule, as reported by MLQ.ai, is as follows:

• H2 2026: availability on Cerebras Cloud for early customers (controlled access)
• Q2 2027: mass production of Helios racks, general availability at partner hyperscalers

This gap of nearly a year between the announcement and general availability is standard in the industry, but it leaves a window open for Nvidia. By Q2 2027, Nvidia will likely have announced its next generation of GPUs (post-Blackwell/Rubin) and will integrate Groq's LPUs into its cloud service offering via Groq LPX.

The current limitations are also important to mention. The 5x tokens/sec/watt benchmarks are internal. Independent tests do not exist yet. The interconnect between Helios and WSE is partially under NDA. And the software frameworks, while compatible in principle, have not yet published performance benchmarks on this specific architecture.

In other words: the announcement is promising, but real-world proof remains to be done.

❌ Common mistakes

Mistake 1: Confusing disaggregated inference with model parallelism

Model parallelism (tensor parallel, pipeline parallel) splits a model that is too large to fit on a single GPU and distributes it across multiple GPUs that all perform both phases. Disaggregated inference separates the phases themselves and assigns them to different hardware. It is not the same thing, and the two can be combined.

Mistake 2: Thinking that 5x tokens/sec/watt applies to all models

The 5x gain is measured on models with over 1 trillion parameters. On a 70B model like Llama 4, the gain will be significantly lower because the prefill is less expensive and the imbalance between phases is reduced. Disaggregated architecture has a break-even point in terms of model size.

Mistake 3: Believing that Cerebras replaces AMD (or vice versa)

Neither Cerebras alone nor AMD alone can do the complete job. The WSE is bad at prefill. The MI455X are suboptimal at sequential decode. It is the combination that creates value. Anyone presenting the announcement as "Cerebras beats AMD" or "AMD beats Cerebras" has not understood the topic.

Mistake 4: Ignoring the cost of the interconnect

A $5.5M Helios rack plus a WSE system represents a considerable upfront investment. The 5x ROI on tokens/sec/watt is only realized under sustained load with massive models. A half-load deployment will likely be less profitable than an equivalent monolithic GPU cluster.

❓ Frequently Asked Questions

Can an individual developer use Helios + Cerebras?

Not directly. This architecture targets hyperscalers and large enterprises. However, you will benefit from it indirectly through the APIs of providers that adopt it (OpenAI, Meta, Oracle). For local use, the meilleurs modèles Ollama remain the pragmatic route.

What use cases is it better than Nvidia GPUs for?

Models with over 1 trillion parameters in high-frequency production. Below 500B, the monolithic GPU remains competitive and simpler to deploy. Above 1T, the energy efficiency of the disaggregated architecture becomes a decisive advantage.

Is Groq LPX dead with this announcement?

No, but its position is more fragile. Groq LPX benefits from Nvidia's sales ecosystem, but the LPU architecture is proprietary and less open than the AMD+Cerebras combination. The battle will be fought on independent benchmarks and real-world availability.

Will the availability on Cerebras Cloud in H2 2026 be accessible to startups?

Likely via an early access program, but prices will be high. It is primarily a tool for enterprise clients (OpenAI, Meta, DOE) that have the budgets and workloads to justify the investment. The APIs IA gratuites via Groq or OpenRouter remain more accessible for small budgets.

How do SGLang or vLLM handle disaggregation in practice?

These frameworks support a mode where prefill and decode are configured as distinct backends. The framework routes the prompt to the prefill backend, retrieves the KV cache, and sends it to the decode backend for generation. The configuration is declarative, but optimizing the interconnect remains an infrastructure task.

✅ Conclusion

Disaggregated inference is no longer a theoretical idea — it's a product architecture with customers, a timeline, and numbers. AMD Helios for prefill, Cerebras WSE for decode, 5x tokens/sec/watt on massive models. The monolithic GPU just received its first credible structural challenge in years.

The real battle won't be over compute, but over the interconnect. And it won't be won by internal benchmarks, but by production deployments at OpenAI and Meta. Q2 2027 will be the moment of truth.