NVIDIA Nemotron 3.5 Super & Lightning: open-weights hybrid Mamba-Transformer MoEs tailored for agentic AI

NVIDIA Nemotron 3.5 Super & Lightning: open-weights hybrid Mamba-Transformer MoEs built for agentic AI

🔎 Architecture becomes the killer weapon again

For years, the LLM race boiled down to stacking attention layers and inflating context windows. It worked. But agentic AI has changed the load profile: an agent working for hours on a code repository or a legal case makes hundreds of tool calls in a row, with context growing at every loop turn. And for every token generated, a classic Transformer has to re-read its entire KV cache. The memory and latency bill explodes.

NVIDIA responds with two open-weights models: Nemotron 3.5 Super (roughly 120 billion total parameters, ~12B active) and Nemotron 3.5 Lightning (~50B total, ~5B active). Two Mixture-of-Experts (MoE) models with a hybrid Mamba-Transformer architecture, described in a technical report published on arXiv and downloadable from NVIDIA's Hugging Face page.

This launch fits into an already visible strategy: after the Nemotron 3 Ultra 550B presented at Computex, NVIDIA is now filling out the middle and bottom of the spectrum, where production agents actually run.

My take in one sentence: this is NVIDIA's most interesting open-weights launch since the start of the Nemotron lineup, because it isn't chasing the biggest model — it's chasing the best cost per agent token. And on that specific battleground, architecture matters more than brute force.

Key Takeaways

• Two hybrid MoE open weights models: Super (~120B total, ~12B active) for quality, Lightning (~50B, ~5B active) for speed and cost.
• Mamba-Transformer architecture: most of the network relies on constant-state SSM blocks, with a few attention layers to preserve accurate information retrieval.
• Near-constant inference memory: the KV cache, the #1 bottleneck for agents on long contexts, is reduced by about 90% at 128K tokens according to the paper.
• 256K token context maintained without collapse in end-of-window recall scores (self-reported figures).
• Permissive license: weights freely downloadable, commercial use permitted under the NVIDIA Open Model License.
• Deliberate target: agents — tool-calling, long sessions, control over inference costs.

Recommended Tools

Nemotron 3.5 Super and Lightning — what exactly are they?

Direct answer: two open-weights LLMs with a hybrid architecture, where Mamba blocks (a state-space model, or SSM) are interleaved with classic attention blocks, all organized as a MoE with only a fraction of the parameters activated per token.

The design is easy to sum up: Super targets the best quality/VRAM trade-off for serious agents, Lightning targets minimal latency and single-GPU deployment. Both share the same architectural recipe, the same 256K-token context, and weights released in FP8 and NVFP4.

According to the technical report, training spans more than 20 trillion tokens, with a ratio of roughly one attention layer for every eight Mamba blocks. Both models also ship a reasoning switch that can be toggled on the fly — handy for balancing latency and quality across the stages of an agentic pipeline.

A family calibrated for agents

Lightning and Super fit into an already dense lineup. The Nemotron 3 Ultra 550B sits at the top of the American open rankings, while the vision-language variant Nemotron 3.5 Super VL, built on the same 120B base with 12B active parameters, covers multimodal agents.

In other words: NVIDIA isn't releasing isolated models — it's building a coherent platform where every component speaks the same tooling language (NeMo, NIM, TensorRT-LLM). That's exactly what teams industrializing agents want.

License: open weights, not open source in the strict sense

An honest clarification: the NVIDIA Open Model License allows commercial use and redistribution of outputs, with no user-count threshold like Llama. But it is not an OSI license in the strict sense — usage restrictions remain. Read it before any sensitive deployment.

Why Mamba is a game changer for agents

Direct answer: because an agent's bottleneck is almost never compute power, but memory and latency on a context that grows with every iteration.

A classic Transformer stores, for every past token, its key and value vectors in the KV cache. That memory grows linearly with the context, and every new token generated requires re-reading that entire cache. At 128K tokens on a 120B MoE, the KV cache alone can weigh several tens of GB — before even counting the weights.

Mamba blocks work differently: they maintain a fixed-size recurrent state, independent of the sequence length. O(1) memory instead of O(n). Direct consequence: at long contexts, the bulk of inference memory becomes… the weights themselves, not the cache.

The paper claims a KV cache reduction of around 90% at 128K tokens compared to an equivalent dense Transformer, and a two- to threefold throughput increase in long-form generation. Stay cautious: these are vendor numbers, measured on their own configurations.

Why keep attention layers then? Because pure SSMs are bad at copying exact information — the famous "needle in a haystack" retrieval problem. The hybrid fixes this flaw: a few attention layers restore recall precision, while Mamba absorbs the volume. It's the best of both worlds, at the cost of non-trivial engineering complexity.

One last nuance, for the sake of completeness: for short chat or simple summarization, a modern Transformer is still perfectly fine. The hybrid really pays off on long-running agent sessions, massive corpora, and multi-step pipelines. If your use case fits within 8K tokens, this article won't change your life.

Benchmarks: where they stand (and how to read these numbers)

Direct answer: according to NVIDIA's self-reported figures, Super lands in the top tier of agentic models — closed models included — while Lightning plays in the efficient open-weights league.

Here's where they stand in our monthly agentic comparison, with Nemotron's self-reported scores marked with an asterisk:

*The Nemotron model scores come from the paper and NVIDIA's announcements, not from independent testing.

Three warnings before crying scandal or revolution. One: self-reported figures should never be compared directly to independent leaderboards — protocols, temperatures and evaluation grids differ. Two: agentic benchmarks mix heterogeneous tasks; a good average score can hide a weakness on your specific use case. Three: what is actually verifiable is the weights — download them and test on your own agent traces.

To put the open competition in perspective: Z.AI is pushing the opposite approach with GLM-5.2, a 753B MoE monster with 1M context under an MIT license — more powerful on paper, but far more resource-hungry to host. NVIDIA, for its part, is betting on efficiency rather than size.

My position, no sugarcoating: if the figures hold up even at 80% in independent testing, Super becomes a serious candidate to replace closed APIs in cost-sensitive agentic pipelines. For an up-to-date selection month after month, our best LLM comparison and our dedicated guide to LLMs for agents track how the landscape evolves month by month.

NVIDIA's Open Strategy Against Llama and Qwen

Direct answer: NVIDIA isn't trying to win the download war — the one dominated by Qwen and the Chinese labs — but the war over per-token agent cost, the one where its hardware decides.

First, look at the landscape. Llama has run out of steam: the latest iterations have disappointed, Meta seems to be hesitating over how open its next big models will be, and the niche of "credible made-in-USA open weights" has been left vacant. On the other side, Qwen crushes the download numbers thanks to an ecosystem of fine-tunes, quantizations, and multilingual support that's hard to dislodge.

NVIDIA attacks from an angle nobody else can hold: hardware/software co-design. A hybrid Mamba-Transformer model optimized for NVFP4, served by vLLM or the NIM containers, makes optimal use of their GPUs. Every point of per-token throughput gained is a selling point for silicon.

Let's be clear-eyed about the motivations: open at NVIDIA is not philanthropy. An agent running locally on open weights still consumes GPUs — preferably theirs. It's a coherent business model, and frankly, it's well played.

One less visible asset remains: enterprise credibility. For a CIO, publishing weights under a permissive license, with professional support behind them, is far more reassuring than a model card signed by some upstart lab. In the on-premise agent segment — banking, healthcare, public sector — it's a real differentiator against Asian open weights, whose ecosystem remains harder to qualify in certain regulated environments.

Running them locally: VRAM, quantization, hardware

Short answer: Lightning targets a single high-end consumer GPU in NVFP4, while Super requires multi-GPU or a datacenter card. In both cases, the near-constant KV cache makes long context affordable.

Add a few GB for the Mamba state and activations. The remarkable point: at 256K context, the extra overhead remains low — this is precisely where the hybrid architecture justifies itself, where an equivalent Transformer would need tens of GB of KV cache.

For common sizing mistakes, see below — the MoE pitfall is covered in detail there. To set up your stack step by step, follow our guide to installing a local LLM and our selection of the best LLMs to run locally.

# Indicative command — check the exact checkpoint name on NVIDIA's Hugging Face
vllm serve nvidia/Nemotron-3.5-Lightning-Instruct-FP4 --max-model-len 262144

One practical warning: hybrid architectures require specific kernels (SSM scan, FlashInfer support). NVIDIA is announcing vLLM support from day one; Ollama and LM Studio builds will follow at the community's pace. Don't plan a production migration on a stack that has never served a Mamba model. Our experience reports on open source AI agents with Ollama detail these ecosystem limitations.

Finally, a pragmatic option: if your agent revolves around an API — orchestration, RAG, scheduled tasks — a simple Hostinger VPS is more than enough to host the orchestration layer, with the model remaining at your provider or on your own GPU rack.

Super or Lightning: which model for which agent?

Direct answer: Lightning for anything that needs to be fast, inexpensive, and single-GPU; Super for long, demanding tasks where decision quality matters more than latency.

The strategy I recommend to serious teams: combine both. A lightweight model like Lightning handles the agent loop — tool calls, routing, intermediate summaries — and escalates to Super only for heavy reasoning steps. This "router" pattern reduces the overall bill without sacrificing final quality.

For agents specifically focused on code, our comparison of the best LLMs for coding remains the best basis for decision-making, since the quality/latency trade-off depends heavily on your execution stack.

❌ Common Mistakes

Mistake 1: calculating VRAM based on active parameters

"12B active, so 12 GB of VRAM" — no. In a MoE, all experts reside in memory; only the computation is sparse. Always budget based on total parameters, in the chosen weight format. Solution: size for ~120 GB in FP8 or ~65 GB in FP4 for Super, and keep a 10-15% margin.

Mistake 2: assuming the entire ecosystem supports the hybrid architecture

"Transformer-only" inference servers will do nothing with these models without dedicated SSM kernels. Solution: start with official vLLM support or NVIDIA's NIM containers, and verify community support before building a pipeline around a third-party runtime.

Mistake 3: swallowing self-reported benchmarks

The scores in the paper measure NVIDIA's configurations, not yours. Solution: build a test suite based on your real agent traces — your tools, your documents, your usual pitfalls — and compare Super, Lightning, and your current solution on that benchmark alone.

Mistake 4: using a cannon to kill a fly

Running Super on a 4K-context chatbot means paying for an architecture whose advantages you're not exploiting. Solution: reserve the hybrid for long contexts and persistent sessions; elsewhere, Lightning — or a smaller model — does the job for less.

❓ Frequently Asked Questions

Are Nemotron 3.5 Super and Lightning really open source?

Open weights, yes; open source in the OSI sense, no. The weights can be freely downloaded from Hugging Face and commercial use is permitted under the NVIDIA Open Model License, but the full training code and data are not published. This is the industry standard—it just needs to be named correctly.

How much VRAM do you need to run Lightning locally?

Count on ~28 GB in NVFP4 for the ~50B total parameters, which fits on a 32 GB RTX 5090—tight, but viable. In FP8, you need ~50 GB, so a datacenter card or multi-GPU setup. The hybrid advantage: at 256K context, the additional memory remains marginal.

Does Super really beat Llama and Qwen?

On self-reported agentic benchmarks, Super ranks among the best open weights. But Qwen still has a more mature ecosystem of fine-tunes and better multilingual support, and direct comparisons remain rare. Test it on your own use cases: that's the only verdict that matters.

Can they be used commercially?

Yes, under the NVIDIA Open Model License: commercial use, derivation, and internal deployment are permitted, with no user-count clause. Some usage restrictions remain—check the license on the Hugging Face repository before shipping anything to end customers.

Where can you follow how these models evolve against the competition?

The Nemotron lineup moves fast, and so does the competition. Our monthly comparison of the best LLMs updates the agentic rankings every month, with verified scores and license changes. It's the best place to track things and decide when to migrate.

✅ Conclusion

With Super and Lightning, NVIDIA isn't just selling open weights: it's pushing an architectural thesis — for agents, memory efficiency beats raw size. Try Lightning on your agent loops this week, and follow the movement in our guide to LLMs for AI agents.