Thinking Machines ships Inkling 2 Mini: 26.4B parameters, 3.4B active, and middle layers that loop 3 times per token
🔎 What changes today
Thinking Machines kept its word. One week after teasing Inkling 2 Small under an Apache 2.0 license, the lab is delivering Inkling 2 Mini: 26.4 billion total parameters, 3.4 billion active per token, and a looped transformer architecture that runs the two middle blocks (layers 6 to 17) three times per token. The result: 5.4 billion effective activations, a memory footprint of ~20 GB in NVFP4, and scores that make it the leader among open-weights models under 100B on DeepSWE v1.1 (48.2%) and Vals Index v2.2 (46.5%). The weights are open on Hugging Face, vLLM is providing day-0 support, and a variable-pass companion model (TML-d24-varloop-research) is shipping alongside the release for research purposes. This is the first time an architecture with temporal loops has reached this level of maturity in open-weight production.
Key takeaways
- Unprecedented architecture: 24 stored layers, 48 applications per token thanks to looping of layers 6-17 (three passes with shared weights).
- Halved inference cost: the unlooped version was 7.5% cheaper to train but required 140% more inference compute to match the loss on 512k-token agentic traces.
- Genuine local deployment: native NVFP4 quantization (last 4 layers in MXFP8), ~20 GB of unified VRAM/RAM, 1M-token context window, natively multimodal.
- Benchmarks: #1 open-weight <100B on DeepSWE v1.1 (48.2%) and Vals Index v2.2 (46.5%), ahead of Gemma 4 31B, Qwen3.8-27B, Dory 1.0 4B, SenseNova-Loop-9B.
- Related research: time horizon without CoT goes from 16s (1 pass) to 37s (3 passes); gains plateau beyond 3 passes; N-Hop Lookup keeps improving up to 10 loops.
Recommended tools
| Tool | Primary use | Price (November 2026) | Ideal for |
|---|---|---|---|
| vLLM | High-performance inference server | Free (open source) | Local production, day-0 support for Inkling 2 Mini |
| Ollama | Simplified local execution | Free | Laptop, quick tests, development |
| LM Studio | Local GUI with quantization | Free | Non-technical users, model comparison |
| Hugging Face Hub | Official weights and model card | Free | Downloads, fine-tuning, research |
Unprecedented architecture: looping at the heart of the model
Inkling 2 Mini is not a classic MoE: it's a looped transformer where computation is reused temporally. The 24 layers are organized into four blocks of six. The two central blocks (layers 6-17) run three times per token with shared weights. Concretely, the model stores 24 layers but applies 48 per token — hence the 5.4 billion effective activations for 3.4 billion active parameters.
Each block of six layers comprises five sliding-window layers (local attention, 1024 tokens) and one full attention layer. The hybrid attention uses 24 query heads and 8 key/value heads. The MoE router selects 8 experts out of 128 per layer, after pruning two-thirds of the initial 384 experts and then healing on 1 trillion tokens distilled from an internal Inkling model.
The communication mechanism between passes is the key. Passes 2 and 3 read "messages" from the previous pass via rank-64 gates that inject information into query, key, value, and the MoE inputs. Each pass keeps its own KV cache — 48 KV caches in total for 24 stored layers. Passes 2-3 share the same starting point (the output of the prelude, layers 0-5) and communicate only through these messages. The coda (layers 18-23) finalizes the processing.
This architecture changes the economic equation: looping does not increase the stored knowledge (a function of the total number of parameters, 26.4 billion), but it amplifies per-token reasoning without inflating model memory. The training cost of the non-looped version was 7.5% lower, but it required 140% additional inference compute to reach the same loss on 512k-token agentic traces. The trade-off is clear: you pay once during training to save massively at inference.
Source: Introducing Inkling 2 Mini - Thinking Machines Lab, Inkling 2 Mini Model Card, Welcome Inkling 2 Mini - Hugging Face Blog
Performance: leader among open-weights models under 100B
On the benchmarks that matter for agentic tasks and long-form reasoning, Inkling 2 Mini dominates its weight class. DeepSWE v1.1 (real-world software engineering evaluation): 48.2%. Vals Index v2.2 (complex, multi-hop reasoning): 46.5%. These scores place the model ahead of Gemma 4 31B, Qwen3.8-27B, Dory 1.0 4B, and SenseNova-Loop-9B — all of which are heavier in active or total parameters.
The model card notes that the 8/128 MoE router, after pruning/healing, retains routing efficiency superior to equivalent dense MoEs. The hybrid local/global attention with a 1024-token window makes it possible to handle the 1M context without quadratic explosion. Native NVFP4 quantization (MXFP8 for the last 4 layers) preserves precision where it matters: the output layers and the critical passages of the loop.
The companion model TML-d24-varloop-research (released on Hugging Face) confirms the trend: without chain-of-thought, the time horizon goes from 16 seconds (1 pass) to 37 seconds (3 passes). Gains plateau beyond 3 passes, but the N-Hop Lookup benchmark keeps improving up to 10 loops. This validates the hypothesis: looping extends effective reasoning depth without explicit CoT, which changes the economics of prompting for complex tasks.
Source: Introducing Inkling 2 Mini, Inkling 2 Mini Model Card, thinkingmachines/Inkling-2-Mini on HF
Quality/Cost Ratio: The New Benchmark Against Qwen4-Max and GPT-6.2 Sol
Inkling 2 Mini redefines the ceiling of what a local model can do — and forces proprietary APIs to justify their margin. Let's compare real inference costs (November 2026):
| Model | Input / Output price (per M tokens) | Type | Context | Notable feature |
|---|---|---|---|---|
| Inkling 2 Mini (local) | ~$0 (electricity/GPU cost only) | Open-weight, local | 1M | 20 GB VRAM, 3-pass looping |
| Qwen4-Max (QwenCloud) | $1.50 / $4.50 | Proprietary API | 1M+ | Input cache $0.075/M |
| GPT-6.2 Sol (OpenAI) | $2.00 / $10.00 | Proprietary API | 1M+ | 1/5th the cost of Astra |
| GPT-6.2 Astra (OpenAI) | $8.00 / $40.00 | Proprietary API | 1M+ | Flagship reasoning |
Sources: GPT-6.2 Sol and Astra pricing - Requesty, Qwen4-Max pricing - DataCamp, GPT-6.2 Sol vs Qwen4-Max - llm-stats
Qwen4-Max is ~1.8x cheaper per token (blended 3:1) than GPT-6.2 Sol, but independent runs are mixed: GPT-6.1 Sol remains cheaper per task and slightly better on Vals; Qwen4-Max is described as slow and verbose. Inkling 2 Mini, for its part, eliminates the API bill for anyone who owns the hardware (RTX 3090/4090 24 GB, M2/M3 Max 32+ GB, or any 20+ GB GPU). On DeepSWE and Vals, it beats both in the open-weight <100B class.
The math is simple: for a continuous agentic workload (e.g., 10M tokens/day), Sol costs ~$60/day, Qwen4-Max ~$33/day, and Inkling 2 Mini $0 marginal. A RTX 4090 (€1,600) pays for itself in ~27 days vs Sol, ~48 days vs Qwen4-Max. And the model runs on your own machine, with no network latency, no data leaks, no rate limits.
This is where the link to our analysis of Qwen4-Max: Alibaba's new flagship lands the day open source weighs 10 takes on its full meaning: the price/performance battle is tipping toward local. And our deep dive on GPT-6.2 Astra and Sol: OpenAI crosses its own self-improvement threshold shows that even the proprietary leader is feeling the pressure.
Local deployment: 20 GB, vLLM day-0, laptop-ready
The "executable on a laptop" promise isn't marketing: native NVFP4 quantization holds up. The ~20 GB memory footprint includes model + KV cache for standard context. On an RTX 4090 (24 GB) or a MacBook Pro M3 Max (36/48/96 GB unified), the model fits with headroom for batching or long context.
vLLM provides day-0 support with a dedicated implementation: three-phase execution (prelude layers 0-5 → core loop 6-17 ×3 passes → coda 18-23) and a 3-layer DSpark draft model that reads hidden states at four layer visits for accelerated speculation. The vLLM blog details the optimization of message passing between passes via the rank-64 gates — critical for not killing throughput.
Ollama and LM Studio will follow in the coming days (weeks). For now, vLLM is the production path. The typical command:
vllm serve thinkingmachines/Inkling-2-Mini --quantization nvfp4 --max-model-len 1000000
NVFP4 quantization is native (no post-training recalibration), and the last 4 layers in MXFP8 protect numerical stability on output. The 1M token context is native — no hacky RoPE scaling, the hybrid local/global architecture handles it structurally.
Source: Inkling 2 Mini on vLLM: Day-0 Support - vLLM Blog, Introducing Inkling 2 Mini
Research: the variable-pass model and the limits of looping
The companion model TML-d24-varloop-research may be more important than Inkling 2 Mini itself for research. It allows varying the number of passes (1 to N) at inference and observing the effect on reasoning. The results are clear:
- Time horizon without CoT: 16 s (1 pass) → 37 s (3 passes). The model "thinks further ahead" without an explicit prompt.
- Ceiling at 3 passes: gains on general tasks stop there. More passes = more compute for zero return.
- N-Hop Lookup: keeps improving up to 10 loops. Multi-hop information retrieval benefits from arbitrary compute depth.
This confirms a strong hypothesis: looping extends the sequential compute depth per token, not knowledge. The 26.4 B parameters store the knowledge; the 3 passes (48 effective layers) allocate "thinking time" per token. Beyond 3 passes, the model has exhausted what its parametric capacity can express in pure sequential reasoning.
This has direct implications for agent architecture: no need for verbose CoT for deep reasoning if the model has an internal loop. Output token cost drops, perceived latency decreases, reliability rises (less drift in long CoT chains). But beware: looping does not create new knowledge. On purely factual or recall tasks, 1 pass is enough. The gain lies in compositional reasoning.
This research also validates Thinking Machines' approach: releasing the production model (fixed 3 passes, optimized) AND the research model (variable passes) simultaneously. The community can audit, extend, and understand. That's rare and healthy.
Source: Introducing Inkling 2 Mini, Welcome Inkling 2 Mini - Hugging Face Blog
❌ Common Mistakes
Mistake 1: Confusing active parameters with effective activations
What goes wrong: Reading "3.4B active" and believing the model is a 3.4B.
The fix: Looping triples effective activations to 5.4B per token. That's the number that determines per-token reasoning capacity. Comparing it to a 3.4B active MoE without looping drastically underestimates its power.
Mistake 2: Thinking that looping increases knowledge
What goes wrong: Expecting 3 passes to improve factual recall or domain coverage.
The fix: Knowledge = f(total parameters = 26.4B). Looping = sequential compute budget per token. On pure recall, 1 pass = 3 passes. Only use multiple passes for reasoning (coding, math, planning, N-hop).
Mistake 3: Ignoring the KV cache × 48 cost
What goes wrong: Calculating VRAM for 24 layers and forgetting that each pass has its own cache.
The fix: 48 KV caches in total. For 1M context, that adds up. vLLM handles the optimization, but if you're implementing it yourself, plan generously. NVFP4 quantization only applies to weights, not to KV caches (FP16/BF16).
Mistake 4: Believing that Inkling 2 Small (Apache 2.0) is already out
What goes wrong: Confusing last week's announcement with today's delivery.
The fix: Inkling 2 Mini is out (Nov 20, open weights). Inkling 2 Small (Apache 2.0 weights) is promised "next week". These are two distinct models. See our article: Thinking Machines unveils Inkling 2 Small: Apache 2.0 weights next week.
❓ Frequently Asked Questions
Is Inkling 2 Mini really open source?
The weights are open (open weights) on Hugging Face. The license is not explicitly Apache 2.0 in the sources (unlike Small, which will be). Check the LICENSE file on the HF repo before commercial use. The vLLM inference code is Apache 2.0.
Can you fine-tune Inkling 2 Mini?
Yes, the weights are available. The looped architecture with rank-64 gates and a pruned/healed MoE requires particular attention: passes 2-3 share the core weights but have distinct gates. LoRA on the experts and gates is the pragmatic path. No official guide yet — wait for community feedback.
What's the minimum GPU to run it comfortably?
20 GB VRAM/unified RAM for the NVFP4-quantized model + standard context KV cache (4k-32k). For long context (100k+), aim for 24 GB+ (RTX 3090/4090, A6000, Mac M2/M3 Max 32 GB+). Below that, CPU offload is possible but slow.
Does looping work on every type of task?
No. Massive gains on sequential reasoning (coding, math, planning, N-hop QA). Zero on factual recall, classification, "one-shot" tasks. The companion model varloop proves it: plateau at 3 passes except for N-Hop Lookup.
Is vLLM the only day-0 supported server?
For now, yes. The vLLM blog details the specific implementation (prelude/loop/coda, DSpark draft model). Ollama, LM Studio, and TGI will follow. For production now: vLLM.
How does Inkling 2 Mini compare to Nemotron 3 Ultra or Llama 4 70B?
Different weight classes. Nemotron 3 Ultra (47B active) and Llama 4 70B (70B dense/70B active) are 10-20x heavier in active parameters. Inkling 2 Mini targets the performance/footprint ratio, not the absolute ceiling. On DeepSWE/Vals <100B, it wins. On MMLU-Pro or GPQA diamond, the 70B+ models stay ahead.
✅ Conclusion
Inkling 2 Mini is not an iteration: it's the proof that the looped transformer architecture has moved from paper to open-weight production. 26.4B stored, 3.4B active, 5.4B effective via 3 passes, 20GB quantized, leader <100B on agentic benchmarks, vLLM day-0. Looping rewrites the economic equation: you pay for sequential compute per token instead of parameters that are always active. For the first time, a local model beats proprietary APIs on their own turf (long reasoning, coding) at zero marginal cost. The ball is now in OpenAI's, Anthropic's, and Alibaba's court: justify the bill.