Needle: the 14 MB model that wants to run agents on watches and robots
🔎 A complete agent in 14 MB: the bet nobody would take
At the top of GitHub Trending this week: cactus-compute/needle, with +3,838 stars in seven days and around 8,000 in total (GitHub Trending, October 2026). A Python repository under the Apache-2.0 license, with no funding round and no keynote. Just a README holding a promise that seemed absurd a year ago: an agent foundation model in a 14 MB binary.
Needle, developed by Cactus Compute, is not yet another pocket chatbot. It's a model specialized in a single skill — calling tools reliably — designed for the devices where cloud LLMs will never reach: watches, sensors, robots, car cabins, microcontrollers.
Why now? Because agentic AI has just hit a physical wall. Every tool call to the cloud costs latency, money, and a bit of privacy. Meanwhile, the wave of micro-models is accelerating: a 169 MB speech-to-text model that beats Whisper Base, the wave of 2-bit ternary models. Needle takes this logic to its conclusion: what if the entire agent could fit in a watch's RAM?
The essentials
- Needle (Cactus Compute) is a tool-calling foundation model for phones, wearables, smart homes, robots, automotive, and microcontrollers, shipped as a single 8 to 29 MB binary (GitHub, 2026).
- Needle 2 (45M parameters) runs a full session in ~28 MB of RAM thanks to CQ2-bit compression — that's 5× to 70× smaller than FunctionGemma 270M, LFM2.5 230M, or Apple FM at comparable performance.
- Needle 3 is a Laddered Simple Attention Network: every depth from 2 to 20 layers is a deployable model, and the 121M model does the arithmetic of a 50M thanks to its engram memory.
- Reliability by construction: a byte-level grammar, compiled from tool schemas, constrains every generated token — the model cannot invent an out-of-schema tool call.
- Try it now: browser demo, weights on Hugging Face,
pip install cactus-needle.
Recommended tools
| Tool | Main use | Price (October 2026) | Ideal for |
|---|---|---|---|
| Needle — official repository | Source code, per-platform engines, documentation | Free (Apache-2.0) | Embedded developers |
| Needle 3 weights on Hugging Face | Download the weights (single 8 to 29 MB file) and the engines | Free | On-device deployment |
| cactus-needle (Python package) | Set up a full agent session in Python | Free | Rapid prototyping |
| Browser demo — Cactus Compute | Try Needle without installing anything | Free | Discovery, evaluation |
What is Needle, exactly? — A direct answer
Needle is a foundation model trained for a single job: deciding when and how to call tools, on hardware where a conventional LLM won't fit. No essay writing, no brainstorming: an execution brain.
The reference paper (Ndubuaku, Mosoyan, Mroz et al., "Needle: Foundation Tool-Calling Model for Tiny Devices", Cactus Compute, 2026) explicitly positions it for six arenas: phones, wearables, smart homes, robots, automotive, and microcontrollers.
The deliverable is as unusual as the ambition: a single binary of 8 to 29 MB depending on the variant, plus a precompiled engine of less than 1 MB per platform. To put the scale in perspective: a mainstream cloud LLM weighs in at hundreds of gigabytes on the datacenter side.
Needle 2 and Needle 3: what changes with each generation
| Needle 2 | Needle 3 | |
|---|---|---|
| Parameters | 45M | 121M (arithmetic efficiency of a 50M) |
| Weights shipped | single 14 MB binary | single file of 8 to 29 MB |
| Compression | CQ2-bit (Cactus Quants) | — |
| RAM in full session | ~28 MB | not published to date |
| Architecture | Simple Attention Network | Laddered SAN, 2 to 20 deployable layers |
| Access | pip install cactus-needle |
Hugging Face weights + engines |
(Sources: README of the official repository and Python package, October 2026.)
If you're only going to remember one version, it's Needle 3 that matters. The "ladder" — the fact that every depth from 2 to 20 layers is a deployable model — changes how you deploy, not just the size of the file. More on that below.
The architecture: why a 121M model does the work of a 50M
Needle fits in 14 MB because it doesn't look like an LLM: the FFN layer, which holds the majority of the parameters in a classic transformer, has been replaced by a Monarch Hadamard MLP that stores almost nothing.
In a standard transformer, the feed-forward layers typically absorb two-thirds of the parameters. These are huge weight matrices, read from memory at every token. On a microcontroller, where memory bandwidth is the real bottleneck, this simple fact disqualifies the classic architecture.
The Monarch Hadamard MLP replaces these matrices with a structured transformation, computed in O(n log n) — much like an FFT — with no weights to read from memory (CoddyKit analysis, October 2026). You compute instead of storing: the memory budget collapses.
Second key idea: the engram memory, an n-gram store read by gather. The majority of the model's parameters live in this lookup table, not in dense multiplications. The announced headline consequence: Needle 3's 121M model does the arithmetic of a 50M. Capacity and compute cost are decoupled — this is the project's most interesting reversal.
The attention is also reworked: grouped-query attention (GQA) enhanced with causal conv taps, short filters that give the model immediate local memory, and multi-lane hyper-connections that carry the signal across several parallel tracks instead of a single one.
Does it work? The design and ablations are documented in the arXiv paper from July 2026. The numbers look good on paper; independent evaluation is still missing, as with any project coming out of the trending charts. But the approach is coherent, and above all reproducible: everything is open source.
The ladder: paying for the model layer by layer
Needle 3 is trained so that every depth from 2 to 20 layers is a deployable model. The same weight file serves a 2-layer model for a sensor costing a few euros and a 20-layer model for a home automation hub. You choose the quality/latency/energy dial at deployment time, not at training time.
This is, in my opinion, the project's most under-commented idea. Post-training quantization cuts precision after the fact; here, depth is a native deployment parameter. You adjust per device, without retraining anything.
256 tokens, 5 tools, zero out-of-schema calls
Needle tackles the dual challenge of RAM/reliability with three mechanisms: a 256-token sliding window where tools are pinned as KV sinks, a retrieval head that returns only the top 5 tools per turn, and a byte-level grammar that constrains every generated token.
The 256-token sliding window is enough for tool calling, because a tool call is a short exchange, not a conversation. Tools pinned as KV sinks remain permanently visible in the attention cache while the window slides over the dialogue: the model never "rediscovers" its tools, they are structurally present.
On the selection side, no catalog of 300 tools stacked into the prompt. A dedicated retrieval head returns only the top 5 candidate tools per turn, and the constrained grammar applies only to that subset. The context stays tiny, the decision stays relevant.
The byte-level grammar, for its part, is compiled from your tool schemas. It constrains every token: the model physically cannot emit a nonexistent tool name or an out-of-type argument. We're far from the free-form JSON of cloud LLMs, which still regularly hallucinates invented keys.
The last piece: a confidence score calibrated by a learned head. The model knows how to say "I'm not sure" — and the host program can then escalate to a bigger model, local or cloud.
This is, in my view, the right interface for embedded systems. An agent's reliability doesn't come from a well-written prompt, but from a structural constraint applied to generation. Needle understood this before most frameworks.
14 MB vs 540 MB: the comparison in numbers
On the authors' benchmarks, Needle 2 delivers performance comparable to models 5× to 70× larger, in a 14 MB file versus 540 MB to 2 GB in f16.
| Model | Parameters | Weights (f16) | Needle 2 (CQ2-bit) |
|---|---|---|---|
| Needle 2 (Cactus Compute) | 45M | 14 MB | — |
| FunctionGemma | 270M | ~540 MB | 14 MB |
| LFM2.5 | 230M | ~460 MB | 14 MB |
| Apple FM | n/a | ~2 GB | 14 MB |
(Sources: repository README and CoddyKit, October 2026.)
The ratio ranges from 5× (LFM2.5) to 70× (Apple FM) in parameters, and the file factor is even more brutal: 2 bits per weight versus 16 in f16, i.e., an 8× compression gain from precision alone. The "comparable performance" qualifier applies to tool-calling tasks, the model's core business — not to writing or general-purpose reasoning.
DroidCall: the fine-tuning that shakes up the hierarchy
The README also announces fine-tunings on DroidCall, a tool-calling dataset: +18 to +36 points for each subnetwork, and as early as 4 layers — about 29M parameters — the fine-tuned subnetwork beats DeepSeek V4 Flash.
To be read with discernment: these are figures published by the authors, on their own benchmarks, in a specific domain. No independent evaluation has weighed in yet. That's the lot of any project coming out of trending: in-house benchmarks side with the house.
But even with a prudence discount, the trajectory is clear. A 29M-parameter model that rivals a giant on its own turf, in 30 MB of RAM, signals that the hierarchy of capabilities is no longer aligned with size.
Where a 14 Mo agent makes sense (and where it doesn't)
Needle targets contexts where the cloud is slow, expensive, or unavailable: wearables, smart homes, robots, automotive, and microcontrollers.
- Wearables: on a watch, an agent that detects a fall and triggers an emergency call cannot afford a network round trip. Local latency, battery preserved.
- Smart home: a local hub that orchestrates lights, shutters, and heating without sending your living habits to a third party.
- Robots: real-time control loops poorly tolerate the 300 ms of a cloud round trip.
- Automotive: commands and adjustments off-network, where 4G coverage disappears.
- Microcontrollers: with an engine under 1 Mo, the target is explicit — sensors and objects costing a few euros.
What Needle is not: a generalist reasoner. An agent that needs to plan a ten-step strategy, write a report, or analyze a document remains the domain of an LLM — our guide to the best LLMs for agents covers this segment in detail.
The winning pattern, in my view, is hybrid: Needle as the local executor, a cloud LLM as the escalation planner, with the confidence score serving as the trigger. In a multi-agent architecture, Needle naturally takes on the role of the proximity agent — the one that acts when the network fails.
How to try Needle today
Three entry points, all free: a browser demo, a pip package, and the weights on Hugging Face.
pip install cactus-needle
The Python package lets you spin up a complete agent session in a few lines. The README reports ~28 MB of RAM for Needle 2 — the budget of a browser tab.
The Needle 3 weights are on Hugging Face as single files of 8 to 29 MB, accompanied by precompiled engines of less than 1 MB per platform.
The browser demo is the best starting point to form an opinion in five minutes, without installing anything.
A testing tip: start with three or four tools with clean schemas rather than an exhaustive catalog. The retrieval head only returns five candidates per turn — the quality of your schemas matters more than their quantity.
To situate Needle in the broader local landscape, see our selection of the best local LLMs of October 2026 and the guide to running AI agents locally with Ollama.
What Needle Reveals About the Direction of AI in 2026
Needle is not an isolated case: it's the most extreme piece of a deeper trend — the extreme specialization of small models for agentic workloads.
Proof lies in the recent ecosystem: a 169 MB speech-to-text model that beats Whisper Base, an open-weights voice model priced at $0.0025 per minute, the 2-bit ternary wave. And on the agents side, Laya offers a 421M-parameter "decision model" to replace the LLM in your agents — same philosophy: the right model for the right job.
On the other side, frameworks are getting organized: Vercel eve wants to do for agents what Next.js did for the web. When orchestration becomes standardized at the top of the stack, the execution layer specializes at the bottom. Needle is that layer.
My bet: the agentic stack of 2027 will look like a sandwich — massive cloud reasoners on top, embedded executors of a few dozen megabytes at the bottom, and a trust protocol in between. Needle's calibrated trust score is a serious candidate for that protocol.
Two unknowns remain: the engram's resilience to large-scale fine-tuning, and the ecosystem's ability to write clean tool schemas — because a structural constraint is only as good as the quality of what it constrains.
❌ Common Mistakes
Mistake 1: Believing that Needle replaces a conversational LLM
Needle is a tool-calling model, not a general-purpose assistant. Asking it to write or analyze means using it outside its domain. The solution: treat it as one member of an agent stack, with a cloud or local LLM on top for reasoning.
Mistake 2: Comparing file sizes without accounting for precision
Needle 2's 14 MB are in CQ2-bit (2 bits), FunctionGemma's ~540 MB in f16 (16 bits). The size ratio includes an 8× compression factor. The solution: benchmark on your own tools and your own devices, not on the README numbers.
Mistake 3: Neglecting the quality of tool schemas
The grammar is compiled from your schemas: an ambiguous schema produces constrained but poor behavior. And the retrieval head only returns 5 tools per turn. The solution: clean schemas, precise descriptions, and curated rather than exhaustive tooling.
Mistake 4: Deploying at the default depth without profiling
The ladder from 2 to 20 layers is a deployment parameter, not an implementation detail. The solution: profile per device, start low, and increase the depth until the confidence score stabilizes on your real-world cases.
❓ Frequently Asked Questions
Is Needle really open source?
Yes. The code is Apache-2.0 licensed, the Needle 3 weights are freely downloadable on Hugging Face, and the architecture paper has been public on arXiv since July 2026. Nothing is gated behind a paid offering, which remains rare for a project of this ambition.
What's the difference between Needle 2 and Needle 3?
Needle 2: 45M parameters, 14 MB binary, full session in ~28 MB of RAM, CQ2-bit compression. Needle 3: 121M parameters (arithmetic efficiency of a 50M), 8 to 29 MB file, ladder of 2 to 20 deployable layers, and engines under 1 MB per platform.
Can Needle really run on a microcontroller?
That's the stated target, with a precompiled engine under 1 MB per platform. The actual RAM budget depends on the depth chosen in the ladder and the number of declared tools: to be validated by profiling on your target, not on the spec sheet.
Does Needle replace GPT-5.5 or Claude Opus 4.7?
No, and that's not the goal. Needle executes tool calls locally; cloud reasoners plan and write. The two are complementary in an agent architecture, with Needle's confidence score serving as the gateway between the two worlds.
How much does Needle cost?
Nothing. Apache-2.0 code, free weights on Hugging Face, engines under 1 MB per platform (October 2026). The only investment is your integration time — and it can be substantial on a real embedded target.
✅ Conclusion
By fitting a reliable tool-calling agent into 14 MB, Needle isn't competing with the cloud giants — it's opening up a front beneath them: the billions of devices that have neither a reliable network nor 2 GB of RAM. Try the browser demo, read the paper, and if you're interested in agents in the broader sense, our selection of the best autonomous AI agents gives you a full tour of the landscape.