The token war has begun: caveman, rtk and OmniRoute, the open source tools that cut 30 to 95% off your LLM bill

The token war has begun: caveman, rtk and OmniRoute, the open source tools that cut 30 to 95% off your LLM bill

🔎 The bill has become the #1 bug of AI development

This week in September 2026, GitHub Trending looks like a finance board meeting. caveman (around 100,000 stars), rtk (around 80,000), headroom (around 70,000) and OmniRoute (around 64,000) dominate the top of aggregators like GitTrend or GitHub Hot. What they have in common: promising to cut 30 to 95% off your token bill.

The trigger can be dated precisely. On June 1, 2026, GitHub switched all Copilot plans to usage-based billing, calculated on tokens at each model's published API rates. At the same time, multi-agent setups multiplied loop iterations — and each iteration re-bills the context. The flat-rate subscription is dead, the variable bill is born.

But here's the twist: the percentages in READMEs don't always survive contact with an independent benchmark. The JetBrains study from July 2026 measures −8.5% for caveman where it claims −65%, and finds that rtk can cost more than it saves. It is precisely this gap between marketing and measurement that this article dissects — tool by tool, number by number.

The essentials

• An entire category of open source token-saving tools dominated GitHub Trending in September 2026: caveman (~100,000 stars), rtk (~80,000), headroom (~70,000), OmniRoute (~64,000).
• The trigger: Copilot switched to token billing on June 1, 2026, and multi-agent agentic sessions are sending consumption through the roof.
• The advertised claims (−60 to −96%) don't always hold up to measurement: JetBrains measured −8.5% for caveman on real agent work, and +7.6% more cost for rtk at low reasoning effort.
• Real gains do exist, but they're localized: output tokens (caveman), command outputs (rtk), browsing compressed ×129 (Caveman 2), JSON (headroom).
• OmniRoute attacks the problem through unit pricing: 352 providers, 90+ free tiers, automatic fallback — the most durable lever in the category.
• Golden rule: only trust the tokens reported by your provider, never the tool's self-reported counter.

Recommended tools

June 1, 2026: The Day the Subscription Became a Bill

If this category explodes precisely in September 2026, it's because on June 1, GitHub turned millions of Copilot subscribers into token consumers billed by the unit.

The mechanism, announced on April 27, 2026 by Mario Rodriguez on the GitHub blog: premium request units disappear in favor of GitHub AI Credits, whose consumption is calculated on tokens (input, output, cached) at each model's published API rates. Code completions remain unlimited and outside of credits. Fallback models, meanwhile, are gone: once credits run out, it's admin budgets that rule.

Subscription prices haven't changed, but credits were bumped up after user feedback, with a new Max plan announced on May 12, 2026:

Why this changes everything: the same prompt can cost ×20 depending on the model. GPT-5 mini is billed at $0.25/M input tokens versus $5.00/M for GPT-5.5; on Anthropic's side, Claude Haiku 4.5 sits at $1.00/M input versus $5.00/M for Claude Opus 4.8, according to The New Stack (June 2, 2026). Copilot's auto mode now routes to the model best suited to the task — the first weapon in the token war is a router, not a compressor.

InfoWorld points out that this is Copilot's second recalibration in under a year: premium requests date back to June 2025 (300/month on Pro, 1,000 on Enterprise, overage at $0.04). The logic is merciless — GitHub was absorbing inference costs that had become unsustainable with increasingly long agentic sessions.

And multi-agent setups make everything worse. Each agent re-reads the context, and some of these hidden re-reads are billed at one-tenth of the price — but they structure the bill. This is exactly where this week's tools claim to step in.

For the details of the switchover, our deep dive GitHub Copilot switches to token billing: the end of the subscription, the start of usage-based billing remains the best starting point.

caveman: ~100,000 stars to make your agents speak in telegraphese

caveman is an MIT-licensed skill that forces your agents to write in telegraphic style — and when it comes to output tokens, the mechanics are real.

JuliusBrussee's project targets 30+ agents: Claude Code, Codex, Gemini, Cursor, Windsurf, Cline, Copilot, and many more. The example in the README speaks for itself: 69 tokens reduced to 19, a −72% reduction.

The in-house benchmark, run on 10 prompts via the real Claude API, shows an average of −65% on output tokens, with a range of 22 to 87%. So far, so good.

But the README is honest about the structural limitation: the skill reduces only output tokens. Input and reasoning tokens remain untouched — and the skill itself adds roughly 1,000 to 1,500 input tokens per turn. On an agent bill dominated by input, the overall arithmetic therefore promises to look far less flattering than −65%.

Caveman 2 adds a more interesting piece: a local proxy that attacks the input. On a Claude Code benchmark pinned at 54 runs — with tokens reported by the provider, not estimated — it claims −33.2% input tokens while passing the 18 exact-answer checks. And compressed browsing is spectacular: 121 tokens versus 15,704 in the Playwright ARIA baseline, i.e., 129.8× smaller.

Around the project, a micro-category has sprung up: simplicio (up to 96% tokens claimed, a single Rust binary), cavemem (persistent cross-agent memory, whose core has been merged into caveman), cavegemma (a LoRA of Gemma 4 31B that speaks caveman natively — to be compared with our pick of the best LLMs to run locally), genshijin (a Japanese version), or ruslanlap/cavemenko (−65-90% claimed, in Ukrainian). The GitHub topics/caveman page gives a sense of the phenomenon's scale.

My take: this is the cheapest compression in the world — no infra, just prompt discipline. But keep −8.5% in mind, not −65%. We'll get to that.

rtk: the Rust proxy that cleans up your outputs before they reach the context

rtk intercepts the output of your commands (git status, tests, logs) and compresses it before it reaches the LLM's context. The idea is sound, the implementation is clean — the accounting, however, will be a problem.

On the engineering side, it's serious stuff: a Rust CLI proxy, a single dependency-free binary, over 100 supported commands, overhead under 10 ms. Claude Code's PreToolUse hook automatically redirects calls:

# Claude Code's PreToolUse hook automatically rewrites:
git status
# into:
rtk git status

The project claims a 60-90% reduction on dev command outputs, and supports 14 agents: Claude Code, Copilot, Gemini CLI, Codex, Cursor, Windsurf, Cline, Kilo Code, Antigravity, Kimi, Pi, OMP, Droid and Hermes.

Two limitations to know before installing. First, Claude Code's native Read/Grep/Glob tools don't go through the hook — yet that's precisely where most of a code agent's context lives. Second, the "saved" tokens are estimated as bytes÷4, without a tokenizer.

To its credit, the README isn't blowing smoke: it admits itself that cutting bash output isn't cutting 90% of the bill, bash being only one component of input tokens, themselves part of the bill. It's this honesty that makes the following benchmark so instructive.

−65% claimed, −8.5% measured: the JetBrains benchmark that pours cold water on it

On real agent work, the advertised gains melt away: caveman goes from −65% claimed to −8.5% measured, and rtk does even worse with +7.6% more cost at low reasoning effort.

The JetBrains benchmark, published on July 20, 2026, is an independent A/B series run on Claude Code 2.1.201 with claude-sonnet-5, at low and high reasoning efforts, measured on SkillsBench. No self-reporting: actually billed tokens.

Results. caveman, advertised at −65%, measures −8.5% on real agent work. rtk, advertised at 60-90%, measures +7.6% in additional cost at low effort (p=0.004, so statistically significant) and ±0% at high effort. In both cases, task quality remains unchanged — these tools don't break anything, they simply don't always do what their counters claim.

rtk's error mechanism is a masterclass in phantom accounting. The tool counts the full raw output as the counterfactual: a cat of a 1.2 MB CSV logs 320,000 tokens "saved" when Claude Code truncates well before that. It estimates tokens as characters÷4 without a tokenizer. And it ignores that most of the input cost comes from hidden re-reads, billed at a tenth of the price.

The kicker: during the study, rtk self-reported 96.2 million tokens "saved" — 99.8% of everything it touched — while the measured bill, for its part, kept climbing.

Important nuance: −8.5% measured and free is small but real. And at high effort, rtk is neutral — not harmful, just useless. The real lesson is methodological: these tools measure a counterfactual, not your bill. The only metric that matters is the token reported by the provider.

It's also worth distinguishing the gain regimes. Caveman 2's compressed browsing (×129 on a specific cost line item) is not the same sport as the global telegraphic style. Compress where it's big and measurable: that's the lesson.

OmniRoute: if you can't reduce tokens, reduce their price

OmniRoute attacks the problem from the other end: a self-hosted gateway that routes every call to the cheapest available provider, with automatic fallback when a quota blows up.

diegosouzapw's project, MIT-licensed, exposes a single OpenAI-compatible endpoint in front of 352 providers (in v3.8.50, up from 290 in v3.8.49 — the growth rate is itself a market signal), including 90+ free tiers, covering roughly 516 models: Claude, GPT, Gemini, DeepSeek, Kimi, GLM, Mistral, MiniMax. Pinggy's analysis sums up the use case: pooling free quotas and routing based on rate limits.

The CLI cuts through the complexity:

omniroute doctor          # diagnose the configuration
omniroute providers test  # test the available providers
omniroute quota           # quota status
omniroute combos switch   # switch provider combo

It's the natural extension of two stories we've been following closely: the open source LLM war — DeepSeek, Kimi, and GLM are routing targets far cheaper than frontier models — and our roundup of the best free LLMs, whose quotas OmniRoute finally lets you pool properly.

On the deployment side, it's a self-hosted service: a small VPS will do (for example at Hostinger, from a few euros a month as of September 2026 — check hostinger.com).

A warning is in order, though. Among the no-auth providers, you'll find reverse-engineered public chatbots — Chipotle Pepper AI, The Old LLM via Playwright auto-generated tokens. It's fragile, legally gray, and strictly off-limits for production. The official gateway is solid; the circumvention part is a grenade with the pin pulled out.

My take: this is the most durable piece in the category. Routing survives the disappointment of compression tools because it plays on unit price — the only lever no independent benchmark can contradict.

headroom and the category: the stars vote, the benchmarks decide

headroom rounds out the lineup by compressing tool outputs and RAG chunks — but the criticism leveled at it on Hacker News sums up the tension across the entire category.

The project, with around 70,000 stars, claims 20% compression on code and 60-95% on JSON, with no degradation in responses. Figures to be taken as claims: I couldn't find any independent benchmark to verify them — and that's the whole problem with this wave.

On Hacker News, the caveman vs headroom debate has settled along the lines of mood: headroom is criticized as invasive and subscription-based. In a category where the competitors are MIT-licensed and free, a subscription for compression that hasn't been independently verified is a nearly disqualifying handicap.

The phenomenon extends well beyond these four projects. The aggregators — agents-radar #130, GitHub Hot, GitTrend, the StartupCorners DevTools Digest — confirm that token savings dominate the trending charts. Developers vote with their stars: the star measures budget anxiety, not actual savings.

And this wave is part of a broader agentic open source spring: OpenHuman, the agent that knows your life before you've even spoken to it (our analysis), or Life-Harness, which promises +88.5% agentic performance without retraining (our feature). Token savings are just one front of the open source battle — the rest is in our roundup of AI news.

What Actually Works: Four Levers, Ranked by ROI

Lever #1 isn't a proxy: it's model routing. The price delta between GPT-5 mini ($0.25/M) and GPT-5.5 ($5.00/M) is ×20 — no compression comes close.

Here are the four levers, from most profitable to most cosmetic:

1. Route to the right model. Copilot's auto mode now does this natively; otherwise, a gateway like OmniRoute. Send subtasks to the cheap models (Haiku 4.5 at $1/M input) and keep the frontier models for what truly deserves them. Our monthly best LLM comparison is made for this, and open source coding is progressing fast — see our deep dive on Kimi K2.7 Code, the 1T-parameter model announcing −30% tokens.
2. Compress where it's big and measurable. Browsing (×129 with Caveman 2), RAG JSON (headroom, to be verified), command outputs (rtk on bash).
3. Telegraphic output. caveman on output tokens: real, but modest on the total bill (−8.5% as measured by JetBrains).
4. Optimize agent runtime. Life-Harness shows you can gain performance without touching the token budget.

The golden rule runs through everything: measure in tokens reported by the provider, on a benchmark pinned to your own tasks. Caveman 2 did it (54 runs, provider tokens); rtk didn't (96.2 million ghost tokens). The difference between these two methods is the difference between actual savings and a belief.

❌ Common Mistakes

Mistake 1: believing the tool's counter

rtk auto-reported 96.2M "saved" tokens (99.8% of what it touched) while the measured bill was going up (JetBrains, July 2026). Internal counters estimate with bytes÷4 and reason counterfactually. Solution: tokens reported by the provider, pinned A/B tests, before/after on your real tasks.

Mistake 2: forgetting that input dominates the bill

caveman only reduces output and adds 1-1.5k input tokens per turn; hidden re-reads, billed at one tenth of the price, shape the cost. Solution: context hygiene, targeted compression (browsing, JSON) and routing to cheaper models.

Mistake 3: hooking up a proxy at low reasoning effort

At low effort, rtk costs +7.6% more (p=0.004); at high effort, it's neutral. Solution: measure under your actual regime before deploying — a savings proxy can cost more than it saves depending on your setup.

Mistake 4: putting reverse-engineered providers into production

OmniRoute's no-auth providers that reverse-engineer public chatbots (Chipotle Pepper AI, The Old LLM) are fragile and legally gray. Solution: official API keys and documented free tiers; reverse-engineering remains personal tinkering, not infrastructure.

❓ Frequently Asked Questions

Do these tools work outside of Claude Code?

Yes, for the most part. caveman targets 30+ agents (Codex, Gemini, Cursor, Windsurf, Cline, Copilot...), rtk supports 14 agents including Gemini CLI, Codex, Cursor, and Kilo Code, and OmniRoute plugs into any OpenAI-compatible client. One caveat though: Claude Code's native Read/Grep/Glob tools bypass rtk's hook.

How much can you really save?

Expect 10-30% on a real agent bill, not 90%. Independent measurements show −8.5% for caveman on real work, ±0% for rtk at high effort. The massive gains exist but are localized: browsing ×129, JSON compressed by 60-95% (claimed). Model routing remains the biggest lever.

Does response quality drop with these tools?

No, based on the available measurements. The JetBrains benchmark finds unchanged task quality for both caveman and rtk, and Caveman 2 passes 18/18 exact-answer checks across 54 runs. But these benchmarks don't cover your domain: test on your own tasks before generalizing.

Is headroom worth a subscription?

That's the doubt raised on Hacker News: headroom is criticized as invasive and subscription-based, in a category that is shifting toward free MIT open source. With caveman, rtk, and OmniRoute free, it's hard to justify a subscription for compression that has no independent benchmark to its name.

Is OmniRoute safe and legal?

The self-hosted MIT gateway poses no particular problem: your keys, your server, your logs. On the other hand, its no-auth providers based on reverse-engineering public chatbots are fragile and legally gray. For production, stick with official API keys and documented free tiers.

✅ Conclusion

The token war isn't won with GitHub stars but with the tokens reported by your provider: route to the right model, compress where it's big and measurable, and beware of any counter that promises 90%. To choose the right model for each task, start with our monthly comparison of the best LLMs.