Infinity: AWS and UC Berkeley suspend 75,000 agents in 8GB — the serverless runtime where a waiting agent costs nothing

Infinity: AWS and UC Berkeley suspend 75,000 agents in 8 GB — the serverless runtime where a waiting agent costs nothing

🔎 An AI agent spends most of its life waiting. And you're paying for that dead time.

On October 23, 2025, AWS released Infinity, an open source agent runtime (Apache 2.0) developed with UC Berkeley as part of the Hydro project. The project tackles a blind spot that nobody was addressing head-on: nearly all harnesses — the execution environment that runs an agent — keep a resident process in memory while the agent... waits. Waits for an API response, a human approval, a three-day CI pipeline.

It's a money problem as much as an architecture problem. In the real world, an agent spends the majority of its cycle waiting on tools, not "thinking." Meanwhile, your RAM is occupied, your container is running, your serverless function is burning through its timeout.

Why now? Because the number of agents is exploding — the coding agent wars, with Grok Build: xAI launches its first coding CLI agent — the coding agent wars intensify, being the latest illustration — and long-running tasks are becoming the norm. The roundup of the best autonomous AI agents keeps filling up with ever more capable harnesses, but the underlying infrastructure hasn't moved. Infinity proposes to move it.

Key takeaways

• The project: Infinity, an open-source agent runtime (Apache 2.0), released on October 23, 2025 by AWS with UC Berkeley, as part of the Hydro project.
• The core idea: every tool call is treated as a suspension point. History is persisted, the agent vanishes from memory, and the result wakes it back up.
• The headline: ~100 KB per idle agent, i.e. "75,000 agents in 8 GB" — arithmetic done on a Raspberry Pi 5, not a measurement.
• The real measurement: 80,000 suspended agents on a single AMD EPYC thread, with a simulated model. With realistic agents (~1 MB of history), capacity drops to ~8,000 agents per 8 GB.
• The architecture: Lambda + SQS + Aurora DSQL, bounded execution slices. An agent waiting on a 3-day CI consumes strictly zero compute.
• The orchestration: spawn_thread / report_to_parent tools, events handled by the child threads.

Recommended tools

The problem: your agents sleep, your bill doesn't

Most harnesses treat waiting on a tool as an active state: the process stays resident, memory stays allocated, the runtime keeps "owning" the agent. Infinity starts from the opposite premise — waiting isn't work, so it shouldn't cost anything.

Consider three mundane cases. A monitoring agent that queries an API every hour. A coding agent waiting for a CI verdict. A research agent waiting for human approval. In all three cases, the classic harness keeps a process and a context in RAM. At scale — 1,000, 10,000 agents — you're allocating hundreds of gigabytes of memory to store, quite literally, nothingness on standby.

Serverless functions alone don't solve this problem: Lambda caps out at 15 minutes per execution. An agent that needs to survive three days must either pay for a permanent container or hack together manual checkpoints. This is exactly the gap Infinity fills, by making suspension a runtime primitive rather than a developer responsibility.

How Infinity works: the tool call as a suspension point

Infinity models agents as stackless coroutines: on every tool call, the agent's minimal state is serialized and persisted, then the process simply disappears.

The technical detail matters. A stackless coroutine has no call stack to preserve — only the essential state, the one that's sufficient to resume execution at the right place. The full cycle:

1. The agent runs in a bounded slice, a slice of computation whose maximum duration is enforced by the runtime.
2. It emits a tool call → the history is persisted, the agent is suspended, the process disappears.
3. The result arrives → an event wakes the agent, which rehydrates and resumes exactly where it left off.

The result: an idle agent costs only its state, roughly 100 KB according to AWS's figures. No process, no thread, no resident RAM. The agent is no longer a running program; it's data that sleeps.

spawn_thread and report_to_parent: delegation as a runtime primitive

Infinity exposes two native tools: spawn_thread, which creates a child agent, and report_to_parent, which sends a result back up to the parent. An important architectural detail: events are handled on the child threads side — the parent doesn't poll in a loop; it's the child threads' slices that process the reports.

It's the same mental model as the sub-agent delegation described in Hermes Agent #14: Task Delegation — orchestrating sub-agents, but pushed down into the infrastructure: orchestration is no longer a framework pattern, it's a runtime primitive, with the "free" suspension that comes with it.

My take: this is the right layer for it. A framework can imitate delegation; only a runtime can make waiting for a sub-agent truly free.

75,000 agents in 8 GB: what the benchmark really says

The headline figure is arithmetic, not a measurement. The measurement is something else — and it remains impressive. Let's break it down, with sourced numbers.

What is measured. AWS ran 80,000 suspended agents on a single thread of an AMD EPYC, with a simulated model — no real LLM calls. This test validates the memory cost of roughly 100 KB per idle agent. Note that the numbers are internally consistent: 80,000 agents × ~100 KB ≈ 8 GB of state. Source: AWS's open source blog.

What is extrapolated. The 8 GB of RAM of a Raspberry Pi 5 divided by ~100 KB gives 75,000 to 80,000 agents. So the "75,000 agents in 8 GB on a Pi" headline is a calculation, not a bench test. ALFW's analysis points this out bluntly.

What isn't said enough. Real agents accumulate tool outputs: logs, JSON dumps, reports. At ~1 MB per agent — already modest for an agent that has actually done work — the Pi's 8 GB only accommodates ~8,000 agents. A factor of 10 between the headline and real-world conditions.

My take: the nuance doesn't take away from the essential point. An idle agent at ~100 KB is a change of magnitude compared to the hundreds of MB of a resident process. But anyone repeating the "75,000 on a Pi" without mentioning the "~8,000 under realistic conditions" is selling a dream. The project itself presents an honest demo; it's up to us — journalists and developers — to relay it as is.

Lambda + SQS + Aurora DSQL: the architecture behind the demo

The Infinity reference deployment is built on three AWS building blocks: Lambda to run bounded slices, SQS to wake up agents, and Aurora DSQL to persist histories.

The flow is simple to describe. A Lambda slice runs the agent up to its next tool call. The state goes into Aurora DSQL, the tool call goes out into the world — API, CI, human. The agent then ceases to exist in memory. When the result comes back, an SQS message triggers a new Lambda slice, which rehydrates the agent from DSQL and resumes execution. Bounded slices also guarantee, as a side effect, that no agent can monopolize a worker: each slice has a time budget.

The economic consequence is the most interesting part. An agent waiting on a CI for three days consumes no compute during those three days — no instance, no idle function, no container. You pay for storage (~100 KB to a few MB) and, at each wake-up, a few milliseconds of Lambda. It's the exact inverse of the "one container per agent" model.

Honesty note: this reference stack is deeply AWS. The runtime is Apache 2.0 and self-hostable, but reproducing the "extreme concurrency" mode elsewhere means rewiring the persistence and wake-up mechanisms yourself. That's not a flaw — it's a choice, and you should know it before committing.

What this changes for agent infrastructure

Infinity moves the agent concurrency bottleneck from a RAM constraint to a storage constraint — and that unlocks workloads that were previously economically impossible.

Three concrete consequences:

1. The marginal cost of an idle agent trends toward zero. Fleets of 10,000 agents for monitoring, supervision, or incident recovery become reasonable to operate.
2. Long-running tasks become normal again. Waiting three days for a CI run, a week for a human review, a month for a domain renewal: no more checkpoint hacks and containers paid to do nothing.
3. The runtime layer is becoming standardized. The movement aligns with recent history: Agent Skills : le framework d'addyosmani qui standardise les workflows des agents IA de codage standardizes workflows, Infinity standardizes execution. When the two meet, an agent becomes a portable artifact — skills on one side, a suspendable runtime on the other.

My blunt take: this is the most interesting piece of agent infrastructure published this fall. Not because the benchmark is flattering — it's honest — but because the primitive is the right one. Frameworks have been fighting over orchestration for two years; the next frontier is the runtime, and today's dominant harnesses will sooner or later have to respond to this kind of project.

Limits: What the Demo Doesn't Tell You

Infinity is promising but very young, and its flagship demo relies on a simulated model. Three caveats apply before you make it your production backbone.

Simulated model ≠ real LLM. The 80,000-agent test doesn't call real models. With actual calls — GPT-5.5, Claude Opus 4.7, Gemini 3 Pro, or an open source model — network latency and API errors change the wake-up dynamics. Not the memory at rest, but the load profile and the cost per wake-up.

Real memory is driven by tool outputs. Histories grow fast: CI logs, verbose API responses, screenshots. Without a pruning policy, the 10x factor between the headline and reality will show up in production.

Maturity. Project released on October 23: potentially unstable API, no production track record, documentation yet to be evaluated. Treat Infinity as a project to watch closely, not as an established standard.

If you'd rather keep control of the entire chain, the alternative remains self-hosting: open source AI agents locally with Ollama on a VPS you control — Hostinger offers VPS plans suited to this use case (check pricing on hostinger.com). You don't get Infinity's free suspension, but you control every byte and every euro.

Trying Infinity in Ten Minutes

The repository is public, and getting started is done through the examples in the README. The bare minimum to get started:

git clone https://github.com/hydro-project/infinity
cd infinity
less README.md

The hydro-project/infinity repository provides agent examples and runtime documentation. Two tips before you dive in: start with a single-tool agent to observe the suspend/wake cycle, then study the handling of bounded slices — that's what guarantees no run goes off the rails. If you're starting from scratch, our guide on how to build an AI agent covers the fundamentals you need to master before choosing a runtime.

❌ Common Mistakes

Mistake 1: Repeating "75,000 agents in 8 GB" as a measured fact

What's wrong with it: it's an arithmetic extrapolation based on ~100 KB per agent, validated on an EPYC thread with a simulated model — not on a Pi with real agents.
The fix: cite both numbers. 80,000 measured (simulated model, EPYC), ~8,000 realistic (1 MB per agent, 8 GB). It's more credible, and more useful to your readers.

Mistake 2: Confusing suspension and memory compression

What's wrong with it: suspension doesn't reduce the size of the history, it removes the process from RAM. A chatty agent with large tool outputs remains a heavyweight agent to persist.
The fix: offload outputs to object storage, systematically prune histories, set an explicit memory budget per agent.

Mistake 3: Migrating your entire fleet to Lambda overnight

What's wrong with it: every wake-up costs something — a Lambda invocation, DSQL reads. An agent that chains short tool calls can cost more in suspend/wake mode than as a resident process.
The fix: profile first. Infinity shines on long and frequent waits, not on agents that fire fifty API calls per minute.

Mistake 4: Forgetting that delegation multiplies state

What's wrong with it: every spawn_thread creates an additional persisted state. A hierarchy of 10 parents × 10 children means 100 histories to manage, wake up, and close out.
The fix: bound delegation depth, trace thread trees, define an end-of-life policy for child threads.

❓ Frequently Asked Questions

Is Infinity really open source?

Yes. The code is published under the Apache 2.0 license in the GitHub repository of the Hydro project, co-developed by AWS and UC Berkeley, and made public on October 23, 2025. You can inspect, modify, and self-host it freely. The AWS reference stack (Lambda, SQS, Aurora DSQL) remains an implementation choice, not a licensing requirement.

Can you really run 75,000 agents on a Raspberry Pi 5?

No one has demonstrated this publicly. The figure is arithmetic (8 GB ÷ ~100 KB) derived from a measured test on AMD EPYC with a simulated model. With realistic agents accumulating ~1 MB of tool outputs, expect closer to ~8,000 agents for 8 GB. The headline is directional, not factual — that's the key point to remember.

Do you need AWS to use Infinity?

No for the runtime, yes for the reference deployment. Infinity is Apache 2.0 and self-hostable, but the documented "extreme concurrency" mode relies on Lambda, SQS, and Aurora DSQL. Reproducing this mode on another cloud or on-premise means implementing the history persistence and wake-up mechanism yourself.

Does Infinity replace LangGraph or agent frameworks?

No, it sits underneath them. Frameworks handle orchestration: step graphs, prompts, application memory. Infinity handles execution: suspension, persistence, wake-up. The two are complementary — a framework could build on this type of runtime to make waiting free without changing its API.

Which LLM models can you use with Infinity?

The runtime is agnostic: the agent calls the model of your choice via API — GPT-5.5, Claude Opus 4.7, Gemini 3 Pro, or a self-hosted open source model. Keep in mind, however, that the published benchmarks rely on a simulated model: latency and cost profiles with real calls remain to be measured in your own context.

✅ Conclusion

By turning every tool call into a suspension point, Infinity (AWS + UC Berkeley, Apache 2.0) drops the cost of an idle agent from "one resident process" to "~100 KB of storage" — and even though the headline figure of 75,000 agents on Pi deserves the context of the ~8,000 realistic agents, the primitive is the right one and it's going to inspire imitators. Clone the repo, try out the suspend/resume cycle, and keep an eye on what the major harnesses will do with it.