Claude generates 180 million lines of formal proofs: 123 open-source libraries verified at the RISC-V assembly level

Claude generates 180 million lines of formal proofs for 123 open-source libraries verified at the RISC-V assembly level

🔎 Why this is a gamechanger

Anthropic and Theorem have just reached a milestone that the industry has been waiting for decades: industrial-scale, AI-driven formal verification on real production code. Published on October 22nd, the experiment covers 123 open-source libraries, 239 entry points from OSS-Fuzz, and produced approximately 180 million lines of Rocq (Coq) proofs—the equivalent of several Linux kernels in volume of mechanically verified specifications. The Rocq kernel re-verifies every proof automatically. Almost no human reviewer is involved. Along the way, Claude corrected bugs that trillions of fuzzing runs had missed, then proved the absence of crashes for any input up to 64 MB. This is the first time formal verification has left the lab to touch delivered software verification, compiled to RISC-V, with a level of automation that makes it cost-effective.

The Essentials

• 180 million Rocq proof lines generated in a few weeks for 123 OSS-Fuzz libraries, verified at the RISC-V assembly level.
• 239 entry points covered, proven property: crash-freedom (absence of crashing) for any input ≤ 64 MB.
• Bugs discovered and patched by Claude that massive fuzzing (trillions of executions) had missed.
• Rocq kernel as a trusted third party: each proof is mechanically re-verified, without significant human review.
• Assumed limitations: crash-freedom scope only (no functional correction), RISC-V compilation, verification independence is questionable.

Recommended Tools

The Anthropic × Theorem Experience: What Happened

Anthropic collaborated with Theorem, a startup specializing in automated formal verification, to apply Claude to 123 C/C++ libraries from OSS-Fuzz—Google's continuous fuzzing program that has been running on thousands of cores for years. The result: 239 entry points (public APIs, parsers, decoders) covered by crash-freedom proofs at the RISC-V assembly level, guaranteeing that no input of 64 MB or less can trigger a crash (null dereference, buffer overflow, division by zero, failed assertion).

The volume is unprecedented: ~180 million lines of Rocq code generated. To give you an idea of the scale, a typical proof represents ~600,000 lines of specifications and lemmas. All of this in "a few weeks" according to Anthropic, with human intervention reduced to validating the compilation chain and auditing samples. The Rocq kernel—the minimal trusted computing base of the proof assistant—mechanically re-verifies every certificate. If the kernel says "valid," the proof holds. Period.

And Claude doesn't just produce proofs: it identifies and fixes bugs along the way. Bugs that trillions of fuzzing executions had not discovered. Once patched, the proofs pass. This is a paradigm shift: the AI doesn't just verify, it repairs in order to prove.

Why the RISC-V assembly level changes everything

Most formal verification work (CompCert, seL4, IronFleet) targets source code or an intermediate language. Here, the proof is on the RISC-V assembly generated by the compiler. This eliminates an entire class of vulnerabilities: compiler optimization bugs, undefined C/C++ behaviors that the behaviors that the compiler "silently exploits," loop transformations that break invariants.

Concretely, the chain of trust is: source code → GCC/Clang compilation → RISC-V binary → assembly specification → Rocq proof. If the proof passes, the exact binary that will execute is covered. This is what Anthropic calls "first of its kind" at this scale.

The choice of RISC-V is not trivial: open ISA, existing formal specification (riscv-isa-formal), and growing adoption in embedded and cloud. But this also poses a limit: the proof is only valid for this compilation target only. Recompiling for x86-64 or ARM64 invalidates the certificate — unless the proof is redone on the target assembly.

What "crash-freedom" Guarantees (and Doesn't)

The proven property is the absence of crashes for any input ≤ 64 MB. This covers:
- Invalid memory access (null dereferencing, out-of-bounds, use-after-free)
- Unmanaged arithmetic overflows
- Division by zero
- abort() assertions reached
- Undefined C/C++ behaviors leading to a hardware trap

What it does not guarantee:
- Functional correctness: the JSON parser may not crash but return an incorrect structure.
- Absence of memory leaks: malloc without free does not crash.
- Temporal properties: no proof of termination (infinite loops (infinite loops possible).
- Business security: authentication bypass, privilege escalation, logical corruption.

Anthropic explicitly states: it is a proof of safety, not correctness. The distinction is crucial. A crash-free XML validator can accept malformed XML and corrupt the downstream application state. Full formal verification (e.g., seL4, CompCert) requires functional specifications — which multiplies the effort by 10 to 100.

Claude's Role: Proof Generator, Not Oracle

Claude does not "know" if the code is correct. It receives: (1) the RISC-V binary, (2) the crash-freedom specification in Rocq, (3) the Theorem API to interact with the proof assistant. It proposes lemmas, constructs loop invariants, attempts lia, omega, auto tactics, and iterates until the Rocq kernel validates.

This is LLM-guided proof search. Theorem provides the environment (CFG extraction, VC generation, Rocq interface), and Claude performs proof engineering. The success rate is not public, but the volume (180 M lines) suggests massive automation with few blocking failures.

This is reminiscent of the Anthropic report on Codex, where 53-58% of bugs were found but with significant failures—here, the proof/repair iteration seems to converge more reliably thanks to the binary feedback from the Rocq kernel. See the Anthropic report on Codex for the comparison.

The bugs that fuzzing missed

This is the most striking point. OSS-Fuzz has been running trillions of inputs on these libraries for years. Yet Claude found bugs reachable only through complex paths that fuzzing doesn't explore deeply enough: boundary conditions in nested loops, interactions between multiple parsers, edge cases of the RISC-V specification (e.g., unaligned instructions, reserved registers).

A concrete example cited by AI Weekly: an image decoder where a while loop with condition ptr < end compiled to assembly as an unsigned comparison that fuzzing had never managed to overflow. Claude produced the loop invariant, Rocq verified it, the bug was patched, the proof passed.

This isn't magic: fuzzing explores randomly, while proof explores the state space systematically via invariants. The two are complementary. But AI makes proof cheap enough to deploy where fuzzing saturates.

The chain of trust: Rocq as an impartial third party

Everything rests on the Rocq kernel (~2000 lines of OCaml). This kernel verifies each proof certificate independently of Theorem, of Anthropic, of Claude. If the kernel accepts, the proof is valid — regardless of who generated it. That's the fundamental difference from "the AI says it's fine."

In practice, the chain is:
1. Theorem extracts the control flow graph (CFR) from the RISC-V binary
2. Generates verification conditions (VCs) for crash-freedom
3. Claude proposes a proof (a sequence of Rocq tactics)
4. The Rocq kernel re-verifies the final certificate
5. If OK → signed artifact (hash of the binary + proof)

No human reads the 180M lines. Samples are audited. Trust is delegated to the kernel. This is the same architecture as CompCert or seL4, but at a scale 1000x larger thanks to LLM automation.

Honest limits: what Anthropic doesn't solve

1. Independence of verification

Theorem is a partner of Anthropic. Claude is Anthropic's model. The proof is generated by the actor that provides the model. No independent third party has re-verified the whole thing. The Rocq kernel is impartial, but the construction of the proof (choice of invariants, lemmas, abstractions) is done by Claude. A too-weak invariant passes the kernel but doesn't prove the intended property — except that here the property (crash-freedom) is encoded in the VCs generated before Claude's intervention. So the risk is low, but not zero.

2. Unverified compilation

The proof covers the RISC-V assembly produced by GCC/Clang. The compiler is not verified (unlike CompCert). A compiler bug could introduce a crash in the binary that doesn't exist in the proof. Low probability, potentially high impact.

3. Incomplete specification

Crash-freedom ≠ security. A parsing library proven crash-free can still be vulnerable to logic attacks (e.g., billion laughs, hash flooding, type confusion). The proof doesn't cover business logic semantics.

4. Hardware threat model

The proof assumes a standard RISC-V model. No hidden channels, no Spectre/Meltdown, no Rowhammer. This is software, not hardware.

5. Maintenance scalability

When the library changes (new version, patch), the proof must be regenerated. Anthropic hasn't published a CI/CD pipeline to maintain the proofs continuously. This is a one-off proof, not a continuous process.

Comparison: where this stands relative to existing work

The innovation is not formal verification — that has existed for 40 years. The innovation is LLM-driven automation of proof engineering, which brings the cost below the break-even threshold for "ordinary" code (not safety-critical).

Implications for the Software Industry

Short term (6-18 months)

• Supply chain adoption: crash-freedom proofs for critical dependencies (zlib, libpng, openssl, sqlite).
• CI/CD with proofs: Rocq certificate generation as a build artifact, verification in the pipeline.
• Software insurance: reduced premiums for proven crash-free components.

Medium term (2-5 years)

• LLM-assisted functional specifications: beyond crash-freedom, business pre/post-conditions.
• Verified compilers by default: pressure to adopt CompCert or formally verify GCC/Clang.
• Regulation: formal proof requirements for critical software (medical, automotive, finance).

Long term

• The end of "testing as the only safety net": tests become examples; proofs become the guarantee.
• AI that writes + proves + maintains: a closed loop from specification → code → proof → deployment.

The open source ecosystem: the counterattack begins

While Anthropic/Theorem close up their code (Theorem is proprietary, Claude is closed), open source is organizing. OpenCode (8M devs, 172K ⭐) is exploring the local coding agent. Qwen3-Coder-Next (80B MoE, 3B active) rivals Claude Sonnet on code. Odysseus from PewDiePie offers a self-hosted AI workspace.

For formal verification, projects like Lean 4 + Mathlib, F* (Fstar), Dafny already integrate LLM backends. The race is on: who will be the first to have an entirely open pipeline (model + proof tool + CI) for crash-freedom at OSS-Fuzz scale?

❌ Common Mistakes

Mistake 1: Confusing crash-freedom with correctness

What's wrong: Saying "this library is proven correct". It is proven not to crash. A JSON parser that returns null instead of an error on invalid input is crash-free but incorrect.
Solution: Always specify the proven property. Ask for the Rocq certificate and inspect it (or have it audited).

Mistake 2: Believing the proof covers the source code

What's wrong: Applying the proof to the GitHub repo. The proof covers the RISC-V binary compiled with specific flags. Recompiling differently invalidates the guarantee.
Solution: Distribute the proven binary + its proof, or lock down the toolchain (reproducible Docker image).

Mistake 3: Neglecting the extended TCB

What's wrong: Thinking only the Rocq kernel matters. The chain includes: CFG extractor (Theorem), VC generator, compiler, RISC-V specification, memory model.
Solution: Audit each link or require end-to-end proofs (e.g., CompCert + VST).

Mistake 4: Underestimating the maintenance cost

What's wrong: Integrating a one-off proof without budgeting to update it at every release.
Solution: Negotiate a proof-regeneration SLA with the vendor, or internalize the expertise.

❓ Frequently Asked Questions

Will this technology replace unit tests?

No. Tests remain essential for functional correction, business regression, and performance. Proof of crash-freedom proof complements tests by covering the infinity of inputs where tests sample.

Can I apply it to my Rust/Go/Python code?

Not directly. The target experience is C/C++ compiled to RISC-V. Rust has memory guarantees at the source level (borrow checker) but not automated assembly proof at this scale. Go/Python have a runtime (GC) which complicates machine-level proof. This is an active area of research.

Can the Rocq kernel have bugs?

Theoretically, yes. Practically, the kernel is minuscule (~2000 lines), formally specified formally, and has been used in critical production since 20 years (CompCert, seL4, CertiKOS). The probability of a kernel bug that validates a false crash-freedom proof is negligible compared to the risk of a bug in unverified code.

How much does it cost?

Anthropic does not publish a price list. Theorem bills per project. Rough estimate: $10k-$100k per library depending on complexity, versus $1M+ for equivalent manual verification. ROI is achieved on critical components with high exposure.

Is it usable in production todaytoday*?

Yes, for critical C/C++ components (parsers, codecs, crypto) compiled for RISC-V. The Theorem + Claude chain is operational. For other targets or languages: not yet packaged.

✅ Conclusion

Anthropic and Theorem have just demonstrated that formal verification at industrial scale has become economical thanks to LLMs. 180 million lines of Rocq proofs, 123 libraries, 239 entry points, down to RISC-V assembly level, in a matter of weeks. This is no longer research — it's a service.

The limitation is clear: crash-freedom only, RISC-V compilation, trust in the Theorem/Claude chain. But the threshold has been crossed. From now on, not proving critical components exposed to untrusted inputs becomes a deliberate risk decision, no longer a technical inevitability.

For teams looking to explore self-hosting coding and proof agents, AI agents with Ollama offer a local starting point. To compare models on code and proof generation, see our comparison of the best LLMs for coding and Claude vs ChatGPT.