🔍 Read the full analysis: What The Move From Fuzz Tests To Proofs Says About Claude on ThorstenMeyerAI.com
Get the latest gadgets delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
Anthropic’s Frontier Red Team and Theorem say Claude proved a no-crash property for 239 fuzz-test entry points across 123 open-source libraries, using compiled RISC-V binaries and the Rocq proof checker. The work also found bugs, but the proofs apply only to specific entry points and modeled builds—not to every version or configuration of the libraries.
Anthropic’s Frontier Red Team and collaborator Theorem reported on October 22 that Claude proved a defined no-crash property for 239 fuzz-test entry points in 123 open-source libraries. The work used compiled RISC-V programs and the Rocq proof checker; the teams also reported finding bugs that earlier fuzzing had not reached. The result is a large-scale demonstration of AI-assisted formal verification, not proof that the libraries are bug-free.
The property covered by the proofs is specific: for every input under 64 megabytes, the compiled program does not crash because of an AddressSanitizer memory error or a failed assertion, subject to the researchers’ stated assumptions. The proof applies to the modeled program and entry point, rather than to every possible use of the library or every build distributed to users.
The pipeline began with existing fuzz tests. The team compiled each library and test into an instrumented RISC-V executable, then used formal models of RISC-V instructions and Linux processes to define the property. Claude generated proofs in Rocq, a proof-assistant system; its checker determines whether a proof follows from the specification. When an attempt failed, the failure could point to a bug, after which Claude could patch the code and try again.
Anthropic and Theorem reported that the resulting proofs contain about 180 million lines of Rocq. A typical proof was about 600,000 lines, and the median binary contained 18,000 RISC-V instructions. The teams said hundreds of bugs were found, including cases involving unusually large or structured inputs and sequences of calls that ordinary fuzzing had not exercised. The report says the process also rediscovered CVE-2026-56132 in libexpat.
Infographic 01 · From test to proof
Existing fuzz tests became specifications. Claude proved them.
Over a few weeks, Claude proved at the machine-code level that hundreds of real fuzz entry points can no longer crash, after patching the bugs it found.
The pipeline
From an existing test to a checked proof
- Compile. The library and its existing fuzz test become a RISC-V executable, instrumented with AddressSanitizer.
- Specify. Theorem’s RISC-V and Linux process semantics plus the no-crash property mechanically yield the specification.
- Prove. Claude writes the proof in Rocq. A failed attempt points at a possible bug; Claude patches it and proves the new spec.
- Check. The Rocq proof checker verifies the proof. Virtually none of the proof needs human review.
Fuzzing vs. proof
Same entry point, different guarantee
| Fuzzing | Proof | |
|---|---|---|
| Inputs | Fuzzing: Generated samples, typically capped at 1 MB | Proof: Every input under 64 MB |
| Paths | Fuzzing: Guided by code coverage, no guarantee | Proof: Every path reachable from the entry point |
| Result | Fuzzing: Bugs found so far | Proof: No crash left to find on the modelled system |
Typical proof: about 600,000 lines for a binary with a median length of 18,000 RISC-V instructions.
“If Claude can prove this spec, the fuzz test on RISC-V becomes superfluous.”Anthropic, 22 Oct 2026
Source: Anthropic, “LLMs are making formal software verification feasible at a much larger scale” (22 Oct 2026).
Powered by Thorsten Meyer AI
From Fuzzing to Exhaustive Checks
Fuzzing tests a program with many generated inputs, but it cannot establish that every possible path has been tested. A successful proof of the stated property instead covers all inputs within the specification and assumptions. That difference can matter when a bug depends on a rare input layout, a long sequence of operations or an input larger than typical fuzzing limits.
The work also speaks to the cost of formal verification. Anthropic’s account says proofs can consume hundreds of millions of tokens, making cost and runtime major constraints. The teams reported that newer models and reuse of shared proofs can reduce the effort, but did not provide a general cost per library in the supplied account. If those costs fall, proof-based checks may become practical for more software; this project alone does not establish that they already are.
For developers and users, the immediate value is narrower but concrete: a proof can offer stronger assurance about a defined property than test coverage alone. It does not certify a whole product, eliminate non-crashing logic flaws, or guarantee that another build behaves the same way.
formal verification software tools
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
What the Proofs Actually Cover
The project uses developers’ existing fuzz tests as the starting point for specifications. A fuzz test identifies an entry point and treats certain outcomes—such as an assertion failure or instrumented memory error—as a crash. The team formalized the claim that no such failure occurs for the modeled program for inputs below the stated size limit.
This avoids requiring researchers to write a complete specification for what an entire library should do. But it also limits the conclusion: a no-crash proof is not a general correctness proof. Some tests check stronger behavior, such as whether decompressing compressed data returns the original input; in those cases, proving the test cannot fail can establish that particular property too.
The reported proofs target single-threaded, static RISC-V binaries linked against musl. The source account says most users instead encounter x86 or ARM builds linked against glibc. Differences between those builds mean the reported results cannot automatically be transferred to common production configurations. Anthropic says it is publishing proofs that did not require code patches, some previously unpublished trusted components, and specifications for tests it could not prove.
As an affiliate, we earn on qualifying purchases.
Where the Guarantees Stop
The proofs establish only the properties written into their specifications, under assumptions encoded in the models. They cover selected fuzz-test entry points and particular compiled binaries, not every function, input condition or deployment of each library. A program can also contain a logic bug that does not cause a crash; the source report describes a file-loading restriction that was not always enforced.
The guarantees depend on trusted components, including the formal models of RISC-V and Linux processes, the theorem statements, the Rocq checker, and supporting hardware, operating-system and compiler elements. The source material does not provide enough detail to assess how independently each component was audited or to compare the project’s full compute cost with conventional verification. Nor does it establish whether the reported findings have all been fixed and released in upstream library versions.
As an affiliate, we earn on qualifying purchases.
Publication and Independent Review
Anthropic says it is making available proofs that required no code patch, trusted components that were not already public, and specifications for fuzz tests the system could not prove. Those materials should let researchers inspect the formal statements and check how far the results extend. The source account does not give a publication schedule for each item or a timeline for upstream fixes to bugs found during the work.
The next useful measures will be whether independent reviewers can reproduce the proofs, whether maintainers patch and release the reported bugs, and how the approach performs on other architectures and software configurations. Until then, the report is evidence that Claude can help produce machine-checked proofs at scale in a constrained setting—not evidence that general-purpose software has been formally certified.
As an affiliate, we earn on qualifying purchases.
Key Questions
What did Claude prove?
It proved that specified fuzz-test entry points in the modeled binaries do not trigger an AddressSanitizer memory error or failed assertion for inputs under 64 megabytes, under the project’s stated assumptions. The result is limited to those properties and builds.
Does this mean the 123 libraries are bug-free?
No. The proofs cover selected entry points and defined properties, not every behavior or configuration. The report also describes a logic bug that did not cause a crash, illustrating why a no-crash proof is not a complete correctness guarantee.
How did the researchers check Claude’s proofs?
Claude generated proofs in Rocq, and Rocq’s proof checker verified them against formal specifications. The approach still relies on trusted models, theorem statements and supporting components.
Can the results be applied to software people use now?
Not automatically. The reported work covers single-threaded, static RISC-V binaries linked against musl. The source says common builds often use x86 or ARM with glibc, so separate evidence would be needed for those configurations.
What happens after the report?
Anthropic says it plans to publish selected proofs and related materials. Independent review, reproducibility, bug fixes by library maintainers and testing on other build configurations remain important next steps.
Source: ThorstenMeyerAI.com
Halloween Picks
halloween
As an affiliate, we earn on qualifying purchases.
