AI Model Crushes Software Verification Costs and Uncovers Hidden Flaws

For software developers in the Pacific Northwest and beyond, the long-standing challenge of proving code is bug-free just got significantly cheaper and faster. Previously, mathematically verifying software required years of specialist labor, limiting the process to critical systems like aircraft avionics. Now, a large language model has successfully automated this complex task, producing millions of lines of proof code in a matter of weeks.

The shift hinges on a collaboration between Anthropic and partner Theorem, which utilized the Claude model to verify existing software rather than writing new code. By translating crash tests into mathematical problems and analyzing compiled machine code, the system generated approximately 180 million lines of proof code for 123 open-source libraries. This volume of work, which would have taken a human team 20 person-years to complete for a single small operating system kernel, was achieved in just a few weeks.

A key benefit of this automated approach is its ability to detect errors that conventional testing often misses. During the verification process, the model identified hundreds of bugs, many of which only surfaced under specific conditions such as unusually large input files. In one notable instance, the team found as many security flaws in a widely used PDF reader in a single week as are typically reported for that software in an entire year. The system also caught a logical error that had remained undetected for over a decade.

While the results are promising, the process comes with strict practical limitations. The proofs are only valid under specific conditions, including a particular processor architecture, single-threaded execution, and input sizes under 64 MB. Additionally, the computational cost remains high, with individual proofs requiring hundreds of millions of tokens. However, newer model versions are becoming more efficient, with the latest iteration solving half the proofs using roughly 40 percent fewer tokens than its predecessor.

The method relies on an independent verification program called Rocq to check the proofs, a process Anthropic describes as needing almost no human reviewers. The researchers refer to the workflow as "fully autonomous" proof generation, although the underlying formal foundations were built by human experts at Theorem. This separation of proof generation and verification allows for a more rigorous check on the AI's output.

Looking ahead, this project suggests a future where AI-generated software can be trusted more readily, reducing the time developers spend on manual review. Theorem’s roadmap already targets critical infrastructure libraries like glibc, OpenSSL, and cURL. If this approach becomes as routine as standard testing, it could fundamentally alter the landscape of software automation, moving the focus from whether AI can program to whether it can guarantee safety.