evaluation

Between September and October, AI labs and their users claimed an enzyme system, the NIST curve seeds, an Enigma break and a genome atlas, and graded on who did the work, what was checked and what replicated, those claims sort by how cheaply an outsider can verify the answer.

Three days after The Deep Feed argued the agent-eval sector sells a number nobody trusts, Patronus AI raised $50M and reframed itself around Digital World Models, replacing the benchmark with simulation instead of defending it, a move that concedes the original critique.

Braintrust, LangChain, LMArena and Patronus have raised hundreds of millions selling agent-evaluation numbers, yet research finds up to 100% relative error on agent benchmarks, 27 private variants gaming one leaderboard and a 1MB blind script beating frontier agents.

Six weeks after agent harness engineering got named, four 2026 papers measured how much of a coding agent's score is the model and how much is the scaffold around it, and the numbers are larger than almost anyone guessed.