benchmarks

Google's Gemini 4 Carbon took first place on Artificial Analysis's index on October 28 with a score of 59, but it launched only to vetted cyber defenders through the Fairwind Program, repeating the staged release Google used for Argon four weeks earlier.

OpenAI released GPT-6.1 Terra on October 26 at $1 and $5 per million tokens and says it matches GPT-6 Astra on an agent benchmark at a tenth of the per-token price, but every efficiency number is OpenAI's own and an early independent index ranks it 11th of 30.

Between September and October, AI labs and their users claimed an enzyme system, the NIST curve seeds, an Enigma break and a genome atlas, and graded on who did the work, what was checked and what replicated, those claims sort by how cheaply an outsider can verify the answer.

Anthropic, OpenAI and Google shipped new flagships between September 1 and October 13, and once independent evaluators reran the headline benchmarks the frontier compressed into a statistical tie, leaving cost, tokens, speed and access as the differences buyers can measure.

Anthropic released Claude Opus 5.5 at $4 and $20 per million tokens and OpenAI released GPT-6 Sol at $2 and $10 about 90 minutes apart on September 22, and because each launch table benchmarks the other lab's previous model, the price sheets are the only like-for-like comparison.

OpenAI released GPT-6 Astra on September 3 at $10 and $50 per million tokens, 2.5 times GPT-5.6 Sol's promotional rate, as the first model it rates Critical in cybersecurity, and several of its biggest launch numbers describe configurations buyers do not get.

Anthropic shipped Claude Fable 5.1 to everyone and the same weights as Mythos 5.1 to vetted US organizations, cutting cache reads 75% to $0.25, but the first independent run found a max-effort task costs 20% more than on Fable 5.

Anthropic shipped Claude Opus 5 as near-frontier intelligence at half the cost of Fable 5, and read from the P&L, the launch is a margin admission that the race has moved from raw capability to cost-per-task.

While Washington spent June turning frontier model releases into licensed events, Meituan open-sourced LongCat-2.0, a 1.6-trillion-parameter agentic-coding model under an MIT license trained entirely on domestic chips, as the counter-move to the export-control regime.

Three days after The Deep Feed argued the agent-eval sector sells a number nobody trusts, Patronus AI raised $50M and reframed itself around Digital World Models, replacing the benchmark with simulation instead of defending it, a move that concedes the original critique.

Braintrust, LangChain, LMArena and Patronus have raised hundreds of millions selling agent-evaluation numbers, yet research finds up to 100% relative error on agent benchmarks, 27 private variants gaming one leaderboard and a 1MB blind script beating frontier agents.

On June 16, GLM-5.2 became the leading open-weight model on the Artificial Analysis Intelligence Index v4.1 at 51, just 3.4 months behind an equally capable closed model, down from a 9.8-month gap in December 2024.

Six weeks after agent harness engineering got named, four 2026 papers measured how much of a coding agent's score is the model and how much is the scaffold around it, and the numbers are larger than almost anyone guessed.

Within three weeks MiniMax M3, Qwen3.7-Max, DeepSeek V4, Kimi K2.6 and Nemotron 3 Ultra all claimed the coding frontier, and third-party leaderboards show a real but small gap, mostly self-graded marquee numbers and a jurisdiction risk no benchmark measures.