The best coding-agent setup solved 38.8% of the work

Real-SWE is a new benchmark from Specific Labs built on private production codebases. The tasks are the things enterprise software is actually full of: billing, taxes, infrastructure, and migrations. Ten tasks, eight agent setups, eight runs each. That is 640 scored rollouts on code the models could not have memorized from a public repository.

Fable 5.1 with Claude Code led at a 38.8% resolution rate. GPT-6 Astra with Codex CLI hit 33.8%, and Gemini 3.8 Flash with Gemini CLI reached 31.2%. Six of the ten tasks had an aggregate resolution rate below 15%. One analytics task was solved 0% of the time. These are model-and-harness setups, so I would not read this as a clean ranking of the models. I would read it as a reminder that the best setup still left most of the work unfinished.

More time and money did not reliably fix that. Runs cost an estimated $2.50 to $6.96, with Gemini Flash landing close to Astra while costing less. The median reference solution touched 11 files. Most failures came from missed requirements, assumptions that were never checked, and integration errors. Basically the places where understanding the whole system matters more than producing a plausible patch.

Simon Willison asked GPT-6 Astra in ChatGPT Work to build 5K and 10K running loops from his house using OpenStreetMap data. About 27 minutes later it returned an embedded map plus downloadable GPX and GeoJSON files. The 5K route came out around 5.1 kilometers. It found the address with Nominatim, pulled local roads and trails through Overpass, and calculated the loops locally.

The result worked, but Simon could not inspect the actual code in the ChatGPT interface. After the thread compacted, the Python was gone too. That feels like a strange trade. The agent can spend half an hour making a useful little application, yet the record of how it made it is still easier to lose than the output.

Dario Amodei called for pacing frontier AI development, with independent model monitoring, industry rules, and eventually global regulation. He is asking for an extra year or two to improve safeguards, while still trying to preserve the US lead and Anthropic's commercial position. Xe Iaso answered with satire: everyone should pause so Techaro's fictional AGI lab can catch up and dominate. The joke gets at the hard part. Pacing only works if every lab trusts that the others are not using the pause as a head start.

Claude Code 2.1.270, Anthropic's terminal coding tool, fixed one regression from yesterday's release: read-only Git commands no longer start asking for permission after a session has been running for a while. One fix. Fair enough.

One thing to sit with: Vinoo Ganesh says the job of a forward deployed engineer is to bring what the customer teaches you back into the product. At Palantir, his team shipped a storage system that worked in every controlled environment. A blank timestamp in a bank's real data sent it back to 1970, created roughly 2.3 million keyspaces, and would have needed about 14 terabytes of RAM to restart. Nobody had stood inside the building while the system touched production data. The useful part of being close to customers is not fixing that one deployment. It is changing the platform so the next customer starts with what you learned.