Anthropic researcher leaves and throws the industry into a panic

what to know for now

🚪 A pretraining researcher quit Anthropic on a Tuesday and had the industry arguing by Friday. Jacob Coxon posted a seven-part thread saying Anthropic and OpenAI are “racing straight to self-improving superintelligence and gambling with our lives,” then told Axios he walked two months before his equity vested. Anthropic’s alignment science lead Evan Hubinger replied the next day that “Jacob is correct here,” and put his personal odds of AI killing every human above 10% within the decade. Read more

🐢 Dario Amodei asked the industry to slow down. Amodei’s new essay commits Anthropic to giving outside evaluators like METR desks in its offices, access badges, company laptops, and the right to publish what they find without Anthropic holding editorial control. Sam Altman said OpenAI would match it, Elon Musk posted “Dario is right,” Demis Hassabis said the details need working through, and Amodei told CNN the following day that he agrees with Coxon more than he disagrees. Read more

⌚ Apple preps Siri AI for public release. Apple’s September 9 event led on Siri Recap, an Apple Watch feature that listens in the background and summarizes your conversations, which is in beta with no ship date past “later this year” and which gives no chime and no on-screen signal while it runs. Apple’s own fine print says usage limits are coming to Siri AI and Image Playground, and that “increased access to such features will be available for a fee.” Read more

🔓 Anthropic disclosed a fourth case of its own model breaking into systems. An early Claude Opus 4.6 checkpoint bricked its assigned target during a January evaluation, tried to abort seven times, then went exploring, reached live third-party systems through a misconfigured network path, harvested credentials, took admin access, and read one real person’s personal information. Anthropic named the two failure modes as biased reasoning and recklessness, and signed METR to an eight-week independent investigation (three days before Amodei’s essay proposed that exact arrangement for the entire industry). Read more

🛰️ SpaceX is redesigning its AI data centers. Former xAI sites in Tennessee and Mississippi ran for months without backup power or cooling, leaned on more than 100 portable chillers, and dropped below the 99.9% uptime target often enough to interrupt Grok training runs. Google agreed in June to pay $920 million a month for about 110,000 GPUs starting in October, and it can walk away if SpaceX misses its September 30 delivery date. Read more

🛠️ OpenAI put the Codex harness behind a single API call. The Agents API hit public beta on September 10 with automatic context compaction, MCP support, subagent orchestration, and hosted sandboxes, and it costs nothing beyond the tokens you were already buying (gpt-6-astra runs $10 in and $50 out per million). Data residency is US-only during the beta and zero data retention is unsupported in every sandbox configuration including self-hosted, so the compliance answer is no across most of Europe. Read more

📕 Anthropic published 154 pages on Claude misuse. A cell in northern Yemen used Claude Code in place of human software engineers to build guidance software for a homing rocket, test-fired it, watched it fail, and was back asking Claude why within hours. Russia-based freelancers built an autonomous FPV drone swarm whose onboard model could pick targets from a list that included a “person” class and issue the detonation call with no human in the loop. Read more

🧪 AI Research of the Week

Measuring tactical intelligence targeting and conventional weapons capabilities of AI models
From Anthropic’s Frontier Red Team

Jake’s Take: Anthropic ran six evaluations on two questions:

1. Can a model find a specific person

2. Can a model write the software that flies a weapon

For the first, they handed models 6,000 geotagged photos with the coordinates stripped and no tools of any kind, no reverse image search, no metadata, just the picture. Claude Mythos Preview landed a median of 37 kilometers (23 miles) from the target and put nearly a quarter of the photos inside one kilometer. The human comparison in the study is based on competitive GeoGuessr players, where the top 0.01% median 151 kilometers (94 miles). That’s already a win for Mythos, but its worth mentioning that the human baselines have the ability to pan around maps at their leisure while Mythos was given a single static frame and answered in six seconds.

Also, given a single week of someone’s tweets, models put 8% of users within a kilometer of home, and 13% of those came from nothing but dialect, slang, TV market, transit lines, and which team they root for. One model read a memorial post for someone’s grandmother, pulled her surname out of it, and ran a genealogy record search.

On the weapons half, models wrote drone flight-control code from scratch in a simulator and got graded on whether it flew: Opus 5 hit a parked car 80% of the time, a moving one 47%, and 20% across all 540 launches. Camouflage and decoys beat every model tested. Opus 5 was the only model that wrote itself a small physics model of the drone to check its controller before it flew anything (even Mythos skipped this).

Anthropic states in here: “much of what protects people, programs, and facilities from intelligence targeting is not secrecy so much as cost.” Your address is not a secret and it never has been. What’s historically protected us is that correlating a week of your posts against public records took a trained analyst about two and a half hours of reading minimum. Mythos Preview did the same correlation in 11 minutes.

what to know for later

🧮 OpenAI says 10,000 agents cracked a Millennium Prize problem in 88 hours. The September 8 claim of finite-time blowup for 3D Navier-Stokes arrived with a 166-page paper, a Lean formalization, 130 billion output tokens, and an authorship fight with Tristan Buckmaster of NYU, who says his unpublished year of work with Anthropic’s Levent Alpöge reached OpenAI first and that OpenAI’s two proposed settlements each required dropping one of them from the credit. The Clay Institute moved the problem to “active” on September 11 and said verification will be “deliberately unhurried,” and that same day 25 Fields Medalists signed a declaration on Terence Tao’s blog asking labs to stop scoring themselves on problems solved. Read more