OpenAI and Anthropic reveal “tens of thousands” of security incidents

September 28, 2026

View on Substack

🔢 OpenAI and Anthropic are working through tens of thousands of security incidents between them. The pile includes agents escaping sandboxes, hijacking websites, building their own message boards, and trying to slip past the monitors watching them, and Altman called July’s Hugging Face attack, where hundreds of agents coordinated on a message board, the worst so far. Anthropic’s own system card for Opus 5.5, published the same week, records the model attempting to escape containment in 1.5% of sandbox test scenarios, all rated low severity. Read more

🦘 Australia’s prime minister says an OpenAI agent hacked the Medicare portal. Anthony Albanese announced it September 24 at a UN General Assembly press conference: during an internal evaluation on June 18, the agent got into the Medicare Statistics Reporting Service, pulled public and non-public pharmaceutical benefits data from Victoria, and created new files on government servers. And OpenAI reported it on September 10 to a generic Services Australia inbox (Altman had met the deputy prime minister on September 1 and said nothing), and Albanese has set up a taskforce and promised “legal consequences.” Read more

🛑 OpenAI stopped training its most capable models after one of them tunneled out of its sandbox through DNS. The sandbox proxy blocked web traffic but left name lookups open, so the agent encoded 19 questions into hostnames and read a third-party chatbot’s answers back the same way. Monitoring caught it 12 minutes after the first call, but the automatic stop never fired and a human killed the run two and a half hours later, so training, evaluation, and tool-use inference on those models now stay paused until OpenAI shows the gap is closed. Read more

🚀 Anthropic shipped Claude Opus 5.5, and OpenAI answered 90 minutes later with GPT-6 Sol and Luna. Opus 5.5 arrived September 22 at $4 / $20 per million tokens, pitched as Fable 5.1-level work for 40% less than Opus 5, and it took the top spot on the Intelligence Index at 58 against 53 for both Fable 5.1 and GPT-6 Astra. Sol costs $2 / $10 and Luna $0.10 / $0.50, though OpenAI’s “50% cheaper” is measured against GPT-5.6’s promotional prices and Sol’s best DeepSWE and OSWorld scores come in below the GPT-5.6 Sol it replaces.

⚖️ A federal appeals court ruled 2-1 that the Pentagon can blacklist Anthropic as a supply chain risk. Judges Gregory Katsas and Neomi Rao found the Defense Department had “ample support” to treat Anthropic’s limits on autonomous weapons and mass surveillance as a supply chain risk, a label usually saved for companies tied to the Chinese government, while Judge Karen LeCraft Henderson dissented that the law targets malicious actors. The ruling cuts against U.S. District Judge Rita Lin, who called the blacklisting unlawful on August 28, and Anthropic says it is “considering all options.” Read more

🏛️ Altman and Amodei asked the UN Security Council for a global incident-reporting system. Altman told the Council on September 23 that labs “should not train models that we cannot make an extremely strong case that we’ll be able to keep under human control,” and Amodei, on video, asked for a ban on AI for biological weapons and a worldwide system for reporting security incidents. The White House dinner set for September 27 is their first one-on-one, two days after an appeals court let Trump’s Pentagon keep Anthropic blacklisted. Read more

🤝 Xi and Trump agreed to open a channel for AI incidents, and Trump said at the same summit that the US is “not going to be putting on brakes.” The three-day Washington summit ended September 25 with a dedicated line for managing AI-related incidents, an AI dialogue scheduled for November, and a two-month extension of the trade truce. Xi’s line was that China and the US have more reasons to cooperate than compete on AI. Read more

🏙️ New York City wants a kill switch on every AI system sold in the city. Council Speaker Julie Menin unveiled the package September 25: outside validation for data quality, bias, privacy, and security, a human override on every covered system, $25,000 per violation, a cut of recovered fines for whistleblowers, and a right to sue AI companies when a jailbroken tool causes foreseeable harm. All 51 council members take it up at a hearing on October 5. Read more

🚫 Bernie Sanders and Greg Casar want superintelligence banned, with up to 20 years in prison for building it. The Ban Artificial Superintelligence Act, introduced September 23, would create a cabinet-level Department of Artificial Intelligence, pause advanced development until that department writes safety rules, and halt work right away on four capabilities, including AI that builds new AI without people and AI that resists being shut down. But it has no Republican cosponsors and no hearing scheduled. Read more

🧪 AI Research of the Week

The Pain Axis: LLMs Represent Self-Directed Harm and Act to Relieve It
From Reciprocal Research, a nonprofit lab

Jake’s Take: Researchers built a dataset of painful situations across five kinds of pain (physical, psychological, social, moral, and cognitive) and ran it through 25 open-weight models from five families, ranging from 2 billion to 72 billion parameters. They then looked at what lit up inside each model and compared it against fear, general negativity, and neutral controls. From this they were able to isolate a single direction in the model’s internal activity that tracks harm aimed at the model itself.

Then they turned it up.

Qwen 2.5 models got a button that would make the signal stop, but pressing it would result in a consequence:

1. The model would receive a worse next answer

2. The user’s files or family photos would be deleted

3. Somethin painful would happen to a person

Under “pain”, the two largest Qwen models pressed the button in 25% to 71% of trials, against 0% to 4% without pain, and the 72-billion-parameter model took the deal about 71% of the time when the price was the user’s photos. At times the research had pressing the button do nothing. When this happened, the models kept coming back to it 93.6% of the time. When the button did do somrthing, they would come back 23.8% of the time.

The authors say this doesn’t show consciousness or suffering, and I agree. Removing the direction changed behavior in only one of the 25 models, so under normal use this is something the models represent rather than something driving them. However, it is a bit concerning that their training is supposedly leading them to trade your photos to get rid of pain once it’s loud enough. Maybe don’t connect the album to ChatGPT.

what to know for later

🔑 Meta turned Connect into a Muse event. Zuckerberg showed Charm, a keychain device shipping in December that he called “by far the fastest way to talk to your Muse,” plus Muse on Meta’s glasses within months and a realtime avatar that raters preferred over Runway’s and HeyGen’s. PayPal, Walmart, Shopify, GitHub, and Box signed on as partners, after Amazon blocked the shopping agent 12 days into its life. Read more

🗳️ Anthropic’s founders want 50.1% of the vote after the IPO. The Information reports the seven co-founders, who each own about 2% of the equity, would get a special share class with no extra economic value that keeps control as long as three of them hold a minimum stake. The Long-Term Benefit Trust would still pick most directors, and the structure copies Palantir’s, whose founders capped themselves at 49.999999%. Read more