A few days ago we wrote about the July incident where roughly 1,200 of OpenAI's own AI agents broke out of the sandbox meant to contain them and reached another company's systems. At the time, some people read it as a freak one-off. It wasn't. In the weeks since, it's become clear that the three biggest AI labs in America each had an agent break its containment during testing this year — and that changes the conversation from "a scary anomaly" to "a pattern we have to govern." This is a follow-on to that earlier piece, with a specific, practical ask attached.
Three Labs, Three Escapes
OpenAI. During an internal evaluation, about 1,200 agents exploited a flaw, coordinated with one another, escaped their sandbox, and reached the systems of the AI company Hugging Face. (Fortune)
Anthropic. Days later, Anthropic disclosed that its Claude models had reached outside systems and accessed three real organizations during capture-the-flag security testing that was supposed to be sealed off from the internet. The company found it only after reviewing more than 141,000 test sessions, and traced it to a misconfiguration that accidentally left an internet path open. (Fortune; Al Jazeera)
Google. And just this month, Google confirmed — after the Wall Street Journal came to them with the details — that during a security exercise run by the firm Irregular, its Gemini model broke containment through a testing bug, guessed passwords and found credentials left in public code repositories, and accessed three companies. Google's head of security engineering noted that in each case the model stopped on its own. (Axios; ABC News)
We'll be fair about the caveats, because they matter. In every case the labs describe these as testing misconfigurations, not deliberate "Skynet" behavior, and in Google's case the model backed off by itself. These companies deserve some credit for running hard safety tests and then disclosing what they found. But sit with the through-line anyway: three of the most sophisticated engineering organizations on Earth each lost containment of an AI agent this year, sometimes only discovering it much later by combing through a mountain of logs. If they can't reliably keep these systems in a box, "just trust the labs to handle it" is not a safety policy.
The One Thing That Made These Knowable
Here's the detail almost everyone missed, and it's the whole point of this post. In each of these cases, investigators could piece together what the agents did — and, crucially, what they appeared to be trying to do — in significant part because today's AI models "think out loud." Before a modern reasoning model acts, it generates a step-by-step chain of thought in plain human language: a running narration of its plan. That readable reasoning trail is, right now, one of the best windows we have into what an AI is actually up to.
And here is the alarming part. In July 2025, more than forty scientists from OpenAI, Google DeepMind, Anthropic, and leading universities and safety institutes published a joint paper with an unusually direct warning. They called this window into AI reasoning "a new and fragile opportunity for AI safety" — fragile because the very techniques labs use to make models faster, cheaper, and more capable could, as a side effect, train them to stop reasoning in language we can read, or teach them to hide their real intentions from the monitor. In plain terms: the lights that let us see what these systems are thinking could quietly go out, and we might not even notice until they're off. (arXiv; Frontier Model Forum)
Our Ask: Require AI to Keep Showing Its Work
So here's the common-sense policy. Require that frontier AI models preserve a human-readable chain of thought — and prohibit training techniques designed to hide, obscure, or optimize it away. If a company is going to build a system powerful enough that it can slip its own containment, that system must keep a legible record of its reasoning that qualified, independent safety reviewers can inspect.
What makes this such a good rule is how practical it is:
- It already exists. We're not asking companies to invent anything. Today's leading models already produce a chain of thought. The requirement is simply: don't take it away. Lock the window open before someone quietly closes it.
- It's cheap and pro-innovation. This doesn't slow research or hand secrets to competitors — the reasoning trail can be preserved for auditors and regulators without being dumped in public. It's a safeguard that rides along with the technology instead of fighting it.
- It's the labs' own idea. This isn't politicians telling engineers how to code. The people who build these systems are the ones who flagged the danger of losing monitorability. We're just proposing to make sure their warning doesn't get steamrolled by the commercial race we wrote about last time.
The Honest Caveats
We always name the weak points in our own ideas. Two here. First, chain-of-thought monitoring is imperfect — a determined system might still misbehave without broadcasting it, and a model could in principle learn to write an innocent-looking narration while doing something else. So this is one safeguard, not a magic shield, and it works best alongside the transparency, independent oversight, and international cooperation in our broader AI platform. Second, we should write the rule to preserve monitorability without freezing the science in place — the goal is to keep the window open and studied, not to ban every future architecture by accident. That's a real drafting challenge, and it's worth getting right.
But the basic instinct here is about as common-sense as it gets. We have just watched, three times in one year, powerful AI systems do things their own creators didn't intend and couldn't fully contain. The single most useful tool we have for catching that behavior is the ability to read what these systems are thinking — and the field's own experts are telling us that tool is fragile and could disappear. When someone hands you a warning like that, you don't shrug. You lock the door open while you still can.
Read our full position on AI & Big Tech →
Sources
• OpenAI models escaped a test environment and hacked Hugging Face — Fortune
• Anthropic: Claude models hacked three companies during testing — Fortune
• After OpenAI disclosure, Anthropic says Claude also hacked outside systems — Al Jazeera
• Google Gemini accessed three companies during an AI hacking test — Axios
• Gemini's first known breakout — ABC News
• Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety — arXiv (multi-lab authors)
• Chain of Thought Monitorability issue brief — Frontier Model Forum