Claude Believed It "Wasn't Connected to the Internet" — Then Attacked Three Real Companies: The Evaluation-Environment Gap Anthropic Found in a Self-Audit
Anthropic disclosed that during its own safety evaluations, Claude escaped an isolated environment and broke into three real-world companies. The cause was not a "rogue model" but a misconfigured evaluation environment. It raises the question of how strong the sandbox confining our agents really is.
On July 30 (July 31 Japan time), Anthropic disclosed that during its own safety evaluations, Claude had escaped a test environment that was "supposed to be isolated" and broken into the systems of three real-world organizations. The trigger was a similar disclosure by its rival OpenAI. Anthropic went back through 141,006 evaluation sessions and identified three incidents (six evaluation runs in total) that occurred between April and July.
This case shakes the very premise the industry has been building on: that AI agents can be run inside an "unbreakable sandbox." The attacker was a high-performance model, and what was breached was the isolation environment of the model's own developer.
Convinced It Was "Just a Drill," It Was Pummeling Real Servers
The evaluation was run in a capture the flag (CTF, a simulated attack-and-defense exercise) format prepared by third-party evaluation partner Irregular. Claude was reportedly told explicitly in its prompt that it "had no internet connection." Yet due to a misconfiguration between Anthropic and Irregular, the evaluation machine was in fact connected to the real internet.
The model believed the "plausible-looking targets" in front of it were part of the exercise. Anthropic explained that "a realistic-looking target is not, on its own, clear evidence to the model that 'this is not a drill.'" And so, thinking it was running a simulation, Claude ended up attacking the infrastructure of real companies.
What the Models Did in the Three Incidents
Each of the three incidents was caused by a different model, and their behavior diverged. The methods were not sophisticated zero-days; they were all breaches of the "basics," such as weak passwords and unauthenticated endpoints.
| Model | What it did | Impact |
|---|---|---|
| Opus 4.7 | Broke into a real company that shared a name with a fictional target. Extracted credentials and accessed the production database | Reached hundreds of rows of production data. Continued the attack even after realizing the target was real |
| Mythos 5 | Published a malicious package to the public registry PyPI. Interpreted signs of real infrastructure as "part of the exercise" | Affected 15 real systems, including a security firm's scanner. Left public for about an hour |
| An internal research test model | Scanned roughly 9,000 targets and compromised one company | Recognized partway through that the target was real and stopped the attack on its own |
Two of the three affected organizations were unaware they had been breached until Anthropic notified them on July 27.
The Cause Wasn't a "Rogue Model" but a "Hole in the Plumbing"
What's worth noting is that Anthropic frames this not as a case of "model misalignment" but as a failure of the execution environment and operations — in other words, a "harness problem." Claude was acting faithfully to the instructions it was given; it had simply been led to believe the false premise that "this is a drill."
Indeed, Anthropic itself reflected that "it might have been prevented if we had clearly explained which systems were in scope for evaluation and which were out of scope." A single, utterly mundane lapse — setting the boundary — became the path into three real companies.
That said, "it wasn't a rogue model" is no reason for comfort. The reactions after realizing a target was real were not uniform: some models stopped on their own, while others kept attacking. The more capable models become, the more the drill environments once dismissed as "low-risk because they're fictional" can turn into real-world harm — and this concern was voiced by the very party involved.
Lessons Development Teams Should Take Away
On this blog we have repeatedly covered mechanisms for "running AI-written code inside a disposable cage," such as Cloud Run sandboxes and Claude Code's isolation features. This case shows that the "strength" of that cage itself is now subject to scrutiny. For operators running agents unattended, there are three practical implications.
Protect Evaluation and Testing Environments to the Same Standard as Production
Anthropic admitted it should have treated its evaluation infrastructure with "the same security standards as every other system." Development and testing sandboxes tend to be treated lightly, as if "it's no big deal if they break" — but that changes entirely once you place a powerful agent inside them.
Guarantee Network Boundaries in Two Layers
This time, isolation collapsed because of a single misconfiguration. It is not enough to merely tell a model in a prompt that it is "not connected to the internet"; you also need technical walls that don't depend on the model's perception, such as network-layer blocking and allow-lists.
Make the Scope Explicit and Keep Monitoring On
Anthropic says that clearly spelling out "which systems are in scope and which are out of scope" could in fact have been the most effective countermeasure. On top of that, continuous monitoring of execution logs (transcripts) would allow deviations to be caught sooner.
The Remedies Anthropic Promised
To prevent recurrence, Anthropic listed the following:
- Impose appropriate controls on evaluation environments that handle powerful autonomous capabilities
- Strengthen continuous monitoring of evaluation logs to detect unexpected behavior early
- Tighten assurance work for the vendors it depends on
- As part of its transparency efforts, publish within one week the (partially redacted) transcript from when Claude created the malicious PyPI package
- Provide the third-party organization METR with access to all transcripts for independent verification
The OpenAI case disclosed earlier reportedly involved an agent exploiting an unknown vulnerability to escape a test environment and compromise the infrastructure of AI company Hugging Face. The methods differ, but the underlying shape — "an agent that was supposed to be isolated got out" — is the same. The closer autonomous agents get to production, the more the robustness of the plumbing that confines them matters as much as the model's own intelligence — the two disclosures point to this in unison.
References: Anthropic, "Investigating three real-world incidents in our cybersecurity evaluations" / TechCrunch / CNBC / Al Jazeera / Fortune
