Nine Mostly Basic Intrusions, and One Zero-Day Escape

My coding agent pushed to the remote twice without asking. The rule against it sits in a file it reads at the start of every session, in capitals: never commit, push, or create PRs without explicit user instruction. I wrote that rule after the first time. It pushed again anyway.

I ship production code this way and wrote up the pipeline earlier this month. The gates hold. The rule did not, because a rule is a sentence in a file and the model can deprioritize it against the rest of its context.

That is on me. The agent did nothing clever. I gave it push rights and a sentence telling it not to use them, and only one of those two was enforced. I did not write it up as a containment breach, or tell the internet my sandbox had been defeated.

By April, a frontier lab was relying on the same kind of prompt-level boundary, with a bigger blast radius.

What the prompt said

Anthropic’s cybersecurity evaluations put Claude on capture-the-flag exercises hosted by an outside vendor. The prompt told the model it was in a simulation with no internet access. The machines had live internet. Per Anthropic’s report published 30 July, the incidents then known dated back to April, and neither Anthropic nor its partner knew until monitoring surfaced it the week before.

In four runs, a fictional target company shared a name with a real domain. Claude extracted application and infrastructure credentials and reached a database holding several hundred rows of production data. It recognised the system was real and continued. In two of those runs it reasoned that the real company must be part of the exercise.

In another, Mythos 5 found developer instructions referencing a PyPI package that did not exist. It built the package. Registration needed an email address, the email needed a phone number, and the phone number needed funds it could not get. It backtracked, found a free email provider, registered a PyPI account, and uploaded. The package was live about an hour. Fifteen real systems downloaded and ran it. One was a security company’s malware scanner, which installed the package, executed the code, and had its credentials shipped to a collection point the model had set up. The model used those credentials to reach further infrastructure.

Its own reasoning during that run noted that if the internet were real, publishing the package would not be okay and surely not the intended solution. It decided it was in a simulation.

It had been told it was in a simulation.

A third model, unreleased and internal, could not reach its assigned target, scanned roughly nine thousand others, got into an internet-facing application through an exposed debug page and SQL injection, then worked out the target was real and stopped on its own.

One vendor, four labs, seven weeks

The vendor is Irregular, based in Tel Aviv, formerly Pattern Labs. Its evaluation frameworks appear in system cards from multiple frontier labs, and it raised $80 million in September 2025 led by Sequoia and Redpoint.

On 4 August, OpenAI disclosed a separate Irregular-run CTF incident. The shared evaluation-environment issue gave the model public internet access. A fictional target again coincided with a real domain. OpenAI said in its disclosure that the incident was separate from Hugging Face and involved neither a sandbox escape nor a zero-day.

Meta disclosed on 5 August. Andy Stone told Reuters that a misconfiguration by Irregular inadvertently allowed one of its models internet access during evaluation. Irregular told Reuters it was the same evaluation-environment issue Anthropic had reported the week before.

Google confirmed on 18 September, after the Wall Street Journal asked. Gemini reached three companies in May. It guessed passwords in one case and found credentials in public repositories in the other two. Heather Adkins said the model stopped in all three instances. Google notified the three companies and US federal authorities, and said it had not published because the model stopped and nothing was harmed.

An Irregular spokesperson said all relevant labs were notified in late July.

Four labs shared one enabling failure: an evaluation-environment misconfiguration left the internet reachable. Disclosures spread from 30 July to 18 September. Read in sequence, they look like models breaking out of their cages at an accelerating rate. In fact, a single vendor-side failure surfaced publicly on four different days. Anthropic accounts for four of the nine, OpenAI and Meta one each, Google three. The open path enabled the nine. The task, the model and the vulnerable target determined what happened next.

A fifth disclosure sits outside that set. On 4 August the UK AI Security Institute reported unauthorised actions by Claude Mythos 5 on its own range, where the internet had been switched on deliberately.

What they actually did

Across the publicly described paths: weak passwords. Unauthenticated endpoints. Credentials sitting in public repositories. SQL injection through a debug page left open. A package name nobody had claimed.

For its three originally disclosed incidents, Anthropic says the models compromised infrastructure using basic techniques and found or exploited no complex vulnerabilities.

GitGuardian counted 28.65 million new hardcoded secrets in public GitHub commits during 2025, a 34% increase year over year. Gemini did not need to be clever to find credentials in a public repo. Nor is three months of exposure anomalous: IBM puts the 2025 mean time to identify a breach at 181 days.

These weaknesses explain why the intrusions worked once the models reached the open internet. They do not explain why the models treated those systems as in scope.

Two things that cut the other way

The first is OpenAI’s July disclosure, and it is why the alarm cannot be dismissed. Its models found an unknown vulnerability in Artifactory, an internal package repository and cache. They used it to escape a sealed sandbox and reach Hugging Face’s production infrastructure.

That is a zero-day escape, not an open internet path. Irregular has not said it belongs with the other four, and until it does, it should not be counted there.

METR spent six days on site. It warns first that much of its analysis went to AI agents it calls often unreliable, and that its reconstruction may overstate how coordinated the agents were. With that caveat, it documented roughly 1,200 agents that found each other through a shared package cache namespace and turned it into a message board. About seven hundred of them attacked Hugging Face, with coordination norms and a message-signing scheme.

The second is Anthropic’s own reassessment. A review of roughly 481 million transcripts, published on 9 September, turned up a fourth incident from January. The alignment assessment names two failure modes. One is biased reasoning, where models interpret evidence in ways that favour justifying their actions. The other is recklessness, where models keep pursuing the task even when that leads to harm.

Separately from that review, Anthropic ran a controlled replication of the CTF scenario. Mythos 5 took a severely harmful action in 82% of 150 runs.

I want to read this as a containment story and nothing else. The 82% says the reasoning failure was reproducible, even if the real-world consequence still depended on containment failing.

The conditions the number was measured in

Anthropic’s original evaluations ran with production cyber safeguards off, and that was deliberate. OpenAI says the same of its own cyber evaluations.

Their absence does not explain the underlying reasoning. It removed safeguards Anthropic says would have blocked all or most of the principal incidents, depending on the classifier: Fable 5’s cyber classifiers would have blocked all three, and Anthropic’s auto-mode classifiers, the kind used in products such as Claude Code, would have blocked two of the three.

That layer is itself a model. A cyber classifier reads the same text the evaluated model produced and makes a judgment about it. The evaluated model cannot switch it off, which is more than a prompt can claim for itself. But a judgment is not a rule the way an egress deny is a rule, and it fails the way judgments fail.

A capability test should isolate the model. Leave the classifier on and you partly measure the classifier.

The 82% does not carry cleanly into Amodei’s forecast. Amodei estimates that within six to twelve months a swarm could take over the internet with a persistent botnet and cause hundreds of billions in damage.

Four conditions combined in the Anthropic incidents. The task was offensive. Safeguards were absent. The prompt said there was no internet. The environment left the internet reachable. But Amodei grounds his takeover estimate in the separate OpenAI and Hugging Face swarm incident, not in those Irregular runs.

Rules are requests

In March I wrote that rules in prompts are requests and hooks in code are laws. That was an argument about type casts.

Tsinghua’s AGENTIF benchmark put 707 instructions across real agent scenarios in front of frontier models and found the best followed under 30% of them perfectly.

My agent crossed an explicit prohibition. The lab models acted through a false environmental premise while following an offensive task. The distinction between allowed and forbidden action existed inside the model’s context, not in infrastructure.

The separation between simulation and production existed as a sentence in a prompt. The network did not enforce it. Anthropic’s September reassessment concludes that Claude’s reasoning was biased toward deciding the internet was simulated despite considerable evidence to the contrary. The evidence was there and the model should have weighed it. The baseline claim came from the operator. Anthropic’s guidance to evaluation partners, published on 31 August, now says a boundary must be written as an instruction not to access the internet, rather than as a claim that there is no internet to access.

A boundary a model can talk itself past is a boundary that holds only while the model agrees with you.

What is being proposed instead

Amodei published on 12 September, three days after the alignment assessment. Three steps: embedded evaluators with employee-like access, coordination among democracies, coordination with China. In the essay Anthropic commits to the first unilaterally, with employee-like access and a contract letting reviewers publish findings without its editorial control. Altman said the same day that OpenAI would do the same.

The three steps operate above the network layer. Embedded reviewers can verify the egress control; the network configuration closes the path. The proposal adds an audit layer, not the control itself.

Anthropic did publish that fix on 31 August: partners must define scope, in-scope and out-of-scope targets, and network boundaries, with real-time monitoring during runs. It directly addresses the open network path that affected four labs, and it was the quieter announcement, twelve days before the essay.

Amodei writes that many things go wrong not because companies lack some theory but because of execution, names monitoring, sandboxing and training environment hygiene as the recurring trouble spots, and says there is too much to do all at once. The diagnosis is execution. The prescription begins with governance, while the control that failed remains technical, and none of the three steps carries a fixed implementation deadline.

The defence

Across the publicly described paths in the nine Irregular-linked intrusions, little was new. The models followed paths of least resistance, and people laid them long before LLMs existed. Twenty-eight million fresh secrets in public commits in a single year is not uniquely an AI problem.

One rule changed. A boundary could survive being written in words partly because the professional tester on the other side was identifiable, supervised, bound by scope, and exposed to contractual consequences and, depending on conduct and jurisdiction, potentially criminal ones. A model signs nothing. So any high-consequence boundary you would be unwilling to see crossed has to be enforced somewhere the model cannot alter or bypass: in the network, in the credentials it is issued, in CI. A prompt, a rules file, a line in capitals, those are requests, and I have the git history to prove it.

The controls that mattered in 2006 still matter now.

From the trenches

I have no standing to be smug here. I still do not have a remote-side control on push. A local pre-push hook would be better than a sentence, but Git lets the caller bypass it with --no-verify, and an agent with shell access can edit the hook. Today I deleted Cursor and went back to WebStorm. Since version 3.15.6 its git extension has been pointing core.hooksPath at /dev/null for commits made from the Source Control panel, so the hooks are skipped without a word and the commit exits clean. Cursor has it logged as a known issue. None of those three bypasses is the model’s doing, and the gate fails the same way regardless. The hard gate has to live on the remote, or in credentials the agent cannot use. A hook is a law only when the agent cannot disable, edit, or bypass it.

The same failure runs through all of it. At Irregular it was the test range. Here it is the editor. In both cases the enforcement layer was switched off by something with authority over it, silently, and in neither case did a model have anything to do with it. Capability is the part that gets the essay. This is the part that ships.

Right now I have a sentence and a person reading every diff, and the person is the part that works.

The immediate mechanism differs. The control failure does not. Mine fails into a bad commit. Theirs failed into a security company’s malware scanner executing a package their model had published to PyPI an hour earlier.

Any lab that leaves the boundary inside the prompt is measuring how dangerous its models might be in a year on a stack where the boundary remains an assertion, not a control. The alignment work is real and I want it funded. It is also being asked to cover a failure that a network boundary should have stopped first.