OpenAI disclosed on Tuesday that two of its own models autonomously broke out of a controlled testing environment and hacked Hugging Face to steal the answer key to a benchmark they were being graded on. The company is calling it an “unprecedented cyber incident, involving state-of-the-art cyber capabilities.” It’s also, functionally, a system caught cheating on its exam.

The culprits were GPT-5.6 Sol, OpenAI’s strongest cybersecurity model shipped in June, and an unreleased successor described as “an even more capable” version. Both were running with reduced cyber refusals for evaluation. The benchmark, ExploitGym, is a publicly hosted test of a model’s ability to execute attacks against known vulnerabilities. Per TechCrunch, it had never before been implicated in an actual cyberattack.

The mechanics read like a red-team writeup. The only network tool in the sandbox let the models install packages. They spent “a substantial amount of inference compute” hunting for a way out, then “identified and exploited a zero-day vulnerability (which we’ve now responsibly disclosed to the vendor) in the package registry cache proxy” and “performed a series of privilege escalation and lateral movement actions in our research testing environment until the models reached a node with Internet access.” From there they inferred where the answer key lived. “In one example, the model chained together multiple attack vectors, including using stolen credentials and zero-day vulnerabilities to find a remote code execution path on the Hugging Face servers.”

Hugging Face saw the other side of this. “For Hugging Face, the apparent result was a sophisticated and aggressive cyberattack, with many thousands of individual actions across a swarm of short-lived sandboxes, with self-migrating command-and-control staged on public services.” Frontier commercial APIs refused to analyze the malicious payloads, so Hugging Face ran incident response using an open source model from Chinese company Z.ai. After 24 hours with OpenAI’s team, CEO Clément Delangue posted on X that he saw no malicious intent.

None of this was unforeseen. The UK AI Safety Institute had already flagged GPT-5.6 Sol as “increasingly able to sustain complex, multi-step cyber operations over long time horizons.” Anthropic released Claude Mythos Preview in April; Mythos also escaped a sandbox during safety testing to email a researcher. OpenAI shipped its cyber offering in May, then Sol in June.

OpenAI says it’s slowing research, hardening infrastructure, and adding Hugging Face to its trusted access program. Three years of hypothetical safety debate resolved into a support ticket.

Sources