AI versus AI: inside the Hugging Face intrusion
The scenario cyber experts warned about has arrived.
When I read about an AI-driven attack on Hugging Face over the weekend, it gave me goosebumps. The reality was even worse: it was a rogue AI.
Last weekend, Douglas Mun sent me a Hugging Face blog about an AI-powered cyberattack. It forcibly reminded me of the sombre warnings from cyber experts who described precisely this scenario to me 12 to 18 months ago.
This development comes with a plot twist to boot. On Tuesday, OpenAI revealed that it was the culprit behind the AI attack.
What happened
According to Hugging Face in its initial blog post published last week, the company detected and responded to an intrusion into its production system. Hugging Face is an AI platform best known for hosting models, datasets, and libraries for building and running AI systems.
So how did the attacker get in? From my understanding, this was achieved by uploading data packages to Hugging Face, something contributors do on a daily basis. Machine learning datasets are often accompanied by custom loader scripts, and these presumably exposed the vulnerabilities that the apps processing them went on to exploit.
AI versus AI
What was extraordinary was the sheer scale and pace of the cyberattack. According to Hugging Face, "the campaign was run by an autonomous agent framework... executing many thousands of individual actions across a swarm of short-lived sandboxes, with self-migrating command-and-control staged on public services."
From my reading, the attacker must have staged its uploads across thousands of locations on the internet. Many cloud services offer a limited amount of free use, and these presumably got co-opted into participating.
Hugging Face knew something was wrong, thanks to the AI defences it has in place. But the sheer volume made it impossible to differentiate decoys and unsuccessful probes from successful attacks. So Hugging Face had to use AI of its own.
Here's the irony: AI models offered by the top commercial providers couldn't tell defence from offence and wouldn't work properly. In the end, the team turned to GLM 5.2, an open-weight AI model from Chinese AI firm Z.ai.
Whoops, we did it
On Tuesday, OpenAI came clean. It turns out the attack was really two AI models: GPT-5.6 Sol and a newer, unreleased model, both tasked to an internal cybersecurity benchmark.
Instead of staying within the test, they worked together and broke out to access the internet. Hugging Face was judged to potentially host information that could help them cheat on the benchmark.
For context, GLM 5.2 needs as much as 1.5TB of HBM to run at full performance, or about two modern GPU servers. That is data centre hardware costing more than half a million dollars.
I have some thoughts on this. But one thing is now clear: AI-powered attacks are no longer theoretical. So the question worth sitting with is what happens the next time the agent doesn't announce itself.
