Hackers used Claude to break into OpenAI

Three developers broke into OpenAI's internal code system using Anthropic's Claude. What in the world is happening with AI and cybersecurity?

Two weeks after experimental AI models at OpenAI tried to break into Hugging Face, OpenAI itself has suffered an AI-powered attack. Here are the implications.

Three people, two subscriptions

Using AI tools such as Claude, security researchers broke into OpenAI. Thankfully, they were part of the company's authorised bug bounty programme and promptly informed OpenAI.

How did they do it? The researchers gained access through an OpenAI employee's ChatGPT account. This eventually gave them access to the company's private repositories, where they could retrieve internal software and suggest changes to the code.

OpenAI said the hackers uncovered two issues: one in a third-party service that hosts its community discussion forum, and a second with the AI company itself. Both have since been resolved.

According to the WSJ, which reported exclusively on this, the effort involved just three people from security firm Hacktron AI "using Claude and Codex subscriptions."

Why AI hackers are different

Over the last two years, I've spoken with cybersecurity experts who argue that AI tools can make sophisticated cyberattacks easier to carry out. That argument is now being borne out in practice.

But what makes AI tools so capable? I believe it boils down to three traits. They are tireless, and will keep trying for hours and days. They are faster, moving too quickly for humans to track. And they are scalable, since agent swarms can be spun up easily.

AI tools also lower the technical barrier, including in areas adjacent to hacking that used to require specialists. The result is that a lone hacker can now do the work that used to take large teams.

For example, I recently used AI to unpack an Android app's APK to figure out how it works for a personal project. Using secrets extracted from it, I could then connect to an external system with my custom-built app. And I'd never even built an Android app before AI, much less reverse engineered one.

AI no longer optional

Which brings us to the uncomfortable conclusion. The problem with AI in cybersecurity is twofold. It lowers the barrier so substantially that even non-hackers can hack, increasing the risks to all. And it hands knowledgeable, well-resourced state hackers far greater capabilities and faster execution speed.

In my view, the proliferation of AI tools has flipped cybersecurity on its head. It's no longer possible to defend organisations from cyberattacks with traditional non-AI tools.

Just as Hugging Face previously turned to AI to thwart OpenAI's rogue agents, AI on the defensive side is no longer optional.