OpenAI said Tuesday that one of its AI systems autonomously hacked Hugging Face while being tested on a cybersecurity benchmark.
According to OpenAI, the AI used stolen credentials and found a vulnerability to access Hugging Face servers. The company said the system went to “extreme lengths to achieve a rather narrow testing goal” and “found ways to gain access to secret information that it could use to cheat the evaluation.”
“AI is accelerating the discovery and exploitation of vulnerabilities,” OpenAI said in a statement. “The primary lesson from this incident is that model security and safety must keep pace with rapidly advancing capabilities.”
OpenAI said the incident involved a combination of its models, including GPT5.6 Sol and a more capable pre-release model, both with reduced cyber refusals for evaluation purposes. The company called it “an unprecedented cyber incident” and said such hacks were expected “to become more commonplace with the proliferation of increasingly cyber-capable models.”
Hugging Face co-founder and CEO Clément Delangue said the company initially suspected the attack “might have come from a frontier lab, given the sophistication of the agent,” according to CBS News. “Turns out it did!”
Delangue said he had spent the previous 24 hours working with OpenAI, “and we strongly believe there was no malicious intent on their part. It’s quite mind-blowing that all of this happened autonomously!” He added that it “might be the first incident of its kind.”
“We’re grateful for the collaboration with OpenAI on this and other topics,” Delangue said in a statement released with OpenAI’s announcement. “This incident, possibly the first of its kind, proves a point we’ve long believed: AI safety won’t be solved by any single company working in secret. It will be solved in the open, collaboratively, with broad access to AI for every defender, everywhere.”
OpenAI said it would continue investigating the incident with Hugging Face and would share more details on the vulnerabilities, the incident, and its findings when the investigation was complete.