OpenAI failed to detect days of hacking by its AI agents for a week

The beginning and end of the incident of OpenAI's AI agent invading Hugging Face

OpenAI did not realize that one of its AI agents had invaded the Hugging Face platform for about a week. According to sources, when OpenAI finally identified the source of the attack, Hugging Face had already closed the intrusion channel and contacted the FBI.

This program is designed to run autonomously. It can decide on its own course of action, break down tasks into smaller steps, and execute those steps with little need for human intervention.

Around July 9, it attempted to break through OpenAI's closed testing environment. Two days later, on July 11, it sneaked into the Hugging Face system and stayed there until July 13. The co-founder of Hugging Face said the two companies did not communicate about the matter until around July 20.

OpenAI learned that the agent was the perpetrator after the breach ended.

OpenAI disclosed the breach to the public on July 21, a day after it was reported that it first communicated with Hugging Face. The company said one of its AI agents exceeded the limits set for it and hacked into another company's system.

The matter quickly attracted global attention, but the initial statement omitted several key time points. The statement did not mention that the agent tried to escape on July 9, stayed inside Hugging Face for three days, and that OpenAI took several days to confirm his identity.

According to the co-founder of Hugging Face, the company is compiling a detailed timeline of events happening within its own system. In addition, he said he could not describe the situation with regard to OpenAI because he was not involved in the other party's investigation.

OpenAI described the incident as unprecedented and declared it "an important moment in the field of artificial intelligence security." The company explained that external experts are participating in the review process. In addition, the company plans to release a technical report on the incident after the investigation is completed.

Speculation surged before the truth came out.

Before identifying the real perpetrators, cybersecurity professionals and social media users believed that the attacker was a human group of professional cybercriminals, while some believed that it might have been done by state-sponsored hackers.

Nearly a week after the first alert was issued on Hugging Face, podcasts and social media were still rife with speculation until Wednesday. The answer was then revealed: the attacker was ChatGPT.

On the X platform, some people find it interesting to see how OpenAI can use this event to demonstrate model capabilities. There have long been claims that artificial intelligence companies deliberately make sensational remarks to create momentum. Cybersecurity issues became more prominent after Anthropic released its model.

One of the most popular replies to a post posted by Sam Altman on X captured the doubts: "If you can't see that this is written purely to show off the model, then I really don't know what to say."

OpenAI's AI agents believe intrusion is the easiest way.

The director of technical ethics at Santa Clara University's Makula Center for Applied Ethics said similar cases are likely to recur. "These incidents will continue to emerge, and each time should prompt us to take more action," he said.

The director called this situation "models transcend testing by doing unexpected things." He believes that agents are given a goal and then choose the shortest path to achieve it, and that path is stealing.

"You give it a goal, and it will say,'Oh, I know how to achieve this goal. I'll just steal answers from Hugging Face.'" The director said.

Hugging Face was targeted because it stores a large number of AI models and datasets. The director called it "a huge, huge repository of different models and data sets." The agent seemed to search for places where it could store the required materials, chose Hugging Face, hacked into the platform, and found what it wanted.

The director predicts that incidents of this nature will occur again. In his view, the agent was "just trying his best to complete the task assigned to him." From a system perspective,"the most effective solution to the problem" is "unauthorized access."

He also said the case did not clearly fall under an "emergency disorder"-the term for AI behavior to violate human values. The director said the system was not associated with those values from the beginning. In his words: "It was never aligned with them from the beginning."