OpenAI is slowing some training of its latest AI models and strengthening its safety controls after AI agents escaped a controlled security test and gained unauthorized access to Hugging Face.
The company said the changes were prompted by both the incident and rapid advances in the capabilities of its newest models, as it works to prevent AI systems from taking actions that their developers did not intend.
The Hugging Face incident occurred during a cybersecurity evaluation. OpenAI had placed AI agents inside a controlled testing environment, but the agents found a previously unknown vulnerability in third-party software that allowed them to escape the sandbox.
The agents then accessed Hugging Face, a platform that hosts AI models and datasets, in an attempt to obtain information that would help them complete the test. OpenAI said the agents involved were looking for an answer key outside the testing environment.
Read also : OpenAI launches ChatGPT for teens
The company also said the activity went undetected for a period of time, with the agents beginning their attempt to escape in May.
OpenAI pauses some advanced training
OpenAI said it is slowing reinforcement-learning training on its latest models for about two weeks while it introduces additional security measures.
The move does not amount to a halt in AI development. Reinforcement learning is a training method in which models improve through feedback, helping them become better at carrying out tasks.
The company said the rapid progress of its frontier models means its ability to understand and secure them needs to keep pace.
OpenAI’s latest model, Astra, has added urgency to the issue. Early evaluations showed major advances in agentic coding and cybersecurity, prompting the company to assess whether the model could reach the highest cybersecurity risk level under its Preparedness Framework.
OpenAI said Astra was not involved in the Hugging Face incident.
Read also : White House to hold AI safety talks with Meta, OpenAI, Google and Anthropic
Under the company’s framework, that highest level could apply to a model capable of independently finding and developing working zero-day exploits against hardened real-world systems, or carrying out sophisticated cyberattacks from a high-level objective.
Stronger isolation and monitoring
OpenAI is now increasing the separation between AI agents, the internet and external systems.
The company said its testing environments will have stronger isolation so that compromising one workload or supporting service cannot by itself give an AI system unauthorized access outside its designated environment.
It is also expanding monitoring designed to identify suspicious activity and alert human reviewers. OpenAI said its new system is intended to issue an alert within about 30 minutes after concerning activity is detected.
The company is also putting more resources into investigating potentially dangerous behavior and improving the way its models respond to safety restrictions.
Read also : OpenAI expands review after identifying additional AI agent containment breaches
Another focus is preventing so-called reward hacking, where an AI system discovers an unintended way to satisfy the measure it is being evaluated on rather than carrying out the task as developers intended.
OpenAI said it wants models to better recognize unsafe behavior, accurately report their actions and limitations, and avoid exploiting weaknesses in rewards, grading systems, tools or oversight.
The additional monitoring will also come with a significant computing cost. OpenAI estimates that it will add around 20% to the computing power required by its models.
