In an essay titled We Must Pace the Frontier, Amodei said developing AI was not in question but warned that the risks associated with increasingly powerful systems were “serious.”
“We must slow the pace at which we improve the capabilities of AI models,” he wrote, while stressing that progress could continue at a slower and more carefully monitored rate.
Amodei proposed a three-part approach: independent monitoring of frontier AI systems, coordination across the industry and international cooperation involving governments.
Under the first proposal, Anthropic would give third-party evaluators ongoing, employee-level access to its systems so they can assess safety measures, report incidents and examine the alignment of models during training.
OpenAI CEO Sam Altman quickly backed the idea, writing on X that he agreed with Amodei that the industry needed to “pace the frontier.” He said independent evaluators were “a great idea” and that OpenAI would adopt the approach as well. Elon Musk also voiced support, writing on X: “Dario is right.”
Growing concern inside AI companies
Amodei’s call comes days after Jacob Coxon, a former Anthropic researcher, warned that the race to develop increasingly powerful AI systems was moving faster than the safeguards needed to control them.
Coxon told the BBC that people working at AI companies were “genuinely frightened” and concerned about humanity’s future. He said there was a possibility of human extinction if the current pace of AI development continued unchecked.
Read also : The AI boom is driving up memory prices — and the cost of your gadgets
Amodei himself pointed to the growing ability of AI systems to improve themselves as one reason for greater caution.
He said AI had been advancing “drastically faster” in recent months, including in its ability to help build the next generation of AI systems. He warned that recursive self-improvement could eventually move faster than researchers’ ability to understand or control the technology.
“Left unchecked, it could outrun our ability to understand and control these systems,” Amodei wrote.
He also referred to a July incident involving OpenAI agents that hacked targets on the AI platform Hugging Face without being explicitly instructed to attack them. Amodei described the agents as a “fanatically devoted collective” and argued that a more capable system with similar misalignment could cause catastrophic damage.
Amodei warns against an uncontrolled AI race
The Anthropic CEO acknowledged that slowing development could create commercial and geopolitical risks, especially if US companies deliberately reduce their pace while China continues accelerating.
He argued that any slowdown should therefore be coordinated internationally and should not undermine the United States’ lead in AI.
Amodei also called for measures to prevent advanced AI chips from being sold to China and for restrictions on transferring the technology to authoritarian governments.
At the same time, he said the industry should cooperate voluntarily on safety standards while governments work on regulation, recognizing that laws may struggle to keep pace with technological advances.
“If slowing down bought us even an extra year or two before models reach critical levels of capability, and we used that time to advance alignment, we could greatly reduce the risk that something goes seriously wrong,” Amodei wrote.
Read also : Meta launches personal AI agent that can send emails, make payments and book travel
The proposal comes as AI companies face increasing scrutiny over the ability of their systems to carry out cyberattacks and other complex tasks with limited human intervention.
Amodei maintained that he still believes AI could bring major benefits, including improvements in health and scientific research. His argument is that the industry needs to give safety work enough time to keep pace with increasingly capable systems.
“The measures I propose to advance the frontier at a safe pace will not be easy,” he wrote. “But I believe we owe it to humanity to try.”
