OpenAI, Anthropic and Google DeepMind are working on the creation of a body to oversee risks linked to artificial intelligence, as calls grow for stronger safeguards against the technology’s potential dangers. The project has already drawn criticism and faces the prospect of political opposition.
The idea grew out of a proposal by Demis Hassabis, chairman of Google DeepMind, who suggested creating a body modeled on the Financial Industry Regulatory Authority (FINRA), the US financial sector’s self-regulatory organization.
Debate over AI risks has intensified since the beginning of the summer, fueled by a series of incidents and dire warnings from several AI professionals.
Several AI models developed by OpenAI and Anthropic have notably left their controlled environments on their own, accessing the Internet and intruding on websites and platforms.
But establishing an industry-led oversight body, even if created solely at the sector’s initiative, would require legislation, particularly to prevent the AI industry from facing accusations of collusion and antitrust violations.
FINRA was created under federal law and operates under the supervision of the US Securities and Exchange Commission (SEC), which reports directly to the executive branch.
“FINRA can devise its own rules, but the SEC has to approve them,” said Andrew Tuch, a law professor at Washington University in St. Louis. All regulations governing FINRA “were either passed by Congress or produced under SEC supervision.”
The issue has become more politically sensitive after Anthropic CEO Dario Amodei called Saturday for a slowdown in AI development.
Donald Trump and several members of his administration have since reiterated their opposition to imposing tighter restrictions on the industry, officially citing concerns that China could gain an advantage.
They have dismissed warnings from several officials and developers, including Jacob Coxon a week earlier, about the risks AI poses to humanity, with the US president describing such warnings as “hoaxes.”
Fears of an AI cartel
Major AI companies have been calling on the US government for months to regulate the sector and oversee AI safety.
The Trump administration launched a process in early August to evaluate new AI models, but participation is voluntary and details of the mechanism have not been disclosed.
In a message posted in mid-July, Hassabis proposed an organization funded by companies in the sector, with a board made up of independent experts and representatives of the open-source AI ecosystem.
The latter includes developers of open AI models that are freely accessible, downloadable and modifiable, unlike closed models such as those developed by OpenAI and Anthropic.
Once established, the organization would require the most advanced AI companies to submit new models for review, with a particular focus on safety, before they could be released commercially.
“I don’t think a small group of AI companies should define the rules for everyone,” Aidan Gomez, CEO of Canadian AI company Cohere, said on CNBC. “There are a huge number of stakeholders who need to be consulted.”
According to Hassabis, the new body could also coordinate a slowdown in AI development if efforts to assess and manage the risks fail to keep pace with the technology’s progress.
“When humanity’s future is at stake, we need binding international rules, not voluntary industry standards,” Democratic Senator Bernie Sanders said Tuesday at the Pro-Human AI event in Washington.
Hassabis has suggested that the proposed self-regulatory body could serve as “a starting point for creating international standards for frontier AI.”
Several observers, particularly on the Republican side, have accused major AI companies of “regulatory capture,” arguing that they could seek to shape their own rules and coordinate to protect their competitive advantage at the expense of competitors and consumers.
David Sacks, an influential adviser to Donald Trump, has warned of the risk of an AI cartel, urging each company to act unilaterally even if that meant slowing its own development. Such a scenario appears unlikely given the scale of investment in the industry’s leading companies.
