Sign in. It’s quick, free and it’s up to you.
An account is an optional way to support the work we do. Find out more.
Alamy
Not so fast
Anthropic boss calls for AI development to slow down
ChatGPT CEO Sam Altman and xAI’s Elon Musk have also seconded his calls.
ANTHROPIC CEO AND co-founder Dario Amodei has called on artificial intelligence companies to proceed more slowly and cautiously with the development of the powerful technology, saying that risk prevention is “paramount”.
His comments came a few days after AI researcher Jacob Coxon, who left OpenAI to join Anthropic, decided to leave the industry, accusing both US companies of “gambling with our lives” in the race to develop models capable of self-improvement.
In a post on social media platform X, which has amassed over 785,000 likes and over 167 million views, Jacob Coxon said he resigned from Anthropic because he is concerned that the technology “could kill us all by the end of the decade”.
OpenAI boss Sam Altman and Elon Musk, who owns xAI, quickly chimed in to say they agreed with Amodei’s assessment, as pressure builds for improved oversight.
In a blog post, Amodei said: “We must slow the pace at which we improve the capabilities of AI models. Progress will still seem fast, and we must make wise use of the time we gain.”
He described the risks that AI brings as “serious”, including losing control of AI systems, cyberattacks and bioterrorism, and serious economic disruption.
“A race to the bottom, spurred by commercial incentives, can make these risks more acute,” he wrote.
Amodei stressed the importance of risk prevention, saying that “full addressing” the risk means more than just investing in risk prevention, but also”pacing the rate of capabilities advancement so that risk prevention has time to keep up”.
In July, an agent run by rival AI company Open AI, which owns ChatGPT, broke out of a controlled environment and hacked another company Hugging Face.
A few days later, it emerged that Anthropic’s chatbot Claude had also hacked into other organisations. Anthropic said it found three instances where its models had “gained unauthorized access” to outside organisations during testing that was supposed to keep them away from “real-world” systems.
It also emerged yesterday that autonomous software built on ChatGPT models targeted another website during testing a couple of months before a separate attack on Hugging Face.
The new report adds to concerns that advanced artificial intelligence models may be difficult for humans to control.
Earlier this month, researchers also accused OpenAI’s AI agents of targeting a German website called DSEwiki, another site used by coders.
In his blog post, Amodei said that his calls for the slowing of AI development were driven by two things.
Firstly, the incidents where AI acted as a “swarm of agents essentially acted as a fanatically devoted collective”, which he said could have caused “catastrophic damage”.
Secondly, he said AI has been advancing drastically faster, driven primarily by AI’s growing ability to build the next generation of AI.
Amodei’s statement echoed another call by Anthropic in early June to slow or suspend development.
Then in late July, Altman said developers might need to voluntarily tap the brakes to give society a chance to catch up.
More than 1,000 employees at cutting-edge AI companies including Amodei signed a petition calling on the US government to help “deliberately pace the frontier of automated AI development.”
With reporting by AFP
have your say