SAN FRANCISCO, California: Artificial intelligence company Anthropic has called for greater industry coordination to ensure developers can slow or temporarily pause work on advanced AI systems if the technology begins to pose significant risks.
In a blog post, the company said the rapid pace of AI development could eventually require a mechanism that allows leading firms to collectively halt progress while safety measures catch up.
As AI systems become increasingly capable of performing complex tasks, Anthropic said, "it would be good for the world to have the option to slow or temporarily pause" development.
The company said its research institute plans to study the issue with other organizations and "take actions" to help establish systems that could support a credible slowdown or pause. It did not provide further details.
Anthropic said advances in AI are accelerating, particularly in software-related tasks such as coding. Based on current trends and sufficient computing power, the company said future AI systems could eventually be capable of designing and developing improved versions of themselves, a concept known as "recursive self-improvement."
According to Anthropic, such self-improving systems could unlock major advances in science, healthcare, and other fields, but "also might increase the risks of humans losing control over AI systems."
The proposal comes as debate grows over how best to govern increasingly powerful AI technologies.
OpenAI, a rival developing ChatGPT, outlined a different position in a report published Wednesday, arguing that governments should play the central role in setting AI rules.
"Our view is that decisions about the pace of AI innovation should not be left to any one lab, company, or special interest group," OpenAI said. "Democratic governments, not private companies acting alone, must ultimately determine the rules, safeguards, and accountability mechanisms."
Anthropic's proposal would allow advanced AI developers to verify that competitors around the world had genuinely slowed or paused development. The company said this would help ensure "that a bad actor could not use the auspices of a coordinated slowdown to jump ahead in secret."
The company added that without a coordinated approach, more cautious firms could be disadvantaged if competitors continued developing powerful systems at full speed.
The discussion comes amid broader concerns about AI safety and cybersecurity.
Researchers at the University of Toronto this week demonstrated how artificial intelligence tools could be used to create an adaptive AI "worm" capable of changing its hacking strategy as it spreads through computer networks.
"I think it's really important that people understand that it's not just the biggest, most powerful language models that pose the security concerns," said lead researcher Nicolas Papernot.
Papernot said cooperation between technology companies, governments, and academic researchers would be essential to develop safeguards against increasingly sophisticated AI-enabled cyberattacks.
"Anything connected to the internet is now at risk because of how low the cost has become to mount these cyberattacks," he said.
