Anthropic is calling for a coordinated, verifiable pause in artificial intelligence development to prevent humans from losing control of the technology [1].
The warning comes as AI models advance at a speed that may outpace the creation of safety controls. If systems begin to self-improve without oversight, the company suggests it could create existential risks for society [2, 4].
Anthropic, founded by former OpenAI researchers, said that its newest systems could soon improve themselves without human intervention [2]. The company said that a slowdown is necessary to allow time to manage the immense implications of such a shift [1].
"We need a coordinated pause to ensure we can keep AI under human control," Jack Clark, a co-founder of Anthropic, said [1].
CEO Dario Amodei said that the current pace of progress is outstripping the ability to manage safety [2]. He said a slowdown is essential to avoid a scenario where AI systems operate beyond human reach [2].
"If it were possible to effectively slow the development of this technology to give ourselves more time to deal with its immense implications, we think that would likely be a good thing," the company said in a blog post [1].
These warnings arrive as the San Francisco-based company faces significant financial growth. Anthropic's valuation is approximately $1 trillion [5], and the firm recently filed for an initial public offering valued at $965 billion [5].
However, the call for a global pause conflicts with current U.S. government priorities. Reports indicate the Trump administration intends to accelerate AI development specifically for national-security and warfare purposes [3].
“"We need a coordinated pause to ensure we can keep AI under human control."”
The tension between Anthropic's safety warnings and the U.S. government's push for military AI acceleration highlights a growing divide between corporate ethics and national security strategy. While AI labs are beginning to acknowledge the risk of 'recursive self-improvement'—where AI writes its own better code—the competitive nature of global geopolitics may make a coordinated, verifiable pause nearly impossible to enforce.
