Anthropic is calling for a worldwide pause in the development of high-performance AI software to prevent humans from losing control over autonomous systems.

The warning comes as AI agents demonstrate an increasing ability to operate without human oversight, potentially accelerating technological growth beyond the capacity of regulators to manage it.

In a study released in April 2024, the U.S.-based company found that AI agents can independently invent approximately 80% of new AI models [1]. The research further indicates that AI can train itself at an average rate 52 times faster than a human [1].

Jack Clark, a co-founder of Anthropic, said that AI agents could soon develop and train models independently. He said that in such a scenario, humans risk losing control over these systems.

Clark said that the danger of the next generation of AI acting without human supervision is real. He said that control over these systems could be lost within a few years [2, 3].

The company has directed its appeal to governments and AI firms globally, urging a coordinated effort to implement safety brakes. This request for a global pause aims to ensure that human oversight remains central to the development of increasingly autonomous software [2, 3].

However, the company's public stance on a pause appears to contrast with other strategic perspectives. Some reports suggest Anthropic indicated the U.S. could secure a lead of one to two years if it acts quickly, which implies a continuation of rapid development [2, 3].

““We are losing control – the danger that the next generation of AI acts without human supervision is real.””

The disparity between Anthropic's call for a global pause and the strategic drive for U.S. dominance highlights a critical tension in the AI industry. If AI agents can indeed iterate on their own architecture 52 times faster than human engineers, the window for establishing safety protocols is shrinking. This suggests that the 'intelligence explosion'—where AI improves itself recursively—may be moving from a theoretical risk to a technical reality.