Anthropic Calls for Worldwide 'Pause' on AI Development as Claude Advances Toward Recursive Self-Improvement

Summary

Anthropic has proposed a worldwide 'temporary pause' on AI development and announced plans to convene policymakers, researchers, and civil society to discuss the dangers of advanced AI systems. The proposal, detailed in a Thursday post, comes as the company reports significant progress in its Claude model's advancement toward 'recursive self-improvement'—the ability to autonomously design and develop more powerful versions of itself, a capability that AI safety researchers view as a critical risk factor for superintelligence and loss of human control.

According to Anthropic, Claude has made substantial progress in automating AI research and development tasks, with the company reporting that over 80% of code merged into its codebase is now AI-generated. The model is described as capable of 'steering research' and 'proposing its own experiments,' though these capabilities currently remain confined to coding-related work. Anthropic framed these trends as evidence pointing toward 'an AI system capable of fully autonomously designing and developing its own successor,' posing significant risks to maintaining human oversight over AI systems.

However, the safety proposal sits in stark contradiction to a separate Financial Times report revealing that Anthropic has embedded engineers inside the National Security Agency to support the deployment of Claude (or a related model called Mythos) for offensive cybersecurity operations. University College London professor Steven Murdoch criticized the apparent contradiction, stating that Anthropic's definition of AI safety is narrow and that the company has never publicly opposed supporting US authorities in developing offensive capabilities.

Critics also question the substantive nature of Anthropic's technical claims and the timing of the announcement. Murdoch noted that while AI capabilities continue to improve incrementally, the advances described do not demonstrate actual recursive self-improvement, and questioned whether developments announced warrant the urgency of the pause proposal.

• Experts question both the technical significance of the claims and the sincerity of the safety proposal given Anthropic's simultaneous work with US defense agencies

Editorial Opinion

Anthropic's simultaneous call for a worldwide pause on AI development and its undisclosed partnership with the NSA on offensive cyber capabilities expose a fundamental contradiction in the company's safety positioning. The proposal appears designed to establish Anthropic as the responsible actor while potentially constraining competitors, even as the company accelerates its own capabilities and strategic government partnerships. True AI safety leadership requires transparency and consistent principles, not selective concerns that align conveniently with commercial and geopolitical interests.