Anthropic Calls for Global Pause in AI Development as 'Self-Improvement' Risks Loom

Summary

Anthropic is urging top artificial intelligence laboratories to slow the pace of AI development, citing significant risks posed by systems that could soon improve themselves without human intervention. In a blog post published Thursday by Marina Favaro and Jack Clark—the heads of its research institute and policy division—the company disclosed internal data documenting how rapidly its most advanced models are advancing, and flagged 'recursive self-improvement' as a critical threshold that could arrive sooner than most institutions are prepared for.

The company proposes establishing a global agreement on slowing frontier AI development, complete with verification mechanisms to ensure competitors comply. While emphasizing that recursive self-improvement has not yet occurred and is not inevitable, Anthropic argues that it would be prudent to create the option to pause development temporarily, allowing societal structures and alignment research to keep pace with technological advancement.

The call for restraint arrives amid Anthropic's own rapid scaling. The company recently closed a funding round that valued it at nearly $1 trillion and has filed confidential paperwork to begin the IPO process. The company's annualized revenue run-rate is on track to reach $50 billion by the end of June 2026, up dramatically from $9 billion at the end of 2025, cementing its position as the frontrunner in a fierce competition with OpenAI for AI supremacy.

• The policy call comes as Anthropic itself accelerates: nearly $1 trillion valuation, IPO filing, and $50B projected annualized revenue by end of Q2 2026