Expert Survey Warns of 10% Catastrophic AI Risk Within 5 Years Without Action
Key Takeaways
- ▸272 AI experts assessed a 10%+ probability of catastrophic AI outcomes within 5 years without intervention
- ▸Top priority risks: AI cyberattacks, weapons development, competitive pressures in AI development, and governance failure
- ▸Even with mitigation efforts, 5 of 24 risk categories remain >10% likely to cause catastrophic harm
Summary
A new research study from the University of Queensland and MIT FutureTech surveyed 272 AI experts from industry, academia, government, and the broader community about catastrophic AI risks. The experts found at least a 10% probability of catastrophic AI outcomes over the next 5 years if no action is taken to reduce risk. Key priorities identified include AI cyberattacks, weapons development, competitive pressures in AI development, and governance failure. Catastrophic outcomes are defined as more than one million deaths, over $100 billion in financial losses, or global collapse of democracy and civil rights.
The research used the Delphi method to anonymously collect expert contributions across 24 AI risk categories. Even if pragmatic mitigations are adopted over the next 5 years, 5 of the 24 risk categories are estimated at more than 10% likely to cause catastrophic harm. The study emphasizes that many sectors are exposed to overlapping risks as AI becomes embedded in decision-critical systems, and highlights the competitive dynamics that disincentivize individual labs and governments from investing sufficiently in safety measures.
- Researchers call for laws, treaties, and collective-action mechanisms to address competitive safety incentives
Editorial Opinion
This study provides sobering evidence that the AI expert community views catastrophic and existential risks as genuinely plausible threats rather than speculative concerns. The fact that even optimistic scenarios with mitigation efforts still leave substantial catastrophic risk suggests current safety approaches are inadequate. Given that comparable risks in nuclear power or aviation would be considered intolerable, policymakers and AI developers should treat these expert assessments as a urgent call to accelerate governance, regulation, and safety investment.
