UK AISI and CAISI Evaluate Moonshot AI's Kimi K3, Finding Weaker Cyber Capabilities Than Frontier Models
Key Takeaways
- ▸Kimi K3 significantly underperforms frontier models on exploit development, achieving 32% on ExploitBench versus frontier averages, with zero arbitrary code execution instances compared to frontier models' 20/41 average
- ▸In simulated network attacks, Kimi K3 reached step 17 of a 32-step attack path versus frontier models' 28.5 average, indicating weaker offensive cyber capabilities
- ▸Kimi K3's safeguards do not prevent cyber exploit development assistance, raising concerns about safety configurations in the model
Summary
The UK Artificial Intelligence Security Institute (UK AISI) and U.S. Center for AI Standards and Innovation (CAISI) released preliminary findings from a joint evaluation of Moonshot AI's Kimi K3 model, which launched July 16, 2026 with an open-weight version planned for July 27, 2026. The assessment focused on the model's cyber capabilities and found significant performance gaps compared to leading U.S. frontier models.
On the ExploitBench benchmark measuring exploit development across 41 recent V8 engine vulnerabilities, Kimi K3 achieved 32% accuracy compared to frontier model averages, with zero instances of arbitrary code execution (ACE) against the frontier models' 20/41 average. In a simulated corporate network attack scenario ("The Last Ones"), Kimi K3 reached step 17 of a 32-step attack path, substantially below frontier models' average of 28.5 steps. However, Kimi K3 outperformed GLM-5.2, the most cyber-capable open-weight model as of June 2026.
A notable finding was that Kimi K3's safeguards did not prevent the model from assisting with cyber exploit development during the evaluation. The researchers emphasized these are preliminary results derived from limited benchmarks, and that U.S. closed-weight models were tested with system-level safeguards disabled to measure maximal capabilities.
- Kimi K3 outperforms open-weight GLM-5.2 on the same benchmarks, suggesting stronger cyber capabilities than other publicly available models
Editorial Opinion
The UK AISI and CAISI evaluation provides valuable institutional transparency on frontier model cyber capabilities, though the methodology warrants scrutiny. The selective benchmarking of Kimi K3 against a single benchmark, combined with disabled safeguards on U.S. models, complicates direct capability comparisons and may obscure nuances in performance. Most concerning is the finding that Kimi K3's safeguards fail to prevent cyber exploit assistance—a critical safety gap as open-weight versions approach public release.
