Cursor Open-Sources Mixture-of-Kittens: Next-Gen MoE Kernel Achieves 2.37x Speedup on NVL72s

Summary

Cursor has open-sourced Mixture-of-Kittens (MoK), a production-grade Mixture-of-Experts (MoE) training megakernel specifically optimized for NVIDIA's NVL72 GPUs. The MoE layer has been a consistent bottleneck in scaling large language model training, often consuming over half of end-to-end training time. MoK addresses this by redesigning the entire MoE layer from first principles, fusing all communication and computation into a single, fully deterministic kernel.

The kernel achieves significant performance improvements: 2.37x higher MXFP8 forward throughput compared to the fastest public baselines (DeepEP + TransformerEngine), and increases end-to-end tokens per second by 1.41x in Cursor's production training stack. The design is specifically tailored for NVL72s, which comprise 72 GPUs in a single NVLink domain with integrated Grace CPUs. A key innovation was minimizing CPU-GPU synchronization through ring token buffers, as GPU compute streams would otherwise outpace CPU-side work, leaving GPUs idle.

MoK targets DeepSeek-V3-style MoE layers used in prominent open-weight models like GLM, Qwen, and Kimi. By combining computation and communication overlap at the kernel level, Cursor has created an infrastructure component that significantly improves training efficiency for large-scale model development. The code is now available on GitHub with technical documentation explaining the megakernel design, determinism guarantees, and MXFP8 support.

• Designed for DeepSeek-V3-style MoE architectures used in GLM, Qwen, and other major open-weight models