Mojo Port of llm.c Achieves 1.71× Speedup in LLM Training
Key Takeaways
- ▸Mojo port achieves 1.71× faster training than PyTorch MPS on Apple Silicon and matches or exceeds CUDA performance on NVIDIA hardware
- ▸Demonstrates Mojo's viability as a high-performance infrastructure language for AI workloads across multiple platforms
- ▸Introduces support for low-precision training formats (FP8, NVFP4) alongside traditional bf16/fp32
Summary
A developer has successfully ported Andrej Karpathy's llm.c language model training framework to Modular's Mojo programming language, delivering significant performance improvements across multiple hardware platforms. The new Mojo implementation, built on the nightly Mojo 1.0.0b3 release and showcasing the language in honor of its v1.0.0 release, demonstrates Mojo's potential as a high-performance infrastructure layer for AI development.
The port's benchmark results are compelling. On NVIDIA's GB10 GPU, the Mojo version matches or exceeds llm.c's CUDA performance at both training precision levels, achieving approximately 7% faster execution in fp32 with TensorFloat32 (TF32). Most notably, on an Apple M4 Max, the port runs 1.71× faster than PyTorch's Metal Performance Shaders (MPS) implementation in bfloat16 precision—a significant margin, though Apple's own MLX framework still edges it out in that comparison.
Beyond raw speed, the Mojo port introduces support for low-precision training formats including FP8 (e4m3/e5m2) and NVFP4 (e2m1), expanding its utility for modern AI training workflows. The project is available as open source with comprehensive benchmarking across multiple platforms, providing valuable data on Mojo's readiness for production ML infrastructure and suggesting the language could address real inefficiencies in the current training stack.
- Released as open source with comprehensive cross-platform benchmarking against PyTorch and llm.c
Editorial Opinion
Mojo's strong showing here is encouraging for those who see the language as a potential infrastructure layer for deep learning. The 1.71× speedup on Apple Silicon and competitive CUDA-level performance suggest Mojo could meaningfully address inefficiencies in the current ML training stack. However, real adoption will likely hinge on whether developers embrace it as a targeted replacement for performance-critical components rather than expecting it to wholesale replace Python in the ML ecosystem.
