K-Search Framework Automatically Translates CUDA Kernels to Apple Silicon, Achieving Near-Expert Performance
Key Takeaways
- ▸K-Search uses an AI-driven evolutionary search process—combining LLM reasoning, code generation, and real-world benchmarking—to automatically optimize GPU kernels, enabling transfer of decades of CUDA expertise without manual reimplementation
- ▸A novel CUDA-to-MLX translation layer encodes hardware rules, optimization patterns, and mathematical constraints to automatically adapt CUDA kernels to Apple Silicon architecture, addressing a significant performance gap in the rapidly-adopted MLX framework
- ▸Early benchmarks show near-expert-level results, with 0.97x speedup on Attention kernels and up to 20x speedup on State Space Model kernels, demonstrating the viability of AI-powered cross-architecture kernel porting
Summary
Researchers have extended K-Search, an AI-powered kernel optimization framework, with a novel CUDA-to-MLX translation layer that automatically converts GPU kernel optimizations from NVIDIA's CUDA ecosystem to Apple's MLX framework for Apple Silicon. This addresses a critical gap in Apple's machine learning ecosystem: while MLX has seen rapid adoption for local AI inference across hundreds of millions of MacBooks, it has lacked the performance-critical optimized kernels—such as paged attention, fused operations, and State Space Model implementations—that the mature NVIDIA ecosystem takes for granted.
The approach uses a structured translation process combined with K-Search's evolutionary kernel optimization, which leverages AI to iteratively refine kernel candidates against real hardware benchmarks. Early results demonstrate near-expert performance, achieving 0.97x speedup compared to MLX's native Attention kernel and up to 20x prefill speedup over community implementations on Mamba SSM kernels. The method is generalizable beyond Apple Silicon and MLX, offering a pathway to accelerate kernel development across emerging AI accelerator ecosystems.
- This work signals a shift in how performance-critical code is ported across hardware ecosystems—from labor-intensive manual translation to AI-assisted automation—with implications for Apple Silicon's competitive positioning for local AI inference
Editorial Opinion
This research represents a meaningful step toward democratizing high-performance AI across diverse hardware ecosystems. By automatically translating decades of CUDA optimization expertise to Apple Silicon, the work addresses a real and growing gap in Apple's MLX ecosystem and establishes a replicable pattern for similar ports to other emerging accelerators. The success of AI-powered kernel optimization suggests we're entering an era where architectural expertise, once locked into specific ecosystems, can be systematically transferred—potentially reshaping the economics of AI hardware fragmentation.
