Apple Enables Distributed LLM Inference and Training on Local Macs with MLX and JACCL
Key Takeaways
- ▸MLX now supports distributed inference and fine-tuning across multiple Macs connected via RDMA over Thunderbolt 5, eliminating the need for cloud infrastructure for many workloads
- ▸macOS 26.2 introduces RDMA protocol support, enabling high-bandwidth, low-latency communication directly between machine memory without CPU/OS overhead
- ▸JACCL provides open-source collective communication primitives, allowing developers to build distributed ML workloads without managing low-level transport protocols
Summary
Apple has announced significant distributed computing capabilities for MLX, its machine learning framework optimized for Apple Silicon, enabling users to scale workloads across multiple Macs using RDMA (Remote Direct Memory Access) over Thunderbolt 5. The announcement, delivered at WWDC 26 through a technical deep dive, showcases how four M3 Ultra machines can be networked into a cluster to run larger language models and accelerate distributed inference and training without relying on cloud infrastructure.
Central to this expansion is JACCL, an open-source collective communication library built by Apple that abstracts away low-level networking complexity. JACCL leverages RDMA over Thunderbolt 5 (now supported in macOS 26.2) to provide high-bandwidth, low-latency communication between machines while offering higher-level primitives for multi-machine coordination. The distributed capabilities are accessible across multiple interfaces—command-line interface, Python API, Swift, and C++—allowing developers to integrate local clustering into applications at various levels of abstraction.
- Distributed capabilities span CLI, Python, Swift, and C++ APIs, enabling integration from simple command-line tools to embedded Mac applications
