RadixArk Launches Miles: Open-Source Framework for Production-Scale LLM Reinforcement Learning
Key Takeaways
- ▸Miles is an open-source framework built natively on SGLang, Megatron-LM, Ray, and PyTorch for coordinating large-scale LLM RL workloads
- ▸The framework addresses distributed systems challenges inherent to modern RL post-training: rollout-training synchronization, MoE routing consistency, weight synchronization, and fault tolerance
- ▸Small-core, pluggable architecture allows researchers and teams to customize rollout logic, reward computation, and loss functions without modifying the framework itself
Summary
RadixArk has released Miles, an open-source framework designed to simplify large-scale LLM reinforcement learning post-training. As LLMs grow larger and transition to mixture-of-experts architectures, RL post-training has become a complex distributed systems challenge—requiring careful coordination between rollout generation, policy training, and weight synchronization across specialized hardware like NVIDIA Blackwell and Hopper GPUs.
Miles addresses this by composing four proven technologies: SGLang for high-throughput rollout generation, NVIDIA Megatron-LM for scalable distributed training, Ray for cluster orchestration, and PyTorch as the underlying compute layer. The framework features a small, pluggable core trainer that researchers and infrastructure teams can customize without forking the codebase.
The system handles critical distributed-systems concerns out of the box: MoE-aware rollout and training alignment, low-precision recipes, fast NVIDIA NCCL/RDMA weight synchronization, observability, checkpointing, and fault tolerance. By separating the compact training loop from pluggable components (rollout logic, reward computation, loss functions, sample filtering, metrics, and hooks), Miles aims to make frontier-scale LLM RL both more reproducible and easier to scale.
- Built-in support for low-precision training, disaggregated/colocated worker placement, and long-running job observability reduces operational complexity
Editorial Opinion
Miles represents a practical answer to a real infrastructure problem: as LLM RL becomes central to model post-training and models scale beyond billions of parameters, the engineering burden has shifted from research to systems. By open-sourcing a composable, production-grade framework, RadixArk is lowering the barrier for the broader community to run frontier-scale RL while maintaining the flexibility researchers need to innovate on algorithms. The focus on pluggable components—rather than a monolithic pipeline—is a smart design choice that should age well as RL techniques evolve.
