miles-rl-training

1# miles: Enterprise-Grade RL for Large-Scale Model Training
2 
3miles is a high-performance, enterprise-ready RL framework optimized for large-scale model post-training. Built as a production fork of slime, it addresses critical challenges in MoE training stability, low-precision training, and train-inference alignment.
4 
5## When to Use miles
6 
7**Choose miles when you need:**
8- Training 1TB+ MoE models (DeepSeek V3, Qwen3-MoE)
9- FP8 or INT4 quantization-aware training
10- Bit-wise identical train-inference alignment
11- Speculative RL for maximum throughput
12- Production stability with enterprise support
13 
14**Consider alternatives when:**
15- You want the research-grade original → use **slime**
16- You need flexible backend swapping → use **verl**
17- You want PyTorch-native abstractions → use **torchforge**
18 
19## Key Features
20 
21### Low-Precision Training
22- **Unified FP8**: End-to-end FP8 for both inference and training
23- **INT4 QAT**: 1TB models on single-machine VRAM (H200)
24- **Rollout Routing Replay (R3)**: Bit-wise expert alignment for MoE
25 
26### Performance Optimizations
27- **Speculative RL**: 25%+ rollout speedup with online SFT draft models
28- **Zero-Copy Weight Sync**: CUDA IPC zero-copy mapping
29- **Partial Rollout**: Recycle half-finished trajectories
30 
31### Train-Inference Alignment
32- **TIS/MIS**: Truncated/Masked Importance Sampling for off-policy correction
33- **Kernel-level optimization**: FlashAttention-3, DeepGEMM integration
34 
35## Installation
36 
37```bash
38# Recommended: Docker
39…