verl-rl-training

1# verl: Volcano Engine Reinforcement Learning for LLMs
2 
3verl is a flexible, efficient, and production-ready RL training library for large language models from ByteDance's Seed team. It implements the HybridFlow framework (EuroSys 2025) and powers models like Doubao-1.5-pro achieving O1-level performance on math benchmarks.
4 
5## When to Use verl
6 
7**Choose verl when you need:**
8- Production-ready RL training at scale (tested up to 671B parameters)
9- Flexibility to swap backends (FSDP ↔ Megatron-LM ↔ vLLM ↔ SGLang)
10- Support for multiple RL algorithms (PPO, GRPO, RLOO, REINFORCE++, DAPO)
11- Multi-turn rollout with tool calling for agentic workflows
12- Vision-language model RL training
13 
14**Consider alternatives when:**
15- You need Megatron-native training → use **slime** or **miles**
16- You want PyTorch-native abstractions with Monarch → use **torchforge**
17- You only need simple SFT/DPO → use **TRL** or **Axolotl**
18 
19## Key Features
20 
21- **Training backends**: FSDP, FSDP2, Megatron-LM
22- **Rollout engines**: vLLM, SGLang, HuggingFace Transformers
23- **Algorithms**: PPO, GRPO, DAPO, RLOO, ReMax, REINFORCE++, SPIN, SPPO
24- **Models**: Qwen-3, Llama-3.1, DeepSeek, Gemma-2 (0.5B to 671B)
25- **Advanced**: LoRA RL, sequence parallelism, expert parallelism, multi-turn tools
26 
27## Installation
28 
29```bash
30# Option 1: pip install
31pip install verl[vllm]  # or verl[sglang] for SGLang backend
32 
33# Option 2: Docker (recommended for production)
34docker pull verlai/verl:vllm011.latest
35 
36# Option 3: From source
37…