ray-train

1# Ray Train - Distributed Training Orchestration
2 
3## Quick start
4 
5Ray Train scales machine learning training from single GPU to multi-node clusters with minimal code changes.
6 
7**Installation**:
8```bash
9pip install -U "ray[train]"
10```
11 
12**Basic PyTorch training** (single node):
13 
14```python
15import ray
16from ray import train
17from ray.train import ScalingConfig
18from ray.train.torch import TorchTrainer
19import torch
20import torch.nn as nn
21 
22# Define training function
23def train_func(config):
24    # Your normal PyTorch code
25    model = nn.Linear(10, 1)
26    optimizer = torch.optim.SGD(model.parameters(), lr=0.01)
27 
28    # Prepare for distributed (Ray handles device placement)
29    model = train.torch.prepare_model(model)
30 
31    for epoch in range(10):
32        # Your training loop
33        output = model(torch.randn(32, 10))
34        loss = output.sum()
35        loss.backward()
36        optimizer.step()
37        optimizer.zero_grad()
38 
39        # Report metrics (logged automatically)
40        train.report({"loss": loss.item(), "epoch": epoch})
41 
42# Run distributed training
43trainer = TorchTrainer(
44    train_func,
45    scaling_config=ScalingConfig(
46        num_workers=4,  # 4 GPUs/workers
47        use_gpu=True
48    )
49)
50 
51result = trainer.fit()
52print(f"Final loss: {result.metrics['loss']}")
53```
54 
55**That's it!** Ray handles:
56- Distributed coordination
57- GPU allocation
58- Fault tolerance
59- Checkpointing
60- Metric aggregation
61 
62## Common workflows
63 
64### Workflow 1: Scale existing PyTorch code
65…