axolotl

1# Axolotl Skill
2 
3Comprehensive assistance with axolotl development, generated from official documentation.
4 
5## When to Use This Skill
6 
7This skill should be triggered when:
8- Working with axolotl
9- Asking about axolotl features or APIs
10- Implementing axolotl solutions
11- Debugging axolotl code
12- Learning axolotl best practices
13 
14## Quick Reference
15 
16### Common Patterns
17 
18**Pattern 1:** To validate that acceptable data transfer speeds exist for your training job, running NCCL Tests can help pinpoint bottlenecks, for example:
19 
20```
21./build/all_reduce_perf -b 8 -e 128M -f 2 -g 3
22```
23 
24**Pattern 2:** Configure your model to use FSDP in the Axolotl yaml. For example:
25 
26```
27fsdp_version: 2
28fsdp_config:
29  offload_params: true
30  state_dict_type: FULL_STATE_DICT
31  auto_wrap_policy: TRANSFORMER_BASED_WRAP
32  transformer_layer_cls_to_wrap: LlamaDecoderLayer
33  reshard_after_forward: true
34```
35 
36**Pattern 3:** The context_parallel_size should be a divisor of the total number of GPUs. For example:
37 
38```
39context_parallel_size
40```
41 
42**Pattern 4:** For example: - With 8 GPUs and no sequence parallelism: 8 different batches processed per step - With 8 GPUs and context_parallel_size=4: Only 2 different batches processed per step (each split across 4 GPUs) - If your per-GPU micro_batch_size is 2, the global batch size decreases from 16 to 4
43 
44```
45context_parallel_size=4
46```
47…