transformer-lens-interpretability

1# TransformerLens: Mechanistic Interpretability for Transformers
2 
3TransformerLens is the de facto standard library for mechanistic interpretability research on GPT-style language models. Created by Neel Nanda and maintained by Bryce Meyer, it provides clean interfaces to inspect and manipulate model internals via HookPoints on every activation.
4 
5**GitHub**: [TransformerLensOrg/TransformerLens](https://github.com/TransformerLensOrg/TransformerLens) (2,900+ stars)
6 
7## When to Use TransformerLens
8 
9**Use TransformerLens when you need to:**
10- Reverse-engineer algorithms learned during training
11- Perform activation patching / causal tracing experiments
12- Study attention patterns and information flow
13- Analyze circuits (e.g., induction heads, IOI circuit)
14- Cache and inspect intermediate activations
15- Apply direct logit attribution
16 
17**Consider alternatives when:**
18- You need to work with non-transformer architectures → Use **nnsight** or **pyvene**
19- You want to train/analyze Sparse Autoencoders → Use **SAELens**
20- You need remote execution on massive models → Use **nnsight** with NDIF
21- You want higher-level causal intervention abstractions → Use **pyvene**
22 
23## Installation
24 
25```bash
26pip install transformer-lens
27```
28 
29For development version:
30```bash
31pip install git+https://github.com/TransformerLensOrg/TransformerLens
32```
33 
34## Core Concepts
35 
36### HookedTransformer
37 
38The main class that wraps transformer models with HookPoints on every activation:
39 
40```python
41…