finetuning

1# Finetuning
2 
3Priors, not rules. Only firm guardrails: held-out eval you never train on, no leakage, trust evo's recorded numbers over the run's self-report. Override anything else against the gate.
4 
5## Pick the technique by reward shape
6 
7Decide on the reward first, technique second. Choosing the comfortable technique over the matching one is the most common failure.
8 
9| Reward shape | Technique |
10|---|---|
11| Verifiable (exact match, unit tests, parser-decidable) | **RL** (GRPO / RLOO / PPO) — reward includes format, so the model learns to emit verifier-acceptable shape |
12| Preference pairs (chosen vs rejected) | **DPO / KTO / ORPO** — cheaper than full RL, no rollouts |
13| Demonstrations only (curated traces, chat data) | **SFT** — install format/tone/capability the base lacks |
14| Have a scorer + want SFT stability | **RFT** — sample, filter by reward, SFT on survivors |
15 
16"SFT-then-RL" is not a law. For a competent base model on a verifiable benchmark, RL-from-base often beats SFT-then-RL end-to-end.
17 
18## Research the literature before the first commit
19 
20The decision tree above is the structural prior. The empirical answer for *this* model on *this* benchmark usually has a recent paper, blog, or HF Space recipe behind it -- and what beats baseline on a 4B base model in 2026 is not what the agent's pre-training data captures. Before picking the technique for `exp_0001` (the first experiment after baseline), invoke `evo:ideator` with a `literature` brief:
21 
22```
23Task(
24…