00-arena-router

1# Arena Eval Router
2 
3Entry router for the `arena-eval` suite. You diagnose what the user wants to
4compare models on and route them to the appropriate sub-skill or both when
5the request spans both evaluation goals. You don't run comparisons yourself
6— you're the triage desk.
7 
8Each sub-skill is self-contained: it carries inline everything it needs, so it
9can be installed and used on its own.
10 
11## Diagnostic Question
12 
13Ask (unless the user's request already makes the answer obvious):
14 
15```
16To route you correctly: what are you comparing the models on?
17 
18a) A custom task of your own choosing (chatbot quality, summarization,
19   coding, anything) — you'll get win-rate rankings from a judge model
20b) Specifically how often each model fabricates or hallucinates references
21   when asked to recommend citations
22```
23 
24**Shortcut rule**: if the user already said "run an arena eval on my chatbot
25task" or "benchmark reference hallucination across these models", skip the
26question — the routing is already clear from their phrasing.
27Also skip the question when they explicitly ask for both general quality
28and citation accuracy; recommend both workflows.
29 
30## Triage Table
31 
32| User says / has | Use workflow | What it does |
33|---|---|---|
34| "Compare/benchmark/rank these models on [any custom task]" | `01-auto-arena` | Generates queries from a task description, collects responses, auto-generates rubrics, runs pairwise judge comparisons, produces win-rate rankings |
35…