advanced-evaluation

1# Advanced Evaluation
2 
3This skill covers production-grade techniques for evaluating LLM outputs using LLMs as judges. It synthesizes research from academic papers, industry practices, and practical implementation experience into actionable patterns for building reliable evaluation systems.
4 
5**Key insight**: LLM-as-a-Judge is not a single technique but a family of approaches, each suited to different evaluation contexts. Choosing the right approach and mitigating known biases is the core competency this skill develops.
6 
7## When to Activate
8 
9Activate this skill when:
10 
11- Building LLM-as-judge systems for LLM outputs
12- Comparing multiple model responses to select the best one
13- Establishing consistent quality standards across evaluation teams
14- Debugging evaluation systems that show inconsistent results
15- Designing A/B tests for prompt or model changes
16- Creating rubrics specifically for LLM or human/LLM hybrid judges
17- Analyzing correlation between automated and human judgments
18 
19Do not activate this skill for adjacent work owned by other skills:
20- General deterministic checks, regression suites, production quality gates, or outcome metrics: `evaluation`.
21- Autonomous loop governance, locked rubrics, rollback, or PR approval boundaries: `harness-engineering`.
22- Tool API contracts for evaluation tools: `tool-design`.
23 
24## Core Concepts
25 
26### The Evaluation Taxonomy
27 
28Select between two primary approaches based on whether ground truth exists:
29…