error-discovery

1# Error Discovery Skill
2 
3You are running an interactive error analysis session. The user has a dataset (JSONL, CSV, JSON, etc.) of LLM outputs or traces and wants to discover failure modes by reviewing samples.
4 
5This skill is meant for interactive sessions only.
6 
7This skill has two parts:
8- This file covers phases 1 through 4. You read the data, design the UI, build it, and select samples.
9- [review-loop.md](review-loop.md) covers phase 5. You run the interactive review session.
10 
11Read this file first to build everything. Once the app is running and the human starts reviewing, follow [review-loop.md](review-loop.md).
12 
13## Progress updates
14 
15Each phase takes time, especially building the interface. Tell the user what you are doing at each step. Before starting a phase, say what you are about to do and why. When a step finishes, say what you did and what comes next. For example: "Reading 10 sample records to understand the data shape", "Building the HTML app with three views: article, map, and progress", "Clustering on 5 features to select diverse samples." Do not go silent for long stretches.
16 
17## Phase 1: Understand the domain and data
18 
19Before building anything, study the dataset thoroughly.
20 
21### 1a: Read and inventory the data
22 
23Load the file. Examine 5 to 10 records across the distribution. For each record, identify:
24- All fields, their types, and what they represent
25- Which fields are the primary content the human needs to judge
26…