analyze-exports

1# Analyze Exports Skill
2 
3## Purpose
4 
5When a tau2-bench task returns reward 0.0 (or fails), the export file contains everything you need to understand why—but it's buried in tool-call summaries and conversation transcripts. This skill systematically analyzes the export to show:
6 
71. **Where it failed** — the exact round (Assistant #X, User #Y) and tool-call sequence
82. **What should have happened** — ground truth from the task explanation
93. **Why it failed** — root cause (missing logic, wrong arguments, bad order, policy misunderstanding)
104. **Was the docs available?** — whether the KB had the answer the agent needed
115. **Evidence** — quotes and citations from both the transcript and source documents
12 
13## Input Format
14 
15An export file (plain text) containing:
16- `RUN:` and `TASK:` metadata
17- `★ TASK EXPLANATION` (success criteria, expected actions)
18- `🛠 TOOLS LOADED` (available tools and KB docs)
19- `⇄ TOOL-CALL SUMMARY` (OURS vs GROUND TRUTH comparison)
20- Full conversation transcript with tool calls and results
21 
22## Output Format & File Organization
23 
24Analysis reports are saved in a standardized folder structure for easy organization and review:
25 
26```
27FAILURE_ANALYSIS/
28├── analysis/
29│   ├── hard_15/
30│   │   ├── task_037_detailed_analysis.md
31│   │   ├── task_038_detailed_analysis.md
32│   │   ├── task_039_detailed_analysis.md
33│   │   └── task_041_detailed_analysis.md
34│   ├── hard_16/
35│   │   ├── task_053_detailed_analysis.md
36│   │   └── task_054_detailed_analysis.md
37…