benchmark

1# Benchmark
2 
3Score a session against the rolling population average and report its percentile on
4cost, tokens, tool count, and complexity using Agent Monitor data.
5 
6## Input
7 
8The user provides: **$ARGUMENTS**
9 
10This may be:
11- A single session ID — benchmark that session
12- "latest" — benchmark the most recent session
13- "latest N" — benchmark the N most recent sessions, each vs the average
14- empty — benchmark the most recent session (default)
15 
16## Data Sources
17 
18| Endpoint | Returns |
19|----------|---------|
20| `GET /api/sessions?limit=N` | Population of sessions with `cost`, `model`, `started_at`, `metadata` (turn_count, total_turn_duration_ms) — builds the rolling baseline |
21| `GET /api/pricing/cost/{sessionId}` | `{ total_cost, breakdown:[{ input_tokens, output_tokens, cache_read_tokens, cache_write_tokens, cost }] }` — the target session's cost and tokens |
22| `GET /api/workflows/{sessionId}` | `complexity` (score), `stats` (tool/event counts), `toolFlow` (distinct tools used) — the target session's tool count and complexity |
23| `GET /api/analytics` | `avg_events_per_session`, `tool_usage`, `daily_sessions` — corroborates population-level averages |
24 
25## Report Sections
26 
27### 1. Build the Baseline
28Fetch the population with `GET /api/sessions?limit=200` (the rolling set). For each
29session gather cost (`GET /api/pricing/cost/{id}` or the list `cost` field), total
30tokens (sum of the 4 token types from the pricing breakdown), tool count and
31…