redteam

1<HARD-GATE>
2NO ASR report WITHOUT paired over-refusal rate measurement.
3NO attack vector distribution WITHOUT reading a policy document first.
4NO regulated-stakes redteam WITHOUT a sign-off block in the audit document.
5</HARD-GATE>
6 
7# Redteam
8 
9Test your application's safety boundaries systematically. This skill generates
10attack prompts from a policy document, measures what gets through, and pairs
11the Attack Success Rate (ASR) with the Over-Refusal Rate so you don't reward
12models that simply refuse everything.
13 
14## When to Activate
15 
16- Pre-deployment safety audit
17- Regulatory compliance check
18- After major model or prompt changes that could affect safety
19- User reports a jailbreak or injection vulnerability
20- Periodic (quarterly) safety re-evaluation
21 
22## Checklist
23 
24You MUST create a task for each item and complete them in order:
25 
261. **Establish policy** — what's prohibited, what's allowed, what's gray
272. **Derive attack vectors from policy** — not a fixed template
283. **Generate attack set + over-refusal set** — paired measurement
294. **Run attacks and evaluate** — ASR per policy category
305. **Compute over-refusal rate** — paired with ASR
316. **Output safety scorecard + audit document**
32 
33## Fast path: run the bundled script
34 
35Once attacks and benign prompts have been run against the system and judged, compute the
36scorecard with the bundled, tested script (`scripts/asr_report.py`, standard library only,
37…