mandela

1Audit whether a validation receives independent outside ground-truth.
2 
3## Goal
4 
5The **Mandela Effect** names a population confidently remembering something that never independently happened. A leaky validation has the same shape: designer, model, and scorer confirm each other. `mandela` uses eight leakage patterns to check whether a *validation* is independent.
6 
7## Workflow
8 
91. Identify the validation: eval, metric, experiment, holdout, or "how we'll know". Name its **components**: model, scorer, designer, and dataset.
102. Ask the **core question**: does external ground-truth enter *independently*?
113. Test all **8 patterns** below. Some apply only to particular components, such as a human subject or a scorer. Report only patterns that fire, each by name.
12   1. **Recall, not reason**: a memorized answer recited instead of one actually derived; the system already knows the result it is supposedly computing.
13   2. **Wrong null hypothesis**: an ablation that removes a surface label but not the underlying signal the system actually exploits, so the "control" still leaks.
14   3. **Shared hallucination**: two components verifying each other; circularity reported as a number.
15   4. **Tautology**: a scorer grading buckets it drew itself.
16   5. **Verifier = designer**: a private, unreproducible recipe in a holdout's clothes.
17   6. **Shared-pool bias**: train and holdout drawn from one labeler pool, so one bias enters both sides.
18…