aaai-reproducibility

1# AAAI Reproducibility
2 
3Use this when a draft needs to survive AAAI review on rigor, not just novelty. AAAI-27 requires the
4reproducibility checklist to be **uploaded separately** from the main PDF, in its own field on the
5submission form (AAAI-26 carried it inside the PDF after the references) — so it is a document a
6reviewer opens on its own, and it has to agree with the paper and supplement rather than read as an
7afterthought. AAAI-27 also states that reviewers assess reproducibility from what was actually
8submitted, and that material promised "after acceptance or publication" is not evidence it exists.
9 
10## Reproducibility audit
11 
12- Map each central claim to submitted evidence: theorem, table, figure, ablation, appendix item,
13  checklist answer, or code/data artifact.
14- Record seeds, splits, preprocessing, hyperparameters, model selection, early stopping, prompt
15  selection, and hardware.
16- Report variance or uncertainty when stochasticity affects conclusions.
17- Document dataset licenses, access constraints, sensitive data, human-subjects issues, and
18  annotation procedures.
19- Separate training compute, inference compute, and experiment search cost.
20- Check the reproducibility checklist for contradictions with the main text and supplement.
21 
22## Common AAAI weaknesses
23 
24- Checklist says code/data are available but supplement lacks runnable commands.
25- Main results rely on one seed, one benchmark, or one prompt family.
26…