From aaai-skills
Audits AAAI paper reproducibility: maps claims to evidence, checks seed/hyperparameter/compute reporting, verifies code/data availability and licensing, and cross-checks the reproducibility checklist for contradictions.
How this skill is triggered — by the user, by Claude, or both
Slash command
/aaai-skills:aaai-reproducibilityThe summary Claude sees in its skill listing — used to decide when to auto-load this skill
Use this when a draft needs to survive AAAI review on rigor, not just novelty. AAAI-26 required a
Use this when a draft needs to survive AAAI review on rigor, not just novelty. AAAI-26 required a reproducibility checklist after references, so the checklist must agree with the paper and supplement rather than read as an afterthought.
AAAI places the reproducibility checklist after the references, and reviewers cross-check each "yes" against the paper and supplement. A "yes" with no backing artifact reads worse than an honest "no", because it signals the checklist was filled in carelessly.
| Checklist answer | Must be backed by | Phase-1 risk if unbacked |
|---|---|---|
| code available | runnable scripts in the ZIP | "claimed but absent" |
| seeds reported | seed list and variance | "single-run cherry-pick" |
| compute disclosed | train vs. inference vs. search cost | "hidden tuning budget" |
| data accessible | license and access path | "irreproducible by anyone" |
A vision-language paper checks "code and data available" but the ZIP holds only PDFs of plots. Audit
verdict: reproducibility grade "fragile", with a checklist conflict between the "yes" and the missing
scripts. The smallest fix is a reproduce.sh that regenerates one headline table from seeds plus a
dataset license note, after which the checklist answer becomes truthful and Phase-1 defensible.
[Reproducibility grade] strong / adequate / fragile / not reviewable
[Checklist conflicts] <answers that contradict paper/supplement>
[Evidence gaps] <claims without submitted verification>
[Compute/data disclosure] complete / incomplete
[Priority fixes] <smallest changes before submission>
npx claudepluginhub brycewang-stanford/awesome-journal-skills --plugin aaai-skillsAudits AAAI experimental evidence including baselines, ablations, statistical significance, robustness, human evaluation, and reproducibility-checklist alignment for Phase-1 survival.
Strengthens AISTATS reproducibility evidence by mapping claims to verifiable locations, auditing checklists, and ensuring turnkey simulation scripts.
Strengthens reproducibility for ICLR papers: maps claims to seeds, splits, commands, and compute; writes reproducibility statements and addresses reviewer concerns about verifiability.