ecopol-robustness

Robustness Strategy (ecopol-robustness)

When to trigger

• The headline policy magnitude moves under plausible alternative specifications
• A discussant could attribute the result to one sample, period, or country
• Inference rests on few clusters / few treated units (common in policy evaluations)
• The result is statistically significant but the magnitude — the part a policymaker uses — is imprecise
• You are pre-empting the academic discussant's "have you tried…" before the conference

Robustness is discussant-proofing, not a checklist dump

At EP, robustness has a specific purpose: the paper will be debated live by two invited discussants who act as the referees, and the academic one will arrive with a list of fragility tests. So organize robustness by the threat each test neutralizes, not as a mechanical appendix of "alternative specifications." A reader (and the policy discussant) should see, for each test, which way of being wrong it rules out. Because EP papers carry a recommendation, the central object to defend is the magnitude and its confidence interval, not just the sign.

The threat-to-test map (build the robustness section from this)

Threat a discussant will raise	The test that answers it
"It's driven by one period/sample"	leave-one-out by year/region; pre/post-crisis split
"Functional form is doing the work"	alternative specifications; semi/non-parametric check
"Parallel trends / pre-trends fail"	event-study leads; honest-DID / pre-trends sensitivity bounds
"Confounders you didn't control for"	coefficient-stability / Oster δ bounds; placebo on unaffected groups
"Inference is too optimistic with few clusters"	wild-cluster bootstrap; randomization inference
"Measurement error in the policy variable"	alternative coding; instrument or bound the error
"It won't generalize beyond this case"	replicate on a second jurisdiction/episode if data allow

Craft moves

• Put a robustness summary in the main text, not buried 40 pages deep: a compact figure or sentence showing the headline number is stable across the threats that matter. The policy discussant will not read the appendix.
• Distinguish robustness from heterogeneity. Stable-everywhere = robust; differs-by-group = a finding to report, not a failure to hide.
• Report the range, not just a star. "The effect lies between X and Y across all specifications" is the EP-useful statement.
• Pre-register the discussant's likely objections (use ecopol-referee-strategy) and answer the top three in the main text proactively — you cannot reply live as easily as in a referee letter, so anticipate.
• Honesty about fragility. If one test moves the number, say what it implies for the policy conclusion rather than hiding it — discussants will find it.

Checklist

• Robustness section organized by threat, each test labeled with the threat it neutralizes
• A main-text robustness summary (figure or sentence) — not appendix-only
• Headline magnitude + CI shown to be stable (or its instability honestly bounded)
• Few-cluster / few-treated inference addressed (wild bootstrap / randomization inference)
• Pre-trends / parallel-trends sensitivity reported where the design needs it
• Coefficient-stability or placebo evidence against unobserved confounders
• Heterogeneity reported as a finding, not disguised as robustness
• Top three likely discussant objections answered proactively in the main text

Anti-patterns

• A 20-table appendix of "alternative specifications" with no map to the threats they address
• Defending only significance while the policy-relevant magnitude swings across specifications
• Standard clustered SEs with five treated jurisdictions and no few-cluster correction
• Burying every robustness result in the appendix where the policy discussant never sees it
• Quietly dropping a specification that breaks the result instead of reporting and interpreting it

Output format

【Journal】Economic Policy (EP)
【Skill】ecopol-robustness
【Headline magnitude】X (CI: [.,.]) — stable across threats? Y/N
【Threats neutralized】[sample / functional form / pre-trends / confounders / inference / measurement / external]
【Main-text summary】figure/sentence present? Y/N
【Few-cluster inference】method used
【Honest fragility note】what moves the number and the policy implication
【Next skill】ecopol-tables-figures