jhe-robustness

Robustness Strategy (jhe-robustness)

When to trigger

• The headline estimate may be sensitive to specification, sample window, or functional form
• Inference is suspect: few clusters (states), serial correlation, or multiple outcomes/subgroups
• A referee will ask whether the result is a coverage/take-up artifact rather than the claimed effect
• The mechanism story is asserted but not separated from competing explanations

Robustness the JHE way: organize by threat, not by appendix

A wall of starred alternative specifications persuades no one. JHE referees want each robustness check mapped to a specific threat to the health-economics claim, with the point estimate's stability — not its significance — as the object. Build the robustness section as a threat-response ledger: name the threat a health economist would raise, run the check that addresses it, and report whether the magnitude moves. The threats that recur at JHE are selection, concurrent policy, measurement of health/utilization, and inference with few policy clusters.

Threat-to-check ledger

Threat to the claim	Check that addresses it
Residual selection into insurance/treatment	bounds (Lee/Manski/Oster); selection-on-observables-to-unobservables (Oster δ)
Concurrent reform contaminates the policy variation	placebo on ineligible group/period; leave-one-reform-out; pre-period falsification
Staggered-timing bias	heterogeneity-robust estimator (CS / SA / dCDH); honest-DID sensitivity
Health/utilization mismeasurement (claims coding, self-report)	alternative outcome definitions; administrative vs. survey cross-check; coding-change robustness
Functional form / sample window	log vs. level, trimming outliers (skewed health spending!), alternative bandwidths, donut RD
Few-cluster inference (states)	wild-cluster bootstrap; randomization inference; correct clustering level
Multiple outcomes/subgroups	MHT adjustment (Romano–Wolf / sharpened q-values); pre-specify the primary outcome
Mechanism is one of several	horse-race the channels; show the competing story predicts a pattern you do not see

Sequencing

1. Lead with the threat the editor/referee will raise first — usually selection or concurrent policy at JHE.
2. Report stability, not stars. Show the point estimate across checks in one figure or compact table; if it moves, say so and explain.
3. Right-size the spending/skew problem. Health expenditure is heavily right-skewed and zero-inflated; defend the estimator (two-part, GLM, IHS) rather than defaulting to OLS on a log.
4. Treat inference as a first-class robustness item, not a footnote — few-state clustering is a classic JHE referee catch.
5. Pre-register the primary outcome where multiple health outcomes invite cherry-picking.
6. Show, do not assert, stability. A single figure plotting the point estimate and CI across every check is worth more than a paragraph claiming robustness; a referee can read it in seconds.

Checklist

• Every robustness check is mapped to a named threat to the health-econ claim
• Point-estimate stability is the reported object (not just significance)
• Selection and concurrent-policy threats are addressed head-on
• Skewed/zero-inflated health spending handled with a defended estimator
• Inference matches the design: correct clustering level + few-cluster correction
• Multiple outcomes/subgroups get MHT adjustment; primary outcome pre-specified
• The mechanism is distinguished from at least one competing explanation

Anti-patterns

• A leave-out or alternative-sample check run but never reconciled when the estimate moves
• An appendix of starred specifications with no map from check to threat
• OLS on log spending with no handling of zeros or skew
• Clustering below the policy level, then claiming significance
• Running every subgroup and reporting the significant ones with no MHT adjustment
• "Results are robust" with no figure showing the point estimate holding
• Dodging the selection threat with more controls instead of a bound or design fix

The skewed-spending decision, made explicitly

Health spending and utilization are the journal's signature dependent variables, and they are right-skewed, zero-inflated, and heavy-tailed — the estimator choice is itself a robustness question a referee will press. Make it a deliberate, defended choice rather than a default:

• Many zeros + continuous positive part → two-part model (probit/logit for any use × GLM for the positive amount); report both parts.
• Skew without excess zeros → GLM with a log link (often gamma), which avoids retransformation bias that plagues OLS-on-log.
• Want to keep zeros and interpret in levels → IHS or Poisson/PPML, stating the elasticity interpretation honestly.
• Whatever you pick, show the result is not an artifact of the functional form by reporting at least one alternative, and never present OLS-on-log as if retransformation were free.

Worked vignette (illustrative)

A provider-payment paper shows intensity rises after a fee change; a referee suspects it is patient selection, not a true behavioral response. The JHE fix: hold the threat ledger explicit — (a) an Oster δ shows selection on unobservables would need to be 2× the observables to overturn the result; (b) a placebo on a fee-unaffected service shows no jump; (c) the spending outcome is re-run with a two-part model given 30% zeros; (d) inference is wild-cluster bootstrapped over 41 providers. The point estimate holds across all four (say 6.2%, stable within ±0.8pp, illustrative). The mechanism — behavioral response, not selection — now survives.

Output format

【Journal】Journal of Health Economics
【Skill】jhe-robustness
【Primary threat】selection / concurrent-policy / staggered-bias / measurement / inference
【Threat→check ledger】[threat: check → estimate movement]
【Spending estimator】OLS / two-part / GLM / IHS — defended? [Y/N]
【Inference】clustering level + few-cluster correction
【MHT】adjusted across outcomes/subgroups? [Y/N]
【Verdict】estimate stable / moves (explained) / fragile
【Next skill】jhe-tables-figures

Handoff boundary

This skill stress-tests an already-identified estimate; it does not fix a broken design (that is jhe-identification) or present the results (that is jhe-tables-figures). If a robustness check reveals the estimate is not actually identified — it swings with the selection bound or fails the placebo — route back to jhe-identification rather than papering over it with more specifications. When the point estimate holds across the threat ledger, hand off to jhe-tables-figures to make the stability legible.