From journal-of-educational-psychology-skills
Hardens study designs for Journal of Educational Psychology manuscripts by addressing nesting, cluster-level power, measurement of learning constructs, ecological validity, and preregistration.
How this skill is triggered — by the user, by Claude, or both
Slash command
/journal-of-educational-psychology-skills:jedpsych-study-designThe summary Claude sees in its skill listing — used to decide when to auto-load this skill
The Journal of Educational Psychology expects designs that are **adequately powered for their nesting
The Journal of Educational Psychology expects designs that are adequately powered for their nesting structure, measure learning constructs well, and have ecological validity for real educational settings. Because JEP studies are usually students nested in classes nested in schools, the single most consequential design decision is matching the unit of randomization, power, and analysis to the level at which the treatment and mechanism operate. This skill hardens the design before data collection.
jedpsych-data-analysis).For a teacher-delivered reading-comprehension trial, justify the number of classrooms before recruiting, tied to the smallest educationally meaningful effect — not a round student count.
Smallest meaningful effect: d = 0.20 (a defensible learning gain for a
classroom literacy intervention).
Nesting: students nested in classrooms; assumed ICC = 0.15; ~23 students
per classroom; pretest covariate (r ≈ .6) absorbs cluster variance.
Power: target 80% power, two-sided alpha .05 → ~48 classrooms
(24 per arm), ~1,100 students; design effect handled via the ICC,
not by counting students as independent.
Stopping: fixed number of clusters; no optional addition of schools.
Covariate: baseline comprehension at student and classroom level.
State the assumed effect size and its source (prior trial, meta-analytic estimate, or a smallest- meaningful-effect argument). Powering on an inflated lab effect, or on student N alone, is the classic JEP design failure.
| Degree of freedom | Lock before data? | Where it lives |
|---|---|---|
| Hypotheses + direction (at the right level) | yes | preregistration / analysis plan |
| Unit of randomization + number of clusters | yes | preregistration |
| Full measure list (all outcomes) | yes | preregistration (prevents cherry-picking) |
| Exclusion / attrition rules | yes | preregistration, with expected attrition |
| Covariates + multilevel model form | yes | analysis plan |
| Fidelity / implementation measures | yes | protocol |
| Exploratory analyses | allowed, but labeled | reported separately, post hoc |
jedpsych-data-analysis).【Unit】randomization / power / analysis level (matched?) [Y/N]
【Sample size】# clusters + size + ICC + smallest meaningful effect
【Measures】validated learning outcome + reliability + transfer? [Y/N]
【Baseline + confounds】equivalence, attrition, fidelity addressed?
【Ecological validity】setting / delivery supports the educational claim?
【Preregistration】confirmatory core locked? where?
【Next】jedpsych-data-analysis
../../resources/external_tools.md — PowerUpR, simr, Optimal Design, multilevel/SEM software, preregistration templates../../resources/official-source-map.md — JARS reporting standards and preregistration policynpx claudepluginhub brycewang-stanford/awesome-journal-skills --plugin journal-of-educational-psychology-skillsDesigns studies for Psychological Science manuscripts, enforcing power analysis, sample-size justification, preregistration, and confound control. Strengthens pre-analysis plans without writing code.
Hardens study design and measurement for JAP manuscripts against common-method variance, weak causal warrants, unmodeled nesting, and construct validity gaps.
Designs a multi-study package for a JPSP manuscript: sequences studies, powers each one, selects experimental/longitudinal/dyadic designs, and plans preregistration.