From journal-of-applied-psychology-skills
Hardens study design and measurement for JAP manuscripts against common-method variance, weak causal warrants, unmodeled nesting, and construct validity gaps.
How this skill is triggered — by the user, by Claude, or both
Slash command
/journal-of-applied-psychology-skills:joap-study-designThe summary Claude sees in its skill listing — used to decide when to auto-load this skill
JAP holds measurement and design to an exacting standard. The recurring killers are **common-method
JAP holds measurement and design to an exacting standard. The recurring killers are common-method variance (CMV), weak causal warrants (cross-sectional single-source data), unmodeled nesting, and construct validity gaps. This skill hardens the design before data collection, where most of these problems can actually be solved.
| Remedy | Type | Note |
|---|---|---|
| Temporal separation (multi-wave) | procedural | predictor and outcome at different waves |
| Source separation (self + other/objective) | procedural | the strongest single defense |
| Measurement/context separation | procedural | different scales/formats for predictor vs outcome |
| Protected anonymity, balanced items | procedural | reduces consistency and acquiescence bias |
| Marker variable / CFA marker technique | statistical | plan a theoretically unrelated marker in advance |
| ULMC (unmeasured latent method construct) | statistical | report alongside, not instead of, procedural remedies |
For the servant-leadership package, justify N at the level the hypotheses live, before collecting.
Multilevel field study (2-2-2 / 2-1-2 mediation):
Constraint: 74 teams (L2) drives power for the team-level indirect effect.
Power target: 80% for the indirect effect (Monte Carlo power for multilevel
mediation), assuming a path ≈ .25, b path ≈ .30, ICC(1) ≈ .15.
Result: target ≥ 70 teams, ~8 members each → ~560–620; we collect 612 in 74.
Lab experiment (causal leg):
Between-subjects, two conditions; power for the interaction (H3 boundary),
N ≈ 240 at 80%, alpha .05; fixed-N, no optional stopping.
Aggregation: report ICC(1), ICC(2), r_wg(j) to justify team-level aggregation
of psychological safety; preregister exclusion rules.
| Degree of freedom | Lock before data? | Where it lives |
|---|---|---|
| Hypotheses + direction + level | yes | preregistration |
| Measures (all scales, all items) | yes | preregistration (prevents scale cherry-picking) |
| CMV remedies (procedural + planned statistical) | yes | design + preregistration |
| Aggregation rules (ICC/r_wg thresholds) | yes | analysis plan |
| Exclusion rules (careless responding, attrition) | yes | preregistration |
| Covariates / model form | yes | analysis plan |
| Exploratory analyses | allowed, labeled | reported separately, post hoc |
joap-data-analysis).【Construct validity】reliability + CFA/invariance evidence? [Y/N]
【Causal warrant】temporal / multi-source / experimental leg present? [Y/N]
【CMV】procedural remedies + planned statistical check declared? [Y/N]
【Nesting】levels, ICC/r_wg, multilevel model justified? [Y/N/NA]
【Sample size】powered for the carrying effect at the right level? [Y/N]
【Next】joap-data-analysis
../../resources/external_tools.md — Mplus/lavaan/lme4, Monte Carlo power, CMV-marker and invariance tools../../resources/official-source-map.md — measurement, design, and reporting expectationsnpx claudepluginhub brycewang-stanford/awesome-journal-skills --plugin journal-of-applied-psychology-skillsDesigns and stress-tests experiments for JCP manuscripts: manipulations, confounds, checks, power, multi-study chains, and pre-registration.
Helps match research design to theoretical claims for JOM manuscripts — construct validity, common-method bias, endogeneity, multilevel structure, and meta-analysis coding protocols.
Designs studies for Psychological Science manuscripts, enforcing power analysis, sample-size justification, preregistration, and confound control. Strengthens pre-analysis plans without writing code.