Science Superpowers
Science Superpowers is a complete computational-science methodology for your research agents, built on a set of composable skills plus initial instructions that make sure your agent actually uses them.
It is a reimplementation of Superpowers (a software-development methodology) for a different domain: doing science with data. The architecture is the same — skills that auto-trigger via a session-start bootstrap — but the workflow is the research lifecycle, and the central discipline is pre-registration instead of test-driven development.
How it works
It starts the moment you fire up your agent. As soon as it sees you're trying to investigate something, it doesn't jump straight into running code on your data. Instead it steps back and helps you turn a fuzzy interest into a precise, falsifiable question.
Once the question is clear, it grounds the work in prior literature and standard methods, designs the analysis, and pre-registers the hypotheses, predictions, and decision rules before looking at the outcomes. That separation — confirmatory vs. exploratory, predictions locked before data — is what protects the work from p-hacking and HARKing (hypothesizing after results are known).
Then it executes the pre-registered plan in a reproducible workspace (pinned environment, fixed seeds, immutable raw data), investigates anomalies by root cause instead of quietly dropping inconvenient data, verifies every claim against fresh reproduced evidence, and red-teams the result before reporting it.
Because the skills trigger automatically, you don't need to do anything special. Your research agent just has Science Superpowers.
The basic workflow
- framing-research-questions — Activates before any analysis. Turns a rough interest into a precise, falsifiable question with hypotheses, the data needed, and what would count as an answer. Saves a question document.
- surveying-prior-work — Grounds the question and chosen methods in what's already known: standard methods, known confounds, prior effect sizes.
- designing-the-analysis — Breaks the work into bite-sized analysis steps with exact datasets, variables, models/tests, power, and decision rules.
- preregistering-analysis — The Iron Law. Locks hypotheses, directional predictions, and decision rules — and the confirmatory/exploratory split — before any outcome is seen.
- setting-up-reproducible-analysis — Isolated, reproducible workspace: pinned environment, fixed seeds, immutable raw data, clean baseline.
- subagent-driven-analysis or executing-analysis — Carries out the pre-registered plan with review checkpoints.
- investigating-anomalous-results — Activates when results look wrong. Root-cause investigation before any adjustment.
- verifying-results-before-claiming — Evidence before claims: re-run, check assumptions, robustness, reproduce.
- requesting-red-team-review / receiving-critical-review — Adversarial review before you believe or report a result.
- reporting-and-archiving-findings — Reproducibility check, then write-up/preprint/iterate/shelve/discard, then archive code + data + environment.
The agent checks for relevant skills before any task. Mandatory workflows, not suggestions.
What's inside
Skills library
Framing
- framing-research-questions — Turn an interest into a falsifiable question (entry gate)
- surveying-prior-work — Ground the question and methods in existing literature
Planning & pre-registration
- designing-the-analysis — Detailed, bite-sized analysis plan
- preregistering-analysis — Lock predictions and decision rules before seeing outcomes (includes statistical-fallacies reference)
Execution
- subagent-driven-analysis — Fresh subagent per analysis step with two-stage review
- executing-analysis — Inline batch execution with checkpoints
Discipline
- investigating-anomalous-results — 4-phase root-cause process for surprising results
- verifying-results-before-claiming — Evidence before claims
Review
- requesting-red-team-review — Dispatch a skeptical reviewer to attack the analysis
- receiving-critical-review — Respond to critique with rigor, not performative agreement
Workspace & reporting
- setting-up-reproducible-analysis — Isolated, reproducible workspace
- reporting-and-archiving-findings — Decide how to report; archive code, data, environment
- dispatching-parallel-investigations — Concurrent independent investigations
Meta
- writing-science-skills — Create new skills following the testing methodology
- using-science-superpowers — Introduction to the skills system
Philosophy
