spml

SPML — ML SuperPowers

SPML is an addon plugin for Superpowers that extends it with ML experiment workflows: Validation Pyramid, experiment-driven development, Watchdog-based single-run supervision, ml-iteration (N-round human-on-the-loop iteration against compound criteria), and Auto Research (protocol-driven metric search).

Superpowers provides the foundation — TDD, code review, subagent architecture, verification. SPML adds the ML domain knowledge on top: what to validate, how to monitor training, how to draw evidence-based conclusions, and how to run a research loop autonomously while a human stays on the loop.

What makes ML different

In traditional software, code runs = result correct. In ML, code runs without errors does NOT mean the result is correct.

"Not working" is reasonable in ML, but the process must be correct. If an implementation error causes poor results, you may misjudge your experimental strategy as ineffective, wasting an entire research direction.

SPML addresses this with:

• Validation Pyramid — 2-level verification (static analysis, runtime + pipeline validation) that separates "implementation bug" from "strategy doesn't work"
• Watchdog — single-run training supervision: restarts from checkpoint on environment failures, async evaluation on new checkpoints, baseline-deviation alerts
• ml-iteration — N-round Supervisor-driven iteration against compound review criteria: Researcher subagents modify code each round, Supervisor reviews and commits or rolls back, human on the loop can interject
• Auto Research — protocol-driven autonomous iteration: Supervisor dispatches fresh Researcher subagents each round, runs the fixed eval script, commits improvements and rolls back regressions, and passes lessons between rounds through an experiences log
• Experiment-driven workflow — hypothesis, independent/dependent/control variables, conclusion recording with metric evidence

Installation

Prerequisites

Install Superpowers first. SPML depends on Superpowers for general development skills (TDD, code review, debugging, etc.).

Claude Code

In Claude Code:

/plugin marketplace add qqhard/superpowers-ML
/plugin install spml

Codex

SPML also works with Codex through native skill discovery.

Install superpowers first, then install SPML:

git clone https://github.com/obra/superpowers.git ~/.codex/superpowers
git clone https://github.com/qqhard/superpowers-ML.git ~/.codex/spml
mkdir -p ~/.agents/skills
ln -s ~/.codex/superpowers/skills ~/.agents/skills/superpowers
ln -s ~/.codex/spml/skills ~/.agents/skills/spml

See docs/README.codex.md for the full Codex guide.

Verify Installation

Start a new session and check that both skill sets are available.

Claude Code:

Ask Claude to use superpowers:brainstorming for general software work, spml:ml-brainstorming for an ML experiment, or spml:autoresearch-create to start an Auto Research project.

Codex:

Ask Codex to use superpowers:brainstorming for general software work, spml:ml-brainstorming for ML experiment work, or spml:autoresearch-create to start an Auto Research project.

How the two plugins work together

General software development:
  /superpowers:brainstorm → superpowers:writing-plans → superpowers:subagent-driven-development
  All skills from Superpowers, SPML not involved.

ML experiments:
  spml:ml-brainstorming → spml:experiment-planning → spml:ml-subagent-dev
  ML workflow from SPML, general discipline (TDD, code review) from Superpowers.

Auto Research (autonomous iteration after a validated baseline):
  spml:autoresearch-create → spml:ml-brainstorming (autoresearch mode) →
  spml:experiment-planning → spml:ml-subagent-dev →
  spml:autoresearch-handoff → spml:autoresearch-run

SPML skills reference Superpowers skills where needed (e.g., superpowers:finishing-a-development-branch, superpowers:using-git-worktrees). Cross-plugin skill invocation works transparently.

The ML Workflow

ml-brainstorming
    Refine hypothesis, collect context, define review_criteria (compound)
    |
experiment-planning
    Break into atomic subtasks with validation criteria
    |
ml-subagent-dev
    Execute each subtask: unit test → implement → Validation Pyramid
    |
training-handoff
    Route between:
    ├── watchdog         (single-run supervision; env restart + async eval)
    └── ml-iteration     (N-round Supervisor-driven iteration against review_criteria)
    |
verification
    Evidence-based conclusion: effective / ineffective / inconclusive

Auto Research (parallel entry for metric search):
    autoresearch-create → ml-brainstorming(autoresearch) → experiment-planning
                       → ml-subagent-dev → autoresearch-handoff → autoresearch

Validation Pyramid