Autoresearch

A native Claude Code plugin that turns Claude into a tireless optimization researcher. Point it at any measurable target — test runtime, bundle size, training loss, Lighthouse score, build time — and it runs the inner loop for you: hypothesize → change code → run a benchmark → measure → keep or discard → commit → repeat. Inspired by karpathy/autoresearch.

This is the Claude Code port of the original pi-autoresearch extension for the pi agent.

What it does

Every kept experiment is auto-committed; every discard/crash auto-reverts your code (session files under .auto/ are always preserved). After a few runs it reports a confidence score so you can tell a real improvement from benchmark noise. A guardrail keeps Claude from overfitting or cheating the benchmark.

Install

From this repository's marketplace:

/plugin marketplace add seanGSISG/cc-autoresearch
/plugin install autoresearch@cc-autoresearch

For local development, load the checkout directly:

claude --plugin-dir /path/to/cc-autoresearch

Requirements: Node.js ≥ 22 on PATH (hooks run TypeScript via Node's built-in type stripping; the MCP server is a pre-bundled, dependency-free mcp/server.js). Git is used for keep/revert.

Quickstart

/autoresearch:start optimize unit-test runtime, keep correctness

On first use for a target, Claude runs the autoresearch-create skill: it gathers (or infers) the goal, benchmark command, primary metric + direction, files in scope, and constraints; creates an autoresearch/<goal>-<date> git branch; and writes .auto/prompt.md + .auto/measure.sh. Then it initializes the session and starts looping.

Watch it live:

/autoresearch:dashboard

Commands

Command	What it does
/autoresearch:start "<goal>"	Enter autoresearch mode and start or resume the loop.
/autoresearch:stop	Leave autoresearch mode (keeps the session log).
/autoresearch:clear	Delete .auto/log.jsonl and reset session state.
/autoresearch:status	Show mode, goal, metric, run count, baseline → best.
/autoresearch:dashboard	Open the live browser dashboard (SSE-updated).
/autoresearch:loop on|off	Toggle the opt-in unattended auto-resume loop (default off).

Tools (MCP)

The bundled MCP server exposes three loop tools (plus dashboard):

• init_experiment — one-time session config (name, metric, unit, direction). Writes the config header to .auto/log.jsonl. Call again to start a new segment with a fresh baseline when the target changes.
• run_experiment — runs a shell command, times wall-clock duration, captures/truncates output, parses METRIC name=value lines, and (if present) runs .auto/checks.sh as a correctness gate.
• log_experiment — records the result. keep → git add -A && git commit; discard/crash/checks_failed → auto-revert code (.auto/** preserved). Reports the confidence score after 3+ runs.

Session files (.auto/)

Everything for a session lives in one folder, so it survives reverts, is easy to .gitignore, and easy to clean up:

File	Purpose
.auto/prompt.md	The playbook — objective, metrics, files in scope, constraints, what's-been-tried. A fresh agent resumes from this alone.
.auto/measure.sh	Benchmark; emits METRIC name=value lines.
.auto/log.jsonl	Append-only result log (source of truth).
.auto/state.json	Mode + loop state (managed by the plugin).
.auto/ideas.md	Optional ideas backlog.
.auto/checks.sh	Optional correctness backpressure.
.auto/config.json	Optional { "maxIterations": N, "workingDir": "..." }.
.auto/hooks/{before,after}.sh	Optional per-iteration hooks (see the autoresearch-hooks skill).

How the loop runs

While autoresearch mode is active, a UserPromptSubmit hook injects the loop guidance ("optimize the primary metric… NEVER STOP… don't cheat the benchmark") each turn, and a SessionStart hook re-injects session context on startup, resume, and after compaction (a deterministic summary of rules + ideas + recent runs, so state survives context loss).

By default you drive each iteration. To let it run unattended, enable the opt-in loop:

/autoresearch:loop on