autoresearch

Try an idea, measure it, keep what works, discard what doesn't, repeat forever.

An extension for pi — an AI coding agent that runs in your terminal. pi-autoresearch gives pi the tools and workflow to run autonomous optimization loops: try an idea, benchmark it, keep improvements, revert regressions, repeat.

Inspired by karpathy/autoresearch. Works for any optimization target: test speed, bundle size, LLM training, build times, Lighthouse scores.

---

---

Quick start

pi install npm:pi-autoresearch

What's included

Extension	Tools + live widget + /autoresearch dashboard
Skill	Gathers what to optimize, writes session files, starts the loop

Extension tools

Tool	Description
init_experiment	One-time session config — name, metric, unit, direction
run_experiment	Runs any command, times wall-clock duration, captures output
log_experiment	Records result, auto-commits, updates widget and dashboard

/autoresearch command

Subcommand	Description
/autoresearch <text>	Enter autoresearch mode. If .auto/prompt.md exists, resumes the loop with <text> as context. Otherwise, sets up a new session.
/autoresearch off	Leave autoresearch mode. Stops auto-resume and clears runtime state but keeps .auto/log.jsonl intact.
/autoresearch clear	Delete .auto/log.jsonl, reset all state, and turn autoresearch mode off. Use this for a clean start.
/autoresearch export	Open a live dashboard in your browser. Auto-updates as experiments run.

Examples:

/autoresearch optimize unit test runtime, monitor correctness
/autoresearch model training, run 5 minutes of train.py and note the loss ratio as optimization target
/autoresearch export
/autoresearch off
/autoresearch clear

Keyboard shortcuts

Shortcut	Description
Ctrl+Shift+F	Open fullscreen scrollable dashboard overlay. Navigate with ↑/↓/j/k, PageUp/PageDown/u/d, g/G for top/bottom, Escape or q to close.

To avoid conflicts with other pi extensions, override or disable these shortcuts in <agent-dir>/extensions/pi-autoresearch.json. <agent-dir> is the active pi profile config directory (usually ~/.pi/agent, or PI_CODING_AGENT_DIR when set):

{
  "shortcuts": {
    "fullscreenDashboard": "ctrl+shift+y"
  }
}

Use null to skip registering a shortcut. Omitted shortcuts keep their defaults.

UI

• Dashboard widget — always visible above the editor: a full results table with columns for commit, metric, status, and description.
• Confidence score — after 3+ runs, shows how the best improvement compares to the session noise floor. ≥2.0× (green) = likely real, 1.0–2.0× (yellow) = above noise but marginal, <1.0× (red) = within noise.
• Fullscreen overlay — Ctrl+Shift+F opens a scrollable full-terminal dashboard. Shows a live spinner with elapsed time for running experiments.

Skills

autoresearch-create asks a few questions (or infers from context) about your goal, command, metric, and files in scope — then writes two files and starts the loop immediately:

autoresearch-finalize turns a noisy autoresearch branch into clean, independent branches — one per logical change, each starting from the merge-base. Groups must not share files, so each branch can be reviewed and merged independently.

autoresearch-hooks (optional) helps author .auto/hooks/before.sh and .auto/hooks/after.sh for a session. It ships with ten reference scripts in skills/autoresearch-hooks/examples/ (external search, learnings journal, native notifications, anti-thrash, idea rotation, and more) — the skill handles the contract, you pick the inspiration. The core autoresearch loop has no hook awareness.

All session files live in a single .auto/ subfolder at the working-directory root — one folder to preserve across reverts, gitignore, and clean up. (Legacy flat autoresearch.* files are still read for in-flight sessions.)