autoharness

autoharness

autoharness is a self-learning skill layer for Claude Code. It learns skills from your real sessions, merges same-scenario ones instead of stacking near-duplicates, updates them in use, and prunes any that stop getting used — so the layer stays clean on its own, touching only the skills it wrote itself.

Same model, different harness — 42% → 78% on CORE-Bench (HAL). The harness does much of the work (swyx's Big Model vs Big Harness), yet it's still rebuilt by hand every model generation. autoharness bets one slice of it — the skill layer — can maintain itself.

Learns from real work	Each episode is distilled into a skill from the session you were already having — no separate data-collection or replay loop.
Groups, doesn't just pile up	A new episode doesn't always add a skill — the reflector compares it against what's there and folds same-scenario skills into one, so the layer consolidates by category instead of accreting near-duplicates.
Validated in use, not on a benchmark	A skill survives by being adhered to in later turns (invocation rate), not a held-out score. No oracle on the active path, and no tokens spent on a dedicated eval.
Only its own skills	Touches only the skills it generated through this plugin — everything else, whether you wrote it or installed it, is left completely alone.
Evidence kept for later	Every create/update logs its scenario and decision to a per-skill ledger — the raw material to build a benchmark from real usage if you ever want one.

Install

Requires python3 on your PATH — autoharness runs entirely as Python (zero third-party dependencies); its hooks and MCP server won't fire without it.

/plugin marketplace add tigerless-labs/autoharness
/plugin install autoharness@autoharness

Run /reload-plugins (or restart Claude Code), then check it's live:

/plugin    # autoharness@autoharness — enabled
/mcp       # stage_skill — connected

Zero config. It now watches your sessions and lands learned skills into .claude/skills/ in the background. Cadence and lifecycle thresholds are tunable — see Configuration.

Update

/plugin marketplace update autoharness              # pull the latest release
/plugin update autoharness                          # update the installed copy
/reload-plugins                                     # apply in the running session

/reload-plugins re-arms the hooks and the MCP server in place; restarting Claude Code works too. The installed copy is cached by the version in plugin.json; releases always bump it, which is what makes the update take effect.

Uninstall

claude plugin uninstall autoharness@autoharness     # stops the hooks + MCP server
claude plugin marketplace remove autoharness        # optional — also drops the install source

Uninstalling only stops it from running — the skills it landed and its own state live outside the plugin and stay on disk. To clear those too, delete its state dir (~/.claude/autoharness/ global, <repo>/.claude/autoharness/ per project) and the self-authored skills under .claude/skills/ (each carries a self-authored ledger marker, so they're easy to tell from yours). Your own skills are never touched.

Configuration

Every knob is an AUTOHARNESS_* environment variable with a built-in default — nothing to configure unless you want to change the pace.