Improve LLM apps and agents from real production traces: capture traces, build evals, run local optimization (GEPA), compare models/providers, and route through the Understudy gateway to reduce cost/latency and raise quality/reliability.
Use when a developer wants to build an eval from their real LLM app before changing anything — "measure how my app is doing today", "build an eval from my workload", "we have no baseline", "is my current model actually good". Turns the workload into auditable local artifacts (harness, metric, frozen splits, baseline); has a public-benchmark on-ramp when no traces exist.
Use when a developer asks "is Understudy causing my errors", "which workloads are routed", "is my routing config actually taking effect", "is my provider healthy", "are there 500s on staging", "what's our error rate", "where is my gateway spend going", or wants self-service diagnostics without asking the team. Reads the hosted reporting endpoints with the developer's sk_* key.
Use when a developer wants to compare candidate models — any mix of local, gateway, or frontier — on the same eval and see quality, latency, cost, and reliability side by side. "Which model should I use", "sweep these models on my benchmark", "compare Gemma vs the frontier on my eval". To stand up and serve a local candidate first, use run-local-model-lab.
Use when you need to know HOW two model runs differ behaviorally on the same tasks, not just THAT one scores higher — per-task trajectory diffing that classifies the gap as persistence/recovery, knowledge, or format/parsing. "why does the bigger model pass these", "is this gap RL-shaped", "diff these two trajectory runs", "where do the trajectories diverge", "what would distillation buy me". The behavioral complement to compare-model-sweep.
Use to turn loose per-task trajectory JSONs (a Lilac export or local rollout corpus) into a queryable, provenance-tracked, contamination-safe dataset that hard-blocks any selection leaking frozen dev/holdout into a train/RL/distill pool. "curate my trajectories", "which rollouts are train-safe", "exclude the holdout rows before RL", "is this distill pool contaminated", "make a hash-stamped selection", "stop hand-filtering trajectories in bash".
Based on adoption, maintenance, documentation, and repository signals. Not a security audit or endorsement.
Type /understudy:onboard in Claude Code — or the matching onboarding
command in your agent — and it walks your LLM app from captured traces to a
measured, cheaper — often local — model, with evidence you can trust.
Works with Claude Code · Cursor · Codex · OpenCode · Hermes Agent · Devin
Understudy is a public, MIT-licensed skill library plus a thin CLI. It gives your coding agent the playbooks to evaluate an AI workload locally, optimize it, and route it to a better-value model — local-first, with no uploads, no provider calls, and nothing spent by default.
curl -fsSL https://raw.githubusercontent.com/UnderstudyLabs/understudy-agent-tools/main/install.sh | bash
The installer sets up the CLI, asks which coding agents to attach to (Claude Code, Cursor, Codex, OpenCode, Hermes Agent, all, or CLI-only), and — if you pick Claude Code — opens it in the current directory. Then, in Claude Code:
/reload-plugins
/understudy:onboard
That's it. Onboarding profiles your machine, interviews you about your workload, and starts the improvement loop. No registration required. On other agents the activation command differs slightly — see Pick your platform.
The installer intentionally does not download model weights, start MLX, launch the ladder server, or make frontier calls. Those happen inside the skill flow, where the agent explains tradeoffs and asks consent first.
Non-interactive installs keep script-friendly autodetect behavior:
curl -fsSL https://raw.githubusercontent.com/UnderstudyLabs/understudy-agent-tools/main/install.sh | bash -s -- --yes
Override the agent prompt with
--agents auto|all|claude-code|cursor|codex|opencode|hermes|devin|none.
The installer is resumable. It writes step markers under
~/.understudy/agent-tools/install-state; after a failed run:
curl -fsSL https://raw.githubusercontent.com/UnderstudyLabs/understudy-agent-tools/main/install.sh | bash -s -- --resume
Or jump directly to a step:
curl -fsSL https://raw.githubusercontent.com/UnderstudyLabs/understudy-agent-tools/main/install.sh | bash -s -- --from-step 2
Developer install from a clone:
npm install
npm run build
node dist/bin.js --help
After package publication:
npm install -g @understudylabs/understudy-agent-tools
understudy spine
The whole OSS loop runs locally and is skill-led:
flowchart LR
A["Capture<br/>evidence"] --> B["Attach harness +<br/>environment"]
B --> C["Confirm metric,<br/>validator, holdout"]
C --> D["Rerun<br/>baseline"]
D --> E["Optimize<br/>workload"]
E --> F["Conservative<br/>claim packet"]
Registration is not required for that loop. Hosted gateway access is available
after understudy login; browser, channel, daemon, and desktop-runtime
commands remain outside this public CLI until intentionally extracted.
Understudy is agent-platform-neutral at the skill layer. All six
supported coding agents share the same skills/ tree and differ
only in a thin adapter: manifest, install path, reload step, and onboarding
invocation. The adapter registry lives in
docs/agent-platform-adapters.md and is
exposed through understudy platforms.
npx claudepluginhub understudylabs/understudy-agent-tools --plugin understudyLazy senior dev mode. Forces the simplest, shortest solution that actually works: YAGNI, stdlib first, no unrequested abstractions.
Comprehensive skill pack with 66 specialized skills for full-stack developers: 12 language experts (Python, TypeScript, Go, Rust, C++, Swift, Kotlin, C#, PHP, Java, SQL, JavaScript), 10 backend frameworks, 6 frontend/mobile, plus infrastructure, DevOps, security, and testing. Features progressive disclosure architecture for 50% faster loading.
Develop, test, build, and deploy Godot 4.x games with Claude Code. Includes GdUnit4 testing, web/desktop exports, CI/CD pipelines, and deployment to Vercel/GitHub Pages/itch.io.
Tools to maintain and improve CLAUDE.md files - audit quality, capture session learnings, and keep project memory current.
A growing collection of Claude-compatible academic workflow bundles. Covers scientific figures, manuscript writing and polishing, reviewer assessment, citation retrieval, data availability, paper reading, literature search, response letters, paper-to-PPTX conversion, and evidence-grounded Chinese invention patent drafting. Rules are organized as reusable skill folders with explicit workflows and quality checks.
Evidence-gated AI coding workflow: scan → analyze → plan → TDD → execute → fix → verify → review, powered by Codebase Memory MCP >= 0.9.0 with optional Serena LSP intelligence. Includes blast-radius planning, test/cycle gates, independent review, and Windows Git Bash hook auto-resolution.