By zjunlp
Mechanist — AI Systems as Scientific Instruments for Understanding the Mechanisms of Intelligence.
The claim agent of /auto. Runs the `/auto-claim` skill under two orthogonal axes — `BEHAVIOR_SOURCE` (given / given-validation / discovery) sets where the behavior comes from and whether it is validated; `MECHANISM` (given / discovery) sets who picks the mechanism method. `discovery` generates ranked, novelty-checked ideas; `given` / `given-validation` faithfully enumerate the behavior/claims in task.md. All combinations emit a refined proposal + experiment plan that downstream stages consume. Use this agent when the orchestrator wants the claim stage executed end-to-end with isolated context.
The experiment agent of /auto. Wraps the /auto-experiment skill, which folds mechanism-family routing inline before implementing, code-reviewing, and deploying the experiment suite. Supports two-step invocation — first call returns candidate families for the orchestrator's mini-prompt, second call (with chosen_family) commits the routing and runs the full pipeline.
The iteration agent of /auto. Runs the /auto-iteration-loop skill — an autonomous review loop that consumes /auto-verify's four-state output (PASS / FAIL / INCONCLUSIVE / ZERO_ELIGIBLE_VARIANTS) plus the orthogonal deferred bucket and routes each claim to the right back-edge (① variant-only fix / ② baseline-script fix / ③ claim-stage re-entry / ⓪ narrative-only) under a unified MAX_ITERATIONS budget with a MAX_CLAIM_REENTRIES sub-budget. Semantics are owned by skills/auto-iteration-loop/SKILL.md — this wrapper only forwards flags, relays the awaiting_upstream handoff, and reports the result.
The verify agent of /auto. Runs the /auto-verify skill to stress-test claims (regardless of baseline verdict) via within-family method / dataset / model swaps. Two mandatory integrity gates — Phase 2 per-claim baseline audit (runs for every target claim) and Phase 9 per-claim variant audit on Phase 3 step 0's top-K picked claims (each = /experiment-audit + /mechanism-audit combined by max_severity). Each claim ends in one of five states: PASS / FAIL / INCONCLUSIVE / ZERO_ELIGIBLE_VARIANTS / INTEGRITY_ONLY. Sonnet by default since the task is more mechanical than creative. Semantics are owned by skills/auto-verify/SKILL.md — this wrapper only forwards flags and reports the result.
Use when main results pass result-to-claim (claim_supported=yes or partial) and ablation studies are needed for paper submission. The external LLM reviewer (via llm-chat MCP) designs ablations from a reviewer's perspective, CC reviews feasibility and implements.
Analyze ML experiment results, compute statistics, generate comparison tables and insights. Use when user says "analyze results", "compare", or needs to interpret experimental data.
Search, download, and summarize academic papers from arXiv. Use when user says "search arxiv", "download paper", "fetch arxiv", "arxiv search", "get paper pdf", or wants to find and save papers from arXiv to the local paper library.
Workflow 1: Claim-stage pipeline, controlled by two orthogonal axes. BEHAVIOR_SOURCE selects the behavior stage: `given` (default; behavior taken from task.md and assumed to hold — no ideation, no novelty, no M0), `given-validation` (behavior taken from task.md but the experiment plan opens with an M0 phenomenon-validation gate), or `discovery` (mine a NEW behavior via /mechanism-behavior-discovery + full ideation: research-lit → idea-creator → novelty-check → impact-check → research-review → research-refine-pipeline, impact-first ranking; plan opens with M0). MECHANISM selects the mechanism stage: `discovery` (default; system routes — claim loads /mechanism-explore to shape direction) or `given` (user named the mechanism method/family in task.md — capture it and forward as CHOSEN_FAMILY, no routing). resource_fidelity:strict (exact models/datasets, no downscaling) is stamped iff BEHAVIOR_SOURCE=given AND MECHANISM=given (the reproduction combination). Always emits `idea-stage/IDEA_REPORT.md` + `refine-logs/{FINAL_PROPOSAL,EXPERIMENT_PLAN}.md`. Use when user says "idea discovery pipeline", "找idea全流程", "从零开始找方向", "复现 task.md", or wants the claim stage of /auto.
The single place for every data constraint an experiment must satisfy — dataset provenance (existing → adapted → constructed), clear train / validation / test splits, labels that reflect the target behavior, and the minimum data amount. Use whenever an experiment chooses, adapts, or constructs a dataset, defines splits, or sets a sample size — for phenomenon validation (M0), mechanism exploration, intervention, or tuning. Domain-general: no assumption about model family, modality, or task.
Admin access level
Server config contains admin-level keywords
Requires secrets
Needs API keys or credentials to function
Own this plugin?
Verify ownership to unlock analytics, metadata editing, and a verified badge. GitHub access is read-only (username + org membership).
Sign in to claimOwn this plugin?
Verify ownership to unlock analytics, metadata editing, and a verified badge. GitHub access is read-only (username + org membership).
Sign in to claimBased on adoption, maintenance, documentation, and repository signals. Not a security audit or endorsement.
Uses power tools
Uses Bash, Write, or Edit tools
Uses power tools
Uses Bash, Write, or Edit tools
Autonomous Research Agent for LLM Mechanistic Interpretability
Overview · Installation · Quick Start · Usage Guide · Acknowledgements · 中文
Mechanist converts a research question about the internal mechanisms of large language models into evidence-backed findings. It coordinates a complete research workflow: literature retrieval, hypothesis formulation, experiment implementation and execution, robustness validation, and iterative refinement — all within a single autonomous pipeline.
Mechanist is distributed as a Claude Code plugin. You do not need to clone this repository to use it (see Installation).
| Stage | Description |
|---|---|
| Literature Review | Queries a 14k-paper interpretability corpus, a 157M-node cross-disciplinary citation graph, and web sources. |
| Hypothesis Formulation | Proposes novelty-checked claims, or extracts claims from user-provided material. |
| Experiment Execution | Generates experiment code, runs evaluations, and records results against a mechanism-aware plan. |
| Verification | Evaluates claims under alternate models, datasets, and methods for robustness. |
| Iteration | Reviews failed or weak results, updates the plan, and reruns the relevant stages. |
research question ──▶ claim ──▶ experiment ──▶ verify ──▶ iterate ──▶ findings
(hypothesis) (execution) (validation) (refinement)
The /auto pipeline consists of an orchestrator that dispatches four sequential stages, each running in an independent sub-agent:
All results are tracked in a Claim Ledger (CLAIMS_LEDGER.md) that records every claim's journey from hypothesis to final verdict.
Download and install Claude Code, then log in:
# Download and install Claude Code
curl -fsSL https://claude.ai/install.sh | bash
# Restart your terminal, then verify
claude --version
[!IMPORTANT] Mechanist requires the Opus 4.7 model. Launch each session with
--model claude-opus-4-7, or select Opus 4.7 within a session using/model.claude --model claude-opus-4-7
The Mechanist MCP servers use uv to bootstrap temporary Python environments:
# Download and install uv
curl -LsSf https://astral.sh/uv/install.sh | sh
# Restart your terminal, then verify
uv --version
npx claudepluginhub zjunlp/mechanist --plugin mechanistPhD-level research capabilities: literature review, multi-source investigation, critical analysis, hypothesis-driven exploration, quantitative/qualitative methods
Autonomous research loops with 10 commands. Generalizes Karpathy's autoresearch loop to any domain with mechanical evaluation, overnight persistence, and zero dependencies.
Idea discovery, proposal refinement, experiment execution, and deep literature orchestration
Three AI models, one synthesis — multi-model research workflow for scientific domains
Autonomous, personalized research loops for Claude Code. Set a topic, walk away, come back to a quality-gated report adapted to your projects.
Comprehensive Research Planning agents specializing in synthesising hypothesis and claims, researching related work and challenging assumptions.