science-agent

Science Agent

Verify what AI writes about science — citations, claims, and cross-file consistency.

A README linked to PMID 12078741 as the foundational paper on Restricted Focus Viewers in vision science. The actual paper at that ID? "Determination of true ileal amino acid digestibility... in barley samples for growing-finishing pigs." The correct PMID was 12723780 — off by 645,039.

Try it now

npx github:andyed/science-agent audit ./docs --bibtex=./refs.bib

Or verify a single DOI against CrossRef:

npx github:andyed/science-agent verify 10.1038/nn.2889

No install, no clone — runs straight from this repo.

What it does

Citation Verification (shipped)

Catches AI-confabulated academic citations before they ship. Verifies inline references against BibTeX and CrossRef:

Pattern	How
Wrong title	Fuzzy title matching against BibTeX + CrossRef
Fabricated co-authors	CrossRef author list verification
Wrong DOI	CrossRef DOI resolution — checks that the DOI points to the claimed paper
Compound confabulation	CrossRef + title search detects merged citations
Ambiguous citation	Surname+year collision detection across BibTeX entries
Orphan citation	Inline reference with no BibTeX entry

Notebook Claim Verification (shipped)

Research notebooks produce numbers. Papers and READMEs cite those numbers. Over time, numbers drift — a notebook gets re-run with new data, but the prose still quotes the old value. Science Agent makes this auditable.

The idea: Each notebook declares its load-bearing results in a ## Key Claims table. Prose references them as [NB14:K3] (notebook 14, claim 3). Science Agent verifies every reference resolves to a real claim with a real value.

## Key Claims

| ID | Claim | Value | Verified |
|----|-------|-------|----------|
| K1 | Sample size after exclusions | N = 2,719 trials | 2026-04-09 |
| K2 | Main effect | ρ = −0.618, p = 0.0426 | 2026-04-09 |

Then in your paper or README:

The position × cognitive load correlation [NB14:K2] suggests...

Audit it:

# Generate aggregate from notebooks (one-time setup)
science-agent aggregate ./notebooks/ -o docs/notebook-key-claims.md

# Audit all claim references in prose
science-agent notebook-audit ./docs \
  --aggregate=./docs/notebook-key-claims.md \
  --notebooks=./notebooks/ \
  --cross-repo=../downstream-repo

Detects:

• Dangling references — [NB14:K3] cited in prose but K3 doesn't exist in NB14
• Missing Key Claims blocks — notebook is cited but has no auditable claims table
• Stale cross-repo values — downstream repo quotes pre-fix numbers from upstream

If you don't use notebooks or don't need claim tracking, ignore this — audit and verify work standalone. If you do, see the full setup guide: docs/notebook-conventions.md

Works with any AI coding assistant

Science Agent is a CLI tool — any assistant that can run shell commands can use it. No API keys, no plugins, no vendor lock-in.

Claude Code:

> check my citations against refs.bib
# Claude runs: npx github:andyed/science-agent audit ./docs --bibtex=./refs.bib

ChatGPT / Codex / GitHub Copilot in terminal:

> run science-agent to verify the citations in my paper
# GPT runs: npx github:andyed/science-agent audit ./paper --bibtex=./references.bib

Gemini Code Assist / Cursor / Windsurf / any terminal AI:

> audit my bibtex citations for confabulation
# Assistant runs: npx github:andyed/science-agent audit . --bibtex=./refs.bib

The pattern is the same everywhere: point at a directory of prose and a BibTeX file. The tool does the rest.

For deeper integration, see agent.md (Claude Code agent) or docs/github-actions.md (CI/CD).

Claude Code plugin (recommended)

Install the whole toolkit — agents, slash commands, CLI — in one step:

/plugin install andyed/science-agent

This pulls the repo, installs npm dependencies into the plugin data dir, and registers:

• Slash commands: /science-agent:audit, /science-agent:figure-audit, /science-agent:notebook-audit, /science-agent:verify, /science-agent:search, /science-agent:aggregate, /science-agent:arxiv
• Sub-agents (auto-invoked when relevant): figure-audit, rigor-audit

prose-audit is not in the plugin yet — its rule table currently lives in muriel (Python) and would force a Python install on every plugin user. Available locally as node cli.js prose-audit <file> if you have muriel cloned. Native Node port tracked.

A planned fourth agent — claim-audit — will cover semantic claim correctness (does the cited paper actually say this?). It slots into agents/ when Phase 4 of PLAN.md ships.