literature-sweep

/literature-sweep

This engine is the parallel-dispatch worked example: the first OMCR engine that fans out across several @literature-curator instances in a single message and then re-collects their work. The other Phase 2 engines (/respond-reviewer, /figure-bake, /outline-expand) all reference this skill's phase 03 for the parallel-dispatch pattern.

If you are reading this because Claude Code's skill auto-discovery surfaced it: invoke it via /literature-sweep <topic>. Do not hand-edit references.bib, the summary CSV, or .claude/omcr-state/citations.json while a run is in flight.

When this skill is invoked, immediately execute the workflow below. Do not only restate or summarize these instructions back to the user.

Signature

/literature-sweep <topic> [--n N] [--depth basic|deep] [--source crossref|openalex|both] [--parallel P]

Flag	Default	Purpose
--n	20	Target number of verified entries (final). Phase 02 pulls 3 * N candidates; phase 04 dedupes and trims; phase 05 verifies.
--depth	basic	basic extracts title/authors/year/venue/abstract. deep adds a per-entry curator dispatch for methodology + key findings.
--source	both	crossref | openalex | both. both queries each source and merges.
--parallel	1	Phase 03 concurrent curator instances. Clamped to 1 ≤ P ≤ 4 (Phase 2 decision §1). Phase 01 warns and clamps values > 4; aborts on < 1.

Examples:

• /literature-sweep "neural manifolds in motor cortex" — defaults: 20 entries, basic depth, both sources, sequential.
• /literature-sweep "diffusion models for protein design" --n 30 --parallel 4 — 30 entries, four curator instances in parallel.
• /literature-sweep "transformer attention sparsity" --depth deep --source crossref — deeper extraction, CrossRef only, sequential.

The pipeline

phase 01 — precheck         (validate topic, ping APIs, resolve bib + CSV paths, clamp --parallel)
phase 02 — search           (CrossRef and/or OpenAlex → 3N candidates, dedupe by DOI)
phase 03 — parallel-read    (@literature-curator × P, each handles a batch — BibTeX + CSV row per paper)
phase 04 — deduplicate      (merge batches, dedupe by DOI, score, trim to N)
phase 05 — verify           (verify-citation skill per survivor — HARD GATE)
phase 06 — finalize         (append to references.bib + CSV; write citations.json.last_sweep; append to _run-log.jsonl)

Phases 02–05 form the read-then-verify pipeline. Unlike /iterate-revision, there is no inner loop — each phase runs once. The parallel concurrency lives entirely in phase 03 and is transparent to phases 04 and 05.

Phase execution

Execute phases sequentially. For each phase, read the linked file and follow its instructions exactly.

1. Phase 1 — Precheck. Read phases/01-precheck.md.
2. Phase 2 — Search. Read phases/02-search.md.
3. Phase 3 — Parallel read. Read phases/03-parallel-read.md.
4. Phase 4 — Deduplicate. Read phases/04-deduplicate.md.
5. Phase 5 — Verify. Read phases/05-verify.md.
6. Phase 6 — Finalize. Read phases/06-finalize.md.

Composition

This engine imports the following primitives from skills/orchestrate/:

• phases/01-state-read.md — read citations.json in phase 01 (bootstrap if missing) and re-read in phase 06 for the durable last_sweep write.
• phases/02-dispatch.md — dispatch @literature-curator. Phase 03 uses it in two shapes: a single sequential call per batch when --parallel 1, or multiple concurrent calls in one assistant message when --parallel > 1 (Claude Code Agent-tool parallel invocation).
• phases/04-loop.md — used only for the _run-log.jsonl phase: "start" + phase: "end" records and run_id generation. There is no inner iteration loop in this engine (one shot, not iterative), so phase 04 of orchestrate is invoked with max_iter = 1 and a trivial always-after-n verdict rule that emits DONE immediately. The real engine logic lives in phases 02–06 here, not inside the loop body.

This engine does not import orchestrate/phases/03-evaluate.md — there is no severity-threshold verdict to compute. The verification gate in phase 05 is a hard pass/fail per entry, not an aggregate verdict over the run.

Integration with verify-citation

Phase 05 calls the standalone skills/verify-citation/SKILL.md skill once per surviving entry (after phase 04's dedupe), via its --doi <doi> mode (we have a DOI by then; we are not auditing an existing BibTeX file). The skill returns one of PASS | MISMATCH | NOT_FOUND | NOT_VERIFIED per entry.

The hard gate (Phase 2 decision §1, locked):

• PASS → entry is added to references.bib and to the summary CSV; recorded in citations.json.verified.
• MISMATCH / NOT_FOUND / NOT_VERIFIED → entry is rejected; recorded in citations.json.last_sweep.rejected with reason. Never written to references.bib.

The user reviews citations.json.last_sweep.rejected after the run and decides whether to manually rescue any rejected entries (typo in DOI, network outage, etc.).

Cost model

Per-entry cost is dominated by:

• One @literature-curator dispatch per --parallel-batch in phase 03 (handles up to 3N / P candidates per dispatch).
• One verify-citation call per surviving entry in phase 05 (cheap; a single HTTP roundtrip per source, stdlib-only Python).
• An optional second curator dispatch per survivor when --depth deep (rough cost doubling for the dispatch portion).

With --n 20 --parallel 1 --depth basic: ~1 curator dispatch over 60 candidates + 20 verify-citation calls. The frontmatter cost_estimate_tokens: 50000 is the coarse upper bound for /supervisor-drive budget gating (Phase 3 §6); actuals land in _run-log.jsonl post-hoc per Phase 0 decision §6.

--parallel 4 does not multiply token cost — the work is sharded, not duplicated — but it does multiply concurrent token throughput, which is the relevant constraint for the runtime's rate limiter.

Locked decisions honored

From develop/phase-2-decisions.md §1:

• Sequential default. --parallel defaults to 1. Power users opt in.
• Clamp 1 ≤ N ≤ 4. Phase 01 warns and clamps > 4; aborts on < 1.
• Parallel failure mode. If one batch errors during the parallel dispatch, the engine falls back to sequential for the remaining batches and records a one-line warning in citations.json.last_sweep.notes. No in-loop retry; the user can re-run with --parallel 1 if they want a clean sequential pass.
• Hard verify-gate. Unverified entries never enter references.bib. Always recorded in citations.json.last_sweep.rejected with a reason so the user can audit.

What this engine does NOT do

• Does not fabricate references. Every entry survives verify-citation before being added — by construction, fabricated DOIs cannot pass.
• Does not decide bucket, our_use, or cited_sections columns in the summary CSV. Those are human-curated columns that the literature-curator agent leaves blank in a sweep; the user (or a later curator pass keyed to a specific manuscript section) fills them.
• Does not invoke /iterate-revision, /respond-reviewer, /figure-bake, or /outline-expand. Engines are leaves (Phase 2 decision §5).
• Does not auto-resolve [CITE: ...] placeholders inside the manuscript. That is the standalone @literature-curator agent's day-job. /literature-sweep is about building a bibliography, not patching a specific placeholder.
• Does not commit to git or push to a remote. Run /sync afterward if you want a commit-and-push snapshot.
• Does not re-fetch full PDFs. Abstract-level metadata only. Deeper claim-fit checks need a human (or a dedicated future engine) to read the paper.

Re-running policy

• Topic already in citations.json.last_sweep.topic and the user re-runs the same topic → the engine still runs. Newer candidates may exist on CrossRef/OpenAlex since the last sweep; dedupe against the BibTeX file (not against last sweep) ensures no double-adds.
• A DOI already in references.bib (case-insensitive) → phase 03 / 04 skip it, count it toward the --n target, and the curator dispatch never re-extracts it. The summary CSV row is left as-is.
• A DOI already in citations.json.verified (from a prior sweep) but missing from references.bib (e.g., user hand-edited the bib) → treated as new; re-verified; re-added.
• Network outage during phase 02 or phase 05 → engine emits a partial result with notes: "API <name> unreachable; partial result" in last_sweep. Any entries that did verify are still added.

See also

• ../orchestrate/SKILL.md — the 4 primitives this engine composes.
• ../verify-citation/SKILL.md — the standalone verifier called from phase 05.
• ../iterate-revision/SKILL.md — pattern reference; voice and structure of this skill mirror it.
• ../../agents/literature-curator.md — the primary dispatched persona; phase 03 inlines its body via orchestrate/phases/02-dispatch.md.
• ../../develop/example-state/README.md — citations.json schema reference, including the last_sweep block this engine writes.
• ../../develop/phase-2-additional-engines.md — full design rationale (Engine 2).
• ../../develop/phase-2-decisions.md — locked decisions §1 (concurrency).