deep-research

Deep Research

One entrance, two collection modes, the same quality bar on both.

Three things below are measured, not assumed. They are why this pipeline is shaped the way it is:

• Self-reported confidence is decorative. Calibration error above 0.377; on a 1–5 scale RLHF models answer "5" in 96.5% of cases. Never ask the model how sure it is — count independent confirmations instead.
• Agents that debate each other collapse into agreement. Three independent 2025 studies found multi-agent debate degrading into sycophancy and consensus collapse, often below single-agent performance. So: parallel independent agents, reconciled by source quality, never by vote.
• Guessing is rational unless you make it costly. Most benchmarks penalise "I don't know" exactly as much as a wrong answer, so models learn to guess. Two models on one task: 52% abstention / 26% error versus 1% abstention / 75% error.

The overriding rule: an admitted gap beats a plausible invention. Every phase has a way to say "not established". Use it.

---

Phase 0 — Contract

Before any search, ask the user three questions. Do not skip them and do not guess the answers.

1. Depth. Quick scan (one round per aspect) · standard (until saturation) · exhaustive (saturation plus an adversarial pass).
2. Source classes. Docs and articles only, or also native material — video transcripts, forum threads, practitioner posts, incident reports? Native material is where practice lives; it is off by default because it costs more to collect.
3. Done means what. One sentence from the user: which decision this research feeds.

Record all three verbatim at the top of the report. They are the acceptance criteria — Phase 4 checks against them.

---

Phase 1 — Decompose

Break the topic into orthogonal aspects: no overlap, and together they cover the question.

• 3–6 aspects. Fewer means the topic was narrow — say so rather than padding. More means it was two topics.
• Each aspect must be answerable independently. If B needs A's answer first, they are one aspect.
• Name what would change the conclusion. If nothing could, this is not research but a search for confirmation — say that and stop.

Show the decomposition. Wait for confirmation before Phase 2. A bad decomposition wastes the entire downstream budget.

---

Phase 2 — Collect

Choosing the mode

	Mode A — sequential	Mode B — ensemble
When	≤3 aspects, or aspects share context	≥4 genuinely independent aspects
How	aspects one at a time in the main context; tools within an aspect in parallel	one agent per aspect, launched together
Cost	one context	N contexts plus reconciliation

Before launching Mode B, tell the user the agent count and the expected order of cost, and wait for a yes. Do not use Mode B to split one modest job — independence is the criterion, not size.

Depth and breadth move together

Raising one alone does not help; both must clear a floor. Per aspect: at least two rounds of search and at least two distinct source types. An aspect covered by one round, or by one type of source, is not covered.

When an aspect is done

Stop on saturation, not on a fixed count.

• 6–12 substantive sources per aspect is the working band (from qualitative methodology, where saturation has twenty years of study)
• stop earlier when two consecutive rounds surface nothing new
• marginal return drops sharply after the third search round on one aspect — if a fourth feels necessary, the aspect is probably mis-scoped; return to Phase 1 instead of grinding

If saturation is not reached within the chosen depth, say so. Do not pad to look complete.

Recording a claim

Every claim carries four fields. This is not bookkeeping — it is the fix for the failure where a derived claim silently becomes a verified one while passing between agents or phases.

Field	Values
source	URL, or derived
basis	read (source opened) · snippet (search result only) · derived (inferred) · assumed
support	count of independent confirmations
conflict	set when any source contradicts this

basis: read is allowed only if the source was actually opened in this session. A snippet is not a read.

Source assessment — two axes, not one

Platform type is metadata, not a verdict. A forum thread carrying a reproducible config beats a vendor post carrying none. Discarding practice by format is exactly how a research report ends up as an academic residue with no operational value.

Axis 1 — platform tier (record it; do not judge by it): official docs, specs, peer-reviewed work, repos >5K stars · reputable technical publications, repos >1K stars · attributed personal writing, Q&A sites · forums, undated or anonymous material.

Axis 2 — content quality signals, 0–6, one point each:

• reproducible specifics: versions, configs, commands, numbers
• states its own limitations, or what failed
• concrete rather than general
• dated, and recent enough for this topic
• first-hand observation rather than restatement
• internally consistent

A tier-C source scoring 5–6 outranks a tier-A source scoring 1–2. When that happens, say so in the report.

Platform popularity signals are weak evidence. Vote counts and accepted answers are distorted by the first-answer advantage; on at least one major Q&A site reputation does not correlate with expertise, accepted answers create a self-reinforcing vote loop, and vote-selling markets exist. Use them as a tiebreak, never as proof.

Independence is not source count

Three articles quoting one original study are one confirmation. Before incrementing support, check: do these sources cite each other or a common origin? Is one a reprint or a paraphrase? Could one plausibly have been generated from the others? Corroboration among non-independent sources is the most common route to a confidently wrong report.

---

Phase 3 — Reconcile

A separate phase, before synthesis. Not part of writing — that is precisely when the discipline gets lost.

1. Collect every claim carrying conflict.
2. Resolve by source quality, never by how many agents or sources said it. Correlated error is the default failure mode of a majority vote.
3. What cannot be resolved goes into the report as an open contradiction, with both positions and their evidence. Never pick a side silently.
4. Where documentation contradicts practitioner reports, record both and note the pattern: docs lag releases and undocumented behaviour is common. Neither side wins automatically.

Causal check

Any claim shaped "X drives Y" is challenged before it enters the report. Strength of association is not evidence of a lever. Look for a source that manipulated X and observed Y; if none exists, downgrade to "associated with" and state that the causal direction is untested.

Learned expensively: a feature with the highest measured association turned out to have zero causal effect under direct manipulation. Stopping at the correlation would have produced a confident and useless recommendation.

---

Phase 4 — Quality gate

Numeric, checked before writing. Failing is a legitimate outcome.

Check	Pass condition
Saturation	every aspect reached it, or the shortfall is stated
Coverage	every aspect has ≥2 independent sources
Diversity	≥3 distinct domains per aspect
Grounding	≥70% of claims are basis: read, not snippet
Conflicts	all resolved, or explicitly recorded as open
Answers the question	the Phase 0 sentence is addressed

On failure: report which checks failed and what would close them, then offer another round. Never write a confident report over a failed gate — that is the exact failure this skill exists to prevent.

---

Phase 5 — Report

Confidence is computed, not felt

Independent confirmations	Label
3+	HIGH
2	MEDIUM
1	LOW — mark [single source]
0, inferred	[DERIVED] — say so in the sentence itself

Never assign a label from impression. If the confirmations cannot be counted, the claim is not ready.

Structure

# {Topic}

**Date** · **Depth requested** · **Sources: N** · **Gate: PASS/FAIL**
**Question this answers:** {user's Phase 0 sentence, verbatim}

## Not established

{First, deliberately. What was looked for and not found, what stayed
contradictory, which aspects fell short of saturation. An empty section
here on a non-trivial topic is a warning sign, not an achievement.}

## Summary

{3–5 bullets, each usable in a decision}

## Findings by aspect

### {Aspect}

{Claim} [HIGH] — [source](url), [source](url)
{Claim} [LOW, single source] — [source](url)

## Contradictions

{Both positions, their evidence, why unresolved}

## Recommendations

{Numbered, each traceable to a finding above}

## Sources

| # | Title | URL | Tier | Quality 0-6 | Date | Basis |

Banned

• A confidence label not derived from a count
• A source in the table that was not actually consulted
• Filler openers: "It's important to note", "In today's landscape", "As we all know", «Стоит отметить», «В современном мире»
• Any number without a source
• An empty "Not established" section on a topic with real uncertainty

Where to save

.claude/research/YYYY-MM-DD-topic.md. Ensemble mode: a directory with one file per agent plus MASTER.md.

---

Ensemble protocol (Mode B)

Basis note: assembled from practice. No validated academic standard exists for preserving reliability metadata across agent handoffs — this is the best available construction, not a citation.

Each agent's briefing must contain — a subagent inherits no conversation history, so anything omitted is simply absent:

1. its aspect, and explicitly what is not its aspect
2. the four claim fields above, as a required output format
3. the source assessment rules, both axes
4. instruction to mark gaps rather than fill them
5. the output path
6. the search tool, named exactly — see below

Name the tool, or do not launch the track. A briefing that says "use Exa or Tavily if available" hands the availability check to the agent, and an agent that finds the named tool missing does not stop — it silently falls back to generic web search and returns official pages instead of practice. Write the exact tool id (mcp__exa2__web_search_exa), and verify it resolves before launching. If it does not resolve, stop and tell the user; do not substitute.

One key per agent when the provider is rate-limited per key. Parallel agents sharing a single credential queue behind one limit while the rest of the pool idles. Where the pool is exposed as separate MCP servers (exa1..exaN on EXA_KEY1..N), agent N is briefed on server exa[(N mod poolSize) + 1] and no other. Rotate to the next server on 429 / 402 / 401 / 403 / 5xx; do not rotate on 200 or 400 — a 400 is a malformed request, and it will be malformed for every key.

Write to the file, not to the chat. A finding reported as a message and not written to the agent's output path is lost when the context closes: the parent sees a summary, the file keeps a placeholder, and the acceptance check reads the file. Messages carry status only.

Agents do not talk to each other. Independence is the whole point; cross-talk reproduces the debate failure described at the top.

The assembler does not synthesise first. Order is: collect all reports → Phase 3 reconciliation → Phase 4 gate → only then write. An assembler that starts writing while reading is how reliability markup disappears.

The assembler re-checks rather than trusts. A claim arriving as derived or assumed keeps that basis in the master report unless the assembler opens the source itself.

---

Failure modes to watch

Symptom	What it actually is
Everything confirms the initial hypothesis	the search was framed for confirmation; redo Phase 1 asking what would falsify it
Long report, decides nothing	aspects were not orthogonal, or the Phase 0 question was never answered
Many sources, all saying the same thing	check independence — likely one origin restated
Confidence high everywhere	labels assigned by feel instead of counted
"Not established" is empty	the gaps were smoothed over
Depth appeared only after the user pushed back	the first pass was lazy; the standard is to exhaust available sources proactively