Carta
Maps, connects, and remembers your documentation.
Carta is a Claude Code plugin that keeps your project docs honest — auditing for contradictions, embedding reference material into a searchable knowledge base, and surfacing the right context exactly when you need it.
The problem (or: how this got built)
Fast-moving projects accumulate documentation debt quietly. You write a spec. An AI agent writes a dozen more files based on it. The spec changes. Three weeks later, four different documents describe the same API endpoint four different ways, and nobody — human or AI — knows which one is right.
This problem gets worse the more you lean on AI agents to help you work. Agents are only as good as the context they can see, and when your docs/ folder is a fog of contradictions and stale frontmatter, you're giving your agent a map that leads off a cliff.
Carta started as a happy accident. While working through a project with a lot of PDFs, datasheets, and fast-changing markdown — the kind of repo where the hardware changes on Thursday and the docs are still describing Wednesday — we built a small structural scanner to flag stale and broken cross-references. Then we added a semantic pass. Then a vector store. Then a /doc-search skill so Claude could query the embedded knowledge directly.
At some point we looked at what we had and realized: this is a thing. It works. It's small, it runs locally, it requires no new services beyond what an LLM-augmented developer already has running. So we generalized it.
What Carta does
Three things, tightly integrated:
1. Audit
A two-pass system that runs on a schedule or on demand:
- Structural scanner (zero LLM calls) — detects stale docs, broken
related: links, homeless markdown files, and orphaned content. Runs fast, runs often.
- Semantic audit (Claude) — reads the scanner output and checks changed doc pairs for contradictions: version numbers, API endpoints, config values, whatever matters in your domain. Writes a rolling
docs/AUDIT_REPORT.md with stable AUDIT-NNN issue IDs that persist across runs.
2. Embed
Ingests your reference material — PDFs, datasheets, manuals, audio transcripts — into a local Qdrant vector store via Ollama. Generates spec_summary blocks for dense documents so the audit agent can cross-reference them without re-reading 200 pages.
3. Search
Natural language recall over everything that's been embedded. Ask Claude what the docs say about rate limiting, authentication flows, power supply constraints, sample naming conventions — whatever's in your knowledge base — and get cited answers back.
Retrieval quality
Search is hybrid (dense + BM25 with Reciprocal Rank Fusion) by default, with an optional
ColPali visual layer for image-heavy PDF pages. Measured on a real technical-docs corpus
(~160 markdown docs + 214 datasheet PDFs, local models — nomic-embed-text + Qdrant/bm25):
Text retrieval — markdown eval, 20 queries:
| Pipeline | recall@5 | MRR |
|---|
| Dense only (cosine) | 0.550 | 0.402 |
| Hybrid (BM25 + dense, RRF) | 0.700 | 0.546 |
On an expanded 62-query set over the same corpus (adds datasheet, supplier, and patent
reference docs): hybrid alone scores 0.790 / 0.641, and the LLM reranker (qwen3.5:9b,
candidate pool 40) lifts it to 0.871 / 0.778 — with rerank: applied on 61/62 queries
confirming the reranker actually ran on every scored query but one.
Visual retrieval — datasheet eval, 14 queries:
| Pipeline | recall@5 | MRR |
|---|
| Text / OCR only | 0.500 | 0.429 |
| + ColPali visual (two-pass) | 0.857 | 0.589 |
The datasheet set includes 6 "visual-only" queries whose answer lives on a diagram, package
drawing, or derating curve that text search structurally can't reach — ColPali lifts those from
0/6 to 5/6. Text and visual hits are fused by rank (RRF), so the visual layer never crowds
out text results.
These are one project's eval sets, not a public benchmark — they show the delta each layer
adds on real technical docs, not an absolute SOTA claim.
When search.rerank.enabled is true, carta eval also prints rerank: applied on N/M queries
— and fails (exit 1) if the reranker ran on zero queries, so a silent fail-open (wrong model
name, Ollama down, reasoning-model misconfig) can never masquerade as a reranked result.
