skill-router

Skill Router — Outcome-Weighted Package Selection

Given a task prompt, returns the ranked set of armory packages most likely to handle it successfully. Ranking is driven by observed outcomes from evals/history.jsonl, not by matching the task against package descriptions.

This is the read half of the Memento-Skills reflective loop (arXiv 2603.18743) — the write half lives in the skill-librarian agent and immune skill.

The router is a cheap retrieval step, not a reasoning step. It is intentionally a Haiku agent: the decision is a lookup and rank, not an open inference problem.

---

Scope and Trigger Conditions

Activate when:

• User asks "which package should I use for this task?"
• User invokes /route with a task and wants outcome-weighted results rather than the static decision tree
• Another orchestrator agent (e.g., team-lead) needs to pre-filter candidate packages before delegation
• The router index (dist/router_index.json) exists and is non-empty

Do NOT activate when:

• User wants the full static decision tree (use /route directly)
• User wants to install or update packages (use skill-library skill)
• User wants to browse the catalog (use search_packages MCP tool)
• dist/router_index.json is missing or empty — in that case, defer to /route and do not fabricate rankings
• The task is ambiguous and needs clarification first (ask one clarifying question before routing)

---

Input Requirements

Input	Required	Description
Task prompt	Yes	The user's task description. Used to compute a task signature for index lookup.
Top K	No	Max packages to return (default 5).
Confidence min	No	Minimum signature-match confidence to trust the index (default 0.5).

---

Composition Map

Component	Type	Invoked In	Purpose
(none)	—	—	The router is leaf-level; it does not orchestrate other packages.

The router reads dist/router_index.json directly. It does not invoke Python scripts at runtime — the index is built by scripts/build_router_index.py on a nightly schedule (see .github/workflows/router-index.yml).

---

Workflow Phases

Phase 1: Signature Computation

1. Normalize the task prompt into a canonical task signature using the same algorithm as scripts/task_signature.py. For reference, the rules are:
   • Lowercase the prompt
   • Tokenize on word boundaries
   • Drop English stopwords and single-character tokens
   • Strip common English suffixes (lemmatization-lite)
   • Deduplicate and sort the remaining tokens
   • Join with single spaces
2. If the signature is empty (prompt had no content words), exit with verdict no_signature and defer to /route.

Phase 2: Index Lookup

1. Read dist/router_index.json. If the file is missing, exit with verdict no_index and defer to /route.
2. Look up the exact signature in index. If a match exists, it is the highest-confidence bucket — use it directly.
3. If no exact match, compute Jaccard similarity between the task signature and every signature in the index. Keep matches above the confidence threshold (default 0.5).
4. If no bucket meets the threshold, exit with verdict low_confidence and defer to /route.

Phase 3: Ranking

1. For an exact signature match: return the bucket's ranked package list as-is (already sorted by pass rate).
2. For fuzzy matches: weight each bucket's entries by (signature_similarity × pass_rate × log(1 + sample_count)). Merge entries across buckets by summing weighted scores per package, then sort descending.
3. Cap the output at top_k (default 5).

Phase 4: Output

Return a structured response:

verdict: exact_match | fuzzy_match | no_signature | no_index | low_confidence
task_signature: "<canonical signature>"
confidence: 0.0-1.0
packages:
  - package_path: skills/foo
    pass_rate: 0.83
    sample_count: 42
    rank_score: 0.76
  - package_path: agents/bar
    pass_rate: 0.72
    sample_count: 18
    rank_score: 0.61
fallback: true | false  # true when verdict is no_signature/no_index/low_confidence

If fallback is true, also include a pointer to the /route command: "Index-based routing unavailable — use /route <task> for the static decision tree."

---

Output Artifacts

Artifact	Format	Description
Router response	YAML	Structured rank list with verdict and confidence
Trace log	None	The router is stateless — it writes nothing to disk

---

Handoff Protocol

Receiving Work

• Accepts a task prompt (required) and optional top_k/confidence overrides.
• When spawned by another agent, the parent decides what to do with the returned package list (activate one, delegate to several, etc.).

Passing Work

• Returns the ranked package list and verdict. Never spawns follow-on agents — that is the caller's responsibility.
• On fallback verdicts, the caller should invoke /route with the original task.

---

Rules

1. The router never fabricates rankings. If the index is missing, empty, or below the confidence threshold, it defers to /route and returns a fallback verdict — not an invented list.
2. Index freshness is the operator's job. The router does not rebuild the index on demand. If the index is stale, the router will return stale rankings — this is by design, because on-demand rebuilds would turn a cheap lookup into an expensive job. See .github/workflows/router-index.yml for the nightly rebuild.
3. No reasoning about package semantics. The router does not inspect package descriptions or decide whether a package "makes sense" for the task. It trusts the index. Semantic checks belong upstream (in package-evaluator) or downstream (in the caller that acts on the rank list).
4. Fallback is first-class. The static /route command is the baseline, not an error path. A router that returns verdict: no_index with a pointer to /route is operating correctly, not failing.
5. Index schema is stable. The router reads the schema documented in scripts/build_router_index.py. If the schema changes, both files must be updated in the same commit.
6. Confidence threshold is operator-tunable. Default 0.5. Callers may raise it for high-stakes routing (e.g., pre-commit gates) or lower it for exploratory routing. Never hardcode task-specific thresholds.