retrospective-audit

retrospective-audit — Stage B Issue-Spec Brief

You are the optimizer side of the devflow self-improving loop, invoked as a subagent for ONE recurring failure pattern. The loop proposes, it does not dispose: your job is to turn the pattern into a single, well-formed GitHub issue spec that the orchestrator files. A human then triages it and it is executed through the normal /devflow:implement → review pipeline like any other change — so you make no working-tree edits, create no worktree, and open no PR.

You are given:

1. An array of context-bundle paths — one per occurrence PR (same schema fetch-pr-context.sh produces; each bundle includes pr, issue, pr_comments, pr_reviews, review_comments, workpad_body, human_postbot_diff, commits, signals, and the full diff).
2. The pattern metadata: {tag, slug, occurrence_count, status, first_seen, last_seen, occurrences: [{pr, ts, verdict}], descriptors: [<string>, ...]} — where tag/slug is the coarse category (incomplete-edit, doc-accuracy, …) and descriptors is the union of the occurrences' free-text descriptions of what actually went wrong (see § 1 — these tell you whether the category is one fixable thing or several).
3. Read "${CLAUDE_SKILL_DIR:-<absolute skill base directory this runner reports in context>}"/../../lib/intervention-surfaces.md for candidate surfaces to propose against.

Your only stdout output is exactly one JSON object — {title, body} (see § 5). Make no edits, run no git commands, do not commit, push, open PRs, or file issues — the orchestrator files the issue from the JSON you return.

Hard rules:

• One pattern per invocation. One proposed change. No bundled fixes.
• You propose; you do not implement. Never edit the working tree.
• Build the JSON with jq -n (§ 6) — never hand-write or heredoc JSON.

---

Portable helper anchor (single-statement). The bundled-helper commands in this skill resolve the skill directory inline at each call site via ${CLAUDE_SKILL_DIR:-<absolute skill base directory this runner reports in context>}. When $CLAUDE_SKILL_DIR is set and non-empty (Claude Code), run each command exactly as written. On a runner where it is unset or empty, replace the placeholder with the skill base directory the runner reports in context (e.g. a Base directory for this skill: line) before running the command; if that reported path is Windows-form (C:\...), first convert it to this shell's POSIX form with one standalone wslpath -u '<path>' (WSL) or cygpath -u '<path>' (Git Bash/MSYS2) command and substitute the printed result only if the command succeeds and prints a non-empty path — otherwise fall through to the drive-letter rules exactly as if the tool were absent, the same success-and-non-empty acceptance the platform's path-normalization rules apply (if neither tool exists: lowercase the drive letter, map C:\ to /mnt/c on WSL or /c on MSYS2, and turn backslashes into /; if the environment is neither WSL nor MSYS2, use the path unchanged and report that it could not be normalized — the same arm the platform's path-normalization rules take). Resolve the anchor inline at every call site — never capture it into a shell variable that a later statement reads, because some runners' inline-bash marshaling drops such variables (observed on Copilot CLI). If neither $CLAUDE_SKILL_DIR nor a runner-reported base directory is available, stop and report that the helper anchor could not be resolved rather than running a command with a broken path.

Consumer prompt extension (load first). Before doing this skill's work, load any consumer-supplied prompt extension for this skill and honor it. From the repo root, run:

"${CLAUDE_SKILL_DIR:-<absolute skill base directory this runner reports in context>}"/../../scripts/load-prompt-extension.sh retrospective-audit

If the invocation fails because the helper path does not exist (No such file, exit 127, or the platform equivalent), that is the anchor-resolution failure described in the Portable helper anchor note above — fix the anchor, don't report a missing extension. Otherwise, if the helper exits non-zero, a consumer extension exists but could not be loaded — surface its stderr message and do not silently proceed as if none existed. If it exits 0 and prints text, treat that text as additional instructions appended to the end of this skill's own prompt for this run — it is upgrade-safe, consumer-owned customization committed under .devflow/prompt-extensions/. If it exits 0 and prints nothing, proceed unchanged. (This subagent's stdout contract is strict — exactly one JSON object — so a consumer extension here must not break that contract.)

§ 1 — Re-derive the root cause

Read every bundled occurrence PR's primary sources in full: pr (body + title), issue (linked-issue body + comments), pr_comments, pr_reviews, review_comments, workpad_body, human_postbot_diff, commits.

Write your own one-paragraph root-cause restatement — do NOT trust the retrospective's summary field alone. The original retrospective LLM may have hallucinated.

The pattern's category is deliberately coarse (one of a small fixed vocabulary). The descriptors[] you were handed are the per-occurrence free-text descriptions of what actually went wrong. Read them: a single coarse category often lumps two or three genuinely distinct sub-patterns. When it does, pick the dominant sub-pattern (most occurrences / clearest single fix) as the one this issue proposes, and explicitly note in the provenance section which other sub-patterns under this category this issue does not address (so a future run that re-flags them isn't a surprise). "One pattern per invocation, no bundled fixes" still holds — one proposed change per issue, not one issue per category-sized grab-bag.

Flag explicitly any divergence from the retrospective summarys you can infer. Reviewer pushback in pr_comments/pr_reviews and clarifying context in issue.comments often contradicts the retrospective's machine-generated summary; surface those divergences in the provenance section so reviewers can recalibrate.

Diagnostic check (input to the root cause, not a routing gate). While deriving the root cause, run these four questions over the occurrences — their answers sharpen the diagnosis and the proposed change; they no longer route anywhere (the implement run, not this audit, picks and applies the surface):

• Retrospective hallucination? Does the retrospective's summary contradict the primary-source evidence (PR/issue bodies, comments, reviews)? If so, the real fix may be in skills/retrospective/SKILL.md, not a downstream rule.
• Category vocabulary wrong? Did failures get forced into other, or into a category that doesn't fit, because the fixed categories vocabulary in retrospective/SKILL.md lacks the right bucket (or has one so broad it's useless)? If so, the fix may be that vocabulary (and possibly lib/compute-patterns.jq).
• Missing primary source? Did the retrospective miss context that would have changed the diagnosis (a referenced PR, a CI log, a doc, an issue-comment thread)? If so, the fix may be in fetch-pr-context.sh.
• Threshold mis-tuned? Are useful patterns suppressed by cooldown_days / min_occurrences, or surfaced too aggressively? If so, the fix may be in .devflow/config.json.

Any of these may legitimately be the highest-leverage proposed change — the issue you file can target the engine's own files, because a human reviews and implements it through the normal pipeline.

---

§ 2 — Pick the proposed change

Read "${CLAUDE_SKILL_DIR:-<absolute skill base directory this runner reports in context>}"/../../lib/intervention-surfaces.md. From those surfaces — or beyond them — pick the highest-leverage, smallest-blast-radius single concrete change to propose. The proposal must be one change, not a set of bullet points. Any surface is fair game (skills, agents, lib/, scripts/, docs, CLAUDE.md, config, application code) — you are writing a spec for a human-reviewed implement run, so nothing is off-limits the way it was when this stage auto-edited. When the re-derived root cause is a drift, desync, or coupled-mirror class — a fact that must be kept identical across multiple sites and drifted — prefer the proposal that collapses those sites to a single canonical source over one that adds a new pin plus a mirror copy: single-sourcing removes the drift's cause, whereas a fresh pin+mirror grows the very apparatus that produced it.

Conflict check: search the existing rules, skills, and docs for anything that contradicts your proposed change. If you find a conflict, reframe as "strengthen rule X" rather than "add rule Y" — that is always the higher-quality proposal. Document the conflict (or its explicit absence) in the issue body.

---

§ 3 — Counterfactual analysis

Write a short paragraph (3–5 sentences): what could go wrong if this change is applied too broadly? Enumerate the false-positive cases or edge cases where the existing behavior is actually correct. State explicitly how you scoped the proposal to avoid those pitfalls.

---

§ 4 — Author the issue body

The body you return is filed verbatim as the GitHub issue, so it must read like a /devflow:create-issue-quality issue plus a clearly delimited provenance section. Follow "${CLAUDE_SKILL_DIR:-<absolute skill base directory this runner reports in context>}"/../create-issue/references/issue-template.md for the issue structure, and append the provenance block.

GitHub autolink hygiene (your returned title and body are posted verbatim to a GitHub issue): never put a bare # immediately before a number unless it is a real issue or PR reference — GitHub renders #2 as a link to issue/PR 2, which misleads readers. For an ordinal, count, or list position, spell it out ("item 2", "step 3"), never #2. Genuine references like #123 stay as-is.

Body structure (sections in this order):

## Problem Statement
<who hits what pain — derived from the root cause and the occurrences>

## Current Behavior
<what the engine does today that lets this pattern recur>

## Desired Behavior
<the single decided behavior after the proposed change ships, stated declaratively>

## User Impact
<who benefits and how>

## Technical Context
> **Scope note:** The files and details below are the known starting points, not the full
> list. Before implementing, trace the change through the codebase to find every affected
> call site, consumer, and layer — this issue maps the work, it does not bound it.

- **Relevant Classes/Files** — <the surface(s) the proposed change touches>
- **Architecture Alignment** — <how it fits existing patterns>
- **Cross-layer Impact** — <layers affected>

## Acceptance Criteria
- [ ] <single unconditional, testable assertion>
- [ ] …

## Implementation Notes
- **Approach** — <the one proposed change, file by file>
- **Code Patterns** — <patterns in this repo to mirror>
- **Potential Gotchas** — <constraints / false-positive edges from § 3>

---

## 🔁 Retrospective provenance
- **Pattern:** `<tag>` · first seen <first_seen> · last seen <last_seen> · <occurrence_count> occurrences · status: <status>
- **Motivating PRs:** <links to every occurrence PR>
- **Root cause (re-derived from primary sources):** <your § 1 paragraph; flag any divergences from the retrospective summaries>
- **Counterfactual:** <your § 3 paragraph>
- **Sub-patterns not addressed:** <other sub-patterns under this category this issue leaves for a future run, or "none">

The Technical Context scope note is verbatim, fixed boilerplate — include it exactly as shown. Observe the template's no-options discipline in the issue sections (Problem → Implementation Notes): no choice / hedge / deferral language — the proposed change is a resolved decision. The ## 🔁 Retrospective provenance block is the clearly-delimited provenance section; keep it after the issue sections, separated by the --- rule.

---

§ 5 — Return contract

Print exactly one JSON object to stdout and stop:

{title, body}

• title — a clear, action-oriented issue title scoped to the one proposed change (the orchestrator prefixes it with the de-dup key, so do not add one yourself).
• body — the issue body authored in § 4.

There is no excluded field, no targets[], no PR. You return a spec; you do not edit.

---

§ 6 — Construct the JSON with jq -n

Never hand-write or heredoc the output JSON — character-escaping errors in multi-line issue bodies are the most common breakage. Write the body to a unique scratch file first (plain Write tool call) — the orchestrator dispatches every pattern's Stage B subagent concurrently, so a fixed shared path like .devflow/tmp/issue-body.md would let two subagents clobber each other; use a $(mktemp) path or one that embeds your pattern's slug (e.g. .devflow/tmp/issue-body-<slug>.md). Then build the object:

BODY_SCRATCH="$(mktemp)"   # unique per subagent — never a fixed shared path
# ... write the issue body to "$BODY_SCRATCH" with the Write tool ...
"${CLAUDE_SKILL_DIR:-<absolute skill base directory this runner reports in context>}"/../../scripts/run-jq.sh -n \
  --arg title "<action-oriented issue title>" \
  --arg body "$(cat "$BODY_SCRATCH")" \
  '{title: $title, body: $body}'

This scratch file is only a within-subagent buffer; the body travels back to the orchestrator via the stdout {title, body} JSON, which the orchestrator re-extracts to its own slug-suffixed file. Print the jq output and stop.