blueprint-derive-tests

/blueprint:derive-tests

Analyze git history to identify fix and feature commits lacking corresponding test changes, then generate a structured Test Regression Plan (TRP) document as a prioritized test backlog.

Use case: Systematically close test coverage gaps by mining commit history for bug fixes and features that shipped without regression tests.

When to Use This Skill

Use this skill when...	Use alternative when...
Bug fixes ship without regression tests	You need to run existing tests (/test:run)
Want a prioritized test backlog from history	Writing tests for a specific feature (manual TDD)
Onboarding a project and assessing test health	Checking current test coverage metrics
Need to find which fixes lack test coverage	Designing a test strategy from scratch (/test:architecture)

Context

• Git repository: !git rev-parse --git-dir
• Blueprint initialized: !find docs/blueprint -maxdepth 1 -name 'manifest.json' -type f
• Total commits: !git rev-list --count HEAD
• Test framework: !find . -maxdepth 3 \( -name 'vitest.config.*' -o -name 'jest.config.*' -o -name 'pytest.ini' -o -name 'pyproject.toml' -o -name 'Cargo.toml' -o -name 'go.mod' \) -type f -print -quit
• Test files: !find . -maxdepth 4 -type f \( -name '*.test.*' -o -name '*.spec.*' -o -name 'test_*' -o -name '*_test.*' \) -print
• Conventional commits sample: !git log --format="%s" --max-count=10

Parameters

Parse these from $ARGUMENTS:

• --quick: Fast scan (last 50 commits only)
• --since DATE: Analyze commits from specific date (e.g., --since 2024-06-01)
• --scope AREA: Filter to commits touching a specific area/scope (e.g., --scope auth)

Default behavior without flags: Analyze last 200 commits.

For detailed templates, severity matrix, and test mapping rules, see REFERENCE.md.

Execution

Execute this test regression plan derivation workflow:

Step 1: Verify prerequisites

Check context values above:

1. If git repository is empty → Error: "This directory is not a git repository. Run from project root."
2. If total commits = "0" → Error: "Repository has no commit history."
3. If Blueprint initialized is empty → Ask user: "Blueprint not initialized. Initialize now (Recommended) or continue without manifest tracking?"
   • If "Initialize now" → Use Task tool to invoke /blueprint:init, then continue
   • If "Continue without" → Skip manifest updates in Step 7

Step 2: Determine analysis scope

Parse $ARGUMENTS for --quick, --since, and --scope:

1. If --quick → scope = last 50 commits
2. If --since DATE → scope = commits from DATE to now
3. If --scope AREA → filter commits to those with scope matching AREA or touching files in AREA directory
4. Otherwise → scope = last 200 commits

Store scope parameters for git log commands in subsequent steps.

Step 3: Detect test infrastructure

Scan for test framework and conventions:

1. Identify test framework from context (vitest, jest, pytest, cargo test, go test)
2. Detect test file naming convention:
   • *.test.ts, *.spec.ts (JS/TS)
   • test_*.py, *_test.py (Python)
   • *_test.rs, tests/ directory (Rust)
   • *_test.go (Go)
3. Map source directories to test directories (e.g., src/ → tests/, src/ → src/__tests__/)
4. Record framework, naming pattern, and directory mapping for Step 5

If no test framework detected → Warn user, continue with file-based detection only.

Step 4: Extract and classify commits

Extract fix and feature commits within scope:

1. Primary targets — fix: commits (highest priority for regression tests):

   git log --format="%H %s" {scope} | grep -E "^[a-f0-9]+ fix(\(.*\))?:"
   

2. Secondary targets — feat: commits (should have accompanying tests):

   git log --format="%H %s" {scope} | grep -E "^[a-f0-9]+ feat(\(.*\))?:"
   

3. Fallback — If conventional commit percentage < 20%, use keyword detection:

   git log --format="%H %s" {scope} | grep -iE "(fix|bug|hotfix|patch|resolve|correct)"
   

For each commit, record: SHA, subject, date, files changed, scope (if conventional).

Step 5: Analyze test coverage gaps

For each commit from Step 4, check for corresponding tests:

1. Inline test changes — Did the same commit modify test files?

   git diff-tree --no-commit-id --name-only -r {SHA} | grep -E "(test|spec|_test\.|\.test\.)"
   

2. Nearby test commits — Within 5 commits after the fix, was a test commit added?

   git log --format="%H %s" {SHA}..{SHA~5} | grep -iE "^[a-f0-9]+ test(\(.*\))?:|add.*test|test.*for"
   

3. Test file exists — For each modified source file, does a corresponding test file exist? Use the source-to-test mapping from Step 3 (see REFERENCE.md for rules per language).

Classify each gap using the severity matrix from REFERENCE.md:

Severity	Criteria
Critical	fix: commit, no test changes, no test file exists for modified source
High	fix: commit, no inline test changes but test file exists (test not updated)
Medium	feat: commit, no test changes, core module affected
Low	feat: commit, no inline tests but nearby test commit exists

Step 6: Generate TRP document

1. Create output directory: mkdir -p docs/trps
2. Determine TRP ID:
   • If manifest exists, read id_registry.last_trp, increment by 1
   • Otherwise start at TRP-001
3. Generate slug from scope or date range (e.g., regression-gaps-2024-q3)
4. Write TRP document to docs/trps/{slug}.md using template from REFERENCE.md

Include in the document:

• YAML frontmatter with id, status: Active, scope, date_range, commits_analyzed
• Executive summary with gap counts by severity
• Detailed gap table: commit SHA, subject, severity, affected files, suggested test type
• Recommended test creation order (Critical first, then High, etc.)
• Suggested test type per gap (see REFERENCE.md)

Step 7: Update manifest

If Blueprint is initialized:

1. Update id_registry.last_trp with the new TRP number
2. Register the document in id_registry.documents:

   {
     "TRP-NNN": {
       "path": "docs/trps/{slug}.md",
       "title": "{TRP title}",
       "status": "Active",
       "created": "{date}"
     }
   }
   

3. Update task registry:

   jq --arg now "$(date -u +%Y-%m-%dT%H:%M:%SZ)" \
     --arg sha "$(git rev-parse HEAD 2>/dev/null)" \
     --argjson analyzed "{commits_analyzed}" \
     --argjson gaps "{gaps_found}" \
     '.task_registry["derive-tests"].last_completed_at = $now |
      .task_registry["derive-tests"].last_result = "success" |
      .task_registry["derive-tests"].stats.runs_total = ((.task_registry["derive-tests"].stats.runs_total // 0) + 1) |
      .task_registry["derive-tests"].stats.items_processed = $analyzed |
      .task_registry["derive-tests"].stats.items_created = $gaps |
      .task_registry["derive-tests"].context.commits_analyzed_up_to = $sha' \
     docs/blueprint/manifest.json > tmp.json && mv tmp.json docs/blueprint/manifest.json
   

Step 8: Report results and suggest next actions

Print summary:

Test Regression Plan Generated!

**Analysis Summary**
- Commits analyzed: {N} ({date_range})
- Fix commits found: {N}
- Feature commits found: {N}

**Coverage Gaps Found**
- Critical: {N} (fix commits with no tests at all)
- High: {N} (fix commits with stale test files)
- Medium: {N} (feature commits missing tests)
- Low: {N} (feature commits with nearby tests)

**Document**: docs/trps/{slug}.md (TRP-{NNN})

**Top Priority Gaps**
1. {commit subject} — {severity} — {affected file}
2. {commit subject} — {severity} — {affected file}
3. {commit subject} — {severity} — {affected file}

Prompt user for next action:

• "Create PRPs for top-priority gaps (Recommended)" — Generate PRP documents for Critical/High gaps
• "Review the TRP document" — Open the generated TRP for manual review
• "Run again with different scope" — Re-run with --since or --scope
• "Done for now" — Exit with document saved

Agentic Optimizations

Context	Command
Fix commits only	git log --format="%H %s" | grep -E "^[a-f0-9]+ fix"
Check test in commit	git diff-tree --no-commit-id --name-only -r {SHA} | grep -E "test|spec"
Files changed	git diff-tree --no-commit-id --name-only -r {SHA}
Fast scan	Use --quick for last 50 commits
Scope filter	Use --scope auth to limit to specific area

---

For detailed templates, severity classification matrix, test mapping rules, and error handling, see REFERENCE.md.