gavel-analyze

Gavel Analyze

Post-run suite analysis. Framework-adaptive.

When to Use

• After running a full or partial test suite
• "Analyze results", "why did tests fail", "pass rate by area"

Failure Classification

Category	Signals	Action
env	Connection refused, timeout, DNS failure	Fix env, re-run
seed	404 on seeded data, empty grids, "not found"	Re-seed database
app bug	Assertion fails on correct test logic	File bug report
test bug	Wrong locator, stale auth, wrong config	Fix test
test-maintenance-drift	Clustered failures, same route/area, element-not-found or renamed-control pattern after recent deploy	gavel-impact → gavel-healer
app-regression	5xx responses, unhandled exceptions, stack traces in test output	gavel-bug (confirm + report)
flake	Intermittent failures, race conditions, retry-dependent results	gavel-flake (triage)
expected fail	Marked expected failure (test.fail, @ExpectedFailure)	Document, link bug

Detecting test-maintenance-drift

Flag when most of these are true:

• ≥3 failures in the same feature area, route, or spec directory
• Errors are locator timeouts, missing text/roles, or "not attached" — not 5xx/auth
• Pass rate was high recently; failures coincide with application merges
• Retries do not help (not flaky — consistently missing DOM)

Hand off to gavel-impact for commit correlation, then gavel-healer to update automation. Do not classify as app-bug without evidence the product regressed.

Output Template

## Suite Analysis -- [project] -- [date]

### Summary
| Metric | Count |
|--------|------:|
| Passed | |
| Failed (unexpected) | |
| Skipped | |
| Expected fail | |
| Duration | |

### Pass Rate by Area
| Area | Passed | Total | Rate |
|------|-------:|------:|-----:|

### Failures
| Test ID | File | Classification | Notes |
|---------|------|----------------|-------|

### Recommendations (ranked)
1. [Highest impact action]

Report Ingestion

When CI artifacts are available, parse them before manual triage:

# JUnit XML
node scripts/parsers/junit.js path/to/results.xml --json

# Allure results directory
node scripts/parsers/allure.js path/to/allure-results --json

# Auto-detect file or directory
node scripts/parsers/index.js path/to/report --json

# Cluster failures for drift vs env signals
node scripts/parsers/junit.js results.xml --json | node scripts/cluster-failures.js

# Playwright JSON report
node scripts/parsers/playwright.js playwright-report.json --json

# Cypress JSON report
node scripts/parsers/cypress.js cypress-results.json --json

# Full CI analysis with optional commit correlation (gavel-impact)
node scripts/analyze-ci.js path/to/report.json \
  --app-repo path/to/app-repo \
  --area-map fixtures/config/area-map.example.json \
  --commits 15 --json

# Playwright HTML report directory
node scripts/parsers/playwright-html.js playwright-report/ --json

# One-shot: HTML report dir → parse + Gavel Result envelope
node scripts/analyze-ci.js playwright-report/ --envelope --project MySuite

# JSON envelope (schema-versioned, machine-readable)
node scripts/analyze-ci.js path/to/report.json --json-envelope --project MySuite

# Result envelope markdown (paste into gavel-analyze response)
node scripts/analyze-ci.js path/to/report.json --envelope --project MySuite
node scripts/analyze-ci.js path/to/report.json --app-repo ../app --area-map ./area-map.json --envelope

analyze-ci.js --envelope renders the standard Gavel Result block with Lead Summary (one-line scan) and Worker Handoff (full table). Use --json-envelope for machine-readable output with schema version gavel-result-envelope/1.0.0.

analyze-ci.js JSON includes per-cluster classification, suspectCommits (when --app-repo is set), and nextAction. Commit correlation runs for every cluster (not just test-maintenance-drift). Use suspect commits as the starting point for manual gavel-impact validation — do not treat git keyword search as proof.

Use parser output to pre-fill Summary, Pass Rate by Area, and Failures tables. When a cluster shows locator-timeout in the same area (≥3 failures), classify as test-maintenance-drift and hand off to gavel-impact → gavel-healer.

Result Envelope

Analysis-only skill. Return templates/result-envelope.md when the report is complete.

• Status DONE — summary, clusters, and ranked recommendations present
• Status INCOMPLETE — raw failures listed but clustering or classification unfinished
• Omit Changes and Verification unless you also fixed tests
• Next Action must name gavel-impact, gavel-healer, gavel-env, or none