Track and analyze regression statistics, trends, and health indicators.
Analyzes regression test data to calculate metrics, identify trends, and generate health dashboards.
/plugin marketplace add jmagly/aiwg/plugin install sdlc@aiwgThis skill inherits all available tools. When active, it can use any tool Claude has access to.
Track and analyze regression statistics, trends, and health indicators.
This skill provides regression analytics by:
When triggered, this skill:
Collects regression data:
Calculates key metrics:
Identifies patterns:
Analyzes trends:
Generates visualizations:
Produces actionable insights:
regression_rate:
description: Number of regressions per time period
formula: regressions_detected / time_period
units: regressions per sprint/week/release
targets:
excellent: "< 2 per sprint"
good: "2-5 per sprint"
acceptable: "5-10 per sprint"
poor: "> 10 per sprint"
calculation:
count: new regressions introduced
period: sprint, release, or month
exclude: known issues, flaky tests
mttd:
description: Average time from regression introduction to detection
formula: sum(detection_time) / regression_count
units: hours or days
targets:
excellent: "< 4 hours"
good: "< 24 hours"
acceptable: "< 7 days"
poor: "> 7 days"
calculation:
detection_time: commit_time_to_failure_report
includes: automated and manual detection
mttf:
description: Average time from detection to fix deployment
formula: sum(fix_time) / regression_count
units: hours or days
targets:
critical: "< 4 hours"
high: "< 24 hours"
medium: "< 7 days"
low: "< 30 days"
calculation:
fix_time: detection_to_fix_deployed
severity_weighted: true
escape_rate:
description: Percentage of regressions reaching production
formula: (production_regressions / total_regressions) * 100
units: percentage
targets:
excellent: "< 5%"
good: "5-10%"
acceptable: "10-20%"
poor: "> 20%"
calculation:
production_regressions: found by users/monitoring
total_regressions: all detected including pre-release
recurrence_rate:
description: Percentage of regressions that recur after fix
formula: (recurring_regressions / total_fixed) * 100
units: percentage
targets:
excellent: "< 5%"
good: "5-10%"
acceptable: "10-15%"
poor: "> 15%"
indicates:
- insufficient test coverage
- lack of regression tests
- poor fix quality
# Regression Metrics Dashboard
**Period**: Last 30 Days (2025-12-29 to 2026-01-28)
**Project**: User Service
## Executive Summary
| Metric | Current | Target | Status | Trend |
|--------|---------|--------|--------|-------|
| Regression Rate | 4.2/sprint | < 5 | ✅ Good | ↓ Improving |
| MTTD | 8.5 hours | < 24h | ✅ Good | ↓ Improving |
| MTTF | 18.7 hours | < 24h | ⚠️ Close | → Stable |
| Escape Rate | 12% | < 10% | ⚠️ Above Target | ↑ Worsening |
| Recurrence Rate | 7% | < 10% | ✅ Good | → Stable |
**Overall Health**: ⚠️ Good with Concerns
**Priority Focus**: Reduce production escapes
## Regression Trend (Last 6 Sprints)
Sprint 8: ██████████ 10 regressions Sprint 9: ████████ 8 regressions Sprint 10: ██████ 6 regressions Sprint 11: █████ 5 regressions Sprint 12: ████ 4 regressions Sprint 13: ████ 4 regressions ↓ -60% improvement since Sprint 8
**Analysis**: Significant improvement trend. Stabilizing around 4-5 per sprint.
## Detection Speed Trend
Week 1: 24h ████████████████████████ Week 2: 18h ██████████████████ Week 3: 12h ████████████ Week 4: 9h █████████ Week 5: 8h ████████ ↓ -67% improvement in 5 weeks
**Analysis**: Automation improvements paying off. Most regressions now caught within hours.
## Component Heatmap
Regressions by component (last 30 days):
| Component | Regressions | Change | Risk Level |
|-----------|-------------|--------|------------|
| src/auth/ | 🔴🔴🔴 3 | +1 | High |
| src/api/ | 🟡🟡 2 | 0 | Medium |
| src/db/ | 🟡🟡 2 | -1 | Medium |
| src/user/ | 🟡 1 | -2 | Low |
| src/utils/ | 🟢 0 | 0 | Low |
**Hotspot Alert**: `src/auth/` showing increased regression rate
## Root Cause Analysis
| Root Cause | Count | % | Trend |
|------------|-------|---|-------|
| Missing test coverage | 5 | 42% | → |
| Integration not tested | 3 | 25% | ↑ |
| Edge case not considered | 2 | 17% | ↓ |
| Flaky test masking issue | 1 | 8% | → |
| Breaking dependency change | 1 | 8% | → |
**Insight**: 67% of regressions preventable with better coverage/integration testing
## Severity Distribution
| Severity | Count | MTTF | Status |
|----------|-------|------|--------|
| Critical | 1 | 3.2h | ✅ Fast response |
| High | 4 | 12.5h | ✅ Within target |
| Medium | 6 | 28.4h | ⚠️ Above target |
| Low | 1 | 72h | ✅ Acceptable |
## Time-to-Detection Analysis
Detection Method: Automated Tests: 75% (avg 4.2h detection) Manual Testing: 17% (avg 32h detection) Production: 8% (avg 96h detection)
**Insight**: Automation catching most issues early. Need to reduce production escapes.
## Time-to-Fix Analysis
Fix Duration by Severity: Critical: ▓▓▓ 3.2h (target: 4h) ✅ High: ▓▓▓▓▓▓ 12.5h (target: 24h) ✅ Medium: ▓▓▓▓▓▓▓▓▓▓▓▓▓▓ 28.4h (target: 24h) ⚠️ Low: ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓ 72h ✅
**Issue**: Medium-severity regressions taking slightly longer than target
## Regression Recurrence
| Original Issue | Recurred | Reason |
|----------------|----------|--------|
| AUTH-101 | ✅ Yes | Missing regression test |
| API-205 | ❌ No | Regression test added |
| DB-089 | ❌ No | Regression test added |
| USER-145 | ❌ No | Regression test added |
**Recurrence Rate**: 25% (1 of 4) - One regression lacked test
## Production Escapes
Regressions that reached production:
| Issue | Severity | Detection | Impact | MTTD |
|-------|----------|-----------|--------|------|
| AUTH-203 | High | User report | 500 users | 12h |
**Analysis**: 1 escape this period. Auth module regression bypassed staging tests.
## Recommendations
### High Priority
1. **Add integration tests for auth flows**
- Reason: 3 regressions in auth, 1 production escape
- Impact: Reduce auth regressions by ~60%
- Effort: 2 days
2. **Improve staging test coverage**
- Reason: Production escape indicates gap
- Impact: Reduce escape rate to <5%
- Effort: 1 week
3. **Reduce medium-severity MTTF**
- Reason: 28.4h vs 24h target
- Impact: Faster user impact resolution
- Effort: Process improvement
### Medium Priority
4. **Add regression tests for all fixes**
- Reason: 25% recurrence rate on fixes without tests
- Impact: Zero recurrence for tested fixes
- Effort: Ongoing discipline
5. **Monitor auth module closely**
- Reason: Highest regression count
- Impact: Early detection of issues
- Effort: Weekly review
## Historical Comparison
| Period | Reg Rate | MTTD | MTTF | Escape % |
|--------|----------|------|------|----------|
| 3 months ago | 8.2 | 36h | 48h | 18% |
| 2 months ago | 6.5 | 24h | 36h | 15% |
| 1 month ago | 5.1 | 12h | 24h | 13% |
| Current | 4.2 | 8.5h | 18.7h | 12% |
**Trend**: All metrics improving. Regression rate down 49%, detection 76% faster.
## Goals for Next Period
| Metric | Current | Goal | Strategy |
|--------|---------|------|----------|
| Regression Rate | 4.2 | < 4 | Improve auth testing |
| MTTD | 8.5h | < 8h | Add more automation |
| MTTF | 18.7h | < 18h | Faster review process |
| Escape Rate | 12% | < 10% | Better staging tests |
## Data Sources
- Regression tests: `.aiwg/testing/regression-results/`
- Bisect reports: `.aiwg/testing/regression-bisect-*/`
- Baseline comparisons: `.aiwg/testing/baseline-comparisons/`
- Issue tracker: GitHub Issues (label: regression)
- CI/CD logs: GitHub Actions
User: "Show regression metrics"
Skill executes:
1. Aggregate data from last 30 days
2. Calculate key metrics
3. Generate dashboard
4. Identify trends
Output:
"Regression Metrics (Last 30 Days)
Overall Health: ⚠️ Good with Concerns
Key Metrics:
- Regression Rate: 4.2/sprint ✅ (target < 5)
- MTTD: 8.5 hours ✅ (target < 24h)
- MTTF: 18.7 hours ⚠️ (target < 24h)
- Escape Rate: 12% ⚠️ (target < 10%)
Hotspots:
🔴 src/auth/ - 3 regressions this period
🟡 src/api/ - 2 regressions
Top Recommendation: Add integration tests for auth
Full dashboard: .aiwg/testing/regression-metrics-dashboard.md"
User: "Regression trends over time"
Skill analyzes:
- Last 6 sprints of data
- Calculate trend direction
- Identify patterns
Output:
"Regression Trends (Last 6 Sprints)
Sprint 8: 10 regressions
Sprint 9: 8 regressions (-20%)
Sprint 10: 6 regressions (-25%)
Sprint 11: 5 regressions (-17%)
Sprint 12: 4 regressions (-20%)
Sprint 13: 4 regressions (stable)
Overall: ↓ -60% improvement
Status: Stabilizing around 4-5/sprint
MTTD: 36h → 8.5h (-76%)
MTTF: 48h → 18.7h (-61%)
Conclusion: Strong improvement trend. Approaching best-in-class levels."
User: "Which components have most regressions?"
Skill generates:
"Component Regression Heatmap (Last 30 Days)
High Risk:
🔴 src/auth/ - 3 regressions (+1 from last period)
Most common: Missing integration tests
Medium Risk:
🟡 src/api/ - 2 regressions (no change)
🟡 src/db/ - 2 regressions (-1 from last period)
Low Risk:
🟢 src/user/ - 1 regression (-2 from last period)
🟢 src/utils/ - 0 regressions
Recommendation: Focus testing efforts on auth module"
This skill uses:
regression-bisect: Import bisect findingsregression-baseline: Analyze baseline drift patternstest-coverage: Correlate coverage with regression ratesproject-awareness: Detect sprint/release boundariesagents:
analysis:
agent: metrics-analyst
focus: Statistical analysis and trends
visualization:
agent: technical-writer
focus: Dashboard and report generation
recommendations:
agent: test-architect
focus: Process improvement suggestions
collection_config:
data_sources:
- regression_test_results
- bisect_reports
- baseline_comparisons
- issue_tracker
- ci_cd_logs
update_frequency: daily
retention: 90 days
aggregation: sprint, week, month
thresholds:
regression_rate:
excellent: 2
good: 5
acceptable: 10
mttd_hours:
excellent: 4
good: 24
acceptable: 168 # 7 days
mttf_hours:
critical: 4
high: 24
medium: 168 # 7 days
escape_rate_percent:
excellent: 5
good: 10
acceptable: 20
alerts:
regression_spike:
condition: regression_rate > 10
severity: high
notification: team-channel
escape_rate_high:
condition: escape_rate > 20%
severity: critical
notification: leadership
mttd_degrading:
condition: mttd_trend_increase > 50%
severity: medium
notification: test-team
.aiwg/testing/regression-metrics-dashboard.md.aiwg/testing/regression-trends.json.aiwg/testing/regression-heatmap.json.aiwg/testing/metrics-history/Activates when the user asks about AI prompts, needs prompt templates, wants to search for prompts, or mentions prompts.chat. Use for discovering, retrieving, and improving prompts.
Search, retrieve, and install Agent Skills from the prompts.chat registry using MCP tools. Use when the user asks to find skills, browse skill catalogs, install a skill for Claude, or extend Claude's capabilities with reusable AI agent components.
Creating algorithmic art using p5.js with seeded randomness and interactive parameter exploration. Use this when users request creating art using code, generative art, algorithmic art, flow fields, or particle systems. Create original algorithmic art rather than copying existing artists' work to avoid copyright violations.