From sdlc
Tracks and analyzes regression metrics like rates, MTTD, MTTF, escape rate, trends, patterns, and health indicators from test results and history.
npx claudepluginhub jmagly/aiwg --plugin sdlcThis skill uses the workspace's default tool permissions.
Track and analyze regression statistics, trends, and health indicators.
Enforces C++ Core Guidelines for writing, reviewing, and refactoring modern C++ code (C++17+), promoting RAII, immutability, type safety, and idiomatic practices.
Provides patterns for shared UI in Compose Multiplatform across Android, iOS, Desktop, and Web: state management with ViewModels/StateFlow, navigation, theming, and performance.
Implements Playwright E2E testing patterns: Page Object Model, test organization, configuration, reporters, artifacts, and CI/CD integration for stable suites.
Track and analyze regression statistics, trends, and health indicators.
This skill provides regression analytics by:
When triggered, this skill:
Collects regression data:
Calculates key metrics:
Identifies patterns:
Analyzes trends:
Generates visualizations:
Produces actionable insights:
regression_rate:
description: Number of regressions per time period
formula: regressions_detected / time_period
units: regressions per sprint/week/release
targets:
excellent: "< 2 per sprint"
good: "2-5 per sprint"
acceptable: "5-10 per sprint"
poor: "> 10 per sprint"
calculation:
count: new regressions introduced
period: sprint, release, or month
exclude: known issues, flaky tests
mttd:
description: Average time from regression introduction to detection
formula: sum(detection_time) / regression_count
units: hours or days
targets:
excellent: "< 4 hours"
good: "< 24 hours"
acceptable: "< 7 days"
poor: "> 7 days"
calculation:
detection_time: commit_time_to_failure_report
includes: automated and manual detection
mttf:
description: Average time from detection to fix deployment
formula: sum(fix_time) / regression_count
units: hours or days
targets:
critical: "< 4 hours"
high: "< 24 hours"
medium: "< 7 days"
low: "< 30 days"
calculation:
fix_time: detection_to_fix_deployed
severity_weighted: true
escape_rate:
description: Percentage of regressions reaching production
formula: (production_regressions / total_regressions) * 100
units: percentage
targets:
excellent: "< 5%"
good: "5-10%"
acceptable: "10-20%"
poor: "> 20%"
calculation:
production_regressions: found by users/monitoring
total_regressions: all detected including pre-release
recurrence_rate:
description: Percentage of regressions that recur after fix
formula: (recurring_regressions / total_fixed) * 100
units: percentage
targets:
excellent: "< 5%"
good: "5-10%"
acceptable: "10-15%"
poor: "> 15%"
indicates:
- insufficient test coverage
- lack of regression tests
- poor fix quality
# Regression Metrics Dashboard
**Period**: Last 30 Days (2025-12-29 to 2026-01-28)
**Project**: User Service
## Executive Summary
| Metric | Current | Target | Status | Trend |
|--------|---------|--------|--------|-------|
| Regression Rate | 4.2/sprint | < 5 | ✅ Good | ↓ Improving |
| MTTD | 8.5 hours | < 24h | ✅ Good | ↓ Improving |
| MTTF | 18.7 hours | < 24h | ⚠️ Close | → Stable |
| Escape Rate | 12% | < 10% | ⚠️ Above Target | ↑ Worsening |
| Recurrence Rate | 7% | < 10% | ✅ Good | → Stable |
**Overall Health**: ⚠️ Good with Concerns
**Priority Focus**: Reduce production escapes
## Regression Trend (Last 6 Sprints)
Sprint 8: ██████████ 10 regressions Sprint 9: ████████ 8 regressions Sprint 10: ██████ 6 regressions Sprint 11: █████ 5 regressions Sprint 12: ████ 4 regressions Sprint 13: ████ 4 regressions ↓ -60% improvement since Sprint 8
**Analysis**: Significant improvement trend. Stabilizing around 4-5 per sprint.
## Detection Speed Trend
Week 1: 24h ████████████████████████ Week 2: 18h ██████████████████ Week 3: 12h ████████████ Week 4: 9h █████████ Week 5: 8h ████████ ↓ -67% improvement in 5 weeks
**Analysis**: Automation improvements paying off. Most regressions now caught within hours.
## Component Heatmap
Regressions by component (last 30 days):
| Component | Regressions | Change | Risk Level |
|-----------|-------------|--------|------------|
| src/auth/ | 🔴🔴🔴 3 | +1 | High |
| src/api/ | 🟡🟡 2 | 0 | Medium |
| src/db/ | 🟡🟡 2 | -1 | Medium |
| src/user/ | 🟡 1 | -2 | Low |
| src/utils/ | 🟢 0 | 0 | Low |
**Hotspot Alert**: `src/auth/` showing increased regression rate
## Root Cause Analysis
| Root Cause | Count | % | Trend |
|------------|-------|---|-------|
| Missing test coverage | 5 | 42% | → |
| Integration not tested | 3 | 25% | ↑ |
| Edge case not considered | 2 | 17% | ↓ |
| Flaky test masking issue | 1 | 8% | → |
| Breaking dependency change | 1 | 8% | → |
**Insight**: 67% of regressions preventable with better coverage/integration testing
## Severity Distribution
| Severity | Count | MTTF | Status |
|----------|-------|------|--------|
| Critical | 1 | 3.2h | ✅ Fast response |
| High | 4 | 12.5h | ✅ Within target |
| Medium | 6 | 28.4h | ⚠️ Above target |
| Low | 1 | 72h | ✅ Acceptable |
## Time-to-Detection Analysis
Detection Method: Automated Tests: 75% (avg 4.2h detection) Manual Testing: 17% (avg 32h detection) Production: 8% (avg 96h detection)
**Insight**: Automation catching most issues early. Need to reduce production escapes.
## Time-to-Fix Analysis
Fix Duration by Severity: Critical: ▓▓▓ 3.2h (target: 4h) ✅ High: ▓▓▓▓▓▓ 12.5h (target: 24h) ✅ Medium: ▓▓▓▓▓▓▓▓▓▓▓▓▓▓ 28.4h (target: 24h) ⚠️ Low: ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓ 72h ✅
**Issue**: Medium-severity regressions taking slightly longer than target
## Regression Recurrence
| Original Issue | Recurred | Reason |
|----------------|----------|--------|
| AUTH-101 | ✅ Yes | Missing regression test |
| API-205 | ❌ No | Regression test added |
| DB-089 | ❌ No | Regression test added |
| USER-145 | ❌ No | Regression test added |
**Recurrence Rate**: 25% (1 of 4) - One regression lacked test
## Production Escapes
Regressions that reached production:
| Issue | Severity | Detection | Impact | MTTD |
|-------|----------|-----------|--------|------|
| AUTH-203 | High | User report | 500 users | 12h |
**Analysis**: 1 escape this period. Auth module regression bypassed staging tests.
## Recommendations
### High Priority
1. **Add integration tests for auth flows**
- Reason: 3 regressions in auth, 1 production escape
- Impact: Reduce auth regressions by ~60%
- Effort: 2 days
2. **Improve staging test coverage**
- Reason: Production escape indicates gap
- Impact: Reduce escape rate to <5%
- Effort: 1 week
3. **Reduce medium-severity MTTF**
- Reason: 28.4h vs 24h target
- Impact: Faster user impact resolution
- Effort: Process improvement
### Medium Priority
4. **Add regression tests for all fixes**
- Reason: 25% recurrence rate on fixes without tests
- Impact: Zero recurrence for tested fixes
- Effort: Ongoing discipline
5. **Monitor auth module closely**
- Reason: Highest regression count
- Impact: Early detection of issues
- Effort: Weekly review
## Historical Comparison
| Period | Reg Rate | MTTD | MTTF | Escape % |
|--------|----------|------|------|----------|
| 3 months ago | 8.2 | 36h | 48h | 18% |
| 2 months ago | 6.5 | 24h | 36h | 15% |
| 1 month ago | 5.1 | 12h | 24h | 13% |
| Current | 4.2 | 8.5h | 18.7h | 12% |
**Trend**: All metrics improving. Regression rate down 49%, detection 76% faster.
## Goals for Next Period
| Metric | Current | Goal | Strategy |
|--------|---------|------|----------|
| Regression Rate | 4.2 | < 4 | Improve auth testing |
| MTTD | 8.5h | < 8h | Add more automation |
| MTTF | 18.7h | < 18h | Faster review process |
| Escape Rate | 12% | < 10% | Better staging tests |
## Data Sources
- Regression tests: `.aiwg/testing/regression-results/`
- Bisect reports: `.aiwg/testing/regression-bisect-*/`
- Baseline comparisons: `.aiwg/testing/baseline-comparisons/`
- Issue tracker: GitHub Issues (label: regression)
- CI/CD logs: GitHub Actions
User: "Show regression metrics"
Skill executes:
1. Aggregate data from last 30 days
2. Calculate key metrics
3. Generate dashboard
4. Identify trends
Output:
"Regression Metrics (Last 30 Days)
Overall Health: ⚠️ Good with Concerns
Key Metrics:
- Regression Rate: 4.2/sprint ✅ (target < 5)
- MTTD: 8.5 hours ✅ (target < 24h)
- MTTF: 18.7 hours ⚠️ (target < 24h)
- Escape Rate: 12% ⚠️ (target < 10%)
Hotspots:
🔴 src/auth/ - 3 regressions this period
🟡 src/api/ - 2 regressions
Top Recommendation: Add integration tests for auth
Full dashboard: .aiwg/testing/regression-metrics-dashboard.md"
User: "Regression trends over time"
Skill analyzes:
- Last 6 sprints of data
- Calculate trend direction
- Identify patterns
Output:
"Regression Trends (Last 6 Sprints)
Sprint 8: 10 regressions
Sprint 9: 8 regressions (-20%)
Sprint 10: 6 regressions (-25%)
Sprint 11: 5 regressions (-17%)
Sprint 12: 4 regressions (-20%)
Sprint 13: 4 regressions (stable)
Overall: ↓ -60% improvement
Status: Stabilizing around 4-5/sprint
MTTD: 36h → 8.5h (-76%)
MTTF: 48h → 18.7h (-61%)
Conclusion: Strong improvement trend. Approaching best-in-class levels."
User: "Which components have most regressions?"
Skill generates:
"Component Regression Heatmap (Last 30 Days)
High Risk:
🔴 src/auth/ - 3 regressions (+1 from last period)
Most common: Missing integration tests
Medium Risk:
🟡 src/api/ - 2 regressions (no change)
🟡 src/db/ - 2 regressions (-1 from last period)
Low Risk:
🟢 src/user/ - 1 regression (-2 from last period)
🟢 src/utils/ - 0 regressions
Recommendation: Focus testing efforts on auth module"
This skill uses:
regression-bisect: Import bisect findingsregression-baseline: Analyze baseline drift patternstest-coverage: Correlate coverage with regression ratesproject-awareness: Detect sprint/release boundariesagents:
analysis:
agent: metrics-analyst
focus: Statistical analysis and trends
visualization:
agent: technical-writer
focus: Dashboard and report generation
recommendations:
agent: test-architect
focus: Process improvement suggestions
collection_config:
data_sources:
- regression_test_results
- bisect_reports
- baseline_comparisons
- issue_tracker
- ci_cd_logs
update_frequency: daily
retention: 90 days
aggregation: sprint, week, month
thresholds:
regression_rate:
excellent: 2
good: 5
acceptable: 10
mttd_hours:
excellent: 4
good: 24
acceptable: 168 # 7 days
mttf_hours:
critical: 4
high: 24
medium: 168 # 7 days
escape_rate_percent:
excellent: 5
good: 10
acceptable: 20
alerts:
regression_spike:
condition: regression_rate > 10
severity: high
notification: team-channel
escape_rate_high:
condition: escape_rate > 20%
severity: critical
notification: leadership
mttd_degrading:
condition: mttd_trend_increase > 50%
severity: medium
notification: test-team
.aiwg/testing/regression-metrics-dashboard.md.aiwg/testing/regression-trends.json.aiwg/testing/regression-heatmap.json.aiwg/testing/metrics-history/