ADD Verify Skill v0.5.0

ADD Verify Skill v0.5.0

Execute quality gates to verify code meets production standards. This skill runs automated checks and produces a structured pass/fail report.

Overview

The Verify skill is the final checkpoint before deployment. It runs a sequence of quality gates determined by context and provides clear go/no-go status for each gate. Output is a structured report with pass/fail status and next steps.

The five-gate system ensures code quality at every stage:

1. Gate 1 (Local): Lint & code formatting
2. Gate 2 (Local): Type checking (if applicable)
3. Gate 3 (CI): Unit tests & coverage
4. Gate 4 (Deploy): Spec compliance & integration tests
5. Gate 5 (Smoke): Post-deploy health checks

Pre-Flight Checks

1. Read .add/config.json

   • Load gate definitions: which commands to run, thresholds
   • Load ci.gates array (ordered list of gates)
   • Load test.minCoverage threshold (default 80%)
   • Load code.lint configuration
   • Load code.types configuration
   • Load environment tier settings

2. Count active rules for maturity level

   • Read maturity level from .add/config.json (default: alpha)
   • Scan all files in rules/ directory
   • For each rule file, read the YAML frontmatter maturity: field
   • Count rules where the maturity field is at or below the current level
     • Maturity hierarchy: poc < alpha < beta < ga
     • A rule with maturity: poc is active at all levels
     • A rule with maturity: beta is active at beta and ga only
   • Include in report header: "{N} rules active at {maturity} level"

3. Determine execution level

   • Use --level flag or infer from environment
   • If not specified, default to 'local'
   • Levels control which gates run:
     • local: Gates 1-2 (lint, types) for development
     • ci: Gates 1-3 (+ unit tests) for continuous integration
     • deploy: Gates 1-4 (+ spec compliance) for production deploy
     • smoke: Gate 5 only (post-deploy health check)

4. Verify required files exist

   • Test files exist (unless smoke level)
   • Implementation files exist
   • Config files exist (package.json, tsconfig.json, etc.)
   • CI/deployment scripts available

5. Check environment

   • Verify tools are installed (eslint, tsc, jest, pytest, etc.)
   • Check Node/Python version if applicable
   • Verify dependencies are installed

6. Check for session handoff

   • Read .add/handoff.md if it exists
   • Note any in-progress work or decisions relevant to this operation
   • If handoff mentions blockers for this skill's scope, warn before proceeding

Execution Steps

Gate 1: Lint & Code Formatting

Purpose: Ensure code follows style conventions and has no obvious issues.

Steps:

1. Detect linter(s) from config or package.json

   • JavaScript/TypeScript: eslint, prettier, biome
   • Python: black, flake8, pylint
   • Go: gofmt, golangci-lint
   • etc.

2. Run linter(s)

   # JavaScript
   npx eslint --max-warnings 0 src/ tests/
   
   # Python
   python -m flake8 src/ tests/ --max-line-length 100
   
   # Go
   gofmt -w ./...
   golangci-lint run ./...
   

3. Capture output

   • Count errors and warnings
   • List problematic files
   • Save for report

4. Handle --fix flag

   • If --fix provided, auto-fix formatting issues
   • Re-run linter to verify fixes
   • Report what was fixed

5. Pass/Fail

   • PASS: 0 errors, 0 warnings (or configured threshold)
   • FAIL: Any errors or warnings
   • WARN: Warnings only (configurable as pass/fail)

Report:

Gate 1: Lint & Formatting
- Errors: 0
- Warnings: 0
- Status: ✓ PASS

Files checked: 12
Issues fixed (--fix): 0

Gate 2: Type Checking

Purpose: Catch type errors and ensure type safety.

Steps:

1. Detect type checker(s)

   • TypeScript: tsc --noEmit
   • Flow: flow
   • MyPy (Python): mypy src/
   • etc.

2. Run type checker

   # TypeScript
   npx tsc --noEmit --strict
   
   # Python with MyPy
   python -m mypy src/ --strict
   

3. Capture output

   • Count type errors
   • List files with issues
   • Report error details

4. Pass/Fail

   • PASS: 0 type errors
   • FAIL: Any type errors
   • SKIP: Not applicable for untyped languages

Report:

Gate 2: Type Checking
- Errors: 0
- Status: ✓ PASS

Type checker: TypeScript (--strict)
Files checked: 12

Gate 3: Unit Tests & Coverage

Purpose: Verify all tests pass and code coverage meets threshold.

Steps:

1. Run test suite

   # JavaScript
   npm test -- --coverage --silent
   
   # Python
   python -m pytest --cov=src --cov-report=term-summary tests/
   

2. Parse test output

   • Count tests run
   • Count tests passed
   • Count tests failed
   • Capture timing

3. Parse coverage output

   • Extract line coverage %
   • Extract branch coverage %
   • Extract function coverage %
   • Compare to minCoverage threshold

4. Identify coverage gaps (if below threshold)

   • Which files are below threshold?
   • Which lines are uncovered?
   • Which branches are untested?

5. Pass/Fail criteria

   • PASS: All tests pass AND coverage >= threshold
   • FAIL: Any test fails OR coverage < threshold
   • WARN: Tests pass but coverage marginally below (90% of threshold)

Report:

Gate 3: Unit Tests & Coverage
- Tests Run: 32
- Tests Passed: 32
- Tests Failed: 0
- Duration: 2.3s
- Line Coverage: 87% (threshold: 80%) ✓
- Branch Coverage: 82% (threshold: 80%) ✓
- Status: ✓ PASS

Coverage gaps (< 80%):
  - src/utils.ts: 73%
  - src/api.ts: 79%

Gate 4: Spec Compliance & Integration Tests

Purpose: Verify implementation meets spec and integration points work.

Steps:

1. Read spec file (specs/{feature}.md)

   • Extract all acceptance criteria
   • Load test mapping (tests/{feature}-mapping.md)

2. Verify test coverage

   • Every AC has at least one test
   • Every AC test is in the "passing" list
   • Every UT is covered by tests

3. Run integration tests (if separate suite)

   npm test -- --testMatch="**/*.integration.test.ts"
   # or
   python -m pytest tests/integration/ -v
   

4. Spec compliance check

   • Parse test mapping
   • Verify mapping completeness
   • Cross-check with spec requirements

5. Pass/Fail

   • PASS: All ACs tested and passing, all integration tests pass
   • FAIL: AC missing test, test failing, or integration test failing
   • WARN: AC has only happy-path test (no edge cases)

Report:

Gate 4: Spec Compliance & Integration Tests
- Spec: Feature X v1.0
- Acceptance Criteria: 5 total
  - AC-001: ✓ Tested and passing
  - AC-002: ✓ Tested and passing
  - AC-003: ✓ Tested and passing
  - AC-004: ✓ Tested and passing
  - AC-005: ✓ Tested and passing
- Integration Tests: 8 total
  - 8 passed, 0 failed
- Status: ✓ PASS

Test mapping file: tests/feature-mapping.md
All requirements traced and verified.

Gate 5: Smoke Tests (Post-Deploy)

Purpose: Quick health check after deployment to catch obvious breakage.

Steps:

1. Identify smoke test script

   • From config: ci.smokeTestScript
   • Typical: npm run test:smoke or ./scripts/smoke-tests.sh

2. Run smoke tests against deployed environment

   ENVIRONMENT=staging npm run test:smoke
   # or
   ./scripts/smoke-tests.sh production
   

3. Capture output

   • Tests run and result
   • Any errors or timeouts
   • Performance baseline check

4. Pass/Fail

   • PASS: All smoke tests pass within timeout
   • FAIL: Any test fails or timeout exceeded
   • SKIP: Not running (smoke level only)

Report:

Gate 5: Smoke Tests (Post-Deploy)
- Environment: staging
- Smoke tests run: 6
- Passed: 6
- Failed: 0
- Duration: 15s
- Status: ✓ PASS

Endpoints verified:
  ✓ GET /api/health
  ✓ GET /api/version
  ✓ POST /api/submit (with test data)
  ✓ GET /api/status
  ✓ Database connection
  ✓ Cache layer

Maturity-Scaled Checks (All Gates)

After each gate's core checks, run maturity-scaled checks from the quality-gates rule. These checks progressively tighten with project maturity.

Execution Pattern (repeat for each gate):

1. Read maturity level from .add/config.json (default: alpha)
2. Load maturity-scaled checklist from rules/quality-gates.md → Maturity-Scaled Checks section
3. Filter checks for this gate using the Gate Distribution mapping:
   • Gate 1: Code quality (complexity, duplication, file/function length), secrets scan, readability (naming, nesting), branch naming
   • Gate 2: Dependency audit, OWASP review, docstrings on exports, N+1/blocking async detection, CHANGELOG/LICENSE
   • Gate 3: Bundle size, PR template, README completeness, dependency freshness
   • Gate 4: Auth pattern review, PII/data handling, response time baselines, stale branch cleanup
   • Gate 5: Response times vs baselines, secure headers verification
4. Execute checks using agent tools:
   • Use Grep to scan for patterns (secrets, magic numbers, N+1 queries, blocking async calls)
   • Use Read to inspect files (function length, nesting depth, docstrings, README content)
   • Use Bash to run tools (dependency audit, bundle size, lint with complexity rules)
   • Use Glob to find files (LICENSE, CHANGELOG, PR template, .gitignore, stale branches)
5. Classify findings as blocking or advisory per maturity level
   • Load override thresholds from .add/config.json qualityChecks if present
   • Apply enforcement level: blocking findings fail the gate, advisory findings are warnings
6. Include in gate report — append maturity-scaled results to the gate's report section

Report Section (added to each gate's output):

Maturity-Scaled Checks ({maturity level}):
  Code Quality: ✓ PASS (complexity max: 12, threshold: 15)
  Security: ✓ PASS (no secrets, OWASP clean)
  Readability: ⚠ ADVISORY (2 functions missing docstrings)
  Performance: ⊘ SKIPPED (not checked at alpha)
  Repo Hygiene: ✓ PASS (branch naming ok, .gitignore exists)

Advisory findings (non-blocking):
  - src/utils.ts:45 — function missing docstring on export
  - src/api.ts:12 — function missing docstring on export

Maturity-Scaled Summary (Overall Report)

Add this section to the overall verification report, after the gate summary table:

## Maturity-Scaled Checks Summary
Maturity Level: {level} (from .add/config.json)

| Category | Status | Blocking | Advisory | Details |
|----------|--------|----------|----------|---------|
| Code Quality | ✓ PASS | 0 | 0 | All metrics within thresholds |
| Security | ✓ PASS | 0 | 2 | OWASP spot-check: 2 minor findings |
| Readability | ⚠ WARN | 0 | 3 | 3 exports missing docstrings |
| Performance | ⊘ SKIP | — | — | Not checked at alpha maturity |
| Repo Hygiene | ✓ PASS | 0 | 0 | All hygiene checks pass |

Advisory Findings (non-blocking):
1. [Security] src/api.ts:34 — input not sanitized before template literal
2. [Security] src/auth.ts:89 — password comparison not constant-time
3. [Readability] src/utils.ts:45 — exported function missing docstring
4. [Readability] src/api.ts:12 — exported function missing docstring
5. [Readability] src/form.ts:78 — magic number 86400 (use named constant)

Execution by Level

Level: local (development)

Runs Gates 1-2:

1. Lint & formatting
2. Type checking

Typical use: Before committing code locally

Level: ci (continuous integration)

Runs Gates 1-3:

1. Lint & formatting
2. Type checking
3. Unit tests & coverage

Typical use: CI/CD pipeline on every push

Level: deploy (production deploy)

Runs Gates 1-4:

1. Lint & formatting
2. Type checking
3. Unit tests & coverage
4. Spec compliance & integration tests

Typical use: Before deploying to production

Level: smoke (post-deploy)

Runs Gate 5 only:

1. Smoke tests

Typical use: After deployment to verify health

Overall Report Format

Generate a comprehensive verification report:

# Quality Gates Verification Report

## Execution Context
- Level: {level}
- Timestamp: {ISO timestamp}
- Feature: {feature-name}
- Branch: {git branch}
- Active Rules: {N} at {maturity} level

## Summary
Overall Status: ✓ ALL GATES PASSED [or ✗ GATES FAILED]

| Gate | Name | Status | Details |
|------|------|--------|---------|
| 1 | Lint & Formatting | ✓ PASS | 0 errors, 0 warnings |
| 2 | Type Checking | ✓ PASS | 0 type errors |
| 3 | Tests & Coverage | ✓ PASS | 32/32 tests, 87% coverage |
| 4 | Spec Compliance | ✓ PASS | 5/5 ACs tested |
| 5 | Smoke Tests | ⊘ SKIPPED | Not applicable at this level |

## Gate 1: Lint & Formatting
- Linter: eslint
- Status: ✓ PASS
- Errors: 0
- Warnings: 0
- Files checked: 12

## Gate 2: Type Checking
- Type checker: TypeScript (strict mode)
- Status: ✓ PASS
- Type errors: 0

## Gate 3: Unit Tests & Coverage
- Status: ✓ PASS
- Tests run: 32
- Passed: 32
- Failed: 0
- Duration: 2.3s
- Coverage:
  - Line: 87% (target: 80%) ✓
  - Branch: 82% (target: 80%) ✓
  - Function: 100% ✓

## Gate 4: Spec Compliance & Integration Tests
- Status: ✓ PASS
- ACs tested: 5/5
- Integration tests: 8 passed, 0 failed
- Spec: Feature X v1.0 (fully compliant)

## Gate 5: Smoke Tests
- Status: ⊘ SKIPPED (not applicable at 'deploy' level)

---

## Recommendations

Ready to proceed: ✓ YES
- All gates passed
- Code meets quality standards
- Safe to merge and deploy

Next steps:
1. [If all gates pass] Run /add:deploy to commit and push
2. [If gates fail] Fix issues and re-run /add:verify

Detailed gate results:
- No critical issues
- No warnings
- Coverage healthy

---

## Configuration Used
- test.framework: jest
- test.minCoverage: 80%
- code.lint: eslint with airbnb config
- ci.gates: [lint, types, tests, spec-compliance]

Progress Tracking

Use TaskCreate and TaskUpdate to report progress through the CLI spinner. Create tasks at the start of each major phase and mark them completed as they finish.

Tasks to create:

Phase	Subject	activeForm
Pre-flight	Running pre-flight checks	Running pre-flight checks...
Gate 1	Lint and formatting	Checking lint and formatting...
Gate 2	Type checking	Running type checks...
Gate 3	Tests and coverage	Running tests and checking coverage...
Gate 4	Spec compliance	Verifying spec compliance...
Gate 5	Smoke tests	Running smoke tests...
Maturity	Maturity-scaled checks	Running maturity-scaled checks...
Report	Generating verification report	Generating verification report...

Mark each task in_progress when starting and completed when done. This gives the user real-time visibility into skill execution.

Error Handling

Gate fails

• Report which gate failed and why
• List specific errors or failures
• Provide remediation guidance
• Exit with non-zero code

Tools not installed

• Report missing tool (eslint, tsc, jest, etc.)
• Provide installation command
• Suggest: npm install or pip install

Coverage below threshold

• Report current vs. target
• Identify which files are below threshold
• Suggest: add tests or check coverage report

Tests timeout

• Report which test timed out
• Suggest running individual test
• Suggest: npm test -- --testNamePattern="test_name"

Environment issues

• Smoke tests fail due to unavailable environment
• Report connectivity issues
• Suggest checking deployment status

--fix Flag Behavior

When --fix is provided:

1. Gate 1 (Lint): Auto-fix formatting issues

   • Run eslint --fix or prettier
   • Re-run linter to verify fixes applied
   • Report what was fixed

2. Gate 2 (Types): Cannot auto-fix type errors

   • Report errors but don't halt
   • Suggest: review type errors manually

3. Gate 3 (Tests): Cannot auto-fix test failures

   • Report failures
   • Suggest: check coverage, add missing tests

4. Gate 4 (Spec): Cannot auto-fix spec mismatches

   • Report issues
   • Suggest: update tests or implementation

5. Gate 5 (Smoke): Cannot auto-fix smoke test failures

   • Report failures
   • Suggest: check deployment

Integration with Other Skills

• Used by /add:tdd-cycle during VERIFY phase
• Can be run standalone to check code quality
• Output informs /add:deploy decision
• Feeds back to /add:implementer or /add:reviewer if gates fail

Configuration in .add/config.json

{
  "test": {
    "framework": "jest",
    "minCoverage": 80,
    "convention": "test_*.test.ts",
    "integrationConvention": "*.integration.test.ts"
  },
  "code": {
    "lint": "eslint",
    "types": "tsc --strict",
    "style": "prettier"
  },
  "ci": {
    "gates": ["lint", "types", "unit-tests", "spec-compliance", "integration-tests"],
    "smokeTestScript": "npm run test:smoke"
  }
}

Process Observation

After completing this skill, do BOTH:

1. Observation Line

Append one observation line to .add/observations.md:

{YYYY-MM-DD HH:MM} | verify | {one-line summary of outcome} | {cost or benefit estimate}

If .add/observations.md does not exist, create it with a # Process Observations header first.

2. Learning Checkpoint

Write a structured JSON learning entry per the checkpoint trigger in rules/learning.md (section: "After Verification"). Classify scope, write to the appropriate JSON file (.add/learnings.json or ~/.claude/add/library.json), and regenerate the markdown view.