Skill: Test Execution

Purpose

Execute test suites, analyze results, and report findings to ensure quality gates are met.

TDD Context

In our TDD workflow, 100% test pass rate and minimum 80% code coverage are mandatory at all quality gates.

Gate	Test Requirement	Coverage Requirement
Code Review	100% unit tests passing	Minimum 80% code coverage
QA Gate	100% integration + E2E tests passing	Minimum 80% code coverage
Deployment Gate	100% full test suite passing	Minimum 80% code coverage

No exceptions to 100% pass rate or 80% coverage threshold. If tests fail or coverage is below threshold, deployment is blocked.

When to Use

• TDD Red Phase: Confirming tests fail before implementation
• TDD Green Phase: Confirming tests pass after implementation
• Quality Gates: Verifying 100% pass rate and 80% coverage at code review, QA, and deployment
• Running tests for a PR/branch
• Validating story completion
• Regression testing before release
• Investigating test failures
• Verifying code coverage meets minimum threshold

---

Execution Types

Test Run Types

Type	When	Scope
Smoke	After deployment	Critical paths only
Regression	Before release	Full suite
Feature	PR/branch	Changed code + related
Performance	Release candidate	Load/stress tests
Security	Periodic/release	Vulnerability scans

---

Test Execution Commands

Running Tests

# All tests
npm test

# Specific test file
npm test -- path/to/test.ts

# Tests matching pattern
npm test -- --grep "UserService"

# With coverage
npm test -- --coverage

# Watch mode (development)
npm test -- --watch

# CI mode (no interaction)
npm test -- --ci --reporters=default --reporters=jest-junit

# Parallel execution
npm test -- --maxWorkers=4

# Only changed files
npm test -- --changedSince=main

Coverage Report

# Generate coverage report
npm test -- --coverage --coverageReporters=text --coverageReporters=html

# Coverage with thresholds (mandatory 80% minimum)
npm test -- --coverage --coverageThreshold='{
  "global": {
    "branches": 70,
    "functions": 80,
    "lines": 80,
    "statements": 80
  }
}'

Note: Minimum 80% code coverage is mandatory at all gates. Coverage below this threshold will block deployment.

---

Result Analysis

Result Interpretation

## Test Execution Summary

**Date:** {{date}}
**Branch:** {{branch}}
**Commit:** {{commitHash}}
**Environment:** {{environment}}

### Results

| Metric | Value | Status |
|--------|-------|--------|
| Total Tests | {{total}} | - |
| Passed | {{passed}} | ✅ |
| Failed | {{failed}} | {{#if failed}}🔴{{else}}✅{{/if}} |
| Skipped | {{skipped}} | {{#if skipped}}⚠️{{else}}✅{{/if}} |
| Duration | {{duration}} | - |

### Coverage

| Type | Actual | Target | Status |
|------|--------|--------|--------|
| Lines | {{lineCov}}% | 80% | {{status}} |
| Branches | {{branchCov}}% | 70% | {{status}} |
| Functions | {{funcCov}}% | 80% | {{status}} |
| Statements | {{stmtCov}}% | 80% | {{status}} |

Failure Analysis Template

## Test Failure Analysis

### Failed Test: {{testName}}

**File:** `{{testFile}}:{{line}}`
**Failure Type:** Assertion | Timeout | Error | Setup

### Error Details

{{errorMessage}}

### Stack Trace

{{stackTrace}}

### Analysis
**Root Cause:** {{rootCause}}
**Classification:** Bug | Flaky | Environment | Test Issue

### Recommended Action
{{recommendedAction}}

### Related
- Story: {{storyKey}}
- PR: {{prNumber}}

---

Flaky Test Management

Identifying Flaky Tests

## Flaky Test Report

### Criteria
Tests failing intermittently (pass rate 50-99% over last 10 runs)

### Identified Flaky Tests

| Test | Pass Rate | Last Failure | Category |
|------|-----------|--------------|----------|
| {{testName}} | {{passRate}}% | {{lastFailure}} | {{category}} |

### Categories
- **Timing:** Race conditions, async issues
- **State:** Shared state, order-dependent
- **External:** Network, third-party services
- **Resource:** Memory, file system

Flaky Test Resolution

## Flaky Test Fix: {{testName}}

### Problem
{{problemDescription}}

### Root Cause
{{rootCause}}

### Fix Applied
{{fixDescription}}

### Verification
- [ ] Test passes 10 consecutive runs
- [ ] No impact on other tests
- [ ] Fix reviewed and approved

---

Test Reporting

PR Test Report

## Test Results for PR #{{prNumber}}

### Summary
{{#if allPassed}}
✅ All tests passed
{{else}}
🔴 {{failedCount}} test(s) failed
{{/if}}

### Changed Files Coverage
| File | Coverage | Change |
|------|----------|--------|
| {{file}} | {{coverage}}% | {{change}} |

### New Tests Added
{{#each newTests}}
- {{testName}}
{{/each}}

### Test Execution
- **Duration:** {{duration}}
- **Total:** {{total}}
- **Passed:** {{passed}}
- **Failed:** {{failed}}

{{#if failures}}
### Failures
{{#each failures}}
#### {{testName}}

{{errorMessage}}

{{/each}}
{{/if}}

Release Test Report

## Release {{version}} Test Report

**Date:** {{date}}
**Environment:** {{environment}}
**Tested By:** QA Engineer Agent

### Executive Summary
{{executiveSummary}}

### Test Execution Summary

| Suite | Total | Passed | Failed | Skipped | Duration |
|-------|-------|--------|--------|---------|----------|
| Unit | {{unit.total}} | {{unit.passed}} | {{unit.failed}} | {{unit.skipped}} | {{unit.duration}} |
| Integration | {{int.total}} | {{int.passed}} | {{int.failed}} | {{int.skipped}} | {{int.duration}} |
| E2E | {{e2e.total}} | {{e2e.passed}} | {{e2e.failed}} | {{e2e.skipped}} | {{e2e.duration}} |
| **Total** | **{{total.total}}** | **{{total.passed}}** | **{{total.failed}}** | **{{total.skipped}}** | **{{total.duration}}** |

### Coverage Summary

| Area | Lines | Branches | Functions |
|------|-------|----------|-----------|
| Overall | {{overall.lines}}% | {{overall.branches}}% | {{overall.functions}}% |
| New Code | {{new.lines}}% | {{new.branches}}% | {{new.functions}}% |

### Feature Testing

| Feature | Status | Notes |
|---------|--------|-------|
| {{feature}} | ✅/🔴/⚠️ | {{notes}} |

### Defects Found

| ID | Severity | Title | Status |
|----|----------|-------|--------|
| {{bugKey}} | {{severity}} | {{title}} | {{status}} |

### Recommendation
{{recommendation}}

### Sign-off
- [ ] All critical tests passed
- [ ] Coverage targets met
- [ ] No critical defects open
- [ ] Performance within targets

---

CI/CD Integration

GitHub Actions Example

name: Tests

on:
  pull_request:
  push:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest

    services:
      postgres:
        image: postgres:14
        env:
          POSTGRES_DB: test
        ports:
          - 5432:5432

    steps:
      - uses: actions/checkout@v3

      - name: Setup Node
        uses: actions/setup-node@v3
        with:
          node-version: '18'
          cache: 'npm'

      - name: Install dependencies
        run: npm ci

      - name: Run tests
        run: npm test -- --ci --coverage
        env:
          DATABASE_URL: postgres://postgres:postgres@localhost:5432/test

      - name: Upload coverage
        uses: codecov/codecov-action@v3
        with:
          files: ./coverage/lcov.info

      - name: Test Report
        uses: dorny/test-reporter@v1
        if: success() || failure()
        with:
          name: Test Results
          path: junit.xml
          reporter: jest-junit

Quality Gates

# Quality gate configuration (MANDATORY)
quality_gates:
  coverage:
    lines: 80          # Minimum 80% line coverage (mandatory)
    branches: 70       # Minimum 70% branch coverage
    functions: 80      # Minimum 80% function coverage
    minimum_overall: 80  # Overall minimum threshold

  tests:
    pass_rate: 100     # 100% pass rate (mandatory, no exceptions)
    max_duration: 600  # seconds

  blocking:
    - coverage_below_threshold  # Blocks if < 80% coverage
    - test_failures             # Blocks if any test fails
    - security_vulnerabilities

Note: Both 100% test pass rate AND minimum 80% code coverage are mandatory requirements at every gate. Neither can be waived.

---

Jira Integration

Log Test Results to Jira

// Log test execution to story
Atlassian:addCommentToJiraIssue({
  cloudId: "{{jira.cloudId}}",
  issueIdOrKey: "{{storyKey}}",
  commentBody: `🤖 **QA Engineer Agent** - Test Execution

**Status:** {{#if allPassed}}✅ All Tests Passed{{else}}🔴 Tests Failed{{/if}}

**Results:**
- Total: {{total}}
- Passed: {{passed}}
- Failed: {{failed}}

**Coverage:** {{coverage}}%

{{#if failures}}
**Failures:**
{{#each failures}}
- {{testName}}: {{error}}
{{/each}}
{{/if}}

**Next Steps:** {{nextSteps}}`
})

Create Bug for Test Failure

// Auto-create bug for test failure
Atlassian:createJiraIssue({
  cloudId: "{{jira.cloudId}}",
  projectKey: "{{jira.bugs.projectKey}}",
  issueTypeName: "Bug",
  summary: "Test Failure: {{testName}}",
  description: `## Test Failure

**Test:** {{testName}}
**File:** {{testFile}}
**Story:** {{storyKey}}

### Error
\`\`\`
{{errorMessage}}
\`\`\`

### Stack Trace
\`\`\`
{{stackTrace}}
\`\`\`

### Environment
- Branch: {{branch}}
- Commit: {{commit}}
- CI Run: {{ciRunUrl}}`,
  labels: ["test-failure", "automated"]
})

---

Used By Agents

Agent	Test Execution Use
QA Engineer	Primary - executes all test types, analyzes results
Senior Developer	Reviews failures, fixes issues
DevOps Engineer	CI/CD test pipeline management
Project Manager	Test status reporting

Best Practices

1. Automate execution - Tests should run without manual intervention
2. Fast feedback - Run relevant tests first
3. Clear reporting - Anyone should understand the results
4. Track trends - Monitor flakiness and duration over time
5. Block on failures - Don't merge failing tests
6. Preserve evidence - Keep logs and artifacts for debugging