Characterization Test Generator Skill

Generates characterization tests (also known as golden master tests or approval tests) to capture existing system behavior before migration, ensuring functional equivalence after changes.

Purpose

Enable behavior preservation during migration through:

Golden master test creation
Approval test generation
Edge case discovery
Input/output recording
Behavior snapshot capture
Regression baseline establishment

Capabilities

1. Golden Master Test Creation

Capture current system outputs for given inputs
Store outputs as reference "golden" files
Generate comparison test infrastructure
Support multiple output formats
Enable baseline versioning

2. Approval Test Generation

Create human-reviewable test outputs
Generate diff-based comparison tests
Support approval workflows
Enable incremental approval
Track approval history

3. Edge Case Discovery

Analyze code paths for boundary conditions
Generate boundary value inputs
Identify error handling scenarios
Map exception paths
Create comprehensive test coverage

4. Input/Output Recording

Instrument code for I/O capture
Record API request/response pairs
Capture database interactions
Log external service calls
Store timing and sequence data

5. Behavior Snapshot

Capture state at key points
Record side effects
Document non-deterministic behavior
Identify environmental dependencies
Map configuration impacts

6. Regression Baseline Establishment

Create baseline test suites
Define acceptance thresholds
Configure tolerance levels
Set up CI/CD integration
Generate coverage reports

Tool Integrations

This skill can leverage the following external tools when available:

Tool	Purpose	Integration Method
ApprovalTests	Approval testing framework	Library
Jest snapshots	JavaScript snapshot testing	Framework
pytest-snapshot	Python snapshot testing	Plugin
TextTest	Golden master testing	CLI
Verify	.NET approval testing	Library
Scientist	Safe refactoring library	Library
AI Testing MCP	AI-powered test generation	MCP Server

Usage

Basic Test Generation

# Invoke skill for characterization test generation
# The skill will analyze code and generate tests

# Expected inputs:
# - targetPath: Path to code to characterize
# - testFramework: 'jest' | 'pytest' | 'junit' | 'nunit' | 'auto'
# - outputDir: Directory for generated tests
# - captureMode: 'snapshot' | 'approval' | 'recording'

Generation Workflow

Analysis Phase
- Identify testable units (functions, methods, endpoints)
- Map input parameters and types
- Detect output formats
- Find external dependencies
Input Discovery Phase
- Extract existing test inputs
- Generate boundary values
- Identify representative cases
- Create combinatorial inputs
Capture Phase
- Execute code with inputs
- Record outputs and side effects
- Capture state changes
- Log external interactions
Test Generation Phase
- Generate test code structure
- Create golden master files
- Set up comparison logic
- Configure CI integration

Output Schema

{
  "generationId": "string",
  "timestamp": "ISO8601",
  "target": {
    "path": "string",
    "language": "string",
    "framework": "string",
    "unitsAnalyzed": "number"
  },
  "testsGenerated": {
    "total": "number",
    "byType": {
      "snapshot": "number",
      "approval": "number",
      "recording": "number"
    },
    "coverage": {
      "functions": "number",
      "branches": "number",
      "lines": "number"
    }
  },
  "characterizations": [
    {
      "unit": "string",
      "type": "function|method|endpoint|class",
      "inputs": [
        {
          "name": "string",
          "type": "string",
          "values": ["any"],
          "source": "existing|generated|boundary"
        }
      ],
      "outputs": {
        "type": "string",
        "goldenMasterPath": "string",
        "checksum": "string"
      },
      "testFile": "string",
      "baselineApproved": "boolean"
    }
  ],
  "edgeCases": [
    {
      "unit": "string",
      "case": "string",
      "input": "any",
      "expectedBehavior": "string",
      "covered": "boolean"
    }
  ],
  "dependencies": {
    "external": ["string"],
    "mocked": ["string"],
    "recorded": ["string"]
  },
  "artifacts": {
    "testSuite": "string",
    "goldenMasters": "string",
    "recordings": "string",
    "coverageReport": "string"
  }
}

Integration with Migration Processes

This skill integrates with the following Code Migration/Modernization processes:

migration-testing-strategy: Primary tool for test baseline creation
code-refactoring: Ensure behavior preservation during refactoring
framework-upgrade: Verify functionality after upgrade
monolith-to-microservices: Validate service extraction

Configuration

Skill Configuration File

Create .characterization-tests.json in the project root:

{
  "testFramework": "auto",
  "outputDir": "./tests/characterization",
  "captureMode": "snapshot",
  "goldenMasterDir": "./tests/golden-masters",
  "inputGeneration": {
    "boundary": true,
    "combinatorial": true,
    "maxCombinations": 100,
    "includeNulls": true,
    "includeEmpty": true
  },
  "outputCapture": {
    "format": "json",
    "normalizeWhitespace": true,
    "ignorePaths": ["$.timestamp", "$.requestId"],
    "tolerance": {
      "numeric": 0.001,
      "dateTime": "1s"
    }
  },
  "dependencies": {
    "mockExternal": true,
    "recordMode": false,
    "replayMode": true
  },
  "approval": {
    "requireApproval": true,
    "approvalDir": "./tests/approvals",
    "reportFormat": "markdown"
  },
  "ci": {
    "failOnNewTests": false,
    "updateGoldenOnPass": false,
    "generateCoverageReport": true
  }
}

MCP Server Integration

When AI Testing MCP Server is available:

// Example AI-powered test generation
{
  "tool": "ai_testing_generate",
  "arguments": {
    "target": "./src/services/user.ts",
    "framework": "jest",
    "style": "characterization"
  }
}

Test Patterns

Snapshot Testing (Jest)

// Generated characterization test
describe('UserService', () => {
  describe('calculateDiscount', () => {
    it('should match snapshot for standard customer', () => {
      const result = userService.calculateDiscount({
        customerId: 'C001',
        purchaseAmount: 100,
        loyaltyPoints: 500
      });
      expect(result).toMatchSnapshot();
    });

    it('should match snapshot for premium customer', () => {
      const result = userService.calculateDiscount({
        customerId: 'C002',
        purchaseAmount: 100,
        loyaltyPoints: 5000,
        isPremium: true
      });
      expect(result).toMatchSnapshot();
    });

    // Edge cases
    it('should match snapshot for zero amount', () => {
      const result = userService.calculateDiscount({
        customerId: 'C001',
        purchaseAmount: 0,
        loyaltyPoints: 0
      });
      expect(result).toMatchSnapshot();
    });
  });
});

Approval Testing (ApprovalTests)

// Generated approval test
public class UserServiceCharacterizationTest {

    @Test
    public void calculateDiscount_standardCustomer() {
        UserService service = new UserService();
        DiscountResult result = service.calculateDiscount(
            new DiscountRequest("C001", 100.0, 500)
        );
        Approvals.verify(result);
    }

    @Test
    public void calculateDiscount_boundaryValues() {
        UserService service = new UserService();

        // Boundary: minimum values
        Approvals.verify(service.calculateDiscount(
            new DiscountRequest("C001", 0.01, 0)
        ), "minimum");

        // Boundary: maximum values
        Approvals.verify(service.calculateDiscount(
            new DiscountRequest("C001", 999999.99, 999999)
        ), "maximum");
    }
}

Recording Tests (Python)

# Generated recording-based test
import pytest
from tests.recordings import PlaybackRecorder

class TestUserServiceCharacterization:

    @pytest.fixture
    def recorder(self):
        return PlaybackRecorder('tests/recordings/user_service')

    def test_get_user_profile_recorded(self, recorder):
        """Replay recorded external API interactions"""
        with recorder.playback('get_user_profile_c001'):
            service = UserService()
            result = service.get_user_profile('C001')

            assert result == recorder.expected_output()

    def test_update_user_settings_recorded(self, recorder):
        """Verify database interactions match recording"""
        with recorder.playback('update_settings_c001'):
            service = UserService()
            result = service.update_settings('C001', {'theme': 'dark'})

            recorder.verify_database_calls()
            assert result == recorder.expected_output()

Edge Case Categories

Boundary Values

Minimum/maximum values
Zero values
Empty strings/arrays
Null/undefined values
Type boundaries (INT_MAX, etc.)

Error Conditions

Invalid inputs
Missing required fields
Type mismatches
Constraint violations
Authentication failures

Concurrent Scenarios

Race conditions
Deadlocks
Timeout scenarios
Partial failures
Retry behaviors

Environmental

Configuration variations
Timezone differences
Locale changes
Feature flag states
Permission levels

Best Practices

Start Before Changes: Generate characterization tests before any migration work
Capture Everything: Record all outputs, side effects, and state changes
Version Golden Masters: Store golden masters in version control
Review Approvals: Human review of initial golden masters is essential
Handle Non-Determinism: Normalize timestamps, IDs, and random values
Incremental Updates: Update baselines incrementally as changes are approved
CI Integration: Fail builds on unexpected behavior changes

characterization-test-generator