Harness Integration Test

Harness Integration Test

Service boundary testing, API contract verification, and consumer-driven contract validation. Ensures services communicate correctly without requiring full end-to-end infrastructure.

When to Use

• Testing API endpoints with real HTTP requests against a running service
• Validating consumer-driven contracts between microservices (Pact, Spring Cloud Contract)
• Verifying database interactions through repository or data access layers
• NOT when testing pure business logic with no I/O (use unit tests or harness-tdd instead)
• NOT when testing full user flows through a browser (use harness-e2e instead)
• NOT when performing load or stress testing on APIs (use harness-load-testing instead)

Process

Phase 1: DISCOVER -- Map Service Boundaries and Dependencies

1. Identify service boundaries. Scan the project structure for:

   • API route definitions (Express routers, FastAPI endpoints, Spring controllers, Go HTTP handlers)
   • Service client code (HTTP clients, gRPC stubs, message queue publishers/consumers)
   • Shared type definitions or API schemas (OpenAPI specs, proto files, GraphQL schemas)

2. Map inter-service dependencies. For each service, catalog:

   • Upstream dependencies: services this service calls
   • Downstream consumers: services that call this service
   • Shared resources: databases, message queues, caches

3. Inventory existing integration tests. Glob for test files in tests/integration/, __integration__/, tests/api/, and contract-tests/. Classify by type:

   • API tests: send HTTP requests and assert responses
   • Contract tests: verify provider/consumer agreements
   • Repository tests: test data access against a real database

4. Identify coverage gaps. Cross-reference discovered endpoints and service boundaries against existing tests. Flag untested:

   • API endpoints with no request/response validation
   • Service boundaries with no contract tests
   • Error scenarios (4xx responses, timeout handling, retry behavior)

5. Select test strategy. Based on the architecture:

   • Monolith: API tests with supertest/httptest against the running application
   • Microservices: consumer-driven contract tests with Pact plus API tests per service
   • Event-driven: message contract tests plus async handler integration tests

Phase 2: MOCK -- Configure Test Doubles and Infrastructure

1. Set up test database. Choose the appropriate strategy:

   • Testcontainers: Spin up a real database in Docker for each test suite. Preferred for PostgreSQL, MySQL, MongoDB.
   • In-memory database: SQLite in-memory for lightweight tests. Only when schema compatibility is confirmed.
   • Transaction rollback: Wrap each test in a transaction and roll back. Fast but requires careful connection management.

2. Configure mock services for external dependencies. For each upstream dependency:

   • Create a mock server using the framework's built-in tools (Pact mock, WireMock, nock, MSW)
   • Define request/response pairs from the API contract or OpenAPI spec
   • Configure realistic error responses (500, 503, timeout) for error path testing

3. Set up contract broker (if using Pact). Configure:

   • Pact broker URL and authentication
   • Consumer and provider version tagging strategy
   • Webhook configuration for provider verification on deploy

4. Create test fixtures and seed data. Generate:

   • Database seed scripts for required reference data
   • Request/response fixtures for common API payloads
   • Factory functions for building test entities with sensible defaults

5. Verify mock infrastructure starts. Run a smoke test that:

   • Starts the test database and confirms connectivity
   • Starts mock services and confirms they respond
   • Seeds baseline data and confirms it is queryable

Phase 3: IMPLEMENT -- Write Integration Tests

1. Write API endpoint tests. For each endpoint, test:

   • Happy path: valid request returns expected response with correct status code and body
   • Validation: invalid input returns 400 with descriptive error messages
   • Authentication: unauthenticated requests return 401, unauthorized return 403
   • Not found: requests for non-existent resources return 404
   • Edge cases: empty collections, pagination boundaries, large payloads

2. Write consumer-driven contract tests (when applicable). For each consumer-provider pair:

   • Consumer side: define interactions (request/response pairs) the consumer expects
   • Provider side: verify the provider satisfies all consumer contracts
   • Use Pact matchers for flexible verification (type matching, regex, array-like)

3. Write repository/data access tests. For each data access layer:

   • CRUD operations with valid data
   • Constraint violations (unique, foreign key, not-null)
   • Query correctness (filters, sorting, pagination)
   • Transaction behavior (isolation, rollback on error)

4. Write error handling and resilience tests. Verify:

   • Timeout behavior: service responds within SLA when dependency is slow
   • Retry logic: transient failures trigger retries with backoff
   • Circuit breaker: repeated failures open the circuit and return fallback
   • Graceful degradation: partial dependency failure does not crash the service

5. Organize tests by execution speed. Tag or separate:

   • Fast integration tests (in-memory mocks, < 5 seconds): run on every commit
   • Slow integration tests (testcontainers, external services, > 5 seconds): run on PR

Phase 4: VALIDATE -- Execute and Verify Contract Compliance

1. Run the full integration test suite. Execute all tests with verbose output. Collect:

   • Pass/fail counts per test category (API, contract, repository)
   • Execution time per test and per suite
   • Any tests that require external services to be running

2. Verify contract compliance. If using Pact:

   • Publish consumer pacts to the broker
   • Run provider verification against published pacts
   • Confirm the can-i-deploy check passes for the target environment

3. Validate test isolation. Run tests in random order (if the framework supports it). Any test that fails only when run after a specific other test has a shared-state bug. Fix immediately.

4. Run harness validate. Confirm the project passes all harness checks including the new integration test infrastructure.

5. Generate coverage report. Summarize:

   • Endpoints tested vs. total endpoints discovered
   • Contract coverage: consumer-provider pairs with verified contracts
   • Error scenarios covered vs. identified
   • Recommended next steps for remaining gaps

Graph Refresh

If a knowledge graph exists at .harness/graph/, refresh it after code changes to keep graph queries accurate:

harness scan [path]

Harness Integration

• harness validate -- Run in VALIDATE phase after all integration tests are implemented. Confirms project-wide health.
• harness check-deps -- Run after MOCK phase to verify test infrastructure dependencies do not leak into production bundles.
• emit_interaction -- Used at checkpoints to present contract verification results and coverage gaps to the human.
• Grep -- Used in DISCOVER phase to find route definitions, HTTP client usage, and service boundary patterns.
• Glob -- Used to catalog existing integration tests and contract files.

Success Criteria

• Every API endpoint has at least one integration test covering the happy path
• Every consumer-provider boundary has a verified contract (when using microservices)
• Error scenarios (400, 401, 403, 404, 500, timeout) are tested for all public endpoints
• All integration tests pass with test doubles -- no dependency on external staging environments for CI
• Test isolation is verified: tests pass in any execution order
• harness validate passes with the integration test suite in place

Examples

Example: Express API with Supertest and Testcontainers

DISCOVER output:

Framework: Express 4.18 with TypeScript
Database: PostgreSQL via Prisma
Endpoints: 14 routes across 4 controllers (users, projects, tasks, auth)
Existing tests: 3 integration tests in tests/integration/ (auth only)
Coverage gaps: projects CRUD, tasks filtering, user profile update

IMPLEMENT -- API endpoint test with supertest:

// tests/integration/projects.test.ts
import request from 'supertest';
import { app } from '../../src/app';
import { prisma } from '../../src/db';
import { createTestUser, generateAuthToken } from '../helpers/auth';

describe('POST /api/projects', () => {
  let authToken: string;

  beforeAll(async () => {
    const user = await createTestUser(prisma);
    authToken = generateAuthToken(user.id);
  });

  afterAll(async () => {
    await prisma.project.deleteMany();
    await prisma.user.deleteMany();
  });

  it('creates a project with valid data', async () => {
    const response = await request(app)
      .post('/api/projects')
      .set('Authorization', `Bearer ${authToken}`)
      .send({ name: 'Test Project', description: 'Integration test' });

    expect(response.status).toBe(201);
    expect(response.body).toMatchObject({
      name: 'Test Project',
      description: 'Integration test',
    });
    expect(response.body.id).toBeDefined();
  });

  it('returns 400 when name is missing', async () => {
    const response = await request(app)
      .post('/api/projects')
      .set('Authorization', `Bearer ${authToken}`)
      .send({ description: 'No name' });

    expect(response.status).toBe(400);
    expect(response.body.errors).toContainEqual(expect.objectContaining({ field: 'name' }));
  });

  it('returns 401 without auth token', async () => {
    const response = await request(app).post('/api/projects').send({ name: 'Unauthorized' });

    expect(response.status).toBe(401);
  });
});

Example: Pact Consumer-Driven Contract Test

IMPLEMENT -- Consumer side (frontend):

// contract-tests/consumer/project-service.pact.ts
import { PactV3, MatchersV3 } from '@pact-foundation/pact';
import { ProjectClient } from '../../src/clients/project-client';

const { like, eachLike, uuid } = MatchersV3;

const provider = new PactV3({
  consumer: 'Dashboard',
  provider: 'ProjectService',
});

describe('ProjectService contract', () => {
  it('returns a list of projects', async () => {
    await provider
      .given('projects exist for user')
      .uponReceiving('a request for user projects')
      .withRequest({
        method: 'GET',
        path: '/api/projects',
        headers: { Authorization: like('Bearer token-123') },
      })
      .willRespondWith({
        status: 200,
        body: eachLike({
          id: uuid(),
          name: like('Project Alpha'),
          createdAt: like('2026-01-15T10:30:00Z'),
        }),
      })
      .executeTest(async (mockServer) => {
        const client = new ProjectClient(mockServer.url);
        const projects = await client.listProjects('token-123');
        expect(projects).toHaveLength(1);
        expect(projects[0].name).toBe('Project Alpha');
      });
  });
});

Rationalizations to Reject

Rationalization	Why It Is Wrong
"Testing the happy path is sufficient -- error scenarios are edge cases"	The success criteria require error scenarios (400, 401, 403, 404, 500, timeout) for all public endpoints. Error paths are where real-world failures happen.
"We can test against the staging environment instead of setting up local mocks"	No integration tests that require external staging environments for CI. Tests must run with local test doubles.
"The consumer contract changed, so I will update the consumer test to match the provider"	Contract changes must be coordinated. The provider may have introduced a bug, not an intentional change.
"Tests pass when I run them in order, so they are fine"	Phase 4 requires running tests in random order. Any test that fails only in a specific order has a shared-state bug.

Gates

• No integration tests that require external staging environments for CI. Every integration test must run with local test doubles (mocks, containers, in-memory databases). Tests that fail without a staging VPN are not integration tests -- they are environment tests.
• No shared mutable state between tests. Each test must set up and tear down its own data. If tests fail when run in random order, shared state exists. Fix it before proceeding.
• No testing implementation details. Integration tests assert on API contracts (status codes, response shapes, headers) and observable data changes -- not on internal function calls or database column values that are not part of the public contract.
• Contract changes must be coordinated. If a provider contract test reveals a breaking change, do not silently update the consumer expectation. Flag it as a coordination point between teams.

Escalation

• When a service dependency has no API documentation or schema: Cannot write accurate contract tests without knowing the contract. Escalate to the dependency team to provide an OpenAPI spec, proto file, or at minimum a Pact broker with published contracts.
• When Testcontainers fails in CI (Docker-in-Docker issues, resource limits): Fall back to in-memory alternatives where possible. For databases that have no in-memory mode, escalate to DevOps to configure CI runners with Docker support.
• When contract verification fails on the provider side: This indicates a real incompatibility between consumer expectations and provider implementation. Do not adjust the consumer test to match the provider bug. Escalate to the provider team with the failing interaction details.
• When integration tests exceed 5 minutes for a single service: Triage by separating fast tests (mocked dependencies) from slow tests (testcontainers). Run fast tests on every commit, slow tests on PR only.