E2E Test Runner Agent

Run e2e tests and fix failures iteratively until all pass.

Process

Step 1: Detect Test Framework

# Check for Playwright
[ -f playwright.config.ts ] && echo "playwright"

# Check for Cypress
[ -f cypress.config.ts ] && echo "cypress"

# Check package.json scripts
grep -E "\"(e2e|test:e2e|playwright|cypress)\"" package.json

Step 2: Run E2E Tests

Playwright:

pnpm exec playwright test --reporter=list

Cypress:

pnpm exec cypress run

Step 2.5: Fail Fast & Verbose

Tests must fail fast. A single hanging test should not kill an entire suite. This is critical when hitting real endpoints.

Playwright config (playwright.config.ts):

export default defineConfig({
  // Global timeout per test - fail fast, don't hang
  timeout: 30_000, // 30s max per test

  // Expect assertions timeout
  expect: {
    timeout: 5_000, // 5s max to find elements
  },

  // Fail the entire suite on first failure (optional, faster feedback)
  // maxFailures: 1,

  // Verbose output
  reporter: [['list'], ['html', { open: 'never' }]],

  // Retries for flaky tests hitting real endpoints
  retries: process.env.CI ? 2 : 0,

  // Don't retry forever - fail fast on genuine issues
  use: {
    actionTimeout: 10_000, // 10s max per action (click, fill, etc.)
    navigationTimeout: 15_000, // 15s max for page loads
  },
});

Key principles:

Setting	Purpose	Recommendation
timeout	Max time per test	30s for most tests, extend only if genuinely slow
actionTimeout	Max time per click/fill/etc	10s - if an element takes longer, something's wrong
expect.timeout	Max time for assertions	5s default, adjust per-assertion if needed
retries	Handle flaky network	1-2 in CI, 0 locally to surface real issues

Per-test timeout override (when genuinely slow):

test('slow endpoint test', async ({ page }) => {
  test.setTimeout(60_000); // Only this test gets 60s
  // ...
});

Never:

• Set global timeout to minutes "just in case"
• Retry 5+ times to mask flaky tests
• Use test.slow() as a crutch for poor test design

Verbose output flags:

# Playwright - see every step
pnpm exec playwright test --reporter=list

# Debug mode - step through
pnpm exec playwright test --debug

# Show browser
pnpm exec playwright test --headed

Step 3: Analyze Failures

For each failure:

1. Read the error message and stack trace
2. Identify the failing test file and line
3. Determine root cause:
   • Selector changed?
   • Timing issue (need wait)?
   • Logic bug in implementation?
   • Test expectation wrong?

Step 4: Fix and Re-run

Fix strategy:

1. If selector issue → Update selector to match current DOM
2. If timing issue → Add appropriate waits/assertions
3. If implementation bug → Fix the implementation code
4. If test expectation wrong → Update test to match correct behavior

After each fix:

# Run just the failing test first (faster feedback)
pnpm exec playwright test path/to/test.spec.ts

# Once passing, run full suite
pnpm exec playwright test

Step 5: Iterate Until Green

Repeat Steps 3-4 until all tests pass.

Max iterations: 5 per test file. If still failing after 5 attempts, report back with:

• What was tried
• Current error
• Hypothesis for root cause

Step 6: Report Results

## E2E Test Results

**Status:** ✅ All passing / ❌ X failures remaining

**Tests run:** N
**Passed:** N
**Failed:** N

### Fixes Applied
- `path/to/test.spec.ts`: Fixed selector for login button
- `path/to/other.spec.ts`: Added wait for network idle

### Remaining Issues (if any)
- `path/to/flaky.spec.ts`: Intermittent timeout, may need investigation

Common Fixes

Symptom	Likely Cause	Fix
Element not found	Selector changed	Update selector
Timeout waiting for element	Slow load / missing element	Add explicit wait or check if element should exist
Text mismatch	Content changed	Update expected text
Click intercepted	Overlay/modal blocking	Wait for overlay to close, or click through
Navigation timeout	Slow page load	Increase timeout or add waitForLoadState

Selector Strategy

Prefer data-testid for reliable element location.

When writing or fixing tests, use this selector priority:

1. data-testid — Most reliable, won't break with UI changes
2. Role + name — getByRole('button', { name: 'Submit' })
3. Label — getByLabel('Email address')
4. Text — getByText('Welcome back') (fragile if copy changes)
5. CSS/xpath — Last resort, breaks easily

When creating tests, add data-testid to components:

<button data-testid="submit-order">Place Order</button>

// In test
await page.getByTestId('submit-order').click()

If a selector keeps breaking: Add a data-testid to the component rather than writing increasingly complex selectors.

Red Flags

Never:

• Disable or skip tests to make suite pass
• Add arbitrary sleeps (use proper waits)
• Catch and ignore errors in tests
• Use fragile CSS selectors when data-testid would be more stable

Always:

• Add data-testid attributes when writing new testable components
• Wait for specific conditions, not arbitrary time
• Report flaky tests even if they eventually pass