webapp-testing

Web Application Testing

To test local web applications, write TypeScript E2E tests using Playwright Test (@playwright/test).

CRITICAL: E2E Test File Placement

• ALWAYS place E2E test files in tests/e2e/ or e2e/ directory at the project root
• NEVER place test scripts in .artifacts/ - that's for evidence only (screenshots, videos)
• E2E tests should be permanent project assets, not disposable artifacts

Decision Tree: Choosing Your Approach

User task → Is it static HTML?
    ├─ Yes → Read HTML file directly to identify selectors
    │         ├─ Success → Write Playwright test using selectors
    │         └─ Fails/Incomplete → Treat as dynamic (below)
    │
    └─ No (dynamic webapp) → Is the server already running?
        ├─ No → Use webServer config in playwright.config.ts
        │        to auto-start the dev server
        │
        └─ Yes → Reconnaissance-then-action:
            1. Navigate and wait for networkidle
            2. Take screenshot or inspect DOM
            3. Identify selectors from rendered state
            4. Execute actions with discovered selectors

Playwright Test Setup

playwright.config.ts

import { defineConfig, devices } from '@playwright/test';

export default defineConfig({
  testDir: './tests/e2e',
  fullyParallel: true,
  forbidOnly: !!process.env.CI,
  retries: process.env.CI ? 2 : 0,
  workers: process.env.CI ? 1 : undefined,
  reporter: 'html',
  use: {
    baseURL: 'http://localhost:3000',
    trace: 'on-first-retry',
    screenshot: 'only-on-failure',
    video: 'retain-on-failure',
  },
  projects: [
    { name: 'chromium', use: { ...devices['Desktop Chrome'] } },
  ],
  webServer: {
    command: 'npm run dev',
    url: 'http://localhost:3000',
    reuseExistingServer: !process.env.CI,
    timeout: 120 * 1000,
  },
});

Example Test: tests/e2e/login.spec.ts

import { test, expect } from '@playwright/test';

test.describe('Login Flow', () => {
  test('should login successfully with valid credentials', async ({ page }) => {
    await page.goto('/login');
    await page.waitForLoadState('networkidle');

    await page.getByLabel('Email').fill('test@example.com');
    await page.getByLabel('Password').fill('password123');
    await page.getByRole('button', { name: 'Login' }).click();

    await expect(page).toHaveURL('/dashboard');
    await expect(page.getByRole('heading', { name: 'Welcome' })).toBeVisible();
  });

  test('should show error for invalid credentials', async ({ page }) => {
    await page.goto('/login');

    await page.getByLabel('Email').fill('invalid@example.com');
    await page.getByLabel('Password').fill('wrong');
    await page.getByRole('button', { name: 'Login' }).click();

    await expect(page.getByText('Invalid credentials')).toBeVisible();
  });
});

Running Tests

# Run all E2E tests
npx playwright test

# Run specific test file
npx playwright test tests/e2e/login.spec.ts

# Run with UI mode (interactive debugging)
npx playwright test --ui

# Run headed (visible browser)
npx playwright test --headed

# Generate test code interactively
npx playwright codegen http://localhost:3000

Reconnaissance-Then-Action Pattern

When you don't know the page structure:

import { test, expect } from '@playwright/test';

test('discover and interact with page elements', async ({ page }) => {
  await page.goto('/');
  await page.waitForLoadState('networkidle');

  // 1. Take screenshot for reconnaissance
  await page.screenshot({ path: '/tmp/inspect.png', fullPage: true });

  // 2. Log all buttons for analysis
  const buttons = await page.getByRole('button').all();
  for (const button of buttons) {
    console.log('Button:', await button.textContent());
  }

  // 3. Get page content for selector discovery
  const content = await page.content();
  console.log(content);
});

Evidence Collection for PR Reviews

When collecting evidence (screenshots/videos) for PR reviews, use this pattern:

# Run tests with evidence collection
FEATURE=${FEATURE:-feature}
mkdir -p .artifacts/$FEATURE/{images,videos}

npx playwright test tests/e2e/your-feature.spec.ts \
  --headed \
  --output=.artifacts/$FEATURE \
  --trace=retain-on-failure \
  --reporter=line

Test with Built-in Evidence Collection

import { test, expect } from '@playwright/test';

test.describe('Feature Demo', () => {
  test('demonstrate feature workflow', async ({ page }, testInfo) => {
    const feature = process.env.FEATURE || 'feature';
    const timestamp = new Date().toISOString().slice(0, 10).replace(/-/g, '');

    await page.goto('/feature');
    await page.waitForLoadState('networkidle');

    // Capture before state
    await page.screenshot({
      path: `.artifacts/${feature}/images/${timestamp}-before.png`,
      fullPage: true
    });

    // Perform actions
    await page.getByRole('button', { name: 'Enable Feature' }).click();
    await expect(page.getByText('Feature Enabled')).toBeVisible();

    // Capture after state
    await page.screenshot({
      path: `.artifacts/${feature}/images/${timestamp}-after.png`,
      fullPage: true
    });
  });
});

Quick One-liner for Ad-hoc Verification

For quick checks without writing a full test file (use TypeScript via tsx):

npx tsx -e "
import { chromium } from 'playwright';

const browser = await chromium.launch();
const page = await browser.newPage();
await page.goto(process.env.BASE_URL || 'http://localhost:3000', { waitUntil: 'networkidle' });
await page.screenshot({ path: '/tmp/webapp.png', fullPage: true });
await browser.close();
console.log('saved: /tmp/webapp.png');
"

Browser CLI Tools（対話的ブラウザ操作）

E2Eテストとは別に、対話的にブラウザを操作するためのCLIツールが3つある。 Dogfooding、手動検証、デバッグに使う。

パッケージ名（正確に）

ツール	パッケージ名	実行方法
browser-use CLI v2	browser-use (PyPI)	uvx browser-use <command>
agent-browser	agent-browser (npm)	npx agent-browser@latest <command>
Playwright CLI	@playwright/mcp (npm)	npx @playwright/mcp@latest (MCP server)

注意: @anthropic-ai/claude-code-playwright は存在しない。Playwright公式は @playwright/mcp と @playwright/cli。

インストール

# 1. browser-use CLI v2（Python / uvx経由）
uv tool install browser-use
# 確認
uvx browser-use --help

# 2. agent-browser（Rust / npx経由）
npx agent-browser@latest --version
# 初回はnpxが自動インストール

# 3. Playwright CLI（Node.js / npx経由）
npx playwright --version
# ブラウザ未インストールなら:
npx playwright install chromium

# 3a. Playwright MCP（Claude Code等のAIツールから使う場合）
npx @playwright/mcp@latest

使い分け

ツール	特徴	向いている場面
browser-use v2	Daemon型で高速（50ms/コマンド）。DOMを4段階圧縮して2-5KBに	サクッと確認したい時。速度重視
agent-browser	Rust製。CDPのAccessibility API直接利用。cursor:pointer要素も検出	深く探りたい時。a11y問題の発見に強い
Playwright CLI	MCP対応。差分snapshot（2回目以降は変更分だけ）	トークン節約したい時。AI連携に最適

ベンチマーク（reviw画面での対話的操作）

	トークン	速度	発見の深さ
Playwright CLI	36K (1位)	1,129秒 (2位)	基本確認
browser-use v2	45K (2位)	264秒 (1位)	中程度
agent-browser	53K (3位)	3,494秒 (3位)	最も深い（4件発見）

基本操作（共通パターン）

browser-use v2

uvx browser-use open http://localhost:3000    # ページを開く
uvx browser-use state                          # DOM状態をテキストで取得（トークン節約）
uvx browser-use click "button#submit"          # クリック
uvx browser-use input "input#name" "テスト"    # テキスト入力（typeではなくinput）
uvx browser-use screenshot /tmp/shot.png       # スクリーンショット
uvx browser-use close                          # ブラウザを閉じる

agent-browser

npx agent-browser@latest open http://localhost:3000
npx agent-browser@latest snapshot              # アクセシビリティツリー（トークン節約）
npx agent-browser@latest click @e5             # ref指定でクリック
npx agent-browser@latest fill @e3 "テスト"     # ref指定で入力
npx agent-browser@latest screenshot /tmp/shot.png
npx agent-browser@latest close

Playwright CLI

npx @playwright/cli@latest open http://localhost:3000
npx @playwright/cli@latest snapshot            # YAML形式アクセシビリティツリー（差分対応）
npx @playwright/cli@latest click --ref=e5      # ref指定でクリック
npx @playwright/cli@latest fill --ref=e3 "テスト"
npx @playwright/cli@latest screenshot          # スクリーンショット
npx @playwright/cli@latest close

トークン節約Tips

• スクリーンショット画像は最低限に。state / snapshot のテキスト出力で状態把握する
• Playwright CLIは差分snapshotを使うので、2回目以降は変更部分だけ返る
• browser-useのstateはviewport内の要素だけ返すのでコンパクト
• agent-browserのsnapshotはインタラクティブ要素にrefを付与するので、意味的に密度が高い

Common Pitfalls

❌ Don't inspect the DOM before waiting for networkidle on dynamic apps ✅ Do wait for page.waitForLoadState('networkidle') before inspection

❌ Don't place E2E test files in .artifacts/ ✅ Do place E2E tests in tests/e2e/ as permanent project assets

❌ Don't write tests in Python ✅ Do write tests in TypeScript using @playwright/test

❌ Don't use Vitest for E2E tests ✅ Do use Playwright Test for E2E, Vitest for unit/integration tests

Best Practices

• Use Playwright Test runner - Not Vitest or other runners for E2E
• Always use TypeScript - Never Python for Playwright tests
• Place tests in tests/e2e/ - Permanent project assets, not disposable
• Use role-based selectors - getByRole, getByLabel, getByText over CSS
• Always close browser - Playwright Test handles this automatically
• Add appropriate waits - waitForLoadState, waitForSelector, expect().toBeVisible()
• Use webServer config - Auto-start dev server in playwright.config.ts

File Structure Convention

project/
├── playwright.config.ts      # Playwright configuration
├── tests/
│   └── e2e/                  # E2E tests (permanent)
│       ├── login.spec.ts
│       ├── checkout.spec.ts
│       └── settings.spec.ts
├── .artifacts/               # Evidence only (temporary)
│   └── <feature>/
│       ├── images/           # Screenshots
│       ├── videos/           # Recorded videos
│       └── REPORT.md         # Review report
└── test-results/             # Playwright auto-generated (gitignored)

Key distinction:

• tests/e2e/ = Permanent E2E test code (committed to repo)
• .artifacts/ = Temporary evidence for PR review (gitignored or LFS)
• test-results/ = Playwright's auto-generated output (gitignored)

Console/Error Collection

test('collect console logs', async ({ page }) => {
  const consoleLogs: string[] = [];
  const errors: string[] = [];

  page.on('console', msg => consoleLogs.push(`${msg.type()}: ${msg.text()}`));
  page.on('pageerror', err => errors.push(err.message));
  page.on('requestfailed', req => errors.push(`Request failed: ${req.url()}`));

  await page.goto('/');
  await page.waitForLoadState('networkidle');

  console.log('Console logs:', consoleLogs);
  if (errors.length > 0) {
    console.error('Errors:', errors);
  }
});

Vitest vs Playwright Test: When to Use Each

Test Type	Tool	Reason
Unit tests	Vitest	Fast, no browser needed
Integration tests	Vitest	Fast, mock external deps
Component tests	Vitest + browser mode	or Storybook
E2E tests	Playwright Test	Full browser, real flows

Do NOT use Vitest for E2E tests. Playwright Test has:

• Built-in parallelization for browser tests
• Automatic retries and trace collection
• Screenshot/video on failure
• Web server management
• Test generator (codegen)