playwright-stealth

Playwright Stealth: Anti-Detection Web Browsing

Intended use: Accessing your own content on sites where automated browsers are incorrectly blocked by bot detection systems. This is personal tooling for content verification, not for unauthorized scraping of third-party data.

Uses Patchright (a Playwright fork that patches CDP-level automation leaks) with a virtual display for headed mode to present as a standard browser.

Why This Approach Works

Bot detection operates at multiple layers. Standard Playwright fails because:

Layer	Detection Signal	Fix
CDP Protocol	Runtime.enable leak, Console.enable	Patchright patches these at source level
Browser Identity	HeadlessChrome in Browser.getVersion	Xvfb + headed mode — browser reports standard Chrome
Automation Flags	--enable-automation, navigator.webdriver=true	Patchright removes flag; init script patches webdriver
JS Fingerprints	Missing plugins, wrong WebGL vendor, no chrome.runtime	Init script patches all of these
Headers	Sec-Fetch-* applied to all requests breaks CORS	Route-based headers — only on document requests
Behavior	Instant actions, no scrolling	Human-like delays between actions

---

Architecture

Xvfb (virtual X display, 1366x768x24)
  +-- Patchright launches Chromium in HEADED mode
        +-- Init scripts patch JS fingerprints
              +-- Navigate to target, challenge resolves in ~6s

Prerequisites (install once):

• xvfb — apt-get install -y xvfb
• patchright — npm install patchright
• System Chromium — /usr/bin/chromium (already installed)

---

Step 1: Launch Stealth Browser

This is the critical step. Use browser_run_code to launch the full stealth stack.

Option A: Via standalone script (for heavy extraction)

When you need full control, run a standalone script via Bash:

xvfb-run --auto-servernum --server-args="-screen 0 1366x768x24" \
  node /path/to/stealth-script.mjs

The script uses Patchright (same API as Playwright):

import { chromium } from 'patchright';

const browser = await chromium.launch({
  executablePath: '/usr/bin/chromium',
  headless: false,  // HEADED — Xvfb provides the display
  args: [
    '--no-sandbox',
    '--disable-gpu',
    '--disable-dev-shm-usage',
    '--no-first-run',
    '--disable-infobars',
    '--window-size=1366,768',
    '--disable-blink-features=AutomationControlled',
  ],
});

const context = await browser.newContext({
  userAgent: 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/144.0.0.0 Safari/537.36',
  viewport: { width: 1366, height: 768 },
  locale: 'en-US',
  timezoneId: 'America/New_York',
});

Option B: Via Playwright MCP (for interactive browsing)

When using the MCP tools directly, apply stealth patches at the MCP level. The MCP plugin can be reconfigured for stealth by updating its config:

claude mcp add playwright -- npx @playwright/mcp@latest \
  --executable-path /usr/bin/chromium \
  --no-sandbox \
  --init-script ${CLAUDE_PLUGIN_ROOT}/playwright-stealth-init.js \
  --user-agent "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/144.0.0.0 Safari/537.36" \
  --viewport-size 1366x768

Important: The MCP approach lacks Patchright's CDP patches and Xvfb headed mode, so it will NOT bypass JS challenge pages. Use Option A for sites with aggressive bot detection. Use the MCP for sites with lighter or no detection.

---

Step 2: Register Stealth Init Scripts

Add these patches BEFORE navigating. They run in the page context before any page JS.

await context.addInitScript(() => {
  // 1. Remove webdriver flag
  Object.defineProperty(navigator, 'webdriver', {
    get: () => undefined, configurable: true
  });

  // 2. Mock chrome runtime (exists in real Chrome, missing in automation)
  window.chrome = {
    runtime: {
      onMessage: { addListener() {}, removeListener() {} },
      sendMessage() {},
      connect() { return { onMessage: { addListener() {} }, postMessage() {} }; }
    },
    loadTimes() { return {}; },
    csi() { return {}; },
    app: {
      isInstalled: false,
      InstallState: { DISABLED: 'disabled', INSTALLED: 'installed', NOT_INSTALLED: 'not_installed' },
      RunningState: { CANNOT_RUN: 'cannot_run', READY_TO_RUN: 'ready_to_run', RUNNING: 'running' }
    }
  };

  // 3. Fake plugins (real Chrome has 3, headless has 0)
  Object.defineProperty(navigator, 'plugins', {
    get: () => {
      const p = [
        { name: 'Chrome PDF Plugin', filename: 'internal-pdf-viewer', description: 'Portable Document Format', length: 1 },
        { name: 'Chrome PDF Viewer', filename: 'mhjfbmdgcfjbbpaeojofohoefgiehjai', description: '', length: 1 },
        { name: 'Native Client', filename: 'internal-nacl-plugin', description: '', length: 1 }
      ];
      p.refresh = () => {};
      return p;
    }
  });

  // 4. Languages & vendor consistency
  Object.defineProperty(navigator, 'languages', { get: () => ['en-US', 'en'] });
  Object.defineProperty(navigator, 'vendor', { get: () => 'Google Inc.' });
  Object.defineProperty(navigator, 'hardwareConcurrency', { get: () => 8 });

  // 5. Fix permissions API (headless returns 'denied' for notifications)
  const origQuery = window.Permissions?.prototype?.query;
  if (origQuery) {
    window.Permissions.prototype.query = function(params) {
      if (params.name === 'notifications') return Promise.resolve({ state: 'prompt', onchange: null });
      return origQuery.call(this, params);
    };
  }

  // 6. Connection info
  Object.defineProperty(navigator, 'connection', {
    get: () => ({ effectiveType: '4g', rtt: 50, downlink: 10, saveData: false })
  });

  // 7. Screen properties
  Object.defineProperty(screen, 'colorDepth', { get: () => 24 });
  Object.defineProperty(screen, 'pixelDepth', { get: () => 24 });

  // 8. WebGL vendor/renderer spoof (hide SwiftShader)
  const getParam = WebGLRenderingContext.prototype.getParameter;
  WebGLRenderingContext.prototype.getParameter = function(param) {
    if (param === 37445) return 'Intel Inc.';
    if (param === 37446) return 'Intel Iris OpenGL Engine';
    return getParam.call(this, param);
  };
  const getParam2 = WebGL2RenderingContext?.prototype?.getParameter;
  if (getParam2) {
    WebGL2RenderingContext.prototype.getParameter = function(param) {
      if (param === 37445) return 'Intel Inc.';
      if (param === 37446) return 'Intel Iris OpenGL Engine';
      return getParam2.call(this, param);
    };
  }
});

---

Step 3: Navigate and Handle Bot Challenges

const page = await context.newPage();
const response = await page.goto(targetUrl, {
  waitUntil: 'domcontentloaded',
  timeout: 30000
});

// Check for JS challenge page
const title = await page.title();
if (title.includes('Just a moment') || title.includes('Attention Required')) {
  // Wait for challenge to auto-resolve (typically 4-8 seconds with stealth)
  for (let i = 0; i < 20; i++) {
    await new Promise(r => setTimeout(r, 2000));
    const currentTitle = await page.title();
    if (!currentTitle.includes('Just a moment')) break;
  }
  // Wait for actual content to load after challenge resolves
  await page.waitForLoadState('networkidle', { timeout: 30000 }).catch(() => {});
  await new Promise(r => setTimeout(r, 3000));
}

If Challenge Doesn't Resolve

If still blocked after 40 seconds, the site may have additional protection. Escalation options:

1. Ask user to solve manually — user opens the URL in their real browser, solves the challenge, exports the clearance cookie for reuse
2. Use platform API — many sites offer APIs for accessing your own content
3. Try cached version — search engine caches may have the content

---

Step 4: Handle Common Obstacles

Cookie Consent Banners

Take a snapshot, find "Accept" / "Accept All" / "I Agree" button, click it.

Login Wall Detection

If the page shows a login form:

1. Tell the user what you see and ask if they want to log in
2. Ask for credentials (NEVER store or log them)
3. Type slowly for login fields:

   await page.fill('#email', email);
   await new Promise(r => setTimeout(r, 1000 + Math.random() * 1000));
   await page.fill('#password', password);
   await new Promise(r => setTimeout(r, 500 + Math.random() * 500));
   await page.click('button[type="submit"]');
   

4. After login, re-navigate to the original target URL if redirected

CAPTCHA

Tell the user: "A CAPTCHA appeared. I can't solve these automatically. Please solve it manually, then tell me to continue." Wait for user signal.

---

Step 5: Content Extraction

Page Metadata

const metadata = await page.evaluate(() => ({
  title: document.title,
  url: window.location.href,
  canonical: document.querySelector('link[rel="canonical"]')?.href,
  description: document.querySelector('meta[name="description"]')?.content,
  ogTitle: document.querySelector('meta[property="og:title"]')?.content,
  ogDescription: document.querySelector('meta[property="og:description"]')?.content,
  ogImage: document.querySelector('meta[property="og:image"]')?.content,
  author: document.querySelector('meta[name="author"]')?.content,
  publishDate: document.querySelector('meta[property="article:published_time"]')?.content
    || document.querySelector('time')?.getAttribute('datetime'),
  lang: document.documentElement.lang
}));

Main Content

const content = await page.evaluate(() => {
  const selectors = [
    'article', 'main', '[role="main"]',
    '.post-content', '.entry-content', '.article-content',
    '.content', '#content', '#main-content'
  ];
  let el = null;
  for (const s of selectors) {
    el = document.querySelector(s);
    if (el && el.textContent.trim().length > 200) break;
  }
  if (!el) el = document.body;
  return el.innerText;
});

When using the MCP tools (Option B)

Use the same extraction pattern via MCP tools:

browser_snapshot                           -> page structure
browser_evaluate: () => document.title     -> metadata
browser_evaluate: () => document.body.innerText -> content
browser_console_messages: level="warning"  -> debug info
browser_network_requests: includeStatic=false -> detect blocks

---

Artifact Output Paths (MANDATORY)

All Playwright-generated files MUST be saved to .artifacts/playwright/ — never to the project root. The cleanup-artifacts Stop hook will move any stray root-level media files automatically, but the preferred approach is to write them to the right place from the start.

File type	Destination
Screenshots (.png, .jpg)	.artifacts/playwright/screenshots/YYYY-MM-DD_HHmm/
Videos (.mp4, .webm)	.artifacts/playwright/videos/YYYY-MM-DD_HHmm/
HAR files (.har)	.artifacts/playwright/har/YYYY-MM-DD_HHmm/

Creating the directory before saving

Always create the directory first:

ARTIFACT_DIR=".artifacts/playwright/screenshots/$(date +%Y-%m-%d_%H%M)"
mkdir -p "$ARTIFACT_DIR"

Then reference it in your Playwright script:

const artifactDir = `.artifacts/playwright/screenshots/${new Date().toISOString().slice(0,16).replace('T','_').replace(':','')}`;
await fs.mkdir(artifactDir, { recursive: true });
await page.screenshot({ path: `${artifactDir}/page.png` });

Via Playwright MCP tools

When using browser_take_screenshot, save to the artifact path explicitly:

browser_take_screenshot -> save as .artifacts/playwright/screenshots/YYYY-MM-DD_HHmm/name.png

Do NOT leave screenshots or videos in the project root directory.

---

Step 6: Output Format

Present extracted data as structured markdown:

# Page Extraction Report

## Metadata
- **URL:** [actual URL after redirects]
- **Title:** [page title]
- **Author:** [if found]
- **Published:** [if found]

## Content
[Main extracted content — headings, paragraphs, lists preserved]

## Diagnostics
- **Bot Detection:** [type detected / none]
- **Resolution:** [Auto-resolved in Xs / No challenge / Manual intervention needed]
- **Console Errors:** [Any relevant errors]

---

Reference Scripts

Stealth init script (standalone file)

Location: ${CLAUDE_PLUGIN_ROOT}/playwright-stealth-init.js

Quick Reference: Approach Selection

Site Protection	Approach	Works?
No bot detection	MCP tools directly	Yes
Light detection (basic JS checks)	MCP + init script	Yes
JS challenge pages	Patchright + Xvfb (Option A)	Yes (tested)
Public profiles on social platforms	Patchright + Xvfb or MCP	Yes (tested)
CAPTCHA required	Any — needs manual user intervention	Partial

Key Dependencies

Package	Purpose	Install
patchright	Playwright fork (patches CDP leaks)	npm install patchright
xvfb	Virtual X display for headed mode without real monitor	apt-get install -y xvfb
System Chromium	Real browser binary at /usr/bin/chromium	Pre-installed

Environment Notes (proot-distro ARM64)

• Always use --no-sandbox and --disable-gpu flags
• Use xvfb-run --auto-servernum to provide virtual display
• Generous timeouts — everything runs 2-5x slower
• Use browser_snapshot (not screenshots) when using MCP tools
• JS challenges typically resolve in 4-8 seconds with this stack