Obscura Headless Browser

---
name: obscura-headless-browser
description: Use Obscura, the lightweight Rust-based headless browser for AI agents and web scraping with CDP, Puppeteer, and Playwright support.
triggers:
  - headless browser for scraping
  - use obscura browser
  - puppeteer with obscura
  - playwright headless rust browser
  - web scraping with CDP
  - run headless chrome alternative
  - automate browser with obscura
  - stealth web scraping rust
---

# Obscura Headless Browser

> Skill by [ara.so](https://ara.so) — Daily 2026 Skills collection.

Obscura is a headless browser engine written in Rust for web scraping and AI agent automation. It runs real JavaScript via V8, implements the Chrome DevTools Protocol (CDP), and acts as a drop-in replacement for headless Chrome — with 30 MB memory usage, instant startup, and built-in anti-detection.

## Installation

### Download Binary (Recommended)

```bash
# Linux x86_64
curl -LO https://github.com/h4ckf0r0day/obscura/releases/latest/download/obscura-x86_64-linux.tar.gz
tar xzf obscura-x86_64-linux.tar.gz
sudo mv obscura /usr/local/bin/

# macOS Apple Silicon
curl -LO https://github.com/h4ckf0r0day/obscura/releases/latest/download/obscura-aarch64-macos.tar.gz
tar xzf obscura-aarch64-macos.tar.gz
sudo mv obscura /usr/local/bin/

# macOS Intel
curl -LO https://github.com/h4ckf0r0day/obscura/releases/latest/download/obscura-x86_64-macos.tar.gz
tar xzf obscura-x86_64-macos.tar.gz
sudo mv obscura /usr/local/bin/

# Windows: download .zip from releases page and extract manually

Single binary. No Chrome, no Node.js, no dependencies required.

Build from Source

git clone https://github.com/h4ckf0r0day/obscura.git
cd obscura
cargo build --release

# With stealth mode (anti-detection + tracker blocking)
cargo build --release --features stealth

Requires Rust 1.75+. First build ~5 min (V8 compiles from source, cached after).

CLI Quick Reference

obscura fetch — Render a Single Page

# Get page title via JS eval
obscura fetch https://example.com --eval "document.title"

# Dump rendered HTML (after JS executes)
obscura fetch https://news.ycombinator.com --dump html

# Dump plain text content
obscura fetch https://example.com --dump text

# Dump all links
obscura fetch https://example.com --dump links

# Wait for network to be idle (SPAs, dynamic content)
obscura fetch https://example.com --wait-until networkidle0

# Wait for a specific CSS selector to appear
obscura fetch https://example.com --selector "#main-content"

# Enable stealth mode for anti-bot sites
obscura fetch https://example.com --stealth --eval "document.title"

# Suppress banner output
obscura fetch https://example.com --quiet --dump html

obscura serve — Start CDP WebSocket Server

# Basic server on default port
obscura serve --port 9222

# With stealth mode enabled
obscura serve --port 9222 --stealth

# With proxy
obscura serve --port 9222 --proxy http://proxy.example.com:8080
obscura serve --port 9222 --proxy socks5://proxy.example.com:1080

# Multiple parallel workers
obscura serve --port 9222 --workers 4 --stealth

# Respect robots.txt
obscura serve --port 9222 --obey-robots

obscura scrape — Parallel Scraping

# Scrape multiple URLs in parallel
obscura scrape https://site1.com https://site2.com https://site3.com \
  --concurrency 25 \
  --eval "document.querySelector('h1').textContent" \
  --format json

# Output as plain text
obscura scrape https://site1.com https://site2.com \
  --eval "document.title" \
  --format text

Puppeteer Integration

npm install puppeteer-core

import puppeteer from 'puppeteer-core';

// Start obscura first: obscura serve --port 9222
const browser = await puppeteer.connect({
  browserWSEndpoint: 'ws://127.0.0.1:9222/devtools/browser',
});

const page = await browser.newPage();
await page.goto('https://news.ycombinator.com');

// Extract structured data
const stories = await page.evaluate(() =>
  Array.from(document.querySelectorAll('.titleline > a'))
    .map(a => ({ title: a.textContent, url: a.href }))
);
console.log(stories);

await browser.disconnect();

Puppeteer with Stealth Mode

# Start with stealth enabled
obscura serve --port 9222 --stealth

import puppeteer from 'puppeteer-core';

const browser = await puppeteer.connect({
  browserWSEndpoint: 'ws://127.0.0.1:9222/devtools/browser',
});

const page = await browser.newPage();

// Set custom headers if needed
await page.setExtraHTTPHeaders({
  'Accept-Language': 'en-US,en;q=0.9',
});

await page.goto('https://bot-protected-site.com', {
  waitUntil: 'networkidle0',
});

const content = await page.content();
await browser.disconnect();

Playwright Integration

npm install playwright-core

import { chromium } from 'playwright-core';

// Start obscura first: obscura serve --port 9222
const browser = await chromium.connectOverCDP({
  endpointURL: 'ws://127.0.0.1:9222',
});

const context = await browser.newContext();
const page = await context.newPage();

await page.goto('https://en.wikipedia.org/wiki/Web_scraping');
console.log(await page.title());

// Wait for selector
await page.waitForSelector('#content');
const text = await page.locator('h1').textContent();
console.log(text);

await browser.close();

Playwright Form Submission & Login

import { chromium } from 'playwright-core';

const browser = await chromium.connectOverCDP({
  endpointURL: 'ws://127.0.0.1:9222',
});

const page = await browser.newContext().then(ctx => ctx.newPage());
await page.goto('https://quotes.toscrape.com/login');

// Fill and submit form
await page.fill('#username', process.env.SCRAPE_USERNAME);
await page.fill('#password', process.env.SCRAPE_PASSWORD);
await page.click('[type="submit"]');

// Obscura handles POST, follows 302 redirect, maintains cookies
await page.waitForNavigation();
console.log('Logged in:', page.url());

await browser.close();

Common Automation Patterns

Scrape Behind Login (Puppeteer)

import puppeteer from 'puppeteer-core';

const browser = await puppeteer.connect({
  browserWSEndpoint: 'ws://127.0.0.1:9222/devtools/browser',
});
const page = await browser.newPage();

// Login
await page.goto('https://example.com/login');
await page.evaluate((user, pass) => {
  document.querySelector('#username').value = user;
  document.querySelector('#password').value = pass;
  document.querySelector('form').submit();
}, process.env.SITE_USERNAME, process.env.SITE_PASSWORD);

await page.waitForNavigation({ waitUntil: 'networkidle0' });

// Now scrape authenticated content
await page.goto('https://example.com/dashboard');
const data = await page.evaluate(() => ({
  title: document.title,
  content: document.querySelector('.data-table')?.innerHTML,
}));

await browser.disconnect();

Request Interception (Fetch Domain)

import puppeteer from 'puppeteer-core';

const browser = await puppeteer.connect({
  browserWSEndpoint: 'ws://127.0.0.1:9222/devtools/browser',
});
const page = await browser.newPage();

// Intercept and modify requests
await page.setRequestInterception(true);
page.on('request', (req) => {
  if (req.resourceType() === 'image') {
    req.abort(); // Block images for faster scraping
  } else {
    req.continue();
  }
});

await page.goto('https://example.com');
await browser.disconnect();

Cookie Management

import puppeteer from 'puppeteer-core';

const browser = await puppeteer.connect({
  browserWSEndpoint: 'ws://127.0.0.1:9222/devtools/browser',
});
const page = await browser.newPage();

// Set cookies before navigation
await page.setCookie({
  name: 'session_token',
  value: process.env.SESSION_TOKEN,
  domain: 'example.com',
});

await page.goto('https://example.com/protected');

// Get cookies after navigation
const cookies = await page.cookies();
console.log(cookies);

await browser.disconnect();

Parallel Scraping with Node.js

import puppeteer from 'puppeteer-core';

const urls = [
  'https://example.com/page1',
  'https://example.com/page2',
  'https://example.com/page3',
];

// Start obscura: obscura serve --port 9222 --workers 4
const browser = await puppeteer.connect({
  browserWSEndpoint: 'ws://127.0.0.1:9222/devtools/browser',
});

const scrape = async (url) => {
  const page = await browser.newPage();
  await page.goto(url, { waitUntil: 'networkidle0' });
  const result = await page.evaluate(() => ({
    title: document.title,
    h1: document.querySelector('h1')?.textContent,
  }));
  await page.close();
  return { url, ...result };
};

// Run with concurrency limit
const results = await Promise.all(urls.map(scrape));
console.log(JSON.stringify(results, null, 2));

await browser.disconnect();

DOM-to-Markdown (LP Domain)

Obscura exposes LP.getMarkdown for converting pages to LLM-friendly Markdown:

import puppeteer from 'puppeteer-core';

const browser = await puppeteer.connect({
  browserWSEndpoint: 'ws://127.0.0.1:9222/devtools/browser',
});

const page = await browser.newPage();
await page.goto('https://en.wikipedia.org/wiki/Rust_(programming_language)');

// Use CDP directly for LP domain
const client = await page.target().createCDPSession();
const { markdown } = await client.send('LP.getMarkdown');
console.log(markdown); // Clean markdown for LLM ingestion

await browser.disconnect();

Stealth Mode Details

Enable with --stealth flag or --features stealth at build time.

Anti-fingerprinting protections:

• Per-session fingerprint randomization (GPU, screen, canvas, audio, battery)
• Realistic navigator.userAgentData (Chrome 145, high-entropy values)
• event.isTrusted = true for dispatched events
• navigator.webdriver = undefined
• Native function masking (Function.prototype.toString() → [native code])
• 3,520 tracker/analytics domains blocked

# Always use stealth for anti-bot protected sites
obscura serve --port 9222 --stealth --workers 2

CDP Domains Reference

Domain	Key Methods
Target	createTarget, closeTarget, attachToTarget, createBrowserContext, disposeBrowserContext
Page	navigate, getFrameTree, addScriptToEvaluateOnNewDocument, lifecycleEvents
Runtime	evaluate, callFunctionOn, getProperties, addBinding
DOM	getDocument, querySelector, querySelectorAll, getOuterHTML, resolveNode
Network	enable, setCookies, getCookies, setExtraHTTPHeaders, setUserAgentOverride
Fetch	enable, continueRequest, fulfillRequest, failRequest
Storage	getCookies, setCookies, deleteCookies
Input	dispatchMouseEvent, dispatchKeyEvent
LP	getMarkdown

Performance Comparison

Metric	Obscura	Headless Chrome
Memory	30 MB	200+ MB
Binary size	70 MB	300+ MB
Startup	Instant	~2s
Static HTML load	51 ms	~500 ms
JS + XHR load	84 ms	~800 ms

Troubleshooting

Connection refused on ws://127.0.0.1:9222:

# Ensure obscura serve is running
obscura serve --port 9222
# Check it's listening
curl http://127.0.0.1:9222/json/version

Page content missing (SPA/dynamic):

# Use networkidle0 wait strategy
obscura fetch https://example.com --wait-until networkidle0

# Or wait for a specific element
obscura fetch https://example.com --selector "#app-content"

Bot detection triggered:

# Always use stealth mode for protected sites
obscura serve --port 9222 --stealth

High memory usage with many pages:

# Always close pages after use in code
await page.close();
# Use --workers to isolate processes
obscura serve --port 9222 --workers 4

Proxy authentication:

# Include credentials in proxy URL
obscura serve --port 9222 --proxy http://$PROXY_USER:$PROXY_PASS@proxy.example.com:8080

Build taking too long:

• Normal: first build ~5 min (V8 compiles from source)
• Subsequent builds use cached V8 artifacts and are much faster
• Use the pre-built binary for fastest setup