web-scraper-skills

Web Scraper Skills: turn website actions into structured APIs

Make your AI agent scrape and interact with websites programmatically, even when the site does not expose the API you need.

web-scraper-skills is an open-source library of website-specific browser actions for AI agents. Each supported website is packaged as a standard SKILL.md skill with focused actions, documented inputs, and structured outputs.

Why

Have you ever tried to point an AI agent at a web app you use every day, only to watch it crawl through the page, inspect huge DOM dumps, guess selectors, miss fields, and burn tokens?

Browsing skills make common web actions fast and repeatable:

• No selector research loop. The agent loads the right action reference instead of rediscovering the page from scratch.
• One page execution call. After navigation, the agent runs maintained action code with page.evaluate() or Chrome Bridge /run-action.
• Lower token and time cost. Less DOM reading, fewer repair loops, fewer failed extraction attempts.
• Cleaner results. Actions return structured data in a predictable shape.

We benchmark each action against a no-skill browser agent and track both time and tokens in BENCHMARKS.md. In one Booking.com search benchmark, the skill path used 3,903 tokens and finished in ~9.5s; the no-skill DOM-inspection loop used 49,290 tokens and took ~82.5s.

Supported Websites

The library currently includes skills for:

Website	Skill	Available actions
Reddit	skills/reddit.com	subreddit-feed, search, post-thread, user-profile, current-user-list
X	skills/x.com	search, post-data, profile-data
LinkedIn	skills/linkedin.com	post-data
TikTok	skills/tiktok.com	get-post-analytics, get-posts-list, download-post-video
Facebook	skills/facebook.com	marketplace-search, marketplace-listing-data, marketplace-seller-data
Amazon	skills/amazon.com	search, product-data, grocery-storefront, cart-data, add-to-cart
Walmart	skills/walmart.com	search, product-data, cart-data, add-to-cart
Target	skills/target.com	search, product-data, cart-data, add-to-cart
AliExpress	skills/aliexpress.com	search, product-data, cart-data, add-to-cart
Booking.com	skills/booking.com	search, hotel-data, reviews, book-room
Airbnb	skills/airbnb.com	search, listing-data, reviews, availability-price

Browsing skills are most useful for web applications that do not expose the API an agent needs. Sometimes that is intentional, as with many social platforms; sometimes the product simply has an old, incomplete, or unavailable API. In those cases, the browser UI is the integration surface, and a skill turns that UI into a repeatable action.

Not on the list? The generic scraper covers any site — extract text, links, tables, structured metadata, and CSV datasets from any page without prebuilt selectors. No API key, no Firecrawl, no external service.

Three ways to install

A. One command with skill.sh

npx skills add OMIXEC/web-scraper-skills

Installs all per-site skills into your coding agent (Claude Code, Cursor, Codex, OpenCode, etc.). CLI auto-detects which agents you have. Run npx skills list to see installed skills.

B. The umbrella (I want all supported sites)

Point your agent at the top-level SKILL.md. It tells the agent which sites are supported and how to load each site's skill on demand:

https://raw.githubusercontent.com/OMIXEC/web-scraper-skills/main/SKILL.md

This is the right choice if your agent might interact with any of the supported sites.

C. A single site (I only care about one)

If you're building an agent that only ever works with one site, skip the umbrella — install that site's SKILL.md directly. For example, a LinkedIn-only agent:

https://raw.githubusercontent.com/OMIXEC/web-scraper-skills/main/skills/linkedin.com/SKILL.md

This is leaner — no list of unrelated sites, no routing step. Each site is a complete, standalone skill.

Repo layout