Slash Command

/crawl

Crawl an entire website and gather content from all pages

Install

Run in your terminal

npx claudepluginhub nathanvale/side-quest-marketplace-old --plugin firecrawl

Details

Argument<url> [--wait] [--limit N] [--max-depth N]

Allowed Tools

Bash

Command Content

Other plugins with /crawl

/crawl

批量爬取小红书笔记（搜索 + 详情 + 评论）。用法：/rednote-mining:crawl <关键词> [--limit N] [--comments N]

rednote-mining

/crawl

Crawl a website and return clean markdown content

BashReadWrite+1

cf-crawl

/crawl

Crawl a website and save structured markdown using Crawl4AI

BashReadWrite+1

crawl4ai

/crawl

Advanced crawling with configuration options (depth, format, etc).

Skill(crawling-content)

sys-browser

/crawl

Batch scrape URLs with Firecrawl and build structured knowledge bases. Takes a URL file (one URL per line), scrapes all pages with structured extraction, and generates knowledge.json + knowledge.md. Use when asked to crawl websites, build a knowledge base, scrape URLs, or extract data from web pages.

Bash(*)

firecrawl

Stats

Parent Repo Stars4

Parent Repo Forks0

Last CommitFeb 6, 2026

Actions

View Source View Plugin View on GitHub View README

Key Flags

--wait - Block until complete (always recommended)

--progress - Show progress while waiting

--limit <n> - Max pages to crawl

--max-depth <n> - Max link depth

--include-paths <paths> - Only crawl matching paths

--exclude-paths <paths> - Skip matching paths

-o <path> - Save to file

For full flag reference, see skills/firecrawl/references/crawl-reference.md

Firecrawl Crawl

Crawl the website: $ARGUMENTS

Instructions

Parse the arguments:
- First argument is the URL to crawl
- Always add --wait if not already present (blocks until complete)
- Key flags: --limit, --max-depth, --include-paths, --exclude-paths, --progress
Run the command:
```
npx firecrawl crawl $ARGUMENTS
```
Present the crawled content to the user

Example Usage

/firecrawl:crawl https://docs.example.com --wait --progress --limit 50
/firecrawl:crawl https://example.com --wait --max-depth 2 --include-paths "/api/*"

Key Flags

--wait - Block until complete (always recommended)
--progress - Show progress while waiting
--limit <n> - Max pages to crawl
--max-depth <n> - Max link depth
--include-paths <paths> - Only crawl matching paths
--exclude-paths <paths> - Skip matching paths
-o <path> - Save to file

For full flag reference, see skills/firecrawl/references/crawl-reference.md

Notes

Crawls can take minutes for large sites - always use --wait
Use --limit to prevent runaway crawls
Costs 1 credit per page crawled
Requires FIRECRAWL_API_KEY or firecrawl login