From firecrawl
Crawls websites to bulk extract content from multiple pages or site sections like /docs, supporting depth limits, path filtering, concurrency, and JSON output via firecrawl CLI.
npx claudepluginhub firecrawl/firecrawl-claude-plugin --plugin firecrawlThis skill is limited to using the following tools:
Bulk extract content from a website. Crawls pages following links up to a depth/limit.
Crawls websites to extract content from multiple pages via Tavily CLI. Saves pages as local markdown files with depth/breadth limits, path filtering, and semantic instructions. Use for bulk doc downloads or site content collection.
Automate web scraping of single pages, site crawling, structured data extraction, and batch URL processing using Firecrawl via Composio integration. Ideal for gathering web data in terminal workflows.
Uses Firecrawl CLI to search the web, scrape URLs, crawl sites, and interact with dynamic pages, outputting clean markdown.
Share bugs, ideas, or general feedback.
Bulk extract content from a website. Crawls pages following links up to a depth/limit.
/docs/)# Crawl a docs section
firecrawl crawl "<url>" --include-paths /docs --limit 50 --wait -o .firecrawl/crawl.json
# Full crawl with depth limit
firecrawl crawl "<url>" --max-depth 3 --wait --progress -o .firecrawl/crawl.json
# Check status of a running crawl
firecrawl crawl <job-id>
| Option | Description |
|---|---|
--wait | Wait for crawl to complete before returning |
--progress | Show progress while waiting |
--limit <n> | Max pages to crawl |
--max-depth <n> | Max link depth to follow |
--include-paths <paths> | Only crawl URLs matching these paths |
--exclude-paths <paths> | Skip URLs matching these paths |
--delay <ms> | Delay between requests |
--max-concurrency <n> | Max parallel crawl workers |
--pretty | Pretty print JSON output |
-o, --output <path> | Output file path |
--wait when you need the results immediately. Without it, crawl returns a job ID for async polling.--include-paths to scope the crawl — don't crawl an entire site when you only need one section.firecrawl credit-usage before large crawls.