**Status**: Production Ready ✅
Scrapes web content with JavaScript rendering, anti-bot bypass, and structured data extraction.
npx claudepluginhub secondsky/claude-skillsThis skill inherits all available tools. When active, it can use any tool Claude has access to.
references/common-patterns.mdreferences/endpoints.mdtemplates/firecrawl-crawl-example.pytemplates/firecrawl-scrape-python.pytemplates/firecrawl-scrape-typescript.tstemplates/firecrawl-worker-fetch.tsStatus: Production Ready ✅ Last Updated: 2025-11-21 Official Docs: https://docs.firecrawl.dev API Version: v2.5
Firecrawl is a Web Data API for AI that turns entire websites into LLM-ready markdown or structured data. It handles:
/v2/scrape - Single Page ScrapingScrapes a single webpage and returns clean, structured content.
Use Cases:
Key Options:
formats: ["markdown", "html", "screenshot"]onlyMainContent: true/false (removes nav, footer, ads)waitFor: milliseconds to wait before scrapingactions: browser automation actions (click, scroll, etc.)/v2/crawl - Full Site CrawlingCrawls all accessible pages from a starting URL.
Use Cases:
Key Options:
limit: max pages to crawlmaxDepth: how many links deep to followallowedDomains: restrict to specific domainsexcludePaths: skip certain URL patterns/v2/map - URL DiscoveryMaps all URLs on a website without scraping content.
Use Cases:
/v2/extract - Structured Data ExtractionUses AI to extract specific data fields from pages.
Use Cases:
Key Options:
schema: Zod or JSON schema defining desired structuresystemPrompt: guide AI extraction behaviorFirecrawl requires an API key for all requests.
fc-)NEVER hardcode API keys in code!
# .env file
FIRECRAWL_API_KEY=fc-your-api-key-here
# .env.local (for local development)
FIRECRAWL_API_KEY=fc-your-api-key-here
pip install firecrawl-py # v4.5.0+
from firecrawl import FirecrawlApp
import os
app = FirecrawlApp(api_key=os.environ.get("FIRECRAWL_API_KEY"))
result = app.scrape_url("https://example.com", params={"formats": ["markdown"], "onlyMainContent": True})
print(result.get("markdown"))
bun add @mendable/firecrawl-js # v4.4.1+
import FirecrawlApp from '@mendable/firecrawl-js';
const app = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY });
const result = await app.scrapeUrl('https://example.com', { formats: ['markdown'], onlyMainContent: true });
console.log(result.markdown);
See: templates/ for crawl, extract, and advanced examples
| Use Case | Endpoint | Key Options |
|---|---|---|
| Documentation scraping | crawl_url() | limit: 500, allowedDomains |
| Product data extraction | extract() | Zod schema + systemPrompt |
| News article scraping | scrape_url() | onlyMainContent: true, removeBase64Images |
| URL discovery | map() | Find all pages before crawling |
See: references/common-patterns.md for complete examples.
# Python
try:
result = app.scrape_url("https://example.com")
except FirecrawlException as e:
print(f"Firecrawl error: {e}")
// TypeScript
try {
const result = await app.scrapeUrl('https://example.com');
} catch (error) {
console.error('Error:', error.message);
}
| Best Practice | Why |
|---|---|
Use onlyMainContent: true | Reduces credits, cleaner output |
Set reasonable limit | Avoid excessive costs |
Use map endpoint first | Plan crawling strategy |
| Cache results | Avoid re-scraping |
| Batch extract calls | More efficient for multiple URLs |
Credits: Free tier = 500/month, paid tiers higher.
⚠️ SDK cannot run in Workers (Node.js dependencies). Use direct REST API:
const response = await fetch('https://api.firecrawl.dev/v2/scrape', {
method: 'POST',
headers: {
'Authorization': `Bearer ${env.FIRECRAWL_API_KEY}`,
'Content-Type': 'application/json',
},
body: JSON.stringify({ url, formats: ['markdown'], onlyMainContent: true })
});
See: references/common-patterns.md for complete Workers example with caching.
| ✅ Use Firecrawl | ❌ Don't Use |
|---|---|
| Modern JS-rendered sites | Simple static HTML (use cheerio) |
| Clean markdown for LLMs | Existing Puppeteer setup works |
| RAG/chatbot content | Direct API available |
| Structured data extraction | Budget constraints |
| Bot protection bypass |
| Issue | Cause | Fix |
|---|---|---|
| "Invalid API Key" | Key not set | Check $FIRECRAWL_API_KEY starts with fc- |
| "Rate limit exceeded" | Monthly credits used | Check dashboard, upgrade plan |
| "Timeout error" | Page slow to load | Add waitFor: 10000 |
| "Content is empty" | JS loads late | Add actions: [{type: "wait", milliseconds: 3000}] |
| Feature | Usage |
|---|---|
| Browser actions | actions: [{type: "click", selector: "button"}] |
| Custom headers | headers: {"User-Agent": "Custom Bot"} |
| Webhooks | webhook: "https://your-domain.com/webhook" |
| Screenshots | formats: ["screenshot"] |
See: references/endpoints.md for complete API reference.
| Reference | Load When... |
|---|---|
endpoints.md | Need complete API endpoint documentation |
common-patterns.md | Cloudflare Workers, caching, batch processing, error handling |
| Package | Version |
|---|---|
| firecrawl-py | 4.5.0+ |
| @mendable/firecrawl-js | 4.4.1+ |
| API | v2 |
Note: Node.js SDK requires Node.js >=22.0.0, cannot run in Workers.
Official Docs: https://docs.firecrawl.dev | GitHub: https://github.com/mendableai/firecrawl
Token Savings: ~60% | Production Ready: ✅
Expert guidance for Next.js Cache Components and Partial Prerendering (PPR). **PROACTIVE ACTIVATION**: Use this skill automatically when working in Next.js projects that have `cacheComponents: true` in their next.config.ts/next.config.js. When this config is detected, proactively apply Cache Components patterns and best practices to all React Server Component implementations. **DETECTION**: At the start of a session in a Next.js project, check for `cacheComponents: true` in next.config. If enabled, this skill's patterns should guide all component authoring, data fetching, and caching decisions. **USE CASES**: Implementing 'use cache' directive, configuring cache lifetimes with cacheLife(), tagging cached data with cacheTag(), invalidating caches with updateTag()/revalidateTag(), optimizing static vs dynamic content boundaries, debugging cache issues, and reviewing Cache Component implementations.
Applies Anthropic's official brand colors and typography to any sort of artifact that may benefit from having Anthropic's look-and-feel. Use it when brand colors or style guidelines, visual formatting, or company design standards apply.
Creating algorithmic art using p5.js with seeded randomness and interactive parameter exploration. Use this when users request creating art using code, generative art, algorithmic art, flow fields, or particle systems. Create original algorithmic art rather than copying existing artists' work to avoid copyright violations.