Stats
Actions
Tags
Help us improve
Share bugs, ideas, or general feedback.
From axon
Use when the user wants to crawl an entire website, documentation site, or multiple pages from a domain; index a whole docs section; or follow links deeply across a site. Triggers on "crawl this site", "index the whole docs", "crawl all pages under", "spider this URL", "index the entire", "grab all pages from". Prefer over scrape when breadth matters — multiple pages across a site.
npx claudepluginhub jmagar/lab --plugin axonHow this skill is triggered — by the user, by Claude, or both
Slash command
/axon:crawlThis skill is limited to the following tools:
The summary Claude sees in its skill listing — used to decide when to auto-load this skill
Crawls a site recursively, scraping and embedding all discovered pages.
Guides Next.js Cache Components and Partial Prerendering (PPR): 'use cache' directives, cacheLife(), cacheTag(), revalidateTag() for caching, invalidation, static/dynamic optimization. Auto-activates on cacheComponents: true.
Migrates code, prompts, and API calls from Claude Sonnet 4.0/4.5 or Opus 4.1 to Opus 4.5, updating model strings on Anthropic, AWS, GCP, Azure platforms.
Breaks plans, specs, or PRDs into thin vertical-slice issues on the project issue tracker using tracer bullets. Useful for converting high-level work into grabbable implementation tickets.
Share bugs, ideas, or general feedback.
Crawls a site recursively, scraping and embedding all discovered pages.
{ "action": "crawl", "urls": ["https://docs.example.com"] }
With limits (always set for large sites):
{
"action": "crawl",
"urls": ["https://docs.example.com"],
"max_pages": 200,
"max_depth": 3,
"include_subdomains": false
}
{ "action": "crawl", "subaction": "status", "job_id": "<uuid>" }
{ "action": "crawl", "subaction": "cancel", "job_id": "<uuid>" }
{ "action": "crawl", "subaction": "list", "limit": 10 }
{ "action": "crawl", "subaction": "cleanup" }
{ "action": "crawl", "subaction": "recover" }
axon crawl https://docs.example.com --max-pages 200 --max-depth 3 --wait true
| Option | Default | Notes |
|---|---|---|
max_pages | 0 (uncapped) | Always set for unknown sites |
max_depth | 5 | Link depth from start URL |
include_subdomains | false | Include *.example.com |
render_mode | auto_switch | http, chrome, auto_switch |
Crawl is async — returns a job_id immediately. Poll subaction: "status" until complete.