Skill

scrapingbee-automation

Automates Scrapingbee web scraping via Rube MCP and Composio toolkit. Discovers current tool schemas, manages connections, and executes tasks dynamically.

automation

developer-tools

npx claudepluginhub lunartech-x/superpowers --plugin superpowers

Tool Access

This skill uses the workspace's default tool permissions.

Preview

Automate Scrapingbee operations through Composio's Scrapingbee toolkit via Rube MCP.

SKILL.md

Similar Skills

scrapingant-automation

Automates Scrapingant web scraping tasks via Rube MCP and Composio toolkit. Discovers tools dynamically, manages connections, and executes workflows reliably.

superpowers

scrapingbee-cli

Scrapes web pages, SERPs, e-commerce, YouTube, and ChatGPT via ScrapingBee CLI with smart path extraction and AI rules for LLMs. Handles JS, CAPTCHAs, anti-bot automatically.

20 files

scrapingbee

Stats

Stars13

Forks2

Last CommitMar 29, 2026

Used By2 plugins

Actions

View Source View Plugin View on GitHub View README

Help us improve

Share bugs, ideas, or general feedback.

Scrapingbee Automation via Rube MCP

Automate Scrapingbee operations through Composio's Scrapingbee toolkit via Rube MCP.

Toolkit docs: composio.dev/toolkits/scrapingbee

Prerequisites

Rube MCP must be connected (RUBE_SEARCH_TOOLS available)
Active Scrapingbee connection via RUBE_MANAGE_CONNECTIONS with toolkit scrapingbee
Always call RUBE_SEARCH_TOOLS first to get current tool schemas

Setup

Get Rube MCP: Add https://rube.app/mcp as an MCP server in your client configuration. No API keys needed — just add the endpoint and it works.

Verify Rube MCP is available by confirming RUBE_SEARCH_TOOLS responds
Call RUBE_MANAGE_CONNECTIONS with toolkit scrapingbee
If connection is not ACTIVE, follow the returned auth link to complete setup
Confirm connection status shows ACTIVE before running any workflows

Tool Discovery

Always discover available tools before executing workflows:

RUBE_SEARCH_TOOLS
queries: [{use_case: "Scrapingbee operations", known_fields: ""}]
session: {generate_id: true}

This returns available tool slugs, input schemas, recommended execution plans, and known pitfalls.

Core Workflow Pattern

Step 1: Discover Available Tools

RUBE_SEARCH_TOOLS
queries: [{use_case: "your specific Scrapingbee task"}]
session: {id: "existing_session_id"}

Step 2: Check Connection

RUBE_MANAGE_CONNECTIONS
toolkits: ["scrapingbee"]
session_id: "your_session_id"

Step 3: Execute Tools

RUBE_MULTI_EXECUTE_TOOL
tools: [{
  tool_slug: "TOOL_SLUG_FROM_SEARCH",
  arguments: {/* schema-compliant args from search results */}
}]
memory: {}
session_id: "your_session_id"

Known Pitfalls

Always search first: Tool schemas change. Never hardcode tool slugs or arguments without calling RUBE_SEARCH_TOOLS
Check connection: Verify RUBE_MANAGE_CONNECTIONS shows ACTIVE status before executing tools
Schema compliance: Use exact field names and types from the search results
Memory parameter: Always include memory in RUBE_MULTI_EXECUTE_TOOL calls, even if empty ({})
Session reuse: Reuse session IDs within a workflow. Generate new ones for new workflows
Pagination: Check responses for pagination tokens and continue fetching until complete

Quick Reference

Operation	Approach
Find tools	`RUBE_SEARCH_TOOLS` with Scrapingbee-specific use case
Connect	`RUBE_MANAGE_CONNECTIONS` with toolkit `scrapingbee`
Execute	`RUBE_MULTI_EXECUTE_TOOL` with discovered tool slugs
Bulk ops	`RUBE_REMOTE_WORKBENCH` with `run_composio_tool()`
Full schema	`RUBE_GET_TOOL_SCHEMAS` for tools with `schemaRef`