Browser Automation (GTM)

Browser automation with two engines: Chrome extension (primary) for authenticated session work, Playwright (fallback) for headless/scripted automation.

Which engine to use

	Chrome extension (primary)	Playwright (fallback)
Engine	Claude in Chrome extension (user's Chrome)	Playwright (own browser instance)
Best for	Acting inside the user's authenticated browser session — logged-in SaaS apps, cookie-based workflows, tasks that need the user's extensions/state	Testing, QA, screenshots, CI/CD, scripted multi-step flows, performance audits, video recording
Auth model	Inherits the user's live Chrome session (cookies, localStorage, extensions)	Starts fresh; save/load auth state explicitly
Headless	No — requires a visible Chrome window	Yes (default)
Scriptable	Extension tool surface (read_page, computer, form_input, javascript_tool)	Full Playwright API, helper library, session mode
Works in Docker/CI	No	Yes

Decision flow:

1. Is the Chrome extension connected? Try it first — it inherits the user's session.
2. If the extension is not connected (not installed, claude --chrome not used, tab group not set up), fall back to Playwright via /eng:browser rather than giving up.
3. Need headless, scripted, or CI-compatible automation with no user session? Go directly to Playwright via /eng:browser.
4. Need both (e.g., extract auth from user's Chrome, then run headless tests)? Use this skill's Chrome extension to extract auth, then hand off to /eng:browser for Playwright automation.

How /eng:browser differs from this skill: /eng:browser is Playwright-only — no Chrome extension, no authenticated session inheritance. Use it directly when you know you need pure Playwright. This skill tries the Chrome extension first and falls back to Playwright when the extension isn't available.

---

Chrome Extension: Three Entry Points

The Claude in Chrome extension can be controlled from different surfaces:

Entry Point	How to Connect	Best For
Standalone	Open extension side panel in Chrome	Browser-native workflows, shortcuts, scheduling
Claude Code CLI	Run claude --chrome, verify with /chrome	Dev/test loops, terminal + browser chaining
Cowork / Claude Desktop	Enable "Claude in Chrome" connector in Desktop settings	Knowledge work with browser access

All three use the same extension and share the same core browser capabilities. The difference is what orchestrates the automation (side panel UI vs CLI vs Desktop).

---

Two Automation Patterns

Pattern A: Script Injection Skills

For skills with a scripts/ folder containing reusable JavaScript SDKs:

• Read script from filesystem → inject via javascript_tool → cache in localStorage
• Examples: Custom SDK scripts in ~/.claude/use-browser-skills/{skill-name}/scripts/
• Use the workflow in this document

Pattern B: Direct UI Automation

For tasks that use Chrome extension tools directly (no pre-saved scripts):

• Use read_page, find, computer, form_input, navigate, javascript_tool directly
• No script injection or caching needed
• Follow the Core Workflow below

When to use Pattern B:

• One-off browser tasks (fill a form, scrape a page, test a UI)
• Tasks where you interact with elements rather than inject an SDK
• Ad-hoc automation without a pre-built skill

How to tell which pattern:

• Has scripts/*.js → Pattern A (script injection)
• No scripts/ folder → Pattern B (direct automation)

---

Core Workflow (All Browser Automation)

Regardless of pattern, all browser automation follows this flow:

Step 0: Get Browser Context (Required First)

CRITICAL: Always call tabs_context_mcp first to get valid tab IDs.

1. tabs_context_mcp           → Get available tabs in MCP group
2. tabs_create_mcp            → Create new tab if needed
3. navigate(url, tabId)       → Go to target page

Without this step, you won't have valid tab IDs and tools will fail.

If this step fails (extension not connected, tools not available): fall back to Playwright via /eng:browser.

Step 1: Understand the Page

read_page(tabId)              → Get accessibility tree with element refs (ref_1, ref_2, ...)
find(query, tabId)            → Natural language search for elements
get_page_text(tabId)          → Extract plain text (for reading articles)
computer(action="screenshot") → Visual capture when DOM isn't enough

Step 2: Interact

computer(action="left_click", ref="ref_1")     → Click by reference (preferred)
form_input(ref="ref_2", value="text")          → Fill form fields
computer(action="type", text="hello")          → Type text
computer(action="scroll", direction="down")    → Scroll
javascript_tool(text="...", tabId)             → Execute JS directly

Step 3: Verify

read_page(tabId)                               → Check new state
computer(action="screenshot")                   → Visual verification
read_console_messages(tabId)                   → Check for JS errors
read_network_requests(tabId, urlPattern="/api")→ Verify API calls

---

Overview (Pattern A: Script Injection)

This skill provides a mechanism to:

• Inject browser-skill scripts into any page (including HTTPS)
• Cache scripts in localStorage to avoid re-transmitting large files
• Version scripts via content hash for automatic cache invalidation

Key insight: HTTPS pages block HTTP resources (mixed content). We read scripts from the filesystem and inject via javascript_tool, which uses the Chrome extension's privileged APIs to bypass page security restrictions.

---

Architecture

┌─────────────────────────────────────────────────────────────────────┐
│                     Three Entry Points                               │
├────────────────────┬─────────────────────┬──────────────────────────┤
│  Claude Desktop    │   Claude Code CLI   │  Extension Side Panel    │
│  (incl. Cowork)    │   (claude --chrome) │  (Standalone)            │
│       │            │         │           │         │                │
│       └────────────┴─────────┴───────────┴─────────┘                │
│                              │                                       │
│                              ▼                                       │
│              ┌───────────────────────────────────┐                  │
│              │   Claude in Chrome Extension      │                  │
│              │      (claude-in-chrome MCP)       │                  │
│              └───────────────┬───────────────────┘                  │
│                              │                                       │
│  ┌───────────────────────────┼───────────────────────────────────┐  │
│  │                    Tool Surface                                │  │
│  │  ┌──────────────┐ ┌──────────────┐ ┌────────────────────────┐ │  │
│  │  │ read_page    │ │ computer     │ │ javascript_tool        │ │  │
│  │  │ find         │ │ form_input   │ │ (Pattern A injection)  │ │  │
│  │  │ get_page_text│ │ navigate     │ │                        │ │  │
│  │  └──────────────┘ └──────────────┘ └────────────────────────┘ │  │
│  │  ┌──────────────┐ ┌──────────────┐ ┌────────────────────────┐ │  │
│  │  │ tabs_context │ │ gif_creator  │ │ read_console_messages  │ │  │
│  │  │ tabs_create  │ │ upload_image │ │ read_network_requests  │ │  │
│  │  └──────────────┘ └──────────────┘ └────────────────────────┘ │  │
│  └───────────────────────────────────────────────────────────────┘  │
│                              │                                       │
│                              ▼                                       │
│  ┌───────────────────────────────────────────────────────────────┐  │
│  │              Browser Tab (any origin)                          │  │
│  │  ┌─────────────────┐    ┌──────────────────────────────────┐  │  │
│  │  │  localStorage   │    │  window.{SDK} (injected script)  │  │  │
│  │  │  (cached code)  │    │  e.g. ChatGPTWebSDK              │  │  │
│  │  └─────────────────┘    └──────────────────────────────────┘  │  │
│  └───────────────────────────────────────────────────────────────┘  │
└─────────────────────────────────────────────────────────────────────┘

Script Source (Pattern A): ~/.claude/use-browser-skills/{skill-name}/scripts/{script}.js

---

Prerequisites

Required for Chrome extension automation:

• Chrome extension: Claude in Chrome installed and connected
• Browser tab: Target page open in a tab within the MCP tab group
• Session started: Call tabs_context_mcp first to get valid tab IDs

For Claude Code CLI:

• Run with claude --chrome flag
• Verify connection with /chrome command
• Requires visible Chrome window (no headless mode)

For Cowork / Claude Desktop:

• Enable "Claude in Chrome" connector in Desktop settings
• Enable connector per conversation
• Model restriction: only works with Haiku 4.5, Sonnet 4.5, or Opus 4.5

For Pattern A (script injection) additionally:

• Browser sub-skill present in ~/.claude/use-browser-skills/{skill-name}/ with a scripts/ directory
• For Claude Cowork (VM): Use host-access skill to read files from host filesystem
• For Claude Code (native): Read files directly with the Read tool

---

Browser Sub-Skills

Browser sub-skills are assets consumed by this skill, not standalone skills. They live in a dedicated directory outside the skills tree:

~/.claude/use-browser-skills/
├── use-chatgpt-in-browser/
│   ├── SKILL.md              # Documents the SDK's API and workflows
│   └── scripts/
│       └── chatgpt-web-sdk.js
└── {your-skill}/
    ├── SKILL.md
    └── scripts/
        └── {script-name}.js

Each browser sub-skill has its own SKILL.md documenting:

• What the script does
• Available methods/APIs
• Usage workflows
• Quality bar and troubleshooting

Browser sub-skills are user/team-specific. They are NOT part of the shared skill and are not discovered by the skills CLI. This skill scans this directory as part of Pattern A.

Discovering available browser sub-skills

ls ~/.claude/use-browser-skills/

---

Pattern A Workflow: Script Injection

For injecting pre-built SDKs from ~/.claude/use-browser-skills/. For direct automation without injection, use the Core Workflow above.

Step 1: Identify the browser sub-skill to inject

List available sub-skills: ls ~/.claude/use-browser-skills/

Step 2: Read the sub-skill's SKILL.md

Load: ~/.claude/use-browser-skills/{skill-name}/SKILL.md

Understand:

• What global object the script creates
• Prerequisites (user logged in, specific page, etc.)
• Available methods

Step 3: Get browser context

// Use tabs_context_mcp to get available tabs
// Create a new tab if needed with tabs_create_mcp
// Navigate to target page with navigate tool

Step 4: Read script and compute hash

For Claude Code (native):

1. Read: ~/.claude/use-browser-skills/{skill-name}/scripts/{script}.js
2. Compute hash (first 8 chars of simple hash)

For Claude Cowork (VM):

1. Use host-access to read file via Command Relay
2. Compute hash from content

Hash function (JavaScript):

function simpleHash(str) {
  var hash = 0;
  for (var i = 0; i < str.length; i++) {
    hash = ((hash << 5) - hash) + str.charCodeAt(i);
    hash = hash & hash;
  }
  return Math.abs(hash).toString(16).substring(0, 8);
}

Step 5: Try loading from cache

Inject the minified loader via javascript_tool (replace {S} and {H}):

(function(s,h){var k="_skill_"+s,c=localStorage.getItem(k);if(c){try{var d=JSON.parse(c);if(d.version===h){eval(d.code);return{status:"cached",skill:s}}return{status:"stale",cached:d.version,expected:h}}catch(e){return{status:"error",msg:e.message}}}return{status:"not_cached"}})("{S}","{H}")

Returns: {status:"cached"|"stale"|"not_cached"|"error",...}

Step 6: If not cached, inject full script + store

If loader returns "stale" or "not_cached":

1. Inject the full script content via javascript_tool
2. Store in cache with minified store (replace {S}, {H}, {C}):

(function(s,h,c){try{localStorage.setItem("_skill_"+s,JSON.stringify({version:h,code:c}));return{status:"stored",skill:s,size:c.length}}catch(e){return{status:"error",msg:e.message}}})("{S}","{H}","{C}")

Note: {C} must be the script code with quotes escaped (replace " with \").

Step 7: Verify injection

typeof window.{GlobalObject} !== "undefined" ? "ready" : "not loaded"

Step 8: Use the skill's API

Follow the workflows documented in the skill's own SKILL.md.

---

Why This Works (HTTPS Bypass)

Approach	Works on HTTPS?	Why
<script src="http://localhost/...">	No	Mixed content blocked
fetch("http://localhost/...")	No	Mixed content blocked
javascript_tool with inline code	Yes	Extension injects directly via chrome.scripting.executeScript

The Chrome extension has elevated privileges. When we pass code to javascript_tool, the extension injects it directly into the page's JavaScript context. The browser sees this as extension-injected code, not as the page loading an external resource.

---

Caching Strategy

• Scripts cached in localStorage per-origin
• Key format: _skill_{skillName}
• Value: JSON with { version: hash, code: scriptContent }
• Cache invalidates when content hash changes

---

Quality Bar

Must have:

• Correct tab selected (in MCP tab group)
• Script file exists and is readable
• Global object created after injection
• No JavaScript errors in console

Should have:

• Cache utilized for repeat injections
• Hash matches current script version

---

Troubleshooting

Issue	Cause	Solution
"Tab not found"	Tab closed or wrong ID	Call tabs_context_mcp to refresh
"Invalid tab ID"	Didn't call tabs_context_mcp first	Always start with tabs_context_mcp
Extension not connecting	Claude Code not started with --chrome	Restart with claude --chrome
Connector not working	Wrong model or not enabled	Use Haiku/Sonnet/Opus 4.5, enable per conversation
Script not loading	JavaScript syntax error	Check console with read_console_messages
Cache stale	Script was updated	Loader auto-detects via hash mismatch
"undefined" after injection	Wrong global object name	Check skill's SKILL.md for correct name
Mixed content error	Using HTTP URL approach	Use inline injection (this skill's approach)
Element ref not found	Page changed since read_page	Re-run read_page to get fresh refs
Click not working	Using coordinates on dynamic page	Prefer ref-based clicks over coordinates
Chrome extension not available	Extension not installed or not connected	Fall back to /eng:browser (Playwright) for the task

---

Quick Reference: Minified Templates

Loader (~185 chars)

(function(s,h){var k="_skill_"+s,c=localStorage.getItem(k);if(c){try{var d=JSON.parse(c);if(d.version===h){eval(d.code);return{status:"cached",skill:s}}return{status:"stale",cached:d.version,expected:h}}catch(e){return{status:"error",msg:e.message}}}return{status:"not_cached"}})("SKILL","HASH")

Store (~175 chars + code)

(function(s,h,c){try{localStorage.setItem("_skill_"+s,JSON.stringify({version:h,code:c}));return{status:"stored",skill:s,size:c.length}}catch(e){return{status:"error",msg:e.message}}})("SKILL","HASH","CODE")

Hash function (~130 chars)

(function(t){var h=0,i=0;for(;i<t.length;i++){h=((h<<5)-h)+t.charCodeAt(i);h=h&h}return Math.abs(h).toString(16).slice(0,8)})("INPUT")

Verify injection

typeof window.SDK !== "undefined"

---

When to Load References

Situation	Load
Core browser automation
Need tool parameter details	references/chrome-extension-tools.md
Choosing interaction approach (refs vs JS vs coords)	references/interaction-patterns.md
Batch/repeatable automation	references/batch-automation.md
Debugging with console/network	references/debugging.md
Pattern A: Script injection
Need cache loader/store templates	scripts/loader.js, scripts/store.js
Debugging cache issues	references/caching-strategy.md
Writing injectable scripts (ES5 syntax)	references/javascript-constraints.md
Browser-skills (sub-skills)
Add your own browser sub-skills	Create a directory in ~/.claude/use-browser-skills/{your-skill}/ with SKILL.md + scripts/