Skill

claude-code-computer-use

Runs bounded UI automation loops in Claude Code using screenshot→decide→act cycles with click/type/scroll tools, safety guardrails, and step limits for desktop/browser control.

Bash

Anthropic

automation

Install

npx claudepluginhub jleechanorg/claude-commands

Tool Access

This skill uses the workspace's default tool permissions.

Preview

Run Claude Code as a bounded UI-control agent that observes screenshots, takes one action, re-checks state, and repeats until success or stop conditions.

SKILL.md

Similar Skills

kotlin-ktor-patterns

Provides Ktor server patterns for routing DSL, plugins (auth, CORS, serialization), Koin DI, WebSockets, services, and testApplication testing.

everything-claude-code

163.2k

deep-research

Conducts multi-source web research with firecrawl and exa MCPs: searches, scrapes pages, synthesizes cited reports. For deep dives, competitive analysis, tech evaluations, or due diligence.

everything-claude-code

163.2k

inventory-demand-planning

Provides demand forecasting, safety stock optimization, replenishment planning, and promotional lift estimation for multi-location retailers managing 300-800 SKUs.

everything-claude-code

163.2k

Stats

Stars26

Forks4

Last CommitMar 24, 2026

Actions

View Source View Plugin View on GitHub View README

Claude Code Computer Use

Run Claude Code as a bounded UI-control agent that observes screenshots, takes one action, re-checks state, and repeats until success or stop conditions.

Required loop

Restate the user goal in one sentence.
Define a max step budget (default 20).
For each step, do exactly one UI action.
After each action, capture a fresh screenshot.
Verify expected change before continuing.
Stop on success, ambiguity, or budget exhaustion.

Tool contract to enforce

Require these tool primitives (or closest equivalents):

screenshot()
click(x,y) / double_click(x,y)
type(text) / key(combo)
scroll(delta)
Optional: ocr(region) / accessibility_tree()

Never take multiple UI actions in a single reasoning step.

Safety policy

Allowlist app/site scope before starting.
Ask before destructive actions (delete, send, purchase, publish, submit).
If target is unclear for 2 consecutive steps, pause and ask human.
Log every step with: intention, action, result.

Claude Code invocation pattern

Use non-interactive Claude Code mode for deterministic logs:

claude --print --permission-mode bypassPermissions "<task prompt>"

Task prompt template:

You are controlling a UI via tools.
Goal: <goal>
Allowed scope: <apps/sites>
Max steps: <N>
Rules:
- One action per step.
- After each action, screenshot and verify.
- If uncertain for 2 steps, stop and ask.
- Ask before destructive actions.
Return a step log and final status: success | blocked | needs-human.

Completion criteria

Declare completion only when:

Goal state is visible in screenshot evidence, or
You are blocked and provide the exact blocker + required human input.

Always return concise evidence:

Final status
Last 3 step logs
Screenshot/file references (if available)