peekaboo-cli

Peekaboo CLI

macOS UI automation command-line tool providing complete desktop control capabilities.

Installation

Copy this skill to your agent's skills directory:

# Claude Code
mkdir -p ~/.claude/skills/peekaboo-cli
cp -r . ~/.claude/skills/peekaboo-cli/

# OpenClaw
mkdir -p ~/.openclaw/skills/peekaboo-cli
cp -r . ~/.openclaw/skills/peekaboo-cli/

Prerequisites

• Peekaboo CLI installed (brew install steipete/tap/peekaboo)
• macOS Screen Recording and Accessibility permissions granted

Pre-execution Validation

Before executing any CLI commands, verify permission status first.

1. Check Permission Status

peekaboo permissions status

2. Grant Permissions (if needed)

If permissions are not granted, guide the user to enable them:

Peekaboo requires the following macOS permissions:
- Screen Recording: For capturing UI screenshots
- Accessibility: For controlling UI elements

System Settings → Privacy & Security → Screen Recording/Accessibility → Add Terminal app

Then verify:

peekaboo permissions grant

3. Verify Configuration

peekaboo list apps

Quick Start

# List running applications
peekaboo list apps

# Capture current window UI with element annotations
peekaboo see --annotate --path /tmp/ui.png

# Click specific element (ID from see command)
peekaboo click --on elem_42

# Type text
peekaboo type "Hello World" --return

Core Commands

UI Capture & Analysis

# Capture UI element map (recommended before click/type)
peekaboo see --json --annotate --path /tmp/ui.png

# Target specific app/window
peekaboo see --app "Safari" --window-title "GitHub"

# Capture menubar popovers
peekaboo see --menubar

# Capture entire screen
peekaboo see --mode screen

Element Interaction

# Click element by ID
peekaboo click --on elem_42
peekaboo click "Submit" --wait-for 8000

# Coordinate-based click
peekaboo click --coords 100,200

# Double-click / Right-click
peekaboo click --on elem_42 --double
peekaboo click --coords 100,200 --right

# Type text
peekaboo type "user@example.com" --return
peekaboo type --clear "new text"  # Clear field first

# Key presses and hotkeys
peekaboo press return
peekaboo hotkey "cmd,w"   # Cmd+W

App Management

# Launch app
peekaboo app launch "Safari"
peekaboo app launch "Xcode" --open ~/Projects/MyApp.xcodeproj

# Quit app
peekaboo app quit --app "Safari"
peekaboo app quit --all --except "Finder,Terminal"

# Switch focus
peekaboo app switch --to Safari --verify

Window Management

# List windows
peekaboo list windows --app "Safari" --include-details bounds,ids

# Move/Resize window
peekaboo window move --app Safari -x 100 -y 100
peekaboo window resize --app Safari -w 1200 --height 800

# Focus/Close/Minimize/Maximize
peekaboo window focus --app Terminal --space-switch
peekaboo window close --app Safari

Menu Operations

# Click menu item
peekaboo menu click --app Safari --path "File,New Tab"

# List menubar status items
peekaboo list menubar

Scroll & Drag

# Scroll
peekaboo scroll --direction down --amount 3

# Drag
peekaboo drag --from 100,100 --to 300,300

# Move mouse
peekaboo move --coords 500,500

Common Workflows

Form Filling

# 1. Capture UI to get element IDs
peekaboo see --json --annotate --path /tmp/form.png

# 2. Click input field and type
peekaboo click --on elem_10
peekaboo type "username" --tab
peekaboo type "password" --return

App Automation

# 1. Launch app
peekaboo app launch "Safari" --wait-until-ready

# 2. Wait and perform actions
peekaboo sleep 2
peekaboo hotkey "cmd,l"  # Focus address bar
peekaboo type "https://github.com" --return

Output Format

Most commands support --json output:

peekaboo see --json | jq '.data.ui_elements[] | {id, label, role}'
peekaboo list apps --json | jq '.data[] | select(.frontmost == true)'

Troubleshooting

Permission Errors

peekaboo permissions status
peekaboo permissions grant

Element Not Found

1. Re-capture UI: peekaboo see --json
2. Use --wait-for to increase wait time
3. Use --coords for coordinate-based targeting

Focus Issues

1. Use --space-switch for cross-Space windows
2. Use peekaboo app switch --to <app> --verify to confirm focus
3. Use --no-auto-focus when already focused manually

Detailed Documentation

• UI Capture references/see.md
• Element Click references/click.md
• Text Input references/type.md
• Key Press references/press.md
• Hotkey references/hotkey.md
• App Management references/app.md
• Window Management references/window.md
• Menu Operations references/menu.md
• Menubar references/menubar.md
• Scroll references/scroll.md
• Drag references/drag.md
• Mouse Move references/move.md
• Swipe Gesture references/swipe.md
• Long-running Capture references/capture.md
• Image Processing references/image.md
• Autonomous Agent references/agent.md
• MCP Server references/mcp.md
• Config Management references/config.md
• Permissions references/permissions.md
• Clipboard references/clipboard.md
• List Queries references/list.md
• Open Files references/open.md
• Sleep references/sleep.md
• Space Management references/space.md
• Dialog references/dialog.md
• Dock references/dock.md
• Clean Cache references/clean.md
• Bridge Service references/bridge.md
• Daemon references/daemon.md
• Tools references/tools.md
• Learn references/learn.md
• Run references/run.md
• MCP Capture Meta references/mcp-capture-meta.md
• Visualizer references/visualizer.md