Peekaboo: Full macOS UI Automation from the Command Line

16,362 downloads, 823 installs, 44 stars. Peekaboo by @steipete is the most-installed Apple ecosystem skill on ClawHub — and for good reason. It wraps the Peekaboo CLI into an OpenClaw skill, giving Claude complete programmatic access to the macOS user interface: screenshots, element inspection, clicks, typing, keyboard shortcuts, window management, and more.

If you've ever wanted your AI agent to actually use your Mac the way you do — not just write scripts, but interact with any application's UI — this is the skill that makes it possible.

The Problem It Solves

Most AI agent tools can read files, call APIs, and run terminal commands. But the macOS UI is largely opaque to them. Applications with no public API, dialog boxes that pop up unexpectedly, settings panels buried in system preferences, legacy software with no CLI — none of these are accessible to agents without a dedicated UI automation layer.

Peekaboo fills that gap. Instead of requiring every application to expose an API, it works at the macOS accessibility and screen capture layer — the same layer used by assistive technology. Any app that supports macOS accessibility (which is most of them) becomes scriptable.

How It Works

The skill requires the Peekaboo CLI to be installed and running. Once installed, Claude can invoke any Peekaboo command through the skill to capture the screen state, inspect what UI elements are present, and then interact with them.

The architecture has two phases:

1. Observation — capture the current screen/window state, list available UI elements
2. Interaction — click, type, scroll, drag, or press keyboard shortcuts on those elements

All commands support --json output for scripting and can share a snapshot cache for efficiency.

Core Commands

Capturing Screen State

# Screenshot of the entire screen
peekaboo image --output screen.png
 
# Screenshot of a specific window
peekaboo image --app Safari --window-title "GitHub"
 
# List all running apps and their windows
peekaboo list apps
 
# List UI elements in the frontmost window
peekaboo list windows

Inspecting UI Elements

# List interactive elements (buttons, links, inputs)
peekaboo list elements --app "Finder"
 
# Check accessibility permissions
peekaboo permissions

Interacting with the UI

# Click by element ID or natural language query
peekaboo click --id "submit-button"
peekaboo click --query "Save button"
 
# Type text into the focused element
peekaboo type "Hello, world!"
peekaboo type --clear "Replacing existing text"
 
# Keyboard shortcuts
peekaboo hotkey cmd,s           # ⌘S (save)
peekaboo hotkey cmd,shift,t     # ⌘⇧T
peekaboo hotkey escape
 
# Scroll within an element
peekaboo scroll --direction down --target "main-content"
 
# Drag between elements or coordinates
peekaboo drag --from "source" --to "destination"

App and Window Management

# Launch an application
peekaboo run --app "TextEdit"
 
# Bring an app to foreground
peekaboo list apps   # see running apps first
 
# Clipboard operations
peekaboo paste "Text to paste into current app"

Scripting with JSON Output

All commands support structured output for piping into scripts:

peekaboo list elements --app "Safari" --json
# Returns: [{id: "...", role: "button", label: "...", frame: {...}}, ...]

Installation

clawhub install peekaboo

Peekaboo CLI must be installed separately on macOS:

brew install steipete/tap/peekaboo

Then grant the necessary macOS permissions:

• Screen Recording — for screenshots and screen capture
• Accessibility — for UI element inspection and interaction

peekaboo permissions   # shows current permission status

Practical Use Cases

macOS app QA automation. Instead of manual click-testing after each build, write a Peekaboo script that Claude runs to verify critical UI paths. The --json output lets you assert on element states programmatically.

IT administration tasks. Configure system settings, install certificates, manage users — all via Claude, without requiring custom scripts for each app. Just describe what you want done and let the agent navigate the UI.

Legacy app integration. Old software with no API and no CLI becomes accessible. Claude can read data from legacy desktop apps by screenshotting, extracting text, and interacting with controls.

Tutorial generation. Have Claude walk through a multi-step workflow in any app while capturing screenshots at each step. Combine with peekaboo capture for live sequences.

Form automation. Repetitive data entry in any desktop application — CRM systems, local utilities, forms in apps — driven by Claude with typed input and tab navigation.

Comparison with Similar Skills

Peekaboo is the right tool when you need to interact with native macOS applications — apps built for macOS where accessibility APIs are available. For pure browser automation, use browser-automation. For operating any screen pixel regardless of accessibility support, use desktop-control.

Considerations

• macOS 15 (Sequoia) or later required. The skill uses ScreenCaptureKit and modern Accessibility APIs that require macOS 15+. If you're on an older macOS version, you'll need to upgrade before using Peekaboo.
• Accessibility permissions required. Without granting Screen Recording and Accessibility, the skill cannot function. These are system-level permissions that require user approval.
• Fragile on UI changes. If an app updates and changes its UI, element IDs and positions may shift. Queries like --query "Save button" are more resilient than hard-coded IDs.
• Headless environments don't work. Peekaboo requires a live macOS desktop session. CI/CD use requires a real or virtual display.
• Session persistence. Some interactions require the app to stay in the foreground. Multi-step workflows should include explicit waits or checks between steps.

The Bigger Picture

The most useful thing an AI agent can do is work in the same environment that humans work in. Humans use desktop applications. They navigate UIs. They click buttons and fill forms in apps that have no API. Peekaboo is a key component of making Claude a genuine desktop collaborator rather than a terminal-only tool.

The 823 installs (the highest absolute install count of any skill in this batch) suggests developers are keeping it in their standard toolkit — not just evaluating it, but committing to it as an ongoing part of their workflow. For macOS-centric development, it's hard to see why you'd work without it.

View the skill on ClawHub: peekaboo