mac-use - Control macOS GUI Apps Visually
Take screenshots, detect UI elements, click, scroll, type, and press keys to control macOS applications through their graphical interface.
Tags
Updated: 2026-05-28Capabilities
Typical Inputs
Typical Outputs
What this skill does
- capture screenshot
- detect text elements
- annotate image
- click numbered element
- click at coordinates
- scroll window
- type text
- press key
- list windows
Inputs
- target app name
- window ID
- element number
- coordinates
- text to type
- key combination
Outputs
- annotated screenshot
- element list
- window list
- click confirmation
Requirements
- macOS platform
- Apple Vision framework
- python3
- screencapture utility