mac-use - Control macOS GUI Applications Visually
Control macOS GUI applications through visual interaction using screenshots, clicks, scrolling, and typing
Tags
Updated: 2026-05-28Capabilities
Typical Inputs
Typical Outputs
What this skill does
- list visible windows
- capture screenshot
- detect text elements
- annotate image
- click element
- click coordinates
- scroll window
- type text
- press key
Inputs
- target application
- window coordinates
- text input
- key combination
Outputs
- annotated screenshot
- element list
- window list
Requirements
- macOS
- Python 3
- Apple Vision framework