mac-use - Control macOS GUI apps visually
Control macOS GUI apps via screenshots, OCR, and mouse/keyboard actions
Tags
Updated: 2026-05-28Capabilities
Typical Inputs
Typical Outputs
What this skill does
- list windows
- capture screenshot
- click element
- click coordinates
- scroll window
- type text
- press key
Inputs
- app name
- window ID
- text content
- key combination
- scroll direction
- scroll amount
Outputs
- annotated screenshot image
- element list JSON
- window list JSON
- click confirmation JSON
Requirements
- macOS
- Python 3
- Apple Vision framework
