geo-crawlers - Analyze AI crawler access to websites
Analyze robots.txt, meta tags, and HTTP headers to map AI crawler access and provide visibility recommendations.
Tags
Updated: 2026-10-03Capabilities
Typical Inputs
Typical Outputs
What this skill does
- Inspect robots.txt rules
- Inspect meta tags
- Inspect HTTP headers
- Determine crawler access
- Map crawler access
- Provide visibility recommendations
Inputs
- Website URL
- Website content
Outputs
- AI crawler access map
- Visibility recommendations