Overview
The Technical category checks the technical foundations required for AI crawlers to reach and understand your content.
The Checks
A1: robots.txt exists
Checks that robots.txt exists and is accessible.
A2: AI crawlers allowed
Verifies that GPTBot, anthropic-ai, PerplexityBot, etc. are not blocked.
A3: HTTPS enabled
Ensures the site uses SSL/TLS encryption.
A4: Sitemap.xml exists
Checks that a sitemap exists and is valid.
A5: Canonical URLs
Verifies correct use of canonical tags.
A6: Load time
Measures page load time (goal: under 3 seconds).
A7: Mobile-responsive
Checks that the page works on mobile devices.
A8: No JavaScript dependencies
Verifies that main content doesn't require JavaScript.
A9-A14: Additional checks
Includes meta tags, structured errors, redirect chains, etc.
Common Issues
- Blocked crawlers - Update robots.txt
- Slow loading - Optimize images and caching
- JavaScript rendering - Use server-side rendering