Missing llms.txt
No /llms.txt file for AI crawlers
Code: missing_llms_txt
Why it matters
No /llms.txt file for AI crawlers. AI systems are increasingly how people discover information. Sites that make it easy for AI crawlers to access and attribute content gain visibility in AI-generated answers.
How to fix
Create /llms.txt in your site root
http# yourdomain.com
> Brief description of your site and what it offers.
## Docs
- [Main Page](https://yourdomain.com/): Overview of the site
- [About](https://yourdomain.com/about): About the organizationRelated checks in AI Readiness
AI crawlers blocked
robots.txt blocks AI bots
AI crawler access matrix
Per-bot allow/block status across the 14 major AI crawlers (GPTBot, OAI-SearchBot, ClaudeBot, Claude-User, PerplexityBot, Google-Extended, Bytespider, etc.)
llms.txt missing H1
/llms.txt exists but lacks the `# Project Name` top-level heading required by llmstxt.org
llms.txt has no sections or links
/llms.txt is missing `## Section` headings and curated markdown links — AI crawlers have nothing to ingest
llms.txt links are broken
URLs listed in /llms.txt return 4xx/5xx — wastes AI crawler budget and risks bad citations
Run a free scan to check your site
Get a complete audit in under 2 minutes. No account required.
Start Free Scan