You’ve probably heard of robots.txt — the file that tells search engine crawlers what they can and can’t access. An llms.txt file serves a similar purpose, but specifically for AI systems. And while almost every website has a robots.txt, less than 2% have an llms.txt.
What exactly is llms.txt?
An llms.txt file is a plain text file deployed at yourdomain.com/llms.txt. It contains a curated list of your most important pages with brief descriptions. Think of it as a table of contents for AI — a structured guide that tells AI crawlers “here are the pages that matter most.”
What should an llms.txt file include?
Your llms.txt should list 10 to 30 of your most important pages. For a typical local service business: your homepage, each service page, your about page, your contact page, and your best blog posts. Each entry should include the URL and a one-line description.
How does llms.txt improve AI visibility?
In our scoring methodology, the absence of an llms.txt file imposes a hard ceiling on semantic structure scores. No site without llms.txt can score above 30 out of 100 on that dimension, regardless of how good the content is.
How do I create one?
Creating an llms.txt file takes less than an hour. Open a text editor, list your key pages with URLs and descriptions, save it as “llms.txt,” and upload it to your website’s root directory. No code changes, no plugins, no developer required.