llms.txt Explained: The New robots.txt for AI Engines (2026 Guide)
What llms.txt is, why ChatGPT and Perplexity care, and exactly what to put in yours — with a copy-paste template you can ship today.
llms.txt is the robots.txt of the answer-engine era. It's a simple Markdown file at the root of your domain that tells large language models which pages on your site are canonical, structured, and safe to cite. Adoption is still early — but the upside is asymmetric, and the cost is one file.
What is llms.txt?
llms.txt is a proposed open standard, served at /llms.txt, that gives LLM-powered crawlers a curated map of your most authoritative content. Unlike robots.txt (which blocks or allows), llms.txt actively recommends — listing your pillar pages, documentation, and key resources in a clean, parseable Markdown format.
Does ChatGPT actually read llms.txt?
Not every model honors it yet. But Anthropic, Perplexity, and several emerging AI search startups have signalled support, and the trajectory mirrors how robots.txt and sitemap.xml became universally honored within a year of becoming useful. Shipping one now is cheap insurance and a small ranking-style signal of being a serious, AI-aware publisher.
What goes in a good llms.txt
A short brand description, a link to your most important pillar pages and pricing/about, a list of canonical product or service URLs, and (optionally) links to API docs or open data. Keep it under 100 lines. Mirror the structure of your sitemap but curate aggressively — only your strongest content belongs here.
Where llms.txt fits in your AEO program
Think of it as one of five surfaces AI crawlers read: robots.txt (allow GPTBot, PerplexityBot, ClaudeBot), sitemap.xml (full URL set with lastmod), llms.txt (curated highlights), JSON-LD schema (per-page semantics), and your content itself. Together they form the AEO foundation. The full stack is covered in the pillar: "What is AEO? How to Get Cited by ChatGPT in 2026."
Frequently asked
It's a community proposal, not yet an IETF or W3C standard. But it has growing momentum and several major AI companies have indicated they're crawling it. Treat it like sitemap.xml in 2005 — early but obvious.
No. They're complementary. robots.txt controls access; llms.txt curates and recommends. Ship both.
At the root of your domain — yourdomain.com/llms.txt — exactly like robots.txt. It must be plain text or Markdown and served with a 200 status.
Want to be cited by ChatGPT, Perplexity & Gemini?
I run a dedicated AEO & GEO program for brands serious about AI search visibility — entity SEO, schema, and citation-worthy content, shipped end-to-end.
See the AEO & GEO serviceContinue reading the AEO cluster
Start with the pillar: What is AEO? How to Get Cited by ChatGPT in 2026. Then keep going below.