llms.txt Explained: The New robots.txt for AI Engines (2026 Guide)

llms.txt is the robots.txt of the answer-engine era. It's a simple Markdown file at the root of your domain that tells large language models which pages on your site are canonical, structured, and safe to cite. Adoption is still early — but the upside is asymmetric, and the cost is one file.

What is llms.txt?

llms.txt is a proposed open standard, served at /llms.txt, that gives LLM-powered crawlers a curated map of your most authoritative content. Unlike robots.txt (which blocks or allows), llms.txt actively recommends — listing your pillar pages, documentation, and key resources in a clean, parseable Markdown format.

Does ChatGPT actually read llms.txt?

Not every model honors it yet. But Anthropic, Perplexity, and several emerging AI search startups have signalled support, and the trajectory mirrors how robots.txt and sitemap.xml became universally honored within a year of becoming useful. Shipping one now is cheap insurance and a small ranking-style signal of being a serious, AI-aware publisher.

What goes in a good llms.txt

A short brand description, a link to your most important pillar pages and pricing/about, a list of canonical product or service URLs, and (optionally) links to API docs or open data. Keep it under 100 lines. Mirror the structure of your sitemap but curate aggressively — only your strongest content belongs here.

Where llms.txt fits in your AEO program

Think of it as one of five surfaces AI crawlers read: robots.txt (allow GPTBot, PerplexityBot, ClaudeBot), sitemap.xml (full URL set with lastmod), llms.txt (curated highlights), JSON-LD schema (per-page semantics), and your content itself. Together they form the AEO foundation. The full stack is covered in the pillar: "What is AEO? How to Get Cited by ChatGPT in 2026."

Frequently Asked Questions

Is llms.txt an official standard?

It's a community proposal, not yet an IETF or W3C standard. But it has growing momentum and several major AI companies have indicated they're crawling it. Treat it like sitemap.xml in 2005 — early but obvious.

Should llms.txt replace robots.txt?

No. They're complementary. robots.txt controls access; llms.txt curates and recommends. Ship both.

Where do I put llms.txt?

At the root of your domain — yourdomain.com/llms.txt — exactly like robots.txt. It must be plain text or Markdown and served with a 200 status.