How to Audit Your llms.txt File: A Practical Checklist

Correction — 31 October 2026: an earlier version presented an original study without supporting records. Those sample sizes, findings and dataset promises have been removed. This is now a practical guide, not an empirical research report.

What does llms.txt do?

llms.txt is a proposal for providing a concise, machine-readable overview of website content. It is not an access-control file, a replacement for robots.txt, or proof that a provider will use your pages. The proposal can evolve: check its current format before editing your file.

Decide whether the file solves a real problem

Start with a reader task: finding your API documentation, understanding a product, or locating your most useful guides. If your navigation already solves the task and maintenance time is limited, prioritise clear pages and working links. A manifest cannot repair inaccessible or inaccurate content.

Check the response and the content

Open /llms.txt and confirm it returns the intended text, not a login screen, error page or HTML fallback. Record the URL, date, HTTP status and any redirect. Compare its heading, overview and grouped links with the current proposal; do not describe a format check as certification by an AI provider.

Inspect every included link

Follow each link. Remove broken or retired destinations, replace unnecessary redirects with the final URL, and keep descriptions factual. Avoid listing private dashboards or customer records. A link in this file is public information, not a permission boundary. Make sure the destination explains the topic in readable text.

Curate instead of exporting your entire sitemap

Group destinations by what someone needs: documentation, product information, support or reference material. Explain why each link matters. Keep one maintained source for the list and assign an owner. When a page moves or its purpose changes, review both the page and its entry.

Measure usefulness without inventing attribution

Use server logs to see whether the file was requested, while recognising that a request does not prove the content influenced an answer. Record actual cited URLs separately. Do not infer an AI-citation increase from publishing the file alone; other site changes and answer variability can explain movement.

Your audit record

Keep a worksheet with file URL, checked date, status, format notes, broken destinations, owner and next action. Recheck after navigation or documentation changes. Google says no special AI files or special schema are required for its AI features; keep normal SEO fundamentals ahead of this optional task.

Sources and further reading

llms.txt proposal

Google guidance on AI features