What it is
A Markdown-formatted text file at /llms.txt listing your main sections and key pages, each with a short factual description. The idea is that parsed HTML is noisy and a curated summary gives you some editorial control over how your site is understood.
That is the whole proposition. It is a map, not a lever.
Being accurate about its status
It is a proposal that gained traction quickly, not a standard any major model provider has committed to honouring as specified. Some crawlers may use it, some ignore it entirely.
This matters because a lot of Indian agency marketing currently presents llms.txt as an AEO service with a meaningful fee attached. The implementation is a generated file. The value is real and small.
The failure mode nobody mentions
A file published once and never updated becomes actively misleading. It lists pages that no longer exist and omits half the site added since, which describes you worse than having no file at all.
Generate it from the same data that produces your sitemap so it cannot drift. On this site it is produced by a PHP script that reads the same constants the sitemap uses, which means adding a page updates both automatically.
What to put in it
- A one-line description of what the business actually does
- Main sections with factual one-line descriptions, not marketing copy
- Key pages a model would benefit from knowing exist
- Valid Markdown link syntax, since malformed links get flagged by accessibility and PageSpeed audits
The thing that actually matters alongside it
Check robots.txt permits the AI crawlers you want citing you. A site that publishes a careful llms.txt while blocking the crawlers in robots.txt has done the decorative half of the job and skipped the functional half.
Key takeaways
- Worth implementing, not worth a significant fee
- Generate it from your sitemap data so it never drifts out of date
- Check robots.txt actually permits the crawlers you want