llms.txt

AI assistants answer questions by reading web pages, and cluttered pages are harder to read correctly. A tidy summary of the key pages could help a model describe a business accurately. The honest status is that this is an early, unofficial standard. Major AI providers have not publicly committed to reading it, and evidence that it changes how often a site is cited is thin. Treat it as a cheap precaution, not a ranking lever.

• Write a short summary of what the business does and who it serves.
• List only the pages worth reading closely, such as product, pricing, key guides and policies, with one line each.
• Keep the file current. An out-of-date index misleads a model more than none at all.
• Some sites also publish a longer llms-full.txt with the full text of key pages. This is optional.
• Check server logs now and then to see whether AI crawlers actually request the file.

An llms.txt file lives at an address such as yoursite.com/llms.txt. It was proposed in September 2024 by Jeremy Howard of Answer.AI. It is written in markdown, so both people and AI models can read it. A typical file has a title, a one-paragraph summary of the business, and short sections of links to the pages that matter most, each with a one-line note on what it contains.

It is easy to confuse with robots.txt, but the two do different jobs. Robots.txt gives crawlers rules about where they may go. An llms.txt file gives no rules. It is a reading list, a clean map for a model that would otherwise have to wade through navigation menus, cookie banners and scripts.

Dumping every URL on the site into the file, which defeats the purpose. Expecting a visible jump in AI citations. Treating it as a replacement for clear content and structured data on the pages themselves.