In short. llms.txt is a proposed Markdown convention for presenting selected site resources. It replaces neither robots.txt nor a sitemap, and use by search engines or assistants is not guaranteed. Treat it as a complementary editorial document that must remain accurate and maintainable.
llms.txt was proposed in September 2024 as a readable entry point for large language models. The format can help a team gather its reference pages in one document. It is neither a crawl directive nor a proven method for gaining visibility or citations.
What is llms.txt?
An llms.txt file is commonly published at https://example.com/llms.txt. Written in Markdown, it can include an organisation name, a short description, sections and links to important pages. The project describes the convention as a way to make site information easier to use in LLM contexts. It is not a W3C or IETF standard.
llms.txt, robots.txt and sitemap.xml have different jobs
These files do not solve the same problem. Confusing them creates needless work and unrealistic expectations.
| File | Main role | Important limit |
|---|---|---|
| robots.txt | Declare access rules for user agents. | It can also address AI crawlers. A rule covering /llms.txt can block that path. |
| sitemap.xml | Signal URLs for discovery. | A sitemap need not be exhaustive. One sitemap file can contain up to 50,000 URLs. |
| llms.txt | Present an editorial selection of resources. | No crawling, indexing or citation outcome is guaranteed. |
What is known about use by AI systems
Publishing a file does not prove that a crawler, search engine or assistant has read it. Keep three things separate: the file exists at an accessible URL, it is integrated into documentation or a tool, and a third-party system actually uses it. Public statements and available tests do not justify claiming that ClaudeBot, GPTBot or PerplexityBot systematically consume llms.txt.
Google states in its guide to optimisation for Search and AI experiences that Google Search does not use llms.txt, including for its generative features. That does not establish what other services do. It does rule out treating llms.txt as a requirement for appearing in Google.
Create a useful file without overclaiming
Start with information you can verify and maintain: official name, short description, scope, product pages, documentation, help, contact and editorial resources. Keep canonical links, test them and remove deleted pages. Link count, section order and review cadence are editorial choices. There is no official numeric threshold or mandatory quarterly review.
A simple structure is # Organisation name, one positioning sentence, then ## Documentation, ## Products and ## Resources. A short factual note can accompany every link. That note primarily helps people reviewing the file and tools that display it; it does not prove a ranking effect.
Publishing: verify the URL that is actually served
The result to check is an accessible HTTP response at /llms.txt. For a static site, the file must be in the published directory. In Flask or Django, putting it in a static folder is not always enough: static-file configuration or a dedicated route must actually serve that URL. Also check that robots rules and access controls do not cover the path by mistake.
The useful check is straightforward: open the public URL and inspect the HTTP status, content type, links and any unexpected redirect. This confirms document availability, not third-party use.
A localised example: select pages that exist
An example should use URLs consistent with the served language. For AI Labs Audit, an English section could link to the REST API tutorial, pricing, the FAQ and this guide. Other language versions should point to their localised routes, rather than to URLs that redirect to French.
Two documented reference points show the difference between a selection and an inventory: on 31 October 2026, the AI Labs Audit Observatory had 10 published studies in 7 languages. Its public REST API tutorial presents more than 100 endpoints. These points describe published resources, not an effect caused by llms.txt.
Limits and an editorial decision
Adopt llms.txt if you have a stable selection of pages, a clear maintenance owner and a publicly verifiable route. Do not do so at the expense of basics: accessible pages, accurate content, clear navigation and maintained sources. Schema.org and experience or expertise signals can have their own uses, but together with llms.txt they are not a proven GEO lever.
The useful question is not “Will this file generate citations?” It is “Does this document help our team publish a faithful map of our essential resources without creating stale information?”
Frequently asked questions about llms.txt
What is the llms.txt file?
llms.txt is a proposed Markdown convention, usually published at a site root, that presents selected information and links. It does not replace robots.txt or a sitemap, and use by AI systems is not guaranteed.
How do I create an llms.txt file?
Create a Markdown file named llms.txt, publish it at an accessible URL, and describe only verifiable pages and information. Then check the HTTP response and the links actually served.
Is llms.txt mandatory for AEO?
No. llms.txt is optional. It can be a complementary editorial document, but no evidence supports presenting it as a condition for visibility or citations.
What information should I include in llms.txt?
You can include the official name, a short description, products or services, contact pages, documentation and other important links, provided they are accurate and maintained.
How often should I update llms.txt?
Update it when its information or links change. The cadence depends on your publishing rhythm; there is no official schedule to follow.
Questions and answers
What is the llms.txt file?
llms.txt is a text file placed at the root of your site that provides AI models with structured information about your business: description, products, services, contact details. It's the robots.txt for AI.
How do I create an llms.txt file?
Create a text file named llms.txt at your site root containing: company name, description, list of products/services, contact information, and links to your key pages.
Is llms.txt mandatory for AEO?
No, it's not mandatory but strongly recommended. It facilitates AI access to your essential information and can improve the accuracy of citations about your business.
What information should I include in llms.txt?
Include: official company name, industry sector, one-sentence description, list of main products/services, geographic coverage area, and links to important pages (about, contact, products).
How often should I update llms.txt?
Update llms.txt whenever your key information changes: new product launch, description update, new contact details. An outdated file can generate inaccurate citations.
Agencies, consultants and in-house teams: compare the answers generated for your own scenarios, languages and selected models. Our infrastructure, your scenarios, your expertise.