llms.txt is a proposal, not an official standard. The idea is simple: you give AI systems and agents a short, curated summary of the most important content on your site. Useful to know about, but at this point not a measure that demonstrably increases your visibility.
Short answer: what you need to know about llms.txt
An llms.txt is an optional Markdown file that acts as a signpost for language models and AI agents. The proposal comes from Jeremy Howard and was published on 3 September 2024 and has been at version 2 since 10 August 2026. The document consistently calls itself a proposal, not a standard.
So stay level-headed about it. The file is cheap to make and it does no harm if you maintain it. But it is no substitute for technical SEO, Google Search does not use it and it is not a fast route to visibility in AI answers.
We don’t build pages, we build logic.
Giacomo Perticara, founder and SEO strategist at GRP Digital
That logic starts with crawlability, clear source pages, internal links and measurability.
What is an llms.txt file?
An llms.txt is a machine-readable summary in Markdown. You bundle the most important pages of a site or section into it, with a short note for each link. Think of it as a table of contents with explanation, not as a complete index.
That distinction is the whole point. A sitemap.xml gives an overview of all indexable URLs. A robots.txt gives guidelines about access for automated systems. llms.txt tries something else: giving context about which pages really matter and why.
Just as we go through the table of contents of a manual, LLMs & search engines always scan your sitemap and robots.txt first to get an overall picture.
Giacomo Perticara, founder and SEO strategist at GRP Digital
That is exactly why an extra file rarely makes the difference. If your structure is already readable, llms.txt adds little. If your structure is unreadable, a separate small file does not repair that.
In practice llms.txt is often made out to be bigger than it is. Treat it as an additional convention within broader AI visibility, not as a separate channel with its own rules.
Where do you put llms.txt?
You place the file in the root or on a sub-path. Version 2 makes explicit that an llms.txt describes the pages under its own path, and that the most specific file wins.
In practice:
/llms.txtdescribes your entire site/docs/llms.txtdescribes everything under/docs/- if both exist, then for a URL under
/docs/the specific file applies
Version 2 also added link relations, so systems can find the file without guessing at it:
rel="alternate" type="text/markdown"points to the Markdown version of a pagerel="describedby"points to the llms.txt that describes that page
Those relations may appear as <link> in the HTML, but also as an HTTP Linkheader. That last option is handy, because you arrange it in your web server or CDN without touching a single page.
What goes into an llms.txt?
According to the proposal, the file consists of four parts:
- an H1 with the name of the project or the site
- a blockquote with a short summary
- zero or more Markdown sections with explanation
- zero or more H2 sections with lists of links
Only that H1 is required. The rest is room for selection and explanation, and that is exactly where the work sits: choosing what does not go in.
A short example:
# GRP Digital
> A short guide to our most important knowledge base articles on SEO, AI visibility and measurability.
This page describes the knowledge base for people and agents who need context fast.
## AI visibility
- [AI visibility](https://grp-digital.com/en/knowledge-base/ai-visibility/): what it is and how it relates to SEO
- [Generative Engine Optimization](https://grp-digital.com/en/knowledge-base/ai-visibility/generative-engine-optimisation/): optimising for generative search systems
## Technical
- [Technical SEO](https://grp-digital.com/en/knowledge-base/technical-seo/): the technical foundation of visibility
- [technical AI visibility audit](https://grp-digital.com/en/knowledge-base/technical-seo/ai-audit-technical/): how to test your site’s technical setupVersion 2 changed two more details that are easily missed. For the Markdown version of a page, alongside page.html.md now also page.md. And the Optional section has lost its mechanical meaning: it is now only a convention for secondary links.
Does it actually work?
The honest answer: you see it reflected in tooling, but not as a proven lever for visibility in search engines.
Google Search Console is clear about it on the page about AI features: “You don’t need to create new machine readable files, AI text files, or markup to appear in these features.” That page was updated on 10 December 2025. Source: Google Search Console.
So you do not need an llms.txt to appear in Google's AI features.
What is supported or visible
At the same time, the file does live in a few places.
Chrome Lighthouse checks for it, within the checks for agentic browsing. If the file is missing, that audit appears as Not Applicable, precisely because it is optional. The Chrome documentation was updated on 5 May 2026.
That seems to contradict what Google says about Search, but it does not. The Lighthouse audit is about agents using your site to carry out a task. It says nothing about ranking in Google Search. Two different questions, two different answers.
On top of that, the big AI labs publish an llms.txt for their own developer documentation:
There is a line here that often gets crossed. That these parties publish such a file for their documentation is no confirmation that their models read your llms.txt to make you more visible. The first is about their docs, the second about your marketing. Those are not the same thing, and the difference often gets left out in practice because the first half sounds better.
What is not confirmed
There is no confirmation that major search or AI platforms use llms.txt for ranking, inclusion or broader visibility. If you look at the documentation the providers publish about their own crawlers, it points in only one direction: robots.txt. OpenAI describes there how you allow or block GPTBot, OAI-SearchBot and ChatGPT-User via robots.txt, and Anthropic writes that its bots respect “do not crawl” signals through the usual rules in robots.txt. llms.txt appears in neither as a file they read.
Be precise here. Google has said it out loud. OpenAI and Anthropic have never said it, in either direction: they’ve never claimed they use your llms.txt, but they’ve never explicitly denied it either. So that is silence. Anyone promising visibility gains on that basis is filling in the silence themselves.
Also llms-full.txt deserves nuance. You come across that name at documentation platforms, but it is not in the proposal and is not part of the specification. It is a convention from tooling, nothing more.
llms.txt vs robots.txt vs sitemap.xml
Put llms.txt alongside your existing files. They answer different questions.
| File | Purpose | Format | Main function | Status |
|---|---|---|---|---|
llms.txt | Context and selection | Markdown | Curated summary with links to important content | Proposal |
robots.txt | Access guidelines | Plain text | Indicating what automated systems may or may not access | Existing web convention |
sitemap.xml | URL overview | XML | List of indexable pages | Existing, widely supported standard |
A useful analogy: sitemap.xml is the full floor plan, robots.txt is the fencing around it, and llms.txt is the note at the entrance saying “these are the three halls you really have to see”.
How do you make an llms.txt?
Start small. If you have a lot of knowledge content or documentation, pick only the pages that give direction. A file with two hundred links is no longer a summary.
Basic structure according to the specification
Stick to this fixed structure:
- H1 with the project or site name
- short blockquote summary
- optionally a few explanatory paragraphs
- H2 sections with link lists per topic
If you want to make the connection with your regular pages explicit, use the link relations from version 2: rel="alternate" to the Markdown version and rel="describedby" to the llms.txt itself. That is mainly useful for documentation you build up in a structured way.
When do you use Optional and llms-full.txt?
In version 2 the Optional section is only a convention for secondary links. Use it as an extra layer, not as a load-bearing part of your file.
llms-full.txt you come across at documentation platforms, but it is not in the proposal. Only publish it if your tooling asks for it, and be clear internally that it is a convention and not the specification.
Writing guidelines for good output
Keep the file short and maintainable:
- choose only genuine source pages, not a complete sitemap
- give each section a clear label
- write summaries without internal jargon
- link only to pages that are current
- record who maintains the file and how often
An outdated llms.txt works against you: you are then actively steering systems towards content that no longer holds up. That weighs more heavily the faster your documentation changes.
In practice: llms.txt in WordPress and on other platforms
The practical rule is simple: look at what your platform actually supports and always check the output yourself.
WordPress
Yoast SEO has generated an llms.txt since June 2025. You switch it on via Yoast SEO, Settings, Site features, AI tools, LLMS.txt. The file is updated weekly and contains the five most recently updated items by default, with priority for cornerstone content. There is also a manual mode in which you decide yourself which pages go in.
Our estimate: that manual mode is almost always the better choice. The five most recently updated items are rarely your five most important pages.
Other platforms
AIOSEO generates it too, and Wix, GitBook and Mintlify do it for the sites and documentation they host. For Shopify no official support has been confirmed. There you publish the file manually or through custom work, and then check whether it is reachable and whether the content matches your current structure.
Even with tooling the question stays the same: are these genuinely your best source pages, or just your newest?
When is llms.txt worth it?
When to use it
llms.txt makes sense if you:
- have a lot of documentation or knowledge content
- can point out clear source pages
- can organise content at section level
- can genuinely keep the file up to date
For such sites it is a neat extra layer on top of a good information architecture.
When to skip it
Skip it if you:
- have a small site without room for maintenance
- have few substantive source pages
- are still looking for the right content structure
- do not yet have internal links and crawlability in order
In those cases llms.txt does not solve a problem you have.
What does demonstrably work for visibility and AI readability?
If you are working seriously on visibility in AI answers, start with the fundamentals: crawlability, clear internal links, answers that can be quoted literally, structured data and measurability. For the technical side, an AI visibility audit of your technical setuphelps; for the outcome, it helps to measure AI visibility structurally instead of relying on isolated observations.
Focus on making it easy to understand quickly. A search engine and an LLM only get a moment for your site.
Giacomo Perticara, founder and SEO strategist at GRP Digital
Why those fundamentals and not an extra file? Because the weighting has shifted. An SEO result used to weigh roughly like this: a quarter technical, half content, a quarter authority. Content was king. What we see now is roughly 30 per cent technical, 30 per cent content and 40 per cent authority. This is our estimate based on our experience over recent years, with no research behind it. But the direction is unmistakable: authority has become the biggest factor. An llms.txt doesn’t touch that factor.
SEO as the basis, AEO and GEO as deepening
“SEO is not dead. AI makes it more important, because an AI answer is only as reliable as the sources beneath it.”
Giacomo Perticara, founder and SEO strategist at GRP Digital
SEO is the overarching discipline. Answer Engine Optimisation and Generative Engine Optimization are deepenings of it, not separate disciplines and certainly not a replacement. First you make sure systems can find, understand and connect your content. Then you optimise for answer opportunities and source selection.
Practical checklist
- make your most important pages properly crawlable
- answer questions directly and quotably, high on the page
- strengthen the internal links between your main topics
- add structured data where it fits the page type
- use clear titles and short summaries
- measure your visibility over time instead of per spot check
- only publish llms.txt if you can maintain it
Frequently asked questions about llms.txt
What is an llms.txt file?
An llms.txt is a Markdown file with a short, machine-readable summary of a site or section, intended for language models and AI agents.
Is llms.txt mandatory?
No. The proposal explicitly describes the file as optional, and Chrome Lighthouse treats it that way too.
Is llms.txt worth it?
Sometimes. Mainly if you have a lot of documentation or structured knowledge content and can keep the file current.
How do you make an llms.txt?
Make a Markdown file with an H1, a short blockquote summary and H2 sections with links to your most important pages. Put it in the root or on the path it describes.
What is the difference between llms.txt and robots.txt?
robots.txt is about access: what may automated systems reach. llms.txt is about context: which content is important and why.
What is the difference between llms.txt and a sitemap?
A sitemap gives an overview of all indexable URLs. llms.txt is a curated selection with explanation.
Does Google Search use llms.txt?
No, there is no confirmation of that. Google Search Console writes precisely that you do not need to create new machine-readable files or AI text files to appear in the AI features.
Do ChatGPT and Claude use llms.txt?
Not that OpenAI or Anthropic confirms anywhere. Their own crawler documentation is entirely about robots.txt and does not name llms.txt as a file they read. They do publish an llms.txt for their developer documentation, but that says something about their docs and nothing about your site.
Conclusion
llms.txt is an interesting proposal, but today it isn’t a measure that demonstrably increases your visibility. The real question is whether you can maintain it. See it as an extra layer for documentation and knowledge content, not as a replacement for your SEO foundation.
The advice is therefore simple. If you have a lot of knowledge content, can make a compact selection and can maintain it, then by all means add llms.txt. It is a small job and it does no harm. If that basis is missing, invest first in crawlability, quotable answers, internal links, structured data and measurability. That is the work that does have a demonstrable effect, with or without an extra file in your root.
