Is llms.txt actually needed, and what does a good one look like?
llms.txt is still a proposal that no vendor has publicly committed to reading, so its benefit today is small and unproven. But it is cheap. A good file is short, maintained by hand, and lists live URLs with one line of context each — not a sitemap copy, and not a list that went stale after the first restructure.
llms.txt is one of those things that is easy to sell a service around: the
file is small, it takes an hour, and the name sounds current. So both sides are
worth stating.
The numbers draw on roughly 500 audits run on our earlier platform and on what we see today. The exports from that work are gone, so these are ranges rather than precise statistics.
What it actually delivers today
The honest answer: an unproven benefit. It is a proposal published at
llmstxt.org, and no major vendor has publicly committed to reading it. Anyone
claiming otherwise should point at the vendor's documentation.
It also costs one file and breaks nothing. So our position is: worth having, not worth paying for separately, and certainly not worth delaying the things that work today — access, speed, structure.
What a broken file looks like
A sitemap copy. A hundred URLs with no context. That is not a map, it is a
list the machine already has in a better format: sitemap.xml.
Stale URLs. Written once and forgotten; after a restructure half the links lead to 404s. Worse than not having it — you are declaring something that does not exist.
Marketing copy. Half a page about how unique the company is, instead of a map. There is nothing for a model to take from it.
Private paths. Sometimes URLs end up in there that the site deliberately
does not publish. llms.txt is a public file — everything in it is published.
What a good one looks like
Short. One sentence about what the site is, then a dozen or so key URLs, each with a single line saying what is there.
And, above all, maintained: if the file cannot be updated alongside the site, it is better not to have it. Same principle as any declaration to a machine — declaring what does not exist is a lie, not a gap.
What to do before it
If you are choosing what to do today, llms.txt is not first — it is about
fifth:
- a
robots.txtthat deliberately says what AI bots should do; - text reachable without JavaScript;
- questions with answers under them;
- structured data with an author and dates;
- and only then
llms.txt.
The wider definition and history: what is llms.txt. What a scan actually verifies: GEO in 2026.
Related articles
- What is llms.txt and do you really need it
- Markdown for agents: a cheaper site for bots
- Does your robots.txt let AI bots in
- The state of AI search: where we stand
What gets checked is listed under what we check.
Sources
What this text is based on — primary sources, not retellings:
Frequently asked
Does `llms.txt` replace `robots.txt`?
No. They answer different questions: `robots.txt` says where a bot may go, `llms.txt` suggests what is worth reading first. Every major bot honours the first; nobody has publicly committed to the second.
Will `llms.txt` improve my visibility in AI answers?
There is no evidence for that. Visibility comes from access, text without JavaScript, and content worth quoting. `llms.txt` is a cheap addition to those, not a substitute.
How many URLs should it list?
As many as you can maintain — usually ten to thirty. A file nobody updates becomes a list of 404s within a year.
Do I need a separate `llms-full.txt`?
Only if the content genuinely fits and you intend to update it. A large file generated once goes stale just as quietly as a small one.
Check yours
A free scan shows which of the things described here your site already does, and which it does not.