What is llms.txt, and do I actually need it?
llms.txt is a plain text file at the root of a site that tells AI systems what the site is and which pages to read first. It is not a standard and no vendor has publicly committed to reading it — useful, but not the first job on the list.
The idea is simple and old: robots.txt says where you may go,
sitemap.xml says which addresses exist, and llms.txt tries to say
what all of it means.
What it looks like
The file is Markdown, not configuration. The structure is fixed: one #
heading, a one-sentence blockquote, then free text with no headings, and only
then ## sections with link lists.
The important part is not the links but that one sentence and the text under it. That is where a model learns whether you are a shop, an auditor or a news site.
What it actually gives you
Today, not much, and indirectly. No large vendor has announced that it reads the file. But it is cheap, costs nothing to keep, and some tools — MCP servers for Claude Desktop, for instance — already consume it directly.
The practical benefit is often different: writing it forces you to state what your site is in one sentence. Many people discover that sentence appears nowhere on their pages.
What it does not replace
It does not replace robots.txt. If that blocks GPTBot, llms.txt will never
be read, because nobody comes in.
It is also not a place for "content for robots". Writing things there that do not exist on the pages creates a second version of the truth — the same idea as hidden text, which used to earn penalties.
When it is worth doing
When the technical side is already in order: AI crawlers allowed, structured
data present, pages answering questions. Then llms.txt is a last five-minute
job.
It is not worth doing instead of those things.
A free scan checks it along with what matters more: whether the bots get through and whether the content is worth citing.
Frequently asked
Does Google read llms.txt?
It has not said so publicly. Google uses `robots.txt` and `sitemap.xml` for indexing; `llms.txt` appears as a signal in none of its documentation.
What is the difference between llms.txt and llms-full.txt?
The first is a map with links, the second is the full content in one file so a model does not have to walk the pages. The second only makes sense when there is little content or it changes rarely.
Can it hurt SEO?
No. It is a separate file that search engines do not index as a page and that changes no ranking signal.
Where does it belong?
At the root: `yourdomain.com/llms.txt`. Anywhere else and no tool will find it, because the convention says to look exactly there.
Check yours
A free scan shows which of the things described here your site already does, and which it does not.