llms.txt is a text file you place at the root of your site to give AI models a clean, ordered summary of your content. The idea is sensible. The problem is that, as of today, the big AI engines barely read it: in the most rigorous experiment to date —90 days and more than 62,000 AI bot visits to a single site— only 0.1% of those visits went to the file. If your plan for getting ChatGPT to recommend you runs through dropping an llms.txt, the data says you are optimizing for a reader who never shows up.
And yet almost everything written about this —in Spanish, dozens of guides from agencies and from the big SEO suites— sells it as the new must-have piece for AI visibility. Let us look at the data, which is what almost nobody has done.
What is llms.txt?
llms.txt is a Markdown file, hosted at yourdomain.com/llms.txt, that offers language models a curated, noise-free version of your site: a summary of what you are plus links to your most important content in clean text. It was proposed by Jeremy Howard, founder of Answer.AI, in September 2024.
The problem it claims to solve is real. A modern website is full of menus, cookie banners, scripts and footers that are interface to a human but, to a model reading the page, are noise that costs tokens and hampers comprehension. The idea of llms.txt is to hand the machine the content already distilled, so it does not have to dig.
The industry has christened it a thousand ways —the "AI sitemap", the "tasting menu", the "press kit for AI"—. All those metaphors share an assumption worth putting on the table before going on: that there is a model on the other side reading the file. That is where it falls apart.
Does AI read your llms.txt?
As of today, not in any meaningful way. No major AI provider has confirmed that its production systems use your llms.txt to discover, cite or recommend you, and server logs —which is where the truth shows— confirm it.
The most solid data point was published by OtterlyAI, an AI-visibility measurement tool, after a 90-day experiment. They put an llms.txt at the root of a site and measured what the AI bots did. Of more than 62,000 AI bot visits to the site, only 84 went to the file: 0.1%. For context, a normal page on that same site received about 265 bot visits on average. The llms.txt performed three times worse than any random page, and barely better than a stray PDF. In their own summary: to the bots, the llms.txt is all but invisible.
It is not an isolated case. An independent thirty-day audit across a thousand domains found that not a single AI bot —not OpenAI's, not Anthropic's, not Perplexity's— came to fetch the file; the only one touching it was Google's regular crawler, the one that requests everything it finds. Search Engine Land ran the experiment on its own site for three months and logged zero visits from AI crawlers, with no measurable improvement in its presence inside generative answers. And Ahrefs settled it without anesthesia: there is not a shred of proof that any model uses that file to decide whom it summarizes or cites.
Google has been the most explicit. It has said its search systems neither read nor act on llms.txt, and has gone as far as comparing it to the "keywords" meta tag —that tag SEOs filled in religiously fifteen years ago and that search engines ended up ignoring entirely—. The analogy stings because it is precise.
Why it does not work the way you think
Because nobody is obliged to read your llms.txt, and because AI does not form its opinion of you by reading what you say about yourself.
The trap is the analogy with robots.txt. The industry reasoning goes: robots.txt controls crawlers, the sitemap helps you get discovered, therefore llms.txt will improve my site in AI answers. It sounds logical and it is false. robots.txt works for a reason llms.txt lacks: twenty years ago search engines agreed to respect it. There is a pact, there is someone on the other side paying attention. llms.txt is a sign you hang in your shop window aimed at a customer who does not walk down your street. It does not matter how well written it is.
There is a deeper reason, and it is the one we measured in our study The Invisible Share. When a freelancer asks ChatGPT which invoicing software to use, the model does not consult the file you left at the door. It travels the associations it has built, from reading half the internet, between that person's intent and the brands that appear again and again tied to that intent. Your place in the model's mental network is not something you decide by declaring yourself in a .txt. It is decided by your footprint: how much and how the rest of the web names you. That is why a file at your root does not move the needle. It is not badly made; it is aimed at the wrong place.
The big engines —Google, Perplexity, OpenAI— already have hugely expensive machinery dedicated to cleaning HTML, identifying the main content and ranking it. They do not need you to leave them a summary to understand your page. They have spent years investing in doing exactly that themselves, at scale. llms.txt solves a problem they had already solved.
What llms.txt is actually for
It is for the AI tools that integrate your content —and that lack Google's infrastructure— to consume it more cheaply and cleanly. That is the use it was designed for, and there it does work.
Think of a coding assistant like Cursor or Copilot, or a copilot embedded inside a SaaS, or a support bot that pulls from your documentation. When those tools need one of your pages, they have to fetch it and clean it on the fly, which is expensive and messy: calling a big provider's search API can cost between 10 and 14 dollars per thousand queries, and that explodes at scale. A well-made llms.txt hands them your content already in clean Markdown, with no banners or clutter, spending a fraction of the tokens. For that case it is a technical courtesy that improves speed, cost and even answer quality.
That is why the AI companies themselves —OpenAI, Anthropic, Perplexity, Stripe— publish their llms.txt: so that code assistants consume their documentation. And here is the misunderstanding that drives the whole market. Someone sees that Anthropic has an llms.txt and concludes "I need one so AI cites me". But a company publishing a clean manual for machines to read has nothing to do with that company's crawler coming to your site to read yours in order to recommend you. It confuses putting the instruction manual on an accessible shelf with having a billboard. They are different things that never touch.
Should you add an llms.txt?
If your goal is AI visibility, adding it will not hurt you, but do not expect it to move anything in the short term. If you have technical documentation or an API that others integrate into their own tools, then yes, it makes sense and is a reasonable future bet.
Implementing it takes ten minutes and there are plugins that generate it for you, so as insurance for the day this changes, go ahead. Even Jeremy Howard himself admits the standard is in its infancy and predicts that one day a formal crawling protocol for AI will emerge, perhaps an heir to this one. Hedging is legitimate.
Two warnings first. The first: do not even think about publishing the full version, llms-full.txt, which dumps all your content into a single file. That is serving your competitors and any crawler your entire library in clean, easy-to-copy text. Transparency is fine; giving away years of work is not.
The second is the one that matters. Do not file llms.txt under "AI visibility strategy", because it has no business there, and the budget you spend on it thinking that is the work is wasted budget. The real work is elsewhere: in AI having you built as a sharp entity —knowing precisely what you are and who you are for— and in your footprint being densely tied to the intents that feed you, in the sources the model actually reads. That is slow, expensive and hard to fake. That is why almost nobody does it, and that is why it works. llms.txt is the exact opposite: easy, free, and with the comforting feeling of having done something. What is cheap to make tends to be cheap to ignore, and that is exactly what the models are doing with it.
Even OtterlyAI, which sells GEO tools, has removed the llms.txt checker from its audit, because its real impact on how AI discovers you is marginal and distracts from what actually moves visibility.
llms.txt vs robots.txt vs sitemap.xml
All three are files at the root of your domain, and that is where the resemblance ends. robots.txt is access control: it tells crawlers where they cannot go, and it works because search engines agreed to obey it. sitemap.xml is inventory: it lists all your URLs so search engines discover and crawl them efficiently. llms.txt is a guidance proposal: it aims to signal to AI models which of your contents are most relevant. The decisive difference is that the first two have recipients who have respected them for years, and the third, for now, does not.
Frequently asked questions about llms.txt
Does AI read the llms.txt file? As of today, barely. In the largest available experiment, only 0.1% of AI bot visits went to the file, and Google has confirmed it does not use it in its AI features.
Does llms.txt improve my rankings or my AI citations? There is no evidence that it does. No major AI engine uses it to rank or cite, and log studies find no correlation between having it and appearing more in generative answers.
So what is llms.txt actually for? For AI tools that integrate content —code assistants, copilots, support bots— to consume your site more cleanly and cheaply. It is infrastructure for integrations, not a visibility lever.
Should I implement llms.txt? If you have documentation or APIs that others integrate, yes, as a low-cost future bet. If your goal is for AI to recommend you to your customers, your effort pays off more in content, entity and citations across the web.
What is llms-full.txt and should I use it? It is a version that dumps all your content into a single file. It is not advisable: it amounts to handing your entire library to competitors and crawlers in easy-to-copy text.
Is llms.txt the same as robots.txt? No. robots.txt controls crawler access and has been respected for years; llms.txt is a guidance proposal for AI models that, for now, those models do not read in any meaningful way.
An analysis by Elevam Labs. If you want to understand where it is really decided whether AI recommends you —and not in a file at your root—, it is in our study The Invisible Share.
