Does llms.txt Do Anything? What Google and the Data Say

What is llms.txt supposed to do?

The llmstxt.org proposal, published in 2024, addresses a genuine problem. When a language model fetches a web page at inference time, it gets HTML full of navigation, cookie banners, ad slots and scripts, inside a context window that is expensive to fill. A curated Markdown file — here is what this site is, here are the URLs that matter, here is how to cite them — is a reasonable engineering answer to that.

The file lives at /llms.txt, is written in Markdown, and typically contains a title, a short summary, and annotated link lists grouped by section. As a specification it is sane and cheap to implement.

The problem is entirely on the consumption side: a hint file only works if somebody reads it.

What does Google say?

Google's position is unambiguous. Documentation updated in June 2026 states that llms.txt has no effect — positive or negative — on Search rankings or AI Overviews, and that Search simply ignores the file.

John Mueller of Google gave the structural reason, reported by Search Engine Journal:

To me, it's comparable to the keywords meta tag — this is what a site-owner claims their site is about.

That analogy is the whole argument, and it is a good one. The keywords meta tag died because a self-reported claim cannot rank anything. If every site declares itself the authoritative source, the declaration carries no information. Any signal a search or answer engine can trust has to be something you cannot simply assert about yourself — how others reference you, what your pages actually contain, whether the claims hold up.

This is also consistent with Google's general AI features guidance, which states directly that you don't need to create new machine readable files, AI text files, or markup to appear in these features.

What does the field data show?

The most useful study is Ahrefs' analysis of 137,210 domains that received traffic in May 2026. The findings:

Measure	Result
Domains publishing an llms.txt	28%
Valid files analysed	~38,000
Files receiving zero requests	97%
Of requests that did occur, from bots	96%
Share of total requests from AI retrieval bots	~1%

Two details in that data are more damning than the headline. First, of the roughly 38,000 valid files, only about 1,100 were fetched by anything at all. Second — and this is the decisive one — no requests came from AI bots probing for files that did not already exist. Crawlers are not looking for llms.txt. If they were, you would see speculative 404s across the sample. You do not.

Adoption estimates vary by sample (other trackers report figures closer to 10%), but no independent study has found meaningful AI-bot consumption. And as of writing, no major provider — OpenAI, Anthropic, Google, Meta or Mistral — has committed to using it as a signal in a production search or answer surface.

Why does the confusion persist?

Three reasons, and it is worth being able to spot each one.

The name. llms.txt sounds like robots.txt, so people assume it inherits its authority. It does not. robots.txt works because crawler software is built to fetch and obey it. llms.txt has the format of a standard without the adoption of one.

Correlation sold as causation. Sites that ship an llms.txt are, as a group, technically attentive sites with good information architecture and real documentation. They tend to do well in AI citations. The file is a marker of that competence, not a cause of it. Every case study you will be shown has this confound and none of them control for it.

It is easy to sell. "Add this file" is a concrete deliverable with a visible artefact at the end. "Write genuinely better answers than the incumbent" is not billable in the same tidy way.

When is it still worth shipping?

There are honest reasons to have one, and they have nothing to do with rankings.

• You are the consumer. If you run a documentation assistant, an internal RAG pipeline, or a support bot over your own content, a curated Markdown index is genuinely useful — you control the reader, so adoption is not a question.
• It costs you nothing. If your site can generate the file from data it already has, the marginal cost is a route handler. This site serves /llms.txt generated from the same published-post query that feeds the sitemap, so it cannot go stale. That is a defensible reason to keep it: near-zero cost, small option value if adoption ever arrives.
• You want the option value. The proposal could still be adopted. Shipping a correct file today costs little and positions you if that changes.

What is not a good reason: expecting rankings, expecting AI citations, or paying a vendor to produce one. If an agency lists llms.txt implementation as a line item on an AI-visibility proposal, that line item is worth close to nothing, and its presence tells you something about the rest of the proposal.

What actually works instead?

The signals that correlate with being cited in AI answers are unglamorous and mostly old:

• Be genuinely referenced elsewhere. Analyses through 2026 consistently find off-site brand mentions correlating more strongly with AI citation than link metrics or on-page optimisation.
• Write self-contained, specific passages. Retrieval works on passages. See what AI Overviews are and how they work for the mechanism.
• Let the right crawlers in. If you block OAI-SearchBot or PerplexityBot in robots.txt, you cannot be cited in those products regardless of what any hint file says. That is a real control with real consequences — see AI crawlers and robots.txt.
• Be a source worth naming. Named authors, dated corrections, primary evidence — the credibility signals that work on humans are the ones an answer engine is willing to attribute to.

The short version: ship llms.txt if it is free and you have a use for it. Do not believe anyone who tells you it moves rankings, and do not pay for it.

Sources: llmstxt.org — the proposal · Ahrefs — We analyzed 137K sites: 97% of llms.txt files never get read · Search Engine Journal — Google says llms.txt is purely speculative for now · Google Search Central — AI features and your website