Do llms.txt Files Actually Matter?

So it's worth revisiting our earlier thoughts — including where the picture is more nuanced than either the hype or the headlines suggest.

What an llms.txt file is

llms.txt is a proposed standard: a plain-text file at the root of your site that maps your most important content for AI models, so they don't have to infer it from your navigation and markup. The idea is sensible. A clean, curated index of what matters on your site, written for machines, sounds like exactly the kind of thing AI tools would want. The question is whether anything actually reads it.

What Google now says

In its AI features and your website guide, Google is unusually direct: "You don't need to create new machine readable files, AI text files, markup, or Markdown to appear in Google Search." And on the file specifically: creating one "will neither harm nor help your site's visibility or rankings in Google Search." It says the same about adding special schema markup for AI — it isn't required, and there's no secret AI-specific format to feed the system. That's about as clear as Google gets. The evidence, directly from the source, is that Google themselves aren't using llms.txt for search. For AI Overviews, AI Mode, and ordinary Search, the file does nothing.

So is the whole idea dead?

Nope.

It was proposed for the broader set of AI tools: assistants pulling a page into a context window, AI search engines building an answer, that sort of thing. Google answering for Google doesn't settle whether the rest of the field finds it useful. And there's a telling signal here.

Anthropic — the company behind Claude — publishes an llms.txt file on its own documentation website. A frontier AI lab putting one on its own documentation is not doing it for Google's benefit. They're doing it because they think it helps AI tools read and use their content, and it's hard to think of a more credible vote of confidence in the format than the people building the models using it themselves.

So our position is that llms.txt still has a place. Google has told us plainly it isn't a factor for search, but "Google doesn't use it" and "nobody uses it" are different claims, and the second one looks wrong.

Where this leaves you

It costs almost nothing to generate an llms.txt file, it does no harm, and at least one of the major AI labs clearly believes in it enough to run one.

That's a reasonable bet to take. We generate and maintain these files on the sites we look after, and we'll keep doing so. What we'd avoid is treating it as a substitute for the real work. It's a small, sensible addition — not a lever that lifts your rankings or guarantees you a citation. If you already have one, leave it. If you don't, add it, then move on to the things that actually move the needle.

What actually moves the needle

The more interesting part of Google's guidance is what it tells you to do instead, and it's the same advice we gave for AI search generally: Make your content crawlable. AI features draw on pages that are publicly accessible and indexable. Anything behind a gate or blocked from crawling can't be cited. Write non-commodity content. Google's own word. Original, first-hand, genuinely useful material — the things a model can't generate for itself. Keep the structure clean. Semantic HTML, a logical hierarchy, good titles and descriptions. This helps every reader, human or machine. Don't game it. Don't chop content into artificial chunks or spin up AI-only versions of pages. Google calls these out specifically, and they tend to backfire. In other words: an llms.txt file is worth having, but it sits on top of the fundamentals — it never replaces them. The work that earns citations is the same work that has always earned good rankings.

Where to start

Add the llms.txt file by all means — but spend the real effort on a content and technical audit: crawlability, metadata, structure, and which of your pages are actually strong enough to be cited. That's the audit we run, and we fold the monitoring into our maintenance service so you can see what AI is making of your site over time.

Email us at hello@mutual.agency.