Why llms.txt still won't make AI systems cite you

New data released this week confirms what sceptics have argued for months: llms.txt, the markdown file many brands added to their websites hoping it would boost AI citations, has almost no measurable effect on visibility in ChatGPT, Google AI Overviews or Perplexity. Large-scale studies covering hundreds of thousands of domains found no correlation between having the file and being cited, and server logs show AI crawlers barely request it. For founders who spent budget implementing it this year, the lesson is that structured markup is not a substitute for content AI systems actually want to retrieve.

What changed?

A fresh wave of research published in the past day has put hard numbers behind a debate that has simmered in the GEO community all year. An Ahrefs analysis of 137,210 domains examined Google's official guide on optimising for generative AI features in Search, which states that website owners don't need to create new machine readable files, AI text files, markup or markdown to appear in generative AI search.

Separately, SE Ranking ran its own analysis of nearly 300,000 domains to test whether having an llms.txt file correlates with how often a domain gets cited by AI systems, and the answer was no. A third study, from OtterlyAI, tracked server logs over 90 days and found that out of more than 62,100 total AI bot visits to the site, only 84 of those visits targeted /llms.txt, roughly 0.1% of all bot traffic.

This builds on an earlier finding that the major AI crawlers simply don't look for the file. A separate study of 500 million LLM bot traffic events found that GPTBot, ClaudeBot, PerplexityBot, OAI-SearchBot and Google-Extended overwhelmingly skip the file and crawl HTML directly.

Google has also been explicit about its own position. According to Security Boulevard, on 15 June 2026 Google logged a change called "Clarifying guidance on llms.txt files" and added a note to its AI optimisation guide stating the file is not needed to appear in Google Search, and that keeping one will neither harm nor help visibility or rankings because Google Search ignores it.

Google's John Mueller has gone further in characterising the file. Per the dev.to analysis, when pressed on the apparent contradiction between different teams at Google, Mueller described llms.txt as "not done for search", calling it a "temporary crutch, perhaps to save some tokens" for AI coding tools parsing developer documentation.

There's also an unexpected security wrinkle. The same Ahrefs dataset identified that the largest single research crawler it tracked identified itself as prompt-injection-survey/1.0, meaning someone is systematically studying llms.txt as a prompt injection vector, because AI agents are designed to ingest and trust whatever the file says.

Why does this matter for your business?

If you've invested time building and maintaining an llms.txt file in the hope it would get your brand cited more often in AI answers, this data suggests that effort was largely wasted. llms.txt does nothing for your visibility in Google Search, AI Overviews or AI Mode, and Google has said so explicitly and repeatedly, with correlation studies finding no relationship between having the file and being cited.

That doesn't mean the file is entirely pointless. It does have one legitimate use case: developer documentation consumed by coding agents. If you run a developer-facing product or API, giving coding assistants a clean index of your docs can still be worthwhile. But for most consumer and B2B brands chasing AI search visibility, it's the wrong lever.

This matters because the GEO industry has produced a lot of advice that sounds authoritative but isn't backed by evidence. As Security Boulevard put it, anyone selling llms.txt as an AI visibility lever is selling the exact thing that makes GEO sceptics sound right. Adoption has still spread widely despite the lack of proof, with roughly 10% of domains now running the file, which tells you more about marketing advice than about efficacy.

The prompt injection angle raises the stakes further. If AI agents are designed to trust whatever an llms.txt file says without verification, a file that does nothing for legitimate visibility could still become an attack surface. Brands that added the file without thinking about who else might edit or exploit it should revisit that decision.

What should you do now?

Stop treating llms.txt as a checkbox for AI visibility. The mechanisms that actually drive citations in ChatGPT, Gemini and AI Overviews are the same fundamentals that have mattered all year: clear, well-structured HTML content that answers specific questions, strong entity signals, genuine third-party mentions, and pages that are easy for standard crawlers to fetch and parse. None of the major platforms require a special file to find or trust your content.

If you maintain developer documentation and want coding agents to navigate it efficiently, keep llms.txt as a low-cost addition, since it's unlikely to cause harm. But redirect the bulk of your GEO budget towards content quality, structured data that's actually supported, such as schema markup, and making sure your most important pages are genuinely crawlable by GPTBot, ClaudeBot, PerplexityBot and Google-Extended.

It's also worth checking what's actually happening with your own site today rather than assuming a file you added months ago is doing any work. Running Sited's free audit can show you which AI systems are actually citing your brand right now, which is a far more reliable signal than whether you've ticked the llms.txt box.

Finally, if you're relying on agencies or vendors who are still pitching llms.txt implementation as a core GEO service, ask them for evidence. The data published this week gives you a clear benchmark to hold them to.

Frequently asked questions

Does llms.txt help with ranking in ChatGPT or Perplexity?

No current evidence supports this. The major AI crawlers, including GPTBot, ClaudeBot, PerplexityBot and OAI-SearchBot, overwhelmingly skip the file and crawl HTML directly, so there is nothing to influence even if the file is present.

Is there any legitimate reason to add llms.txt?

Yes, for developer-facing products. It has one proven use case, which is developer documentation consumed by coding agents. Outside of that context, it has no demonstrated value for AI search visibility.

Did Google ever support llms.txt for search?

No. Google's official guidance confirms the file is not needed to appear in Google Search, and that keeping one will neither harm nor help visibility or rankings because Google Search ignores it entirely.

Could llms.txt actually be risky?

Potentially, yes. Researchers found that the largest research crawler studying llms.txt files identified itself as a prompt injection survey tool, examining the file as a possible injection vector because AI agents are built to trust its contents without verification.

What should I focus on instead of llms.txt?

Prioritise crawlable, well-structured content that directly answers the questions your customers ask, along with genuine citations and mentions across the web. These signals consistently correlate with AI search visibility, unlike a standalone markdown file.