The llms.txt V2 Update Solved a Real Problem. Now We're Waiting for Agents to Care.
2026-08-17 — management consulting India
Two years in, thousands of sites are publishing llms.txt files and no one's sure if it matters.
That's the honest starting point here.
Jeremy Howard, the Answer.AI founder who created the standard, shipped version 2 in August. It's the first real revision since the format launched in 2024, and it fixes something specific: the gap between where an llms.txt file points agents and how agents actually locate the markdown versions of your pages.
You needed a map to find your own map.
What Changed
V1 had a weak point. The llms.txt file pointed agents toward pages, but nothing told them where the markdown versions actually lived. Sites had to pick one of two patterns—either add.md to the full filename like /docs/tutorial.html.md or replace the extension entirely and serve /docs/tutorial.md—and agents just had to guess right.
V2 introduces formal link relations to solve this.
An HTML page can now include a rel="alternate" link with type="text/markdown" that points directly to its markdown version. Another link, rel="describedby", points back to the llms.txt file that covers it. Sites can add these either as standard HTML link elements in the page head or through an HTTP Link response header—which means you can add them via CDN or server config without touching your templates.
It's elegant, actually.
For sites already running v1, updating to v2 is minor. You're not rebuilding. You're just adding link elements or headers that tell agents where to look.
The Adoption Momentum (Which Doesn't Guarantee Anything)
Howard says the update is based on two years of practical experience: thousands of sites now publish an llms.txt file, documentation platforms like Mintlify generate one automatically for every site they host, and even Google's Chrome team added a check for the file to Lighthouse's Agentic Browsing category. Anthropic, OpenAI, and Google's Gemini team now publish their own llms.txt files for their developer documentation.
That's real adoption.
But adoption isn't the same as integration. Most consultants get this wrong, including us sometimes. Just because thousands of sites publish something doesn't mean thousands of agents are reading it.
Google's Search team has said for over a year that support for llms.txt isn't on their agenda. The company's AI optimization guide states that Google Search itself doesn't use these files, and maintaining one will neither harm nor help your site's visibility or rankings. Chrome's Lighthouse tool checks for the file and flags server errors, but finding an llms.txt file on a site is optional—a missing file returns a 404 and Lighthouse marks it as Not Applicable.
So here's the actual situation.
Actually, that's not contradictory.
One is about discovery—whether a search engine or AI overview feature finds your page in the first place. The other is about usability once an agent lands on your site. They're different jobs. Without an llms.txt file, agents may spend more time crawling the site to understand its high-level structure and primary content. That's a friction problem for browsers and agents, not a ranking problem for search.
Why This Matters (And Why It Might Not)
The v2 update assumes that agents are actively looking for Markdown versions of your pages and following link relations to find them. That's plausible for coding assistants like Cursor, which need clean documentation. AI systems struggle with HTML pages wrapped in navigation, ads, and JavaScript, wasting tokens on non-essential content. For a developer trying to build against your API, cleaner input means fewer hallucinations.
But the practical question is messier.
Most chat-based tools have not committed to looking for llms.txt by default. A developer asking Claude or Perplexity about an API probably wants an accurate answer. The company building that API would rather the answer come from their docs than a six-month-old Stack Overflow thread. So there's motivation on both sides.
For news sites, restaurants, or retail stores? The case is less obvious when your content isn't technical documentation.
V2 removes one friction point—making it easier for agents that do look to find markdown versions. That's useful. But it doesn't answer the larger question: do the agents you care about actually read llms.txt at all?
Most of the time, the honest answer is: we don't know yet.
Sites are publishing them. Platforms are auto-generating them. Google Chrome is auditing them. And none of that tells you whether Claude or Perplexity or a custom agent is actually using your file to serve better answers. The adoption curve is real. The evidence that it moves the needle is still catching up.
So what do you do with a file that's technically sound, widely discussed, and measurably unused?
You publish it anyway, because the cost is near zero and the scenario where it matters is worth hedging for.