llms.txt in 2026: Google Ignores It. Should You Still Ship One?

Key points before the detail:

• llms.txt is a proposed convention, not a ratified standard. It is a short Markdown file that points language models and agents at the pages that matter.
• Google's guidance for generative AI features lists the file as something you do not need for Google Search. Chrome's Lighthouse tool still checks for it, and treats a missing file as optional.
• As of the Semrush article this page draws on, no major AI platform has confirmed that it reads the file, and the one test in the sources found no link to better AI results.
• Adoption is rising from a tiny base: 951 domains in July 2025 and 16,670 by September 2026, per Semrush.
• The decision rule at the end: ship it when it is cheap or automatic and your readers include agents, skip it when you would be hand-maintaining it to please Google.

What is llms.txt?

llms.txt is a proposed convention in which a website publishes a Markdown file that gives language models and agents a concise guide to the site. The proposal lives at llmstxt.org, written by its author and now in a second version. It asks for the file at the site root, or at any path within it, covering the pages under that path.

The problem it targets is that web pages are built for people. According to the proposal, an HTML page wraps its information in navigation, ads and JavaScript, converting it back into clean text is imprecise, and context windows are still too small for most websites. The file offers a short, curated starting point, with the detail behind the links fetched only when needed.

Semrush frames the same idea for marketers: instead of letting AI crawlers wander a site, you hand them a curated list of your most important content. Its article is also blunt that this is a proposal, and that the speculation about visibility gains has no proof behind it as of its writing.

What goes inside an llms.txt file?

Only one element is required: an H1 with the name of the site or project. According to the llmstxt.org specification, everything else is optional, and the elements must appear in a fixed order. The table below sets out each part as the proposal describes it.

A minimal file for an invented site called Example Co, using the reserved example.com domain, would look like this. It is an illustration built from the format above, not a file taken from any real site.

# Example Co

> Example Co makes invoicing software for freelancers.

## Docs

- [Quick start](https://example.com/docs/start.md): Set up an account in five steps

The proposal chose Markdown over a stricter format such as XML because it expects language models and agents to read these files, while noting the structure is still precise enough for ordinary parsers. Semrush adds that richer structure, such as sub-headings, is fine as long as the Markdown is valid, and shows that Hugging Face, Vercel, Zapier and Cal.com each organize their files differently.

What does Google say about llms.txt?

Google's guide to generative AI features treats llms.txt as something you do not need for Google Search. The guide is Optimizing your website for generative AI features on Google Search, and it has a section titled “Mythbusting generative AI search: what you don't need to do”.

Here is the limit of this page's evidence. The text under that heading was not in the extract this article was built from, so Google's exact sentence on llms.txt is not quoted here. Kanso's weekly growth checklist, which read the guide on October 13, 2026, records it as saying that Google Search does not use llms.txt files, so creating one neither helps nor harms visibility in Google Search.

Two other points from the same guide matter more for what you do next. Its rule for being eligible in Google's generative AI features is plain: a page must be indexed and eligible to show in Search with a snippet. And the guide says those features rely on Google's core Search ranking and quality systems, which is why it says SEO best practices remain relevant.

Semrush adds that Google's John Mueller confirmed on Bluesky that no major AI platform has confirmed it uses the file. The post itself was not in the material this page was built from, so it is not quoted. If you want a primary-source quote for a client or a boss, take it from Google's guide directly.

Which systems actually read llms.txt?

The evidence supports a narrow list: developer tools and documentation platforms that look for or generate the file, and a proposal author who says coding agents use it. It does not support a claim that any major AI assistant relies on it. The table brings the sources together, grading each claim by who made it.

How the table was put together: every row restates one claim from one of the pages in the Sources section, and the last column labels who made the claim. A claim from the proposal's author about the proposal's use is weaker evidence than a vendor documenting its own feature, and both are weaker than an official platform saying what its own systems do. No row comes from a Kanso test.

The most useful pairing is Lighthouse against Google Search. Chrome's documentation says that without the file, agents may spend more time crawling a site to understand it, and marks a missing file Not Applicable because providing it is optional “at the moment”. Semrush points out that a Lighthouse audit and Google's live crawling are separate systems, so the audit is not evidence that Google's crawlers use the file to answer queries.

How many sites publish an llms.txt file?

Semrush reports 16,670 domains with an llms.txt as of September 2026, up from 951 in July 2025. That is the only adoption count in the sources, and it comes from a single trade article, so treat it as one estimate and not a census.

The arithmetic is simple. The rise is 16,670 minus 951, which is 15,719 domains. The ratio is 16,670 divided by 951, about 17.5, over the 14 months from July 2025 to September 2026, which matches the roughly 17.5 times growth Semrush states.

Semrush's own caveat is the right one: 16,670 domains is a small share of the hundreds of millions of active websites. Fast growth from a small base says the habit is spreading among developers and documentation teams. It does not say the file works. The proposal's author also says thousands of sites publish one and that documentation platforms generate it automatically.

Does llms.txt improve visibility in AI answers?

The sources contain no evidence that it does, and one test that found no link. Semrush says no major AI platform, including ChatGPT, Google's AI Overviews and Anthropic's Claude, has confirmed that it reads or relies on the file.

Semrush also describes its own test. In March 2025 it implemented the file on a sister site, Search Engine Land, and in its testing found no correlation between implementing llms.txt and improved performance in AI results. It reports that from mid-August to late October 2025 the llms.txt page received no visits from Google-Extended, GPTBot, PerplexityBot or ClaudeBot.

Treat that as one site, one period and one publisher's account, with no method published in the material read here. It is a reason for doubt, not a final measurement. The opposing case is only an analogy: llmstxtgenerate.com compares the file to adding structured data in 2010, when early adopters had a quiet advantage once search engines caught up. That is a vendor's opinion about the future, and the same page admits there is no strong evidence yet that models use the file in ways that move the needle.

One more fact that explains the confusion: Anthropic publishes an llms.txt on its own website. Semrush's reading is that this does not mean Anthropic's crawler uses such files, only that the company is probably open to the idea.

Is llms.txt an SEO file?

No: it is closer to documentation for agents than to a ranking signal. According to the proposal, robots.txt tells automated tools what access is acceptable, such as for search indexing, while llms.txt information is used on demand, when an agent needs information about a topic while helping a user.

It is also not a sitemap. The proposal says a sitemap lists everything indexable for people, often lacks the clean Markdown versions of pages, leaves out external links, and covers far more than fits in a context window. An llms.txt is a curated overview that can complement both, for example by giving context for the content robots.txt allows.

For Google, the work that the guide does name is on your own pages. It asks for crawlable content, pages that are indexed and snippet-eligible, good page experience, less duplicate content, and a unique point of view, and it warns that producing a separate page for every query variation to manipulate AI responses violates Google's scaled content abuse policy. If you are unsure whether your pages are even indexed, working out which stage is broken between discovery and ranking is a better first hour than an llms.txt. Kanso's guide to Search Console's Generative AI performance report covers the one Google report that does count your impressions in these features.

What are llms-full.txt and Markdown page versions?

Both are companions to llms.txt that put the actual content in a form agents can fetch. The proposal asks sites to provide a clean Markdown version of each useful page at the same URL, either with .md added or with the extension replaced by .md, and the llms.txt links should point at those versions.

llms-full.txt is a different convention in the sources. Mintlify generates one that combines the whole documentation site into a single file for AI tools, listing each page with its title, source URL, description and full Markdown content. The supplied text of the proposal does not define llms-full.txt, so Mintlify's behavior is a vendor's implementation and not a rule you can cite as the standard.

The proposal also suggests two link relations so clients can find these files: one pointing from a page to its Markdown version, and one pointing from a page to the llms.txt that covers it. They can go in HTML or in an HTTP Link header, which can be set in server or CDN configuration without editing pages. Mintlify goes further and adds a Link header and an X-Llms-Txt header to every response, including 404 pages, so tools can discover the files without knowing where they sit.

How do you publish and check an llms.txt file?

Create the Markdown file, serve it at /llms.txt on your domain, and verify it opens. The steps below combine the proposal, a third-party generator's install guide and Lighthouse's behavior.

1. Decide which pages an agent should read first: the ones that explain what the site is and where the detail lives. Keep the file small enough to fit in a model's context.
2. Write the file in the format above, with at least the H1. Save it as lowercase llms.txt, the filename the specification uses.
3. Place it at the root of your domain. In a Next.js, React or Nuxt project that is the public folder, per llmstxtgenerate.com, and on a WordPress host it is the web root and not wp-content. On Shopify, that guide says to upload it through Files and add a redirect from /llms.txt.
4. Open yourdomain/llms.txt in a browser and confirm you see the Markdown. A server error is the case Lighthouse flags, while a 404 is simply marked Not Applicable.
5. Validate it. The llmstxtchecker.net tool tests for valid Markdown, an H1, a summary blockquote, optional detail sections and H2 sections of linked file lists, and it tests that the links resolve.
6. Revisit it when the site changes. The generator vendor suggests regenerating after major content updates; that is its advice and not a rule from the proposal.

Two constraints from Mintlify's documentation are useful even if you use another host. Its automatically generated index cannot exceed 100,000 characters, and when it would, the main file becomes a directory pointing to index files under /_llms/. Page descriptions are cut at 300 characters, and on fully authenticated sites AI tools that cannot sign in cannot read the files.

Should you still ship an llms.txt file?

Ship one when it costs minutes or is generated for you and agents are among your readers, and skip it when you would be maintaining it by hand to improve Google results. That rule rests on three criteria: what the file costs to produce and keep current, whether a plausible reader will fetch it, and whether Google is the goal.

How the rule was put together: each row applies the three criteria to a fact in the sources, such as the documentation-platform behavior in Mintlify's page or the test in Semrush's article. The verdicts themselves are judgment, not measurement. Nobody in the sources measured the payoff of an llms.txt, so the case for shipping is that the downside is small, not that the upside is proven.

A practical stop rule follows from that. If you publish one as an experiment, write down the date and what you will look at, such as whether any named AI crawler requests the file in your server logs. If nothing visits it by then, you have lost little and learned something specific to your site.

Where does Kanso's generator fit, and what is Kanso's stake?

Kanso has a planned free llms.txt generator, but it is not a working tool yet. The page at /tools/llms-txt-generator is a coming-soon page as of October 21, 2026, so this article does not tell you to use it.

Kanso also gains if you think llms.txt matters, which is a reason to weigh its view with care. The view in this article is that the file has a low cost, an unproven upside for AI answers and no effect on Google Search, and that is the same conclusion whether or not a generator ships. Third-party generators and checkers exist today, including the two in the Sources section.

If you want to see how readable your site is to AI systems more broadly, Kanso's GEO Agent checks how readable your site is to AI systems, estimates how citable it looks and turns the gaps into actions. That citability view is a model estimate: Kanso does not query ChatGPT, Gemini, Claude or Perplexity, so it is not a measurement of AI citations.

What is documented and what is analysis?

Everything in the first list below is attributed to a named source, and everything in the second is Kanso's inference from it. Keeping them apart lets you disagree with the analysis without doubting the facts.

Documented:

• The proposal requires only an H1, and allows the file at the root or at any path (llmstxt.org, v2).
• Lighthouse checks for the file, marks a 404 Not Applicable because it is optional, and flags a server error (Chrome for Developers, updated May 5, 2026).
• Semrush reports 951 domains with a file in July 2025 and 16,670 as of September 2026, and no confirmation from a major AI platform.
• Semrush reports no correlation between adding the file and AI-result performance in its test on Search Engine Land, and no visits from four named crawlers in the period it describes.
• Google's AI guide says its AI features rely on core Search ranking and quality systems; Kanso's checklist records the guide as treating llms.txt as unnecessary for Google Search.

Analysis:

• The downside of shipping a small, accurate file is close to zero, and the upside is unproven.
• A hand-edited file for a large site tends to go stale, which makes it worse than no file.
• Lighthouse checking for the file is a signal about developer tooling, not about Google Search ranking.
• The strongest likely reader is a coding agent reading documentation, because that is the use the proposal itself leads with.

What is unknown: whether any major assistant fetches the file at answer time, how often, and whether it changes what gets cited. Check server logs and Google's own guide for any update before relying on a figure here.

Frequently asked questions

Sources

• The /llms.txt file, v2, llmstxt.org, the proposal's own site
• llms.txt (Lighthouse audit), Chrome for Developers, page last updated May 5, 2026
• Optimizing your website for generative AI features on Google Search, Google Search Central
• What is llms.txt & should you use it?, Semrush Blog
• llms.txt, Mintlify documentation
• Free LLMs.txt Generator, llmstxtgenerate.com, a third-party tool
• llms.txt Checker, llmstxtchecker.net, a third-party tool
• The Weekly Growth Checklist: SEO, AI Search, Content, Competitors and Distribution, Kanso, October 13, 2026