llms.txt for SEO: Does It Help AI Search Visibility?

llms.txt will not rank you in Google Search. Google has said they do not use special AI files for ranking, including llms.txt. If you still want the file, write it so it is accurate. Do not treat it as a ranking switch.

I keep this article because founders ask about it every month. The honest answer is split: some AI crawlers and tooling communities care about a site-level markdown map. Google Search, including AI Overviews and AI Mode, does not use that file as a special ranking input.

Google's wording is in the mythbusting section of optimizing for generative AI features: you do not need new machine-readable files, AI text files, or special markup to appear in Google Search, and Google Search itself does not use them.

What llms.txt is supposed to be

The proposal is a /llms.txt file at the site root, usually Markdown, that points language-model crawlers at the pages you consider canonical explanations. Think of it as a curator note: here is the docs index, here is the product, here are the policies, please do not treat the marketing blog as the spec.

The proposal lives at llmstxt.org. It is not a Google ranking document. It is a community convention. Treat it that way.

That can be useful when you have a large docs set, a lot of duplicate marketing URLs, and you want a polite hint for crawlers that look for the file. It is not a substitute for robots.txt, sitemaps, canonicals, or content quality.

Some writeups also mention /llms-full.txt as a longer dump. Same rules. Longer is not better if it is a paste of the blog. If you cannot maintain a short file, you will not maintain a long one. I would rather ship twenty lines that stay true than a novel that rots after the next Webflow slug change.

What Google Search actually says

Google may discover, crawl, and index many file types. Discovering a file is not the same as treating it as a ranking lever. Their AI optimization guide groups llms.txt with other 'special' markup you can ignore for Google Search.

The same guide says structured data is not required for generative AI search. The AI features doc says there are no additional technical requirements beyond ordinary Search eligibility. If your GEO vendor puts llms.txt on the critical path for 'AI Overview rankings,' they are not reading Search Central.

For eligibility, stay with AI features and your website. Indexed pages, snippet-eligible, crawl allowed. That is the Google story.

A file Google does not use for ranking cannot be the reason you appeared in an Overview. Do not write the case study that way.

If a consultant reports 'AI Overview wins' the week they added llms.txt, ask what else shipped. Titles, indexation, and a new article are more likely causes. Correlation with a file Google says they do not use for ranking is a story, not a result. Keep the chronology honest in your own reports too.

Who might actually fetch it

Some documentation hosts, independent AI crawlers, and internal RAG pipelines look for a site map written for models. That set changes. I will not invent a market-share table. If a specific crawler you care about documents llms.txt support, follow that crawler's rules. If they do not, you are decorating the root.

robots.txt remains the control for Googlebot's crawling for Search. Google-Extended is a separate conversation for some other Google systems. Do not confuse those controls with a Markdown wishlist in /llms.txt.

If your goal is to limit training or grounding in some Google systems outside Search, read Google-Extended documentation, not a blog about llms.txt. If your goal is to stay in Search AI features, do not noindex the pages you want cited. Those are opposite knobs. Mixing them is how teams 'optimize for AI' and accidentally leave Search.

How to write an accurate llms.txt if you keep one

Accuracy is the only reason to have the file. If it lists stale URLs, private staging hosts, or blog posts you no longer stand behind, you trained a crawler to trust the wrong map.

1. Put it at https://yourdomain.com/llms.txt, served as text or Markdown, publicly readable.
2. Start with a one-paragraph description of the company that matches the homepage.
3. Link the canonical product, docs, pricing, and policy URLs you want quoted.
4. Link one about or author URL so the entity is not anonymous.
5. Skip the 200-URL dump of every tag page and paginated archive.
6. Update it when those canonical URLs change, the same week you update the sitemap.

Write in plain language. Do not stuff keywords. Do not claim awards you cannot show on the site. The file should be a shorter, duller version of the truth already on your key URLs.

On a documentation site, point at the getting-started URL, the API reference, the changelog, and the pricing page. On a service site, point at services, proof, about, and booking. Do not point at tag archives, search result URLs, or filtered CMS lists. Those are convenience paths for humans, not canonical explanations.

If you have localized sites, either keep one file that links language homepages clearly, or maintain per-host files that do not mix languages in a single unlabeled list. Ambiguous maps create ambiguous quotes.

A shape that stays honest

• H1: company or product name.
• A short paragraph of what you do and who it is for.
• A small list of 'start here' URLs with one-line labels.
• Optional: optional docs you do not want treated as the spec, labeled as such.

What not to put in it

• Noindex URLs, login walls, and staging hosts.
• Every blog post from 2019 that you would not send a client.
• Affiliate landing pages you do not want quoted as your offer.
• Invented statistics or 'we rank first for AI search.'
• Instructions that contradict robots.txt.

If a URL is not fit to be cited, it does not belong in a file whose job is to suggest citations. That sounds obvious. I still see dumps of the entire sitemap copied into Markdown and called a strategy.

Also skip legal theater in the file. A paragraph that says 'by reading this you agree models may not quote us' is not how crawlers work, and it is not a robots rule. Use robots.txt, preview controls, and noindex when you actually want those behaviors. Do not hide a terms-of-service argument in Markdown and call it SEO.

robots.txt, sitemap, schema: still the real files

robots.txt tells crawlers where they may go. The XML sitemap lists canonical indexable URLs for search engines that use it. Article schema on articles helps Google understand headline, author, and dates. Those are documented. llms.txt is optional commentary.

If you care about schema that matches the page, that is a separate job from this file. I will cover the operator version in schema markup for SEO. For architecture and citations, use the AEO strategy hub. For why GEO products overclaim files like this, read generative engine optimization without the hype.

If you keep one, maintain it

Add llms.txt to the same release checklist as sitemap and robots. When you migrate platforms, map the file or you will advertise dead paths. When you kill a service, remove it. An unmaintained llms.txt is worse than none because it looks authoritative.

Do not report 'we shipped llms.txt' as an SEO win in a client deck. Report indexation, the URLs that earned clicks, and whether the commercial pages are still crawlable. If a non-Google crawler you care about starts using the file, you will see it in their logs, not in a Google ranking report.

A simple maintenance rule: if the sitemap changed, open llms.txt the same day. If a service URL 301s, update the map. If you are not willing to do that, delete the file. An abandoned map is not neutral. It is a confident list of the wrong pages.

The decision I actually make

Small marketing sites: skip it unless a partner or a docs platform you use asks for it. Large docs sites: a short, maintained file can be a courtesy. Google-only SEO programs: do not spend the sprint here.

If you are already doing the first-fixes work and the architecture work, adding llms.txt is a small optional artifact. If you are not doing that work, the file is a costume. I would rather you ship a quoteable service page this week than a Markdown index of pages that still say 'we craft experiences.'

The visibility work is still the unglamorous list: titles, extractable answers, HTML, identity, internal links, unique URLs. That is the AEO first-fixes article. llms.txt does not replace it and does not rank it.

If you want a one-line policy for the team: llms.txt is optional documentation for some machines. Google Search ranking is not one of those machines. Write it only if you will keep it true. Spend the hours you would have spent arguing about the file on the pages the file would have listed.

That is the whole argument. Useful as a map. Useless as a ranking lever at Google. Accurate or it is a liability. Optional for most brochure sites. Never a substitute for pages that can actually be quoted.

If a vendor made llms.txt the center of your AI search proposal, send me the proposal. I will mark what Google will not use, and what is still worth doing on the pages themselves.