What llms.txt is for
llms.txt is a plain Markdown file at the root of your domain that tells language models which pages matter and what each one contains. It is to AI crawlers roughly what robots.txt is to search crawlers, except that instead of restricting access it offers a curated route through your site.
The format is deliberately simple: an H1 with your name, a blockquote summary, then Markdown link lists grouped under headings. Anything under an ## Optional heading is understood to be skippable when a crawler is working to a token budget.
Why it is worth ten minutes
Traditional search rankings no longer carry over to AI answers the way people assume. Google's AI Mode and Perplexity draw roughly 90% of brand citations from Google's top ten results, but ChatGPT pulls only about 30% from that pool, with just 8% overlap with Google and Bing top results. Ranking well is no longer sufficient to be cited.
Adoption of llms.txt is still growing and no engine treats it as authoritative. It costs nothing, it cannot hurt you, and where it is read it makes the difference between a model summarising your homepage and a model finding the page that actually answers the question.
Where it fits
Think of it as one of three layers. This file handles discovery. Structured data handles machine comprehension of individual pages. Third-party presence — the communities, comparisons and mentions a model encounters elsewhere — handles credibility. The third is by far the heaviest, and it is why Reddit presence matters so much to AI visibility.
Questions
Does llms.txt actually do anything yet?
Support is partial and evolving. Several crawlers read it, none treat it as binding. It is a low-cost bet: no downside, meaningful upside if adoption continues.
Where does the file go?
The root of your domain — yoursite.com/llms.txt — served as text/plain or text/markdown, exactly like robots.txt.
What is llms-full.txt?
An optional companion holding expanded content rather than links, for crawlers that want the full text without following URLs. Worth adding once the basic file is in place.
Does this replace robots.txt or a sitemap?
No. They do different jobs and you should have all three. robots.txt controls access, the sitemap enumerates everything, llms.txt curates and explains.