llms.txt Explained: What It Is and How to Implement It

Last updated: June 20, 2026

If you publish anything online, you've probably seen the term llms.txt turn up in SEO threads and Webflow forums, usually attached to a promise that it will get your site into ChatGPT. Here's the plain version. An llms.txt file is a markdown file placed at the root of your website (yoursite.com/llms.txt) that gives large language models a curated guide to your most important content, helping AI tools find and understand your site without wading through navigation, scripts, and other clutter. This guide covers what it actually is, whether it's worth your time, and exactly how to add one, written for Vancouver and Canadian site owners and the developers who build for them.

---

What Is llms.txt?

llms.txt is a plain markdown file you place at the root of your domain, at yoursite.com/llms.txt, that hands large language models a short, curated map of what your site contains and which pages matter most. Rather than letting an AI tool stumble through your menus, cookie banners, and half a megabyte of JavaScript, you give it a clean summary with links to the content you actually want it to read.

The idea came from Jeremy Howard, co-founder of Answer.AI and fast.ai, who published the proposal on September 3, 2024. The specification lives at llmstxt.org. It's worth being precise about its status, because a lot of marketing copy isn't: llms.txt is a voluntary, community-driven convention, not an official standard that every AI engine has agreed to follow. No standards body has ratified it.

Howard's original problem was narrow and technical. Language models have limited context windows, and raw HTML burns most of that space on layout and tracking scripts rather than meaning. His reference implementation was the documentation for his own Python framework, FastHTML. In other words, the file was designed first for developer docs and coding assistants pulling API references at the exact moment a question is asked. That origin matters, and we'll come back to it when we get to whether the thing actually works.

---

llms.txt vs robots.txt vs sitemap.xml

You'll often see llms.txt described as "robots.txt for AI." It's a catchy comparison and a slightly misleading one. These three files solve different problems, and only one of them is about handing instructions to language models.

robots.txt tells crawlers what they're allowed to touch. sitemap.xml hands search engines a full directory of your URLs. llms.txt does neither. It doesn't grant or deny permission, and it's not meant to be exhaustive. It's a curated, human-written summary that points a model at your best pages and explains, in plain words, what each one is for. If robots.txt is a rulebook and sitemap.xml is an index, llms.txt is closer to a guided tour.

---

Does llms.txt Actually Work? (Current Adoption)

This is where a lot of articles go vague, so let's be direct. As of mid-2026, llms.txt is widely discussed and thinly adopted, and no major AI provider has publicly committed to using it as a ranking or citation signal.

The companies you'd expect to be on board mostly are. Anthropic, Stripe, Cloudflare, Perplexity, Cursor, Mintlify, and Zapier all publish llms.txt files. But look at where those files live, and the picture sharpens: almost all of them sit on developer documentation subdomains. They exist so coding assistants like Cursor and Claude Code can pull clean API references when a developer asks. That's the original use case working exactly as intended. It isn't evidence that a general AI search engine reads the llms.txt on an arbitrary marketing site before answering a question about that business.

Google has been unusually blunt here. Its official guidance on optimising for generative AI features states that you don't need to create special machine-readable files, and that Google Search, including its AI features, doesn't use llms.txt. OpenAI and the other large providers have made no commitment in either direction.

So should you bother? For most sites, the honest answer is a qualified yes, but for the right reasons. The file is cheap to create and almost free to maintain. It gives you a clean, accurate description of your business that an AI tool can use if it chooses to, and that reduces the odds of a model inventing something about you from stale third-party pages. And if adoption does accelerate, you're already set up rather than scrambling. Treat it as low-cost insurance and a tidy content signal, not as a traffic strategy. Anyone selling it as a guaranteed lever into ChatGPT is promising more than the evidence currently supports.

---

What Goes in an llms.txt File

The format is deliberately simple, which is part of why it caught on. A valid llms.txt needs only a few elements:

• An H1 with the name of your site or project, on the very first line.
• A blockquote summary: one sentence describing what the site is.
• One or more sections, each a markdown H2 followed by a bulleted list of links, with a short description after each link.
• An optional section at the end for pages that are useful but not essential.

Because it's plain markdown, any language model can parse it without a special schema, and you can read and edit it in any text editor. Here's a realistic example for a small studio site:

# Parabolic Studio

> Vancouver web and brand design studio building fast, custom websites for small businesses and nonprofits.

## Services
- [Web Design](https://www.parabolicstudio.ca/services/web-design): Custom website design and build on Webflow and WordPress.
- [Brand Design](https://www.parabolicstudio.ca/services/brand-design): Logo, identity, and visual systems for growing businesses.

## Guides
- [Webflow vs WordPress for business sites](https://www.parabolicstudio.ca/blog/webflow-vs-wordpress-shopify-squarespace-business-sites): How to choose a CMS based on design, performance, and cost.

## Optional
- [Contact](https://www.parabolicstudio.ca/contact): Start a project or request a quote.

There's also a companion file, llms-full.txt, which holds the actual full text of your key pages flattened into one markdown document rather than just links to them. It's heavier, and mainly relevant for documentation-driven sites that want an assistant to ingest everything in a single pass. For a typical business site, the standard llms.txt is plenty to start with.

---

How to Create and Add llms.txt to Your Site

Building the file is the easy part. Getting it to the root of your domain is where the platform you use starts to matter.

1. Draft the file. Start with your site name as the H1, a one-line blockquote summary, and a handful of sections. Keep it short and specific. A page of well-chosen links beats a wall of everything.
2. List your most important pages with accurate descriptions. Pick the 10 to 20 pages that genuinely represent what you do: core services, your best guides, about, and contact. Write a plain, honest one-liner for each, the kind you'd say out loud to describe the page.
3. Host it at your site root. The file has to resolve at yoursite.com/llms.txt, not inside a subfolder. How you get it there depends on your platform, covered below.

Webflow

Webflow added native support for this. In your site settings you can upload an llms.txt file, and Webflow serves it at the root of your custom domain while keeping it out of the search index. Two things to know going in: the file only appears on your published custom domain, not on the .webflow.io staging domain, and Webflow's implementation is scoped to the llms.txt file itself rather than generating markdown versions of every page. If you're weighing platforms more broadly, our breakdown of choosing between Webflow and WordPress covers the wider trade-offs.

WordPress

If you run Yoast SEO, there's a built-in toggle under Settings, in the Site Features area under AI tools. Switch it on and Yoast generates the file and refreshes it weekly. One caveat worth flagging: by default Yoast lists only a handful of your most recent posts and pages, which is rarely the curated set you actually want. Use the manual page selection option, or reach for a dedicated plugin such as Website LLMs.txt, Rank Math, SEOPress, or All in One SEO, all of which can generate the file and respect your noindex rules. You can also skip plugins entirely and upload a hand-written file to your site root over FTP or your host's file manager.

Other Platforms and Custom Builds

On a custom-coded site, a static host, or a framework build, you simply place the llms.txt file in the public web root and deploy it like any other static asset. Once it's live, confirm it by visiting yoursite.com/llms.txt directly and checking that it loads as clean plain text with no formatting wrapped around it.

---

llms.txt Best Practices and Common Mistakes

The line between a file that helps and one that quietly hurts is accuracy. A few habits keep yours on the right side of it.

• Curate, don't dump. Link the pages that define your business, not every URL you own. A short, sharp file is more useful to a model than a mirror of your sitemap.
• Write descriptions for a stranger, not for search. Webflow's own guidance puts it well: write the file as if you're briefing a new intern who has never seen your site. Say what each page does. Skip the "cutting-edge solutions for everyone" language, because it tells a model nothing.
• Keep it canonical. Point only to canonical URLs, and make sure each one resolves. Links that redirect or 404 make the file worse than having none.
• Keep private things private. Anything you'd not want surfaced in an AI answer, internal pages, client-only areas, thank-you URLs, doesn't belong in here.
• Treat it as living, not finished. When your services, pricing, or key pages change, update the file. An llms.txt that describes last year's business is actively misleading, which is the one outcome you're trying to avoid.

If keeping a file like this current alongside everything else isn't realistic for your team, that's a fair reason to hand it off. Our AI search optimization services cover the setup and the upkeep as part of a broader plan.

---

Frequently Asked Questions

---