Most bad llms.txt files fail for a boring reason: they are trying too hard to be complete. They include too many links, too much adjacent content, and not enough judgment.
That usually comes from a reasonable instinct. Nobody wants to leave out something important. But llms.txt is one of those cases where completeness is not the goal.
The job of the file is to narrow the search space for an agent.
So the real question is not "what can we include?" It is "what should an agent read first if it is trying to actually use this product?"
If you have not read the basics yet, start with What Is llms.txt? A Practical Guide for SaaS Docs Teams. If you are deciding how to structure the file mechanically, read How to Add llms.txt to Your Documentation Site.
This post is about the harder editorial question: what belongs in the file, and what should stay out?
Start From the Job the Docs Need to Do
For most SaaS products, an agent using your docs is trying to do one of a few things:
- complete first-time setup
- authenticate correctly
- call the right endpoint or use the right SDK
- fix an error
- configure an integration
- understand a concept that blocks implementation
That should shape the file.
llms.txt is not a brand surface. It is not a docs directory. It is not a page inventory. It is a curated task map.
If a page does not materially help one of those tasks, it probably does not belong in the first version.
What to Include
For most SaaS docs teams, the default keep-list is pretty stable.
1. The real quickstart
Not the page you call the quickstart. The page that actually gets someone to a successful first integration.
If your current quickstart is mostly marketing framing or architectural background, do not pretend it is the best first step. Use the page that gets the work done.
2. Authentication
This is almost always worth including.
If your product supports more than one auth model, make the recommended path obvious.
3. Core API or product entry point
For API docs, that might be the main API overview or a reference landing page.
For product docs, it might be the setup overview or the main workflow page.
4. Errors, rate limits, retries, or troubleshooting
These are disproportionately useful for agents because they show up mid-task, not just at the start.
5. Webhooks, SDKs, or integrations that represent real adoption paths
Include the things people actually use, not every possible integration surface.
6. One or two concept pages if the product has real architecture
If your docs require understanding environments, tenancy, sync models, or permissions before implementation makes sense, include those pages explicitly.
What to Skip
This is where most files get noisy.
1. Marketing pages
Do not include:
- product overview pages written for buyers
- pricing
- generic landing pages
- announcement posts
Even if those pages are technically part of the same site, they do not usually help an agent solve a docs task.
2. Changelog noise
A changelog can be useful, but most products should not treat it as core llms.txt material.
If one release introduced a breaking auth change or a new required integration path, maybe that deserves a mention elsewhere. But a long release note archive should not dominate the file.
3. Thin or outdated pages
If you would not send a new engineer there, do not send an agent there either.
llms.txt is not the place to preserve internal guilt about docs debt.
4. Duplicate versioned docs
If you have multiple versions of the same guide, choose the canonical one unless older versions are still active and materially different.
5. Internal tools, dashboards, or auth-gated pages
This sounds obvious, but it is worth saying. The file should reflect the useful public docs surface.
6. Giant link lists with no structure
Even if every page is individually valid, a wall of links is still a weak file.
The Useful Test: Would You Hand This to a New Engineer?
This is probably the best editorial test I know for llms.txt.
Open the file and ask:
If a competent engineer joined the team today, would this be a good starting map?
If the answer is yes, you are probably close.
If the answer is no, the problem is usually one of three things:
- too many links
- the wrong links
- groupings that reflect internal org charts instead of user tasks
That test is simple, but it catches most of the mistakes.
What Different SaaS Docs Surfaces Usually Need
Not every docs site should include the same categories.
API docs
Bias toward:
- quickstart
- authentication
- API overview
- errors
- rate limits
- webhooks
- primary SDKs
Help center
Bias toward:
- getting started
- workspace or account setup
- permissions
- billing
- high-volume troubleshooting
- key integrations
Developer platform docs
Bias toward:
- quickstart
- local development
- CLI
- deployment
- architecture or data model
- auth
- SDKs
The point is not to force one universal structure. The point is to choose the pages that fit the actual job of the docs surface.
Pages That Feel Important but Usually Do Not Help
Some pages get included because they feel official.
Examples:
- glossary pages
- generic "overview" pages that say very little
- category landing pages with no real content
- press-announcement-style release posts
- "all integrations" directories
These are not always bad pages. They are just usually not high-leverage pages for agent guidance.
An agent benefits more from one strong webhook guide than from a top-level "Integrations" page that links to twenty products.
A Good File Usually Has a Point of View
This is the subtle part.
The best llms.txt files are not neutral. They express a recommendation:
- start here
- this is the canonical auth path
- these are the common failure modes
- this SDK is the usual default
- this concept matters before you implement anything else
That point of view is what makes the file helpful.
Without it, you still have documentation. You just do not have guidance.
That is also why auto-generated llms.txt files tend to feel mediocre. Automation is great at collecting links. It is bad at deciding which few links matter most.
A Practical Keep-or-Skip Checklist
When you are deciding whether a page belongs in llms.txt, ask:
- Does this page help someone complete a real implementation or support task?
- Is this page canonical, current, and worth trusting?
- Would I want an agent to read this before browsing the rest of the docs?
- Does it reduce ambiguity, or add more of it?
If the answer is mostly no, leave it out.
That is not a failure. It is the point of curation.
The Short Version
For SaaS docs, a good llms.txt usually includes:
- quickstart
- authentication
- core task pages
- operational docs like errors and rate limits
- key SDK or integration paths
- concept pages that unblock implementation
And it usually skips:
- marketing pages
- changelog noise
- stale docs
- duplicate versions
- auth-gated surfaces
- giant undifferentiated link dumps
The format is easy.
The real value comes from deciding what deserves to be there at all.
That is why llms.txt is less like a machine-generated inventory and more like a small editorial artifact for your docs system.