← Back to blog/Tutorials

Make Astro and Starlight Docs Readable to AI Agents

Keep Starlight's design and Astro's publishing model. Add useful discovery files and retrieval interfaces, then test the deployed pages instead of assuming a plugin guarantees AI visibility.

F
Faizan Khan
• 8 min read

You do not need to leave Astro or Starlight to make documentation readable to AI agents. Starlight already provides a strong documentation reading experience. The work is to make sure agents can retrieve the right content and that those interfaces remain correct after a release.

This guide covers a practical self-managed setup. If the decision is about who should own hosting and upkeep, compare Fumadocs vs Starlight or DocsAlot vs Starlight. If you need the broader implementation checklist, start with making documentation AI-readable.

Separate the reader, the index and the connected tool

LayerJobWhat you maintain
Published HTMLA usable, accessible documentation siteContent, navigation and deployment
Markdown or a text representationA cleaner document for compatible retrieval toolsRoutes, content conversion and version alignment
llms.txtA useful map to the documentationIncluded pages, descriptions and working links
Optional MCPSearch/read capabilities for a connected assistantHosting, retrieval, access controls and client compatibility

These are complementary. A search index for your website is not automatically an MCP server, and a bundle of Markdown is not automatically discoverable by every assistant.

1. Verify the existing public pages first

Before installing anything, fetch a real article from the deployed site:

Bash
1curl --fail --location --max-time 20 \
2 --output /tmp/starlight-page.html \
3 "https://docs.example.com/guides/first-task/"

Check for a distinctive sentence from the article in the saved response. Verify the final URL and make sure the content is not a homepage fallback, an authentication screen or a CDN challenge.

Then use the site on a phone and with a keyboard. Adding an AI layer should not make the human reading experience worse. Starlight's official documentation and plugin catalog are the starting points for supported configuration and community options.

2. Generate a discovery index with an actual Starlight plugin

One community option listed in the Starlight ecosystem is starlight-llms-txt. Its documented purpose is generating llms.txt and related context bundles from the documentation content.

Follow the maintainer's compatibility guidance and review the dependency before adding it. Do this on a branch and keep your existing configuration rather than replacing it with a starter example.

Bash
1npm install starlight-llms-txt

The integration shape is:

JavaScript
1import { defineConfig } from 'astro/config';
2import starlight from '@astrojs/starlight';
3import starlightLlmsTxt from 'starlight-llms-txt';
4
5export default defineConfig({
6 site: 'https://docs.example.com',
7 integrations: [
8 starlight({
9 title: 'Example documentation',
10 plugins: [starlightLlmsTxt()],
11 }),
12 ],
13});

This is a minimal configuration example, not a replacement for your site's existing integrations, sidebar, locales or plugins. Merge the new plugin into the appropriate array and use the actual production site URL.

Check the plugin output in development, then build and test the deployment. A correct local index can still contain the wrong absolute links if the production origin or subpath is configured incorrectly.

Publishing this file does not train a model on demand or guarantee citations. Treat it as an additional discovery interface for consumers that use it.

3. Decide whether you need per-page Markdown

An llms.txt plugin can generate context bundles without necessarily providing a separate Markdown URL for every article. Check the actual plugin output instead of assuming those are the same feature.

The Starlight plugin catalog lists additional approaches, including raw Markdown routes and page-copy actions. Review which one fits your Astro version and deployment mode.

Whichever approach you choose, test a page containing more than plain paragraphs:

  • A warning or other callout with information the reader must not miss.
  • A code block with an example that requires explanation.
  • A link to another guide and a relative image.
  • A custom MDX or Astro component.
  • A translated or versioned page if your site has them.

Raw MDX source is not always an adequate substitute for rendered information. An agent that receives only a component tag may miss the instructions inside that component. Provide a meaningful text equivalent and preserve the links it needs to continue.

4. Test the URLs your site actually emits

Find the generated paths in the page or index, then fetch those exact URLs. Do not assume every Starlight setup uses the same suffix.

Bash
1curl --fail --location --max-time 20 \
2 --dump-header /tmp/starlight-text-headers.txt \
3 --output /tmp/starlight-page.txt \
4 "https://docs.example.com/REPLACE-WITH-GENERATED-TEXT-PATH"

Inspect both headers and body. A 200 response containing your normal HTML shell is not a successful Markdown export merely because the requested URL looked like a text path.

For a site deployed under a subpath, verify the public URLs under that subpath. Check that the source page, alternate representation, sitemap and canonical point to the same intended document. Astro's sitemap integration guide explains its site-URL configuration.

Do not put secrets or internal pages into a public index. Removing a page from the sidebar is not an access-control mechanism.

5. Add MCP only when there is a connected-client use case

If your customers use an assistant that can connect to documentation tools, MCP can give it a search/read interface. It is an optional additional layer, not a requirement for a normal public docs website.

Starlight's ecosystem includes community integrations for agent access. Review their hosting needs, data flow and authorization behavior before selecting one. A local protocol demo is not the same deployment as a reliable public endpoint.

Use this acceptance test with the intended client:

  1. Connect to the server using the documented connection method.
  2. Ask a task-specific question without supplying the target page URL.
  3. Check that search finds the intended source.
  4. Read the returned page and inspect its references.
  5. Ask a question your documentation does not answer and inspect the fallback.
  6. Verify that public access cannot retrieve private content.

An external answer engine will not necessarily discover or connect to that MCP server automatically. Keep connected retrieval results separate from the AI visibility measurement you do in unconnected search or chat products.

6. Build a release check instead of a one-time checklist

Choose a page changed in the release and a page that should remain unchanged. After publishing, compare the intended content with the deployed HTML, text representation and connected retrieval result.

Release checkExample failure it catches
Current text exists in HTMLAn old deployment or broken route
Text output contains the same updateA stale export or incomplete conversion
Discovery index links workA renamed page or wrong public origin
Retrieval finds the current procedureAn index that was not refreshed
Private content stays inaccessibleAn export or tool route bypassing access checks

Keep the checks small enough to run after ordinary releases. A large audit nobody repeats is less valuable than three important pages checked reliably.

When a managed docs platform is worth evaluating

Keep Starlight if Astro components, its design model and control of the codebase are important to you. It is a strong choice, and you can implement the agent-facing layer yourself.

Shortlist DocsAlot when you want the published design, hosting, search, Markdown and documentation MCP managed together. The advantage is less operational ownership, not that Astro cannot produce fast or attractive docs.

Review a separate draft before migrating. Custom components may need adaptation, and existing URLs need a deliberate mapping. Your current site can stay live while you evaluate the new reading experience.

Open live DocsAlot docs, compare DocsAlot with Starlight, or create a docs draft.

Continue the workflow

Continue This Guide