From d6ebb1607813e9df6a5f071119cdd6d498725f36 Mon Sep 17 00:00:00 2001 From: Justin Dunham Date: Thu, 30 Jul 2026 10:04:24 -0700 Subject: [PATCH 1/2] Add learn page on optimizing API docs for LLMs and AI search. --- .../ai-optimize-api-docs-llms-ai-search.md | 69 +++++++++++++++++++ learn/ai-for-docs/sidebars.yaml | 2 + 2 files changed, 71 insertions(+) create mode 100644 learn/ai-for-docs/ai-optimize-api-docs-llms-ai-search.md diff --git a/learn/ai-for-docs/ai-optimize-api-docs-llms-ai-search.md b/learn/ai-for-docs/ai-optimize-api-docs-llms-ai-search.md new file mode 100644 index 000000000..28862ab97 --- /dev/null +++ b/learn/ai-for-docs/ai-optimize-api-docs-llms-ai-search.md @@ -0,0 +1,69 @@ +--- +seo: + title: Use AI to optimize your API docs for LLMs and AI search + description: How to structure, format, and test API documentation so LLMs and AI search tools retrieve it correctly, plus where llms.txt actually helps. +--- + +# Use AI to optimize your API docs for LLMs and AI search + +Developers increasingly ask an AI assistant how to authenticate instead of reading a guide start to finish, and AI search results now summarize a page before anyone clicks through to it. When that summary is wrong, the reader blames the API, not the model that misread the docs. + +The fix is not a new tool. It is writing documentation the way an AI system actually reads it: in chunks, by heading, and by exact terminology, so the fragment that gets retrieved is the right one. This article covers [how to optimize your docs for LLMs](https://redocly.com/blog/optimizations-to-make-to-your-docs-for-llms), where `llms.txt` genuinely helps, and how to test the result before a reader finds the gap for you. + +## Why documentation written for humans confuses AI systems + +A person skimming a page uses layout, color, and position to separate ideas even when the prose runs together. An AI system gets none of that. It processes text in paragraph-sized "chunks" and relies on "vector embeddings" to decide which chunk answers a given question, so when a section mixes topics, such as rate limits described inside an authentication guide, the system has no visual cue to pull them apart. + +That same chunking causes a second failure mode: retrieval by similarity, not by meaning. If two sections read alike but cover different endpoints, an assistant can hand back the wrong one entirely, which is common in API docs that describe several authentication methods or endpoints with near-identical names. Both problems trace back to the same root cause: the page was structured for a reader who could see the whole thing at once, not for a system that only sees the piece it retrieved. + +## Give AI a predictable structure to navigate + +Headings work like a map, and a system that skips levels loses the map. A page that jumps from an H1 straight to an H3, then back up to an H2, gives an AI system no reliable way to tell whether a topic is a subpoint or a sibling, so it can misjudge which surrounding text belongs to which claim. + +This is confusing for AI: + +``` +# API Reference +### Rate Limits +#### Authentication +## Error Codes +``` + +This works better: + +``` +# API Reference +## Authentication +### API Key Setup +### Token Management +## Rate Limits +## Error Codes +``` + +The same discipline applies to code. An install command written inline with prose, "install the SDK with `npm install @company/api-client` and then import it," can get merged or truncated when an assistant reformats it for an answer. A fenced code block for the command and a separate one for the import statement removes that ambiguity, because there is nothing left to misparse. If your documentation platform generates this structure from an OpenAPI description, most of that formatting discipline happens automatically, which frees a writer to focus on accuracy rather than markup. + +## Pick one term per concept and watch for drift + +AI systems have to guess whether "API key," "access token," and "auth credential" mean the same thing or three different things, and that guess is a probability, not a fact. Naming the same concept the same way every time removes the guess. It can read a little repetitive to a person, but it gives an assistant one clear answer to retrieve instead of three ambiguous ones. + +That consistency erodes on its own over time. A feature gets renamed, an older page keeps the old name, and a reader asking about the new name gets an answer built on the outdated one, an effect worth watching for because it compounds: a handful of stale terms today becomes a pattern of conflicting answers in a year, and conflicting answers are what turn into support tickets. A quick way to check for it: ask an AI tool about a feature using your current name, and see whether it responds with terminology you retired months ago. + +## Where llms.txt and AI search actually help + +`llms.txt` is a plain-text index, configured under [`seo.llmstxt`](https://redocly.com/docs/realm/config/seo) in `redocly.yaml`, that lists a site's highest-value pages so a tool can find them without crawling everything. [Publishing that index signals which guides matter most, though it does not replace good page structure](https://redocly.com/learn/ai-for-docs/ai-help-developers-find-understand-apis), and it works best when the pages it points to already separate tasks cleanly. + +It is worth being honest about how far that signal reaches. [Redocly ran its own tests on `llms.txt`](https://redocly.com/blog/llms-txt-overhyped) and found that no model acted on the file unless someone pasted its contents directly into a conversation, and access logs showed the file itself is rarely fetched at all. Pasting it in also produced weaker answers than pasting the actual docs would have. + +The same round of testing found real value in Docs "MCP" (Model Context Protocol) integrations instead, which connect an assistant to documentation directly rather than asking it to notice a static index. Treat `llms.txt` as a small, easy signal worth publishing, not the mechanism that makes your docs AI-search-ready: that work still happens on the page itself. + +## Ask AI the same questions your readers would + +The most reliable check on any of this is direct: ask a general-purpose AI tool the same questions a developer would, using only your published docs as its source. Try "how do I authenticate with this API," "show me a request for this endpoint," and "what does this error code mean," then read the answer the way a new integrator would. + +An accurate, complete answer means your structure is working. A confused or incomplete one points to exactly the page that needs a heading fix or a renamed term, which is faster feedback than waiting for a support ticket to reveal the same gap. This is a narrower version of [using AI as a usability tester](https://redocly.com/learn/ai-for-docs/ai-usability-testing): instead of watching an assistant complete a full workflow, you are watching it answer one question, and grading the fragment it retrieved. + +None of this makes deterministic checks unnecessary. [Retrieval-augmented generation](https://redocly.com/learn/ai-for-docs/ai-modern-api-docs), or "RAG," grounds an assistant's answer in your actual docs instead of only its training data, but it can only ground the answer in text that is structured well enough to retrieve correctly in the first place. Structure and terminology are what you control, and the model's summary is what you are testing. + +## How Redocly can help + +Optimizing docs for AI search starts with the same page discipline this article covers, but publishing it is where [Revel](https://redocly.com/revel) fits. Revel is Redocly's external developer portal, and it carries hosted search and AI assistant configuration on the same pages that generate your `llms.txt` index, so the structure a reader sees and the structure an assistant retrieves come from one source. Teams that need the same task-shaped page discipline applied internally can pair Revel with [Reef](https://redocly.com/reef), Redocly's internal catalog, so external AI search and internal discovery both reward the same well-structured pages instead of two different sets of docs drifting apart. diff --git a/learn/ai-for-docs/sidebars.yaml b/learn/ai-for-docs/sidebars.yaml index 7dd3c8899..ab14ad4f9 100644 --- a/learn/ai-for-docs/sidebars.yaml +++ b/learn/ai-for-docs/sidebars.yaml @@ -30,6 +30,8 @@ label: Use AI to build a searchable API catalog for your team - page: ai-help-developers-find-understand-apis.md label: Use AI to help developers find and understand your APIs faster +- page: ai-optimize-api-docs-llms-ai-search.md + label: Use AI to optimize your API docs for LLMs and AI search - page: ai-check-terminology-consistency-across-docs.md label: Use AI to check terminology consistency across your docs - page: ai-usability-testing.md From bd3cfba416262bfee8f3ee29f75171b0092d6226 Mon Sep 17 00:00:00 2001 From: Justin Dunham Date: Thu, 30 Jul 2026 11:30:48 -0700 Subject: [PATCH 2/2] Refresh learn/ai-for-docs draft from latest Sonnet 5 run. --- .../ai-optimize-api-docs-llms-ai-search.md | 64 ++++++++----------- 1 file changed, 27 insertions(+), 37 deletions(-) diff --git a/learn/ai-for-docs/ai-optimize-api-docs-llms-ai-search.md b/learn/ai-for-docs/ai-optimize-api-docs-llms-ai-search.md index 28862ab97..d09d63558 100644 --- a/learn/ai-for-docs/ai-optimize-api-docs-llms-ai-search.md +++ b/learn/ai-for-docs/ai-optimize-api-docs-llms-ai-search.md @@ -1,69 +1,59 @@ --- seo: title: Use AI to optimize your API docs for LLMs and AI search - description: How to structure, format, and test API documentation so LLMs and AI search tools retrieve it correctly, plus where llms.txt actually helps. + description: How to structure headings, terminology, and code examples so LLMs and AI search tools retrieve accurate answers from your API docs. --- # Use AI to optimize your API docs for LLMs and AI search -Developers increasingly ask an AI assistant how to authenticate instead of reading a guide start to finish, and AI search results now summarize a page before anyone clicks through to it. When that summary is wrong, the reader blames the API, not the model that misread the docs. +Developers increasingly ask an AI assistant how to authenticate instead of reading a reference page in order. When the underlying docs are not ready for that kind of reading, the assistant returns a mix of old and new terms, a broken code snippet, or an answer pulled from the wrong section entirely. The developer opens a support ticket, and the team never learns that the docs, not the assistant, caused the confusion. -The fix is not a new tool. It is writing documentation the way an AI system actually reads it: in chunks, by heading, and by exact terminology, so the fragment that gets retrieved is the right one. This article covers [how to optimize your docs for LLMs](https://redocly.com/blog/optimizations-to-make-to-your-docs-for-llms), where `llms.txt` genuinely helps, and how to test the result before a reader finds the gap for you. +This is already showing up in the numbers. A recent Stack Overflow Developer Survey found that 84% of developers now use, or plan to use, AI tools in their workflow, while 46% say they do not trust the accuracy of what those tools return, up from 31% a year earlier, according to [Redocly's analysis of how to optimize your docs for LLMs](https://redocly.com/blog/optimizations-to-make-to-your-docs-for-llms). This article walks through why that mistrust builds and what to change in your headings, terminology, formatting, and testing so both people and AI search tools get the right answer on the first try. -## Why documentation written for humans confuses AI systems +## Why AI search changes what counts as good docs -A person skimming a page uses layout, color, and position to separate ideas even when the prose runs together. An AI system gets none of that. It processes text in paragraph-sized "chunks" and relies on "vector embeddings" to decide which chunk answers a given question, so when a section mixes topics, such as rate limits described inside an authentication guide, the system has no visual cue to pull them apart. +A human reader skims a page, notices the visual hierarchy, and fills in missing detail from context. An AI model does none of that. Most retrieval systems break a page into paragraph-sized chunks and store each one as a "vector embedding," a numeric representation used to find the passage that best matches a question. When a single section mixes two topics, such as authentication and rate limits, the model cannot tell them apart, so a question about one pulls in details about the other. -That same chunking causes a second failure mode: retrieval by similarity, not by meaning. If two sections read alike but cover different endpoints, an assistant can hand back the wrong one entirely, which is common in API docs that describe several authentication methods or endpoints with near-identical names. Both problems trace back to the same root cause: the page was structured for a reader who could see the whole thing at once, not for a system that only sees the piece it retrieved. +The same mechanism causes wrong-section errors. If two pages describe similar-sounding operations, the embedding for one can sit close enough to the other that the assistant retrieves the wrong page and answers with confidence anyway. Fixing this is less about writing more content and more about organizing what already exists so each chunk answers one question well. -## Give AI a predictable structure to navigate +## Structure your docs so AI can parse them -Headings work like a map, and a system that skips levels loses the map. A page that jumps from an H1 straight to an H3, then back up to an H2, gives an AI system no reliable way to tell whether a topic is a subpoint or a sibling, so it can misjudge which surrounding text belongs to which claim. +### Keep heading hierarchy predictable -This is confusing for AI: +AI systems use headings as a map of how concepts relate, so skipping levels (jumping from an `H1` straight to an `H3`) breaks that map and can cause the model to misread which section a detail belongs to. A predictable pattern, main topic as `H2`, subtopics as `H3`, lets an assistant find the authentication section and know that API key setup and token management both belong under it. Documentation platforms that generate structure from an OpenAPI description handle much of this automatically, which frees your team to check accuracy instead of fixing heading levels by hand. -``` -# API Reference -### Rate Limits -#### Authentication -## Error Codes -``` +### Use one term for each concept -This works better: +Calling the same thing an "API key" in one paragraph and an "access token" in the next forces a model to guess whether you mean two different things or one. Pick a single term per concept and use it everywhere, even when the repetition reads a little stiff to a human. Linting rules, the kind Redocly CLI can enforce across an entire spec, catch this kind of drift automatically once you decide on the term. -``` -# API Reference -## Authentication -### API Key Setup -### Token Management -## Rate Limits -## Error Codes -``` +### Format code examples so they survive processing -The same discipline applies to code. An install command written inline with prose, "install the SDK with `npm install @company/api-client` and then import it," can get merged or truncated when an assistant reformats it for an answer. A fenced code block for the command and a separate one for the import statement removes that ambiguity, because there is nothing left to misparse. If your documentation platform generates this structure from an OpenAPI description, most of that formatting discipline happens automatically, which frees a writer to focus on accuracy rather than markup. +Inline commands mixed into a sentence often get merged or altered when an assistant reformats them into an answer. Put install commands and import statements in their own fenced code blocks instead of folding them into a sentence, and keep one action per block. That small change is often the fastest win available, because it fixes the exact spot where developers copy and paste directly from an AI's response. -## Pick one term per concept and watch for drift +### Describe diagrams and screenshots in words -AI systems have to guess whether "API key," "access token," and "auth credential" mean the same thing or three different things, and that guess is a probability, not a fact. Naming the same concept the same way every time removes the guess. It can read a little repetitive to a person, but it gives an assistant one clear answer to retrieve instead of three ambiguous ones. +An assistant working from text alone cannot see your authentication flow diagram, so anything the image communicates needs a text version nearby. A short numbered list of the same steps the diagram shows gives the model something to retrieve and quote, instead of leaving that information invisible to it. -That consistency erodes on its own over time. A feature gets renamed, an older page keeps the old name, and a reader asking about the new name gets an answer built on the outdated one, an effect worth watching for because it compounds: a handful of stale terms today becomes a pattern of conflicting answers in a year, and conflicting answers are what turn into support tickets. A quick way to check for it: ask an AI tool about a feature using your current name, and see whether it responds with terminology you retired months ago. +### Say exactly what you mean -## Where llms.txt and AI search actually help +Pronouns such as "it" or "this" force an assistant to guess which noun they refer to, and it often guesses wrong. Naming the config file, the server, or the endpoint again, even when a human reader would find the repeat unnecessary, removes an entire category of retrieval error. -`llms.txt` is a plain-text index, configured under [`seo.llmstxt`](https://redocly.com/docs/realm/config/seo) in `redocly.yaml`, that lists a site's highest-value pages so a tool can find them without crawling everything. [Publishing that index signals which guides matter most, though it does not replace good page structure](https://redocly.com/learn/ai-for-docs/ai-help-developers-find-understand-apis), and it works best when the pages it points to already separate tasks cleanly. +## Watch for terminology drift over time -It is worth being honest about how far that signal reaches. [Redocly ran its own tests on `llms.txt`](https://redocly.com/blog/llms-txt-overhyped) and found that no model acted on the file unless someone pasted its contents directly into a conversation, and access logs showed the file itself is rarely fetched at all. Pasting it in also produced weaker answers than pasting the actual docs would have. +Feature names change, branches get renamed, and old pages linger in a vector index long after the product moves on. When an API evolves from a "master branch" to a "main branch" but earlier docs never get updated, a developer asking about the current workflow can receive instructions built for the old one. Because inconsistencies compound the longer they sit, a periodic audit for outdated terms is worth scheduling rather than leaving until a support ticket flags it. -The same round of testing found real value in Docs "MCP" (Model Context Protocol) integrations instead, which connect an assistant to documentation directly rather than asking it to notice a static index. Treat `llms.txt` as a small, easy signal worth publishing, not the mechanism that makes your docs AI-search-ready: that work still happens on the page itself. +## Design pages for people and assistants together -## Ask AI the same questions your readers would +None of this means writing two versions of your docs. [Task-shaped headings](https://redocly.com/learn/ai-for-docs/ai-help-developers-find-understand-apis) such as "get an access token" or "list webhooks" work well for a developer skimming a table of contents and for an assistant retrieving one fragment to answer one question, because both readers are looking for the same thing. External developers using Revel and internal teams using Reef can share the same task-based headings, so search inside either surface returns familiar language instead of two competing vocabularies. -The most reliable check on any of this is direct: ask a general-purpose AI tool the same questions a developer would, using only your published docs as its source. Try "how do I authenticate with this API," "show me a request for this endpoint," and "what does this error code mean," then read the answer the way a new integrator would. +Publishing an [llms.txt](https://redocly.com/llms.txt) file gives AI tools a short, machine-readable index of your most important pages, such as authentication, quickstarts, and changelog, without asking them to crawl the entire site. Redocly [automates the creation of an llms.txt file](https://redocly.com/blog/updates-2025-05) from the same OpenAPI-driven build that generates the rest of your docs, so the index stays current as pages change instead of drifting out of sync. -An accurate, complete answer means your structure is working. A confused or incomplete one points to exactly the page that needs a heading fix or a renamed term, which is faster feedback than waiting for a support ticket to reveal the same gap. This is a narrower version of [using AI as a usability tester](https://redocly.com/learn/ai-for-docs/ai-usability-testing): instead of watching an assistant complete a full workflow, you are watching it answer one question, and grading the fragment it retrieved. +## Test how AI actually reads your docs -None of this makes deterministic checks unnecessary. [Retrieval-augmented generation](https://redocly.com/learn/ai-for-docs/ai-modern-api-docs), or "RAG," grounds an assistant's answer in your actual docs instead of only its training data, but it can only ground the answer in text that is structured well enough to retrieve correctly in the first place. Structure and terminology are what you control, and the model's summary is what you are testing. +The most reliable way to know whether these changes worked is to ask an AI tool the same questions a developer would: how do I authenticate, show me a request example for this endpoint, what does this error code mean. When the answer is accurate and complete, the structure behind it is doing its job. When the model mixes up terms or skips a step, that mismatch points to the exact page worth fixing next, a practice covered in more depth in [testing your documentation's usability](https://redocly.com/learn/ai-for-docs/ai-usability-testing) with AI directly. + +Because [OpenAPI-driven documentation](https://redocly.com/learn/ai-for-docs/ai-modern-api-docs) generates consistent headings and formatting by default, teams that build docs this way start the testing process with fewer problems to find. Track time to a developer's first successful call rather than page views alone, since a rising view count paired with a flat success rate usually means AI search is sending readers to a page that looks relevant but does not answer their question. ## How Redocly can help -Optimizing docs for AI search starts with the same page discipline this article covers, but publishing it is where [Revel](https://redocly.com/revel) fits. Revel is Redocly's external developer portal, and it carries hosted search and AI assistant configuration on the same pages that generate your `llms.txt` index, so the structure a reader sees and the structure an assistant retrieves come from one source. Teams that need the same task-shaped page discipline applied internally can pair Revel with [Reef](https://redocly.com/reef), Redocly's internal catalog, so external AI search and internal discovery both reward the same well-structured pages instead of two different sets of docs drifting apart. +Once your docs are structured for AI retrieval, the surface developers actually search still matters. [Revel](https://redocly.com/revel) is Redocly's external developer portal, and its hosted [search configuration](https://redocly.com/docs/realm/config/search) and [AI assistant configuration](https://redocly.com/docs/realm/config/ai-assistant) let you tune how both traditional search and conversational AI features retrieve answers from the same task-shaped pages this article describes. Instead of hoping a generic search bar finds the right fragment, Revel gives you direct control over the surface where people and AI search tools meet your docs.