Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 22 additions & 4 deletions .claude/skills/docs-agent-ready/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ name: docs-agent-ready
description: Use when adding a new docs section or product area, editing llms.ts / the llms.txt or llms/[...slug] / llms-full.txt routes / get-llm-text / skill.md / .well-known endpoints, or working on the "agent score", "llms.txt", or anything "agent-ready" in the docs and site apps. Explains the invariants the Mintlify agent-readiness audit measures and how to hold them.
metadata:
author: Prisma
version: "2026.7.21"
version: "2026.7.24"
---

# Docs agent-readiness
Expand All @@ -15,8 +15,10 @@ Keep Prisma's docs machine-readable so the Mintlify **agent-score** audit does n
- **Root `llms.txt` < 50k bytes** (warn at 35k). It links to per-area section indexes, not every page.
- **Each section index < 50k bytes** (warn at 40k). Over budget means split the section.
- **Every page is reachable** — each `filterPagesForLLMsIndex` page appears in a section file or the root "Other pages" list. The guard asserts against the generated content, not just membership.
- **Directives in HTML + Markdown** — every page's Markdown (`getLLMText`) starts with the hidden `llms.txt` directive blockquote; the HTML page keeps a hidden directive as its first child.
- **HTML/Markdown parity** via `data-markdown-ignore` on human-only chrome so the Markdown mirrors the page.
- **Directives in HTML + Markdown** — every page's Markdown (`getLLMText`) starts with the hidden `llms.txt` directive blockquote; the HTML keeps a hidden directive as the **first child of `<body>` in the root layout** (`apps/docs/src/app/layout.tsx`), NOT inside the page component. Audits measure the directive's byte position in the body and warn when it sits past 50%, which is where it lands if rendered after the sidebar markup.
- **HTML/Markdown parity** via `data-markdown-ignore` on human-only chrome so the Markdown mirrors the page. The OpenAPI explorer (`APIPage` wrapper in `src/components/api-page.tsx`) carries `data-markdown-ignore` because the interactive reference has no markdown equivalent — the `.md` serves the generated API summary instead.
- **Markdown keeps real headings** — fumadocs' processed output emits headings as bare `Text [#anchor]` lines; `getLLMText` restores `##` markers from the page toc (`restoreHeadingMarkers` in `llm-markdown.ts`). Without them, parity checkers strip list-like heading text ("## 1. Set up …") and agents see prose instead of structure.
- **`<details>` blocks are converted** to a bold summary line + dedented body (`formatDetails` in `llm-markdown.ts`); serialized `<details>` children are 2-space indented, which silently breaks the code fences inside for markdown consumers.
- **`llms-full.txt` excludes** legacy `/orm/v6` and the Accelerate/Optimize products (`getLLMsFullPages`).
- **Skill + MCP endpoints live at BOTH roots**: `www.prisma.io` (apps/site) and `/docs` (apps/docs).

Expand All @@ -32,6 +34,8 @@ Keep Prisma's docs machine-readable so the Mintlify **agent-score** audit does n
| `/docs/.well-known/mcp[.json]` | `apps/docs/src/lib/mcp-discovery.ts` |
| `/skill.md`, `/.well-known/agent-skills/*` | `apps/site/src/lib/agent-skills.ts` (`buildSkillMarkdown`) |
| `/.well-known/mcp*` (site) | `apps/site/src/lib/agent-skills.ts` (`buildMcpDiscovery`, server cards) |
| `/docs/mcp` (MCP proxy) | `apps/docs/src/app/mcp/route.ts` → proxies protocol traffic to `mcp.prisma.io/mcp` |
| `/mcp` (site, MCP traffic) | header-matched `beforeFiles` rewrites in `apps/site/next.config.mjs` (browser GETs still get the marketing page) |

The route handlers are thin wrappers: shared builders in `llms.ts` are the single source of truth, so the guard measures exactly what the routes serve.

Expand All @@ -41,18 +45,32 @@ The route handlers are thin wrappers: shared builders in `llms.ts` are the singl

**(b) Section over budget.** When a section fails/warns on size, split it into two sections in `llmsSections` (narrower `prefixes`, or carve a sub-tree out with a new slug). Re-run the guard.

**(c) Changing page chrome** in `apps/docs/src/app/(docs)/(default)/[[...slug]]/page.tsx`: keep the hidden `llms.txt` directive as the first child, and put `data-markdown-ignore` on any human-only chrome (banners, nav, badges) so it stays out of the Markdown.
**(c) Changing page chrome.** The hidden `llms.txt` directive lives in `apps/docs/src/app/layout.tsx` as the first child of `<body>` — keep it there (before `<Banner`), never move it into the page component where the sidebar markup would push it past 50% of the HTML. In `[[...slug]]/page.tsx`, put `data-markdown-ignore` on any human-only chrome (banners, nav, badges) so it stays out of the parity comparison. New interactive/human-only MDX components should get `data-markdown-ignore` on their wrapper plus a markdown fallback in `normalizeProcessedMarkdown` (`llm-markdown.ts`), following `APIPage`/`formatApiPage`.

**(d) Changing the CLI workflow or MCP tools** in docs content: update the skill copy in `apps/site/src/lib/agent-skills.ts` AND `apps/docs/src/lib/agent-skill.ts` — they quote real commands and tool names. Keep them in sync with the Prisma Postgres quickstart and `content/docs/ai/tools/mcp-server.mdx`. The `commonQueries` links in `llms.ts` must point to existing pages (the guard fails on stale links).

**(e) Verification.**

```bash
pnpm --filter docs lint:agent-ready # all invariants + size table
pnpm --filter docs test:llm-markdown # markdown pipeline fidelity snapshots
pnpm --filter docs types:check # types
curl -s https://www.prisma.io/docs/llms.txt | head
curl -s https://www.prisma.io/docs/skill.md | head
curl -s https://www.prisma.io/.well-known/mcp
```

The guard prints a size table with per-file headroom so reviewers see how close each file is to its budget.

**(f) Reproducing the audit.** The audit is the `afdocs` npm CLI (https://afdocs.dev). To reproduce a report locally:

```bash
# version pinned against supply-chain surprises — bump deliberately
npx afdocs@0.18.7 check https://www.prisma.io/docs --sampling deterministic -v
# parity needs its upstream checks in the same run:
npx afdocs@0.18.7 check https://www.prisma.io/docs \
--checks markdown-url-support,content-negotiation,markdown-content-parity \
--sampling deterministic --format json -v
```

Audit gotchas encoded in the invariants above: the HTML directive check needs an `<a href*="/llms.txt">` within the first 10% of the (nav/script/style-stripped) `<body>` on sampled pages and warns when every match sits past 50%; the parity check compares HTML text segments against the `.md`, strips `data-markdown-ignore` elements from the HTML side, and only treats a fenced code block as protected when the fence starts at column 0 — which is why `<details>` bodies must be dedented and headings must keep their `#` markers. The separate "MCP Server Discoverable" check probes `<origin>/mcp` with an MCP initialize request (discovery documents alone do not count), which is what the `/docs/mcp` proxy route and the site `/mcp` rewrites are for.
177 changes: 153 additions & 24 deletions apps/docs/scripts/lint-agent-ready.ts
Original file line number Diff line number Diff line change
Expand Up @@ -146,6 +146,8 @@ if (unmatched.length > CATCHALL_WARN) {
// fast enough to cover the full set.
const directiveFailures: string[] = [];
const missingDescription: string[] = [];
const headingMarkerFailures: string[] = [];
const detailsLeaks: string[] = [];
for (const page of indexPages) {
let text: string;
try {
Expand All @@ -168,6 +170,27 @@ for (const page of indexPages) {
if (description && !text.includes(description)) {
missingDescription.push(page.url);
}

// Heading markers: the processed markdown emits headings as bare
// "Text [#anchor]" lines; getLLMText restores the `#` markers from the toc.
// If a toc anchor appears in the output, the line carrying it must be a real
// markdown heading — otherwise agents see prose and the afdocs parity check
// strips list-like heading text ("## 1. Set up …") on the markdown side.
for (const item of page.data.toc ?? []) {
if (typeof item.url !== "string" || !item.url.startsWith("#")) continue;
const anchorRef = `[${item.url}]`;
const anchorLine = lines.find((line) => line.includes(anchorRef));
if (anchorLine !== undefined && !/^#{1,6} /.test(anchorLine)) {
headingMarkerFailures.push(`${page.url} (${item.url})`);
}
}

// <details> blocks must be converted to plain markdown (formatDetails in
// llm-markdown.ts); a leaked <details> means its body is still 2-space
// indented, which breaks code fences for markdown consumers.
if (text.includes("<details")) {
detailsLeaks.push(page.url);
}
}

if (directiveFailures.length > 0) {
Expand All @@ -192,39 +215,88 @@ if (missingDescription.length > 0) {
pass("Description in markdown", "all frontmatter descriptions present in markdown");
}

if (headingMarkerFailures.length > 0) {
fail(
"Heading markers restored",
`${headingMarkerFailures.length} toc heading(s) rendered without markdown markers (restoreHeadingMarkers in llm-markdown.ts regressed):\n ${headingMarkerFailures
.slice(0, 10)
.join("\n ")}`,
);
} else {
pass("Heading markers restored", "all toc anchors in markdown output sit on real headings");
}

if (detailsLeaks.length > 0) {
fail(
"No <details> leakage",
`${detailsLeaks.length} page(s) leak raw <details> into markdown (formatDetails in llm-markdown.ts regressed):\n ${detailsLeaks
.slice(0, 10)
.join("\n ")}`,
);
} else {
pass("No <details> leakage", "all <details> blocks converted to plain markdown");
}

// ── Check 5b: HTML surface source guard ──────────────────────────────────────
// The rendered HTML page carries the same directive via a hidden element.
// Rendering React in this script is not worth it; instead guard at the source
// level that the docs page component emits a hidden element referencing llms.txt
// BEFORE it renders <DocsPage. This is a source-level guard, not a render test.
const docsPagePath = join(
scriptDir,
"..",
"src",
"app",
"(docs)",
"(default)",
"[[...slug]]",
"page.tsx",
);
// The rendered HTML page carries the same directive via a hidden element that
// must be the FIRST child of <body> in the ROOT LAYOUT — not inside the page
// component. Agent-readiness audits (afdocs "llms-txt-directive-html") measure
// the directive's byte position within the body and warn when it sits past 50%,
// which is where it lands if it renders inside the content area after the
// sidebar markup. Rendering React in this script is not worth it; instead guard
// at the source level that layout.tsx links llms.txt between <body> and the
// first real child (<Banner). This is a source-level guard, not a render test.
const docsLayoutPath = join(scriptDir, "..", "src", "app", "layout.tsx");
try {
const docsPageSource = readFileSync(docsPagePath, "utf8");
const docsPageRenderIndex = docsPageSource.indexOf("<DocsPage");
const llmsRefIndex = docsPageSource.indexOf("llms.txt");
if (docsPageRenderIndex === -1) {
fail("HTML directive source guard", `<DocsPage not found in ${docsPagePath}`);
const layoutSource = readFileSync(docsLayoutPath, "utf8");
const bodyIndex = layoutSource.indexOf("<body");
const llmsRefIndex = layoutSource.indexOf('href="https://www.prisma.io/docs/llms.txt"');
const bannerIndex = layoutSource.indexOf("<Banner");
const ignoreIndex = layoutSource.indexOf("data-markdown-ignore");
if (bodyIndex === -1) {
fail("HTML directive source guard", `<body not found in ${docsLayoutPath}`);
} else if (llmsRefIndex === -1) {
fail("HTML directive source guard", `page.tsx does not reference llms.txt`);
} else if (llmsRefIndex >= docsPageRenderIndex) {
fail("HTML directive source guard", "layout.tsx does not link llms.txt");
} else if (llmsRefIndex < bodyIndex) {
fail(
"HTML directive source guard",
"layout.tsx links llms.txt before <body>; the hidden directive must be the first child of <body>",
);
} else if (bannerIndex !== -1 && llmsRefIndex > bannerIndex) {
fail(
"HTML directive source guard",
"layout.tsx links llms.txt after <Banner; the hidden directive must be the first child of <body> so audits find it near the top of the HTML",
);
} else if (ignoreIndex === -1 || ignoreIndex > llmsRefIndex) {
fail(
"HTML directive source guard",
"page.tsx references llms.txt only after <DocsPage; the hidden directive must precede it",
"the hidden directive in layout.tsx must carry data-markdown-ignore so it stays out of the HTML/markdown parity comparison",
);
} else {
pass("HTML directive source guard", "hidden llms.txt directive precedes <DocsPage");
pass("HTML directive source guard", "hidden llms.txt directive is the first child of <body>");
}
} catch (error) {
fail("HTML directive source guard", `could not read ${docsPagePath}: ${String(error)}`);
fail("HTML directive source guard", `could not read ${docsLayoutPath}: ${String(error)}`);
}

// ── Check 5c: APIPage parity guard ───────────────────────────────────────────
// The interactive OpenAPI explorer on /management-api/endpoints/* has no
// markdown equivalent (per-language code samples, auth widgets); the wrapper in
// api-page.tsx must carry data-markdown-ignore so parity checkers compare only
// the generated markdown API reference.
const apiPagePath = join(scriptDir, "..", "src", "components", "api-page.tsx");
try {
const apiPageSource = readFileSync(apiPagePath, "utf8");
if (!apiPageSource.includes("data-markdown-ignore")) {
fail(
"APIPage parity guard",
"api-page.tsx no longer wraps the OpenAPI explorer in data-markdown-ignore; management-api endpoint pages will fail markdown/HTML parity",
);
} else {
pass("APIPage parity guard", "OpenAPI explorer is excluded from parity comparison");
}
} catch (error) {
fail("APIPage parity guard", `could not read ${apiPagePath}: ${String(error)}`);
}

// ── Check 6: common queries resolve to existing pages ────────────────────────
Expand Down Expand Up @@ -369,6 +441,63 @@ if (mcpErrors.length > 0) {
);
}

// ── Check 9b: MCP protocol endpoints (docs + site) ───────────────────────────
// Discovery documents alone do not satisfy "MCP server discoverable" audits —
// they probe the conventional `<origin>/mcp` endpoints with an MCP initialize
// request. /docs/mcp is a proxy route in this app; www.prisma.io/mcp is a
// marketing page, so MCP traffic there is routed by header-matched rewrites in
// the site next.config. Guard both at the source level.
const mcpEndpointErrors: string[] = [];
const docsMcpRoutePath = join(scriptDir, "..", "src", "app", "mcp", "route.ts");
try {
const routeSource = readFileSync(docsMcpRoutePath, "utf8");
if (!routeSource.includes(`"${MCP_URL}"`)) {
mcpEndpointErrors.push(`docs: src/app/mcp/route.ts does not proxy to "${MCP_URL}"`);
}
for (const handler of ["GET", "POST", "DELETE"]) {
if (!new RegExp(`export (async )?function ${handler}\\b`).test(routeSource)) {
mcpEndpointErrors.push(`docs: src/app/mcp/route.ts is missing the ${handler} handler`);
}
}
} catch (error) {
mcpEndpointErrors.push(`docs: could not read ${docsMcpRoutePath}: ${String(error)}`);
}

const siteNextConfigPath = join(scriptDir, "..", "..", "site", "next.config.mjs");
try {
const siteConfigSource = readFileSync(siteNextConfigPath, "utf8");
// All three header conditions are needed to cover the MCP Streamable HTTP
// transport: POST messages (content-type), the server event stream (accept),
// and session teardown (mcp-session-id). Split the config into per-rewrite
// chunks (each starts at its `source:` and ends before the next one) so a
// header key and the MCP destination must appear together in the SAME
// rewrite entry — matching across neighbouring entries would let a dropped
// header condition slip through.
const mcpRewriteChunks = siteConfigSource
.split(/(?=source: ")/)
.filter((chunk) => chunk.startsWith('source: "/mcp"'));
const requiredMcpHeaderKeys = ["accept", "content-type", "mcp-session-id"];
const missingMcpHeaderRewrites = requiredMcpHeaderKeys.filter(
(key) =>
!mcpRewriteChunks.some(
(chunk) => chunk.includes(`key: "${key}"`) && chunk.includes(`destination: "${MCP_URL}"`),
),
);
Comment thread
coderabbitai[bot] marked this conversation as resolved.
if (missingMcpHeaderRewrites.length > 0) {
mcpEndpointErrors.push(
`site: next.config.mjs is missing a /mcp rewrite to "${MCP_URL}" for header key(s): ${missingMcpHeaderRewrites.join(", ")}`,
);
}
} catch (error) {
mcpEndpointErrors.push(`site: could not read ${siteNextConfigPath}: ${String(error)}`);
}

if (mcpEndpointErrors.length > 0) {
fail("MCP protocol endpoints", `\n ${mcpEndpointErrors.join("\n ")}`);
} else {
pass("MCP protocol endpoints", "/docs/mcp proxy route + site /mcp rewrites present");
}

// ── Check 10: placeholder protectors are collision-safe ──────────────────────
// Regression coverage for the protect/restore pipeline shared by llm-markdown.ts
// and get-llm-text.ts. Adversarial inputs embed text that LOOKS like the internal
Expand Down
24 changes: 3 additions & 21 deletions apps/docs/src/app/(docs)/(default)/[[...slug]]/page.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -31,29 +31,11 @@ export default async function Page({ params }: { params: Promise<PageParams> })
const aiPromptSlug = (page.data as { aiPrompt?: string }).aiPrompt;
const promptContent = aiPromptSlug ? await getPromptContent(aiPromptSlug) : null;

const pageMarkdownUrl = `https://www.prisma.io${withDocsBasePath(page.url)}.md`;

return (
<>
<div
data-markdown-ignore
style={{
position: "absolute",
width: 1,
height: 1,
padding: 0,
margin: -1,
overflow: "hidden",
clip: "rect(0, 0, 0, 0)",
whiteSpace: "nowrap",
border: 0,
}}
>
For the complete Prisma documentation index optimized for AI agents, see{" "}
<a href="https://www.prisma.io/docs/llms.txt">https://www.prisma.io/docs/llms.txt</a>. A
markdown version of this page is available at{" "}
<a href={pageMarkdownUrl}>{pageMarkdownUrl}</a> (append <code>.md</code> to any docs URL).
</div>
{/* The hidden llms.txt directive for AI agents lives in the root layout
(src/app/layout.tsx) as the first child of <body> — agent-readiness
audits require it near the top of the HTML, before the sidebar. */}
<TechArticleSchema page={page} />
<BreadcrumbSchema page={page} />
<DocsPage
Expand Down
Loading
Loading