docs(docs-audit): state the recall denominator in both headers (#13600)

#13306 (maintainer ruling 2026-08-31): a recall figure for affected-docs.mjs
must use the corpus it claims to cover — hand-written docs, excluding
content/docs/references/** — as its denominator. Excluding generated pages
is constructive design, not an omission: they are never hand-edited, so
prompting an author about one is wrong advice. The one figure ever computed
against a wider denominator (all content/docs edits) was measuring how often
the docs generator ran in the sampling window, not this tool's recall (31 of
its 46 ground-truth entries were generated pages the tool can never list).

Adds that statement to affected-docs.mjs's own header (the authoritative
half) and a short cross-reference in docs-drift-check.yml's header (which
already documents the separate, page-level structural-invisibility class
from #11356/#11357) so the two halves compose instead of each stating their
own thing. No behavior change; no new recall figure computed.

Co-authored-by: Claude <noreply@anthropic.com>

Author: os-project-manager

.github/workflows/docs-drift-check.yml

@@ -38,6 +38,22 @@ name: Docs Drift Check
 # run the tool on their own worktree. A reader who did that got a different list and
 # reported a correct row as a false positive, and the investigation of the non-existent
 # defect cost a full round. Same posture, one more thing the run knows and used not to say.
+#
+# THE RECALL DENOMINATOR (#13306, maintainer ruling 2026-08-31) — the OTHER half of the
+# structural-invisibility paragraphs above. #11356/#11357 are about a page that IS already
+# in this tool's scope and still goes unlisted (it restates a rule without naming what this
+# tool anchors on). Separately, and wider: scripts/docs-audit/affected-docs.mjs's own header
+# declares the CORPUS this whole check claims to cover — hand-written docs, excluding
+# `content/docs/references/**` (generated, never hand-edited; excluding it is this check's
+# constructive design, not an omission). Any recall figure for this check must use THAT
+# corpus as its denominator; one computed against all edits under content/docs measures how
+# often the docs generator ran in the sampling window, not this check — measured 2026-08-30,
+# 31 of 46 ground-truth entries in the one figure ever computed that way were generated
+# pages this check can never list (method and correction in #13306). This check stays
+# advisory-only (see the top of this file) and the ceiling does not license widening the
+# corpus to improve the number. Full statement in affected-docs.mjs's header — this
+# paragraph only points at it so the two files compose instead of each stating their own
+# half.
 
 on:
   pull_request:

scripts/docs-audit/affected-docs.mjs

@@ -13,6 +13,33 @@
 // Scope: hand-written docs only = content/docs/**/*.mdx MINUS content/docs/references/**
 // (references are generated from packages/spec and handled by a separate regenerate pass).
 //
+// THE ABOVE SCOPE IS THE RECALL DENOMINATOR (#13306, maintainer ruling 2026-08-31). Any
+// recall figure for this tool — "of the docs pages that should have been listed, how many
+// were" — MUST use the scope above (hand-written docs, generated pages excluded) as its
+// denominator. Excluding `content/docs/references/**` is not a gap this tool happens to
+// have: those pages are AUTO-GENERATED and nobody hand-edits them, so telling an author
+// "you may have affected this page" would be WRONG ADVICE, not missing advice — the
+// exclusion is constructive, by design, same as the Scope line states.
+//
+// ⇒ a recall ratio computed against a WIDER denominator — every edit under content/docs,
+// generated pages included — is not measuring this tool at all. It is measuring how often
+// the docs generator happened to run inside whatever window was sampled, because every one
+// of those runs counts as a "miss" this tool could structurally never have avoided. This is
+// exactly how the one such figure ever computed got it wrong: measured 2026-08-30 over a
+// 91-commit window ending at `c4ecf0c49` (method, replay and the corrected re-derivation in
+// #13306), 31 of its 46 ground-truth entries were `content/docs/references/**` pages this
+// tool cannot list on any run, at any recall — the ratio moved by a factor of 2.7 just from
+// widening the sampling window, which is the signature of a denominator not measuring the
+// thing.
+//
+// ⛔ Maintainer ruling: this ceiling is honest, not a defect, and does not license widening
+// the corpus to improve the number — that trades a real regression (prompting authors about
+// pages they must never touch) for a paper gain. The tool's advisory-only posture (see
+// .github/workflows/docs-drift-check.yml's own header) is unchanged. That file documents
+// the OTHER half of "why a listed number can still miss something" — a page already IN this
+// scope that goes unlisted because it restates a rule without naming what this tool anchors
+// on. Read both; they compose into one picture, not two competing ones.
+//
 // DERIVATION (#9192): a doc is "affected" when it NAMES SOMETHING THE CHANGE TOUCHED —
 // an ANCHOR — not when it merely mentions the changed package.
 //