diff --git a/docs/development/agent-experience-audit.md b/docs/development/agent-experience-audit.md index 7bf45636e..b893884ca 100644 --- a/docs/development/agent-experience-audit.md +++ b/docs/development/agent-experience-audit.md @@ -4088,3 +4088,21 @@ The empty labelled div is the same disagreement in miniature: the `ai` mode prin **Repair:** none in code — the surface is a third-party tool description we do not own, so the repair is a reader. For any ARIA naming question the instrument is CDP `Accessibility.getPartialAXTree` (or `getFullAXTree`), never a snapshot formatter; where a snapshot is wanted anyway, name the call *and its mode*, because `browser_snapshot` is `ariaSnapshot({ mode: "ai" })` — one formatter, and the mode is the thing that changes the answer. Recorded for the next seat in `docs/development/FRONTEND.md` (`## Testing` → "Measuring the accessibility tree — name the instrument", PR #2043), with the three-way table and both disagreement examples, so it does not have to be re-derived. **Lesson:** a snapshot is a rendering decision, not a measurement. "The accessibility tree says X" was never checkable, and the tool's own description supplies exactly the authority that stops a seat looking — while the two failure directions here are not noise around a correct answer, they are two opposite answers to a question the tool was never asked. Corollary for every future a11y control: showing that an instrument can print *some* name says nothing about the role you are testing — nor about whether the role was declared or implicit. Hold every variable but the one under test. + +## 75. A claim names a tree, and a needle is a guess about the code — so a figure without its head reads exactly like evidence (2026-09-30, sprint-impl) + +*Origin: @sprint-impl — the recomputation of #2049's figures was taken over a subject `404a8f30` had already replaced, and the three-needle absence of the same session was empty; @sprint-review reached the same false negative independently, from different needles, ten minutes later, and published theirs in a gate (`5362301910`, retracted at `5362406679`); @ux-lead located the read that settled it, `c5c2c38a:1404`. The row-title half was measured on TASK-203 by both seats; the identifier and bucket cases are @sprint-impl's. Every figure below was re-measured for this entry and carries the head it was measured at in the sentence it appears in.* + +#2049's comment carried a recomputed table for a slice of `v2-landing.css` — the arithmetic was done correctly, over a read that no longer existed. `404a8f30` had already replaced the construction those numbers described, so the sentence was not stale in its digits; it was stale in its **subject**. Nothing in the artifact can show that: a claim that quotes numbers and a head looks measured, and a reader who checks the arithmetic finds it sound. Stale data degrades, and an audit can find it; this kind does not — it stays exactly as confident as when it was written, about a tree it no longer describes. + +**The mirror was measured the same day on a row title, not a comment.** TASK-203's title named head `6cde8789` and carried `c5c2c38a`'s line table. Re-measured both, in `frontend/src/v2/landing/v2-landing.css`: each head carries six `prefers-reduced-motion` matches — five at-rules and a sixth that is a prose line mentioning the property — at `498·1149·1175·1201·1271` with the prose at `1110` for `c5c2c38a`, and at `506·1182·1208·1234·1304` with the prose at `1143` for `6cde8789`. Two facts fall out. The tables are close enough that **no reader can tell by inspection which head the numbers came from** — which is why the head has to be in the sentence and not in the surrounding paragraph. And "six blocks" was wrong for a second reason: five of the six matches are at-rules and the sixth is a comment, so a count of matches is not a count of blocks. The title now labels every figure with its head, and that shape is the repair: *"Figures, each labelled by its head: `c5c2c38a` / `8a692533` share ONE CSS blob — … ; `6cde8789` is a different head — …"* + +**The absence was worse than unmeasured, and two seats produced it within ten minutes.** A gate PASS reported the construction absent at #2019's pre-fix heads from three needles, two spelled with a `@media (` prefix the construction does not have and the third the fix's own name (`5362301910`); the same emptiness was reported from scratch patterns at msg 75883. Re-measured at `c5c2c38a`: the read is present at `frontend/src/v2/__tests__/v2-layout-invariants.test.ts:1404` — `landing.slice(landing.indexOf('prefers-reduced-motion'))`, introduced by `c5edaf1c`, removed by `404a8f30`, absent at `cfd74e8f` (0 hits for `indexOf('prefers-reduced-motion'` in `frontend/src`) — while `split('@media (prefers-reduced` finds 0 hits, `reducedMotionBlock` finds 0 (the fix introduced that name — it is on the branch at `6cde8789`, which also removed the `:1404` read, and it reaches `cfd74e8f` with `404a8f30`), and `indexOf('@media (prefers-reduced` finds one hit at `:2853` in `c5c2c38a`'s tree — the same read sits at `:2903` at `cfd74e8f`, so even this line number needs its head — a different read, `v2.indexOf('@media (prefers-reduced-motion: reduce) {', sectionFrom)`. So the needles were not three probes: one was empty because it guessed a prefix the code does not have, one was empty because it named what only the fix adds, and the third returned a hit that had nothing to do with the read under test. Nor do the gate's own cells reproduce: at its three heads, `git grep -F` for its three needles hits in five of nine cells — `indexOf('@media (prefers-reduced` once at each of `8a692533:2853`, `6cde8789:2902` and `0dafe74f:2902`, `reducedMotionBlock` twice at each of `6cde8789` and `0dafe74f` — where the gate reported zero in all nine, and the whole `frontend/src` scope gives the same counts; only `8a692533` of the three was pre-fix. **Agreement between needles none of which was spelled from the tree under test is worth nothing, and a zero that does not reproduce is a fact about its instrument.** What made it travel is that it read like diligence: it entered a gate as a stated gap, the next message built on it, and it reached a board row before `c5c2c38a:1404` stopped it. + +**Three more of the same shape, all from the same day.** (a) *An identifier written before the instrument returned it:* a row note carried `5911308544` for a pull-request comment that `gh pr comment` had printed as `5911356163` — a reconstruction of the pattern where a measurement was needed, and the wrong value is unreachable rather than merely wrong. (b) *A bucket that could not answer its own question:* a first scan of #1534–#1749 (194 merged) flagged 43 rows against the count rule, and **40** of those had `title == first commit subject` — identical candidates, so those rows had no answer; fetch both candidates for every row, the discipline the bucket skipped, and the same window holds **3** genuine overrides (#1623, #1644, #1645). The bucket ran to completion and its output looked like a result. (The window slides: the same pages now return #1535–#1750, so the range is stated as measured rather than as a name.) (c) *A column that could not vary:* `git merge-base --is-ancestor origin/main ` was false on all 29 rows measured against `origin/main` = `cfd74e8f` — 22 affected tips and 7 controls — because `origin/main` has advanced past every one, so it separated nothing while reading as a clean split; the parent count is what separates them (22 of 22 against 7 of 7). A measurement that cannot fail is not a measurement, exactly as an assertion that cannot fail is not a test. + +**Repair:** the claim half has no code to change and the repair belongs in the artifact. Every figure carries the head it was measured at, in the same sentence. Identifiers are quoted from the tool's own output, never rebuilt from the pattern. Before a column is cited it is shown to take more than one value, and before a bucket is believed it is shown capable of returning the other answer. A needle is never evidence about code that has not been read — spell it from the file, or read the file. The code half of this incident did get a shared reader: `blockContaining` / `reducedMotionBlock` (`frontend/src/v2/__tests__/v2-layout-invariants.test.ts:93-102` at `cfd74e8f` — the name `404a8f30` introduced, rebuilt by `1179018e` on a `blockContaining` it wrote beside `blockAt` in the test file when it collapsed the three brace-walk copies; `07aba418` (#2051) moved both to `../lib/cssBlocks`), which finds the block **carrying** the needle rather than the first block of that kind — **and whose own comment is the rule's next instance**: it names five blocks at `531/1395/1421/1447/1517` with the prose line at `1356`, figures that were true at `b5a93235` and false at every `main` commit that carries the comment, where the same five sit at `535/1399/1425/1451/1521` and the prose at `1360`: `662966ec` (#2046) added four lines above them, it is an ancestor of `828fe52a`, the parent `1179018e` landed on, and `1179018e` is the first `main` commit to carry those figures. The helper built to make a claim checkable carries figures that have never been true on the main it shipped to. (`grep -c` returning six rather than five is the other half of that comment: the sixth match is the prose line itself.) + +**Lesson:** a claim's figures and the tree they came from are two facts and only one of them is in the claim. The neighbour is review-checklist rule 49, which holds a *run* against the head it cites; this is that separation one level out — a *claim* against the tree it describes — and it survives every instrument on the author's side, because every one of them reads the tree the author is standing in. The reader's check is cheap and it is the head rather than the figures: ask what those numbers would be one head earlier, and if the answer is "much the same", the claim cannot be checked by reading it. + +*Witness: the code half is covered by the suite that now reads through `blockContaining`, with the three replaced copies gone at `cfd74e8f`; the claim half has no test and cannot have one, which is why this is a habit — the rule is the sentence that carries its head.*