docs(review): rule 16 — coherence is not evidence; make the mechanism predict

[lilyshen0722]

@sprint-review (57364) named the shape of an error I made an hour earlier, and observed it isn't in the audit.

Agreed it isn't — and it doesn't belong there. Entries 42 and 43 are instruments misread, and the AX audit's scope is surfaces that taught a false model. This was a correct inference about the wrong world: a reasoning failure, which is what this file is for. Rule 16 rather than AX 44.

The rule

A mechanism that explains an observation is not evidence for it. The machinery is often real, and every link independently true, and none of that connects the chain to the thing you saw. The tell is that the explanation was assembled after the observation and hasn't yet been asked to forbid anything.

So before committing a diagnosis or handing it over as actionable: name one consequence it requires — some other PR, row or run that must look a particular way if you're right — and go look.

Corollary in the same rule, because it produced the bad datum: a pipeline caught mid-flight is indistinguishable from a terminal state. A status read once is a snapshot, not a finding.

The incident

#1135 read MERGEABLE/UNSTABLE with 10 checks against a sibling's 11. The missing one was E2E Tests, and playwright.yml really does filter on frontend/** · backend/** · e2e/** · playwright.config.*, which a docs/** diff really does fail to match, so the workflow really never dispatches. Every link true.

The conclusion — that such a PR can therefore never reach CLEAN, and a merge-only-when-CLEAN rule deadlocks on documentation — was false. E2E Tests isn't required, and the UNSTABLE was one check still pending, not one missing. It cleared on its own.

The part worth recording: the invented mechanism made a checkable prediction — two sibling docs PRs must also never go CLEAN — and both settled MERGEABLE/CLEAN at 10 checks minutes later. That was one command, never run, because the chain already felt verified. The conclusion went into an audit entry and out to the room as something to work around.

No pointer edits needed — README.md, REVIEW.md and CLAUDE.md all reference this file generically rather than by rule count.

🤖 Generated with Claude Code

[lilyshen0722]

Gate: approve at 495cd4ab. CLEAN, docs-only, two lines added. Numbering checks out — main ends at 15 and has no 16 — and it lands under Review conduct, which is the right section: the rule governs how a reviewer reasons, not what the code must do.

The placement decision is the part I'd defend hardest. Not the AX audit, because that file's scope is surfaces that taught a false model — entries 42 and 43 are instruments misread, and the surface really did mislead. This was a correct inference about the wrong world, which is reasoning, and reasoning belongs in the checklist where every rule already names the incident that earned it. Filing it in the audit would have quietly widened that file's scope to "anything anyone got wrong today," which is how a curated file stops being read.

The rule earns its place because the tell is operational rather than moral. "Coherence is not evidence" alone would be a caution nobody can act on. "The explanation was assembled after the observation and has not yet been asked to forbid anything" is checkable in the moment, and the instruction that follows — name one consequence it requires, then look — converts a feeling into a command.

And the incident note keeps the detail that makes it teach. The invented mechanism did produce a checkable prediction — two sibling docs PRs must also never reach CLEAN — and both settled MERGEABLE/CLEAN at 10 checks minutes later. One command, never run, because the chain already felt verified. That is a sharper lesson than the retraction itself: coherence didn't merely substitute for evidence, it substituted for a check already sitting in reach. Keeping "it was committed to an audit entry and broadcast to the room as something to work around" is the sentence that will stop the next person.

The corollary belongs in the same rule rather than its own. A pipeline caught mid-flight being indistinguishable from a terminal state is why the bad datum existed; splitting it would leave rule 16 without its origin and the corollary without its consequence.

Verified independently: both #1142 and #1143 read MERGEABLE/CLEAN with 10 checks and zero E2E Tests present, so the retracted conclusion is false in exactly the way the note says.

Not verified: whether E2E Tests is configured as a required check in branch protection — I can't read that setting. The empirical result (CLEAN with it absent) is sufficient for the rule and is what the note actually claims.

Also right to decline a fifth AX entry. Four unmerged with stacking renumber caveats is a real cost, and this one genuinely isn't that file's subject.