Skip to content

docs(testing): the fourth instrument rule — a mutation that reddens arms it cannot reach - #1950

Merged
lilyshen0722 merged 2 commits into
mainfrom
kai/testing-doc-fourth-rule
Sep 27, 2026
Merged

lilyshen0722 merged 2 commits into
mainfrom
kai/testing-doc-fourth-rule

Conversation

@lilyshen0722

@lilyshen0722 lilyshen0722 commented Sep 27, 2026 •

Copy link
Copy Markdown
Contributor

Follow-up to #1947, cut from its merge commit 6f966eec. Vera ruled it should be its own docs PR rather than folded into TASK-167's pass, and #1947 was already in the merge queue when the bullet was ready (a force-push to a queued branch is refused, which is the protection working).

What it adds

A fourth instrument rule, beside the three #1947 landed:

A mutation that reddens arms it cannot reach means the instrument did not run.

The two rules beside it are visible in the instrument's own output — an unapplied replace leaves an empty diff; a fixture that is not the real shape answers the wrong question. This one is visible only in the shape of the red: a mutation to one module reddening a suite that never imports it is not a cause, and the run has already said it is lying. The audit trail is the restore helper open(dest, 'w').write(open(src).read()), which truncates the destination before reading the backup; a missing backup left the module under test at zero bytes, and the symptom was a controller mutation reddening all six arms of an unrelated script suite. The tell was never the count — the reds had no causal path to the mutation. Fix: read the source before opening the destination for write, and assert the backup is non-empty.

Vera named this as a distinct class and supplied the second instance in the same night: a ts-jest compile error that printed Tests: 20 passed with exit 1.

Two probe corrections folded into the array-wrapper bullet, both hers:

  • !! in a probe coerces undefined to false — three distinguishable states reduced to two, with the printed result still looking like a measurement. Her first measurement of isMongooseDocumentArray was reported as false for exactly this reason.
  • constructor.name is not a shape control: it reads 'Array' for both a patched array and a plain one, so it fails silently where isMongooseArray discriminates.

Verification


Gate follow-up (head fee86ca9)

Vera's correction to her own contribution in the fourth bullet, applied. The bullet had said "a green summary from a run that never executed". The run did execute: 3 of 4 suites ran and accounted for all 20 passes; the fourth never compiled, so the summary was complete for the suites that ran and silent about the one that did not. The accurate form is now in the document, together with the operational half that follows from it: a suite that fails to run shows up as a missing contribution, not as a red test, so the check is the count against the baseline (91 → 20) rather than a scan for failures. That is a sharper trap than "never executed" — the number was internally consistent, which is exactly why it read as a normal green.

Re-check: this is one sentence in backend/TESTING.md, and the witness is docReferences 6/6.

…th to the mutation

Cut from `6f966eec` (#1947's merge) rather than folded into any code row, per
Vera's ruling: a docs bullet held as a patch while a code row's scope shifts is
the thing that gets forgotten, and this bullet is about instruments that silently
don't run.

Two additions to `backend/TESTING.md`:

- **A mutation that reddens arms it cannot reach means the instrument did not
  run.** The two rules beside it are visible in the instrument's own output — an
  unapplied replace leaves an empty diff, a fixture that is not the real shape
  answers the wrong question. This one is visible only in the shape of the red.
  Audit trail: the restore helper `open(dest, 'w').write(open(src).read())`
  truncates the destination before reading the backup; a missing backup left the
  module at zero bytes and a controller mutation reddened all six arms of an
  unrelated script suite. Vera named the class and supplied the second instance
  (ts-jest printing `Tests: 20 passed` with exit 1).
- **Two probe corrections folded into the array-wrapper bullet**, both hers:
  `!!` coerces `undefined` to `false` — three distinguishable states printed as
  two, still looking like a measurement — and `constructor.name` is not a shape
  control (it reads `'Array'` for both shapes, where `isMongooseArray`
  discriminates).

Witness: `__tests__/unit/scripts/docReferences.test.js` 6/6. Docs-only, no code
path touched.
…suite contributed zero, it did not fail to run

Vera's correction (74723) to her own contribution in the fourth bullet. It said
"a green summary from a run that never executed"; the run did execute — 3 of 4
suites ran and accounted for all 20 passes, and the fourth never compiled, so the
summary was complete for the suites that ran and silent about the one that did
not. Left as written, the sentence sends a reader scanning for failures when the
tell is a missing contribution: the check is the count against the baseline
(91 → 20).

Comment-only, and the bullet's subject is precision, so the imprecise version is
the one thing it could not carry.
@lilyshen0722
lilyshen0722 added this pull request to the merge queue Sep 27, 2026
Merged via the queue into main with commit b39eb79 Sep 27, 2026
15 of 19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant