docs(testing): the fourth instrument rule — a mutation that reddens arms it cannot reach - #1950
Merged
Merged
Conversation
…th to the mutation Cut from `6f966eec` (#1947's merge) rather than folded into any code row, per Vera's ruling: a docs bullet held as a patch while a code row's scope shifts is the thing that gets forgotten, and this bullet is about instruments that silently don't run. Two additions to `backend/TESTING.md`: - **A mutation that reddens arms it cannot reach means the instrument did not run.** The two rules beside it are visible in the instrument's own output — an unapplied replace leaves an empty diff, a fixture that is not the real shape answers the wrong question. This one is visible only in the shape of the red. Audit trail: the restore helper `open(dest, 'w').write(open(src).read())` truncates the destination before reading the backup; a missing backup left the module at zero bytes and a controller mutation reddened all six arms of an unrelated script suite. Vera named the class and supplied the second instance (ts-jest printing `Tests: 20 passed` with exit 1). - **Two probe corrections folded into the array-wrapper bullet**, both hers: `!!` coerces `undefined` to `false` — three distinguishable states printed as two, still looking like a measurement — and `constructor.name` is not a shape control (it reads `'Array'` for both shapes, where `isMongooseArray` discriminates). Witness: `__tests__/unit/scripts/docReferences.test.js` 6/6. Docs-only, no code path touched.
…suite contributed zero, it did not fail to run Vera's correction (74723) to her own contribution in the fourth bullet. It said "a green summary from a run that never executed"; the run did execute — 3 of 4 suites ran and accounted for all 20 passes, and the fourth never compiled, so the summary was complete for the suites that ran and silent about the one that did not. Left as written, the sentence sends a reader scanning for failures when the tell is a missing contribution: the check is the count against the baseline (91 → 20). Comment-only, and the bullet's subject is precision, so the imprecise version is the one thing it could not carry.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #1947, cut from its merge commit
6f966eec. Vera ruled it should be its own docs PR rather than folded into TASK-167's pass, and #1947 was already in the merge queue when the bullet was ready (a force-push to a queued branch is refused, which is the protection working).What it adds
A fourth instrument rule, beside the three #1947 landed:
The two rules beside it are visible in the instrument's own output — an unapplied replace leaves an empty diff; a fixture that is not the real shape answers the wrong question. This one is visible only in the shape of the red: a mutation to one module reddening a suite that never imports it is not a cause, and the run has already said it is lying. The audit trail is the restore helper
open(dest, 'w').write(open(src).read()), which truncates the destination before reading the backup; a missing backup left the module under test at zero bytes, and the symptom was a controller mutation reddening all six arms of an unrelated script suite. The tell was never the count — the reds had no causal path to the mutation. Fix: read the source before opening the destination for write, and assert the backup is non-empty.Vera named this as a distinct class and supplied the second instance in the same night: a ts-jest compile error that printed
Tests: 20 passedwith exit 1.Two probe corrections folded into the array-wrapper bullet, both hers:
!!in a probe coercesundefinedtofalse— three distinguishable states reduced to two, with the printed result still looking like a measurement. Her first measurement ofisMongooseDocumentArraywas reported asfalsefor exactly this reason.constructor.nameis not a shape control: it reads'Array'for both a patched array and a plain one, so it fails silently whereisMongooseArraydiscriminates.Verification
__tests__/unit/scripts/docReferences.test.js— 6/6 (the witness docs(testing): three instrument rules the TASK-166 gate earned — fixture shape, a mutation that never applied, and load-red #1947 established for this file: it checks everypathandfile:linereference in the docs, so a docs PR that moves a reference reddens here).Gate follow-up (head
fee86ca9)Vera's correction to her own contribution in the fourth bullet, applied. The bullet had said "a green summary from a run that never executed". The run did execute: 3 of 4 suites ran and accounted for all 20 passes; the fourth never compiled, so the summary was complete for the suites that ran and silent about the one that did not. The accurate form is now in the document, together with the operational half that follows from it: a suite that fails to run shows up as a missing contribution, not as a red test, so the check is the count against the baseline (91 → 20) rather than a scan for failures. That is a sharper trap than "never executed" — the number was internally consistent, which is exactly why it read as a normal green.
Re-check: this is one sentence in
backend/TESTING.md, and the witness isdocReferences6/6.