docs(testing): three instrument rules the TASK-166 gate earned — fixture shape, a mutation that never applied, and load-red - #1947
Merged
Conversation
samxu01
force-pushed
the
kai/testing-doc-fixture-shape
branch
from
September 27, 2026 06:52
faee296 to
3e63aee
Compare
lilyshen0722
changed the title
…ure shape, a mutation that never applied, and load-red All three came out of the TASK-166/#1945 verification on 2026-09-27 and all three arrive looking like a result. Fixture shape: a `DocumentArray`'s cast is part of the measurement. `doc.members.includes('<hex>')` is true on a hydrated document and false on the bare array of the same values — while `JSON.parse(JSON.stringify(doc)).members .includes(hex)` is also true, because serialisation turned the values into strings. A green `true` has two mechanisms behind it and the test cannot tell them apart, so the assertion names which one it is. A mutation is evidence only once the replacement applied: an old-string built by a shell pipeline came out empty, matched 11,445 sites, and reported a normal green run — indistinguishable from a surviving mutant unless the edit counts its matches. And the load case: a timeout in a cold parallel run is contention before it is a regression. Nine booting `MongoMemoryServer`s pushed a suite past the global 30 s with the diff untouched; the same suites then passed 95/95 three ways. Docs only; no code, no version bump.
samxu01
force-pushed
the
kai/testing-doc-fixture-shape
branch
from
September 27, 2026 07:09
3e63aee to
81310ed
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Docs only, no version bump.
Three authoring rules, all earned during the TASK-166/#1945 verification, all in the class that arrives looking like a result:
A document array's cast is part of the measurement.
doc.members.includes('<hex>')→ true (hydrated wrapper casts),[objectId].includes('<hex>')→ false (same values, bare array),JSON.parse(JSON.stringify(doc)).members.includes('<hex>')→ true for a different reason (serialisation made them strings). Two mechanisms, onetrue, indistinguishable in a green run — so the test says which.doc.toObject()is the third shape and answers false. Earned by @kai and @vera building the wrong fixture independently while confirming the sameleavePodpremise.A mutation is evidence only once the replacement has applied. Assert the occurrence count before replacing; an old-string built by a pipeline can come out empty, match everywhere, and print a green run — a no-op indistinguishable from a surviving mutant. Earned 2026-09-27 by @vera (11,445 sites, 95/95 green).
A timeout in a cold parallel run is contention before it is a regression. Nine booting
MongoMemoryServers pushed a suite past the global 30 s with the diff untouched; the same suites then passed 95/95 at default workers, 95/95 at-w 2, and 15/15 alone. Also @vera's, recorded so the next seat re-runs rather than diagnoses.Witness:
__tests__/unit/scripts/docReferences.test.js6/6 green — the guard that reads these docs. No code touched, so no mutation ledger applies.Gate: @vera.