From 81310ed4e6c7554390b02e872f86fe022267287b Mon Sep 17 00:00:00 2001 From: Lily Shen <115414357+lilyshen0722@users.noreply.github.com> Date: Sat, 26 Sep 2026 23:52:08 -0700 Subject: [PATCH] =?UTF-8?q?docs(testing):=20three=20instrument=20rules=20t?= =?UTF-8?q?he=20TASK-166=20gate=20earned=20=E2=80=94=20fixture=20shape,=20?= =?UTF-8?q?a=20mutation=20that=20never=20applied,=20and=20load-red?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit All three came out of the TASK-166/#1945 verification on 2026-09-27 and all three arrive looking like a result. Fixture shape: a `DocumentArray`'s cast is part of the measurement. `doc.members.includes('')` is true on a hydrated document and false on the bare array of the same values — while `JSON.parse(JSON.stringify(doc)).members .includes(hex)` is also true, because serialisation turned the values into strings. A green `true` has two mechanisms behind it and the test cannot tell them apart, so the assertion names which one it is. A mutation is evidence only once the replacement applied: an old-string built by a shell pipeline came out empty, matched 11,445 sites, and reported a normal green run — indistinguishable from a surviving mutant unless the edit counts its matches. And the load case: a timeout in a cold parallel run is contention before it is a regression. Nine booting `MongoMemoryServer`s pushed a suite past the global 30 s with the diff untouched; the same suites then passed 95/95 three ways. Docs only; no code, no version bump. --- backend/TESTING.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/backend/TESTING.md b/backend/TESTING.md index f32b8e3f7..f4a814245 100644 --- a/backend/TESTING.md +++ b/backend/TESTING.md @@ -83,8 +83,10 @@ The branch is controlled by `process.env.INTEGRATION_TEST === 'true'`. `__tests_ - **Tier 0 tests don't cross-import `mongoServer` / `pgDb`.** The real-services branch doesn't export them. Use the helpers; if you need direct access, add a narrow helper in `testUtils.js` that works in both tiers. - **Real PG needs `pgcrypto` for `gen_random_uuid()`.** `setupPgDb` creates the extension for Tier 1 — don't call `gen_random_uuid()` in a test that only runs under Tier 0 unless you're also registering the pg-mem function. - **FK ordering matters under real PG.** `pod_members.pod_id` and `messages.pod_id` reference `pods(id) ON DELETE CASCADE`. Tests that insert raw rows must insert into `pods` first. `clearPgDb()` uses `TRUNCATE … CASCADE` to sidestep this on teardown. -- **Timeouts.** Real Mongo operations are slower than in-memory. `jest.setTimeout(30000)` is set globally in `__tests__/setup.js`; avoid hardcoded shorter timeouts in Tier 1 tests. +- **Timeouts.** Real Mongo operations are slower than in-memory. `jest.setTimeout(30000)` is set globally in `__tests__/setup.js`; avoid hardcoded shorter timeouts in Tier 1 tests. **A timeout in a cold parallel run is contention before it is a regression**: nine `MongoMemoryServer` boots in one cold pass can push a suite past the global 30 s on their own, and the same nine suites then pass three runs in a row (95/95 at default workers, 95/95 at `-w 2`, and the suite that timed out passing 15/15 alone — that third figure is one suite, not nine). Re-run the suite alone before reading the timeout as the diff's. - **New route registration? It needs a rate limiter ahead of auth.** `__tests__/unit/routes/routeRateLimitGuard.test.js` scans every `router.(path, …)` under `routes/` and fails when a registration has no `*RateLimit*` middleware, or has one behind `auth`/`agentRuntimeAuth`/`dualAuth` (the auth lookup is the DB work CodeQL's js/missing-rate-limiting flags). A file-level `router.use(auth)` counts as auth for every route after it, the last argument is the handler (never a limiter), and `router.route(path)` chains are scanned per verb. Shape to copy: `routes/agentHooks.ts`. Pre-existing violations sit in `routeRateLimitGuard.baseline.json`, which may only shrink — fix the route, never extend the list; delete a row once its route is compliant. +- **The cast performed by mongoose's array wrapper is part of the measurement, and two mechanisms return the same `true`.** `pod.members` is wrapped in a `MongooseArray` on a hydrated document, so `doc.members.includes('')` is **true** (the wrapper casts) while `[objectId].includes('')` over the same values is **false**. But `JSON.parse(JSON.stringify(doc)).members.includes('')` is **also true** — for a different reason: serialisation turned the values into hex *strings*. So a `true` can come from the cast or from the values having stopped being ObjectIds, and the two are indistinguishable in a green run: say in the test which mechanism produced it, because a `true` from a serialised fixture is a fact about the fixture and proves nothing about the real document. (`doc.toObject()` is the third shape and answers **false**.) Name the mechanism precisely, because the name sends the next reader to a specific place in mongoose: `pod.members` is a genuine `Array` that mongoose has patched (`Array.isArray` true, `isMongooseArray` true) — **not** a `DocumentArray`, whose marker `isMongooseDocumentArray` is *absent* here (`undefined`, not `false`), and which is the subdocument-array case this schema does not use. So the cast comes from mongoose's array patching; a plain array literal carries neither marker and returns `false`. (Measured on mongoose 7.8.6 with both predicates, not inferred from the property name.) *(Earned 2026-09-27, TASK-166/#1945, from two different mistakes. @kai's fixture declared an unlisted creator and `Pod`'s pre-save hook silently repaired it — the error was in the fixture, not in the assertion, and the state that reached the assertion was never the state the test named. @vera ran the hydrated document and the bare array in one probe as a deliberate contrast, which is what made the two mechanisms separable, and contributed the JSON third shape. The rule is the same for both: the instrument, not the answer, is what the green run certifies.)* +- **A mutation is evidence only once the replacement has been shown to apply.** Assert the occurrence count before replacing — a mutating script whose old-string is built by a pipeline can come out empty, match everywhere, and print a normal green run — and read a green result whose diff is empty as a broken instrument rather than a surviving mutant. The tell is in the edit, not the result: count the matches, and refuse the mutation when the count is not what the plan says. *(Earned 2026-09-27, TASK-166/#1945: @vera's first attempt at the invite-route term built its old-string from a shell pipeline that came out empty — the replace matched 11,445 sites and the run reported 95/95 green, an unapplied mutation that would have been filed as a survivor.)* - **New test file, which tier?** Put it under `__tests__/service/` if it exercises real query semantics (Mongo index behavior, regex, ObjectId coercion, PG ILIKE, transactions). Put it under `__tests__/unit/` or similar if a mocked DB is sufficient. - Never use Jest `{ virtual: true }` for a module that exists on disk; under shared workers it can resolve a different module ID and silently bypass the mock (#1691).