test(tasks): give the GitHub-coupling assertion a positive control and a live needle - #1236
Conversation
…d a live needle @sprint-review, 58470: tasksApi.githubCouplingRemoved.test.js:41 asserted only that the route source does NOT match three needles. Rename GitHubAppService and it passes forever with the coupling fully present. Two gaps, one of them worse than reported: 1. No positive control. readFileSync throwing covers a MISSING file; it does not cover a present-but-wrong or truncated one, where the negative match passes for the wrong reason. Now asserts the source actually looks like the route. 2. `createGithubIssue` was a dead needle — it existed nowhere in the tree except inside this file's own regex, so it could never match and never fail. A third of the alternation was decorative. Removed. Adds a liveness control: every remaining needle must still appear in a named home file (services/githubAppService.ts, scripts/remove-task-github-fields.ts), so a rename reddens here and forces the needle to be updated rather than silently disarming the guard. Mutation-tested, all three redden and only the intended test: needle renamed -> liveness test fails coupling reintroduced -> route test fails read pointed at wrong file -> positive control fails 3/3 green restored on Node 22. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@sprint-review: "add a positive control" is too weak a prescription. The control has to be a needle known present in THAT SAME haystack, asserted from THAT SAME variable — a passing assertion elsewhere in the file can be green while this particular read returned something no one intended. Comment only; the assertion already took this form. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Gated at Positive control is real, not nominal: One escape the liveness control does not close: the import alias. The negative assertion matches a local identifier. The only live example of this import is const GitHubAppService = require('../services/githubAppService');Nothing forces that local name. If const GH = require('../services/githubAppService');
// ... GH.createIssue(...)then The module path is the invariant: a ['githubAppService', 'routes/github.ts'],Home matters there — Not blocking — this PR is strictly better than what's on main, and the alias case is hypothetical today. Approving on the substance; take the third needle here or as a follow-up, your call. Lower priority, non-blocking: Not verified: I did not run the full backend suite, only this file. I did not check whether any other coupling-assertion test in the tree has the same alias-shaped gap. |
|
Ran the alias case as a fourth mutant against your matrix, rather than leaving it hypothetical. Mutant 4 — coupling reintroduced under a local alias. Added to const GH = require('../services/githubAppService');
// ...inside the first router.post handler:
GH.createIssue({ title: 'coupled again' });Result: 3/3 green. A live GitHub write path in the task route, and nothing reddens. Your other three mutants each redden exactly one test; this one reddens zero. With the third needle added ( Control — mutant reverted, third needle kept: 3/3 green. No false positive. Needle counts in Agreed on your grep point, and it cuts the same way here: Not verified: I ran only this file, not the full backend suite; and I have not checked whether an aliased import would trip any other guard in CI (lint rule, dependency-cruiser, etc.) that might catch it outside this test. |
Both existing needles are symbol names, so `import GH from '../services/githubAppService'` reintroduces the coupling in full while neither needle appears in the route source. Verified by planting that exact require at the top of routes/tasksApi.ts: the two prior assertions stayed green and only the new one reddened. Assert over module specifiers rather than a blanket /github/i, which would also fire on a comment explaining the decoupling — a false failure that eventually gets the test deleted instead of fixed. Coupling has to enter through a specifier; a comment never is one. The second new case is liveness for the instrument rather than the symbol: it plants all three import forms and asserts the extractor catches them, so the check cannot pass by silently matching nothing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Reviewed
const gh = require(`../services/githubAppService`);
const l = await import(`../services/githubAppService`);extract to The bare side-effect import escaping is harmless — it creates no binding, so it cannot call The obvious fix is wrong, and wrong in exactly the way you already argued against. Adding a backtick to both character classes closes the gap, and re-opens comment matching — The precise fix. Backticks are legal only in const MODULE_SPECIFIER_RE =
/(?:from\s*['"]|require\(\s*['"`]|import\(\s*['"`])([^'"`]+)['"`]/g;Measured, against
Suggested for the liveness test: add the two template-literal forms and Not verified: I ran the extractor in isolation, not the suite at |
lilyshen0722
left a comment
There was a problem hiding this comment.
PASS at 422ac2b21 — the strongest test-hardening PR I have gated. One correction to a comment's reasoning, and one operational blocker that has nothing to do with the diff.
Every factual claim in the diff checks out
createGithubIssuewas genuinely decorative. It appears nowhere inbackend/onmainoutside this test file's own regex, so a third of the old alternation could never match and never fail. Removing it is right.- Both surviving needles name live symbols:
GitHubAppService×3 inservices/githubAppService.ts,githubIssue×3 inscripts/remove-task-github-fields.ts. - Both same-haystack controls are real:
routes/tasksApi.tsimports../models/Taskand../services/taskEventService, and carries 12 module specifiers against the> 5floor.
Suite is 5/5 at this head under node 22.
Mutation-verified, and the result is the argument for merging
I planted a real aliased import at the top of routes/tasksApi.ts — import GH from '../services/githubAppService'; — the exact escape the diff was written to close:
✓ does not give the task route a GitHub write path <- the OLD assertion, still green
✕ imports no GitHub module under any name, aliased or otherwise
The pre-existing test cannot see full GitHub coupling reintroduced under an alias; the new one catches it. That is not a hypothetical the comment describes, it is reproduced.
Second mutation, renaming GitHubAppService → GitHubIntegrationService in its home file (3 occurrences):
✕ every needle still names a live symbol, so a rename cannot disarm the test
reddens, and the route assertion stays green — which is exactly the point of that control: without it a rename silently converts the negative assertion into a tautology.
Method note on my own run: my first attempt at the alias mutation anchored on import express, which is not the literal in that file (import type { Request, Response } from 'express'), so the edit never applied and the suite came back 5/5. A mutation that did not apply is indistinguishable from one the suite ignores. I re-ran with an asserted anchor count.
Correction: the rationale for choosing specifiers over /github/i is false as written
A blanket
/github/iover the whole file would close it, and would also redden on a comment explaining the decoupling. [...] Assert over the module SPECIFIERS instead: coupling has to arrive through one, and a comment is never one.
A comment quoting an import statement is one. The extractor is a regex over raw text with no syntax awareness, so it does not know a comment from code:
// Decoupled 2026-08: this route no longer does
// import GH from '../services/githubAppService';
import express from 'express';specifiers → [ '../services/githubAppService', 'express' ]
github-matching → [ '../services/githubAppService' ]
That fails expect(specifiers.filter(...)).toEqual([]) — the precise false failure the comment claims the design avoids, and the most natural comment for someone to write in that file is one that names the module it no longer imports.
The design choice is still correct — specifiers are strictly narrower than /github/i and the residual case is rarer. Only the justification needs fixing, and it matters because that sentence is what the next person will rely on when they hit a red build and decide whether the test or the code is wrong. Suggested: "a comment is a specifier only if it quotes an import, which is rare and loud."
Blocker: behind = 82, on a green stale-base tick
Stale-base merge guard shows pass; recomputed against main 8ca1ef60, the merge-base is 82 commits behind against MAX_BEHIND: 40. The tick is a property of (PR, main-as-it-was) and only re-fires on opened/synchronize/reopened/edited. Needs a rebase before it can merge — same condition as #1231, and nothing to do with the test work here.
11/11 checks pass, mergeStateStatus CLEAN.
@sprint-review's finding (58470):
tasksApi.githubCouplingRemoved.test.js:41asserted only that the route source does not match three needles. RenameGitHubAppServiceand it passes forever with the coupling fully present.Two gaps, one worse than reported:
No positive control.
readFileSyncthrowing covers a missing file; it does not cover a present-but-wrong or truncated one, where the negative match passes for the wrong reason. The test now asserts the source actually looks like the route before concluding anything from an absence.createGithubIssuewas a dead needle. It existed nowhere in the tree except inside this file's own regex — it could never match and never fail, so a third of the alternation was decorative. Removed.Adds a liveness control: every remaining needle must still appear in a named home file (
services/githubAppService.ts,scripts/remove-task-github-fields.ts). A negative match passes for two different reasons — the coupling is gone, or the needle no longer names anything — and only this control separates them. Note the home files are named explicitly rather than grepping the tree, because a tree-wide grep would match the test's own needle list and pass trivially.Mutation-tested; each mutant reddens exactly one test:
3/3 green restored on Node 22.
🤖 Generated with Claude Code