Skip to content

Commit 4e8416f

Browse files
claude[bot]claude
andauthored
fix(scripts): admit the equality --self-test dispatch to the floor census (#15517)
`measure-self-test-floor`'s DISPATCH criterion matched only `includes()` and `has()`, so six tracked scripts that dispatch on an already-extracted flag (`flag === '--self-test'`) were outside the census entirely -- five of them gates lint.yml already invokes with `--self-test`. An absence from a census is not a clearance: no batch of #13799 could be dispatched against those files, and nothing recorded that they were never floored. The criterion is now published in two halves, MEMBERSHIP and EQUALITY, the way the failure-producer criterion already is, so a control can read each on its own. The equality half admits `==`/`===`, all three quote styles and the literal on either side, and only when the compared operand is a BARE IDENTIFIER: a comparison against a property or a call result is a tool examining the flag as data (`renderedArgv(' --self-test').args === '--self-test'`, live in `scripts/pm/dispatch-gates.mjs`), and admitting those would seed the census with its own instruments. Two fixtures and nine inline controls, running on every invocation like the rest: a gate dispatching by equality must enter the population (and must not match the membership half, so the verdict cannot pass for the wrong reason), each spelling variant is admitted, and a file that only MENTIONS the flag -- in prose, in a comparison against a call result, and as a literal handed to a child process -- must not enter it, while matching unmasked so the control reads the comment masking rather than assuming it. Measured: the census goes from 170 to 176 files, the six named files enter, no row leaves, and no pre-existing row is reclassified. All six classify NONE, so #13799's population grows by six; flooring them is that card's work, not this one's, and none of the six is touched here. Claude-Session: https://claude.ai/code/session_012zGPuVVX3deAx9LdjK8jCk Co-authored-by: Claude <noreply@anthropic.com>
1 parent 26144c2 commit 4e8416f

1 file changed

Lines changed: 109 additions & 2 deletions

File tree

scripts/measure-self-test-floor.mjs

Lines changed: 109 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -90,6 +90,11 @@
9090
* rather than behind a `--self-test` flag, precisely so they cannot become
9191
* unrun; that is the `inline` route `check-self-test-wired.mjs` records.
9292
*
93+
* The POPULATION CRITERION is controlled here too, and for the sharper version
94+
* of the same reason: a dispatch spelling it cannot see does not produce a
95+
* generous classification, it produces an ABSENCE -- and an absent row is
96+
* indistinguishable from a file that was never in scope.
97+
*
9398
* The controls have already earned their place once: an earlier revision of
9499
* `classifyFloor` keyed on the NAME `SELF_TEST_BATTERIES` rather than on a
95100
* comparison that produces a failure, and called a fixture floored after the
@@ -107,8 +112,45 @@ import { maskComments } from './js-comment-mask.mjs';
107112

108113
const ROOT = join(fileURLToPath(new URL('.', import.meta.url)), '..');
109114

110-
/** A `--self-test` DISPATCH: argv membership tested, not a literal passed to a child. */
111-
const DISPATCH = /(?:includes|has)\(\s*['"`]--self-test['"`]\s*\)/;
115+
/**
116+
* A `--self-test` DISPATCH: argv examined, not a literal passed to a child.
117+
*
118+
* Published in two halves so a control can test each on its own, the same way
119+
* the failure-producer criterion below is:
120+
*
121+
* MEMBERSHIP argv membership tested -- `argv.includes('--self-test')`, and
122+
* the `Set` spelling `has('--self-test')`.
123+
* EQUALITY a flag ALREADY EXTRACTED into a variable, compared against the
124+
* literal -- `flag === '--self-test'`. `==` and `===`, all three
125+
* quote styles, the literal on either side.
126+
*
127+
* The membership half alone left SIX tracked files outside the census, five of
128+
* them gates lint.yml already invokes with `--self-test`, so no batch of #13799
129+
* could be dispatched against them and nothing recorded that they were never
130+
* floored. Absence from a census is not a clearance -- it is the one reading
131+
* that looks like a clean bill of health while saying nothing at all (#15421).
132+
*
133+
* BOUNDARY -- the compared operand must be a BARE IDENTIFIER. A property or a
134+
* call result (`renderedArgv(' --self-test').args === '--self-test'`, live in
135+
* `scripts/pm/dispatch-gates.mjs`'s own self-test) is a tool EXAMINING the flag
136+
* as data, not a script dispatching on it, and those comparisons cluster in
137+
* exactly the tools that reason about self-tests -- so admitting them would
138+
* seed this census with its own instruments. The `.` in front of the operand is
139+
* what excludes them.
140+
*
141+
* WARNING: like the floor criterion below, this reads SPELLINGS. Its error runs
142+
* in one direction, and that direction is INVISIBILITY rather than a NONE: a
143+
* dispatch spelled some way neither half knows is not classified generously, it
144+
* is not in the population at all. Two further spellings have no tracked
145+
* carrier in this tree today and are deliberately left unadmitted rather than
146+
* written blind -- an inequality guard (`arg !== '--self-test'`) and a `switch`
147+
* case label. Widen the same way this half was: a control in both directions,
148+
* and the delta in the census measured and published.
149+
*/
150+
const DISPATCH_MEMBERSHIP = /(?:includes|has)\(\s*['"`]--self-test['"`]\s*\)/;
151+
const DISPATCH_EQUALITY =
152+
/(?:^|[^\w$.])[A-Za-z_$][\w$]*\s*={2,3}\s*['"`]--self-test['"`]|['"`]--self-test['"`]\s*={2,3}\s*[A-Za-z_$][\w$]*/;
153+
const DISPATCH = new RegExp(`${DISPATCH_MEMBERSHIP.source}|${DISPATCH_EQUALITY.source}`);
112154

113155
/** Marker injected by the probe. Its presence on disk is the mutation's proof. */
114156
const PROBE_MARKER = 'OS_SELF_TEST_FLOOR_PROBE';
@@ -483,6 +525,48 @@ const TERNARY_EXIT_GATE = [
483525
/** The one dispatch line of `TERNARY_EXIT_GATE`, the anchor the variants replace. */
484526
const TERNARY_EXIT_DISPATCH = 'runSelfTest() ? 0 : 1';
485527

528+
/**
529+
* The EQUALITY dispatch, reduced: the flag pulled out of argv into a variable
530+
* and compared against the literal. This is the shape all six files the
531+
* membership half could not see carry -- among them `scripts/pnpm-filter-
532+
* targets.mjs`, whose `--self-test` `check:pnpm-filter-targets` runs in lint.yml
533+
* (#15421). Like the ternary fixture above it is READ, never spawned: the
534+
* population criterion is a pure function of source text.
535+
*/
536+
const EQUALITY_DISPATCH_GATE = [
537+
'#!/usr/bin/env node',
538+
'function selfTest() {',
539+
' const failures = [];',
540+
" if (1 !== 1) failures.push('x');",
541+
" console.log('fixture self-test: 1 case passes');",
542+
' return failures.length;',
543+
'}',
544+
'const flag = process.argv[2];',
545+
"if (flag === '--self-test') process.exit(selfTest());",
546+
'',
547+
].join('\n');
548+
549+
/** The one comparison that IS the dispatch, the anchor the spellings replace. */
550+
const EQUALITY_DISPATCH = "flag === '--self-test'";
551+
552+
/**
553+
* The shapes that MENTION the flag without dispatching on it, in one file:
554+
* prose describing a dispatch, a comparison against a CALL RESULT (the
555+
* `scripts/pm/dispatch-gates.mjs` shape, where the flag is the data a gate is
556+
* examining), and the literal handed to a child process -- the boundary this
557+
* criterion has drawn since it was one line long. None of them may put a file
558+
* in the population, and the comment is written so that it WOULD match unmasked,
559+
* so the control below reads the masking rather than assuming it.
560+
*/
561+
const NON_DISPATCH_MENTION_GATE = [
562+
'#!/usr/bin/env node',
563+
"// Gates are dispatched with `if (arg === '--self-test') selfTest();` -- prose, not code.",
564+
'const rendered = (argv) => ({ args: argv.trim() });',
565+
"if (rendered(' --self-test').args === '--self-test') console.log('the renderer kept the flag');",
566+
"spawnSync(process.execPath, [target, '--self-test']);",
567+
'',
568+
].join('\n');
569+
486570
/**
487571
* Both instruments, against both directions. Returns the failures; the caller
488572
* refuses on any. Nothing here reads the repo, so a control failure is always
@@ -492,6 +576,29 @@ export function runControls() {
492576
const failures = [];
493577
const say = (cond, label) => { if (!cond) failures.push(label); };
494578

579+
// The POPULATION criterion, both halves and both directions. What is at stake
580+
// here is not a classification but a ROW: a spelling this criterion cannot see
581+
// removes its file from the census silently (#15421).
582+
say(DISPATCH.test(maskComments(HOLED_GATE)),
583+
'POPULATION CONTROL FAILED: a gate dispatching by argv membership was not admitted to the population');
584+
say(DISPATCH.test(maskComments(EQUALITY_DISPATCH_GATE)),
585+
"POPULATION CONTROL FAILED: a gate dispatching by equality on an extracted flag (`flag === '--self-test'`) was not admitted to the population");
586+
say(!DISPATCH_MEMBERSHIP.test(maskComments(EQUALITY_DISPATCH_GATE)),
587+
'CONTROL FIXTURE INVALID: the equality fixture also carries a membership dispatch, so the verdict above would pass without the equality half being read at all');
588+
say(EQUALITY_DISPATCH_GATE.includes(EQUALITY_DISPATCH),
589+
'CONTROL FIXTURE INVALID: the equality fixture no longer carries the dispatch line the spellings below replace, so every variant is the SAME file');
590+
const equalityDispatch = (spelling) => maskComments(EQUALITY_DISPATCH_GATE.replace(EQUALITY_DISPATCH, spelling));
591+
say(DISPATCH.test(equalityDispatch('flag == "--self-test"')),
592+
'POPULATION CONTROL FAILED: loose equality against a double-quoted literal was not admitted');
593+
say(DISPATCH.test(equalityDispatch("'--self-test' === flag")),
594+
'POPULATION CONTROL FAILED: the literal on the LEFT of the comparison was not admitted');
595+
say(DISPATCH.test(equalityDispatch('flag === `--self-test`')),
596+
'POPULATION CONTROL FAILED: a template-literal spelling of the flag was not admitted');
597+
say(!DISPATCH.test(maskComments(NON_DISPATCH_MENTION_GATE)),
598+
'POSITIVE CONTROL FAILED: a file that only MENTIONS the flag -- in prose, in a comparison against a call result, and as a literal handed to a child -- entered the population; the census would then be seeded with the very tools that reason about self-tests');
599+
say(DISPATCH.test(NON_DISPATCH_MENTION_GATE),
600+
'CONTROL FIXTURE INVALID: the mention fixture does not match even UNMASKED, so the verdict above says nothing about comments being masked away');
601+
495602
// Instrument 1, both directions.
496603
say(classifyFloor(maskComments(HOLED_GATE)) === 'NONE',
497604
'POSITIVE CONTROL FAILED: a self-test deciding success by failures.length alone was not classified NONE');

0 commit comments

Comments
 (0)