Skip to content

Commit 19fa30c

Browse files
claude[bot]claude
andauthored
refactor(devx): share the regex-literal walk, keep the position rule a parameter (#15532)
* refactor(devx): share the regex-literal WALK, keep the position rule a parameter `scripts/js-comment-mask.mjs` and `scripts/check-dispatcher-error-vocabulary.mjs` each carried a regex-literal recogniser: a byte-identical 14-word `REGEX_AFTER_KEYWORD` set, an `IDENT_CHAR` class, and a body walk. The keyword set is not decorative — js-comment-mask's own header records that dropping `return` from it passes all 23 of its self-test cases and is caught only by the corpus sweep, so the copy that drifts drifts silently in exactly the direction its owner warns about. Split by NUMBER OF RIGHT ANSWERS rather than by module: the WALK has one (ECMA-262's), so it lives once — `walkRegexBody`, exported from js-comment-mask.mjs alongside `IDENT_CHAR` and `REGEX_AFTER_KEYWORD`; the POSITION RULE has two, because the consumers fail in deliberately opposite directions — a masker over-masks safely, a scanner must never skip a span it invented — so it is a parameter (`makeRegexRecogniser`), with no default and a throw when one is omitted. `walkRegexBody` returns `{ end, closed }` rather than a verdict: an unclosed body is where the two directions part, so the shared half reports the outcome and each caller decides. `scanSource` over-masks to `end` and reads on (198 positions on this tree, every one a JSX closing tag); the gate answers -1. Zero behaviour change, measured both sides: scanSource(BASE) vs scanSource(HEAD) over the same 5929-file corpus population — comment, literal, interpolation, maskComments and stripComments all byte-identical: 0 files differ, 0 flag bytes differ. check-dispatcher-error-vocabulary normal, --report and --self-test output byte-identical, self-test still 10 shapes + 322 assertions. #14742's two cross-fixture self-test cases are KEPT and still mean what they said: the two POSITION RULES are still two, so pinning them against each other on the agreeing fixtures and on the `}` divergence still guards a real thing. js-comment-mask --self-test gains a 9-assertion shared-recogniser section driven in BOTH position modes, with a literal roster and floor of its own: 23 mask/strip corpus + 12 interpolation view + 9 shared recogniser = 44. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012zGPuVVX3deAx9LdjK8jCk * fix(devx): makeRegexRecogniser takes a REQUIRED options parameter `check-declaration-mirrors` went red in CI: scripts/js-comment-mask.d.mts:106 declares `makeRegexRecogniser` with 1 required parameter(s), but scripts/js-comment-mask.mjs implements 0. `Function.length` counts the parameters BEFORE the first defaulted one, so `makeRegexRecogniser({ mayBeginAt } = {})` reports zero required arguments. The declaration is the honest side — there is no default position rule, so the options really are required — and the module is what was wrong. Dropped the `= {}` and read `mayBeginAt` longhand behind an `options == null` guard, which keeps the existing TypeError message rather than letting a bare destructuring TypeError replace it with one that explains nothing. No behaviour change beyond the arity: both `makeRegexRecogniser()` and a non-function rule still throw, with the same message. The self-test row that already covered "omitting the rule throws" now pins all three facts at once — both throws, the message, and `makeRegexRecogniser.length === 1` — so the module's own `--self-test` reds on this regression instead of only a gate one layer away. Ablated: restoring the `= {}` gives `Function.length` 0, `js-comment-mask --self-test` EXIT=1 on that row, and `check-declaration-mirrors` EXIT=1 with the exact CI message above. The row name and count are unchanged, so the verdict still reads 44 cases (23 mask/strip corpus, 12 interpolation view, 9 shared recogniser). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012zGPuVVX3deAx9LdjK8jCk --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 0c5e973 commit 19fa30c

3 files changed

Lines changed: 373 additions & 59 deletions

File tree

scripts/check-dispatcher-error-vocabulary.mjs

Lines changed: 24 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -215,7 +215,21 @@
215215
*/
216216

217217
import { readFileSync, readdirSync, statSync, existsSync } from 'node:fs';
218-
import { maskComments, scanSource } from './js-comment-mask.mjs';
218+
// [#15487] The SHARED half of the regex-literal recogniser: the 14-word
219+
// `REGEX_AFTER_KEYWORD` set, `IDENT_CHAR`, and the literal WALK (character
220+
// class, backslash sequences, line-terminator refusal, flags) live ONCE, in
221+
// `js-comment-mask.mjs`, which already owns "which bytes are literal". What
222+
// stays in this file is `regexMayBeginAt` — the POSITION RULE, the one half the
223+
// two consumers answer differently and on purpose. See `walkRegexBody`'s
224+
// docblock for why that is the line, and `makeRegexRecogniser`'s for why the
225+
// rule is bound once here rather than passed at each of this file's call sites.
226+
import {
227+
IDENT_CHAR,
228+
REGEX_AFTER_KEYWORD,
229+
makeRegexRecogniser,
230+
maskComments,
231+
scanSource,
232+
} from './js-comment-mask.mjs';
219233
// [#12925] The OTHER gate's detector, imported so the kebab declaration below is
220234
// pinned against its real recognizers rather than a paraphrase of them. That gate
221235
// imports nothing from here, so this is a one-way edge.
@@ -1481,16 +1495,14 @@ function unwrapExpression(expr) {
14811495
* divergence itself, so neither the copy nor the deliberate difference can
14821496
* drift unnoticed.
14831497
*/
1484-
const REGEX_AFTER_KEYWORD = new Set([
1485-
'return', 'typeof', 'instanceof', 'in', 'of', 'case', 'delete', 'void',
1486-
'yield', 'await', 'new', 'do', 'else', 'throw',
1487-
]);
1488-
1489-
/** IdentifierPart, near enough for the ASCII this population is written in. */
1490-
const IDENT_CHAR = /[\w$]/;
1491-
1492-
/** LineTerminator — the four the grammar names, not just `\n`. */
1493-
const isLineTerminator = (c) => c === '\n' || c === '\r' || c === '\u2028' || c === '\u2029';
1498+
// `REGEX_AFTER_KEYWORD` and `IDENT_CHAR` are IMPORTED (#15487). They used to be
1499+
// byte-identical copies of the two constants in `scripts/js-comment-mask.mjs`,
1500+
// and that module's own header records what a drifting copy costs: dropping
1501+
// `return` from the set passes all 23 of its self-test cases and is caught only
1502+
// by the corpus sweep, on a file this tree writes today. A copy that can drift
1503+
// silently in exactly the direction its owner warns about is not one to keep.
1504+
// The LineTerminator test went with the walk, which is the only thing that
1505+
// consulted it.
14941506

14951507
/**
14961508
* [#14742] Is the `/` at `at` in a position where an expression may BEGIN?
@@ -1544,32 +1556,7 @@ function regexMayBeginAt(src, at) {
15441556
* so a `/` whose body does not close on its line was a division operator, and
15451557
* answering "not a regex" skips nothing.
15461558
*/
1547-
export function regexLiteralAt(src, at) {
1548-
if (src[at] !== '/') return -1;
1549-
const first = src[at + 1];
1550-
// RegularExpressionFirstChar excludes `*` and `/`, which is the grammar's own
1551-
// reason comments are unambiguous.
1552-
if (first === undefined || first === '*' || first === '/') return -1;
1553-
if (!regexMayBeginAt(src, at)) return -1;
1554-
let inClass = false;
1555-
for (let i = at + 1; i < src.length; i += 1) {
1556-
const c = src[i];
1557-
if (isLineTerminator(c)) return -1;
1558-
if (c === '\\') {
1559-
// RegularExpressionBackslashSequence :: `\` RegularExpressionNonTerminator
1560-
if (i + 1 >= src.length || isLineTerminator(src[i + 1])) return -1;
1561-
i += 1;
1562-
continue;
1563-
}
1564-
if (inClass) {
1565-
if (c === ']') inClass = false;
1566-
continue; // ⭐ a `/`, a quote or a backtick in here is an ORDINARY CHARACTER
1567-
}
1568-
if (c === '[') { inClass = true; continue; }
1569-
if (c === '/') return i; // flags are IdentifierPartChars — code bytes, walked as such
1570-
}
1571-
return -1;
1572-
}
1559+
export const regexLiteralAt = makeRegexRecogniser({ mayBeginAt: regexMayBeginAt });
15731560

15741561
/**
15751562
* [#14742] Does a literal OPEN at `at`? The dispatch the four primitives share,

scripts/js-comment-mask.d.mts

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -65,6 +65,48 @@ export function stripComments(source: string): string;
6565
*/
6666
export function maskComments(source: string): string;
6767

68+
/**
69+
* The outcome of walking the body of the regex literal opening at `at`.
70+
*
71+
* `closed: true` means `end` is the index of the CLOSING `/`. `closed: false`
72+
* means the body ran into a LineTerminator or EOF and `end` is where it
73+
* stopped — what THAT means is the caller's to decide, and the two consumers of
74+
* this walk decide it differently on purpose (see the module's header).
75+
*/
76+
export interface RegexBodyWalk {
77+
end: number;
78+
closed: boolean;
79+
}
80+
81+
/** IdentifierPart, near enough for the ASCII this tree is written in. */
82+
export const IDENT_CHAR: RegExp;
83+
84+
/**
85+
* Keywords after which a `/` opens a REGEX rather than a division. Shared, so
86+
* that a second copy cannot drift: dropping `return` from it passes every
87+
* pinned case and is caught only by the corpus sweep.
88+
*/
89+
export const REGEX_AFTER_KEYWORD: ReadonlySet<string>;
90+
91+
/** LineTerminator — the four the grammar names, which a regex body excludes. */
92+
export function isRegexLineTerminator(c: string): boolean;
93+
94+
/**
95+
* Walk the body of the regex literal opening at `at`, which the CALLER has
96+
* already decided sits where an expression may begin.
97+
*/
98+
export function walkRegexBody(src: string, at: number): RegexBodyWalk;
99+
100+
/**
101+
* Bind a POSITION RULE to the shared walk and get back a recogniser answering
102+
* "index of the closing `/`, or -1". There is no default rule and omitting one
103+
* throws — a default would be one of two opposite failure directions, handed
104+
* silently to whichever caller forgot.
105+
*/
106+
export function makeRegexRecogniser(options: {
107+
mayBeginAt: (src: string, at: number) => boolean;
108+
}): (src: string, at: number) => number;
109+
68110
/**
69111
* Drive the scanner over its own fixture corpus, printing a line per case.
70112
*

0 commit comments

Comments
 (0)