test(scripts): read spawned children's verdicts as numbers, not prose

[baozhoutao]

Fixes #7897

Base 01c27c431. Head at the time every reading below was taken: db58b216b.

The card asked for a sweep in four steps: census, convert the unsafe subset to a machine-readable verdict, drop GITHUB_ACTIONS from children that are expected to fail, and — the step triage called the core of the card — check that each assertion actually discriminates the outcomes it names. All four are below, including one falsified premise and one step whose target set turned out to be empty.

STEP 1 — the census

Population: scripts/__tests__/*.test.ts on 01c27c431 — 112 test files, 29 of which spawn a child (execFileSync / spawnSync / execSync / spawn). Triage's upper bound was 27 of 110; the dispatching seat read 29. 29 is confirmed.

⚠️ The dispatching seat's assumption "all 29 spawn vitest" is falsified, and the way it is false matters: all 29 files mention vitest, because every test file in this repo imports describe/it/expect from it. Exactly one file spawns the vitest binary — check-vi-mock-inherit.test.ts, the worked instance PR #7889 already converted, and one of the four names excluded here. Measured with grep -rn "node_modules/.bin" scripts/__tests__/.

Split criterion, from the card: asserting on a repo gate's own verdict line is safe because gates do not colour; asserting on a third-party tool's presentation is unsafe. ⭐ That premise was measured rather than assumed — grep -rlE 'chalk|picocolors|kleur|ansi-colors|\\u001b\[|\\x1b\[|\\033\[' over scripts/ returns three files, and one of them, scripts/shadcn-sync.js, is spawned by a test in this census. So "repo script ⇒ safe" is not true in this tree; "repo gate ⇒ safe" is (no check-*.mjs colours).

#	file	child spawned	asserted on	inherits GITHUB_ACTIONS	discriminates?	verdict
1	bash32-floor-wiring	node check-bash32-floor.mjs --self-test; git ls-files	gate verdict line; git data	yes (child is not vitest)	no — \d+ cases pass accepts 0	converted
2	catalog-index-regenerable-4633	python3 scripts/regenerate-catalog-index.py (repo generator)	generator prose, distinct strings + status	yes	yes	safe
3	check-changeset-overwrite	node gate; git (fixture setup)	gate verdict + exact counts	yes	yes	safe
4	check-changeset-presence	node gate; git (fixture setup)	gate verdict + exact counts	yes	yes	safe
5	check-control-bytes	git (setup); GNU grep	grep's human-readable output	yes	no — see below, the negative half could not fail	converted
6	check-doc-component-types	node gate; git ls-files	gate stderr prose	yes	yes	safe
7	check-doc-expression-carriage	node gate (+--self-test)	gate verdict, distinct strings + status	yes	yes	safe — excluded, PR #8105
8	check-doc-fence-languages	node gate --self-test	gate verdict line	yes	no — \d+ accepts 0	converted
9	check-doc-links	git ls-files	data	yes	n/a	safe
10	check-doc-snippet-types	bash on a workflow run: step, executables shimmed	step status, stderr, GITHUB_OUTPUT file	yes	yes	safe — excluded, #5174 batch 24
11	check-governed-queue-guard	node gate --self-test	gate verdict line	yes	no — \d+ accepts 0	converted
12	check-half-states	node scripts/pm/check-half-states.mjs --self-test	gate verdict line	yes	no — \d+ accepts 0	converted
13	check-i18n-en-drift	node gate; git (setup)	gate verdict + exact counts	yes	yes	safe
14	check-pre-install-import-graph	node gate --self-test	toContain('self-test:') + /^✓/	yes	no — asserts no count at all	converted
15	check-shell-escape-residue	node gate	verdict + census count, floors pinned in-process	yes	yes	safe
16	check-skills-paths	node gate	: OK + exit 0, floors pinned in-process	yes	yes	safe
17	check-vi-mock-inherit	vitest, node gate, git grep	child's JSON reporter; prose only after stripAnsi	no — removed deliberately	yes	already converted by PR #7889 — excluded, #6892 slice 5
18	check-vi-mock-specifiers	node gate; git init	gate verdict + exact census counts	yes	yes	safe
19	ensure-chromium-ready	bash scripts/ensure-chromium-ready.sh (repo script, no colour)	script prose + a marker file	yes	yes	safe
20	entry-guard-wiring	node gate --self-test	gate verdict line	yes	no — \d+ accepts 0	converted
21	js-comment-mask-jsx-6891	node scripts/js-comment-mask.mjs --self-test	gate verdict line	yes	no — \d+ accepts 0	converted
22	one-authority-per-exported-name-6273	git ls-files -z	data	yes	n/a	safe
23	package-files-exist	git ls-files -z, git check-ignore	data / exit code	yes	n/a	safe
24	shadcn-sync-fetch-cache	node scripts/shadcn-sync.js --list — colours unconditionally	two substring matches on RAW stdout	yes	no — passes on a run that prints the two headers and no components	converted
25	side-effects-declaration-consistency	git ls-files -z	data	yes	n/a	safe
26	site-next-agent-files-4160	git check-ignore, git ls-files --error-unmatch	exit codes	yes	n/a	safe
27	turbo-build-outputs-cover-published-entries	git ls-files	data	yes	n/a	safe
28	unconsumed-widget-option-claim-6186	git ls-files -z	data	yes	n/a	safe
29	upstream-port-parity-wiring	node gate --self-test and plain	two gate verdict lines	yes	no, twice — \d+ cases pass, and ported file(s) match which 0 ported file(s) match satisfies	converted

Totals: 29 spawning files · 3 assert on a third-party tool's presentation (rows 5, 17, 24) · 26 assert on a repo gate's verdict or on machine-readable git data · 7 carry no presentation assertion at all (rows 9, 22, 23, 25, 26, 27, 28) · 10 files converted, carrying 13 assertion sites · 4 excluded by name.

⚠️ One of the four named exclusions, check-i18n-call-site-keys.test.ts (PR #8101), is not in this census at all: it spawns no child process. Nothing was owed on it and nothing is left behind there.

The two defects the census found, in full

check-control-bytes — a negative assertion that could not fail. The pair was:

const out = execFileSync('grep', ['-n', needle, file], { cwd: repoRoot, encoding: 'utf8' });
expect(out).toMatch(new RegExp(needle));
expect(out).not.toMatch(/binary file matches/);

Measured on GNU grep 3.11 — the build this container carries and the one the file's own header already cites — a declined binary file produces:

exit=0   stdout=(empty)   stderr=grep: probe.ts: binary file matches

execFileSync returns stdout only. The message the negative assertion hunts is written on stderr, so that assertion was matching a stream it can never appear on: green for every file on earth, including the one it exists to refuse. This is precisely the card's near-miss shape — an assertion silently weaker than it reads, which no CI run catches, because it passes. The whole pin was carried by the positive half beside it (a declined file yields empty stdout, so toMatch(needle) did fail). Both halves are real now: spawnSync, both streams read, both of grep's historical spellings of the refusal recognised, and a new case hands grep a NUL-bearing fixture as the control that the negative half can go red.

shadcn-sync-fetch-cache — a colouring child, survived by luck. scripts/shadcn-sync.js writes SGR unconditionally: no tty check, no NO_COLOR. Component List and Custom ObjectUI Components: happened to be wrapped whole, so a substring match on raw stdout still hit — but every per-component line is • ${name.padEnd(20)} ESC[2m${description}ESC[0m, an escape sitting between the two things a reader would want to assert across. That is the same byte layout that broke PR #7889 in CI. The two old substring checks also passed on a run that printed the headers and zero components, which is the silent no-op the case says it exists to refuse. Now: ANSI stripped, then every entry of packages/components/shadcn-components.json reconciled against the printer's own format. ⛔ No machine-readable channel was preferable here — --list has no JSON mode and this card's surface is test files only, so no gate script was touched; stripping is the belt and the manifest is the machine-readable half that was available.

STEP 2 — the conversions

scripts/__tests__/helpers/child-verdict.ts, one shared reader (earned: 9 call sites, well past the three-conversion bar), with its own unit case in scripts/__tests__/child-verdict.test.ts:

• stripAnsi — SGR built from the escape's code point, never a raw control byte.
• verdictCount(output, pattern, what) — strips, then captures one number; throws naming the whole output when the pattern misses, so an absent verdict can never be read back as a real zero.
• selfTestCases(output, gate) — the <gate> self-test: N cases pass line every gate here ends with, prefix-agnostic (✓, OK, or none).

Every converted call site keeps the original prose match (now applied to the stripped output — the second belt) and adds the count assertion, so no assertion is weaker than it was. upstream-port-parity-wiring goes further: the ported file(s) match count is reconciled against scripts/upstream-port-pin.json as shipped in the same commit, so an emptied pin can no longer read as a tree at parity.

⭐ The helper's unit case pins the non-equivalence of the old and new spellings in both directions — 0 cases pass satisfies /\d+ cases pass/ and is refused by the count; coloured bytes fail the raw regex and are read correctly after stripping. Without that, the next reader is free to conclude the two spellings are interchangeable, and they are not: that is the whole content of the module.

STEP 3 — GITHUB_ACTIONS in child envs: measured, and the target set is empty

The leg is real. Reproduced independently of any repo file, on a throwaway fixture with one deliberately failing test and vitest resolved by symlink:

with    GITHUB_ACTIONS=true : exit=1  ::error lines = 1
without GITHUB_ACTIONS      : exit=1  ::error lines = 0

::error file=.../fails.test.mjs,title=fails.test.mjs > fails on purpose ...,line=2,column=77::AssertionError: expected 1 to be 2 ...

Same 1-vs-0 the card measured on PR #7889. But there is nothing left to change. Only vitest children emit those annotations, and the census finds exactly one test that spawns vitest — check-vi-mock-inherit.test.ts, which already deletes GITHUB_ACTIONS from the child env (PR #7889) and is excluded here. The three repo scripts that print ::error themselves (check-merge-queue-head.mjs, dependabot-merge-gate.mjs, shadcn-check-report.mjs) are imported by four tests in this census but spawned by none. So step 3 is reported as measured-and-satisfied rather than applied.

STEP 4 — discrimination

Ten of the thirteen converted assertion sites were non-discriminating; they are listed per-row in the census table and detailed above. The pattern in eight of them is one shape: a count written \d+, which accepts 0 — a self-test whose case table went empty prints 0 cases pass and exits 0, and the pin reads green. Every one of them now asserts the count, not the shape.

Ablation — one converted file, both spellings on the same run

On shadcn-sync-fetch-cache.test.ts, with the conversion committed first, the pre-conversion spelling (the printer-format match against the child's raw stdout) was re-introduced on disk beside the converted one and both were run together with FORCE_COLOR=1 in the parent env. Mutation proved on disk by blob hash and marker count before restoring under a trap with absolute paths:

HEAD blob   = cff66ba7954f9bc8f1d6e3976008cf9f64e8e781
before mut  = cff66ba7954f9bc8f1d6e3976008cf9f64e8e781   marker count before = 0
after mut   = ce03b8486f17bfc8d04f5c8394261abfe007f0f2   marker count after  = 2

× matches the printer format on the RAW stdout                        <- the old spelling
AssertionError: expected '\n===================================…'
                to contain '  • button-group         Custom Objec…'
Test Files  1 failed (1)
     Tests  1 failed | 26 passed (27)                                 <- the converted one is in the 26

restored    = cff66ba7954f9bc8f1d6e3976008cf9f64e8e781
git diff HEAD -- scripts/__tests__/shadcn-sync-fetch-cache.test.ts : EMPTY
blob identical to HEAD                                marker count restored = 0

⚠️ Worth stating plainly, because it changes how the result reads: this child colours unconditionally, so FORCE_COLOR=1 is not what makes the old spelling fail — it fails on every machine. That makes the ablation deterministic rather than environment-dependent, and it also means this particular file was one manifest description away from failing locally, not only in CI.

Gates — all at db58b216b, exit codes captured by redirect-then-capture

gate	exit	verdict line
pnpm exec vitest run scripts/__tests__/	0	Test Files 113 passed (113) · Tests 3356 passed (3356) · 214.00s. No flake, so no second run was owed.
the 11 changed/added test files, alone	0	Test Files 11 passed (11) · Tests 219 passed (219)
pnpm type-check:scripts	0	tsc -p tsconfig.scripts.json, no diagnostics
pnpm lint:root	0	✖ 32 problems (0 errors, 32 warnings) — none of the 32 is in a changed file (grepped by name)
pnpm check:doc-example-readers	0	OK 80 documented symbol(s), 3947 call site(s), 11 slot(s)…
pnpm check:control-bytes	0	✅ check-control-bytes: OK (scanned 6487 tracked text file(s); skipped 85 binary).
manual control-byte scan of all 12 changed paths	1 (no hits)	grep -naP '[\x00-\x08\x0b\x0c\x0e-\x1f\x7f]' — empty
node scripts/check-changeset-presence.mjs	0	12 file(s) changed, 0 of them published source… · ✅ no changeset is owed
node scripts/check-governed-queue-guard.mjs --test <12 paths>	0	✅ NOT GOVERNED — 12 path(s) checked against 5 governed surface(s); none matched.

Re-derived beyond the dispatched list, because the diff adds a .ts file under scripts/ and a new test file: check:unreferenced-sources 0, check:entry-guard 0 (70 scripts/ file(s)), check:vi-mock-specifiers 0, check:vi-mock-inherit 0, check-lint-coverage.mjs 0 (46/46 packages linted). All green.

Live E2E (informational) is red on every branch today for an upstream reason (#7990, objectstack#16186) — not this branch's.

Scope

Test files and one test-only helper. ⛔ No gate script under scripts/*.mjs, ⛔ no packages/** source, ⛔ no vitest.config.*, ⛔ no timeout raised anywhere. Changeset: none owed, per the presence script's own verdict above. Related and named without closing anything: PR #7889 (the worked instance this copies), #8062 and #8074 (the TIMEOUT class, a different class, left alone here).

🤖 Generated with Claude Code

https://claude.ai/code/session_01FhBNJcLRZLe8M87VcUgpKr

---

Generated by Claude Code

[baozhoutao]

Standing down on Live E2E (informational) for this PR, read 2026-09-06T16:30Z.

• Failing check: Live E2E (informational) — informational, not a required context, and red on main and on every open branch today for an upstream reason tracked in ci(live-e2e): Live E2E (informational) is red on main and every branch since the backend pin moved to 17.2.0 — the published plugin-auth float (objectstack#16186), not an objectui diff #7990 / objectstack#16186. This diff touches only scripts/__tests__/** (one helper, one unit case, ten converted pin tests) and nothing the live E2E exercises.
• Why it is not this PR's: base-red anchored on the two cards above; the failure names a service the diff does not touch.
• Fix ported: none exists yet in either repo; nothing to port.
• Re-run: not spent — a re-run of a base-red informational job cannot change the reading.

Every other check on db58b216b is green or still running; this PR flips to ready once the required set converges, then goes to the merge queue by squash.

---

Generated by Claude Code

[baozhoutao]

Armed — squash auto-merge enabled and the PR is in the merge queue, domain:devx @ objectui execution seat, PM session session_01FhBNJcLRZLe8M87VcUgpKr, R46.

• Head db58b216b: 28 non-informational checks completed green at 16:39Z (the ci-wait reading); Live E2E (informational) red per the standing-down note above.
• ready_for_review at 16:42:17Z; the post-flip Governed Surface Queue Guard run started after that timestamp and completed success at 16:42:33Z.
• added_to_merge_queue at 16:42:41Z on the REST timeline (read 16:42:47Z).

On MERGED: content probe on re-fetched origin/main (the helper scripts/__tests__/helpers/child-verdict.ts present, stripAnsi in a converted test, and a control that an untouched spawning test is unchanged), then the LANDED note here and the closing stroke on #7897.

---

Generated by Claude Code

[baozhoutao]

LANDED — merged at 2026-09-06T16:57:56Z as fde3dcf41 (squash, via the merge queue; all 17 merge-group runs green), domain:devx @ objectui execution seat, PM session session_01FhBNJcLRZLe8M87VcUgpKr, R46.

Content probe on re-fetched origin/main (fde3dcf41, read 16:58:19Z):

• scripts/__tests__/helpers/child-verdict.ts present.
• stripAnsi appears in 13 files under scripts/__tests__/ (the helper, its unit case, ten converted tests, and the reader count matches the PR's file list); selfTestCases(out present in bash32-floor-wiring.test.ts, and its old expect(out).toMatch(/check-bash32-floor self-test spelling is gone.
• Control: check-changeset-presence.test.ts (row 4 of the census, judged safe) still spawns (11 execFileSync/spawnSync sites) and carries no stripAnsi — an untouched member reads unchanged.

#7897 closed by the Fixes at 16:57:58Z; the seat strips pm:dispatched and clears the assignee there in the same stroke.

---

Generated by Claude Code