You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
finding(test-infra): tests that spawn a child process and assert on its HUMAN-READABLE output pass locally and fail only under GitHub Actions' colouring — one instance cost a CI cycle, the class is unswept #7897
Filed unassigned and unlabelled by the domain:devx @ objectui execution seat, PM session session_01KDq78vMMSzCGWGmhUYBabh, R43. ⛔ Grading and routing are the triage seat's. Surfaced by the dev on #7337 during the patch round on PR #7889 and offered rather than filed there, because filing it properly needed a dedupe pass that round had no budget for. Filing it here so a measured class does not die in a report.
The defect shape
A test spawns a child process and asserts on the child's human-readable stdout with a regex. It passes locally and fails only in CI, because the child colours its output there.
Measured on the real failure (job 101407488095, PR #7889): the child vitest's summary line in CI is
⇒ SGR sequences sit between Tests and 1 failed, so \s+ cannot match. The assertion reads as obviously-correct in review and in local runs; only CI disagrees.
⚠️A second, independent leg of the same shape: a spawned child that inherits GITHUB_ACTIONS=true switches on vitest's github-actions reporter and writes ::error file=… annotations into the parent's CI log. Where the child is supposed to fail (a fixture proving a failure mode), the parent run is decorated with error annotations from fixtures behaving correctly. Measured both ways on #7889: 1 annotation line with the variable, 0 without.
Why the class is worth a sweep rather than one fix
⛔ The failure direction is the expensive one: green locally, red only in CI, so it is discovered by burning a CI cycle on an unrelated PR, by whoever happens to touch that test next. #7889 paid exactly that cost.
⭐ And the near-miss is worse than the failure. Fixing the instance revealed that the original assertions were not mutually exclusive — two different child outcomes both print Test Files 1 failed, so a prose regex that "passed" was not discriminating between them. ⇒ This shape does not only break in CI; it can also be silently weaker than it reads, which no CI run catches at all.
Scope named, ⛔ not measured by this seat
scripts/__tests__/ holds several execFileSync / spawnSync gate-runner tests. The dev's own read: the ones asserting on a gate's own verdict line are safe, because those gates do not colour. The unsafe ones are those asserting on a third-party tool's presentation — vitest above all.
⛔ This seat did not enumerate them; the count is the sweep's job.
Suggested shape for whoever takes it
Enumerate the child-spawning assertions in scripts/__tests__/ and split them by whether the child is a repo gate or a third-party tool.
PR #7889 is the worked instance — reproduction from the real CI bytes, the JSON-reporter fix, and ablation legs D and E proving the two outcomes are not interchangeable.
Dedupe bound, declared
Searched objectstack-ai/objectuistate=alllabels=domain:devx, pages 1–3, title-matching ansi / colour|color+output|child / spawn+assert. No hits. ⚠️ Bounded to titles across ~300 cards; bodies were not searched and the label set exceeds one page. /search/* is 403 from this container (local egress proxy), so a semantic search was not available.
Filed unassigned and unlabelled by the
domain:devx@objectuiexecution seat, PM sessionsession_01KDq78vMMSzCGWGmhUYBabh, R43. ⛔ Grading and routing are the triage seat's. Surfaced by the dev on #7337 during the patch round on PR #7889 and offered rather than filed there, because filing it properly needed a dedupe pass that round had no budget for. Filing it here so a measured class does not die in a report.The defect shape
A test spawns a child process and asserts on the child's human-readable stdout with a regex. It passes locally and fails only in CI, because the child colours its output there.
Measured on the real failure (job
101407488095, PR #7889): the child vitest's summary line in CI is⇒ SGR sequences sit between
Testsand1 failed, so\s+cannot match. The assertion reads as obviously-correct in review and in local runs; only CI disagrees.GITHUB_ACTIONS=trueswitches on vitest's github-actions reporter and writes::error file=…annotations into the parent's CI log. Where the child is supposed to fail (a fixture proving a failure mode), the parent run is decorated with error annotations from fixtures behaving correctly. Measured both ways on #7889: 1 annotation line with the variable, 0 without.Why the class is worth a sweep rather than one fix
⛔ The failure direction is the expensive one: green locally, red only in CI, so it is discovered by burning a CI cycle on an unrelated PR, by whoever happens to touch that test next. #7889 paid exactly that cost.
⭐ And the near-miss is worse than the failure. Fixing the instance revealed that the original assertions were not mutually exclusive — two different child outcomes both print
Test Files 1 failed, so a prose regex that "passed" was not discriminating between them. ⇒ This shape does not only break in CI; it can also be silently weaker than it reads, which no CI run catches at all.Scope named, ⛔ not measured by this seat
scripts/__tests__/holds severalexecFileSync/spawnSyncgate-runner tests. The dev's own read: the ones asserting on a gate's own verdict line are safe, because those gates do not colour. The unsafe ones are those asserting on a third-party tool's presentation — vitest above all.⛔ This seat did not enumerate them; the count is the sweep's job.
Suggested shape for whoever takes it
scripts/__tests__/and split them by whether the child is a repo gate or a third-party tool.@object-ui/i18nmock factories inherit the real surface, and the recogniser stops mis-reading a nested generic (objectui#7337) #7889's fix reads the child's JSON reporter) over regexing presentation; strip ANSI only as the second belt.GITHUB_ACTIONSfrom spawned child envs where the child is expected to fail, so deliberate fixtures stop annotating the parent run.@object-ui/i18nmock factories inherit the real surface, and the recogniser stops mis-reading a nested generic (objectui#7337) #7889 found its real weakness, and a presentation fix alone would have preserved it.Reference
PR #7889 is the worked instance — reproduction from the real CI bytes, the JSON-reporter fix, and ablation legs D and E proving the two outcomes are not interchangeable.
Dedupe bound, declared
Searched⚠️ Bounded to titles across ~300 cards; bodies were not searched and the label set exceeds one page.
objectstack-ai/objectuistate=alllabels=domain:devx, pages 1–3, title-matchingansi/colour|color+output|child/spawn+assert. No hits./search/*is 403 from this container (local egress proxy), so a semantic search was not available.Generated by Claude Code