Observed
On #1044, Ubuntu - parallel extended and Alpine - parallel extended failed together with an identical diff:
✗ Failed: Parallel does not lose same named files in different dirs
Expected '2 passed 2 total 2 passed 2 total '
but got '0 total 0 total 2 passed 2 total 2 passed 2 total '
gh run rerun --failed on the same commit went green, so it is not deterministic — but two independent platforms failing in the same run, the same way, is not the shape of ordinary flakiness and is worth understanding rather than papering over with reruns.
What the diff says
tests/acceptance/bashunit_parallel_consistency_test.sh:51 scrapes counts with
grep -oE '[0-9]+ (passed|total)' over the whole captured output of a nested run. The parallel
side picked up an extra 0 total pair ahead of the real summary — i.e. the capture contains
the tail of a second bashunit run that found no tests ("Tests: 0 total" / "Assertions: 0 total").
So the question is where a zero-test summary comes from inside that capture. Two candidates,
neither confirmed:
- Output from a concurrently running acceptance test leaking into this one's capture under
--parallel (the suite runs many nested ./bashunit invocations).
- The nested run genuinely selecting nothing on the first of its two invocations, under load.
Why the assertion cannot tell us
The test compares scraped counts, not structure, so any stray summary text anywhere in the
capture changes the result. It already carries a assert_contains "2 total" guard against both
sides collapsing vacuously — but that guard passes here, because the real counts are present;
they just have extra ones in front.
Suggested direction
- Anchor the scrape to the summary block rather than the whole capture (e.g. take the last
passed/total pair, or match the Tests: line specifically), so unrelated output cannot
shift the comparison.
- If leakage is the cause, that is worth knowing on its own: it would affect every acceptance
test that captures a nested run's output.
Repro
Not reproducible locally: 3× targeted and 2× full ./bashunit --parallel tests/ on macOS all
green. Needs CI load, or --repeat against the acceptance suite.
Acceptance criteria
Observed
On #1044,
Ubuntu - parallel extendedandAlpine - parallel extendedfailed together with an identical diff:gh run rerun --failedon the same commit went green, so it is not deterministic — but two independent platforms failing in the same run, the same way, is not the shape of ordinary flakiness and is worth understanding rather than papering over with reruns.What the diff says
tests/acceptance/bashunit_parallel_consistency_test.sh:51scrapes counts withgrep -oE '[0-9]+ (passed|total)'over the whole captured output of a nested run. The parallelside picked up an extra
0 totalpair ahead of the real summary — i.e. the capture containsthe tail of a second bashunit run that found no tests ("Tests: 0 total" / "Assertions: 0 total").
So the question is where a zero-test summary comes from inside that capture. Two candidates,
neither confirmed:
--parallel(the suite runs many nested./bashunitinvocations).Why the assertion cannot tell us
The test compares scraped counts, not structure, so any stray summary text anywhere in the
capture changes the result. It already carries a
assert_contains "2 total"guard against bothsides collapsing vacuously — but that guard passes here, because the real counts are present;
they just have extra ones in front.
Suggested direction
passed/totalpair, or match theTests:line specifically), so unrelated output cannotshift the comparison.
test that captures a nested run's output.
Repro
Not reproducible locally: 3× targeted and 2× full
./bashunit --parallel tests/on macOS allgreen. Needs CI load, or
--repeatagainst the acceptance suite.Acceptance criteria