Skip to content

fix(reports): every report format was empty under --parallel - #1006

Merged
Chemaclass merged 1 commit into
mainfrom
fix/1004-parallel-reports
Aug 9, 2026
Merged

fix(reports): every report format was empty under --parallel#1006
Chemaclass merged 1 commit into
mainfrom
fix/1004-parallel-reports

Conversation

@Chemaclass

Copy link
Copy Markdown
Member

🤔 Background

Related #1004

--report-junit, --report-tap, --report-json, --report-html and --log-junit all produced a report of zero tests whenever the run was parallel. The tests ran and passed; only the report was empty.

./bashunit --report-junit r.xml tests/unit/util              # tests="49"
./bashunit --parallel --report-junit r.xml tests/unit/util   # tests="0"

That's worse than an obviously broken file. --parallel is the mode people reach for in CI, junit is what CI consumes, and a junit file reporting no tests renders as a green, empty run. A suite could be fully green, fully reported, and communicating nothing.

🔍 Cause

reports::add_test is called from runner/exec.sh, which under --parallel runs inside the per-test worker. The arrays it appends to died with the worker, and state/parallel.sh — which rebuilds counters from the .result files — had no knowledge of them. Counters crossed the fork boundary; report rows did not.

💡 Fix

Each row is also spooled to a run-scoped file — the same mechanism --snapshot-report-unused already uses to cross that boundary — and replayed in the parent immediately before the writers run. Fields are base64-encoded because a failure message carries newlines and arbitrary text.

The arrays are still filled rather than skipped when parallel. My first version returned early there, and 32 reports unit tests started failing — they call add_test directly in the parent and assert the arrays. The parent never reaches that path for a real parallel test, so replaying the spool cannot double-count.

✅ Verification

All three text reports generated from one fixture both ways and compared: junit identical; tap and json identical in content, differing only in row order, which is inherent to parallel execution. Statuses, durations, assertion counts and the multi-line failure message all survive.

The new acceptance test runs the same suite both ways and compares counts, so they cannot drift apart again.

make sa · make lint · bash build.sh bin -v✅ Build verified ✅ · fork budget unchanged · 1735 sequential / 1694 parallel-simple-strict.

Closes #1004.

--report-junit, --report-tap, --report-json, --report-html and --log-junit all
produced a report of zero tests whenever the run was parallel. The tests ran and
passed; only the report was empty.

    ./bashunit --report-junit r.xml tests/unit/util              tests="49"
    ./bashunit --parallel --report-junit r.xml tests/unit/util   tests="0"

That is worse than an obviously broken file. --parallel is the mode people reach
for in CI and junit is what CI consumes, and a junit file reporting no tests
renders as a green, empty run in most systems. A suite could be fully green,
fully reported, and communicating nothing.

bashunit::reports::add_test is called from runner/exec.sh, which under
--parallel runs inside the per-test worker. The arrays it appends to died with
the worker, and state/parallel.sh -- which rebuilds counters from the .result
files -- had no knowledge of them. Counters crossed the fork boundary; report
rows did not.

Each row is now also spooled to a run-scoped file, the same mechanism
--snapshot-report-unused already uses to cross that boundary, and replayed in
the parent immediately before the writers run. Fields are base64-encoded because
a failure message carries newlines and arbitrary text.

The arrays are still filled in add_test rather than skipped when parallel. The
first version returned early there, which was wrong: the reports unit tests call
add_test directly in the parent, and 32 of them started asserting against arrays
nothing had touched. The parent never reaches that path for a real parallel
test, so replaying the spool cannot double-count.

Verified by generating all three text reports from the same fixture both ways
and comparing: identical for junit, and identical in content for tap and json --
they differ only in row order, which is inherent to parallel execution. Statuses,
durations, assertion counts and the multi-line failure message all survive.

The new acceptance test runs the same suite sequentially and in parallel and
compares the counts, so the two cannot drift apart again.

1735 sequential / 1694 parallel; fork budget unchanged.
@Chemaclass Chemaclass added the bug Something isn't working label Aug 9, 2026
@Chemaclass Chemaclass self-assigned this Aug 9, 2026
@Chemaclass
Chemaclass merged commit afcfafb into main Aug 9, 2026
37 checks passed
@Chemaclass
Chemaclass deleted the fix/1004-parallel-reports branch August 9, 2026 12:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant