perf(coverage): coverage costs 4.7-7.8x, and macOS system bash always gets the slowest engine

[Chemaclass]

Summary

Running with --coverage multiplies wall-clock time by 4.7x to 7.8x depending on Bash version and engine. Measured on tests/unit/util (49 tests), same machine:

	Bash 5.3.15	Bash 3.2.57 (macOS system bash)
no coverage	691 ms	707 ms
--coverage with xtrace	3255 ms (4.7x)	not available
--coverage with trap	4622 ms (6.7x)	5496 ms (7.8x)
--coverage with auto	3458 ms (5.0x) — picks xtrace	5496 ms — falls back to trap

auto resolves correctly on both. The problem is what it resolves to.

The part worth acting on first

bashunit::coverage::xtrace_is_supported requires Bash 4.1+. macOS ships 3.2 as /bin/bash, so on a stock Mac every coverage run gets the trap engine — the slowest path — and setting BASHUNIT_COVERAGE_ENGINE=xtrace changes nothing, silently.

That is correct behaviour (the ADR-009 engine needs BASH_XTRACEFD), but it means the audience most likely to run coverage locally is the audience that gets the worst of it, with no signal explaining why. A developer comparing notes with a Linux colleague sees 7.8x against their 4.7x and has nothing to tell them why.

DX observations

• The cost discourages local use. At 5x, --coverage stops being something you run while working and becomes a CI-only step. That is a real loss: coverage is most useful while writing the test, not after merging it.
• Silent engine downgrade. BASHUNIT_COVERAGE_ENGINE=xtrace on Bash 3.2 is accepted and ignored. --verbose does not say which engine ran. Worth surfacing, even just once per run.
• No diff coverage. The report is whole-file. For a PR the useful question is "are the lines I touched covered", and answering it today means reading a full report and doing the diff by eye. This is the single biggest DX win available, and it is independent of the perf work.
• Good news, verified: coverage totals are identical under --parallel and sequential (129/286 both ways), so the aggregation is sound. Worth stating because the report writers are not — see fix(reports): every report format is empty under --parallel #1004.

Directions worth investigating, roughly in order

1. Measure where the trap engine actually spends its time before optimising it. .claude/rules/perf-fork-budget.md documents the shim-census method used for the runner; the same approach applied to a --coverage run would say whether the cost is the DEBUG trap firing, the per-line bookkeeping, or the report pass. No optimisation should be chosen before that is known.
2. Report which engine ran, at least under --verbose, and warn when an explicit BASHUNIT_COVERAGE_ENGINE request was downgraded.
3. Diff coverage — restrict the report to lines changed against a base ref. Large DX win, no engine work needed.
4. Reconsider the 4.1 floor for xtrace. Bash 3.2 has no BASH_XTRACEFD, so this is likely a hard no; worth confirming and writing down so it is not re-asked.

Constraints

• Bash 3.0+ floor. The trap engine has to keep working there regardless of what happens to xtrace.
• Per-test paths stay fork-free — see .claude/rules/perf-fork-budget.md. Note the existing budget already allows ~3 forks per newly-seen file under --coverage.
• Coverage is exercised by tests/unit/coverage/ and gated by a nightly workflow, not by PR CI, so a regression here is not caught on the PR — see fix(ci): coverage workflow globbed a flat tests/unit that no longer exists #980.
• Any measurement quoted in a PR should state the Bash version and engine, since the two differ by 1.4x on the same machine.

Acceptance criteria

• A profile exists showing where --coverage time goes, per engine, before any optimisation lands
• The engine in use is discoverable at runtime, and a downgraded explicit request is not silent
• A decision is recorded on diff coverage: in scope, or its own issue
• Any perf change keeps the trap engine correct on Bash 3.0, and keeps parallel and sequential totals identical
• make sa · make lint · ./bashunit --parallel --simple --strict tests/ · bash build.sh bin -v

[Chemaclass]

Done in #1031 (merged).

Profile first (bash 5.3, tests/unit/util, 3 files / 286 lines), per the first acceptance criterion:

phase	trap	xtrace
tests + startup	2434ms	1448ms
finalize	4ms	372ms
precompute_file_stats	788ms	929ms
report_text	15ms	17ms
report_lcov	1555ms	1144ms
report total	2367ms	2467ms

The report phase was ~50-60% of the run and near-identical for both engines — so the premise that this was an engine problem was wrong. is_executable_line forked grep -E per unclassified line, and every tracked line is classified twice per run: 1055 grep forks vs 0 without --coverage.

After (interleaved best-of-5, old vs new on the same machine):

bash	engine	before	after
5.3	trap	4260ms	2514ms	1.7x
5.3	xtrace	3213ms	1396ms	2.3x
3.2	trap	4655ms	2990ms	1.6x

Multiplier over a no-coverage run drops from 4.7-7.8x to roughly 2-4x. macOS system bash still gets trap, but it is no longer the worst case by much.

Acceptance criteria

• Profile exists, per engine, before any optimisation landed
• Engine discoverable (--verbose prints it); a downgraded explicit request warns instead of being silent
• Diff coverage decision recorded: its own issue, feat(coverage): diff coverage — restrict the report to lines changed against a base ref #1032 — it shares no code with the perf work
• Trap engine still correct on the 3.0 floor; totals unchanged (129/286 trap, 174/286 xtrace) and parallel == sequential per engine
• make sa · make lint · ./bashunit --parallel --simple --strict tests/ · bash build.sh bin -v

Item 4 (reconsidering the 4.1 floor for xtrace) is confirmed a hard no and now written down in docs/coverage.md: Bash 3.2 has no BASH_XTRACEFD, so xtrace could only reach stderr, where it would interleave with the output of the code under test.

Correctness evidence: both classifier implementations were run over the same 41173 lines across 395 files on Bash 3.2 and 5.3 with zero disagreements, and the differential harness was mutation-tested so it is known to discriminate. Two ERE quirks are preserved deliberately — inside a POSIX bracket expression a backslash is a literal set member, so [^\)] also excludes \ and [\{\}] also matches a bare line continuation.