Problem
--retry mitigates flakiness once it has already burned a CI run. There is no way to go looking for it: to prove a test is deterministic, you have to run the suite N times by hand and diff the outputs.
Related: #1012 classifies a test that only passed on retry as flaky — that tells you a test was flaky after the fact; --repeat lets you find out before merging.
Proposal
--repeat <n> Run each selected test n times (default 1); the test fails if any iteration fails
- Applies per test, inside the runner, so
--repeat 50 --filter flaky_candidate is the natural "hammer this one test" invocation.
- A test is reported once, with the aggregate outcome. On failure, the message names the failing iteration:
failed on iteration 7 of 50.
- Counters count the test once, not n times.
- Composes with
--parallel (each worker repeats its own test) and with --retry (repeat is the outer loop, retry the inner one) — document the interaction explicitly.
--repeat 0 is a usage error, not a silent no-op.
Where to change
src/runner/exec.sh:312 — the retry while :; loop is the natural place; repeat wraps it.
src/main/test.sh, bashunit::main::require_non_negative_int_or_exit (src/main/validate.sh:19) for validation, src/config/env.sh (BASHUNIT_REPEAT), .env.example.
Acceptance criteria
Repo checklist (agent)
- TDD: RED → GREEN → REFACTOR. Write the failing test first.
- Bash 3.0+ only: no
printf -v, no += append, no declare -A, no [[ ]], no ${var,,}, no &>>, no ${arr[-1]}. Expanding a possibly-empty array under set -u needs ${arr[@]+"${arr[@]}"}.
- A new CLI flag must be wired in all of these or a parity test fails:
- Gates:
make sa, make lint, ./bashunit tests/, ./bashunit --parallel tests/. Never run shfmt -w.
- Docs: update
docs/command-line.md. Editing docs/assertions.md invalidates the bashunit doc acceptance snapshot — regenerate it.
- CHANGELOG.md: add one line under
## Unreleased.
- Fixtures under
tests/acceptance/fixtures/ must not end in *test.sh.
- One issue = one PR.
Problem
--retrymitigates flakiness once it has already burned a CI run. There is no way to go looking for it: to prove a test is deterministic, you have to run the suite N times by hand and diff the outputs.Related: #1012 classifies a test that only passed on retry as flaky — that tells you a test was flaky after the fact;
--repeatlets you find out before merging.Proposal
--repeat 50 --filter flaky_candidateis the natural "hammer this one test" invocation.failed on iteration 7 of 50.--parallel(each worker repeats its own test) and with--retry(repeat is the outer loop, retry the inner one) — document the interaction explicitly.--repeat 0is a usage error, not a silent no-op.Where to change
src/runner/exec.sh:312— the retrywhile :;loop is the natural place; repeat wraps it.src/main/test.sh,bashunit::main::require_non_negative_int_or_exit(src/main/validate.sh:19) for validation,src/config/env.sh(BASHUNIT_REPEAT),.env.example.Acceptance criteria
--repeat 3runs each test body three times--repeat 1is byte-identical to no flag--repeat abcand--repeat -1exit non-zero with a usage error (--jobs with a non-integer value hangs on Bash 3.x and is silently ignored on Bash 4.3+ #873 class of bug)--retryis documented and covered by a test--parallelset_up/tear_down) run once per iteration;set_up_before_scriptdoes nottests/acceptance/Repo checklist (agent)
printf -v, no+=append, nodeclare -A, no[[ ]], no${var,,}, no&>>, no${arr[-1]}. Expanding a possibly-empty array underset -uneeds${arr[@]+"${arr[@]}"}.src/main/test.sh(report-style flags needexport -n, seesrc/main/test.sh:188-196for why)bashunit::main::validate_config_or_exit(src/main/validate.sh:60) — unvalidated input used to run the wrong thing and exit 0 (Unknown options are silently ignored: a typo'd flag runs a different suite and exits 0 #871, --jobs with a non-integer value hangs on Bash 3.x and is silently ignored on Bash 4.3+ #873)src/config/env.shand a documented line in.env.example--helptext in the same block it belongs tocompletions/bashunit.bashandcompletions/_bashunit(anti-drift test feat(cli): bash and zsh completion scripts with an anti-drift test #778 fails otherwise)make sa,make lint,./bashunit tests/,./bashunit --parallel tests/. Never runshfmt -w.docs/command-line.md. Editingdocs/assertions.mdinvalidates thebashunit docacceptance snapshot — regenerate it.## Unreleased.tests/acceptance/fixtures/must not end in*test.sh.