From bf745dc29454b937fb01c52b3775e33fe138d2f6 Mon Sep 17 00:00:00 2001 From: Chemaclass Date: Sat, 8 Aug 2026 15:41:10 +0200 Subject: [PATCH] fix(runner): recognise runtime errors from the exit code, not only English text Closes #998. detect_runtime_error matched English phrases -- "command not found" and about twenty more. bash translates its diagnostics, so under any locale with the message catalogue installed none of them matched and a genuine failure-to-run was reported as a plain assertion failure, losing the Error/Failed distinction that exists to tell those apart. The repo runs Spanish, Brazilian and Japanese locale jobs, so this was a supported configuration, not a hypothetical. Exit codes carry the same fact without any text: the shell reserves 127 for "could not find it" and 126 for "found it, could not run it". Both are consulted only after the text scan draws a blank, so English behaviour is unchanged -- including the more specific message the text produces, which is still preferred when available. The gain is wider than the locale case. A test that redirects the diagnostic away still exits 127: function test_error() { set -e invalid_function_name arg1 arg2 &>/dev/null } That is a real fixture in this suite. Its Error message used to be empty, because the text the scan needed had been sent to /dev/null; it now reads "command not found (exit code 127)". Snapshot updated to match, and the diff is worth reading -- it replaces an empty message with the cause. Two structural notes. extract_result_counts now runs before detect_runtime_error so the exit code is available to it; that is safe because it is documented and verified as a pure read that commits nothing. And the line scan moved into its own helper: it used to `return` from the caller on a text miss, which would have skipped the new fallback entirely. 1717 sequential / 1676 parallel; baseline + 4, RED first. --- CHANGELOG.md | 1 + src/runner/diagnostics.sh | 99 ++++++++++++------- src/runner/exec.sh | 8 +- ..._a_test_fail_and_exit_immediately.snapshot | 3 +- tests/unit/runner/diagnostics_test.sh | 34 +++++++ 5 files changed, 107 insertions(+), 38 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index b3ee13c4..201aa803 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -19,6 +19,7 @@ - Internal: Split `src/runner.sh` and `src/coverage.sh` into focused modules with no behavior change; see [ADR-010](adrs/adr-010-src-module-directories.md) (#924, #925) ### Fixed +- Runtime errors are recognised from the exit code when the diagnostic text cannot be matched. bash translates its messages, so under a Spanish or Japanese locale a genuine `command not found` was reported as a plain assertion failure; the same now applies when a test redirects the diagnostic away, where the message used to be empty (#998) - A failing test whose output quotes a shell-error phrase is no longer also reported as a runtime `Error`. The classifier scanned the whole capture -- which includes bashunit's own failure rendering -- for strings like `command not found`, so one cause was reported twice. It now requires the source-and-line prefix bash puts on a real diagnostic (#992) - `assert_have_been_called_times` and `assert_have_been_called_nth_with` report a usage error when their numeric argument is not a number, instead of leaking `[: my_cmd: integer expression expected` from inside bashunit. The common cause is swapping the count and the spy, which the message now names (#984) - `assert_false` no longer passes when the command does not exist. Exit code 127 is non-zero, so a typo in the command name satisfied the assertion while testing nothing; 127 and 126 now fail both `assert_true` and `assert_false`, because they mean the command never ran diff --git a/src/runner/diagnostics.sh b/src/runner/diagnostics.sh index f79a8aab..e25ed311 100644 --- a/src/runner/diagnostics.sh +++ b/src/runner/diagnostics.sh @@ -19,6 +19,50 @@ function bashunit::runner::record_profile() { # rather than exiting, since exiting would only kill the worker. A sequential # run exits with EXIT_CODE_STOP_ON_FAILURE, which main.sh's EXIT trap turns # into the final summary. No-op when the flag is off. +## +# Sets _BASHUNIT_RUNNER_RUNTIME_ERROR_OUT when a line of $1 is a real bash +# diagnostic: one carrying the source-and-line prefix as well as a known +# phrase. Split out so a text miss falls through to the exit-code check rather +# than returning from the caller. +## +function bashunit::runner::_scan_diagnostic_lines() { + local runtime_output=$1 + # A phrase alone is not enough. The capture also carries bashunit's own + # rendering of the failure, and a test whose subject is error handling will + # legitimately quote one of these strings as data -- both used to be misread + # as runtime errors and reported twice, as Failed and as Error, for one cause. + # Requiring the prefix on the same line separates what the shell said from what + # we said about it; bashunit's own output never carries it. + local line + while IFS= read -r line; do + case "$line" in + *": line "[0-9]*": "*) ;; + *) continue ;; + esac + + case "$line" in + *"command not found"* | *"unbound variable"* | *"permission denied"* | \ + *"no such file or directory"* | *"syntax error"* | *"bad substitution"* | \ + *"division by 0"* | *"bad file descriptor"* | \ + *"illegal option"* | *"argument list too long"* | \ + *"readonly variable"* | *"missing keyword"* | \ + *"cannot execute binary file"* | *"invalid arithmetic operator"* | \ + *"ambiguous redirect"* | *"integer expression expected"* | \ + *"too many arguments"* | *"value too great"* | \ + *"not a valid identifier"* | *"unexpected EOF"*) + # Extract from the whole capture, not the matched line: the message shape + # (leading source stripped, newlines removed) is pinned by + # tests/unit/runner/diagnostics_test.sh. + local runtime_error="${runtime_output#*: }" + _BASHUNIT_RUNNER_RUNTIME_ERROR_OUT="${runtime_error//$'\n'/}" + return + ;; + esac + done <