Skip to content

fix(runner): a quoted shell-error phrase is no longer a runtime error - #997

Merged
Chemaclass merged 2 commits into
mainfrom
fix/992-runtime-error-classifier
Aug 8, 2026
Merged

fix(runner): a quoted shell-error phrase is no longer a runtime error#997
Chemaclass merged 2 commits into
mainfrom
fix/992-runtime-error-classifier

Conversation

@Chemaclass

Copy link
Copy Markdown
Member

🤔 Background

Related #992

detect_runtime_error decided whether a failing test also hit a shell error by scanning its captured output for phrases like command not found. That capture includes bashunit's own rendering of the failure, so any message quoting one of those strings was misread and the test reported twice — once as Failed, once as Error — for a single cause.

It's reachable without touching the framework. A test whose subject is error handling naturally has these strings in its output: asserting a script prints a useful message, capturing stderr from a failed command. Passing tests were never affected, which is why it stayed invisible until a test started failing — exactly when clear reporting matters.

💡 The discriminator

The prefix bash puts on its own diagnostics:

/tmp/rd2.sh: line 2: /etc/passwd: Permission denied

Checked across command-not-found, unbound variable, permission denied, no such file, syntax error, division by 0, readonly variable and integer-expression-expected, from inside a sourced test file. All carry : line N: . bashunit's own output never does.

Two groups, because they're prefixed for different reasons. Parser and expansion diagnostics require the prefix on the same line as the phrase. killed, segmentation fault and cannot allocate memory come from job control rather than the parser and can arrive without it, so they keep the whole-capture match — and none appears in bashunit's own output, so they carry no false-positive risk.

⚠️ The first attempt was wrong

Applying the prefix rule to everything broke two existing contracts — one pinning "process: killed", one pinning the extracted message's shape. Both were right to fail. The extraction still runs over the whole capture, not the matched line, so the message is byte-identical to before.

🔓 Lifts the #991 workaround

assert_true couldn't say "command not found" without triggering the duplicate. Verified lifted by restoring that wording and confirming no Error appears. The phrasing stays unknown command because it's already documented and released, not because it has to be — the comment in assert/core.sh now says so.

✅ Verification

2 new acceptance tests (the false positive, and a regression guard that a real shell error is still reported), RED first. make sa · make lint · bash build.sh bin -v✅ Build verified ✅ · 1708 sequential / 1667 parallel-simple-strict.

Closes #992.

detect_runtime_error decided whether a failing test also hit a *shell* error by
scanning its captured output for phrases like "command not found". That capture
includes bashunit's own rendering of the failure, so any message quoting one of
those strings was misread and the test reported twice -- once as Failed, once as
Error -- for a single cause.

It is reachable without touching the framework. A test whose subject is error
handling naturally has these strings in its output: asserting that a script
prints a useful message, capturing stderr from a command that failed. When such
a test fails, the phrase is in the capture and the failure is misclassified.
Passing tests were never affected, which is why this stayed invisible until a
test started failing -- exactly when clear reporting matters.

The discriminator is the prefix bash puts on its own diagnostics:

    /tmp/rd2.sh: line 2: /etc/passwd: Permission denied

Checked across command-not-found, unbound variable, permission denied, no such
file, syntax error, division by 0, readonly variable and integer-expression-
expected, from inside a sourced test file. All carry ": line N: ". bashunit's
own output never does.

Two groups, because they are prefixed for different reasons. Parser and
expansion diagnostics require the prefix on the same line as the phrase.
"killed", "segmentation fault" and "cannot allocate memory" come from job
control rather than the parser and can arrive without it, so they keep the old
whole-capture match; none of them appears in bashunit's own output, so they
carry no false-positive risk.

The first attempt applied the prefix rule to everything and broke two existing
contracts -- one pinning "process: killed" and one pinning the extracted
message's shape. Both were right to fail: the extraction still runs over the
whole capture, not the matched line, so the message is byte-identical to before.

This also lifts the constraint #991 worked around: assert_true could not say
"command not found" without triggering the duplicate. Verified by restoring that
wording and confirming no Error appears. The phrasing stays "unknown command"
because it is already documented and released, not because it has to be; the
comment in assert/core.sh says so now.

1708 sequential / 1667 parallel; baseline + 2, RED first.
@Chemaclass Chemaclass added the bug Something isn't working label Aug 8, 2026
@Chemaclass Chemaclass self-assigned this Aug 8, 2026
bash translates its diagnostics and the classifier's phrase list is English
only, so under the Spanish and Japanese CI locales a genuine 'command not
found' is never recognised. That is pre-existing -- the list has always been
English -- and not what this test is about, so it forces the C locale and keeps
testing the classification mechanics. The localization gap is filed separately.
@Chemaclass
Chemaclass merged commit c1d7aa1 into main Aug 8, 2026
37 checks passed
@Chemaclass
Chemaclass deleted the fix/992-runtime-error-classifier branch August 8, 2026 13:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant