Surface var-less errors in JUnit output - #50
Merged
Conversation
Fixture-init throws (:once/:each) and namespace load/compile errors reach the JUnit reporter with no associated test var. clojure.test/eftest still counts them toward the run's error total, and mb.hawk.core derives the exit code from that total, so such an error fails the run -- but the namespace->var keyed writer had nowhere to put an error with no var, so it was dropped from JUnit entirely. The exit code and the JUnit output then disagreed, and any consumer reconstructing the failed set from JUnit (e.g. to compute a narrow rerun selector) would silently miss the error. Collect var-less errors during the run and write them to their own file (mb_hawk_var_less_errors.xml) at :summary. Each is emitted as a <testcase> with a non-empty name but deliberately no classname, so a consumer that attributes failures by resolving classname/name to ns/var treats it as unattributable rather than targeting a nonexistent var. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
bronsa
reviewed
Jul 24, 2026
bronsa
left a comment
bronsa
left a comment
There was a problem hiding this comment.
looks solid to me, just a few implementation notes
| (write-element! | ||
| w "testcase" | ||
| ;; NOTE: intentionally no `classname` -- see `var-less-error-name`. | ||
| {:name (var-less-error-name result)} |
| (str (or (not-empty message) "Uncaught error with no associated test var") | ||
| (when where (format " (%s)" where))))) | ||
|
|
||
| (defn- throwable->string |
There was a problem hiding this comment.
could be (with-out-str (clojure.stacktrace/print-cause-trace t))
| ;;;; We collect them here and emit them into their own file at `:summary`. Each becomes a `<testcase>` with a | ||
| ;;;; non-empty `name` but deliberately NO `classname` -- see `var-less-error-name`. | ||
|
|
||
| (defonce ^:private var-less-errors (atom [])) |
There was a problem hiding this comment.
feels a bit weird to hold error states in this write namespace. I'd move it to the junit instead, and have the write functions take collected errors as args, e.g.
;; in mb.hawk.junit
(defonce ^:private var-less-errors (atom []))
(defmethod handle-event!* :begin-test-run
[_]
(write/clean-output-dir!)
(reset! var-less-errors [])
(write/create-thread-pool!))
(defmethod handle-event!* :summary
[_]
(write/write-var-less-errors! @var-less-errors)
(write/wait-for-writes-to-finish))
Member
Author
There was a problem hiding this comment.
Moved to mb.hawk.junit in b8cf402 — write-var-less-errors! now takes the errors as an arg.
Replace the hand-rolled throwable->string helper with the standard `(with-out-str (clojure.stacktrace/print-cause-trace t))`, which also walks the cause chain. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The name is built from the exception message, which may carry ANSI color codes and unprintable characters; run it through decolorize-and-escape like the other JUnit text. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The write namespace is otherwise stateless XML-writing logic; the atom accumulating var-less errors fits better alongside the event handlers that produce them. Move it to mb.hawk.junit and have write-var-less-errors! take the collected errors as an argument. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
camsaul
approved these changes
Jul 27, 2026
camsaul
left a comment
camsaul
left a comment
Member
There was a problem hiding this comment.
I feel like there's too much duplication here with existing code but after looking at it for a bit I don't have a ton of concrete suggestions about how to minimize it so I'm going to go ahead an approve it for now.
bronsa
approved these changes
Jul 28, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Resolves DEV-2429
Premise
Fixture-init throws (:once/:each) and namespace load/compile errors reach the JUnit reporter with no associated test var. clojure.test/eftest still counts them toward the run's error total, and mb.hawk.core derives the exit code from that total, so such an error fails the run -- but the namespace->var keyed writer had nowhere to put an error with no var, so it was dropped from JUnit entirely. The exit code and the JUnit output then disagreed, and any consumer reconstructing the failed set from JUnit (e.g. to compute a narrow rerun selector) would silently miss the error.
Summary
Collect var-less errors during the run and write them to their own file (mb_hawk_var_less_errors.xml) at :summary. Each is emitted as a with a non-empty name but deliberately no classname, so a consumer that attributes failures by resolving classname/name to ns/var treats it as unattributable rather than targeting a nonexistent var.