feat(cli): --order-by to run the last run's failures first - #1039
Merged
Conversation
--rerun-failed replays the recorded failures and drops everything else, which is the wrong tool for CI and for the final check before pushing. There you want the whole suite, just with the known-bad tests first, so --stop-on-failure trips in seconds instead of minutes. --order-by defects reads the same .bashunit/last-failed cache but reorders instead of narrowing: recorded files lead the file list and recorded functions lead their file, both in the order the cache holds them, and everything else follows unchanged. With no cache the order falls back to defined. defined names the current behaviour so it can be set explicitly, and random is the existing --random-order under the new flag. Both spellings stay live because either can arrive through the environment rather than the parser, so is_random_order_enabled accepts each. Closes #1011
8 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🤔 Background
Related #1011
--rerun-failedreplays the recorded failures and drops everything else. That is the right tool while fixing one failure, and the wrong one for CI or the final check before pushing, where you want the full suite with the known-bad tests first.💡 Changes
--order-by <mode>picks the execution order:defined(default),defectsorrandom.defectsreads the same.bashunit/last-failedcache as--rerun-failedbut reorders instead of narrowing, so the whole suite still runs. Paired with--stop-on-failureit fails in seconds instead of minutes.defined. An unknown mode exits non-zero listing the valid ones.--random-orderand--seedkeep working unchanged;--order-by randomis the same mode under the new name.