fix: set ok=False when TestAbortAll is raised in run_suite_class#1006
fix: set ok=False when TestAbortAll is raised in run_suite_class#1006devimano2011 wants to merge 1 commit intogoogle:masterfrom
Conversation
Set ok to False when TestAbortAll is caught to indicate failure.
|
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). View this failed invocation of the CLA check for more information. For the most up to date status, view the checks section at the bottom of the pull request. |
|
In our usage, abort is usually intentional, so we don't necessarily consider this "not ok". So I'm not sure whether this should be considered "not ok". |
|
Thank you for the feedback! That's a valid point — I wasn't aware that abort is often intentional in your usage (e.g. device doesn't support the feature under test). Looking at PR #997 more carefully, that fix was motivated by cases where Given your clarification, I see a few options:
|
Related to #950 and #960
Problem
In
run_suite_class()insuite_runner.py, whensignals.TestAbortAllis raised, theexceptblock only haspassand does not setok = False. This means if a test aborts before any failures, the suite exits with code 0 instead of 1.This is the same issue that was fixed in
test_runner.pyby PR #997 for issue #960, butrun_suite_class()insuite_runner.pywas missed.Fix
Change
passtook = Falsein theexcept signals.TestAbortAllblock so that aborting a test suite always results in a non-zero exit code.