Skip to content

Fix compositions returning wrong results when fast_fail is off - #214

Open
gaoflow wants to merge 1 commit into
horejsek:masterfrom
gaoflow:fix-fast-fail-composition
Open

Fix compositions returning wrong results when fast_fail is off#214
gaoflow wants to merge 1 commit into
horejsek:masterfrom
gaoflow:fix-fast-fail-composition

Conversation

@gaoflow

@gaoflow gaoflow commented Jul 26, 2026

Copy link
Copy Markdown
Contributor

fast_fail=False is documented as a way to collect all the errors, so it must not change
whether data is valid. It does:

schema = {'anyOf': [{'type': 'integer'}, {'type': 'string'}]}
fastjsonschema.compile(schema)('abc')                   # ok
fastjsonschema.compile(schema, fast_fail=False)('abc')  # JsonSchemaValuesException: data must be integer

fastjsonschema.compile({'not': {'type': 'integer'}}, fast_fail=False)('abc')
fastjsonschema.compile({'contains': {'type': 'number'}}, fast_fail=False)(['abc', 1])

Every keyword that only tries out a subschema decides on a raised
JsonSchemaValueException, and in collecting mode nothing raises: the except branch is dead
and the errors of a rejected attempt leak into the result. That hits anyOf, oneOf (every
branch counts as a match, so the count is never 1), not (the else always fires), if (its
else is unreachable), contains and propertyNames. A subschema behind a $ref is a second
case: it stays its own function and reports failure with JsonSchemaValuesException, which
except JsonSchemaValueException does not catch, so it escapes the whole validator.

I ran the bundled JSON-Schema-Test-Suite through both modes for draft-04/06/07/2019-09:
171 of 5293 cases got a different verdict with fast_fail=False, 146 of them where the
default mode is right. The generated-code path is the same, 110 divergences on the composition
files. After this change both are 0, and 0 across the
fast_fail x detailed_exceptions x use_default matrix as well.

Two halves, both in code generation: subschemas that are only tried out are generated in
raising mode (trial_validation()) so the try/except works as before, and those except
clauses also catch JsonSchemaValuesException. The second part is needed even inside a trial
block, because the exception type of a $refed function follows the top-level fast_fail, not
the local one ({'anyOf': [{'not': {'$ref': ...}}]}). While there, a plain $ref call in
collecting mode now merges the referenced function's errors into the caller's list instead of
aborting the rest of the document - {'a': {'$ref': ...}, 'b': {'type': 'string'}} used to
report only the errors of a.

Generated code for the default fast_fail=True only changes in that wider except tuple; no
measurable difference in an anyOf/not hot loop. Existing suite unchanged (4408 passed / 449
xfailed / 936 xpassed), pylint stays at 9.91.

Tests in tests/test_fast_fail.py: a verdict-parity matrix over the six keywords (valid and
invalid data, nested compositions, $refed variants), error-list assertions for a rejected
attempt and for a genuinely failing composition, error collection through $ref, and the same
check on compile_to_code output. 16 of them fail on master.

anyOf, oneOf, not, if/then/else, contains and propertyNames only try their
subschemas out and decide on the raised JsonSchemaValueException. With
fast_fail=False nothing is raised, so the except branch is dead and the errors
of a rejected attempt leak into the result - valid data was rejected.

Subschemas of these keywords are now generated in raising mode, and the except
also catches JsonSchemaValuesException, which is what a subschema behind a $ref
raises from its own function. That $ref call is now merged into the caller's
errors as well instead of aborting the rest of the validation.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant