Repository navigation
test(jsoniter): a real oracle for the scanner prefix property — 77 → 58 survivors in +5 test lines - #110
Closed
kryptt wants to merge 3 commits into
Closed
test(jsoniter): a real oracle for the scanner prefix property — 77 → 58 survivors in +5 test lines#110kryptt wants to merge 3 commits into
kryptt wants to merge 3 commits into
Conversation
…x property — kills the end-of-input guard cluster
The prefix property drove every skip*/find* guard already; its oracle was
`Try(...).isSuccess`, i.e. "did not throw". Every mutant in the cluster turns
a Miss into a wrong-but-non-throwing Span, so the oracle was blind to all of
them.
Replaced in place with a model-generator + three-part oracle:
(a) ABSOLUTE — the generator returns the rendered children alongside the
document, so each top-level step`s expected span text is known without
a second parse. A purely differential oracle compares two runs of the
SAME mutated code and passes any uniformly-wrong scanner.
(b) BOUNDS — 0 <= start <= end <= len on both `find` and `findAll`.
(c) DIFFERENTIAL — R1/R2/R3 between prefix and full.
Separators became ", " and child counts became frequency-weighted so empty
containers occur in value position and a post-comma-space cut is reachable.
Exception = failure now, so the no-throw guarantee is kept for free.
Subsumed and deleted: section 8`s rootHit/negHit/untermOk clauses, and
literalCases rows 1-6 (valid + truncated literals). Widened the existing
numberGen exponent alternatives with digit-less "e"/"E".
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0194EHFR4NamCpTHiqy7B74V
…can reach — kills 11 mutants in 9 data rows
Each row is a byte layout whose broken syntax is followed by bytes that
accidentally spell the probed member/element, so a dropped guard mis-parses
onto the target instead of being rejected by a later guard:
1"target":2 a step applied to a non-container value (78/85/108/113/118)
{"a":} a value position holding no value at all (240:41)
{Xk0":1} key-quote guard; skipString finds the CLOSING quote (168)
{"k0"1 2} colon guard (173:8)
...nested twins the same two inside a SKIPPED value (254:8, 258:8)
The existing malformed block`s oracle widened from `!find(...).isHit` to also
require `findAll(...) == Nil`, which is what makes the walkAll guards
observable; the widening covers the six pre-existing rows at no extra lines.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0194EHFR4NamCpTHiqy7B74V
…umed, 0 kills lost The new absolute oracle asserts each top-level child`s span text by index on every generated document, which is exactly what these two enumerated sweeps were: a target-position sweep plus absent-key / index-past-end / negative-index Miss rows plus a skip-over-a-sibling assertion. Measured, not assumed: stryker before the deletion 328K/58S, after 328K/58S — zero regressions by mutant key (file, line, column, mutator, replacement). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0194EHFR4NamCpTHiqy7B74V
Contributor
Author
|
Superseded by #111, which cherry-picks this branch's commits unchanged and adds the core artifacts, a consolidation fold and the QA-page equivalent-mutant documentation. Close this one unmerged rather than merging both — #111 is based on the same |
Contributor
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
improve-test-leverage, jsoniter module. Measured with stryker4s, diffed bymutant key
(file, line, column, mutator, replacement).19 kills for 5 net lines (0.26 lines/kill), 0 regressions, 2 fewer tests.
The actual problem
JsonScannerRobustnessSpec's prefix property already drove all 69 ofJsonPathScanner's surviving end-of-input guards — every prefix of a generateddocument, five paths, both surfaces. Its oracle was
Try(JsonPathScanner.find(prefix, p)).isSuccess: "did not throw". Every oneof those mutants turns a Miss into a wrong-but-non-throwing Span, so the oracle
was structurally blind to all of them. This was an oracle bug, not a coverage
gap — which is why the fix is a replacement rather than an addition.
What changed
1. The prefix property, rewritten in place. The generator now returns the
rendered children alongside the document, so the oracle has three parts:
find(full, Nil) == Span(0, n), and each top-level step's spantext equals the known child. Not optional: a purely differential oracle
compares two runs of the same mutated code, so a uniformly-wrong scanner
(
skipObjectreturning -1 for{}on prefix and full alike) passes it.0 <= start <= end <= lenon bothfindandfindAll.the full span fits inside the prefix, the prefix answer is the full answer;
R3
findAllagrees on every span ending strictly inside the prefix.R2 is one-way on purpose. "The span must be a Miss otherwise" is false here —
the scanner is permissive about truncated numbers, so
find("{\"k0\":12", $.k0)returns the Hit
1.Two generator changes carry specific mutants: members join with
", "(thepost-comma space is the only way a cut can leave
skipObjectLoopat a positionwhose whitespace run reaches end-of-input — line 264 advances without a
skipWs, unlike line 181), and child counts are frequency-weighted so{}/[]occur in value position.
Dropping the
Trywrappers means an exception now fails the property directly,so the original no-throw guarantee is kept for free.
2. Nine malformed constants for guards no document generator can reach —
each is a byte layout whose broken syntax is followed by bytes that accidentally
spell the probed member, so a dropped guard mis-parses onto the target instead
of being caught by a later one.
1"target":2(a step applied to a non-containervalue),
{"a":}(a value position holding no value),{Xk0":1}(dropped key-quotecheck makes
skipStringfind the closing quote, so the compared key really isk0),{"k0"1 2}, and the nested twins of the last two. The existing malformedblock's oracle widened from
!find(...).isHitto also requirefindAll(...) == Nil— that widening is what makes thewalkAllguardsobservable, and it covers the six pre-existing rows at no extra lines.
3. Deletions, all measured. The valid/truncated
literalCasesrows, three ofthe four
small examplesclauses, and — verified by a second full stryker run at328K/58S, identical — the entire object-count and array-index sweeps (61
lines, 2 properties). The absolute clause subsumes them exactly.
The 58 remaining survivors are mostly equivalent
51 of
JsonPathScanner's and all ofPathParser's are defence-in-depth guardswhere a later guard in the same function rejects the same input identically
(
51:8,62:8,130:8,159:8,206:8, …), or>=→==on an index thatskipWs/skipValueguarantee never exceeds the bound.JsoniterPrism.scala:130:8is equivalent too: the else arm calls
readFromSubArray(bytes, -1, -1), whoseArrayIndexOutOfBoundsExceptionis caught two lines below into the sameAffine.Miss.JsoniterTraversal56/170/171 all fall through to a general paththat hands a 0- or n-length array to
PSVec.unsafeWrap, which normalises bylength. Chasing these would add lines without adding kill capacity.
One bonus kill the plan had mis-classified as equivalent:
254:13>=→>.With
>the guard is dead-false on the left, sobytes(kpos)is evaluated atkpos == bytes.length— the post-comma-space cut throws.🤖 Generated with Claude Code
https://claude.ai/code/session_0194EHFR4NamCpTHiqy7B74V