Skip to content

test(jsoniter): a real oracle for the scanner prefix property — 77 → 58 survivors in +5 test lines - #110

Closed
kryptt wants to merge 3 commits into
fix/nominal-resolution-onto-mainfrom
test/kill-survivors-jsoniter
Closed

kryptt wants to merge 3 commits into
fix/nominal-resolution-onto-mainfrom
test/kill-survivors-jsoniter

Conversation

@kryptt

@kryptt kryptt commented Sep 18, 2026

Copy link
Copy Markdown
Contributor

Merge order: this must merge AFTER #105 (fix/nominal-resolution-onto-main), which is its base. If #105 lands first, rebase this onto main.

improve-test-leverage, jsoniter module. Measured with stryker4s, diffed by
mutant key (file, line, column, mutator, replacement).

baseline after
Killed 309 328
Survived 77 58
Timeout 4 4
Ignored 30 30
spec blocks 8 6
net test lines — +5

19 kills for 5 net lines (0.26 lines/kill), 0 regressions, 2 fewer tests.

The actual problem

JsonScannerRobustnessSpec's prefix property already drove all 69 of
JsonPathScanner's surviving end-of-input guards — every prefix of a generated
document, five paths, both surfaces. Its oracle was
Try(JsonPathScanner.find(prefix, p)).isSuccess: "did not throw". Every one
of those mutants turns a Miss into a wrong-but-non-throwing Span, so the oracle
was structurally blind to all of them. This was an oracle bug, not a coverage
gap — which is why the fix is a replacement rather than an addition.

What changed

1. The prefix property, rewritten in place. The generator now returns the
rendered children alongside the document, so the oracle has three parts:

  • absolute — find(full, Nil) == Span(0, n), and each top-level step's span
    text equals the known child. Not optional: a purely differential oracle
    compares two runs of the same mutated code, so a uniformly-wrong scanner
    (skipObject returning -1 for {} on prefix and full alike) passes it.
  • bounds — 0 <= start <= end <= len on both find and findAll.
  • differential — R1 a prefix hit is a truncation of the full hit; R2 when
    the full span fits inside the prefix, the prefix answer is the full answer;
    R3 findAll agrees on every span ending strictly inside the prefix.

R2 is one-way on purpose. "The span must be a Miss otherwise" is false here —
the scanner is permissive about truncated numbers, so find("{\"k0\":12", $.k0)
returns the Hit 1.

Two generator changes carry specific mutants: members join with ", " (the
post-comma space is the only way a cut can leave skipObjectLoop at a position
whose whitespace run reaches end-of-input — line 264 advances without a
skipWs, unlike line 181), and child counts are frequency-weighted so {}/[]
occur in value position.

Dropping the Try wrappers means an exception now fails the property directly,
so the original no-throw guarantee is kept for free.

2. Nine malformed constants for guards no document generator can reach —
each is a byte layout whose broken syntax is followed by bytes that accidentally
spell the probed member, so a dropped guard mis-parses onto the target instead
of being caught by a later one. 1"target":2 (a step applied to a non-container
value), {"a":} (a value position holding no value), {Xk0":1} (dropped key-quote
check makes skipString find the closing quote, so the compared key really is
k0), {"k0"1 2}, and the nested twins of the last two. The existing malformed
block's oracle widened from !find(...).isHit to also require
findAll(...) == Nil — that widening is what makes the walkAll guards
observable, and it covers the six pre-existing rows at no extra lines.

3. Deletions, all measured. The valid/truncated literalCases rows, three of
the four small examples clauses, and — verified by a second full stryker run at
328K/58S, identical — the entire object-count and array-index sweeps (61
lines, 2 properties). The absolute clause subsumes them exactly.

The 58 remaining survivors are mostly equivalent

51 of JsonPathScanner's and all of PathParser's are defence-in-depth guards
where a later guard in the same function rejects the same input identically
(51:8, 62:8, 130:8, 159:8, 206:8, …), or >= → == on an index that
skipWs/skipValue guarantee never exceeds the bound. JsoniterPrism.scala:130:8
is equivalent too: the else arm calls readFromSubArray(bytes, -1, -1), whose
ArrayIndexOutOfBoundsException is caught two lines below into the same
Affine.Miss. JsoniterTraversal 56/170/171 all fall through to a general path
that hands a 0- or n-length array to PSVec.unsafeWrap, which normalises by
length. Chasing these would add lines without adding kill capacity.

One bonus kill the plan had mis-classified as equivalent: 254:13 >= → >.
With > the guard is dead-false on the left, so bytes(kpos) is evaluated at
kpos == bytes.length — the post-comma-space cut throws.

🤖 Generated with Claude Code

https://claude.ai/code/session_0194EHFR4NamCpTHiqy7B74V

kryptt and others added 3 commits September 18, 2026 14:52
…x property — kills the end-of-input guard cluster

The prefix property drove every skip*/find* guard already; its oracle was
`Try(...).isSuccess`, i.e. "did not throw". Every mutant in the cluster turns
a Miss into a wrong-but-non-throwing Span, so the oracle was blind to all of
them.

Replaced in place with a model-generator + three-part oracle:
  (a) ABSOLUTE — the generator returns the rendered children alongside the
      document, so each top-level step`s expected span text is known without
      a second parse. A purely differential oracle compares two runs of the
      SAME mutated code and passes any uniformly-wrong scanner.
  (b) BOUNDS — 0 <= start <= end <= len on both `find` and `findAll`.
  (c) DIFFERENTIAL — R1/R2/R3 between prefix and full.

Separators became ", " and child counts became frequency-weighted so empty
containers occur in value position and a post-comma-space cut is reachable.
Exception = failure now, so the no-throw guarantee is kept for free.

Subsumed and deleted: section 8`s rootHit/negHit/untermOk clauses, and
literalCases rows 1-6 (valid + truncated literals). Widened the existing
numberGen exponent alternatives with digit-less "e"/"E".

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0194EHFR4NamCpTHiqy7B74V
…can reach — kills 11 mutants in 9 data rows

Each row is a byte layout whose broken syntax is followed by bytes that
accidentally spell the probed member/element, so a dropped guard mis-parses
onto the target instead of being rejected by a later guard:

  1"target":2         a step applied to a non-container value (78/85/108/113/118)
  {"a":}              a value position holding no value at all (240:41)
  {Xk0":1}            key-quote guard; skipString finds the CLOSING quote (168)
  {"k0"1 2}           colon guard (173:8)
  ...nested twins     the same two inside a SKIPPED value (254:8, 258:8)

The existing malformed block`s oracle widened from `!find(...).isHit` to also
require `findAll(...) == Nil`, which is what makes the walkAll guards
observable; the widening covers the six pre-existing rows at no extra lines.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0194EHFR4NamCpTHiqy7B74V
…umed, 0 kills lost

The new absolute oracle asserts each top-level child`s span text by index on
every generated document, which is exactly what these two enumerated sweeps
were: a target-position sweep plus absent-key / index-past-end / negative-index
Miss rows plus a skip-over-a-sibling assertion.

Measured, not assumed: stryker before the deletion 328K/58S, after 328K/58S —
zero regressions by mutant key (file, line, column, mutator, replacement).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0194EHFR4NamCpTHiqy7B74V
@kryptt

kryptt commented Sep 18, 2026

Copy link
Copy Markdown
Contributor Author

Superseded by #111, which cherry-picks this branch's commits unchanged and adds the core artifacts, a consolidation fold and the QA-page equivalent-mutant documentation. Close this one unmerged rather than merging both — #111 is based on the same fix/nominal-resolution-onto-main and must merge after #105.

@kryptt

kryptt commented Sep 18, 2026

Copy link
Copy Markdown
Contributor Author

Superseded by #111, which carries these commits unchanged and is now rebased onto main. Closing unmerged: this PR's base fix/nominal-resolution-onto-main was squash-merged as #105 (75c7eaa) and is no longer an ancestor of main, so merging here would land nothing on main.

@kryptt kryptt closed this Sep 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant