Skip to content

test(circe): differential index-bounds oracle — 6 mutants killed, 10 fewer test lines - #108

Closed
kryptt wants to merge 1 commit into
fix/nominal-resolution-onto-mainfrom
test/kill-survivors-circe
Closed

kryptt wants to merge 1 commit into
fix/nominal-resolution-onto-mainfrom
test/kill-survivors-circe

Conversation

@kryptt

@kryptt kryptt commented Sep 18, 2026

Copy link
Copy Markdown
Contributor

Merge order: this must merge AFTER #105 (fix/nominal-resolution-onto-main), which is its base. If #105 lands first, rebase this onto main before merging.

What

One artifact, produced by the improve-test-leverage skill (phase 4 Execute + phase 5 Eval) against the freshly measured circe survivor set.

JsonWalk carries two copies of the array bounds check idx < 0 || idx >= arr.length — one in readPath (JsonWalk.scala:56), one in modifyPath (:81). JsonIndexBoundsSpec was written for the :56 copy (its comment says so) but exercised only .at(i).modify(identity)(...), which routes through modifyPath. Result: :81 was 11/11 killed while all six operator mutants on :56 survived.

The fix is an oracle replacement, not an addition. The hard-coded example becomes one forAll(Gen.chooseNum(-2, 5)) property over a fixed size-3 array, whose expected values are derived from the backing Vector rather than written down, and which drives three surfaces per index:

surface call reaches
read p.get(basket) JsonWalk.readPath:56 — the uncovered copy
Ior write p.modify(identity)(basket) JsonWalk.modifyPath:81 — preserves today's kills
silent write p.modifyUnsafe(identity)(basket) the .getOrElse(json) pass-through

Why the range is -2 .. 5 and the array size is fixed

The five discriminating classes against a size-3 array are i<0, i=0, 0<i<3, i=3 (exactly length), i>3 (strictly past length). Each matters to a specific mutant:

  • i = 0 alone kills both < variants (<→<=, <→==) — they turn a legitimate Hit at index 0 into a Miss. The pre-existing read coverage in JsonPrismSpec reads at index 1, where 1 <= 0 and 1 < 0 agree.
  • i = 3 exactly kills >=→>; i >= 4 does not (4 > 3 fires correctly).
  • i >= 4 only kills >=→==; i = 3 does not (3 == 3 fires correctly).

A generated array size was considered and rejected: size 0 discriminates nothing extra (at size 0 with i=0, <= and == agree with the original), while a second generator dilutes each critical (size, i) cell to ~1/24 per trial and makes the i=3 / i=4 kills flaky. With a fixed size, each class lands with p = 1/8 per trial, so over ScalaCheck's default 100 trials the miss probability for any one class is < 2e-6.

Deletions (part of the artifact, not a favour)

  • JsonIndexBoundsSpec — the "indices 0 and 1 read their elements; -1 and length fail IndexOutOfRange" example, replaced in place. Its three assertions are a strict subset of the property's index classes.
  • JsonPrismSpec — basket1 / json1 plus the unsafeOOR, defaultOOR and negIndex assertions and their covers: lines. defaultOOR was already redundant with JsonFailureSpec.scala:66-71, which asserts the same Ior.Both and the same exact IndexOutOfRange(PathStep.Index(5), 1) on the same modify surface. unsafeOOR / negIndex are subsumed by the property's silent conjunct, which asserts pass-through across every out-of-range class rather than two constants.

Measured (not predicted)

sbt circeIntegration/test — 42 passed, 0 failed. Stryker re-run diffed against the baseline by the full mutant key (file, line, column, mutator, replacement):

baseline after
Killed 38 44
Survived 12 6
Ignored 36 36
score 76.00% 88.00%

Exactly the six predicted mutants flipped Survived → Killed, with no regressions and no new survivors:

JsonWalk.scala:56:20 ConditionalExpression → false
JsonWalk.scala:56:24 EqualityOperator      → <=
JsonWalk.scala:56:24 EqualityOperator      → ==
JsonWalk.scala:56:28 LogicalOperator       → &&
JsonWalk.scala:56:35 EqualityOperator      → ==
JsonWalk.scala:56:35 EqualityOperator      → >

Two distinct failure modes are exercised, both legitimate kills: a value mismatch for the :24 pair, and an escaping IndexOutOfBoundsException from Vector.apply for :20 / :28 / :35 — sound specifically because readPath has no try/catch, unlike modifyPath. That asymmetry is exactly why :56 survived while :81 died.

Objective terms:

  • net_test_lines_added = −10 (scalafmt-arbitrated, comments excluded)
  • suite_test_count delta = 0 (one example replaced by one property; the JsonPrismSpec removals are assertions inside an existing block, not whole tests)
  • suite_kill_density rises on both terms: +6 detected mutants over 10 fewer test LOC

circe is now at its killable ceiling

The 6 remaining survivors are provably equivalent mutants and should be dropped from future sweep action lists:

  • JsonWalk.scala:42:12 and :67:12 — i >= n → i == n on the readPath / modifyPath loop heads. Both loops are entered at 0 and the only recursive calls are i + 1 from inside the i < path.length arm, so i can never exceed path.length. This is verbatim the equivalent pattern the skill's triage table lists.
  • JsonFocus.scala:77:10, :100:10, :114:10, :135:10 — path.length == 0 → false on the four empty-path shortcuts in Leaf. The general branch with an empty path computes the identical value in every case (readPath returns Right(json) at i = 0; modifyPath applies the terminal function at i = 0 and cannot miss), differing only by one stack frame and one closure allocation.

Gates

  • sbt circeIntegration/test ✅
  • sbt "scalafmtCheckAll; scalafixAll --check" ✅

🤖 Generated with Claude Code

https://claude.ai/code/session_0194EHFR4NamCpTHiqy7B74V

…0 lines

`JsonWalk` carries two copies of `idx < 0 || idx >= arr.length`: one in
`readPath` (:56), one in `modifyPath` (:81). `JsonIndexBoundsSpec` was written
for :56 but drove only `.modify(...)`, so all six operator mutants on the read
copy survived while the write copy was 11/11 killed.

Replace the hard-coded example with one `forAll(Gen.chooseNum(-2, 5))` property
over a fixed size-3 array whose expectations are DERIVED from the backing
`Vector`, driving three surfaces per index: `get` (read → :56), `modify` (Ior
write → :81) and `modifyUnsafe` (silent pass-through). The range straddles all
five discriminating classes — i<0, i=0, 0<i<3, i=3, i>3 — which is what the
two `>=` variants need (i=3 alone kills `>=`→`>`, i>=4 alone kills `>=`→`==`).

Deletions: the subsumed `unsafeOOR` / `defaultOOR` / `negIndex` assertions in
JsonPrismSpec. `defaultOOR` was already redundant with JsonFailureSpec:66-71;
the other two are subsumed by the property's `silent` conjunct, which asserts
pass-through across every out-of-range class rather than two constants.

Measured: circe 38K/12S → 44K/6S (76.00% → 88.00%), net -10 test lines, suite
test count unchanged. The remaining 6 survivors are provably equivalent mutants
(`i >= n` → `i == n` on two `+ 1`-incremented loop heads, and four empty-path
shortcuts whose general branch computes the same value), so circe is now at its
killable ceiling.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0194EHFR4NamCpTHiqy7B74V
@kryptt

kryptt commented Sep 18, 2026

Copy link
Copy Markdown
Contributor Author

Superseded by #111, which cherry-picks this branch's commits unchanged and adds the core artifacts, a consolidation fold and the QA-page equivalent-mutant documentation. Close this one unmerged rather than merging both — #111 is based on the same fix/nominal-resolution-onto-main and must merge after #105.

@kryptt

kryptt commented Sep 18, 2026

Copy link
Copy Markdown
Contributor Author

Superseded by #111, which carries these commits unchanged and is now rebased onto main. Closing unmerged: this PR's base fix/nominal-resolution-onto-main was squash-merged as #105 (75c7eaa) and is no longer an ancestor of main, so merging here would land nothing on main.

@kryptt kryptt closed this Sep 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant