Skip to content

perf(avro): one cached name index per schema, not a scan per case field (#103) - #107

Merged
kryptt merged 8 commits into
mainfrom
perf/nominal-preindex-103
Sep 18, 2026
Merged

kryptt merged 8 commits into
mainfrom
perf/nominal-preindex-103

Conversation

@kryptt

@kryptt kryptt commented Sep 18, 2026 •

Copy link
Copy Markdown
Contributor

PR #98's nominal rung is correct and stays correct. This PR changes only what it costs.

Closes #103. #105 has landed on main (squash 75c7eaa); this branch is rebased onto main and its diff is its own work only.

The mechanism

AvroWalk.totalNominalIndex calls nominalIndex once per case field. nominalIndex tries Avro's
exact-name hash and, on a miss, linear-scans every schema field calling sameFieldName — with
no early exit, deliberately, because it has to see a second match before it can call a name
ambiguous.

So a drilled hop costs arity x fields.size x nameLength. And the bill lands on exactly the
population the rung exists for: a name transform (withSnakeCaseFieldNames, a custom
transformFieldNames, a vulcan.Codec rename map) guarantees the exact-name miss, while an
identity-named codec answers from the hash and never scans at all — and was already resolving
correctly by position before #98 existed.

Two multipliers on top: the work is redone per drilled hop (a k-hop chain into one record pays
k times) and a def-shaped or parameterised-given optic re-resolves per operation.

The fix. One O(n) pass over the record's fields builds a normalisedName -> position index;
each case field is then one lookup. The index is cached per record schema, so the k-hop chain and
the per-operation re-resolution both pay once.

Two details worth naming:

  • The exact-name hash stays the first rung, untouched and allocation-free. An identity-named
    codec never builds or consults an index, so the control path is bit-for-bit what it was — and its
    allocation count is measured byte-identical. The transformed path pays for the time it saves:
    see It trades bytes for time.
  • Normalisation is per-CHARACTER (Character.toLowerCase), not String.toLowerCase. The latter
    is locale-sensitive and can change a string's length (U+0130 lower-cases to two chars), so it
    would not agree with the pairwise cursor walk it replaces. Per-char keeps the key exactly as
    discriminating as sameFieldName was.
  • Ambiguity detection is preserved and merely moved. A normalised key already present while
    building the index is the same signal the scan produced on its second hit — detected once per
    schema instead of once per query.

Before / after

Correction to an earlier revision of this section. It claimed the cold column improved at
n=2 (3,230 -> 2,690 ns). It does not, and the change's own cost spec says so: building the index
costs more than the scan it replaces when there is almost nothing to scan, so the first
resolution against a one- or two-field record is slower. Everything below is re-measured —
min of 25 trials, both implementations timed in the same JVM (the frozen pre-index rung is kept in
the test tree as NominalResolutionOracle), schema construction outside the clock — and the
regression is stated rather than averaged away.

ns per resolution:

codec shape fields warm before warm after cold before cold after
identity-named 2 34 36 108 107
identity-named 12 194 196 289 256
identity-named 66 2,128 2,122 5,902 4,067
snake_cased 2 490 224 605 757
snake_cased 12 17,271 1,361 18,367 3,963
snake_cased 66 496,939 8,745 500,700 20,466

warm = repeated resolution against one schema (what a def-shaped or parameterised-given optic
pays per operation); cold = first resolution against each of 200 never-before-seen schemas,
index build included.

The cold path regresses on small records

Median of 25 trials, snake_cased, two independent runs of the same harness:

fields cold before cold after delta
1 147 / 148 434 / 395 +287 / +247 ns 2.7-3.0x slower
2 719 / 723 831 / 765 +113 / +41 ns 1.1-1.2x slower
3 1,339 / 1,320 1,126 / 1,169 -213 / -151 ns crossover
4 2,296 / 2,294 1,538 / 1,518 -758 / -776 ns 1.5x faster
6 4,944 / 5,976 2,142 / 2,364 -2.8 / -3.6 us 2.3-2.5x faster

Crossover at n=3. The penalty is bounded and one-time: a few hundred nanoseconds per schema,
paid on the first hop and amortised over every hop after it, against a 24x cold and 57x warm win at
n=66. The trade is obviously right — it is here because the section previously asserted the
opposite.

It trades bytes for time, and only on the transformed path

The sameFieldName cursor walk it replaces was allocation-free by design; normalisedName
materialises a String per case field per resolution. Deterministic counts, via
com.sun.management.ThreadMXBean.getThreadAllocatedBytes, identical across repeat runs:

codec shape fields B/resolution before B/resolution after
identity-named 2 / 12 / 33 / 66 24 / 64 / 152 / 280 24 / 64 / 152 / 280 unchanged, as claimed
snake_cased 2 24 256 +104 B per case field
snake_cased 12 64 1,336 +104 B per case field
snake_cased 33 152 3,608 +104 B per case field
snake_cased 66 280 7,168 +104 B per case field

The identity-named row is the whole of what that path allocates: the out array, 16 B of header
plus 4 B per case field, and nothing else. It is the control, and it did not move.

This is per operation for a def-shaped or parameterised-given optic, not construction-time
only, so it is nursery traffic on the hot path — and "B/op gates, ns/op advises" is this repo's own
benchmark doctrine, so a future gate will see it. ~490 us of CPU for ~8 KB of nursery at n=66 is
the right way round, but it belongs on the record, not in a test comment.

Correction. An earlier revision of this section said the index lookup had been hoisted out of
the per-case-field loop and measured byte-for-byte unchanged. Half of that was wrong, and the
numbers above are the post-hoist ones. The hoist is now in (perf(avro): resolve the name index once per resolution): the index is threaded through the @tailrec loop as a recursion parameter
rather than re-fetched per case field, which removes one ConcurrentHashMap probe and one
throwaway SchemaQuery key per case field beyond the first. Measured saving: exactly
(arity - 1) x 24 B at every size — 24 B at n=2, 1,560 B at n=66 — so the key was not being
scalar-replaced by escape analysis, contrary to what "byte-for-byte unchanged" implied. Net
-17.9% on the transformed path at n=66.

Threaded, not hoisted above the loop: hoisting would make the identity-named codec fetch an index
it never reads, and that path has to stay exactly as it is.

What the hoist does not do is close the regression, and that half was right: the normalised
String is the remaining ~104 B per case field. Closing it properly means probing the index by a
cursor-computed hash of the normalised name and confirming the candidate with the old
allocation-free pairwise comparison — a larger change than #103 asked for, and deliberately not
taken here.

Both costs are now stated in the CHANGELOG and in the scaladoc beside the code that pays them.

The shape changed, which is the actual claim:

ratio before after linear quadratic
t(2n) / t(n), snake_cased 3.97 2.2 ~2 ~4
t(4n) / t(n), snake_cased 15.44 5.1 ~4 ~16
snake_cased / identity-named at n=66 242x 4.0x

Identity-named fast path: unchanged. Warm is flat to within a few nanoseconds at every size and
the allocation count is byte-identical; the code path is provably untouched, since the index is
reached only from the exact-miss branch.

Cache effectiveness. First hop into a schema (cold, n=66 snake) 20,466 ns; second and every hop
after (warm) 8,745 ns, with the index instance identical across hops —
NominalNameIndexCacheSpec pins that by reference (beTheSameAs), so "a k-hop chain rebuilds k
times" is refuted deterministically rather than by timing.

The verdict diff — empty

This is the load-bearing correctness step: a cost rewrite of a resolution rule is only safe if it
is verdict-identical, and "the suite is still green" cannot show that — an example pins one cell,
while the rung's semantics are a function over (schema, case-field list, declaration index).

NominalResolutionOracle is a verbatim copy of #98's implementation, frozen in the test tree.
NominalResolutionParitySpec diffs the live rung against it over two corpora:

=== nominal verdict diff, real fixtures (case field -> schema field, or ABSTAIN) ===
  Three                schema{a,computed,b,c}
    a                was a                now a               same
    b                was b                now b               same
    c                was c                now c               same
  Pair                 schema{beta,alpha}
    alpha            was alpha            now alpha           same
    beta             was beta             now beta            same
  SnakeComputed        schema{click_id,computed,landing_page_id}
    clickId          was click_id         now click_id        same
    landingPageId    was landing_page_id  now landing_page_id same
  ...
  Tw                   schema{userId_,user_id}
    userId           was ABSTAIN          now ABSTAIN         same
    tag              was ABSTAIN          now ABSTAIN         same
  mismatches: 0
=== nominal verdict diff, synthetic: 259 schemas x 585 case-name lists (151515 pairs, every declIdx in [-1..arity]) ===
  mismatches: 0
  • Exhaustive synthetic: every ordered schema-field list of length 1-3 over a pool of
    deliberately confusable names (a, A, _a, a_b, ab, aB, b — exact, case and separator
    collisions), crossed with every case-name list of length 0-3 over that pool plus one unmatched
    name, crossed with every declIdx in [-1..arity]. Duplicated case names included, because a
    duplicate is the sharpest injectivity probe.
  • Real fixtures: every codec behind DivergentCodecs, ResolutionFalsePositiveSpec,
    AvroNominalResolutionSpec and ResolutionResidualSpec — 27 codecs, printed field by field.

Each semantic that was easy to lose is also pinned as its own named example, so a failure says
which rule broke rather than printing a synthetic triple:

semantic existing coverage added
exact name beats a normalised match elsewhere implied by Doc / Acct explicit example
more than one normalised match => ambiguous, never first-wins ResolutionResidualSpec at3 (Tw) explicit example on a 3-field record
arity > record.getFields.size short-circuits FpRow, FpClick explicit example
injectivity across the WHOLE case-field list FpVisit (partial: not-total, not not-injective) added — two case fields normalising onto one schema field, both abstain
empty caseNames (NamedTuple parent) ThreeNt indirectly explicit example
declIdx < 0 or >= arity none added

Two were genuinely uncovered before: whole-list injectivity (the suite had "not total" fixtures
but no "not injective" one) and the out-of-range declIdx guard.

Cache design, and the leak story

A ConcurrentHashMap side table keyed on the schema's reference identity behind weak keys.

  • Identity, not Schema as the key. Schema.equals is deep-structural — a record compares
    every field, recursively — and Schema is mutable: addProp resets its cached hash. A
    structural key would pay a deep compare per lookup and could lose an entry when a schema is
    annotated after insertion. (Verified against the Avro 1.11/1.12 sources: hashCode() memoises
    into a field that addProp invalidates; RecordSchema.equals walks the field list.)
  • Not Avro's own per-schema property storage, which was the obvious slot to check first, as the
    issue asked. Refuted: a prop is part of Schema.equals, of toString, and of the JSON and
    canonical forms — parking a cache there would change what the schema serialises as, and
    therefore its fingerprint. Hence a side table.
  • Weak, not bounded. Schemas are usually module-level and immortal, where the two are
    equivalent; but a registry client resolving writer schemas per message would leak them forever
    under a strong map, or thrash under a bound. Weak keys make the entry's lifetime the schema's own.
    The cached value holds no reference back to the schema — only Strings copied out of the field
    names — so the entry is genuinely collectable, and cleared keys are drained from a
    ReferenceQueue on the next cache miss (never on the hot path). NominalNameIndexCacheSpec
    proves the no-retention claim by resolving through a schema, dropping every strong reference, and
    watching it be collected.
  • Lookups use a plain throwaway query key (not a WeakReference), so a cache hit costs no GC
    registration; the stored key is the weak one, and equals checks reference identity first so a
    cleared key can still be removed by the purge.

Scaladoc

Three additions.

One sentence on the nominal rung's known-residual note, per the investigation that settled it:
neither the record's schema name nor its parsing fingerprint can detect the residual — the
residual pair is two renamed, swapped columns of the same physical type inside one record, so both
candidate readings carry the same fullname and the same fingerprint by construction, and a
schema-derived value only ever sees one of the two operands a comparison would need. The only
mechanism that reaches the class is the differential codec probe, #100. This is there to stop the
idea being re-proposed.

And the pre-index's two costs, stated beside the code that pays them: the ~104 B per case field the
normalised key materialises on a miss, and the small-n cold regression with its crossover at
n=3.

Review pass: three fixes on top of the green PR

Three independent audit lenses read the pre-index. All three findings were real; two were code, one
was a policy call.

1. The cache probe was still per case field

The multiplier #103 exists to remove had survived one level down. totalNominalIndex asked a
per-case-field helper for each name, and every helper call that missed the exact-name hash called
normalisedNameIndex, which probes the cache with a freshly allocated SchemaQuery. On the
transformed path every exact lookup misses by construction, so at arity 66 that is 66 CHM probes
and 66 throwaway keys per resolution, all answering with the same map.

Fixed by threading the resolved index through the @tailrec loop. The exact-name rung is untouched
and still first. Numbers in the allocation section above; the identity path is byte-identical.

2. normalisedNameIndex handed out the live mutable map

private[avro] is a Scala-only fence — the method is public in bytecode — and what it returned was
the live java.util.HashMap sitting in a process-wide, per-schema cache. One put on that
reference silently and permanently corrupts field resolution for that schema for the life of the
JVM: wrong slot, valid wire bytes, no error.

Now wrapped in Collections.unmodifiableMap, with the wrapper stored in the cache so the wrap
is one allocation per schema rather than one per lookup. NominalNameIndexCacheSpec pins instance
identity with beTheSameAs and still passes unchanged against the stored wrapper — no test needed
weakening.

3. Wall-clock assertions gated the whole CI matrix

NominalResolutionCostSpec put three timing-ratio assertions into the always-run root sbt test,
so they gated the temurin@17 / @21 / @25 matrix on shared GitHub runners. That contradicts this
repo's own standing rule — a nanosecond on a shared box is +/-15-50%, only B/op and within-run
ratios are load-bearing, which is why timing claims live in the JMH pipeline. The margins are
healthy, but 2n/n's bound of 3.0 against a measured 2.18 is a 38% excursion from turning the whole
matrix red for a reason unrelated to the diff under test. (Worth noting: the tightest bound is
2n/n, not the snake/identity one with the noisy denominator.)

Chosen: run them under the quality workflow. The three assertions are armed by
-Deo.costGate=true and skip — visibly, with the reason printed — without it. quality.yml arms
them in a job of its own, not a step in the coverage+mutation job: a timing flake must not abort
that sweep and cost the tag its QA artifacts, a runner doing nothing else is the only place a
wall-clock ratio means anything, and a separate job never sees scoverage-instrumented bytecode,
which would make any timing meaningless. It is the one gate in a workflow that is otherwise all
reports, and it is isolated accordingly. Nothing is weakened: same sizes, same bounds, same
interleaving. The table above them still prints on every lane, gates nothing, and costs ~4 s. An
explicit discarded warm-up over all three sizes was added too, since the gate no longer runs right
after the table that used to warm it.

Rejected — widen and keep in the default lane. The widening would have to survive an arbitrarily
contended runner, at which point the assertion stops separating linear from quadratic, and it
leaves a clock on the critical path of every diff in the repo.

Rejected — replace with a non-timing assertion. Preferred, and searched for first; it does not
exist. The deterministic halves of the claim are already pinned (NominalNameIndexCacheSpec pins
one index instance per schema and no rebuild across resolutions; NominalResolutionParitySpec pins
every verdict). What is left is "how much work per case field", which is not observable from
outside the method: Schema's constructors are package-private so it cannot be subclassed to count
field accesses, and an internal probe counter would put mutable state on the hot path this issue
exists to make cheap. Allocated bytes — deterministic here, and this repo's other load-bearing
metric — cannot see the regression that matters either: the quadratic scan allocated less than
the index does (280 B vs 7,168 B at n=66), so a return to it would read as an improvement. The
reasoning is recorded in the spec's scaladoc so it is not re-searched.

Also

One stale comment: the parity spec's injectivity example described a, _a and A, none of which
appear in it. Rewritten to describe the actual fixture — two case fields normalising onto one schema
field, so the map is non-injective and the rung abstains entirely rather than letting the first one
win. The example was and is correct.

Gates

JDK 25 (Gentoo Temurin 25.0.4.1), so the kyo module was in the root aggregate
(KyoOpticsSpec ran).

gate result
sbt compile pass
sbt test (root aggregate, incl. kyo) pass — 878 examples, 0 failures, 2 skipped
sbt avroIntegration/test pass — 228 examples, 226 passed, 2 skipped
sbt benchmarks/compile pass
sbt mimaReportBinaryIssues pass (vacuous: mimaPreviousArtifacts is empty on every module in this branch state)
sbt docs/mdoc; docs/laikaSite pass
sbt scalafmtCheckAll; scalafixAll --check; scalafmtSbtCheck; benchmarks/scalafmtCheck pass, run last

The 2 skipped are the cost spec's ratio examples, disarmed as described above. The example count is
unchanged at 878 — nothing was deleted, the two assertions moved lanes.

The verdict diff was re-run after the resolution-path change and is still empty: 151,515
synthetic cells plus every real fixture, zero mismatches.

Every gate above was re-run after the correction pass (which touched the CHANGELOG, two scaladoc
blocks and two commit messages) and the format gate ran last, clean, with no files rewritten.

One thing worth knowing: scalafmtCheckAll reported success once from a stale cache while three
test files were in fact unformatted. It reproduced on the review pass — a check run straight after a
format run reads the cache that run just wrote and examines almost nothing. The final gate here was
therefore run with every target/streams/*/scalafmt* and */scalafix* cache deleted first, so all
22 source groups (Compile + Test for every module, avro included) were genuinely re-read: clean.
Also worth knowing: scalafixAll and scalafmtAll disagree about a trailing comma in a multi-line
import — scalafix removes what scalafmt accepts — so the two must be run in that order, not one or
the other.

Commits

  1. test(avro) — the RED harness. Fails on the current implementation with 2n/n = 3.97, 4n/n =
    15.44 and 472,474 ns per resolution at 66 snake_cased fields.
  2. perf(avro) — the pre-index and its cache.
  3. docs(avro) — the residual sentence, the cost disclosures (allocation and the small-n cold
    regression), and the CHANGELOG entry.

Review pass, one commit per finding:

  1. perf(avro) — resolve the name index once per resolution, not per case field.
  2. fix(avro) — hand out an unmodifiable view of the cached name index.
  3. docs(avro) — the injectivity comment describes the example below it.
  4. test(avro) — arm the wall-clock cost ratios only where a clock means something.
  5. docs(changelog) — the measured per-field cost, and the unmodifiable index view.

Out of scope on purpose: no alias rung (#104), no new resolution rungs, no refactors the pre-index
did not need.

🤖 Generated with Claude Code

https://claude.ai/code/session_0194EHFR4NamCpTHiqy7B74V

@kryptt
kryptt force-pushed the perf/nominal-preindex-103 branch 2 times, most recently from cf65c60 to 855d48e Compare September 18, 2026 13:07
@kryptt
kryptt added this pull request to stack #112 September 18, 2026 13:29
Base automatically changed from fix/nominal-resolution-onto-main to main September 18, 2026 13:30
kryptt and others added 8 commits September 18, 2026 09:31
…dicts

RED for issue #103. Nothing here changes behaviour; it makes both halves of
the problem executable before the fix touches anything.

`NominalResolutionOracle` is a VERBATIM copy of PR #98's nominal rung. It is
the differential oracle: a cost rewrite of a resolution rule is only safe if
it is verdict-identical, and "the examples still pass" cannot show that — an
example pins one cell, while the rung's semantics are a function over
(schema, case-field list, declaration index).

`NominalResolutionParitySpec` diffs the live rung against that oracle over
259 synthetic schemas x 585 case-name lists (151,515 pairs, every declIdx in
[-1..arity]) built from deliberately confusable names — exact, case and
separator collisions — plus every codec behind the three #98 suites. It also
names each semantic separately, so a failure says WHICH rule broke:
exact-beats-normalised, two-normalised-hits-is-none, the `arity >
fields.size` short-circuit, whole-list injectivity, the empty-caseNames
(NamedTuple) abstention, and the out-of-range declIdx abstention.

`NominalResolutionCostSpec` is the failing half. It asserts nothing in
nanoseconds — this box is noisy and only within-run ratios are load-bearing
— but on the SHAPE: double the field count and a quadratic rung costs ~4x, a
linear one ~2x. Against the current implementation:

  snake_cased n=66  warm  472,474 ns per resolution
  2n/n = 3.97 (quadratic)      4n/n = 15.44 (quadratic)
  n=66 snake / identity = 242x

`totalNominalIndex` widens from `private` to `private[avro]` so the parity
spec can read its verdicts directly rather than inferring them from a
resolved field name.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0194EHFR4NamCpTHiqy7B74V
…ld (#103)

GREEN for issue #103. The nominal rung's doctrine is unchanged — total and
injective or abstain entirely, exact before normalised, two normalised hits
is no hit. Only its cost moves.

`nominalIndex` tried Avro's exact-name hash and, on a miss, linear-scanned
every schema field calling `sameFieldName`, deliberately without an early
exit: it has to see a SECOND match before it can call a name ambiguous. Run
once per case field, that is `arity x fields.size x nameLength` per drilled
hop — and it is charged to exactly the codecs the rung exists for, because a
name transform (`withSnakeCaseFieldNames`, a custom `transformFieldNames`, a
`vulcan.Codec` rename map) GUARANTEES the exact-name miss. An identity-named
codec answers from the hash and never scanned, so the population the rung was
written for was the only one paying.

Now: one O(n) pass over the record's fields builds a normalised-name ->
position index, and each case field is one lookup. Collision detection is
preserved and merely MOVED — a key already present while BUILDING is the same
signal the scan produced on its second hit, seen once instead of per query.
Normalisation is per-CHARACTER (`Character.toLowerCase`), not
`String.toLowerCase`, which is locale-sensitive and can change a string's
length; that keeps the key exactly as discriminating as the cursor walk it
replaces.

The exact-name path is untouched and still allocation-free, so the
identity-named codec neither builds nor consults an index.

Cache: a `ConcurrentHashMap` side table keyed on the schema's REFERENCE
identity behind WEAK keys.
  - identity, not `Schema` itself: `Schema.equals` is deep-structural and
    `Schema` is mutable (`addProp` resets its cached hash), so a structural
    key would pay a deep compare per lookup and could lose an entry under
    annotation;
  - not Avro's own per-schema props, the obvious slot: a prop is part of
    `equals`, of `toString` and of the JSON and canonical forms, so a cache
    parked there would change what the schema SERIALISES AS;
  - weak, not bounded: schemas are usually module-level and immortal, where
    both are equivalent, but a registry client resolving writer schemas per
    message would leak under a strong map and thrash under a bound. The value
    holds no reference back to the schema, so the entry is genuinely
    collectable; `NominalNameIndexCacheSpec` proves it by dropping a schema
    and watching it be collected.

Measured, ns per resolution, MIN of 25 trials with both implementations timed
in the same JVM run (the frozen pre-index rung is kept in the test tree as
`NominalResolutionOracle`); schema construction is outside the clock:

  identity     n= 2   warm       34 ->       36     cold      108 ->      107
  identity     n=12   warm      194 ->      196     cold      289 ->      256
  identity     n=66   warm    2,128 ->    2,122     cold    5,902 ->    4,067
  snake_cased  n= 2   warm      490 ->      224     cold      605 ->      757
  snake_cased  n=12   warm   17,271 ->    1,361     cold   18,367 ->    3,963
  snake_cased  n=66   warm  496,939 ->    8,745     cold  500,700 ->   20,466

  2n/n  3.97 -> 2.19        4n/n  15.44 -> 5.12
  n=66 snake / identity  242x -> 3.9x

warm = repeated resolution against one schema, which is what a `def`-shaped
or parameterised-`given` optic pays per operation. cold = first resolution
against each of 200 never-before-seen schemas, index build included.

The cold column REGRESSES on small records, and says so: building the index
costs more than the scan it replaces when there is almost nothing to scan.

  cold, snake_cased, med of 25 trials (two runs)
  n=1   147 ->  434 / 148 ->  395    +287 / +247 ns   2.9x  SLOWER
  n=2   719 ->  831 / 723 ->  765    +113 /  +41 ns   1.2x  SLOWER
  n=3  1339 -> 1126 / 1320 -> 1169    -213 / -151 ns  0.85x
  n=4  2296 -> 1538 / 2294 -> 1518    -758 / -776 ns  0.67x

Crossover at n=3; the penalty is a few hundred nanoseconds, once per schema,
amortised over every later hop into it.

And it trades bytes for time on the transformed path, which the scan did not:
`normalisedName` materialises a String where `sameFieldName` walked two
cursors, so a miss now allocates ~128 B per case field (deterministic, via
`getThreadAllocatedBytes`):

  identity     n= 2     24 ->     24 B      n=12    64 ->    64 B
  identity     n=66    280 ->    280 B
  snake_cased  n= 2     24 ->    280 B      n=12    64 -> 1,600 B
  snake_cased  n=66    280 ->  8,728 B

Per OPERATION, not per construction, for a `def`-shaped optic. ~490 us of CPU
for ~8 KB of nursery at n=66 is the right way round, but "B/op gates, ns/op
advises" is this repo's doctrine, so it is on the record rather than implied.
Hoisting the index lookup out of the per-case-field loop does not close it:
the normalised String is ~104 B of the 128 and would survive untouched, and
that hoist was built and measured at byte-for-byte unchanged. Closing it
means probing the index by a cursor-computed hash of the normalised name and
confirming the candidate with the allocation-free pairwise comparison — a
larger change than #103 asked for.

The 151,515-cell verdict diff against the frozen implementation is EMPTY.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0194EHFR4NamCpTHiqy7B74V
…probe

Three changes, all documentation.

The nominal rung's known-residual note now says what CANNOT close the
residual, not only that it is open. A 31-agent investigation settled this
today and the idea keeps being re-proposed: neither the record's schema NAME
nor its parsing FINGERPRINT can detect the residual pair (two renamed,
swapped columns of the same physical type within one record), because both
candidate readings carry the same fullname and the same fingerprint by
construction, and a schema-derived value only ever sees ONE of the two
operands a comparison would need. The only mechanism that reaches the class
is the differential codec probe — issue #100.

The pre-index's two COSTS are now on the record, in the CHANGELOG and beside
the code that pays them, because the entry as written read as if nothing
regressed:

  - `normalisedName` materialises a String, so a resolution that misses the
    exact-name hash allocates ~128 B per case field where the pairwise cursor
    walk it replaced allocated nothing: 24 -> 280 B at n=2, 280 -> 8,728 B at
    n=66 per resolution (deterministic, via getThreadAllocatedBytes). The
    identity-named path is byte-identical, as claimed. This is per OPERATION
    for a `def`-shaped optic, so it is nursery traffic on the hot path, not
    construction-time only — and "B/op gates, ns/op advises" is the repo's
    own doctrine, so a future gate will see it.
  - Building the index costs more than the scan it replaces on a one- or
    two-field record, so the FIRST resolution against one is slower. The
    crossover is at n=3.

Both are the right trade — ~490 us of CPU per resolution traded for ~8 KB of
nursery at n=66, and a few hundred nanoseconds once per small schema — but
they belong in the record rather than in a test comment.

Plus the CHANGELOG entry for #103.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0194EHFR4NamCpTHiqy7B74V
…field

The multiplier issue #103 set out to remove survived one level down.
`totalNominalIndex` asked `nominalIndex` once per case field, and every
`nominalIndex` that missed the exact-name hash called `normalisedNameIndex`,
which probes the cache with a freshly allocated `SchemaQuery` key. On the
transformed path — where EVERY exact lookup misses by construction, which is
the whole population the nominal rung exists for — that is `arity` ConcurrentHashMap
probes and `arity` throwaway keys per resolution, all of them answering with the
same map. At arity 66: 66 probes, 66 keys, one map.

The index is now threaded through the `@tailrec` loop as a recursion parameter:
`null` until some case field needs it, the resolved map afterwards. Threaded and
not hoisted above the loop, because hoisting would make the identity-named codec
build or fetch an index it never reads — that path must stay untouched, and it is:
the exact-name hash rung is still FIRST, still `record.getField(scalaName)`, still
allocation-free.

`nominalIndex` disappears as a separate method (it was the per-field wrapper), its
two rungs inlined into the loop where the index can be carried across iterations.
Verdicts are unchanged by construction — same rungs, same order, same `Ambiguous`
demotion — and `NominalResolutionParitySpec` re-confirms it: 151,515 synthetic
(schema, case-name list, declIdx) cells plus every real #98 fixture, zero
mismatches against the frozen pre-index oracle.

Measured with `com.sun.management.ThreadMXBean.getThreadAllocatedBytes`, warm,
bytes per resolution (the harness the PR's disclosed figures came from):

  shape        n=2    n=12    n=33    n=66     per case field
  identity      24      64     152     280     4 B  (the `out` array, unchanged)
  snake before 280   1,600   4,376   8,728     132 B
  snake after  256   1,336   3,608   7,168     108 B

The saving is exactly `(arity - 1) x 24 B` — one `SchemaQuery` per case field
beyond the first — confirming the key was NOT being scalar-replaced by escape
analysis. Net: -17.9% on the transformed path at n=66. What remains is the
normalised `String` itself, ~104 B per case field, which is the disclosed trade
and needs the cursor-hash probe to close. The identity-named path is
byte-identical before and after.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0194EHFR4NamCpTHiqy7B74V
`normalisedNameIndex` returned the live `java.util.HashMap` it had just built
and cached. `private[avro]` is a Scala-only fence — the method is PUBLIC in
bytecode — and the map it returns is a process-wide singleton per schema behind
a `ConcurrentHashMap`, so a single in-module (or reflective) `put` on the
returned reference silently and permanently corrupts nominal field resolution
for that schema for the life of the JVM. Every later resolution would read the
corrupted entry as if the index had been built that way: wrong slot, valid wire
bytes, no error. Nothing in the type or the name warned against it.

The map is now wrapped in `Collections.unmodifiableMap`, and the WRAPPER — not
the backing map — is what goes into the cache, so the wrap is paid once per
schema at build time rather than per lookup, and repeated lookups keep handing
back the same instance.

That instance stability is what `NominalNameIndexCacheSpec` pins with
`beTheSameAs`, and it still holds: the spec's "built once per schema and handed
back by reference" and "repeated resolution never rebuilds the index" examples
pass unchanged against the stored wrapper, as does the ambiguity example, which
reads a value out of it. No test needed weakening.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0194EHFR4NamCpTHiqy7B74V
The comment on `NominalResolutionParitySpec`'s injectivity example talked about
`a`, `_a` and `A` — an earlier fixture. None of those names appear in the
example: it builds `record{a_b, b}` and asks for case fields `ab` and `aB`.

Rewritten to say what that fixture actually shows: both case fields normalise to
the key `ab`, which names the single schema field `a_b`, so the map is
non-injective and the rung abstains for EVERY field rather than letting the first
case field keep slot 0. Comment only — the example was and is correct.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0194EHFR4NamCpTHiqy7B74V
…omething

`NominalResolutionCostSpec` put three timing-ratio assertions — 2n/n < 3.0,
4n/n < 10.0, snake/identity < 15.0 — into the always-run root `sbt test`, which
means they gate the temurin@17 / @21 / @25 matrix on every push and every PR,
on shared GitHub runners. That contradicts the project's own standing rule: a
nanosecond on a shared box is +/-15-50%, only B/op and within-run ratios are
load-bearing, and timing claims belong in the JMH bench pipeline. The margins
are healthy (2n/n measures 2.09-2.24, 4n/n 4.61-5.35, snake/identity 3.30-5.08)
but 3.0 against 2.18 is a 38% excursion from turning the entire matrix red for a
reason unrelated to the diff under test.

The three assertions are now armed by `-Deo.costGate=true` and skip — visibly,
with the reason printed — without it. `quality.yml` arms them, on release tags
and on demand, in a job of its OWN rather than a step in the coverage+mutation
job: a timing flake must not abort that sweep and cost the tag its QA artifacts,
a runner doing nothing else is the only place a wall-clock ratio means anything,
and a separate job never sees scoverage-instrumented bytecode, which would make
any timing meaningless. Nothing is weakened: same sizes, same bounds, same
interleaving, run where a clock means something. It is the one gate in a
workflow that is otherwise all reports, and it is isolated accordingly.

The table above them still prints on every lane. It gates nothing, costs ~4 s,
and is how a human notices drift between release-tag runs.

Also added, since the gate now runs in a fresh JVM rather than after the table:
an explicit discarded warm-up pass over all three sizes before the first timed
round. The spec is `sequential`, so the table example did warm those paths, but
that was an ordering coincidence, not a contract.

Why not the two alternatives:

  - WIDEN the bounds and keep them in the default lane. The widening would have
    to be large enough to survive an arbitrarily contended runner, at which point
    the assertion stops separating linear from quadratic — and it leaves a clock
    on the critical path of every diff in the repo. Note also that the tightest
    bound is 2n/n at 3.0, not snake/identity at 15.0; widening only the one with
    the noisy denominator would have addressed the smaller half of the problem.

  - REPLACE with a non-timing assertion. Preferred, and not available. The
    deterministic halves of the claim are already pinned: `NominalNameIndexCacheSpec`
    pins one index instance per schema and no rebuild across repeated resolutions,
    `NominalResolutionParitySpec` pins every verdict. What is left is "how much
    work per case field", which is not observable from outside the method:
    `Schema`'s constructors are package-private so it cannot be subclassed to
    count field accesses, and an internal probe counter would put mutable state on
    the hot path this issue exists to make cheap. Allocated bytes — the repo's
    other load-bearing metric, and deterministic here — cannot see the regression
    that matters either: the quadratic scan allocated LESS than the index does
    (280 B vs 7,168 B at n=66), so a return to it would read as an improvement.

The reasoning is recorded in the spec's own scaladoc, so the next reader does not
re-run the search.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0194EHFR4NamCpTHiqy7B74V
…dex view

The #103 entry quoted ~128 B per case field and 8,728 B at n=66. Hoisting the
cache probe out of the per-case-field loop removed one `SchemaQuery` per field
beyond the first — exactly `(arity - 1) x 24 B`, so the key was not being
scalar-replaced — and the figures are now ~104 B per case field, 24 -> 256 B at
n=2 and 280 -> 7,168 B at n=66. Also states that the identity-named path is
byte-identical before and after, which is the invariant the hoist had to keep,
and adds the unmodifiable-view entry.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0194EHFR4NamCpTHiqy7B74V
@kryptt
kryptt force-pushed the perf/nominal-preindex-103 branch from 855d48e to d9e4dda Compare September 18, 2026 13:31
@github-actions

Copy link
Copy Markdown
Contributor

🚀 Cloudflare Pages preview for perf/nominal-preindex-103 is live:

https://573af685.cats-eo-docs.pages.dev

Branch alias: https://perf-nominal-preindex-103.cats-eo-docs.pages.dev

Built from commit d9e4ddaebc15dca435c5f63840bbdcf03f3ce4d2 · updated on every push.

@github-actions

Copy link
Copy Markdown
Contributor

Benchmark A/B

Allocation (B/op) — authoritative

Benchmark params base head Δ
OrderAvroBench.eoModifyNames size=8 2,160.0 1,992.0 -7.8%
AvroJsonBridgeBench.eoWideToJson - 1,472.0 1,408.0 -4.3%
AvroVulcanBench.encode_bridged - 1,224.0 1,240.0 +1.3%
OrderAvroBench.monocleModifyNames size=8 5,448.0 5,400.0 -0.9%
AvroBytesBench.eoModifyPartner - 3,272.0 3,256.0 -0.5%
AvroJsonBridgeBench.eoClickToAvro - 9,400.0 9,368.0 -0.3%
OrderAvroBench.naiveModifyStreet size=64 20,856.2 20,880.2 +0.1%
OrderAvroBench.naiveModifyNames size=64 27,912.2 27,936.3 +0.1%
OrderAvroBench.monocleReadStreet size=512 69,797.9 69,765.6 -0.0%
OrderAvroBench.monocleModifyStreet size=512 169,072.9 169,084.6 +0.0%
OrderAvroBench.eoModifyNames size=512 97,408.0 97,405.4 -0.0%
OrderAvroBench.naiveModifyNames size=512 226,288.4 226,287.3 -0.0%
OrderAvroBench.naiveReadStreet size=512 69,797.8 69,797.5 -0.0%
OrderAvroBench.monocleModifyNames size=64 39,866.8 39,866.7 -0.0%
OrderAvroBench.naiveModifyStreet size=512 169,060.9 169,061.0 +0.0%
OrderAvroBench.monocleModifyNames size=512 382,757.3 382,757.5 +0.0%
OrderAvroBench.eoModifyStreet size=512 328.0 328.0 -0.0%
OrderAvroBench.monocleReadStreet size=64 8,848.1 8,848.1 -0.0%
OrderAvroBench.eoReadStreet size=512 88.0 88.0 +0.0%
AvroJsonBridgeBench.naiveWideToJson - 4,376.0 4,376.0 -0.0%
38 more benchmarks
Benchmark params base head Δ
AvroBytesBench.prunedReadCountry - 1,976.0 1,976.0 -0.0%
AvroBytesBench.naivePassthroughPayload - 10,600.1 10,600.1 +0.0%
AvroVulcanBench.fieldGet_bridged - 432.0 432.0 +0.0%
OrderAvroBench.eoModifyStreet size=64 328.0 328.0 -0.0%
AvroBytesBench.naiveReadCountry - 4,256.0 4,256.0 -0.0%
AvroBytesBench.prunedReadPartner - 1,592.0 1,592.0 +0.0%
OrderAvroBench.monocleModifyStreet size=64 20,904.2 20,904.2 +0.0%
OrderAvroBench.eoModifyNames size=64 12,560.1 12,560.1 -0.0%
AvroVulcanBench.fieldGet_native - 432.0 432.0 -0.0%
OrderAvroBench.monocleReadStreet size=8 1,208.0 1,208.0 -0.0%
OrderAvroBench.eoReadStreet size=8 88.0 88.0 -0.0%
AvroJsonBridgeBench.eoClickToJson - 4,000.0 4,000.0 -0.0%
AvroJsonBridgeBench.naiveClickToJson - 4,696.0 4,696.0 -0.0%
OrderAvroBench.naiveModifyNames size=8 3,752.0 3,752.0 -0.0%
AvroBytesBench.naiveModifyCountry - 7,616.0 7,616.0 +0.0%
AvroBytesBench.naiveReadPartner - 4,264.0 4,264.0 -0.0%
OrderAvroBench.naiveReadStreet size=8 1,208.0 1,208.0 -0.0%
OrderAvroBench.eoModifyStreet size=8 328.0 328.0 -0.0%
AvroBytesBench.eoReadPartner - 480.0 480.0 -0.0%
OrderAvroBench.naiveReadStreet size=64 8,848.1 8,848.1 -0.0%
AvroJsonBridgeBench.naiveWideToAvro - 3,504.0 3,504.0 -0.0%
AvroVulcanBench.rootGet_bridged - 1,472.0 1,472.0 -0.0%
AvroJsonBridgeBench.naiveClickToAvro - 3,928.0 3,928.0 +0.0%
OrderAvroBench.monocleModifyStreet size=8 2,992.0 2,992.0 -0.0%
OrderAvroBench.eoReadStreet size=64 88.0 88.0 -0.0%
AvroBytesBench.naiveModifyPartner - 7,536.0 7,536.0 +0.0%
AvroVulcanBench.decode_bridged - 880.0 880.0 -0.0%
AvroVulcanBench.decode_native - 48.0 48.0 +0.0%
AvroVulcanBench.decode_vulcanRaw - 880.0 880.0 -0.0%
AvroBytesBench.eoReadCountry - 520.0 520.0 +0.0%
AvroVulcanBench.encode_native - 56.0 56.0 +0.0%
AvroBytesBench.eoGraftPayload - 720.0 720.0 +0.0%
AvroJsonBridgeBench.eoWideToAvro - 6,584.0 6,584.0 +0.0%
AvroBytesBench.eoModifyCountry - 3,200.0 3,200.0 -0.0%
OrderAvroBench.naiveModifyStreet size=8 2,968.0 2,968.0 +0.0%
AvroVulcanBench.encode_vulcanRaw - 1,224.0 1,224.0 -0.0%
AvroVulcanBench.rootGet_native - 600.0 600.0 -0.0%
AvroBytesBench.eoSliceGraftPayload - 1,192.0 1,192.0 +0.0%
Timing (ns/op) — directional only, same-VM but shared runner
Benchmark params base head Δ
OrderAvroBench.monocleReadStreet size=512 29,098.8 22,448.2 -22.9%
OrderAvroBench.naiveModifyStreet size=64 5,555.1 6,351.6 +14.3%
OrderAvroBench.naiveModifyNames size=64 7,026.4 7,787.6 +10.8%
AvroVulcanBench.fieldGet_bridged - 84.6 91.9 +8.6%
AvroVulcanBench.fieldGet_native - 90.2 84.5 -6.3%
OrderAvroBench.eoModifyNames size=512 33,502.8 31,437.6 -6.2%
OrderAvroBench.monocleModifyNames size=8 1,422.6 1,335.9 -6.1%
AvroJsonBridgeBench.naiveWideToJson - 1,667.0 1,584.8 -4.9%
OrderAvroBench.eoModifyNames size=8 562.5 539.2 -4.1%
OrderAvroBench.naiveModifyNames size=8 1,043.5 1,018.4 -2.4%
AvroBytesBench.naivePassthroughPayload - 3,942.5 4,021.3 +2.0%
AvroBytesBench.naiveReadCountry - 1,458.4 1,429.8 -2.0%
AvroVulcanBench.encode_bridged - 208.4 212.5 +2.0%
OrderAvroBench.monocleReadStreet size=8 449.5 440.9 -1.9%
AvroJsonBridgeBench.naiveClickToJson - 2,536.9 2,490.1 -1.8%
AvroBytesBench.prunedReadCountry - 822.2 807.1 -1.8%
OrderAvroBench.monocleReadStreet size=64 3,708.9 3,647.4 -1.7%
AvroVulcanBench.decode_bridged - 187.1 184.0 -1.7%
OrderAvroBench.naiveReadStreet size=8 448.7 441.5 -1.6%
AvroJsonBridgeBench.eoClickToJson - 2,608.0 2,567.9 -1.5%
AvroBytesBench.prunedReadPartner - 584.2 593.1 +1.5%
AvroBytesBench.eoSliceGraftPayload - 268.7 272.8 +1.5%
OrderAvroBench.eoModifyStreet size=8 119.2 117.5 -1.5%
AvroVulcanBench.rootGet_bridged - 357.5 352.5 -1.4%
AvroJsonBridgeBench.naiveWideToAvro - 1,152.4 1,136.6 -1.4%
AvroVulcanBench.decode_vulcanRaw - 181.5 179.3 -1.2%
OrderAvroBench.eoModifyStreet size=64 119.0 117.6 -1.2%
AvroBytesBench.naiveModifyCountry - 2,554.3 2,584.4 +1.2%
OrderAvroBench.naiveReadStreet size=512 28,952.6 28,612.6 -1.2%
AvroBytesBench.eoModifyCountry - 592.9 585.9 -1.2%
AvroBytesBench.eoReadPartner - 178.1 176.2 -1.1%
OrderAvroBench.naiveModifyNames size=512 60,119.2 60,781.4 +1.1%
OrderAvroBench.monocleModifyNames size=512 92,317.6 93,306.0 +1.1%
OrderAvroBench.monocleModifyStreet size=8 881.9 872.7 -1.0%
AvroBytesBench.naiveReadPartner - 1,635.4 1,618.9 -1.0%
OrderAvroBench.monocleModifyStreet size=64 6,332.3 6,389.8 +0.9%
OrderAvroBench.eoModifyNames size=64 3,844.7 3,812.5 -0.8%
AvroBytesBench.eoModifyPartner - 641.7 636.4 -0.8%
AvroJsonBridgeBench.naiveClickToAvro - 1,535.1 1,547.3 +0.8%
AvroVulcanBench.decode_native - 15.0 15.1 +0.7%
AvroVulcanBench.encode_native - 12.2 12.3 +0.7%
OrderAvroBench.naiveModifyStreet size=512 48,825.6 49,101.7 +0.6%
AvroJsonBridgeBench.eoWideToAvro - 1,343.1 1,349.9 +0.5%
AvroJsonBridgeBench.eoClickToAvro - 3,374.4 3,391.2 +0.5%
AvroVulcanBench.rootGet_native - 143.2 142.5 -0.5%
AvroVulcanBench.encode_vulcanRaw - 210.1 209.2 -0.4%
OrderAvroBench.monocleModifyStreet size=512 48,669.7 48,885.1 +0.4%
OrderAvroBench.monocleModifyNames size=64 9,796.5 9,753.8 -0.4%
AvroJsonBridgeBench.eoWideToJson - 560.1 557.9 -0.4%
OrderAvroBench.eoModifyStreet size=512 118.3 117.8 -0.4%
AvroBytesBench.eoReadCountry - 150.1 150.7 +0.4%
AvroBytesBench.naiveModifyPartner - 2,771.8 2,780.3 +0.3%
OrderAvroBench.naiveReadStreet size=64 3,662.9 3,652.4 -0.3%
OrderAvroBench.eoReadStreet size=8 36.1 36.0 -0.2%
OrderAvroBench.eoReadStreet size=64 36.1 36.0 -0.2%
OrderAvroBench.eoReadStreet size=512 36.1 36.1 +0.1%
AvroBytesBench.eoGraftPayload - 130.9 131.0 +0.1%
OrderAvroBench.naiveModifyStreet size=8 873.8 873.9 +0.0%

base_sha: 75c7eaa712a23c7503b51858dcdfc61347e600c8 · head_sha: d9e4ddaebc15dca435c5f63840bbdcf03f3ce4d2 · jdk: temurin-21 · runner: ubuntu-22.04 · jmh_params: -i 3 -wi 2 -f 1 -t 1 -foe true -prof gc -rf json · profile: pr:-i3-wi2-f1-t1-gc

@kryptt
kryptt merged commit 9fd3e21 into main Sep 18, 2026
16 checks passed
kryptt added a commit that referenced this pull request Sep 18, 2026
…Walk

#107 landed while this branch was open and rewrote the code the spec targets:
`sameFieldName` and the fuzzy per-field scan are gone, replaced by
`normalisedName` plus a weak-keyed `normalisedNameIndex` cache. The spec itself
needed no change — it was written against the public seam `AvroWalk.fieldNameAt`
precisely so it would survive an internals rewrite, and it agrees with the new
implementation cell for cell — but its `covers:` block named deleted symbols and
pre-#107 line numbers.

Re-aims it at the surviving constructs by name and drops the line numbers, and
states the division of labour with #107's own `NominalResolutionOracle`: that
one is the OLD ALGORITHM frozen verbatim (a refactor gate — "the new rung agrees
with the one it replaced"), this one is written from the DOCTRINE without
reading the implementation. Parity cannot detect a wrong doctrine, which is the
#104 failure mode; the doctrine oracle still passing against wholly rewritten
internals is the evidence that #107 preserved the contract and not just the code
path.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0194EHFR4NamCpTHiqy7B74V
kryptt added a commit that referenced this pull request Sep 18, 2026
…led, 14 fewer examples (#114)

* test(avro): nominal-resolution doctrine oracle — kills 12 mutants in 68 lines

PR #98 added an all-or-nothing nominal rung to `AvroWalk.fieldNameAt`: resolve
`.field(_.x)` by NAME when every case field maps to a distinct schema field,
otherwise abstain to position. Four clauses; only TOTALITY was pinned.
Injectivity, exact-beats-normalised, and ambiguity-implies-abstain had no
fixture that could tell a correct implementation from a broken one — the sole
ambiguity fixture (`ResolutionResidualSpec` "at3") passes for the wrong reason,
because an unrelated unmapped field forces totality-driven abstention first and
masks the ambiguity branch entirely.

That is not cosmetic. Issue #104 recorded an experimental alias rung silently
breaking a case the rung gets right while all 212 avro tests stayed green: a
MIS-TARGETED optic satisfies get-put, put-get, put-put and modify fusion
perfectly, so no optic law family can see this.

`AvroNominalDoctrineSpec` states the doctrine as a declarative oracle and checks
`fieldNameAt` against it EXHAUSTIVELY over ~66k cells: every injective field
list of length 1..3 from a 6-name pool (each name both a legal Avro field name
and a legal Scala identifier), crossed with every case-name list of length 0..3
WITH repetition, crossed with declIdx from -1 to arity+1. The oracle is written
from the doctrine, not copied from the implementation — parity against a frozen
copy of the code cannot detect a wrong doctrine, which is precisely the #104
failure mode.

Enumeration, not `Gen`: the discriminating cells are threshold cells a default
generator would essentially never produce — a duplicate resolution landing on
slot 0, a fuzzy ambiguity whose FIRST hit is at index 0 versus at index >= 1,
and `declIdx` exactly equal to the schema's field count.

Verified by applying each mutant to `AvroWalk.scala` and confirming the property
goes red: 291 (exact fast path), 301 / 304 (the injectivity scan), 169 / 171 /
173 (ambiguity => abstain), 213 (`sameFieldName`'s char inequality), 310
(`declIdx >= fields.size` at the exact boundary), and 242 / 244 / 245 / 249 (the
four disjuncts of the all-or-nothing precondition).

Deletes the subsumed `"fieldNameAt: non-record parent and out-of-range declIdx
both throw loudly"` example: it probed `declIdx = 99` against a 2-field schema,
a cell where `>=` and a `>` mutant agree, so it pinned neither boundary.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0194EHFR4NamCpTHiqy7B74V

* test(avro): count-preserving JSON structure faults — kills 7 mutants in 13 lines

`AvroJsoniter.readRecord`'s strict exact-cover parse has three independent
refusals — unknown key, duplicate key, and the trailing arity check — plus
end-or-comma refusals in the array, map and byte-array readers. The module had
no fixture for any of the delimiter faults, and its one duplicate-key arm was
MASKED: appending `"i":42` makes 14 keys against 13 schema fields, so the arity
check refuses the document first and the `seen` scan is never exercised. Same
masking shape as the ambiguity fixture this branch's first commit replaced.

Every new arm is deliberately COUNT-PRESERVING:

  - the duplicate REPLACES a sibling key (`"l":...` becomes a second `"i":42`),
    leaving 13 keys with `l` unset, so `seen` is the only thing that can refuse;
  - the four delimiter faults close a nested record / array / map / bytes array
    with the wrong bracket, which the outer reader then resumes past, so the
    document parses end-to-end unless the inner reader refuses.

Kills 54 (`seen(field.pos) = true`), 57 and 59 (the `(field eq null) ||
seen(...)` guard and its operator), 71 (the record's objectEndOrCommaError), and
39 / 55 / 62 (the array / map / byte-array end-or-comma refusals).

Drops the now-dominated 14-key duplicate arm. The triage's proposed "unknown
JSON key" arm is NOT added: under both surviving guard mutants an unknown key
dereferences a null `field`, the NPE is caught by `parseSlice`'s `NonFatal`
handler, and the result is byte-identical to the real refusal — an equivalent
path, not a gap.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0194EHFR4NamCpTHiqy7B74V

* test(avro): decoded-map keys and union diagnostic payloads — kills 6 mutants in 46 lines

Three gaps in the in-memory walker, all of the same species: the existing block
asserts the SHAPE of the answer and never its content.

Map keys. The map-walk example used a hand-built `LinkedHashMap[String, _]`, so
`asMap.get(name: String)` hits directly and the `direct == null` Utf8 retry is
dead code under test. A decoded map is keyed by `Utf8` — the shape production
code actually sees — and only the retry finds anything in it. The new arm puts
the record through the binary codec and probes every key of a 6-name pool
against map sizes 0 / 1 / 2 / 5, so present and absent probes both occur at
every size. Kills 322 (the Utf8 retry) and 338 (the PathMissing refusal, whose
removal returns `Right(null)`).

Null union branch. No fixture walked `UnionBranch("null")` into a null payload,
the one branch name for which a null focus is correct rather than a failure.
Kills 191.

Union diagnostics. `UnionResolutionFailed` carries the declared alternative
list, and every existing assertion matched it with a wildcard. Recovering that
list walks back to the nearest Field step, so it needs a union two records deep
to separate the parents cursor's `pIdx >= 0` guard from a `pIdx == 0` one (kills
183), and a path whose recovered Field step names an ARRAY rather than a union
to pin the empty-list degradation — asking a non-union for its alternatives
throws an AvroRuntimeException straight out of the walk (kills 254 and 256).

Folded into the existing per-shape composite blocks per this file's stated
consolidation convention: no new registrations.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0194EHFR4NamCpTHiqy7B74V

* test(avro): byte-cursor refusal identities and the strictness policy — kills 11 mutants in 83 lines

Every guard in `AvroBinaryCursor`'s path walk, when removed, lets the cursor run
on into the Avro runtime, which throws, which `locateFrom` catches and reports
as `BinaryParseFailed`. A `Left` either way — so the module's `isLeft`
assertions pin none of them, and the structured diagnostic the `.record` face
publishes degrades silently to "parse failed". These rows assert WHICH
`AvroFailure` comes back, never that one does.

Called at the `AvroBinaryCursor` seam (`private[avro]`, same package) rather
than through a prism: several rows need a path the drilling macros refuse to
build, and the strictness flag is not reachable from the public surface at all.

Kills 592 (Field step against a non-record), 500 and 597 (UnionBranch step
against a non-union, and the diagnostic's own non-union guard), 601 and 605
(the ordinal scan running off the end of the alternative list), 534 (the
`requested < 0` refusal under the LENIENT policy — the strict path refuses for a
different reason and masks it), 575 (locateElements' non-array terminal) and 506
(locateElements resolving its prefix strictly).

Two rows carry their own fixtures:

  - 542, the terminal-vs-interior union split. An interior union step must not
    anchor the returned span. Every downstream observable still round-trips
    under the mutant, because the mis-anchored span still opens on a decodable
    value; only `span.valueSchema` gives it away.
  - 440, `AvroPrism`'s byte-face read resolving its terminal union strictly. The
    existing union fixtures all mismatch INTO a decode error, which `=== None`
    cannot tell apart from a correct refusal. New `Twin` fixture: two branches
    with identical field shapes and different names, so `TwinA(100)` and
    `TwinB(100)` encode to the same bytes and a lenient read hands back a
    well-formed value of the wrong type. The lenient policy is correct for the
    graft/write path only.

The triage listed 440's witness as a null-branch payload with a trailing long
field. That does not discriminate: `decodeSlice` reads a LENGTH-BOUNDED slice,
and a null branch skips zero bytes, so the mutant hits EOF and Misses exactly
like the real path. Replaced with the `Twin` fixture above.

Also pins AvroTraversal 359 — `.each`'s prefix must resolve to an array or throw
loudly. The public macro rejects a non-array focus at compile time, so the guard
is only reachable at the internal seam, which is what a future drilling entry
point would use.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0194EHFR4NamCpTHiqy7B74V

* test(avro): array framing survives a write — kills 4 mutants in 32 lines

The traversal's write path has three guards nobody could see: the no-op guard
(`active.isEmpty || arity mismatch`), the canonical-splice vs re-frame split,
and the initial canonical flag the block walk threads. Mutating any of them
still produces a payload that DECODES to the right value, which is why the
Optional-law property block at the foot of this file — and every other
write-side assertion in the module — leaves all four alive. What changes is the
FRAMING: a payload that arrives blocked and leaves canonical, or arrives
multi-block and leaves single-block, is a rewrite of bytes the caller never
asked us to touch.

Two cells, both asserted at byte level, and neither is reachable from the
existing fixtures:

  - all-miss under NON-canonical framing (every element on the other union
    branch). The write must be the identity on the bytes. On canonical framing
    the mutants re-frame to something byte-identical, so only the blocked
    fixture separates them. Kills 364 and 361.
  - canonical MULTI-BLOCK framing plus a length-preserving edit. Spec-legal, and
    the only shape where an in-place splice and a whole-region re-frame differ:
    collapsing two positive-count blocks into one costs a count byte, so the
    payload shrinks while decoding identically. Every other fixture here is
    single-block, where a re-frame reproduces the input exactly. Kills 356 and
    586. Hand-assembled, since neither avro encoder emits it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0194EHFR4NamCpTHiqy7B74V

* test(avro): fold 27 examples into 9 — 18 fewer, 0 kills lost

The mandatory consolidation phase. Seven folds, chosen by MEASURED redundancy
rather than by reading: stryker4s records the killing test in `statusReason`,
so the report itself says which examples carry kills. Of avro's 210 registered
tests only 65 are ever the credited killer of a mutant; every example folded
below is in the other 145, which makes zero kill regression a property of the
report and not a hope — the credited killer of each of the 431 killed mutants
survives every fold untouched. It was then measured anyway.

  1. AvroCodecDecoderReuseSpec — SEVEN properties differing only in the
     (generator, writer schema, reader schema) triple and in which decode entry
     point they called, collapsed to ONE. The triple is now data
     (`reuseScenarios`), and each scenario is put through BOTH entry points —
     the thread-local cache and the `threadLocalStorage = false` opt-out —
     where before each was checked by one example each. Strictly more coverage
     per shape.
  2. AvroFieldNamingSpec — nine examples to three. The four byte-face examples
     were one fixture on one code path (fixture guard, `.field` read, `.field`
     modify, `selectDynamic`); record face and nested descent were a second
     pair. The two `.fieldNamed` examples are duplicates of
     `AvroNominalResolutionSpec`'s, which asserts strictly more of the refusal
     message, so they go. `.fields` stays its own example: it is the one shape
     in the module where the FOCUS codec's names diverge from the parent's.
  3. ConfluentReaderSpec — the two unframed-refusal examples (`reader` /
     `recordReader`) become one over a shared predicate, keeping both payload
     shapes; the MOVED-field and PROMOTED-field examples become one over a
     shared `readFramed` rig.
  4. AvroWriteCorrectnessSpec — the three `.each.union[Branch]` examples
     (blocked framing, canonical framing, record face) were one fixture and one
     optic handed to three carriers: now one example, every assertion kept.
  5. AvroWriteCorrectnessSpec — the two `.each.fields` examples likewise
     differed only in how the same basket was framed.
  6. AvroJsonSpec — the enum/bytes/fixed RENDER example and the enum/bytes/fixed
     ROUND-TRIP example built two near-identical three-leaf schemas. One schema,
     one value, and the render half now asserts the WHOLE document
     (`avroToJson(record) === json`) instead of three field lookups — stronger
     than what it replaced.

MEASURED, not assumed. `project avroIntegration; stryker` re-run and diffed
against the pre-consolidation report by the full (file, line, column, mutator,
replacement) key: 431 Killed / 38 Survived / 17 NoCoverage / 6 CompileError /
122 Ignored BEFORE, and exactly the same after, with zero keys added, zero
removed, zero Killed -> anything. Mutation score unchanged at 88.68% total /
91.90% of covered. Ran twice during the phase (after folds 1-3 and after 4-6);
both diffs clean. Against this run's original baseline the whole branch is
+40 kills with zero regressions.

No assertion was weakened and no fixture dropped: every fold keeps every
`must`/`===` it merged, and three of them add one.

avro suite: 177 -> 159 `>>` registrations, 5,201 -> 5,122 test LOC. Against the
run's starting point (173 / 4,952) that is 14 FEWER examples for +40 mutant
kills; the +170 lines are the net of 242 lines of new high-leverage tests minus
79 removed here, i.e. 4.25 net lines per additional kill.

Gates on JDK 25: `avroIntegration/test` Passed 198, Failed 0, Errors 0;
`scalafixAll; scalafmtAll` then `scalafmtCheckAll; scalafixAll --check` clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0194EHFR4NamCpTHiqy7B74V

* docs(avro): re-aim the doctrine spec's covers-block at post-#107 AvroWalk

#107 landed while this branch was open and rewrote the code the spec targets:
`sameFieldName` and the fuzzy per-field scan are gone, replaced by
`normalisedName` plus a weak-keyed `normalisedNameIndex` cache. The spec itself
needed no change — it was written against the public seam `AvroWalk.fieldNameAt`
precisely so it would survive an internals rewrite, and it agrees with the new
implementation cell for cell — but its `covers:` block named deleted symbols and
pre-#107 line numbers.

Re-aims it at the surviving constructs by name and drops the line numbers, and
states the division of labour with #107's own `NominalResolutionOracle`: that
one is the OLD ALGORITHM frozen verbatim (a refactor gate — "the new rung agrees
with the one it replaced"), this one is written from the DOCTRINE without
reading the implementation. Parity cannot detect a wrong doctrine, which is the
#104 failure mode; the doctrine oracle still passing against wholly rewritten
internals is the evidence that #107 preserved the contract and not just the code
path.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0194EHFR4NamCpTHiqy7B74V

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant