build(deps): hearth 0.4.2 / kindlings 0.3.2 — break the 23-element tuple ceiling on .fields - #102
Merged
Merged
Conversation
kryptt
changed the base branch from
fix/nominal-field-resolution-hazard
to
main
September 18, 2026 11:04
…ling (#96) `WideFieldsArityCeilingSpec` drives `codecPrism[A].fields(...)` with 22, 23 and 40 selectors and NO hand-written `AvroCodec[NamedTuple[...]]` given in scope, so every row goes through kindlings' NamedTuple derivation and hearth's `SyntheticNamedTupleConstructor`. On the current pins (hearth 0.4.0 via kindlings-avro-derivation 0.3.0) the 22 row compiles and passes; the 23 and 40 rows fail to compile with `wrong number of arguments at inlining ... expected: 0, found: 23` / `found: 40` -- hearth's `n < 23` branch calling the `*:` cons chain's zero-value-param primary constructor. That is the failure the hearth 0.4.2 / kindlings 0.3.2 bump is meant to remove. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0194EHFR4NamCpTHiqy7B74V
…ple ceiling `.fields(_.x, _.y, ...)` synthesises a `NamedTuple` whose value tuple has no `TupleN` spelling past arity 22 — it IS a `*:` cons chain. hearth 0.4.0's `SyntheticNamedTupleConstructor.unsafeApply` built every NamedTuple by calling the underlying tuple type's primary constructor, and `*:` declares no value parameters, so every 23+-selector `.fields` died with `wrong number of arguments at inlining … expected: 0, found: N`. hearth 0.4.1 added an arity dispatch (issues #313/#314): `case 0 => EmptyTuple`, `case n if n < 23 => new TupleN(...)`, `case _ => Tuple.fromArray(Array[Object](...))`. 0.4.2 is byte-identical to 0.4.1 for that file (0.4.2 carries a macro-loader metaspace-leak fix and `Expr.semiEval` on `ValueOf`). kindlings 0.3.2 contributes nothing to the arity fix itself — its NamedTuple decoder rule changed only in formatting — it is the release that pins hearth 0.4.2. The `n < 23` boundary lines up exactly with `MacroSelectors.tupleTypeOf`, which spells `TupleN` up to 22 and folds a cons chain above. Both branches stay load-bearing: a cons chain at arity 5 still fails on 0.4.2 (measured: 17 errors, `too many arguments for constructor *:`) while the TupleN spelling at arity 5 compiles clean. Transitive pin decisions, each justified rather than defaulted: - apache-avro 1.12.1 -> 1.12.2. kindlings-avro-derivation 0.3.2 depends on 1.12.2, so leaving the explicit pin at 1.12.1 would have turned a visibility pin into a silent DOWNGRADE of the transitive. The pin moves with it. - jackson-core/-databind 2.21.5 -> 2.22.1. avro 1.12.2's parent POM raises jackson-bom 2.20.0 -> 2.22.1. 2.22.1 is exactly the release that re-fixed CVE-2026-54515 (which 2.22.0 regressed and the old comment was waiting for); it is now on Central. Holding 2.21.5 would have downgraded core/databind while jackson-annotations — not in the override list — moved to 2.22, splitting the BOM. The pin lifts to the version avro resolves, so the override is again a regression floor, not a shift, and never 2.22.0. - commons-lang3 3.18.0 -> 3.20.0, same reasoning: the floor tracks what avro 1.12.2 resolves instead of downgrading it. Still above the CVE-2025-48924 range. commons-compress is unchanged at 1.28.0. `.scala-steward.conf`'s jackson series pin moves 2.21.x -> 2.22.x to match, and every stale comment in the pin block is rewritten (it claimed hearth 0.4.0). The `-Xmacro-settings:{circe,cats,avro}Derivation.timeout=30` line needs no change: `DerivationTimeout` is byte-identical in 0.3.2 and the namespaces are unchanged. kindlings 0.3.1's new derivation-policy gate shares that namespace but defaults to `always-allowed`, i.e. current behaviour, and eo sets no policy key. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0194EHFR4NamCpTHiqy7B74V
…nch is permanent `NamedTupleSpellingSpec`'s last row asserted the FAILURE — `typeCheckErrors` over a 23-selector `.fields` had to contain `too many arguments for constructor *:`. Its own comment said the bump would break it and that the break was the signal, not a regression. It is now the positive assertion it asked for: the same given-free `Wide23` fixture, `.fields` over all 23 selectors, decoded and read back, with the focus's runtime class asserted to be `scala.runtime.TupleXXL` — the `Tuple.fromArray` branch actually executing, not merely compiling. A new row replaces it as the negative: a cons-spelled NamedTuple at arity 2 is STILL unconstructible on hearth 0.4.2, because the fix is scoped to arity >= 23 and the `n < 23` arm is unchanged. That makes `MacroSelectors.tupleTypeOf`'s split spelling a permanent requirement of the hearth contract rather than a workaround the bump retires, and a future "simplify the branch away" pass fails loudly instead of silently re-breaking arity 2..22. `FieldsMacroErrorSpec`'s expected macro hint moves with the version string in `JsonPrismMacro` (next commit). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0194EHFR4NamCpTHiqy7B74V
…tated it PR #96's reply and #97's body documented a "2 to 22 selectors" ceiling to users. Every statement of it is now wrong, so each one moves: - `AvroPrism.fields` scaladoc: no arity ceiling; what a wide selection costs is compile time (superlinear derivation), which is an `-Xss` / `-Xmacro-settings:avroDerivation.timeout` question, not a correctness one. - `MacroSelectors.tupleTypeOf` scaladoc: says explicitly that BOTH branches are load-bearing permanently, since hearth's sub-23 arm still cannot build a cons chain — the one thing a future simplification pass would get wrong. - `site/docs/integrations/avro.md`: the "**2 to 22**" paragraph, plus the stale "kindlings-avro-derivation 0.1.2 / apache-avro 1.12.1" backend note. - `AvroCodec` scaladoc, `site/docs/integrations/circe.md`, `JsonPrismMacro`'s missing-codec hint and `CLAUDE.md`: stale library versions (the hint's expected text is pinned by `FieldsMacroErrorSpec`, updated with it). - CHANGELOG: an Added entry for the broken ceiling and a Changed entry for the bump, including the full transitive shift and the explicit note that NO derived Avro schema text moved. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0194EHFR4NamCpTHiqy7B74V
…rrides protect consumers, state the true `.fields` ceiling
Four audit findings on the hearth/kindlings bump, each verified before fixing.
**The macro-expansion timeout was inert.** The build spelled it
`-Xmacro-settings:{circe,cats,avro}Derivation.timeout=30`, but kindlings'
`DerivationTimeout` parses with
`^\s*(\d+)\s*(ms|millis|milliseconds|s|seconds?|m|minutes?)\s*$` and the call
site is `flatMap(parse).getOrElse(Default)` — a bare integer never matches and
falls through silently, so every derivation had been running on the 5s default,
not 30s. Confirmed in the 0.3.2 bytecode (`Default = 5 SECONDS`) and by canary:
`=1ms` fails with "timed out after 1ms", `=1` compiles clean (so a bare integer
is neither seconds nor millis — it is discarded), `=30s` compiles. All spellings
gain the unit: `=30s` build-wide, `=120s` for mdoc fences. The recurring
"derivation timed out" CI flake this setting was added for was never actually
mitigated until now.
**The jackson/commons-lang3 `dependencyOverrides` never reached a consumer.**
sbt's overrides are build-local and emit no `<dependencyManagement>`; verified
that all four generated POMs (cats-eo, -avro, -circe, -generics) carry zero
`<dependencyManagement>`, zero jackson, zero commons-lang3 entries. Downstream
always resolved jackson through avro's own parent BOM, i.e. jackson-databind
2.20.0 before this bump — 5 OSV advisories, 2 HIGH (checked against OSV) — while
eo's CI compiled against the overridden version and `dependency-submission`
reported that safe one. The override made the exposure invisible, not absent.
So the overrides are removed rather than re-pointed. The jackson half is now a
measured no-op (the avro compile classpath is jar-for-jar identical with and
without it) and was a live hazard besides: only two thirds of the BOM-managed
trio was ever overridden — `jackson-annotations` never was — so the next
jackson-bom lift inside avro-parent would have moved annotations alone while
core/databind stayed frozen, splitting the BOM. Dropping the commons-lang3 half
moves this build 3.20.0 -> 3.18.0, which is exactly what a consumer resolves
(commons-lang3 arrives via commons-compress 1.28.0, whose POM declares 3.18.0;
avro-parent's 3.20.0 property manages only avro's own direct deps). 3.18.0 is
past CVE-2025-48924 and OSV-clean. The build now resolves avro's transitives as
consumers do, so `dependency-submission` reports the real consumer graph.
**The scala-steward pin re-armed the trap it had just sprung.** `version =
"2.22."` is a series lock, not a floor: it blocks 2.23.x forever and matches
2.22.0 anyway, so it does not even express "never 2.22.0". The previous `"2.21."`
pin is the proof — 2.22.1 shipped OSV-clean and required by avro 1.12.2 while the
build sat on 2.21.5 until a human edited it. Steward cannot propose a version
below the current one, so the pin only strands. Replaced with an
`updates.ignore` naming the single bad version.
**`.fields` does have a ceiling; the docs said it did not.** Two of them, and the
second bites first. 254 selectors is hard and permanent — `.fields` selects from
a case class and the JVM caps a parameter list at 254 slots (measured: 254
compiles, 255 fails with "Platform restriction: a parameter list's length cannot
exceed 254", on the case class itself). Well below that the binding limit is the
compile thread's stack, since the derivation recurses per field. Measured on a
full-cover probe varying only -Xss: 1m (the JVM default) derives 32 and overflows
at 36; 2m derives 66, overflows at 100; 4m (sbt's launcher default) derives 150,
overflows at 254; 8m — this repo's .jvmopts, and the only reason its suites reach
254 — derives 254. A downstream build inherits none of that, because the
published artifact cannot carry an -Xss. avro.md, the spec scaladoc, the
`AvroPrism.fields` scaladoc and the CHANGELOG now say so.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0194EHFR4NamCpTHiqy7B74V
kryptt
force-pushed
the
chore/bump-hearth-kindlings-tuple23
branch
from
September 18, 2026 16:19
d6e4eac to
97ecf3f
Compare
Contributor
|
🚀 Cloudflare Pages preview for https://7b83ce65.cats-eo-docs.pages.dev Branch alias: https://chore-bump-hearth-kindlings.cats-eo-docs.pages.dev Built from commit |
Contributor
Benchmark A/BAllocation (B/op) — authoritative
433 more benchmarks
Timing (ns/op) — directional only, same-VM but shared runner
base_sha: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Breaks the 23-element tuple ceiling on
.fields(_.x, _.y, …)by moving thederivation backend: hearth 0.4.0 → 0.4.2, kindlings-{avro,cats,circe}-derivation
0.3.0 → 0.3.2.
No longer stacked. #97 and #105 have both landed on
main(as squash merges75c7eaaand its predecessor), so this branch has been rebased withgit rebase --onto origin/main <old base>and now carries only its own 5 commits /13 changed files — down from the 13 commits / 30 files it showed while the two
ancestors were still open. The RED witness for the ceiling (
WideFieldsArityCeilingSpec,introduced against #97's spelling fix) is the first of those 5; the rest turn it green.
Re-gated on top of
mainafter the rebase, including #107's cached-name-index rewrite inAvroWalk— the arity result is unchanged by it.What moved, and why
.fieldssynthesises aNamedTuplewhose value tuple has noTupleNspellingpast arity 22 — at 23 components it is a
*:cons chain. hearth 0.4.0 builtevery NamedTuple by calling the underlying tuple type's primary constructor, and
*:declares no value parameters, so every 23+-selector.fieldsfailed tocompile:
hearth 0.4.1 added an arity dispatch to
SyntheticNamedTupleConstructor.unsafeApply(upstream issues #313 / #314):
0.4.2 is byte-identical to 0.4.1 for that file (it carries a macro-loader metaspace
leak fix and
Expr.semiEvalonValueOf). kindlings 0.3.2 contributes nothing tothe arity fix itself — its NamedTuple decoder rule changed only in formatting — it
is simply the release that pins hearth 0.4.2.
Ceiling: before / after
Measured, not assumed. Spelling = how the macro writes the NamedTuple's value tuple.
TupleN*:expected: 0, found: Nexpected: 0, found: N— unchanged*:(forced — noTupleNexists)expected: 0, found: 23scala.runtime.TupleXXL*:expected: 0, found: 40*:The spelling ceiling is gone. Two real ceilings remain, and the second bites first —
both are now stated in avro.md, the spec scaladoc,
AvroPrism.fields's scaladoc and theCHANGELOG, where the page previously claimed "no arity ceiling":
254 selectors, hard and permanent.
.fieldsselects from a case class and the JVMcaps a parameter list at 254 slots. Measured: a 254-field case class compiles, a
255-field one fails with
Platform restriction: a parameter list's length cannot exceed 254— raised on the case class itself, before.fieldsis reached.The compile thread's stack, which binds far lower. The derivation recurses per field.
Measured on a full-cover
.fieldsprobe, varying only-Xss:-Xss1m2m4m8m.jvmoptsThis repo's
-Xss8mis the only reason its own suites reach 254. A downstream consumerinherits none of it — the published artifact cannot carry an
-Xss— so on a default1 MB compile thread a wide
.fieldsstarts overflowing between 32 and 36 selectors,barely past the old ceiling. The docs now say to raise
-Xssin the consumer's build.Compile time is the third cost, and the
avroDerivation.timeoutlever for it only startedworking in this PR (see above).
The
TupleNbranch stays — permanentlyhearth's fix is scoped to arity ≥ 23 only. Measured on this exact classpath:
AvroDecoder.derivedtoo many arguments for constructor *: in class *:TupleN-spelled NamedTupleSo #97's "
TupleN≤ 22, cons fold above" split inMacroSelectors.tupleTypeOfis apermanent requirement of the hearth contract, not a workaround this bump retires.
Collapsing it to a uniform cons fold would re-break every
.fieldscall of arity 2–22.Answering the follow-up question directly: do not simplify the branch. A follow-up
worth filing is upstream, not here — ask hearth to extend the
Tuple.fromArraypathbelow 23 (or to dealias before choosing), which would make the branch genuinely
redundant. Until then the split is load-bearing, and the new negative row in
NamedTupleSpellingSpecmakes a future simplification pass fail loudly.Transitive dependency diff
Captured with
show <module>/{Compile,Test}/dependencyClasspathacrossavroIntegration,generics,circeIntegration,tests, diffed against thepre-bump baseline. Identical in every configuration that carries them:
commons-compressis unchanged at 1.28.0, and so iscommons-lang3at 3.18.0 —an earlier revision of this PR showed it moving to 3.20.0, but that was the (now removed)
dependencyOverridestalking, not the bump. Nothing else in any configuration moved.Dependency decisions (each one deliberate, not a default)
Holding the explicit pin at 1.12.1 would have turned a visibility pin into a silent
downgrade of the transitive, contradicting the comment's own stated purpose.
dependencyOverridesare REMOVED, not re-pointed.See "Supply chain" below — they never reached a consumer, the jackson half is now a
measured no-op, and keeping two thirds of a BOM-managed trio pinned was a live trap.
.scala-steward.conf's jackson series pin is removed, replaced by anupdates.ignorenaming the single bad version (2.22.0). Aversion = "2.22."pin is aseries lock: it would block 2.23.x forever, and it matches 2.22.0 anyway so it does not
even express "never 2.22.0". The old
"2.21."pin is the proof of that failure mode —2.22.1 shipped OSV-clean and required by avro 1.12.2 while this build sat on 2.21.5 until
a human edited the pin by hand.
Supply chain: the overrides were protecting nobody
This is the finding with real downstream impact, and it is fixed by the avro bump rather
than by anything eo was doing deliberately.
sbt's
dependencyOverridesis BUILD-LOCAL. It emits no<dependencyManagement>, and eo'spublished POMs carry none — verified on all four:
So a consumer of
cats-eo-avroresolved jackson through avro's own parent BOM, neverthrough eo's override. Before this PR that meant jackson-databind 2.20.0; OSV reports
5 advisories against it, 2 HIGH:
avro 1.12.2's parent POM raises
jackson-bom2.20.0 → 2.22.1, so consumers now land on2.22.1, which OSV reports clean. Meanwhile ci.yml's
dependency-submissionjob submitsthe build's resolved graph — with overrides applied — so Dependabot only ever saw the
safe version and the downstream exposure never raised an alert here.
Why the overrides are removed rather than completed:
avroIntegration/Compile/dependencyClasspathisjar-for-jar identical with and without it (measured both ways on this branch).
(annotations dropped its patch component at 2.20), and
jackson-annotationswas neverin the override list. The next jackson-bom lift inside avro-parent would have moved
annotations alone while core/databind stayed frozen — splitting the BOM across two
minors, the precise hazard the override existed to prevent.
3.18.0, which is exactly what a consumer resolves: commons-lang3 arrives via
avro → commons-compress 1.28.0, whose POM declares 3.18.0 outright, and avro-parent'scommons-lang3.version3.20.0 property manages only avro's OWN direct dependencies.3.18.0 is the first release patched against CVE-2025-48924 and is OSV-clean.
Net effect: the build now resolves avro's transitives exactly as consumers do, so
dependency-submissionreports the real consumer graph instead of a locally-sweetened one,and a future advisory against avro's transitives surfaces here as a genuine alert.
The macro-expansion timeout was inert — now it isn't
-Xmacro-settings:{circe,cats,avro}Derivation.timeout=30did nothing. kindlings'DerivationTimeoutparses the value withand the call site is
flatMap(parse).getOrElse(Default)withDefault = 5 SECONDS(read out of the 0.3.2 class file). A bare integer has no unit, never matches, and is
discarded silently — no warning, no error. Every derivation in this build had been running
on 5s, not 30s, since the 0.3.0 pin.
Canary, same file and JVM, only the flag changing:
Every spelling now carries the unit:
=30sbuild-wide,=120sfor mdoc fences. This iswhat finally mitigates the recurring "derivation timed out" CI flake the setting was added
for. The build comment records the regex and the canary procedure for the next bump.
Derived-schema behaviour: nothing changed
kindlings 0.3.1 shipped two real Avro changes (a rewritten sealed-trait decoder
name-match that drops the
transformConstructorNamescall, andderiveSelfContainedSchemamarking the root type as self-derived). No derived Avroschema text moved: record names, field names, namespaces and field order are
byte-identical across the bump. The full 216-example avro suite — including
AvroUnionSpec,AvroFieldNamingSpec,AvroNominalResolutionSpec,AvroWriteCorrectnessSpec,ConfluentGraftSpec,ResolutionResidualSpec,ResolutionFalsePositiveSpec,AvroVulcanSpecandAvroPrismLawsSpec— passesunmodified; no expected string was touched anywhere. eo's only non-identity config
is a
withSnakeCaseFieldNames(a field-name transform), which the rewritten pathdoes not touch. Same for kindlings 0.3.2's new circe built-in-type rules: the circe
and
testssuites pass unmodified.kindlings 0.3.1's new derivation-policy gate shares eo's
-Xmacro-settingsnamespacebut defaults to
always-allowed(= 0.3.0 behaviour) and eo sets no policy key, so nopolicy key is added here.
DerivationTimeoutis byte-identical in 0.3.2 — which is alsowhy the
timeoutspelling bug above is a pre-existing defect this PR fixes rather thanone the bump introduced.
Gates
Run on JDK 25.0.4.1, sbt 1.12.13, Scala 3.8.4 — so the
kyomodule was in theroot aggregate and was exercised (
KyoOpticsSpecran; the kyo module compiled).scalafmtCheckAllscalafixAll --checkscalafmtSbtCheckbenchmarks/scalafmtCheckcompile(root aggregate)test(root aggregate)NominalResolutionCostSpec's wall-clock rows, disarmed unless-Deo.costGate=true)avroIntegration/testmaingained from #103/#107/#111)benchmarks/compile(outside the aggregate)mimaReportBinaryIssuesdocs/mdoc+docs/laikaSiteMiMa is not a real gate here:
mima.sbtsetsThisBuild / tlMimaPreviousVersions := Set.emptybuild-wide for the 0.x line, so thetask passes vacuously. Worth stating because hearth and kindlings-avro are compile
scope in
cats-eo-generics/cats-eo-avroand therefore appear in those publishedPOMs — downstream consumers inherit the new versions. Both upstream releases declare
MiMa green against their predecessors, and the one signature change near eo's surface
(
ClassViewResult.Incompatiblestopped being acase class) is source- andbinary-compatible by design and is a type eo never pattern-matches.
Test changes
WideFieldsArityCeilingSpec(the RED witness, first commit of this PR) now passes: arity 22over a 25-field record, arity 23 over the same, arity 40 over a 42-field record with
String/Int/Long/Double/Booleaninterleaved. Each row checks written slotsby schema name, un-selected siblings still carrying their original values and
reference-identical, the focus's runtime class, and a named-accessor read-back. The
mixed-type row matters specifically because the
Tuple.fromArraybranch boxes everyelement to
Object, so a homogeneous probe would miss a mis-unboxing.NamedTupleSpellingSpec's inverted ceiling row is flipped to a positive round-trip(still given-free, asserting
scala.runtime.TupleXXL), exactly as its own commentinstructed. A new negative row replaces it: a cons-spelled NamedTuple below 23 is
still unconstructible — the executable proof that the
TupleNbranch is permanent.Nothing left broken
No skipped gate, no expected value adjusted to make something pass, no
@Ignore.🤖 Generated with Claude Code
https://claude.ai/code/session_0194EHFR4NamCpTHiqy7B74V