From 916f5e25bf8cb9d13bc48e6b7988e7ec26989ff7 Mon Sep 17 00:00:00 2001 From: Rodolfo Hansen Date: Tue, 22 Sep 2026 16:41:07 +0200 Subject: [PATCH 1/6] =?UTF-8?q?feat(avro):=20derived=20whole-record=20buil?= =?UTF-8?q?der=20=E2=80=94=20recordBuilder[A],=20recursive=20sub-record=20?= =?UTF-8?q?levels=20(#95)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The #95 filer's production path is a hand-built leaf-by-leaf .put builder: fast (744-768 B/op on a 66-leaf nested ClickInfo) but one hand-maintained line per leaf. The positional builder over held per-field leaf codecs removes the boilerplate but re-enters Codec[Sub].encode for every nested sub-record — the filer implemented it, measured 7.5x time / 16.9x allocation against their hand-built path, and rejected it: it strips only the outermost shell of vulcan's composition. recordBuilder[A] derives the hand-built shape at compile time instead: - the macro walks A's case fields into a plain runtime RecordShape IR (one assembly call emitted; no staged code); - construction resolves every case field's schema slot by NAME via a new all-or-nothing AvroWalk.recordSlots rung, and validates every arm against the schema it writes into; - toRecord is pure positional puts: primitives direct, nested case classes RECURSE into sub-record levels (self-recursive types resolve through the runtime level chain), None puts null exactly as vulcan's OptionCodec, everything else through the field type's own vulcan.Codec — summoned at the derivation site, so a missing leaf codec is a compile error naming the field. Construction is total: every refusal (unresolved case field, two case fields claiming one column, arm/schema disagreement, non-record schema) comes back as the Exception half of Exception | WholeRecordBuilder[A], naming the field and the record. The one documented difference from codec.encode: schema-only (computed/derived) columns keep their in-record default — the hand-built .put contract, round-trip-safe through the codec's decode. Doctrine pass on the vulcan surface per review: no naked throws. AvroVulcan.codec is two forms — codec(schema) (total; schema in hand) and codec[A]: Either[Exception, AvroCodec[A]] — schemaOf deleted; the opt-in given stays the one documented eager-failure site. Measured (benchmarks ClickRecordBench, -prof gc, the filer's shape): derived 744 B/op / hand-built 768 / rejected positional 23,912 / full codec 28,408 — the allocation gate is hand-built-equal, and ns/op stays sub-microsecond through codecPrism[...].record.reverseGet (asAvroCodec wiring, +0.001 B/op). Gates: avroIntegration/test 235 passed (21 new across the builder spec, macro-error catalogue and the bridge spec), root aggregate compile, scalafmt on touched modules, .github/bench tooling unittests. --- .github/bench/bench_tools.py | 8 +- CHANGELOG.md | 28 + .../dev/constructive/eo/avro/AvroWalk.scala | 94 ++++ .../eo/avro/vulcan/AvroVulcan.scala | 92 +++- .../eo/avro/vulcan/RecordBuilderMacro.scala | 164 ++++++ .../eo/avro/vulcan/WholeRecordBuilder.scala | 402 +++++++++++++++ .../eo/avro/vulcan/AvroVulcanSpec.scala | 14 +- .../vulcan/WholeRecordBuilderFixtures.scala | 347 +++++++++++++ .../WholeRecordBuilderMacroErrorSpec.scala | 32 ++ .../avro/vulcan/WholeRecordBuilderSpec.scala | 227 +++++++++ .../eo/bench/AvroVulcanBench.scala | 5 +- .../eo/bench/ClickRecordBench.scala | 478 ++++++++++++++++++ site/docs/integrations/avro.md | 82 ++- 13 files changed, 1946 insertions(+), 27 deletions(-) create mode 100644 avro/src/main/scala/dev/constructive/eo/avro/vulcan/RecordBuilderMacro.scala create mode 100644 avro/src/main/scala/dev/constructive/eo/avro/vulcan/WholeRecordBuilder.scala create mode 100644 avro/src/test/scala/dev/constructive/eo/avro/vulcan/WholeRecordBuilderFixtures.scala create mode 100644 avro/src/test/scala/dev/constructive/eo/avro/vulcan/WholeRecordBuilderMacroErrorSpec.scala create mode 100644 avro/src/test/scala/dev/constructive/eo/avro/vulcan/WholeRecordBuilderSpec.scala create mode 100644 benchmarks/src/main/scala/dev/constructive/eo/bench/ClickRecordBench.scala diff --git a/.github/bench/bench_tools.py b/.github/bench/bench_tools.py index 24283e68..a0778a21 100644 --- a/.github/bench/bench_tools.py +++ b/.github/bench/bench_tools.py @@ -40,7 +40,13 @@ # new module without a mapping entry must never silently skip benchmarks. MODULE_BENCHES = { - "avro/": ["AvroBytesBench", "AvroJsonBridgeBench", "AvroVulcanBench", "OrderAvroBench"], + "avro/": [ + "AvroBytesBench", + "AvroJsonBridgeBench", + "AvroVulcanBench", + "OrderAvroBench", + "ClickRecordBench", + ], "circe/": ["JsoniterBench", "OpticBuildBench", "OrderCirceBench", "PlatedBench"], "jsoniter/": [ "AvroJsonBridgeBench", diff --git a/CHANGELOG.md b/CHANGELOG.md index 6048e1a4..52bdb480 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,34 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +### Added + +- **`cats-eo-avro`: the derived whole-record builder — `AvroVulcan.recordBuilder` (#95)**: + `A ⇒ GenericData.Record`, leaf by leaf, at hand-built cost with zero hand-maintained lines. The + macro walks `A`'s case fields at expansion into a plain runtime `RecordShape` IR and emits ONE + assembly call; construction resolves every case field's schema slot by NAME (all-or-nothing, the + #105 doctrine via a new total `AvroWalk.recordSlots` rung) and validates every arm against the + schema it writes into — so `toRecord` is pure positional puts. Per field: primitives put the value + itself, nested case classes RECURSE into a sub-record level (the piece the positional builder the + filer benchmarked and rejected lacked — theirs re-entered `Codec[Sub].encode`, keeping vulcan's + per-sub-record composition, which measured 7.5x time / 16.9x allocation on the real nested + ClickInfo), `None` puts null exactly as vulcan's `OptionCodec`, and everything else (enums, bytes, + logical types, collections, sums, value classes) falls back to the field type's own + `vulcan.Codec` — summoned at the derivation site, so a missing leaf codec is a compile error + naming the field. Construction is TOTAL (`Exception | WholeRecordBuilder[A]`; `AvroWalk.recordSlots` + returns the failure instead of throwing), self-recursive case classes terminate through the runtime + level chain, and the one documented difference from `codec.encode` is schema-only columns + (computed/derived fields keep their in-record default — the hand-built `.put` contract, + round-trip-safe through the codec's decode). `WholeRecordBuilder.asAvroCodec` installs it as an + `AvroCodec.encode` in one line (`given AvroCodec[ClickInfo] = + clickBuilder.fold(e => throw e, _.asAvroCodec)`), keeping drilled `.field` reads and + `codecPrism[...].record.reverseGet` on the same fast path. The bridge also gains the doctrine's + shape overall: `AvroVulcan.codec` is two forms — `codec(schema)` (total; schema in hand) and + `codec[A]: Either[Exception, AvroCodec[A]]` (resolves from the codec) — with no naked throws on + any construction path; the opt-in given stays the one documented eager-failure site. Allocation + gate: `benchmarks` `ClickRecordBench` (66-leaf nested ClickInfo) — derived 744 B/op vs hand-built + 768 vs the rejected positional 23,912 vs full codec 28,408. + ## [0.16.0] - 2026-09-18 ### Added diff --git a/avro/src/main/scala/dev/constructive/eo/avro/AvroWalk.scala b/avro/src/main/scala/dev/constructive/eo/avro/AvroWalk.scala index f261b8d4..152d3ccc 100644 --- a/avro/src/main/scala/dev/constructive/eo/avro/AvroWalk.scala +++ b/avro/src/main/scala/dev/constructive/eo/avro/AvroWalk.scala @@ -593,6 +593,100 @@ private[avro] object AvroWalk: loop(i + 1, t, here) if loop(0, caseNames, null) then out(declIdx) else -1 + /** ALL-OR-NOTHING slot resolution for a whole-record builder level (issue #95's builder). + * + * The same two rungs as [[totalNominalIndex]] per case field — the EXACT name first, through + * Avro's own per-record hash (O(1), allocation-free), then the cached normalised index (issue + * #103) — but the verdict is INVERTED. [[totalNominalIndex]] resolves ONE drilled hop and + * abstains to the positional rung when the case-field list doesn't map: there, a + * positionally-1:1 codec is already correct and must not be re-aimed. A BUILDER has no + * currently-correct fallback — it writes fresh slots from scratch — so a positional guess would + * be exactly the silent corruption the nominal rung exists to prevent, and the only safe verdict + * is TOTAL (every case field resolves) and INJECTIVE (no two case fields claim one schema + * column). Any miss or collision comes back as the [[IllegalArgumentException]] the caller + * throws, naming the field, the record and its field list — construction-time, before any record + * is built. + * + * Schema-only fields (computed/derived columns the case class doesn't hold) are deliberately NOT + * a failure — the case-field list must map INTO the schema, not ONTO it — and keep their + * in-record value: `GenericData.Record(schema)` zero-fills unwritten slots, so a nullable + * computed column serialises as null and a required one fails loudly at write time. That is the + * same contract a hand-built `.put` builder has, which is the point. + * + * `private[avro]` so the builder machinery resolves through this one implementation of the + * doctrine rather than duplicating [[normalisedName]]. + */ + private[avro] def recordSlots( + record: Schema, + caseNames: List[String], + who: String, + ): Exception | Array[Int] = + val out = new Array[Int](caseNames.size) + val names = caseNames.toArray + // Injectivity by scanning the filled prefix of `out` rather than a `Set[Int]`, as in + // [[totalNominalIndex]]: this runs once per builder construction over the level's fields, + // and a Set here costs a boxed Integer and a new Set node per field. + @tailrec def seen(j: Int, idx: Int): Boolean = + j < 0 || (out(j) != idx && seen(j - 1, idx)) + // The name of the earlier case field that already claimed `idx` (the prefix is filled). + @tailrec def earlierClaimant(j: Int, idx: Int): String = + if out(j) == idx then names(j) else earlierClaimant(j - 1, idx) + // The first failing case field with its reason, or none. `index` is the normalised index + // once some case field has needed it, `null` until then — threaded as a recursion parameter + // exactly as in [[totalNominalIndex]], so the identity-named codec never builds an index. + @tailrec def loop( + i: Int, + rest: List[String], + index: JMap[String, Integer] | Null, + ): None.type | (String, String) = + rest match + case Nil => None + case n :: t => + val exact = record.getField(n) + val resolved: Integer | Null = + if exact != null then Integer.valueOf(exact.pos) + else + val here = if index != null then index else normalisedNameIndex(record) + here.get(normalisedName(n)) + resolved match + case null => + (n, "does not name a schema field") + case ambiguous if ambiguous.intValue < 0 => + ( + n, + "matches more than one schema field up to `_`/`-`/`.`/case normalisation", + ) + case idxVal => + val idx = idxVal.intValue + if !seen(i - 1, idx) then + ( + n, + s"collides with case field '${earlierClaimant(i - 1, idx)}' — both resolve to" + + s" schema position $idx", + ) + else + out(i) = idx + loop(i + 1, t, index) + if record.getType != Schema.Type.RECORD then + IllegalArgumentException( + s"$who: the schema is a ${record.getType}, not a record — a whole-record builder mirrors" + + " case fields onto record fields; use the codec for non-record shapes." + ) + else + loop(0, caseNames, null) match + case None => out + case (n, why) => + val fields = record.getFields + IllegalArgumentException( + s"$who: case field '$n' $why — record '${record.getFullName}' fields: " + + fields.asScala.map(_.name).mkString(", ") + + ". A whole-record builder resolves EVERY case field by NAME (exact first, then up to" + + " `_`/`-`/`.`/case, all-or-nothing over the whole case-field list); schema-only fields" + + " (computed columns) are fine and keep their schema default, but a case field must" + + " exist and claim one column. If the codec renames or drops this field, encode" + + " through the codec (or keep the hand-built builder) for this type instead." + ) + /** `_` / `-` / `.` stripped and everything lower-cased: the canonical form under which * `landingPageId`, `landing_page_id`, `LANDING_PAGE_ID` and `landing-page-id` are one name. * diff --git a/avro/src/main/scala/dev/constructive/eo/avro/vulcan/AvroVulcan.scala b/avro/src/main/scala/dev/constructive/eo/avro/vulcan/AvroVulcan.scala index 8f9bd84f..0235ae1b 100644 --- a/avro/src/main/scala/dev/constructive/eo/avro/vulcan/AvroVulcan.scala +++ b/avro/src/main/scala/dev/constructive/eo/avro/vulcan/AvroVulcan.scala @@ -17,14 +17,17 @@ import org.apache.avro.Schema * val p = codecPrism[ClickInfo].field(_.entities) // summons through the bridge * }}} * - * or, named and explicit, `given AvroCodec[ClickInfo] = AvroVulcan.codec`. + * The bridge comes in two shapes. `codec(schema)` is total: the schema is in hand, there is + * nothing to resolve. `codec(using)` resolves the schema from the codec and can fail, so it + * returns `Either[Exception, AvroCodec[A]]`. The opt-in given below is the one site with no + * failure channel — a summon must produce a codec — so a schema that will not resolve fails + * eagerly, at the given site. * * Error mapping — vulcan is `Either`-typed where eo is total or `Throwable`-typed: - * - `schema` is resolved ONCE at construction and throws vulcan's own `AvroError.throwable` if - * invalid — fail fast: an invalid schema is a codec-definition bug, not a per-record - * condition, and eo treats schemas as static. - * - `encode` throws on error for the same reason (eo's `encode(a: A): Any` is total; an encode - * failure under a matching schema is a definition bug). + * - schema resolution: `Left` from `codec(using)`; impossible for `codec(schema)`; eager at the + * given site for the given (see above). + * - `encode` throws on error (eo's `encode(a: A): Any` is total — an encode failure under a + * matching schema is a codec-definition bug, not a per-record condition). * - `decode` errors surface as `Left` via `AvroError.throwable`, matching [[AvroCodec]]'s * structured-failure convention. * @@ -35,17 +38,78 @@ import org.apache.avro.Schema */ object AvroVulcan: - /** Bridge the in-scope `vulcan.Codec[A]` into an [[AvroCodec]][A]. Throws at construction if the - * vulcan schema is invalid (see the object docs for the full error mapping). + /** Bridge `c` under an EXPLICIT schema — the total form: nothing is resolved, so nothing can fail + * at construction. Use when the schema is already in hand (a parsed `.avsc`, a registry lookup, + * the filer's own maintained schema). */ - def codec[A](using c: VCodec[A]): AvroCodec[A] = new AvroCodec[A]: - val schema: Schema = c.schema.fold(e => throw e.throwable, identity) - def encode(a: A): Any = c.encode(a).fold(e => throw e.throwable, identity) - def decodeEither(any: Any): Either[Throwable, A] = c.decode(any, schema).left.map(_.throwable) + def codec[A](schema: Schema)(using c: VCodec[A]): AvroCodec[A] = + val v = c + val s = schema + new AvroCodec[A]: + val schema: Schema = s + def encode(a: A): Any = v.encode(a).fold(e => throw e.throwable, identity) + def decodeEither(any: Any): Either[Throwable, A] = + v.decode(any, schema).left.map(_.throwable) + + /** Bridge the in-scope `vulcan.Codec[A]`, resolving its schema — `Left` (vulcan's own error, as + * an [[Exception]]) when the schema will not resolve. The schema-resolved codec is returned + * whole; nothing throws. + */ + def codec[A](using c: VCodec[A]): Either[Exception, AvroCodec[A]] = + c.schema + .fold( + e => + Left(e.throwable match + case ex: Exception => ex + case other => IllegalStateException("vulcan schema resolution failed", other), + ), + schema => Right(codec(schema)) + ) + + /** The derived whole-record builder (issue #95): `A ⇒ GenericData.Record`, leaf by leaf, at + * hand-built cost with zero hand-maintained lines. + * + * The macro walks `A`'s case fields at COMPILE time; construction resolves every case field's + * schema slot by NAME (all-or-nothing, issue #105's doctrine) and validates every arm against + * the schema it writes into — so [[WholeRecordBuilder.toRecord]] is pure positional puts. This + * is the positional-over-held-leaf-codecs builder the issue filer benchmarked and rejected, with + * the missing piece: it RECURSES into nested case classes (each sub-record built by the same + * rule against its own schema) instead of re-entering `Codec[Sub].encode` — the composition + * their prototype kept paying per sub-record, which is what cost them the 7.5x time / 16.9x + * allocation regression on the real nested ClickInfo. + * + * Per-field arms, classified from the case-class shape at expansion: + * - Boolean / Int / Long / Float / Double / String → the value itself (the Avro datum); + * - nested case class → a sub-record level (recursive; self-recursive types resolve through + * the runtime level chain, so recursive case classes terminate); + * - `Option[X]` → `None` puts null exactly as vulcan's `OptionCodec`, `Some(v)` recurses; + * - everything else (enums, bytes, logical types, collections, sums, value classes) → the + * field type's own `vulcan.Codec`, summoned HERE — a missing leaf codec is a compile error + * pointing at the field. + * + * Construction is TOTAL: every way assembly can fail comes back as the [[Exception]] half, + * naming the field and the record, before any record is built — a case field naming no schema + * column (computed/derived schema columns are fine and keep their in-record default), two case + * fields claiming one column, an arm disagreeing with its schema field's shape, or the codec's + * schema itself refusing to resolve. Hold the builder in a `val` / `given`: construction is the + * once-cost, `toRecord` is the hot path. + * + * Wire-compatible with [[codec]]'s encode on the fields the case class holds; the one documented + * difference is schema-only columns (see [[WholeRecordBuilder]]). The common install: + * + * {{{ + * val clickBuilder = AvroVulcan.recordBuilder[ClickInfo] // Exception | WholeRecordBuilder + * given AvroCodec[ClickInfo] = clickBuilder.fold(e => throw e, _.asAvroCodec) + * }}} + */ + inline def recordBuilder[A](using c: VCodec[A]): Exception | WholeRecordBuilder[A] = + ${ RecordBuilderMacro.builderImpl[A]('c) } /** `import dev.constructive.eo.avro.vulcan.given` makes every in-scope `vulcan.Codec[A]` usable * wherever eo demands `AvroCodec[A]` evidence. Opt-in by import — don't combine with * kindlings-derived `AvroCodec` givens for the same `A` in one scope, or the summon turns - * ambiguous. + * ambiguous. A given must produce its value, so a schema that will not resolve fails HERE — prefer + * the explicit [[AvroVulcan.codec]] forms where an `Either` can be surfaced. */ -given vulcanAvroCodec[A](using VCodec[A]): AvroCodec[A] = AvroVulcan.codec[A] +given vulcanAvroCodec[A](using VCodec[A]): AvroCodec[A] = + AvroVulcan.codec[A].fold(e => throw e, identity) diff --git a/avro/src/main/scala/dev/constructive/eo/avro/vulcan/RecordBuilderMacro.scala b/avro/src/main/scala/dev/constructive/eo/avro/vulcan/RecordBuilderMacro.scala new file mode 100644 index 00000000..50de702a --- /dev/null +++ b/avro/src/main/scala/dev/constructive/eo/avro/vulcan/RecordBuilderMacro.scala @@ -0,0 +1,164 @@ +package dev.constructive.eo.avro.vulcan + +import scala.quoted.* + +import _root_.vulcan.Codec as VCodec +import dev.constructive.eo.avro.vulcan.WholeRecordBuilder.{ + CodecKind, + DirectKind, + FieldShape, + Kind, + OptionKind, + RecordKind, + RecordShape, + SelfKind, +} +import org.apache.avro.Schema + +/** The macro behind [[AvroVulcan.recordBuilder]] — walks `A`'s case fields at EXPANSION and emits + * the [[WholeRecordBuilder.RecordShape]] the runtime assembly resolves against the codec's schema. + * + * The macro's whole job is classification: it never emits per-field code. Each case field becomes + * one [[WholeRecordBuilder.FieldShape]] arm — + * + * - `Option[X]` (dealiased) → [[WholeRecordBuilder.OptionKind]] over `X`'s classification; + * - Boolean / Int / Long / Float / Double / String → [[WholeRecordBuilder.DirectKind]]; + * - a case class (Case-flagged class, not sealed, not a module, not an AnyVal) → + * [[WholeRecordBuilder.RecordKind]] with the sub-shape, or [[WholeRecordBuilder.SelfKind]] + * when the type is already an ancestor on the derivation path (recursive case classes + * terminate at compile time and resolve through the runtime level chain); + * - everything else → [[WholeRecordBuilder.CodecKind]] holding the field type's own + * `vulcan.Codec`, summoned HERE so the caller's scope answers for its leaves — a missing leaf + * codec is a compile error pointing at the exact field. + * + * Because the emitted value is plain data, everything schema-dependent (slot resolution, arm + * validation) happens at builder construction in [[WholeRecordBuilder.derive]] — ordinary + * testable Scala, no staged code. The whole classification lives in [[builderImpl]] as local defs + * under ONE `Quotes`: `TypeRepr` / `Symbol` are path-dependent on the Quotes instance, so a + * `(using Quotes)`-taking helper called from inside a quote would type against a different path. + */ +object RecordBuilderMacro: + + /** Cap on nested record levels per shape. A diamond-heavy case-class graph re-expands shared + * branches per occurrence, so the shape tree grows multiplicatively even though no type ever + * repeats on one path (a repeat becomes a [[WholeRecordBuilder.SelfKind]]); beyond this depth + * the shape is pathological for a whole-record builder and the deep part belongs to its codec. + */ + private val MaxDepth = 24 + + /** Entry: `AvroVulcan.recordBuilder[A]`. Requires a case class `A` (sums encode through their + * codec, not a builder) and the in-scope `vulcan.Codec[A]` whose schema the builder writes. + */ + def builderImpl[A: Type](codec: Expr[VCodec[A]])(using Quotes): Expr[Exception | WholeRecordBuilder[A]] = + import quotes.reflect.* + + def recordShapeOf( + tpe: TypeRepr, + who: String, + ancestors: List[Symbol], + depth: Int, + ): Expr[RecordShape] = + if depth > MaxDepth then + report.errorAndAbort( + s"$who: shape derivation exceeded $MaxDepth nested record levels — a diamond-heavy" + + " case-class graph; derive the builder at a shallower type and encode the deep part" + + " through its codec." + ) + val fieldExprs = tpe.typeSymbol.caseFields.map { fieldSym => + val name = fieldSym.name + val kind = fieldKind(name, tpe.memberType(fieldSym).dealias, ancestors, depth, who) + '{ FieldShape(${ Expr(name) }, $kind) } + } + '{ RecordShape(${ Expr(tpe.show) }, ${ Expr.ofList(fieldExprs) }) } + + def fieldKind( + name: String, + t: TypeRepr, + ancestors: List[Symbol], + depth: Int, + who: String, + ): Expr[Kind] = + t.dealias match + case AppliedType(tc, arg :: Nil) if tc =:= TypeRepr.of[Option] => + '{ OptionKind(${ fieldKind(name, arg.dealias, ancestors, depth, who) }) } + case _ => + val tt = t.widen.dealias + directSchemaType(tt) match + case null => + val tsym = tt.typeSymbol + if isCaseClass(tt) then + ancestors.indexOf(tsym) match + case -1 => + '{ RecordKind(${ recordShapeOf(tt, s"$who → $name", tsym :: ancestors, depth + 1) }) } + case d => '{ SelfKind(${ Expr(d) }) } + else summonLeafCodec(name, tt, who) + case st => '{ DirectKind(${ schemaTypeExpr(st) }) } + + /** The schema type a direct (value-is-the-datum) leaf writes, or null when the type has no + * exact direct arm. Exact kinds only: the builder never widens numerically, so a case field of + * the wrong primitive falls to its codec rather than to a silent promotion. + */ + def directSchemaType(t: TypeRepr): Schema.Type | Null = + if t =:= TypeRepr.of[Boolean] then Schema.Type.BOOLEAN + else if t =:= TypeRepr.of[Int] then Schema.Type.INT + else if t =:= TypeRepr.of[Long] then Schema.Type.LONG + else if t =:= TypeRepr.of[Float] then Schema.Type.FLOAT + else if t =:= TypeRepr.of[Double] then Schema.Type.DOUBLE + else if t =:= TypeRepr.of[String] then Schema.Type.STRING + else null + + /** The quoted `Schema.Type` constant — Java enums do not lift, so each arm is spelled out. */ + def schemaTypeExpr(st: Schema.Type): Expr[Schema.Type] = + st match + case Schema.Type.BOOLEAN => '{ Schema.Type.BOOLEAN } + case Schema.Type.INT => '{ Schema.Type.INT } + case Schema.Type.LONG => '{ Schema.Type.LONG } + case Schema.Type.FLOAT => '{ Schema.Type.FLOAT } + case Schema.Type.DOUBLE => '{ Schema.Type.DOUBLE } + case Schema.Type.STRING => '{ Schema.Type.STRING } + case other => + report.errorAndAbort( + s"RecordBuilderMacro: internal — unhandled direct schema type $other" + ) + + def isCaseClass(t: TypeRepr): Boolean = + val sym = t.typeSymbol + sym.isClassDef && sym.flags.is(Flags.Case) && !sym.flags.is(Flags.Sealed) + && !sym.flags.is(Flags.Module) && !(t <:< TypeRepr.of[AnyVal]) + + def summonLeafCodec(name: String, t: TypeRepr, who: String): Expr[Kind] = + t.asType match + case '[x] => + Expr.summon[VCodec[x]] match + case Some(codecE) => '{ CodecKind(${ codecE }.asInstanceOf[VCodec[Any]]) } + case None => + report.errorAndAbort( + s"$who: field '$name' of type ${Type.show[x]} has no given vulcan.Codec in scope." + + " The builder fast-paths Boolean/Int/Long/Float/Double/String leaves and nested" + + " case-class fields; everything else (enums, bytes, logical types, collections," + + " sums, value classes) falls back to the field type's own vulcan codec — provide" + + " it or change the field's type." + ) + + val who = s"AvroVulcan.recordBuilder[${Type.show[A]}]" + val tpe = TypeRepr.of[A].dealias + if !isCaseClass(tpe) then + report.errorAndAbort( + s"$who: ${Type.show[A]} is not a case class — the builder mirrors a case class onto record" + + " fields; sums (sealed traits, enums, unions) encode through their codec." + ) + val shape = recordShapeOf(tpe, who, tpe.typeSymbol :: Nil, depth = 0) + // Total: the codec's schema failure and the assembly failure both come back as the union's + // Exception half (the vulcan error wrapped, its own throwable as the cause). + '{ + ${ codec }.schema.fold( + e => + IllegalArgumentException( + ${ Expr(who) } + ": the vulcan codec's schema did not resolve", + e.throwable, + ), + schema => WholeRecordBuilder.derive[A](schema, $shape, ${ Expr(who) }), + ) + }.asExprOf[Exception | WholeRecordBuilder[A]] + +end RecordBuilderMacro diff --git a/avro/src/main/scala/dev/constructive/eo/avro/vulcan/WholeRecordBuilder.scala b/avro/src/main/scala/dev/constructive/eo/avro/vulcan/WholeRecordBuilder.scala new file mode 100644 index 00000000..c43384d7 --- /dev/null +++ b/avro/src/main/scala/dev/constructive/eo/avro/vulcan/WholeRecordBuilder.scala @@ -0,0 +1,402 @@ +package dev.constructive.eo.avro.vulcan + +import scala.annotation.tailrec +import scala.jdk.CollectionConverters.* + +import _root_.vulcan.Codec as VCodec +import dev.constructive.eo.avro.{AvroCodec, AvroWalk} +import org.apache.avro.Schema +import org.apache.avro.generic.GenericData + +/** A compile-time-derived whole-record builder: `A ⇒ GenericData.Record`, leaf by leaf, with no + * codec composition on the hot path (issue #95). + * + * '''The use case.''' Building a fresh generic record from a typed value on a hot path — an ingest + * side, a replay, a batch flush. The full vulcan `Codec[A].encode` pays its composition once per + * FIELD, per LEVEL: a `FreeApplicative.analyze`, an `Either` + `Chain.one` per field, and a + * `put(name, value)` hash probe — and a nested sub-record field redoes all of it inside + * `Codec[Sub].encode`, which is what the filer measured as ~384–468 B/field on their real + * ClickInfo. A hand-built `.put(pos, value)` builder avoids all of it but costs one hand-maintained + * line per leaf — the exact complaint the filer opened the issue with. + * + * '''What a derived builder is.''' [[AvroVulcan.recordBuilder]] walks `A`'s case fields at + * COMPILE time (the [[WholeRecordBuilder.RecordShape]] IR) and emits one runtime assembly call; + * construction resolves every case field's schema slot by NAME (all-or-nothing, issue #105's + * doctrine) and validates every arm against the schema it will write into — so `toRecord` itself + * is nothing but positional puts: `new GenericData.Record(schema)`, then per field either the + * value itself (a primitive leaf), a recursive sub-record build (a nested case class — the + * recursion the filer's positional prototype lacked, which is what made nested shapes pay vulcan's + * per-sub-record composition), `null` (a `None`), or the field type's own leaf codec (everything + * the fast arms don't cover). Schema-only fields (computed/derived columns the case class doesn't + * hold) stay at their in-record default. + * + * '''Allocation is the gate, and it is hand-built-equal.''' Per record: the `GenericData.Record` + * values array plus one boxed value per primitive leaf — exactly what the hand-built builder + * allocates; the plans and slots are construction-time. ns/op stays within a small multiple of the + * hand-built form (one erasure-level dispatch per non-primitive leaf; the primitive bulk is a + * tight positional loop) and far below the codec composition it replaces — the `benchmarks` + * `ClickRecordBench` measures all of it side by side. + * + * '''Construction is total.''' Every way assembly can fail — a case field no schema column answers + * for, two case fields claiming one column, an arm disagreeing with its schema field's shape, a + * non-record schema — comes back as the [[Exception]] half of the `recordBuilder` / `derive` + * result, naming the field and the record, BEFORE any record is built. `toRecord` itself is total + * for values matching `A` (a codec-leaf arm that fails encode still throws, per `AvroCodec`'s + * total-encode convention — that is a codec-definition bug, not a construction condition). + * + * '''The one behavioural difference from `Codec[A].encode`,''' stated because a wire-compat claim + * without it would be false precision: a schema-only (computed/derived) column. The codec fills it + * during encode; the builder leaves the slot at its in-record value (null on a fresh + * `GenericData.Record`) — identical to the hand-built `.put` builder the issue benchmarked, and + * round-trip-safe through the codec's own decode (which reconstructs `A` from the fields it + * knows). + * + * @param schema + * the record schema the builder writes into — the codec's own schema object, so a record built + * by the builder and a record decoded by the codec share one identity. + */ +final class WholeRecordBuilder[A] private[avro] ( + val schema: Schema, + level: WholeRecordBuilder.RecordLevel, +): + + /** Build the whole record from `a`. Total for values matching `A`; every case field's slot is + * written, schema-only columns keep their in-record default. + */ + def toRecord(a: A): GenericData.Record = level.build(a) + + /** The [[AvroCodec]] with THIS builder as its encode and the in-scope vulcan codec as its decode + * over THIS builder's schema — the one-line replacement for the hand-written + * `given AvroCodec[A] with { def encode(a) = buildRecord(a) … }` the issue's recommendation + * asked for. Decode errors surface as `Left`; encode is total (see [[AvroVulcan]] for the error + * mapping). + */ + def asAvroCodec(using c: VCodec[A]): AvroCodec[A] = + val v = c + new AvroCodec[A]: + val schema: Schema = WholeRecordBuilder.this.schema + def encode(a: A): Any = toRecord(a) + def decodeEither(any: Any): Either[Throwable, A] = + v.decode(any, WholeRecordBuilder.this.schema).left.map(_.throwable) + +object WholeRecordBuilder: + + // ---- The compile-time IR (macro-emitted, hand-authorable) ------------------- + // + // The `RecordShape` for a case class is the WHOLE derivation the macro emits: case fields in + // declaration order, each classified into one of five arms. It is deliberately a plain runtime + // VALUE (not a macro-only tree) so the assembly below is ordinary testable Scala, and so a shape + // the macro cannot classify can be authored by hand. + + /** One record level: `typeName` for failure messages, case fields in declaration order. */ + final case class RecordShape(typeName: String, fields: List[FieldShape]) + + /** One case field: its name and how `toRecord` obtains the datum it puts. */ + final case class FieldShape(name: String, kind: Kind) + + sealed trait Kind + + /** Primitive leaf written directly: the runtime value IS the Avro datum (Boolean / Int / Long / + * Float / Double / String). Validated against the schema field type at construction. + */ + final case class DirectKind(schemaType: Schema.Type) extends Kind + + /** `Option[X]`: `None` puts null (vulcan's own `OptionCodec`), `Some(v)` puts through the inner + * arm against the non-null union branch. + */ + final case class OptionKind(inner: Kind) extends Kind + + /** Nested case class: a sub-record level built by the same rule against the field's own record + * schema — the recursion that makes nested shapes cost leaf-by-leaf, not codec-per-sub-record. + */ + final case class RecordKind(sub: RecordShape) extends Kind + + /** A self-recursive ancestor already on the derivation path (`depth` levels up, 0 = the level + * being built): resolved to that level at construction, so recursive case classes terminate. + */ + final case class SelfKind(depth: Int) extends Kind + + /** Everything else — enums, bytes, logical types, collections, sums, value classes — encoded by + * the field type's own vulcan codec, summoned at the derivation site and held here. + */ + final case class CodecKind(codec: VCodec[Any]) extends Kind + + // ---- Runtime assembly ------------------------------------------------------ + // + // Walks the `RecordShape` against the codec's schema: slots by name (AvroWalk.recordSlots, + // all-or-nothing), each arm validated against the schema it will write into, sub-levels built + // recursively. Everything happens HERE — once at builder construction, TOTAL (failures come back + // as the Exception half, never thrown) — so `toRecord` is pure positional puts. + + /** Assemble a [[WholeRecordBuilder]] for `A` from `schema` (the codec's) and the compile-time + * `shape`. Returns the failure — naming the field and the record — instead of a builder, when a + * case field names no schema column, two case fields claim one, or an arm disagrees with its + * schema field's shape. Never throws. + */ + def derive[A](schema: Schema, shape: RecordShape, who: String): Exception | WholeRecordBuilder[A] = + if schema.getType != Schema.Type.RECORD then + IllegalArgumentException( + s"$who: the codec's schema is a ${schema.getType}, not a record — the builder mirrors a case" + + " class onto record fields; use the codec for non-record shapes." + ) + else + buildLevel(schema, shape, who, parent = null) match + case e: Exception => e + case level: RecordLevel => new WholeRecordBuilder[A](schema, level) + + /** One record level of the builder: resolved slots plus the per-field plans. Built shell-first (a + * `SelfKind` arm captures the level being built), then sealed — `buildLevel` seals only on + * success, so `build` never sees the empty arrays. + */ + private[avro] final class RecordLevel( + val schema: Schema, + val parent: RecordLevel | Null, + ): + private var directs: Array[DirectSlot] = Array.empty + private var unusual: Array[UnusualSlot] = Array.empty + + private[avro] def seal(d: Array[DirectSlot], u: Array[UnusualSlot]): Unit = + directs = d + unusual = u + + /** Build the record: the primitive bulk first (no dispatch — a positional put loop), then the + * option / sub-record / codec arms. Positions are already resolved; every put is positional. + */ + def build(a: Any): GenericData.Record = + val record = new GenericData.Record(schema) + val product = a.asInstanceOf[Product] + putDirects(product, record, 0) + putUnusual(product, record, 0) + record + + @tailrec private def putDirects(product: Product, r: GenericData.Record, i: Int): Unit = + if i < directs.length then + val d = directs(i) + r.put(d.slot, product.productElement(d.decl)) + putDirects(product, r, i + 1) + + @tailrec private def putUnusual(product: Product, r: GenericData.Record, i: Int): Unit = + if i < unusual.length then + val u = unusual(i) + u.plan.put(r, product.productElement(u.decl)) + putUnusual(product, r, i + 1) + + private[avro] final case class DirectSlot(slot: Int, decl: Int) + private[avro] final case class UnusualSlot(decl: Int, plan: FieldPlan) + + /** One non-primitive arm of a record level: how `toRecord` turns the field's value into the datum + * it puts at `slot`. `put` receives the value as erased `Any` (the case field's value, already + * boxed by `productElement`), so a level dispatches without per-type closures. + */ + private[avro] sealed trait FieldPlan: + def slot: Int + def put(r: GenericData.Record, value: Any): Unit + + private[avro] final case class DirectPlan(slot: Int) extends FieldPlan: + def put(r: GenericData.Record, value: Any): Unit = r.put(slot, value) + + /** `None` → null (vulcan's `OptionCodec`), `Some(v)` → the inner arm. The option field owns ONE + * schema slot; both branches put there. + */ + private[avro] final case class OptionPlan(slot: Int, inner: FieldPlan) extends FieldPlan: + def put(r: GenericData.Record, value: Any): Unit = + value match + case None => r.put(slot, null) + case Some(v) => inner.put(r, v) + + private[avro] final case class SubRecordPlan(slot: Int, sub: RecordLevel) extends FieldPlan: + def put(r: GenericData.Record, value: Any): Unit = r.put(slot, sub.build(value)) + + /** The field type's own codec — resolved once at construction; encode errors throw (eo's total + * encode convention, matching [[AvroVulcan.codec]]). + */ + private[avro] final case class CodecPlan(slot: Int, codec: VCodec[Any]) extends FieldPlan: + def put(r: GenericData.Record, value: Any): Unit = + r.put(slot, codec.encode(value).fold(e => throw e.throwable, identity)) + + private[avro] def buildLevel( + schema: Schema, + shape: RecordShape, + who: String, + parent: RecordLevel | Null, + ): Exception | RecordLevel = + val level = new RecordLevel(schema, parent) + AvroWalk.recordSlots(schema, shape.fields.map(_.name), who) match + case e: Exception => e + case slots: Array[Int] => + val fields = schema.getFields + val directBuf = List.newBuilder[DirectSlot] + val unusualBuf = List.newBuilder[UnusualSlot] + // The first arm failure short-circuits the level; the shell is discarded with it. + @tailrec def each(i: Int, rest: List[FieldShape]): Exception | Null = + rest match + case Nil => null + case f :: t => + planFor(f.kind, slots(i), f.name, fields.get(slots(i)).schema, level, who) match + case e: Exception => e + case d: DirectPlan => + directBuf += DirectSlot(d.slot, i) + each(i + 1, t) + case plan: FieldPlan => + unusualBuf += UnusualSlot(i, plan) + each(i + 1, t) + each(0, shape.fields) match + case e: Exception => e + case null => + level.seal(directBuf.result().toArray, unusualBuf.result().toArray) + level + + /** The `FieldPlan` for one arm, validating the arm against the schema it will write into. */ + private def planFor( + kind: Kind, + slot: Int, + name: String, + schema: Schema, + level: RecordLevel, + who: String, + ): Exception | FieldPlan = + kind match + case DirectKind(expected) => + unwrapNullable(schema, name, who) match + case e: Exception => e + case inner: Schema => + if inner.getType != expected then typeMismatch(name, expected, inner.getType, who) + else DirectPlan(slot) + case RecordKind(sub) => + unwrapNullable(schema, name, who) match + case e: Exception => e + case inner: Schema => + if inner.getType != Schema.Type.RECORD then + typeMismatch(name, Schema.Type.RECORD, inner.getType, who) + else + buildLevel(inner, sub, s"$who → $name (${sub.typeName})", parent = level) match + case e: Exception => e + case subLevel: RecordLevel => SubRecordPlan(slot, subLevel) + case SelfKind(depth) => + climb(level, depth) match + case e: Exception => e + case target: RecordLevel => SubRecordPlan(slot, target) + case CodecKind(codec) => + CodecPlan(slot, codec) + case OptionKind(innerKind) => + if schema.getType != Schema.Type.UNION || !hasNullBranch(schema) then + notNullUnion(name, schema.getType, who) + else + val innerSchema = if schema.getTypes.size == 2 then nonNullBranch(schema) else schema + innerPlanFor(innerKind, slot, name, innerSchema, schema, level, who) match + case e: Exception => e + case inner: FieldPlan => OptionPlan(slot, inner) + + /** The inner arm of an `OptionPlan`. Primitive / nested-record inners require a 2-branch nullable + * pair (they write the non-null branch's shape); codec / self / nested-option inners consume no + * schema shape and accept wider unions. + */ + private def innerPlanFor( + kind: Kind, + slot: Int, + name: String, + innerSchema: Schema, + fieldSchema: Schema, + level: RecordLevel, + who: String, + ): Exception | FieldPlan = + kind match + case DirectKind(expected) => + if fieldSchema.getTypes.size != 2 then pairMismatch(name, fieldSchema.getTypes.size, who) + else if innerSchema.getType != expected then + typeMismatch(name, expected, innerSchema.getType, who) + else DirectPlan(slot) + case RecordKind(sub) => + if fieldSchema.getTypes.size != 2 then pairMismatch(name, fieldSchema.getTypes.size, who) + else if innerSchema.getType != Schema.Type.RECORD then + typeMismatch(name, Schema.Type.RECORD, innerSchema.getType, who) + else + buildLevel(innerSchema, sub, s"$who → $name (${sub.typeName})", parent = level) match + case e: Exception => e + case subLevel: RecordLevel => SubRecordPlan(slot, subLevel) + case other => planFor(other, slot, name, innerSchema, level, who) + + /** The schema behind a plain (non-`Option`) arm: itself, or the non-null branch of a 2-branch + * nullable pair (a non-`Option` case field under a nullable schema — the codec writes the value, + * never null). + */ + private def unwrapNullable(schema: Schema, name: String, who: String): Schema | Exception = + if schema.getType != Schema.Type.UNION then schema + else + val branches = schema.getTypes + if branches.size != 2 then multiBranch(name, branches.size, who) + else + val i = + if branches.get(0).getType == Schema.Type.NULL then 1 + else if branches.get(1).getType == Schema.Type.NULL then 0 + else -1 + if i < 0 then noNullBranch(name, who) + else branches.get(i) + + // ---- Failure factories (one message per refusal, returned — never thrown) ---- + + private def typeMismatch( + name: String, + expected: Schema.Type, + actual: Schema.Type, + who: String, + ): IllegalArgumentException = + IllegalArgumentException( + s"$who: case field '$name' needs a $expected schema field but schema field '$name' is a" + + s" $actual — the schema must agree with the case-class shape; encode through the codec or" + + " align the schema." + ) + + private def pairMismatch(name: String, branches: Int, who: String): IllegalArgumentException = + IllegalArgumentException( + s"$who: case field '$name' is an Option whose inner arm is a primitive leaf or a nested case" + + s" class, but schema field '$name' is a $branches-branch union — the builder's fast arms" + + " handle plain fields and 2-branch nullable pairs only; wider unions belong to the field's" + + " own codec." + ) + + private def notNullUnion( + name: String, + actual: Schema.Type, + who: String, + ): IllegalArgumentException = + IllegalArgumentException( + s"$who: case field '$name' is Option[...] but schema field '$name' is a $actual, not a" + + " null-union — vulcan pairs an optional case field with a null-union, and the builder" + + " mirrors the codec's encode (None → null), not the schema default." + ) + + private def multiBranch(name: String, branches: Int, who: String): IllegalArgumentException = + IllegalArgumentException( + s"$who: schema field '$name' is a $branches-branch union — the builder's non-Option arms" + + " handle plain fields and 2-branch nullable pairs only; wider unions belong to the field's" + + " own codec." + ) + + private def noNullBranch(name: String, who: String): IllegalArgumentException = + IllegalArgumentException( + s"$who: schema field '$name' is a union without a null branch — the builder handles plain" + + " fields and 2-branch nullable pairs only." + ) + + private def hasNullBranch(schema: Schema): Boolean = + schema.getTypes.asScala.exists(_.getType == Schema.Type.NULL) + + private def nonNullBranch(schema: Schema): Schema = + val branches = schema.getTypes + if branches.get(0).getType == Schema.Type.NULL then branches.get(1) else branches.get(0) + + /** `depth` levels up the parent chain — 0 is `level` itself (a self-recursive field of the level + * being built). The depth is compile-time-derived from the ancestor stack, so the failure below + * is an invariant check, not an expected one. + */ + @tailrec private def climb(level: RecordLevel, depth: Int): Exception | RecordLevel = + if depth == 0 then level + else + level.parent match + case p: RecordLevel => climb(p, depth - 1) + case null => + IllegalStateException( + s"whole-record builder: self-reference depth $depth exceeds the level chain —" + + " internal invariant broken" + ) diff --git a/avro/src/test/scala/dev/constructive/eo/avro/vulcan/AvroVulcanSpec.scala b/avro/src/test/scala/dev/constructive/eo/avro/vulcan/AvroVulcanSpec.scala index 4d254242..18d3debb 100644 --- a/avro/src/test/scala/dev/constructive/eo/avro/vulcan/AvroVulcanSpec.scala +++ b/avro/src/test/scala/dev/constructive/eo/avro/vulcan/AvroVulcanSpec.scala @@ -2,8 +2,8 @@ package dev.constructive.eo.avro.vulcan import scala.language.implicitConversions -import _root_.vulcan.Codec as VCodec import cats.syntax.all.* +import _root_.vulcan.Codec as VCodec import dev.constructive.eo.avro.circe.AvroJson import dev.constructive.eo.avro.{codecPrism, AvroCodec} import org.apache.avro.generic.IndexedRecord @@ -24,12 +24,18 @@ class AvroVulcanSpec extends Specification: "AvroVulcan.codec" should { "round-trip encode → decodeEither through the bridged codec" in { - val bridged = AvroVulcan.codec[Combo] - bridged.decodeEither(bridged.encode(original)) must beRight(original) + AvroVulcan.codec[Combo] match + case Right(bridged) => bridged.decodeEither(bridged.encode(original)) must beRight(original) + case Left(e) => ko(e.getMessage) } "surface decode failures as Left, never throw" in { - AvroVulcan.codec[Combo].decodeEither("not a record") must beLeft + AvroVulcan.codec[Combo].map(_.decodeEither("not a record")) must beRight(beLeft) + } + + "bridge under an EXPLICIT schema — the total form" in { + val bridged = AvroVulcan.codec(summon[VCodec[Combo]].schema.toOption.get) + bridged.decodeEither(bridged.encode(original)) must beRight(original) } } diff --git a/avro/src/test/scala/dev/constructive/eo/avro/vulcan/WholeRecordBuilderFixtures.scala b/avro/src/test/scala/dev/constructive/eo/avro/vulcan/WholeRecordBuilderFixtures.scala new file mode 100644 index 00000000..b94040e0 --- /dev/null +++ b/avro/src/test/scala/dev/constructive/eo/avro/vulcan/WholeRecordBuilderFixtures.scala @@ -0,0 +1,347 @@ +package dev.constructive.eo.avro.vulcan + +import java.time.Instant + +import cats.syntax.all.* +import _root_.vulcan.Codec as VCodec + +// ---- Top-level so the vulcan record codecs and the derived builders see plain classfiles. ---- + +enum TrafficClass: + case Organic, Paid, Social, Referral + +object TrafficClass: + given VCodec[TrafficClass] = VCodec.enumeration( + name = "TrafficClass", + namespace = "dev.constructive.eo.avro.vulcan", + symbols = Seq("Organic", "Paid", "Social", "Referral"), + encode = _.toString, + decode = symbol => Right(TrafficClass.valueOf(symbol)), + ) + +final case class Geo(country: String, region: String, city: String, latitude: Double, longitude: Double) + +object Geo: + given VCodec[Geo] = VCodec.record(name = "Geo", namespace = "dev.constructive.eo.avro.vulcan") { fb => + ( + fb("country", _.country), + fb("region", _.region), + fb("city", _.city), + fb("latitude", _.latitude), + fb("longitude", _.longitude), + ).mapN(Geo.apply) + } + +final case class UserAgentInfo( + browser: String, + browserVersion: String, + os: String, + device: String, + language: String, + doNotTrack: Boolean, + robot: Boolean, +) + +object UserAgentInfo: + given VCodec[UserAgentInfo] = VCodec.record( + name = "UserAgentInfo", + namespace = "dev.constructive.eo.avro.vulcan", + ) { fb => + ( + fb("browser", _.browser), + fb("browserVersion", _.browserVersion), + fb("os", _.os), + fb("device", _.device), + fb("language", _.language), + fb("doNotTrack", _.doNotTrack), + fb("robot", _.robot), + ).mapN(UserAgentInfo.apply) + } + +final case class PostClick( + conversionTimestamp: Long, + orderId: Option[String], + revenue: Double, + currency: String, + funnelStep: Int, + completed: Boolean, + attributionWindow: Option[Long], + landingPage: String, + referrer: String, +) + +object PostClick: + given VCodec[PostClick] = VCodec.record( + name = "PostClick", + namespace = "dev.constructive.eo.avro.vulcan", + ) { fb => + ( + fb("conversionTimestamp", _.conversionTimestamp), + fb("orderId", _.orderId), + fb("revenue", _.revenue), + fb("currency", _.currency), + fb("funnelStep", _.funnelStep), + fb("completed", _.completed), + fb("attributionWindow", _.attributionWindow), + fb("landingPage", _.landingPage), + fb("referrer", _.referrer), + ).mapN(PostClick.apply) + } + +/** Twenty same-flavoured flags — the filer's "Ivt alone has 20 fields" leg. */ +final case class Ivt( + ivtGeneralInvalid: Boolean, + ivtSophisticated: Boolean, + ivtBoth: Boolean, + ivtGenuine: Boolean, + ivtUndetermined: Boolean, + ivtScore: Int, + ivtTag1: Int, + ivtTag2: Int, + ivtTag3: Int, + ivtTag4: Int, + ivtTag5: Int, + ivtTag6: Int, + ivtTag7: Int, + ivtTag8: Int, + ivtTag9: Int, + ivtTag10: Int, + ivtFlags: Long, + ivtSchemaVersion: String, + ivtVendor: String, + ivtNote: String, +) + +object Ivt: + given VCodec[Ivt] = VCodec.record( + name = "Ivt", + namespace = "dev.constructive.eo.avro.vulcan", + ) { fb => + ( + fb("ivtGeneralInvalid", _.ivtGeneralInvalid), + fb("ivtSophisticated", _.ivtSophisticated), + fb("ivtBoth", _.ivtBoth), + fb("ivtGenuine", _.ivtGenuine), + fb("ivtUndetermined", _.ivtUndetermined), + fb("ivtScore", _.ivtScore), + fb("ivtTag1", _.ivtTag1), + fb("ivtTag2", _.ivtTag2), + fb("ivtTag3", _.ivtTag3), + fb("ivtTag4", _.ivtTag4), + fb("ivtTag5", _.ivtTag5), + fb("ivtTag6", _.ivtTag6), + fb("ivtTag7", _.ivtTag7), + fb("ivtTag8", _.ivtTag8), + fb("ivtTag9", _.ivtTag9), + fb("ivtTag10", _.ivtTag10), + fb("ivtFlags", _.ivtFlags), + fb("ivtSchemaVersion", _.ivtSchemaVersion), + fb("ivtVendor", _.ivtVendor), + fb("ivtNote", _.ivtNote), + ).mapN(Ivt.apply) + } + +final case class MavenEntities( + pixelHash: Array[Byte], + entityCount: Int, + sessionEntities: String, + mavenSegment: String, + taxonomyVersion: String, + viewability: Double, + engagement: Float, + mrcViewable: Boolean, + trafficClass: TrafficClass, +) + +object MavenEntities: + given VCodec[MavenEntities] = VCodec.record( + name = "MavenEntities", + namespace = "dev.constructive.eo.avro.vulcan", + ) { fb => + ( + fb("pixelHash", _.pixelHash), + fb("entityCount", _.entityCount), + fb("sessionEntities", _.sessionEntities), + fb("mavenSegment", _.mavenSegment), + fb("taxonomyVersion", _.taxonomyVersion), + fb("viewability", _.viewability), + fb("engagement", _.engagement), + fb("mrcViewable", _.mrcViewable), + fb("trafficClass", _.trafficClass), + ).mapN(MavenEntities.apply) + } + +/** The filer's ClickInfo shape, scaled to the fixtures: nested sub-records, a nullable sub-record, + * nullable primitives, a logical-type leaf (Instant) and enum / bytes leaves — every arm of the + * builder's dispatch table in one wire record. + */ +final case class ClickInfo( + clickId: String, + servedAt: Instant, + sessionId: String, + geo: Geo, + userAgent: UserAgentInfo, + postClick: Option[PostClick], + ivt: Ivt, + mavenEntities: MavenEntities, + valid: Boolean, + clickTimestamp: Option[Long], +) + +object ClickInfo: + given VCodec[ClickInfo] = VCodec.record( + name = "ClickInfo", + namespace = "dev.constructive.eo.avro.vulcan", + ) { fb => + ( + fb("clickId", _.clickId), + fb("servedAt", _.servedAt), + fb("sessionId", _.sessionId), + fb("geo", _.geo), + fb("userAgent", _.userAgent), + fb("postClick", _.postClick), + fb("ivt", _.ivt), + fb("mavenEntities", _.mavenEntities), + fb("valid", _.valid), + fb("clickTimestamp", _.clickTimestamp), + ).mapN(ClickInfo.apply) + } + +/** Codec whose field list is REORDERED against the case class — name resolution must map by NAME, + * never by declaration index (issue #105's doctrine, on the build side). + */ +final case class Reordered(alpha: Int, beta: String) + +object Reordered: + given VCodec[Reordered] = VCodec.record( + name = "Reordered", + namespace = "dev.constructive.eo.avro.vulcan", + ) { fb => + (fb("beta", _.beta), fb("alpha", _.alpha)).mapN((beta, alpha) => Reordered(alpha, beta)) + } + +/** Codec whose schema names are snake_cased against the case class — the normalised-name rung. */ +final case class Snakey(userId: Int, userName: String) + +object Snakey: + given VCodec[Snakey] = VCodec.record( + name = "Snakey", + namespace = "dev.constructive.eo.avro.vulcan", + ) { fb => + (fb("user_id", _.userId), fb("user_name", _.userName)).mapN(Snakey.apply) + } + +/** A schema-only computed column: `derived` has no case field; the codec fills it on encode, the + * builder leaves it at its in-record default, and the codec's decode ignores it. + */ +final case class WithComputed(base: Int) + +object WithComputed: + given VCodec[WithComputed] = VCodec.record( + name = "WithComputed", + namespace = "dev.constructive.eo.avro.vulcan", + ) { fb => + // `derived` is a schema-only column: nullable (the accessor's Option summons the union codec), + // so the builder's null default stays decodable and round-trips. + (fb("base", _.base), fb("derived", (c: WithComputed) => (Some(c.base * 2): Option[Int]))) + .mapN((base, _) => WithComputed(base)) + } + +/** A leaf type with NO vulcan codec in scope — the macro's missing-leaf refusal. */ +final class NoCodecLeaf(val s: String) + +final case class WithNoCodec(name: String, opaque: NoCodecLeaf) + +object WithNoCodec: + // The ROOT codec exists; the case-class field `opaque` has none — the macro must be the one to + // refuse it. + given VCodec[WithNoCodec] = VCodec.record( + name = "WithNoCodec", + namespace = "dev.constructive.eo.avro.vulcan", + ) { fb => + fb("name", _.name).map(n => WithNoCodec(n, NoCodecLeaf(""))) + } + +// ---- Construction-failure fixtures --------------------------------------------------------------- + +/** `beta`'s column is renamed `zeta` — beyond normalisation, so the builder must refuse. */ +final case class Renamed(alpha: Int, beta: String) + +object Renamed: + given VCodec[Renamed] = VCodec.record( + name = "Renamed", + namespace = "dev.constructive.eo.avro.vulcan", + ) { fb => + (fb("alpha", _.alpha), fb("zeta", _.beta)).mapN((alpha, _) => Renamed(alpha, "")) + } + +/** `Option` case field over a NON-nullable schema column — the codec lies about optionality. */ +final case class OptMismatch(x: Option[Int]) + +object OptMismatch: + given VCodec[OptMismatch] = VCodec.record( + name = "OptMismatch", + namespace = "dev.constructive.eo.avro.vulcan", + ) { fb => + fb("x", _.x.getOrElse(0)).map(x => OptMismatch(Some(x))) + } + +/** Case-class case field whose column is a STRING — the codec flattens it; no record to recurse. */ +final case class FlatInner(v: Int) + +object FlatInner: + given VCodec[FlatInner] = VCodec.record( + name = "FlatInner", + namespace = "dev.constructive.eo.avro.vulcan", + ) { fb => + fb("v", _.v).map(FlatInner.apply) + } + +final case class RecMis(inner: FlatInner) + +object RecMis: + given VCodec[RecMis] = VCodec.record( + name = "RecMis", + namespace = "dev.constructive.eo.avro.vulcan", + ) { fb => + fb("inner", _.inner.v.toString).map(s => RecMis(FlatInner(s.length))) + } + +/** LONG case field over an INT column — the builder never widens silently. */ +final case class LongField(x: Long) + +object LongField: + given VCodec[LongField] = VCodec.record( + name = "LongField", + namespace = "dev.constructive.eo.avro.vulcan", + ) { fb => + fb("x", _.x.toInt).map(x => LongField(x.toLong)) + } + +/** One schema column reachable by two normalised spellings — the ambiguity sentinel must refuse. */ +final case class Ambig(USERID: Int, other: String) + +object Ambig: + given VCodec[Ambig] = VCodec.record( + name = "Ambig", + namespace = "dev.constructive.eo.avro.vulcan", + ) { fb => + (fb("userId", _.USERID), fb("user_id", _.USERID), fb("other", _.other)) + .mapN((u, _, o) => Ambig(u, o)) + } + +/** Two case fields claiming ONE schema column — the injectivity half of all-or-nothing. */ +final case class Collide(aCol: Int, a_col: Int) + +object Collide: + given VCodec[Collide] = VCodec.record( + name = "Collide", + namespace = "dev.constructive.eo.avro.vulcan", + ) { fb => + (fb("a_col", _.aCol), fb("b", _.a_col)).mapN((a, _) => Collide(a, a)) + } + +/** Self-recursive shape for the runtime knot — derived via a HAND-AUTHORED shape (derive needs no + * codec, and vulcan's eager record codec cannot reference itself from a plain given). + */ +final case class Node(value: Int, next: Option[Node]) diff --git a/avro/src/test/scala/dev/constructive/eo/avro/vulcan/WholeRecordBuilderMacroErrorSpec.scala b/avro/src/test/scala/dev/constructive/eo/avro/vulcan/WholeRecordBuilderMacroErrorSpec.scala new file mode 100644 index 00000000..4fd2362f --- /dev/null +++ b/avro/src/test/scala/dev/constructive/eo/avro/vulcan/WholeRecordBuilderMacroErrorSpec.scala @@ -0,0 +1,32 @@ +package dev.constructive.eo.avro.vulcan + +import scala.compiletime.testing.typeCheckErrors + +import org.specs2.mutable.Specification + +/** Compile-error catalogue for `AvroVulcan.recordBuilder` — the two refusals the macro owns (the + * rest are construction-time, covered by [[WholeRecordBuilderSpec]]). Substring-matched like + * `FieldsMacroErrorSpec`, so message tweaks don't ripple. + */ +class WholeRecordBuilderMacroErrorSpec extends Specification: + + "a leaf field whose type has no vulcan codec names the field and the fallback rule" in { + val errs = typeCheckErrors( + "import dev.constructive.eo.avro.vulcan.*\n" + + "AvroVulcan.recordBuilder[WithNoCodec]" + ) + ( + errs.exists(_.message.contains("has no given vulcan.Codec")) + && errs.exists(_.message.contains("'opaque'")) + ) must beTrue + } + + "a non-case-class root refuses at compile time" in { + val errs = typeCheckErrors( + "import dev.constructive.eo.avro.vulcan.*\n" + + "AvroVulcan.recordBuilder[List[Int]]" + ) + errs.exists(_.message.contains("is not a case class")) must beTrue + } + + diff --git a/avro/src/test/scala/dev/constructive/eo/avro/vulcan/WholeRecordBuilderSpec.scala b/avro/src/test/scala/dev/constructive/eo/avro/vulcan/WholeRecordBuilderSpec.scala new file mode 100644 index 00000000..776adb01 --- /dev/null +++ b/avro/src/test/scala/dev/constructive/eo/avro/vulcan/WholeRecordBuilderSpec.scala @@ -0,0 +1,227 @@ +package dev.constructive.eo.avro.vulcan + +import java.time.Instant + +import scala.language.implicitConversions + +import _root_.vulcan.Codec as VCodec +import dev.constructive.eo.avro.{codecPrism, AvroCodec} +import org.apache.avro.Schema +import org.apache.avro.generic.GenericRecord +import org.specs2.mutable.Specification + +/** The derived whole-record builder's contract (issue #95): byte-for-byte fidelity with the full + * vulcan codec on the fields the case class holds, round-trips through the codec's decode, the + * name-resolution doctrine on the build side, the computed-column difference, the recursive-case + * knot, and every construction-time refusal — surfaced as the union's Exception half, never + * thrown. + */ +class WholeRecordBuilderSpec extends Specification: + + /** Unwrap a construction that the suite asserts SUCCEEDS (the refusal cases assert the other + * half below). + */ + private def built[A](r: Exception | WholeRecordBuilder[A]): WholeRecordBuilder[A] = r match + case b: WholeRecordBuilder[A] => b + case e: Exception => sys.error(e.getMessage) + + private val vraw = summon[VCodec[ClickInfo]] + private val builder = built(AvroVulcan.recordBuilder[ClickInfo]) + + private val rich = ClickInfo( + clickId = "ck_123", + servedAt = Instant.ofEpochMilli(1726000000000L), + sessionId = "s_abc", + geo = Geo("US", "NY", "New York", 40.71, -74.01), + userAgent = UserAgentInfo( + "Firefox", + "119.0", + "Linux", + "desktop", + "en-US", + doNotTrack = true, + robot = false, + ), + postClick = Some( + PostClick( + 1726000001000L, + Some("ord_9"), + 129.99, + "USD", + 3, + completed = true, + Some(7L), + "/lp", + "google", + ) + ), + ivt = Ivt( + true, false, false, true, false, 87, 1, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 42L, "v2", "acme", "clean", + ), + mavenEntities = MavenEntities( + Array[Byte](1, 2, 3), + 12, + "se1,se2", + "seg_a", + "tax7", + 0.87, + 0.5f, + true, + TrafficClass.Paid, + ), + valid = true, + clickTimestamp = Some(1726000000500L), + ) + + private val bare = rich.copy(postClick = None, clickTimestamp = None) + + "recordBuilder[ClickInfo]" should { + + "produce the codec's own record, field for field — deep equality incl. every nested record" in { + builder.toRecord(rich) must beEqualTo(vraw.encode(rich).toOption.get) + } + + "serialise to byte-identical wire bytes under the codec's schema" in { + val derived: Array[Byte] = + AvroCodec.encodeRecord(builder.toRecord(rich), builder.schema).toOption.get + val coded: Array[Byte] = + AvroCodec.encodeRecord(vraw.encode(rich).toOption.get, vraw.schema.toOption.get).toOption.get + derived.toSeq must beEqualTo(coded.toSeq) + } + + "round-trip through the codec's decode, Some and None variants" in { + vraw.decode(builder.toRecord(rich), builder.schema).toOption.get must beEqualTo(rich) + vraw.decode(builder.toRecord(bare), builder.schema).toOption.get must beEqualTo(bare) + } + + "leave the schema-only computed column at its default while the codec fills it — and still round-trip" in { + val b = built(AvroVulcan.recordBuilder[WithComputed]) + val c = summon[VCodec[WithComputed]] + val rec = b.toRecord(WithComputed(21)) + rec.get("derived") must beNull + c.encode(WithComputed(21)).toOption.get.asInstanceOf[GenericRecord].get("derived") must beEqualTo( + Int.box(42) + ) + c.decode(rec, c.schema.toOption.get).toOption.get must beEqualTo(WithComputed(21)) + } + + "resolve by NAME, never by declaration index: a reordered codec field list" in { + val b = built(AvroVulcan.recordBuilder[Reordered]) + val c = summon[VCodec[Reordered]] + val v = Reordered(7, "bee") + b.toRecord(v) must beEqualTo(c.encode(v).toOption.get) + } + + "resolve through the normalised-name rung: snake_case columns" in { + val b = built(AvroVulcan.recordBuilder[Snakey]) + val c = summon[VCodec[Snakey]] + val v = Snakey(9, "ada") + b.toRecord(v) must beEqualTo(c.encode(v).toOption.get) + } + + "install as the AvroCodec encode and drive codecPrism reads and writes" in { + given AvroCodec[ClickInfo] = builder.asAvroCodec + codecPrism[ClickInfo].record.reverseGet(rich) must beEqualTo(builder.toRecord(rich)) + val bytes = AvroCodec.encodeValue(rich).toOption.get + codecPrism[ClickInfo].getOption(bytes) must beSome(rich) + codecPrism[ClickInfo].field(_.sessionId).getOption(bytes) must beSome("s_abc") + } + } + + "a self-recursive shape" should { + + "terminate construction and build the chain through the level knot" in { + // create-then-setFields: the recursive schema's own object fills its `next` union — + // no self-referencing initialiser. + val nodeSchema: Schema = + val rec = Schema.createRecord( + "Node", + "recursive node fixture", + "dev.constructive.eo.avro.vulcan", + false, + ) + val next = + Schema.createUnion(Schema.create(Schema.Type.NULL), rec) + rec.setFields( + java.util.List.of( + new Schema.Field("value", Schema.create(Schema.Type.INT)), + new Schema.Field("next", next), + ) + ) + rec + val shape = WholeRecordBuilder.RecordShape( + "Node", + List( + WholeRecordBuilder.FieldShape( + "value", + WholeRecordBuilder.DirectKind(Schema.Type.INT), + ), + WholeRecordBuilder.FieldShape( + "next", + WholeRecordBuilder.OptionKind(WholeRecordBuilder.SelfKind(0)), + ), + ), + ) + WholeRecordBuilder.derive[Node](nodeSchema, shape, "spec[Node]") match + case e: Exception => sys.error(e.getMessage) + case b: WholeRecordBuilder[Node] => + val rec = b.toRecord(Node(1, Some(Node(2, Some(Node(3, None)))))) + rec.get("value") must beEqualTo(Int.box(1)) + val n2 = rec.get("next").asInstanceOf[GenericRecord] + n2.get("value") must beEqualTo(Int.box(2)) + val n3 = n2.get("next").asInstanceOf[GenericRecord] + n3.get("value") must beEqualTo(Int.box(3)) + n3.get("next") must beNull + } + } + + "construction-time refusal" should { + + "refuse a case field renamed beyond normalisation, naming field and record" in { + AvroVulcan.recordBuilder[Renamed] match + case e: IllegalArgumentException => + e.getMessage must (contain("'beta'") and contain("does not name a schema field")) + case e: Exception => ko(e.getMessage) + case _: WholeRecordBuilder[Renamed] => ko("expected a construction refusal") + } + + "refuse an Option case field over a non-nullable column" in { + AvroVulcan.recordBuilder[OptMismatch] match + case e: IllegalArgumentException => e.getMessage must contain("not a null-union") + case e: Exception => ko(e.getMessage) + case _: WholeRecordBuilder[OptMismatch] => ko("expected a construction refusal") + } + + "refuse a case-class field whose column is not a record" in { + AvroVulcan.recordBuilder[RecMis] match + case e: IllegalArgumentException => + e.getMessage must (contain("needs a RECORD schema field") and contain("is a STRING")) + case e: Exception => ko(e.getMessage) + case _: WholeRecordBuilder[RecMis] => ko("expected a construction refusal") + } + + "refuse a LONG case field over an INT column (never widens silently)" in { + AvroVulcan.recordBuilder[LongField] match + case e: IllegalArgumentException => + e.getMessage must (contain("needs a LONG schema field") and contain("is a INT")) + case e: Exception => ko(e.getMessage) + case _: WholeRecordBuilder[LongField] => ko("expected a construction refusal") + } + + "refuse a column ambiguous under normalisation (the ambiguity sentinel)" in { + AvroVulcan.recordBuilder[Ambig] match + case e: IllegalArgumentException => + e.getMessage must contain("matches more than one schema field") + case e: Exception => ko(e.getMessage) + case _: WholeRecordBuilder[Ambig] => ko("expected a construction refusal") + } + + "refuse two case fields claiming one column (the injectivity half)" in { + AvroVulcan.recordBuilder[Collide] match + case e: IllegalArgumentException => + e.getMessage must contain("collides with case field 'aCol'") + case e: Exception => ko(e.getMessage) + case _: WholeRecordBuilder[Collide] => ko("expected a construction refusal") + } + } diff --git a/benchmarks/src/main/scala/dev/constructive/eo/bench/AvroVulcanBench.scala b/benchmarks/src/main/scala/dev/constructive/eo/bench/AvroVulcanBench.scala index df12d7c2..335840c5 100644 --- a/benchmarks/src/main/scala/dev/constructive/eo/bench/AvroVulcanBench.scala +++ b/benchmarks/src/main/scala/dev/constructive/eo/bench/AvroVulcanBench.scala @@ -50,7 +50,10 @@ object AvroVulcanImpls: } val nativeCodec: AvroCodec[Hit] = summon[AvroCodec[Hit]] - val bridgedCodec: AvroCodec[Hit] = AvroVulcan.codec[Hit](using vulcanCodec) + + val bridgedCodec: AvroCodec[Hit] = + AvroVulcan.codec[Hit](using vulcanCodec).fold(e => throw e, identity) + val vulcanSchema: Schema = bridgedCodec.schema val hit: Hit = Hit("ada", 42L, active = true) diff --git a/benchmarks/src/main/scala/dev/constructive/eo/bench/ClickRecordBench.scala b/benchmarks/src/main/scala/dev/constructive/eo/bench/ClickRecordBench.scala new file mode 100644 index 00000000..12be0e2d --- /dev/null +++ b/benchmarks/src/main/scala/dev/constructive/eo/bench/ClickRecordBench.scala @@ -0,0 +1,478 @@ +package dev.constructive.eo +package bench + +import java.util.concurrent.TimeUnit + +import cats.syntax.all.* + +import org.openjdk.jmh.annotations.* + +import _root_.vulcan.Codec as VCodec + +import avro.AvroCodec +import avro.codecPrism +import avro.vulcan.{AvroVulcan, WholeRecordBuilder} +import org.apache.avro.generic.GenericData + +/** Whole-record encode shoot-out on the issue-#95 filer's production shape: a ~66-leaf ClickInfo + * whose leaves live in 5 nested sub-records (Geo 8, UserAgentInfo 9, PostClick 11, Ivt 20, + * MavenEntities 10) plus 8 root primitives — every leaf a primitive or nullable primitive, so the + * arms differ ONLY in mechanism: + * + * - '''vulcanFull''' — the full `vulcan.Codec[ClickInfo].encode` (the pre-#95 baseline): pays + * vulcan's per-field composition (FreeApplicative analyze, Either + Chain per field, put-by-name + * hash probe) once per field PER LEVEL. + * - '''positional''' — the whole-record builder over held per-field leaf codecs the filer + * implemented and REJECTED: positional puts of `leafCodec.encode(value)`, nested sub-records + * through `subCodec.encode` — strips ONE level of composition (the outermost shell) and keeps + * paying the rest, which is what measured 7.5x time / 16.9x allocation on their real shape. + * - '''hand''' — the leaf-by-leaf `.put(pos, value)` builder the filer hand-maintains today: one + * line per leaf across every nesting level. + * - '''derived''' — `AvroVulcan.recordBuilder[ClickInfo]`: the compile-time-derived builder — the + * hand-built shape with no hand-maintained lines (recursive sub-record levels, primitive bulk + * as positional puts). + * - '''derivedThroughPrism''' — `codecPrism[ClickInfo].record.reverseGet` over the derived codec + * (`asAvroCodec`): the eo-idiomatic surface at the call site, which is how the filer wires it. + * + * '''B/op (`-prof gc`) is the gate; ns/op advises''' (project doctrine, and these boxes are noisy). + * The gate the filer set: derived ≈ hand (both put raw values; String-emitting, Utf8 materialises + * at serialise time), positional ≈ vulcan-per-sub-record. + */ +object ClickRecordImpls: + + final case class Geo( + country: String, + region: String, + city: String, + postalCode: String, + latitude: Double, + longitude: Double, + accuracyM: Long, + geoHash: String, + ) + + object Geo: + given VCodec[Geo] = VCodec.record(name = "Geo", namespace = "dev.constructive.eo.bench") { fb => + ( + fb("country", _.country), + fb("region", _.region), + fb("city", _.city), + fb("postalCode", _.postalCode), + fb("latitude", _.latitude), + fb("longitude", _.longitude), + fb("accuracyM", _.accuracyM), + fb("geoHash", _.geoHash), + ).mapN(Geo.apply) + } + + final case class UserAgentInfo( + browser: String, + browserVersion: String, + os: String, + osVersion: String, + device: String, + language: String, + userAgentRaw: String, + doNotTrack: Boolean, + robot: Boolean, + ) + + object UserAgentInfo: + given VCodec[UserAgentInfo] = VCodec.record( + name = "UserAgentInfo", + namespace = "dev.constructive.eo.bench", + ) { fb => + ( + fb("browser", _.browser), + fb("browserVersion", _.browserVersion), + fb("os", _.os), + fb("osVersion", _.osVersion), + fb("device", _.device), + fb("language", _.language), + fb("userAgentRaw", _.userAgentRaw), + fb("doNotTrack", _.doNotTrack), + fb("robot", _.robot), + ).mapN(UserAgentInfo.apply) + } + + final case class PostClick( + conversionTimestamp: Long, + orderId: Option[String], + revenue: Double, + currency: String, + funnelStep: Int, + completed: Boolean, + attributionWindow: Option[Long], + landingPage: String, + referrer: String, + campaignId: String, + channel: String, + ) + + object PostClick: + given VCodec[PostClick] = VCodec.record( + name = "PostClick", + namespace = "dev.constructive.eo.bench", + ) { fb => + ( + fb("conversionTimestamp", _.conversionTimestamp), + fb("orderId", _.orderId), + fb("revenue", _.revenue), + fb("currency", _.currency), + fb("funnelStep", _.funnelStep), + fb("completed", _.completed), + fb("attributionWindow", _.attributionWindow), + fb("landingPage", _.landingPage), + fb("referrer", _.referrer), + fb("campaignId", _.campaignId), + fb("channel", _.channel), + ).mapN(PostClick.apply) + } + + final case class Ivt( + ivtGeneralInvalid: Boolean, + ivtSophisticated: Boolean, + ivtBoth: Boolean, + ivtGenuine: Boolean, + ivtUndetermined: Boolean, + ivtScore: Int, + ivtTag1: Int, + ivtTag2: Int, + ivtTag3: Int, + ivtTag4: Int, + ivtTag5: Int, + ivtTag6: Int, + ivtTag7: Int, + ivtTag8: Int, + ivtTag9: Int, + ivtTag10: Int, + ivtFlags: Long, + ivtSchemaVersion: String, + ivtVendor: String, + ivtNote: String, + ) + + object Ivt: + given VCodec[Ivt] = VCodec.record( + name = "Ivt", + namespace = "dev.constructive.eo.bench", + ) { fb => + ( + fb("ivtGeneralInvalid", _.ivtGeneralInvalid), + fb("ivtSophisticated", _.ivtSophisticated), + fb("ivtBoth", _.ivtBoth), + fb("ivtGenuine", _.ivtGenuine), + fb("ivtUndetermined", _.ivtUndetermined), + fb("ivtScore", _.ivtScore), + fb("ivtTag1", _.ivtTag1), + fb("ivtTag2", _.ivtTag2), + fb("ivtTag3", _.ivtTag3), + fb("ivtTag4", _.ivtTag4), + fb("ivtTag5", _.ivtTag5), + fb("ivtTag6", _.ivtTag6), + fb("ivtTag7", _.ivtTag7), + fb("ivtTag8", _.ivtTag8), + fb("ivtTag9", _.ivtTag9), + fb("ivtTag10", _.ivtTag10), + fb("ivtFlags", _.ivtFlags), + fb("ivtSchemaVersion", _.ivtSchemaVersion), + fb("ivtVendor", _.ivtVendor), + fb("ivtNote", _.ivtNote), + ).mapN(Ivt.apply) + } + + final case class MavenEntities( + entityCount: Int, + sessionEntities: String, + mavenSegment: String, + taxonomyVersion: String, + audienceId: String, + viewability: Double, + engagement: Float, + mrcViewable: Boolean, + dwellMs: Long, + qualified: Boolean, + ) + + object MavenEntities: + given VCodec[MavenEntities] = VCodec.record( + name = "MavenEntities", + namespace = "dev.constructive.eo.bench", + ) { fb => + ( + fb("entityCount", _.entityCount), + fb("sessionEntities", _.sessionEntities), + fb("mavenSegment", _.mavenSegment), + fb("taxonomyVersion", _.taxonomyVersion), + fb("audienceId", _.audienceId), + fb("viewability", _.viewability), + fb("engagement", _.engagement), + fb("mrcViewable", _.mrcViewable), + fb("dwellMs", _.dwellMs), + fb("qualified", _.qualified), + ).mapN(MavenEntities.apply) + } + + final case class ClickInfo( + clickId: String, + servedAt: Long, + sessionId: String, + geo: Geo, + userAgent: UserAgentInfo, + postClick: PostClick, + ivt: Ivt, + mavenEntities: MavenEntities, + valid: Boolean, + clickTimestamp: Option[Long], + sourceIp: String, + ttlMs: Long, + retained: Boolean, + ) + + object ClickInfo: + given VCodec[ClickInfo] = VCodec.record( + name = "ClickInfo", + namespace = "dev.constructive.eo.bench", + ) { fb => + ( + fb("clickId", _.clickId), + fb("servedAt", _.servedAt), + fb("sessionId", _.sessionId), + fb("geo", _.geo), + fb("userAgent", _.userAgent), + fb("postClick", _.postClick), + fb("ivt", _.ivt), + fb("mavenEntities", _.mavenEntities), + fb("valid", _.valid), + fb("clickTimestamp", _.clickTimestamp), + fb("sourceIp", _.sourceIp), + fb("ttlMs", _.ttlMs), + fb("retained", _.retained), + ).mapN(ClickInfo.apply) + } + + val click = ClickInfo( + clickId = "ck_9f2b7c31", + servedAt = 1726000000123L, + sessionId = "s_8814d0aa", + geo = Geo("US", "NY", "New York", "10001", 40.7128, -74.006, 25L, "dr5regw3"), + userAgent = UserAgentInfo( + "Firefox", + "119.0", + "Linux", + "6.6.0", + "desktop", + "en-US", + "Mozilla/5.0 (X11; Linux x86_64) Gecko/20100101 Firefox/119.0", + doNotTrack = true, + robot = false, + ), + postClick = PostClick( + 1726000001456L, + Some("ord_4417"), + 129.99, + "USD", + 3, + completed = true, + Some(7L), + "/lp/spring", + "google", + "cmp_2281", + "cpc", + ), + ivt = Ivt( + false, false, false, true, false, 87, 1, 0, 2, 0, + 0, 1, 0, 0, 3, 0, 42L, "v2", "acme", "clean", + ), + mavenEntities = MavenEntities( + 12, + "se1,se2,se3", + "seg_a", + "tax7", + "aud_991", + 0.87, + 0.5f, + true, + 3400L, + true, + ), + valid = true, + clickTimestamp = Some(1726000000500L), + sourceIp = "203.0.113.9", + ttlMs = 86400000L, + retained = true, + ) + + val schema = summon[VCodec[ClickInfo]].schema.toOption.get + + /** The filer's REJECTED prototype: every case field's leaf codec resolved once at init, encode = + * N positional puts of `codec.encode(value)`, nested sub-records through `subCodec.encode` — one + * level of composition stripped, the rest paid. + */ + final class PositionalBuilder(schema: org.apache.avro.Schema): + private val cString = VCodec.string + private val cLong = VCodec.long + private val cBool = VCodec.boolean + private val cGeo = summon[VCodec[Geo]] + private val cUa = summon[VCodec[UserAgentInfo]] + private val cPc = summon[VCodec[PostClick]] + private val cIvt = summon[VCodec[Ivt]] + private val cMaven = summon[VCodec[MavenEntities]] + + private def enc[A](c: VCodec[A], a: A): Any = c.encode(a).fold(e => throw e.throwable, identity) + + def toRecord(a: ClickInfo): GenericData.Record = + val r = new GenericData.Record(schema) + r.put(0, enc(cString, a.clickId)) + r.put(1, enc(cLong, a.servedAt)) + r.put(2, enc(cString, a.sessionId)) + r.put(3, enc(cGeo, a.geo)) + r.put(4, enc(cUa, a.userAgent)) + r.put(5, enc(cPc, a.postClick)) + r.put(6, enc(cIvt, a.ivt)) + r.put(7, enc(cMaven, a.mavenEntities)) + r.put(8, enc(cBool, a.valid)) + a.clickTimestamp match + case Some(ts) => r.put(9, enc(cLong, ts)) + case None => r.put(9, null) + r.put(10, enc(cString, a.sourceIp)) + r.put(11, enc(cLong, a.ttlMs)) + r.put(12, enc(cBool, a.retained)) + r + + /** The hand-maintained builder: one line per leaf, every level, raw values. */ + final class HandBuilder(schema: org.apache.avro.Schema): + def toRecord(a: ClickInfo): GenericData.Record = + val r = new GenericData.Record(schema) + r.put(0, a.clickId) + r.put(1, a.servedAt) + r.put(2, a.sessionId) + r.put(3, geoRecord(a.geo)) + r.put(4, uaRecord(a.userAgent)) + r.put(5, postClickRecord(a.postClick)) + r.put(6, ivtRecord(a.ivt)) + r.put(7, mavenRecord(a.mavenEntities)) + r.put(8, a.valid) + a.clickTimestamp match + case Some(ts) => r.put(9, ts) + case None => r.put(9, null) + r.put(10, a.sourceIp) + r.put(11, a.ttlMs) + r.put(12, a.retained) + r + + private def geoRecord(g: Geo): GenericData.Record = + val r = new GenericData.Record(summon[VCodec[Geo]].schema.toOption.get) + r.put(0, g.country) + r.put(1, g.region) + r.put(2, g.city) + r.put(3, g.postalCode) + r.put(4, g.latitude) + r.put(5, g.longitude) + r.put(6, g.accuracyM) + r.put(7, g.geoHash) + r + + private def uaRecord(u: UserAgentInfo): GenericData.Record = + val r = new GenericData.Record(summon[VCodec[UserAgentInfo]].schema.toOption.get) + r.put(0, u.browser) + r.put(1, u.browserVersion) + r.put(2, u.os) + r.put(3, u.osVersion) + r.put(4, u.device) + r.put(5, u.language) + r.put(6, u.userAgentRaw) + r.put(7, u.doNotTrack) + r.put(8, u.robot) + r + + private def postClickRecord(p: PostClick): GenericData.Record = + val r = new GenericData.Record(summon[VCodec[PostClick]].schema.toOption.get) + r.put(0, p.conversionTimestamp) + p.orderId match + case Some(o) => r.put(1, o) + case None => r.put(1, null) + r.put(2, p.revenue) + r.put(3, p.currency) + r.put(4, p.funnelStep) + r.put(5, p.completed) + p.attributionWindow match + case Some(w) => r.put(6, w) + case None => r.put(6, null) + r.put(7, p.landingPage) + r.put(8, p.referrer) + r.put(9, p.campaignId) + r.put(10, p.channel) + r + + private def ivtRecord(i: Ivt): GenericData.Record = + val r = new GenericData.Record(summon[VCodec[Ivt]].schema.toOption.get) + r.put(0, i.ivtGeneralInvalid) + r.put(1, i.ivtSophisticated) + r.put(2, i.ivtBoth) + r.put(3, i.ivtGenuine) + r.put(4, i.ivtUndetermined) + r.put(5, i.ivtScore) + r.put(6, i.ivtTag1) + r.put(7, i.ivtTag2) + r.put(8, i.ivtTag3) + r.put(9, i.ivtTag4) + r.put(10, i.ivtTag5) + r.put(11, i.ivtTag6) + r.put(12, i.ivtTag7) + r.put(13, i.ivtTag8) + r.put(14, i.ivtTag9) + r.put(15, i.ivtTag10) + r.put(16, i.ivtFlags) + r.put(17, i.ivtSchemaVersion) + r.put(18, i.ivtVendor) + r.put(19, i.ivtNote) + r + + private def mavenRecord(m: MavenEntities): GenericData.Record = + val r = new GenericData.Record(summon[VCodec[MavenEntities]].schema.toOption.get) + r.put(0, m.entityCount) + r.put(1, m.sessionEntities) + r.put(2, m.mavenSegment) + r.put(3, m.taxonomyVersion) + r.put(4, m.audienceId) + r.put(5, m.viewability) + r.put(6, m.engagement) + r.put(7, m.mrcViewable) + r.put(8, m.dwellMs) + r.put(9, m.qualified) + r + + val vulcanCodec: VCodec[ClickInfo] = summon[VCodec[ClickInfo]] + val positional: PositionalBuilder = PositionalBuilder(schema) + val hand: HandBuilder = HandBuilder(schema) + + /** The derived builder (construction unwrapped — the once-cost is not the hot path). */ + val derived: WholeRecordBuilder[ClickInfo] = + AvroVulcan.recordBuilder[ClickInfo] match + case b: WholeRecordBuilder[ClickInfo] => b + case e: Exception => throw e + + /** The derived builder installed as the AvroCodec encode — the filer's wiring. */ + val derivedCodec: AvroCodec[ClickInfo] = derived.asAvroCodec + val derivedRoot = codecPrism[ClickInfo](using derivedCodec) + +@State(Scope.Benchmark) +@BenchmarkMode(Array(Mode.AverageTime)) +@OutputTimeUnit(TimeUnit.NANOSECONDS) +@Fork(3) +@Warmup(iterations = 3, time = 1) +@Measurement(iterations = 5, time = 1) +class ClickRecordBench extends JmhDefaults: + + import ClickRecordImpls.* + + @Benchmark def encode_vulcanFull: Any = vulcanCodec.encode(click) + + @Benchmark def encode_positional: GenericData.Record = positional.toRecord(click) + + @Benchmark def encode_hand: GenericData.Record = hand.toRecord(click) + + @Benchmark def encode_derived: GenericData.Record = derived.toRecord(click) + + @Benchmark def encode_derivedThroughPrism: Any = derivedRoot.record.reverseGet(click) diff --git a/site/docs/integrations/avro.md b/site/docs/integrations/avro.md index c137da0e..a65681f0 100644 --- a/site/docs/integrations/avro.md +++ b/site/docs/integrations/avro.md @@ -822,13 +822,81 @@ import dev.constructive.eo.avro.vulcan.given val countL = codecPrism[ClickInfo].field(_.count) ``` -or, named and explicit, -`given AvroCodec[ClickInfo] = AvroVulcan.codec`. The schema is -resolved once at construction (an invalid vulcan schema fails at -the `given` site, not on the first record); encode errors throw -(eo's `encode` is total — an encode failure under a matching -schema is a codec-definition bug); decode errors surface as -`Left` like every other `AvroCodec`. +The bridge comes in two shapes. `AvroVulcan.codec(schema)` is +total — the schema is already in hand, nothing is resolved. The +given-backed `AvroVulcan.codec[A]` resolves the schema from the +codec and can fail, so it returns +`Either[Exception, AvroCodec[A]]`; the opt-in given is the one +site with no failure channel (a summon must produce a codec), so +a schema that will not resolve fails eagerly at the given site. +Encode errors throw (eo's `encode` is total — an encode failure +under a matching schema is a codec-definition bug); decode +errors surface as `Left` like every other `AvroCodec`. + +### The whole-record builder — `AvroVulcan.recordBuilder` + +Building a fresh `GenericData.Record` from a typed value on a +hot path used to be a choice between two costs: the full +`codec[A].encode` — vulcan's per-field composition, per level, +measured at ~384–468 B/field on wide records — or a hand-built +`.put(pos, value)` builder, fast but one hand-maintained line +per leaf (issue #95). `recordBuilder[A]` derives the hand-built +shape at compile time: + +```scala +val clickBuilder = AvroVulcan.recordBuilder[ClickInfo] +// Exception | WholeRecordBuilder[ClickInfo] — construction is +// total. A case field no schema column answers for, two case +// fields claiming one column, or an arm that disagrees with its +// schema field's shape comes back as the Exception half, +// naming the field and the record, before any record is built. + +given AvroCodec[ClickInfo] = + clickBuilder.fold(e => throw e, _.asAvroCodec) +``` + +`toRecord` is pure positional puts. Every case field's schema +slot is resolved by NAME at construction — all-or-nothing, the +same doctrine the drill-down optics resolve with — so the hot +path never touches a hash lookup. Per field, classified from +the case class at expansion: + +- Boolean / Int / Long / Float / Double / String — the value + itself is the Avro datum; +- a nested case class — a sub-record level built by the same + rule against the field's own record schema (self-recursive + case classes resolve through the runtime level chain, so + they terminate); +- `Option[X]` — `None` puts null exactly as vulcan's own + `OptionCodec`; `Some(v)` recurses; +- everything else — enums, bytes, logical types, collections, + sums, value classes — the field type's own `vulcan.Codec`, + summoned at the derivation site; a missing leaf codec is a + compile error pointing at the field. + +The recursion is the point. A positional builder over held +per-field leaf codecs strips only the OUTERMOST shell of +vulcan's composition and keeps paying it inside every nested +`subCodec.encode` — measured on the issue's real ClickInfo as a +7.5x time / 16.9x allocation regression against the hand-built +builder, because most of its ~50 leaf fields live in five +nested sub-records. The derived builder expands every nested +case class the same way a hand-built one would, so its +allocation is hand-built-equal — `benchmarks`' +`ClickRecordBench` measures all the arms side by side, and +B/op is the gate. + +Two documented differences from `codec[A].encode`. A +schema-only column (a computed/derived field your case class +does not hold) keeps its in-record default instead of being +computed — identical to what a hand-built `.put` builder does, +and round-trip-safe through the codec's own decode; a REQUIRED +schema-only column then fails at write time, which is the same +contract the hand-built builder has. And the builder requires +every case field to name a schema column (exact, or uniquely up +to `_`/`-`/`.`/case): a codec that renames or drops a case +field has no silent positional fallback here by design — keep +the hand-built builder (or the codec) for that type. vulcan is an `Optional` dependency of `cats-eo-avro` — add it to your own build to use this sub-package; avro-only users never From 10b39046f3e6c7322be657915be259ed06665d8d Mon Sep 17 00:00:00 2001 From: Rodolfo Hansen Date: Tue, 22 Sep 2026 21:38:40 +0200 Subject: [PATCH 2/6] =?UTF-8?q?style(avro):=20scalafmtAll=20over=20the=20b?= =?UTF-8?q?uilder=20sources=20=E2=80=94=20the=20scope=20the=20manual=20gat?= =?UTF-8?q?e=20missed?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit CI (PR #116) caught what my per-module 'scalafmt' run didn't cover: the Test-config sources and parts of the macro file were unformatted. Full 'scalafmtAll ; scalafmtCheckAll' is green; avroIntegration/test re-run: 235 passed / 0 failed. --- .../eo/avro/vulcan/RecordBuilderMacro.scala | 40 ++++++---- .../eo/avro/vulcan/WholeRecordBuilder.scala | 40 +++++----- .../vulcan/WholeRecordBuilderFixtures.scala | 45 ++++++++--- .../WholeRecordBuilderMacroErrorSpec.scala | 2 - .../avro/vulcan/WholeRecordBuilderSpec.scala | 76 +++++++++++++------ 5 files changed, 134 insertions(+), 69 deletions(-) diff --git a/avro/src/main/scala/dev/constructive/eo/avro/vulcan/RecordBuilderMacro.scala b/avro/src/main/scala/dev/constructive/eo/avro/vulcan/RecordBuilderMacro.scala index 50de702a..466d45a3 100644 --- a/avro/src/main/scala/dev/constructive/eo/avro/vulcan/RecordBuilderMacro.scala +++ b/avro/src/main/scala/dev/constructive/eo/avro/vulcan/RecordBuilderMacro.scala @@ -32,9 +32,9 @@ import org.apache.avro.Schema * codec is a compile error pointing at the exact field. * * Because the emitted value is plain data, everything schema-dependent (slot resolution, arm - * validation) happens at builder construction in [[WholeRecordBuilder.derive]] — ordinary - * testable Scala, no staged code. The whole classification lives in [[builderImpl]] as local defs - * under ONE `Quotes`: `TypeRepr` / `Symbol` are path-dependent on the Quotes instance, so a + * validation) happens at builder construction in [[WholeRecordBuilder.derive]] — ordinary testable + * Scala, no staged code. The whole classification lives in [[builderImpl]] as local defs under ONE + * `Quotes`: `TypeRepr` / `Symbol` are path-dependent on the Quotes instance, so a * `(using Quotes)`-taking helper called from inside a quote would type against a different path. */ object RecordBuilderMacro: @@ -49,7 +49,9 @@ object RecordBuilderMacro: /** Entry: `AvroVulcan.recordBuilder[A]`. Requires a case class `A` (sums encode through their * codec, not a builder) and the in-scope `vulcan.Codec[A]` whose schema the builder writes. */ - def builderImpl[A: Type](codec: Expr[VCodec[A]])(using Quotes): Expr[Exception | WholeRecordBuilder[A]] = + def builderImpl[A: Type](codec: Expr[VCodec[A]])(using + Quotes + ): Expr[Exception | WholeRecordBuilder[A]] = import quotes.reflect.* def recordShapeOf( @@ -89,7 +91,11 @@ object RecordBuilderMacro: if isCaseClass(tt) then ancestors.indexOf(tsym) match case -1 => - '{ RecordKind(${ recordShapeOf(tt, s"$who → $name", tsym :: ancestors, depth + 1) }) } + '{ + RecordKind(${ + recordShapeOf(tt, s"$who → $name", tsym :: ancestors, depth + 1) + }) + } case d => '{ SelfKind(${ Expr(d) }) } else summonLeafCodec(name, tt, who) case st => '{ DirectKind(${ schemaTypeExpr(st) }) } @@ -116,7 +122,7 @@ object RecordBuilderMacro: case Schema.Type.FLOAT => '{ Schema.Type.FLOAT } case Schema.Type.DOUBLE => '{ Schema.Type.DOUBLE } case Schema.Type.STRING => '{ Schema.Type.STRING } - case other => + case other => report.errorAndAbort( s"RecordBuilderMacro: internal — unhandled direct schema type $other" ) @@ -124,14 +130,14 @@ object RecordBuilderMacro: def isCaseClass(t: TypeRepr): Boolean = val sym = t.typeSymbol sym.isClassDef && sym.flags.is(Flags.Case) && !sym.flags.is(Flags.Sealed) - && !sym.flags.is(Flags.Module) && !(t <:< TypeRepr.of[AnyVal]) + && !sym.flags.is(Flags.Module) && !(t <:< TypeRepr.of[AnyVal]) def summonLeafCodec(name: String, t: TypeRepr, who: String): Expr[Kind] = t.asType match case '[x] => Expr.summon[VCodec[x]] match case Some(codecE) => '{ CodecKind(${ codecE }.asInstanceOf[VCodec[Any]]) } - case None => + case None => report.errorAndAbort( s"$who: field '$name' of type ${Type.show[x]} has no given vulcan.Codec in scope." + " The builder fast-paths Boolean/Int/Long/Float/Double/String leaves and nested" @@ -151,14 +157,16 @@ object RecordBuilderMacro: // Total: the codec's schema failure and the assembly failure both come back as the union's // Exception half (the vulcan error wrapped, its own throwable as the cause). '{ - ${ codec }.schema.fold( - e => - IllegalArgumentException( - ${ Expr(who) } + ": the vulcan codec's schema did not resolve", - e.throwable, - ), - schema => WholeRecordBuilder.derive[A](schema, $shape, ${ Expr(who) }), - ) + ${ codec } + .schema + .fold( + e => + IllegalArgumentException( + ${ Expr(who) } + ": the vulcan codec's schema did not resolve", + e.throwable, + ), + schema => WholeRecordBuilder.derive[A](schema, $shape, ${ Expr(who) }), + ) }.asExprOf[Exception | WholeRecordBuilder[A]] end RecordBuilderMacro diff --git a/avro/src/main/scala/dev/constructive/eo/avro/vulcan/WholeRecordBuilder.scala b/avro/src/main/scala/dev/constructive/eo/avro/vulcan/WholeRecordBuilder.scala index c43384d7..9243827b 100644 --- a/avro/src/main/scala/dev/constructive/eo/avro/vulcan/WholeRecordBuilder.scala +++ b/avro/src/main/scala/dev/constructive/eo/avro/vulcan/WholeRecordBuilder.scala @@ -16,11 +16,11 @@ import org.apache.avro.generic.GenericData * FIELD, per LEVEL: a `FreeApplicative.analyze`, an `Either` + `Chain.one` per field, and a * `put(name, value)` hash probe — and a nested sub-record field redoes all of it inside * `Codec[Sub].encode`, which is what the filer measured as ~384–468 B/field on their real - * ClickInfo. A hand-built `.put(pos, value)` builder avoids all of it but costs one hand-maintained - * line per leaf — the exact complaint the filer opened the issue with. + * ClickInfo. A hand-built `.put(pos, value)` builder avoids all of it but costs one + * hand-maintained line per leaf — the exact complaint the filer opened the issue with. * - * '''What a derived builder is.''' [[AvroVulcan.recordBuilder]] walks `A`'s case fields at - * COMPILE time (the [[WholeRecordBuilder.RecordShape]] IR) and emits one runtime assembly call; + * '''What a derived builder is.''' [[AvroVulcan.recordBuilder]] walks `A`'s case fields at COMPILE + * time (the [[WholeRecordBuilder.RecordShape]] IR) and emits one runtime assembly call; * construction resolves every case field's schema slot by NAME (all-or-nothing, issue #105's * doctrine) and validates every arm against the schema it will write into — so `toRecord` itself * is nothing but positional puts: `new GenericData.Record(schema)`, then per field either the @@ -133,7 +133,11 @@ object WholeRecordBuilder: * case field names no schema column, two case fields claim one, or an arm disagrees with its * schema field's shape. Never throws. */ - def derive[A](schema: Schema, shape: RecordShape, who: String): Exception | WholeRecordBuilder[A] = + def derive[A]( + schema: Schema, + shape: RecordShape, + who: String + ): Exception | WholeRecordBuilder[A] = if schema.getType != Schema.Type.RECORD then IllegalArgumentException( s"$who: the codec's schema is a ${schema.getType}, not a record — the builder mirrors a case" @@ -148,7 +152,7 @@ object WholeRecordBuilder: * `SelfKind` arm captures the level being built), then sealed — `buildLevel` seals only on * success, so `build` never sees the empty arrays. */ - private[avro] final class RecordLevel( + final private[avro] class RecordLevel( val schema: Schema, val parent: RecordLevel | Null, ): @@ -181,36 +185,38 @@ object WholeRecordBuilder: u.plan.put(r, product.productElement(u.decl)) putUnusual(product, r, i + 1) - private[avro] final case class DirectSlot(slot: Int, decl: Int) - private[avro] final case class UnusualSlot(decl: Int, plan: FieldPlan) + final private[avro] case class DirectSlot(slot: Int, decl: Int) + final private[avro] case class UnusualSlot(decl: Int, plan: FieldPlan) /** One non-primitive arm of a record level: how `toRecord` turns the field's value into the datum * it puts at `slot`. `put` receives the value as erased `Any` (the case field's value, already * boxed by `productElement`), so a level dispatches without per-type closures. */ - private[avro] sealed trait FieldPlan: + sealed private[avro] trait FieldPlan: def slot: Int def put(r: GenericData.Record, value: Any): Unit - private[avro] final case class DirectPlan(slot: Int) extends FieldPlan: + final private[avro] case class DirectPlan(slot: Int) extends FieldPlan: def put(r: GenericData.Record, value: Any): Unit = r.put(slot, value) /** `None` → null (vulcan's `OptionCodec`), `Some(v)` → the inner arm. The option field owns ONE * schema slot; both branches put there. */ - private[avro] final case class OptionPlan(slot: Int, inner: FieldPlan) extends FieldPlan: + final private[avro] case class OptionPlan(slot: Int, inner: FieldPlan) extends FieldPlan: + def put(r: GenericData.Record, value: Any): Unit = value match case None => r.put(slot, null) case Some(v) => inner.put(r, v) - private[avro] final case class SubRecordPlan(slot: Int, sub: RecordLevel) extends FieldPlan: + final private[avro] case class SubRecordPlan(slot: Int, sub: RecordLevel) extends FieldPlan: def put(r: GenericData.Record, value: Any): Unit = r.put(slot, sub.build(value)) /** The field type's own codec — resolved once at construction; encode errors throw (eo's total * encode convention, matching [[AvroVulcan.codec]]). */ - private[avro] final case class CodecPlan(slot: Int, codec: VCodec[Any]) extends FieldPlan: + final private[avro] case class CodecPlan(slot: Int, codec: VCodec[Any]) extends FieldPlan: + def put(r: GenericData.Record, value: Any): Unit = r.put(slot, codec.encode(value).fold(e => throw e.throwable, identity)) @@ -230,7 +236,7 @@ object WholeRecordBuilder: // The first arm failure short-circuits the level; the shell is discarded with it. @tailrec def each(i: Int, rest: List[FieldShape]): Exception | Null = rest match - case Nil => null + case Nil => null case f :: t => planFor(f.kind, slots(i), f.name, fields.get(slots(i)).schema, level, who) match case e: Exception => e @@ -284,8 +290,8 @@ object WholeRecordBuilder: else val innerSchema = if schema.getTypes.size == 2 then nonNullBranch(schema) else schema innerPlanFor(innerKind, slot, name, innerSchema, schema, level, who) match - case e: Exception => e - case inner: FieldPlan => OptionPlan(slot, inner) + case e: Exception => e + case inner: FieldPlan => OptionPlan(slot, inner) /** The inner arm of an `OptionPlan`. Primitive / nested-record inners require a 2-branch nullable * pair (they write the non-null branch's shape); codec / self / nested-option inners consume no @@ -395,7 +401,7 @@ object WholeRecordBuilder: else level.parent match case p: RecordLevel => climb(p, depth - 1) - case null => + case null => IllegalStateException( s"whole-record builder: self-reference depth $depth exceeds the level chain —" + " internal invariant broken" diff --git a/avro/src/test/scala/dev/constructive/eo/avro/vulcan/WholeRecordBuilderFixtures.scala b/avro/src/test/scala/dev/constructive/eo/avro/vulcan/WholeRecordBuilderFixtures.scala index b94040e0..febdc315 100644 --- a/avro/src/test/scala/dev/constructive/eo/avro/vulcan/WholeRecordBuilderFixtures.scala +++ b/avro/src/test/scala/dev/constructive/eo/avro/vulcan/WholeRecordBuilderFixtures.scala @@ -11,6 +11,7 @@ enum TrafficClass: case Organic, Paid, Social, Referral object TrafficClass: + given VCodec[TrafficClass] = VCodec.enumeration( name = "TrafficClass", namespace = "dev.constructive.eo.avro.vulcan", @@ -19,17 +20,25 @@ object TrafficClass: decode = symbol => Right(TrafficClass.valueOf(symbol)), ) -final case class Geo(country: String, region: String, city: String, latitude: Double, longitude: Double) +final case class Geo( + country: String, + region: String, + city: String, + latitude: Double, + longitude: Double +) object Geo: - given VCodec[Geo] = VCodec.record(name = "Geo", namespace = "dev.constructive.eo.avro.vulcan") { fb => - ( - fb("country", _.country), - fb("region", _.region), - fb("city", _.city), - fb("latitude", _.latitude), - fb("longitude", _.longitude), - ).mapN(Geo.apply) + + given VCodec[Geo] = VCodec.record(name = "Geo", namespace = "dev.constructive.eo.avro.vulcan") { + fb => + ( + fb("country", _.country), + fb("region", _.region), + fb("city", _.city), + fb("latitude", _.latitude), + fb("longitude", _.longitude), + ).mapN(Geo.apply) } final case class UserAgentInfo( @@ -43,6 +52,7 @@ final case class UserAgentInfo( ) object UserAgentInfo: + given VCodec[UserAgentInfo] = VCodec.record( name = "UserAgentInfo", namespace = "dev.constructive.eo.avro.vulcan", @@ -71,6 +81,7 @@ final case class PostClick( ) object PostClick: + given VCodec[PostClick] = VCodec.record( name = "PostClick", namespace = "dev.constructive.eo.avro.vulcan", @@ -113,6 +124,7 @@ final case class Ivt( ) object Ivt: + given VCodec[Ivt] = VCodec.record( name = "Ivt", namespace = "dev.constructive.eo.avro.vulcan", @@ -154,6 +166,7 @@ final case class MavenEntities( ) object MavenEntities: + given VCodec[MavenEntities] = VCodec.record( name = "MavenEntities", namespace = "dev.constructive.eo.avro.vulcan", @@ -189,6 +202,7 @@ final case class ClickInfo( ) object ClickInfo: + given VCodec[ClickInfo] = VCodec.record( name = "ClickInfo", namespace = "dev.constructive.eo.avro.vulcan", @@ -213,6 +227,7 @@ object ClickInfo: final case class Reordered(alpha: Int, beta: String) object Reordered: + given VCodec[Reordered] = VCodec.record( name = "Reordered", namespace = "dev.constructive.eo.avro.vulcan", @@ -224,6 +239,7 @@ object Reordered: final case class Snakey(userId: Int, userName: String) object Snakey: + given VCodec[Snakey] = VCodec.record( name = "Snakey", namespace = "dev.constructive.eo.avro.vulcan", @@ -237,13 +253,14 @@ object Snakey: final case class WithComputed(base: Int) object WithComputed: + given VCodec[WithComputed] = VCodec.record( name = "WithComputed", namespace = "dev.constructive.eo.avro.vulcan", ) { fb => // `derived` is a schema-only column: nullable (the accessor's Option summons the union codec), // so the builder's null default stays decodable and round-trips. - (fb("base", _.base), fb("derived", (c: WithComputed) => (Some(c.base * 2): Option[Int]))) + (fb("base", _.base), fb("derived", (c: WithComputed) => Some(c.base * 2): Option[Int])) .mapN((base, _) => WithComputed(base)) } @@ -253,6 +270,7 @@ final class NoCodecLeaf(val s: String) final case class WithNoCodec(name: String, opaque: NoCodecLeaf) object WithNoCodec: + // The ROOT codec exists; the case-class field `opaque` has none — the macro must be the one to // refuse it. given VCodec[WithNoCodec] = VCodec.record( @@ -268,6 +286,7 @@ object WithNoCodec: final case class Renamed(alpha: Int, beta: String) object Renamed: + given VCodec[Renamed] = VCodec.record( name = "Renamed", namespace = "dev.constructive.eo.avro.vulcan", @@ -279,6 +298,7 @@ object Renamed: final case class OptMismatch(x: Option[Int]) object OptMismatch: + given VCodec[OptMismatch] = VCodec.record( name = "OptMismatch", namespace = "dev.constructive.eo.avro.vulcan", @@ -290,6 +310,7 @@ object OptMismatch: final case class FlatInner(v: Int) object FlatInner: + given VCodec[FlatInner] = VCodec.record( name = "FlatInner", namespace = "dev.constructive.eo.avro.vulcan", @@ -300,6 +321,7 @@ object FlatInner: final case class RecMis(inner: FlatInner) object RecMis: + given VCodec[RecMis] = VCodec.record( name = "RecMis", namespace = "dev.constructive.eo.avro.vulcan", @@ -311,6 +333,7 @@ object RecMis: final case class LongField(x: Long) object LongField: + given VCodec[LongField] = VCodec.record( name = "LongField", namespace = "dev.constructive.eo.avro.vulcan", @@ -322,6 +345,7 @@ object LongField: final case class Ambig(USERID: Int, other: String) object Ambig: + given VCodec[Ambig] = VCodec.record( name = "Ambig", namespace = "dev.constructive.eo.avro.vulcan", @@ -334,6 +358,7 @@ object Ambig: final case class Collide(aCol: Int, a_col: Int) object Collide: + given VCodec[Collide] = VCodec.record( name = "Collide", namespace = "dev.constructive.eo.avro.vulcan", diff --git a/avro/src/test/scala/dev/constructive/eo/avro/vulcan/WholeRecordBuilderMacroErrorSpec.scala b/avro/src/test/scala/dev/constructive/eo/avro/vulcan/WholeRecordBuilderMacroErrorSpec.scala index 4fd2362f..4dab4de6 100644 --- a/avro/src/test/scala/dev/constructive/eo/avro/vulcan/WholeRecordBuilderMacroErrorSpec.scala +++ b/avro/src/test/scala/dev/constructive/eo/avro/vulcan/WholeRecordBuilderMacroErrorSpec.scala @@ -28,5 +28,3 @@ class WholeRecordBuilderMacroErrorSpec extends Specification: ) errs.exists(_.message.contains("is not a case class")) must beTrue } - - diff --git a/avro/src/test/scala/dev/constructive/eo/avro/vulcan/WholeRecordBuilderSpec.scala b/avro/src/test/scala/dev/constructive/eo/avro/vulcan/WholeRecordBuilderSpec.scala index 776adb01..068b329e 100644 --- a/avro/src/test/scala/dev/constructive/eo/avro/vulcan/WholeRecordBuilderSpec.scala +++ b/avro/src/test/scala/dev/constructive/eo/avro/vulcan/WholeRecordBuilderSpec.scala @@ -18,14 +18,14 @@ import org.specs2.mutable.Specification */ class WholeRecordBuilderSpec extends Specification: - /** Unwrap a construction that the suite asserts SUCCEEDS (the refusal cases assert the other - * half below). + /** Unwrap a construction that the suite asserts SUCCEEDS (the refusal cases assert the other half + * below). */ private def built[A](r: Exception | WholeRecordBuilder[A]): WholeRecordBuilder[A] = r match case b: WholeRecordBuilder[A] => b case e: Exception => sys.error(e.getMessage) - private val vraw = summon[VCodec[ClickInfo]] + private val vraw = summon[VCodec[ClickInfo]] private val builder = built(AvroVulcan.recordBuilder[ClickInfo]) private val rich = ClickInfo( @@ -56,8 +56,26 @@ class WholeRecordBuilderSpec extends Specification: ) ), ivt = Ivt( - true, false, false, true, false, 87, 1, 0, 0, 0, - 0, 0, 0, 0, 0, 0, 42L, "v2", "acme", "clean", + true, + false, + false, + true, + false, + 87, + 1, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 42L, + "v2", + "acme", + "clean", ), mavenEntities = MavenEntities( Array[Byte](1, 2, 3), @@ -86,7 +104,10 @@ class WholeRecordBuilderSpec extends Specification: val derived: Array[Byte] = AvroCodec.encodeRecord(builder.toRecord(rich), builder.schema).toOption.get val coded: Array[Byte] = - AvroCodec.encodeRecord(vraw.encode(rich).toOption.get, vraw.schema.toOption.get).toOption.get + AvroCodec + .encodeRecord(vraw.encode(rich).toOption.get, vraw.schema.toOption.get) + .toOption + .get derived.toSeq must beEqualTo(coded.toSeq) } @@ -96,11 +117,15 @@ class WholeRecordBuilderSpec extends Specification: } "leave the schema-only computed column at its default while the codec fills it — and still round-trip" in { - val b = built(AvroVulcan.recordBuilder[WithComputed]) - val c = summon[VCodec[WithComputed]] + val b = built(AvroVulcan.recordBuilder[WithComputed]) + val c = summon[VCodec[WithComputed]] val rec = b.toRecord(WithComputed(21)) rec.get("derived") must beNull - c.encode(WithComputed(21)).toOption.get.asInstanceOf[GenericRecord].get("derived") must beEqualTo( + c.encode(WithComputed(21)) + .toOption + .get + .asInstanceOf[GenericRecord] + .get("derived") must beEqualTo( Int.box(42) ) c.decode(rec, c.schema.toOption.get).toOption.get must beEqualTo(WithComputed(21)) @@ -144,10 +169,13 @@ class WholeRecordBuilderSpec extends Specification: val next = Schema.createUnion(Schema.create(Schema.Type.NULL), rec) rec.setFields( - java.util.List.of( - new Schema.Field("value", Schema.create(Schema.Type.INT)), - new Schema.Field("next", next), - ) + java + .util + .List + .of( + new Schema.Field("value", Schema.create(Schema.Type.INT)), + new Schema.Field("next", next), + ) ) rec val shape = WholeRecordBuilder.RecordShape( @@ -164,7 +192,7 @@ class WholeRecordBuilderSpec extends Specification: ), ) WholeRecordBuilder.derive[Node](nodeSchema, shape, "spec[Node]") match - case e: Exception => sys.error(e.getMessage) + case e: Exception => sys.error(e.getMessage) case b: WholeRecordBuilder[Node] => val rec = b.toRecord(Node(1, Some(Node(2, Some(Node(3, None)))))) rec.get("value") must beEqualTo(Int.box(1)) @@ -181,31 +209,31 @@ class WholeRecordBuilderSpec extends Specification: "refuse a case field renamed beyond normalisation, naming field and record" in { AvroVulcan.recordBuilder[Renamed] match case e: IllegalArgumentException => - e.getMessage must (contain("'beta'") and contain("does not name a schema field")) - case e: Exception => ko(e.getMessage) + e.getMessage must (contain("'beta'").and(contain("does not name a schema field"))) + case e: Exception => ko(e.getMessage) case _: WholeRecordBuilder[Renamed] => ko("expected a construction refusal") } "refuse an Option case field over a non-nullable column" in { AvroVulcan.recordBuilder[OptMismatch] match - case e: IllegalArgumentException => e.getMessage must contain("not a null-union") - case e: Exception => ko(e.getMessage) + case e: IllegalArgumentException => e.getMessage must contain("not a null-union") + case e: Exception => ko(e.getMessage) case _: WholeRecordBuilder[OptMismatch] => ko("expected a construction refusal") } "refuse a case-class field whose column is not a record" in { AvroVulcan.recordBuilder[RecMis] match case e: IllegalArgumentException => - e.getMessage must (contain("needs a RECORD schema field") and contain("is a STRING")) - case e: Exception => ko(e.getMessage) + e.getMessage must (contain("needs a RECORD schema field").and(contain("is a STRING"))) + case e: Exception => ko(e.getMessage) case _: WholeRecordBuilder[RecMis] => ko("expected a construction refusal") } "refuse a LONG case field over an INT column (never widens silently)" in { AvroVulcan.recordBuilder[LongField] match case e: IllegalArgumentException => - e.getMessage must (contain("needs a LONG schema field") and contain("is a INT")) - case e: Exception => ko(e.getMessage) + e.getMessage must (contain("needs a LONG schema field").and(contain("is a INT"))) + case e: Exception => ko(e.getMessage) case _: WholeRecordBuilder[LongField] => ko("expected a construction refusal") } @@ -213,7 +241,7 @@ class WholeRecordBuilderSpec extends Specification: AvroVulcan.recordBuilder[Ambig] match case e: IllegalArgumentException => e.getMessage must contain("matches more than one schema field") - case e: Exception => ko(e.getMessage) + case e: Exception => ko(e.getMessage) case _: WholeRecordBuilder[Ambig] => ko("expected a construction refusal") } @@ -221,7 +249,7 @@ class WholeRecordBuilderSpec extends Specification: AvroVulcan.recordBuilder[Collide] match case e: IllegalArgumentException => e.getMessage must contain("collides with case field 'aCol'") - case e: Exception => ko(e.getMessage) + case e: Exception => ko(e.getMessage) case _: WholeRecordBuilder[Collide] => ko("expected a construction refusal") } } From b3920cd35aa4be8dce3acb2c2467f40cfc14fe1a Mon Sep 17 00:00:00 2001 From: Rodolfo Hansen Date: Tue, 22 Sep 2026 23:54:17 +0200 Subject: [PATCH 3/6] =?UTF-8?q?style:=20scalafmt=20ClickRecordBench=20+=20?= =?UTF-8?q?scalafix=20import=20order=20=E2=80=94=20what=20the=20pre-commit?= =?UTF-8?q?=20hook=20flagged?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit benchmarks sits outside the root aggregate, so scalafmtAll never sees it: ClickRecordBench's later edits drifted. scalafixAll --check also ordered the new files' import blocks. Both surfaced by the pre-commit hook on this commit. --- .../eo/avro/vulcan/RecordBuilderMacro.scala | 2 +- .../eo/avro/vulcan/AvroVulcanSpec.scala | 2 +- .../vulcan/WholeRecordBuilderFixtures.scala | 5 +- .../avro/vulcan/WholeRecordBuilderSpec.scala | 3 +- .../eo/bench/ClickRecordBench.scala | 61 +++++++++++++------ 5 files changed, 48 insertions(+), 25 deletions(-) diff --git a/avro/src/main/scala/dev/constructive/eo/avro/vulcan/RecordBuilderMacro.scala b/avro/src/main/scala/dev/constructive/eo/avro/vulcan/RecordBuilderMacro.scala index 466d45a3..138234f1 100644 --- a/avro/src/main/scala/dev/constructive/eo/avro/vulcan/RecordBuilderMacro.scala +++ b/avro/src/main/scala/dev/constructive/eo/avro/vulcan/RecordBuilderMacro.scala @@ -11,7 +11,7 @@ import dev.constructive.eo.avro.vulcan.WholeRecordBuilder.{ OptionKind, RecordKind, RecordShape, - SelfKind, + SelfKind } import org.apache.avro.Schema diff --git a/avro/src/test/scala/dev/constructive/eo/avro/vulcan/AvroVulcanSpec.scala b/avro/src/test/scala/dev/constructive/eo/avro/vulcan/AvroVulcanSpec.scala index 18d3debb..ce908f24 100644 --- a/avro/src/test/scala/dev/constructive/eo/avro/vulcan/AvroVulcanSpec.scala +++ b/avro/src/test/scala/dev/constructive/eo/avro/vulcan/AvroVulcanSpec.scala @@ -2,8 +2,8 @@ package dev.constructive.eo.avro.vulcan import scala.language.implicitConversions -import cats.syntax.all.* import _root_.vulcan.Codec as VCodec +import cats.syntax.all.* import dev.constructive.eo.avro.circe.AvroJson import dev.constructive.eo.avro.{codecPrism, AvroCodec} import org.apache.avro.generic.IndexedRecord diff --git a/avro/src/test/scala/dev/constructive/eo/avro/vulcan/WholeRecordBuilderFixtures.scala b/avro/src/test/scala/dev/constructive/eo/avro/vulcan/WholeRecordBuilderFixtures.scala index febdc315..b09025d2 100644 --- a/avro/src/test/scala/dev/constructive/eo/avro/vulcan/WholeRecordBuilderFixtures.scala +++ b/avro/src/test/scala/dev/constructive/eo/avro/vulcan/WholeRecordBuilderFixtures.scala @@ -1,9 +1,8 @@ package dev.constructive.eo.avro.vulcan -import java.time.Instant - -import cats.syntax.all.* import _root_.vulcan.Codec as VCodec +import cats.syntax.all.* +import java.time.Instant // ---- Top-level so the vulcan record codecs and the derived builders see plain classfiles. ---- diff --git a/avro/src/test/scala/dev/constructive/eo/avro/vulcan/WholeRecordBuilderSpec.scala b/avro/src/test/scala/dev/constructive/eo/avro/vulcan/WholeRecordBuilderSpec.scala index 068b329e..1f187758 100644 --- a/avro/src/test/scala/dev/constructive/eo/avro/vulcan/WholeRecordBuilderSpec.scala +++ b/avro/src/test/scala/dev/constructive/eo/avro/vulcan/WholeRecordBuilderSpec.scala @@ -1,11 +1,10 @@ package dev.constructive.eo.avro.vulcan -import java.time.Instant - import scala.language.implicitConversions import _root_.vulcan.Codec as VCodec import dev.constructive.eo.avro.{codecPrism, AvroCodec} +import java.time.Instant import org.apache.avro.Schema import org.apache.avro.generic.GenericRecord import org.specs2.mutable.Specification diff --git a/benchmarks/src/main/scala/dev/constructive/eo/bench/ClickRecordBench.scala b/benchmarks/src/main/scala/dev/constructive/eo/bench/ClickRecordBench.scala index 12be0e2d..0afa25ca 100644 --- a/benchmarks/src/main/scala/dev/constructive/eo/bench/ClickRecordBench.scala +++ b/benchmarks/src/main/scala/dev/constructive/eo/bench/ClickRecordBench.scala @@ -20,23 +20,23 @@ import org.apache.avro.generic.GenericData * arms differ ONLY in mechanism: * * - '''vulcanFull''' — the full `vulcan.Codec[ClickInfo].encode` (the pre-#95 baseline): pays - * vulcan's per-field composition (FreeApplicative analyze, Either + Chain per field, put-by-name - * hash probe) once per field PER LEVEL. + * vulcan's per-field composition (FreeApplicative analyze, Either + Chain per field, + * put-by-name hash probe) once per field PER LEVEL. * - '''positional''' — the whole-record builder over held per-field leaf codecs the filer * implemented and REJECTED: positional puts of `leafCodec.encode(value)`, nested sub-records * through `subCodec.encode` — strips ONE level of composition (the outermost shell) and keeps * paying the rest, which is what measured 7.5x time / 16.9x allocation on their real shape. * - '''hand''' — the leaf-by-leaf `.put(pos, value)` builder the filer hand-maintains today: one * line per leaf across every nesting level. - * - '''derived''' — `AvroVulcan.recordBuilder[ClickInfo]`: the compile-time-derived builder — the - * hand-built shape with no hand-maintained lines (recursive sub-record levels, primitive bulk - * as positional puts). + * - '''derived''' — `AvroVulcan.recordBuilder[ClickInfo]`: the compile-time-derived builder — + * the hand-built shape with no hand-maintained lines (recursive sub-record levels, primitive + * bulk as positional puts). * - '''derivedThroughPrism''' — `codecPrism[ClickInfo].record.reverseGet` over the derived codec * (`asAvroCodec`): the eo-idiomatic surface at the call site, which is how the filer wires it. * - * '''B/op (`-prof gc`) is the gate; ns/op advises''' (project doctrine, and these boxes are noisy). - * The gate the filer set: derived ≈ hand (both put raw values; String-emitting, Utf8 materialises - * at serialise time), positional ≈ vulcan-per-sub-record. + * '''B/op (`-prof gc`) is the gate; ns/op advises''' (project doctrine, and these boxes are + * noisy). The gate the filer set: derived ≈ hand (both put raw values; String-emitting, Utf8 + * materialises at serialise time), positional ≈ vulcan-per-sub-record. */ object ClickRecordImpls: @@ -52,6 +52,7 @@ object ClickRecordImpls: ) object Geo: + given VCodec[Geo] = VCodec.record(name = "Geo", namespace = "dev.constructive.eo.bench") { fb => ( fb("country", _.country), @@ -78,6 +79,7 @@ object ClickRecordImpls: ) object UserAgentInfo: + given VCodec[UserAgentInfo] = VCodec.record( name = "UserAgentInfo", namespace = "dev.constructive.eo.bench", @@ -110,6 +112,7 @@ object ClickRecordImpls: ) object PostClick: + given VCodec[PostClick] = VCodec.record( name = "PostClick", namespace = "dev.constructive.eo.bench", @@ -153,6 +156,7 @@ object ClickRecordImpls: ) object Ivt: + given VCodec[Ivt] = VCodec.record( name = "Ivt", namespace = "dev.constructive.eo.bench", @@ -195,6 +199,7 @@ object ClickRecordImpls: ) object MavenEntities: + given VCodec[MavenEntities] = VCodec.record( name = "MavenEntities", namespace = "dev.constructive.eo.bench", @@ -230,6 +235,7 @@ object ClickRecordImpls: ) object ClickInfo: + given VCodec[ClickInfo] = VCodec.record( name = "ClickInfo", namespace = "dev.constructive.eo.bench", @@ -281,8 +287,26 @@ object ClickRecordImpls: "cpc", ), ivt = Ivt( - false, false, false, true, false, 87, 1, 0, 2, 0, - 0, 1, 0, 0, 3, 0, 42L, "v2", "acme", "clean", + false, + false, + false, + true, + false, + 87, + 1, + 0, + 2, + 0, + 0, + 1, + 0, + 0, + 3, + 0, + 42L, + "v2", + "acme", + "clean", ), mavenEntities = MavenEntities( 12, @@ -311,13 +335,13 @@ object ClickRecordImpls: */ final class PositionalBuilder(schema: org.apache.avro.Schema): private val cString = VCodec.string - private val cLong = VCodec.long - private val cBool = VCodec.boolean - private val cGeo = summon[VCodec[Geo]] - private val cUa = summon[VCodec[UserAgentInfo]] - private val cPc = summon[VCodec[PostClick]] - private val cIvt = summon[VCodec[Ivt]] - private val cMaven = summon[VCodec[MavenEntities]] + private val cLong = VCodec.long + private val cBool = VCodec.boolean + private val cGeo = summon[VCodec[Geo]] + private val cUa = summon[VCodec[UserAgentInfo]] + private val cPc = summon[VCodec[PostClick]] + private val cIvt = summon[VCodec[Ivt]] + private val cMaven = summon[VCodec[MavenEntities]] private def enc[A](c: VCodec[A], a: A): Any = c.encode(a).fold(e => throw e.throwable, identity) @@ -342,6 +366,7 @@ object ClickRecordImpls: /** The hand-maintained builder: one line per leaf, every level, raw values. */ final class HandBuilder(schema: org.apache.avro.Schema): + def toRecord(a: ClickInfo): GenericData.Record = val r = new GenericData.Record(schema) r.put(0, a.clickId) @@ -445,7 +470,7 @@ object ClickRecordImpls: val vulcanCodec: VCodec[ClickInfo] = summon[VCodec[ClickInfo]] val positional: PositionalBuilder = PositionalBuilder(schema) - val hand: HandBuilder = HandBuilder(schema) + val hand: HandBuilder = HandBuilder(schema) /** The derived builder (construction unwrapped — the once-cost is not the hot path). */ val derived: WholeRecordBuilder[ClickInfo] = From 2ab680c76a42d88ef4c6cdeaddc845900639d7c6 Mon Sep 17 00:00:00 2001 From: Rodolfo Hansen Date: Wed, 23 Sep 2026 00:46:08 +0200 Subject: [PATCH 4/6] =?UTF-8?q?ci:=20coursier/ivy/boot=20caching=20across?= =?UTF-8?q?=20all=20workflows=20=E2=80=94=20refresh-on-run,=20not=20save-o?= =?UTF-8?q?nce?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit setup-java's 'cache: sbt' preset only SAVES on a primary-key miss, so dependencies fetched after a key was minted are re-downloaded from Central on every later run — and one flaky fetch fails the job. That is the scalafix-cli_3.8.4:0.14.7 failure on this PR: the artifact was on Central, main's identical step resolved it green, but the PR runners had to re-fetch it and hit a blip. Twice. Replace it with actions/cache@v4 over ~/.cache/coursier, ~/.ivy2/cache and ~/.sbt, keyed on the build definition with a broad restore-keys prefix, in ci.yml (via the generator — githubWorkflowJobSetup now strips the setup-java cache param, injects the cache step after each setup-java leg, and rewires the 'sbt update' warmups to the cache step's cache-hit), plus the five hand-written workflows. actions/cache saves on every non-exact-key run, so late-fetched dependencies enter the cache chain for every subsequent run instead of fossilising with the key. --- .github/workflows/bench-pr.yml | 11 +- .github/workflows/bench-sweep.yml | 11 +- .github/workflows/benchmarks.yml | 11 +- .github/workflows/ci.yml | 180 ++++++++++++++++++++++++++---- .github/workflows/deploy-site.yml | 11 +- .github/workflows/quality.yml | 22 +++- build.sbt | 38 ++++++- 7 files changed, 253 insertions(+), 31 deletions(-) diff --git a/.github/workflows/bench-pr.yml b/.github/workflows/bench-pr.yml index d2d10199..93cb4fec 100644 --- a/.github/workflows/bench-pr.yml +++ b/.github/workflows/bench-pr.yml @@ -127,7 +127,16 @@ jobs: with: distribution: temurin java-version: 25 # kyoIntegration (KyoDiBench dep) targets JDK 25 since v0.15.0 - cache: sbt + - name: Cache coursier + ivy + sbt boot + uses: actions/cache@v4 + with: + path: | + ~/.cache/coursier + ~/.ivy2/cache + ~/.sbt + key: ${{ runner.os }}-sbt-${{ hashFiles('**/*.sbt', 'project/build.properties', 'project/**/*.scala') }} + restore-keys: | + ${{ runner.os }}-sbt- - name: Base run (${{ steps.plan.outputs.base }}) id: base diff --git a/.github/workflows/bench-sweep.yml b/.github/workflows/bench-sweep.yml index b3cd0e8b..c2d025d1 100644 --- a/.github/workflows/bench-sweep.yml +++ b/.github/workflows/bench-sweep.yml @@ -123,7 +123,16 @@ jobs: with: distribution: temurin java-version: 25 # kyoIntegration (KyoDiBench dep) targets JDK 25 since v0.15.0 - cache: sbt + - name: Cache coursier + ivy + sbt boot + uses: actions/cache@v4 + with: + path: | + ~/.cache/coursier + ~/.ivy2/cache + ~/.sbt + key: ${{ runner.os }}-sbt-${{ hashFiles('**/*.sbt', 'project/build.properties', 'project/**/*.scala') }} + restore-keys: | + ${{ runner.os }}-sbt- - name: Run full JMH sweep if: steps.ctx.outputs.skip == 'false' diff --git a/.github/workflows/benchmarks.yml b/.github/workflows/benchmarks.yml index 201fc146..419a8120 100644 --- a/.github/workflows/benchmarks.yml +++ b/.github/workflows/benchmarks.yml @@ -61,7 +61,16 @@ jobs: with: distribution: temurin java-version: 21 - cache: sbt + - name: Cache coursier + ivy + sbt boot + uses: actions/cache@v4 + with: + path: | + ~/.cache/coursier + ~/.ivy2/cache + ~/.sbt + key: ${{ runner.os }}-sbt-${{ hashFiles('**/*.sbt', 'project/build.properties', 'project/**/*.scala') }} + restore-keys: | + ${{ runner.os }}-sbt- - name: Run JMH # `inputs.*` are only populated for workflow_dispatch; the `|| 'default'` diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index fe4f1d09..6978522b 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -48,10 +48,21 @@ jobs: with: distribution: temurin java-version: 25 - cache: sbt + + - name: Cache coursier + ivy + sbt boot + id: coursier-cache-temurin-25 + if: matrix.java == 'temurin@25' + uses: actions/cache@v4 + with: + path: | + ~/.cache/coursier + ~/.ivy2/cache + ~/.sbt + key: ${{ runner.os }}-sbt-${{ hashFiles('**/*.sbt', 'project/build.properties', 'project/**/*.scala') }} + restore-keys: ${{ runner.os }}-sbt- - name: sbt update - if: matrix.java == 'temurin@25' && steps.setup-java-temurin-25.outputs.cache-hit == 'false' + if: matrix.java == 'temurin@25' && steps.coursier-cache-temurin-25.outputs.cache-hit != 'true' run: sbt +update - name: Setup Java (temurin@17) @@ -61,10 +72,21 @@ jobs: with: distribution: temurin java-version: 17 - cache: sbt + + - name: Cache coursier + ivy + sbt boot + id: coursier-cache-temurin-17 + if: matrix.java == 'temurin@17' + uses: actions/cache@v4 + with: + path: | + ~/.cache/coursier + ~/.ivy2/cache + ~/.sbt + key: ${{ runner.os }}-sbt-${{ hashFiles('**/*.sbt', 'project/build.properties', 'project/**/*.scala') }} + restore-keys: ${{ runner.os }}-sbt- - name: sbt update - if: matrix.java == 'temurin@17' && steps.setup-java-temurin-17.outputs.cache-hit == 'false' + if: matrix.java == 'temurin@17' && steps.coursier-cache-temurin-17.outputs.cache-hit != 'true' run: sbt +update - name: Setup Java (temurin@21) @@ -74,10 +96,21 @@ jobs: with: distribution: temurin java-version: 21 - cache: sbt + + - name: Cache coursier + ivy + sbt boot + id: coursier-cache-temurin-21 + if: matrix.java == 'temurin@21' + uses: actions/cache@v4 + with: + path: | + ~/.cache/coursier + ~/.ivy2/cache + ~/.sbt + key: ${{ runner.os }}-sbt-${{ hashFiles('**/*.sbt', 'project/build.properties', 'project/**/*.scala') }} + restore-keys: ${{ runner.os }}-sbt- - name: sbt update - if: matrix.java == 'temurin@21' && steps.setup-java-temurin-21.outputs.cache-hit == 'false' + if: matrix.java == 'temurin@21' && steps.coursier-cache-temurin-21.outputs.cache-hit != 'true' run: sbt +update - name: Check formatting @@ -140,10 +173,21 @@ jobs: with: distribution: temurin java-version: 25 - cache: sbt + + - name: Cache coursier + ivy + sbt boot + id: coursier-cache-temurin-25 + if: matrix.java == 'temurin@25' + uses: actions/cache@v4 + with: + path: | + ~/.cache/coursier + ~/.ivy2/cache + ~/.sbt + key: ${{ runner.os }}-sbt-${{ hashFiles('**/*.sbt', 'project/build.properties', 'project/**/*.scala') }} + restore-keys: ${{ runner.os }}-sbt- - name: sbt update - if: matrix.java == 'temurin@25' && steps.setup-java-temurin-25.outputs.cache-hit == 'false' + if: matrix.java == 'temurin@25' && steps.coursier-cache-temurin-25.outputs.cache-hit != 'true' run: sbt +update - name: Setup Java (temurin@17) @@ -153,10 +197,21 @@ jobs: with: distribution: temurin java-version: 17 - cache: sbt + + - name: Cache coursier + ivy + sbt boot + id: coursier-cache-temurin-17 + if: matrix.java == 'temurin@17' + uses: actions/cache@v4 + with: + path: | + ~/.cache/coursier + ~/.ivy2/cache + ~/.sbt + key: ${{ runner.os }}-sbt-${{ hashFiles('**/*.sbt', 'project/build.properties', 'project/**/*.scala') }} + restore-keys: ${{ runner.os }}-sbt- - name: sbt update - if: matrix.java == 'temurin@17' && steps.setup-java-temurin-17.outputs.cache-hit == 'false' + if: matrix.java == 'temurin@17' && steps.coursier-cache-temurin-17.outputs.cache-hit != 'true' run: sbt +update - name: Setup Java (temurin@21) @@ -166,10 +221,21 @@ jobs: with: distribution: temurin java-version: 21 - cache: sbt + + - name: Cache coursier + ivy + sbt boot + id: coursier-cache-temurin-21 + if: matrix.java == 'temurin@21' + uses: actions/cache@v4 + with: + path: | + ~/.cache/coursier + ~/.ivy2/cache + ~/.sbt + key: ${{ runner.os }}-sbt-${{ hashFiles('**/*.sbt', 'project/build.properties', 'project/**/*.scala') }} + restore-keys: ${{ runner.os }}-sbt- - name: sbt update - if: matrix.java == 'temurin@21' && steps.setup-java-temurin-21.outputs.cache-hit == 'false' + if: matrix.java == 'temurin@21' && steps.coursier-cache-temurin-21.outputs.cache-hit != 'true' run: sbt +update - name: Download target directories (3) @@ -229,10 +295,21 @@ jobs: with: distribution: temurin java-version: 25 - cache: sbt + + - name: Cache coursier + ivy + sbt boot + id: coursier-cache-temurin-25 + if: matrix.java == 'temurin@25' + uses: actions/cache@v4 + with: + path: | + ~/.cache/coursier + ~/.ivy2/cache + ~/.sbt + key: ${{ runner.os }}-sbt-${{ hashFiles('**/*.sbt', 'project/build.properties', 'project/**/*.scala') }} + restore-keys: ${{ runner.os }}-sbt- - name: sbt update - if: matrix.java == 'temurin@25' && steps.setup-java-temurin-25.outputs.cache-hit == 'false' + if: matrix.java == 'temurin@25' && steps.coursier-cache-temurin-25.outputs.cache-hit != 'true' run: sbt +update - name: Setup Java (temurin@17) @@ -242,10 +319,21 @@ jobs: with: distribution: temurin java-version: 17 - cache: sbt + + - name: Cache coursier + ivy + sbt boot + id: coursier-cache-temurin-17 + if: matrix.java == 'temurin@17' + uses: actions/cache@v4 + with: + path: | + ~/.cache/coursier + ~/.ivy2/cache + ~/.sbt + key: ${{ runner.os }}-sbt-${{ hashFiles('**/*.sbt', 'project/build.properties', 'project/**/*.scala') }} + restore-keys: ${{ runner.os }}-sbt- - name: sbt update - if: matrix.java == 'temurin@17' && steps.setup-java-temurin-17.outputs.cache-hit == 'false' + if: matrix.java == 'temurin@17' && steps.coursier-cache-temurin-17.outputs.cache-hit != 'true' run: sbt +update - name: Setup Java (temurin@21) @@ -255,10 +343,21 @@ jobs: with: distribution: temurin java-version: 21 - cache: sbt + + - name: Cache coursier + ivy + sbt boot + id: coursier-cache-temurin-21 + if: matrix.java == 'temurin@21' + uses: actions/cache@v4 + with: + path: | + ~/.cache/coursier + ~/.ivy2/cache + ~/.sbt + key: ${{ runner.os }}-sbt-${{ hashFiles('**/*.sbt', 'project/build.properties', 'project/**/*.scala') }} + restore-keys: ${{ runner.os }}-sbt- - name: sbt update - if: matrix.java == 'temurin@21' && steps.setup-java-temurin-21.outputs.cache-hit == 'false' + if: matrix.java == 'temurin@21' && steps.coursier-cache-temurin-21.outputs.cache-hit != 'true' run: sbt +update - name: Submit Dependencies @@ -315,10 +414,21 @@ jobs: with: distribution: temurin java-version: 25 - cache: sbt + + - name: Cache coursier + ivy + sbt boot + id: coursier-cache-temurin-25 + if: matrix.java == 'temurin@25' + uses: actions/cache@v4 + with: + path: | + ~/.cache/coursier + ~/.ivy2/cache + ~/.sbt + key: ${{ runner.os }}-sbt-${{ hashFiles('**/*.sbt', 'project/build.properties', 'project/**/*.scala') }} + restore-keys: ${{ runner.os }}-sbt- - name: sbt update - if: matrix.java == 'temurin@25' && steps.setup-java-temurin-25.outputs.cache-hit == 'false' + if: matrix.java == 'temurin@25' && steps.coursier-cache-temurin-25.outputs.cache-hit != 'true' run: sbt +update - name: Setup Java (temurin@17) @@ -328,10 +438,21 @@ jobs: with: distribution: temurin java-version: 17 - cache: sbt + + - name: Cache coursier + ivy + sbt boot + id: coursier-cache-temurin-17 + if: matrix.java == 'temurin@17' + uses: actions/cache@v4 + with: + path: | + ~/.cache/coursier + ~/.ivy2/cache + ~/.sbt + key: ${{ runner.os }}-sbt-${{ hashFiles('**/*.sbt', 'project/build.properties', 'project/**/*.scala') }} + restore-keys: ${{ runner.os }}-sbt- - name: sbt update - if: matrix.java == 'temurin@17' && steps.setup-java-temurin-17.outputs.cache-hit == 'false' + if: matrix.java == 'temurin@17' && steps.coursier-cache-temurin-17.outputs.cache-hit != 'true' run: sbt +update - name: Setup Java (temurin@21) @@ -341,10 +462,21 @@ jobs: with: distribution: temurin java-version: 21 - cache: sbt + + - name: Cache coursier + ivy + sbt boot + id: coursier-cache-temurin-21 + if: matrix.java == 'temurin@21' + uses: actions/cache@v4 + with: + path: | + ~/.cache/coursier + ~/.ivy2/cache + ~/.sbt + key: ${{ runner.os }}-sbt-${{ hashFiles('**/*.sbt', 'project/build.properties', 'project/**/*.scala') }} + restore-keys: ${{ runner.os }}-sbt- - name: sbt update - if: matrix.java == 'temurin@21' && steps.setup-java-temurin-21.outputs.cache-hit == 'false' + if: matrix.java == 'temurin@21' && steps.coursier-cache-temurin-21.outputs.cache-hit != 'true' run: sbt +update - name: Generate site diff --git a/.github/workflows/deploy-site.yml b/.github/workflows/deploy-site.yml index 59771133..d6852b8e 100644 --- a/.github/workflows/deploy-site.yml +++ b/.github/workflows/deploy-site.yml @@ -69,7 +69,16 @@ jobs: with: distribution: temurin java-version: 25 - cache: sbt + - name: Cache coursier + ivy + sbt boot + uses: actions/cache@v4 + with: + path: | + ~/.cache/coursier + ~/.ivy2/cache + ~/.sbt + key: ${{ runner.os }}-sbt-${{ hashFiles('**/*.sbt', 'project/build.properties', 'project/**/*.scala') }} + restore-keys: | + ${{ runner.os }}-sbt- - name: Build site run: sbt 'docs/tlSite; unidocs/unidoc' diff --git a/.github/workflows/quality.yml b/.github/workflows/quality.yml index d887fe8c..780bafa5 100644 --- a/.github/workflows/quality.yml +++ b/.github/workflows/quality.yml @@ -60,7 +60,16 @@ jobs: with: distribution: temurin java-version: 25 - cache: sbt + - name: Cache coursier + ivy + sbt boot + uses: actions/cache@v4 + with: + path: | + ~/.cache/coursier + ~/.ivy2/cache + ~/.sbt + key: ${{ runner.os }}-sbt-${{ hashFiles('**/*.sbt', 'project/build.properties', 'project/**/*.scala') }} + restore-keys: | + ${{ runner.os }}-sbt- - name: Cost ratios (armed) run: sbt -batch -Deo.costGate=true "avroIntegration/testOnly *NominalResolutionCostSpec" @@ -87,7 +96,16 @@ jobs: with: distribution: temurin java-version: 25 - cache: sbt + - name: Cache coursier + ivy + sbt boot + uses: actions/cache@v4 + with: + path: | + ~/.cache/coursier + ~/.ivy2/cache + ~/.sbt + key: ${{ runner.os }}-sbt-${{ hashFiles('**/*.sbt', 'project/build.properties', 'project/**/*.scala') }} + restore-keys: | + ${{ runner.os }}-sbt- - name: Unused public code (WarnUnusedCode) # xuwei-k's unused-code-plugin: scans every module's Compile sources and diff --git a/build.sbt b/build.sbt index f14eadbd..f0946b80 100644 --- a/build.sbt +++ b/build.sbt @@ -232,9 +232,45 @@ ThisBuild / githubWorkflowGeneratedDownloadSteps ~= { steps => // generates the newer versions upstream — see // https://github.com/typelevel/sbt-typelevel/releases. ThisBuild / githubWorkflowJobSetup ~= { steps => - steps + val bumped = steps .map(bumpActionVersion("actions", "checkout", "v7")) .map(bumpActionVersion("actions", "setup-java", "v6")) + + // Explicit coursier/ivy/boot caching instead of setup-java's `cache: sbt`: + // that preset only SAVES on a primary-key miss, so anything the jobs fetch + // after a key was minted (scalafix-cli when a check step lands, Steward + // bumps, ...) is re-downloaded from Central on every later run — and one + // flaky fetch fails the job (PR #116: `scalafix-cli_3.8.4:0.14.7` flaked + // twice while main's identical step stayed green). restore-keys keep the + // chain warm across key changes, and actions/cache saves on every + // non-exact-key run, so the cache refreshes instead of fossilising. + bumped.flatMap { + case s: WorkflowStep.Use if s.id.exists(_.startsWith("setup-java-")) => + val javaId = s.id.get.stripPrefix("setup-java-") + val cacheStep = WorkflowStep.Use( + UseRef.Public("actions", "cache", "v4"), + params = Map( + "path" -> Seq( + "~/.cache/coursier", + "~/.ivy2/cache", + "~/.sbt", + ).mkString("\n"), + "key" -> "${{ runner.os }}-sbt-${{ hashFiles('**/*.sbt', 'project/build.properties', 'project/**/*.scala') }}", + "restore-keys" -> "${{ runner.os }}-sbt-", + ), + id = Some(s"coursier-cache-$javaId"), + name = Some("Cache coursier + ivy + sbt boot"), + cond = s.cond, + ) + Seq(s.withParams(s.params - "cache"), cacheStep) + case s: WorkflowStep.Sbt + if s.cond.exists(_.contains("outputs.cache-hit == 'false'")) => + val rewired = s.cond.get + .replace("setup-java-", "coursier-cache-") + .replace("outputs.cache-hit == 'false'", "outputs.cache-hit != 'true'") + Seq(s.withCond(Some(rewired))) + case other => Seq(other) + } } ThisBuild / githubWorkflowAddedJobs ~= { jobs => From 9e280b607854e5a722530a6cde5dfcb0ef637204 Mon Sep 17 00:00:00 2001 From: Rodolfo Hansen Date: Wed, 23 Sep 2026 01:25:45 +0200 Subject: [PATCH 5/6] =?UTF-8?q?style(build):=20scalafmtSbt=20the=20jobSetu?= =?UTF-8?q?p=20transformer=20=E2=80=94=20CI=20checks=20what=20the=20hook?= =?UTF-8?q?=20doesn't?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The pre-commit hook runs scalafmtCheckAll/benchmarks/scalafmtCheck but not scalafmtSbtCheck, so the unformatted build.sbt edit passed locally and failed in CI's 'Check formatting' step. --- build.sbt | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/build.sbt b/build.sbt index f0946b80..31ff9346 100644 --- a/build.sbt +++ b/build.sbt @@ -263,9 +263,10 @@ ThisBuild / githubWorkflowJobSetup ~= { steps => cond = s.cond, ) Seq(s.withParams(s.params - "cache"), cacheStep) - case s: WorkflowStep.Sbt - if s.cond.exists(_.contains("outputs.cache-hit == 'false'")) => - val rewired = s.cond.get + case s: WorkflowStep.Sbt if s.cond.exists(_.contains("outputs.cache-hit == 'false'")) => + val rewired = s + .cond + .get .replace("setup-java-", "coursier-cache-") .replace("outputs.cache-hit == 'false'", "outputs.cache-hit != 'true'") Seq(s.withCond(Some(rewired))) From 910183c2ddcd0323cd619abbc91f6a821b70aaa2 Mon Sep 17 00:00:00 2001 From: Rodolfo Hansen Date: Wed, 23 Sep 2026 01:44:11 +0200 Subject: [PATCH 6/6] =?UTF-8?q?ci(hooks):=20scalafmtSbtCheck=20in=20pre-co?= =?UTF-8?q?mmit=20=E2=80=94=20mirror=20CI's=20'Check=20formatting'?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The hook checked .scala/.md formatting and benchmarks/, but not the build's *.sbt files — an unformatted build.sbt edit passed the hook and failed CI's 'Check formatting' step (caught on PR #116). CI runs scalafmtSbtCheck right after scalafmtCheckAll; the hook now does too. --- .githooks/pre-commit | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/.githooks/pre-commit b/.githooks/pre-commit index d2fc9585..9bec54b7 100755 --- a/.githooks/pre-commit +++ b/.githooks/pre-commit @@ -3,6 +3,8 @@ # # Covered: # * scalafmtCheckAll — formatting matches .scalafmt.conf across all modules +# * scalafmtSbtCheck — ditto for the build's *.sbt files (mirrors CI's +# 'Check formatting' step, which checks them separately) # * benchmarks/scalafmtCheck — ditto for benchmarks/ (outside the root aggregate) # * scalafixAll --check — scalafix rules in .scalafix.conf are clean # * docs/mdoc — every `scala mdoc` fence in site/docs/*.md compiles @@ -23,4 +25,4 @@ else fi echo "[pre-commit] running scalafmtCheckAll + benchmarks/scalafmtCheck + scalafixAll --check + docs/mdoc + docs/laikaSite ..." -sbt -no-colors 'scalafmtCheckAll; benchmarks/scalafmtCheck; benchmarks/Test/scalafmtCheck; scalafixAll --check; docs/mdoc; docs/laikaSite' +sbt -no-colors 'scalafmtCheckAll; scalafmtSbtCheck; benchmarks/scalafmtCheck; benchmarks/Test/scalafmtCheck; scalafixAll --check; docs/mdoc; docs/laikaSite'