fix(qa): score zio and kyo on the QA page, and unblock zio's stryker run - #113
Merged
Merged
Conversation
Contributor
|
🚀 Cloudflare Pages preview for https://43c69ef7.cats-eo-docs.pages.dev Branch alias: https://fix-qa-report-zio-kyo-module.cats-eo-docs.pages.dev Built from commit |
`site/tools/gen-qa-report.py`'s `MUTATION_MODULES` stopped at `jsoniter`, so
the two effect-system integrations never had a row on the quality-assurance
page even though `mutationAll` has been running stryker over both. The
generator looked for reports it was never told to look for, which is the
quietest possible failure: no error, no empty row, just absence.
`.github/workflows/quality.yml`'s per-module loop had drifted the same way,
and further: it was missing `schemesLaws` too. A module absent from that loop
produces no report.json on the release-tag run, so its row falls back to
dashes — or, worse, keeps whatever numbers a developer's local run happened to
leave on disk. The loop now matches the `mutationAll` alias, and both lists
carry a comment saying they must stay in step. The loop takes sbt PROJECT IDS
(`zioIntegration`), `MUTATION_MODULES` takes DIRECTORIES (`zio`); getting that
backwards is what silently finds nothing, so it is written down.
Two honest edge cases the rows had to render correctly:
- `zio` generates ZERO mutants on main — it is `ZEnvironment` / `ZLayer` /
`Ref` wiring that delegates straight into ZIO's API, with no operator,
literal or branch for a mutator to change. That is `n/a`, NOT 0%: a module
with nothing to mutate has not failed a test-quality bar, it has no bar to
fail. The old code divided only when the denominator was non-zero, but it
rendered the empty case as the same em-dash it uses for "no report at all",
conflating "nothing to score" with "nothing ran". `n/a` is also what
stryker's own console prints here.
- `kyo` scores 38.5% total / 83.3% covered, and the entire no-coverage block
is `RecordIsoMacro` — quoted-macro code that expands at compile time, the
same structural reason `generics` scores 0%. The Notes column now says so,
so the total score does not read as a coverage hole.
The all-NoCoverage shape (`generics`: 0.0% total, covered undefined) is
deliberately left alone — those mutants exist, they are simply never executed.
Rows verified end to end: stryker was run per-module in a clone
(`project zioIntegration; stryker`, same for kyo — never `<module>/stryker`)
and the generator read the resulting report.json for both.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0194EHFR4NamCpTHiqy7B74V
The rebase onto main invalidated this branch's own premise. When it was written, `zio` was `ZEnvironment` / `ZLayer` / `Ref` wiring that delegated straight into ZIO's API, so stryker generated ZERO mutants for it and the honest row was `n/a`. #92 then landed the ecosystem optics — the zio-schema `DynamicValue` kit, the zio-json AST and `JsonCursor` bridge, the STM focus ops, `Chunks` — and with them a great deal of genuinely mutatable code. `project zioIntegration; stryker` on the rebased tree does not print `n/a`. It aborts, with the same UnableToFixCompilerErrorsException that #109 fixed for kyo, on `json/JsonOptics.scala`: 'Extension without extension methods' 'Not found: type To' / 'Not found: self' / 'Not found: type A' Same mechanism, same fix. stryker4s 0.20.3 re-prints every mutated file through scalameta; a SINGLE-METHOD significant-indentation `extension` clause is not a `Term.Block`, so it comes back as the one-line form with the method's leading Scaladoc replayed between the two, and the forced newline lands `def` in column 0. Four such clauses exist in the module — two in `json/JsonOptics` (`JsonCursor.optional`, `JsonCodec.stringPrism`) and two in `schema/SchemaOptics` (`Schema.dynamicPrism`, `BinaryCodec.prism`). Bracing their bodies makes the printer emit the braced form and the newline harmless. `ZioOptics`'s single-method clause already hoists its Scaladoc above `extension`, which dodges the same trap, so it is left alone. Verified, not asserted: `javap -p -c` over `JsonOptics$package$` and `SchemaOptics$package$` is byte-for-byte identical before and after, and `zioIntegration/test` passes unchanged (268 examples). With that, both effect-system rows are real measurements rather than placeholders, regenerated by `site/tools/gen-qa-report.py` from the actual report.json: | `zio` | 37 | 0 | 6 | 0 | 0 | 86.0% | 86.0% | | `kyo` | 30 | 0 | 5 | 28 | 0 | 47.6% | 85.7% | zio is a clean high-signal row: 37 killed, 6 survived, and no no-coverage block at all. kyo moved off #109's 20/4/28 (38.5% / 83.3%) because #111's survivor-killing pass landed in between; its 28 no-coverage mutants are still all `RecordIsoMacro`, so the note stands. The generator's `n/a` branch stays. Nothing hits it today, but rendering a zero-mutant module as the same em-dash it uses for "no report at all" conflates "nothing to score" with "nothing ran", and printing 0.0% would read as a test-quality failure where there is no bar to fail. The page now says so explicitly rather than pointing at zio as the example. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0194EHFR4NamCpTHiqy7B74V
kryptt
force-pushed
the
fix/qa-report-zio-kyo-modules
branch
from
September 18, 2026 16:30
62b097b to
eee9a12
Compare
Contributor
Benchmark A/BAllocation (B/op) — authoritative
Timing (ns/op) — directional only, same-VM but shared runner
base_sha: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The bug
site/tools/gen-qa-report.pybuilds the mutation table from a hard-codedMUTATION_MODULESlist, and that list stopped atjsoniter.zioIntegrationand
kyoIntegrationhave been in themutationAllalias for a while, so strykerhas been running over both — but the generator never looked for their reports, so
neither module has ever had a row on
quality-assurance.md. No error, no empty row,
just absence.
The two lists use different identifiers, and that is the trap the bug is made
of:
quality.yml's loop takes sbt project ids (zioIntegration,schemesLaws),MUTATION_MODULEStakes on-disk directories (zio,schemes-laws, whichlatest_reportglobs). Getting it backwards silently findsnothing — exactly the failure being fixed — so both lists now carry a comment
saying so and pointing at each other.
A third instance of the same drift, fixed here:
quality.yml's per-module loopwas
generics schemes circeIntegration avroIntegration jsoniterIntegration—missing
schemesLawsas well as both integrations. A module absent from thatloop produces no
report.jsonon the release run, so its row falls back todashes, or keeps whatever a developer's local run left on disk. The loop now
matches the
mutationAllalias.What the rebase changed
ziono longer scoresn/a. When this PR was written,zio/was a 98-lineDI integration —
ZEnvironment/ZLayer/Refwiring delegating straight intoZIO's API — and stryker generated zero mutants for it, so
n/awas the honestrow. #92 then landed the ecosystem optics (zio-schema
DynamicValuekit, zio-jsonAST +
JsonCursor, STM focus ops,Chunks) and with them a lot of genuinelymutatable code.
On the rebased tree
project zioIntegration; strykerdoes not printn/a. Itaborts, with the same exception #109 fixed for kyo, now in
zio/json/JsonOptics.scala:Same mechanism, same fix, so the second commit applies it: stryker4s 0.20.3
re-prints every mutated file through scalameta, and a single-method
significant-indentation
extensionclause is not aTerm.Block, so it comes backas the one-line form with the method's leading Scaladoc replayed between the two —
the forced newline lands
defin column 0. Four such clauses exist in the module:JsonCursor.optionalandJsonCodec.stringPrisminjson/JsonOptics.scala,Schema.dynamicPrismandBinaryCodec.prisminschema/SchemaOptics.scala.Bracing their bodies makes the printer emit the braced form and the newline
harmless.
ZioOptics's single-method clause already hoists its Scaladoc aboveextension, which dodges the same trap, so it is untouched.Verified rather than asserted:
javap -p -coverJsonOptics$package$andSchemaOptics$package$is byte-for-byte identical before and after thebracing, and
zioIntegration/testpasses unchanged (268 examples).kyo's numbers moved too, because #111's survivor-killing pass landed inbetween: 20 K / 4 S / 38.5% became 30 K / 5 S / 47.6%.
Evidence: the generated rows
Produced by the patched generator reading the real
report.jsonof a run onthis tree, not written by hand:
ziois a clean, high-signal row: 37 killed, 6 survived, and nono-coverage block at all. Everything stryker can mutate — the
DynamicValueand
zio.jsontree navigation, theJsonCursorwrite descent, theChunkindex guards — is genuinely executed by the suite. It needs no Notes caveat,
so it has none.
kyo's low total score is a macro artefact, not a coverage hole. All 28no-coverage mutants are in
RecordIsoMacro— quoted-macro code that expands atcompile time and leaves no runtime footprint for a test run to cover, the
same structural reason
genericsscores 0%. The covered column (85.7%) isthe one that describes the hand-written optics, and the Notes column says so.
Only the mutation table's two new rows are touched; the coverage table and the
composition matrix are left byte-identical, because a full in-place generator run
in a clone without a scoverage aggregate would have replaced the real coverage
numbers with "No scoverage aggregate report found".
The
n/abranch stays, without zio as its exampleNothing hits it today. It is kept anyway: the old code rendered a zero-mutant
module as the same em-dash it uses for "no report at all", conflating "nothing
to score" with "nothing ran", and printing
0.0%would read as a test-qualityfailure for a module that offers a mutator no purchase.
n/ais also whatstryker's own console prints. The page now states the distinction as a general
rule instead of pointing at zio.
The all-NoCoverage shape is deliberately left alone:
genericskeeps0.0%totalwith an em-dash covered score, because those mutants do exist — they are simply
never executed.
n/an/aIgnored(StringLiteral-excluded) mutantsn/an/aNoCoverage(thegenericsshape)0.0%—86.0%86.0%report.json——(+ the "runsbt mutationAll" note)Conflict resolution against
mainBoth conflicting hunks were keep-both — main and this branch edited the same
regions for unrelated reasons.
site/tools/gen-qa-report.py, comment block: kept test: 29 mutants killed in −2 net test lines — jsoniter oracle rewrite, circe bounds property, core exists/Index, and the equivalent-mutant catalogue #111's reworded"a note that claims a module is unscoreable MUST be retired" paragraph and
this branch's directory-vs-project-id comment above it.
site/tools/gen-qa-report.py,MUTATION_MODULES: kept test: 29 mutants killed in −2 net test lines — jsoniter oracle rewrite, circe bounds property, core exists/Index, and the equivalent-mutant catalogue #111's retiredavro/jsoniter notes (the old "Not scored:" strings are gone for good) and
appended the zio/kyo entries after them.
site/docs/quality-assurance.md, mutation table: same — test: 29 mutants killed in −2 net test lines — jsoniter oracle rewrite, circe bounds property, core exists/Index, and the equivalent-mutant catalogue #111's correctedavro/jsoniter rows plus the two new ones.
site/docs/quality-assurance.md, prose: main and this branch had each grown azio/kyo explainer, in different places, saying overlapping things. They are
merged into one block, keeping fix(kyo): brace the valuePrism extension so stryker4s can mutate the module #109's braced-
extensiondetail from main andthe covered-vs-total framing from here.
.github/workflows/quality.ymlauto-merged: main's newcostjob and thisbranch's stryker-loop edit are disjoint.
Nothing from either side was dropped. #111's Known equivalent mutants
catalogue is untouched.
Gates
Clone on Temurin 25, never the primary repo.
kyoconfirmed in the aggregate.sbt "++ 3" compile test— success, 0 errors;kyocompiled and its foursuites ran.
sbt benchmarks/compile— success (outside the root aggregate, breakssilently).
sbt mimaReportBinaryIssues— success.sbt "docs/mdoc; docs/laikaSite"— success; renderedquality-assurance.htmlcarries both new rows.
sbt scalafmtCheckAll "scalafixAll --check"— success, run last, afterpurging the
target/streams/*/scalafmt*and*scalafix*caches; it genuinelyre-read 20 source groups rather than reporting a cached pass.
zioIntegration/test— 268 examples, 0 failures, with the braces applied.project zioIntegration; stryker/project kyoIntegration; stryker, in theproject-switch form — never
<module>/stryker, which reports 100% NoCoverage.javap -p -cdiff over the two affected package objects — identical.🤖 Generated with Claude Code
https://claude.ai/code/session_0194EHFR4NamCpTHiqy7B74V