Skip to content

Convert the remaining tolerance-bound fixture specs to exact f32 match counting #256

Description

@wormeyman

Split out of #253. Continues the programme #162 started.

Why

A bound cannot tell "close" from "identical". #253 measured four defects that passed the bound they were meant to be caught by, while destroying bit-exactness wholesale:

op planted defect worst exact its bound
plain multioctave drop f32 on amp*basis 3.725e-7 173/266 < 5e-7 passes
plain multioctave drop f32 on the amp chain 4.768e-7 215/266 < 5e-7 passes
variable persistence drop f32 on the gain multiply 1.907e-6 252/266 < 4e-6 passes
quick multioctave scale/amp by powers not a chain 4.971e-5 143/190 < 5e-5 passes

93 points stop matching the game in row 1 and the bound never moves off green. A bound is not "tight" because it sits near the residual - it is blind by construction.

Two bounds found carrying enormous slack in passing: amplitude_corrected at < 5e-3 against a measured 1.788e-7 (~28,000x), and cliffiness_basic at < 5e-6 against 5.960e-8 (~84x, tightened in #253).

Scope

A sweep found roughly 30 more specs that grade a ported primitive or expression against a committed oracle fixture with toBeLessThan / toBeLessThanOrEqual / toBeCloseTo, or a tolerance predicate wrapped in .toBe(true).

Nearly all of their fixtures are 100% f32-representable, which is what makes exact-match scoring legal. Measured per fixture by walking the compared arrays and counting Math.fround(v) === v failures. Some of the larger ones:

spec bound(s) fixture f32?
vulcanusResources.spec.ts 13 bounds, 6e-5 to 3.5e-3 oracle-vulcanus-resources 15190/15190
regularPatches.spec.ts abs 1.0 / rel 1e-2 oracle-resource-regular 16420/16420
cliffOrientationMargin.spec.ts < 0.1 ...corner-fields-entity-regions 12675/12675
fulgoraExpressions.spec.ts ~67 sites, 23 already at 0 4 fulgora fixtures all clean
vulcanusBiomes.spec.ts 8 bounds, 1e-4 to 5e-4 oracle-vulcanus-biomes 3472/3472
treeOracle.spec.ts 1e-3 / 9e-4 / 2e-4 oracle-trees, -controls 442+51
cliffFields.spec.ts, cliffResidual.spec.ts abs 1.0, < 1e-3 oracle-cliff-elevation 2048/2048
vulcanusRocks.spec.ts 5 bounds, 5e-5 to 5e-4 oracle-vulcanus-rocks 434/434
elevationNauvis / elevationLakes / elevationIsland 8e-3, 1e-4 4 elevation fixtures 26-52 each
basisNoiseSeeding.spec.ts < 1e-5 basis-noise-seeding.game 547/547
startingSpotAtAngle.spec.ts < 3e-6 oracle-starting-spot 152/152
randomPenalty.spec.ts < 1e-6 oracle-random-penalty 40/40
vulcanusSpawn / vulcanusCracks / vulcanusHelpers / vulcanusClimate / vulcanusElevation various 5 fixtures all clean

One genuine exception found: spot-selection.game.json's spots[][2] peak field is 0/413 f32 - it was captured with deliberate 3-decimal rounding, per that spec's own comment. A bound is the correct instrument there. Do not convert it.

Also note regularPatches.spec.ts's own header already documents its abs/rel tolerance pair hiding two independent opposite-signed bugs at once, so it is a good early candidate.

How to do one

  1. Confirm the fixture's compared values are all f32 (Math.fround(v) === v), and add a test asserting it, so the scoring cannot quietly stop being valid.
  2. Confirm its positions are on the 1/256 grid - see moisture and aux keep a 1-4 ulp residual that the 1/256 snap does not explain #255. If they are not, the bound grades the capture and converting it is premature.
  3. Measure the current exact count and worst residual.
  4. If worst is already 0, assert toBe(0) and the exact count and delete the bound. If not, that gap is a finding, not a tolerance to keep - the last three times it was chased it turned out to be an f64 evaluation or an aliased constant.
  5. Plant a defect and confirm the new assertion goes red. Verify the planted break actually applied.

Good subagent fan-out: the specs barely interact. Give any agent that edits files its own worktree.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requesttoolchain-ciBuild, test, CI, dependencies, developer tooling

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions