Skip to content

fix(fractal-dim): oracle-correct box-count & perimeter dimensions#366

Open
sameeul wants to merge 4 commits into
mainfrom
fix/fractal-dim-oracle-correctness
Open

fix(fractal-dim): oracle-correct box-count & perimeter dimensions#366
sameeul wants to merge 4 commits into
mainfrom
fix/fractal-dim-oracle-correctness

Conversation

@sameeul

@sameeul sameeul commented Jul 11, 2026

Copy link
Copy Markdown
Member

Summary

Corrects the accuracy of the 2D fractal-dimension features (FRACT_DIM_BOXCOUNT,
FRACT_DIM_PERIMETER) and validates them against external oracles. Follows up the sign/aggregation
fix already on main and makes the values correct against known ground truth.

Before → after:

shape feature before after truth / oracle
filled square box-count 1.75 2.000 2.000
Sierpinski triangle box-count 1.39 1.585 log2(3)
Koch snowflake perimeter out of range 1.266 log4/log3 = 1.262
154-px ROI perimeter out of range (0.32) 1.101 cross-method oracle 1.163 (agree 6%)

Root cause

Box counting is grid-registration sensitive. Power2PaddedImageMatrix padded to a power of two
strictly larger than the ROI (closest_pow2) and centered it, so the ROI never lined up
with the coarse boxes — halving the box size didn't cleanly quadruple the count, biasing the
dimension low on every shape. The perimeter path additionally truncated the perimeter to int
and added a broken tail chord (variable length + squared distance instead of the Euclidean chord),
pushing FRACT_DIM_PERIMETER out of the valid [1,2] range.

Changes (4 commits)

  1. feat(helpers): ceil_pow2 — tight next-power-of-two (256→256, not 512).
  2. fix: align the ROI to the box-counting gridceil_pow2 + origin placement in
    Power2PaddedImageMatrix and its oversized _NT variant.
  3. fix: adaptive box counting and a clean divider perimeter
    • Box-count: count boxes on a tight, origin-aligned grid and auto-switch on ROI size
      large ROIs use a single grid (fast early-exit tile scan), small ROIs shift the grid over 4
      origins and take the minimum count (as FracLac does). Perf-neutral (large-ROI ~16 ms, same as
      before; small ROIs ~35 µs).
    • Estimator: least-squares loglog_slope shared by both features (box -slope, perimeter
      1-slope).
    • Perimeter: clean closed-contour Richardson divider — double perimeter, dropped tail chord,
      mean-ruler-length scale.
  4. test: oracle-validated goldens and analytic/arbitrary-ROI tests — analytic-shape oracle
    tests (tests/python/test_fractal_dim_oracle.py, numpy-only, runs in CI); and the C++ fractal
    assertions moved to the 154-px pixelIntensityFeaturesTestData ROI, which fits enough box
    sizes for both features to be oracle-validated (the 26-px morphology fixture only fits 3
    scales, where the perimeter has no convergent oracle).

Oracle validation

Every row is a validation, not a self-referential snapshot. "Same method" = compared to a tool
running the identical algorithm (tight). "Cross-method" = compared to an independent method that
estimates the same dimension (convergent validity — agreement within ~10% is the accepted
criterion for two numerical methods mutually validating).

shape feature truth oracle (ImageJ/FracLac) nyxus validation
filled square box-count 2.000 2.000 2.000 analytic + same-method ✓
straight line box-count 1.000 1.000 1.000 analytic + same-method ✓
Sierpinski triangle box-count 1.585 1.585 1.585 analytic + same-method ✓
disk perimeter 1.000 1.02 analytic ✓
Koch snowflake perimeter 1.262 1.266 analytic ✓ (err 0.004)
154-px ROI box-count 1.389 1.389 same-method ✓ (C++ 1% + python)
154-px ROI perimeter 1.163 (edge box-count) 1.101 cross-method ✓ (6% < 10%) (C++ + python)

The last row is the key point: nyxus's divider perimeter (1.101) and an independent ImageJ
box-count-of-the-edge (1.163) — two different algorithms — agree to 6%. That is a genuine oracle
validation by convergent validity, asserted in both the C++ gtest (10% tolerance) and the python
suite. So the perimeter test is both a regression guard and an oracle check.

Both fractal features are validated on the 154-px ROI because it fits 5 box sizes; the smaller 26-px
morphology fixture (only 3 scales) is no longer used for them — there the two methods diverge (26%)
and the perimeter has no convergent oracle. C++ gtest: full suite passes. Python oracle: 6/6.
FracLac is GUI-only, so the box-count oracle was reproduced with a headless shifting-grid macro;
oracle values are baked in as offline constants.

Why the 26-px fixture was dropped for the fractal features

A fractal dimension is a scaling measurement — it only means something when it is fit across enough
scales. The 26-px shape2d_morphology_mask is a synthetic non-fractal blob authored to exercise basic
morphology and topology (its interior hole is what sets EULER_NUMBER = 0), not fractal analysis:

  • Too few scales. Its 6×7 bounding box pads to 8, giving only 3 box sizes (2, 4, 8), and its
    edge is just 18 pixels. A log–log slope over 3 points is barely determined and dominated by
    discretization — the box-count landing exactly on log2(3) there was a coincidence of that coarse
    coverage, not a robust measurement.
  • No convergent oracle at that resolution. With 3 scales the divider and box-count-of-edge disagree
    by 26% (1.263 vs 1.585), so the perimeter cannot be oracle-validated — asserting it would pin
    noise, not the feature.
  • The 154-px ROI clears the bar. It fits 5 box sizes, where the two methods converge to 6% and
    both features become genuinely oracle-checkable.

So the fractal features moved to the ROI that can actually validate them; the 26-px fixture keeps the
shape/topology goldens it was designed for. (Nothing about the algorithm changed with the fixture —
the analytic-shape tests, which do have ground truth, still guard correctness independently.)

Research note: divider vs box-counting for FRACT_DIM_PERIMETER

The 6% divider-vs-box-count spread on the 154-px ROI is small enough to validate (above), and it is
also exactly the well-documented divergence between the two numerical approaches:

  • They provably coincide for strictly self-similar fractals (box-counting / Minkowski,
    divider/compass, Hausdorff dimensions all equal) — which is why nyxus's divider recovers the Koch
    snowflake to 0.004. Divergence appears only on finite, digitized, non-self-similar curves, and
    it shrinks with resolution (26-px fixture: 26% at 3 box sizes; 154-px: 6% at 5).
  • The direction of divergence is not universal. A benchmark on the Australia coastline reports
    box-count 1.143 vs divider 1.130 (Δ≈0.013) and states "the divider dimension more accurately
    represents the irregularity of a coastline"
    (Sci. Rep. 2021).
  • The divider is the theoretically preferred boundary dimension (Mandelbrot's original coastline
    method); box-counting was adopted mainly because it removes the divider's bookkeeping ambiguities
    ("multiple forward intersections at a particular stepsize … the leftover portion of the curve") and
    automates for any set — not because it is more accurate for lines. See Klinkenberg,
    A review of methods used to determine the fractal dimension of linear features
    (Math. Geology 1994).

We keep the divider (its DIN ISO 9276-6 "structured walk" heritage) because it recovers the one
analytic ground truth essentially exactly. A multi-start averaging variant (the divider analog of
shifting grids) was implemented and rejected — it degraded the Koch recovery from 1.266 to 1.37.
A box-count-of-boundary variant would match FracLac's number exactly but is a different method,
not an accuracy upgrade; it remains an option if same-tool matching is later preferred.

🤖 Generated with Claude Code

sameeul and others added 3 commits July 11, 2026 02:12
Smallest power of two >= a, unlike closest_pow2 which is strictly greater. Used to pad
an ROI to a box-counting grid without wasting an octave (ceil_pow2(256)=256 vs 512).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Box counting is grid-registration sensitive. Power2PaddedImageMatrix (and its oversized
_NT variant) padded to a power of two strictly larger than the ROI and centered it, so the
ROI never lined up with the coarse boxes: halving the box size did not cleanly quadruple the
count and the dimension came out systematically low (a filled square read 1.75 instead of
2.0, Sierpinski 1.39 instead of 1.585). Pad tight (ceil_pow2) and place the ROI at the grid
origin. Validated against the ImageJ/FracLac box-count oracle.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Box counting (calculate_boxcount_fdim): count boxes directly from the ROI pixel cloud on a
tight, origin-aligned grid, and auto-switch on ROI size - large ROIs use a single grid via the
padded mask matrix (fast early-exit tile scan), small ROIs shift the grid over 4 origins and
take the minimum count (as FracLac does), which removes the residual registration bias that
only bites when few box sizes fit. Recovers analytic shapes exactly (square->2, line->1,
Sierpinski->log2(3)) and reproduces the ImageJ/FracLac shifting-grid oracle on real ROIs.

Estimator: replace the old mean-of-local-slopes (an endpoint slope) with loglog_slope, a
least-squares fit of log(measure) vs log(scale) shared by both features (box-count D = -slope,
perimeter D = 1 - slope).

Perimeter (calculate_perimeter_fdim): rewrite as a clean closed-contour Richardson divider -
perimeter as a double (was truncated to int), drop the broken tail chord (it added a
variable-length closing segment and used squared distance instead of the Euclidean chord),
and scale by the actual mean ruler length. Recovers the Koch snowflake (log4/log3=1.2619) and
a smooth disk (1.0); the single-vertex walk matches analytic truth (a multi-start average was
tried and rejected - it degraded the Koch recovery to 1.37).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…I tests

Validate both fractal features against external oracles:

- Analytic shapes (tests/python/test_fractal_dim_oracle.py): box-count square->2, line->1,
  Sierpinski->log2(3); perimeter disk->1, Koch snowflake->log4/log3. numpy-only, runs in CI.
- Arbitrary 154-px ROI: box-count vs the ImageJ shifting-grid oracle (same method, 1.389, tight)
  and the divider perimeter vs box-count-of-edge (cross-method, 1.163, agree ~6%).

Move the C++ shape2d fractal assertions to the 154-px pixelIntensityFeaturesTestData ROI
(test_shape2d_fractal_dimension_154px_oracle): it fits 5 box sizes, enough for BOTH features to
be oracle-validated - box-count same-method (1% tol) and perimeter cross-method (10% tol). The
26-px morphology fixture only fits 3 scales, where the perimeter has no convergent cross-method
oracle, so the fractal features are no longer asserted there. Other shape2d goldens
(convex hull, extrema, circles, ...) stay on the 26-px fixture.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@sameeul sameeul force-pushed the fix/fractal-dim-oracle-correctness branch from a946076 to 38f6989 Compare July 11, 2026 09:37
@sameeul sameeul requested a review from darkclad July 11, 2026 10:23
// and biased the dimension low (a filled square read 1.75 instead of 2.0).
int bigSide = std::max(aabb.get_width(), aabb.get_height());
StatsInt paddedSide = Nyxus::closest_pow2 (bigSide);
StatsInt paddedSide = Nyxus::ceil_pow2 (bigSide);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nyxus::closest_pow2 is not used anymore, can be removed

Comment thread src/nyx/helpers/helpers.h
inline int ceil_pow2(const int a)
{
int p = 1;
while (p < a)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overflow issue for a > 2^30, p <<= 1 goes negative and goes into infinite loop
Suggesting either rewrite it like this
inline uint32_t ceil_pow2(uint32_t a) {
if (a == 0) return 1;
if (a > (1u << 31)) return 0; // or UINT_MAX / assert — pick a policy
uint32_t x = a - 1;
x |= x>>1; x|=x>>2; x|=x>>4; x|=x>>8; x|=x>>16;
return x + 1;
}

or switch to uint64_t

#include
inline uint64_t ceil_pow2(uint32_t a) {
return a <= 1 ? 1 : std::bit_ceil<uint64_t>(a);
}

std::vector<std::pair<double, double>> coverage;

coverage.push_back({ s, cnt });
if (paddedSide > 32)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Magic number, need a comment on why we look for 32

// boundary dimension); Nyxus' Richardson divider agrees to ~6% (cross-method
// convergent validity), asserted with a 10% tolerance -> 1.163
static std::unordered_map<std::string, double> oracle_fractal_154px_golden_values{
{"FRACT_DIM_BOXCOUNT", 1.3891},

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit (non-blocking): flag that 1.3891 is a same-method pin, not external ground truth.
This value comes from the ImageJ/FracLac shifting-grid box count — the same algorithm Nyxus runs — so the 1% assertion below is really a regression/self-consistency pin, not an independent oracle. That's fine and the header comment (L72) already says "SAME method as Nyxus", but since it sits in a map literally named ...oracle... next to the genuinely cross-method FRACT_DIM_PERIMETER, it's easy to misread 1.3891 as external corroboration. Suggest a one-line inline marker so the distinction survives a skim, e.g.:

{"FRACT_DIM_BOXCOUNT", 1.3891}, // self-consistency pin: SAME shifting-grid method as Nyxus (not independent validation)
{"FRACT_DIM_PERIMETER", 1.163}, // cross-method (edge box-count vs divider) — the real external check
The independent validation here is the analytic shapes (square→2, line→1, Sierpinski→log₂3, …) and the cross-method perimeter; the 154-px box-count is a pin.

# algorithm than nyxus' Richardson divider, but estimates the SAME boundary-complexity
# dimension (box dim == compass/divider dim for boundary curves). The two agree to
# 0.06 here - cross-method convergent validity -> 1.163.
ORACLE_BOXCOUNT_154 = 1.389 # same-method oracle; nyxus computes 1.389 (<0.001)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit (non-blocking): same as the C++ side — mark this as a same-method pin.
ORACLE_BOXCOUNT_154 = 1.389 is the offline ImageJ shifting-grid box count, i.e. the same method Nyxus uses, so abs(bc - 1.389) < 0.02 is a self-consistency check rather than independent corroboration (the trailing comment "<0.001" makes that clear — a near-exact match is expected precisely because it's the same algorithm). Consider renaming to make intent obvious at the call site, or at minimum keep the comment explicit:

ORACLE_BOXCOUNT_154 = 1.389 # SAME-method pin (ImageJ shifting-grid == nyxus); not independent
ORACLE_PERIMETER_154_EDGE = 1.163 # cross-method (edge box-count vs divider) — independent check
No change to the assertion itself needed; the tight 0.02 tolerance is correct for a same-method pin. The genuinely independent Python checks remain the analytic-shape tests and the cross-method perimeter.


std::vector<Pixel2> K;
r.merge_multicontour(K);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is worth checking that K is truly single countour
// The divider assumes ONE closed contour. Multi-contour ROIs (holes / disconnected
// fragments) concatenate into K and inject spurious seam chords that bias the dimension.
assert(r.contour_count() <= 1 && "perimeter fdim assumes single contour");

// Fractal dimensions are validated on the irregular 154-px pixelIntensityFeaturesTestData ROI, which
// fits enough box sizes for both features to be oracle-validated (the 26-px morphology fixture only
// fits 3 scales, where the perimeter has no convergent cross-method oracle).
static void calculate_fractal_154px_feature_values(std::vector<std::vector<double>>& fvals)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit (non-blocking, maintenance watch-point): this harness hand-copies the production ROI-preprocessing path.
load_test_roi_data only fills raw_pixels, so L339–342 re-implement inline what the real masked loader does — make_nonanisotropic_aabb() → aux_image_matrix.allocate(...) → calculate_from_pixelcloud(...) → initialize_fvals() (plus the hardcoded label 102). That's correct for the current loader and the oracle values are meaningful, but nothing keeps it in sync: if production ever changes how an ROI is finalized (AABB/anisotropy convention, matrix fill, a new normalization step), this fixture keeps building the ROI the old way and would still pass — validating fractal dimensions against an ROI the real pipeline would never produce.
Two ways to de-risk, in order of preference:

route the test through the same finalize entry point the loader uses instead of duplicating its steps, so a production change flows in automatically; or
if no shared entry point exists, add a // KEEP IN SYNC WITH marker next to the make_nonanisotropic_aabb block and an assert pinning the resulting AABB/matrix dims, so drift is visible. Not a blocker for this PR (the reconstruction is already commented and matches production today) — just flagging the hidden coupling so whoever touches ROI preprocessing later updates it here too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants