Skip to content

Add histogram axes, accumulators, and frequency views - #108

Merged
jmh530 merged 46 commits into
libmir:masterfrom
jmh530:histogram
Sep 13, 2026
Merged

jmh530 merged 46 commits into
libmir:masterfrom
jmh530:histogram

Conversation

@jmh530

@jmh530 jmh530 commented Sep 13, 2026

Copy link
Copy Markdown
Contributor

Add histogram functionality to mir-stat, including axis construction,
incremental counting, and frequency calculations.

The implementation provides:

  • Integral, regular, transformed, variable, enum, and category axes.
  • Configurable endpoint closure, underflow, overflow, and circular endpoints.
  • Histogram accumulators with caller-supplied storage, plus reference-counted
    construction through rchistogram.
  • Read-only bin views that reflect current counts.
  • Frequency accumulators with optional output-type selection, relative and
    cumulative frequencies, and live frequency views.
  • Owning cumulative-frequency snapshots and caller-supplied output buffers.
  • Sturges, Scott, and Freedman–Diaconis bin-count selection.

Boundary validation and lookup account for floating-point rounding. Tests
cover extreme values, qualifiers, borrowed-view lifetimes, and @nogc
operations. CI now includes DIP1000 and optimized release configurations.

Current limitations

Multiple axes record separate marginal counts, not joint multidimensional
bins. Joint storage and flow bins, weighted histograms, per-bin statistical
accumulators, and dynamically widening counters remain deferred.

Borrowed frequency views require their source accumulator to remain alive
and in place. Their safe use depends on compiler escape checking.

Validation

All six local configurations pass with 46 modules each:

  • DMD: default, DIP1000, and optimized release.
  • LDC: default, DIP1000, and optimized release.

Hosted Linux/Windows CI remains to be verified on the PR.

Assistance

Developed with AI assistance from Codex (GPT-6 Astra Medium).

jmh530 added 30 commits May 19, 2021 10:24
AI-Assisted-By: Codex (GPT-6 Astra Medium)
AI-Assisted-By: Codex (GPT-6 Astra Medium)
AI-Assisted-By: Codex (GPT-6 Astra Medium)
AI-Assisted-By: Codex (GPT-6 Astra Medium)
AI-Assisted-By: Codex (GPT-6 Astra Medium)
AI-Assisted-By: Codex (GPT-6 Astra Medium)
AI-Assisted-By: Codex (GPT-6 Astra Medium)
AI-Assisted-By: Codex (GPT-6 Astra Medium)
AI-Assisted-By: Codex (GPT-6 Astra Medium)
Store read-only handles while preserving shared count updates and
reference-counted ownership. Allow const views to be saved and sliced
into mutable cursors, and add VariableAxis.lightConst support.

Add documentation examples and regression tests for const access,
storage lifetime, and range operations. All 49 test modules pass.

AI-Assisted-By: Codex (GPT-6 Astra Medium)
Add frequencyBins with live counts and frequencies, const access,
slicing, and independent cursors. Isolate escape-check detection so
borrowing requires @System when the necessary checks are unavailable.

Co-locate bin views with their accumulators and extract shared internal
helpers. Add documentation examples and behavior and lifetime tests.

Enable the histogram suite in the DIP1000 build. Fix VariableAxis
lookup safety by borrowing the search slice while retaining ownership
in the axis.

AI-Assisted-By: Codex (GPT-6 Astra Medium)
Add cumulativeFrequency with a configurable floating-point output type,
defaulting to double. Sum current counts through the selected bin,
including underflow, and divide by the total including overflow.
Return NaN when the total is zero.

Make both histogram flow accessors const. Add documentation examples
and tests covering output types, flow options, storage types, const
access, insertion, merging, and invalid indices.

AI-Assisted-By: Codex (GPT-6 Astra Medium)
Add cumulativeFrequencies with a configurable floating-point output
type, defaulting to double. Compute all ordinary-bin cumulative
frequencies in one pass and return an independently owned,
reference-counted Mir slice.

Include underflow in cumulative counts and overflow in the total.
Return NaN values when the total is zero.

Add documentation examples and tests for flow handling, output types,
const access, and snapshot independence after source updates and
destruction.

AI-Assisted-By: Codex (GPT-6 Astra Medium)
Add a cumulativeFrequencies overload for writable floating-point arrays
and one-dimensional Mir slices. Infer output precision from destination
elements and require one element per ordinary bin.

Reuse the destination overload when allocating snapshots. Document the
requirement that output storage not overlap accumulator counts.

Add documentation examples and tests for buffer reuse, strided slices,
const sources, stack output, and invalid destinations.

AI-Assisted-By: Codex (GPT-6 Astra Medium)
Use shared boundary calculations for regular and transformed axes,
preserving the supplied endpoints. Check estimated bin indices against
the original observation and fall back to binary search when needed.

Reject invalid grids and coincident adjacent boundaries during
assertion-enabled construction.

Add documentation examples and regression tests covering shared edges,
extreme float/double/real values, interval closure, circular endpoints,
and transformed-bin membership.

AI-Assisted-By: Codex (GPT-6 Astra Medium)
Reject nonpositive IntegralAxis bin counts and upper bounds that do
not exceed the lower bound. For integral count and bin types, check
that the count fits BinType and the endpoint addition cannot overflow.

Require VariableAxis boundaries to be strictly increasing with a bin
count that fits CountType. Preserve constant-time const conversion by
reusing validated boundaries.

Check that accumulator storage matches its axes. Document multiple-axis
marginal histograms and add regression tests for invalid axes, integral
endpoint limits, and mismatched storage.

AI-Assisted-By: Codex (GPT-6 Astra Medium)
Treat each argument as a separate observation for single-axis
histograms and frequency accumulators. Require exactly one compatible
coordinate per axis for multiple-axis histograms.

Share observation-type checks and accept const and immutable numeric
inputs. Update frequency totals after each successful insertion.

Add compiler checks and regression tests for argument counts, invalid
types, qualified inputs, category strings, flows, and partial batches.

AI-Assisted-By: Codex (GPT-6 Astra Medium)
Make underflow, overflow, and their relative-frequency accessors
callable on const frequency accumulators.

Add @nogc tests for histogram construction, insertion, merging,
category handling, bin views, frequency reads, cumulative snapshots,
caller-supplied output, and break selection.

Check borrowed frequency-view traversal without GC allocation in
both escape-checking modes.

AI-Assisted-By: Codex (GPT-6 Astra Medium)
Replace histogram-specific test guards with mir_stat_test and restore
the full suite in default, coverage, and DIP1000 builds.

Rename the DIP1000 lifetime-test flag to mir_stat_test_lifetime.

AI-Assisted-By: Codex (GPT-6 Astra Medium)
Move floating-to-count conversion into a helper that DMD does not
inline, avoiding compiler crashes when callers compare bin indices
with zero. Keep the rest of axis lookup inlineable.

Add regression coverage for inlined lookup with float, double, and
real inputs under both interval conventions.

AI-Assisted-By: Codex (GPT-6 Astra Medium)
Compare the real and imaginary components using a small tolerance
instead of exact equality, accommodating rounding differences in
optimized builds.

AI-Assisted-By: Codex (GPT-6 Astra Medium)
 AI-Assisted-By: Codex (GPT-6 Astra Medium)
Reject nonfinite bounds and collapsed floating-point bin boundaries.
Use shared rounded boundaries for bin descriptions and lookup, while
preserving direct integer lookup and the DMD conversion workaround.

Add regression coverage for float, double, and real at precision limits
and fractional origins, including both closure options and circular axes.

AI-Assisted-By: Codex (GPT-6 Astra Medium)
Use exact arithmetic for integer spans and integer widths, preserving
distances between nearby large values. Handle overflowing floating-point
spans and ratios that underflow to zero.

Support integer arrays directly and validate inputs, widths, and count
representability before conversion. Document constant-input behavior.

Add regression coverage for extreme values, integer widths, invalid
inputs, and CountType limits, including @nogc tests.

AI-Assisted-By: Codex (GPT-6 Astra Medium)
Correct parameter descriptions, sturges references, and documentation
typos. Clarify that the remaining storage-shape work concerns joint
histograms; marginal storage already has validation.

Remove trailing whitespace from histogram modules while preserving
CRLF line endings and final newlines.

AI-Assisted-By: Codex (GPT-6 Astra Medium)
Provide self-contained guidance for routine Mir development, numerical
and performance work, and compiler investigations.

Encourage reuse of established findings and proportional verification.
Allow optional use of other available skills without requiring companion
skills. Retain useful regression tests when removing compiler workarounds.

AI-Assisted-By: Codex (GPT-6 Astra Medium)
Check transform compatibility after BinType is deduced. Probe template
transforms through the Mir invocation path so incompatible candidates
are rejected without hard compilation errors.

Add regression coverage for explicit and inferred types and inverses,
axis options, boundary counts, and invalid transforms.

AI-Assisted-By: Codex (GPT-6 Astra Medium)
Move the elementCount import into the shared template scope so array
inputs compile with both default and explicit count types.

Add regression coverage for mutable and const arrays and agreement
with slice inputs.

AI-Assisted-By: Codex (GPT-6 Astra Medium)
Wrap assignments in statement blocks inside __traits(compiles) to avoid parser errors while preserving checks that read-only data cannot be modified.

AI-Assisted-By: Codex (GPT-6 Astra Medium)
Avoid unreachable statements in compile-time argument validation. Check borrowed-view escape rejection in @safe code without requiring older compilers to enforce the same restriction in @System code.

AI-Assisted-By: Codex (GPT-6 Astra Medium)
Import DefaultCountType within each documentation example that uses it so the examples compile independently of the axis module's imports.

AI-Assisted-By: Codex (GPT-6 Astra Medium)
Add missing parameter descriptions and remove entries that do not match their overloads. Correct VariableAxis parameter names so histogram documentation builds with warnings treated as errors.

AI-Assisted-By: Codex (GPT-6 Astra Medium)
@jmh530
jmh530 marked this pull request as ready for review September 13, 2026 17:26
@jmh530
jmh530 merged commit 04f1564 into libmir:master Sep 13, 2026
9 checks passed
@jmh530
jmh530 deleted the histogram branch September 15, 2026 23:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant