Skip to content

Address opendp-num fuzz findings (DASHU-007/008/015/022/026)#94

Merged
cmpute merged 5 commits into
masterfrom
worktree-opendp-bugfix
Jul 24, 2026
Merged

Address opendp-num fuzz findings (DASHU-007/008/015/022/026)#94
cmpute merged 5 commits into
masterfrom
worktree-opendp-bugfix

Conversation

@cmpute

@cmpute cmpute commented Jul 23, 2026

Copy link
Copy Markdown
Owner

Reviewed the opendp-num differential/property-fuzz findings against the current source (~/opendp-num/findings). Of the nine dashu findings:

  • DASHU-020 / DASHU-021 — already fixed by Round to_f64/to_f32 once, to the subnormal-aware precision #91 (verified: convert_base_odd/significand_bits + the subnormal-halfway and high-precision regression tests are present).
  • DASHU-023 / DASHU-024 — deferred (directed rounding at FBig's exponent-range extremes; both masked by the OpenDP adapter). Tracked with TODO comments at the exp_internal overflow branch and Context::powi. Fixing properly needs mode-aware range saturation — out of scope here.
  • The five below are fixed.

DASHU-015 (dashu-int) — to_f64 reports inexact as Exact

to_f64_small used a saturating f as DoubleWord round-trip for the exactness test; at DoubleWord::MAX the value rounds up to 2^BITS, the cast clamps back to DoubleWord::MAX, and the comparison reports Equal → wrongly tagged Exact. Detect the saturation case first. Same guard applied to the 16/32-bit RefLarge → f64 path.

DASHU-026 (dashu-float) — round_fract debug assert OOMs

The debug assert built B^precision. For a sparse sticky tail (where precision is the exponent gap — reached via exp_m1 of a large-magnitude input) this allocated gigabytes in debug builds. Replaced with a log2_bounds-based precondition check that allocates nothing and only fires on a proven violation.

DASHU-022 (dashu-float) — exact results panic under unlimited precision

assert_limited_precision ran before the exact zero/one shortcuts, so precision-0 values (FBig::try_from(0.0), FBig::ONE/ZERO) panicked in exp/exp_m1/sqrt/ln/ln_1p despite exact results. Shortcuts hoisted above the assertion.

DASHU-008 (dashu-int) — GCD scratchpad under-allocation

GCD reserved scratch once from the initial operand lengths, but each euclidean step dispatches on the current lengths, so a later lopsided Burnikel-Ziegler step was under-reserved → "internal error: not enough memory allocated" (runtime .expect, so debug and release). Now sized for the loop-wide worst case: mul::memory_requirement_up_to(rhs_len, rhs_len / 2) (independent of lhs_len; returns zero_layout for small operands automatically). Applies to both the gcd and extended-gcd reservations.

DASHU-007 (dashu-float) — add correctly-rounded log2

Previously only the f32-precision log2_bounds estimate existed, so directed log2 was wrong by many ULPs. Added FBig::log2 / Context::log2 / CachedFBig::log2 computed as ln(x)/ln(2) at an elevated working precision. The result magnitude tracks the division's error amplification, so a few guard digits certify the final round across the whole range. log2(2^k) = k; log2(1) = 0 is certified Exact via the shortcut.

Verification

  • cargo test --workspace --exclude dashu-python — all pass (incl. new regression tests; 85 float doctests)
  • cargo clippy --all-features --all-targets --workspace --exclude dashu-python -- -D warnings — clean
  • cargo fmt --all -- --check — clean
  • Sharpness: stashing the DASHU-008 fix reproduces the exact memory.rs:150 panic; the DASHU-026 regression (exp_m1(-2^62), which under the old assert would try to allocate ~8×10¹⁷ bits) completes in 0.00 s.
  • Changelogs updated (## Unreleased) for dashu-int, dashu-float, dashu-ratio.

🤖 Generated with Claude Code

Reviewed the opendp-num differential/property-fuzz findings against the
current source. DASHU-020/021 are already fixed by PR #91; DASHU-023/024
(directed rounding at FBig's exponent-range extremes) are tracked with
TODO comments and deferred. This commit fixes the remaining five.

DASHU-015 (dashu-int, incorrect-result): `to_f64_small` reported an inexact
conversion as `Approximation::Exact` at the `DoubleWord::MAX` boundary. The
exactness test used a saturating `f as DoubleWord` round-trip, which clamps
back to `DoubleWord::MAX` when the value rounds up to `2^BITS`. Detect that
saturation case before the comparison; apply the same guard to the 16/32-bit
RefLarge -> f64 fast path.

DASHU-026 (dashu-float, debug panic/OOM): `round_fract`'s debug assert built
`B^precision`; for a sparse sticky tail (precision == the exponent gap, e.g.
from `exp_m1` of a large-magnitude input) this exhausted memory in debug
builds. Replace it with a `log2_bounds`-based precondition check that
allocates nothing and only fires on a proven violation.

DASHU-022 (dashu-float, panic): `assert_limited_precision` fired before the
exact zero/one shortcuts, so values carrying unlimited precision (precision
0) -- `FBig::try_from(0.0)`, `FBig::ONE`/`ZERO` -- panicked in exp/exp_m1/
sqrt/ln/ln_1p despite mathematically exact results. Hoist the exact
shortcuts above the assertion.

DASHU-008 (dashu-int, panic): GCD of two large, similarly-sized integers
aborted with "internal error: not enough memory allocated". The scratchpad
was reserved once from the initial operand lengths, but each euclidean
step's division dispatches on the current lengths, so a later lopsided
Burnikel-Ziegler step (divisor > 48 words, quotient > 32 words) was
under-reserved. Size it for the worst case reachable in the loop:
`mul::memory_requirement_up_to(rhs_len, rhs_len / 2)`. Applies to both the
gcd and extended-gcd reservations.

DASHU-007 (dashu-float, missing feature): add a correctly-rounded
`FBig::log2` / `Context::log2` / `CachedFBig::log2`, computed as
`ln(x)/ln(2)` at an elevated working precision. Previously only the
f32-precision `log2_bounds` estimate existed, so directed log2 was wrong by
many ULPs. The result magnitude tracks the division's error amplification,
so a few guard digits certify the final round across the whole range.

Each fix includes a regression test; the DASHU-008 test reproduces the exact
memory.rs panic under the old code.

Co-Authored-By: Claude <noreply@anthropic.com>
@Shoeboxam

Copy link
Copy Markdown
Contributor

Some admittedly robotic feedback that is nonetheless helpful:

log2 finding

The PR computes ln(x) / ln(2) once at a fixed elevated precision and then rounds to the requested precision. That is only near-correct: an intermediate result near a rounding boundary can double-round to the adjacent value. For example, with (B=2), (p=53), and (x=492\cdot2^{-135}), the PR uses 62 working bits and returns a result one ULP too low.

Directed rounding is also not preserved by rounding both operands in the same direction: Up(ln(x)) / Up(ln(2)) is not necessarily an upper bound because enlarging the positive denominator lowers the quotient. The current implementation is at lines 687–693 of the diff. ([GitHub]1)

Suggested implementation

Rebase on the Ziv machinery from PR #92 and use outward interval division, retrying until both interval endpoints round to the same target value. PR #92 already follows this certified-error-interval approach for transcendental functions. ([GitHub]2)

impl<R: ErrorBounds> Context<R> {
    fn log2_internal<const B: Word>(
        &self,
        x: &Repr<B>,
        mut cache: Option<&mut ConstCache>,
    ) -> Rounded<FBig<R, B>> {
        if x.is_one() {
            return Exact(FBig::ZERO);
        }

        self.ziv_interval(self.base_guard_digits::<B>() + 4, |guard| {
            let p = self.precision + guard;
            let work = Context::<mode::HalfEven>::new(p);

            // Each result satisfies true value ∈ [value - error, value + error].
            let (lx, ex) =
                work.ln_compute(x, p, false, reborrow_cache(&mut cache));

            let two = Repr::new(IBig::from(2), 0);
            let (l2, e2) =
                work.ln_compute(&two, p, false, reborrow_cache(&mut cache));

            let nx_lo = &lx.repr - &ex.repr;
            let nx_hi = &lx.repr + &ex.repr;
            let d_lo = &l2.repr - &e2.repr;
            let d_hi = &l2.repr + &e2.repr;

            debug_assert!(d_lo > Repr::zero());

            let down = Context::<mode::Down>::new(p);
            let up = Context::<mode::Up>::new(p);

            // Denominator is positive. Select endpoint pairing according
            // to the numerator sign.
            if nx_lo >= Repr::zero() {
                (
                    down.div(&nx_lo, &d_hi).unwrap().value(),
                    up.div(&nx_hi, &d_lo).unwrap().value(),
                )
            } else if nx_hi <= Repr::zero() {
                (
                    down.div(&nx_lo, &d_lo).unwrap().value(),
                    up.div(&nx_hi, &d_hi).unwrap().value(),
                )
            } else {
                // The ln(x) interval still crosses zero. Returning a broad
                // interval forces ziv_interval to retry at higher precision.
                (
                    down.div(&nx_lo, &d_lo).unwrap().value(),
                    up.div(&nx_hi, &d_lo).unwrap().value(),
                )
            }
        })
    }
}

ziv_interval should round both outward endpoints under R; it accepts only when they produce the same target value, and otherwise increases guard. It should report Exact only when the interval is a singleton; otherwise conservatively report Inexact.

Jacob Zhong and others added 3 commits July 24, 2026 23:19
The DASHU-NNN tokens reference an external fuzz-findings corpus that other
developers don't have context for. Rephrase the affected comments (two
production TODOs and several test docstrings) to describe the limitation or
regression in self-contained terms.

Co-Authored-By: Claude <noreply@anthropic.com>
Replace the xorshift-based `big_from_seed` in the two DASHU-008 regression
tests (`test_gcd_large_lopsided_reduction`, `rbig_reduce_large_lopsided`)
with fixed all-ones operands sized by word count. The tests still reproduce
the exact `memory.rs:150` panic under the old code (verified) but now use
deterministic inputs.

Add a Code-style rule to AGENTS.md: in-crate tests must use fixed,
deterministic inputs — never random/property-test generators — except when
testing the `rand` integration itself. Randomized input belongs under
`fuzz/`.

Co-Authored-By: Claude <noreply@anthropic.com>
On 32-bit targets `isize` tops out at ~2.1e9, so for x = -2^62 the value
floor(x/ln2) ≈ -6.6e18 overflows isize and exp_internal takes its (deferred)
overflow branch — returning Exact(-1) — instead of the round_fract path this
test targets. A sharp OOM needs an exponent gap large enough that 2^gap
exceeds memory, yet still fitting isize; that window only exists on 64-bit.
Gate the test accordingly. The underlying round_fract fix is arch-independent.

Co-Authored-By: Claude <noreply@anthropic.com>
@cmpute

cmpute commented Jul 24, 2026

Copy link
Copy Markdown
Owner Author

@Shoeboxam Thanks for your note on the guaranteed rounding. The whole Ziv thing is on another PR (#92), however unfortunately that requires break changes, so I will keep the simple version of log2 here.

The earlier ID cleanup dropped the TODO tag along with the issue ID.
Keep `TODO:` so the limitations still surface in a TODO grep.

Co-Authored-By: Claude <noreply@anthropic.com>
@cmpute
cmpute merged commit 1b96010 into master Jul 24, 2026
16 checks passed
cmpute pushed a commit that referenced this pull request Jul 24, 2026
Resolve conflicts from PR #94 landing on master while ziv rewrote the same
surfaces with the Ziv engine:

- log2: both branches added it. Keep ziv's guaranteed-correct version (Ziv
  interval division + power-of-two exact shortcut); drop #94's near-correct
  single-division variant, its log2_internal, and its log2 tests (subsumed by
  ziv's directed-vs-oracle regressions in both bases).
- exp.rs/log.rs: keep ziv's Ziv rewrites. Restore assert_limited_precision in
  ln_internal/exp_internal after the exact shortcuts (the merge had dropped it,
  so ln(2)/exp(2) at unlimited precision stopped panicking — #94's DASHU-022
  moved the shortcuts ahead of the assert, but the assert itself must remain
  for the non-exact path).
- fbig_cached_ops: keep ziv's R: ErrorBounds forwarding block (ziv's
  transcendentals require ErrorBounds); drop #94's R: Round additions.
- integer/CHANGELOG: combine both fix lists (ziv's NTT zero-operand fix +
  #94's to_f64 saturation and GCD scratchpad fixes).

Drop test_exp_m1_large_negative_no_oom: exp_m1(-2^62) under a directed mode
needs to certify that -1 + 2^-6.6e18 exceeds -1, which requires ~6.6e18 bits —
the Ziv loop runs until precision explodes. That is the deferred directed-
rounding-at-exp-extremes limitation (the non-Ziv single-round exp #94 added it
for didn't have this). The DASHU-026 round_fract concern it also covered is
already present on ziv.

Auto-merged cleanly (kept): #94's to_f64 saturation, GCD scratchpad, sqrt
precision-0 shortcut, round_fract debug-assert, and the AGENTS.md fixed-input
test rule.

Co-Authored-By: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants