Skip to content

perf: build PAGE trace from dense memory store#858

Open
ColoCarletti wants to merge 3 commits into
mainfrom
perf-page-dense-gen
Open

perf: build PAGE trace from dense memory store#858
ColoCarletti wants to merge 3 commits into
mainfrom
perf-page-dense-gen

Conversation

@ColoCarletti

Copy link
Copy Markdown
Collaborator

What

Generate the PAGE trace tables by reading each cell's final (value, timestamp) straight from the dense per-page memory store, instead of first building a sparse FinalStateMap and doing one (mostly-miss) lookup per page offset.

Why

Those per-offset sparse-map lookups were the dominant cost of PAGE generation.
Reading the dense page slice directly is one indexed read per offset, no hashing.

Impact (ethrex 5-tx block, RTX 5090)

  • PAGE table generation: 658ms → 260ms
  • Trace generation: 2.63s → 2.25s (−15%)
  • Total prove: 12.8s → 12.5s (−2.4%)

Produces the exact same PAGE trace — prove + verify pass.

Changes (3 files)

  • prover/src/paged_mem.rs: add PagedMem::page_data() (dense per-page slice).
  • prover/src/tables/page.rs: add generate_page_trace_from_dense().
  • prover/src/tables/trace_builder.rs: generate_page_tables uses the dense path.

@ColoCarletti

Copy link
Copy Markdown
Collaborator Author

/bench 5

@ColoCarletti

Copy link
Copy Markdown
Collaborator Author

/bench-gpu

@github-actions

github-actions Bot commented Jul 23, 2026

Copy link
Copy Markdown

GPU Benchmark (ABBA) — 44bcbf24f3 vs main (14 pairs)

RTX 5090 · Vast.ai datacenter @ $0.8423148148148147/hr · prover/cuda · drift-free A/B/B/A

=== ABBA paired result  (improvement: - = PR faster) ===
  pairs: 14   mean A (PR): 17.797s   mean B (base): 18.178s

  [parametric] paired-t   mean -2.09%   sd 1.11%   se 0.30%
               95% CI: [-2.73%, -1.45%]   (t df=13 = 2.16)
  [robust]     median -2.21%   Wilcoxon W+=0 W-=105  p(exact)=0.0001  (z=-3.26)

  --- server stability (this run; compare across servers) ---
  run-to-run jitter:    A CV 0.67%   B CV 0.84%        (lower = steadier)
  within-session drift: +0.73% over the run, 1st->2nd half +0.25%
    (jitter -> Tier-1 cached gate floor; drift -> whether the cached baseline can be trusted)

  VERDICT: REAL IMPROVEMENT - PR faster by ~2.09% (t-CI and Wilcoxon agree)

  raw pairs: /tmp/abba_run/pairs.csv

- = PR faster. Trust the verdict when paired-t and Wilcoxon agree.

@github-actions

github-actions Bot commented Jul 23, 2026

Copy link
Copy Markdown

Benchmark — ethrex 20 transfers (median of 3)

Table parallelism: auto (cores / 3)

Metric main PR Δ
Peak heap 73028 MB 72504 MB -524 MB (-0.7%) ⚪
Prove time 37.771s 37.531s -0.240s (-0.6%) ⚪

✅ No significant change.

🔬 Looks like a small speedup (-0.6%) — below what 3 runs can confirm. Comment /bench-abba to run the drift-free ABBA tiebreaker (paired-t CI + exact Wilcoxon). Note: it occupies the bench server for ~30–40 min.
Optional pair count: /bench-abba 32 (20 resolves ~1%, 32 for ~0.6%).

✅ Low variance (time: 1.6%, heap: 0.6%)

Commit: 44bcbf2 · Baseline: cached · Runner: self-hosted bench

@ColoCarletti

Copy link
Copy Markdown
Collaborator Author

/bench 10

@ColoCarletti
ColoCarletti force-pushed the perf-page-dense-gen branch from 086761e to 82a3d7a Compare July 24, 2026 18:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants