Conversation
Add two Criterion bench targets, real_bulk_load and real_incremental, that mirror bulk_load and incremental but run against a locally captured cluster dump instead of the checked-in testdata/ fixtures. They time bulk load (interpreter vs DD session) and incremental add_fact (interpreter vs DD) at realistic scale. They are separate targets with their own benchmark names so Criterion's per-name regression history for the testdata-based benches is not polluted. The dump is read from testdata-real/ (override with PALLOGRAPH_REAL_FIXTURES), which is now gitignored. If the directory is absent the benches print a message and exit instead of failing, so a plain cargo bench still works for everyone. To support this, factor load_bench_fixtures_from(dir) out of load_bench_fixtures. No behavior change to the tool itself. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
ajwdev
force-pushed
the
ajw-bench-real-dumps
branch
from
October 2, 2026 17:07
c8f98e6 to
0d7283f
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds two Criterion bench targets,
real_bulk_loadandreal_incremental, that mirrorbulk_loadandincrementalbut run against a real cluster dump instead of the synthetictestdata/fixtures. Also addsload_bench_fixtures_from(dir)tosrc/engine.rs(withload_bench_fixturesnow delegating to it) and gitignores/testdata-real.This is a benchmark-only change. There is no behavior change to the tool.
What it measures / how to run
real_bulk_load: interpreter evaluate vs DD session startup (graph build, worker spawn, initial EDB settle).real_incremental: per-factadd_factcost, interpreter (add + full re-evaluate) vs DD (incremental commit).They use their own benchmark names so Criterion's history for the
testdata/benches stays comparable.Input is a directory of Kubernetes JSON/YAML manifests, default
testdata-real/(gitignored), overridable withPALLOGRAPH_REAL_FIXTURES. The header ofbenches/real_bulk_load.rsshows akubectl get ... -o jsoncommand to produce one. If the directory is missing the bench prints a message and exits without failing.Testing
cargo check --benchespasses.cargo testpasses (56 + 56 + 22 tests, 0 failed).🤖 Generated with Claude Code