Skip to content

Measure the engine store the same way at both ends of a run - #119

Merged
tamnd merged 1 commit into
mainfrom
store-du
Aug 24, 2026
Merged

Measure the engine store the same way at both ends of a run#119
tamnd merged 1 commit into
mainfrom
store-du

Conversation

@tamnd

@tamnd tamnd commented Aug 24, 2026

Copy link
Copy Markdown
Owner

Part of #56, the disk half of the resource measurement section.

A served engine reported an unknown footprint because its files live in a container this process cannot walk. That is why ladybug and neo4j returned -1 BytesOnDisk and why the disk rows of the resource table were empty for every rival. Each container spec now names the directory its server keeps the database in (/data for neo4j, /var/lib/memgraph, /var/lib/postgresql, /data/db), and du runs inside the container to size it.

The reading is taken twice, after the load and after the measured run, by one function chosen per plane, and the loader's own figure is used only where the harness cannot measure the store itself. That matters beyond the served engines. An adapter answers for whatever it thinks its store is, which is one file for zu and a whole directory for ladybug, and the growth figure was that number against a directory walk taken here. Both ends are the same ruler now, so the growth is the bytes the measured run made durable rather than a difference between two rulers.

du runs with --apparent-size so the count means what the local planes report by adding up file sizes, with a kilobyte-block fallback for an image without GNU coreutils.

Measured on this box, linkbench smoke count 400 concurrency 4, zu inproc against ladybug inproc:

dataset on disk                 551.0 KiB  551.0 KiB
store after load                3.8 MiB    2.9 MiB
store after run                 5.9 MiB    3.0 MiB
store growth                    2.1 MiB    104.0 KiB

zu's store after load was the size of bench.zu1 before this and is the size of the database directory now, so the growth figure shrinks rather than grows: it no longer counts the sidecar files as bytes the run added.

No engine adapter changed. go test ./... green.

A served engine reported an unknown footprint because its files live in a
container this process cannot walk, so the disk rows were empty for every
rival and the resource claims had nothing to stand on. Each container spec
now names the directory its server keeps the database in, and du runs
inside the container to size it.

The reading is taken twice, after the load and after the measured run, by
one function per plane, and the loader's own figure is used only where the
harness cannot measure the store itself. That matters beyond the served
engines: an adapter answers for whatever it thinks its store is, one file
for zu and a whole directory for ladybug, and the growth figure was that
against a directory walk taken here. Both ends are the same ruler now.
@tamnd
tamnd merged commit 35c1f35 into main Aug 24, 2026
2 checks passed
@tamnd
tamnd deleted the store-du branch August 24, 2026 11:41
@tamnd tamnd mentioned this pull request Aug 24, 2026
12 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant