v0.3.0: complete rewrite from first principles

[tamnd]

What this is

v0.2 measured whatever each adapter happened to run and printed the results side by side. This rewrite makes the comparison defensible. A workload defines a logical question once, the harness asks it of every engine in that engine's dialect, and every engine is validated against one engine-independent reference before any number gets published.

Spec is in notes/Spec/2064g/bench/00-10.

Architecture

• Engine SPI: Engine/Session/Tx/Result, plus an explicit Info{Name, Plane, Dialects, Caps}. Adding an engine is one package and one registration. The runner has no if name == "zu" branches left.
• Dialect resolution: a query carries Texts per dialect, an engine carries a preference-ordered chain. First match wins. No match is a SKIP with reason no-dialect-text, never a silent fallback. mage gets its own dialect so Memgraph's procedure surface stops resolving as cypher and failing to parse. A parse failure is a FAIL, and one FAIL discards the whole workload.
• Verify gate: every query runs its oracle before measurement. One FAIL discards the measurement for every query in that workload x engine. A SKIP does not poison, a FAIL does.
• Capability filters run before dialect resolution, so a missing kernel is reported as the more precise reason. Algorithm, NeedsPathPredicate, NeedsShortestPath, and write-without-transactions all SKIP rather than handing an engine a text it cannot parse.
• Planes (inproc, bolt, subprocess, native) are stamped per row, so an in-process number is never quietly compared against a wire number.

Engines: zu, Neo4j 2026.06.0, Ladybug 0.19.1, Memgraph. The Bolt and cgo adapters sit behind the bolt and ladybug build tags.

Workloads: micro (read, write, mix, power-law, uniform, ER), SNB Interactive v2 (short, complex, update, mix), FinBench-derived, LinkBench-derived, LSQB, Graphalytics, GAP, Graph500.

Three fixes that invalidated published numbers

Write bracketing was verification-only. Setup and Teardown now surround every firing of a write, including warmup and measured repetitions, untimed, with repetitions of a bracketed query serialized.

Without the bracket a keyed insert is a trajectory, not a distribution. An engine that enforces the key errors on repetitions 2..n and reports the cost of a constraint violation (Ladybug's snb-iu1 errored 100/100). An engine that does not enforce it accumulates duplicates and reports inserts into a table growing under the measurement. The silent case is worse: a write whose Setup staged its input matches nothing once that input is consumed, so every later repetition reports the cost of an index probe that found no rows, which renders as a very fast write. Every setup-dependent SNB and LinkBench write was measuring roughly 25x under its true cost.

Verification cannot catch this. It fires each write exactly once, and once is the repetition that always succeeds.

ColdRun never assigned P50. It accumulated Max, called it P99, and left P50 at the zero value, so every cold p50 ever printed was 0.00ms. That reads as an instant cold first access, the opposite of what a cold pass exists to show. It now collects samples and summarizes through the same code as the warm run.

An all-errors query rendered 0.00ms, the fastest-looking cell in its row, because errors are excluded from the percentile slice. It now renders ERR(n/n).

Rebase notes

Rebased onto main after #49 and #54 landed. #49's micro-powerlaw and micro-uniform workloads are ported into the new API, along with the micro-khop3, micro-varlen, micro-sp, and micro-sp-bidir queries they need. The two path queries are the reason NeedsShortestPath exists: zu declares no shortest-path support, and without the flag its dialect chain falls through to the Cypher text and fails on shortestPath(), which would discard the entire micro measurement instead of skipping two queries.

The LSQB brute-force validation from #51 and #52 is rewritten against this branch's nine query texts (main and the rewrite define different queries under the same lsqb-qN ids). 23 fixtures, shaped to break the shortcuts a grouped count can take: an edge that must not join, a hub that fans out, a duplicate counted with multiplicity, and cycles sharing edges.

#53 is not merged. It fails to compile against the current tamnd/gr (unknown field MaxPoolPages), and it lives entirely in adapter/gr/, which this branch deletes.

Verification

go build ./..., go vet ./..., gofmt -l . clean. Full test suite passes untagged and with -tags "bolt ladybug", including new regression tests: TestBracketMakesWriteRepetitionStationary, TestBracketNotTimed, TestColdRunReportsP50.

Fast-profile runs on two hosts (server3, gamingpc) show 0 FAILs and 0 errored repetitions. A bounded full-profile pass (7 workloads x 3 engines) agrees with the fast-profile numbers within noise, which confirms the truncated fast counts are sound.

Benchmark output (results/, results-full/) is gitignored now. The previously committed run artifacts are removed, except one kept as a report/testdata/ fixture.

zu now answers in zuQL

zu ran four queries before this. zu shell and zu query were both listed in zu help and both replied "unknown command", so the probe fell back to primitive mode and everything outside micro-point, micro-point-miss, micro-khop1, micro-edge skipped with zu-no-query-verb. tamnd/zu#72 adds the query verb, and this branch drives it: parameters go across as repeated -p name=value flags sorted by name, so the command line is stable run to run.

The dialect chain drops Cypher. zu accepts a Cypher-shaped syntax for the subset it implements but it is not Cypher: labels are case-sensitive and the loader names its tables node and edge, so a text written for Neo4j fails on the label before it reaches anything interesting. Keeping Cypher in the chain turned every query without a zuQL text into a FAIL, and one FAIL discards the whole workload. With a one-dialect chain those queries skip with no-dialect-text, which is the honest outcome.

The micro texts follow. The existing zuQL ones are lowercased, micro-scan-stats and the two triangle counts gain zuQL texts, and micro-edge counts in a WITH and compares in the RETURN, because zu's binder takes a bare aggregate call in a projection and nothing else.

All nine micro-read queries verify against zu now, as do micro-uniform, micro-er, micro-mix, and micro-powerlaw, where the two shortest-path queries skip on the declared capability as intended.