Skip to content

Parse a cold statement once instead of three times - #674

Open
tamnd wants to merge 1 commit into
mainfrom
parse-once-miss
Open

Parse a cold statement once instead of three times#674
tamnd wants to merge 1 commit into
mainfrom
parse-once-miss

Conversation

@tamnd

@tamnd tamnd commented Aug 23, 2026

Copy link
Copy Markdown
Owner

Closes #658.

A text the plan cache has not seen was parsed three times: once to
decide it was a query rather than one of the four things that is not,
once to read the USE in front of it, and once to compile it. Same
text, same tree, three times. #657 took the first of those off the warm
path. This carries the tree through the miss path so there is one parse
on it.

What changed:

  • query::categorise is not_a_query handing back what it parsed
    rather than a verdict, and not_a_query is a thin wrapper over it for
    the callers that only want the verdict.
  • Session::categorised answers a three-way Kind: the text has a plan
    already, or it is a query and here is the tree, or it is not a query
    and here is what it is. run_within and stream_in both match on it.
  • plan_from and compiled_from take the tree when the caller has one
    and parse when it does not, so nothing that arrives with only the text
    changed.
  • query::run_with, the one-shot path with no cache to read, categorises
    and hands the tree to a new prepare_from.
  • The read-only refusal in db.rs sits in front of the session and
    deciding whether a statement writes is a parse of its own. A text the
    session already holds a plan for is one that refusal let through
    before, so it is answered from the cache now.

What did not change: the warm plan cache hit, at 0.04 us either side.
This is the other path, the one a shell session, a conformance corpus
and the first minute of any workload are made of.

benches/compile.rs is new and measures the cold send the way
benches/session.rs measures the warm one. Two binaries built from
before and after, interleaved, four rounds, on an M-series laptop:

before after
cold compile through a session 7.83, 8.12, 8.25 us 5.83, 6.04, 6.21 us
cold send on a read-only connection 15.75, 15.96, 16.21, 16.62 us 11.83, 12.08, 12.50, 12.54 us

About a quarter off either way. Round one of the old binary read 11.29
on the session line, which is the first-run outlier every A/B on this
machine has, and it is left in the log rather than dropped from it.

One thing is deliberately left: a cold statement on a read-only
connection still pays the refusal parse, about 1.7 us of the 11.8, and
threading the tree past it would mean a parsed variant of run,
run_streaming, prepare and profile. The repeated statement, which
is what a read-only connection actually serves, is answered from the
cache.

cargo test -p zu is green, 328 lib tests and every integration binary,
and clippy and fmt are clean.

A text the plan cache has not seen was parsed once to say it was a
query rather than one of the four things that is not, once again to
read the `USE` in front of it, and a third time to compile it. It is
the same text and the same tree every time. #657 took the first of
those off the warm path; this carries the tree through the miss path
so there is one parse on it.

`query::categorise` is `not_a_query` handing back what it parsed, and
`Session::categorised` answers a three-way `Kind`: the text has a plan
already, or it is a query and here is the tree, or it is not a query
and here is what it is. `plan_from` and `compiled_from` take the tree
when the caller has one and parse when it does not, so nothing that
arrives with only the text changed. `run_with` in query.rs, which has
no cache to read and so paid both parses on every call, categorises
and hands the tree to `prepare_from`.

The read-only refusal in db.rs sits in front of the session, and
deciding whether a statement writes is a parse of its own. A text the
session already holds a plan for is one that refusal let through
before, so it is answered from the cache now. A cold statement on a
read-only connection still pays it, about 1.7 us, because threading
the tree past it would mean a parsed variant of run, run_streaming,
prepare and profile.

benches/compile.rs is new and measures the cold send the way the
session bench measures the warm one. Two binaries, interleaved, four
rounds, on this laptop:

  cold compile through a session   7.8 to 8.3 us  ->  5.8 to 6.2 us
  cold send on a read-only conn   15.8 to 16.6 us -> 11.8 to 12.5 us

About a quarter off either way. A warm plan cache hit is 0.04 us and
is unchanged, which is the point: this is the path a shell session, a
conformance corpus and the first minute of a workload are made of.

While here, plan_for's doc comment had been left stranded on top of
`categorised` when #655 landed, so it goes back where it belongs.

Closes #658
@tamnd

tamnd commented Aug 24, 2026

Copy link
Copy Markdown
Owner Author

I was wrong earlier in this thread when I said there was no mechanism by which this could move the write gate. There is a signal, it is consistent across three runs a side, and here is what it looks like.

The absolute numbers overlap completely, which is why it read as flake at first:

branch write_cpu_nosync_us ceiling on that host
main d1c231f 49.6 59.3
main e93b79f 61.8 70.7
main 0f306b8 75.5 81.3
674 67.3 63.3
674 75.5 68.1
674 69.3 66.4

Main's own values run 49.6 to 75.5, so 674's 67 to 75 is inside main's range and no single run tells you anything. What separates them is the ratio of the value to the ceiling, and that is the right thing to look at because both sides are scaled by the same host calibration, so the box speed cancels:

branch value over ceiling
main 0.84, 0.87, 0.93
674 1.06, 1.11, 1.04

Three and three with no overlap. That is not flake.

Now what it means, which is the part I cannot settle from CI alone. The ceiling is 30.0 us scaled by calibrate(), and calibrate() times MATCH (p:person) WHERE p.age = {age} RETURN count(p) AS n built with format! on every iteration. The gated statement is MATCH (p:person) WHERE p.age = {age} SET p.age = {age}, also built with format! on every iteration. Both are cold statements and both go through the path this PR changes, so this PR takes the same parse off both of them.

Take a fixed amount P off a read costing R and off a write costing W, with W larger than R. The ceiling moves by (R-P)/R and the value moves by (W-P)/W, and the second is closer to one than the first, so the ratio of value to ceiling goes up. Solving that against the observed 21 percent shift puts P at about 9 us a statement on a hosted runner, which is the right order for two parses of a short statement on a box running two to three times slower than the reference.

So the same data supports two readings:

  1. The write path got slower by about 20 percent and the gate caught it.
  2. Both paths got faster by the same absolute amount, the read is the smaller of the two so it moved further in proportion, the ceiling tightened with it, and the gate is punishing a change that made both sides faster.

The comment on calibrate() anticipates one direction of this and says so plainly: a read that got slower would relax the ceiling, and that is the trade it accepts. The other direction is not covered. A read that got faster tightens the ceiling, and any change that takes a shared fixed cost off both paths will always tighten it by more than it lowers the value, because the read is the smaller number. That is a blind spot in the gate design regardless of which reading is right here, and I will file it separately.

CI cannot separate the two, because the hosted runners never give the same host twice. What separates them is one A to B on one quiet box: write_cpu_nosync_us and the raw calibrate() microseconds, main and this branch, back to back. Queued behind a sweep that is holding the only idle machine I have. Not merging until that comes back.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

A statement that misses the plan cache is still parsed twice

1 participant