Skip to content

Count every check invocation, and sample the clock - #6081

Open
vlsi wants to merge 2 commits into
google:masterfrom
vlsi:vs/timings-slot-prototype
Open

Count every check invocation, and sample the clock#6081
vlsi wants to merge 2 commits into
google:masterfrom
vlsi:vs/timings-slot-prototype

Conversation

@vlsi

@vlsi vlsi commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Stacked on #6080. The first commit here is that pull request unchanged; only the second one, Count every check invocation, and sample the clock, is this change. It wants to merge after #6080, and I will rebase once that lands.

Why

This is the third path @cpovirk listed in #6079: make the timing cheap enough that it does not hurt. #6080 is the second path, and the two compose — a flag still decides whether anything is collected, and this decides what collecting costs.

Timing a check on every invocation costs about what the cheapest invocations cost. On error_prone_core, 45% of spans are shorter than 128 ns and carry 2.9% of the recorded time, while a pair of System.nanoTime calls measures 24 ns to 27 ns on the machine below. The checks a report is read for are at the other end and are rare: on that workload every top entry by total time runs 948 times, once per compilation unit.

What

A check keeps its own CheckTiming, so the scanner reaches its state through a field rather than a lookup by canonical name. The invocation count is exact. The elapsed time is sampled: below 256 invocations every one is timed, above it one in count / 256, and each sample counts for the invocations it stands in for. A check whose mean invocation stays above a microsecond is timed on every one of them, so the ones the report is about are measured rather than estimated, and one measurement perturbs such an invocation by a few percent at most.

That bound is the only tunable, and it does not want to be a flag: a clock read can be measured at startup in well under a millisecond, and the bound follows from it.

The report gains two columns. The invocation count, and the longest single invocation — which is what separates a check that is slow from one that paid a one-off cost. On Calcite InjectOnBugCheckers reads 7 ms over 21416 calls, and one run charged it 307 ms because the first constructor it saw triggered a type lookup the classpath could not answer.

Error Prone ran 457 checks in 34585 ms, and spent 326 ms initializing
      8567 ms   24.8%        859655 calls  max 175288458 ns  NullAway
      1222 ms    3.5%        101964 calls  max  67279500 ns  ParameterName
      1179 ms    3.4%         21416 calls  max  15422250 ns  MissingFail

What it costs

Both workloads compiled in process so the numbers are the compiling thread's own, macOS on Apple silicon. Each variant is compared against a build whose spans are no-ops, alternating the two inside one JVM so every pair meets the same machine state.

spans opened Stopwatch, as today this change
error_prone_core, 948 sources, JDK 24 14771508 +656 ms of 13.0 s, 8 pairs of 8 +209 ms, 6 pairs of 8
Calcite :core, 1655 sources, JDK 21, NullAway 28493608 +1216 ms of 37 s, 4 pairs of 4 not distinguishable from no spans

On Calcite the closure also costs 0.42 GiB of the compiling thread's allocation, which this removes:

allocated
Stopwatch, spans on 21.03 GiB
spans off 20.61 GiB
this change 20.60 GiB

The closure does not show up on error_prone_core, where C2 inlines span into processMatchers and scalar-replaces it. It does on Calcite, whose larger check set defeats that inlining, so whether the allocation is real depends on the JIT and a project cannot tell which case it is in.

Sampling costs accuracy, and the question is how much against the noise a timing report already carries. Comparing per-check shares, over checks above 50 ms, across four runs of each build:

within 10% within 20%
exact against exact — the noise floor 78% 91%
exact against sampled 65% 86%

How to verify

mvn -pl check_api,core test -Dtest=CheckTimingTest,ErrorProneTimingsTest,ErrorProneOptionsTest,ErrorProneJavaCompilerTest

CheckTimingTest drives the sampling schedule and the weighted total through a supplied elapsed time rather than a clock, so the arithmetic the report prints is pinned without timing anything, and its expected values are worked out by hand from the documented rule rather than from the implementation's own expression. ErrorProneTimingsTest covers both ways span reaches a check's state, and that one canonical name folds every slot reported under it.

Each test was watched failing against the change it guards: reverting the stride weighting, the mean test that keeps an expensive check exactly timed, the open flag that rejects a nested span, the sampling reset that keeps a second close from recording again, the list registry, and the instanceof BugChecker arm each fail their own test and no other.

If this is not the shape you want

Close it. #6080 stands on its own, and the measurements above are the part of this that is worth keeping either way — in particular for internal CL 933288891: a span is roughly half clock and half lookup on these workloads, so hoisting the lookup out of the per-node loop should take out about that half and leave the rest.

vlsi added 2 commits August 31, 2026 16:41
ErrorProneScanner opens a timing span for every matcher on every AST node, and
nothing reads what the spans record: ErrorProneTimings.timings() and
initializationTime() have no callers in check_api or core, and no test asserts on
either. Compiling error_prone_core opens 14771508 spans, and Calcite's :core
opens 28493608; measured against a build whose spans are no-ops, they cost a
median of 656 ms of a 13.0 s compile and 1216 ms of a 37 s one.

The spans are now gated on -XepPrintTimings, which also prints the per-check
totals once the compilation finishes, so a build that does not ask for the data
does not collect it and a build that does can read it. Issue google#6079 has the
measurements and proposes making the collection itself cheap enough to leave on;
that is a larger change and is not this one.

Assisted-by: Claude Code (claude-opus-5)
Timing a check on every invocation costs about as much as the cheapest
invocations it measures: on error_prone_core 45% of spans are shorter than 128 ns
and carry 2.9% of the recorded time, while a pair of System.nanoTime calls
measures 24 ns to 27 ns. So the count is now exact and the elapsed time is
sampled, and a sample counts for the invocations it stands in for.

A check keeps its own CheckTiming, so the scanner reaches it without looking the
check up by name, and a check whose mean invocation stays above a microsecond is
timed on every one of them -- those are the checks a report is read for, and
timing them perturbs each by a few percent at most. Measured against a build that
opens no span at all, this costs a median of 209 ms of a 13.0 s compile where the
Stopwatch it replaces costs 656 ms, and on Calcite's :core it is not
distinguishable from having no spans, allocation included.

The report gains the invocation count and the longest single invocation, which
separate a check that is slow from one that paid a one-off cost: on Calcite
InjectOnBugCheckers reads 7 ms over 21416 calls, and one run charged it 307 ms
because the first constructor it saw triggered a type lookup the classpath could
not answer.

Estimates are worth what they can be checked against: CheckTimingTest drives the
schedule and the weighted total through a supplied elapsed time rather than a
clock, and ErrorProneTimingsTest covers both ways span reaches a check's state.

See google#6079. This sits on top of the flag in google#6080 and is not proposed for merge
until the shape of the collection is settled.

Assisted-by: Claude Code (claude-opus-5)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant