[Pratt Parser] Cache source content size in Lexer to avoid repeated variant dispatch in character loops. - #2274
Open
copybara-service[bot] wants to merge 1 commit into
Open
copybara-service[bot] wants to merge 1 commit into
copybara-service[bot] wants to merge 1 commit into
Conversation
…ariant dispatch in character loops. `SourceContentView::size()` is defined out-of-line and dispatches via `absl::visit` over a 4-element `absl::variant`. Because the character scanning loops in `Lexer` also invoke out-of-line methods on `content_`, the compiler cannot hoist `content_.size()` out of loop conditions, causing it to be re-evaluated on every character scanned. Caching `content_size_` in `Lexer` improves Pratt parser CPU time across `pratt_parser_benchmark` by ~5.2% geomean (~6.7% geomean on valid expressions). Measured with `pratt_parser_benchmark` (built `-c opt --dynamic_mode=off`). "Pratt before" is this CL's parent (`p4head`), so the last column is this CL's own contribution. The ANTLR column is the series baseline and is unaffected by this change. CPU time, median across 36 interleaved repetitions: | Case | ANTLR | Pratt before | Pratt after | Pratt vs ANTLR | Delta this CL | | :--- | ---: | ---: | ---: | ---: | ---: | | ParseCommon | 210,982 ns | 11,606 ns | 11,003 ns | 19.2x faster | -5.2% | | ParseArithmeticChain/10 | 55,289 ns | 2,781 ns | 2,608 ns | 21.2x faster | -6.2% | | ParseArithmeticChain/50 | 291,952 ns | 13,794 ns | 13,265 ns | 22.0x faster | -3.8% | | ParseArithmeticChain/100 | 559,317 ns | 27,506 ns | 25,724 ns | 21.7x faster | -6.5% | | ParseLogicalChain/10 | 43,318 ns | 3,306 ns | 2,986 ns | 14.5x faster | -9.7% | | ParseLogicalChain/50 | 207,612 ns | 15,118 ns | 13,940 ns | 14.9x faster | -7.8% | | ParseLogicalChain/100 | 442,381 ns | 30,089 ns | 28,675 ns | 15.4x faster | -4.7% | | ParseMemberChain/10 | 34,693 ns | 1,819 ns | 1,705 ns | 20.4x faster | -6.3% | | ParseMemberChain/50 | 161,104 ns | 8,633 ns | 7,889 ns | 20.4x faster | -8.6% | | ParseMemberChain/100 | 335,750 ns | 16,917 ns | 15,724 ns | 21.4x faster | -7.1% | | ParseNestedParentheses/10 | 28,084 ns | 803 ns | 752 ns | 37.3x faster | -6.3% | | ParseNestedParentheses/50 | 123,579 ns | 2,841 ns | 2,620 ns | 47.2x faster | -7.8% | | ParseCommonSyntaxErrors | 429,515 ns | 18,317 ns | 17,561 ns | 24.5x faster | -4.1% | | ParseArithmeticChainSyntaxError/10 | 72,212 ns | 3,256 ns | 3,078 ns | 23.5x faster | -5.5% | | ParseArithmeticChainSyntaxError/50 | 295,287 ns | 14,321 ns | 13,419 ns | 22.0x faster | -6.3% | | ParseArithmeticChainSyntaxError/100 | 628,385 ns | 28,006 ns | 26,414 ns | 23.8x faster | -5.7% | | ParseLogicalChainSyntaxError/10 | 58,855 ns | 3,776 ns | 3,553 ns | 16.6x faster | -5.9% | | ParseLogicalChainSyntaxError/50 | 235,134 ns | 16,104 ns | 15,147 ns | 15.5x faster | -5.9% | | ParseLogicalChainSyntaxError/100 | 458,219 ns | 31,111 ns | 29,472 ns | 15.5x faster | -5.3% | | ParseMemberChainSyntaxError/10 | 50,335 ns | 2,144 ns | 2,051 ns | 24.5x faster | -4.3% | | ParseMemberChainSyntaxError/50 | 182,649 ns | 8,962 ns | 8,516 ns | 21.4x faster | -5.0% | | ParseMemberChainSyntaxError/100 | 346,654 ns | 17,161 ns | 16,230 ns | 21.4x faster | -5.4% | | ParseNestedParenthesesSyntaxError/10 | 241,350 ns | 6,098 ns | 5,982 ns | 40.3x faster | -1.9% | | ParseNestedParenthesesSyntaxError/50 | 1,561,548 ns | 28,725 ns | 27,390 ns | 57.0x faster | -4.6% | | ParseRepeatedSyntaxErrors/10 | 170,609 ns | 6,218 ns | 6,044 ns | 28.2x faster | -2.8% | | ParseRepeatedSyntaxErrors/50 | 172,554 ns | 8,602 ns | 8,620 ns | 20.0x faster | +0.2% | | ParseRepeatedSyntaxErrors/100 | 180,960 ns | 9,324 ns | 9,532 ns | 19.0x faster | +2.2% | | **Geomean** | | | | **22.5x faster** | **-5.2%** | PiperOrigin-RevId: 983495524
copybara-service
Bot
force-pushed
the
test_983495524
branch
from
September 18, 2026 00:31
fc6c722 to
1cc4816
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
[Pratt Parser] Cache source content size in Lexer to avoid repeated variant dispatch in character loops.
SourceContentView::size()is defined out-of-line and dispatches viaabsl::visitover a 4-elementabsl::variant. Because the character scanning loops inLexeralso invoke out-of-line methods oncontent_, the compiler cannot hoistcontent_.size()out of loop conditions, causing it to be re-evaluated on every character scanned.Caching
content_size_inLexerimproves Pratt parser CPU time acrosspratt_parser_benchmarkby ~5.2% geomean (~6.7% geomean on valid expressions).Measured with
pratt_parser_benchmark(built-c opt --dynamic_mode=off). "Pratt before" is this CL's parent (p4head), so the last column is this CL's own contribution. The ANTLR column is the series baseline and is unaffected by this change.CPU time, median across 36 interleaved repetitions: