Repository navigation
perf(calcite): push a prefix LIKE into the scan as a string range - #558
Merged
Merged
Conversation
Calcite matched LIKE row by row, and its interpreter compiled the pattern to a regex for every row (Like.sqlToRegexLike + Pattern.compile were 30% of a 2.5M-row title LIKE 'Star Wars%'). RexFilterTranslator now translates col LIKE 'prefix%' on a string column into col >= prefix AND col < successor(prefix), where the successor raises the prefix's last code point by one (skipping the surrogate block; a trailing U+10FFFF is dropped first). Code-point order is the UTF-8 byte order RowFilter compares strings in, so the range holds exactly the strings starting with the prefix: it prunes zones from their string bounds and the scan enforces it, so Calcite stops re-checking the row. A pattern with another wildcard or an ESCAPE clause is left to Calcite; LIKE '%' keeps every non-null row. Raincloud imdb title (2.53M rows), best of 3: count(*) where title like 'Star Wars%' 1049 -> 206 ms Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WtuVGQ5h5Lk6Zn4JnkxF1K
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Calcite matched
LIKErow by row, and its interpreter compiled the pattern to a regex per row (Like.sqlToRegexLike+Pattern.compile≈ 30% of a 2.5M-rowtitle LIKE 'Star Wars%').RexFilterTranslatornow translatescol LIKE 'prefix%'on a string column intocol >= prefix AND col < successor(prefix)(last code point + 1, skipping surrogates). Code-point order is the UTF-8 byte orderRowFiltercompares strings in, so the range is exact: it prunes zones and the scan enforces it, so Calcite stops re-checking. Other wildcards andESCAPEstay with Calcite;LIKE '%'isIS NOT NULL.Raincloud imdb title (2.53M rows):
count(*) where title like 'Star Wars%'1049 → 206 ms.Tests
LikePushDownTest: 10 patterns (incl. the Latin-1 boundaryÿ%, untranslateda_c%,%c,abc) equal Calcite's ownLIKEovers || '';successoredge cases.🤖 Generated with Claude Code
https://claude.ai/code/session_01WtuVGQ5h5Lk6Zn4JnkxF1K