Repository navigation
perf(reader): a filtered scan copies the matching rows into flat arrays (#602) - #606
Merged
Merged
Conversation
…ys (#602) SelectedArrays wrapped every selected column in an index-mapped view (get(i) = inner.get(rows[i])) over a long[rowCount] built per chunk. It now copies the selected rows into flat buffers in the chunk's arena, as Rust's filter does: a flat buffer is copied by a typed per-row gather at density <= 0.5 and by one bulk copy per run of consecutive rows above it; other leaves copy through their accessors; dictionaries still copy only their codes. The selection is a sorted int[] sized to the matches. The ten view records go away, and with them the java:S6218 findings that failed the Sonar quality gate (supersedes #605). JavaVsJniFilterBenchmark.javaFilterClose, ohlc.vortex, -f 3, ops/s (the 0.001 label selects about 11% of rows on this file): selectivity main branch alloc/op main -> branch 0.001 6581 ± 605 6505 ± 532 913 KB -> 195 KB 0.01 6918 ± 563 6740 ± 239 913 KB -> 196 KB 0.1 6001 ± 88 5194 ± 311 987 KB -> 363 KB 1.0 3144 ± 148 2285 ± 343 1429 KB -> 1247 KB The dense end is slower: the copy moves rows the view never touched, and this benchmark reads each row once, the view's best case. A BitSet selection was tried first and lost at every density: walking it with nextSetBit once per column cost more than the filter's own row test. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SwhK4W5cgj7y7s4ZPCif5x
This was referenced Oct 24, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #602. Supersedes #605 (the SelectedArrays view records it suppressed are deleted).
A filtered scan copies the matching rows into flat arrays, as Rust's
filterdoes, instead of index-mapped views over along[rowCount]. Benchmark table and the dense-case trade-off are in the commit message.🤖 Generated with Claude Code
https://claude.ai/code/session_01SwhK4W5cgj7y7s4ZPCif5x