bindings-typescript: share the string codec and add an ASCII fast path to writeString - #5783
Open
TheUltDev wants to merge 1 commit into
Open
bindings-typescript: share the string codec and add an ASCII fast path to writeString#5783TheUltDev wants to merge 1 commit into
TheUltDev wants to merge 1 commit into
Conversation
…h to writeString `BinaryWriter.writeString` allocated a fresh `TextEncoder` and a temporary `Uint8Array` for every string it wrote, and `BinaryReader.readString` a fresh `TextDecoder` for every string it read. Both run once per string column per row on every insert, update, find and scan, so the per-call allocations dominated the cost of string columns. Writer: pure-ASCII strings are written straight into the buffer, one byte per char; non-ASCII input falls back to a shared encoder. Reader: one shared decoder. Measured inside a 2.8.2 module (20k inserts into a table with 12 string columns): 255-300 ms before, 78-88 ms after; a single-blob-column table went from ~80 ms to ~60 ms since its one string benefits too. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01S8Pm6ygGY3L9M75ZkBmCSj
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description of Changes
BinaryWriter.writeStringallocates a freshTextEncoderplus a temporaryUint8Arrayfor every string it writes, andBinaryReader.readStringa freshTextDecoderfor every string it reads. Both run once per string column per row on every insert/update/find/scan inside a module, so the per-call allocations dominate the cost of string columns.This PR:
TextEncoder. (TextEncoder.encodeIntowould be the natural tool, but the module host'sTextEncoderdoes not provide it, so the fast path is a plain loop.)TextDecoderinstead of one per call. The decode itself is unchanged; a per-char ASCII loop was measured and is slower than a singledecodeat typical field lengths, so the reader keeps it.Output is byte-identical to the previous implementation (tests compare against
TextEncoderfor empty, ASCII, Latin-1, CJK, astral/surrogate pairs, mixed, 10k-char strings, and buffer growth from a 1-byte initial capacity).Measured inside a module on a stock 2.8.2 server (20k inserts,
Date.now()around the loop):The same bytes through fewer allocations; the f64 control shows the rest of the insert path was never the bottleneck.
API and ABI breaking changes
None. Same wire bytes, same public surface.
Expected complexity level and risk
Testing
crates/bindings-typescript:vitest run(29 files, 301 tests) including the newwriteStringsuite (encodes likeTextEncoderfor empty / ASCII / Latin-1 / CJK / astral / mixed / 10k-char inputs, grows from a 1-byte buffer, round-trips throughreadString, consecutive writes stay contiguous).eslintandprettier --checkon the touched files.🤖 Generated with Claude Code