Skip to content

bindings-typescript: share the string codec and add an ASCII fast path to writeString - #5783

Open
TheUltDev wants to merge 1 commit into
clockworklabs:masterfrom
TheUltDev:ts-string-codec-fastpath
Open

bindings-typescript: share the string codec and add an ASCII fast path to writeString#5783
TheUltDev wants to merge 1 commit into
clockworklabs:masterfrom
TheUltDev:ts-string-codec-fastpath

Conversation

@TheUltDev

@TheUltDev TheUltDev commented Aug 23, 2026

Copy link
Copy Markdown

Description of Changes

BinaryWriter.writeString allocates a fresh TextEncoder plus a temporary Uint8Array for every string it writes, and BinaryReader.readString a fresh TextDecoder for every string it reads. Both run once per string column per row on every insert/update/find/scan inside a module, so the per-call allocations dominate the cost of string columns.

This PR:

  • Writer: writes pure-ASCII strings straight into the writer buffer, one byte per char, with no intermediate allocation; non-ASCII input falls back to a single shared TextEncoder. (TextEncoder.encodeInto would be the natural tool, but the module host's TextEncoder does not provide it, so the fast path is a plain loop.)
  • Reader: one shared TextDecoder instead of one per call. The decode itself is unchanged; a per-char ASCII loop was measured and is slower than a single decode at typical field lengths, so the reader keeps it.

Output is byte-identical to the previous implementation (tests compare against TextEncoder for empty, ASCII, Latin-1, CJK, astral/surrogate pairs, mixed, 10k-char strings, and buffer growth from a 1-byte initial capacity).

Measured inside a module on a stock 2.8.2 server (20k inserts, Date.now() around the loop):

table before after
12 string columns 255-300 ms 78-88 ms
12 f64 columns (control) ~42-124 ms unchanged
1 string column holding a 305-byte JSON blob ~80 ms ~60 ms

The same bytes through fewer allocations; the f64 control shows the rest of the insert path was never the bottleneck.

API and ABI breaking changes

None. Same wire bytes, same public surface.

Expected complexity level and risk

  1. Two methods, no behavior change beyond allocation; covered by byte-equality tests.

Testing

  • crates/bindings-typescript: vitest run (29 files, 301 tests) including the new writeString suite (encodes like TextEncoder for empty / ASCII / Latin-1 / CJK / astral / mixed / 10k-char inputs, grows from a 1-byte buffer, round-trips through readString, consecutive writes stay contiguous).
  • eslint and prettier --check on the touched files.
  • Exercised end to end by a module publishing to a 2.8.2 server with string, option and mixed-type rows, including non-ASCII round trips through the HTTP call path.

🤖 Generated with Claude Code

…h to writeString

`BinaryWriter.writeString` allocated a fresh `TextEncoder` and a temporary
`Uint8Array` for every string it wrote, and `BinaryReader.readString` a
fresh `TextDecoder` for every string it read. Both run once per string
column per row on every insert, update, find and scan, so the per-call
allocations dominated the cost of string columns.

Writer: pure-ASCII strings are written straight into the buffer, one byte
per char; non-ASCII input falls back to a shared encoder. Reader: one
shared decoder.

Measured inside a 2.8.2 module (20k inserts into a table with 12 string
columns): 255-300 ms before, 78-88 ms after; a single-blob-column table
went from ~80 ms to ~60 ms since its one string benefits too.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S8Pm6ygGY3L9M75ZkBmCSj
@CLAassistant

CLAassistant commented Aug 23, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants