Skip to content

[codex] add predefined RowBinary encoders#374

Draft
ruslandoga wants to merge 1 commit into
masterfrom
ruslandoga-conductor/predefine-rowbinary-encoders
Draft

[codex] add predefined RowBinary encoders#374
ruslandoga wants to merge 1 commit into
masterfrom
ruslandoga-conductor/predefine-rowbinary-encoders

Conversation

@ruslandoga
Copy link
Copy Markdown
Collaborator

Summary

Adds Ch.RowBinary.define_encoder/1 for compile-time RowBinary encoders from a non-Ecto schema shape such as schema: [id: "UInt64", text: "String"] or schema: %{"id" => "UInt64"}.

The generated encoder can:

  • encode one atom-keyed or string-keyed row map/struct
  • encode many rows with rows: true
  • encode a full INSERT INTO ... FORMAT RowBinaryWithNamesAndTypes body when table: is provided, with the names/types prefix precomputed

The first specialization covers common hot insert paths and falls back to Ch.RowBinary.encode/2 for less common types.

Benchmark

Updated bench/encode.exs to compare current list/map paths against the generated encoder. Local Apple M2 result for 1,000,000 rows:

Predefined atom map encoder stream of 100k row chunks: 2.22 ips / 450.21 ms
Predefined atom map encoder: 1.96 ips / 509.09 ms
Predefined full insert body: 1.92 ips / 521.85 ms
RowBinary row lists: 1.00 ips / 1004.43 ms
RowBinary atom maps via fields: 0.81 ips / 1229.04 ms

Follow-up Issues

Validation

  • mix compile --warnings-as-errors
  • MIX_ENV=test mix compile --warnings-as-errors
  • MIX_ENV=test mix format
  • git diff --check
  • mix run bench/encode.exs
  • custom mix run -e smoke check comparing generated encoder bytes against encode_row/2, encode_rows/2, and encode_names_and_types/2

mix test test/ch/row_binary_test.exs could not run in this workspace because the Docker daemon/OrbStack socket is unavailable, so ClickHouse from compose.yml could not be started.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant