feat(eql): give EQL v4 range query domains and eql_v4.between

[coderdan]

Summary

EQL v4 can now take a BETWEEN query as one operand. EQL is the SQL library we install into a customer's PostgreSQL database to store and search encrypted values. EQL v4 is the copy of it that Stack Encrypt (our Rust engine) writes into.

ADR-0009 decision 7 makes a range query one term carrying both bounds, so a caller can't pair a low bound from one column with a high bound from another. Until now there was nothing in SQL to bind that term to. This PR adds a range query domain for each ordered family and ordering scheme, and an inlinable eql_v4.between(column, range) that uses the column's order-term index.

This PR is based on next. It was stacked on #1241, which has since merged. It doesn't depend on #1244, and #1244 doesn't depend on it. The two PRs only edit the same changelogs and changeset.

Changes

SQL (EQL v4 only)

• Range query domains: eql_v4.query_<family>_range_ore and eql_v4.query_<family>_range_ope, one per (family, ordering term), as decision 8 says. That's 18 in total, over 9 families.
  • Wire shape: {"v":4,"i":{…},"lo":<bound>,"hi":<bound>}. Each bound is the value the single-bound domain stores under ob (ORE, an array of hex blocks) or op (OPE, a hex string).
  • The CHECK requires v, i, lo and hi, forbids c, and checks each bound's shape. Its version check is NULL-safe (COALESCE(VALUE->>'v' = '4', VALUE IS NULL)), so "v": null is refused. The other domains have the same hole; that's EQL domain CHECKs accept a payload whose version is JSON null ("v": null) #1285.
• eql_v4.between(column, range), inclusive at both ends. It is LANGUAGE sql, IMMUTABLE, not STRICT and has no SET clause, so the planner inlines it and uses the functional index on the order term.
  • There are 29 overloads, one per stored ordered domain: _ord_ore and _search_ore take the ORE range, while _ord, _ord_ope and _search take the OPE range.
  • eql_v4_internal.lo(r) and eql_v4_internal.hi(r) read a bound with the term's own constructor. They're internal because only between calls them.
• An ORE column's index needs the ore_block_256 btree operator class. A non-superuser install skips that class and disables the ORE column domains. The docs and the generated notes on the ORE overloads say so, and point to the OPE columns.
• The range parameter must be cast to its exact range domain ($1::eql_v4.query_integer_range_ore). An uncast $1, a jsonb parameter, or a range of another family or term fails with 42725 "function … is not unique". The single-bound operators work differently: they accept an untyped $1. The function reference, the SQL comments and both READMEs say this.
• between has to be written schema-qualified, because BETWEEN is an SQL keyword.

Keeping v3 unchanged

• One source tree renders both EQL versions. A new file directive, -- ONLY EQL VERSION: 4, confines the range files to v4:
  • eql-codegen select <3|4> filters the files for each version's build. It refuses a kept file that -- REQUIRE:s a dropped one.
  • The renderer refuses to render a confined file as the other version.
  • The installer-completeness and Doxygen scripts honour the directive.
• The v3 installer and both uninstallers are byte-identical to the base branch. No line of the v4 installer is lost; adding the range files only shifts where a few existing blocks sit in topological order.
• docs:validate:documented-sql now also validates the v4-only files. It runs the whole v4 install order in one transaction and then rolls it back, so every statement in those files actually runs.

Bindings

• Rust: 18 …RangeOreQuery and …RangeOpeQuery structs in eql_bindings::v4, listed by v4::all_range_query(). They implement no Stack Encrypt traits. DecryptInto never applies, because a query operand doesn't decrypt. EncryptFrom is follow-up No SDK can build an EQL v4 range operand for eql_v4.between — range bindings implement no EncryptFrom #1286: the derive can't describe two bounds under one column context, ORE bounds wait on block ORE, and query_term has nothing to call it with yet.
• Go: 18 []byte types in encrypt/eql/eql_gen.go, with Value, Scan and JSON methods. They are not in the Types table.

Docs and release notes

• New "Range queries in EQL v4" section in docs/reference/eql-functions.md.
• packages/eql/AGENTS.md, the eql-bindings and Go READMEs and CHANGELOGs, and .changeset/eql-v4-bindings.md. The changeset says the v4 bundle reaches users as the eql-v4-install.sql attached to each Go release, not through npm or the crate.

Verification

• mise run build, with the v3 installer and uninstallers compared byte for byte against the base branch: identical.
• mise run test:encryption:postgres against a disposable postgres:17: text_eq 1, v4_types 3 and v4_range 6 tests passed. The v4_range tests cover:
  • Integer ORE and both OPE column domains: rows inside, outside and exactly at both bounds; across zero; a one-value range; an empty range; the full i32 range; an inverted range; a NULL range.
  • Text ORE and OPE.
  • EXPLAIN: an index scan with both bounds in the Index Cond, including for text. This holds for a literal range and for a $1 under a generic plan.
  • The domain CHECKs.
  • A catalog-driven smoke test over all 9 ordered families and all 29 between overloads.
  • The exact-cast rule, pinned with prepared statements.
• Inlining: when one between overload was made STRICT by hand, the EXPLAIN test failed, so the test does catch lost inlining. The reviewer also confirmed an Index Scan under a generic plan with a real $1.
• mise run test:crates, test:encryption, test:codegen, codegen:parity, types:check and typescript:check all passed.
• The docs validators passed: required tags, 100% coverage, documented-sql (286 files, including the 18 range files) and doc anchors.
• The other EQL gates passed: installer-complete, symbol order, self-contained v3, v4 isolation, schemas parity, public identifiers, and the matrix inventory and coverage. eql_v4.lints() returns no rows.
• pnpm test:scripts: 1362 passed.
• Go: all guests rebuilt, go vet ./... and go test ./... -count=1 passed, and go generate ./... leaves no diff. mise run wasm:guest:test: 71 of 71.
• After the rebase onto next, these passed again:
  • build, test:codegen, test:crates and test:encryption;
  • the DB-free gates;
  • docs:validate:source, docs:validate:documented-sql (286 files) and test:doc-anchors;
  • types:check and codegen:parity;
  • test:encryption:postgres against a local PostgreSQL 17.5: all 10 tests.
• • After merging next and the NULL-version fix: test:encryption:postgres against postgres:17, 17 tests across the five suites, plus codegen:parity, test:v4_isolation, test:installer_complete and docs:validate:documented-sql.

Not run: the SQLx suite (test:sqlx). It needs CipherStash credentials to generate fixtures through ZeroKMS. The database tests live in packages/eql/tests/encryption instead, which installs the v4 bundle, whereas SQLx installs v3.

Where the test order terms come from. The ordered EQL types aren't producible yet (#1236, #1132). So the tests take order terms from cipherstash-client's OreIndexer and OpeIndexer over the fake key source's index key, the same pattern as ste_vec_selectors.rs. Stack Encrypt is untouched, and nothing in this PR derives order terms.

Related

Closes #1237

Review notes

• I had an independent review run on the first commit. It found nothing blocking; its accuracy, coverage and validator findings are the three follow-up commits.
• Follow-ups, deliberately not in this PR:
  • No SDK can build an EQL v4 range operand for eql_v4.between — range bindings implement no EncryptFrom #1286: EncryptFrom on the range structs, range fields in the Go Types table and the Rust Target table, and Stack Encrypt building the {v, i, lo, hi} envelope for a range on an EQL target (query_term returns Error::Internal for that case today).
  • EQL domain CHECKs accept a payload whose version is JSON null ("v": null) #1285: the NULL-safe version check for every other domain, which changes the v3 bundle.
  • between over a json_entry (OPE), which waits until EQL v4 has SteVec bindings.
  • ADR-0009 says a range's ORE bounds will be "left ciphertexts" once block ORE lands. But EQL's compare_ore_block_256_term reads the right half from the query side, so query bounds need full ciphertexts. I've noted this on stashgen refuses TextOrdOre and TextSearchOre — EQL stores block ORE terms and the engine derives CLLW ORE #1132, where it needs settling before block ORE is built.

https://claude.ai/code/session_01BZhkvKLpNtKuzj3HCfMXoL

[changeset-bot]

⚠️ No Changeset found

Latest commit: ad19081

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes changesets to release 12 packages

Name	Type
@cipherstash/eql	Minor
stash	Patch
@cipherstash/stack-prisma	Patch
@cipherstash/basic-example	Patch
@cipherstash/e2e	Patch
@cipherstash/prisma-example	Patch
@cipherstash/stack-drizzle	Patch
@cipherstash/stack-supabase	Patch
@cipherstash/stack	Patch
@cipherstash/wizard	Patch
@cipherstash/bench	Patch
@cipherstash/test-kit	Patch

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

• Configuration used: Organization UI
• Review profile: CHILL
• Plan: Advanced
• Run ID: 3310e29d-f67b-406f-9357-caf9dd5c2932

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

_{Comment @coderabbitai help to get the list of available commands.}

[coderdan]

Rebased onto next. Two things needed fixing:

• languages/golang/CHANGELOG.md conflicted with the integer-widths entry from next. Both entries are kept.
• tests/v4_range.rs used eql_bindings::encryption::plaintext::Int32. next removed that type, so the test now encrypts the i32 directly. The fix is folded into the first commit.

There were no review comments to address. The branch no longer depends on #1241, and it is independent of #1244.

I re-ran the checks locally after the rebase, and these passed:

• build, test:codegen, test:crates, test:encryption;
• the installer, symbol-order, v4-isolation, schemas-parity, public-identifier and matrix gates;
• docs:validate:source, docs:validate:documented-sql and test:doc-anchors;
• types:check and codegen:parity;
• test:encryption:postgres against a local PostgreSQL 17.5: all 10 tests, including the 6 v4_range ones.

I couldn't run SQLx, since it needs credentials, so that's left to CI.

[coderdan]

Second pass at high effort, on the rebased head 8cfaf37 against next. Six inline comments from the first pass are already on the PR (the ORE opclass caveat in the between docs and templates, select not checking -- REQUIRE: closure, documented-sql.sh counting a missing eql_v4 schema as validated, the duplicated term_label, the stale verify_no_v3_in_v4.sh header, and the garbled Go changelog sentence). I have not repeated those; the three below are what that pass did not cover.

Rebase check. Nothing is stale. cargo check --all-targets --all-features passes for packages/eql/tests/encryption on this head, no FfiValue or encryption::plaintext reference survives in the PR, and the test crate builds its stored rows through keyset.encrypt_as as next now expects. (The only workspace-wide cargo check failure is the SQLx crate's include_str! of credential-generated fixtures, which predates this PR.)

What I ran without credentials. cargo test -p eql-domains -p eql-codegen and cargo test -p eql-bindings --features stack-encrypt --lib: all green (166 + 99 + 108 and the smaller suites). The Postgres tests were not run locally; I confirmed test-eql.yml runs mise run test:encryption:postgres, so the 29-overload smoke test is on CI.

What I checked in the SQL and found sound. Both comparisons keep the column on the comparator's left, so the bound is the operand compare_ore_block_256_term reads its right half from, which is the only shape that works with full-ciphertext bounds (the left-ciphertext question is already on #1132 as the PR says). Bounds are inclusive at both ends and the tests pin rows exactly at both bounds. NULL range and inverted range both yield no rows, and TermError::InvertedRange in stack-encrypt/src/dynamic/query.rs backs the comment that the client refuses lo > hi. pin_search_path_v3.sql's structural skip (LANGUAGE sql, IMMUTABLE, a jsonb-backed domain argument) leaves between, lo and hi unpinned, so post-install inlining survives. The uninstaller is DROP SCHEMA ... CASCADE, so the new objects are covered. No dynamic SQL anywhere in the change; the CHECKs are parsed at creation time and the function bodies resolve pg_catalog names like every existing unpinned extractor. build.sh assigns selected on its own line, so a failing select aborts under set -e.

Findings, most severe first. None is blocking.

1. Medium (test coverage): the index-engagement test can only see a folded constant, not a parameter, so it cannot detect between losing inlining under a generic plan. Inline comment on v4_range.rs.
2. Low (surface): lo and hi are user-facing names in eql_v4 with no user purpose. Inline comment on the template.
3. Low (scope): json_entry leaves carry the OPE cross-type comparisons but get no between, so a range over an encrypted JSON path is still two predicates. Inline comment on spec.rs; may be deliberate.

[coderdan]

Incremental re-review at high effort of b7a793c against the previous head 8cfaf37 (same next base, git range-diff of the four commits). I only reviewed what changed.

Prior findings: all nine are addressed. I checked each one in the code, not in the commit messages.

1. ORE opclass caveat in the between docs and template: fixed. The reference example now uses an OPE column. A bullet names the ore_block_256 opclass and what a non-superuser install does instead. The generated @note appears only on the ORE overloads, and the_ore_overloads_name_the_opclass_they_need pins that. There is a small follow-up below (the range ORE operand is not part of the fallback).
2. select not checking -- REQUIRE: closure: fixed. render::select collects the dropped set and refuses any edge from a kept file into it, and two unit tests cover it. I also checked that the check works on the real input: build.sh feeds it src/deps-ordered-v3.txt, whose src/v3/... paths are the same form requires_of returns. surface_order has already validated every target by then, so no target can fall outside both sets.
3. documented-sql.sh counting a missing eql_v4 schema as validated: fixed. I ran the task against a local PostgreSQL 15: 286 validated, 0 errors. Then I appended a broken CREATE FUNCTION to the rendered query_integer_range_functions.sql, and the task failed with 18 errors (one bad file counts as all 18, which is cosmetic).
4. Duplicated term_label and key list in bindings.rs: fixed. It now uses range::term_label and range::range_required_keys().
5. Stale verify_no_v3_in_v4.sh header: fixed.
6. Garbled Go changelog sentence: fixed.
7. Index test only covering the constant-folded path: fixed. The generic-plan PREPARE / EXPLAIN EXECUTE assertion is in place. I ran v4_range --ignored locally against the built v4 bundle on PostgreSQL 15: 6/6 pass.
8. lo / hi in the public eql_v4 schema: fixed. After install, eql_v4_internal has 18 lo and 18 hi, and eql_v4 has none. I checked the four things you asked about:
   • proconfig is NULL on all of them and on all 29 between overloads, so pin_search_path_v3.sql's structural skip covers the internal schema too (owned_schemas() includes it).
   • The v4 uninstaller drops both schemas with CASCADE.
   • Grants are unchanged in practice. Ordering already needed USAGE on eql_v4_internal (ord_term inlines ope_cllw, see permissions.md). As a non-superuser role with only USAGE on eql_v4 and eql_v4_internal, a generic-plan between(c, $1::eql_v4.query_integer_range_ope) still runs as an Index Scan whose Index Cond is decode(...) >= eql_v4_internal.lo($1)::bytea AND ... <= eql_v4_internal.hi($1)::bytea, and it returns the right rows.
   • No reference to eql_v4.lo / eql_v4.hi is left anywhere in the tree.
9. json_entry having no between: resolved as deliberate. The range_columns doc comment and the follow-up list say so.

New findings, most severe first. None is blocking.

1. Low: on a non-superuser install, ore_fallback.sql disables query_<T>_ord_ore but not query_<T>_range_ore, so a range operand gets past the refusal that gives the single-bound operand its OPE hint. Inline on the reference.
2. Low (conflicts with #1235): the reference and a test comment say the client refuses a range whose low bound is above its high bound. #1235 (open against next) removes TermError::InvertedRange and lets that range through. Inline on the reference.
3. Nit: the generated ORE @note tells every family to use _search, which only text has. Inline on the template.

Checks run (no credentials): cargo test -p eql-codegen -p eql-domains and cargo clippy -p eql-codegen -p eql-domains --all-targets -- -D warnings (green). mise run build. docs:validate:documented-sql against a local PostgreSQL 15, both clean and with the injected failure. cargo test -p eql-encryption-tests --test v4_range -- --ignored against the built v4 bundle. A non-superuser install of release/cipherstash-encrypt-v4.sql. CI is green on this head.

[coderdan]

Low (conflicts with #1235): "the client refuses to build one" will be wrong once #1235 lands.

#1235 (open, also against next) removes TermError::InvertedRange, so Between with the low bound above the high bound now derives its terms, as SQL BETWEEN 20 AND 10 does. The SQL behaviour this sentence describes is unchanged: the range matches no row, and the tests pin that. Only the client half becomes false. The same claim is in the test comment at packages/eql/tests/encryption/tests/v4_range.rs:366 ("the client refuses one, and the SQL does not reorder it").

Suggest wording that is true whichever PR merges first: "So does a range whose low bound is above its high bound, as with SQL BETWEEN; the SQL does not reorder the bounds."

[coderdan]

Fixed in f3539d6 with your wording, in the reference, the range types template (so all 9 generated files) and the v4_range.rs comment. Each now says such a range matches no row, as with SQL BETWEEN, and that the SQL does not reorder the bounds. None of them says the client refuses it any more.

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-19T06:12:54.341828Z	b7a793c	Draft marked ready
🔒 Security Review	✅ Completed 2026-10-19T06:12:32.096321Z	b7a793c	Draft marked ready

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b7a793c494

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".