Repository navigation
EQL domain CHECKs accept a payload whose version is JSON null ("v": null) #1285
Description
Background
EQL (Encrypt Query Language) is the SQL library we install into a customer's PostgreSQL database. It stores each encrypted value as a JSON payload, and it gives each kind of value a PostgreSQL domain: a named type over jsonb with a CHECK constraint. Examples are public.eql_v4_integer_ord_ope for a stored column and eql_v4.query_integer_ord_ope for a query operand. The CHECK is what stops the wrong payload from getting into a column. Part of its job is to check the payload's version field, v, so that a v3 payload can't be stored in a v4 column and a v4 payload can't be stored in a v3 one. The same SQL source is installed twice, as the eql_v3 and eql_v4 schemas (ADR-0002).
Problem
Every generated domain CHECK tests the version like this:
CHECK (
jsonb_typeof(VALUE) = 'object'
AND VALUE ? 'v'
...
AND VALUE->>'v' = '4'
)VALUE ? 'v' only says that the key is present. When the payload has "v": null, VALUE->>'v' is SQL NULL, so VALUE->>'v' = '4' is NULL too, and so is the whole CHECK. PostgreSQL treats a CHECK that evaluates to NULL as passing. So a payload with a JSON null version gets past the version check of every domain.
What this means in practice: a payload with no real version can be stored in, or used to query, a column of either EQL version. The version check is meant to rule that out, but it doesn't. Nothing produces such a payload today, so this is a hole in the guard, not data we know is corrupt. But the guard exists to catch payloads from a buggy or hostile producer, and that is exactly the kind of producer that would write one.
Nothing catches this today. The tests of each domain's shape cast a wrong version ("v": 3), not a null one.
Verified against the EQL v4 bundle built from next and installed in PostgreSQL 17:
SELECT '{"v":null,"i":{"t":"a","c":"b"},"c":"x","op":"00"}'::jsonb::public.eql_v4_integer_ord_ope;
-- accepted
SELECT '{"v":null,"i":{"t":"a","c":"b"},"op":"00"}'::jsonb::eql_v4.query_integer_ord_ope;
-- acceptedFrom source, not run: v3 is rendered from the same templates, so eql_v3 has the same hole.
Where the predicate comes from:
packages/eql/crates/eql-codegen/templates/types.sql.j2:25(stored domains)packages/eql/crates/eql-codegen/templates/query_types.sql.j2:36(query operands)- 89 occurrences across 20 generated files under
packages/eql/src/v3/scalars/onnext
The range query domains added by #1243 had the same predicate. That PR fixed them only, because they are the one part of the bundle that exists in v4 alone; fixing the others changes the v3 bundle.
Proposal
- In both templates, make the version comparison NULL-safe without changing what a NULL value of the domain does:
This is
AND COALESCE(VALUE->>'v' = '{{ eql_version }}', VALUE IS NULL)
falsefor"v": nulland still lets a SQLNULLvalue through, as a domain must. It is the form feat(eql): give EQL v4 range query domains and eql_v4.between #1243 uses for the range domains, and it keeps the one spellingtasks/test/verify_no_v3_in_v4.shaccepts. (A plainCOALESCE(..., false)refuses SQLNULLvalues and breaks every nullable column.) - Regenerate
src/v3/scalars(cargo run -p eql-codegen), and update any hand-written domains that use the same predicate (checkbooleanandjson). - Add a
"v": nullcase to the domain shape tests, for a stored domain and a query operand, in both the SQLx suite (v3) andpackages/eql/tests/encryption(v4). Check that it fails before step 1 is applied. - Decide what an upgrade does with a row that already holds
"v": null: whether the changedCHECKis validated against existing rows, and if so, how such a row is found first. - Add an
@cipherstash/eqlchangeset. This changes the v3 bundle, so it ships in the npm package as well as the v4 release asset.
Activity