Skip to content

EQL domain CHECKs accept a payload whose version is JSON null ("v": null) #1285

Open

Description

@coderdan

Background

EQL (Encrypt Query Language) is the SQL library we install into a customer's PostgreSQL database. It stores each encrypted value as a JSON payload, and it gives each kind of value a PostgreSQL domain: a named type over jsonb with a CHECK constraint. Examples are public.eql_v4_integer_ord_ope for a stored column and eql_v4.query_integer_ord_ope for a query operand. The CHECK is what stops the wrong payload from getting into a column. Part of its job is to check the payload's version field, v, so that a v3 payload can't be stored in a v4 column and a v4 payload can't be stored in a v3 one. The same SQL source is installed twice, as the eql_v3 and eql_v4 schemas (ADR-0002).

Problem

Every generated domain CHECK tests the version like this:

CHECK (
  jsonb_typeof(VALUE) = 'object'
  AND VALUE ? 'v'
  ...
  AND VALUE->>'v' = '4'
)

VALUE ? 'v' only says that the key is present. When the payload has "v": null, VALUE->>'v' is SQL NULL, so VALUE->>'v' = '4' is NULL too, and so is the whole CHECK. PostgreSQL treats a CHECK that evaluates to NULL as passing. So a payload with a JSON null version gets past the version check of every domain.

What this means in practice: a payload with no real version can be stored in, or used to query, a column of either EQL version. The version check is meant to rule that out, but it doesn't. Nothing produces such a payload today, so this is a hole in the guard, not data we know is corrupt. But the guard exists to catch payloads from a buggy or hostile producer, and that is exactly the kind of producer that would write one.

Nothing catches this today. The tests of each domain's shape cast a wrong version ("v": 3), not a null one.

Verified against the EQL v4 bundle built from next and installed in PostgreSQL 17:

SELECT '{"v":null,"i":{"t":"a","c":"b"},"c":"x","op":"00"}'::jsonb::public.eql_v4_integer_ord_ope;
-- accepted
SELECT '{"v":null,"i":{"t":"a","c":"b"},"op":"00"}'::jsonb::eql_v4.query_integer_ord_ope;
-- accepted

From source, not run: v3 is rendered from the same templates, so eql_v3 has the same hole.

Where the predicate comes from:

  • packages/eql/crates/eql-codegen/templates/types.sql.j2:25 (stored domains)
  • packages/eql/crates/eql-codegen/templates/query_types.sql.j2:36 (query operands)
  • 89 occurrences across 20 generated files under packages/eql/src/v3/scalars/ on next

The range query domains added by #1243 had the same predicate. That PR fixed them only, because they are the one part of the bundle that exists in v4 alone; fixing the others changes the v3 bundle.

Proposal

  1. In both templates, make the version comparison NULL-safe without changing what a NULL value of the domain does:
    AND COALESCE(VALUE->>'v' = '{{ eql_version }}', VALUE IS NULL)
    This is false for "v": null and still lets a SQL NULL value through, as a domain must. It is the form feat(eql): give EQL v4 range query domains and eql_v4.between #1243 uses for the range domains, and it keeps the one spelling tasks/test/verify_no_v3_in_v4.sh accepts. (A plain COALESCE(..., false) refuses SQL NULL values and breaks every nullable column.)
  2. Regenerate src/v3/scalars (cargo run -p eql-codegen), and update any hand-written domains that use the same predicate (check boolean and json).
  3. Add a "v": null case to the domain shape tests, for a stored domain and a query operand, in both the SQLx suite (v3) and packages/eql/tests/encryption (v4). Check that it fails before step 1 is applied.
  4. Decide what an upgrade does with a row that already holds "v": null: whether the changed CHECK is validated against existing rows, and if so, how such a row is found first.
  5. Add an @cipherstash/eql changeset. This changes the v3 bundle, so it ships in the npm package as well as the v4 release asset.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    SDKbugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions