Repository navigation
Conversation
The live json_tree oracle checked a fixed corpus and a fixed list of needles. The only randomized containment check compared the projection with an in-memory model of jsonb containment, never with PostgreSQL, so nothing generated was ever answered by the database itself. postgres_containment_over_generated_documents_is_jsonb_containment stores 300 generated documents encrypted beside their plaintexts and asks 500 needles: random values, sub-documents pruned from stored documents, and those sub-documents with one edit. For every needle the rows the encrypted @> and <@ select must be the rows jsonb @> and <@ select over the plaintexts, as PostgreSQL computes them. It runs once with the planner held to the GIN index and once with every index path off, and checks each @> plan with EXPLAIN. <@ may differ only where the SQL says it does (a stored root scalar inside a needle root array), and the test asserts that is the whole difference. It also checks -> at 20 generated key paths: the entry exists exactly where jsonb #> finds a node, and a leaf opens to the value jsonb holds there. The seed is fixed (1336), logged on every run, and replaced by EQL_JSON_TREE_SEED. A failure prints the seed, the needle and every stored document the two answers disagree on. test:encryption:postgres passes --show-output so the seed line reaches the CI log. Claude-Session: https://claude.ai/code/session_01XQN1QpaWVDGjHEg39KEJF3
|
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the
|
The generated containment oracle drew its documents and needles from a hand-rolled SplitMix64 generator, one fixed batch per run, and could not shrink a failure. It now draws them from proptest strategies: a prop_recursive JSON value with the same coverage (nested objects and arrays, repeated keys, arrays of objects and of arrays, duplicate elements, numbers equal by value in different representations, numeric strings, booleans, null, empty containers, root scalars and arrays), and needles that are random values, prunings of a stored document, or such a pruning with one edit. A derived needle names its document by an Index and its pruning by a keep mask, so it follows the documents as they shrink and shrinks toward a smaller needle itself. A fault injected into the <@ exception shrinks to one stored 0 and the needle [0]. A case is a batch of documents and needles, checked in a few set-based statements: one correlated subquery per needle, which the planner serves from the GIN index when sequential scans are off. The table has no primary key, so the GIN index is the only one there is to take. Every case asserts with EXPLAIN that each plan is the one it asked for. The three properties (@> on both plans, <@ on both plans less its documented exception, and -> at generated key paths) are separate tests under given_generated_documents_in_postgres, and a database-free test holds the document strategy to every feature the census counts. The module lives in tests/json_tree/generated.rs. The RNG is ChaCha with seed 1336 and 64 cases by default. PROPTEST_RNG_SEED and PROPTEST_CASES override them, as for any proptest run, and both are logged. Failures persist to the crate's proptest-regressions/json_tree.txt through FileFailurePersistence::Direct, because SourceParallel resolves file!() from the working directory, which is not where this nested workspace member's paths are rooted. Claude-Session: https://claude.ai/code/session_01XQN1QpaWVDGjHEg39KEJF3
|
Reworked in dd9bc51 (pushed as a new commit, no history rewrite):
|
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
…ntainment with proptest tests/json_index.rs held 28 flat tests named for whole sentences, with bare assertions and a hand-rolled Lcg for its only randomized check. It now follows the repository's Rust testing conventions: one given_<scenario> module per situation and a test named for what it expects, a scenario() helper for the cipher and plan, an assertions module for the refusals and round trips the tests repeat, a message on every assertion, and unwrap allowed at the crate root, which is the test module of an integration test. No assertion was dropped. The longest tests were split along the expectations they made (an accessor's matching, ordering and refusals are five tests now; a subtree key's derivation, gates and root handling are three), and four near-identical assertions became loops. The run goes from 28 tests to 47. The random containment check becomes two proptest properties, one for @> and one for <@, over a generated stored document and a needle: a random value, the document itself, a pruning of it, or a pruning with one edit. They hold the projection to the in-file model of jsonb, which given_the_containment_model pins to PostgreSQL's answers. A failure shrinks to a small pair: breaking the <@ exception shrinks to stored 0 and the needle [0]. The RNG is ChaCha, seed 1336, 512 cases per property. PROPTEST_RNG_SEED and PROPTEST_CASES replace them. Failures persist to the crate's proptest-regressions/json_index.txt. A census test fails if the document strategy stops producing a feature the property relies on. The strategy duplicates the EQL live oracle's, which sits in another Cargo workspace. proptest is a dev-dependency with default features off, as stack-guest-abi takes it, resolving to the 1.11.0 already in the lock. The plan document's description of both oracles is updated to match. Claude-Session: https://claude.ai/code/session_01XQN1QpaWVDGjHEg39KEJF3
Mutation testing (cargo-mutants,
|
|
Added f2e998c, which restructures
|
…roperties Brings in the rename of the plan-layer Plan::value().encrypt_as(layout) to encrypt_with_layout(layout), the generic-plan assertions in the planner test and the row-provenance wording fix. The one conflict was in packages/stack-encrypt/tests/json_index.rs, which this branch rewrites: the rewritten file is kept and the rename applied to its two plan-layer call sites. The cipher-layer encrypt_as calls are unchanged. Claude-Session: https://claude.ai/code/session_01N7bCqa3ZVJMcEE99RzVqEu
…roperties Cargo.lock conflict only: the base branch's lock, which now carries next, plus this branch's proptest edge. Claude-Session: https://claude.ai/code/session_01N7bCqa3ZVJMcEE99RzVqEu
Summary
EQL v4's
json_treecolumn lets PostgreSQL answer JSON containment over encrypted documents:@>("does this document contain that one?") and<@(the reverse). The promise is that it answers exactly as PostgreSQL's ownjsonboperators would answer over the plaintexts. Until now that promise was checked against live PostgreSQL only for a fixed, hand-picked set of documents. The only randomized check compared against a Rust model ofjsonbcontainment, never against PostgreSQL itself.This PR adds property tests built on proptest. Each test generates batches of documents and queries and checks every answer against PostgreSQL's real
jsonbanswer, on both the index plan and the sequential-scan plan. A failure shrinks to a small document and query. It is test-only.It also restructures
packages/stack-encrypt/tests/json_index.rs, the in-memory half of the same oracle (no database: it holds the encrypted projection to a Rust model ofjsonb). Its tests now follow the repository's Rust testing conventions, and its hand-rolled random check is replaced by proptest properties like the ones above. No assertion was dropped; the old-to-new mapping is below.Changes
packages/eql/tests/encryption/tests/json_tree/generated.rs(new, included fromjson_tree.rsasmod generated):What a case is: a batch of up to 64 stored documents and up to 96 needles (the queries). The default run is 64 cases.
Document strategy: a recursive (
prop_recursive) JSON value. It covers:1/1.0/1e0,100/1e2,0/-0.0,0.5/5e-1){}and[]Needle strategies:
A derived needle refers to its document by position and to its pruning by a keep mask. When proptest shrinks the documents, the needle still refers to a valid document, and the needle itself shrinks toward smaller.
Oracle: each document is stored twice in the same row, encrypted in a
public.eql_v4_json_treecolumn and as plaintextjsonb. A case is checked in a few set-based statements. Each statement answers every needle at once with one correlated subquery per needle. PostgreSQL computes thejsonbanswer, and the encrypted answer must equal it for every needle.Plans: every case runs on both plans: sequential scans off (the GIN index), and every index path off.
EXPLAINconfirms each plan. The table has no primary key, so the GIN index is the only index the planner can take.The three properties, as separate tests under
given_generated_documents_in_postgres:contains_selects_the_rows_jsonb_contains_on_both_plans(@>)contained_by_selects_the_rows_jsonb_does_except_a_root_scalar_in_a_root_array(<@). The only allowed difference is the documented one (a stored root scalar inside a needle root array). The test asserts that this is the whole difference and that the run hits the case at least once.arrow_opens_a_leaf_to_the_value_jsonb_holds(->at 20 key paths). The encrypted column has an entry exactly wherejsonb #>finds a node, and a leaf opens (decrypts) to the valuejsonbholds there.Generator coverage:
given_the_document_strategy::produces_every_feature_the_oracle_relies_onneeds no database. It runs intest:encryptionand fails if the generator stops producing any feature listed above.Structure: follows the Rust testing conventions:
given_*scenario modules, test names that state the expectation, ascenario()helper, anassertionssubmodule, a message on every assertion, and#[allow(clippy::unwrap_used)]on the module.packages/eql/tests/encryption/proptest-regressions/json_tree.txt: where a failing case's seed is recorded. Every later run replays it first. The file is set explicitly withFileFailurePersistence::Direct, because proptest's default location is resolved in a way that does not fit this nested crate.packages/eql/tests/encryption/Cargo.toml,packages/eql/Cargo.lock:proptest = "1"as a dev-dependency. That is the same requirementeql-domainsandtests/sqlxcarry, so the lockfile only gains one dependency edge on the existing 1.11.0, not a second copy.packages/eql/mise.toml:test:encryption:postgrespasses--show-output, so the seed, case count and summary lines appear in the CI log even when the tests pass.json_tree.rs: the hand-rolled generated test from this PR's first commit is replaced by the module above. The file's other tests are unchanged.packages/stack-encrypt/tests/json_index.rs(rewritten, 28 tests → 47):given_<scenario>module per situation, each test named for what it expects. Ascenario()helper builds the cipher and plan. Anassertionsmodule holds the checks that repeat: shape-mismatch, tamper, term and two-context refusals, round trips, and both containment directions. Every assertion has a message.#![allow(clippy::unwrap_used)]sits at the crate root, which is the test module of an integration test.Lcgrandom check becomes two properties,@>and<@, over one generated stored document and one needle per case. A needle is a random value, the document itself, a pruning of it, or a pruning with one edit. The oracle is the in-filejsonbmodel, whichgiven_the_containment_modelpins to PostgreSQL's answers.PROPTEST_RNG_SEEDandPROPTEST_CASESoverride them.packages/stack-encrypt/proptest-regressions/json_index.txt.packages/stack-encrypt/Cargo.toml,Cargo.lock:proptest = { version = "1.7", default-features = false, features = ["std"] }as a dev-dependency, written the waystack-guest-abiwrites it. It resolves to the 1.11.0 already in the root lockfile.docs/plans/2026-10-17-json-tree-index.md: the description of both oracles now names the proptest properties. No file in the repo referred to the old test names or tojson_index.rs:<line>.Old → new test names in
json_index.rs(28 → 47)a_document_seals_one_entry_per_node_and_opens_againgiven_a_document::seals_one_entry_per_node_and_opens_againthe_deepest_document_the_index_takes_opens_againgiven_the_deepest_document::opens_again_and_one_level_deeper_is_refused_when_sealedthe_wire_form_is_what_the_sql_domain_storesgiven_the_fixture_document::stores_the_wire_form_the_sql_domain_checksthe_chain_fields_sit_where_the_design_saysgiven_the_fixture_document::{places_the_chain_fields_where_the_design_says, gives_equal_values_at_one_path_one_value_selector_and_two_ciphertexts, chains_the_top_level_fields_from_an_open_root}an_ordering_term_sits_only_at_an_ordered_pathgiven_ordered_paths::{puts_a_term_only_on_a_number_or_string_leaf_at_an_ordered_path, refuses_a_position_or_a_text_that_is_not_a_path}a_token_derived_from_the_path_alone_unlocks_exactly_its_subtreegiven_a_subtree_key::{unlocks_exactly_its_subtree_from_the_path_alone, gates_each_key_step_below_its_head, has_no_token_at_a_sealed_root_and_derives_from_an_open_one}a_selector_is_the_stored_one_and_folds_positions_into_the_wildcardgiven_the_fixture_document::derives_the_stored_selector_from_a_path_folding_positions_into_the_wildcardan_extracted_node_opens_to_its_scalar_and_a_subtree_to_its_valuegiven_an_extracted_node::{opens_to_its_scalar_and_a_walked_subtree_to_its_value, round_trips_with_the_envelope_the_sql_grafts}subtree_rows_the_server_repeats_or_lines_up_are_refusedgiven_rows_the_server_chose::refuses_a_repeated_row_or_a_line_of_containersa_moved_reordered_or_regrafted_entry_fails_to_opengiven_a_tampered_document::fails_to_open_a_moved_reordered_or_regrafted_entrythe_containment_model_gives_postgresqls_answersgiven_the_containment_model::gives_postgresqls_pinned_answerscontainment_over_the_projection_is_postgresqls_containmentgiven_the_design_corpus::answers_containment_both_ways_as_jsonb_does;given_generated_documents::{contains_as_jsonb_contains, is_contained_by_as_jsonb_except_a_root_scalar_in_a_root_array};given_generated_documents::given_the_document_strategy::produces_every_feature_the_property_relies_ona_query_term_serializes_as_the_needle_operand_and_derives_no_keygiven_a_containment_query::serializes_as_the_needle_operand_and_derives_no_keyan_accessor_asks_at_a_pathgiven_an_accessor::{matches_the_value_at_its_path, orders_as_the_stored_leaf_at_an_ordered_path, refuses_a_bad_path_an_unordered_leaf_or_a_path_the_plan_does_not_order, orders_at_a_wildcard_and_refuses_a_position, refuses_a_container_for_equality}a_laid_out_plan_opens_what_it_wrotegiven_a_laid_out_plan::{reports_its_layout, opens_what_it_wrote_one_document_or_a_batch, refuses_an_open_that_names_another_context, answers_a_named_index_with_its_own_options}equality_at_the_root_is_not_containmentgiven_equality_at_the_root::matches_the_root_value_and_not_an_array_holding_ita_query_under_other_options_is_refused_by_the_plangiven_a_plan::refuses_a_query_under_other_optionsopening_through_the_plan_is_the_typed_opengiven_a_plan::opens_what_it_wrote_as_the_typed_open_does(now also compares with the typed open)the_json_index_has_a_wire_form(featuredynamic)given_the_json_index_spec::has_a_wire_formno_stored_tag_is_a_tokengiven_the_fixture_document::stores_no_tokena_position_reaches_no_object_membergiven_a_position::{reaches_no_object_member, gates_on_the_elements_selector_below_a_key_step}gated_walk_rows_open_to_the_reached_nodes_valuegiven_gated_walk_rows::open_to_the_reached_nodes_valueeach_stored_form_opens_as_a_record_fieldgiven_a_derived_record::opens_each_stored_form_as_its_fieldthe_sealed_root_a_walk_fell_back_to_opens_at_the_pathgiven_a_fallback_root::{opens_at_the_keys_path, refuses_one_value_where_the_path_reaches_none_or_two, is_only_the_sealed_root_alone_at_depth_zero}the_key_built_fallback_opens_to_what_the_walk_opens_togiven_a_fallback_root::opens_to_what_the_walk_opens_toa_walked_row_carries_its_depthgiven_an_extracted_node::carries_no_depth_until_a_walk_gives_it_onea_document_without_the_sealed_root_opens_from_its_entriesgiven_an_open_root::keeps_the_root_as_an_entry_and_opens_from_the_entriesan_accessor_is_refused_before_its_keyset_loadsgiven_an_accessor::is_refused_before_its_keyset_loadsEncryption uses the test crate's fake key source, so no CipherStash credentials are needed.
Replaying a failure
The run is deterministic: a ChaCha RNG with seed 1336, 64 cases. Every run logs, for example:
PROPTEST_RNG_SEEDandPROPTEST_CASESoverride the seed and the case count, the standard proptest variables. A failure prints the shrunk case: each stored document and needle as JSON, the two answers, and the rows they disagree on. It also appends the case's seed toproptest-regressions/json_tree.txt. Commit that line with the fix.json_index.rsworks the same way, with seed 1336 and 512 cases. For example:A failure prints the minimal stored document and needle, and appends the seed to
packages/stack-encrypt/proptest-regressions/json_index.txt.Verification
Locally, against the EQL tasks' own Postgres (
mise run postgres:up postgres-N, port 7432), with the v4 bundle built from this tree:mise run test:encryption:postgrespasses on PostgreSQL 17 and 14.24. Thejson_treesuite took 15.4 s on 17 and 16.5 s on 14. The three properties run in parallel; the slowest is<@, at about 15 s.<@exception, and 254 leaves opened and 1,957 containers found by->.<@exception on purpose. The test failed and shrank to one stored document0and the needle[0], which is exactly the documented case. The change was reverted, along with the regression line it wrote.cargo fmt --checkand the task's clippy lines (-p eql-bindings -p eql-encryption-tests --all-features --all-targets -D warnings, and-p eql-bindings --features stack-encrypt) are clean.cargo test --locked -p eql-encryption-testspasses.vitest --config scripts/vitest.config.mjs: 1,371 passed, 29 skipped), including the ones that parsepackages/eql/mise.toml, the Cargo manifests and the workflows. All 78 files are vitest suites, sonode --testdoes not apply to them.For
stack-encrypt:mise x --env test -- cargo nextest run --locked -p stack-encrypt --all-features: 784 passed, 1 skipped, including all 47json_indextests and the trybuilduibinary. The two properties take about 0.76 s for 512 cases.cargo test -p stack-encrypt --no-default-features --test json_index(the shapewasm:no-http-testbuilds) passes 46 tests. Thedynamic-only wire-form test is left out there.cargo fmt --all --checkand the workflow'scargo clippy --locked --no-deps --workspace --all-targets --all-features -- -D warningsare clean.<@exception in the property on purpose. It shrank to stored0and needle[0], and was then reverted.CI: run 40047240427 is green,
ci-requiredincluded. InRust encryption with PostgreSQL 17, all three properties passed (->in 4.6 s,@>in 11.4 s,<@in 15.1 s; the suite took 15.1 s). The log shows the same seed and counts as the local run. The database-free census test passed inRust workspace crates. On a pull request,test-eql.ymlruns PostgreSQL 17 only. Versions 14–16 run on a push tomainand nightly, so for this PR PG 14 was checked locally only (above). The first commit's version passed CI on PG 17 in run 39983784023, and locally on 14, 15 and 16.CI for the
json_indexcommit (f2e998c): all 48 checks pass. The 2 skipped are the nightly fuzz campaign and the Claude review, which this event does not run. InTests (crates)(fmt, clippy, nextest, doctests, rustdoc), nextest ran all 47json_indextests. TheMutantsgate reported no mutants to test, since the diff touches nosrc, and passed.Unused dependencies,CRAP,Fuzzregression andMirialso pass, and so doesTest EQL(ci-required).No changeset: test-only.
Review notes
test:encryption:postgresrunscargo test -- --ignoredbehind a listing guard that other suites share, and changing the runner is outside this PR.json_index.rsduplicates the EQL generator rather than sharing it, because the two are separate Cargo workspaces. Its properties hold the projection to the in-file Rust model ofjsonb; the EQL tests are the ones that check against PostgreSQL itself.json_index.rswas restructured. The otherstack-encrypttest files andtests/commonare unchanged.Related
Refs #1336 (stacked on it: base
feat/json-tree-gates).https://claude.ai/code/session_01XQN1QpaWVDGjHEg39KEJF3