Skip to content

fix(proxy): refuse COPY FROM into encrypted columns, check statements after COPY FROM STDIN - #488

Merged
tobyhede merged 2 commits into
mainfrom
claude/project-thread-ajkwfx
Oct 26, 2026
Merged

tobyhede merged 2 commits into
mainfrom
claude/project-thread-ajkwfx

Conversation

@tobyhede

@tobyhede tobyhede commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

Requested by Toby

Ports the COPY ... FROM fix from #483 (release/2.2) to main. main's frontend has a single per-statement route decision, a batch view and schema-qualified resolution, so this is a re-implementation rather than a cherry-pick.

The problem

  • Proxy does not encrypt COPY data, but forwarded a COPY ... FROM (client stream, file or program) into an encrypted column, so plaintext was stored in it.
  • The SQL parser expects the copy data of COPY ... FROM STDIN to follow in the SQL. Without a trailing ; the statement was refused as invalid SQL, so plaintext bulk loads (\copy, driver COPY APIs) failed. With a ; every later statement in the simple Query was read as copy data and forwarded unchecked, so COPY notes FROM STDIN; INSERT INTO users (email) VALUES ('plaintext') reached PostgreSQL in plaintext.

The change

  • Parser (parser.rs): when the SQL has a COPY ... FROM STDIN, parse each ;-separated statement on its own, ending the COPY with the ; the parser expects. Statements after it are now parsed and routed like any other. RESET normalisation still applies.
  • Route (frontend.rs): a COPY ... FROM into a table is checked against the same resolver a write would use (the batch view, including temporary tables). With encrypted columns configured it is refused when any column it fills is encrypted (or legacy EQL v2), the table is unresolvable or ambiguous, a named column is unknown, or the table's columns are unknown (CREATE TABLE ... (LIKE ...), which the parser misreads as a column named LIKE). This holds whatever the mapping-errors setting is. Otherwise it is forwarded as NothingEncrypted. Without encrypted columns the COPY's table is not checked, though a COPY after encryption-changing DDL in the same Query is refused, as an INSERT would be.
  • Batch rule: a simple Query with a COPY and a statement Proxy maps is refused (CopyWithEncryptedStatement), as the rewritten Query cannot carry the COPY.
  • eql_mapper::requires_schema: requires_type_check was also used as "this statement needs the schema" (waiting for in-flight DDL, failing closed after encryption-changing DDL or settings changes in a batch, and the prepared-statement staleness check). A COPY ... FROM now counts too.
  • New errors: CopyIntoEncryptedColumn, CopyIntoUnknownColumn, CopyWithEncryptedStatement, documented in docs/errors.md. Unresolvable and ambiguous tables reuse the existing errors.

Out of scope: COPY stream encryption and decryption, and COPY ... TO STDOUT of encrypted columns, which returns ciphertext as before. COPY (INSERT ... RETURNING) TO was already refused on main.

One deliberate difference from #483: there, Proxy recorded DDL before PostgreSQL ran it, so the COPY check also checked the loaded table. On main, overlays are confirmed after execution and the batch view already handles IF NOT EXISTS, so the COPY check uses the batch view on its own, the same as INSERT.

Tests

Written first and seen failing:

  • Parser: statements after COPY ... FROM STDIN are statements.
  • Frontend: COPY into encrypted columns is refused (simple and extended, STDIN, file and program), COPY into native columns is forwarded (including temporary tables and tables created earlier in the Query), unresolvable table, unknown column and LIKE tables are refused, statements after a COPY are checked, COPY in a mapped Query is refused with nothing encrypted, and COPY after encryption-changing DDL in the same Query is refused.
  • Integration (Rust via tokio-postgres copy_in, and Python via psycopg): plaintext COPY round-trips, and COPY into an encrypted column is refused while the connection stays usable.

cargo test -p cipherstash-proxy --lib (476) and -p eql-mapper pass locally, and clippy is clean. Integration tests run in CI.

Acknowledgment

By submitting this pull request, I confirm that CipherStash can use, modify, copy, and redistribute this contribution, under the terms of CipherStash's choice.


Generated by Claude Code

@tobyhede tobyhede self-assigned this Oct 25, 2026
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 25, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-25T01:04:23.173390Z 1695ceb PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1695ceb40f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/cipherstash-proxy-integration/src/copy_from_stdin.rs Outdated

@cipherstash-agent cipherstash-agent left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review attestation — automated review by Claude Code (/code-review medium, then company-skills:cross-check-review-findings) at head 1695ceb.

Result: no critical, blocking or new findings.

  • Checked: COPY ... FROM routing in frontend.rs (route, check_copy_into), the batch rule CopyWithEncryptedStatement, requires_schema at all call sites, and the COPY ... FROM STDIN statement splitting in parser.rs.
  • Candidate defects were traced and none had a reachable trigger. Existing PR feedback holds no findings (only a Codex status stub), so nothing is duplicated or contradicted.
  • Not verified here: the integration tests (Rust copy_in, Python psycopg). They were still running at review time, so treat CI as the gate.

Generated by Claude Code

@tobyhede
tobyhede requested a review from freshtonic October 25, 2026 22:30
… after COPY FROM STDIN

Proxy does not encrypt COPY data, but forwarded a `COPY ... FROM` (a
client stream, a file or a program) into an encrypted column, storing
plaintext in it.

The SQL parser also expects a `COPY ... FROM STDIN` to be followed by its
copy data in the SQL: without a `;` it refused the statement, so plaintext
bulk loads failed, and with one it read every later statement in the
simple Query as copy data, so they were forwarded unchecked.

- Parse each `;`-separated statement on its own when the SQL has a
  `COPY ... FROM STDIN`, so the statements after it are checked.
- Route a `COPY ... FROM` through its own schema check against the batch
  view: refuse it when any column it fills is encrypted, when the table
  is unresolvable or ambiguous, or a named column is unknown, whatever
  the mapping errors setting. Without encrypted columns it is forwarded.
- Refuse a simple Query that has a `COPY` and a statement Proxy maps, as
  the rewritten Query cannot carry the `COPY`.
- Treat a `COPY ... FROM` as needing the schema (`requires_schema`), so
  it waits for in-flight DDL and fails closed after DDL or settings
  changes, like a type-checked statement.
@tobyhede
tobyhede force-pushed the claude/project-thread-ajkwfx branch from 5d40c36 to e4cad15 Compare October 25, 2026 22:36

@freshtonic freshtonic left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed head e4cad150. I found no fail-open path. The port also covers the points from my #483 review: schema-qualified COPY uses the same resolver as a write (public.encrypted and archive.encrypted are tested), and COPY (INSERT … RETURNING) TO was already refused on main.

Merge hazard with #489

  • The two PRs merge cleanly as text, but not as behaviour.
  • copy_from_stdin_into_encrypted_column_is_rejected_by_postgres in #489 expects that Proxy forwards COPY … FROM STDIN; into an encrypted column and that PostgreSQL rejects it with 23514. After this PR, Proxy refuses the Parse with CopyIntoEncryptedColumn. The copy_in(…).await.unwrap() in that test then panics.
  • The new Stale route entry in #489's CONTEXT.md also says that the domain CHECK is what stops plaintext in COPY … FROM STDIN. After this PR, that is false.
  • Whichever PR merges second must change that test to expect Proxy's refusal (or delete it, because copy_from_stdin.rs here covers it), and must remove the COPY sentence from CONTEXT.md.

Nits

  • The empty-target refusal has no test. In check_copy_into (frontend.rs:1426), I changed if targets.is_empty() to if false, and all COPY tests still passed. I think the branch cannot be reached for the CREATE TABLE … AS case in the comment: in the same Query, that DDL fails closed first, and across Queries a permanent table is loaded from the catalog with its columns. Keep it as a defence, but change the comment so that it does not name AS as the case it handles.
  • "Without encrypted columns every COPY is forwarded" (PR description, and docs/errors.md). This is not fully true. requires_schema makes a COPY count in simple_query_requires_fail_closed, settings_simple_query_requires_fail_closed and ensure_schema_modelled, and those checks do not look at has_encrypted_columns. So with no encrypted columns, CREATE TABLE t AS SELECT 1 AS a; COPY t FROM STDIN is now refused. This agrees with how INSERT behaves, so I think the behaviour is correct, but the docs should not say "every".
  • is_copy_from_stdin also matches COPY (SELECT * FROM stdin) TO STDOUT, because it finds FROM followed by STDIN anywhere in the statement. The only effect is that the Query is parsed one statement at a time, so a BEGIN ATOMIC body in the same Query fails to parse and is refused. It fails closed, so a comment is sufficient.

What I checked and found correct

  • Statement splitting. The split is on tokens, so a ; in a literal, a quoted identifier, a dollar-quoted body or a comment does not split. PostgreSQL parses the whole Query string as SQL, so Proxy now sees the same statements that PostgreSQL runs. Bodies that contain several statements (BEGIN ATOMIC, CREATE RULE … DO (…; …)) fail to parse and are refused.
  • INHERITS, LIKE, CLONE and AS. In the same Query, these are encryption-changing DDL, so a later COPY gets DependentStatementAfterDdl. A temporary table created with them is unmodelled, so the connection fails closed. CREATE TABLE … (LIKE …) is refused by the LIKE check.
  • The route order. UNSAFE_DISABLE_MAPPING still forwards first, as intended. A COPY never reaches the type check or the fallback. A COPY in a mapped Query is refused before anything is encrypted (the encrypt counter is 0).
  • Extended protocol. A COPY sent with Parse is routed by the same function. needs_mapping is now true for it, so the prepared-statement staleness check covers it.
  • Mutations. I removed each of these and its test failed: the LIKE check, the requires_schema change in simple_query_requires_fail_closed, and the CopyWithEncryptedStatement batch rule.
  • CI: all 9 checks pass.

What I ran

  • cargo test -p cipherstash-proxy --lib: 475 of 476 passed. The one failure, config::tandem::tests::prometheus_config, is the environment race. The config:: tests with --test-threads=1: 37 of 37 passed.
  • cargo test -p eql-mapper --lib: 225 passed.
  • The mutations above. I reverted all changes after the runs.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

Copy link
Copy Markdown
Contributor Author

Thanks @freshtonic. I pushed 1904b7a, a fast-forward with no rewrite, for the three nits:

  • The empty-target comment in check_copy_into now calls the branch a defence. It no longer names CREATE TABLE ... AS as the case it handles.
  • docs/errors.md and the PR description no longer say that every COPY is forwarded when no columns are encrypted. They now say the COPY's table is not checked, and that a COPY after encryption-changing DDL in the same Query is still refused, as an INSERT is.
  • is_copy_from_stdin now has a comment saying it also matches COPY (SELECT * FROM stdin) TO STDOUT, and that this fails closed.

On #489: I agreed with that PR's thread that whichever of the two merges second removes the COPY test and the COPY sentence in the Stale route entry.


Generated by Claude Code

@tobyhede
tobyhede merged commit 2a0a469 into main Oct 26, 2026
9 checks passed
@tobyhede
tobyhede deleted the claude/project-thread-ajkwfx branch October 26, 2026 00:42
tobyhede pushed a commit that referenced this pull request Oct 26, 2026
Address the skill-driven review of the routing model.

- Split the unchecked statement kind into CopyFrom, NestedWrite and
  Opaque, and route them as Frontend::route does since #488:
  check_copy_into refuses COPY ... FROM into an encrypted or unresolvable
  column, and a nested write is refused while encrypted columns are
  configured. The Bind settings check is gated by requires_schema.
- A3 and F2 now cover only DO, CALL and SQL EXECUTE. New assumption A6:
  a nested write takes no parameters. A5 is documented as an
  over-approximation, which widens F1. Decision D4 keeps Bind and Execute
  as one step.
- Parse is only of an unused name, a refused Parse leaves it unused, and
  an unknown Bind under UNSAFE_DISABLE_MAPPING is forwarded, as in Proxy.
- New witnesses for Bind, Close, transaction end and the mapping setting,
  so every action has one only it can reach. New mutations M5 to M8.
- Two-statement batches hold type-checked statements only, which keeps
  check.sh all at about 6 minutes.
- Fill in the source-file correspondence table and the AGENTS.md pointer.

No property changes. TLC: asBuilt 56,994 states, concurrentDdl 445,094,
safety holds in both; all expected violations and M1 to M8 reproduce.

Signed-off-by: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants