Skip to content

fix: escape raw SQL written into generated migrations - #877

Closed
grempe wants to merge 2 commits into
ash-project:mainfrom
grempe:migration-sql-escaping
Closed

grempe wants to merge 2 commits into
ash-project:mainfrom
grempe:migration-sql-escaping

Conversation

@grempe

@grempe grempe commented Oct 1, 2026

Copy link
Copy Markdown
Contributor

Fixes #876.

The migration generator puts SQL from a resource into the generated migration's Elixir source without escaping it, so Elixir reads a different string from the migration than the one the resource declares:

  • check_constraint check: (alone, or combined with the base filter) goes into a """ heredoc in AddCheckConstraint.up/1 and RemoveCheckConstraint.down/1. check: ~S[code ~ '^\d{4}$'] reaches Postgres with U+007F in place of \d, so the constraint refuses the rows it was written to accept, and #{x} in the SQL is evaluated. The snapshot holds the declared SQL, so mix ash.codegen --check reports nothing pending.
  • custom_statements without code?: true go into an execute("""...""") heredoc in AddCustomStatement, with the same effect.
  • An identity's unique index on a resource with base_filter_sql is written as where: "(<base_filter_sql>)" in AddUniqueIndex.up/1. A quoted identifier such as "archived" = false ends the string early, and mix ash.codegen raises a SyntaxError while formatting the file.

#578 moved check constraints into heredocs so that double quotes survive, which they now do, but a heredoc still processes backslash escapes and interpolation. The existing test for it compares the migration's text, which matches the declaration even though Elixir reads something else from it: the check in that test, ~S[title ~= '("\"\\"\\\"\\\\"\\\\\")'], is read back as title ~= '(""\"\"\\"\\")'.

This adds Helper.escape_heredoc/1, which escapes \, #{ and """, and applies it to the four check constraint heredocs and the two custom statement heredocs. The layout of generated migrations is unchanged, and a heredoc whose SQL contains none of those three sequences is generated exactly as before. The unique index filter is written with inspect(base_filter, printable_limit: :infinity); the limit is there because inspect/1 otherwise truncates strings over 4096 bytes. custom_indexes where: already goes through inspect/1 (via option/2) and was not affected.

One behaviour change to be aware of: anyone who worked around this by doubling backslashes in their resource (as suggested on #576) will get the doubled backslashes in the next migration generated for that constraint or statement. Migrations that were already generated are untouched, and since snapshots do not change, this does not generate any new migrations by itself.

Tests: a new describe "raw SQL in generated migrations" block in test/migration_generator_test.exs covers a check constraint with \d, #{x} and """ (and its down), a check combined with a base filter, a custom statement's up and down, and the identity under a quoted base_filter_sql. The new helpers parse the generated migration with Code.string_to_quoted!/1 and compare the strings Elixir reads, and the existing quote-heavy check test now does the same. On main all four new tests and the updated one fail; with the change, test/migration_generator_test.exs passes (119 tests).

Full suite on main (63cf7ca) and on this branch, PostgreSQL 18.6, Elixir 1.20.1 / OTP 29: 1039 passed on main, 1043 on this branch (the four new tests), with the same 81 failures on both: 77 in AshPostgres.TemporalTest (unique_violation on subscription_pkey) and the four sort tests in JoinSubquerySortTest, UniqAggregateSortTest and FromManyAggregateSortTest. mix format --check-formatted reports only test/support/temporal/domain.ex, which is unformatted on main and untouched here; mix credo --strict and mix sobelow are clean. I did not get a local mix dialyzer result: building the dependency PLT ran out of memory on this machine (past 28 GB) before it reached this project's code, so CI's run is the check for that.

Found by an AI agent working with a human while generating migrations for their own application. The reproduction outside this repo is https://github.com/grempe/ash-fuzz-repros/blob/main/test/ash_postgres/migration_sql_not_escaped_test.exs, which generates and runs the migration against Postgres; its four failing cases pass against this branch.

Contributor checklist

Leave anything that you believe does not apply unchecked.

  • I accept the AI Policy, or AI was not used in the creation of this PR.
  • Bug fixes include regression tests
  • Chores
  • Documentation changes
  • Features include unit/acceptance tests
  • Refactoring
  • Update dependencies

Check constraint SQL (with its base filter) and custom statement SQL were
interpolated as is into `"""` heredocs in the generated migration, so Elixir
reinterpreted backslash escapes and `#{` before the SQL reached Postgres:
`\d` became U+007F and `#{x}` was evaluated. An identity's unique index on a
resource with `base_filter_sql` wrote the filter between plain double quotes,
so a quoted identifier produced an invalid migration.

The heredoc bodies are now escaped (`\`, `#{` and `"""`), and the unique
index filter is written with `inspect/1`. The existing check constraint test
compared the migration's text, which matched even though Elixir read a
different string from it; it now compares the string Elixir reads.
@grempe

grempe commented Oct 1, 2026

Copy link
Copy Markdown
Contributor Author

@zachdaniel A heads-up on CI: none of the failures on this PR come from the change. Each one fails the same way on main at 63cf7ca:

  • mix test: since babae7b, config/config.exs sets port: 5433 for the three test repos, but the CI Postgres service listens on 5432. The job stops at "The database for AshPostgres.TestRepo couldn't be created" and no tests run. I ran the suite locally on 5432; the results are in the description.
  • mix dialyzer: the job is cancelled ("The runner has received a shutdown signal") while adding 3132 modules to the deps PLT. The last passing run I found is 945073e. The lock has changed since then (ash and ash_sql now point at git refs), and the job rebuilds that PLT. Locally the same step grew past 28 GB before I stopped it.
  • audit: mint 1.10.1 has three advisories (CVE-2026-91043, CVE-2026-92103, CVE-2026-94194), all fixed in mint 1.10.2.
  • mix format --check-formatted: test/support/temporal/domain.ex needs formatting.

Happy to open a separate PR for any of these if that helps.

@grempe

grempe commented Oct 2, 2026

Copy link
Copy Markdown
Contributor Author

Update: with main (7006d03) merged into this branch, mix dialyzer now runs and passes on all five Postgres versions (0 errors), using the PLT cached for the new lockfile. The other failures are unchanged and still match main: the test database cannot be created on port 5433, test/support/temporal/domain.ex is unformatted, and the mint 1.10.1 advisories.

@zachdaniel

Copy link
Copy Markdown
Contributor

See my coment on the issue for how this should be handled.

🚀 Thank you for your contribution! 🚀

@zachdaniel zachdaniel closed this Oct 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

The migration generator writes raw SQL into Elixir string literals without escaping it

2 participants