diff --git a/documentation/dsls/DSL-AshPostgres.DataLayer.md b/documentation/dsls/DSL-AshPostgres.DataLayer.md index cb13f0ab..8e14f28e 100644 --- a/documentation/dsls/DSL-AshPostgres.DataLayer.md +++ b/documentation/dsls/DSL-AshPostgres.DataLayer.md @@ -44,9 +44,9 @@ end | [`storage_types`](#postgres-storage_types){: #postgres-storage_types } | `keyword` | `[]` | A keyword list of attribute names to the ecto type that should be used for that attribute. Only necessary if you need to override the defaults. | | [`migration_types`](#postgres-migration_types){: #postgres-migration_types } | `keyword` | `[]` | A keyword list of attribute names to the ecto migration type that should be used for that attribute. Only necessary if you need to override the defaults. | | [`migration_defaults`](#postgres-migration_defaults){: #postgres-migration_defaults } | `keyword` | `[]` | A keyword list of attribute names to the ecto migration default that should be used for that attribute. The string you use will be placed verbatim in the migration. Use fragments like `fragment(\\"now()\\")`, or for `nil`, use `\\"nil\\"`. For custom `Ash.Type` modules, see `c:AshPostgres.Type.value_to_postgres_default/3`. | -| [`calculations_to_sql`](#postgres-calculations_to_sql){: #postgres-calculations_to_sql } | `keyword` | | A keyword list of calculations and their SQL representation. Used when creating unique indexes for identities over calculations | -| [`identity_wheres_to_sql`](#postgres-identity_wheres_to_sql){: #postgres-identity_wheres_to_sql } | `keyword` | | A keyword list of identity names and the SQL representation of their `where` clause. See `AshPostgres.DataLayer.Info.identity_wheres_to_sql/1` for more details. | -| [`base_filter_sql`](#postgres-base_filter_sql){: #postgres-base_filter_sql } | `String.t` | | A raw sql version of the base_filter, e.g `representative = true`. Required if trying to create a unique constraint on a resource with a base_filter | +| [`calculations_to_sql`](#postgres-calculations_to_sql){: #postgres-calculations_to_sql } | `keyword` | | A keyword list of calculations and their SQL representation. Used when creating unique indexes for identities over calculations. On a temporal resource, the migration generator writes this SQL into an `execute("...")` call in the migration's Elixir source as is, so Elixir interprets backslash escapes and `#{` in it and a `"` ends the string; escape them for Elixir there. On other resources it is escaped for you. | +| [`identity_wheres_to_sql`](#postgres-identity_wheres_to_sql){: #postgres-identity_wheres_to_sql } | `keyword` | | A keyword list of identity names and the SQL representation of their `where` clause. See `AshPostgres.DataLayer.Info.identity_wheres_to_sql/1` for more details. On a temporal resource, the migration generator writes this SQL into an `execute("...")` call in the migration's Elixir source as is, so Elixir interprets backslash escapes and `#{` in it and a `"` ends the string; escape them for Elixir there. On other resources it is escaped for you. | +| [`base_filter_sql`](#postgres-base_filter_sql){: #postgres-base_filter_sql } | `String.t` | | A raw sql version of the base_filter, e.g `representative = true`. Required if trying to create a unique constraint on a resource with a base_filter. The migration generator does not escape this SQL consistently. It is written into the migration's Elixir source as is in check constraints (inside a `"""` heredoc), in the unique index of an identity without a `where`, and in the identities of a temporal resource (both inside a double-quoted string). There, Elixir interprets backslash escapes and `#{` in it, and a `"` ends a double-quoted string. In custom indexes and in the unique index of an identity with a `where`, it is escaped for you. Prefer SQL without backslashes, double quotes and `#{` here, and check the generated migration if you need them. | | [`simple_join_first_aggregates`](#postgres-simple_join_first_aggregates){: #postgres-simple_join_first_aggregates } | `list(atom)` | `[]` | A list of `:first` type aggregate names that can be joined to using a simple join. Use when you have a `:first` aggregate that uses a to-many relationship , but your `filter` statement ensures that there is only one result. Optimizes the generated query. | | [`skip_unique_indexes`](#postgres-skip_unique_indexes){: #postgres-skip_unique_indexes } | `atom \| list(atom)` | `[]` | Skip generating unique indexes when generating migrations | | [`unique_index_names`](#postgres-unique_index_names){: #postgres-unique_index_names } | `list({list(atom), String.t} \| {list(atom), String.t, String.t})` | `[]` | A list of unique index names that could raise errors that are not configured in identities, or an mfa to a function that takes a changeset and returns the list. In the format `{[:affected, :keys], "name_of_constraint"}` or `{[:affected, :keys], "name_of_constraint", "custom error message"}` | @@ -210,8 +210,8 @@ end | Name | Type | Default | Docs | |------|------|---------|------| -| [`up`](#postgres-custom_statements-statement-up){: #postgres-custom_statements-statement-up .spark-required} | `String.t` | | How to create the structure of the statement | -| [`down`](#postgres-custom_statements-statement-down){: #postgres-custom_statements-statement-down .spark-required} | `String.t` | | How to tear down the structure of the statement | +| [`up`](#postgres-custom_statements-statement-up){: #postgres-custom_statements-statement-up .spark-required} | `String.t` | | How to create the structure of the statement. Unless `code?` is true, the migration generator writes this SQL into an `execute("""...""")` heredoc in the migration's Elixir source as is, so when the migration compiles, Elixir interprets backslash escapes and `#{` in it. Escape them for Elixir: for SQL containing `'^\d+$'`, write `'^\\d+$'`, for example inside `~S(...)`. | +| [`down`](#postgres-custom_statements-statement-down){: #postgres-custom_statements-statement-down .spark-required} | `String.t` | | How to tear down the structure of the statement. Escape it as described for `up`. | | [`code?`](#postgres-custom_statements-statement-code?){: #postgres-custom_statements-statement-code? } | `boolean` | `false` | By default, we place the strings inside of ecto migration's `execute/1` function and assume they are sql. Use this option if you want to provide custom elixir code to be placed directly in the migrations | | [`global?`](#postgres-custom_statements-statement-global?){: #postgres-custom_statements-statement-global? } | `boolean` | `false` | By default, a multi-tenant resource's custom statements will be written into the tenant migration folder. Set this to true for statements that create global, shared structures so they are written into the public migration folder even when defined on a tenant resource. | | [`after_tables`](#postgres-custom_statements-statement-after_tables){: #postgres-custom_statements-statement-after_tables } | `list(String.t)` | `[]` | Table names whose structural operations must be complete before this statement's `up` runs. This does not wait for custom statements declared on those tables. Use this for raw SQL that references another table's columns or indexes. | @@ -396,7 +396,7 @@ check_constraint :price, "price_must_be_positive", check: "price > 0", message: | Name | Type | Default | Docs | |------|------|---------|------| | [`message`](#postgres-check_constraints-check_constraint-message){: #postgres-check_constraints-check_constraint-message } | `String.t` | | The message to be added if the check constraint fails | -| [`check`](#postgres-check_constraints-check_constraint-check){: #postgres-check_constraints-check_constraint-check } | `String.t` | | The contents of the check. If this is set, the migration generator will include it when generating migrations | +| [`check`](#postgres-check_constraints-check_constraint-check){: #postgres-check_constraints-check_constraint-check } | `String.t` | | The contents of the check. If this is set, the migration generator will include it when generating migrations. The migration generator writes this SQL into the migration's Elixir source as is, inside a `"""` heredoc, so when the migration compiles, Elixir interprets backslash escapes and `#{` in it. Escape them for Elixir: for the SQL `code ~ '^\d{4}$'`, write `check: ~S(code ~ '^\\d{4}$')`. | diff --git a/lib/check_constraint.ex b/lib/check_constraint.ex index 16ab815a..a0f01b5c 100644 --- a/lib/check_constraint.ex +++ b/lib/check_constraint.ex @@ -26,8 +26,11 @@ defmodule AshPostgres.CheckConstraint do ], check: [ type: :string, - doc: - "The contents of the check. If this is set, the migration generator will include it when generating migrations" + doc: ~S""" + The contents of the check. If this is set, the migration generator will include it when generating migrations. + + The migration generator writes this SQL into the migration's Elixir source as is, inside a `\"""` heredoc, so when the migration compiles, Elixir interprets backslash escapes and `#{` in it. Escape them for Elixir: for the SQL `code ~ '^\d{4}$'`, write `check: ~S(code ~ '^\\d{4}$')`. + """ ] ] end diff --git a/lib/data_layer.ex b/lib/data_layer.ex index 34414857..5ef2b90a 100644 --- a/lib/data_layer.ex +++ b/lib/data_layer.ex @@ -336,18 +336,27 @@ defmodule AshPostgres.DataLayer do ], calculations_to_sql: [ type: :keyword_list, - doc: - "A keyword list of calculations and their SQL representation. Used when creating unique indexes for identities over calculations" + doc: ~S""" + A keyword list of calculations and their SQL representation. Used when creating unique indexes for identities over calculations. + + On a temporal resource, the migration generator writes this SQL into an `execute("...")` call in the migration's Elixir source as is, so Elixir interprets backslash escapes and `#{` in it and a `"` ends the string; escape them for Elixir there. On other resources it is escaped for you. + """ ], identity_wheres_to_sql: [ type: :keyword_list, - doc: - "A keyword list of identity names and the SQL representation of their `where` clause. See `AshPostgres.DataLayer.Info.identity_wheres_to_sql/1` for more details." + doc: ~S""" + A keyword list of identity names and the SQL representation of their `where` clause. See `AshPostgres.DataLayer.Info.identity_wheres_to_sql/1` for more details. + + On a temporal resource, the migration generator writes this SQL into an `execute("...")` call in the migration's Elixir source as is, so Elixir interprets backslash escapes and `#{` in it and a `"` ends the string; escape them for Elixir there. On other resources it is escaped for you. + """ ], base_filter_sql: [ type: :string, - doc: - "A raw sql version of the base_filter, e.g `representative = true`. Required if trying to create a unique constraint on a resource with a base_filter" + doc: ~S""" + A raw sql version of the base_filter, e.g `representative = true`. Required if trying to create a unique constraint on a resource with a base_filter. + + The migration generator does not escape this SQL consistently. It is written into the migration's Elixir source as is in check constraints (inside a `\"""` heredoc), in the unique index of an identity without a `where`, and in the identities of a temporal resource (both inside a double-quoted string). There, Elixir interprets backslash escapes and `#{` in it, and a `"` ends a double-quoted string. In custom indexes and in the unique index of an identity with a `where`, it is escaped for you. Prefer SQL without backslashes, double quotes and `#{` here, and check the generated migration if you need them. + """ ], simple_join_first_aggregates: [ type: {:list, :atom}, diff --git a/lib/migration_generator/operation.ex b/lib/migration_generator/operation.ex index 5f217a2b..e5bab790 100644 --- a/lib/migration_generator/operation.ex +++ b/lib/migration_generator/operation.ex @@ -1121,6 +1121,10 @@ defmodule AshPostgres.MigrationGenerator.Operation do "create unique_index(:#{as_atom(table)}, [#{Enum.map_join(keys, ", ", &inspect/1)}], #{join(["name: \"#{index_name}\"", option("prefix", schema), option("nulls_distinct", nils_distinct?), option("where", where), concurrently_option])})" base_filter -> + # 3.0: escape `base_filter` here, as the other branches do through `option/2`. + # Today it is written into a plain string as is: a `"` in `base_filter_sql` ends + # the string and Elixir reinterprets backslash escapes and `#{`, so the same SQL + # comes out differently here than with an identity `where`. See #876. base_filter = "(#{base_filter})" "create unique_index(:#{as_atom(table)}, [#{Enum.map_join(keys, ", ", &inspect/1)}], where: \"#{base_filter}\", #{join(["name: \"#{index_name}\"", option("prefix", schema), option("nulls_distinct", nils_distinct?), concurrently_option])})" @@ -1163,6 +1167,11 @@ defmodule AshPostgres.MigrationGenerator.Operation do where_sql = if predicate, do: " WHERE (#{predicate})", else: "" + # 3.0: escape `predicate` and `exclude_cols` (from `identity_wheres_to_sql`, + # `base_filter_sql` and `calculations_to_sql`) for the generated `execute("...")`. + # Today they are written into it as is: a `"` ends the string and Elixir + # reinterprets backslash escapes and `#{`, unlike the non-temporal branches, + # which escape the same SQL through `option/2`. See #876. body = "ADD CONSTRAINT \\\"#{index_name}\\\" EXCLUDE USING gist (#{exclude_cols})#{where_sql}" @@ -1227,6 +1236,11 @@ defmodule AshPostgres.MigrationGenerator.Operation do @moduledoc false defstruct [:statement, :table, :schema, no_phase: true] + # 3.0: escape `up` and `down` in the generated `execute("""...""")` heredoc + # when `code?` is false. Today they are written into it as is, so Elixir + # reinterprets backslash escapes and `#{` in the SQL, and users escape it + # themselves to compensate (see #876). `SerialSequenceTransition` also renders + # through here and relies on `#{prefix()}` being interpolated at runtime. def up(%{statement: %{up: up, code?: false}}) do """ execute(\"\"\" @@ -1706,6 +1720,10 @@ defmodule AshPostgres.MigrationGenerator.Operation do }) do prefix = if schema, do: ", " <> option(:prefix, schema), else: "" + # 3.0: escape `check` and `base_filter` in the generated migration, for example + # with a `~S"""` heredoc. Today they are written into the `"""` heredoc as is, so + # Elixir reinterprets backslash escapes and `#{` in the SQL, and users escape it + # themselves to compensate (see #876). Escaping it changes what those users get. if base_filter do ~s''' create constraint(:#{as_atom(table)}, :#{as_atom(name)}, check: """ @@ -1751,6 +1769,7 @@ defmodule AshPostgres.MigrationGenerator.Operation do }) do prefix = if schema, do: ", " <> option(:prefix, schema), else: "" + # 3.0: escape `check` and `base_filter` here too; see `AddCheckConstraint.up/1`. if base_filter do ~s''' create constraint(:#{as_atom(table)}, :#{as_atom(name)}, check: """ diff --git a/lib/statement.ex b/lib/statement.ex index 726977e3..2faa7f3b 100644 --- a/lib/statement.ex +++ b/lib/statement.ex @@ -42,14 +42,16 @@ defmodule AshPostgres.Statement do ], up: [ type: :string, - doc: """ - How to create the structure of the statement + doc: ~S""" + How to create the structure of the statement. + + Unless `code?` is true, the migration generator writes this SQL into an `execute(\"""...\""")` heredoc in the migration's Elixir source as is, so when the migration compiles, Elixir interprets backslash escapes and `#{` in it. Escape them for Elixir: for SQL containing `'^\d+$'`, write `'^\\d+$'`, for example inside `~S(...)`. """, required: true ], down: [ type: :string, - doc: "How to tear down the structure of the statement", + doc: "How to tear down the structure of the statement. Escape it as described for `up`.", required: true ], after_tables: [