diff --git a/.changeset/cast-blob-compile-option-claim.md b/.changeset/cast-blob-compile-option-claim.md new file mode 100644 index 0000000000..62ff4360e6 --- /dev/null +++ b/.changeset/cast-blob-compile-option-claim.md @@ -0,0 +1,14 @@ +--- +"@objectstack/spec": patch +"@objectstack/driver-turso": patch +"@objectstack/driver-sql": patch +"@objectstack/service-analytics": patch +--- + +Correct the documented reason for rejecting `CAST(col AS BLOB) LIKE ?` as a portable case-exact construct. + +Four headers stated, as a universal fact about SQLite, that the construct "was measured to return NOTHING". That is not a property of SQLite: whether `LIKE` is false for a BLOB operand is fixed when SQLite is compiled, by `SQLITE_LIKE_DOESNT_MATCH_BLOBS`, and the two SQLite builds this project ships disagree about it. Measured over the shared `FILTER_TEXT_ROWS` fixture, `{ name: { $contains: 'acme' } }` compiled to that construct returns `[]` on better-sqlite3 13.0.3 (SQLite 3.53.4, flag compiled in) and `['1','2']` on sql.js 1.14.1 (SQLite 3.49.1, flag absent) — the latter being exactly the ASCII case-folding defect the construct was being considered to avoid. + +No behaviour changes and no conclusion changes: all four sites still reject the construct and still choose `GLOB`. The rejection is now stated in a form that does not depend on any particular return value — a construct whose meaning is decided by an upstream compile flag cannot carry a read scope, because it means two different things on the two builds shipped here. Two supporting readings are recorded alongside it: `typeof CAST(name AS BLOB)` is `'blob'` on both builds, so the CAST is not the part that differs, and `GLOB` answers identically on both. + +Documentation only. `@objectstack/spec` and `@objectstack/driver-turso` ship the corrected text in their published type declarations (and `spec` also publishes the corrected source file directly, via its `src/**/*.zod.ts` entry); for `@objectstack/driver-sql` and `@objectstack/service-analytics` the change reaches published output only through sourcemaps. diff --git a/packages/drivers/driver-sql/src/sql-driver.ts b/packages/drivers/driver-sql/src/sql-driver.ts index ce45e8ded1..e57febd750 100644 --- a/packages/drivers/driver-sql/src/sql-driver.ts +++ b/packages/drivers/driver-sql/src/sql-driver.ts @@ -2864,10 +2864,32 @@ function mysqlAsciiLowerBinary(expr: string): string { * - **SQLite → `GLOB`.** `LIKE`'s ASCII fold cannot be turned off per-statement; * `PRAGMA case_sensitive_like` is a CONNECTION-global switch, so one query * would change every other query's meaning. Of the operand-level tricks, - * `CAST(col AS BLOB) LIKE ?` was measured to return NOTHING at all (SQLite's - * LIKE is false for a BLOB operand), so the operator has to change. `GLOB` is - * case-exact by definition and carries its own escape mechanism - * ({@link escapeGlobComparand}). `lower()` in front of it is still the + * `CAST(col AS BLOB) LIKE ?` is disqualified by something worse than + * failing: it means TWO DIFFERENT THINGS on the two SQLite builds this repo + * ships. Whether `LIKE` is false for a BLOB operand is not a property of + * SQLite the language — it is set when SQLite is COMPILED, by + * `SQLITE_LIKE_DOESNT_MATCH_BLOBS`. Measured over the shared + * `FILTER_TEXT_ROWS` fixture, `{name: {$contains: 'acme'}}` compiled to that + * construct: + * + * | build | `SQLITE_LIKE_DOESNT_MATCH_BLOBS` | rows | + * |---|---|---| + * | better-sqlite3 13.0.3 (SQLite 3.53.4) | compiled in | `[]` | + * | sql.js 1.14.1 (SQLite 3.49.1) | absent | `['1','2']` — `ACME Corp` AND `acme corp` | + * + * So one build silently answers nothing and the other silently answers + * exactly the ASCII over-fold this whole function exists to end, and which + * one a caller gets is decided by a flag upstream of us. That divergence is + * the rejection on its own: a construct whose meaning depends on how the + * driver's SQLite was BUILT cannot carry a read scope (#3948) whatever value + * it happens to return in any one container — the `[]` above is a build's + * answer, not SQLite's. The CAST itself is not the part that differs: + * `typeof CAST(name AS BLOB)` is `'blob'` on BOTH builds, so what diverges is + * purely `LIKE`'s rule for a blob operand, which is the compile-time half. So + * the operator has to change. `GLOB` is case-exact by definition, answers + * `['2']` on BOTH builds above, and carries its own + * escape mechanism ({@link escapeGlobComparand}). `lower()` in front of it is + * still the * `$icontains` fold, and still ASCII-only: measured, `lower('CAFÉ')` is * `'cafÉ'`, so `lower(name) GLOB '*café*'` answers row 4 and `'*cafÉ*'` * answers row 3 — the Q1 = A boundary, executed rather than argued. diff --git a/packages/drivers/driver-turso/src/remote-transport.ts b/packages/drivers/driver-turso/src/remote-transport.ts index e01130df15..8ea56d50d6 100644 --- a/packages/drivers/driver-turso/src/remote-transport.ts +++ b/packages/drivers/driver-turso/src/remote-transport.ts @@ -3036,7 +3036,20 @@ export class RemoteTransport { * over-matching rather than a near miss. The fold cannot be switched off per * statement (`PRAGMA case_sensitive_like` is connection-global, so one query * would silently redefine every other query on the same connection), and - * `CAST(col AS BLOB) LIKE ?` was measured to match NOTHING at all. `GLOB` is + * `CAST(col AS BLOB) LIKE ?` cannot replace it either: whether `LIKE` is false + * for a BLOB operand is fixed when SQLite is COMPILED, by + * `SQLITE_LIKE_DOESNT_MATCH_BLOBS`, so the construct means two DIFFERENT + * things on the two SQLite builds this repo ships. Measured over the shared + * `FILTER_TEXT_ROWS` fixture, `{name: {$contains: 'acme'}}` compiled to it: + * better-sqlite3 13.0.3 (SQLite 3.53.4, flag compiled in) answers `[]`, and + * sql.js 1.14.1 (SQLite 3.49.1, flag absent) answers `['1','2']` — the very + * over-match described above. `typeof CAST(name AS BLOB)` is `'blob'` on BOTH, + * so the CAST is not the part that differs; `LIKE`'s blob rule is. That + * divergence disqualifies it here without any claim about return values, and + * it bites hardest on THIS face: a remote transport cannot pin the build its + * libSQL server was compiled from, so the flag is not merely upstream, it is + * across the wire. `GLOB` carries no such dependency — measured, it answers + * `['2']` on both builds. It is * SQLite's case-exact pattern operator and is what both SQLite faces now * emit — `SqlDriver.applyLike`'s `textMatchPredicate` reaches the identical * decision for the local transport, and `turso-local-remote-*` parity suites diff --git a/packages/services/service-analytics/src/text-match-sql.ts b/packages/services/service-analytics/src/text-match-sql.ts index 56cb378e6d..4c3e69e340 100644 --- a/packages/services/service-analytics/src/text-match-sql.ts +++ b/packages/services/service-analytics/src/text-match-sql.ts @@ -31,8 +31,16 @@ * Postgres and MySQL. * - `CAST(… AS BINARY)` is byte-wise on MySQL, is not a type on Postgres, and * takes NUMERIC affinity on SQLite (it would compare a number). - * - `CAST(col AS BLOB) LIKE ?` was measured on the driver side to return - * NOTHING at all — SQLite's LIKE is false for a BLOB operand. + * - `CAST(col AS BLOB) LIKE ?` means two DIFFERENT things on the two SQLite + * builds this repo ships, which disqualifies it more firmly than any single + * wrong answer would. Whether `LIKE` is false for a BLOB operand is fixed + * when SQLite is COMPILED, by `SQLITE_LIKE_DOESNT_MATCH_BLOBS`, so it is + * not a portable property at all: measured on this same fixture, + * `{ name: { $contains: 'acme' } }` answered `[]` on better-sqlite3 13.0.3 + * (SQLite 3.53.4, flag compiled in) and `['1','2']` on sql.js 1.14.1 + * (SQLite 3.49.1, flag absent) — the latter being precisely the over-fold + * above. A construct that a read scope's correctness rests on cannot be one + * whose meaning an upstream build flag decides. * - The portable primitives that ARE case-sensitive everywhere (`replace()`) * express "occurs somewhere" but not "occurs at the start / at the end" * without character-length arithmetic that is spelled differently on every diff --git a/packages/spec/src/data/filter.zod.ts b/packages/spec/src/data/filter.zod.ts index 9a711e03cf..86b3b74026 100644 --- a/packages/spec/src/data/filter.zod.ts +++ b/packages/spec/src/data/filter.zod.ts @@ -1082,11 +1082,28 @@ export function matchesLikePattern(value: string, pattern: string, foldAscii = f * #6518 measured every way out of that and landed on `GLOB`, which is * case-exact by definition: `PRAGMA case_sensitive_like` is CONNECTION-global, * so one query would redefine every other query on the connection, and - * `CAST(col AS BLOB) LIKE ?` was measured to match NOTHING. Three of the five - * backends are SQLite underneath (driver-sql on better-sqlite3, - * driver-sqlite-wasm, driver-turso on both transports), so without this - * translation `$like` would mean one thing on Postgres and another on SQLite — - * the divergence #6518 closed for `$contains`, re-opened one operator over. + * `CAST(col AS BLOB) LIKE ?` is not portable enough to be the answer — whether + * `LIKE` is false for a BLOB operand is fixed when SQLite is COMPILED, by + * `SQLITE_LIKE_DOESNT_MATCH_BLOBS`, so that construct means two DIFFERENT + * things on the two SQLite builds this repo ships. Measured over the shared + * `FILTER_TEXT_ROWS` fixture, `{name: {$contains: 'acme'}}` compiled to it: + * + * | build | `SQLITE_LIKE_DOESNT_MATCH_BLOBS` | rows | + * |---|---|---| + * | better-sqlite3 13.0.3 (SQLite 3.53.4) | compiled in | `[]` | + * | sql.js 1.14.1 (SQLite 3.49.1) | absent | `['1','2']` — `ACME Corp` AND `acme corp` | + * + * That divergence is disqualifying on its own, without any claim about what the + * construct returns: the flag is upstream of us, so the meaning is a property of + * how a backend's SQLite was BUILT. And this file is exactly where that bites — + * three of the five backends are SQLite underneath (driver-sql on + * better-sqlite3, driver-sqlite-wasm, driver-turso on both transports), and + * those are NOT the same build: the two rows above ARE two of the three. `GLOB` + * has no such dependency — measured, it answers `['2']` on BOTH builds — and + * `typeof CAST(name AS BLOB)` is `'blob'` on both, so the CAST is not the part + * that differs; `LIKE`'s blob rule is. Without this translation `$like` would + * mean one thing on Postgres and another on SQLite — the divergence #6518 + * closed for `$contains`, re-opened one operator over. * * `GLOB` has a DIFFERENT pattern language, which is the whole reason this is a * translation and not an escape: