Merge remote-tracking branch 'origin/main' into claude/issue-15631-15632-error-docs-wire-shape

Author: claude

.changeset/analytics-icontains-per-dialect-fold.md

@@ -0,0 +1,19 @@
+---
+"@objectstack/service-analytics": patch
+---
+
+Analytics `$icontains` no longer compiles a `translate()` call on the `sqlite` and `mysql` dialects. On **SQLite** that function does not exist and the statement failed to parse — measured on the engine, not inferred. On **MySQL** the same construct was emitted and its arm is repaired the same way, but nothing was ever executed there: the MySQL arm is asserted as emitted TEXT only, on this face and on `driver-sql`'s alike, so no MySQL parse failure is claimed as measured.
+
+`$icontains` folds ASCII case on both sides of the comparison (#4706 Q1 = A). All three of this package's SQL compilers — the query's own `where` (`NativeSQLStrategy.buildFilterClause`), the ADR-0021 D-C read scope (`compileScopedFilterToSql`) and the `ObjectQLStrategy` echo of that statement — spelled that fold as `translate(col, 'ABC…', 'abc…')` on all four dialect values a compiler can see: `sqlite`, `mysql`, `postgres` and `unknown`, onto which `normalizeSqlDialect` maps everything else, an unset hook and `'oracle'` included. `translate()` is PostgreSQL/Oracle; SQLite has none. Measured on sql.js 1.14.1 (SQLite 3.49.1, the engine `driver-sqlite-wasm` runs), `SELECT translate('ABC','ABC','abc')` answers `no such function: translate` — so this was not a filter that returned the wrong rows, it was a statement the engine refused. On a SQLite datasource, an analytics `where` carrying `$icontains` and an **RLS read scope** carrying it were both unusable.
+
+The fold is now chosen per dialect, on the same construct table the case-exact text family already used, reached through one `fold` flag:
+
+- **SQLite** — `lower(col) GLOB lower(?)`. SQLite's `lower()` is ASCII-only (measured: `lower('CAFÉ')` is `cafÉ`), so this is the ruled fold rather than an approximation of it, and it runs.
+- **PostgreSQL** and the `unknown` residue — `translate()`, byte-for-byte what those two arms emitted before. Measured set for that word: this package's own suite pins six cells verbatim — `{NativeSQLStrategy, ObjectQLStrategy echo, compileScopedFilterToSql} × {dialect unset, 'postgres'}` for `{name: {$icontains: 'acme'}}`, full emitted SQL and the exact bound params — and the round-1 contract review widened it to **2,721 cells** (2,720 = `{undefined, 'postgres', 'unknown', 'oracle'} × 5 compiler paths × 8 filter shapes × 17 comparands`, plus the bare `{dialect: undefined}` cell), emitted at the merge-base blobs (all five hash-verified) and again at this head: **0 changed cells, 0 error cells**. Outside that set nothing is claimed — no PostgreSQL server was contacted, and on `sqlite` and `mysql` the bytes deliberately changed (340 of 680 cells each, all inside the four `$icontains` shapes).
+- **MySQL** — the nested-`REPLACE` fold over `CAST(… AS BINARY)`, matching what `driver-sql` emits for the same operator; the review measured the two faces byte-equal on 60 of 60 MySQL cells. Asserted as text only — no MySQL server is provisionable in the container that wrote this, so that cell is a declared skip, not a claimed pass.
+
+⚠️ Carve-out, stated because it is the surviving half of the defect and not an aside: an `unknown` dialect that is really SQLite is **not** fixed by this change. The residue is reached by four constructions the round-1 contract review drove rather than reasoned — a `SqlDriver` given a **class** client or an unrecognised spelling (`'libsql'`), a host hook answering knex's own `'sqlite3'`, a directly-constructed public `AnalyticsService` with the optional `sqlDialect` omitted, and a `data` service without `getDriverForObject`. For each of them `translate()` still reaches the engine and still fails to parse, on the `where` path, the read scope and the echo alike. No in-repo SQLite driver lands there — `SqliteWasmDriver` and `TursoDriver` both answer `"sqlite"`, measured — so this is an embedder-composition population, not a shipped-driver one. Tracked as #16028.
+
+`$icontains` and the case-sensitive `$contains` family remain two separate constructs on every dialect the compilers accept — collapsing them would give `$contains` back the case fold #4706 Q2 = A took away from it. Measured set for that word: 510 cells (six dialect names — the four values above plus `'oracle'` and an unset hook, which both normalize to `unknown` — × 5 compiler paths × 17 comparands), 0 of them identical between the two families and no `$contains` cell carrying a fold.
+
+⚠️ One deliberate divergence from `driver-sql`, recorded here rather than only in this package's source: `driver-sql`'s own `unknown` arm folds with `LOWER()`, this one keeps `translate()`. Each face keeps the residue it already had, and adopting `LOWER()` here would silently restore on PostgreSQL the Unicode fold #4706 Q1 = A rules out. The pointer exists on this side only; `driver-sql` carries no cross-reference back.

.changeset/analytics-measure-result-type-string-family.md

@@ -0,0 +1,25 @@
+---
+"@objectstack/service-analytics": minor
+---
+
+A `min`/`max` over a string-valued field is described as `string`, not `number` (#16098)
+
+The sibling population of the temporal fix. `min` and `max` return a value **of the aggregated field's own type**, so a `min` over a `text` / `select` / `lookup` / `autonumber` column carries a string — and `POST /api/v1/analytics/dataset/query` described every one of those columns as `type: "number"`, exactly as it did for the temporal family before the temporal half landed.
+
+What changed:
+
+- **`measureResultType` now answers `string` for the string-valued field types too**, in the same one table it already answered `time` from. No second mechanism and no new call site: the rule still answers `undefined` for "no correction", and `queryDataset`'s ADR-0021 result-column enrichment still applies it once, downstream of all four producers of the shape.
+- **The corrected spelling is `string`**, the `DimensionType` word a `lookup` or `string` DIMENSION column in the same response already carries (`dataset-compiler.dimensionType`). A textual measure spelled `text` would have been a sixth word in a five-word wire vocabulary, leaving every existing consumer branch unreached — the same argument that chose `time` over `datetime`.
+- **Membership is composed from `@objectstack/spec`'s own value classes** (`STRING_VALUE_TYPES`, `SINGLE_OPTION_TYPES`, `REFERENCE_VALUE_TYPES`) rather than re-listed, so what the platform says a field type STORES and what this rule says a `min` over it RETURNS cannot drift.
+
+Corrected: `text`, `textarea`, `email`, `url`, `phone`, `password`, `secret`, `markdown`, `html`, `richtext`, `code`, `color`, `signature`, `qrcode`, `select`, `radio`, `lookup`, `master_detail`, `tree`, `user`, `autonumber` — twenty-one members, each verdict read off the two shipped statements of what the type stores (the spec value contract and `driver-sql`'s DDL column switch).
+
+Deliberately NOT corrected, with the measurement recorded rather than a guess shipped as a declaration:
+
+- **`boolean` / `toggle`** — Postgres has no `min(boolean)` at all, SQLite answers `0`/`1` as numbers, and the driver seam has been recorded answering `false`/`true`. Three readings that disagree about whether a value exists and what kind it is. `DimensionType` does carry a `boolean` word, so the correction is spellable; it is not made.
+- **The JSON-column classes** (`multiselect` / `checkboxes` / `tags`, `composite` / `repeater` / `record` / `location` / `address` / `vector`, `json`) — no `min` over `jsonb` on Postgres, serialized TEXT on SQLite.
+- **The file types** (`image` / `file` / `avatar` / `video` / `audio`) — their stored form is mid-migration under ADR-0104 D3: the value contract already says an opaque `sys_file` id while the DDL still gives them a JSON column.
+- **`formula`** — its result type IS declared, on `FieldSchema.returnType`, but that key is not on `AnalyticsServiceConfig.sourceFieldMeta`'s return shape and is itself optional.
+- **`summary`** — measured NUMERIC on both shipped statements (the spec's `NUMERIC_VALUE_TYPES`, and `driver-sql`'s `table.float` column), so the `number` it already carried is correct rather than merely unexamined.
+
+Every member of `FieldType` now carries an explicit verdict, pinned by a test that walks the enum: a field type added to the spec fails that pin instead of silently inheriting the flat `number`.

.changeset/analytics-measure-result-type-temporal.md

@@ -0,0 +1,24 @@
+---
+"@objectstack/service-analytics": minor
+"@objectstack/spec": patch
+---
+
+A dataset measure's `fields[].type` stops contradicting the value beside it: a `min`/`max` over a temporal field is described as `time`, not `number` (#15768)
+
+`POST /api/v1/analytics/dataset/query` described **every** measure column as `type: "number"`, including a `min`/`max` over a `date` / `datetime` / `time` field whose value in the same response is an ISO instant. Measured on a real boot (`@objectstack/cli` 17.3.0, SQLite dev datasource):
+
+```json
+{"rows":[{"oldest_last_update_at":"2026-07-04T07:00:00.000Z"}],
+ "fields":[{"name":"oldest_last_update_at","type":"number","label":"Oldest touch","format":"relative"}]}
+```
+
+`min` and `max` return a value **of the aggregated field's own type**, so that column carries an instant and the metadata denied it — which is enough on its own to keep a formatter that branches on the declared type from ever reaching a temporal branch.
+
+What changed:
+
+- **The measure column's type is resolved from the authored measure plus the source field's declared type**, in `AnalyticsService.queryDataset`'s ADR-0021 result-column enrichment — the same block that already resolves `label` / `format` / `currency` / `percentScale`, and the one seam every producer of the shape passes through on the way to the route, which relays that method's return verbatim. The rule itself is `measureResultType` in the new `measure-result-type.ts`, so the per-aggregate verdict has one home instead of four copies.
+- **The corrected spelling is `time`**, the `DimensionType` word a temporal DIMENSION column in the same response has always carried. A second temporal word in one wire position would have left every existing consumer branch unreached.
+- **Only `min` and `max` move.** `count` and `count_distinct` are numeric however temporal the column they read is; `sum` / `avg` over a temporal column are refused by no layer and answered by the backend (an epoch mean on SQLite, an error on Postgres), so there is no single value for a type to describe and none is invented; a derived measure is numeric by construction, because `computeDerived` coerces its operands with `Number()`. Row values are untouched on every path.
+- **Tiered "cannot answer, do not block".** A host with no source-field metadata wired, and a measure over a relationship PATH (which the source-field lookup resolves against the base object and therefore cannot answer), both leave the column exactly as the query layer produced it.
+
+`AnalyticsResult.fields[].type` and the `AnalyticsResultResponse` schema now state the vocabulary this position speaks and what each aggregate answers; neither declaration widens — the wire type was, and remains, a string.

.changeset/analytics-preview-min-max-operand-type.md

@@ -0,0 +1,25 @@
+---
+"@objectstack/service-analytics": minor
+---
+
+A draft-preview `min`/`max` answers the operand's own type instead of `0`, and a preview dimension column is described by its own type
+
+`POST /api/v1/analytics/dataset/query` has two producers of one response: the engine, and — when the request renders the as-if-published world over a pending seed draft (ADR-0037 P3) — `evaluateAnalyticsQueryOverRows`. The second one coerced every aggregate operand with `Number()` and dropped the non-finite ones, so a `min` / `max` over a non-numeric field answered `0`. Measured on one dataset and one row set, with two services differing only in whether a pending seed draft exists:
+
+```
+live     {"category":"travel","latest_spend":"2026-05-12"}
+preview  {"category":"travel","latest_spend":0}
+```
+
+That is not a mislabelled column: it is a different, wrong answer to the same query, with no refusal and no warning, on the path an author is looking at *while* authoring the dataset.
+
+What changed, per member of the closed `AggregationFunction` vocabulary:
+
+- **`min` / `max` return the winning operand in its own type.** Ordering goes through this file's shared `compare` — so an ISO date orders as a date, a BSON `Date` orders as its instant against wire text, and text orders the way `MIN(text_col)` does on a SQL face — with a numeric arm so a numeric column written as text (`'800'`) still orders numerically. `cross-object-rebucket.ts` settled the identical question for the recombination path: the value these two pick is a value OF the column, so it must come back in the shape the row carried.
+- **A group whose operand is null throughout answers `null`, not `0`** — `emptyGroupValueFor` (`@objectstack/spec/data`) rules `min` / `max` over nothing unanswerable, and `0` reads as a measurement nobody made.
+- **`count_distinct` answers a cardinality again.** Its arm was spelled `countDistinct`, a word no producer mints (`dataset-compiler` copies the spec's `count_distinct` through), so it was unreachable and the measure fell to the numeric default — answering a row count under the author's `count_distinct` name (measured: `3` where the live path says `2`).
+- **`count` stays a row count and `sum` / `avg` stay arithmetic.** Counting dates is still counting.
+- **`sum` / `avg` over a TEMPORAL operand is deliberately unchanged.** There is no defined answer — the SQL faces do not agree on one either — and refusing an incoherent aggregate/field-type pair is an open decision, not this fix's to invent.
+- **A dimension column is typed from the cube dimension**, the same expression both live producers use (`d?.type || 'string'`), so a `date` dataset dimension is `time` on the preview path as it already was on the live one. A MEASURE column keeps the `number` every producer mints; correcting that is the ADR-0021 descriptor pass's one rule, not a second copy here.
+
+Derived measures are untouched: `computeDerived` still coerces with `Number()` and answers `null` for a non-finite operand — but a derived ratio over a temporal `min` / `max` now sees a date instead of the spurious `0`, so it answers `null` on the preview path exactly as it already did on the live one.

.changeset/analytics-text-family-case-exact-per-dialect.md

@@ -0,0 +1,18 @@
+---
+"@objectstack/service-analytics": minor
+"@objectstack/driver-sql": minor
+---
+
+The analytics SQL compilers compile the case-sensitive text family per dialect, so a `$contains` policy on SQLite stops admitting rows it excludes (#15684)
+
+`$contains` / `$notContains` / `$startsWith` / `$endsWith` are case-SENSITIVE on every backend (#4706 Q2 = A). All three of `service-analytics`' SQL compilers emitted `col LIKE ? ESCAPE ?` on every dialect, and SQLite's `LIKE` folds ASCII case unconditionally — the fold cannot be turned off per statement, because `PRAGMA case_sensitive_like` is a connection-global switch. Measured on sql.js over the shared `FILTER_TEXT_ROWS` fixture, `{ name: { $contains: 'acme' } }` answered `['1','2']` — `ACME Corp` **and** `acme corp` — where `FILTER_TEXT_CASES` says `['2']`.
+
+On two of the three compilers that is a wrong chart. The third is `read-scope-sql.ts`, the ADR-0021 D-C read scope: a scope that **admits** rows the policy's case-sensitive predicate excludes is over-reach, not a loose filter — the same reading that file already applied to its own `LIKE` escaping. The `/analytics/sql` echo was wrong in a third way: it printed `LIKE` while the statement it claims to reproduce ran through a driver that has emitted `GLOB` on the SQLite dialects since #6518.
+
+What changed:
+
+- **The construct is chosen per dialect** (`text-match-sql.ts`), arm for arm with `driver-sql`'s own table: `GLOB` on SQLite (case-exact by definition, with its own `*` / `?` / `[` escaped class and no `ESCAPE` clause), `LIKE` over `CAST(… AS BINARY)` on MySQL, and `LIKE` **unchanged** on Postgres, where it is already exactly the ruled semantics. There is no single construct that is case-exact and parses on all three, so the dialect had to become an input rather than a guess.
+- **The dialect arrives from the driver that will execute the statement.** New optional `AnalyticsServiceConfig.sqlDialect`, wired by `AnalyticsServicePlugin` from `IDataEngine.getDriverForObject`. `SqlDriver.dialectName` is now public so that answer can be read without a second dialect-resolution table drifting behind the driver's own knex spellings; it is derived and read-only.
+- **A host that answers no dialect keeps the `LIKE` it always got** — "cannot answer, do not block". Postgres deployments see byte-identical SQL.
+
+`$icontains` is untouched: it keeps its own ASCII-only fold on both sides, and collapsing the two families onto one path would hand the case-exact family back the fold the ruling took away from it. `LIKE` escaping is unchanged wherever a `LIKE` is still emitted.