Skip to content

Commit 9cdffbe

Browse files
os-muskclaude
andauthored
feat(spec,driver-sql,cli): one physical representation for the NUMERIC column family, read by all three producers (#16887)
* wip: inherited unverified draft from the predecessor session (#16318) The container restart that killed the previous os-dev left this in the worktree, uncommitted. Committed verbatim so it is not the only copy; every number in it is re-measured before anything is claimed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ADLdAs2pVcH17h9tZKWMBg * feat(spec,driver-sql,cli): one per-field-type physical representation for the NUMERIC column family Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ADLdAs2pVcH17h9tZKWMBg * fix(driver-sql): the numeric read coercion runs on every dialect Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ADLdAs2pVcH17h9tZKWMBg * test(spec,driver-sql,cli): pin the numeric representation on all three producers Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ADLdAs2pVcH17h9tZKWMBg * test(cli): the multiple-JSON pin takes NOT NULL from storage.notNull (ADR-0113) Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ADLdAs2pVcH17h9tZKWMBg * chore(spec): regenerate api-surface and export-origins for the numeric column table Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ADLdAs2pVcH17h9tZKWMBg * chore: changeset for the numeric column representation table Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ADLdAs2pVcH17h9tZKWMBg * fix(docs,driver-sql,spec): the numeric column's documentation, its live-dialect pins and the read-path bound The protocol doc stated the opposite of the DDL this branch emits, and three claims the branch makes had no instrument behind them. - content/docs/protocol/objectql/types.mdx: `number`, `currency` and `percent` said "a floating-point column (REAL / FLOAT)" and the type-conversion matrix said `REAL` / `FLOAT` / `REAL` for the family and for `summary`. All of it now states the exact-decimal column, the "new columns only" bound, and the SQLite and per-dialect `rating` consequences, with `slider` / `progress` / `rating` given rows the matrix never carried. - packages/spec/src/api/sortability.zod.ts (and its generated content/docs/references/api/sortability.mdx): the `summary` fossil said "an engine-maintained `table.float`". - packages/services/service-analytics/src/measure-result-type.ts: the same fossil, as the reason `summary` needs no correction. The reason is unchanged; only the column it names moved. New live-dialect cell (`sql-driver-16318-numeric-representation-live-dialects.test.ts`): `numeric_precision` / `numeric_scale` read off the server's own catalog and compared against the spec table, the `typeof === 'number'` read that SQLite cannot exercise, and `rating`'s fractional disposition asserted PER DIALECT — PostgreSQL refuses, MySQL rounds. The PG half was executed against a live PostgreSQL 16.13; the MySQL half runs in the "Temporal Conformance (live PG + MySQL)" job. `formatOutput`'s numeric read coercion is scoped per dialect. It reads the authorable `NUMERIC_VALUE_TYPES` half on the server dialects and keeps the wider `NUMERIC_SCALAR_TYPES` set on SQLite, where the legacy TEXT-affinity repair lives. The aliases `integer` / `int` / `float` are how an external, introspected column reaches the driver, and node-postgres hands `bigint` back as a string precisely because it does not fit a JS double: coercing it would round above 2^53 on a table this change never created, outside the "new columns only" bound. Changeset: `rating`'s two dialects stated separately, the residual bound restated as "30 fractional digits are kept", the binary64 read seam named, and the ADR-0087 disposition argued positively instead of resting on the detector miss the gate's own docblock warns about. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018rzQyhLGC5iVs11V3TzRs5 * docs(spec): state the residual bound the changeset states, and point the ADR-0087 marker at the right bullet F1 — `packages/spec/src/data/numeric-column-representation.ts`'s "residual bound" section said only that magnitudes below 1e-30 round to zero, while the shipped changeset states the column keeps 30 fractional digits, that precision loss BEGINS around |x| < 1e-13 and is TOTAL below 1e-30, and that magnitudes at or above 1e35 are refused. That docblock is the canonical home both `sql-driver.ts` and `os generate migration` point readers at ("the spec module carries the measurements, the ruling and the residual bound"), and it says of itself that the bound is "stated here so no reader has to rediscover it" — so it now states both ends of the bound in the changeset's own terms. F2 — `.changeset/numeric-column-representation.md` said "Three consequences" above four bullets, and its ADR-0087 marker claimed "bullet 3 IS a prescription" where bullet 3 is the read-seam bullet and the prescription is bullet 4 (the `NOT NULL` / `storage.notNull` one). The marker is the auditable artifact the gate prints verbatim, so a wrong internal pointer defeats its purpose. Exactly one `adr-0087:` marker, same category and same argument. Docblock and changeset prose only: no behaviour change, no schema move, no test edit. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017Js5kTpTtxieBjPyScgxJ3 --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 07150b3 commit 9cdffbe

16 files changed

Lines changed: 1421 additions & 76 deletions
Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
---
2+
'@objectstack/spec': minor
3+
'@objectstack/driver-sql': minor
4+
'@objectstack/cli': minor
5+
---
6+
7+
One physical representation for the NUMERIC column family, read by every producer of DDL
8+
9+
`packages/spec` now states, per field type, what column a numeric field gets, and all three
10+
producers read it: `SqlDriver.createColumn`, `os generate migration --format sql` and
11+
`os generate migration --format typescript`. Measured on live PostgreSQL 16.13, one object
12+
through all three producers, before and after:
13+
14+
```
15+
BEFORE AFTER
16+
driver sql gen ts gen all three
17+
number real numeric(18,2) numeric(8,2) numeric(65,30)
18+
currency real numeric(18,2) numeric(8,2) numeric(65,30)
19+
percent real numeric(5,2) numeric(8,2) numeric(65,30)
20+
slider real numeric(18,2) numeric(8,2) numeric(65,30)
21+
summary real numeric(18,2) numeric(8,2) numeric(65,30)
22+
progress real numeric(5,2) numeric(8,2) numeric(65,30)
23+
rating real integer integer integer
24+
```
25+
26+
7 of 7 columns diverged before, 0 of 7 after. Every arm of the old split lost data in its own
27+
direction: `real` is IEEE-754 binary32, so a `currency` of `1234567.89` read back `1234567.9`;
28+
`numeric(5,2)` and `numeric(18,2)` silently ROUND a legitimate `33.333` to `33.33` (round
29+
half-up — executed, not inferred); `numeric(8,2)` refused `1234567.89` outright. `65,30` is
30+
MySQL's documented `DECIMAL` maximum and therefore the portable one, and it is the only
31+
candidate measured to lose nothing on a nine-value corpus.
32+
33+
Both migration formats also take the physical `NOT NULL` from `storage.notNull` and never from
34+
`required`, which is where `SqlDriver.createColumn` has taken it since ADR-0113: `required` is
35+
the write-time contract the record validator enforces, and binding the DDL to it made every
36+
post-deploy tightening a destructive migration.
37+
38+
**BREAKING** — new columns only; no existing column is retyped, no migration is planned, and no
39+
backfill runs. Four consequences to know before creating new tables:
40+
41+
- `rating` is an INTEGER column, and the two server dialects dispose of a fractional star count
42+
DIFFERENTLY — do not read one answer for both. PostgreSQL REFUSES `4.5` outright, where a
43+
`real` column accepted it. MySQL does NOT refuse: it ROUNDS, and `4.5` becomes `5` with no
44+
error, which is a silent alteration and the reason to declare a `slider` (in the exact-decimal
45+
set) for anything that wants fractional values. SQLite refuses nothing either: it stores `4.5`
46+
as a REAL in an INTEGER-affinity column, unchanged from today.
47+
- An exact-decimal column is bounded where a float is not, in BOTH directions. It keeps 30
48+
fractional digits: a magnitude whose significant digits run past the 30th decimal place loses
49+
the tail silently — `1.2345678901234567e-15` stores as `0.000000000000001234567890123457`, so
50+
the loss begins around |x| < 1e-13 and is total below 1e-30 — and magnitudes at or above 1e35
51+
are REFUSED, where `real` kept about seven significant digits out to ~1e38. A refusal is loud;
52+
the rounding it replaces was not.
53+
- Reads are bounded by the wire contract, not by the column. `find()` hands back a JS number
54+
(`z.number().finite()`), so a value that was never a JS double does not survive the round trip
55+
exactly — `1234567890123456.123` reads back `1234567890123456`, and 2^53+1 reads back 2^53.
56+
The fidelity this buys is an exact COLUMN read through a double: values written by this
57+
platform round-trip exactly, and SQL-side writers, `summary` roll-ups computed in SQL and any
58+
magnitude at or above 2^53 are bounded by the read seam. Widening that is a wire-contract
59+
change and is not in this release.
60+
- A generated migration no longer emits `NOT NULL` for a field marked only `required: true`.
61+
Declare `storage: { notNull: true }` for a physical constraint — which is what the platform's
62+
own table has always done since ADR-0113, and what `os migrate meta` deliberately does NOT
63+
supply on your behalf (the conversion that stamped it was withdrawn by maintainer ruling on
64+
2026-09-08). A source author who wants the column they had must write that block themselves;
65+
`required: true` keeps its own meaning, the write-time contract the record validator enforces.
66+
67+
SQLite emits byte-identical DDL for the six exact-decimal members: knex compiles both
68+
`table.decimal(name, p, s)` and `table.float(name)` to the same `float` column there.
69+
70+
<!-- adr-0087: not-required (no-migration-prescription) Claimed on a POSITIVE argument, not on the detector finding nothing — the failure mode this gate's own docblock names (#8277). Stated plainly: bullet 4 (the `NOT NULL` one) IS a prescription, and it is a prescription for a SOURCE AUTHOR, not for a metadata upgrader, which is the distinction ADR-0087's D8 addendum says this category cannot mechanically tell apart. The ledger serves `objectstack migrate meta`; the only ledger entry this change could carry is the `field-required-notnull-explicit` conversion, and that conversion was WITHDRAWN by maintainer ruling on 2026-09-08 (decision batch #85, #16693/#16890) on the ground that stamping `storage.notNull` wherever `required: true` appears is the implication ADR-0113 abolished — `packages/spec/src/conversions/registry.ts` now carries a tombstone saying re-adding one is the mistake it exists to stop. So `registered` is FORBIDDEN here, not merely unnecessary. The other four are closed on facts: the bumped packages publish (not `unpublished`); no id pre-dates the base (not `already-registered`); no named symbol is a non-metadata runtime interface (not `runtime-interface-only`); and `type-surface-only` fails its predicate 2, since this diff adds a module under `packages/spec/**`. The numeric half prescribes nothing at all — no spec key, no export and no config field is removed or renamed, existing sources parse and publish unchanged, and existing columns are untouched by the ruling that authorized this (「不考虑现有数据」). ⚠️ The residual is declared rather than hidden: the vocabulary has no category for a source-author prescription the ledger must not carry, which is D8's blind spot reached from a second direction; raised for the maintainer in the PR report rather than resolved by dropping the BREAKING banner. -->

content/docs/protocol/objectql/types.mdx

Lines changed: 48 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -263,9 +263,19 @@ quantity:
263263
- `min`/`max`: Range validation
264264

265265
**Database mapping:**
266-
- SQL driver: a floating-point column (`REAL` on PostgreSQL/SQLite, `FLOAT` on
267-
MySQL). `precision`/`scale` are validation and display metadata — the DDL does
268-
**not** emit `NUMERIC(precision, scale)`.
266+
- SQL driver: an **exact decimal** column at a fixed, portable size —
267+
`NUMERIC(65,30)` on PostgreSQL, `DECIMAL(65,30)` on MySQL, `float` on SQLite
268+
(which applies neither precision nor scale). The size comes from one
269+
per-field-type table in `packages/spec`
270+
(`NUMERIC_COLUMN_REPRESENTATION`), which the driver and both
271+
`os generate migration` formats all read, so one declaration produces one
272+
column whoever builds the table.
273+
⚠️ The field's own `precision`/`scale` remain validation and display
274+
metadata: the DDL does **not** emit `NUMERIC(precision, scale)` from *your*
275+
declared numbers — the column is the fixed pair above whatever the field
276+
declares.
277+
⚠️ **New columns only.** Nothing retypes a column that already exists; a
278+
table created before this keeps its `REAL`/`FLOAT` columns and their values.
269279
- MongoDB: `Number`
270280

271281
**Use cases:**
@@ -309,8 +319,17 @@ deprecated in the spec.
309319
- `precision` (0–10, default 2) for decimal places
310320

311321
**Database mapping:**
312-
- SQL driver: a floating-point column (`REAL` / `FLOAT`) — one column, no
313-
companion currency column and no JSON blob
322+
- SQL driver: the same **exact decimal** column as `number`
323+
(`NUMERIC(65,30)` / `DECIMAL(65,30)`; `float` on SQLite) — one column, no
324+
companion currency column and no JSON blob. Money is the member where the
325+
binary32 `REAL` this replaced lost a correctness question rather than a
326+
display one: `1234567.89` read back `1234567.9` from a `REAL` column.
327+
⚠️ The column is **not** a blanket `DECIMAL(18,2)`: the platform's own CLDR
328+
table carries 0-digit currencies (JPY, KRW) and 3-digit ones (BHD, KWD), and
329+
the currency-code schema fails open for crypto and custom codes, so a money
330+
column that fixes two decimals is wrong for a set the platform declines to
331+
close.
332+
⚠️ **New columns only** — see `number` above.
314333
- MongoDB: `Number`
315334

316335
---
@@ -331,7 +350,11 @@ discount_rate:
331350

332351
**Storage:** the percentage **number itself** — `25.5` means 25.5%, matching the
333352
`min: 0` / `max: 100` bounds above. It is *not* rescaled to a 0–1 ratio on write.
334-
Physically it is the same floating-point column as `number`.
353+
Physically it is the same **exact decimal** column as `number`
354+
(`NUMERIC(65,30)` / `DECIMAL(65,30)`; `float` on SQLite), on new tables only.
355+
That width is what holds a legitimate `33.333` — the narrow `NUMERIC(5,2)` the
356+
`--format sql` generator used to emit rounded it half-up to `33.33`, and
357+
rounded the 0–1 fraction storage of the same value to `0.33`.
335358

336359
<Callout>
337360
The separate `percent` **template filter** (`{{ record.rate | percent }}`) does
@@ -1174,15 +1197,16 @@ The column each type gets from the SQL driver, per dialect:
11741197
|---------------|------------|-------|--------|
11751198
| `text` / `textarea` / `html` | `TEXT` \* | `TEXT` \* | `TEXT` \* |
11761199
| `email` / `url` / `phone` / `password` | `VARCHAR(maxLength)` † | `VARCHAR(maxLength)` † | `VARCHAR(maxLength)` † |
1177-
| `number` / `currency` / `percent` | `REAL` | `FLOAT` | `REAL` |
1200+
| `number` / `currency` / `percent` / `slider` / `progress` | `NUMERIC(65,30)` ‡ | `DECIMAL(65,30)` ‡ | `float` ‡ |
1201+
| `rating` | `INTEGER` ‡ | `INT` ‡ | `INTEGER` ‡ |
11781202
| `date` | `DATE` | `DATE` | `TEXT` (`YYYY-MM-DD`) |
11791203
| `datetime` | `TIMESTAMPTZ` | `DATETIME(3)` | `TEXT` (canonical `…Z`) |
11801204
| `time` | `TIME` | `TIME(3)` | `TEXT` (`HH:MM:SS[.fff]`) |
11811205
| `boolean` / `toggle` | `BOOLEAN` | `BOOLEAN` | `INTEGER` `0`/`1` |
11821206
| `select` / `radio` | `VARCHAR(255)` | `VARCHAR(255)` | `VARCHAR(255)` |
11831207
| `multiselect` / `tags` | `JSON` | `JSON` | `TEXT` (JSON) |
11841208
| `lookup` / `master_detail` / `tree` | `VARCHAR(255)` | `VARCHAR(255)` | `VARCHAR(255)` |
1185-
| `summary` | `REAL` | `FLOAT` | `REAL` |
1209+
| `summary` | `NUMERIC(65,30)` ‡ | `DECIMAL(65,30)` ‡ | `float` ‡ |
11861210
| `autonumber` | `VARCHAR(255)` | `VARCHAR(255)` | `VARCHAR(255)` |
11871211
| `formula` | *(no column — virtual)* | *(no column)* | *(no column)* |
11881212
| `json` / `location` / `address` | `JSON` | `JSON` | `TEXT` (JSON) |
@@ -1209,6 +1233,22 @@ Note the neighbouring rows that deliberately do **not** follow this rule:
12091233
runtime-issued number — in none of those is the stored string the value the
12101234
field's `maxLength` describes, so all of them keep `VARCHAR(255)`.
12111235

1236+
‡ **The numeric family is NEW COLUMNS ONLY.** The size is one per-field-type
1237+
table in `packages/spec` (`NUMERIC_COLUMN_REPRESENTATION`) that
1238+
`SqlDriver.createColumn` and both `os generate migration` formats read, so the
1239+
three producers no longer disagree; before it they emitted `REAL`,
1240+
`NUMERIC(18,2)`/`NUMERIC(5,2)` and `NUMERIC(8,2)` for the same declaration.
1241+
`65,30` is MySQL's documented `DECIMAL` maximum and therefore the portable one.
1242+
Nothing retypes an existing column, plans a migration, or reports drift over the
1243+
difference — a table created before this keeps its `REAL`/`FLOAT` columns, and a
1244+
new numeric field added to it gets an exact-decimal column beside them.
1245+
On SQLite the exact-decimal members compile to the same `float` column the
1246+
driver emitted before (knex's SQLite `decimal` compiler is the literal `float`),
1247+
so SQLite keeps REAL affinity and gains no exactness; `rating` moves to INTEGER
1248+
affinity there and SQLite still accepts a fractional value as a REAL. The
1249+
refusal `rating` gains is a PostgreSQL/MySQL effect: PostgreSQL refuses a
1250+
fractional star count outright, and MySQL **rounds** it (4.5 arrives as 5).
1251+
12121252
Any field flagged `multiple: true` becomes a `JSON` column regardless of its
12131253
type. Relationship columns are plain id strings with no database `FOREIGN KEY`
12141254
constraint (see `lookup` above). The MongoDB driver is schemaless — it issues no

content/docs/references/api/sortability.mdx

Lines changed: 8 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -65,10 +65,14 @@ measured degradation is not refused; the projection covers all four:
6565
## Considered and deliberately NOT members
6666

6767
- `summary` / `autonumber` — the other two `COMPUTED_VALUE_TYPES`. They sort
68-
CORRECTLY (`summary` is an engine-maintained `table.float`, `autonumber`
69-
an engine-assigned `table.string`; measured on #6924), which is exactly
70-
why virtuality is judged by the storage predicate and never by the write
71-
contract — widening would refuse the two types that work.
68+
CORRECTLY (`summary` is an engine-maintained numeric column — `table.float`
69+
when #6924 measured it, an exact `table.decimal` on new tables since
70+
#16318's stated representation — and `autonumber` an engine-assigned
71+
`table.string`), which is exactly why virtuality is judged by the storage
72+
predicate and never by the write contract — widening would refuse the two
73+
types that work. ⚠️ The column TYPE is not what makes them sortable —
74+
having a PROVISIONED column is — which is why #16318's retype of the
75+
numeric family moved nothing in this projection.
7276
- `encrypted` / `secret` / `json` / `vector` and the other heavy or masked
7377
types — every one has a stored column, neither door refuses an ORDER BY
7478
over one, and the drivers execute it. Marking them unsortable here would

packages/cli/src/commands/generate-multiple-json-column.pin.test.ts

Lines changed: 23 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -207,15 +207,29 @@ describe('#14829 — `multiple: true` is one answer across all three surfaces',
207207
expect(tsColumn('multi_text')).toBe("table.jsonb('multi_text')");
208208
});
209209

210-
it('nullability still comes from `required`, not from the flag', () => {
211-
const out = generateMigrationSql({
212-
objects: { probe: { name: 'probe', fields: { tags_req: { type: 'lookup', multiple: true, required: true } } } },
213-
});
214-
expect(out).toContain('"tags_req" JSONB NOT NULL');
215-
const ts = generateMigrationTs({
216-
objects: { probe: { name: 'probe', fields: { tags_req: { type: 'lookup', multiple: true, required: true } } } },
217-
});
218-
expect(ts).toContain("table.jsonb('tags_req').notNullable();");
210+
/**
211+
* ⚠️ [#16318] The VEHICLE changed, the subject did not. This pin is about
212+
* `multiple` not deciding nullability; `required` was merely how a NOT NULL
213+
* was spelled when it was written. Both generators now take the physical NOT
214+
* NULL from `storage.notNull` and never from `required` (ADR-0113, which took
215+
* `SqlDriver.createColumn` off `required` because binding the DDL to it made
216+
* every post-deploy tightening a destructive migration) — so the constrained
217+
* case is spelled the new way, and the `required`-only case is asserted
218+
* BESIDE it: it must now be nullable in both formats, which is the half that
219+
* would have caught this change silently reverting.
220+
*/
221+
it('nullability comes from `storage.notNull`, not from the flag and not from `required`', () => {
222+
const constrained = { type: 'lookup', multiple: true, storage: { notNull: true } };
223+
const writeOnly = { type: 'lookup', multiple: true, required: true };
224+
const config = { objects: { probe: { name: 'probe', fields: { tags_nn: constrained, tags_req: writeOnly } } } };
225+
226+
const out = generateMigrationSql(config);
227+
expect(out).toContain('"tags_nn" JSONB NOT NULL');
228+
expect(out).toMatch(/"tags_req" JSONB(?! NOT NULL)/);
229+
230+
const ts = generateMigrationTs(config);
231+
expect(ts).toContain("table.jsonb('tags_nn').notNullable();");
232+
expect(ts).toContain("table.jsonb('tags_req').nullable();");
219233
});
220234

221235
// ── The authority, read where it lives ──────────────────────────────────

0 commit comments

Comments
 (0)