feat(spec,driver-sql,cli): one physical representation for the NUMERIC column family, read by all three producers

.changeset/numeric-column-representation.md

@@ -0,0 +1,55 @@
+---
+'@objectstack/spec': minor
+'@objectstack/driver-sql': minor
+'@objectstack/cli': minor
+---
+
+One physical representation for the NUMERIC column family, read by every producer of DDL
+
+`packages/spec` now states, per field type, what column a numeric field gets, and all three
+producers read it: `SqlDriver.createColumn`, `os generate migration --format sql` and
+`os generate migration --format typescript`. Measured on live PostgreSQL 16.13, one object
+through all three producers, before and after:
+
+```
+             BEFORE                                  AFTER
+             driver  sql gen        ts gen           all three
+number       real    numeric(18,2)  numeric(8,2)     numeric(65,30)
+currency     real    numeric(18,2)  numeric(8,2)     numeric(65,30)
+percent      real    numeric(5,2)   numeric(8,2)     numeric(65,30)
+slider       real    numeric(18,2)  numeric(8,2)     numeric(65,30)
+summary      real    numeric(18,2)  numeric(8,2)     numeric(65,30)
+progress     real    numeric(5,2)   numeric(8,2)     numeric(65,30)
+rating       real    integer        integer          integer
+```
+
+7 of 7 columns diverged before, 0 of 7 after. Every arm of the old split lost data in its own
+direction: `real` is IEEE-754 binary32, so a `currency` of `1234567.89` read back `1234567.9`;
+`numeric(5,2)` and `numeric(18,2)` silently ROUND a legitimate `33.333` to `33.33` (round
+half-up — executed, not inferred); `numeric(8,2)` refused `1234567.89` outright. `65,30` is
+MySQL's documented `DECIMAL` maximum and therefore the portable one, and it is the only
+candidate measured to lose nothing on a nine-value corpus.
+
+Both migration formats also take the physical `NOT NULL` from `storage.notNull` and never from
+`required`, which is where `SqlDriver.createColumn` has taken it since ADR-0113: `required` is
+the write-time contract the record validator enforces, and binding the DDL to it made every
+post-deploy tightening a destructive migration.
+
+**BREAKING** — new columns only; no existing column is retyped, no migration is planned, and no
+backfill runs. Three consequences to know before creating new tables:
+
+- `rating` is an INTEGER column on PostgreSQL and MySQL, so a fractional star count is now
+  REFUSED there where a `real` column accepted it. A field that wants fractional values is a
+  `slider`, which is in the exact-decimal set. SQLite is unaffected: it stores `4.5` as a REAL
+  in an INTEGER-affinity column and refuses nothing.
+- An exact-decimal column is bounded where a float is not: magnitudes below 1e-30 round to zero
+  and magnitudes at or above 1e35 are refused, where `real` kept about seven significant digits
+  out to ~1e38. A refusal is loud; the rounding it replaces was not.
+- A generated migration no longer emits `NOT NULL` for a field marked only `required: true`.
+  Declare `storage: { notNull: true }` for a physical constraint — which is what the platform's
+  own table has always done.
+
+SQLite emits byte-identical DDL for the six exact-decimal members: knex compiles both
+`table.decimal(name, p, s)` and `table.float(name)` to the same `float` column there.
+
+<!-- adr-0087: not-required (no-migration-prescription) Nothing an author writes is removed or renamed — no spec key, no export, no config field. The change is the physical column a NEW table gets; existing sources parse and publish unchanged, and existing columns are untouched by the ruling that authorized this (「不考虑现有数据」). There is no FROM -> TO edit to prescribe. -->

packages/cli/src/commands/generate-multiple-json-column.pin.test.ts

@@ -207,15 +207,29 @@ describe('#14829 — `multiple: true` is one answer across all three surfaces',
     expect(tsColumn('multi_text')).toBe("table.jsonb('multi_text')");
   });
 
-  it('nullability still comes from `required`, not from the flag', () => {
-    const out = generateMigrationSql({
-      objects: { probe: { name: 'probe', fields: { tags_req: { type: 'lookup', multiple: true, required: true } } } },
-    });
-    expect(out).toContain('"tags_req" JSONB NOT NULL');
-    const ts = generateMigrationTs({
-      objects: { probe: { name: 'probe', fields: { tags_req: { type: 'lookup', multiple: true, required: true } } } },
-    });
-    expect(ts).toContain("table.jsonb('tags_req').notNullable();");
+  /**
+   * ⚠️ [#16318] The VEHICLE changed, the subject did not. This pin is about
+   * `multiple` not deciding nullability; `required` was merely how a NOT NULL
+   * was spelled when it was written. Both generators now take the physical NOT
+   * NULL from `storage.notNull` and never from `required` (ADR-0113, which took
+   * `SqlDriver.createColumn` off `required` because binding the DDL to it made
+   * every post-deploy tightening a destructive migration) — so the constrained
+   * case is spelled the new way, and the `required`-only case is asserted
+   * BESIDE it: it must now be nullable in both formats, which is the half that
+   * would have caught this change silently reverting.
+   */
+  it('nullability comes from `storage.notNull`, not from the flag and not from `required`', () => {
+    const constrained = { type: 'lookup', multiple: true, storage: { notNull: true } };
+    const writeOnly = { type: 'lookup', multiple: true, required: true };
+    const config = { objects: { probe: { name: 'probe', fields: { tags_nn: constrained, tags_req: writeOnly } } } };
+
+    const out = generateMigrationSql(config);
+    expect(out).toContain('"tags_nn" JSONB NOT NULL');
+    expect(out).toMatch(/"tags_req" JSONB(?! NOT NULL)/);
+
+    const ts = generateMigrationTs(config);
+    expect(ts).toContain("table.jsonb('tags_nn').notNullable();");
+    expect(ts).toContain("table.jsonb('tags_req').nullable();");
   });
 
   // ── The authority, read where it lives ──────────────────────────────────

packages/cli/src/commands/generate-numeric-column-representation.pin.test.ts

@@ -0,0 +1,230 @@
+// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
+
+/**
+ * THE #16318 PIN: the NUMERIC column both migration generators emit is the
+ * numeric column `driver-sql` actually creates, and both read it from the one
+ * table `packages/spec` states.
+ *
+ * ## The defect
+ *
+ * One object, seven plain numeric declarations, three producers, driven into
+ * live PostgreSQL 16.13 and read back out of `information_schema.columns` with
+ * `numeric_precision` / `numeric_scale` — the half a bare `data_type` read
+ * hides:
+ *
+ * ```
+ *              driver    sql gen         ts gen
+ * number       real      numeric(18,2)   numeric(8,2)
+ * currency     real      numeric(18,2)   numeric(8,2)
+ * percent      real      numeric(5,2)    numeric(8,2)
+ * slider       real      numeric(18,2)   numeric(8,2)
+ * summary      real      numeric(18,2)   numeric(8,2)
+ * progress     real      numeric(5,2)    numeric(8,2)
+ * rating       real      integer         integer
+ * ```
+ *
+ * 7 of 7 diverged, and six of them THREE ways rather than the two the report
+ * named — `table.decimal(name)` with no arguments is knex's `decimal(8, 2)`, so
+ * the two halves of one command never agreed with each other either. A control
+ * family already unified (#16091: `text` / `email` / `boolean` / `date`) came
+ * back 0-of-4 divergent in the same run, so AGREE is a reading the instrument
+ * can produce. After the repair the same command reports 0 of 7.
+ *
+ * ## Why this pin reads the spec instead of asserting the numbers
+ *
+ * The same reason `generate-string-family-width.pin.test.ts` gives for the
+ * character widths: the whole shape of this card is "the producers disagree",
+ * so a pin that transcribed `DECIMAL(65,30)` would re-create the defect one
+ * layer up and stay green on the day the table moves. Every expectation here is
+ * derived from `numericColumnFor`, and the membership of the family from
+ * `NUMERIC_VALUE_TYPES` — neither is listed in this file.
+ *
+ * ## The oracle
+ *
+ * `SqlDriver.initObjects` on an in-memory better-sqlite3 database, read back
+ * with `PRAGMA table_info`, exactly as the #16091 pin does it: it runs the real
+ * `createColumn` chain and reports the column that actually exists. ⚠️ SQLite
+ * is where the oracle can run in a unit test, and SQLite applies no precision
+ * and no scale — `ColumnCompiler_SQLite3.prototype.decimal` is the literal
+ * `'float'`. So the oracle answers the question SQLite can answer (which ARM
+ * each type takes: the exact-decimal one or the integer one) and the PostgreSQL
+ * precision/scale claim is carried by the spec-side equality below plus the
+ * live run recorded in the PR. ⛔ Do not read the oracle as a precision check.
+ *
+ * ## The nullability half (ADR-0113, #16294 cause 1)
+ *
+ * `SqlDriver.createColumn` emits the physical NOT NULL from `storage.notNull`
+ * and deliberately not from `required` — its own arm records why: binding the
+ * DDL to `required` made every post-deploy tightening a destructive migration.
+ * Both generators were still on `required`. Driven on live PostgreSQL 16.13
+ * after the repair, four declaration shapes through all three producers:
+ * `required` alone is NULLABLE on all three, `storage.notNull` is NOT NULL on
+ * all three, 0 of 4 diverge, and the probe still distinguishes the two verdicts.
+ */
+
+import { afterAll, describe, expect, it } from 'vitest';
+
+import { SqlDriver } from '@objectstack/driver-sql';
+import {
+  NUMERIC_VALUE_TYPES,
+  numericColumnFor,
+  type NumericColumnRepresentation,
+} from '@objectstack/spec/data';
+
+import { generateMigrationSql, generateMigrationTs } from './generate.js';
+
+const NUMERIC_TYPES = [...NUMERIC_VALUE_TYPES].sort();
+
+/**
+ * ⭐ THE REAL CHAIN, widened exactly as `generate-string-family-width.pin.test`
+ * widens it: `protected` is a compile-time visibility rule, so a subclass can
+ * publish the driver's own `knex` without copying a character of its logic.
+ * `initObjects` dispatches every field through `createColumn` and hands the
+ * result to knex; `PRAGMA table_info` reports the column that then exists.
+ */
+class DriverOracle extends SqlDriver {
+  public async createdColumns(object: { name: string; fields?: Record<string, unknown> }): Promise<Map<string, string>> {
+    await this.initObjects([object as never]);
+    const rows = (await this.knex.raw(`PRAGMA table_info("${object.name}")`)) as Array<{
+      name: string;
+      type: string;
+    }>;
+    return new Map(rows.map((row) => [row.name, row.type]));
+  }
+}
+
+const ORACLE = new DriverOracle({
+  client: 'better-sqlite3',
+  connection: { filename: ':memory:' },
+  useNullAsDefault: true,
+});
+
+afterAll(async () => {
+  await ORACLE.disconnect();
+});
+
+/** The answer this file is allowed to expect — resolved, never transcribed. */
+function stated(type: string): NumericColumnRepresentation {
+  const answer = numericColumnFor(type);
+  if (!answer) throw new Error(`packages/spec states no column for the numeric type '${type}'`);
+  return answer;
+}
+
+function objectOf(types: readonly string[]): Record<string, unknown> {
+  const fields: Record<string, unknown> = {};
+  for (const t of types) fields[`f_${t}`] = { type: t };
+  return { name: 'num_zoo', fields };
+}
+
+describe('#16318 — both migration formats emit the stated numeric column', () => {
+  it('the family is non-empty and this file did not invent its membership', () => {
+    // Non-vacuity: every assertion below loops over NUMERIC_TYPES, so an empty
+    // set would pass every one of them while measuring nothing.
+    expect(NUMERIC_TYPES.length).toBeGreaterThanOrEqual(7);
+    expect(NUMERIC_TYPES).toContain('currency');
+    expect(NUMERIC_TYPES).toContain('rating');
+  });
+
+  it('the sql format emits the stated column for every member', () => {
+    const sql = generateMigrationSql({ objects: { num_zoo: objectOf(NUMERIC_TYPES) } });
+    for (const type of NUMERIC_TYPES) {
+      const answer = stated(type);
+      const expected =
+        answer.kind === 'integer' ? 'INTEGER' : `DECIMAL(${answer.precision},${answer.scale})`;
+      expect(sql, type).toContain(`"f_${type}" ${expected}`);
+    }
+    // The two shapes this card removed must be gone from the whole emission,
+    // not merely absent from the rows above.
+    expect(sql).not.toContain('DECIMAL(18,2)');
+    expect(sql).not.toContain('DECIMAL(5,2)');
+  });
+
+  it('the typescript format emits the stated column for every member', () => {
+    const ts = generateMigrationTs({ objects: { num_zoo: objectOf(NUMERIC_TYPES) } });
+    for (const type of NUMERIC_TYPES) {
+      const answer = stated(type);
+      const expected =
+        answer.kind === 'integer'
+          ? `table.integer('f_${type}')`
+          : `table.decimal('f_${type}', ${answer.precision}, ${answer.scale})`;
+      expect(ts, type).toContain(expected);
+    }
+    // Knex's argument-less `decimal` is `decimal(8, 2)` — the third answer in
+    // the divergence, and the one no reader of this file would guess.
+    expect(ts).not.toMatch(/table\.decimal\('f_[a-z]+'\)/);
+  });
+
+  it('the two formats agree with each other on every member', () => {
+    const sql = generateMigrationSql({ objects: { num_zoo: objectOf(NUMERIC_TYPES) } });
+    const ts = generateMigrationTs({ objects: { num_zoo: objectOf(NUMERIC_TYPES) } });
+    for (const type of NUMERIC_TYPES) {
+      const answer = stated(type);
+      if (answer.kind === 'integer') {
+        expect(sql, type).toContain(`"f_${type}" INTEGER`);
+        expect(ts, type).toContain(`table.integer('f_${type}')`);
+      } else {
+        expect(sql, type).toContain(`"f_${type}" DECIMAL(${answer.precision},${answer.scale})`);
+        expect(ts, type).toContain(`table.decimal('f_${type}', ${answer.precision}, ${answer.scale})`);
+      }
+    }
+  });
+
+  /**
+   * The ORACLE. Which ARM the driver puts each type in — the only half of the
+   * claim SQLite can carry, see the docblock.
+   */
+  it('the driver puts every member in the arm the spec states', async () => {
+    const byName = await ORACLE.createdColumns(objectOf(NUMERIC_TYPES) as never);
+
+    // Non-vacuity: the read must have found the columns at all.
+    for (const type of NUMERIC_TYPES) expect(byName.has(`f_${type}`), `f_${type} missing`).toBe(true);
+
+    for (const type of NUMERIC_TYPES) {
+      const answer = stated(type);
+      // knex compiles BOTH `decimal(p, s)` and `float` to the literal `float`
+      // on SQLite, which is exactly the measurement that makes this move
+      // affinity-neutral for the six exact-decimal members.
+      expect(byName.get(`f_${type}`)?.toLowerCase(), type).toBe(
+        answer.kind === 'integer' ? 'integer' : 'float',
+      );
+    }
+    // The discriminating control: the two arms must not have collapsed into
+    // one, or "the driver agrees" would be a constant rather than a reading.
+    expect(new Set(NUMERIC_TYPES.map((t) => byName.get(`f_${t}`)?.toLowerCase())).size).toBe(2);
+  });
+});
+
+describe('#16318 / ADR-0113 — both formats take NOT NULL from `storage.notNull`', () => {
+  const FIELDS = {
+    f_required_only: { type: 'text', required: true },
+    f_storage_only: { type: 'text', storage: { notNull: true } },
+    f_both: { type: 'text', required: true, storage: { notNull: true } },
+    f_neither: { type: 'text' },
+  };
+  const config = { objects: { nn: { name: 'nn', fields: FIELDS } } };
+
+  it('the sql format constrains exactly the columns the driver constrains', () => {
+    const sql = generateMigrationSql(config);
+    expect(sql).toContain('"f_storage_only" TEXT NOT NULL');
+    expect(sql).toContain('"f_both" TEXT NOT NULL');
+    // `required` alone is the WRITE-time contract; it must not reach the DDL.
+    expect(sql).toMatch(/"f_required_only" TEXT(?! NOT NULL)/);
+    expect(sql).toMatch(/"f_neither" TEXT(?! NOT NULL)/);
+  });
+
+  it('the typescript format constrains exactly the same columns', () => {
+    const ts = generateMigrationTs(config);
+    expect(ts).toContain(`table.text('f_storage_only').notNullable()`);
+    expect(ts).toContain(`table.text('f_both').notNullable()`);
+    expect(ts).toContain(`table.text('f_required_only').nullable()`);
+    expect(ts).toContain(`table.text('f_neither').nullable()`);
+  });
+
+  it('the probe distinguishes its two verdicts', () => {
+    // Without this the two tests above would pass against a generator that
+    // emitted NOT NULL for everything, or for nothing.
+    const ts = generateMigrationTs(config);
+    expect(ts).toContain('.notNullable()');
+    expect(ts).toContain('.nullable()');
+  });
+});