diff --git a/.changeset/17502-served-schema-drops-unauthorable-columns.md b/.changeset/17502-served-schema-drops-unauthorable-columns.md new file mode 100644 index 00000000000..b11d0946ab1 --- /dev/null +++ b/.changeset/17502-served-schema-drops-unauthorable-columns.md @@ -0,0 +1,63 @@ +--- +'@objectstack/metadata-protocol': minor +--- + +fix(metadata-protocol): `GET /meta/types` stops publishing properties no instance can satisfy (#17502) + +The served JSON Schema advertised the `retiredKey()` tombstones alongside the +live keys. `retiredKey()` keeps a removed authorable key declared on purpose — +the removal has to be audible — and `z.toJSONSchema` renders that tombstone as +a property node, `{ "description": "[REMOVED] ", "not": {} }`. + +`not: {}` is the JSON Schema spelling of "no instance validates", so a consumer +that reads the subschema is told the truth. A consumer that reads the KEY SET is +not: Studio builds a repeater's column headers from +`items.properties[k].title ?? k`, so a tombstone inside a row shape became a +column an author was invited to fill and `saveMetaItem` then refused. + +`toJsonSchemaSafe` now drops every property whose subschema admits no instance +before it serves or caches the document — structurally, by asking the JSON +Schema question, never by matching the `[REMOVED] ` description prefix, which +would put a second hand-written spelling of "this is a tombstone" in a consumer. +A property that admits nothing and is `required` is kept: dropping it would turn +"this object admits nothing" into "this object admits anything". + +Measured over the whole served registry at `74eaab8614`, this change's merge +base (`@objectstack/spec` SOURCE at 17.4.0, plus the retirements unreleased at +that sha — not the published release): 80 such nodes across 16 types — a +reading taken at that tree, not a standing invariant; it moves as retired keys +land or age out. + +**Nothing is un-retired, and no prescription CHANNEL is destroyed.** The removal is a +property of ONE emitter. `tsc` still types the key `never`, the parse still +refuses it with the prescription byte for byte, `packages/spec`'s +`authorable-surface/` ratchet still lists every retired key as `[RETIRED]`, and +the generated reference pages still print the full prescription in the +description column of a `never`-typed row. What this drops is a fourth copy, on +the one surface whose documented job is to describe what an author MAY write. + +**What an author stops being offered, stated as a class.** A tombstone became +visible wherever a renderer derives its field or column list from the served KEY +SET and reads the subschema for nothing but a label — so the retired key arrived +as an editable input, or as a repeater column, that the publish door then +refused. Three mechanisms put one in front of an author, and one retired key can +reach it through more than one of them: + +- **the flat, schema-driven fallback**, for a served type that carries no + `*.form.ts` layout: its field list *is* the served `properties` map, and a + nested object renders recursively, so a tombstone at any depth becomes a field + with the `[REMOVED] ` prescription as its help text; +- **repeater rows**, whose column headers are `items.properties[k].title ?? k` — + the carrier this card was filed on; +- **server-field grafting**, where an inspector merges the server's top-level + properties into a trailing "More fields" section: a key the UI's own bundled + spec predates is offered *because* the served document is the only place it is + known from. + +No count of the affected sites is given, on purpose. Which nodes reach an author +depends on the renderer and on the Console build this repo pins, so any number +written here would be false at the next pin bump. The invariant is the class: the +served document stops offering what the publish door refuses, and every retired +key keeps the full prescription on its generated reference page. A repeater +column loses no text either way — the row-cell renderer has no `description` +branch — so there the removal only withdraws the offer. diff --git a/packages/metadata-protocol/src/protocol.meta-types-degenerate-derivation.test.ts b/packages/metadata-protocol/src/protocol.meta-types-degenerate-derivation.test.ts index a7a30cb4a32..a9825130b55 100644 --- a/packages/metadata-protocol/src/protocol.meta-types-degenerate-derivation.test.ts +++ b/packages/metadata-protocol/src/protocol.meta-types-degenerate-derivation.test.ts @@ -42,6 +42,28 @@ * instrument agrees with the card wherever the card actually measured, and the * canonicalising comparison is what tells content apart from key ordering. * + * ## [#17502] Why the baseline is now STRIPPED before it is compared + * + * There are two declared reasons a served payload may differ from the raw + * derivation, and this suite owns exactly one of them. #17502 made + * `toJsonSchemaSafe` drop every property whose subschema admits no instance — + * a `retiredKey()` tombstone — so 15 of the served types legitimately differ + * from their raw derivation for a reason that has nothing to do with the + * degeneracy retry. Comparing against the raw document would make this pin red + * for that reason and blind to its own: a later blanket widening to + * `io: 'input'` would arrive inside an already-red assertion nobody could read. + * + * So the baseline has the SAME strip applied — through the emitter's own + * `stripUnauthorableProperties`, never a second spelling — and what remains on + * the two sides of the comparison is exactly the retry's blast radius. The + * assertion is unchanged in strength: widen the retry to every type and 24 + * types move, not one. + * + * The property-count controls keep the CARD's original numbers as their + * authority and add back what the strip removed, so the constant still fails + * when a live property appears or disappears, and the subtraction is derived + * rather than a second hand-maintained table. + * * Harness: the real `getMetaTypes()` on one protocol instance over a stub * engine, so the assertions are about what the endpoint SERVES. A pin taken on * a derivation chosen for convenience would not cover the served path at all — @@ -55,6 +77,10 @@ import { assertEngineDeleteDispatch, assertEngineUpdateDispatch, assertEngineFin import { DEFAULT_METADATA_TYPE_REGISTRY, getMetadataTypeSchema } from '@objectstack/spec/kernel'; import { METADATA_FORM_REGISTRY } from '@objectstack/spec/system'; import { ObjectStackProtocolImplementation } from './protocol.js'; +// [#17502] The emitter's OWN strip and its predicate — the baseline below is +// stripped with the same code the server runs, so this pin can never drift +// into measuring a second, hand-written idea of "admits nothing". +import { acceptsNothing, stripUnauthorableProperties } from './unauthorable-nodes.js'; /** * The whole served surface: every declared metadata type plus every @@ -109,6 +135,41 @@ function preFixDerivation(type: string): Record | undefined { } } +/** + * [#17502] The pre-fix derivation with this card's strip applied — the baseline + * the blast-radius pin compares against, so the only difference left to find is + * the degeneracy retry's. + */ +function preFixServedBaseline(type: string): Record | undefined { + return stripUnauthorableProperties(preFixDerivation(type)); +} + +/** + * [#17502] How many TOP-LEVEL properties the strip removes from this type's + * served document. + * + * Counted on whichever derivation the server can actually use: `action` has no + * properties at all on the default arm, so its three tombstones are visible + * only on the `io: 'input'` retry that #17501 gave it. + */ +function retiredTopLevelCount(type: string): number { + const schema = getMetadataTypeSchema(type); + if (!schema) return 0; + for (const io of ['output', 'input'] as const) { + let json: Record; + try { + json = z.toJSONSchema(schema as z.ZodTypeAny, { unrepresentable: 'any', io }) as Record; + } catch { + continue; + } + const properties = json.properties as Record | undefined; + if (properties && Object.keys(properties).length > 0) { + return Object.values(properties).filter(acceptsNothing).length; + } + } + return 0; +} + /** * Recursive key sort. Two documents that differ only in key ORDER canonicalise * to the same string; anything still different after this is real content. @@ -147,7 +208,12 @@ describe('#17501 — /meta/types serves a real schema for `action`, and moves no const properties = served!.properties as Record; expect(properties, '`action` must name its properties').toBeDefined(); - expect(Object.keys(properties).length).toBe(48); + // [#17502] 48 is the key set `action` DECLARES — 45 accepted plus the + // three that admit no instance and are therefore refused — and that + // declared total stays the pinned authority. The served document no + // longer carries those three, so they are added back rather than the + // constant being lowered — a live key going missing is still red. + expect(Object.keys(properties).length + retiredTopLevelCount('action')).toBe(48); // A sample an author would actually address, and the one #17500's // repeater titles need a node to sit on. for (const key of ['name', 'label', 'objectName', 'type', 'params', 'locations']) { @@ -161,7 +227,7 @@ describe('#17501 — /meta/types serves a real schema for `action`, and moves no const moved: string[] = []; for (const type of SERVED_TYPES) { if (!getMetadataTypeSchema(type)) continue; // absence is not degeneracy — see below - const before = preFixDerivation(type); + const before = preFixServedBaseline(type); const after = served.get(type); if (canon(before) !== canon(after)) moved.push(type); } @@ -178,7 +244,7 @@ describe('#17501 — /meta/types serves a real schema for `action`, and moves no for (const type of SERVED_TYPES) { if (type === 'action' || !getMetadataTypeSchema(type)) continue; - const before = preFixDerivation(type); + const before = preFixServedBaseline(type); const after = served.get(type); // Raw equality first: these must not move at all. expect(JSON.stringify(after), `${type} served payload moved`).toBe(JSON.stringify(before)); @@ -212,7 +278,13 @@ describe('#17501 — /meta/types serves a real schema for `action`, and moves no async (type, count) => { const served = (await servedSchemas()).get(type as string); expect(served, `${type} must be served`).toBeDefined(); - expect(Object.keys(served!.properties as Record).length).toBe(count); + // [#17502] The card's count is the authority; what the strip + // removed is added back, derived, so this stays a control over + // LIVE properties rather than a number quietly rewritten. + expect( + Object.keys(served!.properties as Record).length + + retiredTopLevelCount(type as string), + ).toBe(count); }, ); diff --git a/packages/metadata-protocol/src/protocol.meta-types-unauthorable-columns.test.ts b/packages/metadata-protocol/src/protocol.meta-types-unauthorable-columns.test.ts new file mode 100644 index 00000000000..82896e1c883 --- /dev/null +++ b/packages/metadata-protocol/src/protocol.meta-types-unauthorable-columns.test.ts @@ -0,0 +1,346 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * [#17502] `GET /meta/types` must not offer a column the publish door refuses. + * + * ## The defect, measured on the SERVED payload + * + * `retiredKey()` keeps a removed authorable key declared so the retirement is + * audible, and `z.toJSONSchema` renders that tombstone as a property node: + * + * { "description": "[REMOVED] ", "not": {} } + * + * `not: {}` says "no instance validates", so a consumer reading the SUBSCHEMA + * is told the truth. Studio's repeater table does not read the subschema — it + * builds its column headers from `items.properties[k].title ?? k` — so every + * tombstone in a row shape became a column an author is invited to fill and + * `saveMetaItem` then refuses. + * + * Measured on `origin/main` at 74eaab8614 over the whole served registry: + * **80 tombstone nodes across 16 types**. Five of them are `dashboard.widgets[]`'s + * `actionUrl`, `actionType`, `actionIcon`, `responsive` and `aria` — the row + * this file pins, and the carrier the card was filed on. ⚠️ They are not the + * whole reachable set: a repeater row in another type reaches an author the + * same way, as does the flat schema-driven fallback for a layout-less type and + * an inspector that grafts server-only properties into a "More fields" section. + * How many sites there are at any moment is a function of the renderer and of + * the pinned Console build, so no count of them is pinned here — the class + * guard below is over the whole registry instead. + * + * ⚠️ The card's headline carrier, `flow.nodes[].outputSchema`, is NOT on the + * served path: `flow` takes the output derivation, where `nodes.items` carries + * no properties at all. It is visible only in the `io: 'input'` derivation that + * `packages/spec`'s `repeater-item-titles.test.ts` takes deliberately. The + * empty served `flow.nodes` row is a separate defect and is not this pin's. + * + * ## What this pin asserts, and why each half is here + * + * The verdict is structural, never the `[REMOVED] ` description prefix: a + * prefix match would be a second hand-written spelling of "this is a + * tombstone" living in a consumer, which is the shape this card removes. + * + * Both controls matter. The DARK half (the five columns are gone) passes + * vacuously if the harness never reached the row, so the LIT half pins the + * seventeen live columns that must survive beside them, and a third control + * re-derives the pre-strip payload in-process and requires the nodes to be + * there — which is what makes this file fail on `origin/main` today rather + * than describe a payload nobody produced. + * + * Harness: the real `getMetaTypes()` on one protocol instance over a stub + * engine — the same shape `protocol.meta-types-degenerate-derivation.test.ts` + * uses, so the assertions are about what the endpoint SERVES and not about a + * derivation picked for convenience. + */ +import { describe, expect, it } from 'vitest'; +import { z } from 'zod'; +// [#5619] The producer's OWN write-verb dispatch decisions, so the fake engine +// below cannot accept a call ObjectQL itself refuses. +import { assertEngineDeleteDispatch, assertEngineUpdateDispatch, assertEngineFindOnePredicate, type EngineFindOneQueryInput } from '@objectstack/metadata-core'; +import { DEFAULT_METADATA_TYPE_REGISTRY, getMetadataTypeSchema } from '@objectstack/spec/kernel'; +import { METADATA_FORM_REGISTRY } from '@objectstack/spec/system'; +import { ObjectStackProtocolImplementation } from './protocol.js'; +import { acceptsNothing } from './unauthorable-nodes.js'; + +const SERVED_TYPES = Array.from(new Set([ + ...DEFAULT_METADATA_TYPE_REGISTRY.map((e) => e.type), + ...Object.keys(METADATA_FORM_REGISTRY), +])).sort(); + +function makeProtocol() { + const engine: any = { + async findOne(object: string, query?: EngineFindOneQueryInput) { + assertEngineFindOnePredicate(object, query); return null; + }, + async find() { return []; }, + async insert() { return { id: 'unused' }; }, + async update(_t: string, data: Record, opts?: Record) { + assertEngineUpdateDispatch(data, opts); + return { id: null }; + }, + async delete(_t: string, opts?: Record) { + assertEngineDeleteDispatch(opts); + return { deleted: 1 }; + }, + async count() { return 0; }, + async transaction(fn: (ctx: unknown) => Promise) { return fn(undefined); }, + async execute() { return {}; }, + async getObjectSchema() { return undefined; }, + registry: { + getRegisteredTypes: () => [...SERVED_TYPES], + registerItem: () => {}, + registerObject: () => {}, + unregisterItem: () => {}, + listItems: () => [], + getItem: () => undefined, + getArtifactItem: () => undefined, + }, + }; + return new ObjectStackProtocolImplementation(engine, () => new Map(), undefined) as any; +} + +async function servedSchemas(): Promise | undefined>> { + const listing = await makeProtocol().getMetaTypes(); + const map = new Map | undefined>(); + for (const entry of listing.entries as Array<{ type: string; schema?: Record }>) { + map.set(entry.type, entry.schema); + } + return map; +} + +/** The derivation the endpoint ran BEFORE this card's strip stage. */ +function preStripDerivation(type: string, io: 'output' | 'input' = 'output'): Record | undefined { + const schema = getMetadataTypeSchema(type); + if (!schema) return undefined; + try { + return z.toJSONSchema(schema as z.ZodTypeAny, { unrepresentable: 'any', io }) as Record; + } catch { + return undefined; + } +} + +/** + * What the strip DID to one document, read off the two documents by a parallel + * walk rather than by re-running the strip: `removed` is every key the + * derivation has and the served payload does not, with the node that was + * dropped; `other` is everything else that moved — an addition, a changed + * value, a changed array length. + * + * ⚠️ Deliberately NOT a second implementation of the strip. It asks only + * "what moved"; the assertion supplies the verdict, so a defect in the strip + * cannot appear on both sides of the comparison and cancel itself out. + */ +function strippedDiff( + before: unknown, + after: unknown, +): { removed: Array<{ path: string; node: unknown }>; other: string[] } { + const removed: Array<{ path: string; node: unknown }> = []; + const other: string[] = []; + const visit = (b: unknown, a: unknown, p: string): void => { + if (Array.isArray(b) || Array.isArray(a)) { + if (!Array.isArray(b) || !Array.isArray(a) || b.length !== a.length) { other.push(p); return; } + b.forEach((entry, i) => visit(entry, a[i], `${p}[${i}]`)); + return; + } + if (b && typeof b === 'object') { + if (!a || typeof a !== 'object') { other.push(p); return; } + const bo = b as Record; + const ao = a as Record; + for (const [key, value] of Object.entries(bo)) { + if (!(key in ao)) { removed.push({ path: `${p}.${key}`, node: value }); continue; } + visit(value, ao[key], `${p}.${key}`); + } + for (const key of Object.keys(ao)) if (!(key in bo)) other.push(`${p}.${key}`); + return; + } + if (b !== a) other.push(p); + }; + visit(before, after, '$'); + return { removed, other }; +} + +/** + * Every `` in a JSON Schema document whose subschema admits no instance. + * Walks the document generically so a tombstone that moves house — into a + * `$defs` entry, a union arm, a deeper row — is still found. + */ +function unsatisfiablePaths(node: unknown, path = '$'): string[] { + if (Array.isArray(node)) return node.flatMap((n, i) => unsatisfiablePaths(n, `${path}[${i}]`)); + if (!node || typeof node !== 'object') return []; + const out: string[] = []; + for (const [key, value] of Object.entries(node as Record)) { + if (key === 'default' || key === 'const' || key === 'enum' || key === 'examples') continue; + if (key === 'properties' && value && typeof value === 'object') { + for (const [prop, sub] of Object.entries(value as Record)) { + if (acceptsNothing(sub)) out.push(`${path}.properties.${prop}`); + } + } + out.push(...unsatisfiablePaths(value, `${path}.${key}`)); + } + return out; +} + +/** `dashboard.widgets[]`'s row shape, off the served document. */ +function widgetRow(served: Record | undefined): Record { + const widgets = (served?.properties as any)?.widgets; + const items = widgets?.items; + const resolved = typeof items?.$ref === 'string' + ? (served as any).$defs?.[String(items.$ref).replace('#/$defs/', '')] + : items; + return (resolved?.properties ?? {}) as Record; +} + +/** The five columns the parse door refuses (`ui/dashboard.zod.ts` tombstones). */ +const RETIRED_WIDGET_COLUMNS = ['actionUrl', 'actionType', 'actionIcon', 'responsive', 'aria'] as const; + +/** The live columns that must survive beside them — the lit control. */ +const LIVE_WIDGET_COLUMNS = [ + 'chartConfig', 'colorVariant', 'compareTo', 'dataset', 'description', 'dimensions', + 'filter', 'filterBindings', 'id', 'layout', 'options', 'requiresObject', + 'requiresService', 'suppressWarnings', 'title', 'type', 'values', +] as const; + +describe('#17502 — the served repeater row offers no column the parse door refuses', () => { + it('control: the pre-strip derivation really did carry the five tombstone columns', () => { + // Without this, both halves below could pass over a payload that never + // had the nodes — and the pin would be green on `origin/main` too. + const row = widgetRow(preStripDerivation('dashboard')); + for (const key of RETIRED_WIDGET_COLUMNS) { + expect(Object.keys(row), `pre-strip dashboard.widgets row declares ${key}`).toContain(key); + expect(acceptsNothing((row as any)[key]), `${key} is a node that admits nothing`).toBe(true); + expect(String((row as any)[key].description)).toMatch(/^\[REMOVED\] /); + } + expect(Object.keys(row).length).toBe(22); + }); + + it('lit: the served `dashboard.widgets` row still carries every live column', async () => { + const row = widgetRow((await servedSchemas()).get('dashboard')); + expect(Object.keys(row).sort()).toEqual([...LIVE_WIDGET_COLUMNS].sort()); + }); + + it('dark: the five retired columns are gone from the served row', async () => { + const row = widgetRow((await servedSchemas()).get('dashboard')); + for (const key of RETIRED_WIDGET_COLUMNS) { + expect(Object.keys(row), `dashboard.widgets must not offer ${key}`).not.toContain(key); + } + }); + + it('class guard: no served type publishes a property that admits no instance', async () => { + const served = await servedSchemas(); + const offenders: string[] = []; + for (const type of SERVED_TYPES) { + const schema = served.get(type); + if (!schema) continue; + offenders.push(...unsatisfiablePaths(schema, type)); + } + // ⛔ An entry here is a key the endpoint advertises and the publish door + // refuses — file it, never add it to a list. + expect(offenders).toEqual([]); + }); + + it('over-drop guard: the served payload is its derivation MINUS unsatisfiable nodes, nothing else', async () => { + // ⚠️ This is the direction the blast-radius pin in + // `protocol.meta-types-degenerate-derivation.test.ts` CANNOT see. Since + // #17502 its baseline is `stripUnauthorableProperties(preFixDerivation(type))`, + // so a strip that drops too much drops it on BOTH sides of that + // comparison and stays invisible — only `dashboard.widgets`'s lit + // columns and the CARD types' TOP-level counts guard over-dropping + // there. This pin reads the removals themselves, at every depth, for + // every served type. The one question this pin asks: did that node + // admit any instance? + const served = await servedSchemas(); + const unexplained: string[] = []; + const overDropped: string[] = []; + const removedByType = new Map(); + + for (const type of SERVED_TYPES) { + const after = served.get(type); + if (!after) continue; + // The endpoint derives on zod's default arm and retries `io: 'input'` + // only when the default one is degenerate (#17501). Take whichever + // arm the served document is a pure DELETION of, so this pin never + // re-spells `isDegenerateDerivation`, whose only copy belongs in + // the emitter. + const diff = (['output', 'input'] as const) + .map((io) => preStripDerivation(type, io)) + .filter((d): d is Record => Boolean(d)) + .map((before) => strippedDiff(before, after)) + .find((d) => d.other.length === 0); + if (!diff) { unexplained.push(type); continue; } + removedByType.set(type, diff.removed.map((r) => `${type}${r.path.slice(1)}`)); + for (const r of diff.removed) { + if (!acceptsNothing(r.node)) overDropped.push(`${type}${r.path.slice(1)}`); + } + } + + // ⛔ The strip only ever takes keys AWAY. An entry here means the served + // payload is no longer either derivation minus something. + expect(unexplained).toEqual([]); + // ⛔ An entry here is a LIVE node the endpoint stopped serving — the + // over-drop defect. File it, never add it to a list. + expect(overDropped).toEqual([]); + + // Non-vacuity: without this the two assertions above pass over a ledger + // that read nothing at all. `dashboard` is the type that exercises both + // depths — the five repeater-row columns pinned above, and three + // top-level tombstones — so the ledger is proven to reach a row shape + // and not only the surface. Sorted, so key ORDER is not what is pinned. + expect([...(removedByType.get('dashboard') ?? [])].sort()).toEqual([ + ...RETIRED_WIDGET_COLUMNS.map((k) => `dashboard.properties.widgets.items.properties.${k}`), + 'dashboard.properties.refreshInterval', + 'dashboard.properties.aria', + 'dashboard.properties.performance', + // [#17751, arrived with main] `ChartConfigSchema.aria` retired one + // level deeper than the widget row, inside `chartConfig`. + 'dashboard.properties.widgets.items.properties.chartConfig.properties.aria', + ].sort()); + }); + + // ⚠️ This control derives with zod's DEFAULT (output) arm only, which is + // 77 nodes across 15 types. The served payload carries 80 across 16: for + // `action` alone `toJsonSchemaSafe` falls through to the `io: 'input'` + // retry (#17501), and that arm adds `execute` / `shortcut` / `bulkEnabled`. + // The class guard above runs over the SERVED document and covers all 80; + // this control deliberately does not re-spell `isDegenerateDerivation`, + // whose only copy belongs in the emitter. + it('control: the class really is non-empty before the strip — 77 nodes across 15 types on the output arm', () => { + const byType = new Map(); + for (const type of SERVED_TYPES) { + const before = preStripDerivation(type); + if (!before) continue; + const n = unsatisfiablePaths(before, type).length; + if (n > 0) byType.set(type, n); + } + const total = [...byType.values()].reduce((a, b) => a + b, 0); + expect(total).toBeGreaterThan(0); + expect(byType.has('dashboard')).toBe(true); + }); +}); + +describe('#17502 — the removal is payload-only: every prescription channel survives', () => { + it('the parse still refuses the key with the tombstone prescription, byte for byte', async () => { + const dashboard = getMetadataTypeSchema('dashboard') as z.ZodTypeAny; + const result = dashboard.safeParse({ + name: 'ops', label: 'Ops', + widgets: [{ id: 'w1', type: 'metric', actionUrl: '/x' }], + } as never); + expect(result.success, 'a retired widget column is still refused at publish').toBe(false); + const issue = result.error!.issues.find((i) => i.path[i.path.length - 1] === 'actionUrl'); + expect(issue, 'the refusal names the retired key').toBeDefined(); + expect((issue as { expected?: string }).expected).toBe('never'); + // The prescription — the FROM -> TO mapping this retirement exists to + // deliver — is carried by the refusal, which the strip never touches. + expect(issue!.message).toContain('was removed in @objectstack/spec 17.0.0'); + expect(issue!.message).toContain('header: { actions:'); + expect(issue!.message).toContain('os migrate meta --from 16'); + }); + + it('the Zod shape still declares the tombstone — nothing is un-retired upstream', () => { + // `packages/spec`'s `authorable-surface/` ratchet and the generated + // reference pages read this shape, not the served payload, so both keep + // publishing the retirement. The strip is a property of ONE emitter. + const row = widgetRow(preStripDerivation('dashboard')); + for (const key of RETIRED_WIDGET_COLUMNS) { + expect(Object.keys(row)).toContain(key); + } + }); +}); diff --git a/packages/metadata-protocol/src/protocol.ts b/packages/metadata-protocol/src/protocol.ts index ddf3d433d06..2a53643c381 100644 --- a/packages/metadata-protocol/src/protocol.ts +++ b/packages/metadata-protocol/src/protocol.ts @@ -15,6 +15,11 @@ import { postureEnforcesWall } from '@objectstack/spec/security'; import { resolveDiscoveryVersion } from './discovery-version.js'; import type { MetadataHostEngine } from './host-engine.js'; import { omitInternalFieldsFromWriteResponse } from './write-response-internal-fields.js'; +// [#17502] The served JSON Schema publishes what an author MAY write, so a +// property no instance can satisfy — a `retiredKey()` tombstone, rendered +// `{ not: {} }` — is dropped from it. See the module header for the channels +// that keep carrying the retirement's prescription. +import { stripUnauthorableProperties } from './unauthorable-nodes.js'; import { evaluateRuntimeAuthoringGate, CLOSURE_CONTEXT_KEY_BY_TYPE, @@ -476,8 +481,9 @@ function toJsonSchemaSafe(schema: z.ZodTypeAny, typeLabel?: string): Record; if (!isDegenerateDerivation(authoring)) { - _jsonSchemaCache.set(schema, authoring); - return authoring; + const authorable = stripUnauthorableProperties(authoring); + _jsonSchemaCache.set(schema, authorable); + return authorable; } } catch { // Fall through to the loud arm below. diff --git a/packages/metadata-protocol/src/unauthorable-nodes.test.ts b/packages/metadata-protocol/src/unauthorable-nodes.test.ts new file mode 100644 index 00000000000..f6a6cf34ffb --- /dev/null +++ b/packages/metadata-protocol/src/unauthorable-nodes.test.ts @@ -0,0 +1,118 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * [#17502] Unit pins for the strip stage itself. The served-payload assertions + * live in `protocol.meta-types-unauthorable-columns.test.ts`; this file covers + * the two behaviours that have no live carrier today and would therefore never + * be exercised by the registry sweep — the `required` guard and copy-on-write. + */ +import { describe, expect, it } from 'vitest'; +import { z } from 'zod'; +import { acceptsNothing, stripUnauthorableProperties } from './unauthorable-nodes.js'; + +const NEVER = { description: '[REMOVED] `x` was removed. Delete the key.', not: {} }; + +describe('acceptsNothing', () => { + it('recognises the `{ not: {} }` node zod emits for `z.never()`, in both derivations', () => { + const shape = z.object({ live: z.string(), dead: z.never().optional().describe('[REMOVED] gone') }); + for (const io of ['output', 'input'] as const) { + const json = z.toJSONSchema(shape, { unrepresentable: 'any', io }) as any; + expect(acceptsNothing(json.properties.dead), `io=${io}`).toBe(true); + expect(acceptsNothing(json.properties.live), `io=${io}`).toBe(false); + } + }); + + it('is not fooled by a NON-empty `not`, which still admits instances', () => { + expect(acceptsNothing({ not: { type: 'string' } })).toBe(false); + expect(acceptsNothing({ not: [] })).toBe(false); + expect(acceptsNothing({ not: null })).toBe(false); + expect(acceptsNothing(undefined)).toBe(false); + }); +}); + +describe('stripUnauthorableProperties', () => { + it('drops an optional unsatisfiable property at every depth, rows and $defs included', () => { + const out: any = stripUnauthorableProperties({ + type: 'object', + properties: { + top: NEVER, + live: { type: 'string' }, + rows: { type: 'array', items: { type: 'object', properties: { col: NEVER, keep: { type: 'number' } } } }, + map: { type: 'object', additionalProperties: { type: 'object', properties: { inner: NEVER } } }, + }, + $defs: { Shared: { type: 'object', properties: { held: NEVER, kept: { type: 'boolean' } } } }, + }); + expect(Object.keys(out.properties)).toEqual(['live', 'rows', 'map']); + expect(Object.keys(out.properties.rows.items.properties)).toEqual(['keep']); + expect(Object.keys(out.properties.map.additionalProperties.properties)).toEqual([]); + expect(Object.keys(out.$defs.Shared.properties)).toEqual(['kept']); + }); + + it('KEEPS an unsatisfiable property that is `required` — dropping it would widen the shape', () => { + // `{ not: {} }` + required === the object admits nothing. Removing the + // key would turn that into "admits anything", a real widening. No + // `retiredKey()` is ever required (it is `.optional()`), so this guard + // exists for whatever else may derive to the same node. + const input = { type: 'object', required: ['dead'], properties: { dead: NEVER, live: { type: 'string' } } }; + const out: any = stripUnauthorableProperties(input); + expect(Object.keys(out.properties)).toEqual(['dead', 'live']); + expect(out).toBe(input); // nothing to drop ⇒ returned by reference + }); + + it('is copy-on-write: a document with nothing to drop comes back by reference', () => { + const input = { type: 'object', properties: { a: { type: 'string' } }, $defs: { B: { type: 'number' } } }; + expect(stripUnauthorableProperties(input)).toBe(input); + }); + + it('is POSITION-aware: a property literally NAMED `properties` is not a properties map', () => { + // A `properties` / `$defs` value is a map of author-chosen NAMES, not a + // schema node. A walk that reads the map as a node reads the keywords of + // the property named `properties` as property subschemas — and deletes + // any one valued `{ not: {} }`. Both inputs below are pure zod. + const record: any = z.toJSONSchema(z.object({ properties: z.record(z.string(), z.never()) }), { unrepresentable: 'any' }); + expect(record.properties.properties.additionalProperties, 'precondition').toEqual({ not: {} }); + const strippedRecord: any = stripUnauthorableProperties(record); + // `additionalProperties: { not: {} }` is what makes this node admit ONLY + // `{}`. Dropping it lets any object through — a widening of a live node. + expect(strippedRecord.properties.properties.additionalProperties).toEqual({ not: {} }); + expect(strippedRecord).toBe(record); // nothing to drop ⇒ by reference + + const list: any = z.toJSONSchema(z.object({ properties: z.array(z.never()) }), { unrepresentable: 'any' }); + expect(list.properties.properties.items, 'precondition').toEqual({ not: {} }); + const strippedList: any = stripUnauthorableProperties(list); + // `items: { not: {} }` is what makes this node admit ONLY `[]`. + expect(strippedList.properties.properties.items).toEqual({ not: {} }); + expect(strippedList).toBe(list); + }); + + it('is POSITION-aware in `$defs` too, where the entry names are just as free', () => { + const input = { $defs: { properties: { type: 'object', additionalProperties: { not: {} } } } }; + const out: any = stripUnauthorableProperties(input); + expect(out.$defs.properties.additionalProperties).toEqual({ not: {} }); + expect(out).toBe(input); + }); + + it('still strips inside a property whose NAME collides with a data-valued keyword', () => { + // The mirror of the two above: the map's VALUES are schema nodes + // whatever they are called, so `required` and `default` as property + // NAMES must not buy their subtrees an exemption from the walk. + const input = { + type: 'object', + properties: { + required: { type: 'object', properties: { dead: NEVER, live: { type: 'string' } } }, + default: { type: 'object', properties: { dead: NEVER } }, + }, + }; + const out: any = stripUnauthorableProperties(input); + expect(Object.keys(out.properties.required.properties)).toEqual(['live']); + expect(Object.keys(out.properties.default.properties)).toEqual([]); + }); + + it('never rewrites DATA-valued keywords that merely look like a schema', () => { + // `default` carries an author's value, not a subschema. A walk that + // treats it as one silently edits served defaults. + const input = { type: 'object', default: { properties: { dead: { not: {} } } }, properties: { live: { type: 'string' } } }; + const out: any = stripUnauthorableProperties(input); + expect(out.default).toEqual({ properties: { dead: { not: {} } } }); + }); +}); diff --git a/packages/metadata-protocol/src/unauthorable-nodes.ts b/packages/metadata-protocol/src/unauthorable-nodes.ts new file mode 100644 index 00000000000..9ec8cfe07c7 --- /dev/null +++ b/packages/metadata-protocol/src/unauthorable-nodes.ts @@ -0,0 +1,188 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * [#17502] Drop the properties a served JSON Schema publishes but no instance + * can satisfy. + * + * ## What lands in the payload, and why it reads as an offer + * + * `retiredKey()` (`@objectstack/spec` `shared/retired-key.ts`) declares a + * REMOVED authorable key as `z.never({ error: () => guidance }).optional() + * .describe('[REMOVED] ' + guidance)`. The key stays declared on purpose — the + * retirement has to be audible, and the two channels it names are `tsc` (the + * input type is `never`) and the parse (the refusal carries the FROM -> TO + * prescription instead of a bare "unrecognized key"). + * + * `z.toJSONSchema` renders that tombstone as a property node, measured here as + * + * { "description": "[REMOVED] ", "not": {} } + * + * `not: {}` is the JSON Schema spelling of "no instance validates", so a + * consumer that reads the SUBSCHEMA sees the refusal. A consumer that reads the + * KEY SET does not: Studio builds a repeater's column headers from + * `items.properties[k].title ?? k`, so every tombstone in a row shape becomes a + * column an author is invited to fill and the publish door then refuses. That + * is the offer-vs-door defect, and the payload is where it is cheapest to + * close — one emission point instead of one accommodation per renderer. + * + * ## Why the prescription is not lost with the node + * + * The removal keeps every channel that carries the prescription today: `tsc` + * and the parse are properties of the Zod shape and are untouched here; + * `packages/spec`'s `authorable-surface/` ratchet still lists each retired key + * as `[RETIRED]`; and the generated reference pages still print the full + * prescription in the description column of a `never`-typed row (see + * `content/docs/references/ui/dashboard.mdx`). What this drops is a fourth + * copy, on the one surface whose documented job is to describe what an author + * MAY write. + * + * ## The predicate is structural, never the `[REMOVED] ` prefix + * + * Matching the description prefix would put a second, hand-written spelling of + * "this is a tombstone" in a consumer — the very shape this card exists to + * remove. `acceptsNothing()` asks the JSON Schema question instead: does this + * subschema admit any instance at all? Anything that answers "no" is not part + * of an authorable surface, whatever produced it. + * + * ## The one thing it must not do + * + * A property that accepts nothing and is REQUIRED makes its object + * uninhabitable. Dropping such a key would turn "nothing validates" into + * "anything validates" — a real widening, and a lie of exactly the kind + * Route & surface ownership rule 4 forbids. So a key named in the parent's + * `required` array is kept, unsatisfiable and all. `retiredKey()` is + * `.optional()`, so no tombstone is ever in that arm; the guard is for + * whatever else may one day derive to `{ not: {} }`. + * + * ## Which is why the walk is POSITION-aware + * + * The drop decision is legal in exactly one position: an entry of a schema + * node's own `properties` map, where the sibling `required` array is in scope + * to veto it. Everywhere else a `{ not: {} }` is load-bearing — it is what + * `additionalProperties`, `items`, `propertyNames` or `patternProperties` use + * to say "and nothing more" — and removing it widens the node. + * + * So a `properties` / `$defs` / `patternProperties` / `dependentSchemas` value + * is walked as a MAP, never as a schema node: its keys are author-chosen NAMES, + * not keywords. Reading such a map as a node is how a property literally named + * `properties` gets its keywords treated as property subschemas — + * `z.object({ properties: z.record(z.string(), z.never()) })` then loses the + * `additionalProperties: { not: {} }` that made it admit only `{}` — and it is + * also how a property named `required` or `default` buys its whole subtree an + * exemption from the walk. Both directions are pinned in + * `unauthorable-nodes.test.ts`. + */ + +/** JSON Schema keywords whose values are DATA, not subschemas — never walked. */ +const NON_SCHEMA_KEYS: ReadonlySet = new Set([ + 'default', 'const', 'enum', 'examples', 'title', 'description', + '$schema', '$id', '$comment', 'required', +]); + +/** + * JSON Schema keywords whose value is a MAP of author-chosen NAME -> subschema. + * The map is not a schema node; every VALUE in it is. Nothing is ever dropped + * from one of these — `patternProperties` and `$defs` have no `required` array + * that could license a drop, and a `$defs` entry may be the target of a `$ref`. + */ +const SCHEMA_MAP_KEYS: ReadonlySet = new Set([ + 'properties', 'patternProperties', 'dependentSchemas', '$defs', 'definitions', +]); + +/** + * Does this subschema admit no instance at all? + * + * `{ "not": {} }` is the canonical spelling — `{}` accepts everything, so its + * negation accepts nothing — and it is what `z.toJSONSchema` emits for + * `z.never()` in both the output and the authoring derivation. + */ +export function acceptsNothing(node: unknown): boolean { + if (!node || typeof node !== 'object' || Array.isArray(node)) return false; + const not = (node as Record).not; + return ( + typeof not === 'object' + && not !== null + && !Array.isArray(not) + && Object.keys(not).length === 0 + ); +} + +/** + * Return `json` with every unsatisfiable, non-required property removed, at + * every depth. Pure and copy-on-write: a document with nothing to drop is + * returned by reference, so an untouched type's served payload stays + * byte-identical (and reference-identical) to its derivation. + */ +export function stripUnauthorableProperties(json: T): T { + return walkSchema(json) as T; +} + +/** + * Walk a SCHEMA node — the only position in which a property may be dropped, + * because it is the only position where the deciding `required` array is a + * sibling. + */ +function walkSchema(node: unknown): unknown { + if (Array.isArray(node)) { + // `allOf` / `anyOf` / `oneOf` / `prefixItems`: every entry is a schema. + let changed = false; + const out = node.map((entry) => { + const next = walkSchema(entry); + if (next !== entry) changed = true; + return next; + }); + return changed ? out : node; + } + if (!node || typeof node !== 'object') return node; + + const source = node as Record; + let out: Record | undefined; + const write = (key: string, value: unknown) => { + out ??= { ...source }; + out[key] = value; + }; + + const properties = source.properties; + if (properties && typeof properties === 'object' && !Array.isArray(properties)) { + const required = new Set( + Array.isArray(source.required) ? source.required.filter((k): k is string => typeof k === 'string') : [], + ); + let kept: Record | undefined; + for (const [key, value] of Object.entries(properties as Record)) { + if (acceptsNothing(value) && !required.has(key)) { + kept ??= { ...(properties as Record) }; + delete kept[key]; + } + } + if (kept) write('properties', kept); + } + + for (const [key, value] of Object.entries(source)) { + if (NON_SCHEMA_KEYS.has(key)) continue; + // `properties` may already have been pruned above; recurse into that. + const current = out ? out[key] : value; + const next = SCHEMA_MAP_KEYS.has(key) ? walkSchemaMap(current) : walkSchema(current); + if (next !== current) write(key, next); + } + + return out ?? node; +} + +/** + * Walk a MAP of NAME -> schema. The map itself is never read as a schema node, + * so no keyword logic applies to its keys and nothing is dropped here; each + * value is handed back to `walkSchema`, whatever it happens to be called. + */ +function walkSchemaMap(node: unknown): unknown { + if (!node || typeof node !== 'object' || Array.isArray(node)) return node; + const source = node as Record; + let out: Record | undefined; + for (const [key, value] of Object.entries(source)) { + const next = walkSchema(value); + if (next !== value) { + out ??= { ...source }; + out[key] = next; + } + } + return out ?? node; +} diff --git a/scripts/engine-double-contract.pinned.json b/scripts/engine-double-contract.pinned.json index 02d21027f58..a0f21d79569 100644 --- a/scripts/engine-double-contract.pinned.json +++ b/scripts/engine-double-contract.pinned.json @@ -806,6 +806,21 @@ "verb": "update", "pinned": 1 }, + { + "file": "packages/metadata-protocol/src/protocol.meta-types-unauthorable-columns.test.ts", + "verb": "delete", + "pinned": 1 + }, + { + "file": "packages/metadata-protocol/src/protocol.meta-types-unauthorable-columns.test.ts", + "verb": "findOne", + "pinned": 1 + }, + { + "file": "packages/metadata-protocol/src/protocol.meta-types-unauthorable-columns.test.ts", + "verb": "update", + "pinned": 1 + }, { "file": "packages/metadata-protocol/src/protocol.metadata-redaction.test.ts", "verb": "delete",