Merge remote-tracking branch 'origin/main' into claude/issue-13986-structured-logger-volume-census

Author: claude

.changeset/autonumber-default-unique.md

@@ -0,0 +1,73 @@
+---
+"@objectstack/spec": minor
+---
+
+feat(spec): an `autonumber` field is `unique: 'organization'` by default; explicit `unique: false` opts out (#13894)
+
+**BREAKING** emitted-shape change on `FieldSchema` (the accept set is unchanged),
+shipped as `minor` under the repo's launch-window convention for breaking changes.
+
+An auto-number is a business identifier — a contract number, a quote number, a
+case number — and an identifier that may repeat is not one. Yet the platform
+only ever materialized a unique index where the author had written `unique`
+by hand: of hotcrm's nine auto-numbered identifiers, exactly one
+(`crm_case.case_number`, `unique: true`) carried the tenant-composite unique
+index, and the other eight could mint the same number twice (measured:
+objectstack#12394 re-issued `ACC-000009`). Maintainer ruling 2026-08-31
+(hotcrm#1301): the default flips.
+
+- An `autonumber` field that **omits** `unique` now parses to
+  `unique: 'organization'` — one holder per organization, materialized by the
+  drivers exactly as `case_number`'s hand-written declaration was: the NULL-safe
+  tenant-composite index `(COALESCE(organization_id, '__global__'), <field>)` on an
+  organization-scoped object, a plain unique index on an object with no
+  organization key.
+- Every **other** field type keeps `unique: false` as its default, at the same
+  key position — parse output for non-autonumber fields is byte-identical.
+- Every **authored** spelling (`true`, `'organization'`, `'global'`, `false`)
+  parses exactly as before, on every type.
+- The default is materialized at parse time (the `.overwrite()` tail of
+  `FieldSchema`, the type-conditional precedent `deleteBehavior` set), because
+  the drivers read the parsed `unique` value-only; the published JSON Schema
+  therefore no longer carries `default: false` on `Field.unique` — the
+  description states the rule, and the authorable-defaults ratchet records the
+  move as `data/Field:unique = false → (none)`.
+
+**Opting out.** Write `unique: false` explicitly on the autonumber field. That
+is the whole opt-out surface — no second key. It is legitimate only for a
+display-only sequence that nothing uses to identify the record; note that the
+platform's duplicate scan (`os migrate duplicates`) keeps treating every
+autonumber field as an identifier regardless.
+
+**Migration — what an operator with existing duplicates sees.** A table that
+already holds duplicate auto-numbers cannot take the index. On SQLite/Postgres/
+MySQL the SQL driver does not fail the boot and does not skip silently: it logs
+on the `error` channel —
+
+```
+[sql-driver] cannot create NULL-safe unique index 'uniq_crm_quote_organization_id_quote_number' on "crm_quote" — existing rows violate it (duplicates the previous NULL-distinct index admitted, #5030). The constraint 'organization_id, quote_number' is NOT enforced until the data is deduplicated: run "os migrate plan" for the conflicting rows (ADR-0120 D4).
+```
+
+— and the same boot's drift pass names the conflicting key groups with their
+row counts:
+
+```
+[schema-drift] crm_quote: cannot create 'uniq_crm_quote_organization_id_quote_number' as UNIQUE (COALESCE(organization_id, '__global__'), quote_number) — existing rows already violate the NULL-safe unique constraint (duplicates the old index wrongly admitted, #5030): (organization_id="__global__", quote_number="QUO-00009") × 2 rows; (organization_id="org_x", quote_number="QUO-00010") × 2 rows. The op is BLOCKED: apply re-probes and refuses, and the existing index stays in place (ADR-0120 D4). Deduplicate the listed rows, then re-run "os migrate plan".
+```
+
+`os migrate plan` reports the same blocked `create_index` with the same groups
+until the rows are deduplicated; `os migrate duplicates` lists the holder row
+ids of any value minted across organization partitions (the seed/API split).
+Deduplicate — which duplicate keeps its number is a business decision — then
+re-run `os migrate plan` / restart, and the index materializes. An object with
+`tenancy.enabled: false` takes a plain unique index instead, and there the
+driver raises the database's own unique-violation error at boot (it names the
+index, not the rows) — run `os migrate duplicates` / a `GROUP BY <field> HAVING
+COUNT(*) > 1` to find them.
+
+Two landed defects change shape on purpose under the default: a counter that
+re-issues a number after a burned reservation (#12394) and two counters minting
+for one object (#8686) used to produce a *silent* duplicate; they now produce a
+loud unique-violation refusal at the write.
+
+<!-- adr-0087: registered autonumber-default-unique-organization -->

.changeset/bu-tenant-screen-relanding.md

@@ -0,0 +1,11 @@
+---
+'@objectstack/plugin-sharing': patch
+---
+
+Fix: a sharing rule with a business-unit recipient granted nothing when the unit came from seed data — and tenant-screen the member reads that widening exposes.
+
+`BusinessUnitGraphService.orgScope` screened `sys_business_unit` with a strict `organization_id` equality, while the platform's own read-side chokepoint (`SqlDriver.applyTenantScope`) is null-inclusive: `(organization_id = ? OR organization_id IS NULL)`, because a NULL organization marks a platform/seeded row every tenant may see. A sharing rule always carries the caller's organization, but a business unit written by seed data carries none — a seed cannot know the id the runtime mints at boot — so the two never matched. The seed check read the unit as "does not exist", both recipient widths (`business_unit` and `unit_and_subordinates`) expanded to zero users, and the rule stayed active having materialised no `sys_record_share` row and logged nothing. `orgScope` now applies the platform's null-inclusive screen, the same predicate `plugin-approvals` already applies to these very rows and `SharingRuleService.adminOrgScope` applies to the rule table.
+
+The member reads are now tenant-screened, which they were not before. Both `expandUnitMembers` and `expandUsers` queried `sys_business_unit_member` with no organization predicate at all, under a system context that carries no tenant either, so the strict unit screen was the only thing keeping an org-stamped rule away from that unscoped query. Widening the unit screen alone would have turned a silent under-grant into a silent cross-tenant over-grant, since a seeded unit id exists identically in every tenant. The member screen is strict rather than null-inclusive on purpose: seed replay and elevated system writes both leave `sys_business_unit_member.organization_id` NULL, so a NULL there means unknown tenancy rather than platform-global, and an org-scoped rule does not grant to it. The sibling recipient widths already read their membership rows this way.
+
+An active business-unit rule that expands to no recipients now warns once per rule per process, naming the rule, the object, the recipient kind, the unit and the organization. That case — a rule whose unit and membership rows were both seeded — is the one combination that still grants nobody, and it is no longer silent.

.changeset/create-record-duplicate-code.md

@@ -0,0 +1,25 @@
+---
+"@objectstack/service-automation": patch
+---
+
+fix(automation): `create_record` now surfaces the engine's `DUPLICATE_RECORD` code, so a `try_catch` / `fault` edge can finally tell "already there" from "the store is down" (#14419)
+
+`engine.insert` (#14095) already raises `DuplicateRecordError` — `code: 'DUPLICATE_RECORD'` (ADR-0112) — for a unique-constraint violation, driver-independent. The `create_record` node executor threw that away: every failure, from a duplicate key to a downed connection, collapsed into one opaque string (`create_record(<object>) failed: <message>`). A flow's only two error-handling primitives, `try_catch` and a `fault` edge, saw the same shape either way — the only expressible reading of "swallow the duplicate" was "swallow everything".
+
+`NodeExecutionResult` gains an optional `code?: string` field, beside the existing `errorClass`, set when the caught error carries the platform's classified `DUPLICATE_RECORD` code. `AutomationEngine` now copies it onto the `$error` run variable alongside `message` (both the direct `fault`-edge path and the `try_catch` catch-region binding, which previously reconstructed `errorVariable` from the caught exception's message alone and silently dropped it), so a flow can actually branch on `{$error.code}`:
+
+```
+try:   create_record(lead, { email })
+catch: { $error.code === 'DUPLICATE_RECORD' } → swallow, continue
+       else                                   → re-raise / route the fault edge
+```
+
+Additive only — no existing field, message text or routing behaviour changes; an executor that never sets `code` (every one except `create_record` today) is unaffected. Deliberately scoped to `create_record` alone: `update_record` / `delete_record` collapse the same way, but `engine.update` still leaks the raw driver error (#14390, not yet fixed), so those node results have nothing structured to surface yet. `create_record` itself forwards `code` only when it equals `DUPLICATE_RECORD` — narrowly, on purpose, matching the ADR-0112 vocabulary member this repair was actually scoped to surface, not any code an as-yet-unaudited driver error might someday carry.
+
+**Patch round 1 (tier contract review):** `try_catch`'s catch region reads `code` off the run-wide `$error`, but the engine only rewrites `$error` when a failing node *returns* a failure, or *throws* through a node with its own `fault` edge — and a node inside a `try_catch`'s `try` region never has one (the region's synthetic sub-flow carries only the region's own edges). A node that fails by throwing (a `timeoutMs` firing, a dying nested container) therefore used to leave `$error` exactly as an *earlier, unrelated* failure left it — its `code` included. An identity guard (`$error` must have *changed*, not merely still be present, since this attempt started) closes that; two flows now pin it: a `loop` sweeping two rows where row 1 is a genuine duplicate and row 2 times out, and a plain flow where an earlier fault-routed duplicate must not leak into a later, unrelated `try_catch`.
+
+A custom `IDataEngine` implementation whose thrown error already carries `code: 'DUPLICATE_RECORD'` (without being an instance of `@objectstack/objectql`'s `DuplicateRecordError`) is treated as a duplicate too — correct under ADR-0112, since `code` is the classified envelope's public contract, not the concrete class.
+
+**Known gap, filed rather than fixed here (out of this lane's scope):** `packages/spec`'s `TryCatchErrorValueSchema` — the ONE declared shape for the `errorVariable` binding shared by author, engine and run log — does not declare `code` yet, and strips it on a strict parse. `packages/spec` is single-owner (`domain:spec`); tracked as #14954.
+
+<!-- adr-0087: not-required (no-migration-prescription) additive optional field; no authorable key, export removal or rename for an upgrader to migrate -->

.changeset/create-scaffold-manifest-identity.md

@@ -0,0 +1,37 @@
+---
+"@objectstack/cli": patch
+---
+
+fix(cli): `os create example` now scaffolds a manifest the protocol schema accepts
+
+The `objectstack.config.ts` that `os create example <name>` wrote declared
+three manifest keys — `name`, `version`, `description` — and nothing else.
+`ManifestSchema` requires `id` (the reverse-domain package id) and `type`
+(`app` | `plugin` | …), and `namespace` is the mandatory prefix of every object
+name, which decides each object's table name and REST path. Parsed against the
+schema, the emitted block answered `success: false` with
+`invalid_type@id · invalid_value@type`.
+
+`defineStack` throws on exactly that, so the project a documented command had
+just created refused to load on its first run — before the author had written a
+line. The three `os init` templates all stamped the identity block; this was
+the one scaffold that had drifted, and nothing noticed because no test looked
+at these templates as data.
+
+The template now stamps what `os init` stamps: `id`, `namespace` (derived from
+the project name with `init`'s own `sanitizeNamespace`, so both scaffolders
+answer the same way for the same input), `type: 'app'` and
+`engines.protocol`, alongside the `version`, `name` and `description` it
+already carried. `engines.protocol` is stamped from `PROTOCOL_MAJOR` — the same
+constant `init` stamps — and ships with the same self-contained comment
+explaining what the range is and when to move it.
+
+A pin sweeps both scaffolders: every `init` and `create` template that emits an
+`objectstack.config.ts` is rendered through its own emitter, loaded back, and
+its `manifest` parsed through the real `ManifestSchema`. The population is
+derived from the two template maps rather than listed, so a template added
+later is swept the day it is added.
+
+`os create` itself is untouched — it is not removed, deprecated, or redirected
+at `os init`. Whether the two scaffolders should stay separate is a CLI-surface
+decision, not this fix.

.changeset/driver-update-missing-id-null.md

@@ -0,0 +1,64 @@
+---
+"@objectstack/driver-mongodb": minor
+"@objectstack/driver-turso": minor
+---
+
+fix(drivers): `update()` on a missing id answers `null` on MongoDB and on Turso's remote face
+
+**BREAKING** for TypeScript consumers — a published TYPE-surface narrowing alongside a
+runtime behaviour change, shipped as `minor` under the launch-window convention. Two
+published declared returns move: `MongoDBDriver.update()` and `RemoteTransport.update()`
+(both exported from their package index) now declare
+`Promise<Record<string, unknown> | null>` where they declared
+`Promise<Record<string, unknown>>`. A caller that reads fields off either result —
+`result.id`, `result.title` — no longer compiles until it narrows the `null` arm first.
+The narrowing is delivered by the compiler at every call site, and it is the honest
+declaration: the value that arm carries has always been reachable, it was simply being
+answered with a fabricated record instead.
+
+`IDataDriver.update()` declares `Promise<Record<string, unknown> | null>` — the
+not-found arm landed with the ruling on the contract (`packages/spec` is untouched here),
+and it is the answer `InMemoryDriver`, `SqlDriver`, `SqliteWasmDriver` and `TursoDriver`'s
+local face have always given. Two implementations did not honour it. They **invented a
+record** instead:
+
+- `MongoDBDriver.update()` ran `updateOne({ id })`, then `findOne({ id })`, and
+  when nothing came back returned
+  `withoutUndefinedOwnKeys({ id: String(id), ...updateData })` — a row assembled
+  from the caller's own payload plus the `updated_at` it had just stamped, under
+  an id that names no document.
+- `RemoteTransport.update()` ran `UPDATE … WHERE "id" = ?`, then
+  `SELECT * … WHERE "id" = ?`, and when no row came back returned
+  `{ id, ...data }` — the caller's payload with the id stapled on.
+
+Both now return `null`. That is the runtime half of this change, and it is why this
+release is not a pure type-surface move: the value a caller receives for a missing id is
+different at run time, not only in the `.d.ts`.
+
+This is the expensive direction of wrong, not merely the wrong answer: the
+fabricated row said **succeeded** where the truth was **not found**, and said it
+in a shape carrying the caller's own fields back, so nothing about it looked
+wrong. Through the engine's by-id door a REST / SDK / MCP `update` against a
+deleted or mistyped id answered **200 with a record that does not exist** — on
+these two implementations only. A caller, human or agent, read that as a landed
+write and did not retry, alert or roll back.
+
+Two things downstream become correct rather than merely different:
+
+- **One `TursoDriver`, one answer.** Its remote branch passes the transport
+  result through `formatRemoteRow`, which already guards
+  `row && typeof row === 'object'`, so `null` reaches the engine untouched and
+  the two faces converge with no edit at that seam. Previously the same driver
+  answered the same missing id two ways, chosen by `isRemote`.
+- **`RemoteTransport.bulkUpdate()`'s skip stops being dead code.**
+  `if (updated) results.push(updated)` is the cross-driver convention
+  `SqlDriver.bulkUpdate` follows; on this transport `updated` could never be
+  falsy, so a batch over N missing ids answered N invented rows. It now answers
+  the rows that exist.
+
+`upsert()` is untouched on both drivers: an upsert never answers "not found".
+
+No landed test pinned the fabricating posture on either driver, so the
+regression pins added here are net-new coverage rather than a changed baseline.
+
+<!-- adr-0087: not-required (no-migration-prescription) No metadata key is removed, renamed or re-shaped: the moving surfaces are two driver methods' declared return types and the value they answer for an id that names no row, so there is nothing for `objectstack migrate meta`, `spec-changes.json` or the upgrade guide to project, and this changeset prescribes no rewrite. The consumer obligation is a TypeScript narrowing at the call site, delivered by the compiler. `type-surface-only` is NOT claimable here: its predicate 4 (narrowed-from-erased) is false — neither declared return was `any` at the merge base, they were the non-null `Promise<Record<string, unknown>>` — and, independently, runtime behaviour moves in the same diff, which is more than a type surface. Same disposition and reasoning as `.changeset/driver-memory-update-upsert-honest-types.md` (PR #14434), one day earlier in this same series. -->

.changeset/filesystem-loader-list-reachability.md

@@ -0,0 +1,38 @@
+---
+"@objectstack/metadata": minor
+---
+
+fix(metadata): `FilesystemLoader.list()` reports only names `findFile()` / `load()` / `exists()` can resolve
+
+**BREAKING (list output narrows).** Two shapes stop appearing in
+`FilesystemLoader.list()`, and therefore in `MetadataManager.listNames()`:
+
+- **nested files** — `ROOT/TYPE/crm/account.json` was listed as `account`, a
+  name that resolves against `ROOT/TYPE/account.json` and finds nothing;
+- **extension-less files** — `ROOT/TYPE/noext` was listed as `noext`, which
+  resolves under no appended extension at all.
+
+A third shape follows from the same rule rather than from a rule of its own:
+the extensions a name can be resolved under are now the ones belonging to the
+loader's **registered serializers**, so under the manager's default format set
+(`typescript` / `json` / `yaml`) a `.js` file leaves `list()` too. It was
+previously listed and resolvable while `loadMany()` could never return it and
+`load()` threw `No serializer found for format: javascript`. Register the
+`javascript` serializer and it is listed, resolvable and loadable together.
+
+`list()`, `findFile()` and `loadManyKeyed()` now share one name-to-path
+derivation, so `listNames()` and `get()` give the same answer. Previously a name
+could sit in the list while `get()` answered `null` for it — a silent failure an
+author reads as their own typo.
+
+Nothing changes for a tree whose metadata is laid out as `ROOT/TYPE/NAME.json`
+(or `.yaml` / `.yml` / `.ts`), which is the layout ADR-0008 §10 already
+prescribes and `metadata-fs`'s `parseItemPath()` already enforces. `.yaml`,
+`.yml` and `.ts` are unaffected: the extension set follows the registered
+serializers, not §10's `.json`-only rule, which governs the `metadata-fs` store.
+
+`loadMany()` is unchanged and still returns bodies for nested and
+extension-less files; `findFile()` still resolves an explicitly path-shaped
+name such as `crm/account`, which nothing lists.
+
+<!-- adr-0087: not-required (no-migration-prescription) No authorable key, Zod schema or stored row moves: this narrows one runtime loader's `list()` output. The ledger's artifacts project metadata rewrites, and there is nothing here for `objectstack migrate meta` to rewrite — a tree carrying a nested or extension-less file needs the FILE relocated into the two-segment layout ADR-0008 §10 already prescribes, which no migration prescription can express. -->