Skip to content

fix(spec): project a union branch-by-branch, so five filter operators reach a published reference page - #17085

Merged
os-bill merged 8 commits into
mainfrom
claude/issue-16431-filter-operator-schema-projection
Sep 9, 2026
Merged

fix(spec): project a union branch-by-branch, so five filter operators reach a published reference page#17085
os-bill merged 8 commits into
mainfrom
claude/issue-16431-filter-operator-schema-projection

Conversation

@os-bill

@os-bill os-bill commented Sep 9, 2026

Copy link
Copy Markdown
Collaborator

Part of #16431 — option (a) only. Option (c) landed as PR #16908; option (b) was ruled against by triage and is not attempted here. One item stays open on the card and is named at the bottom.

The premise the card recorded is FALSIFIED, and that is the first result

The card hypothesised that the existing io: 'input' retry in build-schemas.ts would project a z.date() branch if it were applied per branch rather than per schema, "since an author writes an ISO string". Measured against zod 4.4.3, that is false, and the reason is structural rather than incidental:

z.toJSONSchema(z.date(), { target: 'draft-2020-12' })              -> THROW Date cannot be represented in JSON Schema
z.toJSONSchema(z.date(), { target: 'draft-2020-12', io: 'input' }) -> THROW Date cannot be represented in JSON Schema
converting EACH branch of number | Date | string separately, io: 'input':
  branch 0 -> {"type":"number"}      branch 1 -> THROW      branch 2 -> {"type":"string"}

dateProcessor in zod/v4/core/json-schema-processors reads only ctx.unrepresentable and never ctx.io, so the two directions are the same answer at every granularity. Per-branch io: 'input' publishes nothing.

⇒ The route triage ruled — teach the generator to project z.date() — is unaffected; only the mechanism the card guessed at is. The one switch zod offers is unrepresentable: 'any', and it is used here as a vehicle, never as the answer.

What landed

A third projection attempt, reached only when both strict directions have already refused the whole export: packages/spec/scripts/lib/union-branch-projection.ts, wired into the emission loop of build-schemas.ts.

  1. Convert with unrepresentable: 'any', marking through override every node that came back with no structural keyword.
  2. Drop the marked nodes that are direct members of an anyOf / oneOf.
  3. A marked node anywhere else — an object property, a record value, an array item — fails the projection, and the export is skipped by re-throwing the error zod produced.

Step 3 is the load-bearing half. A bare {} inside an anyOf accepts every JSON value, so a build that merely set unrepresentable: 'any' would publish universally-permissive schemas for the exports skipped today and report nothing — a worse version of the silence this card is about. Nothing is ever emitted with a {} standing in for a type zod refused, and because the failure path re-throws zod's own error, this change cannot alter why anything is skipped, and so cannot alter any cause recorded in unemitted-schemas.baseline.json.

Two details that measurement forced, both pinned by tests:

  • Detection is a strict re-conversion of the node, not a list of type names. "Emitted no keyword" also describes z.any() and z.unknown() — and describes them through wrappers. FieldOperators.$eq is z.any().optional().describe(...), which comes back as { description }, byte-shaped exactly like an unprojectable z.date().describe(...). A list-based detector marked it and refused the whole projection for the enforced half of the filter contract.
  • Both io directions are projected and the one that drops FEWER branches wins, output breaking a tie — not output-first. A .transform() branch has no output form and a good input one; output-first would delete an authorable shape. Measured case: Data.HookSchema, where output drops the deprecated z.custom handler AND a pipe, and input drops only the z.custom.

Why dropping the branch is not a narrowing

These artifacts describe JSON documents. A JSON document cannot carry a Date INSTANCE, so the set of JSON documents the z.date() branch admits is empty, and an empty member of an anyOf contributes nothing to the union's accept set. Removing it changes which JSON documents validate by exactly nothing. The ordering docblock in src/data/filter.zod.ts states the same thing from the runtime side: the date-macro resolver returns only strings, and the driver only ever sees ISO date / timestamp strings.

⛔ No runtime accept set was narrowed. packages/spec/src/data/filter.zod.ts is not in this diff — orderingComparandSchema and rangeEndpointSchema are byte-identical, which is what keeps this (a) and not (b).

The published prose still names Date — the reference table's Description cell reads "a number, a Date, a string, or a { $field } reference" while its Type cell reads number | string | { $field ... }. That difference is deliberate and is recorded ON the artifact: each of the five new schema files carries an x-unprojectable-branches array naming the exact JSON Pointer and zod type of every branch the projection dropped, e.g. #/properties/$gt/anyOf/1 / date.

The measured effect, repo-wide

A generator change is repo-wide by construction, which is why triage said (a) "要有自己的验证面". Measured on this tree:

BEFORE   Generated: 1588   Skipped: 23  (16 function · 4 date · 2 undefined · 1 custom)
AFTER    Generated: 1593   Skipped: 18  (16 function · 2 undefined)
         Projected: 5 export(s) after dropping 31 union branch(es) with no JSON form

The 23 is re-derived here, not inherited: it matches the population recorded when option (c) landed. Five entries leave the ledger — the ratchet reported each as repaired and refused the build until its line went:

export branches dropped what it publishes now
Data.ComparisonOperatorSchema 4 (date) $gt $gte $lt $lte
Data.RangeOperatorSchema 2 (date) $between
Data.FieldOperatorsSchema 6 (date) the enforced copy of all four families
Data.NormalizedFilterSchema 18 (date) $and / $or / $not
Data.HookSchema 1 (custom) the whole hook metadata type

Data.HookSchema is the ledger's own "widest gap" entry, and it was not sought: its only unprojectable member was the DEPRECATED inline-function handler branch, which is a direct union member. It is the subject of the separately filed #16906, whose measured defect (zero data/Hook: keys under any key-level ratchet) this repairs as a side effect — 22 data/Hook: authorable keys and 4 data/Hook: defaults now enter authorable-surface/ and authorable-defaults/. ⇒ #16906 wants re-triage rather than work; it is not addressed by design here.

⛔ Nothing was emptied that should not be. The 18 remaining entries all fail for a reason a per-branch projection cannot reach, and three ledger reason fields that this PR made FALSE were corrected against measurement rather than left standing:

  • Automation.FlowFunctionEntrySchema — its bare z.function() member IS dropped (1 branch pruned), and the union still does not publish, because the member behind it requires a callable handler in a PROPERTY position.
  • Data.MemoryConfigSchema / Data.MemoryPersistenceConfigSchema — the unprojectable node is at #/properties/persistence/anyOf/0/anyOf/4/properties/adapter/properties/load, nested two levels inside a union branch rather than being that branch.

The card's own consumer-side controls, before and after

Re-run on this tree, with the card's lit control:

                        BEFORE   AFTER
ComparisonOperator         0        7
RangeOperator              0        2
FieldOperator              0        7
SpecialOperator            2        2      (control: the file IS live and the pattern CAN hit)

grep -rlE 'ComparisonOperator|RangeOperator|FieldOperators' content/docs/   ->  0 files BEFORE, 2 files AFTER

content/docs/references/data/filter.mdx grows 10,752 -> 72,626 bytes. The five operators now render with their full .describe() text: 1048 / 1060 / 1045 / 1057 characters on $gt / $gte / $lt / $lte, and 1009 on $between — including the { "$gte": "2026-01-01" } shape the platform's own date-macro resolver produces, which is what #5685 wrote that prose to correct.

Rider 1 from the landing comment, discharged

The summary line read Skipped: N (unsupported types: function, date, bigint, custom) — a hard-coded list that had stopped describing the tree: bigint was in it and is not a cause here, undefined is a cause here and was not in it. It is now DERIVED from the build's own skips through countByCause, so it cannot go stale again. It reads Skipped: 18 (unsupported types: function, undefined) today.

⚠️ One maintainer decision is in this diff, deliberately visible

Publishing ComparisonOperator and RangeOperator makes the import-surface ratchet fire: neither schema has an export type X = z.infer of typeof XSchema alias, so the generated ## TypeScript Usage line omits their names. packages/spec/docs-import-surface.baseline.json therefore grows by exactly two lines (60 -> 62, nothing removed).

The gate's preferred remedy is to add the aliases — but that is an edit to packages/spec/src/data/filter.zod.ts and a new published export name, both outside this dispatch's fence. The precedent points the same way: four of the six operator families already sit in that baseline (EqualityOperator, SetOperator, SpecialOperator, StringOperator), so recording these two beside them is the consistent answer and adding aliases to only two of six would not be. ⇒ Recorded, not decided. If a maintainer prefers the aliases, they are a two-line follow-up.

Verification

  • Gates: 91 families derived by node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack --commands over the 13 changed paths; reconciled with --ran: 90 run, 1 unrun. The one is pnpm check:pm-dispatch-gates, a self-test suite over scripts/pm/dispatch-gates.mjs — a file this diff does not touch — which exceeded this container's foreground command cap on three separate attempts (it makes visible progress and is killed mid-run, exit 124). NOT MEASURED, ⛔ not a red; CI runs it. Seven others returned exit 3 PREREQUISITE NOT MET, each needing a workspace build this container did not have; four of those turned green after pnpm --filter @objectstack/spec build and the rest need sibling packages built, which CI does.
  • Tests: pnpm --filter @objectstack/spec test -> 468 files / 13125 passed, exit 0 under the shared verify lock. pnpm --filter @objectstack/spec typecheck -> exit 0. New scripts/union-branch-projection.test.ts: 19 passed.
  • Ablation, one script, both legs under trap ... EXIT INT TERM with absolute paths and blob-hash proof. Mutation: candidates.length === 0 -> >= 0 in projectByPruningUnionBranches, anchor count 1 -> 0, injected marker count 1, blob 1b23049d -> 4c9f2e87. Under it the unit file went 8 failed / 11 passed and check:authorable-surface exited 1 — reporting Skipped: 23 again and firing the SIBLING disappearance ratchet with "5 previously published schema(s) disappeared from this build", which is the two ratchets guarding their two different directions. Restore: blob back to 1b23049d, git diff HEAD 0 bytes.
  • Lint ran in full rather than narrowed: eslint . --no-inline-config --format json over 6410 files, 0 errors, 0 warnings, exit 0, at 3b78f35e.
  • Control bytes: grep -naP over all 13 changed files -> 0 hits, with a positive control on a 0x01 byte that fired.
  • origin/main merged into this branch before opening; check:generated 15/15 green afterwards.

What is still open on #16431

Not addressed here, and the reason this says Part of:

  • the two missing type aliases above — the card's own observation that the ## TypeScript Usage import line does not name these schemas is now true for two of them for a NEW reason, and the remedy is a maintainer decision on the published export surface;
  • option (b) itself, which triage refused and routed to the decision box; nothing here narrows a runtime accept set, and after (a) the motivation for (b) — "so the document can generate" — no longer exists.

🤖 Generated with Claude Code

https://claude.ai/code/session_01MkQhmuuJAVDjmeWNixwDDH


Generated by Claude Code

@github-actions

github-actions Bot commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

⚠️ 6 changed file(s) yielded no anchor (packages/spec/authorable-defaults/data.json, packages/spec/authorable-surface/data.json, packages/spec/declaration-map/data.json, …), so the pages documenting them are NOT COVERED by this run — this is not a clean bill of health for those files. Nothing else in this diff resolved to a documentable surface (no symbol, route or SDK anchor derived from 1 changed package(s)).

What this run could not see
  • 6 changed file(s) yielded no anchor (packages/spec/authorable-defaults/data.json, packages/spec/authorable-surface/data.json, packages/spec/declaration-map/data.json, …) — pages documenting those are invisible to this run
  • a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.

Coarse fallback — 131 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json 854639b311f714d936ae4ca239ef838ecc785845packageMentionDocs.

@github-actions github-actions Bot added documentation Improvements or additions to documentation tests tooling labels Sep 9, 2026
@os-bill
os-bill marked this pull request as ready for review September 9, 2026 07:38
@os-bill
os-bill added this pull request to the merge queue Sep 9, 2026
Merged via the queue into main with commit 58b36fa Sep 9, 2026
43 checks passed
@os-bill
os-bill deleted the claude/issue-16431-filter-operator-schema-projection branch September 9, 2026 08:05

Copy link
Copy Markdown
Collaborator

Contract review at CONTRACT_REVIEW_TIERPOST-MERGEVerdict: PASS WITH FINDINGS (director seat, summon #18 segment 3, session_017Js5kTpTtxieBjPyScgxJ3, 2026-09-09T09:1xZ)

PR #17085 · head 3b78f35e7040874b08514b6b43cad240daccf4f9 · merged 2026-09-09T08:05:23Z (merge commit 58b36faf, ancestor of origin/main) · reviewed-at 2026-09-09T08:44:14Z.

  • Why post-merge: needs:contract-review was removed at 07:38:11Z, the PR readied 07:38:22Z and queued 07:39:41Z with no tier verdict on this PR, its reviews, or card [finding] Five filter operators ($gt/$gte/$lt/$lte/$between) reach NO published reference page — build-schemas.ts skips their whole schema over an unrepresentable z.date(), and the skip is silent #16431 — fifteen minutes after the domain:spec seat's own review (5597883648) ended 「⛔ 无达档 PASS 不得入队」. This comment supplies the verdict the record lacks.
  • Reviewed-by: isolated claude-fable-5-1 subagent, transcript-verified (129 harness model stamps, all claude-fable-5-1, zero residue; positive control 92 assistant / 40 user role tokens), adopted verbatim below. Method: read-only; PR head extracted with git archive into scratch, pnpm install --filter @objectstack/spec there, the 19 unit tests, four mutations, a full build-schemas.ts run and build-docs.ts --check executed against that copy. Implemented-by: claude/issue-16431-filter-operator-schema-projection, os-bill / session_01MkQhmuuJAVDjmeWNixwDDH (dev mode:subagent of the same session).

F1 (non-blocking, process/record) — PR #17085 timeline 07:23:02Z→07:40:58Z; card #16431 comment 5597883648; #16335 comment 5598037662; #6017 comment 5598082884 — The tier verdict that cleared the label was never filed on either carrier. The seat's own review (07:23Z) says 「⛔ 无达档 PASS 不得入队」; its #16335 comment at 07:36:57Z says the #17085 tier review was still running; needs:contract-review was removed at 07:38:11Z, ready 07:38:22Z, queued 07:39:41Z; the only claim of a PASS is prose on the seat's shift thread #6017 at 07:40:58Z (「达档复核 PASS 零阻塞发现(熔断读数 113 与 89)」) with no verdict text, no Implemented-by:/Reviewed-by: pair and no provenance comment on PR or card — contrary to contract-review.md 「PASS ⇒ … 清标同笔留 provenance 评论引该 PASS」 and 落地前检 ①. Dev and seat are the same session, so the seat review is by construction not the independent tier control. Fix: this comment and its pointer on #16431 are the provenance.

F2 (non-blocking, tests)packages/spec/scripts/union-branch-projection.test.ts:150-155, 226-230; packages/spec/scripts/lib/union-branch-projection.ts:297 — The clause 「refuse when the marked node is anywhere but a direct anyOf/oneOf member」 is pinned only at the root. Mutations on the scratch copy: M2 (projection disabled) → 8 fail; M3 (strict re-conversion removed) → 4 fail; M1 (guard at :297 deleted) → 1 fail (only refuses a union whose every branch is unprojectable, :157); M1b (guard loosened to === '#', nested marks tolerated) → 19/19 pass. The property/record/array pins at :152-154 and the PersistenceAdapterSchema pin at :229 pass under M1b vacuously via the pruned.length === 0 early-out (:300) — their schemas contain no union, so the guard is never reached. Under M1b the FlowFunctionEntry shape publishes {"anyOf":[{"type":"object","properties":{"handler":{"x-os-unprojectable":"function"}},"required":["handler","name"],…},{"type":"string"}]} — a marker leak plus a required annotation-only property, the exact failure the PR body calls load-bearing. CI backstop exists but is indirect: the ledger's repaired ratchet (build-schemas.ts:2774-2779) refuses the build when Automation.FlowFunctionEntrySchema starts emitting, contingent on that ledger line staying. Follow-up: add a unit pin projectByPruningUnionBranches(z.union([z.function(), z.object({ handler: z.function() }), z.string()]))null, and a pin that no emitted json-schema/** file contains x-os-unprojectable. No .skip/.only/.todo (count 0).

F3 (non-blocking, ADR-0078 spirit)packages/spec/scripts/build-schemas.ts:486-498; packages/spec/scripts/build-docs.ts (no match) — x-unprojectable-branches has zero readers at the PR head (git grep: only the writer and the changeset mention it; x-io likewise has none since #2978). build-docs.ts does not render it, so the page the card is about shows Type number | string | { $field … } beside prose reading "a number, a Date, a string" (filter.mdx ComparisonOperator rows) and hook.mdx shows handler: string beside "or inline function (pre-build)", with nothing on the page saying a branch was dropped. Not an ADR-0078/0049 breach (those govern authorable metadata keys; this is an x- annotation on a generated artifact) and the artifact record is real — but the PR body's "recorded ON the artifact" is the whole record. Follow-up (same card as F2): have build-docs.ts emit a one-line note from x-unprojectable-branches, or decide artifact-only is enough and say so in the module docblock.

F4 (non-blocking, scope)packages/spec/authorable-surface/data.json (+22 data/Hook: keys), authorable-defaults/data.json (+4) — The generator change is repo-wide by construction; Data.HookSchema now publishes (input shape, x-io: input, one custom branch dropped at #/properties/handler/anyOf/1). This repairs #16906's measured defect as a side effect; #16906 (still bare: no labels, no assignee) needs re-triage, not work.

F5 (non-blocking, recorded decision)packages/spec/docs-import-surface.baseline.json:14,28 — Two accepted gaps added (60→62) because ComparisonOperator/RangeOperator have no export type alias; the dev's open question was answered by the seat as option A (consistent with the four sibling families already in the baseline; B/C add published export names and edit fenced filter.zod.ts). The reviewer concurs; it is a technical default, not a decision-box item. Residual: the card's own (b)/alias question stays open on #16431 (PR is Part of).

Verified and holding:

  • Published artifacts: the six committed JSON artifacts re-derive byte-identical from the scratch build (Generated: 1593 · Skipped: 18 (function, undefined) · Projected: 5 export(s) after dropping 31 union branch(es), exit 0); tarball-only new files json-schema/data/{ComparisonOperator,RangeOperator,FieldOperators,NormalizedFilter,Hook}.json (gitignored at .gitignore:63, shipped via package.json files[] → 'json-schema'), each carrying x-unprojectable-branches; objectstack.json bundle includes them. Across all 1583 emitted files: 0 x-os-unprojectable leaks, 0 annotation-only union members.
  • Invariant: probe on the PR head — ComparisonOperator.$gt.anyOf after the drop is structurally identical (descriptions stripped) to a strict z.toJSONSchema(z.union([z.number(), z.string(), FieldReferenceSchema])); all 30 filter drops are type: 'date' at …/anyOf/1 positions; hook.zod.ts:246-250 itself states the JSON artifact only ever contains the string handler. For every JSON-schema consumer the accept set over JSON documents is unchanged; only in-process TS callers can pass a Date instance, and filter.zod.ts is not in the diff, so the runtime union is untouched.
  • Semver: @objectstack/spec: patch matches AGENTS.md:1034-1036 (bug fix in a released package → patch); additive only, no removal, no breaking; Clause-② yes correctly declared for new keys on a published payload.
  • Route: inside option (a) as ruled (5579055032): triage's route was "teach the generator"; the per-branch io:'input' idea was a 「线索…值得试」. The dev tried it, posted readings (test :71-81), and did not touch filter.zod.ts. The published schema equals what (b) would have published, reached generator-side with the runtime accept set untouched — precisely the line triage drew. No re-ruling needed; the seat review reached the same conclusion.
  • Docs: all three mdx carry the AUTO-GENERATED banner; build-docs.ts --check in scratch → exit 0, 228 generated files in sync, 62 accepted gap(s); index.mdx 1577→1582 / Data 168→173 consistent with manifest +5. No content/docs/releases/ edit.
  • CI on head: 43 check-runs, 35 success / 8 skipped / 0 failed, all pinned to 3b78f35e70; Lint & Repo Gates success 07:37:34Z, Type Check workspace success 07:33:49Z. The unlabel at 07:38:11Z came 37 s after Lint & Repo Gates went green — CI convergence was genuinely awaited.
  • Docs-drift bot 5597813139: names six artifact files with no anchor; grep at PR head for any page stating the old population → zero hits outside the regenerated pages. It names no page the PR falsified.
{"pr":17085,"post_merge":true,"head":"3b78f35e7040874b08514b6b43cad240daccf4f9","verdict":"PASS WITH FINDINGS","blocking":[],"clause2":"yes","semver_ok":true,"followup_owed":true,"ci":"43 check-runs on head: 35 success, 8 skipped, 0 failed; Lint & Repo Gates + Type Check workspace success before the label was cleared"}

维护者速读

已发布面的变动是纯增量:六个已提交的生成物在隔离副本里逐字节重生成一致,npm 包内新增五个 json-schema/data/*.json(gitignore 但走 files[] 发运),authorable-surface 新进 48 个键(含 22 个 data/Hook:),未发出台账 23→18,所有 1583 个发出文件零标记泄漏、零全接受的 union 成员。「丢弃 Date 分支不收窄接受集」的主张成立:实测 $gt 丢弃后的 anyOf 与直接对 number|string|FieldReference 做严格转换的结果结构相同,30 个 filter 丢弃全是 date,Hook 的 handler 按源码自述 JSON 产物本就只有字符串形;filter.zod.ts 不在 diff 里,运行时接受集未动,路线仍在分诊裁定的 (a) 之内,无需重裁。三个 mdx 全是生成物(build-docs --check 228 文件同步),未碰 releases,head 上 43 个 check 无红。需要跟进的有三件:① 记录缺口——清标发生在席位自己写的「无达档 PASS 不得入队」之后 15 分钟,PR 与卡上均无达档裁决与 provenance 评论,唯一痕迹是 #6017 上的一句散文,本裁决即 provenance;② 一张小工具卡——单元测试对「标记在 union 直接成员之外即拒绝」只钉住了根级,松动为「仅根级拒绝」时 19 个用例全绿(M1b),FlowFunctionEntry 形状会带着 x-os-unprojectable 发布,目前只靠台账 repaired 棘轮兜底,需补嵌套用例与「发出文件不含标记」的钉子,并顺带决定 x-unprojectable-branches(当前零读者)是否要在参考页上渲染一行说明;③ #16906 已被顺手修复,需改判而非派工。合并前的达档复核会给 PASS WITH FINDINGS,无任何本应拦下的内容落地。


Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation size/xl tests tooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants