Skip to content

chore(scripts): measure a strict authoring face over today's corpora (objectui#7581) - #7916

Merged
baozhoutao merged 1 commit into
mainfrom
claude/issue-7581-strict-authoring-face-measurement
Sep 6, 2026
Merged

chore(scripts): measure a strict authoring face over today's corpora (objectui#7581)#7916
baozhoutao merged 1 commit into
mainfrom
claude/issue-7581-strict-authoring-face-measurement

Conversation

@claude

@claude claude Bot commented Sep 6, 2026

Copy link
Copy Markdown
Contributor

Fixes #7581

The first deliverable of the #5250 ruling (maintainer, 2026-09-04, batch #25, option 2 — a strict authoring face as a programme, measurement first). This PR ships nothing and repairs nothing: no schema in packages/types changes, no gate is wired, and .passthrough() on BaseSchemaCore is untouched. It adds one reproducible throwaway, scripts/measure-strict-authoring-face.mjs, plus the reading it produces.

The number, in one line

2100 nodes across the three corpora. 196 are refused by a strict twin; 137 of those are green today and red only under strict; the other 59 are already red under the face as shipped. 58 of the 94 component types seen are strict-clean — zero refusals. A naive .strict() flip over whole documents refuses 304 of 557 node documents, but that number measures the recursion point, not the components (see "the node boundary" below).

How to re-derive it

pnpm --filter @object-ui/types build
pnpm exec tsx scripts/measure-strict-authoring-face.mjs          # the report below
pnpm exec tsx scripts/measure-strict-authoring-face.mjs --json   # machine form

Output is deterministic — every list sorted, nothing timestamped — so two runs diff clean (verified, both formats). The report pins HEAD, the origin/main commit it forked from, and a mechanical answer to "does this tree's corpus still equal main's?", so the numbers can be re-derived after the declaration repairs of #6939 land and move them. #6939 was not merged in: this reading is fa7d66c45's.

The five decisions the measurement rests on

  1. The registry is the arms of AnyComponentSchema (107 component types, zero collisions), not a type map built from all 206 module exports. That is not a style preference — it is measurably load-bearing: FilterFieldSchema declares type: z.enum(['text','select','number','date',…]) for a filter field, and an export-order registry hands it the text and select node types. The first draft of this script did exactly that and reported 718 text nodes with 704 strict refusals; the corrected registry reports 637 text nodes with 0.

  2. Traversal is schema-guided, never "any object with a type key". A field definition { name: 'amount', type: 'currency' } and a node { type: 'text' } are structurally identical, and half the field-type vocabulary (text, number, date, select, html, label) collides with a node type. A value is a node only where the schema declares a SchemaNodeSchema slot; from there descent continues through that node's own schema. A type at a node slot resolving in no schema is counted, not dropped.

  3. The node boundary. Child slots are z.union([SchemaNodeSchema, z.array(SchemaNodeSchema)]), and SchemaNodeSchema resolves to BaseSchemaCore — the ~21 base keys and nothing type-specific. Strict-ifying that recursion point makes every child node's own declared props unrecognised, so it measures the recursion point rather than the components. The per-component tables therefore replace SchemaNodeSchema with z.any() and flatten each document into nodes judged against their own schemas. Both readings are reported; the whole-tree one (304/557) is what a naive flip would really do, and it is the programme's first structural finding: the recursion point must be redirected to AnyComponentSchema before strict can be switched on at all.

  4. Three twins per schema. baseline (the face as shipped, boundary applied) tells you what is red today; strict adds catchall: never everywhere; keyProbe is a shape skeleton — same keys, every leaf z.any(), every member optional, union arms merged key-wise — that cannot fail, so diff(input, keyProbe.parse(input)) enumerates undeclared keys exactly, including on nodes that are red for value reasons and would otherwise yield no parse output to diff. Objects are cloned by patching _zod.def and calling their own constructor rather than rebuilt with z.object(shape), because the latter drops .refine() checks and a twin that quietly lost a refinement would under-report red.

  5. The predicates, stated because the card asks for them. Node document: parses, is a plain object, carries a string type, and that type resolves in the registry. Fragment (docs only): a ```json fence whose body does not parse as JSON — the deliberate-snippet shape; counted, never silently skipped, and this count is the price tag on finding: no gate reads a doc snippet's schema KEYS against the spec — #4823's deferred "second dimension", closed unbuilt #5138. Non-document: parses but fails one of the other three, split into notAnObject / noTypeKey / unresolvedType — the last kept apart because such a document is refused by AnyComponentSchema today.

Corpus enumeration — mine beside the PM's

corpus PM's count mine note
examples/schema-catalog/src/schemas/**/*.json 431 431 agree
```json fence openings under content/docs/** 204 204 agree; mine is a fence-state scan (info strings, indented fences), not a grep
apps/**/*.json 8 8 agree on the count — but all 8 are tooling config (package.json, three tsconfig*.json, manifest.json, vercel.json): zero authored node documents
packages/*/examples/** 6 6 1 .json, 5 .ts

⚠️ The apps half of corpus 3 is nearly empty, and that is the finding. Because apps/**/*.json holds no authored documents, corpus 3 was widened mechanically to .ts modules under apps/** and packages/*/examples/** that carry a type: '…' literal in source; those are imported and their exported plain objects harvested. That yields 19 items and only 2 node documents. The apps' real authored surface is JSX inside *-preview.tsx harnesses and metadata-designer drafts in preview-samples.ts that are spec records (objects, tools, jobs), not node documents. One module could not be imported and is recorded rather than skipped.

Cross-reference against the cards named on #7581

Findings filed separately (⛔ not repaired here)

Two things the measurement surfaced that are not undeclared-key results, deduped against the open backlog before filing:

  • BreadcrumbSchema and ObjectTreeSchema are not exported by name from @object-ui/types/zod — reachable only through the union, so no consumer can validate one alone.
  • Ten z.lazy exports rebuild their schema on every getter call (getter() === getter() is false), so the schema is reconstructed per parse and cannot be compared by identity.

Everything else is listed below rather than filed, which #7581 explicitly permits ("file it … or list it in the report").

Gates

gate verdict line
pnpm check:control-bytes ✅ check-control-bytes: OK (scanned 6398 tracked text file(s); skipped 85 binary). — exit 0
pnpm check:shell-escape-residue ✅ check-shell-escape-residue: OK (5/5 root(s) resolved …; 0 occurrence(s) outside a fence …) — exit 0
pnpm type-check:scripts exit 0, no diagnostics (tsc -p tsconfig.scripts.json)
pnpm lint:root ✖ 33 problems (0 errors, 33 warnings) — exit 0; 0 of them on the new file (pre-existing warnings only)
node scripts/check-governed-queue-guard.mjs --test … ✅ NOT GOVERNED — 2 path(s) checked against 5 governed surface(s); none matched.
reproducibility two runs, markdown and --json, byte-identical both ways

The corpus pin has a verified negative leg: appending a line to one content/docs file flips corpus identical to that main commit to NO, and git checkout HEAD -- content/docs restores the file byte-identically (blob 6aa5e591d59f6a6aa5e59, git diff HEAD -- content/docs empty).


Strict authoring face — measurement (objectui#7581)

  • HEAD 0044a60218eba4993b0a6e64e8697068c98756c4 · forked from origin/main at fa7d66c4505d0fcc866fbecec1901c4990a80c3a
  • corpus identical to that main commit: yes (compared over examples/schema-catalog/src/schemas, content/docs, apps, packages/types/src)
  • zod 4.4.3 · face read from packages/types/dist/zod/index.zod.js
  • node schemas in the registry (arms of AnyComponentSchema): 107 component types

Per corpus

corpus items node documents fragments not an object no type unresolved type nodes nodes strict-refused nodes red today strict-ONLY
catalog 431 412 0 0 0 19 1899 136 23 113
docs 204 143 25 0 20 16 193 58 35 23
authored 19 2 2 0 6 9 8 2 1 1

Headline

metric value
node documents measured 557
nodes measured (documents flattened at declared node slots) 2100
nodes refused by the strict twin 196
…of which strict is the ONLY reason (green today, red strictly) 137
nodes already red under the face as shipped 59
component types seen in the corpora 94
component types strict-CLEAN (zero refusals) 58
registered component types with no document anywhere 13
whole-document strict parse (no node boundary), refused 304 / 557
whole-document parse under the face as shipped, refused 50 / 557

Top undeclared keys — top level (what BaseSchemaCore.passthrough() admits today)

key occurrences
showSubmit 76
action 9
dataSource 9
actions 8
change 3
formType 3
message 3
sections 3
submitButton 3
text 3
value 3
currentPage 2
defaultViewType 2
listViews 2
searchPlaceholder 2
searchableFields 2
separator 2
title 2
breadcrumbs 1
content 1
defaultListView 1
defaultTab 1
emptyText 1
events 1
field 1
filterableFields 1
fullWidth 1
icon 1
itemTemplate 1
items 1
navigation 1
operations 1
optionsFrom 1
pageSize 1
pages 1
pagination 1
required 1
resizableColumns 1
rows 1
showStepIndicator 1

Top undeclared keys — nested inside declared members

path occurrences
widgets.[].value 16
widgets.[].icon 15
widgets.[].trend 15
widgets.[].trendValue 15
widgets.[].options.data 11
widgets.[].options.xField 11
widgets.[].options.yField 11
items.[].description 8
items.[].time 8
items.[].title 8
items.[].variant 8
fields.[].multiple 6
fields.[].rows 5
items.[].icon 4
widgets.[].options.value 4
fields.[].columns 3
fields.[].formula 3
fields.[].precision 3
fields.[].return_type 3
fields.[].summary_field 3
fields.[].summary_object 3
fields.[].summary_type 3
items.[].label 3
items.[].type 3
widgets.[].colSpan 3
widgets.[].rowSpan 3
fields.[].currency 2
fields.[].dimensions 2
fields.[].format 2
fields.[].max 2
fields.[].min 2
fields.[].reference_to 2
items.[].items 2
columns.[].key 1
columns.[].title 1
fields.[].accept 1
fields.[].columnSpan 1
fields.[].defaultChecked 1
fields.[].defaultValue 1
fields.[].maxLength 1

Undeclared keys × does any renderer source mention the identifier?

Coarse triage hint only — see the rendererMentions docblock. A key with 0
mentions is #7077-shaped (nothing reads it); a key with mentions is #6939-shaped
(the renderer reads it, the schema does not declare it). Both still need the call
site read before they become worklist items.

Probe controls — className (must be > 0): 13409; zzzFabricatedControlKey (must be 0): 0 ⇒ probe is live.

key total occurrences in corpora git grep -w hits in packages/*/src (types excluded)
columnSpan 1 0
fullWidth 1 0
itemTemplate 1 0
rowSpan 3 0
summary_field 3 0
summary_object 3 0
totalItems 1 0
defaultChecked 1 1
return_type 3 1
summary_type 3 1
striped 1 2
xField 11 2
optionsFrom 1 3
breadcrumbs 1 4
min_length 1 4
defaultListView 1 5
drawerSide 1 5
showStepIndicator 1 6
singleClickEdit 1 6
trendValue 15 6
resizableColumns 1 8
showViewSwitcher 1 11
filterableFields 1 12
defaultViewType 2 13
yField 11 13
defaultTab 1 22
validations 1 29
searchableFields 2 35
currentPage 2 40
minLength 1 40
showSubmit 76 40
submitButton 3 45
colSpan 3 60
emptyText 1 60
formType 4 76
trend 15 77
precision 3 83
listViews 2 92
searchPlaceholder 2 93
maxLength 1 106
pagination 1 110
reference_to 2 129
pageSize 1 136
operations 1 169
separator 2 179
pattern 1 216
formula 3 225
accept 1 230
dimensions 2 240
multiple 6 293
pages 1 299
events 1 340
currency 2 407
sections 3 413
format 2 598
enabled 2 607
navigation 1 732
change 3 790
content 1 790
time 8 867
required 1 958
max 2 960
actions 8 989
min 2 1062
icon 20 1350
defaultValue 1 1386
mode 1 1437
variant 8 1448
dataSource 9 1465
columns 3 1483
body 1 1695
description 8 1716
rows 6 1860
message 4 2046
action 9 2724
title 11 3291
items 3 3673
data 11 5902
schema 1 6008
key 1 6886
field 1 7647
text 3 7680
value 23 8785
label 3 8799
type 3 11251

Why the nodes that are ALREADY red today are red (top issues)

These are refused by the face as shipped; strict is not what broke them.

component: path — issue nodes
resizable: panels.0.id — invalid_type 7
resizable: panels.1.id — invalid_type 7
carousel: items.0 — invalid_type 5
carousel: items.1 — invalid_type 5
carousel: items.2 — invalid_type 5
flex: direction — invalid_value 5
page: body — invalid_type 5
badge: variant — invalid_value 3
dashboard: (root) — unrecognized_keys 3
dashboard: type — invalid_value 3
filter-builder: conditions — invalid_type 3
filter-builder: field — invalid_type 3
filter-builder: logic — invalid_value 3
filter-builder: operator — invalid_value 3
button: onClick — custom 2
form: onSubmit — custom 2
list-view: objectName — invalid_type 2
object-grid: objectName — invalid_type 2
resizable: panels.2.id — invalid_type 2
select: options.0 — invalid_type 2
select: options.1 — invalid_type 2
select: options.2 — invalid_type 2
alert: variant — invalid_value 1
button: size — invalid_value 1
carousel: items.3 — invalid_type 1
collapsible: content — invalid_type 1
data-table: columns.0.accessorKey — invalid_type 1
data-table: columns.0.header — invalid_type 1
data-table: data — invalid_type 1
detail: groups.0.fields.1.type — invalid_value 1
form: fields — invalid_type 1
form: fields.1.validation.pattern — invalid_type 1
form: fields.2.options.0 — invalid_type 1
form: fields.2.options.1 — invalid_type 1
form: fields.2.options.2 — invalid_type 1
grid: (root) — invalid_type 1
icon: icon — invalid_type 1
list: items — invalid_type 1
object-form: mode — invalid_value 1
object-form: objectName — invalid_type 1

Per component type

type schema nodes strict-refused strict-only red today top-level undeclared keys nested undeclared keys
text TextSchema 637 0 0 0
flex FlexSchema 243 5 0 5
stack StackSchema 154 1 0 1 items×1
button ButtonSchema 134 11 8 3 action×5, events×1, fullWidth×1, text×1
card CardSchema 111 7 7 0 change×3, value×3, dataSource×2, content×1, icon×1
form FormSchema 92 83 79 4 showSubmit×76, submitButton×3, title×1 fields.[].multiple×6, fields.[].rows×5, fields.[].columns×3, fields.[].formula×3, fields.[].precision×3, fields.[].return_type×3, +19 more
icon IconSchema 73 1 0 1
box BoxSchema 72 0 0 0
label LabelSchema 45 1 1 0 required×1
badge BadgeSchema 42 3 0 3 text×2
input InputSchema 38 1 1 0 validations×1
statistic StatisticSchema 33 1 0 1
grid GridSchema 26 1 0 1 actions×1, pagination×1, rows×1, title×1
object-grid ObjectGridSchema 19 4 1 3 dataSource×2, operations×1, resizableColumns×1, singleClickEdit×1, striped×1 pagination.enabled×1, selection.enabled×1, selection.mode×1
container ContainerSchema 18 0 0 0
separator SeparatorSchema 18 0 0 0
toggle ToggleSchema 17 0 0 0
dashboard DashboardComponentSchema 15 13 10 3 widgets.[].value×16, widgets.[].icon×15, widgets.[].trend×15, widgets.[].trendValue×15, widgets.[].options.data×11, widgets.[].options.xField×11, +4 more
avatar AvatarSchema 14 0 0 0
page PageNodeSchema 14 7 1 6 actions×3, breadcrumbs×1 regions.[].body×1
div DivSchema 12 0 0 0
select SelectSchema 11 3 0 3 field×1, optionsFrom×1
checkbox CheckboxSchema 10 0 0 0
data-table DataTableSchema 8 1 0 1 dataSource×1 columns.[].key×1, columns.[].title×1
radio-group RadioGroupSchema 8 0 0 0
date-picker DatePickerSchema 7 0 0 0
empty EmptySchema 7 5 5 0 action×4, message×2
file-upload FileUploadSchema 7 0 0 0
header-bar HeaderBarSchema 7 0 0 0
resizable ResizableSchema 7 7 0 7
scroll-area ScrollAreaSchema 7 0 0 0
sonner SonnerSchema 7 0 0 0
spinner SpinnerSchema 7 0 0 0
switch SwitchSchema 7 0 0 0
toast ToastSchema 7 0 0 0
kanban KanbanSchema 6 0 0 0
tree-view TreeViewSchema 6 0 0 0
aspect-ratio AspectRatioSchema 5 0 0 0
button-group ButtonGroupSchema 5 0 0 0
carousel CarouselSchema 5 5 0 5
combobox ComboboxSchema 5 2 2 0 searchPlaceholder×2, emptyText×1
filter-builder FilterBuilderSchema 5 3 0 3
input-otp InputOTPSchema 5 2 2 0 separator×2
kbd KbdSchema 5 0 0 0
object-form ObjectFormSchema 5 4 3 1 formType×3, sections×3, dataSource×1, defaultTab×1, showStepIndicator×1
progress ProgressSchema 5 0 0 0
table TableSchema 5 0 0 0
timeline TimelineSchema 5 3 3 0 items.[].description×8, items.[].time×8, items.[].title×8, items.[].variant×8, items.[].icon×4, items.[].label×3
alert-dialog AlertDialogSchema 4 4 4 0 actions×4
bar-chart BarChartSchema 4 0 0 0
breadcrumb (not exported by name) 4 0 0 0
chart ChartSchema 4 0 0 0
code-editor CodeEditorSchema 4 0 0 0
object-view ObjectViewSchema 4 4 4 0 defaultViewType×2, listViews×2, searchableFields×2, defaultListView×1, filterableFields×1, navigation×1, +1 more form.drawerSide×1, form.formType×1
textarea TextareaSchema 4 0 0 0
alert AlertSchema 3 1 0 1 message×1
calendar-view CalendarViewSchema 3 0 0 0
chatbot ChatbotSchema 3 0 0 0
list ListSchema 3 2 1 1 itemTemplate×1 items.[].type×3
markdown MarkdownSchema 3 0 0 0
object-gantt ObjectGanttSchema 3 0 0 0
object-map ObjectMapSchema 3 0 0 0
sheet SheetSchema 3 0 0 0
skeleton SkeletonSchema 3 0 0 0
tabs TabsSchema 3 0 0 0
toaster ToasterSchema 3 0 0 0
toggle-group ToggleGroupSchema 3 0 0 0
command CommandSchema 2 0 0 0
dropdown-menu DropdownMenuSchema 2 0 0 0
image ImageSchema 2 0 0 0
list-view ListViewSchema 2 2 0 2 dataSource×2
loading LoadingSchema 2 0 0 0
navigation-menu NavigationMenuSchema 2 2 2 0 items.[].items×2
pagination PaginationSchema 2 2 2 0 currentPage×2, pageSize×1, totalItems×1
view-switcher ViewSwitcherSchema 2 0 0 0
accordion AccordionSchema 1 0 0 0
action ActionSchema 1 0 0 0
app AppComponentSchema 1 1 1 0 pages×1
collapsible CollapsibleSchema 1 1 0 1
context-menu ContextMenuSchema 1 0 0 0
detail DetailSchema 1 1 0 1
detail-view DetailViewSchema 1 0 0 0
dialog DialogSchema 1 0 0 0
drawer DrawerSchema 1 0 0 0
filter-ui FilterUISchema 1 0 0 0
hover-card HoverCardSchema 1 0 0 0
html HtmlSchema 1 0 0 0
menubar MenubarSchema 1 0 0 0
object-kanban ObjectKanbanSchema 1 1 0 1 dataSource×1
popover PopoverSchema 1 1 0 1
sidebar SidebarSchema 1 0 0 0
slider SliderSchema 1 0 0 0
sort-ui SortUISchema 1 0 0 0
tooltip TooltipSchema 1 0 0 0

type values that resolve in no node schema (refused by AnyComponentSchema today)

At a document root:

type occurrences
ui:calendar 5
sidebar-provider 4
action:button 3
article 2
home 2
item 2
module 2
page-header 2
script 2
section 2
aside 1
board 1
component-name 1
donut-chart 1
footer 1
group 1
header 1
main 1
my-component 1
nav 1
object 1
page:tabs 1
pie-chart 1
radar-chart 1
schedule 1
spec-report 1
summary 1
unknown-component 1

At a declared child-node slot:

type occurrences
footer 1
h1 1
header 1
line-chart 1
main 1
my-component 1
nav 1
password 1
ui:calendar 1

Registered component types with zero documents in any corpus

calendar, chatbot-enhanced, chatbot-floating, crud-dialog, object-calendar, object-chart, object-data-table, object-gallery, object-tree, report, report-builder, report-viewer, span

Node schemas not exported by name from @object-ui/types/zod

breadcrumb, object-tree — reachable only through the union, so no consumer can validate one of these alone.

z.lazy exports whose getter rebuilds the schema on every call

ActionSchema, AppMenuItemSchema, FilterBuilderConditionSchema, FilterGroupSchema, MenuItemSchema, NavLinkSchema, NavigationItemSchema, NavigationMenuItemSchema, SchemaNodeSchema, TreeNodeSchemagetter() === getter() is false, so the schema is reconstructed on every parse and no consumer can compare it by identity.

The TypeScript face — where [key: string]: any is declared

packages/types/src declares the index signature at 12 sites, and 120 interfaces extends BaseSchema directly (so they inherit it).

file line declared on
packages/types/src/base.ts 467 BaseSchema
packages/types/src/base.ts 499 ComponentRendererProps
packages/types/src/complex.ts 114 KanbanCard
packages/types/src/data-protocol.ts 68 QueryASTNode
packages/types/src/data.ts 133 QueryParams
packages/types/src/data.ts 1265 AggregateResult
packages/types/src/designer.ts 514 UnifiedViewConfig
packages/types/src/designer.ts 648 DashboardConfig
packages/types/src/form.ts 1109 FormField
packages/types/src/objectql.ts 110 ListViewGalleryConfig
packages/types/src/objectql.ts 120 ListViewTimelineConfig
packages/types/src/ui-action.ts 827 ActionContext

Walker limits — shapes strict could not close

  • opaque custom node — strict cannot reach inside it: (inline custom)
  • opaque function node — strict cannot reach inside it: (inline function)
  • opaque transform node — strict cannot reach inside it: (inline transform)

Authored-module loading (corpus 3)

  • apps/console/src/pages/developer/hooks/useApiDiscovery.ts: Cannot find module '/home/user/objectui-issue-7581/apps/console/node_modules/@object-ui/app-shell/dist/index.js' imported from /home/user/objectui-issue-7581/apps/console/src/pages/developer/hooks/useApiDiscovery.ts

Modules that PRINT when imported (console muted for the run): packages/types/examples/zod-validation-example.ts


🤖 Generated with Claude Code

https://claude.ai/code/session_013uAaxiwgYDybsTNV9xwa1M


Generated by Claude Code

…(objectui#7581)

The first deliverable of the #5250 ruling is a number, not a fix. This adds a
reproducible throwaway that derives a strict twin of every node schema in
`@object-ui/types/zod` (in memory — no schema changes, no gate wiring, and
`.passthrough()` on `BaseSchemaCore` is untouched) and runs three corpora
through it, counting undeclared keys per component.

The traversal is schema-guided rather than "any object with a `type` key": a
field definition `{ name: 'amount', type: 'currency' }` and a node
`{ type: 'text' }` are structurally identical and half the field-type
vocabulary collides with a node type, so nodes are found only where the schema
declares a `SchemaNodeSchema` slot. The registry is taken from the arms of
`AnyComponentSchema` for the same reason: an export-order registry hands the
`text` and `select` node types to `FilterFieldSchema`.

Output is deterministic and pins the `main` SHA it ran against, so the numbers
can be re-derived after the declaration repairs of #6939 land and move them.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013uAaxiwgYDybsTNV9xwa1M
@baozhoutao
baozhoutao marked this pull request as ready for review September 6, 2026 03:03
@baozhoutao
baozhoutao added this pull request to the merge queue Sep 6, 2026
Merged via the queue into main with commit b8e3f83 Sep 6, 2026
33 checks passed
@baozhoutao
baozhoutao deleted the claude/issue-7581-strict-authoring-face-measurement branch September 6, 2026 03:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants