Skip to content

fix(metadata-protocol): insertManyData reports the dropped-field union at BATCH level instead of naming rows it cannot identify - #17688

Merged
os-musk merged 5 commits into
mainfrom
claude/issue-17290-insertmany-dropped-field-row-attribution
Sep 11, 2026
Merged

fix(metadata-protocol): insertManyData reports the dropped-field union at BATCH level instead of naming rows it cannot identify#17688
os-musk merged 5 commits into
mainfrom
claude/issue-17290-insertmany-dropped-field-row-attribution

Conversation

@os-musk

@os-musk os-musk commented Sep 11, 2026

Copy link
Copy Markdown
Collaborator

Fixes #17290

Clause-②: yes

insertManyData attributed a batch-union dropped-field name to every row that supplied it. Maintainer ruling C's per-row hook exemption falsifies that inference, so the union is now reported where it is true — on the response — and no row is named.

The defect, and why the inference cannot be rescued

Every create-side strip is the engine's, and its onFieldsDropped event is the UNION over the batch; the listener signature carries no row index. This seam reconstructed a row set from that union:

const supplied = request.records?.[i] ?? {};
const mine = [...engineDropped].filter((f) => f in supplied);

on the stated premise that "the strip only removes keys the ROW ITSELF supplied". Ruling C (#14147) put the static-readonly strip INSIDE engine.insert, AFTER the beforeInsert hooks, where it exempts keys a hook itself assigned — recorded per row (packages/objectql/src/engine.ts, hookWrittenKeys: rowHookWrittenKeys[i]; packages/objectql/src/validation/rule-validator.ts:1268, if (hookWrittenKeys?.has(name)) continue;). "Supplied N" and "dropped N" are therefore different sets, in two directions:

  • a row whose hook re-assigned the key kept and wrote it, and was still handed a droppedFields naming it — on an outcome whose record carries the written value;
  • a row the batch culled before the strip ran (ok: false) dropped nothing at all, and supplying the name was enough to have it named.

The outcome's own record cannot repair it, which is why the card's suggested post-hoc check is not the cheaper route. A stripped readonly field is RE-DEFAULTED over exactly the keys the strip took (engine.ts, #3043's contract), and a stripped autonumber is refilled by applyAutonumbers afterwards — so the key is PRESENT on the row that really did drop it. Measured, in engine-autonumber-runtime-owned.test.ts against a real engine: both outcomes come back carrying account_number (['ACC-0001', 'ACC-0002']) when only the second row forged one. A "is the key still there?" check would delete TRUE attributions and leave the hook-exempt false one standing. Comparing values fails on the exact case hookWrittenKeys was built for — the hook assigning the value the caller also sent.

⇒ Per triage: attribute per row, or say plainly that the set is batch-level and name no rows — and ⛔ never keep row names under a caveat. The honest set ({rows whose payload carried N} minus {rows whose beforeInsert hook assigned N}) is computed per row upstream and does not cross this seam, so the second exit is the correct one. Restoring row precision would mean giving the engine's drop report a per-row channel, not a reconstruction at the call site.

The change

FROM  { object, outcomes: [{ ok, record, droppedFields? }, …] }
TO    { object, outcomes: [{ ok, record }, …], droppedFields? }

The events are now collected whole and merged the way createManyData merges them, which also restores each event's own reason (the old flatten-into-a-Set relabelled everything 'readonly'). updateManyData and batchData keep per-row droppedFields untouched — they earn it mechanically, one engine call per row.

The docblock dies with the code. Enumerated by CLAIM rather than by spelling — the census instruction #16933's dispatch introduced (ruling 5), which is what produced this card in the first place:

carrier claim corrected
metadata-protocol/src/protocol.tsinsertManyData docblock the authorising text; replaced with why no row can be named
metadata-protocol/src/protocol.tscreateManyData comment its parenthetical said insertManyData "recovers row precision from the same union"
metadata-protocol/src/protocol.tsmergeDroppedFieldEvents docblock "the per-row insertMany/batch paths carry their own per-row droppedFields"
metadata-protocol/src/protocol.dropped-fields.bulk.test.ts — header the channel table's insertManyData row
objectql/src/engine.tsinsertMany docblock "a caller holding the input rows can attribute each name back to the rows that carried it (insertManyData does exactly that)"
spec/src/api/protocol.zod.tsCreateManyDataResponseSchema.droppedFields.describe() the same claim in a string printed AT the customer

This agrees with #16933's landed prose correction (PR #17293, commit 439331895), which is the card triage said to read first: that PR kept the { object, records, count } "no per-row slot" reason and corrected only the falsified one. Located by content — git log -S 'schema-uniform' -- packages/metadata-protocol/src/protocol.ts; the PM's git log --grep=16933 zero was a dead instrument, since squash subjects need not carry the number.

The pins are REPLACED, not amended

Three pin sites restated the same inference in their own comments and titles, and their engine doubles had no hook-write concept at all — looser than the producer, so each was green through exactly the shape it was written for. The double in protocol.dropped-fields.bulk.test.ts now models ruling C's exemption and the re-default, and the block gains the case the card asked for: a hook exempting one row of a batch.

Verification

Head at measurement: a38c0baab5. origin/main moved under this branch mid-run (49cd71548e6465cc0a7c) and was merged in, never rebased; the new commit touches packages/plugins/plugin-approvals only, so the serial file has no second edit on it. Build and test run through scripts/pm/os-verify-lock.sh; every verdict below is quoted from the gate's own line, never from a bare $? after a pipe.

  • pnpm --filter '@objectstack/metadata-protocol...' --filter '@objectstack/objectql...' --filter '@objectstack/spec...' build --concurrency=2VERDICT command-exit 0
  • ... run typecheck on the three packages — VERDICT command-exit 0
  • pnpm --filter @objectstack/metadata-protocol run testTest Files 176 passed | 2 skipped (178) · Tests 2529 passed | 12 skipped (2541)
  • pnpm --filter @objectstack/objectql run testTest Files 296 passed (296) · Tests 4959 passed (4959)
  • pnpm --filter @objectstack/spec run testTest Files 473 passed (473) · Tests 13432 passed (13432)
  • the three pin files re-run on the post-merge head — Tests 22 passed (22) + Tests 32 passed (32)
  • repo-level lint, not narrowed: pnpm exec eslint . --no-inline-config --format json at a38c0baab56633 files linted (population read from eslint's own config, count read from the JSON report), 0 errors, 0 warnings
  • gate families: scripts/pm/dispatch-gates.mjs --commands then --ran with an exit code recorded per family — 112 derived famil(ies) accounted for — 111 run, 1 NOT-MEASURED
  • ratchet families re-run on the post-merge head after the last commit: check:type-check-coverageOK — 76/80 workspace packages type-checked; check:type-check-debtOK — 5 ledger entr(ies) re-measured in 142.0s, 55 raw tsc error(s) total, none above its recorded number

NOT MEASURED: pnpm check:dual-build-cjs-loads, reason: exit 3 PREREQUISITE NOT MET — it reads built output and 8 packages in this checkout have no dist/; clearing it needs the whole-tree build that CI's Build Core job performs. ⛔ Recorded as not measured, never as a pass.

Three gates needed work rather than just a run, and each is in the diff:

  • check:docs went red because a .describe() moved: content/docs/references/api/protocol.mdx is generated and is regenerated here (gen:schema + gen:docs, run on a committed non-merge state; authorable-surface.base.json is untouched, and the regeneration is a one-line change).
  • check-adr-0087-registration demanded a disposition for the **BREAKING** token. The changeset carries not-required (no-migration-prescription) with its argument: no authorable key, no Zod schema and no stored shape moves — packages/spec declares no response schema for this face at all — so objectstack migrate meta has nothing to visit and the compiler at the consumer's own call site is the channel. The changeset therefore describes the break in prose rather than as a rewrite table, which is the shape this repo already uses for a compiler-carried break.
  • check-engine-split-ratio refused on a shallow clone (exit 2, "a ratio derived here would be real, plausible and WRONG"). Deepened per its own remedy (git fetch --shallow-since=2026-06-06) and re-run green.

Reverse verification (one-off, restored)

With the repair committed, the pre-repair reconstruction was re-injected into packages/metadata-protocol/src/protocol.ts behind a unique marker and the pins re-run. The tests import ./protocol.js relative to the same package, so vitest resolves the mutation from src and no rebuild is involved.

  • on-disk proof: injected marker count 1; mutated blob a9a4d568… differs from the HEAD blob 7b43df47…
  • mutated run: Test Files 2 failed (2) · Tests 4 failed | 18 passed (22), including the defect itself —
    AssertionError: a written value must never be reported as dropped: expected { ok: true, …(2) } to not have property "droppedFields" / Received: [{ fields: ["approval_status"], object: "approval_case", reason: "readonly" }]
  • restored under an EXIT INT TERM trap via git checkout HEAD -- PATH: git diff HEAD empty, current blob 7b43df47… equals the HEAD blob, marker residue 0

Acceptance notes

  • ⚠️ Triage's routing line named packages/objectql / the create path; the repair lands in packages/metadata-protocol. Confirmed independently: insertManyData is defined at packages/metadata-protocol/src/protocol.ts, and packages/objectql/src/engine.ts carried only a comment about it. The two packages/objectql files here are a falsified docblock and a pin that asserted the removed behaviour — consequences of the repair, not a widening of it.
  • The cross-package consumer the dispatch named, packages/rest/src/import-runner.ts, does not read droppedFields: its own insertManyData interface declares outcomes: Array<{ ok, record?, error? }> and the call site reads only .outcomes. Clause-② is yes on the mechanical floor anyway — a new key on a published payload.
  • noted, not filed: packages/spec declares no Zod schema for the insertManyData response at all (only CreateManyDataResponseSchema prose mentions it), so this surface's shape is carried by TypeScript alone. Successor: none identified — flagged for the seat rather than filed, since "should this face have a declared response schema" is a contract question and not a defect I measured.

🤖 Generated with Claude Code

https://claude.ai/code/session_01RuoNSXUbBoWHkNS4AknTrM


Generated by Claude Code

@github-actions

github-actions Bot commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

This PR changes 3 package(s): @objectstack/metadata-protocol, @objectstack/objectql, @objectstack/spec, touching 9 documentable anchor(s).

8 hand-written doc(s) NAME something this change touched and may need an implementation-accuracy re-verification:

  • content/docs/api/client-sdk.mdx (via createMany (sdk, the bare tail of client method data.createMany, bound to POST /api/v1/data/:object/createMany), data.batch (sdk, the route ledger binds it to POST /api/v1/data/:object/batch, selected by route anchor /:object/batch), data.createMany (sdk, the route ledger binds it to POST /api/v1/data/:object/createMany, selected by route anchor /:object/createMany), /:object/batch (route, bridged from symbol createManyData — its route source's handler names it))
  • content/docs/api/data-api.mdx (via createMany (sdk, the bare tail of client method data.createMany, bound to POST /api/v1/data/:object/createMany), /:object/batch (route, bridged from symbol createManyData — its route source's handler names it), /:object/createMany (route, bridged from symbol createManyData — its route source's handler names it))
  • content/docs/api/wire-format.mdx (via /:object/batch (route, bridged from symbol createManyData — its route source's handler names it))
  • content/docs/concepts/metadata-lifecycle.mdx (via ObjectStackProtocolImplementation (symbol, a top-level class))
  • content/docs/getting-started/quick-reference.mdx (via /:object/batch (route, bridged from symbol createManyData — its route source's handler names it))
  • content/docs/kernel/contracts/data-engine.mdx (via createManyData (symbol, a method of class ObjectStackProtocolImplementation), insertManyData (symbol, a method of class ObjectStackProtocolImplementation))
  • content/docs/kernel/services-checklist.mdx (via createManyData (symbol, a method of class ObjectStackProtocolImplementation))
  • content/docs/protocol/kernel/http-protocol.mdx (via createMany (sdk, the bare tail of client method data.createMany, bound to POST /api/v1/data/:object/createMany), /:object/batch (route, bridged from symbol createManyData — its route source's handler names it))

4 release-owned page(s) also name something this change touched. These are read-only:

  • content/docs/releases/implementation-status.mdx (via createMany (sdk, the bare tail of client method data.createMany, bound to POST /api/v1/data/:object/createMany), /:object/batch (route, bridged from symbol createManyData — its route source's handler names it), /:object/createMany (route, bridged from symbol createManyData — its route source's handler names it))
  • content/docs/releases/v16.mdx (via ObjectStackProtocolImplementation (symbol, a top-level class), createMany (sdk, the bare tail of client method data.createMany, bound to POST /api/v1/data/:object/createMany), data.batch (sdk, the route ledger binds it to POST /api/v1/data/:object/batch, selected by route anchor /:object/batch))
  • content/docs/releases/v17/17-0.mdx (via ObjectStackProtocolImplementation (symbol, a top-level class))
  • content/docs/releases/v17/17-4.mdx (via createManyData (symbol, a method of class ObjectStackProtocolImplementation), insertManyData (symbol, a method of class ObjectStackProtocolImplementation), /:object/batch (route, bridged from symbol createManyData — its route source's handler names it))

content/docs/releases/ is RELEASE-OWNED (AGENTS.md "Documentation Guardrails"): release
notes are written centrally at release time, and a code PR that edits them is the exact PR
that guardrail exists to stop. They are still audited — read-only. If one of them is actually
wrong, file an issue or open a dedicated docs-only PR; do not edit it here.

What this run could not see
  • 1 anchor(s) matched too much of the corpus to be a work list: ObjectQL (symbol, 68 pages)
  • the SDK route bridge reached 60 of 215 client-bound route-ledger rows — the other 155 have no registrar path: tail to select them, so pages documenting THEIR client methods cannot appear above, on this or any run. Of those 155: 0 are remediable by widening that discovery convention (an in-repo file declares the path; the convention did not scan it); 55 are structural — on a ledger where NOT ONE row is declared in-repo, so no discovery change reaches them at any price; 100 are undecided (no in-repo declaration, on a ledger that has other in-repo registrars — absence and an unreadable spelling are not distinguishable here). The rows themselves: node scripts/docs-audit/affected-docs.mjs --bridge-coverage
  • a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.

Coarse fallback — 137 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json 6465cc0a7c83ba60503bfd4ddd907d35220e4244packageMentionDocs.

Which tree this was computed on

This run read content/docs from a5a07925fba72237d6ad15f3d842a05b971756fb — the merge of head a38c0baab57a287c6e022acfff59d1147222fb75 into base 6465cc0a7c83ba60503bfd4ddd907d35220e4244, which is what actions/checkout gives a pull_request run. Not the PR head.

A worktree cut from an older main holds a different content/docs, so re-deriving there can legitimately return a different list — that is a different tree, not a wrong row. To answer on the same tree:

# while this PR is open — GitHub drops the merge commit once it closes
git fetch origin a5a07925fba72237d6ad15f3d842a05b971756fb && git checkout a5a07925fba72237d6ad15f3d842a05b971756fb
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin 6465cc0a7c83ba60503bfd4ddd907d35220e4244 a38c0baab57a287c6e022acfff59d1147222fb75 && git checkout -B drift-repro 6465cc0a7c83ba60503bfd4ddd907d35220e4244 && git merge --no-ff a38c0baab57a287c6e022acfff59d1147222fb75

node scripts/docs-audit/affected-docs.mjs --json 6465cc0a7c83ba60503bfd4ddd907d35220e4244

⚠️ That checkout carried uncommitted changes, so the commit above does not fully identify what was read.

Advisory only, and a precision-first one (#9192): a page is listed because it names a
symbol, wire route or SDK method this diff touched — not because it mentions a changed
package. Each row says which anchor put it there, so a wrong row is reportable rather than
merely annoying. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs 6465cc0a7c83ba60503bfd4ddd907d35220e4244 → pass the list as
args.docs, on the commit named under Which tree this was computed on.

@github-actions github-actions Bot added documentation Improvements or additions to documentation tooling labels Sep 11, 2026

os-musk commented Sep 11, 2026

Copy link
Copy Markdown
Collaborator Author

Contract review

Head reviewed: a38c0baab5. Card #17290 · domain:engine execution PM seat · R1 · 2026-09-11.
Tier: 默认判断档自审加门禁 (SKILL.md:641), ⛔ outside the downgrade fuse. ⛔ Readings taken by this seat from the tree.

① Derived judgments

# change judgment
1 insertManyData's response type: droppedFields moves off outcomes[i] onto the response object correct, and Clause-②: yes is right — a published TypeScript response shape changes for every consumer. Read from the diff: Array<{ ok; record?; error?; droppedFields? }>Array<{ ok; record?; error? }>; droppedFields?.
2 the false inference const mine = [...engineDropped].filter((f) => f in supplied) is DELETED, and its docblock with it triage's ruling honoured exactly — 「attribute per row, or say plainly the set is batch-level and name no rows」, and ⛔ not 「keep row names and add a caveat」. ⭐ Killing the docblock matters as much as the code: it is what authorised the inference.
3 the second exit was taken because the per-row set is NOT reachable at this seam measured, and the measurement is the good part. The engine emits ONE onFieldsDropped carrying the batch union with no row index; and outcomes[i].record cannot substitute, because a stripped readonly key is re-defaulted and a stripped autonumber refilled — so a post-hoc record check would ⛔ delete TRUE attributions while leaving the hook-exempt false one standing. ⇒ the second exit is the correct answer, ⛔ not a lesser fix.
4 the engine double gained a hook-write concept it never had ✅ ⭐ the sharpest finding in the round: the double 「was green through exactly the shape it was written for」. A pin that cannot model the exemption cannot fail on it.
5 ⚠️ packages/spec/src/api/protocol.zod.ts +4/−2 ⚠️ ACCEPTED AND DECLARED — a cross-package touch into another lane's ownership. Verified from the diff that it is prose only: a .describe() STRING inside CreateManyDataResponseSchema, ⛔ no key, ⛔ no accept set, ⛔ no schema shape. ⭐ Accepted because this PR's own behaviour change is what makes that sentence false, and shipping knowingly-false customer-facing prose is the worse outcome. ⇒ put to triage as the SECOND instance of the precedent question already open from #17617 (does a cross-package addition made necessary by the diff need the cross-domain exception path?). ⛔ Not settled here.
6 content/docs/references/api/protocol.mdx +1/−1 generated, ⛔ not hand-edited — the regeneration the changed .describe() string forces, produced by the repo's own gen:schema + gen:docs. One line, consistent with row 5.

② Semver grading

@objectstack/metadata-protocol: minor + BREAKING in prose · @objectstack/objectql: patch · @objectstack/spec: patch. ✅ Consistent, and Check Changeset is green on this head — a minor exists on a package whose src/** the diff moves, which is exactly the limb that reddened the sibling PR #17694.
ADR-0087 disposition not-required (no-migration-prescription) ⇒ ✅ sound on its own terms, and its strongest limb is checkable: packages/spec declares no response schema for this face at all, so migrate meta has nothing to visit and the compiler is the only channel that reaches consumers. ⭐ That is also why row 5's spec file is a description string rather than a schema.

③ Boundary flags

No open_questions — and the round earned that by falsifying an assumption instead of inheriting it. Zone 2 assumption D (「#16933 landed」) was flagged UNMEASURED with an explicit warning that this seat's git log --grep=16933 returning 0 was a dead instrument, ⛔ not evidence. ⇒ the round located it by content (git log -S 'schema-uniform', after confirming that spelling is absent today, with a positive control of 6 hits on the same file) and found PR #17293 / 439331895. ⭐ Assumption D is FALSIFIED and the seat's zero was correctly not inherited. Assumptions A, B and C (the last flagged a READING) all re-measured and CONFIRMED, C at rule-validator.ts:1268.

Triage's routing line was independently measured wrong by the round too — it named packages/objectql; the repair is in packages/metadata-protocol. Two seats measuring the same thing separately is what made that safe.

⚠️ One self-reported flawed control, and reporting it is the right behaviour: the round's second on-disk ablation counter anchored on a string that also occurs in createManyData, so it read 1 where 0 was expected. It said so, and named the load-bearing evidence instead — the unique marker count and blob-hash inequality against the HEAD blob. ⭐ ⛔ A control that cannot fire is voided, ⛔ not quietly reported.

Two out_of_scope_findings accepted as noted-not-filed, both honest negatives: packages/spec declares no Zod response schema for this face (a contract question, ⛔ not a measured defect), and packages/rest/src/import-runner.ts — the consumer the deleted docblock named as the reason per-row precision mattered — ⭐ never read droppedFields on either shape. ⇒ the old justification dies with the docblock and nothing measured breaks.

⚠️ origin/main moved mid-run (49cd71548e6465cc0a7c) and was MERGED in, ⛔ never rebased — the hot-file discipline this seat ordered, followed. It touches plugin-approvals only, so the serial file protocol.ts carries no second edit.

Independence pair

Implemented-by: claude/issue-17290-insertmany-dropped-field-row-attribution
Reviewed-by: session_01RuoNSXUbBoWHkNS4AknTrM

⚠️ Declared: NOT an independent review. The executor was a mode:subagent dev of the reviewing session — it records its branch because a subagent has no session of its own, so the pair does ⛔ not mechanically read as 「两者同 session」, and letting that stand as independence would be dishonest. ⭐ This is the in-seat default-tier self-review plus gates this lane runs by design (SKILL.md:641; #17294), ⛔ not a 达档 ruling.

Verdict

✅ PASS

Landing pre-checks, all three by this seat: ① this comment · ② --pair 17688 exit 0 · ③ 34 distinct check names, 0 non-green, 0 still running (latest run per name).

⇒ Both carriers stripped in this same stroke.

domain:engine 执行 PM 席 · R1 · session_01RuoNSXUbBoWHkNS4AknTrM · 2026-09-11


Generated by Claude Code

@os-musk
os-musk marked this pull request as ready for review September 11, 2026 15:02
@os-musk
os-musk enabled auto-merge September 11, 2026 15:02
@os-musk
os-musk added this pull request to the merge queue Sep 11, 2026
Merged via the queue into main with commit ada2869 Sep 11, 2026
45 checks passed
@os-musk
os-musk deleted the claude/issue-17290-insertmany-dropped-field-row-attribution branch September 11, 2026 15:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation size/m tests tooling

Projects

None yet

2 participants