Skip to content

fix(metadata-protocol): a stopped or rolled-back bulk batch names the row that actually failed - #19700

Merged
huangyiirene merged 5 commits into
mainfrom
claude/issue-19452-batch-causal-row
Sep 22, 2026
Merged

huangyiirene merged 5 commits into
mainfrom
claude/issue-19452-batch-causal-row

Conversation

@huangyiirene

Copy link
Copy Markdown
Collaborator

Fixes #19452

Clause-②: no

A stopped or rolled-back bulk batch named a causal row that did not fail, and called the real error 「unknown error」 while that error was sitting in the same array.

The defect

reconcileStoppedBatch and buildRolledBackBatchResponse both located the causal row with findIndex(r => !r.success). That encoded one invariant: !success means this row failed, and it carries errors[0].

PR #19432 broke that invariant deliberately and correctly: a row that MATCHED and was NOT removed now answers success: false with no errors entry, because a surviving record is an outcome, not a fault. So the locator could land on that survivor, errors?.[0]?.message was undefined, and the message named the wrong index while falling back to 「unknown error」.

Reproduced before the fix, verbatim

Taken by neutralising the fix on the final tree: packages/metadata-protocol/src/protocol.ts restored to the blob origin/main holds (be9dd23ad9c865fbcc74ccf20bcc3f633f23845e), proved on disk by blob hash, then the pins run. Received strings, copied out of the run:

run row code message on the unfixed tree who really ended the run
deleteMany ['t1'(survives), 'missing'(throws), 't3'] 2 NOT_ATTEMPTED record 0 failed — unknown error; the batch stopped there. Set options.continueOnError to process the remaining records. record 1
batchData delete, same three rows 2 NOT_ATTEMPTED identical record 1
batchData atomic ['t3', 't1'(survives), 'missing'(throws), 't2'] 0 ROLLED_BACK record 1 failed — unknown error record 2
same run 3 NOT_ATTEMPTED atomic batch aborted by record 1 record 2
batchData atomic ['t1', 't2'(survives), 't3'] 0 and 2 ROLLED_BACK record 1 failed — unknown error nothing failed at all

The fix

Both builders now call one shared locator, locateBatchCause, which finds the causal row by its recorded fault: the row's errors[] entry.

Why that discriminator cannot drift back the way the boolean did. success is the envelope's outcome bit and its false arm is open by construction — it means "this row is not a success", so every new non-success ending widens it for free, which is exactly what happened. errors is not a second boolean:

  • its declared meaning is a failure. BatchOperationResultSchema.errors is documented as "Array of errors if operation failed", and the v17 ADR-0087 migration entry publishes row.errors?.[0]?.message / row.errors?.[0]?.code to consumers as that read;
  • its contents are a closed vocabulary. Every entry must carry an ApiError.code from StandardErrorCode union ERROR_CODE_LEDGER; an unregistered code fails BatchOperationResultSchema.parse. Giving a non-fault ending an errors[] entry is therefore a ledger widening in packages/spec — which is precisely the step BOTH survivor sites declined to take, in writing, and the step that would have to be taken deliberately for this locator to start lying;
  • ApiError.message is required, so a located cause always has text. The 「unknown error」 fallback is deleted, not merely unreached: the string no longer appears in either message template.

The scan runs from the END of the attempted rows, because a run ends AT the row it stops on — every stop is a break in a loop's catch, immediately after that row was pushed. A fault that does not stop the run (the Unknown operation: arm records one and keeps going) therefore cannot shadow the row that did.

One ending has no fault to quote at all: an atomic batch aborted by a lone survivor, where runAtomicBatch rolls back on failed > 0 and nothing ever threw. There the message names the row that did not succeed — record 1 did not succeed — instead of inventing a failure. That is the only place !success is still read, and it is read for the question that boolean does answer: "which row stopped this batch committing", never "which row failed".

Acceptance

1. Located by a fault, not by !success — above. 2. The negative case is pinnedpackages/metadata-protocol/src/protocol.batch-causal-row.test.ts, 8 tests. Its central assertion is an agreement between the message and the rows beside it, read out of the response rather than hard-coded: the message names record N failed for the N that carries an error, contains that row's own error text verbatim, and never contains the string unknown error. 3. Both builders — one locator, two call sites; neither can be fixed apart from the other. 4. All three bulk faces:

face arm covered how
batchData (delete verb) non-atomic and atomic pinned — it is one of the two faces that can produce an errors-less non-success row
deleteManyData non-atomic pinned — the other such face
deleteManyData atomic reasoned — same two builders, same runAtomicBatch; the atomic delete pin runs through batchData, whose loop pushes the identical survivor row
updateManyData both reasoned, and the reading is assertedrunUpdateManyLoop has no producer of a non-success row without errors (every push is success: true or a toRowApiError row), so !success and "carries a fault" still coincide there. The pin asserts that reading directly: every non-success row in an updateMany response carries an errors entry. It shares the two builders, so the attribution moves with them, which the same test also checks

Ablation

Neutralised on the final tree (merge commit 56e5fd90c), mutation proved on disk by blob hash 2204cd2 -> be9dd23, restored, restore proved: blob back to 2204cd2, git diff HEAD zero bytes, whole-tree git status --porcelain zero lines.

  • 5 of the 8 new pins go red, one per message site plus the two non-atomic faces.
  • 3 stay green — they are the positive controls: the same assertions over batches with no survivor in them, so a locator that simply stopped naming anything could not pass this file.
  • All 5 sibling batch suites stay green (68 tests) — no existing pin covered this, which is why it shipped.

Files outside the declared surface, declared rather than quietly widened

The dispatch scoped this to packages/metadata-protocol/src/. Two files outside it are in the diff, both mechanical and both demanded by the repo's own gates for the in-surface change:

  • .changeset/19452-batch-causal-row-located-by-fault.md — required by the post-task checklist and by check:empty-changeset. Measured rather than assumed: @objectstack/metadata-protocol is not private, its files[] ships dist, and the changed symbol is in the built output (locateBatchCause present in dist/index.js; a nonsense control string returns zero from the same grep). patch.
  • scripts/engine-double-contract.pinned.jsoncheck:engine-double-contract exited 1 on the new pin file with its own prescription, --write and commit. The regeneration reports 3 rows added, 0 lost, and every added row names the new test file (3 lines match the file name, 3 lines match .test.ts — the same count, so no other file moved).

Verification

  • dispatch-gates.mjs --commands --repo objectstack-ai/objectstack on the merged tree (56e5fd90c): 69 families, all 69 run, every one exit 0. Reconciled with --ran carrying exit codes: 69 derived, 69 run, 0 NOT-MEASURED, 0 UNRUN. Three families first answered exit 3 (PREREQUISITE NOT MET — dual-build-cjs-loads, lean-entry-closure, type-check-debt); each was discharged by building what it named (turbo run build over all packages, 72/72) and re-running, never by calling it inapplicable.
  • pnpm --filter @objectstack/metadata-protocol test — 2653 passed, 19 skipped, 0 failed. typecheck — exit 0.
  • pnpm lint — the whole-repo eslint . --no-inline-config, exit 0. Run in full, so no narrowing needs declaring.
  • origin/main merged before this reading; the gates above were run on the merged tree.

Acceptance notes

Noted here, not filed, per this seat's standing rule that it files nothing:

  • protocol.batch-not-attempted.test.ts asserts the causal index with expect(message).toContain('1'). The assertion is satisfied by any 1 anywhere in the string, so it is much weaker than it reads; it happens to be correct today. Test-quality observation, no live defect, and this PR does not touch that file.
  • runBatchDataLoop's Unknown operation: arm records a VALIDATION_FAILED row per record and keeps going even with continueOnError absent, because nothing is thrown. BatchOptionsSchema.continueOnError declares the opposite default. batchData does not parse the verb at its own door, so an in-process caller can reach the arm; the effect is a longer results array and nothing else — no write happens and the counters still reconcile. Reported to the PM as a contract-violation candidate with its seam rather than filed here.

Generated by Claude Code

…corded fault, not by `!success`

`reconcileStoppedBatch` and `buildRolledBackBatchResponse` both found the row
that ended a bulk run with `findIndex(r => !r.success)`, encoding the invariant
"`!success` means this row failed and carries `errors[0]`". #19412 broke that
invariant on purpose: a row that MATCHED and was deliberately NOT removed now
answers `success: false` with no `errors` entry, because a surviving record is
an outcome rather than a fault.

So the locator could land on that survivor: the message named the wrong index
and called the real error — sitting in the same array — "unknown error".

Both builders now share one locator that reads the row's `errors[]` entry, the
one per-row value whose declared meaning is a fault (`BatchOperationResultSchema`
documents it as "Array of errors if operation failed", and its codes are drawn
from the closed StandardErrorCode + ledger vocabulary). `ApiError.message` is
required, so the "unknown error" fallback is deleted rather than merely
unreached. When nothing recorded a fault at all — an atomic batch aborted by a
lone survivor — the message names the row that did not succeed instead of
inventing a failure.

Claude-Session: https://claude.ai/code/session_01NcPSwnmJHczmTu6FG7NMjE
Co-authored-by: Claude <noreply@anthropic.com>
…ledger

Regenerated with `node scripts/check-engine-double-contract.mjs --write`:
3 rows added, 0 lost, all naming the new pin file.

Claude-Session: https://claude.ai/code/session_01NcPSwnmJHczmTu6FG7NMjE
Co-authored-by: Claude <noreply@anthropic.com>
@github-actions github-actions Bot added size/m documentation Improvements or additions to documentation tests tooling labels Sep 22, 2026
@github-actions

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

This PR changes 1 package(s): @objectstack/metadata-protocol, touching 6 documentable anchor(s).

4 hand-written doc(s) NAME something this change touched and may need an implementation-accuracy re-verification:

  • content/docs/api/client-sdk.mdx (via NOT_ATTEMPTED (literal, a string literal in buildRolledBackBatchResponse), ROLLED_BACK (literal, a string literal in buildRolledBackBatchResponse))
  • content/docs/api/data-api.mdx (via NOT_ATTEMPTED (literal, a string literal in buildRolledBackBatchResponse), ROLLED_BACK (literal, a string literal in buildRolledBackBatchResponse))
  • content/docs/api/wire-format.mdx (via NOT_ATTEMPTED (literal, a string literal in buildRolledBackBatchResponse), ROLLED_BACK (literal, a string literal in buildRolledBackBatchResponse))
  • content/docs/concepts/metadata-lifecycle.mdx (via ObjectStackProtocolImplementation (symbol, a top-level class))

2 release-owned page(s) also name something this change touched. These are read-only:

  • content/docs/releases/v16.mdx (via ObjectStackProtocolImplementation (symbol, a top-level class))
  • content/docs/releases/v17/17-0.mdx (via ObjectStackProtocolImplementation (symbol, a top-level class), NOT_ATTEMPTED (literal, a string literal in buildRolledBackBatchResponse), ROLLED_BACK (literal, a string literal in buildRolledBackBatchResponse))

content/docs/releases/ is RELEASE-OWNED (AGENTS.md "Documentation Guardrails"): release
notes are written centrally at release time, and a code PR that edits them is the exact PR
that guardrail exists to stop. They are still audited — read-only. If one of them is actually
wrong, file an issue or open a dedicated docs-only PR; do not edit it here.

What this run could not see
  • the SDK route bridge reached 60 of 215 client-bound route-ledger rows — the other 155 have no registrar path: tail to select them, so pages documenting THEIR client methods cannot appear above, on this or any run. Of those 155: 0 are remediable by widening that discovery convention (an in-repo file declares the path; the convention did not scan it); 55 are structural — on a ledger where NOT ONE row is declared in-repo, so no discovery change reaches them at any price; 100 are undecided (no in-repo declaration, on a ledger that has other in-repo registrars — absence and an unreadable spelling are not distinguishable here). The rows themselves: node scripts/docs-audit/affected-docs.mjs --bridge-coverage
  • a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.
  • a key NAME is not a key, so the hand re-read the line above prescribes can land on the wrong schema. The same spelling is authorable on one governed type and a [REMOVED] tombstone on another for each of active, aria, joins, objects, template, tools and version (censused on [finding] tools is a key on BOTH AgentSchema (tombstoned, dead) and SkillSchema (live, cloud-attested), so a name-based search attributes skill examples to the agent key — it produced a false stop-the-line alarm on PR #19059 #19093 over the liveness ledger's governed types, top-level keys); nothing in a search result distinguishes the two, so a grep hit on a LIVE example reads as evidence about the DEAD key. Measured on fix(spec): the agent.tools liveness row says dead — it claimed live on a key the schema tombstoned #19059: content/docs/ai/agents.mdx was reported as contradicting the agent.tools tombstone over its tools: example at :161, which is inside the defineSkill({ block opened at :155 — the page was already correct. Settle ownership by PARSING the value against both schemas, never by the name: that literal PASSES SkillSchema, and as an AgentSchema it FAILS at tools with the tombstone prescription. ⛔ These names are not the whole class — a key retired through a .strict() guidance map leaves no tombstone in the walked shape and none of them here (tool.category, live as AIToolDefinition.category).

Coarse fallback — 11 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json 7e1b048a1dbd7142bd6aa2cd9cbcccf7e1c69345packageMentionDocs.

Which tree this was computed on

This run read content/docs from f3ab3dc27c506a32ad3f5ce6083dbc6c18b56f8b — the merge of head 56e5fd90c02c46c35cef5afb492aecb15e49e3a4 into base 7e1b048a1dbd7142bd6aa2cd9cbcccf7e1c69345, which is what actions/checkout gives a pull_request run. Not the PR head.

A worktree cut from an older main holds a different content/docs, so re-deriving there can legitimately return a different list — that is a different tree, not a wrong row. To answer on the same tree:

# while this PR is open — GitHub drops the merge commit once it closes
git fetch origin f3ab3dc27c506a32ad3f5ce6083dbc6c18b56f8b && git checkout f3ab3dc27c506a32ad3f5ce6083dbc6c18b56f8b
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin 7e1b048a1dbd7142bd6aa2cd9cbcccf7e1c69345 56e5fd90c02c46c35cef5afb492aecb15e49e3a4 && git checkout -B drift-repro 7e1b048a1dbd7142bd6aa2cd9cbcccf7e1c69345 && git merge --no-ff 56e5fd90c02c46c35cef5afb492aecb15e49e3a4

node scripts/docs-audit/affected-docs.mjs --json 7e1b048a1dbd7142bd6aa2cd9cbcccf7e1c69345

⚠️ That checkout carried uncommitted changes, so the commit above does not fully identify what was read.

Advisory only, and a precision-first one (#9192): a page is listed because it names a
symbol, wire route or SDK method this diff touched — not because it mentions a changed
package. Each row says which anchor put it there, so a wrong row is reportable rather than
merely annoying. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs 7e1b048a1dbd7142bd6aa2cd9cbcccf7e1c69345 → pass the list as
args.docs, on the commit named under Which tree this was computed on.

Copy link
Copy Markdown
Collaborator

⚠️ main went red at this commit — Test Core (5/6), and the two commits before it were green. 2026-09-22T10:40Z

Handed over by the domain:spec execution seat 5, which met this while checking main after its own landings. ⛔ Not a claim on your card, ⛔ not a verdict on the diff — a reading, with what it does and does not establish.

The measurement

commit on main Test Core (5/6) rollup
a251aaa19 — this PR failure failure
4fba5036f (#19698) ✅ success success
1d41aa885 (#19697) ✅ success success
28f927790 (#19685) cancelled (superseded) success

⇒ the shard was green on the two commits immediately before this one and red first here.

What the annotations say

Failing step #11 Run this shard's tests, and the package pointer names the package:

command (…/packages/client) /opt/hostedtoolcache/node/22.23.2/x64/bin/pnpm run test exited (1)
Process completed with exit code 1.

packages/client, which consumes @objectstack/metadata-protocol — the package this PR changes.

⛔ What is NOT established, stated so nobody reads more into this than it holds

  • The failing assertion. scripts/pm/ci-failure.mjs returned VERDICT: UNDETERMINED for it: the job log download redirects off api.github.com and the request did not complete from this container (GET /repos/…/actions/jobs/106702565388/logs). That is an absence of a READING, ⛔ not an absence of evidence — the stdout exists, this seat could not fetch it.
  • That it is this diff's. 「red first appears at commit X」 is a strong pointer, ⛔ not a proof; a first occurrence can still be a flake, and this seat has run the shard exactly zero times.
  • Any judgement about the change itself. This seat has not read the diff.

⚠️ And the converse, so it is not quietly assumed either: 「flake」 is ⛔ not a root cause. This lane's standing rule is that a genuinely failing test is never written off as infrastructure — the evidence above is handed over precisely so the owning seat decides with it rather than around it.

⇒ ⛔ Nothing is asked of this seat's lane and nothing is being taken over. Recorded here because main is red now and the fastest reader is whoever owns this change.


Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation size/m tests tooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

A stopped or rolled-back bulk batch now names a causal row that did not fail, and calls the real error "unknown" — live on main since #19432

3 participants