F2+F3: transferSale awaitable, block timestamps written as UTC - #62
Merged
Conversation
transferSale was not async and called db.withTransaction(...) without returning it, so the block transaction committed (claiming the action's global_seq in _processed_actions) while the order/order_item writes were still in flight. A failure in the inner transaction — seen live as 'RELEASE SAVEPOINT can only be used in transaction blocks' — was then skipped by every future reindex: a claimed ledger row without applied writes, the same class PR #55 fixed in verifyClaim. Make transferSale async and return the withTransaction promise, matching createCommunity. Audit of every withTransaction in src/: createCommunity (returned), updateCommunity (awaited) and upsertAction (returned) were already correct; transferSale was the only fire-and-forget hit.
Every timestamp column is 'timestamp without time zone', matching the
Elixir backend (which stores UTC via DateTime.utc_now()). node-pg (and
pg-promise, which reuses pg's serializer) formats a JS Date in the
process's LOCAL zone and appends an offset that Postgres ignores for
tz-less columns, so a raw Date landed as host-local wall clock — seen
live: block 2026-08-04T21:04:09Z stored as 23:04:09 on a CEST host.
Indexer rows and app rows disagreed in one database, and
current_month_quantity's date_trunc('month') window lands wrong at
month boundaries.
Add src/dates.js (toUTC -> Date.toISOString) and route every Date we
write through it: blockInfo.timestamp in community/token/escrow
updaters, plus the inserted_at/updated_at 'new Date()' writes, which
had the same host-local defect.
Two fixes on top of the merge with master (F0's watermark claim resolver and F0b's escrow `closedBy`, both kept alongside the toUTC conversion). transferSale returned the inner transaction so ledgered() could await it, but kept `.catch(e => logError(...))` on the end. That catch turns a rejection back into a resolved promise, so ledgered saw the updater succeed and kept the _processed_actions row it had just claimed — the order stayed recorded as applied with none of its writes landed, which is the exact failure the return was added to prevent. Log and rethrow instead: the error propagates through ledgered's non-ResolveError branch, the block rolls back (taking the ledger row with it), and the action is left unprocessed for a restart or reindex. Note on the UTC half: prod runs Etc/UTC (verified on the box today, node reports offset 0), and prod rows match their block times — the last indexed transfer sits at 00:50:51 for a block stamped 00:50:50.5Z. So this is a latent correctness fix for non-UTC hosts, not a live production defect. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
F2 + F3 —
transferSalefire-and-forget, and host-local timestampsTwo packets from the follow-up handoff, both in the updater layer. Two commits:
912efef(F2) and32a32d4(F3).F2 —
transferSalewas fire-and-forget (deployed to prod)Observed in the Phase 1 e2e run:
RELEASE SAVEPOINT can only be used in transaction blocks (SQLSTATE 25P01).transferSalewas notasyncand calleddb.withTransaction(...).catch(...)without returning it, soledgered()could not await it: the block transaction committed and_processed_actionsclaimed the action'sglobal_seqwhile the inner work was still running. Any failure there would then be skipped by every future reindex — the "claimed ledger row without applied writes" mode the reindex runbook warns about (same class as #55'sverifyClaimfix). Pre-existing since68ef3b6.Fix:
async function transferSale+return db.withTransaction(...), matchingcreateCommunity's shape.Audit of every
withTransactioninsrc/:community.js:90createCommunityreturn db.withTransaction(...)community.js:138updateCommunityawait db.withTransaction(...)community.js:280transferSalecommunity.js:445upsertActionreturn db.withTransaction(...)token.js,escrow.jsRelated fire-and-forget shapes found but not changed (out of F2's scope, flagged for a future packet):
updateCommunity's tx callback firestx.communities.update(...).catch(...)unawaited (community.js:130 — harmless today via node-pg per-connection serialization, but an update error is swallowed instead of rolling back);reward(community.js:529,547) andcreateToken/updateToken/setExpiry(token.js) fire DB writes the returned promise doesn't await.F2 acceptance — replay procedure
Local chain (
NODE_ENV=local→cambiatus_local): pushedtransfersale [1, "alice", "bob", "5 TST", 1](block 28092), indexed (order 1 + order_item 1, stock 10→9). Then the handoff procedure: truncate_index_state/_block_number_txid/_processed_actions, full reindex from block 1. Cross-check the replay was complete: registered action types on chain total 196 =_processed_actionscount after reindex.Zero errors in the replay log; stock not double-decremented.
F3 — block timestamps were written in host-local time
Mechanism (verified against the installed libs, not assumed):
GetActionsReader.js:150buildsblockInfo.timestampcorrectly as a UTC instant, but node-pg'sdateToStringserializes aDatein the process's local zone and appends an offset; Postgres ignores the offset fortimestamp without time zoneinput (SELECT '2026-08-04T21:04:09.000+02:00'::timestamp→2026-08-04 21:04:09), so the local wall clock lands in the column. pg-promise's raw-SQL path reuses the same serializer. Rejected the one-linepg.defaults.parseInputDatesAsUTC = trueglobal — it silently changes third-party paths and can't be audited per write.Fix: new
src/dates.jsexportingtoUTC(date) => date.toISOString(), applied to every affected write — block timestamps ANDnew Date()writes alike (fixing only block timestamps would have leftcreated_atUTC butinserted_athost-local):community.js— 14blockInfo.timestampwrites (createCommunity, netlink ×2, transferSale ×5, upsertObjective, upsertAction ×2, claimAction, verifyClaim), 16new Date()writes (subdomains, roles, network_roles, rewards, upsertRole, assignRole), plusupsertAction'sdeadline.token.js—transfer.created_at,issue.created_at.escrow.js—regDeposit$9,closeDeposit$4.F3 acceptance — TZ experiment
Same action indexed twice, fresh rows each run:
(
11:44:22.500→11:44:23istimestamp(0)rounding.) Pre-fix code on this same machine (CEST) stored block 28092 as13:34:03— bug reproduced before, correct after.Caveats surfaced during acceptance (not this PR's scope)
upsertAction'sdeadlineis built withnew Date(parseInt(payload.data.deadline))(ms), but the contract comparesnow() < deadlinein seconds — stores a 1970 instant wheneverdeadline > 0. Worth its own packet._index_statetruncate footgun: demux's handler doesfindOne({id: 1}) || {}thensave()— truncating the table inserts a new row per block and restarts reindex from genesis forever. The runbook'sUPDATE _index_state SET block_number = …form is the safe one; the handoff's acceptance text should be amended.cambiatus.tkhistory is non-monotonic (seqs 100–106 at blocks 137–162, seqs 107–108 at blocks 10404–10405) — cosmetic, the reader handles it.