feat(onchain): transactional outbox and durable slashing saga (#396 #397)

[frienzy514-png]

Closes #396
Closes #397

Summary

#396: transactional outbox for on-chain writes. Intent mutations no longer submit Soroban transactions inside the HTTP request. create / acceptIfOpen / fillIfAccepted / cancelIfOpen now write the intent change and an onchain_outbox row in a single Prisma $transaction. OutboxRelayService submits the rows afterwards.

#397: end-to-end slashing saga. The sweeper now only detects missed fills. A durable pending_slashes saga (detected → challenge_window → submitted → confirmed | cancelled) holds each slash for a challenge window. When the window ends it re-verifies on-chain that no fill landed, then submits through SolverRegistryService.slashSolver. Each cancellation compensates exactly once through rollbackPenalty.

Design

Outbox (#396)

Requirement	How
onchain_outbox (id, intent_id, operation, payload, status, attempts, next_attempt_at, tx_hash)	Migration 20260928000001_onchain_outbox. Also stores envelope_hash, locked_until and last_error
Intent change + outbox row in one transaction	IIntentsUnitOfWork (src/intents/intents.unit-of-work.ts). A lost race enqueues nothing. Payloads are encoded to contract args at enqueue time, so bad input fails the request instead of creating a poison row
Relay claims with SKIP LOCKED	One WITH … FOR UPDATE SKIP LOCKED … UPDATE … RETURNING statement (prisma-outbox.repository.ts)
Ordering per intent_id, parallel across intents	Only the head row per intent can be claimed. Every earlier row must be confirmed/simulated
Idempotency after a crash	The signed envelope hash is stored before broadcast. A row reclaimed after the lease expires looks that hash up first: SUCCESS → confirm without resubmitting, NOT_FOUND → rebuild. This is safe because the lease (120 s) outlives the tx time bound (30 s). All post-claim writes are fenced on attempts
Poison rows → dead with alert	Exponential backoff (capped at 5 min). After OUTBOX_MAX_ATTEMPTS: dead, vortex_outbox_dead_total, alert rule, and the intent stays blocked until POST /api/v1/admin/outbox/:id/requeue
Confirmation tracker	Adds a non-blocking TxConfirmationService.check() next to #394's waitForConfirmation, so workers never block a tick. StellarTxService.invokeContract (the #394 pipeline) only gains an optional beforeSubmit(hash) hook, called after signing and before broadcast. A SUCCESS from the live path confirms the row immediately
Kill switch (#477)	A KillSwitchActiveException puts the row back without consuming an attempt, so an emergency pause can never dead-letter rows

Slashing saga (#397)

Requirement	How
States detected → challenge_window → submitted → confirmed | cancelled	pending_slashes + SlashingPipelineService. Every step is a conditional transition, so a crash resumes from the stored state
Re-verify that no fill landed before submitting	FillVerifierService scans the settlement contract's intent_filled events. If the RPC check fails, the saga retries later and never submits blind
Challenge window (default 10 min), cancellable by fill proof or admin	SLASH_CHALLENGE_WINDOW_SECONDS. POST /api/v1/slashes/:intentId/fill-proof (solver-signed, verified on-chain) and POST /api/v1/admin/slashes/:intentId/cancel
Exactly-once slash per intent	Unique index on intent_id. Checked under concurrent detection against real Postgres
Saga with explicit compensations	cancel() runs only on the winning → cancelled transition: rollbackPenalty (with a durable fallback if the in-memory penalty was lost on restart), intent → filled/expired, audit entry, and an intent_slash_cancelled WS event
Edge: clock skew	Timeliness uses ledger close time against fillDeadline + SLASH_CLOCK_SKEW_TOLERANCE_SECONDS
Kill switch (#477)	A pause on slash/onchain defers the slash without consuming attempts, so the saga never gives up during a pause
Edge: solver deregistered mid-window	The slash still proceeds, since deregistration must not be an escape hatch. A solver with no record is cancelled and compensated
Runbook	docs/runbooks/slash-cancellation.md

Admin routes (/api/v1/admin/slashes, /api/v1/admin/outbox/:id/requeue) use the existing AdminGuard RBAC (x-admin-key / ADMIN_API_KEYS). Each action is written to admin_audit_log before it is applied, and the authenticated admin's id is the actor.

New env vars

OUTBOX_RELAY_ENABLED, OUTBOX_RELAY_INTERVAL_MS, OUTBOX_RELAY_BATCH_SIZE, OUTBOX_MAX_ATTEMPTS, OUTBOX_LEASE_SECONDS, SLASH_CHALLENGE_WINDOW_SECONDS, SLASH_CLOCK_SKEW_TOLERANCE_SECONDS, and SLASH_MAX_SUBMIT_ATTEMPTS. Each one is in env.validation.ts, configuration.ts and all four .env*.example files. check:env-drift reports no drift for any of them.

Docs

The architecture doc (onchain-settlement.md) has new outbox and saga sections. on-call.md has a new Scenario G (dead/backlogged outbox) and config table rows. Prometheus rules are in docs/runbooks/alerts/onchain-writes.rules.yml, and there's a CHANGELOG entry.

Testing

Everything was fully verified on the base the branch was written against (before rebasing), with typecheck and all suites green:

• outbox + saga suites: 66 passed. Branch coverage: slashing-pipeline.service.ts 98.7% (target ≥95%), outbox-relay.service.ts 100%, outbox.repository.ts 95.8%, prisma-outbox.repository.ts 91.7%, outbox-operations.ts 92.3% (target ≥90%).
• Outbox crash-injection scenarios: a crash between DB commit and submit (the next relay submits); a crash after broadcast and before markSubmitted (confirmed, not resubmitted); an envelope that never landed (rebuilt after the lease); a lease lost before broadcast (nothing sent).
• Saga scenarios: late fill, fill-proof, admin cancel (including mid-submit lease and already-submitted), RPC failure during verify/submit/confirm with backoff and give-up, clock skew, deregistered solver, restart-lost penalty record, and every lost-race branch.
• Integration against real PostgreSQL 17 (throwaway local instance): concurrent claimDue calls return disjoint, head-of-intent-only rows; a throw inside the unit of work rolls back both the intent and the outbox row; concurrent detection yields one pending_slashes row; lease/fencing behaves as specified; migration.sql matches schema.prisma; down.sql removes both tables and enums.

After rebasing onto current main (0e6fcb3)

main doesn't currently build (see the PR comment), so full-suite results aren't meaningful yet. What I could verify:

Check	main	This branch
tsc --noEmit errors	76	76, with zero per-file difference (the 43 .ts files I added or changed introduce no type errors)
ESLint errors on lines this PR adds	—	0 (the 2 errors in the touched files are existing parse errors in stellar-signature.ts / env.validation.ts)
prisma validate	valid	valid
check:env-drift for new keys	—	no drift
Suites independent of files broken on main	—	7 suites / 45 tests pass (outbox repo, Prisma adapters, operations, unit of work, pending slashes, admin controller, solvers)

The other 9 suites of mine fail to compile only on errors already on main: intents.service.ts lines 151/581/713, soroban.service.ts getLedger, stellar-tx.service.ts duplicate Operation import, stellar-signature.ts parse errors. Once main compiles they should run as before, and I'll rebase and confirm.

Notes for reviewers

• Contract method names (create_intent, accept_intent, fill_intent, cancel_intent) are provisional until the settlement ADR (Write an ADR for the on-chain settlement architecture #19) lands.
• SolverRegistryService's submit path is still gated pending Wire solver accept() to the solver-registry contract #23. Until then, slashes end at submitted with simulated=true, and the runbook documents this. SlashResult gains a failed flag so the saga can tell "retry" apart from "gated/simulated".
• Rows under ONCHAIN_DRY_RUN=true end as simulated and are not replayed when dry-run is later switched off.
• The outbox, unit of work and saga follow INTENTS_PERSISTENCE. With the default memory, behaviour is in-process only.

[drips-wave]

@frienzy514-png Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits

[frienzy514-png]

Heads-up for maintainers: main (0e6fcb3) doesn't compile right now, so CI on this PR (and on any other open PR) can't go green until it's fixed. These errors come from earlier merges, not from this PR:

• src/intents/intents.service.ts: L151 (required param after optional ones), L581 (stray return this.repo.acceptIfOpen(id, …, nowSec) inside observeAccept, where id/nowSec are undefined), L713
• src/soroban/stellar-tx.service.ts: duplicate Operation import, required param after optional
• src/soroban/soroban.service.ts: getLedger doesn't exist on SorobanRpc.Server
• src/common/stellar-signature.ts: buildDisputeDecisionMessage / buildUpdateSolverMessage are fused together (missing } and /**), which is a parse error
• src/config/env.validation.ts: a stray }); after ORACLE_ALLOWLIST closes the Joi schema early, so the WS / health keys after it are outside the object

This branch has exactly the same tsc error count as main (76), with no per-file difference, so it adds none. I didn't fix the above here to keep the PR scoped to #396/#397. I'm happy to open a separate fix PR if that helps, and I'll rebase this one once main builds.