Skip to content

fix(backend): restore idempotency store with dry-run safe retention sweep (#1375) - #1491

Open
solaawojobi00-bit wants to merge 1 commit into
Junirezz:mainfrom
solaawojobi00-bit:fix/issue-1375-idempotency-dry-run
Open

solaawojobi00-bit wants to merge 1 commit into
Junirezz:mainfrom
solaawojobi00-bit:fix/issue-1375-idempotency-dry-run

Conversation

@solaawojobi00-bit

Copy link
Copy Markdown
Contributor

fix(backend): restore idempotency store with a dry-run safe retention sweep (#1375)

Problem

pruneStaleIdempotencyRecords delegates to idempotencyStore.pruneStaleKeys(retentionMs, dryRun), but commit 5db5f6e0 replaced backend/src/idempotency.ts with the Prisma-backed helpers and deleted the Redis/NodeCache IdempotencyStore. That removed localCache, pruneStaleKeys and its localPruned field. Nine files still import it (idempotencyRetention.ts, vaultEndpoints.ts, transferOrchestrator.ts, index.ts, and five test suites), so there was no safe way to rehearse a retention sweep in production.

Scenario Before After
POST /admin/idempotency/retention/cleanup?dryRun=true TypeError: Cannot read properties of undefined (reading 'pruneStaleKeys') Returns { pruned, localPruned, redisPruned, dryRun: true }; nothing is deleted
Dry-run with stale local-cache entries Crash Entries counted in localPruned and still present afterwards
Dry-run with stale Redis keys Crash Keys counted in redisPruned and still in Redis afterwards
Dry-run effect on sweep metrics n/a lastSweepAt, totalPruned, lastPrunedCount and eviction counter unchanged
Live sweep after a dry-run Crash Prunes the same keys the dry-run reported and records metrics
new IdempotencyStore() in tests IdempotencyStore is not a constructor Works

Separately, backend CI (Backend Governance / Test Coverage) could not get past tsc or prisma generate on main because of unrelated breakage. The build-level parts are fixed here as well (see Changes).

Solution

Restore the store next to the new Prisma helpers, without changing those helpers, and make the dry-run contract explicit:

  • pruneStaleKeys(retentionMs, dryRun) reads the local cache and Redis (SCAN/GET/TTL only) and counts stale entries. It deletes entries and bumps evictions only when dryRun is false. It returns { pruned, localPruned, redisPruned, dryRun }.
  • pruneStaleIdempotencyRecords(dryRun) returns the same breakdown, logs a "dry-run completed" line with wouldPrune, and updates the sweep metrics only for live runs.

Changes

backend/src/idempotency.ts: restores IdempotencyStore, idempotencyStore, IdempotencyConflictError, IdempotentOperationResult, buildIdempotencyFingerprint, getIdempotencyHashThreshold, the private localCache: NodeCache and pruneStaleKeys, taken from the last version before 5db5f6e0. The store's metadata type is renamed IdempotencyStoreKeyMetadata so it doesn't clash with the new Prisma IdempotencyKeyMetadata.

async pruneStaleKeys(retentionMs: number, dryRun = false):
  Promise<{ pruned: number; localPruned: number; redisPruned: number; dryRun: boolean }>

backend/src/idempotencyRetention.ts: new IdempotencyRetentionSweepResult type. A dry run logs what it would prune and doesn't touch retentionState; the result includes the per-backend counts. The admin endpoint spreads the result, so it gets these fields without changes.

backend/src/__tests__/idempotencyRetention.test.ts: the rateLimiter mock can now inject an in-memory Redis fake; four dry-run regression tests added.

Pre-existing build fixes (one or two lines each):

File Fix Why
prisma/schema.prisma Close the WalletTenantAssociation model prisma validate/generate failed (P1012)
src/apiContractSnapshots.ts Close the unterminated if block; key in (x ?? {}) TS1005 syntax error stopped tsc; precedence bug
src/vaultEndpoints.ts Import readsLimiter; order /receipts by timestamp; add a return path in the /strategy cooldown branch ReferenceError: readsLimiter is not defined at module load; Transaction has no createdAt; TS7030
src/tracing.ts new Resource(...) instead of resourceFromAttributes The lockfile resolves @opentelemetry/resources to 1.30.1, which has no resourceFromAttributes
src/schemaSnapshot.ts .shape instead of ._shape Zod v4 API
src/swagger.ts Remove the duplicate checks key TS1117; the fuller object is kept
src/tests/idempotency.test.ts, src/tests/tenantBoundary.test.ts Cast mock requests Request.get overload typing

Regression Tests

Acceptance criterion (#1375) Test
Store exposes localCache and pruneStaleKeys returns localPruned prunes stale local idempotency keys
Dry-run reports what would be pruned (local) without mutating state store dry-run reports stale local keys without deleting them
Dry-run reports what would be pruned (Redis) without mutating state; live run deletes store dry-run reports stale Redis keys without deleting them
Sweep dry-run is safe for production rehearsal (no deletions, no metric changes) sweep dry-run leaves the store and sweep metrics untouched
Dry-run output matches what a real sweep then does a live sweep after a dry-run prunes the same keys and records metrics
Existing dry-run entry point still works supports dry-run retention sweeps

Testing

$ npx jest --runInBand --verbose src/__tests__/idempotencyRetention.test.ts src/__tests__/transferOrchestrator.test.ts
PASS src/__tests__/transferOrchestrator.test.ts (27.702 s)
PASS src/__tests__/idempotencyRetention.test.ts
    √ reports retention policy and store metrics (3 ms)
    √ prunes stale local idempotency keys (3 ms)
    √ supports dry-run retention sweeps (6 ms)
      √ store dry-run reports stale local keys without deleting them (2 ms)
      √ store dry-run reports stale Redis keys without deleting them (3 ms)
      √ sweep dry-run leaves the store and sweep metrics untouched (2 ms)
      √ a live sweep after a dry-run prunes the same keys and records metrics (4 ms)
Test Suites: 2 passed, 2 total
Tests:       52 passed, 52 total

Full backend suite (npx jest --runInBand), upstream main vs this branch:

main:    Test Suites: 34 failed, 57 passed, 91 total   Tests: 60 failed, 638 passed, 698 total
branch:  Test Suites: 16 failed, 75 passed, 91 total   Tests: 87 failed, 1236 passed, 1323 total

No suite that passes on main fails on this branch. More tests run because suites that previously failed to load (missing store exports, readsLimiter, the syntax error) now execute.

npx tsc --noEmit: 1 blocking syntax error on main (which hid the rest) → 15 remaining errors, all in the two pre-existing areas listed below. eslint on the changed files: 0 errors.

Notes for Reviewers

CI will not be fully green, for pre-existing reasons outside #1375. Backend Governance has failed on main since 2026-08-25, and Test Coverage has no successful run. This PR clears the build-level blockers; these remain and need their own issues:

  1. src/sessionAudit.ts (13 TS errors): uses prisma.sessionAuditLog, but no SessionAuditLog model or migration was ever added. The file isn't imported anywhere.
  2. src/middleware/tenantBoundary.ts (2 TS errors): filters Transaction by deletedAt and WebhookEndpoint by tenantId; neither column exists. This is a tenant-isolation check, so it needs a schema and migration decision, not a code workaround.
  3. 16 failing suites (pagination, transactions, OpenAPI contract, referral, allowlist, webhooks, withdrawal limit, and others) caused by logic or contract drift already on main.

Overlap: #1486 also restores the idempotency store (for #1318/#1319/#1322). Whichever merges second will conflict in backend/src/idempotency.ts. This PR keeps the original store implementation and export names, so the resolution should be mechanical.

Risk: Low–Medium. It restores previously shipped behavior that vault endpoints and the transfer orchestrator rely on; the Prisma helpers are unchanged. There's no database migration: the schema fix only adds a missing }.
Rollback: a clean git revert, with no persistent state involved.

Closes #1375

…weep (Junirezz#1375)

Commit 5db5f6e replaced idempotency.ts with the Prisma-backed helpers and
dropped the Redis/NodeCache IdempotencyStore, leaving pruneStaleIdempotencyRecords,
vault endpoints and the transfer orchestrator importing symbols that no longer
existed.

- Restore IdempotencyStore (localCache, pruneStaleKeys) alongside the Prisma
  helpers; pruneStaleKeys reports localPruned/redisPruned and a dryRun flag and
  never mutates state in dry-run mode.
- pruneStaleIdempotencyRecords returns the per-backend breakdown, logs dry-run
  results and leaves sweep metrics untouched on dry runs.
- Add dry-run regression tests for local cache, Redis and the sweep.

Also fixes pre-existing upstream build breakage blocking backend CI:
- schema.prisma: close WalletTenantAssociation model
- apiContractSnapshots.ts: close unterminated block, fix `in`/`??` precedence
- vaultEndpoints.ts: import readsLimiter, order receipts by timestamp,
  fix missing return path in strategy handler
- tracing.ts: use Resource from the locked @opentelemetry/resources 1.x
- schemaSnapshot.ts: use Zod v4 `shape`
- swagger.ts: remove duplicate `checks` key
- tests: fix mock Request.get typings
@drips-wave

drips-wave Bot commented Sep 29, 2026

Copy link
Copy Markdown

@solaawojobi00-bit Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

idempotency retention sweep lacks a dry-run safe path for production rehearsals

1 participant