Skip to content

[High] Durable Event Replay Backed by Redis Streams #457

Description

@james2177

Description:
Replace the in-process ring buffer (REPLAY_BUFFER_SIZE) with a Redis Streams-backed log so clients can resume after deploys, replica switches, or long disconnects within a retention window.

Problem Statement & Context:
The replay buffer in intents.gateway.ts is per-process and lost on restart. A deploy disconnects all clients, and when they reconnect to a new pod they cannot replay, forcing full resyncs.

Scope & Acceptance Criteria:

  • Events appended with XADD (MAXLEN ~ retention); since(seq) served via XRANGE.
  • Retention configurable by count and time; reset message when requested seq is older.
  • Replay throughput: 10k events to a client in < 1 s.
  • Out of scope: infinite history (use REST for that).

Implementation Guidelines:

  1. Key Files/Modules: src/intents/intents.gateway.ts (replay buffer class), src/intents/backplane/.
  2. Design/Architecture: ReplayStore interface with memory and Redis implementations.
  3. Edge Cases/Constraints: Filtered replays (per chain/state) must filter server-side without loading entire stream into memory.
  4. Testing: Restart-and-resume e2e; retention boundary tests.

Definition of "Done": Common DoD.

Resources:


Common Definition of "Done" (applies in addition to the criteria above):

  • Code written, tested, and documented (TSDoc on public APIs, README/runbook/ADR updates where behaviour changes).
  • All acceptance criteria met; npm run lint, npm run typecheck, npm test, npm run test:e2e pass in CI.
  • PR follows .github/PULL_REQUEST_TEMPLATE, uses a Conventional Commit title (enforced by commitlint), includes test output / metrics screenshots, and references the issue.
  • New env vars are added to .env.example variants and src/config/env.validation.ts (the check:env-drift script must pass).
  • Reviewed and approved by at least one CODEOWNER.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Stellar WaveIssues in the Stellar wave program

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions