[High] Solver Liveness Heartbeats with Automatic Offline Detection

[james2177]

Description:
Require authenticated solvers to heartbeat over WS (or REST), track liveness server-side, and automatically mark solvers offline — excluding them from RFQ and routing — after missed heartbeats.

Problem Statement & Context:
markLive/markOffline exist but depend on explicit calls. Dead solvers remain eligible for quotes, causing timeouts and unfilled intents.

Scope & Acceptance Criteria:

• Heartbeat interval negotiated (default 10 s); offline after 3 misses; liveness shared across replicas (Redis).
• Solver status transitions emit WS solver_status_changed and update lastActiveAt.
• RFQ and capability filtering exclude offline solvers.
• Metrics: live solver count per chain.
• Out of scope: penalising downtime.

Implementation Guidelines:

1. Key Files/Modules: src/solvers/solvers.service.ts, src/intents/intents.gateway.ts, src/solvers/solvers.controller.ts.
2. Design/Architecture: Redis keys with TTL per solver; keyspace notifications or periodic scan for expiry.
3. Edge Cases/Constraints: Network partitions of a single replica must not flap all solvers offline.
4. Testing: Fake-timer tests; two-replica integration test.

Definition of "Done": Common DoD.

Resources:

• Design reference: N/A (backend-only)
• src/solvers/solvers.service.ts

---

Common Definition of "Done" (applies in addition to the criteria above):

• Code written, tested, and documented (TSDoc on public APIs, README/runbook/ADR updates where behaviour changes).
• All acceptance criteria met; npm run lint, npm run typecheck, npm test, npm run test:e2e pass in CI.
• PR follows .github/PULL_REQUEST_TEMPLATE, uses a Conventional Commit title (enforced by commitlint), includes test output / metrics screenshots, and references the issue.
• New env vars are added to .env.example variants and src/config/env.validation.ts (the check:env-drift script must pass).
• Reviewed and approved by at least one CODEOWNER.

[james2177]

Related prior work: Builds on #119 / #283: lastActiveAt, markLive, and markOffline exist. This issue adds enforced heartbeats and automatic offline detection shared across replicas.

[DevScoopee]

@DevScoopee has applied to work on this issue as part of the Stellar Wave Program's 9th wave.

i fully understand what needs to be done and can deliver. kindly assign

ℹ️ Repo Maintainers: To accept this application, review their application or assign @DevScoopee to this issue.

[benedictworks-home]

@benedictworks-home has applied to work on this issue as part of the Stellar Wave Program's 9th wave.

Hi maintainer, please assign

ℹ️ Repo Maintainers: To accept this application, review their application or assign @benedictworks-home to this issue.

[drips-wave]

Congratulations, @benedictworks-home! 🎉 Your application was accepted by the repo's maintainers, and the issue is due on September 30, 2026.

🧑‍💻 @benedictworks-home: Please resolve the issue such that the repo's maintainers have enough time to review your contribution before the due date. You'll earn Points for completing the issue on-time, which will make you eligible for a share of the Stellar Wave Program's reward pool.

Warning

When opening a PR, please link it to this issue to ensure it gets tracked accurately. Points are awarded when this issue is marked as completed by the maintainer.

🤠 Repo maintainers: Please keep an eye on the contributor's progress and review their work before the due date. You can manage this issue, including adjusting its complexity and points, here.

🌊 Happy Wave 🌊