Skip to content

fix: check Redis connectivity in /ready health probe - #1513

Open
Jayycob2211 wants to merge 4 commits into
Junirezz:mainfrom
Jayycob2211:fix/issue-1443-health-ready-endpoint-does-not-check-redis
Open

Jayycob2211 wants to merge 4 commits into
Junirezz:mainfrom
Jayycob2211:fix/issue-1443-health-ready-endpoint-does-not-check-redis

Conversation

@Jayycob2211

Copy link
Copy Markdown

Overview

This PR closes a readiness gap where /ready reported healthy even when Redis was unreachable. A pod with a dead cache connection would keep receiving traffic from k8s, and any GET /vaults request hitting the cache path would throw 500 and fall back to the DB, causing a thundering herd. The /ready probe now pings Redis (with a 500ms timeout) and returns 503 when it fails, while treating Redis as optional when REDIS_URL is not configured.

Related Issue

Changes

🩺 Redis Readiness Check

  • [MODIFY] backend/src/redisCache.ts

    • Expose a ping() helper that performs a Redis PING with a 500ms timeout so the health probe can verify cache connectivity.
    • Report whether Redis is configured (i.e. REDIS_URL set) so callers can distinguish "optional and absent" from "configured but down".
  • [MODIFY] backend/src/healthProbe.ts

    • Add a redis entry to the /ready checks.
    • When Redis is configured, run redis.ping(); on failure return 503 with checks: { redis: { status: 'down' } }.
    • When REDIS_URL is not set, report checks: { redis: { status: 'up', optional: true } } instead of down.
  • [MODIFY] backend/src/__tests__/healthReady.test.ts

    • Update /ready contract tests to assert the new redis check is present.
    • Add a test that mocks redis.ping to throw and asserts /ready responds 503.
    • Add a test covering the optional-Redis case (REDIS_URL unset → up with optional: true).
  • [MODIFY] backend/schema-snapshots/get-_ready.json

    • Refresh the /ready response snapshot to include the redis check.

Verification Results

npm test -- backend/src/__tests__/healthReady.test.ts
✅ healthReady tests passed (including redis-down → 503 and optional-redis → up)
Acceptance Criteria Status
Add redis.ping() check with 500ms timeout to /ready; return 503 with checks: {redis: {status: 'down'}} on failure ✅ Implemented in healthProbe.ts / redisCache.ts
Optional redis (when REDIS_URL not set) reports up with optional: true ✅ Covered by updated contract test
Update health/ready contract tests and snapshots to include redis check ✅ healthReady.test.ts + get-_ready.json updated
Test: mock redis.ping to throw and assert /ready is 503 ✅ Added to healthReady.test.ts

Closes #1443

@drips-wave

drips-wave Bot commented Sep 29, 2026

Copy link
Copy Markdown

@Jayycob2211 Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Health /ready endpoint does not check Redis connectivity causing k8s to route to pod with dead cache

1 participant