Skip to content

Sentinel client gets stuck connecting to an unreachable instance #3385

Description

@BenRuns

Description

.

Description

RedisSentinelClient#connect()'s retry cap becomes dead code after the very first successful connection, so once a Sentinel client has connected once, it can retry Sentinel topology rediscovery forever on any later failure — silently, with no error ever emitted for the app's own client.on('error', ...) handler to react to.

Root cause, in sentinel/index.js:

if (!this.#isReady && count > this.#maxCommandRediscovers) {
throw e;
}
await setTimeout(1000); // otherwise: sleep 1s and loop forever

this.#isReady is the Sentinel client's own top-level flag. It's set true once, after the first successful connect, and is only ever reset back to false in .close(). So on any subsequent topology-discovery failure (e.g. after the app has been running fine for hours and then the Sentinel topology becomes unreachable), !this.#isReady is always false — the count > maxCommandRediscovers cap can never trip, and connect() just sleeps 1s and loops indefinitely instead of ever throwing.

Net effect: a long-lived Sentinel client that hits a topology-discovery outage after its initial connect does not recover on its own, does not emit any lifecycle/error event, and does not honor the configured retry cap at all. From the outside the process looks "stuck" — no crash, no error log, just a client silently spinning on the topology-discovery loop.

Related but distinct from #3346 (fixed by #3374) — that one is about the standalone client hanging mid-async-suspension during a socket-death race. This is Sentinel-specific, purely in the post-initial-connect retry-cap logic, and reproduces deterministically once the client has connected at least once.

Node.js Version

20.x (also seen on 22.x)

Redis Server Version

7.x, Sentinel topology (3 sentinel + primary/replica set)

Node Redis Version

@redis/client@6.0.0 — confirmed still present in 6.1.0 and 6.2.0-beta.

Platform

Linux (Kubernetes pods), reproduced locally on macOS via Docker Compose Sentinel topology

Logs

RedisCommandTimeoutError repeating indefinitely with no error/reconnecting/end event ever emitted, once the client has connected successfully at least once and a later topology-discovery attempt starts failing. count > maxCommandRediscovers never trips because #isReady is already true and is never reset outside of .close().

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions