Repository navigation
Sudden multi-key operations RedisCommandExceptions on a standalone/non-clustered Azure Managed Redis #3284
Description
After #3274 being fixed in v3.4.5 we upgraded on Tuesday and it worked like a charm for ~ 24 hours but at some point yesterday we started getting RedisConnectionException like this one:
StackExchange.Redis.RedisConnectionException: SocketClosed on XXX.northcentralus.redis.azure.net:10000/Subscription, Idle/RanToCompletion, last: PING, origin: ReadAllAsync, outstanding: 1, last-read: 0s ago, last-write: 0s ago, unanswered-write: 0s ago, keep-alive: 60s, state: ConnectedEstablished, in: 0, in-pipe: 0, last-heartbeat: 0s ago, last-mbeat: 0s ago, global: 0s ago, v: 3.4.5.54687
which we thought are related to an Azure internal resource maintenance event. Some time after that we started getting RedisCommandException for multi-key operations but we are using a non-clustered Azure Managed Redis resource which shouldn't trigger these:
StackExchange.Redis.RedisCommandException: Multi-key operations must involve a single slot; keys can use 'hash tags' to help this, i.e. '{/users/12345}/account' and '{/users/12345}/contacts' will always be in the same slot
at StackExchange.Redis.ConnectionMultiplexer.SelectServer(Message message) in /_/src/StackExchange.Redis/ConnectionMultiplexer.cs:line 2532
at StackExchange.Redis.RedisTransaction.ExecuteAsync(CommandFlags flags) in /_/src/StackExchange.Redis/RedisTransaction.cs:line 89
...
Reverting back to v3.3.1 fixed the issue and it's all OK now (still observing for exceptions though)
Activity
mgravell commentedon Oct 22, 2026
Ok, I suspect these are unrelated. The first could relate to maintenance - the interesting question to me is: did it recover?
The second: that's super interesting - there is a relevant CLUSTER mode change, but it shouldn't incorrectly identify between standalone and regular.
Is there any chance you'd be willing to run the ExportConfiguration method on the failing version, and let me know what files come back in the zip? Or at least let me know whether it is still reliably reprocible with that version?
vladislav-karamfilov commentedon Oct 22, 2026
Yes,
RedisConnectionExceptions were thrown for only ~40 minutes yesterday.I have deployed the app with v3.4.5 and the issue isn't reproducible at the moment. Reverting back to v3.3.1 though because I don't want to risk it failing during the work hours of our US clients.
mgravell commentedon Oct 22, 2026
~40 minutes would sound like a "nope" to me - I'd expect recovery to happen within seconds, minutes on a bad day. I wonder whether I can auto-enable some better health checks (we have the full health check suite from geo-failover at our fingers)
On the cluster/routing problem: I'm investigating a hypothesis that
HELLO 3reported the mode as cluster during some kind of internal migration event. That by itself IMO isn't sufficient to work usefully as a cluster, so I'm testing a change whereby we only switch to cluster mode following a successfulCLUSTER NODESand/orCLUSTER SLOTS.Maybe @philon-msft can opine on the AMR maintenance internals to see what really happens under the hood here, but I'm going to progress that
HELLOhypothesis in isolation. In particular, I think there's some interaction between RESP3, pub/sub connections, HELLO, and confusing "cluster" reports; until very recently, RESP3+pub/sub connections wasn't a thing, as we shared the connection for both scenarios.Route as a cluster only with a slot map; take HELLO mode/role from th…
mgravell commentedon Oct 22, 2026
fixed by #3285
Merge main into v4 (19 commits to 2026-10-23), with the v3-only fixes…