Release inner cache read lock when outer lock fails - #34
Merged
Conversation
LayeringWrapper.GetValue never released the read lock it took in the inner cache when the outer cache refused to grant one for the same key. The guard was unreachable: `lockOnMiss` is already forced to false earlier in the function whenever `sentinelInner == NoLockSentinel`, so `lockOnMiss && sentinelInner == NoLockSentinel` could never be true. Check `sentinelOuter` instead, mirroring GetValues, and skip the release round trip when there is no inner sentinel to release. Without this, the inner key stayed locked for the full 60s SentinelTTL and the caller got back a combined sentinel that looked valid but that SetValue silently discarded, so the value was never cached. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: efdc36a8-3708-4c0a-977c-70e78d179640
stgarrity
self-requested a review
September 5, 2026 01:19
stgarrity
approved these changes
Sep 5, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
LayeringWrapper.GetValuecould leave a read lock held in the inner cache after the outer cache refused to grant a lock for the same key. When that happened the key stayed locked in the inner cache for the full 60sSentinelTTL, and the caller got back a combined sentinel that looked valid but thatSetValuesilently discarded, so the fetched value was never actually cached.The bug
The guard meant to handle this was unreachable:
Earlier in the same function
lockOnMissis already forced tofalsewheneversentinelInner == NoLockSentinel, so the two halves of that condition can never both be true. The check was also looking at the wrong sentinel: what we care about is whether the outer cache granted a lock.The fix
Check
sentinelOuterinstead, mirroring the correct logic thatGetValuesalready uses:The extra
sentinelInner != NoLockSentinelhalf is not in theGetValuesversion. It avoids a pointless Redis round trip when there is no inner sentinel to release, and is otherwise a no-op.With this,
combineSentinels("", "")returnsNoLockSentinel, so callers correctly observe that they hold no lock rather than acting on a misleading one.Test
Adds
TestLayeringCache/TestReadLockReleasedWhenOuterLockFails, which pre-locks a key in the outer cache only, then does a layeredGetValue(..., lockOnMiss=true). The inner cache grants a read lock, the outer cache refuses, and the test asserts both that the returned sentinel isNoLockSentineland that the inner key was actually released.Confirmed it fails on the unfixed code and passes after the change. The full
infra/cachesuite is green.Notes for reviewers
These tests need a local Redis (inner on DB 0, outer on DB 1) via
tools/start-redis.sh.This fix came out of a broader concurrency audit of
infra/cache. Three other findings are intentionally not addressed here and are worth separate issues:WriteSentinel(redis_provider.go) WATCHes onlykeys[0]whileMSETwrites all keys, and itsMGETruns onc.redisClientrather than thetx, so it is not a consistent snapshot. This defeats the tombstone preservation logic.Flushuses a plain pipeline with noWATCH, accumulatingDels across the entire scan, so a tombstone written mid-scan is deleted even whenflushTombstones=false(the mode the invalidation handler uses).RedisCacheCommunicationProvider.Shutdownsends on an unbuffered channel with adefaultcase, so shutdown can silently no-op and leak the reader goroutine; thePubSubis never closed.