fix: preserve original expiration window on stale claim reclaim#13
fix: preserve original expiration window on stale claim reclaim#13jolovicdev wants to merge 1 commit into
Conversation
When a stale running claim is reclaimed, updating expires_at to now + ttl shifts the expiration window forward. This causes the cache entry to live longer than the caller originally intended if the worker was slow or crashed. Instead, preserve the original expires_at set at commit creation. The task_def.ttl field is still updated so that downstream code sees the current submission's TTL preference, but the actual expiration horizon remains stable. Adds tests for sync and async stale reclaim with cache=True.
There was a problem hiding this comment.
PR Review
PR: fix: preserve original expiration window on stale claim reclaim
Important
Verdict: Request changes — tests must assert that expires_at is preserved after reclaim to prevent regression.
Findings
| Pri | Location | Finding | Action |
|---|---|---|---|
P2 |
tests/test_store.py:325 |
Missing assertion that expires_at is preserved after reclaim |
Add assertion comparing commit's expires_at to original value |
P2 |
tests/test_async_client.py:312 |
Missing assertion that expires_at is unchanged and cache hit works |
Add expires_at assertion and second submission to confirm cache hit |
P3 |
tests/test_async_client.py:282 |
Test name includes "with_ttl" but no TTL is set | Rename test to reflect actual scenario |
Notes
- Both new tests validate that stale claims are reclaimed and function re-executes, but neither checks that the original
expires_atis preserved. Without these assertions, a regression that recalculatesexpires_atwould not be caught. - The async test also does not verify that a subsequent submission returns from cache without re-execution.
What To Do Next
- Add assertions for
expires_atin bothtest_reclaimed_stale_claim_keeps_original_expires_atandtest_cached_task_with_ttl_and_stale_reclaim. - In the async test, add a second
submitcall to confirm the cache hit (counter should not increment).
|
|
||
| log = client.log() | ||
| assert len(log) == 1 | ||
| assert log[0].status.value == "completed" |
There was a problem hiding this comment.
P2 This test does not assert that expires_at is preserved after reclaim. Without it, a regression that recalculates expires_at would not be caught. Add an assertion comparing the commit's expires_at to the original value. For example:
| assert log[0].status.value == "completed" | |
| assert log[0].expires_at is None |
|
[tests/test_async_client.py:312]
|
|
[tests/test_async_client.py:282]
|
|
Closing in favor of a more comprehensive fix. |
Summary
Preserves the original
expires_atvalue when reclaiming a stale running claim, rather than recalculating it from the current time. This prevents cache entries from living longer than intended when a worker crash or slowdown causes a claim to be reclaimed.Changes
src/cashet/async_executor.py— Removed theexpires_atrecalculation on stale claim reclaim. The original expiration window is now kept intact.src/cashet/models.py— Added clarifying comments on theexpires_atfield to document the stable-horizon design intent.tests/test_store.py— Addedtest_reclaimed_stale_claim_keeps_original_expires_atcovering sync stale reclaim.tests/test_async_client.py— Addedtest_cached_task_with_ttl_and_stale_reclaimcovering async stale reclaim.Rationale
When a claim is reclaimed after a worker dies, the previous behavior recalculated
expires_at = now + ttl. This effectively extends the cache lifetime by the full TTL from the reclaim moment, even though the task may have already been running for a while. By preserving the originalexpires_at, callers get a predictable expiration horizon tied to the original submission time.The
task_def.ttlfield is still propagated to the reclaimed claim so that downstream inspection reflects the current submission's preferences, but the actual eviction timestamp remains stable.Testing
All 296 existing tests pass (45 skipped for Redis). The two new tests verify that stale claim reclamation works correctly for both sync and async paths with
cache=True.