caching: continuous cache eviction, fixes #183

[ThomasWaldmann]

Fixes #183.

Problem

The cache was only cleaned up by Store.open() and Store.close(). While the store was in use, a cache with a size limit grew without bounds - reading a lot of data used as much disk space as all the loaded objects together (a partial load that misses caches the full object).

Change

• For each cache namespace with a max_age or size limit, the Store keeps an in-memory index (name, size, last use) while it is opened. Open and close still scan and clean up the namespace, the scan now also fills the index.
• Before an item is put into the cache (by store() or by a load() cache miss), room is made for it: expired items first, then least recently used items. So the namespace total stays <= size also while the store is in use. The per-operation cost is a dict operation, the cache backend is not listed.
• Items bigger than size are not cached (and a previous value of that name is removed from the cache).
• The index is ordered by the last use by this store. The atime only orders items the store has not used itself yet. Filesystems often only update the atime for the first read after a write (relatime-like; measured on APFS: the second read 1 s later does not change it), so an atime-ordered eviction would evict items that are in active use.
• A cache hit never expires anything (as before): max_age is only checked when something is put into the cache and at open / close.

Shared caches

Several clients may share one cache (e.g. a cache directory used by multiple processes):

• A store rescans a size limited namespace after it has put more than size / 4 bytes into it (CACHE_RESCAN_DIVISOR), so it sees what other clients have added or evicted. With N clients the total can temporarily reach about size * (1 + N / 4). Scan cost measured on a local posixfs cache with levels [1]: 8 ms / 500 items, 35 ms / 5000 items, 317 ms / 50000 items.
• An item another client has evicted is a cache miss and gets cached again; a hit on an item another client has cached adds it to the index.
• Not solved here: clients do not see each other's cache hits (that would need a backend touch API), so a client might evict an item another client frequently uses.

Also

Scanning a namespace the cache backend does not have yet (first open of a fresh cache) is not an error any more - it logged a "cache cleanup failed ... ObjectNotFound" warning and counted a cache error.

Tests

13 new tests in tests/test_cache.py: the size limit holds while loading (the scenario of #183) and while storing, LRU order of this session, partial load miss, items bigger than size, index accounting (overwrite, delete, soft delete / undelete, invalidation), three shared cache cases, max_age eviction when storing, eviction errors, no index without limits, first open.

test_close_cleans_up_lru_cache_items_by_size and test_close_cleans_up_expired_before_lru_size_eviction now fill the cache via a second Store sharing it: they exceeded size while storing, which evicts right away now.

Checked with borg (master, BORG_STORE_CACHE, BORG_PACK_CACHE_SIZE=60000000, extracting 120 MB from packs of up to 51 MB): the pack cache peaks at 50.9 MB, with borgstore 0.6.2 it peaks at 117 MB.

No CHANGES.rst entry yet.

🤖 Generated with Claude Code