caching

---

title: Caching permalink: /caching parent: Deep Dives nav_order: 8

The Caching Engine

SimpleZipDrive uses a hybrid, tiered caching design so that browsing feels local while keeping memory bounded.

Tier overview

flowchart LR
    R[Read request] --> S{Stored ZIP entry?}
    S -- "Yes" --> Z["Zero-copy direct read<br/>(4 MB read-ahead)<br/>no cache"]
    S -- "No" --> M{Entry size ≤<br/>per-file RAM limit?}
    M -- "Yes" --> MC["Shared memory cache<br/>decompress once, share buffer"]
    M -- "No" --> DC["Disk cache<br/>extract once to temp file"]
    MC -- "memory limit hit / OOM" --> DC

Loading

Seekable sources: .zar (Zstd seekable) and Xbox XISO images (.iso, .xiso, .cso) feed these cache tiers from random-access entry streams — .zar reads decode only the zstd blocks they touch, .cso decodes blocks on demand, and plain .iso files are read straight from their disc sectors. Seeking in the source is always cheap; only the cache tier determines what gets buffered.

1. Zero-copy path for stored ZIP entries

A ZIP entry that is stored (not deflated), not encrypted, and not part of a solid archive is never decompressed at all. The reader opens a seekable window directly over the archive's bytes at the entry's data offset:

• Reads use System.IO.RandomAccess positional reads on the archive's file handle — no stream-position contention between concurrent readers.
• Sequential access triggers a 4 MB read-ahead buffer.
• This gives near-native throughput (measured 500+ MB/s on SSDs) with a tiny, fixed memory footprint.

2. Shared memory cache for small entries

Compressed entries that fit within the per-entry memory limit are decompressed exactly once into an exact-size buffer:

• Decompress-once, share across handles. Every process opening the file gets its own read-only stream over the same buffer, with an atomic reference count. Opening the same file 100 times costs one decompression.
• Warm buffers. When the last handle closes, the buffer stays in the cache (reference count 0) so the next open is instant. It is evicted only when memory is needed — LRU by last-use time.
• First decompression is serialized per entry (a per-entry semaphore) — parallel opens do not duplicate work.
• Exact-size buffers. Decompression writes directly into the final buffer (no intermediate MemoryStream + ToArray() copy). This halved peak memory in 2.9.0: a 424 MB entry now peaks at ≈ one copy of the data instead of two.

Limits:

Limit	Default / rule
Per-entry RAM limit	MaxMemoryPerFileMb setting, 512 MB default, clamped to 1 MB … 90 % of installed RAM
Total memory-cache budget	90 % of total available memory (GC.GetGCMemoryInfo().TotalAvailableMemoryBytes)

When an allocation would exceed the total budget — or the system throws OutOfMemoryException — the entry falls back to the disk cache instead of failing ("Memory limit approaching. Using disk cache for small file '…'").

Task Manager note: the working set after first open is roughly app base + file size for the largest file you touched. The 2.9.0 fix removed the transient double allocation that made memory appear to "double" between runs (see issue #10).

3. Disk cache for large entries

Entries larger than the per-entry limit (or decompressed when memory is exhausted) are extracted once to a file in the session temp directory:

• Location: %LOCALAPPDATA%\SimpleZipDrive\Temp\<pid>_<guid>\<n>.tmp — one directory per mount session.
• Temp files are created with a secure ACL granting full control to the current user only.
• The extracted file is registered in an in-session cache and reused by all subsequent opens ("Reusing existing temporary cache for '…'").
• Free disk space is checked first: "Insufficient disk space to extract file '…'. Available: … Required: …".
• Reads come from the extracted file via regular file I/O, so Windows' own file cache accelerates repeat access.

Cleanup

• On unmount / exit: the session's temp directory is deleted recursively; all disk-cache files are removed; the memory cache is cleared; per-entry semaphores are disposed.
• On startup (orphan sweep): directories matching <pid>_<guid> whose PID no longer exists — or whose PID was reused by a different process — are deleted. This cleans up after crashes without touching live sessions.
• Manual: the Clean Temp Files menu item runs the same sweep on demand.
• Cross-integrity mount folders (%LOCALAPPDATA%\SimpleZipDrive\Mounts\…) are not auto-deleted — empty folders may accumulate and can be removed manually.

Tuning

• Lower MaxMemoryPerFileMb (e.g. 64–128) on RAM-constrained machines — larger files then go to the disk cache instead of RAM (settings).
• Keep it high (512+ MB) for fastest repeated access to big game/media files, provided you have the RAM.
• The zero-copy stored-entry path is unaffected by these limits — it never buffers the whole file.