Configure raster cache lock waits and add safe retention

CHANGELOG.md

@@ -7,6 +7,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+### Added
+
+- `cacheLockTimeout` to configure the raster cache lock wait independently of upstream HTTP timeouts.
+- `CachePruner::prune()` for scheduled raster tile retention, including obsolete namespaces and unused metadata.
+
+### Fixed
+
+- Close cache lock handles on successful and failed requests, and retry when a cleaner replaces the metadata inode or removes an empty cache directory.
+
 ## [2.4.1] - 2026-08-28
 
 ### Fixed

README.md

@@ -52,6 +52,8 @@ sending output.
 The configuration defines the behavior of the proxy.
 
 * `cacheServerPath`: Base directory for caching tiles and map assets.
+* `cacheLockTimeout`: (Raster tiles, optional) Maximum seconds to wait for a tile's exclusive cache lock.
+  Defaults to `2.85` for compatibility; set it above the full upstream fetch and processing budget.
 * `ops`: (Raster tiles) Operation pipeline for bitmap tile requests.
 * `mapAssetBasePath`: (Mapbox styles) URL path prefix for proxied style JSON, vector tiles, sprites, and glyphs.
 * `styles`: (Mapbox styles) Named style configurations for `MapboxStyleProxy`.
@@ -135,6 +137,42 @@ requests. HTTP(S) fetches go through Guzzle; `file://` URLs are read from disk.
 For tile pipelines, set `upstreamHttp` at the root of the config. A `src` operation can override it with its own
 `upstreamHttp` block.
 
+### Raster cache retention
+
+`cacheServerTtl` controls freshness: an expired tile is refreshed on its next request, and a failed refresh can
+still serve its cached copy. It does not evict unused files.
+
+Call `CachePruner::prune()` from cron or your application's scheduler to remove raster tile files older than a
+separate retention period. This includes obsolete `cacheServerName` namespaces, old metadata without retained
+tiles, and empty directories. The return value counts deleted files. For example, run this daily:
+
+```php
+use OpenMapsight\TileProxy\CachePruner;
+
+$deleted = CachePruner::prune('/var/cache/mapsight-tile-proxy', 30 * 86400);
+```
+
+Choose a retention period longer than your raster `cacheServerTtl` values and any upstream minimum caching
+requirements. Age is measured from tile writes, not filesystem atime; metadata tracks the last completed request.
+Retention limits the age of stored tiles, not total disk usage. The reserved `mapbox-style-proxy` subtree, including
+its empty directories, is excluded from raster cleanup.
+
+The pruner takes the same per-tile lock as `Base`, skips busy tiles without waiting, and rechecks timestamps under
+the lock. Waiting requests reopen metadata that was removed by the pruner before using the cache. Run cleanup
+only after all processes sharing the cache use this version's locking, and leave `yoloOnLockTimeout` disabled.
+Use a POSIX filesystem supporting `flock()` across those processes. Cache directories are reserved for proxy files;
+symlinks are not followed by the pruner.
+
+For an upstream timeout of ten seconds plus tile processing, set `cacheLockTimeout` to a larger budget, for example:
+
+```jsonc
+"cacheLockTimeout": 15,
+"upstreamHttp": { "timeout": 10 }
+```
+
+Zero means a single non-blocking lock attempt. Negative or non-finite timeouts are rejected. Locks are closed on
+both successful requests and failures; the default remains 2.85 seconds for existing deployments.
+
 ### Error logging
 
 Most failures are handled gracefully in HTTP responses, but are otherwise silent unless logging is enabled.

classes/Base.php

@@ -45,23 +45,13 @@ public static function handleTileRequest(
         $cachePath = static::getCachePath($cfg, $reqArgs);
 
         $metadataPath = $cachePath . '-.metadata';
-        Utils::mkdirp(dirname($cachePath));
-        $lockH = fopen($metadataPath, 'cb');
-        for ($lockTry = 1; ; ++$lockTry) {
-            if (flock($lockH, LOCK_EX | LOCK_NB) === true) {
-                break;
-            }
-
-            if (10 <= $lockTry) {
-                if (isset($cfg['yoloOnLockTimeout']) && $cfg['yoloOnLockTimeout'] === true) {
-                    @error_log('Yoloed lock for "' . $metadataPath . '"' . "\n");
-                    break;
-                }
-
+        $lock = FileLock::acquire($metadataPath, (float)($cfg['cacheLockTimeout'] ?? 2.85));
+        if ($lock === null) {
+            if (isset($cfg['yoloOnLockTimeout']) && $cfg['yoloOnLockTimeout'] === true) {
+                @error_log('Yoloed lock for "' . $metadataPath . '"' . "\n");
+            } else {
                 throw new RuntimeException('Can not lock "' . $metadataPath . '"');
             }
-
-            usleep($lockTry * $lockTry * 10 * 1000);
         }
 
         try {
@@ -83,10 +73,13 @@ public static function handleTileRequest(
 
             $res->checkpointCache();
             Metadata::save($meta);
+            if (!touch($metadataPath)) {
+                throw new RuntimeException('Could not update cache access time "' . $metadataPath . '"');
+            }
 
             return static::buildTileResponse($res);
         } finally {
-            flock($lockH, LOCK_UN);
+            $lock?->release();
         }
     }
 

classes/CachePruner.php

@@ -0,0 +1,184 @@
+<?php
+declare(strict_types=1);
+
+namespace OpenMapsight\TileProxy;
+
+use DirectoryIterator;
+use InvalidArgumentException;
+use RecursiveCallbackFilterIterator;
+use RecursiveDirectoryIterator;
+use RecursiveIteratorIterator;
+use RuntimeException;
+use SplFileInfo;
+use UnexpectedValueException;
+
+final class CachePruner
+{
+    /** Delete raster cache files older than maxAgeSeconds; return the number of files deleted. */
+    public static function prune(string $cacheServerPath, int $maxAgeSeconds): int
+    {
+        if ($maxAgeSeconds <= 0) {
+            throw new InvalidArgumentException('Cache retention must be positive');
+        }
+
+        if (!is_dir($cacheServerPath)) {
+            return 0;
+        }
+
+        // Use the same separators for iterator paths and the reserved-subtree comparison.
+        $cacheServerPath = rtrim($cacheServerPath, '/') . '/';
+        $cutoff = time() - $maxAgeSeconds;
+        $deleted = 0;
+        // Mapbox asset writers do not share raster tile locks, even in the same cache root.
+        $mapboxCachePath = $cacheServerPath . 'mapbox-style-proxy';
+        $directories = self::openDirectory(
+            $cacheServerPath,
+            static fn (): RecursiveDirectoryIterator => new RecursiveDirectoryIterator(
+                $cacheServerPath, RecursiveDirectoryIterator::SKIP_DOTS
+            )
+        );
+        if ($directories === null) {
+            return 0;
+        }
+
+        $files = new RecursiveIteratorIterator(
+            new RecursiveCallbackFilterIterator(
+                $directories,
+                static fn (SplFileInfo $file): bool => $file->getPathname() !== $mapboxCachePath
+            ),
+            RecursiveIteratorIterator::CHILD_FIRST,
+            RecursiveIteratorIterator::CATCH_GET_CHILD
+        );
+
+        foreach ($files as $file) {
+            if ($file->isLink()) {
+                continue;
+            }
+
+            if ($file->isDir()) {
+                $deleted += self::pruneDirectory($file->getPathname(), $cutoff);
+                // Non-empty directories and directories in use are left in place.
+                @rmdir($file->getPathname());
+            }
+        }
+
+        $deleted += self::pruneDirectory($cacheServerPath, $cutoff);
+        return $deleted;
+    }
+
+    private static function pruneDirectory(string $path, int $cutoff): int
+    {
+        $files = self::openDirectory($path, static fn (): DirectoryIterator => new DirectoryIterator($path));
+        if ($files === null) {
+            return 0;
+        }
+
+        $groups = [];
+        foreach ($files as $file) {
+            if ($file->isLink() || !$file->isFile()) {
+                continue;
+            }
+
+            if (preg_match('/^(\d+)-\.metadata$/D', $file->getFilename(), $matches) === 1) {
+                $groups[$matches[1]]['metadata'] = $file->getPathname();
+            } elseif (preg_match('/^(\d+)-.*-\d+$/D', $file->getFilename(), $matches) === 1) {
+                $groups[$matches[1]]['tiles'][] = $file->getPathname();
+            }
+        }
+
+        $deleted = 0;
+        foreach ($groups as $group) {
+            if (isset($group['metadata'])) {
+                $deleted += self::pruneTile($group['metadata'], $group['tiles'] ?? [], $cutoff);
+            }
+        }
+
+        return $deleted;
+    }
+
+    /**
+     * @template T of DirectoryIterator
+     * @param callable(): T $open
+     * @return T|null
+     */
+    private static function openDirectory(string $path, callable $open): ?DirectoryIterator
+    {
+        for ($attempt = 0; ; ++$attempt) {
+            try {
+                return $open();
+            } catch (UnexpectedValueException $error) {
+                clearstatcache(true, $path);
+                if (!is_dir($path)) {
+                    return null;
+                }
+                // A request may have recreated the directory after the failed open. Retry once.
+                if ($attempt > 0) {
+                    throw $error;
+                }
+            }
+        }
+    }
+
+    /** @param list<string> $tiles */
+    private static function pruneTile(string $metadataPath, array $tiles, int $cutoff): int
+    {
+        $lock = FileLock::acquire($metadataPath, 0, false);
+        if ($lock === null) {
+            return 0;
+        }
+
+        try {
+            $deleted = 0;
+            $hasTiles = false;
+
+            foreach ($tiles as $tilePath) {
+                if (self::isExpired($tilePath, $cutoff)) {
+                    self::delete($tilePath);
+                    ++$deleted;
+                } else {
+                    $hasTiles = true;
+                }
+            }
+
+            // A failed request can checkpoint a new namespace after the scan without touching
+            // metadata. Recheck the directory under the lock before removing its lock file.
+            if (!$hasTiles && self::isExpired($metadataPath, $cutoff) && !self::hasTiles($metadataPath)) {
+                self::delete($metadataPath);
+                ++$deleted;
+            }
+
+            return $deleted;
+        } finally {
+            $lock->release();
+        }
+    }
+
+    private static function hasTiles(string $metadataPath): bool
+    {
+        $tilePattern = '/^' . preg_quote(basename($metadataPath, '-.metadata'), '/') . '-.*-\d+$/D';
+        foreach (new DirectoryIterator(dirname($metadataPath)) as $file) {
+            if (preg_match($tilePattern, $file->getFilename()) === 1 && !$file->isLink() && $file->isFile()) {
+                return true;
+            }
+        }
+
+        return false;
+    }
+
+    private static function isExpired(string $path, int $cutoff): bool
+    {
+        clearstatcache(true, $path);
+        if (is_link($path)) {
+            return false;
+        }
+        $mtime = @filemtime($path);
+        return $mtime !== false && $mtime < $cutoff;
+    }
+
+    private static function delete(string $path): void
+    {
+        if (!@unlink($path)) {
+            throw new RuntimeException('Could not delete cache file "' . $path . '"');
+        }
+    }
+}

classes/FileLock.php

@@ -0,0 +1,89 @@
+<?php
+declare(strict_types=1);
+
+namespace OpenMapsight\TileProxy;
+
+use InvalidArgumentException;
+use RuntimeException;
+
+/** @internal */
+final class FileLock
+{
+    /** @param resource|null $handle */
+    private function __construct(private mixed $handle)
+    {
+    }
+
+    public static function acquire(string $path, float $timeout, bool $create = true): ?self
+    {
+        if (!is_finite($timeout) || $timeout < 0) {
+            throw new InvalidArgumentException('Lock timeout must be finite and non-negative');
+        }
+
+        $deadline = hrtime(true) / 1e9 + $timeout;
+        do {
+            if ($create) {
+                try {
+                    Utils::mkdirp(dirname($path));
+                } catch (RuntimeException $error) {
+                    // A pruner can remove the directory between mkdirp()'s mkdir() and is_dir().
+                    // Let the open attempt below handle recovery within the same lock budget.
+                    if (hrtime(true) / 1e9 >= $deadline) {
+                        throw $error;
+                    }
+                }
+            }
+
+            $handle = @fopen($path, $create ? 'c+b' : 'r+b');
+            if ($handle === false) {
+                if (!$create) {
+                    return null;
+                }
+
+                // The parent may have been removed before fopen() and recreated before this check.
+                clearstatcache(true, dirname($path));
+                if (hrtime(true) / 1e9 >= $deadline) {
+                    throw new RuntimeException('Could not open lock file "' . $path . '"');
+                }
+            } else {
+                if (flock($handle, LOCK_EX | LOCK_NB) && self::isCurrentFile($handle, $path)) {
+                    return new self($handle);
+                }
+
+                fclose($handle);
+            }
+
+            $remaining = $deadline - hrtime(true) / 1e9;
+            if ($remaining <= 0) {
+                return null;
+            }
+
+            usleep((int)min(50000, ceil($remaining * 1e6)));
+        } while (true);
+    }
+
+    /** @param resource $handle */
+    private static function isCurrentFile($handle, string $path): bool
+    {
+        clearstatcache(true, $path);
+        $current = @stat($path);
+        $opened = fstat($handle);
+
+        // A cleaner may unlink metadata while another request has its old inode open.
+        return $current !== false && $opened !== false
+            && $current['dev'] === $opened['dev'] && $current['ino'] === $opened['ino'];
+    }
+
+    public function release(): void
+    {
+        if (is_resource($this->handle)) {
+            fclose($this->handle);
+            $this->handle = null;
+        }
+    }
+
+    public function __destruct()
+    {
+        $this->release();
+    }
+}