Repository navigation
Merge dev: dependency security upgrades, gallery-dl image sync, reader grid overview - #71
Merged
Merged
Conversation
Both were hit for real while running this migration. The runbook told the operator to pass --entrypoint python. entrypoint.sh ends in `exec gosu appuser "$@"` and the Dockerfile declares no USER, so overriding the entrypoint runs the container as root and leaves root-owned directories under /data that the worker (uid 1042) can no longer write to. Drop the flag and pass the command as arguments. The runbook also stopped api and worker in step 0 and never brought them back. Restarting them assigns new container IPs, and nginx resolves its upstreams once at start/reload and caches them, so every /api/* request returns 502 while every container still reports healthy — the frontend shows the user as logged out and nothing loads. Container healthchecks cannot see this, so the added step reloads nginx and curls the API for a 401 rather than trusting `docker compose ps`. Also documents unfreezing the scheduled scan and regenerating the info.json sidecars, which the rebuild deletes because reconciliation only writes missing ones. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…iter The redis service passed --appendonly yes but never disabled RDB save points, so both persistence engines ran at once. SAQ's steady write rate (~3.3 setex/s) keeps the dirty counter above the default `save 300 100` threshold at all times, so a full dump.rdb was written every ~301s: 4598 snapshots over 16 days uptime. Startup logs show recovery loads the AOF and never dump.rdb, so those snapshots were pure write amplification. With the dataset at 161 MB -- almost entirely cached image bytes under thumb:proxied / thumb:cdn -- that cost ~46 GB/day of NVMe writes against ~0.9 GB/day for the AOF, and scaled linearly toward ~220 GB/day at the 768 MB maxmemory ceiling. Measured 148 MiB written per snapshot; measured 2.96 MiB per 660s with save points off. ADR-0006's noeviction policy and maxmemory are untouched. scripts/backup.sh had to change in the same commit. Its BGSAVE ran redis-cli without credentials and failed with NOAUTH, hidden by `2>/dev/null || true`. That was harmless only because automatic snapshots kept dump.rdb fresh; with them disabled it would have archived an unboundedly stale dump forever. The BGSAVE now authenticates, waits for rdb_bgsave_in_progress to clear instead of sleeping 2s, and skips the copy rather than archiving a stale file on failure. test_compose_redis_uses_non_evicting_policy matched `image:` and `command:` as adjacent lines, so the new comment broke it. Its helper now locates the keys independently and skips the same-named depends_on anchor. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A Redis restart drops every subscription and redis-py does not resubscribe on its own. The two process-scoped listeners treated that as terminal -- they logged a warning and the coroutine returned, with nothing supervising the task -- so a single restart left the process deaf for the rest of its lifetime and surfaced nothing afterwards. Observed live: pubsub_channels fell to 0 after recreating the redis container and stayed there until api and worker restarted. The request-scoped listeners in routers/ws.py were already fine. They re-raise, websocket_endpoint tears the socket down via asyncio.wait(FIRST_COMPLETED), and the PWA reconnects after 3s, so recovery is delegated to the client. The process-scoped ones had no equivalent supervisor. pubsub_listen_forever() in core/redis_client.py now owns the loop: exponential backoff from 1s to 30s, a per-message guard so one bad payload does not cost the subscription, and an explicit CancelledError re-raise, without which stop_listener() and worker shutdown() would hang on the task they await. Reconnecting alone is not enough. Messages published while the connection was down are gone with no way to replay them, so every (re)subscribe runs a resync hook: site_config drops its whole cache, and the worker re-reads the persisted level through apply_log_level_from_redis(), the same call its startup makes. The site_config half was self-limiting anyway (a 30s cache TTL bounded the staleness); the log level had no such fallback and failed silently. This also fixes startup: start_listener() used to subscribe before creating the task, so a Redis that was unavailable at boot disabled invalidation for the whole process lifetime. The subscribe now happens inside the retry loop. static-defense: not applicable, runtime-only Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…th is shared Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…board nav Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…ore measurement The container-width effect ran in useEffect (post-paint), so the first painted frame could still use the initial containerWidth=0 with the default colCount=3, computing a negative cellWidth/cellHeight. Browsers treat a negative CSS height as invalid and fall back to content-driven auto sizing, collapsing every virtual row on top of the others near the top of the screen — reported on mobile as thumbnails "piled up in the top quarter" of the grid overview. Switch the measurement to useLayoutEffect so it resolves before paint, clamp cellWidth to a non-negative value, and gate rendering the row content on cellWidth > 0 (showing a loading spinner otherwise) so a stuck-at-zero measurement can never reach a rendered cell. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
… stale row-offset cache Screenshot from a real device showed the actual root cause: rows spaced GAP apart (e.g. pages 25/28/31/34, three apart = one per column) were rendering on top of each other. @tanstack/react-virtual's own internal layout effect measures and caches per-row positions on the *first* render, before this component's own useLayoutEffect (which reads the real scrollRef.getBoundingClientRect()) ever runs. That cache is never recomputed later just because a subsequent render passes a different estimateSize — it only invalidates on an explicit DOM remeasurement or a count change. Starting containerWidth at 0 fed the virtualizer a near-zero estimate on that very first pass, permanently caching several rows on top of each other near the top of the scroll area. The prior `cellWidth > 0` JSX gate didn't help: it only prevented this component's own render from calling getVirtualItems(), not the library's internal effect from measuring with the same bad estimateSize. Seed containerWidth/colCount from window.innerWidth (same pattern as VirtualGrid.tsx) so the very first render already uses a real width. Also guard the precise-measurement effect so a 0 reading from getBoundingClientRect/ResizeObserver never overwrites a good estimate with a broken one, and account for the scroll container's own px-4 padding when dividing available width into columns (previously computed against space the grid never actually had). Also add safe-area-inset padding to the header and scroll container, so the close button and content clear the iOS notch/status bar and the phone's bottom navigation/gesture bar. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…ng open Scrolling inside the grid overview (a finger swipe upward) bubbled touch events up to useTouchGesture's listener on the outer reader-container, which was misread as swipe-up-to-back and exited the Reader instead of just scrolling the grid. isDisabled only checked zoom state, not whether the grid overview was open. Add state.isGridOpen to the isDisabled check, matching the guard already applied to the Escape-to-back handler and useKeyboardNav for the same reason. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
.heic is an accepted import extension but the worker's Pillow only decoded AVIF, so every HEIC raised "cannot identify image file" and never got a thumbnail, phash or thumbhash. Add pillow-heif and register its opener when worker.thumbnail loads. Regression test decodes a real ftypheic file. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…cies Clears the open Dependabot alerts: next (RCE in image optimization, next/og and Windows hosts), sharp override 0.35.4 (libheif), undici, js-yaml, dompurify, vitest/@vitest/mocker, and brace-expansion. The remaining braces advisory comes from the eslint-config-next dev toolchain and has no upstream fix; npm's suggested fix is a downgrade. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The in-process gallery-dl (extractor detection) comes from the image while downloads run the venv on a persistent volume, so the two silently drift apart across rebuilds. - Bake gallery-dl and psycopg wheels into the image (/opt/gdl-wheels) and add a GDL_VERSION build arg that pins the version and busts the layer cache. - After worker startup, a background task syncs the venv to the image's version offline when the image is newer. It goes through the same guards as an admin upgrade (no queued/paused/running downloads, exclusive lifecycle lock, re-check before swap) and retries while rejected instead of forcing. - Never downgrade; a .synced_image_version marker keeps an admin rollback from being undone by a restart with the same image. - A fresh venv now installs the image's version instead of a hard-coded one. - GET /version returns image and image_newer; the settings page shows the image version and a notice when it is newer than the venv. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The tagger container was removed in 73fc7d2 (ADR 0011), so there is no service left to mount the NAS into. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Security / dependencies (clears the 15 open Dependabot alerts on
main)nextto 16.3.8 (RCE in image optimization,next/ogand Windows hosts),sharpoverride to 0.35.4 (libheif), plusundici,js-yaml,dompurify,vitest/@vitest/mocker,brace-expansion.bracesadvisory comes from theeslint-config-nextdev toolchain and has no upstream fix (npm's suggested fix is a downgrade); it does not ship in the runtime.gallery-dl image/venv drift
GDL_VERSIONbuild arg (pins the version and busts the layer cache)..synced_image_versionmarker keeps an admin rollback from being undone by a restart.GET /api/admin/gallery-dl/versionreturnsimageandimage_newer; the settings page shows the image version and a notice when it is newer than the venv.Reader / other
Test plan
tsc --noEmit, eslint (0 errors), vitest (165 files / 1236 tests),next builddev: worker synced venv 1.32.8 → 1.32.15 offline on startup;/→ 307,/api/settings/features→ 401, all containers healthymainclose🤖 Generated with Claude Code