Repository navigation
feat(monitoring): alert when Plex cannot read its media files - #204
Merged
Merged
Conversation
added 2 commits
September 23, 2026 16:39
The last part of #199. On 2026-09-17 a privacy prompt blocked Plex's access to the NAS for 19 hours. plex-watchdog's only health check asks whether the Plex API answers, which Plex can do while it cannot open a single file. Each plex-watchdog poll now asks Plex itself to check a file. It takes the first movie or episode from /library/recentlyAdded (TV seasons come back as <Directory> and have no file), then requests /library/metadata/<ratingKey>?checkFiles=1 with a 30 s timeout. A <Part> with exists="0" or accessible="0", or no answer in time, is a failure. Two in a row send "[<host>] Plex cannot read media files" through alert_transition, with a reminder every 12 hours and a RESOLVED email when a check passes. Asking Plex matters: the watchdog's /bin/bash is a different TCC identity, so reading the NAS from the script could pass while Plex is blocked. No movie or episode in the list, any other request error (a 404 when the item was removed between the two requests), missing attributes, or Plex being unreachable skip the check without counting it. An open alert stays open while Plex is unreachable, so there is no false recovery email. The check runs right after the prefs fetch, before the hash fast path, which returns early on almost every poll. Step 8 used to rebuild state.json from scratch; it now updates its own keys in place, so the new .media_check_failures and .transitions survive a settings change. On TILSIT, checkFiles=1 on the newest movie returns exists="1" accessible="1" in under a second. A run of the new script as operator, in a throwaway HOME against the live Plex, passed the check and exited 0. Tests: 10 new tests in tests/plex-watchdog.bats, run through the launchd-style /bin/bash 3.2 cycle, with a curl mock that answers each Plex URL from a fixture. Moving the check after the fast path fails 5 of them; going back to rebuilding the state fails 8, including the one written for it. All BATS suites pass; shellcheck and shfmt are clean. Docs: plex-watchdog-README.md describes the check; monitoring-README.md updates the watchdog table and the alert_transition note, and says the check is expected, but not tested, to catch a prompt that blocks Plex. Not deployed. Deploy: render plex-watchdog.sh with the setup script's values, diff against /Users/operator/.local/bin/plex-watchdog, back it up, install it (mode 700, operator:staff), and kickstart the agent. Closes #199.
Found after the previous commit. The media check reads state.json and
updates it with jq before anything else. With a corrupt file, jq failed
under set -e and the watchdog exited 5 on every run. Nothing rewrote the
file, so it would never have recovered. The old code rebuilt the file
from scratch in Step 8, so a corrupt file healed on the next full cycle.
The media check now starts from {} when the state is not a JSON object,
the same guard Step 8 already has. The new test writes "not json" to the
state file and expects exit 0 and a rebuilt state; it fails without the
fix. All 45 plex-watchdog tests pass; shellcheck and shfmt are clean.
Advances #199.
Gate Review — PR #204State preservation (lines 437–451): Changed from Media check function (lines 177–273): Integration (line 322): Runs after token extraction, no blocking failure paths. Test coverage: 18 new tests for alert timing, timeout, recovery, skipped cases, corrupted state. No bugs, regressions, security issues, or data loss risk detected. VERDICT: PASS |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The last part of #199 (PR C in the plan). PR A (#200) fixed alert email under launchd; PR B (#201) added stall-watchdog for privacy prompts, long
transmission-doneruns and the podman supervisor.What changes
Each plex-watchdog poll now asks Plex itself whether it can read a media file:
GET /library/recentlyAdded(first 10). Take the first<Video>. TV seasons come back as<Directory>, with no file to check.GET /library/metadata/<ratingKey>?checkFiles=1, 30 s timeout.exists="0",accessible="0"or a timeout is a failure. Two in a row send[<host>] Plex cannot read media filesthroughalert_transition, with a 12-hour reminder and aRESOLVED:email on recovery. No movie or episode in the list, any other request error (for example a 404 when the item was removed between the calls), missing attributes, or Plex unreachable skip the check without counting it.Asking Plex matters: the watchdog runs as
/bin/bash, a different TCC identity from Plex, so reading the NAS from the script could pass while Plex is blocked.Two fixes to the existing script came with it:
state.jsonfrom scratch, which would have dropped.transitionsand.media_check_failureson every settings change. It now updates its keys in place. The second commit makes the media check start from{}on a corrupt state file; without it the watchdog exited 5 on every run and never recovered.Testing
tests/plex-watchdog.bats(45 total), run through the launchd-style/bin/bash3.2 cycle. The curl mock answers each Plex URL from a fixture.checkFiles=1on the newest movie returnsexists="1" accessible="1"in under a second. The new script, run once as operator in a throwaway HOME against the live Plex, passed the check and exited 0.Not tested: whether a real privacy prompt makes
checkFilesfail. The 2026-09-23 spot check foundcheckFiles=1answering while a prompt was open. The monitoring README says this.Deploy (not done)
Render
plex-watchdog.shwith the setup script's values, diff against/Users/operator/.local/bin/plex-watchdog, back it up, install it (mode 700, operator:staff), and kickstartcom.tilsit.plex-watchdog.Closes #199.