Per-user data directories, one process-safe log, no import side effects

[sequoiap]

Moves PyroLab's files out of the installed package and replaces per-process log files with one log that's safe across processes. Importing pyrolab no longer has side effects. Closes #25, #36, #63, #64, #66, #71, #80. Targets master; it doesn't depend on any other PR.

Where files live (#66)

• Per-user folders: configuration, logs, runtime state, nameserver databases and the update-check cache now live in each user's standard folders, found with platformdirs, the new dependency (>=3.0; it has no dependencies of its own). On Windows that's %LOCALAPPDATA%\pyrolab; on Linux, ~/.config/pyrolab and ~/.local/state/pyrolab. Setting PYROLAB_DATA_DIR puts everything in one folder instead.
• pyrolab --data lists every location.
• Importing creates nothing: folders are created when a file is first written. So pip install --upgrade no longer wipes anyone's config, and a read-only install no longer fails on import.
• One-time copy for existing installs: the first pyrolab command or daemon start after upgrading copies the old pyrolab/data/local config and nameserver databases to the new locations, and says so. It never overwrites anything, and leaves the originals in place. A marker file stops it repeating, so a config removed with config reset isn't brought back. Logs and runtime state aren't copied.

One log, safe across processes (#64, #25)

• One file, one writer: the background daemon is the only process that writes the log, a single pyrolab.log that rotates at 5 MB and keeps 5 old files. Before, each process wrote its own pyrolab_<pid>.log (166 in one checkout), rotating at 30 KB.
• Each child has its own pipe: every nameserver or daemon process sends its log records to the daemon over a private pipe (logs.log_to_pipe in the child, logs.collect_from_pipe in the daemon).
• Why not loguru, or the standard library's shared queue? I prototyped both, since loguru's enqueue=True looked attractive. They work fine until a child is killed partway through writing a log record. The killed child dies holding the shared queue's lock, and from then on every other process blocks the moment it logs, including the background daemon. Make process lifecycle robust: escalating stop, restart backoff, non-fatal registration #93 kills hung children by design, so this would happen eventually. In my tests, both deadlocked in the first round of 20 under spawn and fork, while the pipe-per-child design survived all 20. test_logs.py now checks this with real processes.
• One JSON object per line, which makes reading the log back reliable.

Importing pyrolab changes nothing global (#63)

• What's left on import: the package only adds a NullHandler to the pyrolab logger. It no longer adds a handler to the root logger, replaces sys.excepthook, or changes warning filters.
• Setup moved to the programs: the pyrolab command, through a new cli:run entry point, and the daemon set these up for themselves. CLI commands print log messages only when PYROLAB_LOGLEVEL is set. Client scripts that want Pyro's remote tracebacks now opt in themselves; the docs show how.

Reading the log (#71, #36)

• pyrolab status (new, Getting errors from the logs shortcut #36's request): whether the daemon is running, and the warnings and errors since it started, one line each. It points to logs show for full tracebacks.
• pyrolab logs show (new): the log since the daemon last started, with --level, --all for earlier runs, and --lines.
• pyrolab logs export FILE: text by default, or --json, across rotated files oldest first. Lines it can't read are counted and skipped instead of crashing the command (pyrolab logs export crashes on any log entry without a parseable timestamp #71).
• pyrolab logs clean: refuses while the daemon is running, since the daemon is writing the log.

get_ip() (#80)

• No more crashes on isolated networks: with no route to the internet, it falls back to the address the machine's hostname resolves to, then to 127.0.0.1. It logs a warning that names host: public as the setting to change.
• A correction to the issue: get_ip() exists for host: public. It picks the address a nameserver or daemon listens on and publishes in its URIs, so it has to be an address clients can reach. Clients connect to the PyroLab server, never to instruments, so the issue's "instrument subnet" point didn't apply. For a server on several networks, the docs now say to set host to the address on the clients' network.

Also

• Docs: envvars.rst became "Files, Logs, and Environment Variables". configuration.rst gained a section on choosing host, and clients.rst a note on remote tracebacks.
• Removed dead code: try_itr (two copies), the commented-out condense_logs, and an unused nameserver constant.
• Tests: the data_dir fixture is now autouse, so no test can touch real per-user folders or copy a developer's real old files.

Found by Windows CI

• A thread leak on Windows: Windows hands a pipe's write end to a spawned child by duplicating it inside the parent. The child takes it over only once it has started up. A child killed during startup never does, so the parent's reader thread waited forever, leaking a thread and a pipe handle each time. This only affected the leaked reader, not other processes' logging. The reader now also checks whether its child is alive: it reads everything left in the pipe, then stops once the child is dead (collect_from_pipe(alive=...)). Only the Windows 3.14 job caught it.
• A deadlock in one of my tests, not in PyroLab: it filled a pipe before starting the reader, which blocks on Windows' small pipe buffers. The CI time limits from Make process lifecycle robust: escalating stop, restart backoff, non-fatal registration #93 turned that into a 15-minute failure with a traceback pointing at the exact line, instead of a 6-hour hang. The test is fixed.

Checks run

• Suite: 432 passed, 9 skipped (Windows: 429 passed, 12 skipped; the extra skips are the fork and Unix-socket variants). pre-commit passes on every file.
• No writes to real user folders: I ran the whole suite with HOME and the XDG_* folders pointed at an empty directory. Nothing was written there.
• Import side effects: a fresh-interpreter test checks the root handlers, excepthook, warning filters and created folders. master fails every one of those checks.
• Real child processes, under both spawn and fork:
  • two children logged 202 records into one file, tracebacks included;
  • three rounds of killing children mid-log, after which the next child and the parent still logged.
• The package: uv build plus twine check --strict pass. The wheel requires platformdirs>=3.0, pyrolab points at pyrolab.cli:run, and no runtime files are bundled.
• End-to-end run with real processes, simulating an upgrade (an old config in the package folder, PYROLAB_DATA_DIR set to an empty folder):
  • the first command copied the config and said so;
  • up autolaunched a healthy daemon and one whose service crashes on import, and records from all of them landed in the single pyrolab.log, with the crash tracebacks from the child processes;
  • status, logs show and logs export worked;
  • the log folder held only pyrolab.log and pyrolabd_output.log;
  • down and logs clean left nothing behind.

🤖 Generated with Claude Code