Skip to content

pathlib: use PID liveness as primary stale-lock signal - #14937

Open
jawauntb wants to merge 2 commits into
pytest-dev:mainfrom
jawauntb:jb/tmpdir-pid-liveness
Open

pathlib: use PID liveness as primary stale-lock signal#14937
jawauntb wants to merge 2 commits into
pytest-dev:mainfrom
jawauntb:jb/tmpdir-pid-liveness

Conversation

@jawauntb

Copy link
Copy Markdown

Refs #14935 (part 2 of 3).

What

ensure_deletable in _pytest/pathlib.py now reads the PID stored in a
session's .lock file and probes it for liveness (os.kill(pid, 0) on
POSIX, OpenProcess on Windows) as its primary signal. The historical
LOCK_TIMEOUT = 3 days mtime check is retained as a fallback for locks
whose contents cannot be parsed (empty, corrupted, partially written).

Why

The current implementation treats a .lock as dead purely by mtime age:

if lock_time < consider_lock_dead_if_created_before:
    lock.unlink(); return True

Two failure modes fall out of that:

  1. Dead scratch survives three days. A crashed session's directory
    isn't reclaimable until three days after its lock was last touched, even
    though the process is obviously gone.
  2. Live scratch gets reaped mid-run. A long-running suite whose .lock
    mtime crosses the three-day threshold can have its own scratch removed
    by a concurrent cleanup.

Both are fixed by the same one-line diagnosis: pytest already writes the
PID into the lock (create_cleanup_lock in pathlib.py); it just never
reads it back.

Behavior matrix

Old (mtime only) New (pid primary)
Lock owner alive, mtime new not deletable not deletable
Lock owner alive, mtime > 3d deletable (wrong) not deletable
Lock owner dead, mtime new not deletable (waits 3d) deletable (correct, immediate)
Lock owner dead, mtime > 3d deletable deletable
Lock unreadable/corrupt mtime path mtime path (fallback preserved)

Windows

Windows liveness uses OpenProcess(PROCESS_QUERY_LIMITED_INFORMATION, …)
plus GetExitCodeProcess. ERROR_INVALID_PARAMETER (87) is the canonical
"no such process" return; any other failure is treated conservatively as
"assume alive" so unfamiliar edge cases fall through to the mtime fallback
rather than aggressively deleting.

PID recycling

If a lock's PID has been recycled to a different process, _pid_alive
returns True and cleanup waits. That's conservative but safe: the mtime
fallback still takes over after LOCK_TIMEOUT. This does not regress any
existing behavior.

Tests

  • test_cleanup_locked updated: with the current process's PID in the
    lock, ensure_deletable returns False regardless of mtime threshold —
    which is the point of the change.
  • test_cleanup_dead_pid_deletable_regardless_of_mtime — asserts a lock
    naming a nonexistent PID is deletable immediately, without waiting for
    LOCK_TIMEOUT.
  • test_cleanup_unreadable_lock_falls_back_to_mtime — asserts the mtime
    path still applies when the lock's contents can't be parsed.

Local runs: test_pathlib.py + test_tmpdir.py + test_pytester.py
223 passed, 3 skipped, 1 xfailed.

Scope

Independent of #14936 (the .origin sidecar). Either can land first.
The third planned PR (per-rootdir retention layout) is not in this change.

Changelog

changelog/14935.improvement.rst — same tracking issue as the other two
PRs; towncrier will render them together.

🤖 Generated with Claude Code

https://claude.ai/code/session_017o9EnCzHyTdYKdcXshMyMZ

@psf-chronographer psf-chronographer Bot added the bot:chronographer:provided (automation) changelog entry is part of PR label Aug 25, 2026
`ensure_deletable` previously treated a `.lock` file as dead purely by
mtime age, with LOCK_TIMEOUT = 3 days. The PID inside the lock was
never consulted. Two failure modes followed:

- A crashed session's scratch survives for three days before it can
  be reaped, even though the process is obviously gone.
- A long-running session (some suites do run for hours) whose lock
  happens to be older than three days can have its own scratch
  reaped mid-run.

This change reads the PID out of the lock file and probes it with
`os.kill(pid, 0)` on POSIX and `OpenProcess` on Windows. If the PID
is provably not running, the lock is unlinked and the directory is
reported deletable regardless of clock. If the PID is alive, the
directory is not deletable regardless of clock. The mtime-based
check remains as a fallback for locks whose contents cannot be
parsed (empty, corrupted, or partially written).

Refs pytest-dev#14935.
@jawauntb
jawauntb force-pushed the jb/tmpdir-pid-liveness branch from c48f281 to 8c91612 Compare August 25, 2026 23:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bot:chronographer:provided (automation) changelog entry is part of PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant