Skip to content

fix(explainer-video): scrub PYTHONHOME and PYTHONPATH from pydeps interpreter handover #6162

Description

@kyle-sexton

Problem

scripts/pydeps.py hands over to a different Python than the one that started it, and forwards the
starting interpreter's whole environment to it. When the starting interpreter is a uv-managed
trampoline (for example ~/.local/bin/python3.exe for CPython 3.14), that environment carries a
PYTHONHOME pointing at the 3.14 install. The 3.12 or 3.13 child then loads 3.14's stdlib and dies on
its first import, and the SessionStart notice is a raw traceback.

On origin/main (explainer-video 0.2.2, 551eb2e25):

  • plugins/explainer-video/scripts/pydeps.py:208-209: the handover runs
    subprocess.run([chosen, ...], env={**os.environ, HANDED_OVER: chosen}). PYTHONHOME,
    PYTHONPATH and any UV_INTERNAL__* from the starting interpreter pass straight through.
  • plugins/explainer-video/scripts/pydeps.py:197-198: interpreter() probes each candidate with
    subprocess.run([found, '-c', probe], capture_output=True) and the same inherited environment. A
    supported candidate that crashes at startup under the foreign PYTHONHOME is rejected as
    unsupported, which can end in the misleading "Python 3.12 or 3.13 is required and none is on PATH"
    message at :221-223 even though one exists.
  • plugins/explainer-video/hooks/install-python-deps.sh:24-30 picks the first existing candidate
    (python3.13 python3.12 python3 python), not the first supported one, and relies on pydeps to
    re-pick; :37-38 pastes whatever pydeps printed (here a traceback) into the notice.

Observed notice (explainer-video 0.2.2, Windows, Git Bash):

File ".../explainer-video/0.2.2/scripts/pydeps.py", line 24, in <module>
    import subprocess
File "...\uv\python\cpython-3.14-windows-x86_64-none\Lib\_py_warnings.py", line 49, in <module>
    _use_context = sys.flags.context_aware_warnings
AttributeError: 'sys.flags' object has no attribute 'context_aware_warnings'

Line 24 is import subprocess in the handed-over 3.13 child: 3.13's sys.flags has no
context_aware_warnings, which only 3.14's stdlib reads.

Evidence

Verified this pass (read-only, git show origin/main:...):

Reported by the item and NOT reproduced this pass (no live probes allowed):

  • The uv trampoline sets both PYTHONHOME and UV_INTERNAL__PYTHONHOME to its own install in the
    environment of the Python it launches.
  • Reporter's repro, run from inside the uv 3.14 shim:
    python3 -c "import os,subprocess; subprocess.run([r'C:\...\Python313\python.exe','-c','import subprocess'], env=dict(os.environ))"
    fails the same way.
  • Reporter's workaround: uv python install 3.13 put python3.13.exe in ~/.local/bin, so the
    hook's first candidate is already supported, no handover happens, the hook ran clean (2m13s) and
    pydeps.py check passed all four rows. This masks the bug.

Proposed approach

  1. Add one helper in pydeps.py, e.g. _foreign_env(extra=None), returning os.environ minus
    PYTHONHOME, PYTHONPATH and every key starting with UV_INTERNAL__, plus extra. The child is a
    different interpreter; none of those values can be right for it.
  2. Use it for the candidate probe in interpreter() (:198) and for the handover (:208-209, with
    {HANDED_OVER: chosen} as extra).
  3. Report a crashed handover as a reason plus repair line, in the hook, not in main(). main() does
    not capture the child's output, and capturing it there would swallow render.py's live output under
    run. install-python-deps.sh:37-38 already captures $out: when it contains Traceback, emit
    "explainer-video: the Python handover failed: <last line of $out>; repair: " instead of
    the dump. (Alternative: capture only for install and check in main(); more code, same result.)
  4. Tests in scripts/test_explainer_video_pydeps.py:
    • a platform-independent unit test of the helper: with PYTHONHOME, PYTHONPATH and
      UV_INTERNAL__PYTHONHOME set, none appear in its result and HANDED_OVER does;
    • a posix Launcher case where PYTHONHOME is set to a foreign prefix and the fake python3.12
      logs $PYTHONHOME on both the -c probe and the handover; assert both are empty.
    • extend hooks/install-python-deps.test.sh with a stub pydeps that prints a traceback and exits
      non-zero; assert the notice carries the last line and a repair line, not Traceback.
  5. Version bump in plugins/explainer-video/.claude-plugin/plugin.json and a CHANGELOG entry.

Considered, not part of this fix (follow-up only if wanted):

  • On Windows, prefer py -3.13 / py -3.12 from the launcher, or uv python find 3.13 when uv is
    present, over bare-name PATH order. Here a supported 3.13 existed (python.org and uv-managed), and
    only PATH order decided whether it was found. Scrubbing the env fixes the crash without it.
  • One selection point: the hook and interpreter() both claim to pick "the first supported Python",
    but the hook takes the first existing one and pydeps walks again. Having the hook start any Python
    and let pydeps choose (as it already does) is fine once the handover is safe; collapsing the double
    walk is cleanup. If either change lands, update
    docs/conventions/on-demand-dependencies/README.md:172, which describes the handover.

Acceptance criteria

  • The handover subprocess (pydeps.py main) receives no PYTHONHOME, PYTHONPATH or
    UV_INTERNAL__* key, and does receive EXPLAINER_VIDEO_PYDEPS_INTERPRETER.
  • The candidate probe in interpreter() receives no PYTHONHOME, PYTHONPATH or
    UV_INTERNAL__* key.
  • A unit test of the scrub helper passes on Windows and posix (not skipped on either).
  • A posix launcher test with PYTHONHOME set to a foreign prefix shows an empty PYTHONHOME in
    both the probe and the handed-over child.
  • When the handed-over child exits non-zero with a traceback, the SessionStart notice contains the
    traceback's last line and a repair line, and does not contain the word Traceback.
  • run still streams the script's output live (no capture added on that path).
  • Version bumped and CHANGELOG entry added.

Constraints and gotchas

  • _env() (:86-89) deliberately sets PYTHONPATH to the installed set for the probe, pip and run;
    do not route those through the scrub helper unchanged, or run loses its packages. The scrub
    applies only where pydeps launches a different interpreter (probe and handover). Whether _env()
    should also drop an inherited PYTHONHOME is a judgment call: after the fix the handed-over process
    no longer has one, and when no handover happens the inherited value belongs to that same interpreter.
  • The Windows host cannot run the posix Launcher fakes (shell scripts); that is why the helper test
    must not depend on them.
  • Keep the hook's exit 0 contract (install-python-deps.sh:6): a failed install never blocks the
    session.
  • Test runs must not touch the real plugin data dir; existing tests pass --data-dir and a fixture
    lock.

Context

Source: local handoff item 20261003-220000-explainer-video-pydeps-pythonhome-leak.md (retired into
this issue).

Related: #1678 (closed) hit the same uv-shim PYTHONHOME leak in babysit-prs test wrappers that
call py -3, and was closed as a local-environment artifact, not a repo bug. This case differs: the
shipped pydeps.py itself switches to a different interpreter and forwards the shim's environment to
it, so the fix belongs in the repo. #6017 (open) is about declaring a maximum version in
prerequisites detection, adjacent but separate.

No activity

Activity on this issue will appear here.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    agent-readyFully specified and briefed; eligible for autonomous pickup from the frontier.priority: mediumReal value, no hard deadline; normal backlog flow.work-class: scopedA briefed fix or small feature; blast radius bounded by the brief, tests exist.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions