Port dream setup to Python on Windows - #33
Xingdi (Eric) Yuan (xingdi-eric-yuan) merged 16 commits into
Conversation
Cross-platform Python port of the bash dream-setup.sh. Prints the dream context as JSON (the eval/export env mode is dropped; JSON is the only output). Mirrors the .sh contract: slug/namespace validation, .shadow/ gitignore guard (child-path probe), default-branch detection, namespace resolution (env > TASK_INFO.json > .env > basename), external worktree path, idempotent create-with-retry, and RUN_PREFIX detection. Pins UTF-8 on stdout/stderr and imports the shared _worktree_safety gate (not a subprocess). Auto-GC prefers a dream-gc.py sibling and otherwise falls back to the bash dream-gc.sh; it is best-effort and never breaks setup, cleanly no-opping where no usable shell exists. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 35800a4a-a5d9-4e3a-b0e3-0d093e083b24
Port test_dream_setup_sh.py to invoke dream-setup.py via sys.executable (no bash), so the suite runs on both OSes. The base env starts from the real environment (Python + git need SystemRoot/PATH/TEMP on Windows), pins git config to devnull, and clears inherited DREAM_* vars. Worktree registration is compared via resolved Paths since `git worktree list` prints forward-slash paths on Windows. The auto-GC throttle/opt-out tests (dream-setup's own logic) run on every OS. The two tests that assert an orphan is actually swept still need the bash dream-gc.sh, so they sit in a requires_bash class skipped on Windows. The eval-stdout-purity test becomes a JSON-stdout-purity assertion. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 35800a4a-a5d9-4e3a-b0e3-0d093e083b24
Point the dream skill at the ported dream-setup.py and its JSON contract: frontmatter scripts list, the helper table, the .shadow/ gitignore-guard note, the Script-Failure-Recovery list, the setup-script finder, and the two setup examples now run `dream-setup.py` and parse the JSON object it prints instead of eval-ing shell exports. References to dream-gc.sh being auto-triggered now name dream-setup.py. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 35800a4a-a5d9-4e3a-b0e3-0d093e083b24
The Python dream-setup.py fully replaces the bash entry point, so drop dream-setup.sh and its bash-only test_dream_setup_sh.py. This also retires that module's requires_bash Windows skip. dream-cleanup.sh and dream-gc.sh remain until they are ported to Python. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 35800a4a-a5d9-4e3a-b0e3-0d093e083b24
Route documented setup through a Python interpreter, document the JSON-to-variable mapping, canonicalize repo roots before isolation checks, and avoid throttling auto-GC when the sweeper cannot launch. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Use one platform-aware root resolver for setup and reconciliation, propagate it to auto-GC, and preserve failed cleanup worktrees for recovery. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Protect root propagation, failed auto-GC retry behavior, controlled TASK_INFO validation, safe bridge parsing, and cleanup reporting. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Document the NUL-safe JSON bridge and pass the setup-selected root to reconciliation and Bash cleanup commands. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Canonicalize shared worktree roots and prevent the POSIX GC fallback from running on Windows, avoiding cleanup paths that cannot safely interpret Windows worktree metadata. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Resolve namespaces consistently across setup and reconciliation, propagate the setup namespace explicitly, and prevent cleanup from touching indexed branches outside the selected namespace. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Preserve slug-colliding worktrees whenever Git reports a detached or indeterminate registration, and normalize worktree identity for Windows path casing. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Exercise setup JSON root propagation through reconciler cleanup with a real worktree and bare remote. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Preserve the Python setup and canonical root/namespace flow alongside current pinned tooling, coherent lineage retention, and streamlined documentation. Add cross-history integration coverage. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Xingdi (Eric) Yuan (xingdi-eric-yuan)
left a comment
Xingdi (Eric) Yuan (xingdi-eric-yuan)
left a comment
There was a problem hiding this comment.
Merged current main (4d863a3, including #38 and #39) into this branch without rewriting its history. The resolution preserves the Python setup port, shared namespace/native worktree-root handling, pinned current tooling, coherent ancestor retention, and streamlined docs. The PR is now mergeable. Added coverage for setup dependencies in both pinned installed layouts and for coherent ancestors across namespaces.
467 combined setup/reconciliation/tooling/validator/Nap tests passed. The previously identified TMPDIR-sensitive test was excluded from that combined run and reproduced separately as a failure. I re-ran the production findings against the merged tree using private repositories and local bare remotes; the three comments below still apply and were deliberately not folded into the conflict resolution.
| # Remove the registered worktree before deleting its branch. Git | ||
| # refuses to delete a branch that remains checked out in a worktree. | ||
| _gc_worktree_after_merge( | ||
| repo_root, dream_ns, dream_id, branch, worktree_root, | ||
| ) |
There was a problem hiding this comment.
[P1] Preserve unpublished local commits before removing the worktree/branch. This new ordering removes the worktree before the later git branch -D, so Git no longer protects a checked-out local branch. The artifact/index/default-branch checks establish that an earlier experiment was reconciled, not that the current local tip was published.
Reproduced after this merge: push an experiment, commit/push its reconciled artifacts on main, then make one additional local commit in the still-attached dream worktree without pushing. Cleanup removes that worktree and both branch refs; git for-each-ref --contains <unpublished-tip> becomes empty. Against the original base, the local branch survived because branch deletion happened while it was checked out. The commit object still exists temporarily, but its only durable ref is lost.
Before either removal, verify that the local tip is covered by the archived/persisted experiment tip (and avoid removing dirty worktrees). If it is ahead, diverged, or cannot be established as preserved, retain the worktree and branch with an actionable message. Add a real-Git regression for a clean worktree containing an unpushed local commit.
There was a problem hiding this comment.
Fixed in 9ef4e11. cleanup_branches() in dream-reconcile.py now verifies that local and remote-tracking tips are ancestors of the archived tip_commit before deletion. Registered worktrees are removed without --force; an unsafe or failed removal retains the branch. Real-Git regressions cover unpushed/diverged commits, dirty worktrees, and unverifiable archive tips.
| if r.returncode != 0: | ||
| _err("ERROR: Not in a git repository") | ||
| sys.exit(1) | ||
| return os.path.abspath(r.stdout.strip()) |
There was a problem hiding this comment.
[P2] Do not trim filename whitespace from the repository root. stdout.strip() removes legitimate trailing spaces/tabs from the path returned by Git. An existing POSIX repository named source repo works with the old Bash setup, but the Python port returns .../source repo and fails at os.chdir(repo_root) with FileNotFoundError, even with explicit --repo-root and --namespace.
Remove only Git's output terminator while preserving the actual filename, and add a real repository-path regression. I reproduced this unchanged at bcbeeff.
There was a problem hiding this comment.
Fixed in 9ef4e11. _resolve_repo_root() in dream-setup.py now removes only the final newline emitted by Git, preserving actual filename whitespace. Added real-repository regressions for trailing spaces, tabs, and newlines, with both explicit --repo-root and automatic discovery.
| env_extra={ | ||
| "DREAM_GC_AUTO": "0", | ||
| "TEMP": str(system_temp), | ||
| "TMP": str(system_temp), |
There was a problem hiding this comment.
[P2] Isolate TMPDIR as well as TEMP/TMP in this test. _base_env() copies the host environment, and tempfile.gettempdir() checks TMPDIR before these two overrides. With a valid inherited TMPDIR (as on macOS), setup correctly uses that directory and the assertion at line 250 fails. This also creates the worktree outside the directory the test intended to control.
The focused run with a private inherited TMPDIR reproduced the failure; the same test passed when TMPDIR was removed. Set all three variables to system_temp, or clear the higher-priority variable explicitly. This is a test-isolation problem, not a reason to change the production native-temp resolution.
There was a problem hiding this comment.
Fixed in 9ef4e11. test_uses_system_temp_root_without_override() in test_dream_setup.py now sets TMPDIR, TEMP, and TMP to the same test-owned directory. It also seeds a conflicting inherited TMPDIR and verifies that no worktree is created there.
Preserve reconciliation path preflight alongside shared namespace resolution and consolidate the dated changelog entries. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Check branch tips against the archived experiment before cleanup, retain registered worktrees that cannot be safely removed, and isolate TMPDIR in the native-temp test. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Preserve Python 3.9 execution and lossless NUL-delimited worktree paths, correctly account for remote-only cleanup, and reject newline-terminated identifiers. Extend real-Git regression cases and the existing Python 3.9 smoke job. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
Follow-up fixes in f39bb31:
Added regression coverage and Python 3.9 checks for Dream entry points. |
9249b63
into
microsoft:main
3 participants
Summary
Closes #11.
Part of #7.
Ports the Dream setup entry point from Bash to Python.
dream-setup.pynow emits structured JSON instead of shell code, so the workflow can be consumed safely and consistently on Windows, Linux, and macOS.Reviewer guide
The commits are intentionally small and ordered for review:
dream-setup.py, ports the setup tests, switches the skill to JSON consumption, and removesdream-setup.shplus its Bash test.Behavior and safety changes
eval/ shell-export boundary.DREAM_WORKTREE_BASEoverrides are canonicalized before setup emits or cleanup consumes them.dream-gc.shauto-GC fallback until Phase 5 providesdream-gc.py.Scope boundary
This PR does not port
dream-cleanup.shordream-gc.sh; those scripts remain in place and their Python replacements remain Phase 5 (#12). The smalldream-reconcile.pychanges are limited to maintaining the setup-to-cleanup lifecycle contract and preventing destructive behavior exposed by the new Windows-capable setup entry point.Regression coverage added
.envvalues, and multi-namespace cleanup isolation.🤖 Co-authored with GitHub Copilot.