Skip to content

tecnix: read staged-only dirty paths from the git index - #75

Open
GoodForOneFare wants to merge 1 commit into
mainfrom
gordo-fix-staged-only-index-overlay
Open

GoodForOneFare wants to merge 1 commit into
mainfrom
gordo-fix-staged-only-index-overlay

Conversation

@GoodForOneFare

@GoodForOneFare GoodForOneFare commented Oct 5, 2026 •

Copy link
Copy Markdown
Member

Motivation

Fixes dev up in World failing with tec build errors like path '«unknown»/system/tectonix/modules/dev.nix' does not exist. It happens when a sparse checkout is mid-operation: most often an uncommitted merge of main (e.g. one stopped on a conflict), but also a stopped cherry-pick, revert or rebase. Agents in secondary worktrees hit it most, because they merge main and leave the merge uncommitted.

Context

The repo accessor sends every path git status reports to the working tree. During an uncommitted merge in a sparse checkout, git stages the cleanly merged files outside the sparse set but never writes them: they are skip-worktree entries whose only copy is the index blob. Evaluation then fails on the first one it reads, and only committing the operation clears it.

Changes

  • Parse git status --porcelain=v2 -z instead of v1, keeping each entry's status and index blob. Both names of a rename are dirty; the source of a copy is not (classified by the score field, so C. and .C alike). Unmerged and untracked paths are dirty as before.
  • An ordinary, renamed or copied entry whose worktree column is ., and whose index entry is a regular file, executable or symlink, has no change of its own against the index, so its content is that blob. readFile, readLink, lstat, directory listings and the dirty fingerprint for those paths are served from the index through the GitRepo the accessor already holds.
  • Directories that exist only in the index (a new directory the merge adds outside the sparse set, or a file it replaces with a directory) lstat as directories. A directory the merge replaces with a file now fails readDir with "not a directory", as the committed tree does, instead of listing nothing. That last case was also wrong in fully materialized checkouts.
  • Staged entries fingerprint by blob id and mode rather than content, so a directory fingerprint never reads staged blobs and doesn't depend on whether files are materialized. Staged files get a new fingerprint once (a one-time cache miss).
  • Worktree-modified, unmerged, untracked and deleted entries stay on disk, as do all dirty paths in worldtree sandboxes, which have no libgit2 repository to read blobs from.
  • Edge case: a file that is skip-worktree, staged and also edited locally now evaluates as the staged content, which matches git's view of it.

@GoodForOneFare
GoodForOneFare force-pushed the gordo-fix-staged-only-index-overlay branch 5 times, most recently from 13a65c8 to fcf29ae Compare October 6, 2026 11:44
The repo accessor sends every path `git status` reports to the working
tree. During an uncommitted merge (or cherry-pick, revert, rebase stop)
in a sparse checkout, git stages the cleanly merged files outside the
sparse set but never writes them: they are skip-worktree entries whose
only copy is the index blob. Evaluation then fails on the first one it
reads with `path '«unknown»/<file>' does not exist`, and only committing
the operation clears it. In World this reaches `tec build :__devEnv` in
any sparse worktree that merges main without committing.

Parse `git status --porcelain=v2 -z` instead of v1, keeping each
entry's status and index blob. An ordinary or renamed entry whose
worktree column is `.` has no change of its own against the index, so
its content is that blob: serve readFile, readLink, lstat, directory
listing and the dirty fingerprint for those paths from the index
through the GitRepo the accessor already holds. For a materialized file
the bytes are the same as on disk, so fingerprints are unchanged.
Worktree-modified, unmerged, untracked and deleted entries stay on
disk, as do all dirty paths in worldtree sandboxes, which have no
libgit2 repository to read blobs from.

Co-authored-by: Gord Pearson <gord.pearson@shopify.com>
@GoodForOneFare
GoodForOneFare force-pushed the gordo-fix-staged-only-index-overlay branch from fcf29ae to d10f40c Compare October 6, 2026 12:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants