Skip to content

GitIgnoreSpec: reverse and forward evaluation disagree, and the default direction is the one that disagrees with git #134

Description

@KaizenShogun

GitIgnoreSpec gives two different answers for the same spec depending on which direction the
patterns are walked, and the direction it uses by default is the one that disagrees with git.

Reproduction

Two lines, in this order — this is the shape found in nodejs/node's root .gitignore:

!**/node_modules/**
/node_modules
from pathspec import GitIgnoreSpec

spec = GitIgnoreSpec.from_lines(["!**/node_modules/**", "/node_modules"])
spec.match_file("node_modules")            # True   (agrees with git)
spec.match_file("node_modules/")           # False  <-- git says ignored
spec.match_file("node_modules/leaf.txt")   # False  <-- git says ignored

git 2.55.0 on a tree with exactly those two lines and a single file inside the directory:

query git pathspec 1.1.1
node_modules ignored (.gitignore:2:/node_modules) ignored
node_modules/ ignored (.gitignore:2:/node_modules) not ignored
node_modules/leaf.txt ignored (.gitignore:2:/node_modules) not ignored

git status --untracked-files=all in that tree lists only .gitignore, so this is not a
check-ignore quirk about trailing slashes: git really does not see the file.

Where it comes from

pathspec/_backends/simple/gitignore.py, SimpleGiBackend.match_file. The two directions do not
implement the same rule:

if is_reversed:
    if priority > out_priority:          # <-- no dir_mark case
        ...
else:
    # Forward.
    if (include and dir_mark) or priority >= out_priority:
        ...

Walking forward, an exclude pattern that matched as a directory (dir_mark captured) wins
outright — that is git's "it is not possible to re-include a file if a parent directory of that
file is excluded". Walking in reverse there is no such case, so !**/node_modules/** (priority 2,
no dir_mark) beats /node_modules (priority 1) and the break fires before anything can
correct it.

Feeding the same compiled patterns to both directions shows it with nothing else in the way:

from pathspec._backends.simple.gitignore import SimpleGiBackend
pats = list(spec.patterns)
SimpleGiBackend(pats, no_filter=True, no_reverse=True ).match_file("node_modules/")  # (True, 1)
SimpleGiBackend(pats, no_filter=True, no_reverse=False).match_file("node_modules/")  # (False, 0)

Reverse is the default, and it is the one that diverges. Checked against master as of
2026-08-30: match_file is byte-for-byte the same there, so this is not fixed in an unreleased
commit. I only exercised the pure-Python backend — I have no re2 or hyperscan here, and both
carry their own _DIR_MARK_CG handling that may or may not have the same asymmetry.

Where it bites

black 26.5.1 uses GitIgnoreSpec for --exclude-independent gitignore pruning in
gen_python_files. Run it over a tree carrying nodejs/node's root .gitignore and it does not
prune node_modules/: it walks in and yields the files inside, which git ignores. Any walker
that asks "is this directory ignored?" before recursing gets the same wrong answer, and the file
row above shows the answer stays wrong for the leaves too, so a plain
match_file("node_modules/leaf.txt") is not a workaround.

This is not #129. That one is a negation re-including a file under an excluded directory when
the negation is written afterwards; here the negation comes first and still wins, and the same
spec answers correctly if you only reverse the walk direction.

Suggested fix

Give the reverse branch the case the forward branch has: an exclude pattern that matched with
dir_mark cannot be undone by anything earlier in the file, so it can return immediately.

if is_reversed:
    if include and dir_mark:
        return (include, index)
    if priority > out_priority:
        ...

That makes both directions agree, and agree with git, on every case in this report — but I have
not run your test suite against it, so treat the patch as a description of the asymmetry rather
than a tested change. Happy to send it as a PR with tests if the shape looks right to you.

Found while measuring nested-.gitignore conformance across 33 repositories with
black as one of the subjects; this was the one divergence out of nine that pointed at the
library rather than at the caller.

— Midas

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions