Skip to content

Commit 73d6632

Browse files
committed
feat!: use guarded Git commands for repository storage
GitPython's Python implementations of repository storage couple its behavior to on-disk formats and object-ID widths. Delegate repository discovery, configuration, references, reflogs, object storage, tree construction, index operations, and revision parsing to `git.cmd.Git` so the default backend works with SHA-1/SHA-256 objects and files/reftable references. Require Git 2.52 or newer, retain the deprecated `GitDB` as an explicit choice, and preserve raw `repo.git` access. Use existing unsafe option/protocol primitives together with operand validation, NUL-framed records, and protected option ordering. Suppress implicit hooks, filesystem monitors, maintenance, lazy fetching, external diffs, and default text conversion in managed plumbing. Explicit commit hooks remain supported; signing and custom archive commands require opt-in. Preserve the established clean/smudge-filter behavior of existing worktree operations. Keep common workflows and map recognizable native failures to established exceptions. Remove or restrict low-level binary index/tree, raw reflog, precompressed object, and direct storage mutation APIs that cannot be exposed faithfully through Git. Preserve semantic index edits through private native indexes, discover worktree storage through Git, and reconnect retained submodule metadata without assuming its object or reference backend. Document the API and CLI changes in `doc/source/changes.rst`, add format and injection coverage, update minimum-Git CI and fuzz tooling, and bound the existing throughput benchmarks for subprocess-based operations. This work is planned for GitPython 3.4 some weeks before Git 3's official release, allowing users to test this branch and report compatibility feedback beforehand. Validation: the full pytest run produced 1,328 passes, 79 skips, one expected failure, and three submodule failures that were resolved and passed reruns. A fresh submodule/offline run had 224 passes; its two metadata-alias failures were fixed and retested, followed by eight passing retained-metadata checks. The Git 2.52 matrix passed 127 tests, plus four later quoted-branch checks. Ruff, mypy, basedpyright, Sphinx with warnings as errors, Python 3.8 package smoke tests, and deterministic fuzz-harness/version-guard checks pass. The updated fuzzing Docker image was not built and no long fuzz campaign was run.
1 parent 2d7f566 commit 73d6632

66 files changed

Lines changed: 3332 additions & 6795 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.basedpyright/baseline.json‎

Lines changed: 0 additions & 144 deletions
Original file line numberDiff line numberDiff line change
@@ -8,75 +8,9 @@
88
"endColumn": 45,
99
"lineCount": 1
1010
}
11-
},
12-
{
13-
"code": "reportArgumentType",
14-
"range": {
15-
"startColumn": 41,
16-
"endColumn": 48,
17-
"lineCount": 1
18-
}
19-
},
20-
{
21-
"code": "reportArgumentType",
22-
"range": {
23-
"startColumn": 32,
24-
"endColumn": 39,
25-
"lineCount": 1
26-
}
27-
},
28-
{
29-
"code": "reportCallIssue",
30-
"range": {
31-
"startColumn": 25,
32-
"endColumn": 46,
33-
"lineCount": 1
34-
}
35-
},
36-
{
37-
"code": "reportArgumentType",
38-
"range": {
39-
"startColumn": 30,
40-
"endColumn": 39,
41-
"lineCount": 1
42-
}
43-
}
44-
],
45-
"./git/db.py": [
46-
{
47-
"code": "reportIncompatibleMethodOverride",
48-
"range": {
49-
"startColumn": 8,
50-
"endColumn": 12,
51-
"lineCount": 1
52-
}
53-
},
54-
{
55-
"code": "reportIncompatibleMethodOverride",
56-
"range": {
57-
"startColumn": 8,
58-
"endColumn": 14,
59-
"lineCount": 1
60-
}
6111
}
6212
],
6313
"./git/index/base.py": [
64-
{
65-
"code": "reportArgumentType",
66-
"range": {
67-
"startColumn": 30,
68-
"endColumn": 36,
69-
"lineCount": 1
70-
}
71-
},
72-
{
73-
"code": "reportArgumentType",
74-
"range": {
75-
"startColumn": 28,
76-
"endColumn": 34,
77-
"lineCount": 1
78-
}
79-
},
8014
{
8115
"code": "reportArgumentType",
8216
"range": {
@@ -186,14 +120,6 @@
186120
"endColumn": 25,
187121
"lineCount": 1
188122
}
189-
},
190-
{
191-
"code": "reportArgumentType",
192-
"range": {
193-
"startColumn": 52,
194-
"endColumn": 84,
195-
"lineCount": 1
196-
}
197123
}
198124
],
199125
"./git/objects/tag.py": [
@@ -274,32 +200,6 @@
274200
}
275201
}
276202
],
277-
"./git/refs/log.py": [
278-
{
279-
"code": "reportArgumentType",
280-
"range": {
281-
"startColumn": 30,
282-
"endColumn": 34,
283-
"lineCount": 1
284-
}
285-
},
286-
{
287-
"code": "reportAttributeAccessIssue",
288-
"range": {
289-
"startColumn": 17,
290-
"endColumn": 22,
291-
"lineCount": 1
292-
}
293-
},
294-
{
295-
"code": "reportArgumentType",
296-
"range": {
297-
"startColumn": 28,
298-
"endColumn": 30,
299-
"lineCount": 1
300-
}
301-
}
302-
],
303203
"./git/refs/reference.py": [
304204
{
305205
"code": "reportIncompatibleVariableOverride",
@@ -310,16 +210,6 @@
310210
}
311211
}
312212
],
313-
"./git/refs/symbolic.py": [
314-
{
315-
"code": "reportAttributeAccessIssue",
316-
"range": {
317-
"startColumn": 15,
318-
"endColumn": 20,
319-
"lineCount": 1
320-
}
321-
}
322-
],
323213
"./git/refs/tag.py": [
324214
{
325215
"code": "reportIncompatibleMethodOverride",
@@ -339,22 +229,6 @@
339229
}
340230
],
341231
"./git/remote.py": [
342-
{
343-
"code": "reportAttributeAccessIssue",
344-
"range": {
345-
"startColumn": 26,
346-
"endColumn": 38,
347-
"lineCount": 1
348-
}
349-
},
350-
{
351-
"code": "reportAttributeAccessIssue",
352-
"range": {
353-
"startColumn": 26,
354-
"endColumn": 38,
355-
"lineCount": 1
356-
}
357-
},
358232
{
359233
"code": "reportAttributeAccessIssue",
360234
"range": {
@@ -373,14 +247,6 @@
373247
}
374248
],
375249
"./git/repo/base.py": [
376-
{
377-
"code": "reportReturnType",
378-
"range": {
379-
"startColumn": 15,
380-
"endColumn": 28,
381-
"lineCount": 1
382-
}
383-
},
384250
{
385251
"code": "reportTypedDictNotRequiredAccess",
386252
"range": {
@@ -446,16 +312,6 @@
446312
}
447313
}
448314
],
449-
"./git/repo/fun.py": [
450-
{
451-
"code": "reportReturnType",
452-
"range": {
453-
"startColumn": 11,
454-
"endColumn": 20,
455-
"lineCount": 1
456-
}
457-
}
458-
],
459315
"./test/deprecation/test_basic.py": [
460316
{
461317
"code": "reportUnusedExpression",

‎.github/workflows/git-cli.yml‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
name: Minimum Git CLI
2+
3+
on:
4+
push:
5+
branches: [main]
6+
pull_request:
7+
workflow_dispatch:
8+
9+
permissions:
10+
contents: read
11+
12+
jobs:
13+
git-2-52:
14+
runs-on: ubuntu-latest
15+
steps:
16+
- uses: actions/checkout@v7
17+
- uses: actions/checkout@v7
18+
with:
19+
repository: git/git
20+
ref: v2.52.0
21+
path: .git-source
22+
- uses: actions/setup-python@v7
23+
with:
24+
python-version: '3.12'
25+
- name: Build minimum supported Git
26+
working-directory: .git-source
27+
run: |
28+
make -j2 prefix="$RUNNER_TEMP/git" NO_GETTEXT=YesPlease NO_TCLTK=YesPlease NO_PERL=YesPlease NO_CURL=YesPlease install
29+
echo "$RUNNER_TEMP/git/bin" >> "$GITHUB_PATH"
30+
- name: Install GitPython and test dependencies
31+
run: python -m pip install ./smmap ./gitdb '.[test]'
32+
- name: Verify native format and safety contracts
33+
env:
34+
GIT_CONFIG_NOSYSTEM: '1'
35+
GIT_CONFIG_GLOBAL: /dev/null
36+
GIT_AUTHOR_NAME: GitPython Tests
37+
GIT_AUTHOR_EMAIL: tests@example.invalid
38+
GIT_COMMITTER_NAME: GitPython Tests
39+
GIT_COMMITTER_EMAIL: tests@example.invalid
40+
run: |
41+
git version
42+
python -m pytest --no-cov --tb=short test/test_cli_objects_index.py test/test_cli_safety.py test/test_reflog.py test/test_config.py test/test_remote_cli.py test/test_submodule.py::test_submodule_cli_lifecycle

‎.github/workflows/pythonpackage.yml‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -58,6 +58,19 @@ jobs:
5858
with:
5959
wsl-version: 1
6060

61+
- name: Install current Git (Linux)
62+
if: matrix.os-type == 'ubuntu'
63+
run: |
64+
sudo add-apt-repository --yes ppa:git-core/ppa
65+
sudo apt-get update
66+
sudo apt-get install --yes git
67+
68+
- name: Install current Git (macOS)
69+
if: matrix.os-type == 'macos'
70+
run: |
71+
brew install git
72+
echo "$(brew --prefix git)/bin" >> "$GITHUB_PATH"
73+
6174
- name: Prepare this repo for tests
6275
run: |
6376
./init-tests-after-clone.sh

‎README.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -40,10 +40,10 @@ The project is open to contributions of all kinds, as well as new maintainers.
4040
### REQUIREMENTS
4141

4242
GitPython needs the `git` executable to be installed on the system and available in your
43-
`PATH` for most operations. If it is not in your `PATH`, you can help GitPython find it
43+
`PATH` for repository operations. If it is not in your `PATH`, you can help GitPython find it
4444
by setting the `GIT_PYTHON_GIT_EXECUTABLE=<path/to/git>` environment variable.
4545

46-
- Git (1.7.x or newer)
46+
- Git (2.52 or newer)
4747
- Python >= 3.8
4848

4949
The list of dependencies are listed in [`./requirements.txt`](https://github.com/gitpython-developers/GitPython/blob/main/requirements.txt) and [`./test-requirements.txt`](https://github.com/gitpython-developers/GitPython/blob/main/test-requirements.txt).

‎doc/source/changes.rst‎

Lines changed: 92 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,98 @@
22
Changelog
33
=========
44

5+
Unreleased: Git CLI migration
6+
=============================
7+
8+
GitPython now delegates repository discovery, references, configuration, object
9+
storage, tree construction, index operations, and revision parsing to Git. The
10+
default backend supports SHA-1 and SHA-256 repositories with either files or
11+
reftable reference storage. Repository object IDs must no longer be assumed to
12+
contain 20 binary bytes or 40 hexadecimal characters.
13+
Legacy class-level ``NULL_BIN_SHA`` and ``NULL_HEX_SHA`` constants retain their
14+
SHA-1 values; do not use their width to interpret repository object IDs.
15+
16+
Git executable and safety
17+
-------------------------
18+
19+
* Git **2.52 or newer** is required for library-managed operations. Older versions
20+
raise ``UnsupportedOperation`` before repository mutation; there is no Python
21+
implementation fallback. ``GIT_PYTHON_GIT_EXECUTABLE`` still selects Git.
22+
* Every managed command uses argument sequences without a shell. Existing unsafe
23+
option/protocol checks remain, and operands and stdin records receive additional
24+
validation. NUL-delimited ``cat-file`` requests prevent newline injection.
25+
Previously accepted option-like revision/ref arguments can now be rejected.
26+
* New plumbing calls suppress implicit hooks, filesystem monitors, automatic
27+
maintenance, and lazy network fetches. Index staging continues to store raw
28+
content rather than introduce clean filters. Diffs disable external diff and
29+
text conversion programs; blame disables text conversion unless unsafe options
30+
are explicitly enabled. Explicit commit hooks use ``git hook run`` and honor
31+
``skip_hooks``; Git combines their output, so ``HookExecutionError`` may carry
32+
former stdout text in stderr. Trailer additions reject executable trailer
33+
configuration instead of invoking it.
34+
* Existing working-tree conversions retain Git's behavior: status and working-tree
35+
diffs can run configured clean filters; checkout, clone, and archive can run
36+
smudge filters. Archive defaults to Git's built-in formats and internal gzip;
37+
custom format commands require ``allow_unsafe_options=True``. Tag creation
38+
suppresses configured signing by default; signing, verification, and editor
39+
options require the same opt-in.
40+
* The raw ``repo.git`` interface remains available for direct Git commands.
41+
Existing explicit unsafe-option and unsafe-protocol opt-ins remain independent.
42+
They do not permit argument or stdin-record injection, or reordering command
43+
options past the library's safety flags.
44+
* Known command outcomes retain established GitPython/configparser exceptions
45+
where distinguishable. Other failures raise ``GitCommandError`` with Git's
46+
exit status and diagnostic output; exact error text may differ.
47+
48+
API changes
49+
-----------
50+
51+
* ``GitCmdObjectDB`` no longer inherits ``LooseObjectDB``. Its object reads,
52+
writes, existence checks, and enumeration use Git, including packed objects.
53+
Precompressed input streams and custom object-output writers are unsupported.
54+
The deprecated ``GitDB`` remains explicitly selectable with its existing
55+
warning and limitations; ``gitdb`` remains a dependency for shared types.
56+
* ``Repo.object_format`` and ``Repo.ref_format`` report Git's storage formats.
57+
``Repo.alternates`` is read-only and reports effective absolute alternate
58+
directories, including environment and transitive alternates. Direct editing
59+
of the alternates file through this property is removed.
60+
* Index entries retain their mode, object ID, path, and stage. Raw stat fields,
61+
arbitrary cache flags/extensions/checksums, ``IndexFileSHA1Writer``, and the
62+
standalone binary index read/write/merge helpers are removed. Use
63+
``IndexFile.from_tree()``, ``IndexFile.new()``, ``write()`` and ``write_tree()``.
64+
Git preserves untouched metadata when an existing index is edited. Temporary
65+
indexes isolate tree/merge operations from the real index and working tree.
66+
``version`` is read-only. ``from_tree()`` accepts ``trivial``, ``aggressive``,
67+
and ``verbose`` options; arbitrary ``read-tree`` keyword forwarding is removed.
68+
* Standalone binary tree parsers, serializers, and multi-tree traversal helpers
69+
are removed. Use ``Tree`` traversal/cache operations and the index interface.
70+
Tree/commit serialization adapters that remain write their objects to the
71+
repository to obtain Git-produced bytes.
72+
Commit creation preserves message bytes through plain stdin. Injecting or
73+
reusing arbitrary ``gpgsig`` headers is unsupported; modified signed commits
74+
become unsigned.
75+
* ``RefLog`` is associated with a reference, not a filesystem path. Keep using
76+
``ref.log()``, ``ref.log_entry()`` and ``ref.log_append()``. ``RefLogEntry`` now
77+
contains ``newhexsha``, ``actor``, ``time`` and ``message``; ``oldhexsha`` and
78+
the old tuple layout are removed. Reads follow Git's commit-reflog view, which
79+
omits entries targeting non-commit or unavailable objects. Old IDs are never
80+
inferred from adjacent entries. Raw reflog file/stream read, write, and rewrite
81+
APIs are removed. Reference updates follow Git's reflog creation/update rules,
82+
including updates when ``logmsg`` is omitted.
83+
* ``GitConfigParser`` uses Git's configuration syntax and canonical key spelling,
84+
and no longer subclasses ``configparser.RawConfigParser``.
85+
Common getters, typed values, duplicate values, file/stream inputs and mutation
86+
methods remain. Invalid Git syntax previously accepted by Python is rejected.
87+
Empty-section creation and valueless writes are removed; valueless reads remain.
88+
Relative includes from streams are rejected by Git. Locks cover each native
89+
mutation rather than the writer object's lifetime.
90+
* ``Submodule.rename()`` is removed. Moving a submodule preserves its logical name,
91+
matching Git. Normal removal retains Git's recoverable submodule metadata.
92+
Re-adding with ``no_checkout=True`` can reuse that metadata without changing refs;
93+
incompatible URLs, branches, and clone-only options are rejected before mutation.
94+
Fetch/pull results are derived from command output rather than ``FETCH_HEAD``
95+
file parsing. Revision strings follow Git's native revision grammar.
96+
597
3.2.0
698
=====
799

‎doc/source/intro.rst‎

Lines changed: 3 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -6,18 +6,16 @@ Overview / Install
66

77
GitPython is a python library used to interact with git repositories, high-level like git-porcelain, or low-level like git-plumbing.
88

9-
It provides abstractions of git objects for easy access of repository data, and additionally allows you to access the git repository more directly using either a pure python implementation, or the faster, but more resource intensive git command implementation.
9+
It provides Python objects for repository data and delegates Git operations to the Git executable. The default backend supports both SHA-1 and SHA-256 object IDs and both files and reftable reference storage. The legacy pure-Python GitDB backend remains available but is deprecated.
1010

1111
The object database implementation is optimized for handling large quantities of objects and large datasets, which is achieved by using low-level structures and data streaming.
1212

1313
Requirements
1414
============
1515

1616
* `Python`_ >= 3.8
17-
* `Git`_ 1.7.0 or newer
18-
It should also work with older versions, but it may be that some operations
19-
involving remotes will not work as expected.
20-
* `GitDB`_ - a pure python git database implementation
17+
* `Git`_ 2.52 or newer
18+
* `GitDB`_ - shared data types and the deprecated legacy object database
2119
* `typing_extensions`_ >= 3.7.3.4 (if python < 3.10)
2220

2321
.. _Python: https://www.python.org

0 commit comments

Comments
 (0)