python3 support and improved tests#95
Open
Jwink3101 wants to merge 11 commits into
Open
Conversation
jedbrown
approved these changes
Oct 11, 2019
Owner
jedbrown
left a comment
jedbrown
left a comment
There was a problem hiding this comment.
Works for me; thanks. Just one comment. (Sorry about delayed review.)
Open
MateuszKrawczuk
added a commit
to CybrixSystems/git-fat
that referenced
this pull request
May 7, 2026
Applies the strategy from upstream PR jedbrown#95: introduce `touni()`/`tobytes()` helpers plus `from __future__ import unicode_literals`, then convert bytes/str only at I/O boundaries (subprocess output, filter stdin/stdout, rsync stdin). The encode/decode logic itself stays unchanged. - drop the explicit Python 3 rejection guard - open fat object files in binary mode (rb/wb) - use sys.stdin.buffer / sys.stdout.buffer in clean/smudge filters - feed bytes to Popen.communicate() in push/pull - decode rev-parse, ls-files, cat-file, git config output to str - in cmd_index_filter, decode hash-object stdout before %-formatting (avoids the latent "b'...'" bug present in PR jedbrown#95) test.sh writes binary content to a.fat and asserts via base64 to catch text-mode regressions. .gitignore covers TEST_py* and fat-test*/ used by upstream's run_test.py. Verified end-to-end on Python 3.12: clean/smudge filters, push, clone, pull, verify (with a corrupted object), status --all, and the retroactive flow (find + index-filter + filter-branch). Signed-off-by: Mateusz Krawczuk <mateusz.krawczuk@cybrixsystems.com>
MateuszKrawczuk
pushed a commit
to CybrixSystems/git-fat
that referenced
this pull request
May 7, 2026
Adds Python 3 support to git-fat while remaining compatible with Python 2.7. Introduces touni()/tobytes() helpers and unicode_literals, then converts bytes/str only at I/O boundaries (subprocess output, filter stdin/stdout, rsync stdin) so the encode/decode logic itself stays unchanged. - drop the explicit Python 3 rejection guard - open fat object files in binary mode (rb/wb) - use sys.stdin.buffer / sys.stdout.buffer in clean/smudge filters - feed bytes to Popen.communicate() in push/pull - decode rev-parse, ls-files, cat-file, git config output to str test.sh writes binary content to a.fat and asserts via base64 to catch text-mode regressions; .gitignore covers the new TEST_py* / fat-test*/ test directories. Verified end-to-end on Python 3.12: clean/smudge filters, push, clone, pull, verify (with a corrupted object), status --all, and the retroactive flow (find + index-filter + filter-branch). Originally posted as jedbrown#95 by Jwink3101. Link: jedbrown#95 [mkrawczuk: skip run_test.py; wrap stdout in touni() before %-formatting in cmd_index_filter manage_gitattributes path to avoid the latent "b'...'" bug; minor message wording] Signed-off-by: Mateusz Krawczuk <mateusz.krawczuk@cybrixsystems.com>
MateuszKrawczuk
pushed a commit
to CybrixSystems/git-fat
that referenced
this pull request
May 7, 2026
Adds Python 3 support to git-fat. Originally based on Jwink3101's PR jedbrown#95 (touni()/tobytes() helpers at I/O boundaries), then trimmed to drop Python 2 compatibility entirely (the team has no py2 users) and extended with the Windows binary-mode handling from Jed Brown's jed/python3-win32-compat branch so collaborators can run git-fat under Git for Windows without CRLF corruption of fat objects. - drop the explicit Python 3 rejection guard - open fat object files in binary mode (rb/wb) - new get_binary_stdio() helper used by clean/smudge filters; on win32 it calls msvcrt.setmode(fd, O_BINARY) on the underlying file descriptors so the C runtime stops translating CRLF - feed bytes to Popen.communicate() in push/pull - decode rev-parse, ls-files, cat-file, git config output to str - defensive bytes->str decode in verbose_stderr (avoids "b'/path/...'" in diagnostics, especially under Windows where getfilesystemencoding() is mbcs/cp1252 rather than utf-8) - gen_large_blobs: numlarge counter starts at 0 (cosmetic, diagnostic) test.sh writes binary content to a.fat and asserts via base64 to catch text-mode regressions; .gitignore covers TEST_py* and fat-test*/. Verified end-to-end on Python 3.12 (Linux): clean/smudge filters, push, clone, pull, verify (with a corrupted object), status --all, and the retroactive flow (find + index-filter + filter-branch). Windows path is not exercised locally; the win32-only branch in get_binary_stdio() is small and inspectable. Originally posted as jedbrown#95 by Jwink3101. Link: jedbrown#95 Link: https://github.com/jedbrown/git-fat/tree/jed/python3-win32-compat [mkrawczuk: drop py2 compatibility (no __future__, no unicode alias, no check_output backport, no io.open import); switch shebang to python3; add Windows-only msvcrt.setmode binary mode in get_binary_stdio() (on top of Jwink3101's sys.stdin.buffer pattern, inspired by Jed Brown commit 3534426); add defensive bytes decoding in verbose_stderr (Jed commit eadb17b); fix numlarge off-by-one (Jed commit b700297); wrap stdout in touni() before %-formatting in cmd_index_filter manage_gitattributes path to avoid the latent "b'...'" bug in PR jedbrown#95] Signed-off-by: Mateusz Krawczuk <mateusz.krawczuk@cybrixsystems.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is take II at adding python3 support. First, the test is updated to use a binary file (the downfall of the last attempt) and then
git-fatis updated to handle python 3. There is also a new test runner to more carefully test python2 and 3 support by changing the shebang and bash path inside the test.Insomuch as the tests have good coverage, everything passes in python2 and 3. I have not tested 2.6 since it was already broken. I have also not tested on Windows