-
Notifications
You must be signed in to change notification settings - Fork 149
fix(utils): apply the stdlib data filter on both tarball extraction paths #734
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -169,31 +169,47 @@ def extract_tarball( | |
| shutil.rmtree(extractPath, ignore_errors=True) | ||
| shutil.rmtree(outPath, ignore_errors=True) | ||
|
|
||
| # Both extraction paths below use the stdlib "data" filter so a member with an | ||
| # absolute path or a `../` traversal is rejected instead of being written | ||
| # wherever it points (CVE-2007-4559). | ||
| # | ||
| # This used to be skipped because of https://github.com/python/cpython/issues/107845, | ||
| # where data_filter resolved symlink targets against the destination root rather | ||
| # than against the directory holding the link, and so falsely raised | ||
| # LinkOutsideDestinationError on valid archives. That was a false-rejection bug, | ||
| # never an escape, and it was fixed in 3.10.13 / 3.11.5 / 3.12.0rc2 (2023-08-24). | ||
| # The only affected releases in our range are 3.10.12 and 3.11.4 — and since | ||
| # `extractall(filter=...)` does not exist before 3.10.12 at all, that is the whole | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🟡 Medium — The version reasoning falls short in both directions: Raised by 6 of 8 reviewers (gpt-5.6-sol-max adversarial, kimi-k3-max adversarial, gpt-5.6-sol-max edge-case, gemini-3.1-pro edge-case, claude-opus-5-thinking-max edge-case, kimi-k3-max edge-case). |
||
| # window. On those two, the worst case is a loud error rather than a silent escape. | ||
| if not show_progress: | ||
| with tarfile.open(inPath) as tar: | ||
| tar.extractall(filter=None) | ||
| tar.extractall(filter="data") | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🟠 High — Neither Raised by 4 of 8 reviewers (claude-opus-5-thinking-max adversarial, gpt-5.6-sol-max adversarial, claude-opus-5-thinking-max edge-case, gpt-5.6-sol-max edge-case).
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Not this PR's fault (pre-existing in both paths), but worth a follow-up: |
||
| shutil.move(extractPath, outPath) | ||
| return | ||
|
|
||
| fileSize = inPath.stat().st_size | ||
|
|
||
| _size = 0 | ||
|
|
||
| with _tarball_progress("extracting tarball...", fileSize) as (barProg, barTask, pathProg, pathTask): | ||
|
|
||
| def _filter(tinfo: tarfile.TarInfo, _path: PathLike): | ||
| nonlocal _size | ||
| pathProg.update(pathTask, description=tinfo.path) | ||
| barProg.advance(barTask, _size) | ||
| _size = tinfo.size | ||
|
|
||
| # TODO: ideally we'd use data_filter here, but it's busted: https://github.com/python/cpython/issues/107845 | ||
| # return tarfile.data_filter(tinfo, _path) | ||
| return tinfo | ||
| def _reporting_members(tar: tarfile.TarFile): | ||
| """Yield every member, driving the progress bars as we go. | ||
|
|
||
| Progress reporting used to ride on the ``filter`` argument, which | ||
| meant the extraction filter had to be a custom callable that | ||
| returned members unmodified — silently disabling the CVE-2007-4559 | ||
| checks. The two concerns are separable: ``members`` drives the UI | ||
| and ``filter`` stays the stdlib ``"data"`` filter. | ||
| """ | ||
| size = 0 | ||
| for tinfo in tar: | ||
| pathProg.update(pathTask, description=tinfo.path) | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🟢 Low — Raised by 3 of 8 reviewers (claude-opus-5-thinking-max adversarial, kimi-k3-max adversarial, claude-opus-5-thinking-max edge-case). |
||
| barProg.advance(barTask, size) | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🟢 Low — The bar's total is the compressed archive size ( Raised by 1 of 8 reviewers (gemini-3.1-pro edge-case). |
||
| size = tinfo.size | ||
| yield tinfo | ||
| barProg.advance(barTask, size) | ||
|
|
||
| with tarfile.open(inPath) as tar: | ||
| tar.extractall(filter=_filter) | ||
| barProg.advance(barTask, _size) | ||
| tar.extractall(members=_reporting_members(tar), filter="data") | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🟡 Medium — Raised by 5 of 8 reviewers (claude-opus-5-thinking-max adversarial, kimi-k3-max adversarial, claude-opus-5-thinking-max edge-case, gpt-5.6-sol-max edge-case, kimi-k3-max edge-case). |
||
| pathProg.update(pathTask, description="") | ||
|
|
||
| shutil.move(extractPath, outPath) | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,4 +1,5 @@ | ||
| import io | ||
| import tarfile | ||
| from unittest.mock import MagicMock, patch | ||
|
|
||
| import pytest | ||
|
|
@@ -82,3 +83,79 @@ def test_create_and_extract(self, tmp_path, monkeypatch): | |
|
|
||
| assert (dest / "hello.txt").read_text() == "hello world" | ||
| assert (dest / "sub" / "nested.txt").read_text() == "nested content" | ||
|
|
||
|
|
||
| def _write_member(tar: tarfile.TarFile, name: str, data: bytes) -> None: | ||
| tinfo = tarfile.TarInfo(name) | ||
| tinfo.size = len(data) | ||
| tar.addfile(tinfo, io.BytesIO(data)) | ||
|
|
||
|
|
||
| def _write_symlink(tar: tarfile.TarFile, name: str, target: str) -> None: | ||
| tinfo = tarfile.TarInfo(name) | ||
| tinfo.type = tarfile.SYMTYPE | ||
| tinfo.linkname = target | ||
| tar.addfile(tinfo) | ||
|
|
||
|
|
||
| class TestExtractTarballFiltering: | ||
| """Regression tests for CVE-2007-4559 (see issue #725). | ||
|
|
||
| ``extract_tarball`` extracts into the current working directory, so a member | ||
| named ``../evil.txt`` lands one level above it unless the stdlib ``data`` | ||
| filter rejects the archive. | ||
| """ | ||
|
|
||
| @staticmethod | ||
| def _traversal_tarball(workdir): | ||
| tarball = workdir / "payload.tgz" | ||
| with tarfile.open(tarball, "w:gz") as tar: | ||
| _write_member(tar, "payload/keep.txt", b"benign") | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🟡 Medium — This traversal test only passes because the benign member is written first: Raised by 2 of 8 reviewers (claude-opus-5-thinking-max edge-case, claude-opus-5-thinking-max adversarial). |
||
| _write_member(tar, "../evil.txt", b"pwned") | ||
| return tarball | ||
|
|
||
| @pytest.mark.parametrize("show_progress", [False, True]) | ||
| def test_rejects_path_traversal_member(self, tmp_path, monkeypatch, show_progress): | ||
| """Both extraction paths must refuse to write outside the destination.""" | ||
| workdir = tmp_path / "work" | ||
| workdir.mkdir() | ||
| monkeypatch.chdir(workdir) | ||
|
|
||
| tarball = self._traversal_tarball(workdir) | ||
| escaped = tmp_path / "evil.txt" | ||
|
|
||
| rejection = None | ||
| with patch("comfy_cli.utils.Live"): | ||
| try: | ||
| extract_tarball(tarball, workdir / "out", show_progress=show_progress) | ||
| except tarfile.FilterError as exc: | ||
| rejection = exc | ||
|
|
||
| assert not escaped.exists(), f"traversal member escaped the extraction directory: {escaped}" | ||
| assert rejection is not None, "the traversal member was extracted instead of being rejected" | ||
|
|
||
| @pytest.mark.parametrize("show_progress", [False, True]) | ||
| def test_allows_internal_symlinks(self, tmp_path, monkeypatch, show_progress): | ||
| """Control: the filter must not reject the layout real payloads use. | ||
|
|
||
| python-build-standalone tarballs (the only thing ``StandalonePython`` | ||
| extracts) are full of relative symlinks such as ``bin/python3 -> | ||
| python3.12``. Those stay inside the destination and must survive. | ||
| """ | ||
| workdir = tmp_path / "work" | ||
| workdir.mkdir() | ||
| monkeypatch.chdir(workdir) | ||
|
|
||
| tarball = workdir / "python.tgz" | ||
| with tarfile.open(tarball, "w:gz") as tar: | ||
| _write_member(tar, "python/bin/python3.12", b"#!/bin/sh\n") | ||
| _write_symlink(tar, "python/bin/python3", "python3.12") | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🟢 Low — This control test is meant to back the comment's claim that the 3.10.12/3.11.4 Raised by 2 of 8 reviewers (claude-opus-5-thinking-max adversarial, kimi-k3-max edge-case). |
||
| _write_symlink(tar, "python/bin/python", "python3") | ||
|
|
||
| dest = workdir / "out" | ||
| with patch("comfy_cli.utils.Live"): | ||
| extract_tarball(tarball, dest, show_progress=show_progress) | ||
|
|
||
| assert (dest / "bin" / "python3.12").read_bytes() == b"#!/bin/sh\n" | ||
| assert (dest / "bin" / "python3").is_symlink() | ||
| assert (dest / "bin" / "python").is_symlink() | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🔴 Critical —
extractPathis derived from the archive's own first member (old_name = info.name.split("/")[0]) and handed toshutil.rmtree(..., ignore_errors=True)before any filter runs, sofilter="data"cannot protect it: a tarball whose first member is../evilyieldsold_name == "..", andPath.with_name("..")is accepted, so this recursively deletes the parent of the tarball's directory with the errors swallowed (forcomfy standalone --rehydrate, which passespython.tgz, that is the parent of the user's cwd). An absolute first member such as/etc/passwdinstead yieldsold_name == ""and crashes inwith_name. Validateold_name— reject"",".","..", and any name containing a separator — before using it as an rmtree target.Raised by 5 of 8 reviewers (claude-opus-5-thinking-max adversarial, gpt-5.6-sol-max adversarial, kimi-k3-max adversarial, claude-opus-5-thinking-max edge-case, gpt-5.6-sol-max edge-case).