Skip to content

scripts: add an assert-rewrite diff tool - #14921

Open
RonnyPfannschmidt wants to merge 1 commit into
pytest-dev:mainfrom
RonnyPfannschmidt:ronny/assert-rewrite-diff-tool
Open

scripts: add an assert-rewrite diff tool#14921
RonnyPfannschmidt wants to merge 1 commit into
pytest-dev:mainfrom
RonnyPfannschmidt:ronny/assert-rewrite-diff-tool

Conversation

@RonnyPfannschmidt

Copy link
Copy Markdown
Member

Split out of #14447 at @Pierre-Sassoulas' request — the tooling there was
review aid, not part of the fix, and it is useful on its own.

What it does

scripts/diff-assert-rewrite.py dumps what the rewriter generates for a
snippet and diffs two sides. A side is plain (the source as written),
worktree (this checkout's src/), or a released pytest version, fetched on
demand with uv run --with pytest==VERSION — nothing under comparison has to
be installed. Output is either rewritten source (ast.unparse) or an AST.

The default — plain against worktree, as source — answers @bluetech's
question on #14447 directly, on main as it stands today:

$ python scripts/diff-assert-rewrite.py -c 'def test_walrus_boolop():
    assert (x := side_effect()) and (x := False)
'
--- plain
+++ worktree
@@ -1,2 +1,18 @@
+import builtins as @py_builtins
+import _pytest.assertion.rewrite as @pytest_ar
+
 def test_walrus_boolop():
-    assert (x := side_effect()) and (x := False)
+    @py_assert1 = []
+    @py_assert0 = (x := side_effect())
+    if (x := side_effect()):
+        @py_assert0 = (x := False)
+    if not @py_assert0:
+        @py_format3 = '%(py2)s' % {'py2': @pytest_ar._saferepr((x := side_effect())) if 'x' in @py_builtins.locals() or @pytest_ar._should_repr_global_name((x := side_effect())) else 'x'}
+        @py_assert1.append(@py_format3)
+        if (x := side_effect()):
+            @py_format5 = '%(py4)s' % {'py4': @pytest_ar._saferepr((x := False)) if 'x' in @py_builtins.locals() or @pytest_ar._should_repr_global_name((x := False)) else 'x'}
+            @py_assert1.append(@py_format5)
+        @py_format6 = @pytest_ar._format_boolop(@py_assert1, 0) % {}
+        @py_format8 = ('' + 'assert %(py7)s') % {'py7': @py_format6}
+        raise AssertionError(@pytest_ar._format_explanation(@py_format8))
+    @py_assert0 = @py_assert1 = None

side_effect() appears four times where the user wrote it once — that is
#14445, visible without reading the rewriter. Other modes:

# a behaviour change against a release, over a whole file
$ python scripts/diff-assert-rewrite.py --left 8.3.4 testing/example.py

# as AST, when the source form hides the difference
$ python scripts/diff-assert-rewrite.py --left 8.3.4 --format ast -c 'assert a == b'

# two releases against each other; also reads stdin
$ echo 'assert a == b' | python scripts/diff-assert-rewrite.py --left 6.2.5 --right 8.3.4

It exits 1 when the sides differ, so it works as a check as well as a viewer.

Notes

Down from the three files (410 lines) that were in #14447 to one file
(130 lines): the separate dump script is gone (it existed only to be
re-invoked by the diff script, which now spawns the two workers itself), the
example_asserts.py fixture is replaced by -c/stdin, and the compact AST
format is folded into ast — position attributes were never wanted.

@Pierre-Sassoulas: on the primer idea — running this over a selection of open
source repos to show output changes per PR is a bigger step (corpus choice,
a stored baseline, CI plumbing) and I would rather do it separately. This is
the piece it would be built on: a stable, scriptable "what does the rewriter
emit" that already exits non-zero on a difference.

Made with Claude Code

@RonnyPfannschmidt
RonnyPfannschmidt force-pushed the ronny/assert-rewrite-diff-tool branch from ffea670 to 7f59292 Compare August 22, 2026 05:53
@psf-chronographer psf-chronographer Bot added the bot:chronographer:provided (automation) changelog entry is part of PR label Aug 22, 2026
The rewriter is read through its failure messages; what it actually
generates is invisible unless one hand-writes an ast.unparse harness.
Reviewing a change to it means asking "what does the emitted code look
like now, and how does that differ from what it was".

Add a script that answers exactly that: it dumps a snippet's rewritten
form -- as source, or as an AST -- for the source as written, for this
checkout, or for any released pytest version, and diffs two of them.
Released versions are fetched on demand via ``uv run --with``, so no
version under comparison has to be installed.

By default it diffs the snippet as written against this checkout, which
is the "show me what rewriting does here" case:

    python scripts/diff-assert-rewrite.py -c 'assert (x := f()) and (x := False)'

It exits 1 when the sides differ, so it can also be used as a check.

@Pierre-Sassoulas Pierre-Sassoulas left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a nit, LGTM

src = Path(__file__).parent.parent / "src"
env = os.environ | {"PYTHONPATH": str(src)}
else:
cmd = ["uv", "run", "--no-project", "--with", f"pytest=={spec}"]

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to set the expected python version as 3.14 or more ?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should probably be a cli arg

"--right", default="worktree", metavar="SPEC", help="the same, other side"
)
parser.add_argument("--format", choices=("source", "ast"), default="source")
parser.add_argument("--no-color", action="store_true")

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could set the python interpreter version through uv

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bot:chronographer:provided (automation) changelog entry is part of PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants