test(fixture): add a deterministic fixture repository - #214
Merged
Merged
Conversation
Contributor
|
This pull request changes committed snapshots. View the snapshot report to compare them side by side. |
A scripted history with fixed authors and dates that hashes the same on every machine, covering every diff shape the viewer renders: multi-hunk edits, an added and a deleted file, a rename with edits and a pure rename, a mode change, a binary change, generated files by path, content and gitattribute, a lockfile, a path with a space and non-ASCII letters, an empty file, a file without a trailing newline, CRLF endings and a minified line. The checkout carries a staged, an unstaged and an untracked change for working mode. `npm run fixture -- [dir]` builds it for hands-on testing and prints the comparisons worth opening; `test/fixture/repo.test.ts` pins the hashes so an unintended change to the history fails before any screenshot does. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01D3jjTs2MgCQu4MtVxz1x2E
Playwright scenarios start a real diffle over the fixture repository, drive it
the way a reviewer does and compare screenshots in light and dark, plus
accessibility snapshots of the help, the mode picker and the threads panel.
Every scenario gets its own repository and server, so review state never
leaks between them; `test.use({ revs })` picks the comparison.
The screenshot-change skill's harness moves to `test/e2e/harness.ts` in
TypeScript, so the skill and the tests share one set of verbs; the skill's
private Playwright and font install go away in favour of dev dependencies.
File headers carry `data-path` so the verbs find a file by name rather than
by a title the viewer splits into styled parts.
Also pulls in pixelmatch and pngjs for the snapshot report that follows.
💥 Breaking: the screenshot-change skill no longer has its own package.json;
run `npm install` and `npx playwright install chromium --only-shell` in the
checkout, and run scenarios with `npx tsx`.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01D3jjTs2MgCQu4MtVxz1x2E
One realistic export of a review, the full `--help` texts and the GitHub review payload live as files under `__snapshots__/`, so a change to any of them is judged by reading the diff of what a user or an API receives rather than an edited string literal. The formatter leaves snapshot directories alone: they are test output, byte for byte. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01D3jjTs2MgCQu4MtVxz1x2E
`npm run snapshot-report -- --base <rev>` builds one self-contained HTML page from the committed snapshots that differ between two revisions, without rerunning a test: side by side, an overlay slider and a pixel diff for screenshots, a unified diff for text, and the source of the test behind each one. A pull request that touches a snapshot directory gets the report as an artifact and a sticky comment linking to it. CI compares screenshots inside Playwright's container so fonts and rasterization match the run that produced them, and uploads the expected, actual and diff images with traces when a scenario fails. The `Update snapshots` workflow regenerates them in the same container and commits the result to the branch, so an intended UI change can be accepted from any platform. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01D3jjTs2MgCQu4MtVxz1x2E
The ReviewPane, CommentPanel, FileTreePane and SettingsDialog tests replaced the diff viewer, the API or the store with stubs and asserted on the wiring between them; the e2e scenarios now cover what they showed against the real renderer. The store's race tests stay: a browser cannot provoke those orderings reliably. CLAUDE.md and the README describe the layers and the workflow: the fixture repository, the e2e scenarios, file snapshots, and how to accept changes. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01D3jjTs2MgCQu4MtVxz1x2E
The container hints and rasterizes the same fonts slightly differently from the machine that produced the first set, so the 32 images its failed run captured become the expected ones. The CRLF and minified-line screenshots split into two scenarios: a failing first screenshot hid the second one's result. The manual update workflow is only offered once its file is on the default branch; the workflow notes the artifact as the way to accept images until then. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01D3jjTs2MgCQu4MtVxz1x2E
…ainer The last two images the split scenario produced on its own. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01D3jjTs2MgCQu4MtVxz1x2E
The app ships JetBrains Mono and Inter since #215, so the harness no longer fronts them itself; a page is ready once `document.fonts.ready` resolves. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01D3jjTs2MgCQu4MtVxz1x2E
pavelzw
force-pushed
the
claude/testing-setup-improvements-duq710
branch
from
September 20, 2026 11:29
3549d58 to
e4a5203
Compare
The client now ships every subset of Inter and JetBrains Mono, so glyphs the injected latin subset left to a fallback, the currency symbols among them, render from the design's fonts; every screenshot moved by a few pixels. The images are the ones CI's container produced. The Nix hash follows the merged lockfile. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01D3jjTs2MgCQu4MtVxz1x2E
Refuse nonempty fixture destinations and create fresh temporary repositories by default. Escape snapshot paths in HTML reports and authenticate snapshot pushes with Octo STS so CI runs on updates. BREAKING CHANGE: fixture generation no longer replaces existing directories.
# Conflicts: # test/client/ReviewPane.test.ts
# Conflicts: # test/client/FileTreePane.test.ts # test/client/ReviewPane.test.ts
# Conflicts: # test/client/ReviewPane.test.ts
# Conflicts: # test/client/ReviewPane.test.ts
pavelzw
marked this pull request as ready for review
September 30, 2026 19:39
moritzwilksch
approved these changes
Sep 30, 2026
pavelzw
enabled auto-merge (squash)
September 30, 2026 19:43
pavelzw
disabled auto-merge
September 30, 2026 19:43
This was referenced Sep 30, 2026
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A scripted history with fixed authors and dates that hashes the same on every
machine, covering every diff shape the viewer renders: multi-hunk edits, an
added and a deleted file, a rename with edits and a pure rename, a mode change,
a binary change, generated files by path, content and gitattribute, a lockfile,
a path with a space and non-ASCII letters, an empty file, a file without a
trailing newline, CRLF endings and a minified line. The checkout carries a
staged, an unstaged and an untracked change for working mode.
npm run fixture -- [dir]builds it for hands-on testing and prints thecomparisons worth opening;
test/fixture/repo.test.tspins the hashes so anunintended change to the history fails before any screenshot does.
Co-Authored-By: Claude Fable 5.1 noreply@anthropic.com
Claude-Session: https://claude.ai/code/session_01D3jjTs2MgCQu4MtVxz1x2E