Skip to content

feat(screenshot): add a capture runner and fix e2e verb papercuts - #242

Merged
pavelzw merged 2 commits into
mainfrom
mw/harness-papercuts
Sep 30, 2026
Merged

pavelzw merged 2 commits into
mainfrom
mw/harness-papercuts

Conversation

@moritzwilksch

@moritzwilksch moritzwilksch commented Sep 30, 2026 •

Copy link
Copy Markdown
Owner

Makes screenshotting a change cheap for agents, on top of the e2e harness from #214, and fixes two papercuts users reported. Replaces #238, which targeted the harness #214 removed.

Runner

A scenario exports one interaction function and, optionally, a repo. The runner owns the server, browser, before/after runs and cleanup.

import type { ScenarioContext } from '<checkout>/scripts/screenshot.ts';
export default async ({ page, shot, selectLines }: ScenarioContext) => {
  await selectLines('tally/ledger.py', 3, 3);
  await page.keyboard.type('Wait...');
  await shot('ellipsis', page.getByPlaceholder('Leave a comment…'));
};

npm run screenshot -- lig.ts --before c9dc8d2506 prints only this:

/tmp/…/lig
  ellipsis-after.png 726×128
  ellipsis-before.png 722×128
  ellipsis-compare.png 760×107  ← read this to verify

compare

  • Repo: the fixture repository by default. A scenario can export inline { before, after } files (demoRepo: temp dir, git -C, inline identity) or a repo path.
  • Before state: --before runs a git archive build of that revision with its own server and client, through startDiffle({ root }). It is cached per commit, and never swaps dist/client or registers a worktree.
  • Quiet builds: buildClient prints only on failure and skips a build whose sources haven't changed, which also speeds up the e2e global setup. ffmpeg is quiet and the intermediate webm is removed.
  • Compare image: one labelled image per shot, so one read verifies a change.
  • Failures: the error, the scenario line, and a 1x failure-<side>.png; exit status 1.
  • Docs: SKILL.md and SETUP.md are rewritten around the runner.
-export async function withBaseClient<T>(rev: string, fn: () => Promise<T>): Promise<T>
+export function baseCheckout(rev: string): string
 export interface DiffleOptions {
+  root?: string; // the diffle checkout to run, server and client
 }

💥 Breaking: withBaseClient, installClient, CLIENT_DIR and scenario.template.mts are removed. npm run screenshot now takes a scenario for the runner.

Harness fixes (test/e2e/browser.ts)

  • Off-screen files: the viewer renders only files near the viewport, so helpers acting on a listed but off-screen file timed out (reported). Header helpers now walk there with J/K, keeping a collapsed file collapsed; line helpers use gotoFile, which also expands it.
  • Mid-scroll press: line helpers wait for the target row to stop moving before pressing. Before, a press landed during the scroll correction and opened nothing.
  • Clear failures: a file with no lines (binary, empty, rename-only, mode-only) fails in about 2 s with a message saying so, not a 15 s timeout. A missing line lists the rendered lines. selectLines waits for the composer and says if it never opens.
  • Tooltip: toggleCollapse moves the pointer away first. The app's tooltip removes title from the hovered button, so a second toggle found nothing.
  • Older revisions: fileItem also accepts headers from before data-path, so --before works on older revisions.
  • Thread shape: SKILL.md documents it: anchor.path, messages[0].body, no comments (reported).

test/e2e/harness.spec.ts pins every helper in CI, so markup changes break there instead of in an agent's script.

The viewer renders only files near the viewport, so verbs on a listed but
off-screen file timed out looking for its container. They now move the
cursor there first: header verbs walk with J/K and keep a collapsed file
collapsed, line verbs use gotoFile and expand it.

Line verbs also wait for the target row to stop moving before pressing,
fail fast on files with no rows, list the rendered lines when one is
missing, and selectLines waits for the composer. SKILL.md documents the
thread shape (messages, not comments).
A scenario exports one interaction function and, optionally, a repo; the
runner reviews the fixture by default, owns the server, browser,
before/after runs and cleanup, and prints only output paths plus a
labelled compare image.

- --before runs a cached git-archive build of the base revision with its
  own server and client (startDiffle({ root })), never touching dist/client.
- buildClient is silent unless it fails and skips a fresh build.
- demoRepo builds throwaway repos with -C and an inline identity.
- test/e2e/harness.spec.ts pins every verb in CI.
- toggleCollapse parks the pointer; fileItem also accepts older headers.

BREAKING CHANGE: withBaseClient, installClient, CLIENT_DIR and
scenario.template.mts are removed; npm run screenshot now takes a
scenario for the runner.
@pavelzw
pavelzw merged commit c8da6df into main Sep 30, 2026
18 of 20 checks passed
@pavelzw
pavelzw deleted the mw/harness-papercuts branch September 30, 2026 21:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants