Skip to content

(test): run Playwright journeys against the real Electron app in CI (#304) - #432

Merged
devsuitup merged 2 commits into
mainfrom
test/304-playwright-e2e
Oct 3, 2026
Merged

devsuitup merged 2 commits into
mainfrom
test/304-playwright-e2e

Conversation

@devsuitup

Copy link
Copy Markdown
Owner

Refs #304

What

Five Playwright journeys that launch the real Electron app (_electron.launch()), plus a CI job that runs them on Linux.

  • e2e/fixtures.js: each test gets its own temporary HOME (and USERPROFILE), SWITCHBOARD_DATA_DIR, SWITCHBOARD_TRIGGERS_DIR and git identity, with CLAUDE* variables dropped. It builds fixture git repositories and transcripts, starts a trace, and on failure saves trace.zip and failure.png. Teardown is bounded: if the app has not exited within 10 s, the whole process tree is killed, then the HOME is deleted.
  • e2e/changes.spec.js: tracked-file counts and the header total, an untracked file opened from its row, and an edit saved from the panel editor that reaches disk. The editor must take the panel's width, not two 225 px columns.
  • e2e/panel.spec.js: the panel shell opened with no tab fills the panel and is not a sidebar row; Changes on a directory with no git work tree shows the not-a-repository note and no git output.
  • .github/workflows/test.yml: new e2e job (ubuntu-latest, xvfb-run, timeout-minutes: 20). It checks that better-sqlite3 and node-pty load under Electron, because postinstall swallows a failed rebuild. It uploads e2e/test-results/ and the HTML report on failure. It is not made a required check here.
  • task e2e / npm run e2e. The new files are outside node --test (which reads only test/) and are excluded from c8 and from the patch-coverage gate.
  • Docs: new docs/e2e.md; docs/live-testing.md updated now that @playwright/test is a dev dependency; a quick-orientation row in .ai/shared-guidelines.md; a line in docs/README.md.

There is no browser download. Native modules are already built for Electron by the existing postinstall (electron-builder install-app-deps), and the existing test jobs are unchanged.

The issue's fifth journey expected the Changes button to disappear. #310 made it unconditional, so the journey checks what the panel says instead.

How it was tested

Locally on Windows 11, against this tree, with the main checkout's Electron 41.0.3 and Playwright 1.63.0: 5 passed (48.8 s).

Each mutation below was applied in the worktree, run, and restored:

Mutation Result
.shell-only region rule removed (public/style.css) shell journey red: height ratio 0.26, needs > 0.9
both .shell-only rules removed shell journey red: handle visible
isPanelShellSession skip removed (session-cache.js) shell journey red: panel: ids returned by getProjects
Changes editor default set to side-by-side edit journey red: editor 224.5 px wide, needs > 359 px
not-a-repository branch skipped in refreshChanges no-work-tree journey red: no .changes-note
row shows the deleted count as the added count tracked-counts journey red
up-front untracked counting off, and the count on open removed untracked journey red

Removing only the count on open leaves the untracked journey green, because status has already counted the file up front.

Not verified

  • The Linux CI job has not run yet. Whether the native-module check and xvfb-run work on ubuntu-latest is unproven until it does.
  • Ten consecutive CI runs without an intermittent failure (the issue's third acceptance point).
  • The teardown hardening (bounded close, then a process-tree kill) was added after the local runs and has only been linted. It was added because one local failing run left an Electron process behind.
  • The full task check was not run, because the disk was full. Lint passed with 0 errors, and the related unit files passed (102 tests, 0 failed).

No changelog entry: users see nothing (no-changelog).

jsdom has no layout and the unit suites run over fakes, so the four panel
defects behind #304 were only found by launching the app. Five journeys
now launch it with Playwright under a throwaway HOME, data dir and
triggers dir, and assert structure, geometry and what reaches disk:
Changes counts and totals, an untracked file opened from its row, an edit
saved from the panel editor, the panel shell filling the panel without a
sidebar row, and a directory with no git work tree.

A new e2e job runs them on ubuntu-latest under xvfb-run, checks first that
the native modules load under Electron, and uploads traces and screenshots
on failure. task e2e runs them locally. The files sit in e2e/, outside
node --test and c8.

Refs #304. Not yet shown: a green run on the Linux job, and ten consecutive
runs without an intermittent failure.
@devsuitup devsuitup added the no-changelog The PR changes nothing a user sees; the CHANGELOG.md check is waived label Oct 3, 2026
@devsuitup

Copy link
Copy Markdown
Owner Author

Adversarial review at 3e22c0a: no blocking finding. CI e2e job green (5 passed, 55 s, native modules rebuilt for Electron 41.0.3 and load-checked).

Non-blocking, to fix before merge:

  1. e2e/fixtures.js:96 — on Linux proc.kill('SIGKILL') kills the main pid only, not the process tree the docs claim; the path never ran (close succeeded). Kill the group (detached + process.kill(-pid)) or correct the docs.
  2. e2e/changes.spec.js:97 — "the file stays changed" is asserted on a row already rendered; nothing waits for the post-save refresh. Wait for the refresh, then assert the row's counts.
  3. e2e/fixtures.js:20-33 — inherited HISTFILE, GIT_DIR, GIT_WORK_TREE, GIT_INDEX_FILE are not dropped; a run from a hook could reach real files.

Noted, accepted: the job is not a required check and the ten-run soak is outstanding (hence Refs #304); journey 2 does not pin the count-on-open (status counts untracked up front).

Checked: isolation of HOME//.claude//.switchboard/triggers/userData on Linux and Windows; no real claude spawned; no disguised sleeps; e2e excluded from node --test, c8 and the patch gate; lockfile adds only @playwright/test, playwright, playwright-core 1.63.0 (dev) plus the root engines already in package.json; journey 5 change justified by #310.

On Linux and macOS the teardown sent SIGKILL to the main pid only;
Playwright already starts Electron as a process-group leader there, so
signal the group, falling back to the pid. Windows keeps taskkill /T.

The saved-edit journey asserted a row that was already rendered. It now
checks the row's counts before the edit and polls until the refresh after
the save shows the new line.

The launched environment drops every inherited GIT_* variable and
HISTFILE, so a GIT_DIR or GIT_INDEX_FILE in the runner cannot point git
at another repository.
@devsuitup

Copy link
Copy Markdown
Owner Author

Delta review at 406c9d7: items 1-3 fixed, no finding left. Playwright 1.63.0 spawns Electron detached off Windows (coreBundle.js launchProcess), so the group kill reaches the tree; pid guard excludes 0/-1, ESRCH falls back; the post-save poll waits for {added:2}, which only a refresh from git produces; the env filter keeps what the fixture re-sets. All checks green on this SHA, e2e 5/5. Approved. Still open on #304: the ten-run soak and making e2e a required check.

@devsuitup
devsuitup merged commit 14afc31 into main Oct 3, 2026
12 checks passed
@devsuitup
devsuitup deleted the test/304-playwright-e2e branch October 3, 2026 12:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

no-changelog The PR changes nothing a user sees; the CHANGELOG.md check is waived

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant