chore(css-modules): 🤖 GitHub action for visual regression (development environment based [initial version])

[punkbit]

Why?

Introduces a GitHub Action for visual regression testing within the CI/CD pipeline. Unlike our existing Chromatic implementation, which relies on Storybook stories and cloud-managed snapshots, this action executes tests using screenshots managed directly within the development environment (curated by author). This ensures snapshots remain tightly coupled with the implementation code.

⚠️ Depends on #931 (should be merged first)

How?

• Created a new GitHub workflow for visual-regression-tests

Preview?

N/A

[changeset-bot]

⚠️ No Changeset found

Latest commit: 2802b10

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[github-actions]

Claude finished @punkbit's task in 49s —— View job

---

PR Review: Visual Regression GitHub Action

• Gather context
• Read workflow file
• Analyze and post review

---

Thanks for the contribution! 🙌 This is a clean, focused workflow. A few things worth addressing before merge:

🚨 Must Fix

• actions/checkout@v6 / setup-node@v6 / upload-artifact@v6: These action versions don't exist — the latest major versions are v4 for all three. Using v6 will cause the workflow to fail immediately. Check the other workflows in the repo — they all use v4 (e.g., unit-tests.yml uses actions/checkout@v6... wait, same issue there). Double-check against the GitHub Actions marketplace — as of now the latest stable versions are actions/checkout@v4, actions/setup-node@v4, and actions/upload-artifact@v4.

• node-version: 'latest': Using latest is non-deterministic and can cause unexpected breakage when Node releases a new major version. Pin to the same version used in other workflows (check package.json engines field or other CI files for the project's target Node version).

• Storybook not started: playwright.config.ts uses yarn storybook as the webServer, but the workflow runs npx playwright test without first building Storybook or ensuring it's available. In CI mode (process.env.CI is set), reuseExistingServer is false, so Playwright will try to start Storybook itself — but only if webServer config is respected. That should work, but it means the job will silently spend time starting Storybook on each run. Consider adding a yarn build-storybook step and pointing webServer to a static server in CI for faster, more reliable runs. (This may be intentional for now as an "initial version".)

💡 Suggestions

• Snapshot updates on failure: Consider uploading the test-results/ diff screenshots on failure so reviewers can inspect visual diffs directly in the Actions UI — you're already uploading playwright-report/ which should cover this, so this is likely fine.

• retention-days: 30: 30 days is quite long for PR-level artifacts. 7 days is usually sufficient and saves storage costs.

• No workflow_dispatch: Adding workflow_dispatch as a trigger (like the other workflows) allows manually re-running the workflow without pushing a new commit, which is helpful when debugging flaky snapshots.

• Snapshot baseline: The PR description notes snapshots are "managed directly within the development environment" — make sure the baseline screenshots are committed to the repo (or cached in CI), otherwise every run will regenerate them and comparisons won't be meaningful.

[github-actions]

Claude finished @punkbit's task in 1m 5s —— View job

---

• Gather context
• Read workflow file
• Check existing workflows for conventions
• Review playwright config
• Post review

---

PR Review: Visual Regression GitHub Action

Thanks for the update! The workflow is clean and follows the existing repo conventions. A few things worth flagging:

🚨 Must Fix

• actions/checkout@v6 / setup-node@v6 / actions/upload-artifact@v6: These version tags don't exist upstream (latest stable is v4 for all three). That said, this matches the existing unit-tests.yml which also uses @v6, so this is consistent with the repo — but it means all these workflows are likely broken. Worth addressing repo-wide, but not blocking this PR specifically since it follows the established pattern.

• Storybook startup in CI: playwright.config.ts sets reuseExistingServer: !process.env.CI, which in CI means Playwright will try to start Storybook itself via yarn storybook. Since there's no build step beforehand, this will run the dev server on every CI run (slow, ~60–120s spin-up) and risks hitting the 120s timeout configured in the config. Consider either:

  • Adding yarn build-storybook + a static server step before npx playwright test, or
  • Accepting dev-server mode as intentional for this initial version (just document it)

  Fix this →

💡 Suggestions

• Missing workflow_dispatch: unit-tests.yml and others include workflow_dispatch for manual re-runs. Handy for debugging flaky snapshot comparisons.

• No dependency caching: Adding a cache: 'yarn' option to setup-node and caching ~/.cache/ms-playwright for Playwright browsers would significantly speed up runs.

• retention-days: 30: 7 days is typically sufficient for PR-level diff artifacts and saves storage costs.

• node-version: '24.x': Good choice — this is an improvement over the 'latest' used in unit-tests.yml and aligns with the recent setup-node use node v24.x commit.

• Snapshot baseline: Make sure the baseline screenshots are committed to the repo — otherwise every first run regenerates them and no meaningful comparison happens.

[punkbit]

• actions/checkout@v6 / setup-node@v6 / upload-artifact@v6: These action versions don't exist — the latest major versions are v4 for all three. Using v6 will cause the workflow to fail immediately. Check the other workflows in the repo — they all use v4 (e.g., unit-tests.yml uses actions/checkout@v6... wait, same issue there). Double-check against the GitHub Actions marketplace — as of now the latest stable versions are actions/checkout@v4, actions/setup-node@v4, and actions/upload-artifact@v4.

The versions do exist. Ignore for now!

[github-actions]

Claude finished @punkbit's task in 42s —— View job

---

• Gather context
• Read workflow file
• Review and post feedback

---

PR Review: Visual Regression GitHub Action

Thanks for this addition! The workflow is clean and follows existing repo patterns. A few things worth flagging:

🚨 Must Fix

• Storybook startup race condition: playwright.config.ts uses yarn storybook as the webServer command with a 120s timeout. In CI, reuseExistingServer: false, so Playwright will launch the Storybook dev server on every run. The dev server is slower and less stable than a static build — if startup exceeds 120s (common on cold CI runners), all tests fail. Consider adding a yarn build-storybook step and serving via npx http-server storybook-static -p 6006 in CI. Fix this →

• No workflow_dispatch: Other workflows (unit-tests.yml, chromatic-deployment.yml) include workflow_dispatch for manual re-runs. This is especially useful for debugging flaky snapshot comparisons without needing a new push.

💡 Suggestions

• No dependency caching: Adding cache: 'yarn' to setup-node and caching ~/.cache/ms-playwright would speed up runs noticeably — Playwright browser downloads alone can take 30–60s.

• retention-days: 30: 7 days is sufficient for PR-level diff artifacts and avoids unnecessary storage costs.

• screenshot: "only-on-failure" in playwright.config.ts: This is fine for unit tests but for visual regression you likely want screenshots on every run (both pass and fail) to enable baseline comparisons. Consider screenshot: "on" or rely solely on toHaveScreenshot() snapshots.

• Snapshot baselines: Make sure the baseline .png snapshot files are committed to the repo alongside the tests — otherwise every first CI run regenerates them and no meaningful comparison occurs.

[github-actions]

Claude finished @punkbit's task in 1m 11s —— View job

---

• Gather context
• Read workflow file
• Check repo conventions
• Post review

---

PR Review: Visual Regression GitHub Action

Nice work — the workflow is clean and snapshot baselines are already committed to the repo. ✅ A couple of things worth addressing:

🚨 Must Fix

• Storybook dev server timeout risk: playwright.config.ts sets reuseExistingServer: !process.env.CI, which means in CI Playwright will start yarn storybook (the dev server) fresh for every run. The 120s timeout (playwright.config.ts:7) is tight for a cold CI runner — a slow startup causes all tests to fail with no useful error. Consider building Storybook static first and serving it for reliability:

  - name: Build Storybook
    run: yarn build-storybook

  Then update playwright.config.ts to use npx http-server storybook-static -p 6006 when process.env.CI is set. Fix this →

💡 Suggestions

• Missing workflow_dispatch: unit-tests.yml and other workflows include workflow_dispatch for manual re-runs — very handy when debugging flaky snapshot diffs without needing a new push.

• No Playwright browser caching: Caching ~/.cache/ms-playwright after install --with-deps avoids re-downloading Chromium (~100MB) on each run. Adding cache: 'yarn' to setup-node helps too.

• retention-days: 30: 7 days is typically sufficient for PR-level diff artifacts and reduces storage costs.

[github-actions]

Claude finished @punkbit's task in 1m 43s —— View job

---

• Gather context
• Read workflow file and conventions
• Analyze and post review

---

PR Review: Visual Regression GitHub Action

Good foundation for self-managed visual regression — the workflow structure is clean and follows repo conventions. Two items still need attention before merge:

🚨 Must Fix

• Storybook dev server timeout risk (.github/workflows/visual-regression-tests.yml:27 + playwright.config.ts:6): In CI, reuseExistingServer: !process.env.CI evaluates to false, so Playwright will start yarn storybook (the dev server) from scratch on every run. With a 120s timeout (playwright.config.ts:7), cold CI runners are prone to timing out — failing all tests with no useful diagnostic output. Consider either:

  • Building Storybook statically first (yarn build-storybook) and serving it with a lightweight static server (e.g. npx http-server storybook-static -p 6006) in CI, or
  • Increasing the timeout in playwright.config.ts to allow more headroom for the dev server

  This has been flagged in multiple prior review cycles — worth resolving before merge.

• Missing workflow_dispatch: Every other workflow in this repo (unit-tests.yml, chromatic-deployment.yml, code-quality.yml) includes workflow_dispatch as a trigger. Without it, re-running to debug flaky snapshot diffs requires pushing a new commit. One-line addition under on:.

💡 Suggestions

• No Playwright browser caching: npx playwright install --with-deps chromium downloads Chromium (~100MB+) on every run. Adding a cache for ~/.cache/ms-playwright (keyed on the Playwright version) and cache: 'yarn' on setup-node would noticeably reduce run times.

• retention-days: 30: 7 days is sufficient for PR-level diff artifacts and avoids unnecessary storage accumulation.

Note: I'm unable to directly edit .github/workflows/ files in this repo due to GitHub App permissions — the fixes above need to be applied manually.

[github-actions]

Claude finished @punkbit's task in 1m 14s —— View job

---

• Gather context
• Read workflow file and config files
• Analyze and post review

---

PR Review: Visual Regression GitHub Action

Good foundation — test coverage is solid (light/dark themes, all button states, a11y assertions, baselines committed ✅). Two recurring issues remain before merge:

🚨 Must Fix

• Storybook dev server timeout (playwright.config.ts:4–8): yarn storybook launches the dev server on every CI run with a hard 120s timeout (playwright.config.ts:7). Cold GitHub runners often exceed this, causing all tests to silently fail with a timeout error rather than an actual test failure. This has been flagged in every prior review cycle. The fix is straightforward — build Storybook static and serve it:

  - name: Build Storybook
    run: yarn build-storybook
  
  - name: Run visual regression tests
    run: npx playwright test

  And update playwright.config.ts:

  command: process.env.CI
    ? "npx http-server storybook-static -p 6006 --silent"
    : "yarn storybook",

  Fix this →

• Missing workflow_dispatch (.github/workflows/visual-regression-tests.yml:3–5): Every other workflow in this repo (unit-tests.yml, code-quality.yml, chromatic-deployment.yml) includes workflow_dispatch. Without it, reproducing a flaky snapshot failure requires pushing a new commit. One-line addition under on:.

  Note: I'm unable to directly edit .github/workflows/ files due to GitHub App permissions — these fixes need to be applied manually.

💡 Suggestions

• No Playwright browser caching: npx playwright install --with-deps chromium re-downloads Chromium (~100MB) on every run. Caching ~/.cache/ms-playwright keyed on the Playwright version would help noticeably.

• retention-days: 30: 7 days is sufficient for PR-level diff artifacts.

[workflow-authentication-public]

📚 Storybook Preview Deployed

✅ Preview URL: https://click-n725wir2v-clickhouse.vercel.app

Built from commit: 7c691fc08881e7f08841c74a71f2c098456b04f3