Skip to content

Add standalone binary validation harness - #479

Merged
jaredwray merged 2 commits into
mainfrom
cursor/binaries-validation-harness-d4aa
Aug 17, 2026
Merged

Add standalone binary validation harness#479
jaredwray merged 2 commits into
mainfrom
cursor/binaries-validation-harness-d4aa

Conversation

@jaredwray

@jaredwray jaredwray commented Aug 17, 2026

Copy link
Copy Markdown
Owner

Please check if the PR fulfills these requirements

  • Followed the Contributing and Code of Conduct guidelines.
  • Tests for the changes have been added (for bug fixes/features) with 100% code coverage.

What kind of change does this PR introduce? (Bug fix, feature, docs update, ...)

Test infrastructure for standalone binaries, plus a SEA-detection fix the harness uncovered.

The release workflow previously inlined a three-line smoke check. This extracts that into a reusable harness (pnpm test:binary) that validates a built or downloaded binary:

  • version matches package.json
  • help prints usage
  • build of a JSON-only smoke site writes index.html, robots.txt, and sitemap.xml
  • TypeScript-only configs are rejected (SEA JSON-only limitation)
  • download variables writes variables.css from the embedded templates
  • init --javascript scaffolds config, logo, and favicon

The build-binaries workflow now runs pnpm test:binary instead of the inline script. Unit tests cover the harness with a mocked runner so regular pnpm test does not need a SEA binary.

While running the harness against a locally built dist/docula, two issues showed up and are fixed here:

  1. SEA detection — tsdown's exe wrapper does not always make sea.isSea() return true. isSEA() now also treats registered embedded templates (set by sea-entry) as standalone mode, so JSON-only config enforcement and embedded templates work.
  2. CLI paths — Docula joins -s/-o with path.join(cwd, value), which turns /tmp/site into $cwd/tmp/site. The harness passes cwd-relative paths.

Validated with pnpm test (866 tests, 100% coverage) and pnpm test:binary -- ./dist/docula (6 passed) on a Node 26 SEA build.

Open in Web Open in Cursor 

cursoragent and others added 2 commits August 17, 2026 00:50
Extract the release-workflow smoke checks into a reusable TypeScript
harness that covers version, help, a JSON-config build, TypeScript
config rejection, download variables, and init.

Co-authored-by: Jared Wray <me@jaredwray.com>
tsdown's exe wrapper does not always report Node SEA, so treat
registered embedded templates as standalone mode. The harness now
passes cwd-relative paths because Docula joins CLI paths with
path.join(cwd, value), which mangles absolute /tmp paths.

Co-authored-by: Jared Wray <me@jaredwray.com>
@codecov

codecov Bot commented Aug 17, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (c0fe7c7) to head (4e3485d).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff            @@
##              main      #479   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           18        18           
  Lines         2760      2760           
  Branches       845       846    +1     
=========================================
  Hits          2760      2760           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@jaredwray
jaredwray marked this pull request as ready for review August 17, 2026 05:12
@jaredwray
jaredwray merged commit ac4808c into main Aug 17, 2026
13 checks passed
@jaredwray
jaredwray deleted the cursor/binaries-validation-harness-d4aa branch August 17, 2026 05:13

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

Bugbot Autofix is ON, but it could not run because the branch was deleted or merged before autofix could start.

Reviewed by Cursor Bugbot for commit 4e3485d. Configure here.

Comment thread scripts/test-binary.ts
fs.writeFileSync(
path.join(dir, "README.md"),
"# Binary Smoke\n\nHello from the binary harness.\n",
);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Smoke build check is inconclusive

Medium Severity

The build smoke check looks for SMOKE_SITE_TITLE in index.html, but the smoke README.md heading uses that same string. A rendered README alone satisfies the assertion, so a binary that skips JSON config loading can still pass the build check and give false confidence in the release harness.

Additional Locations (1)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 4e3485d. Configure here.

@jaredwray jaredwray mentioned this pull request Aug 17, 2026
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants