Compile in-process on OCaml domains in OCaml rewatch

[cknitt]

Stacked on #8780.

This PR moves compilation into the OCaml rewatch process. Instead of starting bsc for every parse and compile, rescript links the compiler and runs requests on a bounded pool of OCaml 5 domains. That removes process start-up and pipe handling, which make the subprocess port about 6% slower than Rust on clean builds. It also opens the door to sharing compiler state between requests later. The compiler side, which makes requests safe to run repeatedly and in parallel and captures their output, is #8864.

Changes

• Rewatch schedules independent parse and compile requests, and namespace maps, on worker domains. By default that's at most 8, and one fewer than the CPU count; REWATCH_COMPILER_DOMAINS overrides it, and a count above the machine's limit is lowered to it. The worker that compiled a module also publishes its artifacts; build-state updates stay on the scheduling domain. Workers get a 2M-word minor heap, because with the default one they keep stopping each other for minor collections. Together, these two make clean builds of tests/tests about 20% faster (27% less CPU), for about 200 MB more memory at 8 workers. PPXs and hooks still run as external processes.
• Compiler output is captured in memory, so builds need no temporary directory. Output written before an escaping exception is reported with the failed module.
• A termination signal stops the build at the next poll point, after the modules compiling in-process finish, so a stop never leaves a half-written artifact. Repeated signals are ignored: the npm launcher forwards the Ctrl-C the terminal already delivered.
• A damaged published AST, such as one cut short by a killed build, makes its module parse again instead of failing every later build.
• rescript format also runs on the embedded compiler. PPXs, js-post-build, --after-build and the lock-owner check are the only remaining subprocesses. RESCRIPT_BSC_EXE now only affects rescript-rust, and CONTRIBUTING.md says how to test a local compiler instead.
• The compiler's build identity is a digest of its sources, linked into the driver, so OCaml rewatch rebuilds after a compiler change without fingerprinting an installed executable. lib/bs/compiler-info.json still records the bsc.exe shipped next to rescript.exe; the VS Code extension reads bsc_path to find bsc.exe and the other platform binaries. A development build, which has no bsc.exe next to it, records the one in the installed @rescript/<platform> package.

ThreadSanitizer

An OCaml 5.5.0 ThreadSanitizer build of rescript reported no data races in compiler or rewatch code on clean builds of rewatch/testrepo (4 domains), tests/tests (2, 8 and 12 domains), the gentype example and the React PPX tests, or in the rewatch unit tests. Two reports appeared in some tests/tests runs, both inside the runtime's parallel minor GC (oldify_one against pool_allocate). With the shared Js_analyzer accumulator restored, the same tests/tests build reports 91 races, all in Js_analyzer, so the tool does catch this class of bug. This run predates publishing on the worker domains.

Tests

• The focused suite uses a large module to keep an in-process compile running. With it, it tests:
  • that a second build waits for the build lock;
  • that repeated signals still stop cleanly;
  • that TERM, HUP and QUIT during compilation stop with the signal's status;
  • that removing watch.lock stops a watcher's compile;
  • that an atomic save during a rebuild is built.
• It also tests:
  • that a damaged published AST is re-parsed;
  • that builds need no temporary directory;
  • that a failing PPX's stderr doesn't shift its error;
  • PPX temp files under a relative TMPDIR;
  • that a stop is noticed between namespace maps;
  • that a failed parse stays pending.
• Unit tests check that an exception escaping the compiler fails only its own module and that PPX cancellation returns promptly.
• rewatch-ocaml/bench compares clean-build artifacts across domain counts; it does not run in CI.

🤖 Generated with Claude Code

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 80.94%. Comparing base (002d5dd) to head (6d0571e).

Additional details and impacted files

@@                 Coverage Diff                 @@
##           rewatch-ocaml-port    #8781   +/-   ##
===================================================
  Coverage               80.94%   80.94%           
===================================================
  Files                     453      453           
  Lines                   62839    62839           
===================================================
  Hits                    50865    50865           
  Misses                  11974    11974           

Files with missing lines	Coverage Δ
compiler/bsc/rescript_compiler_driver.ml	81.22% <ø> (ø)

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[pkg-pr-new]

Open in StackBlitz

rescript

npm i https://pkg.pr.new/rescript@8781

@rescript/belt

npm i https://pkg.pr.new/@rescript/belt@8781

@rescript/darwin-arm64

npm i https://pkg.pr.new/@rescript/darwin-arm64@8781

@rescript/darwin-x64

npm i https://pkg.pr.new/@rescript/darwin-x64@8781

@rescript/linux-arm64

npm i https://pkg.pr.new/@rescript/linux-arm64@8781

@rescript/linux-x64

npm i https://pkg.pr.new/@rescript/linux-x64@8781

@rescript/runtime

npm i https://pkg.pr.new/@rescript/runtime@8781

@rescript/win32-x64

npm i https://pkg.pr.new/@rescript/win32-x64@8781

commit: 6d0571e

[cknitt]

@codex review

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-14T18:17:14.994232Z	d2d1c04	Manual request

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 61da00ebaa

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[cristianoc]

This is where I would have expected a detectable per difference.
Or difference in allocation profile.

But perhaps I don't understand the spirit of what is done here.