Skip to content

fix(ci): harden Bun isolate worker teardown and probe timeouts - #1179

Merged
Wibias merged 3 commits into
devfrom
fix/bun-isolate-worker-hangs
Aug 7, 2026
Merged

fix(ci): harden Bun isolate worker teardown and probe timeouts#1179
Wibias merged 3 commits into
devfrom
fix/bun-isolate-worker-hangs

Conversation

@Wibias

@Wibias Wibias commented Aug 7, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • extend the storage Worker post-close OS-join settle to Linux, where Bun 1.3.14 is also showing isolate/reclaim failures
  • keep the longer 1.5s Windows settle while using a 250ms reclaim window on Linux/macOS
  • add a regression test that pins the per-platform settle policy
  • bound the synchronous Claude launcher-context probe with a 3s timeout and hard kill so a wedged child cannot consume the entire CI job

Failure observed

Linux shard test 3/4 repeatedly fails under Bun 1.3.14 with two linked symptoms:

  1. tests/api-storage-policy-already-running.test.ts hits an unhandled Bun runtime failure while creating a stream around Worker activity:
error: EEXIST: file already exists, epoll_ctl
at new WriteStream (internal:fs/streams:244:58)

Bun then reports test-runner corruption (Cannot call beforeEach() after the test run has completed) but continues into later files.

  1. Once the corrupted shard reaches tests/claude-dotenv-provenance-transport.test.ts, its unbounded spawnSync() child can wedge. The log shows killed 1 dangling process, followed by no progress until the 15-minute job timeout.

The repository already documents Bun 1.3.14 Worker-reclaim crashes on Ubuntu, but worker-lifecycle.ts only applied its post-close OS-join settle on Windows/macOS. This PR closes that Linux gap and independently makes the Claude subprocess fail boundedly instead of hanging the whole shard.

Test coverage

  • new pure regression test covers Windows/macOS/Linux settle values
  • existing Claude transport tests continue exercising the real child-process path, now with a bounded subprocess lifetime
  • full cross-platform CI should verify whether the repeated Linux test 3/4 hang is eliminated

Scope

This is intentionally separate from #1175; it changes only CI/runtime hardening for the pre-existing Bun isolate failure and subprocess hang.

Summary by CodeRabbit

  • Bug Fixes

    • Improved worker shutdown timing across Windows, macOS, Linux, and other platforms.
    • Added support for Linux and Bun 1.3.14 behavior during worker termination.
  • Tests

    • Added coverage for platform-specific shutdown delays.
    • Improved child-process test reliability with probe timeouts and explicit termination handling.

@Wibias
Wibias marked this pull request as ready for review August 7, 2026 05:29
@coderabbitai

coderabbitai Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: b131c510-f364-4638-8891-00e616ea91c8

📥 Commits

Reviewing files that changed from the base of the PR and between 6d04574 and a53163f.

📒 Files selected for processing (3)
  • src/storage/worker-lifecycle.ts
  • tests/claude-dotenv-provenance-transport.test.ts
  • tests/storage-worker-os-join-settle.test.ts

📝 Walkthrough

Walkthrough

The worker lifecycle now exports platform-specific OS-join settle durations and applies them during termination. Tests cover Windows, macOS, Linux, and FreeBSD. A child-process probe test now has bounded execution and explicit spawn-error handling.

Changes

Worker OS-join settling

Layer / File(s) Summary
Platform settle policy and termination integration
src/storage/worker-lifecycle.ts
The lifecycle exports storageWorkerOsJoinSettleMs, returning 1,500 ms for Windows, 250 ms for Darwin and Linux, and 0 ms otherwise. Termination preserves the zero-delay yield and sleeps only when the selected duration is positive. Documentation records the supported platform cases.
Platform coverage and probe cleanup
tests/storage-worker-os-join-settle.test.ts, tests/claude-dotenv-provenance-transport.test.ts
Tests verify settle durations for Windows, macOS, Linux, and FreeBSD. The child-process probe uses UTF-8 output, an explicit environment, a 3-second timeout, SIGKILL, and immediate spawn-error propagation.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related PRs

Suggested labels: bug

Suggested reviewers: ingwannu, lidge-jun

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main changes: hardened Bun worker teardown and bounded probe timeouts.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/bun-isolate-worker-hangs

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a53163f0b1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

const result = spawnSync(process.execPath, [probe, ...args], {
encoding: "utf8",
env,
timeout: PROBE_TIMEOUT_MS,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Give the Bun child a contention-safe deadline

On contended cross-platform runners, this 3-second ceiling can turn a healthy cold Bun launch into ETIMEDOUT, and line 46 then fails all three transport tests before they exercise the behavior. The Windows CI notes in tests/codex-catalog-restore.test.ts:41-42 record the same spawnSync(bun ...) pattern taking about 5.4 seconds under load, while tests/helpers/test-budget.ts:33-34 assigns real child processes a 45-second budget. Keep the subprocess bounded, but use a deadline with CI-contention headroom (and an appropriate outer test budget) rather than 3 seconds.

Useful? React with 👍 / 👎.

@github-actions github-actions Bot added the bug Something isn't working label Aug 7, 2026
@github-actions

github-actions Bot commented Aug 7, 2026

Copy link
Copy Markdown

Deterministic PR hygiene checks passed.

@Wibias
Wibias merged commit 44dce33 into dev Aug 7, 2026
22 of 24 checks passed
@Wibias
Wibias deleted the fix/bun-isolate-worker-hangs branch August 7, 2026 06:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant