Skip to content

ci: test on push to main + nightly; fix registry assertion order-dependence - #915

Merged
kriszyp merged 5 commits into
mainfrom
chore/ci-push-trigger-and-test-hygiene
Oct 13, 2026
Merged

kriszyp merged 5 commits into
mainfrom
chore/ci-push-trigger-and-test-hygiene

Conversation

@kriszyp

@kriszyp kriszyp commented Oct 8, 2026 •

Copy link
Copy Markdown
Member

⊙ Problem

.github/workflows/pr.yml's Test workflow only triggers on pull_request, so a merge to main gets no test run (stress.yml already runs on push). Separately, test/shutdown.test.ts asserted on registryStatus()'s global array length — order-dependent, since the native registry is process-global and the whole suite runs with fileParallelism: false (one shared process). The Deno-test retry (~line 237) retried only on Windows, with no stated reason, unlike bun-test's own Windows-only retry which cites a specific tracked race. vitest.config.ts's forks-pool choice for Bun/Deno had no comment explaining the tradeoff.

❓ Your call: as specified, yes on all four — this is exactly four independent test/CI-hygiene fixes with no shared mechanism, scoped to config and test files only.

💡 Solution

🔧 Changes

  • .github/workflows/pr.yml — push/schedule triggers, run-id-scoped concurrency group, Windows-only Deno retry (documented), and a corrected bun-test comment (see below).
  • test/shutdown.test.ts — path-scoped getRegistryStatus() assertions.
  • vitest.config.ts — forks-pool rationale comment.

Unplanned, in-scope fix: fixing the vitest.config.ts comment surfaced a real contradiction with bun-test's pre-existing comment, which claimed Bun runs every test file's DB churn in one process. Installed Bun locally and ran the same two-file process.pid check against it — PIDs differed across files, same as under Node and Deno: Bun is not single-process either. The pre-existing comment's premise was wrong; corrected it to point at worker_threads churn within a run (the actual mechanism behind #695) instead. The retry behavior itself is unchanged.

⚖️ Alternatives

  • Deno retry: extend to every OS (the first attempt) — rejected after cross-model review. An unconditional retry-with-warning on Linux/macOS turns an intermittent native failure from probability p to roughly p², and a GitHub Actions ::warning:: on an otherwise-green run isn't seen by anyone. Reverted to Windows-only.
  • Concurrency: cancel-in-progress: ${{ github.event_name == 'pull_request' }} alone, same shared group (the second attempt) — looked sufficient but only stops cancelling a running run; GitHub still drops an older queued run in the same group regardless of that setting. Needed the run_id-based group split too.

✅ Verification

  • pnpm check (type-check + lint + fmt) clean.
  • pnpm test — 1180 passed, 10 skipped, 0 failed (full suite, including test/shutdown.test.ts and test/registry-status.test.ts).
  • Empirically verified the forks-pool comment: ran a throwaway two-file process.pid check under plain Node (forced --pool=forks) and under Bun (bun --bun ...) — both spawn a fresh process per test file, serialized by fileParallelism: false to one at a time.
  • Checked for an existing registry leak that the new path-scoped assertion might hide: ran the full suite once with shutdown.test.ts's original global-length assertions restored (uncommitted, local-only) — still 1180/10/0. No current leak.
  • .github/workflows/pr.yml parses as valid YAML (yaml.safe_load).
  • Independent cross-model review: 5 rounds (1 full, 3 delta, 1 closing-full), codex + gemini + cursor-composer + harper-domain. Converged with only nit-level findings remaining (below).

❓ Your call: push has no paths-ignore, so a docs-only merge to main still runs the full ~19-job matrix. Left it unfiltered, matching stress.yml's existing push trigger (also unfiltered) — the point of this change is a test signal on every commit to main, and runner minutes on a public repo are free. A narrower paths-ignore on push (keeping schedule unfiltered) is a one-line follow-up if runner queue time becomes a real problem.

❓ Your call: didn't rename pr.yml despite it now running on push/schedule too — the workflow's name: is already Test (what GitHub shows and what branch protection keys on), so the filename is cosmetic. Flagged as optional by review; out of scope for this pass.

❓ Your call: the nightly schedule relies on GitHub's default failure notification (email to whoever last touched the cron line) — no Slack/issue routing added. Decided that's acceptable to start; routing is easy to add later if nightly failures go unnoticed.

No numbered GitHub issue was filed for this; it was worked from a dispatch task (rocksdb-js-ci-on-main-and-test-hygiene).

Related PRs: #900 overlaps, #742 overlaps, 5 others independent

Origin — the dispatch brief this PR was written from

Run the Test workflow on main; registry-assertion and retry hygiene

Dispatch: task rocksdb-js-ci-on-main-and-test-hygiene · queued by unknown · ran by claude/sonnet/xhigh · worker kzyp-xps-1

Review-Coverage: authored=claude; ran=gemini,codex; adjudicated=domain; declined=cursor-grok,cursor-composer,cursor-kimi,cursor-muse; rounds=5; full=2 @ 8574288

Review-Attention: study ~10m (sensitive: pr.yml; decisions: main-push-full-matrix, nightly-failure-routing, path-scoped-shutdown-assertion) @ 8574288

kriszyp and others added 5 commits October 8, 2026 09:24
…ence

pr.yml's Test workflow ran on pull_request only, so a merge to main had no
test signal (stress.yml already runs on push). Add push: branches: [main]
and a nightly schedule.

Make the Deno-test retry OS-consistent: it retried failures only on
Windows, unlike bun-test's documented Windows-only retry (a specific,
cited race). No equivalent Deno-specific root cause is documented, so
retry once on every OS instead.

test/shutdown.test.ts asserted on registryStatus()'s global array length,
which is order-dependent — the native registry is process-global and
vitest runs with fileParallelism: false, so another test file's open
database shows up in the same array. Filter by the test's own db.path,
matching the pattern already used in registry-status.test.ts,
destroy.test.ts, secondary.test.ts, etc.

Document why vitest.config.ts picks the forks pool for Bun/Deno and what
it costs (process-spawn overhead, no --expose-gc propagation).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FF7XCwDWipqGApc3eX32wb
Dispatch-Task: rocksdb-js-ci-on-main-and-test-hygiene
- cancel-in-progress now only applies to pull_request runs. push and
  schedule share the refs/heads/main group, so leaving it unconditional
  meant two quick merges (or a push racing the nightly run) cancelled
  one of them -- silently dropping the per-commit signal this change
  exists to add.
- The Deno retry now logs a ::warning:: on the first failed attempt
  before retrying, so a rising flake rate is visible instead of being
  silently absorbed by the retry on every OS.
- shutdown.test.ts now calls getRegistryStatus() instead of the
  deprecated registryStatus() alias (src/load-binding.ts marks it for
  removal in v3.0.0, and open PR #900 removes it outright).
- Fixed the new vitest.config.ts comment: fileParallelism: false forces
  maxWorkers to 1, so forks here is one process for the whole run, not
  one per test file (matches bun-test's existing comment on the same
  behavior).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FF7XCwDWipqGApc3eX32wb
Dispatch-Task: rocksdb-js-ci-on-main-and-test-hygiene
Review round 2 found two real problems with the round-1 fixes:

- cancel-in-progress only stops a RUNNING run from being cancelled.
  GitHub always drops an older QUEUED run when a newer one joins the
  same concurrency group, regardless of that setting -- so two merges
  (or a push racing the nightly run) still silently dropped one run's
  signal. push/schedule now get a group keyed on github.run_id instead
  of the shared main ref, so they never collide with each other or
  with a PR's group.

- The unconditional Deno retry-with-warning (added in the prior commit)
  doubles this job's failure tolerance on Linux/macOS from p to roughly
  p^2 with no evidence a comparable flake exists there, and the warning
  sits on a run that passes and that nobody opens -- it doesn't actually
  fix the visibility problem it was meant to fix. Reverted to the
  original Windows-only retry, now with a comment explaining why: it's
  covering general Windows-runner flakiness (matching bun-test and the
  Node coverage job above), not a Deno-specific race the way bun-test's
  #695 is, so there's no basis for extending it to Linux/macOS.

Also fixed the vitest.config.ts comment from the prior commit: it's
wrong that fileParallelism collapses forks into one process for the
whole run. Verified with a throwaway two-file process.pid check that
each file still gets its own forked process; fileParallelism only
serializes them to one at a time. bun-test's existing comment about a
single Bun process is a separate, Bun-specific claim this doesn't
contradict or confirm.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FF7XCwDWipqGApc3eX32wb
Dispatch-Task: rocksdb-js-ci-on-main-and-test-hygiene
Shortened the nightly, concurrency, and Deno-retry comments to state
the decision instead of narrating it or arguing with a reviewer, and
dropped vitest.config.ts's "verified via process.pid" aside and its
claim about a Bun-specific exception (the other file's single-process
comment is about Bun's own run, not something this diff can confirm or
contradict).

Also ran the full suite once with shutdown.test.ts's original
global-length assertions restored (not committed) to check the
decision ledger's open question about a possible existing registry
leak: 1180 passed / 10 skipped / 0 failed, so there's no current leak
for the path-scoped assertions to hide.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FF7XCwDWipqGApc3eX32wb
Dispatch-Task: rocksdb-js-ci-on-main-and-test-hygiene
Review flagged a real contradiction: vitest.config.ts's corrected
comment says forks gives each test file its own child process, while
bun-test's pre-existing comment claimed Bun runs every file's DB churn
in one process. Installed Bun locally and ran the same two-file
process.pid check against it (bun --bun ... vitest run): PIDs differed
across files, same as under Node and Deno -- Bun is not single-process
either, so the pre-existing comment's premise was wrong.

The retry itself is unaffected (#695 is real and still the reason for
the extra Windows attempts); only the inaccurate "single-process, every
file's DB churn in one process" explanation is corrected to point at
worker_threads churn within a run instead.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FF7XCwDWipqGApc3eX32wb
Dispatch-Task: rocksdb-js-ci-on-main-and-test-hygiene

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the shutdown tests in test/shutdown.test.ts to use getRegistryStatus instead of registryStatus, querying specific database paths to make assertions more robust. It also adds explanatory comments in vitest.config.ts regarding the use of the 'forks' pool in alternate runtimes. There are no review comments, so I have no feedback to provide.

@github-actions

github-actions Bot commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

📊 Benchmark Results

get-sync.bench.ts

getSync() > random keys - small key size (100 records)

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 lmdb 1 24.00K ops/sec 41.67 39.09 2,159.946 0.173 120,002
🥈 rocksdb 2 11.02K ops/sec 90.75 85.90 5,969.007 0.260 55,100

getSync() > sequential keys - small key size (100 records)

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 lmdb 1 28.90K ops/sec 34.61 32.98 570.798 0.114 144,484
🥈 rocksdb 2 10.85K ops/sec 92.13 87.86 633.891 0.113 54,274

ranges.bench.ts

getRange() > small range (100 records, 50 range)

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 lmdb 1 23.91K ops/sec 41.82 37.30 1,866.179 0.316 119,572
🥈 rocksdb 2 14.60K ops/sec 68.49 59.16 1,079.461 0.156 73,008

realistic-load.bench.ts

Realistic write load with workers > write variable records with transaction log

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 rocksdb 1 421.52 ops/sec 2,372.366 43.28 83,815.95 18.50 845
🥈 lmdb 2 25.93 ops/sec 38,561.796 438.679 1,225,639.675 136.714 64.00

transaction-log.bench.ts

Transaction log > read 100 iterators while write log with 100 byte records

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 rocksdb 1 34.68K ops/sec 28.84 12.86 20,930.323 0.854 173,397
🥈 lmdb 2 440.71 ops/sec 2,269.088 132.913 23,310.425 1.48 2,204

Transaction log > read one entry from random position from log with 1000 100 byte records

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 rocksdb 1 607.37K ops/sec 1.65 1.41 715.9 0.107 3,036,842
🥈 lmdb 2 452.57K ops/sec 2.21 1.19 8,268.236 0.551 2,262,828

worker-put-sync.bench.ts

putSync() > random keys - small key size (100 records, 10 workers)

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 rocksdb 1 830.67 ops/sec 1,203.845 1,031.801 3,006.231 0.373 1,662
🥈 lmdb 2 1.13 ops/sec 881,247.035 806,021.804 1,015,538.255 5.51 10.00

worker-transaction-log.bench.ts

Transaction log with workers > write log with 100 byte records

Implementation Rank Operations/sec Mean (ms) Min (ms) Max (ms) RME (%) Samples
🥇 rocksdb 1 20.54K ops/sec 48.70 30.25 20,464.702 2.08 41,071
🥈 lmdb 2 835.91 ops/sec 1,196.307 235.407 11,258.603 5.08 1,672

Results from commit 2ef5967

@kriszyp
kriszyp marked this pull request as ready for review October 9, 2026 00:24
@kriszyp
kriszyp requested a review from cb1kenobi October 9, 2026 00:24
@kriszyp
kriszyp merged commit 94ae107 into main Oct 13, 2026
28 checks passed
@kriszyp
kriszyp deleted the chore/ci-push-trigger-and-test-hygiene branch October 13, 2026 17:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants