Conversation
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: trueComment |
edolstra
added
the
flake-regression-test
boost::context's manage_exception_state snapshots *__cxa_get_globals() before a fiber context switch and writes it back afterwards, to preserve the libstdc++ per-thread exception state across switches. However, __cxa_get_globals() is declared __attribute__((const)), so the compiler folds the constructor's and the destructor's calls into a single call executed before the switch. When a fiber suspends on one thread and is resumed on another, the destructor then restores the snapshot through the *original* thread's globals pointer, silently overwriting that thread's live exception state (the caughtExceptions list head and the uncaughtExceptions counter) from another thread. With the fiber-based parallel evaluator this caused reproducible "std::terminate() called without exception" aborts and segfaults whenever multiple fibers unwound exceptions concurrently (e.g. during shutdown or on Ctrl-C): a fiber's `throw;` in Value::force() would find its thread's caught-exception list emptied under its feet. This is an upstream boost bug (present in stock 1.89, independent of our 0001 patch, which merely extends manage_exception_state). Add a separate 0002 patch that fetches the globals pointer through a volatile function pointer, forcing a fresh call in the destructor so the write goes to the thread the destructor actually runs on. Assisted-by: Claude Fable 5 <noreply@anthropic.com>
Previously, when an evaluation worker thread hit a thunk that was being evaluated by another thread, it blocked on a condition variable until the thunk was finished. To utilize all cores, this requires oversubscribing threads, and even then most threads can end up blocked on a single popular thunk. Instead, run every work item on its own boost::context fiber. When a fiber hits a pending/awaited thunk, it suspends and the worker thread switches to other work; `finish()` re-enqueues exactly the fibers waiting on that value (the wait list is keyed on the value, so unlike the old hashed-domain condition variable, fiber wakeups are never spurious). Thus we always make progress as long as there is runnable work: e.g. 200 work items can be simultaneously suspended on a shared thunk while 8 worker threads keep evaluating. Notes: * The suspension handshake keeps the waiter-domain mutex locked across the context switch: the fiber's continuation only materializes on the scheduler side of the switch, so the scheduler registers it in the wait list and then releases the lock, preventing another thread from resuming a half-suspended fiber. * Non-fiber contexts (e.g. the main thread) still use the old condition variable path in `waitOnThunk()`. * `myEvalThreadId` is now a per-fiber id (set on every fiber switch-in), since with two fibers on one thread, a per-thread id would produce false "infinite recursion" errors from the self-wait check. * On shutdown/interrupt, all suspended fibers are flushed from the wait lists and resumed so that they observe `quit` (or the interrupt) and unwind their stacks via a normal `Interrupted` exception; a suspended fiber is never destroyed. Work items that haven't started get an `Interrupted` exception on their promise. * Fiber stacks are not yet registered as GC roots, so for now, parallel evaluation requires the GC to be disabled (e.g. via GC_DONT_GC=1). Also not yet done: throttling the number of live fibers, fiber stack pooling, and cross-fiber deadlock detection. Assisted-by: Claude Fable 5 <noreply@anthropic.com>
Upstream boostorg/context already fixed the cross-thread exception-state corruption in manage_exception_state by marking its constructor and destructor BOOST_NOINLINE (which prevents the compiler from CSE'ing the const-attributed __cxa_get_globals() calls across the context switch). That fix predates boost 1.89 but was accidentally reverted by the merge of upstream PR #324, and 1.89 was released in that state; it was re-applied upstream in commit 0921b9fd5c776aec7748475c6c10807e0d51bc6d. Backport that commit instead of our own equivalent fix, so that the patch stack matches upstream and both patches can simply be dropped once we're on a boost release that contains them. Assisted-by: Claude Fable 5 <noreply@anthropic.com>
`EvalState::callDepth` is a thread-local counter, and the `CallDepth` RAII guard holds a reference to it. A fiber can suspend mid-call-chain (with guards alive on its stack) and be resumed on a different thread; the guards would then decrement the original thread's counter through the stale reference, while the new thread's counter never comes back down, corrupting the depth accounting on both threads and producing spurious "max-call-depth exceeded" errors. Give each fiber its own call-depth counter in the `Fiber` record, and make the thread-local `EvalState::callDepthPtr` point to the counter of the current execution context: the fiber's counter while a fiber is running (switched in `Executor::runFiber()`), or the thread's own `callDepth` otherwise. Since the fiber's counter lives in the heap-allocated `Fiber` record, the references held by `CallDepth` guards remain valid no matter which thread runs or unwinds them. Assisted-by: Claude Fable 5 <noreply@anthropic.com>
The scheduler released the waiter-domain mutex through a `std::unique_lock` living on the suspending fiber's stack. However, `unique_lock::unlock()` releases the mutex *before* clearing its owns-flag, and the moment the mutex is released, another thread can extract and resume the fiber, which then reads the owns-flag on its stack concurrently with the scheduler's write. Occasionally the fiber would read a stale `true`, tripping the `!lk.owns_lock()` assertion (and in builds without assertions, the `unique_lock` destructor would unlock a mutex it doesn't hold, which is undefined behavior). This showed up as a flaky abort in `nix search`. Instead, the fiber now calls `lk.release()` (a fiber-local write) before switching out, handing mutex ownership to the scheduler, which unlocks `domain.mutex` directly. That's legal since the mutex was locked by the fiber on the scheduler's own thread. Assisted-by: Claude Fable 5 <noreply@anthropic.com>
Allocating a fresh 60 MiB stack (mmap + guard-page mprotect + munmap, plus first-touch page faults) for every work item is expensive: e.g. `nix search nixpkgs fizzbuzz --no-eval-cache` spawns ~117k fibers, of which only ~45 are ever simultaneously alive, making it ~30% slower than the thread-based executor at the default settings. Keep finished fibers' stacks in a pool and reuse them for new fibers. Reused stacks also come with their previously faulted-in pages. This makes the fiber-based evaluator match the thread-based baseline on the `nix search` benchmark (~3.4s), with only 124 real stack allocations for the ~117k fibers (shown by the new `nrFiberStacksAllocated` statistic). Assisted-by: Claude Fable 5 <noreply@anthropic.com>
Since the worker threads only run the scheduler loop — the actual evaluation happens on fibers, which have their own stacks — they no longer need a 60 MiB stack, so we don't need boost::thread's stack-size attribute anymore and can use plain std::thread with the default stack size. This also lets us drop the boost::thread dependency from libexpr. Note: the workers must still register themselves with the Boehm GC. That's not for the sake of the (now root-free) worker stacks, but because fibers running on a worker allocate from it: without registration, Boehm has no thread-local allocation freelists for the thread and every allocation takes the global allocation lock, making e.g. `nix search nixpkgs fizzbuzz --no-eval-cache` ~3.5x slower (~12s instead of ~3.4s). Assisted-by: Claude Fable 5 <noreply@anthropic.com>
Replace the unordered_multimap<ValueBase *, FiberPtr> wait list with an unordered_map<ValueBase *, std::vector<FiberPtr>>. This expresses the intent (a list of waiters per value) more directly, and lets notifyWaiters() extract all waiters for a value in one splice via node extraction instead of walking equal_range() and erasing node-by-node while holding the domain lock. It's also cheaper when many fibers wait on the same thunk, since the waiters are stored contiguously instead of in one map node per fiber. Assisted-by: Claude Fable 5 <noreply@anthropic.com>
Previously, fiber stacks were invisible to the Boehm GC, so parallel evaluation required disabling the GC (GC_DONT_GC=1): values reachable only from a fiber stack could be collected while alive. * Running fibers: when a collection happens, the thread's captured sp is inside the fiber stack, and the sp corrector (`fixupBoehmStackPointer()`) used to clamp it back to the OS thread stack, so the fiber stack was never scanned. The corrector now detects this case and pushes the fiber stack's used range `[sp, base)` directly onto the mark stack (which is legal there: the corrector runs during root pushing with the GC lock held, same as `GC_push_other_roots`). The worker's own scheduler stack is then excluded from scanning, since it holds no GC roots and scanning it in full would fault in otherwise untouched pages. * Suspended fibers (parked in the wait lists or the ready queue): scanned via a `GC_set_push_other_roots` callback. `suspendFiber()` publishes the used portion of its stack (from just below the current frame — with slack for the register block that the context switch pushes, i.e. bytes that get touched anyway — up to the stack base) right before switching out, and clears it right after being resumed. Thus at every instant, a fiber stack is covered by the thread scan, by the published range, or (briefly, harmlessly) both. The stacks are tracked in an append-only registry with one entry per allocated stack (thanks to the stack pool, only a modest number), updated only through atomic stores. This is required because the GC callbacks run with the world stopped: they cannot take any lock that a frozen thread might hold, and must tolerate threads frozen in the middle of an update. Free (pooled) and unstarted stacks have no published range and are never scanned. Known limitation: if a fiber enters a boost::coroutine2 coroutine, the fiber frames below the coroutine are not scanned (the sp is then inside the coroutine stack, so we can't tell how much of the fiber stack is in use). This extends the existing assumption that coroutine stacks hold no GC roots. Assisted-by: Claude Fable 5 <noreply@anthropic.com>
Now that `CallDepth` guards increment/decrement the thread-local counter of whatever thread they run on (instead of holding a reference to a specific counter), the fiber scheduler no longer needs to redirect the accounting through a pointer. Instead, `runFiber()` simply swaps the thread-local counter with the fiber's saved depth on every switch-in/out: a fiber suspended mid-call-chain carries its depth to whatever thread resumes it, and its guards unwind against the correct value there. Assisted-by: Claude Fable 5 <noreply@anthropic.com>
`EvalState::evalContext` was a thread-local value, which is broken with fibers: a fiber that is suspended and resumed on another thread loses its context, and two fibers on different threads could race non-atomic shared_ptr assignments against the same thread's slot. Make the thread-local a plain *pointer* to the active `EvalContext`: each fiber owns its own context (living in the heap-allocated `Fiber` record, so it travels with the fiber), and `runFiber()` points the thread-local at it on switch-in and restores it on switch-out. Using a pointer means fiber switches don't touch the `provenance` shared_ptr's atomic reference count. Non-fiber contexts (i.e. the main thread) share a global default context, preserving the old behavior. As a side effect, a work item's context can no longer leak into subsequent work items executed on the same worker thread, since every fiber starts with a fresh context. Assisted-by: Claude Fable 5 <noreply@anthropic.com>
Producers used to wake workers unconditionally: `spawn()` did a `notify_all()` (waking every worker for possibly a handful of items) and `enqueueFiber()` an unconditional `notify_one()`. Instead, track the number of sleeping workers (under the state lock, so there are no lost wakeups) and wake exactly `min(nrItems, nrSleeping)` workers — zero futex calls when all workers are busy. Note: on the `nix search nixpkgs --no-eval-cache` benchmark this turns out not to reduce the overall context-switch count measurably: tracing shows that ~65% of the ~750k futex waits in that workload come from the Boehm GC (the global allocation lock `GC_allocate_ml` and the parallel-marker coordination lock), and most of the remainder are genuine executor sleeps between work bursts. Still, exact wakeups are strictly better than the previous thundering herd as the worker count grows. Assisted-by: Claude Fable 5 <noreply@anthropic.com>
It's no longer useful.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Instead of threads blocking when they hit a thunk being evaluated by another thread, we now use fibers. This allows the thread to switch to any other runnable fiber.
Context