Skip to content

Performance Tuning

Jump to bottom
github-actions[bot] edited this page Aug 24, 2026 · 39 revisions

Performance Tuning Guide

This guide covers performance optimization techniques for Elio applications.

Overview

Elio is designed for high performance through:

  • Lock-free data structures (Chase-Lev deque)
  • Work-stealing scheduler
  • Efficient I/O backends (io_uring, epoll)
  • Standard coroutine frame allocation with unrestricted destruction order
  • Minimal synchronization overhead

Actual Performance Numbers

Scheduling Benchmarks

Operation Typical Best Case Notes
Task Spawn Benchmark-dependent - Standard heap frames
Context Switch ~230 ns ~212 ns Suspend and resume
Yield ~30 ns ~16 ns Per 1000 vthreads
MPSC push ~5 ns - Cross-thread scheduling
Chase-Lev push ~13 ns - Local queue operation

I/O Benchmarks

Scenario Latency Throughput
Single-thread file read 1.46 μs/read 685K IOPS
4-thread concurrent read 0.93 μs/read 1.07M IOPS

Scalability

CPU-bound workload with 100K iterations per task:

Threads Throughput Speedup
1 ~18K tasks/sec 1.0x
2 ~33K tasks/sec 1.9x
4 ~56K tasks/sec 3.2x
8 ~86K tasks/sec 4.9x

Scaling efficiency depends on workload characteristics. Tasks with more computation relative to scheduling overhead will show better scaling.

Wake-up Mechanism

Elio uses an eventfd embedded in each worker's I/O backend (epoll/io_uring) for cross-thread notifications. This provides a single unified wait point: both I/O completions and task wake-ups unblock the same poll() call, eliminating the latency gap that exists with separate wait mechanisms. A worker-level pending-wake claim lets a burst of submissions share one eventfd notification.

Built-in Optimizations

Coalesced Submission Wake

The first cross-thread submission in a busy or blocked interval claims and writes an eventfd wake. Later submissions still publish their queue entries but share that outstanding notification:

// In worker_thread::schedule()
if (inbox_->push(handle.address())) {
    wake_for_submission();
}

Before entering a blocking poll, the worker atomically clears the pending-wake claim and rechecks both external queues. A submission published before that clear is found by the recheck; a submission published after it claims a new wake and interrupts the poll. This handshake preserves unconditional-wake correctness while avoiding one eventfd_write syscall per task in a burst.

This is deliberately not the previous idle-flag lazy-wake optimization. The producer never decides that a wake is unnecessary from a sampled worker state; that approach had a race which caused 10 ms tail latency on weak hardware.

The MPSC ring remains the normal submission path. If it stays full after bounded retries, the worker accepts the handle through a locked overflow queue. That slow path trades burst-only synchronization for correctness: a borrowed resume handle stays assigned to its worker rather than running on the producer thread or being destroyed.

Owner-Local Continuations

scheduler::try_schedule() distinguishes continuation resumption from initial task admission. When called by an active worker and the suspended coroutine can legally remain there, it publishes directly to the owner's Chase-Lev deque. This avoids the round-robin counter, external MPSC publication, lifecycle mutex, and eventfd write. The continuation remains visible to work stealing.

Affinity and I/O ownership remain authoritative. A valid affinity for another worker takes the external route, while an active I/O pin stays with the backend owner even while that worker is draining. Movable continuations do not take the local fast path on a draining worker. New independent tasks submitted through go() or spawn() retain normal distributed admission.

Unified Wake Mechanism

Each worker's I/O backend (epoll or io_uring) contains an embedded eventfd. The first task submitted from another thread while no submission wake is outstanding writes to that eventfd. Because the descriptor is registered with the same epoll/io_uring instance that handles I/O completions, both I/O events and task wake-ups unblock the same poll() call.

This unified design has two key benefits:

  1. Single wait point. A worker blocked on I/O poll is immediately woken by a cross-thread task submission. There is no separate condition variable or futex that could introduce a latency gap between "I/O ready" and "task ready" paths.

  2. Safe coalescing. Producers atomically share one outstanding submission wake. The worker clears that claim and rechecks its external queues before every blocking poll, closing the clear-versus-block lost-wake window.

The first submission pays for one eventfd_write; additional tasks in the same burst pay only for the atomic claim check. A blocked worker is still interrupted immediately rather than waiting for a polling timeout.

Wait Strategy

Elio supports configurable wait strategies to balance latency vs CPU usage:

#include <elio/runtime/scheduler.hpp>
#include <elio/runtime/wait_strategy.hpp>

using namespace elio::runtime;

// Pure blocking (default) - lowest CPU usage
scheduler sched(4, wait_strategy::blocking());

// Hybrid spin-then-block - good for low-latency workloads
// Spins for 1000 iterations with yield, then blocks on I/O poll
scheduler sched(4, wait_strategy::hybrid(1000));

// Aggressive spinning - ultra-low latency (uses pause instruction)
scheduler sched(4, wait_strategy::spinning(1000));

// Custom strategy
wait_strategy custom{
    .spin_iterations = 500,   // Spin count before blocking
    .spin_yield = true       // Yield during spin (friendlier to other threads)
};
scheduler sched(4, custom);

Strategy Selection Guide:

Strategy CPU Usage Wake Latency Use Case
blocking() Lowest ~1-10 μs General workloads (default)
hybrid(N) Low-Medium ~1-5 μs Latency-sensitive with mixed load
spinning(N) High ~100-500 ns Ultra-low latency, dedicated CPUs
aggressive(N) Medium-High ~100-1000 ns Low latency, shared CPUs

The spin_yield flag controls whether the spin phase uses std::this_thread::yield() (true) or the CPU pause instruction (false). Yielding is friendlier to other threads but slightly slower.

Runtime Configuration:

// Change per-worker strategy at runtime
auto* worker = sched.get_worker(0);
worker->set_wait_strategy(wait_strategy::spinning(2000));

io_uring Batch Submit

I/O operations are automatically batched:

// In io_uring_backend::poll()
// Auto-submit any pending operations before waiting
if (io_uring_sq_ready(&ring_) > 0) {
    io_uring_submit(&ring_);
}

This reduces the number of io_uring_submit syscalls by batching multiple operations.

Lazy Debug ID Allocation

Debug IDs for coroutines are only allocated when actually accessed, reducing creation overhead in production:

// debug_id_ initialized to 0, allocated on first id() call
// Only available when ELIO_ENABLE_DEBUG_METADATA is enabled
uint64_t id() noexcept {
    if (debug_id_ == 0) {
        debug_id_ = id_allocator::allocate();
    }
    return debug_id_;
}

Optimized Yield Path

Yielding skips affinity checks and scheduler lookups for better performance:

// In yield_awaitable::await_suspend()
auto* worker = runtime::worker_thread::current();
if (worker) {
    // Fast path: directly schedule to local queue
    worker->schedule_local(awaiter);
    return;
}
// Slow path only when no current worker

Scheduler Tuning

Thread Count

#include <elio/runtime/scheduler.hpp>

// Default: matches hardware concurrency
scheduler sched;

// Custom thread count
scheduler sched(8);  // 8 worker threads

// For I/O-bound workloads, consider more threads than cores
scheduler sched(std::thread::hardware_concurrency() * 2);

// For CPU-bound workloads, match core count
scheduler sched(std::thread::hardware_concurrency());

Dynamic Thread Adjustment

The scheduler supports changing the worker thread count at runtime:

// Adjust thread count at runtime
sched.set_thread_count(8);  // Grow to 8 workers
sched.set_thread_count(2);  // Shrink to 2 workers

On shrink, num_threads() reflects the smaller logical scheduling pool as soon as retirement is published, while each removed worker thread continues polling its worker-local I/O until both pending operations and active pins reach zero. The shrink call does not wait for that drain. A later grow can wait when it must reuse a slot whose previous worker is still draining, so bound long-lived I/O with application deadlines or cancellation when resize responsiveness matters. shutdown_force() can interrupt the drain, but is a non-graceful teardown path that may orphan in-flight I/O.

For automatic scaling, use the Autoscaler component. It monitors queue length and automatically scales worker threads based on configurable thresholds:

#include <elio/runtime/autoscaler.hpp>

elio::runtime::autoscaler_config config;
config.overload_threshold = 20;    // Scale up when queue > 20
config.idle_threshold = 5;       // Scale down when queue < 5
config.idle_delay = std::chrono::seconds(30);
config.min_workers = 2;
config.max_workers = 16;

elio::runtime::autoscaler<runtime::scheduler,
    elio::runtime::on_overload<elio::runtime::scale_up<elio::runtime::null>>,
    elio::runtime::on_idle<elio::runtime::scale_down<elio::runtime::null>>,
    elio::runtime::on_block<elio::runtime::log>
> autoscaler(config);

autoscaler.start(&sched);

This is useful for adapting to load changes automatically (see Scheduler Statistics). The on_block trigger samples each non-idle worker's completed-resume counter once per tick_interval. It reports a block when that sampled counter has made no progress for longer than block_threshold; notification is therefore best-effort and can lag the threshold by roughly one sampling interval. Idle time is excluded, and any observed progress restarts the threshold. Block detection does not add a clock read to every coroutine resume.

Call the concrete worker diagnostic enable_task_time_tracking() to explicitly opt that worker into exact per-resume timestamp collection. For backward compatibility, last_task_time() also enables tracking when called directly. Periodic monitoring should prefer worker_tasks_executed(), which stays on the lower-cost counter path.

Thread Affinity

Pin coroutines to specific workers for cache locality:

#include <elio/runtime/affinity.hpp>

coro::task<void> cache_sensitive_work() {
    // Bind to current worker for cache locality
    co_await elio::bind_to_current_worker();

    // All subsequent work stays on this worker
    process_data();
}

// Or set affinity to a specific worker
coro::task<void> pinned_work() {
    co_await elio::set_affinity(2);  // Bind to worker 2 and migrate there
    co_await elio::set_affinity(2, false);  // Bind without migrating

    // Later, allow free migration again
    co_await elio::clear_affinity();
}

set_affinity is an awaitable. When called with migrate=true (the default), the coroutine is immediately rescheduled on the target worker. With migrate=false, the affinity is recorded but migration is deferred until the next scheduling point. clear_affinity removes caller affinity so the task can be stolen when no stronger runtime ownership constraint exists. An active worker-local I/O pin always wins; neither API migrates a pending backend operation.

Spawn-time Pinning with go_to()

When you know the target worker at spawn time, go_to() is more efficient than go() + set_affinity():

// Preferred: affinity is set before first resume, no migration window
elio::go_to(2, cache_sensitive_work);

// Alternative: task may briefly run on another worker before migrating
elio::go([]() -> coro::task<void> {
    co_await elio::set_affinity(2);
    // ...
    co_return;
});

go_to() sets the worker affinity before scheduling the task and initially enqueues it to the target worker when that worker is available. If a later steal attempt observes the task on another queue, the scheduler requeues it to the affinity worker instead of executing it on the wrong worker. This avoids the brief scheduling window where the task could execute on the wrong worker before set_affinity takes effect.

Use a worker id in [0, scheduler.num_threads()) when exact placement matters. Out-of-range ids are not rejected, but they are fallback behavior rather than a stable pinning contract, especially while the pool is being resized.

I/O Backend Selection

io_uring vs epoll

Elio auto-detects the best available backend:

#include <elio/io/io_context.hpp>

// Auto-detect (prefers io_uring)
io::io_context ctx;

// Force specific backend
io::io_context ctx(io::io_context::backend_type::io_uring);
io::io_context ctx(io::io_context::backend_type::epoll);

// Check active backend
std::cout << "Backend: " << ctx.get_backend_name() << std::endl;

These directly constructed contexts are standalone. Drive and serialize them from their owning thread; mutating one from a scheduler worker is rejected. Scheduler coroutines should use io::current_io_context(); a pending operation is pinned to that worker and context generation until completion, so it does not incur a central-reactor hop or migrate between backends.

Why io_uring is preferred:

  • Submission batching. Multiple I/O operations can be queued in the submission ring before a single io_uring_enter syscall, amortizing syscall overhead across many operations.
  • Completion batching. Completions accumulate in the completion ring and can be reaped in bulk without per-operation syscalls, unlike epoll where each I/O still requires a separate read/write/accept call after readiness notification.
  • Registered resources. File descriptors and buffers can be pre-registered with the kernel, reducing per-operation kernel crossing cost by avoiding repeated fget/fput and page table walks.
  • Native async semantics. Operations are inherently asynchronous — submit and forget until completion — which aligns naturally with coroutine suspension and resumption. There is no "readiness" vs "completion" mismatch as with epoll.

epoll fallback:

  • Works on older kernels (pre-5.1)
  • Lower memory overhead (no shared ring buffers)
  • Adequate for moderate workloads where per-operation syscall cost is not the bottleneck

io_uring Kernel Requirements

For best io_uring performance:

  • Linux 5.1+: Basic io_uring
  • Linux 5.6+: Full features
  • Linux 5.11+: Multi-shot accept

Memory Management

Coroutine Frame Allocation

coro::task frames use the standard coroutine heap allocation path. There is no per-vthread bump allocator or LIFO destruction requirement. Keeping frames small still reduces allocation traffic and cache pressure, but callers should not depend on allocator locality or on the worker that eventually frees a frame. The runtime co-allocates each independent logical-vthread execution context and cancellation state in one shared control block. Nested Elio tasks defer that allocation and, when directly awaited by another Elio task, reuse the actual awaiter's context. A deep transparent helper chain therefore pays one control allocation for its root rather than one context plus one cancellation-link node per frame. Independent go/spawn and task-group roots still receive separate contexts; task_scope() also preserves a separate cancellation context by contract. Use those APIs when policy isolation is intentional. Cancellation tokens can retain the shared logical-vthread block after a particular frame is destroyed.

If an application already owns a non-empty lazy task that has not completed, transfer it directly to avoid a callable-wrapper frame and its task-local control state:

// A callable wrapper safely owns arbitrary callable state and arguments.
scheduler.go(make_request, input);

// An already-constructed task can be transferred without that wrapper.
scheduler.go(make_request(input));
auto joined = scheduler.go_joinable(make_request(input));

Do not eagerly invoke an arbitrary temporary coroutine lambda merely to select the direct overload: its returned frame may retain the lambda through this. Passing the callable lets Elio keep that object alive in the wrapper.

Cancellation Callback Registration

Each cancel_token::on_cancel() registration owns one shared callback node. Small nothrow-movable callables use the node's inline buffer; larger callables require a second payload allocation. The node uses a native-width atomic phase for selection, invocation, and teardown. Ordinary register/unregister traffic therefore does not initialize or lock a per-registration mutex or condition variable. The cancellation-state mutex still owns list insertion, selection, and O(N) unlinking, so unregistering an old callback from a long-lived source costs more than removing its newest callback.

cancel_callback_benchmark reports the platform-specific node sizes and allocator-requested bytes for an inline callback, register/unregister latency, newest and oldest unlink latency for several list sizes, cancellation dispatch, immediate registration after cancellation, and cancellation p95/p99 while another thread unregisters callbacks. Build it in Release mode and collect multiple process-level runs on a fixed CPU when comparing revisions:

cmake -S . -B build-release -DCMAKE_BUILD_TYPE=Release \
  -DELIO_BUILD_EXAMPLES=ON
cmake --build build-release --target cancel_callback_benchmark --parallel 2
taskset -c 2 ./build-release/examples/cancel_callback_benchmark 2000

Use --smoke for a short 20-sample termination and output check. The optional numeric argument must be a strict positive sample count; invalid or additional arguments return exit status 2.

Use at least two isolated CPUs for the concurrent-unregister tail diagnostic, for example taskset -c 2,3. Its removed and invoked outcome totals confirm that cancellation and teardown actually overlapped. Keep the one-CPU run for the single-thread register, unlink, dispatch, and immediate-callback metrics. Use an external paired runner across multiple process invocations to calculate revision-to-revision confidence intervals; the in-process percentiles are diagnostic samples, not a substitute for paired confidence intervals.

Avoiding Allocations

Keep coroutine frames small to reduce allocation and cache cost:

// Bad: Large array increases every coroutine frame allocation
coro::task<void> large_frame() {
    char buffer[8192];  // Increases every frame allocation
    co_await read_data(buffer);
}

// Good: Allocate separately
coro::task<void> small_frame() {
    auto buffer = std::make_unique<char[]>(8192);
    co_await read_data(buffer.get());
}

Synchronization Primitives

Event Ready Path

A non-cancellable wait on an already-set manual-reset event completes with an atomic state check and does not allocate. A wait that reaches the unset slow path creates shared wake state for safe dequeue-then-schedule lifetime management. Token-aware waits create that state eagerly because the cancellation callback must be able to select a terminal result before suspension.

Mutex Performance

Elio's mutex uses an atomic, allocation-free fast path for non-cancellable, uncontended locks. It rechecks that fast path at suspension entry, so an unlock that wins the await_ready() / await_suspend() race also avoids allocation. A wait that remains contended at that recheck creates independently owned wake state before taking the queue lock. It enters the FIFO waiter queue only if the final locked acquisition recheck also fails; otherwise it acquires without parking and releases the unused wake state. Token-aware locks create arbitration state eagerly so cancellation can race ownership transfer safely. This deferral favors workloads with a meaningful uncontended fraction; a permanently contended handoff loop can be slightly slower.

#include <elio/sync/primitives.hpp>

sync::mutex mtx;

// Fast path: atomic CAS (~10ns)
// Slow path: suspend and queue (~100ns + context switch)

coro::task<void> critical_section() {
    co_await mtx.lock();
    // ... critical section ...
    mtx.unlock();
}

// Use try_lock to avoid blocking
if (mtx.try_lock()) {
    // Got lock immediately
    mtx.unlock();
} else {
    // Skip or retry later
}

Semaphore Performance

A non-cancellable semaphore::acquire() consumes an immediately available permit without allocating shared wake state. If no permit is ready, the awaiter creates independently owned state before taking the queue lock and rechecks the count under that lock before parking. This preserves permit accounting when release() races the transition from await_ready() to await_suspend(), although that race can create one state that is not used for suspension. Token-aware acquires retain eager allocation for cancellation and permit-handoff arbitration.

For steady-state permit guards, pair each successful acquire with the matching release:

sync::semaphore permits(1);

coro::task<void> use_permit() {
    co_await permits.acquire();
    // ... bounded work ...
    permits.release();
}

Reader-Writer Lock

For read-heavy workloads:

sync::shared_mutex rw_mtx;

// Multiple concurrent readers (atomic counter, no blocking)
coro::task<void> reader() {
    co_await rw_mtx.lock_shared();
    auto data = read_data();
    rw_mtx.unlock_shared();
}

// Exclusive writers
coro::task<void> writer() {
    co_await rw_mtx.lock();
    write_data();
    rw_mtx.unlock();
}

No-token lock_shared() admission is allocation-free when the initial ready CAS or the suspension-entry retry succeeds. A reader that observes an active or waiting writer retains independently owned wake state before entering the waiter queue, so dequeue-to-schedule lifetime and reserved-grant recovery remain safe. Token-aware readers create cancellation arbitration state eagerly.

No-token lock() also avoids wake-state allocation when its ready check acquires exclusive ownership. A writer that enters await_suspend() allocates once before the queue mutex and before publishing WRITER_WAITING; allocation failure cannot leave writer preference, pending-writer accounting, or a queue node behind. Token-aware writers remain eager. Unlike readers, this allocation occurs before writer preference becomes visible, so sustained-reader writer progress remains an explicit correctness and measurement requirement.

Benchmark ready readers and writers, forced writer/reader handoffs, mixed reader/writer loads, and sustained queued-writer pressure with the same final source against baseline and candidate include trees. Use fixed allowed CPUs and at least 30 balanced, interleaved process pairs. Analyze paired log ratios with at least 100,000 stratified bootstrap resamples. A candidate must show a supported ready reader improvement. Because this optimization deliberately moves wake-state allocation from every reader to the contended path, assess forced handoff by absolute cost: its paired point increase must not exceed 3 ns and its 95% confidence upper increase must not exceed 4 ns. Reader and mixed aggregate controls fail if their paired point regresses by more than 2% or their confidence interval establishes a directional regression. Writer p95 and p99 point regressions must not exceed 5%, with 95% upper bounds no greater than 10%. Report the raw maximum as a scheduling-sensitive diagnostic rather than a starvation gate. Every pressure sample must complete the exact configured writer count with zero no-progress observations; reader admission must never pass an already-published writer.

For writer lazy allocation, treat sub-3% controlled differences as approximately neutral unless an exact mechanism count explains them. The ready writer must change from one wake-state allocation per measured operation to zero and show a material latency improvement. Evaluate last-reader and writer-to-writer handoff in absolute nanoseconds as well as ratios. A consistent mixed/pressure point regression above 5%, a p95/p99 point regression above 10%, an incomplete writer count, a timeout, or any zero- progress observation blocks the change. Confidence intervals and raw maxima must still be reported, but a CI-only overrun inside the noise band or a single raw maximum is diagnostic rather than an independent rejection criterion.

Each reader workload runs repeated co_await lock_shared() operations in a persistent coroutine frame. Scheduler construction, task creation, one same- path warmup acquisition, frame destruction, and scheduler shutdown stay outside the timed ready/reader interval. The forced row coordinates persistent reader and driver coroutines on one worker so writer release occurs only after the reader has published, and completion is counted only after the resumed reader releases its shared slot.

The writer-core suite uses one persistent writer frame for repeated ready co_await lock() operations. Its forced rows coordinate a persistent writer and driver on one scheduler worker; the driver releases the held reader or writer only after the target writer has completed await_suspend(), and counts completion only after the resumed writer releases exclusive ownership. One full handoff is warmed outside the timed interval.

Build examples/shared_mutex_reader_benchmark.cpp twice from the exact same final source and compiler flags, changing only the Elio include root. Formal timing builds must not define ELIO_RUNTIME_TEST_HOOKS, because the hook's failure-injection and allocation counters add atomic RMWs to wake-state construction. Build the same source separately as shared_mutex_writer_allocation_probe with hooks enabled to verify the exact ready-writer allocation change; do not use probe timings as performance evidence. Formal timing runs use one strict suite per process:

shared_mutex_reader_benchmark --suite core --iterations 1000000
shared_mutex_reader_benchmark --suite writer-core --iterations 1000000
shared_mutex_reader_benchmark --suite readers --workers 4 --iterations 250000
shared_mutex_reader_benchmark --suite mixed --workers 4 --reader-percent 90 --iterations 100000
shared_mutex_reader_benchmark --suite mixed --workers 4 --reader-percent 50 --iterations 100000
shared_mutex_reader_benchmark --suite pressure --readers 4 --iterations 50000

Run core and the one-reader readers case on one allowed physical CPU. Pin multi-reader and mixed cases to exactly the requested number of physical CPUs. The pressure case uses one affinity-bound scheduler worker per reader, one additional scheduler worker for queued writers, and the calling thread as the driver, so it needs the requested reader CPUs plus two physical CPUs; disclose any oversubscription. Apply an external process timeout, reject unexpected row counts or incomplete operation/progress totals, and alternate baseline/candidate order in every pair. --smoke is only a functional driver, not a formal sample.

Channel Selection

Choose appropriate channel type:

// Rendezvous channel: synchronous hand-off, no buffering (default)
sync::channel<int> ch;       // or equivalently: sync::channel<int> ch(0);

// Bounded channel: back-pressure, bounded memory
sync::channel<int> bch(100);

// Unbounded channel: faster but can grow indefinitely
auto uch = sync::channel<int>::unbounded();

// Low-level bounded MPMC ring: non-blocking try_push/try_pop
// Requires #include <elio/sync/lockfree_ring.hpp>
sync::LockfreeMPMCRing<int> ring(1024);
int value = 42;
bool pushed = ring.try_push(value);
auto popped = ring.try_pop();

The channel<T>::send(...) task stores one by-value T in its coroutine frame. When a bounded or rendezvous send must wait, its intrusive waiter borrows that same frame-owned object rather than moving it into a second T subobject. The reduction is approximately one payload region for large inline types and applies to both ordinary and token-aware sends. It does not change the delivery move: the payload is transferred only after normal completion wins. Public awaiter objects constructed directly by callers still own one independent payload so their lifetime does not depend on a constructor argument.

bench_channel_send_frame reports requested coroutine-frame bytes, allocator-usable bytes where the platform exposes them, construction/destruction cost for naturally aligned 8/64/256/1024-byte inline payloads, ready bounded/unbounded sends, and forced bounded-full/rendezvous handoffs with and without active tokens. Its allocation recorder is isolated from the ordinary bench_channel executable. Compare Release builds with pinned, interleaved baseline/candidate samples; the benchmark intentionally does not impose timing thresholds on shared CI runners. Pass --smoke for reduced iteration counts when validating a Debug build.

After a successful bounded receive, Elio checks the blocked-sender queue while holding the channel mutex. An empty queue returns without taking the separate per-credit refill lock. Token-aware recv(token) folds that empty check into the lock already protecting its successful ring pop. If a sender is queued, the existing finite credit snapshot, per-credit physical-slot check, FIFO claim, cancellation arbitration, and post-unlock scheduling are unchanged. Because sender enqueue uses the same mutex, a sender arriving after an empty decision either uses the newly reusable slot directly or, after out-of-order consumer publication, is handled by the consumer that publishes the next producer slot.

bench_channel_refill isolates ready bounded recv(), try_recv(), and active-token receive paths, forced full-channel sender refill, cancellation and close controls, and 1/1, 2/2, and 4/4 producer/consumer throughput. Build the same final benchmark source against baseline and candidate include trees, then run at least 30 process-level pairs in alternating baseline/candidate order on a fixed allowed CPU set. Analyze paired log ratios with at least 100,000 stratified bootstrap resamples. A formal invocation emits exactly nine unique ns/op rows; reject missing, duplicate, non-finite, or nonzero-exit samples. Ready receive rows must improve. Because the empty-list check deliberately trades a small queued-sender branch cost for removing a redundant lock from the common path, assess forced refill by absolute cost: its paired point increase must not exceed 3 ns and its 95% confidence upper increase must not exceed 6 ns. Producer/consumer throughput may not regress by more than 2%. Cancellation and close rows are semantic controls and must remain stable. Run the 1/1, 2/2, and 4/4 rows on allowed sets containing 2, 4, and 8 distinct physical CPUs respectively; an oversubscribed row is diagnostic only.

cmake --build build-release --target bench_channel_refill --parallel 2
taskset -c 2,3 ./build-release/examples/bench_channel_refill

Wrap every benchmark process in an external timeout because the executable has no internal watchdog, and record any timeout as a failed sample rather than a performance observation.

Use --smoke only for a short build and termination check; it is not a formal performance sample.

Joinable Task Destruction State

join_handle::wait_destroyed() observes a native-width one-way atomic state. Final frame teardown publishes destroyed with release ordering and calls notify_all(); external waiters use acquire load/wait loops. This keeps result completion and final frame destruction separate while avoiding a second mutex/condition-variable protocol and waiter-registration bit. Multiple external threads may wait concurrently. Scheduler workers must not block in this API.

bench_join_destroy_atomic reports join-state layout, destruction publication, already-destroyed waits, 1/2/4/8 blocking waiters, direct joinable spawn/drain, batch spawn/drain, and pending-task release. The blocking suite uses an untimed 50-microsecond stabilization interval before each wake measurement. Build the exact same final source against baseline and candidate include trees, use fixed physical CPU sets and balanced process order, retain raw samples, and wrap each process in an external timeout.

Treat sub-3% controlled timing differences as approximately neutral. The primary acceptance criterion is the simpler one-state synchronization contract; a consistent mean regression above 5%, a stable p95 point regression above 10%, or any correctness/liveness failure blocks the change. With only tens of process-level samples, run-level p99 is effectively a near-maximum statistic; report it and the raw maximum as diagnostics instead of independent rejection gates. Hosted CI should run correctness tests only and must not enforce timing thresholds.

cmake --build build-release --target bench_join_destroy_atomic --parallel 2
taskset -c 2,3 ./build-release/examples/bench_join_destroy_atomic --smoke

Fair TCP Loopback Benchmark

The optional TCP loopback suite exposes two separately attributable comparison axes. Elio, libuv, and standalone Asio clients connect to the same dependency-neutral bench_tcp_reference server. The dependency-neutral bench_tcp_reference_client separately drives the Elio, libuv, and Asio servers. Every pair uses the same versioned wire contract and fixed amount of work. Self-pair measurements combine both halves of the system and therefore must be labelled as integration results, not as isolated client or server scores.

The suite defines three separate workloads:

  • latency keeps exactly one record outstanding. After a fully drained fixed warm-up, each measured sample starts immediately before one complete record write and ends only after the matching echoed record has been completely read and validated. It reports RTT distribution statistics and records per second.
  • message measures logical record rate. Every record is one complete logical write operation in every adapter. There is one writer per TCP stream and at most one write operation in flight. --credit-window limits sent records whose echoes have not yet been validated; it never creates overlapping writes on the same stream.
  • bulk measures application-record throughput. Every adapter uses the same explicit --chunk-bytes logical write size and transfers exactly --bulk-bytes, including the 32-byte benchmark header in each chunk. It reports MiB/s, not an invented I/O-operation count.

All workloads preallocate their hot-path buffers, handle short reads and writes until the negotiated record or chunk is complete, and validate trial identity, sequence, payload, duplicates, loss, and ordering. Warm-up records are fully acknowledged before the measured clock starts. The clock stops only after the last measured echo is verified, so an implementation cannot improve its result by closing with unaccounted work in flight.

The deterministic payload is initialized once and only the fixed header is stamped for each send. Receivers still inspect every payload byte. This keeps the integrity check exact without making bulk timing include per-chunk payload generation and hashing; dedicated runs should still confirm that neither reference peer is saturated by validation work.

Each performance JSON result is self-describing and includes its schema version, implementation, measured role, peer, workload, configured unit size, target and verified counts, logical write submissions and completions, sent/received/verified records, verified bytes, and maximum same-stream write concurrency. counter_scope=adapter identifies client adapter counters. A server performance result uses counter_scope=driver because its interval and request-side counters come from the common POSIX reference client. The server process independently emits per-connection counters to server-evidence.jsonl; the runner checks those server-observed record/byte totals, integrity status, and write concurrency against the scheduled work. A valid result must obey all of the following relevant identities:

configured phase records == write_submissions == write_completions
configured phase records == sent_records == received_records == verified_records
sent_bytes == received_bytes == verified_bytes == records * write_size_bytes
max_write_operations_in_flight <= 1
max_unacknowledged_records <= credit_window
all integrity, write-size, record-per-write, and transport error counts == 0

Run the short conformance suite after building the eight optional targets:

python3 tools/run-tcp-benchmark-conformance.py \
  --reference-server build-release/examples/bench_tcp_reference \
  --reference-client build-release/examples/bench_tcp_reference_client \
  --elio-client build-release/examples/bench_tcp_elio \
  --elio-server build-release/examples/bench_tcp_elio_server \
  --libuv-client build-release/examples/bench_tcp_libuv \
  --libuv-server build-release/examples/bench_tcp_libuv_server \
  --asio-client build-release/examples/bench_tcp_asio \
  --asio-server build-release/examples/bench_tcp_asio_server \
  --output-dir tcp-benchmark-conformance

This smoke run proves protocol compatibility, termination, validation, exact accounting, equivalent logical write sizes, and the single-writer invariant. It also saves a manifest with the revision, environment, executable hashes, and every executed command. Every results.jsonl row from this runner is marked performance_eligible=false; its elapsed fields remain useful for diagnosing a stalled or anomalous run but are unscored. It does not prove that one implementation is faster. The generated summary.md and summary.json expose overall correctness totals, separate client/server matrices, the 3-by-3 cross-runtime interoperability matrix, and diagnostic latency and throughput observations. Runtime rows retain a fixed order and the report contains no winner, ranking, ratio, or statistical-significance claim. GitHub Actions renders the Markdown report in its job summary and links the complete evidence artifact; failed partial runs report completed coverage, the remaining expected coverage, and the failure reason when enough evidence exists to construct an honest summary.

Producing A Performance Baseline

Use dedicated, otherwise-idle hardware for publishable comparisons. Pin the reference peer and client to fixed, disjoint physical CPUs; record the CPU, kernel, compiler, build flags, socket settings, frequency policy, benchmark schema, and revision; and retain every raw JSON result. Keep workload parameters identical across implementations and avoid background frequency or thermal changes.

After building all eight binaries, invoke the dedicated comparison runner (the CPU numbers below are examples; verify they are separate physical cores on the actual host):

python3 tools/run-tcp-performance-comparison.py \
  --reference-server build-release/examples/bench_tcp_reference \
  --reference-client build-release/examples/bench_tcp_reference_client \
  --elio-client build-release/examples/bench_tcp_elio \
  --elio-server build-release/examples/bench_tcp_elio_server \
  --libuv-client build-release/examples/bench_tcp_libuv \
  --libuv-server build-release/examples/bench_tcp_libuv_server \
  --asio-client build-release/examples/bench_tcp_asio \
  --asio-server build-release/examples/bench_tcp_asio_server \
  --client-cpus 2 --server-cpus 4 \
  --dedicated-host \
  --build-metadata /var/tmp/elio-build-metadata.json \
  --blocks 18 --seed 1145 \
  --output-dir /var/tmp/elio-tcp-performance-1145

The runner rejects overlapping or SMT-sibling CPU selections and records the seed, exact randomized order, commands, environment, clean revision, runner and binary hashes, raw trials.jsonl, and label-linked server-evidence.jsonl. Omitting affinity intentionally selects smoke mode: all samples remain performance_eligible=false and no comparison ratio is published. --dedicated-host records the operator's assertion that the host is otherwise idle; without it, results are also smoke-only. A dirty worktree also makes the run unqualified. CPU isolation does not prove that the machine is otherwise idle; checking system load, frequency, thermal state, and background services remains the operator's responsibility.

/var/tmp/elio-build-metadata.json is a caller-maintained JSON object kept outside the worktree so creating it cannot make the source revision dirty. At minimum, record the compiler name/version, build type, compile/link flags, and CMake options, for example:

{
  "compiler": "g++",
  "compiler_version": "14.2.0",
  "build_type": "Release",
  "cxxflags": "-O3 -DNDEBUG",
  "ldflags": "",
  "cmake_options": {"ELIO_ENABLE_DEBUG_METADATA": "OFF"},
  "source_revision": "full-git-head-sha"
}

The runner stores both the object and its SHA-256. Omitting it is allowed for a smoke run but makes every result ineligible for publication; a supplied file with missing fields or a source_revision different from HEAD is rejected. Likewise, fewer than 18 blocks or a measured phase shorter than the default 250 ms is diagnostic-only. The runner rejects --minimum-measured-ms values below 250; the option may only raise the publication floor. Increase the fixed work count when a case is too short.

For each client-reference trial the runner conservatively divides the POSIX reference server's whole-connection process CPU time (including warm-up) by the client's measured-phase wall time. This can overstate server utilization and make a trial ineligible, but cannot hide reference-server saturation. For each server-reference trial it divides the POSIX reference client's measured-phase process CPU time by that same phase's wall time. The default 90% saturation limit is configurable with --reference-peer-max-cpu-percent; reaching it marks the trial ineligible and suppresses aggregate ratios instead of attributing the reference bottleneck to the runtime under test.

Run at least 18 balanced blocks for every workload and parameter set. Each block runs all three implementations once; distribute the six possible process orders evenly so cache, temperature, and time drift are not confounded with one adapter. Report the raw block samples, medians and dispersion, plus paired confidence intervals for comparisons. Investigate outliers rather than silently deleting them, and do not infer a regression or advantage from a percentage smaller than the observed run-to-run noise.

summary.json and summary.md report per implementation, role, workload, and size medians, MAD, and IQR plus paired bootstrap 95% intervals. If the interval cannot resolve a two-percentage-point effect, the result is explicitly inconclusive_at_2_percent. Any failed eligibility condition marks the group unqualified and suppresses its pairwise change and interval.

GitHub-hosted public runners execute only the fixed-work conformance smoke. Their changing hardware, co-tenancy, frequency state, and run order make them unsuitable for stable performance ratios. The workflow therefore publishes no cross-library ranking and applies no timing or performance-ratio gate. The latency and throughput values visible in its summary remain single-run diagnostic observations with performance_eligible=false; visibility does not turn them into a ranking, ratio, or significance result. A real performance comparison must be generated by a separate dedicated-runner invocation following the balanced-block procedure above; public conformance artifacts must never be promoted into that sample set.

Network Performance

Connection Pooling

HTTP client uses connection pooling by default:

http::client_config config;
config.max_connections_per_host = 10;
config.pool_idle_timeout = std::chrono::seconds(60);

http::client client(config);  // No io_context parameter

Buffer Sizes

Tune read buffer sizes for your workload:

http::client_config config;
config.read_buffer_size = 16384;  // 16KB (default: 8KB)

// For large payloads
config.read_buffer_size = 65536;  // 64KB

TCP Settings

Configure TCP options for performance:

// Enable TCP_NODELAY for latency-sensitive applications
net::tcp_stream stream = /* ... */;
stream.set_no_delay(true);  // Note: underscore in method name

// Buffer sizes are set via tcp_options at connection time, not on the stream
net::tcp_options opts;
opts.recv_buffer = 65536;
opts.send_buffer = 65536;

Profiling and Monitoring

Scheduler Statistics

The scheduler exposes individual metric accessors rather than a single stats struct:

// Available scheduler metrics
size_t total = sched.total_tasks_executed();     // Total across all workers
size_t w0 = sched.worker_tasks_executed(0);      // Worker 0's count
size_t pending = sched.pending_tasks();          // Currently pending tasks
size_t threads = sched.num_threads();            // Logical scheduling pool size

These are lightweight atomic reads suitable for periodic monitoring in production. Combine with set_thread_count to implement your own adaptive scaling.

During retirement, pending_tasks() (and active_tasks()) includes load owned by draining workers outside the logical num_threads() range. The execution and steal counters, in contrast, cover only currently visible workers.

Workers publish execution and successful-steal counts from single-writer local counters, so metric collection does not require atomic read-modify-write instructions on coroutine resume or steal paths.

Logging Overhead

Debug logging has overhead; disable in production:

// Set at compile time
// cmake -DELIO_ENABLE_DEBUG_METADATA=OFF ..

// Or at runtime
elio::log::logger::instance().set_level(elio::log::level::warning);

Coroutine Stack Tracing

Use virtual stack for debugging without significant overhead:

// Enable in debug builds only
#if ELIO_ENABLE_DEBUG_METADATA
    auto* frame = coro::promise_base::current_frame();
    auto stack = coro::dump_virtual_stack();
    for (const auto& entry : stack) {
        fmt::print("{}\n", entry);
    }
#endif

Benchmarking Tips

Warm-up

// Warm up allocators and caches
for (int i = 0; i < 1000; i++) {
    elio::go(warmup_task);
}
sched.wait_for_idle();

// Now measure
auto start = std::chrono::steady_clock::now();
// ... actual benchmark ...
auto end = std::chrono::steady_clock::now();

Avoid Measurement Overhead

// Bad: timing inside hot loop
for (int i = 0; i < 1000000; i++) {
    auto start = now();  // Overhead!
    do_work();
    auto end = now();
    record(end - start);
}

// Good: time the whole batch
auto start = now();
for (int i = 0; i < 1000000; i++) {
    do_work();
}
auto end = now();
auto avg = (end - start) / 1000000;

Use Release Builds

Always benchmark with optimizations:

cmake -DCMAKE_BUILD_TYPE=Release ..
cmake --build .

Common Performance Issues

Problem: High Latency Spikes

Causes:

  • Work stealing delays
  • GC pauses in other processes
  • Kernel scheduling

Solutions:

  • Pin critical tasks to workers
  • Use CPU affinity for scheduler threads
  • Consider real-time scheduling

Elio periodically services competing work even if a worker remains continuously runnable: external submissions every 256 completed worker-loop task dispatches and pending I/O every 16,384 dispatches. These constants deliberately use different scales because checking an inbox is cheap while polling a backend has measurable cost. They bound cooperative scheduler turns, not elapsed time. A coroutine that runs CPU work without suspending can still starve its worker; split that work, yield at suitable application boundaries, or use spawn_blocking(). A backend poll may resume a ready completion batch inline; those resumptions are part of the poll, not separate worker-loop dispatches.

Problem: Low Throughput

Causes:

  • Lock contention
  • Inefficient I/O batching
  • Small buffer sizes

Solutions:

  • Profile lock contention
  • Use io_uring for batching
  • Increase buffer sizes

Problem: High Memory Usage

Causes:

  • Unbounded channels
  • Large coroutine frames
  • Connection pool growth

Solutions:

  • Use bounded channels
  • Allocate large buffers separately
  • Limit connection pool size

Running Benchmarks

Elio includes several benchmark tools:

cmake --build build

# Quick benchmark - measures spawn, context switch, yield, reschedule,
# and external submission bursts
./build/examples/quick_benchmark

# Microbenchmarks - individual operation timing
./build/examples/microbench

# Scheduler service latency - armed I/O and remote inbox under runnable load
./build/examples/scheduler_service_benchmark

# I/O benchmark - file read throughput
./build/examples/io_benchmark

# Full benchmark suite
./build/examples/benchmark

# Scalability test - multi-thread scaling
./build/examples/scalability_test

Interpreting Results

Benchmark results can vary significantly (min/max differ by 2-7x) due to:

  • CPU frequency scaling
  • System load
  • Cache state
  • Memory allocation patterns

Run benchmarks multiple times and use minimum values for best-case analysis. For scheduler_service_benchmark, inspect p50, p99, and maximum latency rather than the minimum. The benchmark intentionally keeps one worker's local deque runnable while it delivers one armed I/O completion and one remote submission; it measures scheduler service delay, not the underlying eventfd syscall time. It also reports total backlog yields so a latency improvement can be evaluated against its worst-case pending-I/O throughput cost. Use --smoke only to verify build and termination in CI. Timing values are not CI pass/fail thresholds because shared runners are noisy.

Quick Reference

Scenario Recommendation
I/O-bound 2x core count threads
CPU-bound 1x core count threads
Latency-critical Pin to workers, io_uring
Throughput-critical Large buffers, batching
Memory-constrained Bounded channels, small pools
Read-heavy sync Use shared_mutex

See Also

Clone this wiki locally