Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion context/design.md
Original file line number Diff line number Diff line change
Expand Up @@ -175,7 +175,7 @@ The algorithm's existing Ruby performance motivates the port. Rust performance c

1. Record boundaries and reusable Rust conventions (implemented).
2. Replace stackful execution with async-task and Crossbeam worker queues (implemented). Keep the coroutine prototype in its saved branch.
3. Implement primitive task dispatch, scheduler ownership, cancellation and shutdown (implemented). Runtime-independent cooperative signals and deferred cancellation are implemented; cancellation-aware file/socket operations and transferable task ownership are implemented. Clock cancellation is implemented; exclusive file close and bounded submission-only cleanup are implemented; wider cleanup hooks remain future work. Higher-level task groups are outside the executor for now.
3. Implement primitive task dispatch, scheduler ownership, cancellation and shutdown (implemented). Runtime-independent cooperative signals and deferred cancellation are implemented; cancellation-aware file/socket operations and transferable task ownership are implemented. Clock cancellation is implemented; exclusive file/socket close, directional socket shutdown, and bounded submission-only cleanup are implemented; file advice remains future work. Higher-level task groups are outside the executor for now.
4. Establish minimal clock and I/O contracts with concrete consumers (implemented with Socket, File, Clock and a portable TCP example).
5. Implement native readiness and the Tokio adapter, running the same consumers with both (implemented). Native sleep uses async-io until scheduler integration with `socketry-timers`.
6. Implement io\_uring's owned-buffer lifecycle, socket/file operations, cancellation and runtime probing (implemented). Improve operation reuse, buffer registration and submission backpressure in subsequent work.
Expand Down
4 changes: 2 additions & 2 deletions context/gaps.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ Task cancellation currently drops the future before its next poll. An active pol

`Cancellation` and `defer_cancel` now provide runtime-independent cooperative requests and deferred cleanup. Signals can be shared or linked through child signals to shut down the application or an individual service. They signal intent; task handles still confirm completion.

File and socket I/O operations now accept these signals. Explicit file close omits cancellation, rejects outstanding use, and uses bounded cleanup admission; graceful scheduler shutdown drains accepted closes. Applications explicitly await tasks they want to finish; abandoning an owning task handle requests destruction. Graceful shutdown must keep workers, I/O services, and the scheduler available until draining completes. Cooperative cancellation has no automatic forced abort or internal escalation; a process supervisor handles SIGTERM followed by SIGKILL. Existing task cancellation and scheduler shutdown still destroy futures, so callers must request cooperative cancellation and await work first.
File and socket I/O operations now accept these signals. Explicit file/socket close omits cancellation, rejects outstanding use, and uses bounded cleanup admission; socket shutdown retains the borrowed descriptor. Graceful scheduler shutdown drains accepted cleanup requests. Applications explicitly await tasks they want to finish; abandoning an owning task handle requests destruction. Graceful shutdown must keep workers, I/O services, and the scheduler available until draining completes. Cooperative cancellation has no automatic forced abort or internal escalation; a process supervisor handles SIGTERM followed by SIGKILL. Existing task cancellation and scheduler shutdown still destroy futures, so callers must request cooperative cancellation and await work first.

## 2. Higher-level task grouping is deferred

Expand All @@ -18,7 +18,7 @@ The executor does not expose a Barrier or require a group/owner argument. Librar

## 3. The I/O boundary is still narrow

`Socket` uses one associated resource type for unconnected, bound, listening, connected, and datagram sockets. It exposes `socket_open`, `socket_bind`, syscall-like `socket_listen`, `socket_connect`, `socket_accept`, address inspection, reads, writes, and readiness. The initial API supports IPv4/IPv6 and stream/datagram sockets with optional TCP/UDP protocol selection. OS-resource import is a separate `SocketRegistration` capability. Socket close/shutdown, options, message/address APIs, and ancillary data remain future work. `File` opens associated resources and borrows them for seeking, truncation, synchronization, shared-cursor and positioned I/O. Explicit close/submission now consumes an exclusively owned handle, and optional `FileAllocate` supplies Linux allocation. Metadata hooks remain to implement. File/socket operations support cancellation and owned buffer views. `Clock` supplies monotonic time and cancellable absolute and relative waits.
`Socket` uses one associated resource type for unconnected, bound, listening, connected, and datagram sockets. It exposes `socket_open`, `socket_bind`, syscall-like `socket_listen`, `socket_connect`, `socket_accept`, address inspection, reads, writes, and readiness. The initial API supports IPv4/IPv6 and stream/datagram sockets with optional TCP/UDP protocol selection. OS-resource import is a separate `SocketRegistration` capability. Socket close/shutdown and their admission-only variants are implemented; options, message/address APIs, and ancillary data remain future work. `File` opens associated resources and borrows them for seeking, truncation, synchronization, shared-cursor and positioned I/O. Explicit close/submission now consumes an exclusively owned handle, and optional `FileAllocate` supplies Linux allocation. Metadata hooks remain to implement. File/socket operations support cancellation and owned buffer views. `Clock` supplies monotonic time and cancellable absolute and relative waits.

`Blocking` now adds bounded asynchronous admission for owned synchronous closures, implemented through dedicated native and Tokio pools. Existing file fallbacks still use backend pools. Socket options, message/address APIs, DNS, randomness, and processes remain outside the portable boundary. Deadline policies can now compose `Clock` and cancellation; higher-level timeout helpers remain separate. Ordinary synchronization uses futures and wakers above scheduler primitives. Expand capabilities as real consumers need them, keeping the application's implementation choice explicit. A broader boundary would also allow more operating-system behavior to be substituted in deterministic tests.

Expand Down
15 changes: 12 additions & 3 deletions context/implementation.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ This guide describes the current implementation and its boundaries. Read [the de
- The pool uses a mutex-protected queue, a condition variable for workers, and event listeners for asynchronous admission. Register listeners before inspecting capacity; never invoke wakers or destroy user captures/results under the queue lock.
- Check cancellation before admission and again with abandonment before calling the closure. Queued cancellation need not complete until dequeued. Once started, preserve the real output; closures can cooperate explicitly using a captured signal. Catch job panics and release capacity even when abandoned result destructors panic.
- Close admission and discard queued jobs at scheduler shutdown; started jobs retain their data and finish independently. Do not join pool threads or implicitly drain jobs in `Scheduler::run`. Applications await them before shutting down services. Partial thread startup failure closes the pool and releases already-started workers.
- Existing file fallbacks stay on their backend pools. File close uses a separate cleanup pool built from the same internal implementation: accepted jobs survive abandonment/closure and graceful shutdown drains them. Each pool uses the configured `BlockingOptions` limits. Public pool sharing and idle thread retirement remain separate work.
- Existing file fallbacks stay on their backend pools. File close and socket close/shutdown use a separate cleanup pool built from the same internal implementation: accepted jobs survive abandonment/closure and graceful shutdown drains them. Each pool uses the configured `BlockingOptions` limits. Public pool sharing and idle thread retirement remain separate work.

## Cooperative cancellation

Expand All @@ -58,7 +58,7 @@ This guide describes the current implementation and its boundaries. Read [the de
## I/O and runtime boundaries

- `scheduler.rs` re-exports Socket, File, Interest and Clock from `scheduler/socket.rs`, `file.rs`, `interest.rs` and `clock.rs`. Operations return concrete Send futures; portable consumers receive the required capabilities.
- `File` declares an associated resource and opens it with `FileOptions`; native/Tokio choose `Arc<std::fs::File>` internally. `Socket` creates TCP listeners and exposes address inspection; standard-library resource import belongs to `SocketRegistration`. Blocking truncation, synchronization, shared-cursor, and positioned file helpers live in `scheduler/file_io.rs`. `file_seek` accepts `std::io::SeekFrom` and returns the resulting offset through a cancellable blocking fallback on all current native backends and the Tokio runtime pool; await it and other cursor operations in order. `file_read`/`file_write` advance the OS cursor with no extra per-file state or locking; io\_uring represents current-position requests with `None` internally and encodes them as offset `-1`. Await cursor operations sequentially when ordering matters, and retain buffers through terminal completion as for positioned I/O.
- `File` declares an associated resource and opens it with `FileOptions`; native/Tokio choose `Arc<std::fs::File>` internally. `Socket` creates IP socket descriptors with explicit bind/listen/connect and address inspection; standard-library resource import belongs to `SocketRegistration`. Blocking truncation, synchronization, shared-cursor, and positioned file helpers live in `scheduler/file_io.rs`. `file_seek` accepts `std::io::SeekFrom` and returns the resulting offset through a cancellable blocking fallback on all current native backends and the Tokio runtime pool; await it and other cursor operations in order. `file_read`/`file_write` advance the OS cursor with no extra per-file state or locking; io\_uring represents current-position requests with `None` internally and encodes them as offset `-1`. Await cursor operations sequentially when ordering matters, and retain buffers through terminal completion as for positioned I/O.
- `FileAllocate: File` optionally reserves space with `AllocationMode::Extend`/`KeepSize`. Native readiness and Tokio implement it on Linux using `fallocate64` on their file pools; io\_uring probes optional `Fallocate` and falls back when absent. Other targets currently omit the implementation. Positive length and a range end within `i64` are required; data and cursor are preserved, and filesystem support errors remain runtime results. No resize/zero-write emulation, unsharing, or durability promise. The simulator reserves vector capacity within its 1 MiB bound.
- `File::file_truncate` changes the logical length with zero-filled extension, requiring write access and preserving the cursor. Lengths must fit in `i64`. Readiness and Tokio use `set_len` on their file pools; io\_uring probes optional `Ftruncate` support and falls back to the blocking implementation when absent. Buffer-free ring requests share resource retention and cancellation with sync. Await related operations sequentially; truncation provides neither space reservation nor durability. The simulator uses its 1 MiB bound.
- `File::file_sync` takes `SyncMode::All` or `SyncMode::Data` and waits for OS completion. io\_uring submits buffer-free `Fsync` requests through the same cancellation/completion machinery as transfers and requires opcode support at initialization. Readiness and Tokio use their existing file pools with `sync_all`/`sync_data`; the simulator completes immediately without modeling durability. Await preceding writes first; sync leaves the cursor unchanged and does not flush application buffers or synchronize directory entries. File ownership is retained through terminal completion even after waiter abandonment.
Expand All @@ -75,7 +75,7 @@ This guide describes the current implementation and its boundaries. Read [the de

## Next boundaries

- `socketry-timers`, also available through `socketry::timers`, implements the independent io-event timer queue with batched insertion, constant-time cancellation and lazy compaction. Its optional `sharded` feature adds worker queues, cross-worker assistance and lazy sleep futures; the experimental inbox driver remains optional. The crate has no executor dependency. Scheduler integration, operation pools, registered buffers, general descriptor/UDP interfaces and overlapped Windows file operations remain planned. `Clock::now` and cancellable absolute/relative waits use async-io on native and the selected runtime on Tokio. Relative sleeps start on first poll, and overflow maps to `InvalidInput`. Paused Tokio clocks are preserved; native timer-engine replacement remains separate work.
- `socketry-timers`, also available through `socketry::timers`, implements the independent io-event timer queue with batched insertion, constant-time cancellation and lazy compaction. Its optional `sharded` feature adds worker queues, cross-worker assistance and lazy sleep futures; the experimental inbox driver remains optional. The crate has no executor dependency. Scheduler integration, operation pools, registered buffers, socket options/message interfaces and overlapped Windows file operations remain planned. `Clock::now` and cancellable absolute/relative waits use async-io on native and the selected runtime on Tokio. Relative sleeps start on first poll, and overflow maps to `InvalidInput`. Paused Tokio clocks are preserved; native timer-engine replacement remains separate work.
- Keep runtime requirements explicit; ordinary Future support does not provide another runtime's I/O or timers.
- There is no non-Send task executor or scoped borrowing.
- Do not introduce unsafe Send, stack migration or nested blocking waits.
Expand All @@ -100,3 +100,12 @@ Coverage measures source regions for native and Tokio schedulers on each support
- All operation result notifications must follow release of backend file references, including cancellation and requests discarded before submission. io_uring still retains published requests until original terminal CQEs; cancellation CQEs cannot release them.
- `examples/file_close.rs` explains cancellation clones, keeping the operation future alive through cancellation, completion races, and explicit close before propagating an I/O result. Dropping an operation future abandons completion and may leave backend work running; io_uring requests cancellation, but immediate close can still panic until backend access ends.
- Close consumes the OS handle once and reports syscall errors without retrying. `_submit` execution errors are currently discarded. Ordinary resource destruction remains best effort; native close SQEs/CQE suppression and richer debug tracking are deferred.

## Explicit socket lifecycle

- `Socket` requires directional shutdown with `std::net::Shutdown`, consuming close, and `_submit` variants. Ordinary futures await OS completion; `_submit` awaits admission and discards execution errors. These lazy cleanup operations have no cancellation input.
- Native and Tokio descriptors privately retain their persistent registration through `Arc`, without exposing `Clone` or a common file/socket resource trait. Readiness reads/writes borrow it; accepted shutdown jobs and io_uring operations retain it. Explicit close checks exclusivity before admission and panics on outstanding use; the cleanup job then deregisters and closes on its worker.
- File and socket cleanup share `scheduler/cleanup.rs` and each scheduler's existing bounded cleanup pool. Admission, abandonment, error reporting, and graceful draining use the same contract. Native socket close/shutdown currently use this pool even with io_uring selected; native SQEs are deferred.
- Await prior writes before `Shutdown::Write`, and await shutdown completion before explicit close. `_submit` supplies no completion fence. Completion notifications follow release of retained backend ownership. Request and await I/O cancellation before close; never equate dropping an operation future with completion.
- Windows cleanup switches an exclusively owned, deregistered socket to blocking mode before closing, preserving positive linger behavior on the cleanup worker. Unix close is attempted once. `Drop` remains best effort.
- The simulator models directional EOF and disabled writes immediately, without deferred cleanup. Its `_submit` methods still discard execution errors after accepting the request. `examples/socket_shutdown.rs` explains request/response half-close and final ownership transfer.
Loading
Loading