diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..acf3c07 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,75 @@ +# Linux Systems Programming — project conventions + +Tutorial site: Linux systems programming in C++23, Go, and Rust for +intermediate→advanced developers. Jekyll/GitHub Pages (built in CI — no local +Ruby), examples in `examples/NN-slug/{cpp,go,rust}/`, chapters in +`_docs/NN-slug.md` with the same NN. + +## Scope rules + +- **eBPF is tooling only** (bcc-tools, bpftrace, bpftool) — we observe our own + userspace programs; we never write kernel eBPF. +- Examples run **locally** unless they need root, kernel observation, or a + second host — then the KVM lab: `systems-target` / `systems-peer` + (`scripts/lab/`, LAB_PREFIX=systems, Fedora 44 guests, snapshot `lab-ready`). +- Containers: multi-stage **Containerfile** (never "Dockerfile"), **UBI 10** + bases, rootless podman. + +## Toolchain pins + +- C++23 — system GCC (floor 14) primary, clang first-class alternate presets; + CMake ≥ 3.25, Presets v6, Ninja, `build/${presetName}`; Conan 2 only where + third-party deps exist. +- Go — `go.mod` `go 1.26` + `toolchain go1.26.5`. +- Rust — `rust-toolchain.toml` channel 1.97.1, edition 2024. +- Lua 5.4 (LuaJIT accepted locally) drives `verify.lua`; Python 3 drives + `scripts/test-all-examples.py` and `scripts/validate.py`. + +## Chapter conventions + +- Front matter: `title/order/part/description/duration`; `part:` must equal a + `_parts` `part_name` exactly; quote values containing colons. +- Spine: hook → figure → concept → "How the code works" → "Errors, three ways" + → "Concurrency lens" → "Build, run, observe" → cross-check → "What you + learned" → status footer. +- **Tools used box (required)**: immediately after the intro/figure, a callout + listing every tool the chapter invokes and where it runs, e.g. + `> **Tools used** — `strace` (host), `gdbserver` (systems-target VM), + `podman` (host). All appear in `scripts/check-host.sh` or the VM cloud-init.` + Every tool named must actually be exercised in the chapter. +- Code tabs: `{% include codetabs.html langs="C++|Go|Rust" %}` + exactly one + fenced block per label, in order, each a **verbatim excerpt** of the real + example source. +- Command prefixes: `[host]$`, `[vm]$`, `[peer]$`. +- Wrap literal `{{ }}`/`{% %}` in `{% raw %}…{% endraw %}` in prose. +- Diagrams: paired SVG+Excalidraw via `scripts/generate_diagram.py` + (`g.OUT="assets/diagrams"`); catalogue every diagram in + `assets/diagrams/README.md`; embed via the `excalidraw.html` include with a + `Figure N.x` caption. +- **Verification discipline**: a demo is *verified* only when it produced its + claimed observable effect on the stated environment; footers + (`status--verified` / `status--unverified`) state the behavioral evidence. + Every number quoted as "from a run" must come from a real run. +- **Banned words**: never use "honest", "honestly", "to be honest" — write + "real", "practical", "accurate" instead. + +## Examples + +- Mint new examples with `scripts/new-example.sh NN-slug`; register in + `examples/manifest.yaml` (`mode: local|vm|vm-peer`, `requires: [lgtm]`). +- Identical observable behavior across the three languages — one `verify.lua` + (checks.lua API) asserts behavior, never merely exit 0. +- Demo contract: `./demo.sh [cpp|go|rust] [build|run]`; `TARGET=` deploys + via `scripts/lab/deploy-to-vm.sh` (`SUDO=1` for root demos). +- Runner: `python3 scripts/test-all-examples.py` — VM examples strictly + serial; SKIP (not FAIL) when lab/LGTM is down. + +## Git + +- One branch per iteration (`docs/rNN-…`), PR, squash merge to main, delete + branch. Never commit directly to main. Conventional Commits: + types docs/site/demo/ci/chore/fix/feat/refactor/style; scopes `§N`, + `demo-NN`, `rNN`. +- CI (`validate.yml`) checks site integrity + build smoke only; the + authoritative gate is a host run of the runner with the lab up, recorded in + `_plans/`. diff --git a/_docs/01-prerequisites.md b/_docs/01-prerequisites.md index 9b81bd8..fad3851 100644 --- a/_docs/01-prerequisites.md +++ b/_docs/01-prerequisites.md @@ -25,6 +25,10 @@ every later example reuses. alt="Three toolchain stacks — GCC/CMake/gdb for C++, go1.26.5/go build/dlv for Go, rustc/cargo/clippy for Rust — each converging through its own demo.sh onto one shared demo.sh runner contract." caption="Figure 1.1 — three toolchains, one demo.sh contract shared by every example in the book" %} +> **Tools used** — `gcc`, `cmake`, `ninja`, `go`, `cargo`, `rustup`, `strace` +> (all host — the toolchains are this chapter's subject). Everything here is +> checked by `scripts/check-host.sh`. + ## One host, three toolchains The book targets **Fedora 44** deliberately. Systems programming is the one @@ -104,11 +108,16 @@ The hard requirements, and the reason each is hard: script checks the device node itself. - **`git` and `gh`** — the repo workflow. -The soft rows: **Conan 2.x** (only some C++ demos pull dependencies through -it), **ruby + bundler** (local Jekyll preview of this site only), and the Go -lint/debug trio **golangci-lint, staticcheck, dlv** (CI runs them; locally -they are nice-to-have). Membership in the `libvirt` group is also warn-only — -everything works with `sudo`, it is just tedious without the group. +Hard rows also cover the debug and observation tooling the cross-checks lean +on from here forward: **gdb, valgrind, perf, strace, ltrace**, and the +**sanitizer runtimes** (`libasan`/`libtsan` — without them the `asan` and +`tsan` presets compile but cannot link). The soft rows: **Conan 2.x** (only +some C++ demos pull dependencies through it) and the Go lint/debug trio +**golangci-lint, staticcheck, dlv** (CI runs them; locally they are +nice-to-have). Membership in the `libvirt` group is also warn-only — +everything works with `sudo`, it is just tedious without the group. There is +no Ruby row: the site builds and deploys in CI, so nothing local depends on +Jekyll. ## The demo contract diff --git a/_docs/02-lab-setup.md b/_docs/02-lab-setup.md index 8e98adb..e3ceba1 100644 --- a/_docs/02-lab-setup.md +++ b/_docs/02-lab-setup.md @@ -24,6 +24,10 @@ destroys the guests by name. alt="Lab topology: a host band with libvirtd/KVM, the ~/.cache/systems-lab image cache holding the pristine Fedora Cloud Base image plus one qcow2 overlay per guest, and the scripts/lab directory; below it, the libvirt NAT network 192.168.124.0/24 containing systems-target (4 GB, .7) and systems-peer (2 GB, .95), both on kernel 6.19.10. An amber deploy arrow runs from the scripts to systems-target; a dashed line links the two guests for later two-host chapters; a note marks the lab-ready snapshot taken on both." caption="Figure 2.1 — the systems-* lab: cached base image + per-VM overlays on the host, two guests on libvirt NAT, one amber deploy path" %} +> **Tools used** — `virt-install`, `qemu-img`, `cloud-localds`, `virsh`, `ssh` +> (host). Everything here is checked by `scripts/check-host.sh`; the guest +> toolset itself is installed by cloud-init. + ## Why a disposable lab Three kinds of chapters force the issue. **Privileged demos** — capabilities, diff --git a/_docs/03-observability-stack.md b/_docs/03-observability-stack.md index 7b4a159..712df65 100644 --- a/_docs/03-observability-stack.md +++ b/_docs/03-observability-stack.md @@ -25,6 +25,10 @@ Chapter 38 onward depends on what you stand up now. alt="Telemetry pipeline: a host demo and a VM demo send OTLP HTTP to port 4318 on the lsp-lgtm container, where the OpenTelemetry Collector fans out to Loki for logs, Tempo for traces, and Mimir for metrics; Grafana on port 3000 queries all three and serves the developer's browser. The VM demo reaches the host at the libvirt gateway address 192.168.124.1." caption="Figure 3.1 — one container, whole pipeline: OTLP in on :4318, Grafana out on :3000" %} +> **Tools used** — `podman`, `systemctl`, `curl`, `openssl`, `python3` (host); +> `ip` (systems-target VM). Everything here is checked by +> `scripts/check-host.sh`, ships with Fedora, or is preinstalled in the lab VMs. + ## What LGTM is, and why one container LGTM is Grafana Labs' acronym for its open-source backend quartet: **L**oki diff --git a/_docs/04-syscalls-and-the-abi.md b/_docs/04-syscalls-and-the-abi.md new file mode 100644 index 0000000..83bb123 --- /dev/null +++ b/_docs/04-syscalls-and-the-abi.md @@ -0,0 +1,391 @@ +--- +title: "Syscalls and the ABI" +order: 4 +part: "Foundations" +description: "What a system call really is — the syscall instruction, the register-level kernel ABI, and the vDSO fast path — traced through sysprobe, one specimen program that shows how glibc, the Go runtime, and rustix each reach the same kernel." +duration: 40 minutes +--- + +Chapter 1 ended on a number it did not explain: the C++ hello made 64 syscalls, +the Go hello made 217, and we waved at "the runtime" and moved on. This chapter +is where that debt gets paid. With the lab and observability stack built, Part +1 starts at the only interface that actually matters in this book — the +user/kernel boundary — and answers three questions precisely: what *is* a +system call at the instruction level, why do some libc calls never reach the +kernel at all (the vDSO), and what path does each of our three languages take +from your function call down to the `syscall` instruction. The vehicle is +`sysprobe`, a program designed not to *do* anything interesting but to be +*watched*: four labeled steps, each a different kind of syscall, run under +`strace` three times. + +The code is in `examples/04-syscalls-and-the-abi/`. `./demo.sh` builds and +runs all three implementations; the `README.md` there documents the shared +CLI contract and the exact five-line output that `verify.lua` asserts. + +{% include excalidraw.html + file="04-user-kernel-path" + alt="A user-space band with your code, a language wrapper, and the syscall instruction, plus a dashed vdso page; an amber trap arrow crosses into a kernel band through entry_SYSCALL_64 and the syscall table to sys_getrandom, with a dashed return arrow carrying rax back." + caption="Figure 4.1 — one getrandom(2) crossing the boundary: marshal registers, trap, dispatch by number, return rax — or take the vDSO fast path and never trap at all" %} + +> **Tools used** — `strace`, `objdump`, `ldd`, `python3` (host); `bpftrace` +> (lab VM, exercised in Part 8). Everything here is checked by +> `scripts/check-host.sh` or preinstalled in the lab VMs. + +## The boundary is an instruction, not an API + +A system call is not a function call that happens to be important. Your +process runs in ring 3, where the CPU refuses to touch page tables, device +registers, or other processes' memory; the kernel runs in ring 0, where it +can. The *only* legitimate way across on x86-64 is a handful of trap +instructions, and on modern Linux that means `syscall`. The contract around +that instruction — the **syscall ABI** — is register-level and brutally +simple: the syscall *number* goes in `rax` (318 for `getrandom` on x86-64), +up to six arguments go in `rdi`, `rsi`, `rdx`, `r10`, `r8`, `r9`, and the +result comes back in `rax`, where a small negative value means "error +`-errno`". Note the fourth register: the C function-call ABI would use `rcx`, +but the `syscall` instruction itself clobbers `rcx` to save the return +address, so the kernel ABI substitutes `r10`. That one register swap is why +every language needs at least a sliver of assembly or a wrapper library to +make a syscall — you cannot express "arg 4 in r10" in portable C. + +When the instruction executes, the CPU switches to ring 0 and jumps to the +kernel's `entry_SYSCALL_64`, which moves onto a kernel stack, saves your +registers, and indexes `sys_call_table` by `rax` to dispatch the handler. +The handler runs *in your process*, just in kernel mode — `sys_getrandom` +writes directly into the buffer your `rdi` pointed at. Then the path unwinds +and `sysretq` drops back to ring 3 with the result in `rax`. Every trap is a +full round trip — two mode switches plus the kernel work — which is why +syscall *count* is a performance currency, and why `strace`, which uses +`ptrace` to stop the process at exactly this boundary, sees every trap a +process makes and nothing else. + +## The vDSO: syscalls that never happen + +That "nothing else" has teeth. Some things the kernel exports are read so +often — the clock, above all — that trapping for each read would be absurd. +So the kernel maps a small shared library into every process, the **vDSO**, +visible in any process's memory map: + +```bash +[host]$ grep -E 'vdso|vvar' /proc/self/maps +7f9fc010b000-7f9fc010f000 r--p 00000000 00:00 0 [vvar] +7f9fc010f000-7f9fc0111000 r--p 00000000 00:00 0 [vvar_vclock] +7f9fc0111000-7f9fc0113000 r-xp 00000000 00:00 0 [vdso] +``` + +`[vdso]` is executable kernel-provided code; the `[vvar]` pages are read-only +data the kernel keeps current (timekeeping state in `vvar_vclock`). A +`clock_gettime` through glibc calls into that page, reads the shared data, +and returns — ring 3 the whole way. To see the difference, I compiled a +16-line probe that loops `clock_gettime(CLOCK_MONOTONIC, …)` a thousand +times, then `getrandom(buf, 16, 0)` a thousand times, and ran it under the +book's strace 7.1 build: + +```bash +[host]$ strace -c -e trace=clock_gettime,getrandom ./vdso-probe +% time seconds usecs/call calls errors syscall +------ ----------- ----------- --------- --------- ---------------- + 0.00 0.000000 0 2 getrandom +------ ----------- ----------- --------- --------- ---------------- +100.00 0.000000 0 2 total +``` + +One thousand `clock_gettime` calls: **zero** syscalls. One thousand +`getrandom` calls: **two** — and neither is the one you would predict. +The full trace shows `getrandom("…", 8, GRND_NONBLOCK) = 8` (glibc seeding +internal state at startup) and `getrandom("…", 32, 0) = 32`. Where did our +sixteen-byte request go? Kernel 7.1 plus glibc 2.43 route `getrandom` +through the vDSO's `vgetrandom`: on first use it traps *once* to pull a +32-byte ChaCha20 key from the kernel, then generates every byte you asked +for — all thousand requests — in user space. The lesson generalizes: `strace` +shows you the boundary, not your API calls, and on a current kernel even +"obvious" syscalls like `getrandom` may have quietly stopped being syscalls. +Keep that 32-byte signature in mind; it reappears inside `sysprobe`. + +## Three roads to the same instruction + +{% include excalidraw.html + file="04-runtime-syscall-layers" + alt="Three side-by-side stacks — C++ code over glibc wrappers over syscall instructions in libc, Go code over the Go runtime over 511 syscall instructions in a static binary, Rust code over the rustix linux_raw backend over 32 inlined syscall instructions — all with amber arrows labeled 70, 283, and 73 calls into one dark kernel ABI box." + caption="Figure 4.2 — the per-language layer stacks over the one register ABI, annotated with this chapter's real strace -c totals" %} + +The kernel ABI is one; the roads to it differ, and `file`, `ldd`, and +`objdump` on the three freshly built `sysprobe` binaries map them exactly. +Counting literal `syscall` instructions with +`objdump -d | grep -cw syscall`: + +- **C++ — 0.** The binary is dynamically linked (`ldd` lists `libstdc++`, + `libm`, `libgcc_s`, `libc`, and `linux-vdso.so.1` itself). Your code calls + glibc *wrappers*; every `syscall` instruction lives in `libc.so.6`. The + wrapper layer is not thin: it flips `-errno` into the `errno` global, + implements thread-cancellation points, chooses vDSO fast paths — and + sometimes substitutes a different syscall entirely, as we will see with + `nanosleep`. +- **Go — 511.** The binary is statically linked; `ldd` says + `not a dynamic executable`. Go carries its own syscall stubs, its own + calling convention, and its own stack discipline (goroutine stacks are + small and growable, so the runtime must ensure a syscall never runs on + one that might move). `golang.org/x/sys/unix` is a veneer over + `runtime`-provided entry points — including, since Go 1.24, the vDSO: + `x/sys` v0.47.0 links `unix.Getrandom` straight to `runtime.vgetrandom` + via `go:linkname` and only falls back to the raw syscall where the vDSO + is unsupported. +- **Rust — 32.** The binary still links `libc.so.6` — std uses it for + process startup — but `rustix` compiles with its `linux_raw` backend on + x86-64, inlining raw `syscall` instructions at each call site and + returning the `rax` error directly as `Result<_, Errno>`. No `errno` + global is ever involved, and no glibc policy sits between your call and + the kernel — which is exactly why rustix's `getrandom` shows up in + `strace` and glibc's does not. + +## How the code works + +`sysprobe` performs four steps, chosen so each exercises a different species +of syscall: **open** creates a resource (an fd), **write** moves data through +one, **sleep** blocks in the kernel, and **random** asks the kernel for a +service with partial-result semantics. Each step prints `step= ok`, the +program ends with `sysprobe: 4 steps ok`, and any argument gets `usage: app` +and exit 2 — identical in all three languages, so one `verify.lua` asserts +all five lines appear in order for each. Step 1 is the richest, so it is the +excerpt; all three are the real functions from +`examples/04-syscalls-and-the-abi/`, verbatim. + +{% include codetabs.html langs="C++|Go|Rust" %} + +```cpp +// Step 1: openat(2). Prefer an anonymous O_TMPFILE inode (no name ever +// appears in the directory); fall back to mkstemp(3) + unlink(2) on +// filesystems that lack O_TMPFILE. +[[nodiscard]] std::expected +open_scratch(const std::string& dir) { + if (const int fd = + ::openat(AT_FDCWD, dir.c_str(), O_TMPFILE | O_RDWR | O_CLOEXEC, 0600); + fd >= 0) { + return unique_fd{fd}; + } + if (errno != EOPNOTSUPP && errno != EISDIR && errno != EINVAL) { + return std::unexpected(last_error()); + } + std::string tmpl = dir + "/sysprobe.XXXXXX"; + const int fd = ::mkstemp(tmpl.data()); + if (fd < 0) { + return std::unexpected(last_error()); + } + unique_fd owned{fd}; + if (::unlink(tmpl.c_str()) != 0) { + return std::unexpected(last_error()); + } + return owned; +} +``` + +```go +// openScratch is step 1: openat(2). Prefer an anonymous O_TMPFILE inode (no +// name ever appears in the directory); fall back to a named create+unlink on +// filesystems that lack O_TMPFILE. +func openScratch(dir string) (int, error) { + fd, err := unix.Openat(unix.AT_FDCWD, dir, + unix.O_TMPFILE|unix.O_RDWR|unix.O_CLOEXEC, 0o600) + if err == nil { + return fd, nil + } + if !errors.Is(err, unix.EOPNOTSUPP) && !errors.Is(err, unix.EISDIR) && + !errors.Is(err, unix.EINVAL) { + return -1, fmt.Errorf("openat %s (O_TMPFILE): %w", dir, err) + } + path := fmt.Sprintf("%s/sysprobe.%d", dir, os.Getpid()) + fd, err = unix.Openat(unix.AT_FDCWD, path, + unix.O_CREAT|unix.O_EXCL|unix.O_RDWR|unix.O_CLOEXEC, 0o600) + if err != nil { + return -1, fmt.Errorf("openat %s: %w", path, err) + } + if err := unix.Unlink(path); err != nil { + unix.Close(fd) + return -1, fmt.Errorf("unlink %s: %w", path, err) + } + return fd, nil +} +``` + +```rust +/// Step 1: openat(2). Prefer an anonymous O_TMPFILE inode (no name ever +/// appears in the directory); fall back to a named create+unlink on +/// filesystems that lack O_TMPFILE. +fn open_scratch(dir: &str) -> Result { + let mode = Mode::RUSR | Mode::WUSR; + match fs::openat( + CWD, + dir, + OFlags::TMPFILE | OFlags::RDWR | OFlags::CLOEXEC, + mode, + ) { + Ok(fd) => return Ok(fd), + Err(Errno::OPNOTSUPP | Errno::ISDIR | Errno::INVAL) => {} + Err(e) => return Err(e).with_context(|| format!("openat {dir} (O_TMPFILE)")), + } + let path = format!("{dir}/sysprobe.{}", std::process::id()); + let fd = fs::openat( + CWD, + &path, + OFlags::CREATE | OFlags::EXCL | OFlags::RDWR | OFlags::CLOEXEC, + mode, + ) + .with_context(|| format!("openat {path}"))?; + fs::unlinkat(CWD, &path, AtFlags::empty()).with_context(|| format!("unlink {path}"))?; + Ok(fd) +} +``` + +The `openat` call opens the *directory itself* with `O_TMPFILE`: the kernel +allocates an inode with a link count of zero, so no name ever appears — no +cleanup, no symlink races, gone on close. Not every filesystem supports it, +so each version distinguishes "this filesystem can't" (`EOPNOTSUPP`, or the +`EISDIR`/`EINVAL` older kernels and filesystems report) — recoverable, take +the create-then-`unlink` fallback, which reaches the same anonymous-inode +state through a brief named window — from any other errno, which is a real +failure to propagate. Getting that triage *wrong* is invisible on your +machine and a crash on someone else's; getting it right is most of what +"handling" an errno means. + +Step 2, `write_all`, loops because `write(2)` may legally write *fewer* +bytes than asked, so a single unchecked call is a data-loss bug even when it +"works"; each iteration re-slices the buffer by the returned count. The fd +is then closed *deliberately early* — RAII scope exit in C++, a checked +`unix.Close` in Go, `drop(fd)` in Rust — so the trace shows a clean +`openat → write → close` group before the sleep starts. Step 3 sleeps 10 ms +via `nanosleep(2)`, and on `EINTR` resumes with the *remainder* the kernel +wrote back — resuming with the original request would restart the full 10 ms +every time a signal landed. Step 4 loops `getrandom(2)` over partial reads +for the same reason as `write_all`. `main` in each language is only +sequencing: check `argc`, resolve `$TMPDIR` (default `/tmp`), run the four +steps, print per-step lines as they succeed. + +### Errors, three ways + +All three languages receive the identical fact — `rax` came back negative — +and immediately disagree about what to do with it. glibc stores it into the +thread-local `errno` and returns `-1`, so the C++ code's first act is +`last_error()`: capture `errno` into a `std::error_code` *now*, before any +other libc call can overwrite it, and carry it in `std::expected` as a +value. Go's `unix` package returns a `syscall.Errno` as an ordinary `error`; +the code wraps with `%w` (`"openat %s (O_TMPFILE): %w"`) so `errors.Is(err, +unix.EOPNOTSUPP)` still matches through the added context. rustix never +materializes a global at all: the negative `rax` *is* the `Err(Errno)`, and +the fallback triage becomes a match arm — +`Err(Errno::OPNOTSUPP | Errno::ISDIR | Errno::INVAL)`. Same kernel fact, +three idioms; Chapter 5 turns this three-way comparison into the book's +error-handling policy. + +### Concurrency lens + +`strace -c` needs `-f` for the Go binary only, and the reason previews Part +7. Before `main` ran, the Go runtime made 4 `clone` calls (its worker +threads) and 114 `rt_sigaction` calls (installing handlers so it can +preempt goroutines); while our program slept its one 10 ms sleep, a runtime +thread issued 61 `nanosleep` calls of its own — visible in the trace as +`{tv_nsec=20000}` repeated, then doubling `40000`, `80000`, up to +`2560000` as the runtime's monitor backs off polling an idle program. +C++ and Rust stay a single thread from `execve` to exit, which is exactly +why their traces need no `-f`. The portable lesson is the `EINTR` loops: +once any process has signal handlers (as Go's runtime just demonstrated), +*every* blocking syscall can return early, so retry-on-`EINTR` is not +defensive decoration but the baseline discipline — Chapter 12 (signals) +makes it precise. + +## Build, run, observe + +```bash +[host]$ cd examples/04-syscalls-and-the-abi && ./demo.sh +``` + +Each language builds (CMake `release` preset, `go build`, `cargo build +--release`) and prints the identical five lines: + +``` +step=open ok +step=write ok +step=sleep ok +step=random ok +sysprobe: 4 steps ok +``` + +The book's harness agrees — `python3 scripts/test-all-examples.py --only +04-syscalls-and-the-abi` reports `PASS` for cpp, go, and rust. Now run it +the way it was designed to be run. This book's strace lives at +`scratchpad/usr/bin/strace` (v7.1) on the reference host; the C++ binary, +filtered to the interesting calls: + +```bash +[host]$ strace -e trace=openat,write,close,nanosleep,clock_nanosleep,getrandom ./cpp/build/release/app +... +openat(AT_FDCWD, "/tmp", O_RDWR|O_CLOEXEC|O_TMPFILE, 0600) = 3 +write(3, "sysprobe scratch payload\n", 25) = 25 +close(3) = 0 +clock_nanosleep(CLOCK_REALTIME, 0, {tv_sec=0, tv_nsec=10000000}, 0x7ffecebd5fc0) = 0 +getrandom("\xc3\x2e\x14\xc6...", 32, 0) = 32 +write(1, "step=open ok\nstep=write ok\nstep="..., 77) = 77 +``` + +Read that against the source and two "discrepancies" jump out, both +wrapper-layer fingerprints. The source says `nanosleep`; the trace says +`clock_nanosleep(CLOCK_REALTIME, …)` — glibc's `nanosleep()` is implemented +on the more general syscall. The source asks for 16 random bytes; the trace +shows one 32-byte `getrandom` — the vDSO key-seeding trap from earlier, after +which our request was served in user space. The Rust trace inverts both: +a raw `nanosleep({tv_sec=0, tv_nsec=10000000}, …)` and a literal +`getrandom("…", 16, 0) = 16` (plus `O_LARGEFILE` spelled out in its +`openat` flags — rustix passes explicitly what glibc implies). And Go, whose +`unix.Getrandom` now rides `runtime.vgetrandom`, shows a single 32-byte +`getrandom` for the whole process. One source-level contract, three +distinct boundary signatures — none of them wrong, all of them visible. + +## Cross-check: counting every trap + +Chapter 1 counted 64 syscalls for the C++ hello and 217 for Go's. `strace +-c` on the three `sysprobe` binaries (`-f` for Go) deepens that finding with +a full accounting; trimmed to the load-bearing rows from this session's +runs: + +``` +cpp : 70 total — 21 mmap, 6 openat, 6 close, 2 getrandom, 1 clock_nanosleep +go : 283 total — 114 rt_sigaction, 61 nanosleep, 24 mmap, 11 futex, + 10 sigaltstack, 4 clone, 1 getrandom +rust : 73 total — 12 mmap, 5 openat, 6 write, 2 getrandom, 1 nanosleep +``` + +The program's own work is four steps in every case; the rest is runtime +overhead, and the ratio matches Chapter 1's almost exactly (roughly 4× for +Go, near-parity for C++ and Rust). C++'s 21 `mmap`s are the dynamic loader +mapping four libraries; Rust's 73 is C++-shaped because its *startup* still +goes through glibc even though its syscalls do not. Go's 283 is dominated by +runtime self-management — signals, threads, and its sleeping monitor. As an +independent surface, the `objdump` census (0, 511, and 32 embedded `syscall` +instructions) confirms *where* each binary keeps its boundary crossings: +nowhere (glibc's), everywhere (a static runtime's), and inline at 32 rustix +call sites. Two tools, one story. + +

unverified — On the lab VM: the kernel-side view of this chapter — watching sys_enter_* tracepoints fire with bpftrace as sysprobe runs, counting traps from inside the kernel instead of via ptrace — needs root and a disposable kernel, so it is not run here; chapter 30 (Debugging part) exercises exactly this on systems-target.

+ +## What you learned + +- A syscall is a register protocol around one trapping instruction: number + in `rax`, args in `rdi rsi rdx r10 r8 r9` (`r10` because `syscall` + clobbers `rcx`), result or `-errno` back in `rax`. +- The vDSO answers hot calls without a trap — 1000 `clock_gettime` calls + produced zero syscalls here, and on kernel 7.1/glibc 2.43 even + `getrandom` is served in user space after one 32-byte key-seeding trap. +- The three languages take three roads to the same ABI — glibc wrappers + (0 `syscall` instructions in the binary), the Go runtime's own stubs and + ABI (511, static), rustix's inlined raw backend (32) — and each road + leaves a distinct fingerprint in `strace`. +- `strace -c` puts numbers on runtime overhead: 70 (C++) vs 283 (Go, all + threads) vs 73 (Rust) traps for four steps of real work — the same shape + as Chapter 1's 64-vs-217, now fully attributed. + +Next, **errors, three ways**: the `-errno` fact each wrapper just handed us +becomes a policy — `std::expected`, wrapped Go errors, and `thiserror`-style +Rust — that every later example in the book obeys. + +--- + +

verified — all numbers in this chapter are from this session's runs on the Fedora 44 reference host (kernel 7.1.3-200.fc44, glibc 2.43, strace 7.1): the harness reported PASS for cpp/go/rust and each binary printed the five-line block ending sysprobe: 4 steps ok; strace -c totaled 70 (cpp), 283 (go, -cf), and 73 (rust); the vdso-probe run showed 0 clock_gettime and 2 getrandom syscalls for 1000 calls each, including getrandom("…", 32, 0) = 32; the quoted sequence traces (O_TMPFILE openat, 25-byte write, clock_nanosleep vs raw nanosleep, Go's 20 µs-doubling monitor sleeps) are trimmed real output; objdump counted 0/511/32 syscall instructions; and /proc/self/maps showed the quoted [vdso]/[vvar_vclock] mappings. Not run here: the bpftrace kernel-side count (lab-VM callout above).

diff --git a/_docs/05-errors-three-ways.md b/_docs/05-errors-three-ways.md new file mode 100644 index 0000000..6ce9f74 --- /dev/null +++ b/_docs/05-errors-three-ways.md @@ -0,0 +1,475 @@ +--- +title: "Errors, three ways" +order: 5 +part: "Foundations" +description: "One error's journey from a failing write(2) into std::expected, a wrapped Go error, and a Rust Result — copyx establishes the book's error taxonomy, the strerror(3) message contract, and the EINTR retry policy every later chapter reuses." +duration: "45 minutes" +--- + +Chapter 4 walked a syscall down to the boundary and back, and ended on the +detail this chapter is built from: when a syscall fails, the kernel does not +throw, log, or explain — it puts a small negative integer in a register and +returns. Everything your language calls an "error" is constructed, in user +space, from that one number. This chapter takes a single program — `copyx`, a +file copier — and writes it three times so you can watch the *same* kernel +failure become a `std::expected`, a wrapped Go `error`, and a Rust `Result`, +and still come out the other side as byte-identical output and the same exit +code. Along the way it fixes the policies the rest of the book never +renegotiates: what EINTR means, what a short write demands, and which side of +a failure owns the exit code. + +The code is in `examples/05-errors-three-ways/`. `./demo.sh` there builds all +three implementations and runs a walkthrough of the taxonomy; its `README.md` +states the behavior contract each one must satisfy. + +{% include excalidraw.html + file="05-error-flow-three-ways" + alt="One failure's journey: write(2) on /dev/full returns -ENOSPC in the kernel; in user space that becomes errno 28, which fans out into three columns — C++23's std::expected with std::error_code mapping into a Failure struct, Go's unix.Errno wrapped into *opError with Unwrap for errors.Is/As, and Rust's Result with Errno mapped via map_err into a thiserror CopyError variant — all three converging on one outcome box: exit 3 and the line copyx: write /dev/full: No space left on device." + caption="Figure 5.1 — one failure, three spellings, one contract: ENOSPC from write(2) to the user, in each type system" %} + +> **Tools used** — `strace`, `head`, `cmp`, `ls`, `python3` (host); `bpftrace` +> (lab VM, exercised in Part 8). Everything here is checked by +> `scripts/check-host.sh`, ships with Fedora, or is preinstalled in the lab VMs. + +## errno is an ABI, not a variable + +On x86-64 Linux, a failing syscall returns a value between -1 and -4095 in +`rax`; the number is the negated error code. `ENOSPC` is 28 — not "usually +28", but 28 by kernel ABI, stable for decades, the same for every language +and libc on the platform. What differs is the unwrapping. glibc's syscall +stubs check the return, store the positive code into the thread-local +`errno`, and hand C a `-1`. Go's runtime makes the syscall itself and returns +the code as an ordinary value — `unix.Errno` is just `uintptr` with an +`Error()` method; there is no global to race on. Rust's `rustix` does the +same trick: `Errno` is a wrapper around the raw code, delivered in the `Err` +arm of a `Result`, never through ambient state. + +That shared integer is why a cross-language error *policy* is even possible. +Whatever type each language dresses the failure in, the bottom of the stack +is the same number with the same `strerror(3)` text — and `copyx` treats that +text as part of its interface. The behavior contract, identical in all three +implementations: + +| Outcome | Output | Exit | +| --- | --- | --- | +| Success | stdout: `copied bytes` | 0 | +| Bad usage | stderr: `usage: copyx SRC DST` | 2 | +| Source-side failure (open/read SRC) | stderr: `copyx: ` | 2 | +| Destination-side failure (open/write/close DST) | stderr: `copyx: ` | 3 | + +The split is deliberate: a caller (or a shell script) can distinguish "your +input was bad" from "the place you told me to write is broken" without +parsing prose. Exit codes are the oldest structured error channel on Unix, +and `copyx` uses them as one. + +Two syscall-level policies are also part of the contract, because they will +be true of every program in this book. First, **EINTR is retried**. A +`read(2)` or `write(2)` interrupted by a signal before transferring anything +fails with `EINTR`; nothing happened, so the only correct response is to +reissue the call — treating it as an error turns every signal into data +corruption at a distance. Second, **short writes are resumed**. `write(2)` is +allowed to accept fewer bytes than you offered; the loop must advance past +what the kernel took and continue. (The third sibling, `EAGAIN`, means "would +block" on a non-blocking descriptor — not an error either, but a scheduling +fact; it becomes central when epoll arrives in the Files and I/O part.) One +more subtlety the contract encodes: the write side must *observe* +`close(2)`, because deferred I/O errors can surface there; the read side's +close result carries nothing actionable and is released by RAII or `defer`. + +## How the code works + +Each implementation has the same skeleton: a phase-tagged error type, a +retrying `read_some`, a resuming `write_all`, and a `copy_file` that maps +each failure onto the taxonomy. The error types come first, because they are +where the three type systems diverge most visibly. C++ +(`examples/05-errors-three-ways/cpp/src/main.cpp`) uses a plain struct — + +```cpp +struct Failure { + std::string message; // printed after the "copyx: " prefix + int exit_code; +}; +``` + +— because by the time `copy_file` fails, classification is *done*: the +message is rendered and the exit code chosen. Go +(`examples/05-errors-three-ways/go/main.go`) cannot stop there, since Go +errors are expected to stay inspectable: + +```go +type opError struct { + op string // "read", "write", "close"; "" means open(2) + path string + err unix.Errno // reachable via errors.Is / errors.As through Unwrap + exit int +} +``` + +`opError` keeps the raw errno and implements `Unwrap() error`, so +`errors.Is(err, unix.ENOENT)` still answers truthfully through the wrapper. +Rust (`examples/05-errors-three-ways/rust/src/main.rs`) puts the phase into +the *type* itself, one enum variant per failure site, with `thiserror` +generating the `Display` impl from the `#[error("...")]` attributes: + +```rust +#[derive(Debug, Error)] +enum CopyError { + #[error("{path}: {reason}")] + SrcOpen { path: String, reason: String }, + #[error("read {path}: {reason}")] + Read { path: String, reason: String }, + #[error("{path}: {reason}")] + DstOpen { path: String, reason: String }, + #[error("write {path}: {reason}")] + Write { path: String, reason: String }, + #[error("close {path}: {reason}")] + Close { path: String, reason: String }, +} +``` + +Now the core loops. The three excerpts below are the real `write_all` and +`copy_file` from each implementation — read one, then diff it mentally +against the next: + +{% include codetabs.html langs="C++|Go|Rust" %} + +```cpp +// write(2) until the whole span is written: EINTR restarts the call, a short +// write resumes from where the kernel stopped. +[[nodiscard]] std::expected +write_all(const Fd& fd, std::span buf) { + while (!buf.empty()) { + const ssize_t n = ::write(fd.get(), buf.data(), buf.size()); + if (n > 0) { + buf = buf.subspan(static_cast(n)); + continue; + } + if (n < 0 && errno == EINTR) { + continue; // interrupted: reissue the same span + } + return std::unexpected(last_error()); + } + return {}; +} + +struct Failure { + std::string message; // printed after the "copyx: " prefix + int exit_code; +}; + +[[nodiscard]] std::expected +copy_file(const char* src_path, const char* dst_path) { + auto src = open_fd(src_path, O_RDONLY | O_CLOEXEC); + if (!src) { + return std::unexpected( + Failure{std::format("{}: {}", src_path, src.error().message()), 2}); + } + + auto dst = open_fd(dst_path, O_WRONLY | O_CREAT | O_TRUNC | O_CLOEXEC, 0644); + if (!dst) { + return std::unexpected( + Failure{std::format("{}: {}", dst_path, dst.error().message()), 3}); + } + + std::uint64_t total = 0; + std::array buf{}; + for (;;) { + const auto n = read_some(*src, buf); + if (!n) { + return std::unexpected( + Failure{std::format("read {}: {}", src_path, n.error().message()), 2}); + } + if (*n == 0) { + break; // EOF + } + if (auto w = write_all(*dst, std::span{buf}.first(*n)); !w) { + return std::unexpected( + Failure{std::format("write {}: {}", dst_path, w.error().message()), 3}); + } + total += *n; + } + + // The write side must observe close(2): deferred IO errors land here. + if (auto c = dst->close(); !c) { + return std::unexpected( + Failure{std::format("close {}: {}", dst_path, c.error().message()), 3}); + } + return total; +} +``` + +```go +// writeAll pushes the whole buffer through write(2): EINTR restarts the +// call, a short write resumes from where the kernel stopped. +func writeAll(fd int, buf []byte) error { + for len(buf) > 0 { + n, err := unix.Write(fd, buf) + if errors.Is(err, unix.EINTR) { + continue // interrupted: reissue the same span + } + if err != nil { + return err + } + buf = buf[n:] + } + return nil +} + +func copyFile(srcPath, dstPath string) (int64, error) { + src, err := unix.Open(srcPath, unix.O_RDONLY|unix.O_CLOEXEC, 0) + if err != nil { + return 0, newOpError("", srcPath, fmt.Errorf("open %s: %w", srcPath, err), 2) + } + defer unix.Close(src) // read side: nothing actionable in the close result + + dst, err := unix.Open(dstPath, unix.O_WRONLY|unix.O_CREAT|unix.O_TRUNC|unix.O_CLOEXEC, 0o644) + if err != nil { + return 0, newOpError("", dstPath, fmt.Errorf("open %s: %w", dstPath, err), 3) + } + + var total int64 + buf := make([]byte, 64*1024) + for { + n, err := readSome(src, buf) + if err != nil { + unix.Close(dst) + return 0, newOpError("read", srcPath, err, 2) + } + if n == 0 { + break // EOF + } + if err := writeAll(dst, buf[:n]); err != nil { + unix.Close(dst) + return 0, newOpError("write", dstPath, err, 3) + } + total += int64(n) + } + + // The write side must observe close(2): deferred IO errors land here. + if err := unix.Close(dst); err != nil { + return 0, newOpError("close", dstPath, err, 3) + } + return total, nil +} +``` + +```rust +/// write(2) until the whole slice is written: EINTR restarts the call, a +/// short write resumes from where the kernel stopped. +fn write_all(fd: impl AsFd, mut buf: &[u8]) -> Result<(), Errno> { + while !buf.is_empty() { + match write(&fd, buf) { + Ok(n) => buf = &buf[n..], + Err(Errno::INTR) => continue, // interrupted: reissue the same span + Err(e) => return Err(e), + } + } + Ok(()) +} + +/// close(2) with the result observed: on the write side this is where +/// deferred IO errors surface. `into_raw_fd` transfers ownership out of the +/// `OwnedFd`, so the fd is closed exactly once. +fn close_checked(fd: OwnedFd) -> Result<(), Errno> { + unsafe { try_close(fd.into_raw_fd()) } +} + +fn copy_file(src_path: &str, dst_path: &str) -> Result { + let src = open(src_path, OFlags::RDONLY | OFlags::CLOEXEC, Mode::empty()).map_err(|e| { + CopyError::SrcOpen { path: src_path.to_owned(), reason: strerror(e) } + })?; // OwnedFd: dropped (and closed) on every path below + + let dst = open( + dst_path, + OFlags::WRONLY | OFlags::CREATE | OFlags::TRUNC | OFlags::CLOEXEC, + Mode::from_bits_truncate(0o644), + ) + .map_err(|e| CopyError::DstOpen { path: dst_path.to_owned(), reason: strerror(e) })?; + + let mut total: u64 = 0; + let mut buf = vec![0u8; 64 * 1024]; + loop { + let n = read_some(&src, &mut buf).map_err(|e| CopyError::Read { + path: src_path.to_owned(), + reason: strerror(e), + })?; + if n == 0 { + break; // EOF + } + write_all(&dst, &buf[..n]).map_err(|e| CopyError::Write { + path: dst_path.to_owned(), + reason: strerror(e), + })?; + total += n as u64; + } + + close_checked(dst).map_err(|e| CopyError::Close { + path: dst_path.to_owned(), + reason: strerror(e), + })?; + Ok(total) +} +``` + +The loops are structurally identical — a 64 KiB buffer, read until a zero +return means EOF, `write_all` per chunk, then a checked close — but each +language spends its effort differently. **C++** builds the discipline out of +`std::expected`: every fallible helper returns +`std::expected`, `last_error()` snapshots `errno` into +`std::generic_category()` (the category for POSIX errno values; Chapter 1's +`uname` wrapper used `system_category` — for Linux errno they render the +same, and the book standardizes on `generic_category` from here on), and the +move-only RAII `Fd` guarantees no descriptor outlives its owner. The +asymmetry between `dst->close()` — explicit, `[[nodiscard]]`, result mapped +to exit 3 — and `src` being closed silently by the destructor *is* the close +policy from the contract, written in types. **Go** threads one `error` value +through everything: `readSome` detects EINTR with +`errors.Is(err, unix.EINTR)` rather than `err == unix.EINTR`, so the check +survives any future wrapping; `newOpError` recovers the raw `unix.Errno` +from whatever it is handed using `errors.As`, and `main` gets the exit code +back the same way. Note also what Go must do that the others get free: its +errno strings are lowercase ("no such file or directory"), so `strerror` +capitalizes the first rune to match glibc. **Rust** lets `?` do the +propagation and `map_err` do the classification — each call site names its +variant, so forgetting to classify a failure is a compile error, and +`exit_code()` is an exhaustive `match` the compiler re-checks whenever a +variant is added. Its `strerror` has the inverse problem to Go's: `Errno` +displays through `std::io::Error`, which appends " (os error N)", so the +suffix is stripped to keep the three outputs byte-identical. + +### Errors, three ways + +`copyx` is the policy statement for the thread that runs through every +chapter, so state the doctrine plainly. In **C++**, errors that are part of +normal operation — a missing file, a full disk — travel as values +(`std::expected`), and exceptions are reserved for *subsystem boundaries*: +the places where a whole component (a parser, a plugin, a request handler) +fails as a unit and someone several frames up owns the recovery. Inside +syscall-adjacent code, exceptions obscure control flow exactly where you need +to see it. In **Go**, the equivalent line is drawn between `error` and +`panic`: an `error` is a result, a `panic` is a bug (index out of range, nil +map write) — `copyx` returns errors for everything the kernel can say and +would panic only on programmer mistakes. In **Rust**, the same line separates +`Result` from `panic!`; the book's examples are written to be correct under +`panic = "abort"`, which means a panic is never part of the error path — it +is a crash you fix. On the library-versus-application split: `thiserror` +(used here) is for code whose *callers* match on error variants; `anyhow` is +for binaries that only report. `copyx` is a binary, but its exit codes make +error *identity* part of the interface, which is exactly the case where the +typed enum earns its boilerplate. + +### Concurrency lens + +`copyx` is deliberately single-threaded — one loop, one buffer, no shared +state — but the error machinery is already concurrency-load-bearing. C's +`errno` is thread-local precisely so two threads failing simultaneously do +not corrupt each other's error state, and `last_error()` copies it into a +value *immediately* because the very next libc call on the same thread may +overwrite it. Go and Rust sidestep the global entirely: the errno arrives as +a return value, so there is nothing thread-local to protect. And the EINTR +policy is really a concurrency policy in disguise — signals are asynchronous +interruptions, and a program whose I/O loops retry EINTR is one whose +correctness does not depend on when a signal lands. That property gets +load-tested in the Processes, Signals, Privilege part, when `pmon` starts +handling real signals around these same loops. + +## Build, run, observe + +```bash +[host]$ cd examples/05-errors-three-ways && ./demo.sh +``` + +The walkthrough runs the taxonomy per language. Reproduce it by hand against +any of the three binaries — shown here with a 200 000-byte source (sized so +the final read comes up short of the 64 KiB buffer): + +```bash +[host]$ head -c 200000 /dev/urandom > src.bin +[host]$ ./cpp/build/release/app src.bin dst.bin; echo exit=$? +copied 200000 bytes +exit=0 +[host]$ ./cpp/build/release/app nope.bin out.bin; echo exit=$? +copyx: nope.bin: No such file or directory +exit=2 +[host]$ ./cpp/build/release/app src.bin /dev/full; echo exit=$? +copyx: write /dev/full: No space left on device +exit=3 +``` + +Then run the same three cases with `./go/bin/app` and +`./rust/target/release/app`: on this host all three produced the identical +three lines and the identical exit codes 0, 2, 3, and `cmp src.bin dst.bin` +exited 0 for each copy — byte-identical output from byte-identical input, +which is the whole claim of the chapter. `/dev/full` is the tool that makes +the ENOSPC case repeatable: `ls -li /dev/full` shows a character device with +major:minor `1, 7`, a kernel-provided device whose write handler always +returns `-ENOSPC` — a full disk on tap, no root required. The book's runner +(`python3 scripts/test-all-examples.py --only 05-errors-three-ways`) drives +eleven behavioral checks per language through `verify.lua` — including the +empty-source edge case (`copied 0 bytes`, exit 0) — and reported +`PASS PASS PASS` on this host. + +## Cross-check: watch the error being born + +Everything above shows the error *after* each runtime dressed it up. +`strace -e trace=write` shows the moment it exists — the failing `write(2)` +itself. On the C++ binary: + +```bash +[host]$ strace -e trace=write ./cpp/build/release/app src.bin /dev/full +write(4, "\343\275\2436'\325\377\3074N0\262\2248O\303U}\267R\327|\241?1\205M\265\303\320\f\2"..., 65536) = -1 ENOSPC (No space left on device) +write(2, "copyx: write /dev/full: No space"..., 47copyx: write /dev/full: No space left on device) = 47 +write(2, "\n", 1 +) = 1 ++++ exited with 3 +++ +``` + +Read it bottom-up from the failure: one 65 536-byte `write` to descriptor 4 +(the destination; 3 is the source) fails with `ENOSPC`, and the *next* writes +are to descriptor 2 — the program printing its own error message. The entire +journey in Figure 5.1 happened between those two lines, in user space, +invisible to the kernel. The Rust binary (`strace -e trace=write +./rust/target/release/app src.bin /dev/full`) fails identically but emits the +message as six separate little writes (`"copyx: "`, `"write "`, +`"/dev/full"`, …) — `eprintln!` writing each format fragment straight to +unbuffered stderr. The Go binary (add `-f`; the runtime has threads) shows +the failing 65 536-byte write on descriptor **5**, not 4 — this trace shows +why: the runtime is already holding +`/sys/fs/cgroup/.../cpu.max` open on descriptor 3 for its container-aware +scheduling, so the source and destination land one higher. Three runtimes, +three descriptor layouts and stderr habits — and one identical failing +syscall, one identical message, one identical `+++ exited with 3 +++`. + +> **On the lab VM** — unverified: +> the aggregate view of this chapter — *which* errnos your whole system's +> syscalls are failing with, live — belongs to eBPF tooling in the class of +> bpftrace one-liners counting negative syscall returns by errno. That is not +> runnable on this host without privileges; Chapter 30 (Debugging part) +> exercises it in the `systems-target` VM against these same binaries. + +## What you learned + +- Error codes are kernel ABI: a failing syscall returns `-errno` in a + register, `ENOSPC` is 28 everywhere on the platform, and glibc's + thread-local `errno`, Go's `unix.Errno` value, and rustix's `Errno` are + three deliveries of the same integer. +- `copyx`'s taxonomy — exit 2 for the source side, 3 for the destination + side, reasons in `strerror(3)` form — is expressible in all three type + systems: `std::expected` mapped into a `Failure`, an `*opError` with + `Unwrap` keeping `errors.Is`/`As` truthful, and a `thiserror` enum whose + `exit_code()` the compiler keeps exhaustive. +- The book's retry policy: EINTR means nothing transferred, so reissue; + short writes resume from where the kernel stopped; the write side observes + `close(2)`; EAGAIN is a scheduling fact deferred to the epoll chapters. +- Exceptions (C++) and panics (Go, Rust) are for bugs and subsystem + boundaries, never for outcomes the kernel can report; and `strace` can + catch the exact syscall where an error is born, before any runtime + touches it. + +Next, the second thread that never stops: **concurrency, three ways** — the +same worker pool as `std::jthread`s and atomics, goroutines and channels, +and scoped threads under `Send`/`Sync`. + +--- + +

verified — all evidence in this chapter was produced on the Fedora 44 host this session: each of the three binaries printed copied 200000 bytes (exit 0) with cmp exiting 0, copyx: nope.bin: No such file or directory (exit 2), and copyx: write /dev/full: No space left on device (exit 3); the strace excerpt is real output showing write(4, …, 65536) = -1 ENOSPC before the exit-3; the Go trace showed the failing write on descriptor 5 with cpu.max held on 3, and the Rust trace showed the fragmented stderr writes; ls -li /dev/full showed the 1, 7 character device; and scripts/test-all-examples.py --only 05-errors-three-ways reported PASS for cpp, go, and rust (3 passed, 0 failed, 0 skipped). The lab-VM eBPF callout above is the one unverified item, deferred to Chapter 30.

diff --git a/_docs/06-concurrency-three-ways.md b/_docs/06-concurrency-three-ways.md new file mode 100644 index 0000000..113d39a --- /dev/null +++ b/_docs/06-concurrency-three-ways.md @@ -0,0 +1,420 @@ +--- +title: "Concurrency, three ways" +order: 6 +part: "Foundations" +description: "The second thread that runs the whole book — how each language shares work over the same kernel threads. C++23 jthreads and stop tokens, Go's runtime and channels, Rust's compiler-refereed sharing — one parallel checksummer, byte-identical output, and a SIGINT drain you can watch happen." +duration: 50 minutes +--- + +Chapter 5 pinned down the first of this book's two permanent threads: how each +language turns an `errno` into something a caller can act on. This chapter +pins down the second — how each language *shares work* — and it does it the +only way that is fair: one program, `parhash`, written three times with each +language's native concurrency toolkit, required to produce **byte-identical +stdout**. Underneath, all three are the same thing — kernel threads created +with `clone(2)`, parked and woken with `futex(2)` — so what you are really +comparing is vocabulary: what C++ hands you as parts, what Go's runtime takes +away from you, and what Rust's compiler refuses to let you get wrong. One +scope note up front: everything here is *thread* concurrency. Async — C++ +coroutines, tokio, and what the Go scheduler looks like from below — is +deliberately deferred to Chapter 27, after you have seen the event loops that +motivate it. + +The code is in `examples/06-concurrency-three-ways/`. `./demo.sh` builds and +runs all three implementations; its `README.md` covers the shared contract and +how `verify.lua` checks it. + +{% include excalidraw.html + file="06-three-concurrency-models" + alt="Three columns — C++23, Go, Rust — each showing the same producer, queue, four-worker, cancellation shape: main thread into a mutex/deque/condition_variable_any queue into four jthreads with a sigtimedwait watcher; walker goroutine into an unbuffered channel into four goroutines with signal.NotifyContext; main thread into an Arc-Mutex-wrapped mpsc receiver into four scoped threads with a signal-hook AtomicBool." + caption="Figure 6.1 — one program shape, three vocabularies; same kernel threads underneath." %} + +> **Tools used** — `cmp`, `truncate`, `python3` (host); bcc-tools +> (`offcputime`, `exitsnoop`) (lab VM, exercised in Part 8). Everything here is +> checked by `scripts/check-host.sh`, ships with Fedora, or is preinstalled in +> the lab VMs. + +## Three models over one primitive + +`parhash DIR` walks every regular file under `DIR` (symlinks are never +followed), streams each through an inline FNV-1a 64 hash with a pool of +exactly **4 workers**, then prints one `<16-hex> ` line per file, +sorted, plus a summary. The fixed worker count is part of the contract — the +summary line says `4 workers` in all three languages — and the sort at the end +is what makes byte-identical output possible at all, because the three +languages do not even *walk* the tree in the same order, let alone finish +hashing in one. + +**C++23** gives you parts, not a machine. `std::jthread` improves on +`std::thread` in two ways that this example leans on: it joins in its +destructor (a `std::thread` you forget to join calls `std::terminate`), and it +carries a `std::stop_token` — a cooperative cancellation flag that +`condition_variable_any::wait` understands natively. What the standard does +*not* give you is everything else: there is no work queue, no channel, no +select, and — critically for this chapter — no signal story. Signals are POSIX +territory, and mixing them with threads requires a discipline the standard +never mentions: block the signal in every thread, then let exactly one thread +consume it synchronously. The C++ source is real work: about 230 lines, of +which maybe 60 are concurrency plumbing you had to design yourself. + +**Go** took the opposite bet: the runtime owns the threads and you never see +them. In the GMP scheduler, *goroutines* (G) — cheap, growable-stack tasks — +are multiplexed by the runtime onto a small set of kernel threads (M), each of +which needs a *processor* slot (P, capped by `GOMAXPROCS`, 16 on this host) to +run Go code; when a goroutine blocks in a syscall its M can hand the P to +another thread, and idle Ps steal runnable Gs from busy ones. That is the +entire mental model this book needs until Chapter 27. What you program against +is not the scheduler but the idiom on top: goroutines communicate over +channels, `select` races multiple channel operations, and cancellation is a +`context.Context` whose `Done()` channel closes — one convention so universal +that the standard library will hand you a context pre-wired to SIGINT. + +**Rust** keeps the threads visible — `std::thread::scope` spawns real kernel +threads, one each — and moves the refereeing into the compiler. `Send` and +`Sync` are marker traits computed from a type's contents: `Send` means a value +may move to another thread, `Sync` means `&T` may be shared across threads. +The race you would have written in C++ — four workers pushing into one shared +`Vec` because you forgot the mutex — simply does not compile in Rust: `&mut +Vec` cannot be aliased across threads, and the compiler says so at the exact +line you tried. This example hits a subtler case of the same rule: +`mpsc::Receiver` is not `Sync` — sharing one receiver among four workers is +only legal behind a lock — so the type `Arc>>` in +the source is not a style choice; it is the shortest thing the compiler would +accept. Scoped threads add the ergonomic half: because `scope` guarantees +every spawned thread joins before the scope returns, workers may borrow plain +stack data (the root path, the interrupt flag) without `'static` bounds or +reference counting. + +## How the code works + +All three sources — `examples/06-concurrency-three-ways/cpp/src/main.cpp`, +`go/main.go`, and `rust/src/main.rs` — share one shape: a producer walks the +tree and feeds relative paths into a queue; four workers pull paths, hash +files, and emit `(relpath, hash)` results; main collects, sorts by path, and +prints. The interesting 30 lines of each are the worker/queue mechanics: + +{% include codetabs.html langs="C++|Go|Rust" %} + +```cpp +// Mutex-guarded FIFO. pop() blocks with a stop_token-aware wait, so a +// request_stop() wakes idle workers immediately and makes them refuse +// queued-but-unstarted items; a file a worker is mid-hash on always finishes. +class TaskQueue { +public: + void push(Task task) { + { + std::lock_guard lock(mutex_); + queue_.push_back(std::move(task)); + } + cv_.notify_one(); + } + + void close() { + { + std::lock_guard lock(mutex_); + closed_ = true; + } + cv_.notify_all(); + } + + [[nodiscard]] std::optional pop(std::stop_token st) { + std::unique_lock lock(mutex_); + cv_.wait(lock, st, [this] { return closed_ || !queue_.empty(); }); + if (st.stop_requested() || queue_.empty()) { + return std::nullopt; + } + Task task = std::move(queue_.front()); + queue_.pop_front(); + return task; + } + +private: + std::mutex mutex_; + std::condition_variable_any cv_; + std::deque queue_; + bool closed_ = false; +}; +``` + +```go +func worker(ctx context.Context, root string, paths <-chan string, results chan<- result) { + for { + // Done first: once cancelled, refuse queued-but-unstarted work. + select { + case <-ctx.Done(): + return + default: + } + select { + case <-ctx.Done(): + return + case rel, ok := <-paths: + if !ok { + return + } + sum, err := hashFile(filepath.Join(root, rel)) + if err != nil { + fmt.Fprintf(os.Stderr, "parhash: skipping %s\n", rel) + continue + } + results <- result{rel: rel, sum: sum} + } + } +} +``` + +```rust + std::thread::scope(|scope| { + for _ in 0..WORKERS { + let path_rx = Arc::clone(&path_rx); + let result_tx = result_tx.clone(); + let interrupted = Arc::clone(&interrupted); + let root = &root; + scope.spawn(move || { + loop { + if interrupted.load(Ordering::Relaxed) { + return; // refuse queued-but-unstarted work + } + // Lock only around recv so dequeues serialize but hashing + // runs in parallel across all four workers. + let Ok(rel) = path_rx.lock().expect("receiver lock").recv() else { + return; // walker done and queue drained + }; + match hash_file(&root.join(&rel)) { + Ok(sum) => { + let _ = result_tx.send((rel, sum)); + } + Err(_) => eprintln!("parhash: skipping {rel}"), + } + } + }); + } + drop(result_tx); // main keeps no sender: result_rx ends when workers do + + walk(&root, Path::new(""), &interrupted, &path_tx); + drop(path_tx); // walker done: idle workers see the channel close + + for entry in result_rx { + results.push(entry); // drains until every in-flight hash lands + } + }); +``` + +**C++ — the queue you had to build.** `TaskQueue` is the channel C++ does not +ship: a `std::deque` guarded by a mutex, with a `condition_variable_any` for +blocking pops. The load-bearing call is the three-argument +`cv_.wait(lock, st, pred)`: it sleeps until the predicate holds *or the stop +token fires*, which is exactly why the type is `condition_variable_any` and +not plain `condition_variable` — only the `_any` variant accepts a +`stop_token`. Without it, a `request_stop()` could not wake an idle worker, +and Ctrl-C would hang until the next `push`. On waking, `pop` deliberately +checks `st.stop_requested()` *before* looking at the queue: a cancelled worker +refuses work that is queued but unstarted, while a file it is already mid-hash +on always finishes, because cancellation is only ever observed between pops. +`close()` is the producer's end-of-input signal — `notify_all`, because every +idle worker must wake to see `closed_` and exit. The cancellation source is a +watcher `jthread`: `main` blocks SIGINT process-wide with `pthread_sigmask` +*before creating any thread* (the mask is inherited, so no thread can be +interrupted asynchronously), and the watcher loops on `sigtimedwait` with a +20 ms timeout, consuming the signal synchronously and calling +`cancel.request_stop()` — no async-signal-safety puzzle, no handler, ever. + +**Go — the idiom in eleven lines of select.** The worker's first `select` has +a `default` arm, making it a non-blocking poll of `ctx.Done()`; the second +races `Done()` against the next path. Both are needed: Go's `select` picks +*pseudorandomly* among ready cases, so if paths keep arriving after +cancellation, the second select alone could keep choosing work forever. The +poll first guarantees a cancelled worker takes no new task. `rel, ok := +<-paths` with `ok == false` is Go's equivalent of `close()` above — the walker +`close(paths)`s in a `defer`, and every worker unblocks. The wiring in `main` +is the part worth memorizing: `signal.NotifyContext(context.Background(), +os.Interrupt)` gives a context that SIGINT cancels; a `sync.WaitGroup` +counts the four workers; and a fifth goroutine does `wg.Wait(); close(results)` +so that main's `for r := range results` terminates exactly when every +in-flight hash has landed. That closer goroutine *is* the drain. + +**Rust — sharing only what the types allow.** The four `scope.spawn` closures +borrow `root` directly (`let root = &root;`) — legal only because scoped +threads provably join before `main`'s stack frame dies. Each worker locks the +shared receiver *only around* `recv()`, so dequeues serialize but hashing runs +four-wide; hold the lock across `hash_file` and you would have built a +one-worker pool with extra steps. The two `drop` calls are doing what `close()` +and the closer goroutine did in the other languages: dropping `path_tx` after +the walk makes every `recv()` return `Err`, which is how idle workers learn +there is no more work; dropping main's `result_tx` clone means `result_rx`'s +iterator ends precisely when the last worker (holding the last sender) exits. +Cancellation is the bluntest of the three: `signal_hook::flag::register` +points SIGINT at a shared `AtomicBool`, and walker and workers poll it with +`Ordering::Relaxed` — a plain flag needs no ordering guarantees, a point +Chapter 26 returns to properly. + +### Errors, three ways + +The per-file hash function carries this book's Chapter 5 policy into +concurrent code. C++'s `hash_file` returns +`std::expected` and is the only one of the +three that must handle `EINTR` by hand — its raw `::read` loop retries on +`errno == EINTR`, while Go's `os.File.Read` and Rust's `File::read` retry +inside the runtime. Go wraps with `%w` (`fmt.Errorf("open %s: %w", path, +err)`) so callers can still `errors.Is` through the context; Rust does the +same with `anyhow::Context`. The *policy* is identical everywhere: an +unreadable file is not fatal — `parhash: skipping ` on stderr, keep +hashing; a missing argument is `usage: parhash DIR` and exit 2; and an +interrupt is exit **130**, the shell convention for 128 + SIGINT's number 2. +Exit codes are part of the error surface, and `verify.lua` checks all three. + +### Concurrency lens + +This chapter *is* the lens the rest of the book looks through, so here is the +first reading. The C++ model buys maximum control — you chose the queue, the +wake policy, the signal discipline — at the price of designing all of it. The +Go model is the least code by far and the only one where a runtime stands +between you and the kernel. The Rust model costs you a negotiation with the +compiler up front and repays it by making the forgotten-mutex bug +unrepresentable. From here on, each example states which model it leans on and +why; when the answer becomes "none of these — the problem wants an event +loop," that is Chapter 27's cue. + +## Build, run, observe + +```bash +[host]$ cd examples/06-concurrency-three-ways && ./demo.sh build +[host]$ ./demo.sh cpp run /some/tree +``` + +The Go build is special in one visible way: `go/demo.sh` compiles with +`go build -race`. This is the concurrency chapter, so the race detector +referees every Go run, including all of CI's verification runs. On the book's +five-file fixture (two text files, a nested pair, an empty file), every +language prints exactly: + +``` +bbd23ea491ed9813 a.txt +4d038ef62a7d9c0b b.txt +f839d20567c0a911 sub/c.txt +51d88627df287325 sub/deep/d.bin +cbf29ce484222325 zz-empty.dat +parhash: 5 files, 4 workers +``` + +The empty file's hash is `cbf29ce484222325` — FNV-1a's offset basis, i.e. the +hash of zero bytes — which makes it a nice canary: if that line differs, the +hash loop itself is wrong, not the concurrency. On this host the full runner +pass (`python3 scripts/test-all-examples.py --only 06-concurrency-three-ways`) +reports PASS for all three languages, 8 checks each, Go under `-race`. + +## Cross-check: cmp, /proc, and a live Ctrl-C + +The contract says the three binaries are interchangeable. `cmp` — which +compares byte streams and says nothing only when they are identical — is the +right referee. On the fixture above: + +```bash +[host]$ ./cpp/build/release/app "$FIX" > out-cpp.txt +[host]$ ./go/bin/app "$FIX" > out-go.txt +[host]$ ./rust/target/release/app "$FIX" > out-rust.txt +[host]$ cmp out-cpp.txt out-go.txt && echo identical +identical +[host]$ cmp out-cpp.txt out-rust.txt && echo identical +identical +``` + +Next, the claim that these are all just kernel threads. Give each binary a +tree big enough to keep it busy — `verify.lua`'s trick is 16 small files plus +96 files of 64 MiB created with `truncate`, ~6.1 GiB logical, 64 KiB on disk, +so the workers hash streams of zeros for seconds while your SSD does nothing — +and count entries in `/proc//task`, one per thread, while it runs. On +this host: **C++ 6** (main, four workers, the sigtimedwait watcher), **Rust +5** (main plus four scoped threads — no watcher, since a flag needs none), and +**Go 10** — the runtime decides its own thread count, and under `-race` on a +16-CPU host it settled on ten kernel threads for a program whose source names +only six concurrent activities. Ask *what the threads are doing* and `/proc` +answers that too: + +```bash +[host]$ for t in /proc/$pid/task/*; do echo "$(basename $t) $(cat $t/wchan)"; done +2558332 futex_do_wait +2558334 do_sigtimedwait.isra.0 +2558335 0 +2558336 0 +2558337 0 +2558338 0 +``` + +That is the C++ binary mid-run: the main thread parked in `futex_do_wait` +(the mutex/CV machinery is futexes underneath), the watcher visibly sitting in +`do_sigtimedwait`, and four workers with `wchan` 0 — running, hashing. + +Last, the drain — the behavior Figure 6.2 sequences and the one a validator +can only trust if it was watched happening. Start each binary on the sparse +tree, wait 400 ms, send SIGINT, and collect streams and exit codes. All three +exited **130** with `parhash: interrupted` on stderr, and each printed **20** +completed, well-formed hash lines with no summary line. Twenty is not a +coincidence to wave past: roughly 16 files were fully hashed when the signal +landed, and the **4 in-flight 64 MiB files — one per worker — all completed +after SIGINT**, exactly the finish-what-you-hold guarantee all three +implementations promise. The partial *sets* differ, and that is instructive: +Go's partial output starts `56750b99cb963a67 aa-01.txt` because +`filepath.WalkDir` sorts directory entries, so the small files went first; +C++ and Rust use raw `readdir` order and their partials were twenty +`805f256ad4222325 big-NN.bin` lines (64 MiB of zeros always hashes the same). +Identical *final* output despite different walk orders is precisely why the +program sorts at the end instead of trusting the walk. + +{% include excalidraw.html + file="06-cancellation-drain" + alt="SIGINT fans out as one flipped flag to three parties — the walker stops enumerating, idle workers wake and exit, busy workers finish the file in hand — then the queue closes and results drain into a final box: partial sorted lines, parhash: interrupted, exit 130." + caption="Figure 6.2 — the drain: stop accepting, finish what you hold, report what completed." %} + +> **On the lab VM** — unverified: +> bcc-tools would let you watch the drain from the kernel's side — +> `offcputime` showing the idle workers parked in futex waits, `exitsnoop` +> catching the thread exits as they cascade. Those tools are not runnable on +> this host without root; Chapter 30 (Debugging part) builds that toolkit on +> the `systems-target` VM and points it at these exact binaries. + +One asymmetry to name plainly: Go ran every check under its race detector, +but the C++ pool ran with no equivalent referee — this example's CMake presets +stop at `asan`, and no ThreadSanitizer build was exercised here. +unverified — the host carries +the `libtsan` runtime (`scripts/check-host.sh` checks for it) and a TSan probe +links and runs, but running *this* worker pool under TSan, and what its +reports look like when the mutex is deliberately removed, is Chapter 29's +job. + +## What you learned + +- All three models are `clone(2)` + `futex(2)` underneath — `/proc//task` + showed 6, 10, and 5 kernel threads for the same four-worker program, and + `wchan` showed who was parked where. +- C++23 gives parts (`jthread`, `stop_token`, `condition_variable_any`), and + you assemble the queue and the signal discipline — block SIGINT everywhere, + consume it in one thread via `sigtimedwait`, `request_stop()` the rest. +- Go's cancellation idiom is structural: `signal.NotifyContext`, workers that + `select` on `Done()` before work, and a `WaitGroup`-driven channel close + that *is* the drain. +- Rust turns sharing mistakes into type errors — `mpsc::Receiver` is not + `Sync`, so the compiler forces `Arc>`; scoped threads let workers + borrow stack data because the join is guaranteed. +- Cancellation-as-drain is a contract you can verify: stop accepting, finish + what you hold, report what completed, exit 130 — and all three languages + honored it live. + +Next, Part 2 begins with **file descriptors and the VFS** — the first chapter +of `fwatch`, where the question stops being "who runs the code" and becomes +"what exactly is that small integer the kernel keeps handing us." + +--- + +

verified — every number above +comes from runs on the Fedora 44 reference host (kernel 7.1.3-200.fc44) this +session: the runner reported 06-concurrency-three-ways PASS PASS +PASS (3 passed, 0 failed, Go built with -race); the +fixture outputs of the three binaries were cmp-identical, +including cbf29ce484222325 zz-empty.dat; /proc/<pid>/task +held 6 (C++), 10 (Go), and 5 (Rust) entries mid-run on the ~6.1 GiB sparse +tree, with the quoted wchan values; and SIGINT at ~400 ms +produced exit 130, parhash: interrupted on stderr, and 20 +completed hash lines (the 4 in-flight 64 MiB files finishing after the +signal) in each language. The bcc-tools observation and the TSan run remain +unverified as marked above.

diff --git a/_docs/07-fds-raii-and-the-vfs.md b/_docs/07-fds-raii-and-the-vfs.md new file mode 100644 index 0000000..4d51357 --- /dev/null +++ b/_docs/07-fds-raii-and-the-vfs.md @@ -0,0 +1,549 @@ +--- +title: "fds, RAII, and the VFS" +order: 7 +part: "Files and I/O" +description: "The three-layer model behind every file descriptor — fd table, open file description, inode — proven with /proc and strace, plus openat/fstatat directory-relative walking and an owning fd wrapper in each language, as fwatch v0 takes its first snapshot." +duration: 45 minutes +--- + +This chapter starts the second of the book's long-running artifacts: `fwatch`, +the file watcher that will grow inotify, epoll, io_uring, and finally a +Landlock sandbox over the next several chapters. Version 0 has no events at +all — it is a deterministic snapshot/diff tool — because before you can watch +files change you need to be precise about what a file descriptor actually +*is*. The one new idea here is that an fd is not a file: it is the top of a +three-layer chain — a slot in your process's fd table, pointing at a +system-wide *open file description*, pointing at an *inode* — and every flag, +offset, and ownership question in this part of the book lands on exactly one +of those three layers. Along the way we build the owning fd wrapper each +language uses for the rest of the book, and we walk directory trees the safe +way: relative to a directory fd, never by re-resolving paths. + +The code is in `examples/07-fds-raii-and-the-vfs/`. `./demo.sh` there builds +all three implementations and runs a self-contained snapshot → mutate → diff +demo; its `README.md` specifies the CLI, the snapshot format, and the exit +codes all three languages share. + +{% include excalidraw.html + file="07-fd-table-vfs" + alt="Three stacked layers: the fwatch process fd table holding fds 0-2 and four dirfds 3-6 with cloexec on, each dirfd pointing down to its own kernel open file description holding flags 02700000 and a readdir position, each description pointing down to a VFS inode numbered 69926 through 69929; dashed openat arrows chain fd 3 to fd 4 to fd 5 to fd 6." + caption="Figure 7.1 — one paused fwatch walk: four dirfds in the per-process fd table, four open file descriptions carrying flags and offset, four inodes; every number in this figure comes from the /proc inspection later in the chapter" %} + +> **Tools used** — `strace`, `stat`, `ls`, `lsof`, `touch`, `cmp`, `python3` +> (host); `opensnoop`/`bpftrace` (lab VM, exercised in Part 8). Everything here +> is checked by `scripts/check-host.sh`, ships with Fedora, or is preinstalled +> in the lab VMs. + +## One fd, three tables + +When `openat(2)` returns `3`, three separate kernel structures were involved, +and conflating them is the root of most fd bugs: + +1. **The fd table** is per-process: a plain array indexed by small integers, + which is why fds are always the lowest free number. Each slot holds two + things only — a pointer to an open file description, and the + **close-on-exec bit**. That placement matters: `O_CLOEXEC` is a property + of *your process's slot*, not of the underlying open file. +2. **The open file description** is system-wide: created by each `open`, + shared by `dup(2)` and inherited across `fork(2)`. It owns the status + flags (`O_RDONLY`, `O_NOFOLLOW`, …) and the file offset. Two processes + that inherited the same description share one offset — the classic + surprise where a parent and child interleave writes; two independent + `open`s of the same path get two descriptions and two offsets. +3. **The inode** is the VFS object for the file itself — type, size, mtime, + and the number `ls -li` prints. Many descriptions can point at one inode; + an inode with zero directory entries *and* zero descriptions is when the + data actually goes away (which is why you can unlink a running binary). + +`fwatch snapshot` exercises the whole chain: it opens directories (fd table +entries), enumerates them through their descriptions' readdir offsets, and +records size and mtime — inode fields — for every regular file. The +` ` lines it prints, sorted +bytewise by path, are inode metadata reached through fd-relative syscalls +without ever re-resolving a full path. + +## Open flags, and `O_CLOEXEC` everywhere + +Every directory open in all three implementations uses the same four flags — +this is the C++ spelling, from +`examples/07-fds-raii-and-the-vfs/cpp/src/main.cpp`: + +```cpp +constexpr int kDirFlags = O_RDONLY | O_DIRECTORY | O_CLOEXEC | O_NOFOLLOW; +``` + +`O_DIRECTORY` makes the open fail with `ENOTDIR` if the name stopped being a +directory between `readdir` and `openat` — classification races are real, and +we would rather get an error than enumerate a file. `O_NOFOLLOW` refuses to +traverse a symlink in the final component, so a walk of a tree you do not +fully control cannot be steered out of that tree by swapping a subdirectory +for a symlink to `/etc`. + +`O_CLOEXEC` deserves its own paragraph because the book will put it on +*every* fd it ever opens. Without it, every fd you hold is inherited by every +child process you — or any library you link — ever `fork`/`exec`s. That is +how a watcher ends up keeping a deleted directory alive because some spawned +helper still holds fd 5, or how a privileged fd leaks into an unprivileged +child. The tempting fix, "open, then set `FD_CLOEXEC` with `fcntl`", has a +hole: in a threaded program another thread can `fork`+`exec` in the window +between the two calls, and the fd escapes anyway. `O_CLOEXEC` closes that +window by setting the bit *atomically at open time* — which is why it is +baked into `kDirFlags` rather than applied afterwards. Note which layer it +lives on: the per-process fd table slot. A `dup` of the fd starts with the +bit clear; the open file description neither knows nor cares. + +## `openat`, `fstatat`, and walking without paths + +The naive way to walk a tree is string arithmetic: `readdir`, concatenate +`dir + "/" + name`, `stat` the result, recurse. Every one of those `stat`s +re-resolves the whole path from `/`, so every step is a fresh race: if any +ancestor component is renamed — or replaced by a symlink — between two calls, +you are suddenly operating on a different file than the one `readdir` told +you about. The `*at` syscall family fixes this by making the *directory fd* +the anchor: `openat(dirfd, name, …)` and `fstatat(dirfd, name, …, +AT_SYMLINK_NOFOLLOW)` resolve exactly one component, relative to a directory +you already hold open. Once `fwatch` holds fd 3 on the root of the tree, +renaming that tree out from under it changes nothing — the fd pins the inode, +not the name. `AT_SYMLINK_NOFOLLOW` completes the discipline for the stat +side: classify the entry itself, never what a symlink points at. (`AT_FDCWD`, +which `scan` passes for the very first open, is the pseudo-dirfd meaning +"resolve relative to the current working directory" — the only path-based +resolution in the whole program.) + +This is not only about races. Anchored walks also survive path-length limits, +cross into the capability style of `openat2(2)` and `RESOLVE_BENEATH` that +the Landlock chapter will want, and are *faster* — one component resolved per +call instead of the whole prefix every time. + +## How the code works + +Two data structures carry the whole program. A `Tree` maps relative path → +`(size, mtime)`: C++ uses `std::map` and Rust +`BTreeMap` because both iterate in sorted key order, which +makes snapshot output deterministic for free; Go uses a plain +`map[string]info` and sorts the keys explicitly at print time — same +observable behavior, sorting moved to the edge. `Info` is two `int64`s; two +snapshots differ exactly when a key set or an `Info` differs, which is all +`diff` needs. + +The second structure is the owning fd wrapper, and it is the heart of the +chapter. The C++ one, verbatim: + +```cpp +// Owns a file descriptor; closes it exactly once. +class Fd { +public: + Fd() = default; + explicit Fd(int fd) noexcept : fd_(fd) {} + ~Fd() { reset(); } + + Fd(const Fd&) = delete; + Fd& operator=(const Fd&) = delete; + + Fd(Fd&& other) noexcept : fd_(std::exchange(other.fd_, -1)) {} + Fd& operator=(Fd&& other) noexcept { + if (this != &other) { + reset(); + fd_ = std::exchange(other.fd_, -1); + } + return *this; + } + + [[nodiscard]] int get() const noexcept { return fd_; } + [[nodiscard]] int release() noexcept { return std::exchange(fd_, -1); } + explicit operator bool() const noexcept { return fd_ >= 0; } + +private: + void reset() noexcept { + if (fd_ >= 0) { + ::close(fd_); + fd_ = -1; + } + } + int fd_ = -1; +}; +``` + +Copying is deleted because two owners means two `close`s — a double close is +worse than a leak, since between the two calls the number may already name +someone else's freshly opened file. Moving transfers ownership by +`std::exchange`-ing the source to `-1`, so the moved-from destructor is a +no-op. `release()` is the escape hatch for APIs that *take* ownership, and +the C++ code needs it immediately: `fdopendir(3)` adopts the fd into a +`DIR*`, after which `closedir(3)` is the one and only cleanup. The +`DirStream` class documents that transfer in code — `DirStream::adopt(Fd)` +calls `fd.release()` only after `fdopendir` succeeds, so on the failure path +the `Fd` destructor still closes it. Exactly one owner on every path, +including errors. + +Rust gets the same semantics from the standard library: `rustix::fs::open` +and `openat` return `std::os::fd::OwnedFd`, whose `Drop` is the close and +which is move-only by construction — the borrow checker rejects what C++ +merely deletes. Functions that need the fd without owning it take it as +`BorrowedFd` (via `AsFd`, which is why `walk` can pass `&fd` to `statat` and +`openat` and still drop `fd` at the end). Go has no destructors, so the walk +wraps each raw dirfd in `os.NewFile` and `defer f.Close()` owns it — one +close per directory, on every return path. + +The walk itself is the same function three times: + +{% include codetabs.html langs="C++|Go|Rust" %} + +```cpp +// Walk the directory opened as `fd`, recording every regular file into `tree`. +// `display` is the path for error messages, `rel` the DIR-relative prefix. +[[nodiscard]] std::expected walk(Fd fd, const std::string& display, + const std::string& rel, Tree& tree) { + auto stream = DirStream::adopt(std::move(fd)); + if (!stream) { + return std::unexpected(fail("open directory", display, stream.error())); + } + + while (const dirent* entry = ::readdir(stream->get())) { + const std::string name = entry->d_name; + if (name == "." || name == "..") continue; + + const auto st = stat_at(stream->fd(), name.c_str()); + if (!st) { + if (st.error().value() == ENOENT) continue; // vanished mid-walk + return std::unexpected(fail("stat", join(display, name), st.error())); + } + + if (S_ISREG(st->st_mode)) { + tree[join(rel, name)] = Info{static_cast(st->st_size), + static_cast(st->st_mtim.tv_sec)}; + } else if (S_ISDIR(st->st_mode)) { + auto sub = open_dir_at(stream->fd(), name.c_str()); + if (!sub) { + if (sub.error().value() == ENOENT) continue; + return std::unexpected( + fail("open directory", join(display, name), sub.error())); + } + if (auto walked = + walk(std::move(*sub), join(display, name), join(rel, name), tree); + !walked) { + return walked; + } + } + // Symlinks, sockets, pipes, devices: not part of the v0 snapshot. + } + return {}; +} +``` + +```go +// walk enumerates the directory owned by f, recording every regular file +// into out. display is the path for error messages, rel the DIR-relative +// prefix. f is closed before walk returns. +func walk(f *os.File, display, rel string, out tree) error { + defer f.Close() + + names, err := f.Readdirnames(-1) + if err != nil { + return fmt.Errorf("reading '%s': %w", display, err) + } + + dirfd := int(f.Fd()) + for _, name := range names { + var st unix.Stat_t + if err := unix.Fstatat(dirfd, name, &st, unix.AT_SYMLINK_NOFOLLOW); err != nil { + if errors.Is(err, unix.ENOENT) { + continue // vanished mid-walk + } + return failf("stat", join(display, name), err) + } + + switch st.Mode & unix.S_IFMT { + case unix.S_IFREG: + out[join(rel, name)] = info{size: st.Size, mtime: st.Mtim.Sec} + case unix.S_IFDIR: + childFd, err := unix.Openat(dirfd, name, dirFlags, 0) + if err != nil { + if errors.Is(err, unix.ENOENT) { + continue + } + return failf("open directory", join(display, name), err) + } + child := os.NewFile(uintptr(childFd), join(display, name)) + if err := walk(child, join(display, name), join(rel, name), out); err != nil { + return err + } + } + // Symlinks, sockets, pipes, devices: not part of the v0 snapshot. + } + return nil +} +``` + +```rust +/// Walk the directory owned by `fd`, recording every regular file into +/// `tree`. `display` is the path for error messages, `rel` the DIR-relative +/// prefix. `fd` is dropped (closed) when the walk of this level finishes. +fn walk(fd: OwnedFd, display: &str, rel: &str, tree: &mut Tree) -> Result<()> { + // Dir::read_from dups the fd for its own cursor; `fd` stays usable as the + // anchor for the *at() calls below. + let dir = Dir::read_from(&fd).map_err(|e| fail("open directory", display, e))?; + + for entry in dir { + let entry = entry.map_err(|e| fail("read directory", display, e))?; + let name = entry.file_name().to_string_lossy().into_owned(); + if name == "." || name == ".." { + continue; + } + + let st = match statat(&fd, &name, AtFlags::SYMLINK_NOFOLLOW) { + Ok(st) => st, + Err(rustix::io::Errno::NOENT) => continue, // vanished mid-walk + Err(e) => return Err(fail("stat", &join(display, &name), e)), + }; + + match FileType::from_raw_mode(st.st_mode) { + FileType::RegularFile => { + tree.insert( + join(rel, &name), + Info { size: st.st_size as i64, mtime: st.st_mtime as i64 }, + ); + } + FileType::Directory => { + let sub = match openat(&fd, &name, DIR_FLAGS, Mode::empty()) { + Ok(sub) => sub, + Err(rustix::io::Errno::NOENT) => continue, + Err(e) => return Err(fail("open directory", &join(display, &name), e)), + }; + walk(sub, &join(display, &name), &join(rel, &name), tree)?; + } + // Symlinks, sockets, pipes, devices: not part of the v0 snapshot. + _ => {} + } + } + Ok(()) +} +``` + +Structure first: each `walk` takes *ownership* of the directory fd (by-value +`Fd`, `*os.File` with `defer Close`, by-value `OwnedFd`), enumerates +entries, `fstatat`s each one relative to that fd, records regular files under +`join(rel, name)`, and recurses into subdirectories by `openat`-ing a child +fd and handing ownership down. `display` and `rel` are deliberately separate +strings — errors should say `cannot stat 'tree/sub/b.txt'` with the +user-supplied prefix, while snapshot keys must be root-relative so a snapshot +taken as `fwatch snapshot ./tree` compares cleanly against a rescan of the +same tree by any other spelling of its path. + +Three per-language details are worth dwelling on. In C++, enumeration goes +through `DirStream` because `readdir` needs a `DIR*`, but classification and +descent use `stream->fd()` — `dirfd(3)` recovers the underlying fd from the +stream, so the anchor for the `*at` calls is the same open file description +the stream reads. In Go, `dirfd := int(f.Fd())` extracts the raw integer, +which is safe here *only because* `defer f.Close()` keeps `f` reachable for +the whole function — more on that in the concurrency lens. In Rust, +`Dir::read_from(&fd)` borrows the fd and takes its own handle for the +enumeration cursor (you can see it in the strace below as an extra +`openat(fd, ".")`), leaving `fd` free to serve as the `statat`/`openat` +anchor; entry order comes from the kernel unsorted in all three languages, +and the sorted map makes the output deterministic anyway. + +The wiring is thin by design. `scan` opens the root (`AT_FDCWD`-relative — +the only full-path resolution), seeds the recursion, and returns the `Tree`. +`cmd_snapshot` prints it. `cmd_diff` re-reads a saved snapshot with an +ordinary buffered file read — parsing lines *from the right*, so the last two +space-separated fields are size and mtime and paths containing spaces +round-trip — then walks the sorted union of old and new key sets emitting +`+`/`-`/`~` lines and the final `fwatch: A added, R removed, M modified` +summary, which prints even when all counts are zero. Fragile bits, stated +plainly: mtime has one-second granularity here (`st_mtim.tv_sec`), so a +same-second same-size rewrite is invisible to `diff` — the durability chapter +sharpens this; symlinks, sockets, pipes, and devices are skipped entirely in +v0; and the whole tree lives in memory, which is fine for the trees `fwatch` +will watch and wrong for a backup tool. + +## Errors, three ways + +All three implementations must produce byte-identical diagnostics — the +contract is `fwatch: error: cannot open directory 'X' (errno N)` on stderr +and exit 1 — so each language has to recover the *raw errno* from its own +error machinery. C++ converts `errno` to `std::error_code` at the syscall +wrapper and formats `Failure` messages with `ec.value()`; fallible functions +return `std::expected`, and `main` never touches `errno` directly. Go wraps +with `%w` (`failf` → `fmt.Errorf`), keeps the chain queryable, and digs the +number back out with `errors.As(err, &errno)` — while `errors.Is(err, +unix.ENOENT)` handles the one *recoverable* error: an entry that `readdir` +reported but that vanished before `fstatat`, which every implementation +silently skips rather than failing the walk. Rust matches +`rustix::io::Errno::NOENT` structurally for the same skip and formats +`errno.raw_os_error()` into an `anyhow` error otherwise. Same policy, three +mechanisms: races you expect are handled at the call site; everything else +carries the path and the errno up to one printer. + +## Concurrency lens + +`fwatch` v0 is deliberately single-threaded in all three languages — the walk +is a depth-first recursion with one live fd chain — but two concurrency +points are already load-bearing. First, `O_CLOEXEC`: the fd table is shared +by every thread in the process, and the fork+exec window it closes is a +*threading* hazard — even "single-threaded" Go code runs on a multithreaded +runtime that may spawn processes on your behalf. Second, Go's `os.File` has a +cleanup registered with the runtime: if a `File` becomes unreachable, the +garbage collector closes its fd for you. That sounds like RAII but is not — +it is nondeterministic, and it means `int(f.Fd())` is a loan against `f`'s +liveness: hold the raw integer past the last reference to `f` and the +collector can close the fd while you are still using the number, or worse, +after the kernel has reassigned it. The `defer f.Close()` in `walk` is +therefore doing double duty — deterministic cleanup *and* a liveness anchor +for `dirfd`. Rust makes the same mistake unrepresentable (`BorrowedFd`'s +lifetime is tied to the `OwnedFd`), and C++ trusts you to keep the `Fd` +alive, which is why the book's wrapper never hands out an owning raw int +except through the explicitly named `release()`. + +## Build, run, observe + +```bash +[host]$ cd examples/07-fds-raii-and-the-vfs && ./demo.sh +``` + +Each language builds, snapshots a fixture tree into a temp file, mutates the +tree, and diffs. To drive it by hand the way the rest of this chapter does, +make a small tree with pinned mtimes and snapshot it: + +```bash +[host]$ mkdir -p tree/sub/deep +[host]$ printf hello > tree/a.txt && printf bye > tree/gone.txt +[host]$ printf worldwide > tree/sub/b.txt && printf abc > tree/sub/deep/c.bin +[host]$ touch -d @1000000000 tree/a.txt && touch -d @1000000300 tree/gone.txt +[host]$ touch -d @1000000100 tree/sub/b.txt && touch -d @1000000200 tree/sub/deep/c.bin +[host]$ ./cpp/build/release/app snapshot tree +a.txt 5 1000000000 +gone.txt 3 1000000300 +sub/b.txt 9 1000000100 +sub/deep/c.bin 3 1000000200 +``` + +On this host the Go and Rust binaries print those same four lines, and +`cmp` between the three captured outputs confirms they are byte-identical — +that identical observable behavior is what `verify.lua` asserts, and the +runner (`python3 scripts/test-all-examples.py --only 07-fds-raii-and-the-vfs`) +reports `PASS PASS PASS` for cpp, go, and rust. + +## Cross-check, three ways + +**Snapshot fields against `ls -li` and `stat`.** The snapshot claims sizes +5, 3, 9, 3 and four specific mtimes. Independent tools that read the same +inodes agree, field for field: + +```bash +[host]$ ls -li tree +68626 -rw-r--r-- 1 rsedor rsedor 5 Sep 8 2001 a.txt +68627 -rw-r--r-- 1 rsedor rsedor 3 Sep 8 2001 gone.txt +68624 drwxr-xr-x 3 rsedor rsedor 80 Jul 18 23:02 sub +[host]$ stat --format='%i %s %Y %n' tree/a.txt tree/sub/b.txt +68626 5 1000000000 tree/a.txt +68628 9 1000000100 tree/sub/b.txt +``` + +`ls` renders `mtime` 1000000000 as `Sep 8 2001` — the same inode field, +formatted; the sizes match the snapshot's second column exactly. + +**The syscalls, under `strace`.** Filtering the C++ binary to the calls the +source claims it makes (on x86_64 glibc, `fstatat` arrives in the kernel as +`newfstatat`; nothing in this program uses `statx`, and tracing it confirms +zero calls): + +```bash +[host]$ strace -e trace=openat,newfstatat,statx ./cpp/build/release/app snapshot tree +openat(AT_FDCWD, "tree", O_RDONLY|O_NOFOLLOW|O_CLOEXEC|O_DIRECTORY) = 3 +newfstatat(3, "gone.txt", {st_mode=S_IFREG|0644, st_size=3, ...}, AT_SYMLINK_NOFOLLOW) = 0 +newfstatat(3, "a.txt", {st_mode=S_IFREG|0644, st_size=5, ...}, AT_SYMLINK_NOFOLLOW) = 0 +newfstatat(3, "sub", {st_mode=S_IFDIR|0755, st_size=80, ...}, AT_SYMLINK_NOFOLLOW) = 0 +openat(3, "sub", O_RDONLY|O_NOFOLLOW|O_CLOEXEC|O_DIRECTORY) = 4 +newfstatat(4, "b.txt", {st_mode=S_IFREG|0644, st_size=9, ...}, AT_SYMLINK_NOFOLLOW) = 0 +newfstatat(4, "deep", {st_mode=S_IFDIR|0755, st_size=60, ...}, AT_SYMLINK_NOFOLLOW) = 0 +openat(4, "deep", O_RDONLY|O_NOFOLLOW|O_CLOEXEC|O_DIRECTORY) = 5 +newfstatat(5, "c.bin", {st_mode=S_IFREG|0644, st_size=3, ...}, AT_SYMLINK_NOFOLLOW) = 0 ++++ exited with 0 +++ +``` + +(Loader lines for `ld.so.cache` and the shared libraries trimmed.) Read the +first arguments: only the root open uses `AT_FDCWD`; every later `openat` and +`newfstatat` anchors on fd 3, 4, or 5 — the dirfd chain, exactly as Figure +7.1 draws it, with every open carrying `O_CLOEXEC|O_NOFOLLOW|O_DIRECTORY`. +The kernel also reports unsorted entry order (`gone.txt` before `a.txt`); +the sorted output really does come from the map. The Rust binary shows one +extra `openat(3, ".", …) = 4` per level — `Dir::read_from` acquiring its own +cursor handle, just as its comment says. + +**The fd table of a live walk, in `/proc`.** `snapshot` finishes its walk +before printing, so to catch the dirfds open you have to slow the walk down. +`strace` can inject delays; against a tree `big/l1/l2/l3/` holding 3000 +files, delaying each stat by 20 ms leaves minutes to inspect: + +```bash +[host]$ strace -o /dev/null -e trace=newfstatat -e inject=newfstatat:delay_enter=20000 \ + ./cpp/build/release/app snapshot big > /dev/null & +[host]$ ls -l /proc//fd +lr-x------ 1 rsedor rsedor 64 Jul 18 23:03 3 -> …/fixture/big +lr-x------ 1 rsedor rsedor 64 Jul 18 23:03 4 -> …/fixture/big/l1 +lr-x------ 1 rsedor rsedor 64 Jul 18 23:03 5 -> …/fixture/big/l1/l2 +lr-x------ 1 rsedor rsedor 64 Jul 18 23:03 6 -> …/fixture/big/l1/l2/l3 +[host]$ cat /proc//fdinfo/6 +pos: 1980 +flags: 02700000 +mnt_id: 59 +ino: 69929 +``` + +(fds 0–2 trimmed.) Four dirfds, one per recursion level — the walk holds +`O(depth)` fds, not `O(files)`. The `fdinfo` file reads out the *description* +layer: `pos: 1980` is the readdir cursor partway through l3's 3000 entries, +and `flags: 02700000` decodes as `O_CLOEXEC` (02000000) + `O_NOFOLLOW` +(0400000) + `O_DIRECTORY` (0200000) + `O_LARGEFILE` (0100000, added by the +64-bit ABI) — the close-on-exec bit, verified on a live process. The `ino:` +lines completed the loop on this host: fds 3–6 reported inodes 69926–69929, +and `ls -lid big big/l1 big/l1/l2 big/l1/l2/l3` printed 69926, 69927, 69928, +and 69929 for the same directories, while +`lsof -a -p -d 0-20` listed the four fds as type `DIR` with the same +`NODE` numbers. fd table → description → inode: each layer observed with a +different tool, all agreeing. + +> **On the lab VM** unverified — +> the eBPF view of this chapter is watching `openat` flags fleet-wide with +> `opensnoop`/`bpftrace` instead of per-process `strace`; bcc-tools are not +> runnable on this host without privileges, and chapter 30 (Debugging part) +> exercises exactly that on the `systems-target` VM. + +## What you learned + +- A file descriptor is three layers: a per-process fd-table slot (home of the + close-on-exec bit), a shared open file description (flags and offset — + `/proc//fdinfo` reads it out), and the inode (`ls -li`'s numbers, + `fwatch`'s size and mtime). +- `O_CLOEXEC` at open time is the only race-free spelling, and + `O_DIRECTORY|O_NOFOLLOW` turn classification races into clean errors; + `openat`/`fstatat` anchored on a dirfd resolve one component per call and + are immune to ancestor renames. +- Ownership is the same idea three ways: a move-only `Fd` with an explicit + `release()` for `fdopendir`'s ownership transfer, `os.File` with a deferred + `Close` that also keeps the GC's nondeterministic closer at bay, and + `OwnedFd`/`BorrowedFd` where the compiler enforces the discipline. +- Expected races are policy: `ENOENT` on a vanished entry is a skip at the + call site in all three languages; everything else carries path + errno to + one printer that formats identical text. + +Next, **page cache and durability**: what the kernel does with the bytes you +write, why `fwatch`'s one-second mtimes can lie, and what `fsync` actually +promises. + +--- + +

verified — every number and +output excerpt above was produced on the Fedora 44 reference host this +session: the runner printed 07-fds-raii-and-the-vfs PASS PASS +PASS (3 passed, 0 failed, 0 skipped); all three binaries emitted the +identical four snapshot lines shown (confirmed byte-identical with +cmp); the strace excerpt is real trimmed output showing the +AT_FDCWD root open and the fd 3→4→5 dirfd chain with +O_RDONLY|O_NOFOLLOW|O_CLOEXEC|O_DIRECTORY on every open; and the +delay-injection run yielded the live /proc/<pid>/fd +listing (dirfds 3–6), fdinfo flags 02700000 and +inodes 69926–69929 matching ls -lid and lsof. The +"On the lab VM" eBPF callout is unverified as marked, as is the exact readdir +pos value on filesystems other than tmpfs.

diff --git a/_docs/08-page-cache-and-durability.md b/_docs/08-page-cache-and-durability.md new file mode 100644 index 0000000..903e3f8 --- /dev/null +++ b/_docs/08-page-cache-and-durability.md @@ -0,0 +1,476 @@ +--- +title: "The page cache and durability" +order: 8 +part: "Files and I/O" +description: "What write(2) actually promises — and what durability costs: dirty pages and writeback, fsync vs fdatasync vs sync_file_range, O_DIRECT, and the atomic-rename pattern, all measured with one tri-language benchmark on real hardware." +duration: 45 minutes +--- + +Chapter 7 ended with a file descriptor safely wrapped in RAII and a byte +successfully written through the VFS. This chapter is about the uncomfortable +question that follows: written *where*? A successful `write(2)` puts your +bytes in kernel memory — the **page cache** — and nothing more; if the power +fails a second later, they are gone, and no error will ever tell you so. The +gap between "the syscall returned 0 errors" and "the bytes are on the device" +is where databases earn their reputations, and this chapter measures that gap +with `iobench`, a small write benchmark implemented identically in C++23, Go, +and Rust. On this host the same 64 MiB costs 21 ms to "write" and 138 ms to +write *durably* — and by the end you will have watched the difference happen +in `/proc/meminfo`, `/proc//io`, and an `strace` syscall table. + +The code is in `examples/08-page-cache-and-durability/`. `./demo.sh` there +builds and runs all three implementations; its `README.md` covers the CLI, +output format, and exit codes shared by all three. + +{% include excalidraw.html + file="08-write-path-page-cache" + alt="Three columns over user space, kernel, and storage bands. In buffered mode, write(2) copies into the page cache and data reaches the NVMe device either by later asynchronous writeback or by an fdatasync at the end; in fsync-every mode an fdatasync every N blocks forces it down; in direct mode an O_DIRECT write bypasses the page cache and DMAs straight from the user buffer to the device." + caption="Figure 8.1 — one write path, three flush points: where each iobench mode hands bytes to the device" %} + +> **Tools used** — `strace`, `stat`, `grep`, `python3` (host); +> `cachestat`/`biolatency` (lab VM, exercised in Part 8). Everything here is +> checked by `scripts/check-host.sh`, ships with Fedora, or is preinstalled in +> the lab VMs. + +## What write(2) actually promises + +The contract of a successful buffered `write(2)` is precise and smaller than +intuition suggests: the kernel has copied your bytes into page-cache pages +belonging to the file, marked those pages **dirty**, and made the data +visible to every subsequent `read(2)` from any process. That is all. +Durability is explicitly not included — the manpage says so — and the return +value is really just confirming a memcpy into kernel memory. That is why +`iobench --mode buffered` reports 3,013 MiB/s on this host's NVMe drive: the +timed loop never touches the device. You are benchmarking `memcpy` plus +bookkeeping. + +The cache earns its keep in both directions. On reads, the kernel notices +sequential access and issues **readahead** — it fetches pages you have not +asked for yet, so a sequential reader almost never blocks on the device +(`posix_fadvise(2)` lets you tune or defeat this; `iobench` is write-only, so +readahead sits out this chapter's measurements). On writes, dirty pages +decouple your process from device latency entirely — until something forces +the issue. + +Dirty pages do not accumulate forever. Kernel **writeback** (flusher) threads +write them to the device in the background, driven by two kinds of pressure: +age (`vm.dirty_expire_centisecs`, 30 s by default — no page stays dirty +longer) and volume (`vm.dirty_background_ratio` starts background writeback; +at `vm.dirty_ratio` the kernel begins throttling the writers themselves, and +your "memcpy-speed" `write(2)` suddenly runs at device speed). The systemwide +total is one number: `Dirty:` in `/proc/meminfo`, which the cross-check +section watches balloon by a gigabyte and collapse again. + +## Choosing your flush point: fsync, fdatasync, sync_file_range + +When you need the bytes on the device *now*, you pick one of three calls, in +decreasing order of guarantee: + +- **`fsync(2)`** — writes the file's dirty pages *and* its metadata (size, + timestamps, block allocations) to the device, and asks the device to flush + its own volatile write cache. When it returns, the data survives power + loss. +- **`fdatasync(2)`** — the same, minus metadata that is not needed to read + the data back. An mtime update does not gate your bytes, so skipping that + metadata write can save a device round trip. A changed file *size* is + data-critical metadata and is still flushed. `iobench` uses `fdatasync` + throughout, and the append-heavy write pattern of a database WAL is + exactly the case it was designed for. +- **`sync_file_range(2)`** — the expert-mode footgun: it can start or wait on + writeback for a byte range, but it never writes metadata and never issues a + device cache flush, so it is **not** a durability primitive. Its real use + is pacing — nudging writeback along so a later `fdatasync` has less to do — + and its manpage warning ("extremely dangerous" as a durability mechanism) + is one of the better reads in section 2. + +`iobench` turns the choice into a flag. `--mode buffered` times the bare +`write(2)` loop, then pays for a single closing `fdatasync` on a separate +`fsync_ms=` line, so the price of durability is printed as its own number. +`--mode fsync-every` moves `fdatasync` *inside* the timed loop, every N +blocks — durable-as-you-go, the way a database commits. + +## O_DIRECT, and getting renames to keep promises + +The third mode opts out of the cache entirely. Opening with **`O_DIRECT`** +makes each `write(2)` DMA straight from your user buffer to the device — +no copy into the page cache, no dirty pages, no writeback. The price is a +contract: buffer address, file offset, and transfer length must all be +aligned to the device's logical block size (4,096 bytes covers current +hardware), and a filesystem may simply refuse the flag at `open(2)` with +`EINVAL` — which `iobench` treats as a documented outcome (exit 4), because +it genuinely varies by filesystem. + +Why do databases insist on it? Because they already maintain their own buffer +pool, sized and evicted with knowledge of the workload that the kernel cannot +have; caching every page twice wastes half the RAM. And because a database +needs to *know* when bytes are on disk to order its writes (WAL before data +page), a cache that says "done" when nothing happened is a liability. Note +what `O_DIRECT` does **not** give you: the device's own write cache is still +in play, so even direct I/O needs `fdatasync` (or `O_DSYNC`) for real +durability. + +The last durability tool is not a syscall but a pattern. To replace a config +file so that readers see either the old version or the new one — never a +torn half — you use **atomic rename**: write the new content to a temporary +file in the *same directory*, `fsync` the temp file, `rename(2)` it over the +target (rename within a filesystem is atomic), then `fsync` the *directory* +fd so the rename itself — a directory entry, i.e. metadata living in the +directory, as chapter 7's inode walk showed — is on disk too. Skip step 2 and +a crash can leave you a correctly named empty file; skip step 4 and the old +name can come back after reboot. `iobench` deliberately does not do this — a +benchmark should overwrite in place — but every "save settings" feature you +ship should. + +## How the code works + +All three implementations share one CLI, one output format, and one +structure: parse arguments, open with mode-appropriate flags, write M MiB in +64 KiB blocks with mode-appropriate syncing, report. The heart is the `run` +function, shown here verbatim from each language: + +{% include codetabs.html langs="C++|Go|Rust" %} + +```cpp +[[nodiscard]] int run(const Options& opt) { + auto fd = open_output(opt.file, opt.mode == Mode::direct); + if (!fd) { + if (opt.mode == Mode::direct && fd.error() == std::errc::invalid_argument) { + std::println(stderr, "direct: unsupported on this filesystem"); + return 4; + } + std::println(stderr, "error: open {}: {}", opt.file, fd.error().message()); + return 1; + } + + const auto block = make_block(); + if (!block) { + std::println(stderr, "error: aligned_alloc failed"); + return 1; + } + const std::span data{block.get(), kBlockSize}; + + const std::size_t nblocks = opt.size_mb * (kMiB / kBlockSize); + const auto bytes = static_cast(opt.size_mb) * kMiB; + + const auto t0 = std::chrono::steady_clock::now(); + for (std::size_t i = 0; i < nblocks; ++i) { + if (const auto w = write_all(*fd, data); !w) { + std::println(stderr, "error: write {}: {}", opt.file, w.error().message()); + return 1; + } + if (opt.mode == Mode::fsync_every && (i + 1) % opt.every == 0) { + if (const auto s = datasync(*fd); !s) { + std::println(stderr, "error: fdatasync {}: {}", opt.file, s.error().message()); + return 1; + } + } + } + if (opt.mode == Mode::fsync_every && nblocks % opt.every != 0) { + if (const auto s = datasync(*fd); !s) { + std::println(stderr, "error: fdatasync {}: {}", opt.file, s.error().message()); + return 1; + } + } + const auto t1 = std::chrono::steady_clock::now(); + + const std::int64_t ns = elapsed_ns(t0, t1); + const double mib_per_s = + (static_cast(bytes) / static_cast(kMiB)) / (static_cast(ns) / 1e9); + std::println("mode={} bytes={} ms={} MiB/s={:.1f}", mode_name(opt.mode), bytes, + ns / 1'000'000, mib_per_s); + + if (opt.mode == Mode::buffered) { + const auto t2 = std::chrono::steady_clock::now(); + if (const auto s = datasync(*fd); !s) { + std::println(stderr, "error: fdatasync {}: {}", opt.file, s.error().message()); + return 1; + } + const auto t3 = std::chrono::steady_clock::now(); + std::println("fsync_ms={}", elapsed_ns(t2, t3) / 1'000'000); + } + return 0; +} +``` + +```go +func run(opt options) (int, error) { + flags := unix.O_WRONLY | unix.O_CREAT | unix.O_TRUNC + if opt.mode == "direct" { + flags |= unix.O_DIRECT + } + fd, err := unix.Open(opt.file, flags, 0o644) + if err != nil { + if opt.mode == "direct" && errors.Is(err, unix.EINVAL) { + fmt.Fprintln(os.Stderr, "direct: unsupported on this filesystem") + return 4, nil + } + return 1, fmt.Errorf("open %s: %w", opt.file, err) + } + defer unix.Close(fd) + + block := alignedBlock() + nblocks := opt.sizeMB * (mib / blockSize) + bytes := opt.sizeMB * mib + + t0 := time.Now() + for i := uint64(0); i < nblocks; i++ { + if err := writeAll(fd, block); err != nil { + return 1, fmt.Errorf("%s: %w", opt.file, err) + } + if opt.mode == "fsync-every" && (i+1)%opt.every == 0 { + if err := datasync(fd); err != nil { + return 1, fmt.Errorf("%s: %w", opt.file, err) + } + } + } + if opt.mode == "fsync-every" && nblocks%opt.every != 0 { + if err := datasync(fd); err != nil { + return 1, fmt.Errorf("%s: %w", opt.file, err) + } + } + ns := clampNS(time.Since(t0)) + + mibPerS := (float64(bytes) / float64(mib)) / (float64(ns) / 1e9) + fmt.Printf("mode=%s bytes=%d ms=%d MiB/s=%.1f\n", opt.mode, bytes, ns/1e6, mibPerS) + + if opt.mode == "buffered" { + t2 := time.Now() + if err := datasync(fd); err != nil { + return 1, fmt.Errorf("%s: %w", opt.file, err) + } + fmt.Printf("fsync_ms=%d\n", clampNS(time.Since(t2))/1e6) + } + return 0, nil +} +``` + +```rust +fn run(opt: &Options) -> Result { + let fd = match open_output(&opt.file, opt.mode == SyncMode::Direct) { + Ok(fd) => fd, + Err(Errno::INVAL) if opt.mode == SyncMode::Direct => { + eprintln!("direct: unsupported on this filesystem"); + return Ok(ExitCode::from(4)); + } + Err(e) => return Err(e).context(format!("open {}", opt.file)), + }; + + let block = AlignedBlock::new(); + let nblocks = opt.size_mb * (MIB / BLOCK_SIZE as u64); + let bytes = opt.size_mb * MIB; + + let t0 = Instant::now(); + for i in 0..nblocks { + write_all(&fd, block.as_slice()).with_context(|| format!("write {}", opt.file))?; + if opt.mode == SyncMode::FsyncEvery && (i + 1) % opt.every == 0 { + fdatasync(&fd).with_context(|| format!("fdatasync {}", opt.file))?; + } + } + if opt.mode == SyncMode::FsyncEvery && nblocks % opt.every != 0 { + fdatasync(&fd).with_context(|| format!("fdatasync {}", opt.file))?; + } + let ns = elapsed_ns(t0); + + let mib_per_s = (bytes as f64 / MIB as f64) / (ns as f64 / 1e9); + println!( + "mode={} bytes={} ms={} MiB/s={:.1}", + opt.mode.name(), + bytes, + ns / 1_000_000, + mib_per_s + ); + + if opt.mode == SyncMode::Buffered { + let t2 = Instant::now(); + fdatasync(&fd).with_context(|| format!("fdatasync {}", opt.file))?; + println!("fsync_ms={}", elapsed_ns(t2) / 1_000_000); + } + Ok(ExitCode::SUCCESS) +} +``` + +Walk the shared skeleton. **The open** adds `O_DIRECT` to +`O_WRONLY | O_CREAT | O_TRUNC` only in direct mode, and only the direct-mode +`EINVAL` is treated as the "unsupported filesystem" outcome — an `EINVAL` +from a buffered open would be a real bug and falls through to the generic +error path. **The block** is built once and rewritten every iteration: 64 KiB +of `i & 0xFF` bytes at a 4,096-aligned address. C++ gets alignment from +`std::aligned_alloc` (freed via a `unique_ptr` with a `FreeDeleter`); Go and +Rust over-allocate by one alignment unit and slice to the first aligned +offset inside — Go's comment notes the heap does not move allocations, and +Rust keeps the backing `Vec` alive in a struct so the aligned range stays +valid. The alignment is mandatory for `O_DIRECT` and harmless otherwise, +which is why all three modes share one buffer. + +**The write** is never a bare `write(2)`. Each language has a `write_all` +that loops: a short write advances the slice/span by the returned count, and +`EINTR` — the call interrupted by a signal before writing anything — retries. +A 64 KiB write nearly always completes in one call, but "nearly always" is +exactly the assumption this book keeps telling you not to compile in. +**The sync placement** is the whole experiment: fsync-every calls `fdatasync` +inside the timed region every `--every` blocks (plus a trailing sync when the +block count is not a multiple of N — drop that and up to N−1 blocks quietly +escape the durability guarantee being measured), while buffered mode stops +the clock, prints the report line, and only then times its single `fdatasync` +as `fsync_ms=`. Elapsed time is clamped to at least 1 ns before the +throughput division — on tmpfs a tiny run genuinely can round to 0 ms. + +### Errors, three ways + +Chapter 5's policy gets its first workout against a syscall that fails in +interesting ways. The key move in all three languages is *classify before +you generalize*: the open error is inspected for the one value with a +documented meaning (`EINVAL` in direct mode → message + exit 4) before being +wrapped into the general error path (exit 1). C++ compares the +`std::error_code` against `std::errc::invalid_argument`, keeping the +comparison portable rather than testing raw `errno`. Go wraps with `%w` and +tests with `errors.Is(err, unix.EINVAL)`, so classification still works +through layers of context. Rust matches `Err(Errno::INVAL)` *before* +converting into `anyhow::Error` — the guard sits in the `match` because after +`.context()` the typed errno is boxed away and would need a downcast. Usage +errors are a separate exit code (2) diagnosed entirely in `parse_args`, +before any file is created — `verify.lua` checks that a bad `--mode` leaves +nothing on disk. + +### Concurrency lens + +`iobench` is single-threaded on purpose, but the system it measures is not. +The moment the write loop starts dirtying pages, kernel flusher threads wake +and begin writing them back *concurrently with the loop* — in the cross-check +below, a 2 GiB write peaks at only ~1.1 GiB dirty because writeback was +draining the pool while the program filled it. The final `fdatasync` is +therefore a *rendezvous*, not the whole transfer: it waits for whatever +writeback has not already covered. Language runtimes add one more wrinkle +from chapter 6: a blocking `fdatasync` parks the calling OS thread, so in Go +the runtime hands the P to another thread and other goroutines keep running — +one reason "just fsync in a goroutine" feels free until you count threads. +And timing uses `steady_clock` / `time.Now` monotonic readings / +`Instant` — wall-clock time can step backwards; monotonic clocks are the only +thing you should ever subtract. + +## Build, run, observe + +```bash +[host]$ cd examples/08-page-cache-and-durability +[host]$ ./demo.sh cpp run --mode buffered bench.bin +[host]$ ./demo.sh cpp run --mode fsync-every bench.bin +[host]$ ./demo.sh cpp run --mode direct bench.bin +``` + +On this host (btrfs on NVMe, 64 MiB default, C++ build) the three modes +printed: + +``` +mode=buffered bytes=67108864 ms=21 MiB/s=3013.0 +fsync_ms=17 +mode=fsync-every bytes=67108864 ms=138 MiB/s=462.8 +mode=direct bytes=67108864 ms=81 MiB/s=785.2 +``` + +Read the buffered pair together: 21 ms to "write" 64 MiB, then 17 ms of +`fdatasync` — durability nearly doubles the end-to-end time, and the 3,013 +MiB/s headline was never a device number. Syncing every 8 blocks (512 KiB) +drops throughput 6.5× to 462.8 MiB/s because the device's flush latency now +sits inside the loop 128 times. `O_DIRECT` lands between them at 785.2 MiB/s: +every byte really moved to the device inside the timed region, with no +copy and no cache. Go and Rust agree with the C++ numbers (buffered 3,283.9 +and 3,483.6 MiB/s with `fsync_ms=17` and `14` on the same runs) — the page +cache does not care which language dirtied it. After any run, +`stat -c %s bench.bin` reports exactly 67108864: same bytes, very different +guarantees. + +## Cross-check: watching the bytes move + +The claims above are checkable from the outside. First, the syscall pattern: +`strace -c` on a 4 MiB fsync-every run with `--every 4` should show 64 data +writes (4 MiB / 64 KiB) and 16 `fdatasync`s (64 / 4). It does — the 65th +write is the report line to stdout: + +``` +[host]$ strace -c -e trace=write,fdatasync,openat ./cpp/build/release/app --mode fsync-every --every 4 --size-mb 4 bench.bin +% time seconds usecs/call calls errors syscall +------ ----------- ----------- --------- --------- ---------------- + 54.29 0.001589 99 16 fdatasync + 45.71 0.001338 20 65 write + 0.00 0.000000 0 6 openat +``` + +Second, the dirty-page story. Sampling `Dirty:` from `/proc/meminfo` before, +during, and after a 2 GiB buffered run on this host: + +``` +Dirty: 8684 kB # before +Dirty: 1118476 kB # mid-run, during the write loop +Dirty: 22520 kB # after exit — fdatasync drained it +``` + +A gigabyte of "written" data sat in RAM mid-run (and only ~1.1 GiB of the 2 +GiB was dirty at the sample — concurrent writeback was already draining), and +the closing `fdatasync` — 424 ms on that run, against 661 ms for the whole 2 +GiB write loop — put the rest on the device. Third, the per-process view in +`/proc//io` during that same 2 GiB run: + +``` +[host]$ grep -E '^(wchar|write_bytes)' /proc//io # mid write loop +wchar: 540213256 +write_bytes: 540286976 +[host]$ grep -E '^(wchar|write_bytes)' /proc//io # during the final fdatasync +wchar: 2147483656 +write_bytes: 2147500032 +``` + +`wchar` counts bytes handed to `write(2)`; `write_bytes` counts bytes the +task caused to be destined for the storage layer, charged at page-dirtying +time and in page-sized units (hence it running a few KiB ahead). The second +sample is the chapter's thesis in two lines: all 2 GiB of `write(2)` calls +had returned while the process was still alive *waiting inside `fdatasync`*. +Finally, the same binary on tmpfs (this book's scratchpad) shows why none of +the magnitudes are portable: buffered 3,547.2 MiB/s with `fsync_ms=0`, +fsync-every 5,113.2, direct 5,304.7 — with no device behind the file, +`fdatasync` is nearly free, `O_DIRECT` is accepted but bypasses nothing, and +the three modes converge. That is precisely why `verify.lua` asserts output +shape and exact file sizes, never throughput. + +

unverified — On the +lab VM: the bcc-tools pair cachestat (page-cache +hit/miss counts while iobench runs) and biolatency (the block +I/O latency histogram that separates fsync-every's many small flushes from +direct mode's steady stream) would show the kernel side of these runs. +They need root and a kernel with BPF tooling, so they are not run on this +host; chapter 30 (Debugging part) exercises them on the lab VM.

+ +## What you learned + +- A successful `write(2)` means "copied into the page cache", nothing more; + dirty pages reach the device via writeback (age and pressure driven) or + when you say so — and on this host that "so" cost 17 ms atop a 21 ms + 64 MiB write. +- `fsync` flushes data + metadata, `fdatasync` skips metadata that does not + gate reading the data back, and `sync_file_range` is writeback pacing, not + durability. +- `O_DIRECT` trades the cache for alignment rules and an `EINVAL`-at-open + probe — and databases take that trade to own their caching and write + ordering; crash-safe file replacement is tmpfile → `fsync` → `rename` → + `fsync(dir)`. +- `strace -c`, `/proc/meminfo`'s `Dirty:`, and `/proc//io` let you + audit any program's durability behavior from outside — 2 GiB of `wchar` + with the process still parked in `fdatasync` is the promise gap made + visible. + +Next, **event loops**: `epoll` and `inotify`, where the question stops being +"is the byte durable" and becomes "which of ten thousand fds is ready". + +--- + +

verified — all numbers above +are from runs performed on the reference host (btrfs on NVMe, Fedora 44, +kernel 7.1.3-200.fc44) while writing this chapter: the mode reports +(ms=21 / fsync_ms=17, ms=138, ms=81 at +64 MiB; Go/Rust buffered parity), the strace -c table (16 +fdatasync, 65 write), the Dirty: +sequence 8684 → 1118476 → 22520 kB around a 2 GiB buffered run +(ms=661, fsync_ms=424), the +/proc/<pid>/io samples, the tmpfs convergence numbers, and +the runner gate (test-all-examples.py: cpp PASS, go PASS, rust +PASS — 15 behavioral checks per language). The bcc-tools callout above is the +one unverified item, deferred to the lab VM.

diff --git a/_docs/09-event-loops-epoll-inotify.md b/_docs/09-event-loops-epoll-inotify.md new file mode 100644 index 0000000..0d4c948 --- /dev/null +++ b/_docs/09-event-loops-epoll-inotify.md @@ -0,0 +1,584 @@ +--- +title: "Event loops: epoll and inotify" +order: 9 +part: "Files and I/O" +description: "The readiness model — nonblocking fds multiplexed by one epoll loop, level versus edge triggering, inotify's per-watch queues and overflow, and timerfd/signalfd as ordinary readable fds — as fwatch v1 grows a real-time watch subcommand and Go's select turns out to be the same kernel machinery wearing channels." +duration: 50 minutes +--- + +Chapter 8 closed on a question durability cannot answer: "which of ten +thousand fds is ready?" This chapter answers it with the **readiness model** +— the kernel will tell you *when an fd has something for you*, and a program +built on that promise never blocks anywhere except the one call that asks. +`fwatch` v1 keeps the polling-era `snapshot` and `diff` subcommands from +Chapters 7 and 8 and adds `watch`: a real-time watcher that reports +`created`/`modified`/`deleted` per file, debounced 100 ms per path. The C++ +and Rust versions build it the explicit way — one `epoll` loop over three +kernel fds (inotify, timerfd, signalfd), a single thread, nothing hidden — +while the Go version inverts the picture: the runtime's netpoller owns the +epoll loop, and the same three event sources surface as channels in a +`select`. Same kernel machinery, two programming surfaces, and by the end +you will have watched both under `strace` and read epoll's own interest list +out of `/proc`. + +The code is in `examples/09-event-loops-epoll-inotify/`. `./demo.sh` there +builds all three implementations and runs a 1-second self-demo; its +`README.md` specifies the CLI, the debounce/coalescing rules, and the exit +codes all three languages share. + +{% include excalidraw.html + file="09-event-loop-anatomy" + alt="Two bands. In the user-space band, four loop steps run left to right — arm timerfd, epoll_wait, dispatch on the data token, drain and act — with a return arrow labeled loop. In the kernel band, an eventpoll interest-list box holds tfd 3, 4, 5, fed by three source boxes: the inotify fd with wd 1 and mask 0x3c6, the CLOCK_MONOTONIC timerfd, and the signalfd carrying blocked SIGINT and SIGTERM; an amber readiness arrow carries EPOLLIN and the token back up into epoll_wait." + caption="Figure 9.1 — the anatomy of fwatch watch: one loop, three event sources; every fd number, token, and mask in this figure reappears verbatim in the /proc inspection later in the chapter" %} + +> **Tools used** — `strace`, `mktemp`, `stat`, `ls`, `python3` (host); +> `bpftrace` (lab VM, exercised in Part 8). Everything here is checked by +> `scripts/check-host.sh`, ships with Fedora, or is preinstalled in the lab VMs. + +## The readiness model + +A blocking `read(2)` couples two things that have no business being coupled: +*waiting* for data and *moving* it. With one input source that is fine; with +three, a blocking read on the wrong fd deadlocks you — while you wait for a +file event, the timeout you promised never fires. The classic escape was one +thread per source; the scalable escape is to decouple: put every fd in +**nonblocking** mode (`IN_NONBLOCK`, `TFD_NONBLOCK`, `SFD_NONBLOCK` — all +`O_NONBLOCK` under different names), so `read` either returns data +immediately or fails with `EAGAIN`, and delegate all the *waiting* to one +syscall that can wait on everything at once. + +That syscall is `epoll_wait(2)`, and its shape explains its scalability. You +create a kernel object with `epoll_create1(2)` — it is itself an fd, of type +`anon_inode:[eventpoll]` — then *register* interest once per fd with +`epoll_ctl(EPOLL_CTL_ADD, …)`. The registration carries two things: an event +mask (`EPOLLIN` — readable) and a caller-chosen 64-bit cookie +(`epoll_event.data`) the kernel returns verbatim when that fd fires. The +interest list lives **in the kernel**, which is the whole advantage over the +older `poll(2)`/`select(2)`: those pass the entire fd set on every call, so +each wait is O(fds); epoll pays the setup cost once and each wakeup returns +only the fds that are actually ready. `fwatch` has three fds so any of them +would work — the point of learning epoll on three is that the loop is +*identical* at thirty thousand, which is where Part 6's network servers take +it. + +## Level, edge, and the ONESHOT caveat + +Registration has one decision that changes program correctness, not just +performance: **when** does a ready fd count as ready again? + +- **Level-triggered** (the default): `epoll_wait` reports the fd as long as + data *remains* — a fact about the fd's state. Forget to read everything and + the next wait simply reports it again. +- **Edge-triggered** (`EPOLLET`): the fd is reported only on the + *transition* to readable — a fact about a state change. Miss data and no + further wakeup comes until *new* data arrives; the only correct consumer + is one that drains the fd to `EAGAIN` on every wakeup, without exception. + +This book uses level triggering everywhere it writes an epoll loop, because +level converts a whole class of lost-wakeup bugs into mere extra wakeups. +The discipline it demands instead is smaller but real: **consume what made +the fd readable**. That is why the loop below reads the timerfd's 8-byte +expiration counter even though it only wants "the timer fired" — leave those +8 bytes and the timerfd stays readable, `epoll_wait` returns instantly +forever, and your loop spins at 100% CPU. Note that `fwatch` drains its +inotify fd to `EAGAIN` anyway: with level triggering that is an optimization +(fewer wakeups per burst, and each burst is timestamped coherently); under +edge it would be the difference between working and silently stalling. + +> **EPOLLONESHOT caveat** — there is a third mode you should know exists +> before you need it. When *multiple threads* call `epoll_wait` on one epoll +> fd, a level-triggered fd that still has data can be reported to two +> threads at once, and even an edge can race a second event while thread one +> is mid-read. `EPOLLONESHOT` disarms the registration after each report; +> the owning thread must re-arm with `EPOLL_CTL_MOD` when done. It is the +> standard fix for multi-threaded epoll — and the reason this book's +> single-threaded loops do not need it. One loop, one thread, no re-arm +> protocol: that simplicity is a feature, not a limitation. + +Interestingly, the readiness style is observable per implementation: the +strace later in this chapter shows Go's netpoller registering our inotify fd +with `EPOLLIN|EPOLLOUT|EPOLLRDHUP|EPOLLET` — the runtime is an +edge-triggered consumer (it always drains to `EAGAIN` and parks goroutines +on the transition), while our C++ and Rust loops register plain level +`EPOLLIN`. + +## inotify: per-watch queues, and where they overflow + +`inotify_init1(2)` returns an fd; `inotify_add_watch(2)` attaches a **watch** +to an inode — directory or file — with a mask of event types, and returns a +small integer *watch descriptor* (`wd`) that names the watch in every event. +`fwatch` watches one directory with six mask bits: `IN_CREATE`, `IN_MODIFY`, +`IN_ATTRIB`, `IN_DELETE`, `IN_MOVED_FROM`, `IN_MOVED_TO` — together `0x3c6`, +a number you will meet again in `/proc`. Reading the fd yields packed +variable-length records — `wd`, `mask`, `cookie`, `len`, then `len` bytes of +NUL-padded name — so all three implementations walk a byte buffer, stepping +by `sizeof(header) + len`. A record with `len == 0` is an event on the +watched object itself (we skip those), and a rename arrives as a +`IN_MOVED_FROM`/`IN_MOVED_TO` pair joined by `cookie`. + +Semantics worth being precise about: events queue **per inotify instance**, +in order, and the kernel already coalesces *identical adjacent* events. The +queue is finite — on this host `/proc/sys/fs/inotify/max_queued_events` is +16384 (with `max_user_watches` 524288 and `max_user_instances` 512) — and +when it fills, the kernel does not block the writer and does not drop +silently: it appends one `IN_Q_OVERFLOW` event and discards until you catch +up. A correct watcher treats `IN_Q_OVERFLOW` as "my view is stale, +re-scan" — which is exactly why `fwatch` kept its `snapshot`/`diff` +subcommands: the polling machinery from Chapters 7 and 8 is the recovery +path for the event machinery of Chapter 9. Also note what a directory watch does +*not* give you: recursion. Subdirectories need their own watches — the cost +model behind editors and sync clients burning thousands of `wd`s. + +## Everything is an fd: timerfd and signalfd + +The elegance of the readiness model is that anything that can *become ready* +can join the loop, and Linux has spent two decades converting non-fd events +into fds. `fwatch` uses two of them. `timerfd_create(2)` gives a timer whose +expiration makes an fd readable; reading it returns an 8-byte count of +expirations since the last read. The C++ and Rust loops run exactly **one** +timer for two jobs — each iteration re-arms it (one-shot) to +`min(overall deadline, earliest debounce due)`, so whichever deadline is +nearest is the one wakeup that gets scheduled. One subtlety hides in +`arm_timer`: an `it_value` of all zeros means *disarm*, so a deadline that +is already past is clamped to one nanosecond — "fire immediately" and "never +fire" are one bit pattern apart. + +`signalfd(2)` does the same for signals — with a prerequisite that trips +everyone once: you must **block** the signals first (`sigprocmask(2)`), +otherwise they are still delivered asynchronously to a handler and the fd +reads nothing. Blocked-then-read, SIGINT stops being a control-flow +interruption and becomes data: a `signalfd_siginfo` record you handle at a +place *you* chose, after the current batch, with no reentrancy rules and no +`volatile sig_atomic_t` flag. The loop still guards `EINTR` anyway +(any signal outside the blocked set can still interrupt `epoll_wait`), but +the shutdown path itself is just another `case` in the dispatch. + +## How the code works + +Two data structures carry the whole design. `pending` — `std::map` in C++, `BTreeMap` in Rust, `map[string]*pendingEvent` +in Go — maps each path to its coalesced event kind and its debounce deadline +(`due = now + 100ms`); ordered maps are chosen deliberately so a batch flush +prints in name order and `verify.lua` can assert output deterministically +(Go sorts the keys at flush time instead). The second structure is the token: +C++ registers each fd with `data.u32` set to `tok_inotify`/`tok_timer`/ +`tok_signal` (Rust: `TOK_INOTIFY = 0` … `TOK_SIGNAL = 2`), so dispatch is a +`switch` on a number the kernel echoes back — no fd comparisons, no lookup +table. Coalescing is one shared `merge` rule: within a window, delete wins, +a fresh create stays `created` through later writes, and delete-then-recreate +reads as `modified` — the same visible truth `diff` would have reported. + +Setup is four fallible constructions — `inotify_init1` + `inotify_add_watch`, +`timerfd_create`, `sigprocmask` + `signalfd`, `epoll_create1` + three +`epoll_ctl` ADDs — every fd owned by Chapter 7's RAII wrapper (`Fd` in C++, +`OwnedFd`-backed nix types in Rust, `os.File` in Go). Then the loop; here it +is in all three languages, verbatim from +`examples/09-event-loops-epoll-inotify/`: + +{% include codetabs.html langs="C++|Go|Rust" %} + +```cpp + for (;;) { + // One timer covers both jobs: it is always armed to whichever comes + // first — the overall timeout or the earliest per-path debounce. + auto next = deadline; + for (const auto& [_, p] : pending) { + next = std::min(next, p.due); + } + if (auto armed = arm_timer(*timer, next - steady_clock::now()); !armed) { + std::println(stderr, "fwatch: watch: timerfd_settime: {}", armed.error().message()); + return 1; + } + + std::array events{}; + const int n = ::epoll_wait(epoll->get(), events.data(), events.size(), -1); + if (n < 0) { + if (errno == EINTR) { + continue; + } + std::println(stderr, "fwatch: watch: epoll_wait: {}", last_error().message()); + return 1; + } + + for (int i = 0; i < n; ++i) { + switch (events[i].data.u32) { + case tok_inotify: { + alignas(inotify_event) char buf[4096]; + for (;;) { + const ssize_t got = ::read(inotify->get(), buf, sizeof buf); + if (got <= 0) { + break; // EAGAIN: queue drained + } + const auto now = steady_clock::now(); + for (const char* p = buf; p < buf + got;) { + const auto* ev = reinterpret_cast(p); + p += sizeof(inotify_event) + ev->len; + if (ev->len == 0) { + continue; // event on the directory itself + } + const auto kind = classify(ev->mask); + if (!kind) { + continue; + } + const std::string name{ev->name}; + const auto it = pending.find(name); + const auto old_kind = it == pending.end() + ? std::nullopt + : std::optional{it->second.kind}; + pending.insert_or_assign( + name, Pending{merge(old_kind, *kind), now + kDebounce}); + } + } + break; + } + case tok_timer: { + std::uint64_t expirations = 0; + std::ignore = ::read(timer->get(), &expirations, sizeof expirations); + const auto now = steady_clock::now(); + if (now >= deadline) { + flush(pending, now, /*all=*/true); + std::println("fwatch: exiting (timeout)"); + return 0; + } + flush(pending, now, /*all=*/false); + break; + } + case tok_signal: { + signalfd_siginfo info{}; + std::ignore = ::read(sigfd->get(), &info, sizeof info); + flush(pending, steady_clock::now(), /*all=*/true); + std::println("fwatch: exiting (signal)"); + return 0; + } + default: + break; + } + } + } +``` + +```go + for { + select { + case ev, ok := <-events: + if !ok { + return errors.New("inotify reader stopped unexpectedly") + } + oldKind := "" + if p, exists := pending[ev.name]; exists { + oldKind = p.kind + } + pending[ev.name] = &pendingEvent{ + kind: mergeKinds(oldKind, ev.kind), + due: time.Now().Add(debounce), + } + rearm() + case <-flushTimer.C: + flush(false) + rearm() + case <-timeout: + flush(true) + fmt.Println("fwatch: exiting (timeout)") + return nil + case <-sigs: + flush(true) + fmt.Println("fwatch: exiting (signal)") + return nil + } + } +``` + +```rust + loop { + // One timer covers both jobs: it is always armed to whichever comes + // first — the overall timeout or the earliest per-path debounce. + let next = pending.values().map(|p| p.due).min().map_or(deadline, |d| d.min(deadline)); + arm_timer(&timer, next.saturating_duration_since(Instant::now()))?; + + let mut events = [EpollEvent::empty(); 8]; + let n = match epoll.wait(&mut events, EpollTimeout::NONE) { + Ok(n) => n, + Err(Errno::EINTR) => continue, + Err(e) => return Err(e).context("epoll_wait"), + }; + + for event in &events[..n] { + match event.data() { + TOK_INOTIFY => loop { + let batch = match inotify.read_events() { + Ok(batch) => batch, + Err(Errno::EAGAIN) => break, // queue drained + Err(e) => return Err(e).context("inotify read"), + }; + let now = Instant::now(); + for ev in batch { + let Some(name) = ev.name else { + continue; // event on the directory itself + }; + let Some(kind) = classify(ev.mask) else { + continue; + }; + let name = name.to_string_lossy().into_owned(); + let old = pending.get(&name).map(|p| p.kind); + pending + .insert(name, Pending { kind: merge(old, kind), due: now + DEBOUNCE }); + } + }, + TOK_TIMER => { + let mut buf = [0u8; 8]; + let _ = nix::unistd::read(&timer, &mut buf); + let now = Instant::now(); + if now >= deadline { + flush(&mut pending, now, true); + println!("fwatch: exiting (timeout)"); + return Ok(()); + } + flush(&mut pending, now, false); + } + TOK_SIGNAL => { + let _ = sigfd.read_signal(); + flush(&mut pending, Instant::now(), true); + println!("fwatch: exiting (signal)"); + return Ok(()); + } + _ => {} + } + } + } +``` + +C++ and Rust are the same program in two spellings — arm, wait forever +(`-1` / `EpollTimeout::NONE`; the timerfd *is* the timeout), dispatch by +token, drain or consume, `EINTR` retries the wait. The one visible library +difference: nix's `inotify.read_events()` does the record-walking that C++ +does by hand over `buf`, returning parsed events with `Option` +names — same bytes, one abstraction up. + +Go's excerpt has no epoll in it, and that is the entire point. Its `watch` +is built from a goroutine and four channel operations: `readEvents` (in +`examples/09-event-loops-epoll-inotify/go/main.go`) wraps the `O_NONBLOCK` +inotify fd in an `os.File` — the act that registers it with the runtime's +**netpoller** — and calls `f.Read` in a loop. That read *looks* blocking, +but because the fd is nonblocking and registered, the runtime parks the +*goroutine* (not an OS thread) until its epoll loop reports readiness, then +resumes it to walk the same packed records (`unix.SizeofInotifyEvent`, +`raw.Len`, TrimRight the NUL padding) and send on a channel. The other +sources never touch an fd in user code: `time.After` is the timeout, +a `time.Timer` re-armed to the earliest `due` is the debounce (`rearm` +mirrors the C++ `arm_timer` computation exactly), and `signal.Notify` +delivers SIGINT/SIGTERM on a channel. `select` is `epoll_wait` one level up +— it blocks until any source is ready and dispatches — except the "fds" are +channels and the kernel machinery is rented from the runtime. + +### Errors, three ways + +Setup is where `watch` can fail, and the missing-directory case exercises +each language's Chapter 5 policy end to end. C++ threads four +`std::expected` constructions together, chaining the three `epoll_ctl` adds +with `.and_then(…)` so the first failure short-circuits with its +`std::error_code`; the observed result is +`fwatch: watch: /no/such/dir: No such file or directory`, exit 1. Go gets +`unix.ENOENT` back from `inotify_add_watch`, matches it with +`errors.Is(err, unix.ENOENT)` to emit the friendlier +`no such directory`, and — because the raw fd is not yet wrapped in an +`os.File` at that point — must `unix.Close(ifd)` manually on that one path: +the seam between raw-fd code and RAII-style ownership is exactly one line +wide, and it is the line you forget. Rust's `anyhow::Context` builds the +same chain declaratively (`.with_context(|| dir.to_string())`), printed with +`{err:#}` to get `context: cause` in one line. Runtime errors inside the +loop split by kind: `EINTR` is not an error (retry the wait — explicitly in +C++/Rust, invisibly in Go), `EAGAIN` on the inotify fd is the *success* case +"queue drained", and anything else aborts with context. Usage errors exit 2, +runtime failures exit 1, clean shutdown — timeout or signal — exits 0; all +three verified below. + +### Concurrency lens + +Count the threads doing *your* work: one in C++, one in Rust — and one +goroutine plus `main` in Go, multiplexed by the runtime onto its thread +pool. The single-threaded epoll design is not an accident, it is the +concurrency *strategy*: since only one thread touches `pending`, there are +no locks, no atomics, and no `EPOLLONESHOT` re-arm protocol; the event loop +serializes everything by construction. Go reaches the same freedom-from-locks +differently — `pending` is confined to the `select` goroutine, and other +concerns communicate by channel — which is Chapter 6's "share memory by +communicating" applied to I/O. The costs surface in `strace`: idle for 400 ms, +the C++ watcher makes exactly **1** `epoll_wait` call (armed once, sleeps, +fires at the deadline), while the Go binary makes **10** `epoll_pwait` calls +across its threads — the netpoller is consulted by the scheduler, not just +by your `select`, and no `timerfd_create` or `signalfd4` ever appears +because Go implements timers in the runtime and signals via handlers feeding +channels. Neither number is wrong; one design spends syscalls to buy an +abstraction, the other spends explicitness to buy syscall silence. + +## Build, run, observe + +```bash +[host]$ cd examples/09-event-loops-epoll-inotify && ./demo.sh +``` + +Each language builds and runs a 1-second self-watch. For a run you control, +background a watcher and touch files under it: + +```bash +[host]$ WD=$(mktemp -d) +[host]$ ./cpp/build/release/app watch "$WD" --timeout-ms 1500 & +[host]$ echo hi > "$WD/a.txt"; echo more >> "$WD/a.txt"; sleep 0.4; rm "$WD/a.txt"; wait +``` + +Observed output, all three languages identical: + +``` +event: created a.txt +event: deleted a.txt +fwatch: exiting (timeout) +``` + +Two lines, not four: the create and the immediate append landed in one 100 ms +debounce window and `merge` kept `created`; the `rm` 400 ms later was its own +window. Send SIGINT instead of waiting and the last line becomes +`fwatch: exiting (signal)` (observed, exit 0). The book's harness agrees on +the full behavior matrix — `python3 scripts/test-all-examples.py --only +09-event-loops-epoll-inotify` reports `PASS` for cpp, go, and rust +(3 passed, 0 failed, 0 skipped). + +## Cross-check: the loop under strace, and epoll's own bookkeeping + +The chapter's claims are checkable from outside. First, the C++ watcher +under this book's strace build (`scratchpad/usr/bin/strace`, v7.1), during +exactly the create/append/delete session above; trimmed, long paths elided: + +```bash +[host]$ strace -e trace=epoll_wait,inotify_add_watch,read ./cpp/build/release/app watch "$WD" --timeout-ms 1500 +inotify_add_watch(3, "/tmp/…/fw-demo", IN_MODIFY|IN_ATTRIB|IN_MOVED_FROM|IN_MOVED_TO|IN_CREATE|IN_DELETE) = 1 +epoll_wait(6, [{events=EPOLLIN, data=0}], 8, -1) = 1 +read(3, "\1\0\0\0\0\1\0\0\0\0\0\0\20\0\0\0a.txt\0\0\0\0\0\0\0\0\0\0\0"..., 4096) = 64 +read(3, 0x7ffcfe3fba00, 4096) = -1 EAGAIN (Resource temporarily unavailable) +epoll_wait(6, [{events=EPOLLIN, data=0x1}], 8, -1) = 1 +read(4, "\1\0\0\0\0\0\0\0", 8) = 8 +epoll_wait(6, [{events=EPOLLIN, data=0}], 8, -1) = 1 +read(3, "\1\0\0\0\0\2\0\0\0\0\0\0\20\0\0\0a.txt\0\0\0\0\0\0\0\0\0\0\0", 4096) = 32 +read(3, 0x7ffcfe3fba00, 4096) = -1 EAGAIN (Resource temporarily unavailable) +epoll_wait(6, [{events=EPOLLIN, data=0x1}], 8, -1) = 1 +read(4, "\1\0\0\0\0\0\0\0", 8) = 8 +epoll_wait(6, [{events=EPOLLIN, data=0x1}], 8, -1) = 1 +read(4, "\1\0\0\0\0\0\0\0", 8) = 8 ++++ exited with 0 +++ +``` + +Every mechanism from this chapter is on one screen. The `data=0` wakeups are +the inotify token, `data=0x1` the timer token. The first inotify drain +returns **64 bytes — two packed records in one read** (the create, mask +`\0\1\0\0` = `IN_CREATE`, and the append, coalesced by our merge into one +output line), followed by the mandatory `EAGAIN`. The delete arrives later as +one 32-byte record with mask `\0\2\0\0` = `IN_DELETE`. Three timer wakeups, +three 8-byte expiration reads: two debounce flushes and the final deadline. +Total `epoll_wait` calls for the whole session: five. + +Second — and this is the part few people ever look at — epoll's interest +list is inspectable. With a watcher running (PID via `$!`): + +```bash +[host]$ ls -l /proc/$PID/fd | tail -4 +lr-x------ 1 rsedor rsedor 64 Jul 18 23:06 3 -> anon_inode:inotify +lrwx------ 1 rsedor rsedor 64 Jul 18 23:06 4 -> anon_inode:[timerfd] +lrwx------ 1 rsedor rsedor 64 Jul 18 23:06 5 -> anon_inode:[signalfd] +lrwx------ 1 rsedor rsedor 64 Jul 18 23:06 6 -> anon_inode:[eventpoll] +[host]$ cat /proc/$PID/fdinfo/6 +tfd: 3 events: 19 data: 0 pos:0 ino:426 sdev:11 +tfd: 4 events: 19 data: 1 pos:0 ino:426 sdev:11 +tfd: 5 events: 19 data: 2 pos:0 ino:426 sdev:11 +[host]$ cat /proc/$PID/fdinfo/3 +inotify wd:1 ino:142a5 sdev:35 mask:3c6 ignored_mask:0 fhandle-bytes:c fhandle-type:1 f_handle:6945b24ea542010000000000 +[host]$ cat /proc/$PID/fdinfo/4 +clockid: 1 +ticks: 0 +it_value: (2, 694650159) +``` + +The eventpoll fdinfo *is* Figure 9.1: three `tfd` (target fd) rows, our +tokens 0/1/2 in the `data` column, `events: 19` = `EPOLLIN|EPOLLERR|EPOLLHUP` +(the error bits are always reported; you never register them). The inotify +fdinfo shows `wd:1`, our exact mask `3c6`, and the watched inode `142a5` — +hex for 82597, which `stat -c '%i'` on the watched directory confirms. The +timerfd fdinfo shows `clockid: 1` (`CLOCK_MONOTONIC`) and, caught mid-run, +`it_value: (2, 694650159)` — about 2.69 s remaining on a 3000 ms run, i.e. +armed to the overall deadline because no debounce was pending. + +Third, the Go binary on the same session, tracing what *our code never +wrote*: + +```bash +[host]$ strace -f -e trace=epoll_create1,epoll_ctl,epoll_pwait ./go/bin/app watch "$WD" --timeout-ms 800 +epoll_create1(EPOLL_CLOEXEC) = 5 +epoll_ctl(5, EPOLL_CTL_ADD, 6, {events=EPOLLIN, data=0x5e0998}) = 0 +epoll_ctl(5, EPOLL_CTL_ADD, 4, {events=EPOLLIN|EPOLLOUT|EPOLLRDHUP|EPOLLET, data=0x3fb6883027000001}) = 0 +epoll_pwait(5, [], 128, 0, NULL, 0) = 0 +epoll_pwait(5, [], 128, 799, NULL, 0) = 0 +``` + +The runtime built its own epoll instance, registered its wakeup fd, then +added our inotify fd (4) **edge-triggered** with an opaque runtime cookie +where we used token integers — 9 `epoll_pwait` calls in this run, issued +from several threads, none from `main.go`. The idle comparison quantifies +the two architectures: over a 400 ms idle watch, `strace -c` shows the C++ +binary making exactly 1 `epoll_wait`, 1 `epoll_create1`, 1 `timerfd_create`, +1 `timerfd_settime`, 1 `signalfd4`, and 1 `inotify_add_watch`; the Rust +binary is syscall-for-syscall identical (the same six calls, once each); the +Go binary makes 10 `epoll_pwait` and zero timerfd/signalfd calls. Two tools +— the trace and `/proc` — one consistent story: readiness is a kernel fact, +and both surfaces are views of it. + +

unverified — On the +lab VM: the kernel-side view of this loop — watching wakeup sources +fire with bpftrace on tracepoint:syscalls:sys_enter_epoll_wait +and kprobes on the eventpoll internals, and forcing a real +IN_Q_OVERFLOW by shrinking max_queued_events — +needs root and a disposable kernel, so it is not run here; chapter 30 +(Debugging part) exercises exactly this on systems-target.

+ +## What you learned + +- Readiness decouples waiting from reading: nonblocking fds, one + `epoll_wait`, and a kernel-side interest list you can literally read back + from `/proc//fdinfo/` — tokens, masks, and all. +- Level triggering plus "consume what woke you" is the book's default; + edge demands drain-to-`EAGAIN` always (Go's netpoller does exactly that, + registering with `EPOLLET`), and `EPOLLONESHOT` exists for the + multi-threaded case this design avoids. +- inotify queues events per instance with per-watch `wd`s and a finite + queue (16384 here) that overflows into `IN_Q_OVERFLOW` — the signal to + fall back to `snapshot`/`diff`; watches do not recurse. +- timerfd and signalfd turn timeouts and SIGINT into readable fds — one + re-armed one-shot timer served both the debounce and the deadline, and a + blocked-then-read signal became an ordinary dispatch case. +- Go's goroutine + channels + `select` is the same machinery inverted: 6 + explicit setup syscalls and 1 wait for C++/Rust idle, versus 10 runtime + `epoll_pwait`s and no timerfd/signalfd for Go. + +Next, **io_uring**: the completion model — instead of the kernel telling you +an fd is *ready* so you can do the work, you submit the work and the kernel +tells you it is *done*. + +--- + +

verified — all evidence in +this chapter was produced on the Fedora 44 reference host (kernel +7.1.3-200.fc44, strace 7.1) this session: the runner reported +PASS for cpp, go, and rust (3 passed, 0 failed, 0 skipped); the +create/append/delete session printed exactly event: created +a.txt, event: deleted a.txt, fwatch: exiting +(timeout), and a SIGINT run printed fwatch: exiting +(signal) with exit 0 (missing dir exited 1, bad usage 2); the C++ +strace excerpt is real trimmed output (5 epoll_waits, the +64-byte two-record inotify read, the EAGAINs, three 8-byte +timerfd reads); the /proc quotes are real (tfd +rows 3/4/5 with data 0/1/2, mask:3c6, ino:142a5 +matching stat -c '%i' = 82597, clockid: 1, +it_value: (2, 694650159)); the Go trace showed +epoll_ctl … EPOLLIN|EPOLLOUT|EPOLLRDHUP|EPOLLET on the inotify +fd and 9 epoll_pwaits (10 in the idle -c run, with +0 timerfd/signalfd syscalls), while C++ and Rust each made exactly one of +each setup call and one wait when idle; and +/proc/sys/fs/inotify read 16384 / 524288 / 512. Not run here: +the bpftrace/overflow experiment (lab-VM callout above).

diff --git a/_docs/10-io-uring.md b/_docs/10-io-uring.md new file mode 100644 index 0000000..74f1397 --- /dev/null +++ b/_docs/10-io-uring.md @@ -0,0 +1,523 @@ +--- +title: "io_uring: the completion model" +order: 10 +part: "Files and I/O" +description: "fwatch grows a batched sync engine and meets the second of the two I/O mental models — completion. SQ and CQ rings shared with the kernel, one io_uring_enter carrying sixteen operations, real strace counts showing the syscall collapse, and real timings showing why fewer syscalls is not automatically faster." +duration: 60 minutes +--- + +Chapter 9 ended with `fwatch` blocked in an event loop, and with a promise: +epoll would not be the last word on Linux I/O. Epoll answers one question — +*which of these fds may I touch without blocking?* — and then leaves every +actual `read(2)` and `write(2)` for you to make, one syscall per operation. +This chapter adds `fwatch sync`, a small directory-copy subcommand with two +engines: `--engine rw`, the plain read/write loop you already know, and +`--engine uring`, which never issues a data `read(2)` or `write(2)` at all. +Instead it *describes* batches of operations in memory the kernel shares with +the process, and submits sixteen of them per syscall. That is the +**completion model**, and readiness-versus-completion is the mental-model +pair this whole Part has been building toward. The chapter also delivers two +deliberate surprises: our uring engine is measurably *slower* on this +workload, and one of our three languages refuses to play at all — both on +purpose, both worth understanding. + +The code is in `examples/10-io-uring/`. `./demo.sh` there builds and runs all +three implementations; its `README.md` covers the `scan`/`watch`/`sync` +subcommands and the two-engine contract. + +{% include excalidraw.html + file="10-uring-rings" + alt="Three columns: user-space fwatch code filling SQEs and popping CQEs, the mmap'd shared memory holding the SQ ring, SQE array, and CQ ring, and the kernel consuming submissions and posting completions, with io_uring_enter as the single syscall between them." + caption="Figure 10.1 — the io_uring anatomy: two rings and an SQE array shared via mmap; io_uring_enter is the only syscall in the data path" %} + +> **Tools used** — `strace`, `diff`, `cmp`, `grep`, `python3` (host); +> `bpftrace` (lab VM, exercised in Part 8). Everything here is checked by +> `scripts/check-host.sh`, ships with Fedora, or is preinstalled in the lab VMs. + +## Readiness asks "may I?", completion says "it is done" + +Every I/O interface you have met so far is synchronous at the moment of data +movement: `read(2)` in chapter 7, `write(2)` through the page cache in +chapter 8, and epoll in chapter 9 — which, for all its scalability, only +*schedules* those same calls. Readiness is a hint about the future ("fd 7 +will not block right now"), and it can even be wrong: a readable socket can +still return `EAGAIN` by the time you get there. + +io_uring inverts the contract. You write a **submission queue entry** (SQE) — +a 64-byte struct naming an opcode (`IORING_OP_READ`), an fd, a buffer +address, a length, and a file offset — into an array the kernel mapped into +your address space. You publish it by advancing the tail index of the +**submission queue** (SQ) ring. Some time later the kernel leaves a +**completion queue entry** (CQE) in the CQ ring: `res` is the operation's +result (bytes transferred, or a negative errno), and `user_data` is an opaque +value you attached to the SQE so you can match the completion back to the +request. The syscall `io_uring_enter(2)` exists only to say "I have queued N +things; wake me when M of them are done" — and with `IORING_ENTER_GETEVENTS` +it does both halves in one crossing. One syscall can therefore carry any +number of operations, which is the entire economic argument: the fixed cost +of entering the kernel is amortized across a batch. There is even a mode that +eliminates the syscall entirely — `IORING_SETUP_SQPOLL` starts a kernel +thread that polls the SQ ring so a busy application never traps at all. We +mention it and move on: our engine does not use it, and the manpage itself +warns that it "may sound immediately appealing as an automatic go-faster +flag" but must be evaluated case by case. Hold that thought; our numbers +below make the same point without SQPOLL. + +{% include excalidraw.html + file="10-readiness-vs-completion" + alt="Two columns. Left, readiness: epoll_wait asks which fds are ready, the kernel reports readiness events, the application still performs read and write itself, and the operation can still fail with EAGAIN. Right, completion: the application queues SQEs describing the I/O, one io_uring_enter submits the batch, the kernel performs the I/O inline or via io-wq workers, and CQEs carry the finished results." + caption="Figure 10.2 — the mental-model pair: readiness (epoll) hints per fd; completion (io_uring) reports finished results per operation" %} + +## Where uring wins — and why containers say no + +The batch economics pay off where per-operation syscall overhead is a real +fraction of the work: thousands of small operations in flight against +storage that benefits from deep queues (`O_DIRECT` on NVMe), or servers +juggling enormous socket counts, where features like registered buffers and +multishot operations remove per-op setup cost too. They pay off *least* when +each operation is already almost free — and a buffered read of a page-cache-hot +file is a memcpy. Small trees of warm files are exactly that, which is why +the timings below look the way they do. + +The same power has a security price. io_uring is a large, fast-moving kernel +attack surface, and it lets a process express nearly every syscall family +through one opcode-dispatched interface — which quietly bypasses syscall +filters that were written in terms of `read`/`write`/`connect`. Container +runtimes responded plainly: on this host, podman's default seccomp profile +(`/usr/share/containers/seccomp.json`) sets `"defaultAction": "SCMP_ACT_ERRNO"` +and its allowlist contains **zero** mentions of `io_uring_setup`, +`io_uring_enter`, or `io_uring_register` — deny by default. Run the uring +engine in an unmodified container and `io_uring_setup` fails with `EPERM`. +Chapter 34, when our programs move into containers, deals with that head-on. + +## Go's deliberate absence + +`fwatch sync --engine uring` in Go prints +`engine=uring: unsupported in Go (see chapter 10)` and exits 64. That is not +laziness — it is the chapter's design lesson. Go already solved "many I/O +operations, few OS threads" once, inside the runtime: every goroutine writes +ordinary blocking code, and the **netpoller** multiplexes those goroutines +over epoll, parking them until readiness arrives. The concurrency model *is* +the I/O model. Grafting a completion engine underneath would mean teaching +the scheduler that some I/O finishes on rings instead of unparking on +readiness, deciding who owns the buffers while the kernel holds them, and +doing it all without breaking the promise that `f.Read(buf)` just works. The +tracking issue the source comment cites — golang/go#31908, open since 2019 — +remains on hold for exactly these reasons. Rust, at the other extreme, ships +io_uring as a library (`io-uring`, the crate our Rust engine uses) and lets +runtimes compete; C++ gives you the raw syscalls and a free hand. Three +languages, three answers to the same question: *who owns the event loop?* + +> **Research note — io_uring on this kernel, from primary sources.** Claims +> about io_uring age fast, so this chapter cites only what this host shows. +> Kernel `7.1.3-200.fc44` reported 18 `IORING_FEAT_*` flags in the +> `io_uring_setup` params during our traced run — from `SINGLE_MMAP` and +> `NODROP` through `RECVSEND_BUNDLE`, `MIN_TIMEOUT`, `RW_ATTR`, and +> `NO_IOWAIT` — and the shipped `io_uring(7)`, `io_uring_setup(2)`, and +> `io_uring_enter(2)` manpages document the setup/enter contract used here. +> Feature lists in older articles (and newer marketing) routinely disagree +> with both; when in doubt, trust the flags your kernel returns. + +## How the code works + +`fwatch` v2 carries the earlier chapters' machinery forward — the v0 +polling walk survives as `scan` (now a one-line summary; the per-file +`snapshot`/`diff` pair stays with v1), v1's `watch` keeps its name — and +adds `cmd_sync`: walk the source tree, recreate directories, copy every +regular file, print `synced files bytes engine= ms=`. The +engines differ only in how one file's bytes move. + +The C++ engine is the pedagogical core, because the host has no +`liburing-devel` and we turned that into a feature: a self-contained `Ring` +class built on raw `syscall(2)` wrappers, so every moving part is visible. +`Ring::create` calls `io_uring_setup(2)` with 64 entries, then performs the +three canonical mmaps of the returned fd — the SQ ring (head, tail, mask, +and an index array), the CQ ring (head, tail, and the CQE array), and the +SQE array itself — honoring `IORING_FEAT_SINGLE_MMAP`, which on modern +kernels folds the two rings into one mapping (our trace below shows exactly +that: one 2368-byte mapping at offset 0, one SQE mapping at +`IORING_OFF_SQES`). The interesting discipline is in the hot-path methods, +from `examples/10-io-uring/cpp/src/main.cpp`: + +```cpp + // Claim the next SQE slot (zeroed); nullptr when the SQ ring is full. + [[nodiscard]] io_uring_sqe* try_get_sqe() { + const unsigned head = std::atomic_ref(*sq_head_).load(std::memory_order_acquire); + if (local_tail_ - head > sq_mask_) { + return nullptr; + } + const unsigned idx = local_tail_ & sq_mask_; + io_uring_sqe* sqe = &sqes_[idx]; + std::memset(sqe, 0, sizeof(*sqe)); + sq_array_[idx] = idx; + ++local_tail_; + ++to_submit_; + return sqe; + } + + // Publish queued SQEs and wait for wait_nr completions in one syscall. + [[nodiscard]] expected submit_and_wait(unsigned wait_nr) { + std::atomic_ref(*sq_tail_).store(local_tail_, std::memory_order_release); + return enter(std::exchange(to_submit_, 0), wait_nr); + } +``` + +`try_get_sqe` load-acquires the SQ *head* — the kernel's consumption cursor — +to learn how much room remains; the ring is full when our private +`local_tail_` has run a whole ring-size ahead of it. Filled entries stay +private until `submit_and_wait` store-releases the shared *tail*, which is +the publish: everything written to the SQEs before that store is guaranteed +visible to the kernel after it. The mask trick (`local_tail_ & sq_mask_`) +works because ring sizes are powers of two, so a free-running 32-bit counter +maps onto a slot without ever being reset — the same idiom you will meet +again in the lock-free chapter. + +On top of the ring, each engine copies one file the same way — here is the +same function three times: + +{% include codetabs.html langs="C++|Go|Rust" %} + +```cpp +[[nodiscard]] expected copy_file_uring(Ring& ring, const fs::path& src, + const fs::path& dst) { + auto in = Fd::open(src, O_RDONLY | O_CLOEXEC); + if (!in) { + return unexpected(in.error()); + } + auto out = Fd::open(dst, O_WRONLY | O_CREAT | O_TRUNC | O_CLOEXEC, 0644); + if (!out) { + return unexpected(out.error()); + } + struct stat st{}; + if (::fstat(in->get(), &st) != 0) { + return unexpected(errno_ec()); + } + const auto size = static_cast(st.st_size); + + std::array, kPairs> bufs; + for (auto& b : bufs) { + b.resize(kChunk); + } + + uint64_t off = 0; + while (off < size) { + std::array lens{}; + unsigned pairs = 0; + while (pairs < kPairs && off < size) { + const auto len = static_cast(std::min(kChunk, size - off)); + lens[pairs] = len; + io_uring_sqe* rd = ring.try_get_sqe(); + io_uring_sqe* wr = rd != nullptr ? ring.try_get_sqe() : nullptr; + if (rd == nullptr || wr == nullptr) { + return unexpected(std::make_error_code(std::errc::no_buffer_space)); + } + // READ this chunk into the pair's buffer... + rd->opcode = IORING_OP_READ; + rd->fd = in->get(); + rd->addr = reinterpret_cast(bufs[pairs].data()); + rd->len = len; + rd->off = off; + rd->flags = IOSQE_IO_LINK; // ...then, only if it fully succeeded, + rd->user_data = uint64_t{pairs} << 1U; + // ...WRITE the same buffer at the same offset in dst. + wr->opcode = IORING_OP_WRITE; + wr->fd = out->get(); + wr->addr = reinterpret_cast(bufs[pairs].data()); + wr->len = len; + wr->off = off; + wr->user_data = (uint64_t{pairs} << 1U) | 1U; + off += len; + ++pairs; + } + if (auto s = ring.submit_and_wait(2 * pairs); !s) { + return unexpected(s.error()); + } + bool short_io = false; + auto d = ring.drain(2 * pairs, [&](const io_uring_cqe& cqe) { + const auto slot = static_cast(cqe.user_data >> 1U); + if (cqe.res < 0 || static_cast(cqe.res) != lens[slot]) { + short_io = true; + } + }); + if (!d) { + return unexpected(d.error()); + } + if (short_io) { + return unexpected(std::make_error_code(std::errc::io_error)); + } + } + return size; +} +``` + +```go +func cmdSync(src, dst, engine string) int { + if engine == "uring" { + fmt.Fprintln(os.Stderr, "engine=uring: unsupported in Go (see chapter 10)") + return 64 + } + start := time.Now() + if err := os.MkdirAll(dst, 0o755); err != nil { + fmt.Fprintf(os.Stderr, "fwatch: %v\n", err) + return 1 + } + var files, bytes uint64 + err := walk(src, func(path string, d fs.DirEntry) error { + rel, err := filepath.Rel(src, path) + if err != nil { + return fmt.Errorf("rel %s: %w", path, err) + } + target := filepath.Join(dst, rel) + switch { + case d.IsDir(): + return os.MkdirAll(target, 0o755) + case d.Type().IsRegular(): + n, err := copyFileRW(path, target) + if err != nil { + return err + } + files++ + bytes += n + } + return nil + }) + if err != nil { + fmt.Fprintf(os.Stderr, "fwatch: %v\n", err) + return 1 + } + ms := time.Since(start).Milliseconds() + fmt.Printf("synced %d files %d bytes engine=%s ms=%d\n", files, bytes, engine, ms) + return 0 +} +``` + +```rust +/// io_uring copy: batches of linked READ->WRITE pairs at explicit offsets. +fn copy_file_uring(ring: &mut IoUring, src: &Path, dst: &Path) -> Result { + let input = File::open(src).with_context(|| format!("open {}", src.display()))?; + let output = File::create(dst).with_context(|| format!("create {}", dst.display()))?; + let size = input.metadata()?.len(); + + let mut bufs = vec![vec![0u8; CHUNK as usize]; PAIRS]; + let mut off = 0u64; + while off < size { + let mut lens = [0u32; PAIRS]; + let mut pairs = 0usize; + { + let mut sq = ring.submission(); + while pairs < PAIRS && off < size { + let len = CHUNK.min(size - off) as u32; + lens[pairs] = len; + let buf = bufs[pairs].as_mut_ptr(); + // READ this chunk into the pair's buffer, then — only if it + // fully succeeded (IO_LINK) — WRITE it at the same offset. + let read = opcode::Read::new(types::Fd(input.as_raw_fd()), buf, len) + .offset(off) + .build() + .flags(squeue::Flags::IO_LINK) + .user_data((pairs as u64) << 1); + let write = opcode::Write::new(types::Fd(output.as_raw_fd()), buf, len) + .offset(off) + .build() + .user_data(((pairs as u64) << 1) | 1); + // SAFETY: the buffers and fds outlive the submissions; we wait + // for all completions before the next loop iteration. + unsafe { + sq.push(&read).map_err(|_| anyhow!("submission queue full"))?; + sq.push(&write).map_err(|_| anyhow!("submission queue full"))?; + } + off += u64::from(len); + pairs += 1; + } + } // drop the SQ view: syncs the tail before entering the kernel + ring.submit_and_wait(2 * pairs) + .context("io_uring_enter")?; + let mut seen = 0usize; + while seen < 2 * pairs { + for cqe in ring.completion() { + let slot = (cqe.user_data() >> 1) as usize; + if cqe.result() < 0 || cqe.result() as u32 != lens[slot] { + bail!( + "io_uring short op on {}: got {} want {}", + src.display(), + cqe.result(), + lens[slot] + ); + } + seen += 1; + } + if seen < 2 * pairs { + ring.submit_and_wait(2 * pairs - seen).context("io_uring_enter")?; + } + } + } + Ok(size) +} +``` + +The design is identical in C++ and Rust: for each 128 KiB chunk, queue a +READ into a per-slot buffer and a WRITE of that same buffer at the same +destination offset, and chain them with `IOSQE_IO_LINK` — the kernel starts +the WRITE only after its READ fully succeeded, so ordering *within a pair* +needs no round-trip to userspace. Up to eight pairs (sixteen SQEs, which is +why the ring is sized 64) go into one `submit_and_wait(2 * pairs)`. Every +operation carries an explicit `off` — completions can arrive in any order, +so nothing may depend on a shared file position; `user_data` encodes the +slot (and read/write bit) so each CQE can be checked against the length that +was requested. `drain` then insists on *exactly* `2 * pairs` CQEs, +re-entering the kernel if a wait returned early — a wait can be satisfied +before every linked pair has retired. Rust's version is the same shape on +the `io-uring` crate, which owns the setup and barriers (dropping the +`submission()` view is the tail-publish); the `unsafe` blocks mark the one +contract the compiler cannot check — the kernel holds raw pointers into +`bufs` until we have drained the completions. The fragile bits are +deliberate and visible: `kChunk`/`kPairs` are hardcoded tuning knobs, the +batch waits for *all* its completions before starting the next (no +cross-file pipelining), and the buffers are plain allocations, not +registered buffers — all three are exactly where a production engine would +diverge, and the numbers below show the cost. + +### Errors, three ways + +The completion model moves error handling: failures are no longer a `-1` and +an ambient `errno` at the call site, but a negative errno *value* inside +each CQE, delivered whenever the kernel got around to that operation. The +C++ engine folds every CQE through one check — `cqe.res < 0 || +cqe.res != lens[slot]` — treating short transfers as errors, and returns +`std::errc::io_error` through the same `expected<…, error_code>` channel as +`open` failures; a full SQ ring is `std::errc::no_buffer_space`, and only +`io_uring_enter` itself can still fail the old way (its `EINTR` is retried +in `Ring::enter`). Rust does the same per-CQE audit but `bail!`s with the +got/want lengths, so the error message carries the evidence. And a linked +pair fails as a unit: if a READ comes up short, the kernel cancels its +linked WRITE with `-ECANCELED` — one more CQE, one more `res < 0`, caught by +the same check. Go's error story here is the bluntest and the most +interesting: `exit 64`, refusing at the front door, because the runtime's +contract cannot express the engine at all. + +### Concurrency lens + +Strip away the I/O and Figure 10.1 is a pair of single-producer, +single-consumer lock-free ring buffers — the kernel is simply *the other +thread*. That is why `try_get_sqe`/`submit_and_wait` are written in +acquire/release vocabulary: the store-release of the SQ tail publishes the +filled SQEs, the kernel's load-acquire observes them, and the mirror-image +pair guards the CQ ring in the other direction (`pop` load-acquires the CQ +tail before reading a CQE). `std::atomic_ref` is doing real work here — the +ring indices are plain `unsigned` fields in shared memory laid out by the +kernel, and `atomic_ref` lets us perform atomic operations on memory we do +not get to declare `std::atomic`. Meanwhile the kernel side is concurrent +too: `IORING_FEAT_NATIVE_WORKERS` in our trace is the io-wq worker-thread +pool, which executes any submitted operation that cannot complete +immediately — our buffered file writes among them, a fact that returns in +the timings. Chapter 26 rebuilds this exact structure in user space, barrier +by barrier. + +## Build, run, observe + +```bash +[host]$ cd examples/10-io-uring && ./demo.sh build +[host]$ ./demo.sh cpp run sync --engine uring +``` + +For real numbers, this session generated a scratch tree — 403 files, +19,507,360 bytes: 400 small files of 4–64 KiB across eight directories, +plus three larger files (about 1 MiB, 3 MiB, and 1.1 MiB) sized to straddle +the 128 KiB chunk and the 8-pair batch. Three runs per engine, warm page +cache, `ms=` as printed by the tool: + +| engine | C++ | Rust | Go | +|---|---|---|---| +| `rw` | 11, 12, 11 ms | 13, 12, 12 ms | 21 ms | +| `uring` | 129, 137, 130 ms | 105, 107, 107 ms | exit 64 (by design) | + +Read that plainly: on this workload our uring engine is roughly **ten times +slower**. Nothing is broken — the copies are byte-identical (checked below) +— the model just costs more than it saves here. Every file is page-cache +hot, so each `read(2)` in the rw loop is a short memcpy; the syscalls we so +cleverly eliminated were never the bottleneck. The uring path, meanwhile, +pays for what buffered file I/O cannot do asynchronously inline — the +kernel punts those operations to io-wq worker threads, buying context +switches and wakeups per batch — and our engine waits for each batch to +fully retire before building the next, with unregistered buffers reallocated +per file. This is the batch-size lesson the chapter owes you: completion +wins when queues are deep, operations are genuinely asynchronous, and fixed +costs get amortized across real waiting — not when the "I/O" is a memcpy +that was already cheaper than the bookkeeping. On the small verify fixture +(5 files, 1,348,634 bytes) the gap shrinks to noise: `rw` printed `ms=0`, +uring `ms=2`–`3`. + +## Cross-check: the syscall collapse, counted + +Timing said uring lost; `strace -c` proves it did so while *keeping its +promise* — the promise was fewer syscalls, not faster memcpys. Same tree, +C++ binary, both engines: + +```bash +[host]$ strace -c ./cpp/build/release/app sync /src /dst --engine rw +``` + +The rw run: **850 `read`**, **444 `write`**, 820 `openat` — 3,467 calls +total. The uring run: **407 `io_uring_enter`**, 1 `io_uring_setup`, and the +only remaining `read`/`write` calls (4/1) belong to the dynamic loader. The +arithmetic closes exactly: 400 single-batch small files, plus 2 + 3 + 2 +batches for the three large files (9, 24, and 10 chunks at 8 pairs per +batch) = 407 enters carrying all 19.5 MB. On the 5-file fixture the same +count is 5 enters — one per small file, two for the 9-chunk `d.bin`, zero +for the empty file. The uring table also shows 3,216 `brk` calls — the +allocator churn of building eight 128 KiB buffers per file — a bonus lesson +in how `strace -c` catches *your* overhead, not just the kernel's. + +The setup trace shows the anatomy of Figure 10.1 verbatim +(`strace -e trace=io_uring_setup,io_uring_enter,mmap`, trimmed): + +``` +io_uring_setup(64, {flags=0, sq_entries=64, cq_entries=128, + features=IORING_FEAT_SINGLE_MMAP|IORING_FEAT_NODROP|...|IORING_FEAT_NO_IOWAIT, + sq_off={head=0, tail=4, ...}, cq_off={head=8, tail=12, ...}}) = 3 +mmap(NULL, 2368, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_POPULATE, 3, 0) = 0x7f508be1b000 +mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_POPULATE, 3, 0x10000000) = 0x7f508be1a000 +``` + +One mapping at offset 0 holds both rings (`SINGLE_MMAP`); the second, at +`IORING_OFF_SQES` (0x10000000), is the SQE array. Correctness got its own +independent check: `diff -r` on the uring-copied tree reported no +differences, and `cmp` on the copied `d.bin` (1,048,581 bytes, the +batch-boundary-straddling file) was byte-identical. Finally, the security +claim above is a one-liner you can run yourself: `grep -c io_uring +/usr/share/containers/seccomp.json` prints `0` on this host, against a +`defaultAction` of `SCMP_ACT_ERRNO`. + +> **On the lab VM** — unverified: +> the natural next probe is watching io-wq from the inside — +> `bpftrace -e 'tracepoint:io_uring:io_uring_submit_req { @[comm] = count(); }'` +> while the two engines run. bcc/bpftrace are not part of this host's +> toolchain by design; chapter 30 (Debugging part) builds that eBPF +> observation kit on the lab VM and exercises exactly this. + +## What you learned + +- Readiness and completion are the two I/O mental models: epoll hints that + an fd *may* be usable; io_uring reports that an operation *finished*, with + its result in the CQE — and the SQ/SQE/CQ structures live in memory shared + via three well-known mmaps of the ring fd. +- One `io_uring_enter` can submit and reap a whole batch — 407 enters + replaced 1,294 data syscalls on our 403-file tree — and `IOSQE_IO_LINK` + expresses intra-pair ordering without a userspace round-trip. +- Fewer syscalls is not faster by itself: against a warm page cache our + uring engine ran ~10× slower than the rw loop (129 vs 11 ms), because + buffered-file completions ride io-wq worker threads and our batches are + small and fully synchronous — completion earns its keep at depth, on real + waiting. +- io_uring's power is why container seccomp profiles deny it by default + (podman's allowlist has no io_uring entries on this host — chapter 34), + and why Go declines to expose it at all: the netpoller already owns the + event loop (golang/go#31908 stays on hold). + +Next, Part 3 opens with the **process lifecycle** — `fork`, `exec`, and +`wait` — and the first cut of `pmon`, the process supervisor that stays with +us to the capstone. + +--- + +

verified — all numbers above are from this session's runs on the Fedora 44 host (kernel 7.1.3-200.fc44): the runner table passed 3/3 (cpp, go, rust); the 403-file/19,507,360-byte tree synced with engine=rw in 11–12 ms (C++) and 12–13 ms (Rust) versus 129–137 ms and 105–107 ms with engine=uring; strace -c counted 850 read + 444 write (rw) against 407 io_uring_enter + 1 io_uring_setup (uring, loader-only read/write 4/1); the fixture uring sync used 5 enters and printed synced 5 files 1348634 bytes engine=uring ms=3; the io_uring_setup/mmap trace, the diff -r/cmp byte-equality, the Go refusal (engine=uring: unsupported in Go (see chapter 10), exit 64), and the empty grep -c io_uring against podman's seccomp profile were all reproduced as quoted. Unverified: the bpftrace probe in the lab-VM callout, as marked.

diff --git a/assets/diagrams/04-runtime-syscall-layers.excalidraw b/assets/diagrams/04-runtime-syscall-layers.excalidraw new file mode 100644 index 0000000..322e1fd --- /dev/null +++ b/assets/diagrams/04-runtime-syscall-layers.excalidraw @@ -0,0 +1,1813 @@ +{ + "type": "excalidraw", + "version": 2, + "source": "https://excalidraw.com", + "elements": [ + { + "id": "r1059510276", + "type": "rectangle", + "x": 20, + "y": 20, + "width": 300, + "height": 330, + "angle": 0, + "strokeColor": "#dddddd", + "backgroundColor": "#fafafa", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1961323096, + "versionNonce": 1369088416, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t189082726", + "type": "text", + "x": 32, + "y": 26, + "width": 132.0, + "height": 15.0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 77617101, + "versionNonce": 173661994, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "C++ (dynamic, glibc)", + "fontSize": 12, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "C++ (dynamic, glibc)", + "lineHeight": 1.25, + "baseline": 12 + }, + { + "id": "r394904241", + "type": "rectangle", + "x": 330, + "y": 20, + "width": 300, + "height": 330, + "angle": 0, + "strokeColor": "#dddddd", + "backgroundColor": "#fafafa", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1326746992, + "versionNonce": 1016853206, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1212392161", + "type": "text", + "x": 342, + "y": 26, + "width": 158.4, + "height": 15.0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1046840687, + "versionNonce": 1163609938, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "Go (static, own runtime)", + "fontSize": 12, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "Go (static, own runtime)", + "lineHeight": 1.25, + "baseline": 12 + }, + { + "id": "r1109625020", + "type": "rectangle", + "x": 640, + "y": 20, + "width": 300, + "height": 330, + "angle": 0, + "strokeColor": "#dddddd", + "backgroundColor": "#fafafa", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 677748880, + "versionNonce": 1316167007, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t34910281", + "type": "text", + "x": 652, + "y": 26, + "width": 171.60000000000002, + "height": 15.0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1963947957, + "versionNonce": 1600877822, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "Rust (rustix, raw backend)", + "fontSize": 12, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "Rust (rustix, raw backend)", + "lineHeight": 1.25, + "baseline": 12 + }, + { + "id": "r1232198733", + "type": "rectangle", + "x": 45, + "y": 55, + "width": 250, + "height": 58, + "angle": 0, + "strokeColor": "#2f6db5", + "backgroundColor": "#eef4fb", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1011177040, + "versionNonce": 1109695590, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1514818722", + "type": "text", + "x": 55, + "y": 63, + "width": 61.60000000000001, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 711129428, + "versionNonce": 1495147986, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "main.cpp", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "main.cpp", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1777374346", + "type": "text", + "x": 55, + "y": 83, + "width": 193.60000000000002, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 233672808, + "versionNonce": 1350416053, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "::openat / ::write / ::getrandom", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "::openat / ::write / ::getrandom", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r828500509", + "type": "rectangle", + "x": 45, + "y": 135, + "width": 250, + "height": 58, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 570998137, + "versionNonce": 1214587537, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t818920021", + "type": "text", + "x": 55, + "y": 143, + "width": 200.20000000000002, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1841099465, + "versionNonce": 1054472099, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "glibc wrappers (libc.so.6)", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "glibc wrappers (libc.so.6)", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t665898006", + "type": "text", + "x": 55, + "y": 163, + "width": 175.45000000000002, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1133804426, + "versionNonce": 1196628347, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "errno, cancellation, vDSO use", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "errno, cancellation, vDSO use", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r254277989", + "type": "rectangle", + "x": 45, + "y": 215, + "width": 250, + "height": 58, + "angle": 0, + "strokeColor": "#e8870c", + "backgroundColor": "#fff8ef", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1479136092, + "versionNonce": 630538649, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1100289200", + "type": "text", + "x": 55, + "y": 223, + "width": 231.00000000000003, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1906778629, + "versionNonce": 1974174357, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "syscall insns in libc, not app", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "syscall insns in libc, not app", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t851083461", + "type": "text", + "x": 55, + "y": 243, + "width": 102.85000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1142267322, + "versionNonce": 2941422, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "objdump on app: 0", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "objdump on app: 0", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r461708546", + "type": "rectangle", + "x": 355, + "y": 55, + "width": 250, + "height": 58, + "angle": 0, + "strokeColor": "#2f6db5", + "backgroundColor": "#eef4fb", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1106864874, + "versionNonce": 1241124759, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t187702053", + "type": "text", + "x": 365, + "y": 63, + "width": 53.900000000000006, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 317819050, + "versionNonce": 777203422, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "main.go", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "main.go", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1086847496", + "type": "text", + "x": 365, + "y": 83, + "width": 127.05000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1820221335, + "versionNonce": 508310451, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "golang.org/x/sys/unix", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "golang.org/x/sys/unix", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r847138671", + "type": "rectangle", + "x": 355, + "y": 135, + "width": 250, + "height": 58, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 333233224, + "versionNonce": 1972101637, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t35606689", + "type": "text", + "x": 365, + "y": 143, + "width": 215.60000000000002, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1535004110, + "versionNonce": 1652746425, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "Go runtime: own ABI + stacks", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "Go runtime: own ABI + stacks", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t448048603", + "type": "text", + "x": 365, + "y": 163, + "width": 169.4, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1200433329, + "versionNonce": 855537051, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "Syscall6, runtime.vgetrandom", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "Syscall6, runtime.vgetrandom", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1290778073", + "type": "rectangle", + "x": 355, + "y": 215, + "width": 250, + "height": 58, + "angle": 0, + "strokeColor": "#e8870c", + "backgroundColor": "#fff8ef", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1428295677, + "versionNonce": 1394954097, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1025138695", + "type": "text", + "x": 365, + "y": 223, + "width": 231.00000000000003, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 768711847, + "versionNonce": 417371518, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "syscall insns in static binary", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "syscall insns in static binary", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t761542602", + "type": "text", + "x": 365, + "y": 243, + "width": 114.95, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 577398940, + "versionNonce": 351607873, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "objdump on app: 511", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "objdump on app: 511", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r310150986", + "type": "rectangle", + "x": 665, + "y": 55, + "width": 250, + "height": 58, + "angle": 0, + "strokeColor": "#2f6db5", + "backgroundColor": "#eef4fb", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1260930947, + "versionNonce": 690494818, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1838173048", + "type": "text", + "x": 675, + "y": 63, + "width": 53.900000000000006, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1731374175, + "versionNonce": 1880407203, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "main.rs", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "main.rs", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t168649835", + "type": "text", + "x": 675, + "y": 83, + "width": 187.55, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 443835761, + "versionNonce": 926339134, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "rustix::fs / io / thread / rand", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "rustix::fs / io / thread / rand", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r476500805", + "type": "rectangle", + "x": 665, + "y": 135, + "width": 250, + "height": 58, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 77147535, + "versionNonce": 634128867, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1893588721", + "type": "text", + "x": 675, + "y": 143, + "width": 184.8, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 188667500, + "versionNonce": 1895213071, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "rustix linux_raw backend", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "rustix linux_raw backend", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t814224198", + "type": "text", + "x": 675, + "y": 163, + "width": 199.65, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1991295630, + "versionNonce": 1926816132, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "Result<_, Errno>, no errno global", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "Result<_, Errno>, no errno global", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1113521742", + "type": "rectangle", + "x": 665, + "y": 215, + "width": 250, + "height": 58, + "angle": 0, + "strokeColor": "#e8870c", + "backgroundColor": "#fff8ef", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 376640755, + "versionNonce": 1715621625, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t515703820", + "type": "text", + "x": 675, + "y": 223, + "width": 215.60000000000002, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 245806633, + "versionNonce": 1919347631, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "syscall insns inlined in app", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "syscall insns inlined in app", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t34945878", + "type": "text", + "x": 675, + "y": 243, + "width": 108.9, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1717740296, + "versionNonce": 695753559, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "objdump on app: 32", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "objdump on app: 32", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1535435999", + "type": "rectangle", + "x": 45, + "y": 385, + "width": 870, + "height": 60, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#111111", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1637348784, + "versionNonce": 574319486, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1060409456", + "type": "text", + "x": 55, + "y": 393, + "width": 215.60000000000002, + "height": 17.5, + "angle": 0, + "strokeColor": "#ffffff", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1120104429, + "versionNonce": 1199276878, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "one kernel ABI for all three", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "one kernel ABI for all three", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t750471520", + "type": "text", + "x": 55, + "y": 413, + "width": 490.05, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 543354409, + "versionNonce": 1084966060, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "number in rax - args in rdi rsi rdx r10 r8 r9 - return in rax (negative = -errno)", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "number in rax - args in rdi rsi rdx r10 r8 r9 - return in rax (negative = -errno)", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a1393503229", + "type": "arrow", + "x": 170, + "y": 113, + "width": 0, + "height": 22, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 665728008, + "versionNonce": 255712331, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + 22 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "a1063161969", + "type": "arrow", + "x": 170, + "y": 193, + "width": 0, + "height": 22, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 338751929, + "versionNonce": 955763459, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + 22 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "a1816096783", + "type": "arrow", + "x": 170, + "y": 273, + "width": 0, + "height": 112, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 228158407, + "versionNonce": 1704349354, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + 112 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t694442309", + "type": "text", + "x": 170.0, + "y": 315.0, + "width": 48.400000000000006, + "height": 13.75, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1003761289, + "versionNonce": 806265242, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "70 calls", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "70 calls", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a835001045", + "type": "arrow", + "x": 480, + "y": 113, + "width": 0, + "height": 22, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 724236895, + "versionNonce": 761442035, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + 22 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "a690933158", + "type": "arrow", + "x": 480, + "y": 193, + "width": 0, + "height": 22, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1188755426, + "versionNonce": 247728275, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + 22 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "a1544031532", + "type": "arrow", + "x": 480, + "y": 273, + "width": 0, + "height": 112, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 465082352, + "versionNonce": 57439240, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + 112 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t1647659662", + "type": "text", + "x": 480.0, + "y": 315.0, + "width": 54.45, + "height": 13.75, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 876607452, + "versionNonce": 1759060737, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "283 calls", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "283 calls", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a232685954", + "type": "arrow", + "x": 790, + "y": 113, + "width": 0, + "height": 22, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 404676527, + "versionNonce": 1006105334, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + 22 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "a1488122948", + "type": "arrow", + "x": 790, + "y": 193, + "width": 0, + "height": 22, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1884665724, + "versionNonce": 1173159794, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + 22 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "a1682827917", + "type": "arrow", + "x": 790, + "y": 273, + "width": 0, + "height": 112, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 576034416, + "versionNonce": 432406679, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + 112 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t70394929", + "type": "text", + "x": 790.0, + "y": 315.0, + "width": 48.400000000000006, + "height": 13.75, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1785867163, + "versionNonce": 411821475, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "73 calls", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "73 calls", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t549926088", + "type": "text", + "x": 45, + "y": 310, + "width": 144.375, + "height": 13.125, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 964785815, + "versionNonce": 757370581, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "clock_nanosleep via glibc", + "fontSize": 10.5, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "clock_nanosleep via glibc", + "lineHeight": 1.25, + "baseline": 10.5 + }, + { + "id": "t1866698590", + "type": "text", + "x": 355, + "y": 310, + "width": 236.775, + "height": 13.125, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1173153768, + "versionNonce": 1784508462, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "raw nanosleep; 4 clones, 114 rt_sigaction", + "fontSize": 10.5, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "raw nanosleep; 4 clones, 114 rt_sigaction", + "lineHeight": 1.25, + "baseline": 10.5 + }, + { + "id": "t1941502310", + "type": "text", + "x": 665, + "y": 310, + "width": 207.9, + "height": 13.125, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1243259132, + "versionNonce": 1193365401, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "raw nanosleep; libc only for startup", + "fontSize": 10.5, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "raw nanosleep; libc only for startup", + "lineHeight": 1.25, + "baseline": 10.5 + }, + { + "id": "t1826661559", + "type": "text", + "x": 45, + "y": 362, + "width": 490.05, + "height": 13.75, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1986912287, + "versionNonce": 963608811, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "strace -c totals for one sysprobe run on the reference host (Go: -f, all threads)", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "strace -c totals for one sysprobe run on the reference host (Go: -f, all threads)", + "lineHeight": 1.25, + "baseline": 11 + } + ], + "appState": { + "viewBackgroundColor": "#ffffff", + "gridSize": null + }, + "files": {} +} \ No newline at end of file diff --git a/assets/diagrams/04-runtime-syscall-layers.svg b/assets/diagrams/04-runtime-syscall-layers.svg new file mode 100644 index 0000000..2e9799e --- /dev/null +++ b/assets/diagrams/04-runtime-syscall-layers.svg @@ -0,0 +1,58 @@ + + + +C++ (dynamic, glibc) + +Go (static, own runtime) + +Rust (rustix, raw backend) + +main.cpp +::openat / ::write / ::getrandom + +glibc wrappers (libc.so.6) +errno, cancellation, vDSO use + +syscall insns in libc, not app +objdump on app: 0 + +main.go +golang.org/x/sys/unix + +Go runtime: own ABI + stacks +Syscall6, runtime.vgetrandom + +syscall insns in static binary +objdump on app: 511 + +main.rs +rustix::fs / io / thread / rand + +rustix linux_raw backend +Result<_, Errno>, no errno global + +syscall insns inlined in app +objdump on app: 32 + +one kernel ABI for all three +number in rax - args in rdi rsi rdx r10 r8 r9 - return in rax (negative = -errno) + + + +70 calls +70 calls + + + +283 calls +283 calls + + + +73 calls +73 calls +clock_nanosleep via glibc +raw nanosleep; 4 clones, 114 rt_sigaction +raw nanosleep; libc only for startup +strace -c totals for one sysprobe run on the reference host (Go: -f, all threads) + \ No newline at end of file diff --git a/assets/diagrams/04-user-kernel-path.excalidraw b/assets/diagrams/04-user-kernel-path.excalidraw new file mode 100644 index 0000000..833748f --- /dev/null +++ b/assets/diagrams/04-user-kernel-path.excalidraw @@ -0,0 +1,1405 @@ +{ + "type": "excalidraw", + "version": 2, + "source": "https://excalidraw.com", + "elements": [ + { + "id": "r1267778046", + "type": "rectangle", + "x": 20, + "y": 20, + "width": 920, + "height": 210, + "angle": 0, + "strokeColor": "#dddddd", + "backgroundColor": "#eef4fb", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 557070538, + "versionNonce": 92364962, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1206515689", + "type": "text", + "x": 32, + "y": 26, + "width": 125.4, + "height": 15.0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1996247490, + "versionNonce": 1198487524, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "user space (ring 3)", + "fontSize": 12, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "user space (ring 3)", + "lineHeight": 1.25, + "baseline": 12 + }, + { + "id": "r206676000", + "type": "rectangle", + "x": 20, + "y": 250, + "width": 920, + "height": 150, + "angle": 0, + "strokeColor": "#dddddd", + "backgroundColor": "#f4f4f4", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1248301967, + "versionNonce": 690976128, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1559971535", + "type": "text", + "x": 32, + "y": 256, + "width": 99.00000000000001, + "height": 15.0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 15468869, + "versionNonce": 1004402376, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "kernel (ring 0)", + "fontSize": 12, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "kernel (ring 0)", + "lineHeight": 1.25, + "baseline": 12 + }, + { + "id": "r1606159990", + "type": "rectangle", + "x": 40, + "y": 60, + "width": 190, + "height": 70, + "angle": 0, + "strokeColor": "#2f6db5", + "backgroundColor": "#eef4fb", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1177203613, + "versionNonce": 980173345, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t716179673", + "type": "text", + "x": 50, + "y": 68, + "width": 69.30000000000001, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1356719396, + "versionNonce": 402061272, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "your code", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "your code", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t527953009", + "type": "text", + "x": 50, + "y": 88, + "width": 127.05000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 859839224, + "versionNonce": 904060019, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "getrandom(buf, 16, 0)", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "getrandom(buf, 16, 0)", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r920678395", + "type": "rectangle", + "x": 280, + "y": 60, + "width": 210, + "height": 70, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 371650489, + "versionNonce": 1514027629, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t681638292", + "type": "text", + "x": 290, + "y": 68, + "width": 123.20000000000002, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 973181970, + "versionNonce": 309908842, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "language wrapper", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "language wrapper", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1958953970", + "type": "text", + "x": 290, + "y": 88, + "width": 133.10000000000002, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1542225852, + "versionNonce": 680127779, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "glibc / x/sys / rustix", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "glibc / x/sys / rustix", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t822202332", + "type": "text", + "x": 290, + "y": 103, + "width": 163.35000000000002, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 708690607, + "versionNonce": 155091069, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "marshal args, decode -errno", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "marshal args, decode -errno", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r629233657", + "type": "rectangle", + "x": 540, + "y": 60, + "width": 170, + "height": 70, + "angle": 0, + "strokeColor": "#e8870c", + "backgroundColor": "#fff8ef", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 782684370, + "versionNonce": 1280106159, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1693830944", + "type": "text", + "x": 550, + "y": 68, + "width": 146.3, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1369935781, + "versionNonce": 553311009, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "syscall instruction", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "syscall instruction", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1984452563", + "type": "text", + "x": 550, + "y": 88, + "width": 96.80000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1021913791, + "versionNonce": 1180553057, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "rax=318, args in", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "rax=318, args in", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t274130159", + "type": "text", + "x": 550, + "y": 103, + "width": 127.05000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 611146031, + "versionNonce": 551871017, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "rdi rsi rdx r10 r8 r9", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "rdi rsi rdx r10 r8 r9", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r801409400", + "type": "rectangle", + "x": 760, + "y": 60, + "width": 160, + "height": 70, + "angle": 0, + "strokeColor": "#999999", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "dashed", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1626799609, + "versionNonce": 1318581919, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1064718404", + "type": "text", + "x": 770, + "y": 68, + "width": 84.7, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1035148658, + "versionNonce": 556457468, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "[vdso] page", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "[vdso] page", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1121359743", + "type": "text", + "x": 770, + "y": 88, + "width": 84.7, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1267881353, + "versionNonce": 680989792, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "clock_gettime,", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "clock_gettime,", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t1015892341", + "type": "text", + "x": 770, + "y": 103, + "width": 114.95, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 386744112, + "versionNonce": 1142168850, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "vgetrandom: no trap", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "vgetrandom: no trap", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1166999851", + "type": "rectangle", + "x": 540, + "y": 285, + "width": 170, + "height": 60, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#111111", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 828430967, + "versionNonce": 434057299, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1146015610", + "type": "text", + "x": 550, + "y": 293, + "width": 123.20000000000002, + "height": 17.5, + "angle": 0, + "strokeColor": "#ffffff", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 261435189, + "versionNonce": 1458863321, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "entry_SYSCALL_64", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "entry_SYSCALL_64", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t256902399", + "type": "text", + "x": 550, + "y": 313, + "width": 121.00000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 114459391, + "versionNonce": 1984822324, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "swap to kernel stack", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "swap to kernel stack", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r61936044", + "type": "rectangle", + "x": 280, + "y": 285, + "width": 210, + "height": 60, + "angle": 0, + "strokeColor": "#888888", + "backgroundColor": "#f4f4f4", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 750813693, + "versionNonce": 955726312, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t341837439", + "type": "text", + "x": 290, + "y": 293, + "width": 146.3, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1091421749, + "versionNonce": 21001505, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "sys_call_table[rax]", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "sys_call_table[rax]", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t90982630", + "type": "text", + "x": 290, + "y": 313, + "width": 108.9, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 180120370, + "versionNonce": 763507225, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "dispatch by number", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "dispatch by number", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r390254831", + "type": "rectangle", + "x": 40, + "y": 285, + "width": 190, + "height": 60, + "angle": 0, + "strokeColor": "#888888", + "backgroundColor": "#f4f4f4", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 365558308, + "versionNonce": 675487741, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t766945853", + "type": "text", + "x": 50, + "y": 293, + "width": 115.50000000000001, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 489658639, + "versionNonce": 599626040, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "sys_getrandom()", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "sys_getrandom()", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1912566127", + "type": "text", + "x": 50, + "y": 313, + "width": 102.85000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1306066014, + "versionNonce": 1824295240, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "fills your buffer", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "fills your buffer", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a112376854", + "type": "arrow", + "x": 230, + "y": 95, + "width": 50, + "height": 0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 803910754, + "versionNonce": 122793390, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 50, + 0 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "a1404871642", + "type": "arrow", + "x": 490, + "y": 95, + "width": 50, + "height": 0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1396545467, + "versionNonce": 1308681225, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 50, + 0 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "a1996187928", + "type": "arrow", + "x": 490, + "y": 75, + "width": 270, + "height": 0, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "dashed", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1907786272, + "versionNonce": 258633953, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 270, + 0 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t1338969318", + "type": "text", + "x": 625.0, + "y": 61.0, + "width": 181.50000000000003, + "height": 13.75, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1833473837, + "versionNonce": 868654404, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "fast path: stays in user space", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "fast path: stays in user space", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a1367879852", + "type": "arrow", + "x": 625, + "y": 130, + "width": 0, + "height": 155, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 135842719, + "versionNonce": 571936460, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + 155 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t1791500295", + "type": "text", + "x": 625.0, + "y": 193.5, + "width": 40, + "height": 13.75, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 524764408, + "versionNonce": 1304001318, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "trap", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "trap", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a1524378443", + "type": "arrow", + "x": 540, + "y": 315, + "width": 50, + "height": 0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 546515001, + "versionNonce": 1065314103, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + -50, + 0 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "a1127457462", + "type": "arrow", + "x": 280, + "y": 315, + "width": 50, + "height": 0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 410146541, + "versionNonce": 1269364958, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + -50, + 0 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "a1203871218", + "type": "arrow", + "x": 135, + "y": 285, + "width": 0, + "height": 155, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "dashed", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 600714640, + "versionNonce": 676924323, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + -155 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t360585969", + "type": "text", + "x": 135.0, + "y": 193.5, + "width": 181.50000000000003, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1364151096, + "versionNonce": 81714832, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "return: rax = result or -errno", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "return: rax = result or -errno", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t1328610991", + "type": "text", + "x": 40, + "y": 180, + "width": 417.45000000000005, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 160651678, + "versionNonce": 1808150973, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "one round trip per trap: mode switch + kernel work + mode switch back", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "one round trip per trap: mode switch + kernel work + mode switch back", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t1342029498", + "type": "text", + "x": 40, + "y": 198, + "width": 477.95000000000005, + "height": 13.75, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1789585076, + "versionNonce": 1999597603, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "strace sits on this boundary - it sees every trap, and nothing the vDSO answers", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "strace sits on this boundary - it sees every trap, and nothing the vDSO answers", + "lineHeight": 1.25, + "baseline": 11 + } + ], + "appState": { + "viewBackgroundColor": "#ffffff", + "gridSize": null + }, + "files": {} +} \ No newline at end of file diff --git a/assets/diagrams/04-user-kernel-path.svg b/assets/diagrams/04-user-kernel-path.svg new file mode 100644 index 0000000..b6152e1 --- /dev/null +++ b/assets/diagrams/04-user-kernel-path.svg @@ -0,0 +1,46 @@ + + + +user space (ring 3) + +kernel (ring 0) + +your code +getrandom(buf, 16, 0) + +language wrapper +glibc / x/sys / rustix +marshal args, decode -errno + +syscall instruction +rax=318, args in +rdi rsi rdx r10 r8 r9 + +[vdso] page +clock_gettime, +vgetrandom: no trap + +entry_SYSCALL_64 +swap to kernel stack + +sys_call_table[rax] +dispatch by number + +sys_getrandom() +fills your buffer + + + +fast path: stays in user space +fast path: stays in user space + +trap +trap + + + +return: rax = result or -errno +return: rax = result or -errno +one round trip per trap: mode switch + kernel work + mode switch back +strace sits on this boundary - it sees every trap, and nothing the vDSO answers + \ No newline at end of file diff --git a/assets/diagrams/05-error-flow-three-ways.excalidraw b/assets/diagrams/05-error-flow-three-ways.excalidraw new file mode 100644 index 0000000..7a9290f --- /dev/null +++ b/assets/diagrams/05-error-flow-three-ways.excalidraw @@ -0,0 +1,1730 @@ +{ + "type": "excalidraw", + "version": 2, + "source": "https://excalidraw.com", + "elements": [ + { + "id": "r378220503", + "type": "rectangle", + "x": 20, + "y": 30, + "width": 840, + "height": 110, + "angle": 0, + "strokeColor": "#dddddd", + "backgroundColor": "#f4f4f4", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1718990018, + "versionNonce": 539386052, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1743219994", + "type": "text", + "x": 32, + "y": 36, + "width": 40, + "height": 15.0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 301190239, + "versionNonce": 1365924295, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "kernel", + "fontSize": 12, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "kernel", + "lineHeight": 1.25, + "baseline": 12 + }, + { + "id": "r1758934218", + "type": "rectangle", + "x": 20, + "y": 155, + "width": 840, + "height": 385, + "angle": 0, + "strokeColor": "#dddddd", + "backgroundColor": "#fcfcfc", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 306157431, + "versionNonce": 884013903, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1522507957", + "type": "text", + "x": 32, + "y": 161, + "width": 66.0, + "height": 15.0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1838954109, + "versionNonce": 1990658717, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "user space", + "fontSize": 12, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "user space", + "lineHeight": 1.25, + "baseline": 12 + }, + { + "id": "r527110786", + "type": "rectangle", + "x": 350, + "y": 55, + "width": 180, + "height": 60, + "angle": 0, + "strokeColor": "#888888", + "backgroundColor": "#f4f4f4", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 195941568, + "versionNonce": 335334910, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t879187804", + "type": "text", + "x": 360, + "y": 63, + "width": 161.70000000000002, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1810334843, + "versionNonce": 500766146, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "write(2) on /dev/full", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "write(2) on /dev/full", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1102835442", + "type": "text", + "x": 360, + "y": 83, + "width": 90.75000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1424103811, + "versionNonce": 575647279, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "returns -ENOSPC", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "returns -ENOSPC", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1498580107", + "type": "rectangle", + "x": 350, + "y": 170, + "width": 180, + "height": 60, + "angle": 0, + "strokeColor": "#e8870c", + "backgroundColor": "#fff8ef", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 3670610, + "versionNonce": 7057962, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t520702432", + "type": "text", + "x": 360, + "y": 178, + "width": 77.0, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 577093004, + "versionNonce": 1106006921, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "errno = 28", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "errno = 28", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1905172881", + "type": "text", + "x": 360, + "y": 198, + "width": 96.80000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1114254388, + "versionNonce": 922466889, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "ENOSPC \u2014 the ABI", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "ENOSPC \u2014 the ABI", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1641703930", + "type": "rectangle", + "x": 60, + "y": 275, + "width": 200, + "height": 64, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1998182674, + "versionNonce": 1196162141, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1233897431", + "type": "text", + "x": 70, + "y": 283, + "width": 123.20000000000002, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 813752573, + "versionNonce": 12916410, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "std::expected", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "std::error_code>", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r705295319", + "type": "rectangle", + "x": 60, + "y": 375, + "width": 200, + "height": 64, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 238445022, + "versionNonce": 1995862928, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t574695122", + "type": "text", + "x": 70, + "y": 383, + "width": 53.900000000000006, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 14883370, + "versionNonce": 920968805, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "Failure", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "Failure", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t844648299", + "type": "text", + "x": 70, + "y": 403, + "width": 114.95, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 776655565, + "versionNonce": 640149551, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "message + exit_code", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "message + exit_code", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1914888152", + "type": "rectangle", + "x": 340, + "y": 275, + "width": 200, + "height": 64, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 648182757, + "versionNonce": 1944441385, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t231788958", + "type": "text", + "x": 350, + "y": 283, + "width": 138.60000000000002, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 179243357, + "versionNonce": 607252680, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "error (unix.Errno)", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "error (unix.Errno)", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t886551353", + "type": "text", + "x": 350, + "y": 303, + "width": 102.85000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 231826711, + "versionNonce": 1449438260, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "an ordinary value", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "an ordinary value", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r946757920", + "type": "rectangle", + "x": 340, + "y": 375, + "width": 200, + "height": 64, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1876520511, + "versionNonce": 1639544367, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t524606566", + "type": "text", + "x": 350, + "y": 383, + "width": 61.60000000000001, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 314529455, + "versionNonce": 1348671016, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "*opError", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "*opError", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t677804270", + "type": "text", + "x": 350, + "y": 403, + "width": 127.05000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 851238617, + "versionNonce": 880208600, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "Unwrap \u2192 errors.Is/As", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "Unwrap \u2192 errors.Is/As", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r285679248", + "type": "rectangle", + "x": 620, + "y": 275, + "width": 200, + "height": 64, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1555368165, + "versionNonce": 1061719420, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1822583350", + "type": "text", + "x": 630, + "y": 283, + "width": 123.20000000000002, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1295292575, + "versionNonce": 1701904880, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "Result", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "Result", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1050049203", + "type": "text", + "x": 630, + "y": 303, + "width": 40, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 527419453, + "versionNonce": 119570889, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "rustix", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "rustix", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1195502498", + "type": "rectangle", + "x": 620, + "y": 375, + "width": 200, + "height": 64, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1590164266, + "versionNonce": 1222285677, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t640294386", + "type": "text", + "x": 630, + "y": 383, + "width": 123.20000000000002, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 227771942, + "versionNonce": 1193967392, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "CopyError::Write", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "CopyError::Write", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1856799358", + "type": "text", + "x": 630, + "y": 403, + "width": 139.15, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1768780762, + "versionNonce": 1841880414, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "thiserror + exit_code()", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "thiserror + exit_code()", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r133215127", + "type": "rectangle", + "x": 250, + "y": 485, + "width": 380, + "height": 50, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#111111", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 514386535, + "versionNonce": 622263272, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1933866662", + "type": "text", + "x": 260, + "y": 493, + "width": 46.2, + "height": 17.5, + "angle": 0, + "strokeColor": "#ffffff", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1264556913, + "versionNonce": 593100704, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "exit 3", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "exit 3", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1634340939", + "type": "text", + "x": 260, + "y": 513, + "width": 284.35, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1498927430, + "versionNonce": 347675724, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "copyx: write /dev/full: No space left on device", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "copyx: write /dev/full: No space left on device", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a502574079", + "type": "arrow", + "x": 440, + "y": 115, + "width": 0, + "height": 55, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 281383664, + "versionNonce": 787857407, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + 55 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t364215827", + "type": "text", + "x": 440.0, + "y": 128.5, + "width": 102.85000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 46660791, + "versionNonce": 613211826, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "rax = -28 \u2192 errno", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "rax = -28 \u2192 errno", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a130492079", + "type": "arrow", + "x": 380, + "y": 230, + "width": 200, + "height": 45, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1098670144, + "versionNonce": 352134248, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + -200, + 45 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "a484098285", + "type": "arrow", + "x": 440, + "y": 230, + "width": 0, + "height": 45, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 643785228, + "versionNonce": 293367959, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + 45 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "a581794521", + "type": "arrow", + "x": 500, + "y": 230, + "width": 200, + "height": 45, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1772990777, + "versionNonce": 1932771022, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 200, + 45 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "a1377180964", + "type": "arrow", + "x": 160, + "y": 339, + "width": 0, + "height": 36, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1810965, + "versionNonce": 1705436588, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + 36 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t1959219588", + "type": "text", + "x": 160.0, + "y": 343.0, + "width": 54.45, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1137243378, + "versionNonce": 533454810, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "map phase", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "map phase", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a1996762305", + "type": "arrow", + "x": 440, + "y": 339, + "width": 0, + "height": 36, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 175828627, + "versionNonce": 1725714327, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + 36 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t1140824058", + "type": "text", + "x": 440.0, + "y": 343.0, + "width": 40, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 920476604, + "versionNonce": 801949961, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "wrap", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "wrap", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a790889403", + "type": "arrow", + "x": 720, + "y": 339, + "width": 0, + "height": 36, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1324616943, + "versionNonce": 23557924, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + 36 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t1265364613", + "type": "text", + "x": 720.0, + "y": 343.0, + "width": 42.35, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 461259096, + "versionNonce": 1766640239, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "map_err", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "map_err", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a515104054", + "type": "arrow", + "x": 160, + "y": 439, + "width": 160, + "height": 46, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 143435018, + "versionNonce": 1293377114, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 160, + 46 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "a918382558", + "type": "arrow", + "x": 440, + "y": 439, + "width": 0, + "height": 46, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1734039762, + "versionNonce": 1726646534, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + 46 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "a1721185234", + "type": "arrow", + "x": 720, + "y": 439, + "width": 160, + "height": 46, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1376345237, + "versionNonce": 1571180492, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + -160, + 46 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t1597548752", + "type": "text", + "x": 160, + "y": 255, + "width": 40, + "height": 13.75, + "angle": 0, + "strokeColor": "#1d1d1d", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 440838058, + "versionNonce": 551317178, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "C++23", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "C++23", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t256447480", + "type": "text", + "x": 440, + "y": 255, + "width": 40, + "height": 13.75, + "angle": 0, + "strokeColor": "#1d1d1d", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1764479135, + "versionNonce": 897967127, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "Go", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "Go", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t1675769010", + "type": "text", + "x": 720, + "y": 255, + "width": 40, + "height": 13.75, + "angle": 0, + "strokeColor": "#1d1d1d", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1801080868, + "versionNonce": 501468675, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "Rust", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "Rust", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t1141146634", + "type": "text", + "x": 440, + "y": 12, + "width": 254.10000000000002, + "height": 13.75, + "angle": 0, + "strokeColor": "#1d1d1d", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 419194744, + "versionNonce": 183040143, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "one failure, three spellings, one contract", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "one failure, three spellings, one contract", + "lineHeight": 1.25, + "baseline": 11 + } + ], + "appState": { + "viewBackgroundColor": "#ffffff", + "gridSize": null + }, + "files": {} +} \ No newline at end of file diff --git a/assets/diagrams/05-error-flow-three-ways.svg b/assets/diagrams/05-error-flow-three-ways.svg new file mode 100644 index 0000000..e2b7d15 --- /dev/null +++ b/assets/diagrams/05-error-flow-three-ways.svg @@ -0,0 +1,56 @@ + + + +kernel + +user space + +write(2) on /dev/full +returns -ENOSPC + +errno = 28 +ENOSPC — the ABI + +std::expected<T, +std::error_code> + +Failure +message + exit_code + +error (unix.Errno) +an ordinary value + +*opError +Unwrap → errors.Is/As + +Result<T, Errno> +rustix + +CopyError::Write +thiserror + exit_code() + +exit 3 +copyx: write /dev/full: No space left on device + +rax = -28 → errno +rax = -28 → errno + + + + +map phase +map phase + +wrap +wrap + +map_err +map_err + + + +C++23 +Go +Rust +one failure, three spellings, one contract + \ No newline at end of file diff --git a/assets/diagrams/06-cancellation-drain.excalidraw b/assets/diagrams/06-cancellation-drain.excalidraw new file mode 100644 index 0000000..9543ba2 --- /dev/null +++ b/assets/diagrams/06-cancellation-drain.excalidraw @@ -0,0 +1,1146 @@ +{ + "type": "excalidraw", + "version": 2, + "source": "https://excalidraw.com", + "elements": [ + { + "id": "r855244012", + "type": "rectangle", + "x": 30, + "y": 152, + "width": 150, + "height": 60, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#111111", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 889827705, + "versionNonce": 650495003, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t846720240", + "type": "text", + "x": 40, + "y": 160, + "width": 46.2, + "height": 17.5, + "angle": 0, + "strokeColor": "#ffffff", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 694537616, + "versionNonce": 714266346, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "SIGINT", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "SIGINT", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t504863697", + "type": "text", + "x": 40, + "y": 180, + "width": 90.75000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 72070799, + "versionNonce": 1257604805, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "kill -INT ", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "kill -INT ", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r282546068", + "type": "rectangle", + "x": 262, + "y": 52, + "width": 210, + "height": 56, + "angle": 0, + "strokeColor": "#2f6db5", + "backgroundColor": "#eef4fb", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1760570253, + "versionNonce": 1090409980, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1382387461", + "type": "text", + "x": 272, + "y": 60, + "width": 130.9, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 88970687, + "versionNonce": 1040565832, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "walker / producer", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "walker / producer", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t498092236", + "type": "text", + "x": 272, + "y": 80, + "width": 102.85000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1719994093, + "versionNonce": 950996776, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "stops enumerating", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "stops enumerating", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r612398599", + "type": "rectangle", + "x": 262, + "y": 148, + "width": 210, + "height": 56, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1356767569, + "versionNonce": 1561965757, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t188840403", + "type": "text", + "x": 272, + "y": 156, + "width": 92.4, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1778400426, + "versionNonce": 1943800296, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "idle workers", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "idle workers", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1948618113", + "type": "text", + "x": 272, + "y": 176, + "width": 121.00000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1954796260, + "versionNonce": 1656659322, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "wake from wait, exit", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "wake from wait, exit", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r573816324", + "type": "rectangle", + "x": 262, + "y": 244, + "width": 210, + "height": 56, + "angle": 0, + "strokeColor": "#e8870c", + "backgroundColor": "#fff8ef", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 661968614, + "versionNonce": 746792323, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1332414842", + "type": "text", + "x": 272, + "y": 252, + "width": 92.4, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1406912555, + "versionNonce": 523296984, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "busy workers", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "busy workers", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1994743201", + "type": "text", + "x": 272, + "y": 272, + "width": 139.15, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 725386317, + "versionNonce": 664636565, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "finish the file in hand", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "finish the file in hand", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1156243590", + "type": "rectangle", + "x": 544, + "y": 148, + "width": 170, + "height": 56, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1757837005, + "versionNonce": 671767891, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1623430455", + "type": "text", + "x": 554, + "y": 156, + "width": 100.10000000000001, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 659900372, + "versionNonce": 845806835, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "queue closes,", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "queue closes,", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1132906264", + "type": "text", + "x": 554, + "y": 176, + "width": 78.65, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1392685253, + "versionNonce": 683521578, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "results drain", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "results drain", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1653169311", + "type": "rectangle", + "x": 776, + "y": 132, + "width": 164, + "height": 88, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#111111", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 751109410, + "versionNonce": 1458407430, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1593095794", + "type": "text", + "x": 786, + "y": 140, + "width": 154.0, + "height": 17.5, + "angle": 0, + "strokeColor": "#ffffff", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 597615460, + "versionNonce": 1904244864, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "partial sorted lines", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "partial sorted lines", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1808622974", + "type": "text", + "x": 786, + "y": 160, + "width": 121.00000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 251917723, + "versionNonce": 265241438, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "parhash: interrupted", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "parhash: interrupted", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t745056955", + "type": "text", + "x": 786, + "y": 175, + "width": 48.400000000000006, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 912320612, + "versionNonce": 614081780, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "exit 130", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "exit 130", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a884932331", + "type": "arrow", + "x": 180, + "y": 168, + "width": 82, + "height": 84, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "dashed", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1105590474, + "versionNonce": 867309271, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 82, + -84 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "a1936337414", + "type": "arrow", + "x": 180, + "y": 182, + "width": 82, + "height": 6, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "dashed", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 921612920, + "versionNonce": 1206885276, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 82, + -6 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t582265415", + "type": "text", + "x": 221.0, + "y": 165.0, + "width": 84.7, + "height": 13.75, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 181506797, + "versionNonce": 851179327, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "one flag flips", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "one flag flips", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a1995856654", + "type": "arrow", + "x": 180, + "y": 196, + "width": 82, + "height": 72, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "dashed", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1826728979, + "versionNonce": 614714355, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 82, + 72 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "a919884330", + "type": "arrow", + "x": 472, + "y": 80, + "width": 72, + "height": 80, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 596138864, + "versionNonce": 1762133001, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 72, + 80 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t206858102", + "type": "text", + "x": 508.0, + "y": 106.0, + "width": 72.60000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1025442138, + "versionNonce": 128950727, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "no new tasks", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "no new tasks", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a491519500", + "type": "arrow", + "x": 472, + "y": 176, + "width": 72, + "height": 0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1159410113, + "versionNonce": 1923707221, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 72, + 0 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "a1791726619", + "type": "arrow", + "x": 472, + "y": 272, + "width": 72, + "height": 80, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 344857117, + "versionNonce": 892747594, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 72, + -80 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t623366951", + "type": "text", + "x": 508.0, + "y": 218.0, + "width": 72.60000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1584478200, + "versionNonce": 1813636014, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "last results", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "last results", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a240869950", + "type": "arrow", + "x": 714, + "y": 176, + "width": 62, + "height": 0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1722074829, + "versionNonce": 189031129, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 62, + 0 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t995096505", + "type": "text", + "x": 745.0, + "y": 162.0, + "width": 72.60000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1219512911, + "versionNonce": 480288277, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "sort + print", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "sort + print", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t152598260", + "type": "text", + "x": 480, + "y": 20, + "width": 284.35, + "height": 13.75, + "angle": 0, + "strokeColor": "#1d1d1d", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1293766074, + "versionNonce": 1039125993, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "the drain, observed live on a 6 GiB sparse tree", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "the drain, observed live on a 6 GiB sparse tree", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t1506668714", + "type": "text", + "x": 480, + "y": 334, + "width": 750.2, + "height": 13.75, + "angle": 0, + "strokeColor": "#1d1d1d", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1336248842, + "versionNonce": 887156047, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "observed: all 4 in-flight 64 MiB hashes completed after SIGINT; 20 well-formed lines, no summary, exit 130 in every language", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "observed: all 4 in-flight 64 MiB hashes completed after SIGINT; 20 well-formed lines, no summary, exit 130 in every language", + "lineHeight": 1.25, + "baseline": 11 + } + ], + "appState": { + "viewBackgroundColor": "#ffffff", + "gridSize": null + }, + "files": {} +} \ No newline at end of file diff --git a/assets/diagrams/06-cancellation-drain.svg b/assets/diagrams/06-cancellation-drain.svg new file mode 100644 index 0000000..47828aa --- /dev/null +++ b/assets/diagrams/06-cancellation-drain.svg @@ -0,0 +1,39 @@ + + + +SIGINT +kill -INT <pid> + +walker / producer +stops enumerating + +idle workers +wake from wait, exit + +busy workers +finish the file in hand + +queue closes, +results drain + +partial sorted lines +parhash: interrupted +exit 130 + + +one flag flips +one flag flips + + +no new tasks +no new tasks + + +last results +last results + +sort + print +sort + print +the drain, observed live on a 6 GiB sparse tree +observed: all 4 in-flight 64 MiB hashes completed after SIGINT; 20 well-formed lines, no summary, exit 130 in every language + \ No newline at end of file diff --git a/assets/diagrams/06-three-concurrency-models.excalidraw b/assets/diagrams/06-three-concurrency-models.excalidraw new file mode 100644 index 0000000..2454fad --- /dev/null +++ b/assets/diagrams/06-three-concurrency-models.excalidraw @@ -0,0 +1,2254 @@ +{ + "type": "excalidraw", + "version": 2, + "source": "https://excalidraw.com", + "elements": [ + { + "id": "r1118212221", + "type": "rectangle", + "x": 20, + "y": 48, + "width": 286, + "height": 358, + "angle": 0, + "strokeColor": "#dddddd", + "backgroundColor": "#fafafa", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 706811953, + "versionNonce": 1018445191, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1126397704", + "type": "text", + "x": 32, + "y": 54, + "width": 204.60000000000002, + "height": 15.0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 729846885, + "versionNonce": 1276757035, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "C++23 \u2014 what the standard gives", + "fontSize": 12, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "C++23 \u2014 what the standard gives", + "lineHeight": 1.25, + "baseline": 12 + }, + { + "id": "r72990744", + "type": "rectangle", + "x": 318, + "y": 48, + "width": 286, + "height": 358, + "angle": 0, + "strokeColor": "#dddddd", + "backgroundColor": "#fafafa", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1802651737, + "versionNonce": 1292210574, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t744157292", + "type": "text", + "x": 330, + "y": 54, + "width": 132.0, + "height": 15.0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1513597378, + "versionNonce": 71069817, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "Go \u2014 the GMP runtime", + "fontSize": 12, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "Go \u2014 the GMP runtime", + "lineHeight": 1.25, + "baseline": 12 + }, + { + "id": "r1374276573", + "type": "rectangle", + "x": 616, + "y": 48, + "width": 286, + "height": 358, + "angle": 0, + "strokeColor": "#dddddd", + "backgroundColor": "#fafafa", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1338745290, + "versionNonce": 1629669349, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t801702274", + "type": "text", + "x": 628, + "y": 54, + "width": 158.4, + "height": 15.0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1456056170, + "versionNonce": 588847385, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "Rust \u2014 ownership-checked", + "fontSize": 12, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "Rust \u2014 ownership-checked", + "lineHeight": 1.25, + "baseline": 12 + }, + { + "id": "r394419942", + "type": "rectangle", + "x": 68, + "y": 76, + "width": 190, + "height": 50, + "angle": 0, + "strokeColor": "#2f6db5", + "backgroundColor": "#eef4fb", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 377819782, + "versionNonce": 321989247, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t902489408", + "type": "text", + "x": 78, + "y": 84, + "width": 84.7, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1552281836, + "versionNonce": 1565906434, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "main thread", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "main thread", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1189395713", + "type": "text", + "x": 78, + "y": 104, + "width": 133.10000000000002, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1869876932, + "versionNonce": 1293511197, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "walks the tree, push()", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "walks the tree, push()", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1584986047", + "type": "rectangle", + "x": 68, + "y": 158, + "width": 190, + "height": 60, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1230274395, + "versionNonce": 715460108, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1257587319", + "type": "text", + "x": 78, + "y": 166, + "width": 69.30000000000001, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1084531238, + "versionNonce": 1696421925, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "TaskQueue", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "TaskQueue", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t505501807", + "type": "text", + "x": 78, + "y": 186, + "width": 78.65, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 166458863, + "versionNonce": 1000499035, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "mutex + deque", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "mutex + deque", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t237891608", + "type": "text", + "x": 78, + "y": 201, + "width": 133.10000000000002, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1503389790, + "versionNonce": 944163670, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "condition_variable_any", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "condition_variable_any", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r539442800", + "type": "rectangle", + "x": 68, + "y": 250, + "width": 190, + "height": 50, + "angle": 0, + "strokeColor": "#e8870c", + "backgroundColor": "#fff8ef", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1878387095, + "versionNonce": 986911307, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t789338852", + "type": "text", + "x": 78, + "y": 258, + "width": 123.20000000000002, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1352359021, + "versionNonce": 1380103928, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "4 \u00d7 std::jthread", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "4 \u00d7 std::jthread", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t752388060", + "type": "text", + "x": 78, + "y": 278, + "width": 90.75000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 721100199, + "versionNonce": 966025069, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "pop(stop_token)", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "pop(stop_token)", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r946433234", + "type": "rectangle", + "x": 68, + "y": 332, + "width": 190, + "height": 60, + "angle": 0, + "strokeColor": "#999999", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1352108755, + "versionNonce": 768285651, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1104941242", + "type": "text", + "x": 78, + "y": 340, + "width": 115.50000000000001, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1065148734, + "versionNonce": 77386140, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "watcher jthread", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "watcher jthread", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1289817445", + "type": "text", + "x": 78, + "y": 360, + "width": 121.00000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1556952123, + "versionNonce": 1080793460, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "sigtimedwait(SIGINT)", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "sigtimedwait(SIGINT)", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t1041214032", + "type": "text", + "x": 78, + "y": 375, + "width": 66.55000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1597695859, + "versionNonce": 1975605671, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "stop_source", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "stop_source", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1516809656", + "type": "rectangle", + "x": 366, + "y": 76, + "width": 190, + "height": 50, + "angle": 0, + "strokeColor": "#2f6db5", + "backgroundColor": "#eef4fb", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 730680535, + "versionNonce": 941532246, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t611322442", + "type": "text", + "x": 376, + "y": 84, + "width": 123.20000000000002, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1196463771, + "versionNonce": 261231822, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "walker goroutine", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "walker goroutine", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1342612681", + "type": "text", + "x": 376, + "y": 104, + "width": 133.10000000000002, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1798916135, + "versionNonce": 775677379, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "filepath.WalkDir, send", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "filepath.WalkDir, send", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1447206698", + "type": "rectangle", + "x": 366, + "y": 158, + "width": 190, + "height": 60, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 251775224, + "versionNonce": 1894065649, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1506860411", + "type": "text", + "x": 376, + "y": 166, + "width": 84.7, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 149364892, + "versionNonce": 1851630604, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "chan string", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "chan string", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1891985204", + "type": "text", + "x": 376, + "y": 186, + "width": 60.50000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 9602404, + "versionNonce": 256539195, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "unbuffered", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "unbuffered", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t769779132", + "type": "text", + "x": 376, + "y": 201, + "width": 102.85000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 317589550, + "versionNonce": 904376205, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "results chan back", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "results chan back", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1910330690", + "type": "rectangle", + "x": 366, + "y": 250, + "width": 190, + "height": 50, + "angle": 0, + "strokeColor": "#e8870c", + "backgroundColor": "#fff8ef", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 203786154, + "versionNonce": 930406179, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1408041278", + "type": "text", + "x": 376, + "y": 258, + "width": 107.80000000000001, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 520294931, + "versionNonce": 1729164936, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "4 \u00d7 goroutines", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "4 \u00d7 goroutines", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1293547052", + "type": "text", + "x": 376, + "y": 278, + "width": 102.85000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 642918464, + "versionNonce": 706920211, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "select Done first", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "select Done first", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r644018003", + "type": "rectangle", + "x": 366, + "y": 332, + "width": 190, + "height": 60, + "angle": 0, + "strokeColor": "#999999", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 543309442, + "versionNonce": 127183132, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t769143276", + "type": "text", + "x": 376, + "y": 340, + "width": 154.0, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 688995848, + "versionNonce": 639162916, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "signal.NotifyContext", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "signal.NotifyContext", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1530977171", + "type": "text", + "x": 376, + "y": 360, + "width": 108.9, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1986545682, + "versionNonce": 1222230452, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "SIGINT cancels ctx", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "SIGINT cancels ctx", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t1570713440", + "type": "text", + "x": 376, + "y": 375, + "width": 60.50000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 798711397, + "versionNonce": 1035366830, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "ctx.Done()", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "ctx.Done()", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r966255065", + "type": "rectangle", + "x": 664, + "y": 76, + "width": 190, + "height": 50, + "angle": 0, + "strokeColor": "#2f6db5", + "backgroundColor": "#eef4fb", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1490764381, + "versionNonce": 796698879, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1068551741", + "type": "text", + "x": 674, + "y": 84, + "width": 84.7, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1573686079, + "versionNonce": 316738309, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "main thread", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "main thread", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1296025015", + "type": "text", + "x": 674, + "y": 104, + "width": 121.00000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1028877150, + "versionNonce": 562045344, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "recursive walk, send", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "recursive walk, send", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r974752478", + "type": "rectangle", + "x": 664, + "y": 158, + "width": 190, + "height": 60, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 99812660, + "versionNonce": 1539403838, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t459978716", + "type": "text", + "x": 674, + "y": 166, + "width": 92.4, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1844413347, + "versionNonce": 1373451834, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "mpsc channel", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "mpsc channel", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1895395934", + "type": "text", + "x": 674, + "y": 186, + "width": 121.00000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1934698447, + "versionNonce": 725912109, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "Arc>", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "Arc>", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t101723237", + "type": "text", + "x": 674, + "y": 201, + "width": 102.85000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1563222189, + "versionNonce": 378710550, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "results mpsc back", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "results mpsc back", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1272332895", + "type": "rectangle", + "x": 664, + "y": 250, + "width": 190, + "height": 50, + "angle": 0, + "strokeColor": "#e8870c", + "backgroundColor": "#fff8ef", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 141080786, + "versionNonce": 1280726406, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1622813799", + "type": "text", + "x": 674, + "y": 258, + "width": 138.60000000000002, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 954253823, + "versionNonce": 982019955, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "4 \u00d7 scoped threads", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "4 \u00d7 scoped threads", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t763975106", + "type": "text", + "x": 674, + "y": 278, + "width": 114.95, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 496928877, + "versionNonce": 1035870617, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "thread::scope joins", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "thread::scope joins", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r585761457", + "type": "rectangle", + "x": 664, + "y": 332, + "width": 190, + "height": 60, + "angle": 0, + "strokeColor": "#999999", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 11286546, + "versionNonce": 1293851221, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t77991513", + "type": "text", + "x": 674, + "y": 340, + "width": 123.20000000000002, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1959931651, + "versionNonce": 301501446, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "signal-hook flag", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "signal-hook flag", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t354034821", + "type": "text", + "x": 674, + "y": 360, + "width": 108.9, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1303647022, + "versionNonce": 449889223, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "SIGINT sets shared", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "SIGINT sets shared", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t1285752563", + "type": "text", + "x": 674, + "y": 375, + "width": 60.50000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 388491614, + "versionNonce": 1698313300, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "AtomicBool", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "AtomicBool", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a79889962", + "type": "arrow", + "x": 163, + "y": 126, + "width": 0, + "height": 32, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1221333351, + "versionNonce": 1308710535, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + 32 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t1439936466", + "type": "text", + "x": 163.0, + "y": 128.0, + "width": 40, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1193448326, + "versionNonce": 1529441418, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "paths", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "paths", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a551354791", + "type": "arrow", + "x": 163, + "y": 218, + "width": 0, + "height": 32, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1874918934, + "versionNonce": 1747628358, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + 32 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "a1333321883", + "type": "arrow", + "x": 163, + "y": 332, + "width": 0, + "height": 32, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "dashed", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1562711166, + "versionNonce": 1597721385, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + -32 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t954485402", + "type": "text", + "x": 163.0, + "y": 302.0, + "width": 84.7, + "height": 13.75, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 394718261, + "versionNonce": 395953534, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "request_stop()", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "request_stop()", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a561744262", + "type": "arrow", + "x": 461, + "y": 126, + "width": 0, + "height": 32, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1149612202, + "versionNonce": 1235386544, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + 32 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t1368409887", + "type": "text", + "x": 461.0, + "y": 128.0, + "width": 40, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1339921371, + "versionNonce": 991116884, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "paths", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "paths", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a1203799738", + "type": "arrow", + "x": 461, + "y": 218, + "width": 0, + "height": 32, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1773236431, + "versionNonce": 167309140, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + 32 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "a1832284454", + "type": "arrow", + "x": 461, + "y": 332, + "width": 0, + "height": 32, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "dashed", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1572968972, + "versionNonce": 16260054, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + -32 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t287134793", + "type": "text", + "x": 461.0, + "y": 302.0, + "width": 102.85000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1431035726, + "versionNonce": 324759187, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "context cancelled", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "context cancelled", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a1248690090", + "type": "arrow", + "x": 759, + "y": 126, + "width": 0, + "height": 32, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 355856532, + "versionNonce": 150044032, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + 32 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t1132666867", + "type": "text", + "x": 759.0, + "y": 128.0, + "width": 40, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1242688685, + "versionNonce": 702996180, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "paths", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "paths", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a296755212", + "type": "arrow", + "x": 759, + "y": 218, + "width": 0, + "height": 32, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1346177757, + "versionNonce": 1044412235, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + 32 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "a1141675416", + "type": "arrow", + "x": 759, + "y": 332, + "width": 0, + "height": 32, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "dashed", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1570398807, + "versionNonce": 422430722, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + -32 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t1371226680", + "type": "text", + "x": 759.0, + "y": 302.0, + "width": 66.55000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 347467379, + "versionNonce": 1320190213, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "flag polled", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "flag polled", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t1071127848", + "type": "text", + "x": 461, + "y": 20, + "width": 369.05, + "height": 13.75, + "angle": 0, + "strokeColor": "#1d1d1d", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1673660760, + "versionNonce": 1130121209, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "one program shape, three vocabularies \u2014 byte-identical stdout", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "one program shape, three vocabularies \u2014 byte-identical stdout", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t652717717", + "type": "text", + "x": 461, + "y": 432, + "width": 514.25, + "height": 13.75, + "angle": 0, + "strokeColor": "#1d1d1d", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1760520237, + "versionNonce": 1945440904, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "same kernel threads underneath \u2014 /proc//task while hashing: C++ 6, Go 10, Rust 5", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "same kernel threads underneath \u2014 /proc//task while hashing: C++ 6, Go 10, Rust 5", + "lineHeight": 1.25, + "baseline": 11 + } + ], + "appState": { + "viewBackgroundColor": "#ffffff", + "gridSize": null + }, + "files": {} +} \ No newline at end of file diff --git a/assets/diagrams/06-three-concurrency-models.svg b/assets/diagrams/06-three-concurrency-models.svg new file mode 100644 index 0000000..fafe454 --- /dev/null +++ b/assets/diagrams/06-three-concurrency-models.svg @@ -0,0 +1,74 @@ + + + +C++23 — what the standard gives + +Go — the GMP runtime + +Rust — ownership-checked + +main thread +walks the tree, push() + +TaskQueue +mutex + deque +condition_variable_any + +4 × std::jthread +pop(stop_token) + +watcher jthread +sigtimedwait(SIGINT) +stop_source + +walker goroutine +filepath.WalkDir, send + +chan string +unbuffered +results chan back + +4 × goroutines +select Done first + +signal.NotifyContext +SIGINT cancels ctx +ctx.Done() + +main thread +recursive walk, send + +mpsc channel +Arc<Mutex<Receiver>> +results mpsc back + +4 × scoped threads +thread::scope joins + +signal-hook flag +SIGINT sets shared +AtomicBool + +paths +paths + + +request_stop() +request_stop() + +paths +paths + + +context cancelled +context cancelled + +paths +paths + + +flag polled +flag polled +one program shape, three vocabularies — byte-identical stdout +same kernel threads underneath — /proc/<pid>/task while hashing: C++ 6, Go 10, Rust 5 + \ No newline at end of file diff --git a/assets/diagrams/07-fd-table-vfs.excalidraw b/assets/diagrams/07-fd-table-vfs.excalidraw new file mode 100644 index 0000000..3d0887c --- /dev/null +++ b/assets/diagrams/07-fd-table-vfs.excalidraw @@ -0,0 +1,2508 @@ +{ + "type": "excalidraw", + "version": 2, + "source": "https://excalidraw.com", + "elements": [ + { + "id": "r417373343", + "type": "rectangle", + "x": 20, + "y": 42, + "width": 860, + "height": 118, + "angle": 0, + "strokeColor": "#dddddd", + "backgroundColor": "#eef4fb", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 166083695, + "versionNonce": 1909870030, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1107421352", + "type": "text", + "x": 32, + "y": 48, + "width": 303.6, + "height": 15.0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1770523874, + "versionNonce": 420333153, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "fwatch process \u2014 fd table (ls -l /proc/PID/fd)", + "fontSize": 12, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "fwatch process \u2014 fd table (ls -l /proc/PID/fd)", + "lineHeight": 1.25, + "baseline": 12 + }, + { + "id": "r1927861136", + "type": "rectangle", + "x": 20, + "y": 192, + "width": 860, + "height": 112, + "angle": 0, + "strokeColor": "#dddddd", + "backgroundColor": "#f4f4f4", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 556772095, + "versionNonce": 592310046, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t261403403", + "type": "text", + "x": 32, + "y": 198, + "width": 382.8, + "height": 15.0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1944634857, + "versionNonce": 1291108340, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "kernel \u2014 open file descriptions (flags + offset live here)", + "fontSize": 12, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "kernel \u2014 open file descriptions (flags + offset live here)", + "lineHeight": 1.25, + "baseline": 12 + }, + { + "id": "r1604468625", + "type": "rectangle", + "x": 20, + "y": 336, + "width": 860, + "height": 104, + "angle": 0, + "strokeColor": "#dddddd", + "backgroundColor": "#f4f4f4", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 622006451, + "versionNonce": 1833812449, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t295102277", + "type": "text", + "x": 32, + "y": 342, + "width": 224.4, + "height": 15.0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1667242630, + "versionNonce": 1434161153, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "VFS \u2014 inodes (what ls -li numbers)", + "fontSize": 12, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "VFS \u2014 inodes (what ls -li numbers)", + "lineHeight": 1.25, + "baseline": 12 + }, + { + "id": "r1379055789", + "type": "rectangle", + "x": 40, + "y": 76, + "width": 150, + "height": 64, + "angle": 0, + "strokeColor": "#999999", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "dashed", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1335757456, + "versionNonce": 1222675345, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1938329892", + "type": "text", + "x": 50, + "y": 84, + "width": 92.4, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1011561787, + "versionNonce": 634362535, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "fd 0 \u00b7 1 \u00b7 2", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "fd 0 \u00b7 1 \u00b7 2", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1049829591", + "type": "text", + "x": 50, + "y": 104, + "width": 78.65, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 941739167, + "versionNonce": 1713062566, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "stdin/out/err", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "stdin/out/err", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t191793279", + "type": "text", + "x": 50, + "y": 119, + "width": 66.55000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1437048867, + "versionNonce": 1435006502, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "cloexec off", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "cloexec off", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1247399235", + "type": "rectangle", + "x": 220, + "y": 76, + "width": 140, + "height": 64, + "angle": 0, + "strokeColor": "#2f6db5", + "backgroundColor": "#eef4fb", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 908020425, + "versionNonce": 1412277817, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1689050777", + "type": "text", + "x": 230, + "y": 84, + "width": 40, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1572846092, + "versionNonce": 1534608247, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "fd 3", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "fd 3", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t877183798", + "type": "text", + "x": 230, + "y": 104, + "width": 40, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 406757722, + "versionNonce": 1984252009, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "big/", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "big/", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t96950958", + "type": "text", + "x": 230, + "y": 119, + "width": 60.50000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 140475299, + "versionNonce": 1404203312, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "cloexec on", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "cloexec on", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1385512971", + "type": "rectangle", + "x": 390, + "y": 76, + "width": 140, + "height": 64, + "angle": 0, + "strokeColor": "#2f6db5", + "backgroundColor": "#eef4fb", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 589560010, + "versionNonce": 1240996426, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t490283102", + "type": "text", + "x": 400, + "y": 84, + "width": 40, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1011225556, + "versionNonce": 854189223, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "fd 4", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "fd 4", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1857478996", + "type": "text", + "x": 400, + "y": 104, + "width": 42.35, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1385395750, + "versionNonce": 1370571521, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "big/l1/", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "big/l1/", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t1926911578", + "type": "text", + "x": 400, + "y": 119, + "width": 60.50000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 356909897, + "versionNonce": 717896690, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "cloexec on", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "cloexec on", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1209947032", + "type": "rectangle", + "x": 560, + "y": 76, + "width": 140, + "height": 64, + "angle": 0, + "strokeColor": "#2f6db5", + "backgroundColor": "#eef4fb", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1098321687, + "versionNonce": 1742921713, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t492823294", + "type": "text", + "x": 570, + "y": 84, + "width": 40, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1680307434, + "versionNonce": 822606310, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "fd 5", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "fd 5", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1215429095", + "type": "text", + "x": 570, + "y": 104, + "width": 60.50000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1627371999, + "versionNonce": 1661494269, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "big/l1/l2/", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "big/l1/l2/", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t129185532", + "type": "text", + "x": 570, + "y": 119, + "width": 60.50000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 405635325, + "versionNonce": 966417126, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "cloexec on", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "cloexec on", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1613216210", + "type": "rectangle", + "x": 730, + "y": 76, + "width": 140, + "height": 64, + "angle": 0, + "strokeColor": "#e8870c", + "backgroundColor": "#fff8ef", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 478722170, + "versionNonce": 1175480238, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1016817068", + "type": "text", + "x": 740, + "y": 84, + "width": 40, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1349718727, + "versionNonce": 19211752, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "fd 6", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "fd 6", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1490256053", + "type": "text", + "x": 740, + "y": 104, + "width": 78.65, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1861172675, + "versionNonce": 1242308680, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "big/l1/l2/l3/", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "big/l1/l2/l3/", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t77146161", + "type": "text", + "x": 740, + "y": 119, + "width": 60.50000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 327049556, + "versionNonce": 303011912, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "cloexec on", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "cloexec on", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1965429513", + "type": "rectangle", + "x": 220, + "y": 226, + "width": 140, + "height": 62, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 200408525, + "versionNonce": 273685880, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1477755428", + "type": "text", + "x": 230, + "y": 234, + "width": 84.7, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 965175388, + "versionNonce": 540245323, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "description", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "description", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t395031770", + "type": "text", + "x": 230, + "y": 254, + "width": 84.7, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 807648884, + "versionNonce": 17053784, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "flags 02700000", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "flags 02700000", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t674325677", + "type": "text", + "x": 230, + "y": 269, + "width": 78.65, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1542043779, + "versionNonce": 1456128983, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "pos (readdir)", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "pos (readdir)", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r947083989", + "type": "rectangle", + "x": 390, + "y": 226, + "width": 140, + "height": 62, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 277791274, + "versionNonce": 10764124, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1531980502", + "type": "text", + "x": 400, + "y": 234, + "width": 84.7, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 560327155, + "versionNonce": 1723874956, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "description", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "description", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t567111363", + "type": "text", + "x": 400, + "y": 254, + "width": 84.7, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1596833038, + "versionNonce": 1949758306, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "flags 02700000", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "flags 02700000", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t1270070829", + "type": "text", + "x": 400, + "y": 269, + "width": 78.65, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1788973397, + "versionNonce": 1742393884, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "pos (readdir)", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "pos (readdir)", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1922256666", + "type": "rectangle", + "x": 560, + "y": 226, + "width": 140, + "height": 62, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1229089063, + "versionNonce": 259742160, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1383414523", + "type": "text", + "x": 570, + "y": 234, + "width": 84.7, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1263754989, + "versionNonce": 1920950731, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "description", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "description", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t858711539", + "type": "text", + "x": 570, + "y": 254, + "width": 84.7, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 454596638, + "versionNonce": 1159936522, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "flags 02700000", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "flags 02700000", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t1037636559", + "type": "text", + "x": 570, + "y": 269, + "width": 78.65, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 365473825, + "versionNonce": 460549141, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "pos (readdir)", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "pos (readdir)", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r316611125", + "type": "rectangle", + "x": 730, + "y": 226, + "width": 140, + "height": 62, + "angle": 0, + "strokeColor": "#e8870c", + "backgroundColor": "#fff8ef", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1060322621, + "versionNonce": 1186727775, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1760822131", + "type": "text", + "x": 740, + "y": 234, + "width": 84.7, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 388141955, + "versionNonce": 753414318, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "description", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "description", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t578415652", + "type": "text", + "x": 740, + "y": 254, + "width": 84.7, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 221443726, + "versionNonce": 407094848, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "flags 02700000", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "flags 02700000", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t870359791", + "type": "text", + "x": 740, + "y": 269, + "width": 48.400000000000006, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1135744893, + "versionNonce": 244111331, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "pos 1980", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "pos 1980", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r659219334", + "type": "rectangle", + "x": 220, + "y": 368, + "width": 140, + "height": 56, + "angle": 0, + "strokeColor": "#888888", + "backgroundColor": "#f4f4f4", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 965917329, + "versionNonce": 1918380431, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t40527647", + "type": "text", + "x": 230, + "y": 376, + "width": 84.7, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 984867721, + "versionNonce": 87172645, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "inode 69926", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "inode 69926", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t885149329", + "type": "text", + "x": 230, + "y": 396, + "width": 54.45, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1920716067, + "versionNonce": 1946993320, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "directory", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "directory", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r707685528", + "type": "rectangle", + "x": 390, + "y": 368, + "width": 140, + "height": 56, + "angle": 0, + "strokeColor": "#888888", + "backgroundColor": "#f4f4f4", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 771658382, + "versionNonce": 186979620, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t43920798", + "type": "text", + "x": 400, + "y": 376, + "width": 84.7, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1330045311, + "versionNonce": 530403831, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "inode 69927", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "inode 69927", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t79894342", + "type": "text", + "x": 400, + "y": 396, + "width": 54.45, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 171143578, + "versionNonce": 1358622281, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "directory", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "directory", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1800804193", + "type": "rectangle", + "x": 560, + "y": 368, + "width": 140, + "height": 56, + "angle": 0, + "strokeColor": "#888888", + "backgroundColor": "#f4f4f4", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1293846388, + "versionNonce": 1240781224, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t675066166", + "type": "text", + "x": 570, + "y": 376, + "width": 84.7, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1834142861, + "versionNonce": 242582958, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "inode 69928", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "inode 69928", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t500511117", + "type": "text", + "x": 570, + "y": 396, + "width": 54.45, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1488776979, + "versionNonce": 1455541213, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "directory", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "directory", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r687960920", + "type": "rectangle", + "x": 730, + "y": 368, + "width": 140, + "height": 56, + "angle": 0, + "strokeColor": "#e8870c", + "backgroundColor": "#fff8ef", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 176347527, + "versionNonce": 1137344739, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t207007133", + "type": "text", + "x": 740, + "y": 376, + "width": 84.7, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1903263812, + "versionNonce": 1441005712, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "inode 69929", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "inode 69929", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t303539048", + "type": "text", + "x": 740, + "y": 396, + "width": 72.60000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 667999053, + "versionNonce": 1490129335, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "3000 entries", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "3000 entries", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a562011924", + "type": "arrow", + "x": 290.0, + "y": 140, + "width": 0.0, + "height": 86, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 485930353, + "versionNonce": 955305383, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0.0, + 86 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "a143744889", + "type": "arrow", + "x": 290.0, + "y": 288, + "width": 0.0, + "height": 80, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1853468094, + "versionNonce": 964925807, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0.0, + 80 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "a1957578956", + "type": "arrow", + "x": 460.0, + "y": 140, + "width": 0.0, + "height": 86, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 306840265, + "versionNonce": 1088348665, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0.0, + 86 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "a1524784364", + "type": "arrow", + "x": 460.0, + "y": 288, + "width": 0.0, + "height": 80, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1301114320, + "versionNonce": 1004243803, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0.0, + 80 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "a891330325", + "type": "arrow", + "x": 630.0, + "y": 140, + "width": 0.0, + "height": 86, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 667944124, + "versionNonce": 1009051571, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0.0, + 86 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "a266273583", + "type": "arrow", + "x": 630.0, + "y": 288, + "width": 0.0, + "height": 80, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 671500171, + "versionNonce": 958709580, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0.0, + 80 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "a1298050581", + "type": "arrow", + "x": 800.0, + "y": 140, + "width": 0.0, + "height": 86, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 434924490, + "versionNonce": 551704059, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0.0, + 86 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "a1232853812", + "type": "arrow", + "x": 800.0, + "y": 288, + "width": 0.0, + "height": 80, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 363310587, + "versionNonce": 1000877983, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0.0, + 80 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "a227563049", + "type": "arrow", + "x": 360, + "y": 96, + "width": 30, + "height": 0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "dashed", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 278233819, + "versionNonce": 1584049079, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 30, + 0 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t1901699998", + "type": "text", + "x": 375.0, + "y": 82.0, + "width": 40, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 727154144, + "versionNonce": 354267405, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "openat", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "openat", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a858811362", + "type": "arrow", + "x": 530, + "y": 96, + "width": 30, + "height": 0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "dashed", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1419236838, + "versionNonce": 1205001575, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 30, + 0 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t1623702170", + "type": "text", + "x": 545.0, + "y": 82.0, + "width": 40, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 927811170, + "versionNonce": 297515405, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "openat", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "openat", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a1204276046", + "type": "arrow", + "x": 700, + "y": 96, + "width": 30, + "height": 0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "dashed", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 266001151, + "versionNonce": 743884460, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 30, + 0 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t659360646", + "type": "text", + "x": 715.0, + "y": 82.0, + "width": 40, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 589495750, + "versionNonce": 115684194, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "openat", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "openat", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t1637672013", + "type": "text", + "x": 450.0, + "y": 18, + "width": 726.0000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#1d1d1d", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 714432258, + "versionNonce": 1590536789, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "one paused fwatch walk: four dirfds, one recursion chain \u2014 cloexec is a per-fd bit; flags and offset are per-description", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "one paused fwatch walk: four dirfds, one recursion chain \u2014 cloexec is a per-fd bit; flags and offset are per-description", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t72751164", + "type": "text", + "x": 115.0, + "y": 240, + "width": 151.25, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 241796311, + "versionNonce": 1854565371, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "dup/fork copy fd entries;", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "dup/fork copy fd entries;", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t1717751723", + "type": "text", + "x": 115.0, + "y": 256, + "width": 139.15, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1064006627, + "versionNonce": 1935023006, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "descriptions are shared", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "descriptions are shared", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t50459236", + "type": "text", + "x": 115.0, + "y": 386, + "width": 121.00000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 209678710, + "versionNonce": 1079067552, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "fdinfo ino == ls -li", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "fdinfo ino == ls -li", + "lineHeight": 1.25, + "baseline": 11 + } + ], + "appState": { + "viewBackgroundColor": "#ffffff", + "gridSize": null + }, + "files": {} +} \ No newline at end of file diff --git a/assets/diagrams/07-fd-table-vfs.svg b/assets/diagrams/07-fd-table-vfs.svg new file mode 100644 index 0000000..1a52c3b --- /dev/null +++ b/assets/diagrams/07-fd-table-vfs.svg @@ -0,0 +1,78 @@ + + + +fwatch process — fd table (ls -l /proc/PID/fd) + +kernel — open file descriptions (flags + offset live here) + +VFS — inodes (what ls -li numbers) + +fd 0 · 1 · 2 +stdin/out/err +cloexec off + +fd 3 +big/ +cloexec on + +fd 4 +big/l1/ +cloexec on + +fd 5 +big/l1/l2/ +cloexec on + +fd 6 +big/l1/l2/l3/ +cloexec on + +description +flags 02700000 +pos (readdir) + +description +flags 02700000 +pos (readdir) + +description +flags 02700000 +pos (readdir) + +description +flags 02700000 +pos 1980 + +inode 69926 +directory + +inode 69927 +directory + +inode 69928 +directory + +inode 69929 +3000 entries + + + + + + + + + +openat +openat + +openat +openat + +openat +openat +one paused fwatch walk: four dirfds, one recursion chain — cloexec is a per-fd bit; flags and offset are per-description +dup/fork copy fd entries; +descriptions are shared +fdinfo ino == ls -li + \ No newline at end of file diff --git a/assets/diagrams/08-write-path-page-cache.excalidraw b/assets/diagrams/08-write-path-page-cache.excalidraw new file mode 100644 index 0000000..4a95c1e --- /dev/null +++ b/assets/diagrams/08-write-path-page-cache.excalidraw @@ -0,0 +1,1314 @@ +{ + "type": "excalidraw", + "version": 2, + "source": "https://excalidraw.com", + "elements": [ + { + "id": "r647630499", + "type": "rectangle", + "x": 20, + "y": 42, + "width": 860, + "height": 92, + "angle": 0, + "strokeColor": "#dddddd", + "backgroundColor": "#fafafa", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1389236139, + "versionNonce": 1592025327, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t700613562", + "type": "text", + "x": 32, + "y": 48, + "width": 66.0, + "height": 15.0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1633334980, + "versionNonce": 1410624333, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "user space", + "fontSize": 12, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "user space", + "lineHeight": 1.25, + "baseline": 12 + }, + { + "id": "r83007876", + "type": "rectangle", + "x": 20, + "y": 150, + "width": 860, + "height": 115, + "angle": 0, + "strokeColor": "#dddddd", + "backgroundColor": "#fafafa", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1030848828, + "versionNonce": 1450944859, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1025003133", + "type": "text", + "x": 32, + "y": 156, + "width": 40, + "height": 15.0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 610436674, + "versionNonce": 1897013022, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "kernel", + "fontSize": 12, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "kernel", + "lineHeight": 1.25, + "baseline": 12 + }, + { + "id": "r736052201", + "type": "rectangle", + "x": 20, + "y": 282, + "width": 860, + "height": 96, + "angle": 0, + "strokeColor": "#dddddd", + "backgroundColor": "#fafafa", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 649695390, + "versionNonce": 1797317326, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1789951750", + "type": "text", + "x": 32, + "y": 288, + "width": 46.2, + "height": 15.0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1535072457, + "versionNonce": 1186115497, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "storage", + "fontSize": 12, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "storage", + "lineHeight": 1.25, + "baseline": 12 + }, + { + "id": "r1251270843", + "type": "rectangle", + "x": 70, + "y": 68, + "width": 760, + "height": 52, + "angle": 0, + "strokeColor": "#2f6db5", + "backgroundColor": "#eef4fb", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1371616741, + "versionNonce": 242165918, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1601154648", + "type": "text", + "x": 80, + "y": 76, + "width": 146.3, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 126928608, + "versionNonce": 1325839805, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "iobench user buffer", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "iobench user buffer", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1669078465", + "type": "text", + "x": 80, + "y": 96, + "width": 314.6, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 619895550, + "versionNonce": 1057577005, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "one 64 KiB block, 4096-aligned, written M x 16 times", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "one 64 KiB block, 4096-aligned, written M x 16 times", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1895204902", + "type": "rectangle", + "x": 70, + "y": 176, + "width": 530, + "height": 64, + "angle": 0, + "strokeColor": "#e8870c", + "backgroundColor": "#fff8ef", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1031441910, + "versionNonce": 1378945736, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1561470878", + "type": "text", + "x": 80, + "y": 184, + "width": 77.0, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1585507879, + "versionNonce": 26265263, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "page cache", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "page cache", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1662227052", + "type": "text", + "x": 80, + "y": 204, + "width": 375.1, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1100836471, + "versionNonce": 1400748184, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "write(2) copies in and dirties pages \u2014 Dirty: in /proc/meminfo", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "write(2) copies in and dirties pages \u2014 Dirty: in /proc/meminfo", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r83899401", + "type": "rectangle", + "x": 655, + "y": 176, + "width": 170, + "height": 64, + "angle": 0, + "strokeColor": "#999999", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "dashed", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1583794320, + "versionNonce": 598032049, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t457045398", + "type": "text", + "x": 665, + "y": 184, + "width": 100.10000000000001, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 871284046, + "versionNonce": 1390829336, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "no cache copy", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "no cache copy", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1359560831", + "type": "text", + "x": 665, + "y": 204, + "width": 121.00000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1696415552, + "versionNonce": 1362491145, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "kernel DMAs straight", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "kernel DMAs straight", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t757682111", + "type": "text", + "x": 665, + "y": 219, + "width": 121.00000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1851908900, + "versionNonce": 1444433808, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "from the user buffer", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "from the user buffer", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1895887298", + "type": "rectangle", + "x": 70, + "y": 306, + "width": 760, + "height": 48, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1567134820, + "versionNonce": 1764700922, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1417791838", + "type": "text", + "x": 80, + "y": 314, + "width": 84.7, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1062903251, + "versionNonce": 796678020, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "NVMe device", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "NVMe device", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1515601633", + "type": "text", + "x": 80, + "y": 334, + "width": 260.15000000000003, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1978300116, + "versionNonce": 1939361344, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "bytes are durable only after they land here", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "bytes are durable only after they land here", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a657712328", + "type": "arrow", + "x": 200, + "y": 120, + "width": 0, + "height": 56, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 632749375, + "versionNonce": 719729044, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + 56 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t1742890047", + "type": "text", + "x": 200.0, + "y": 134.0, + "width": 114.95, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1341733092, + "versionNonce": 1515266243, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "write(2) \u2014 a memcpy", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "write(2) \u2014 a memcpy", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a769220139", + "type": "arrow", + "x": 130, + "y": 240, + "width": 0, + "height": 66, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "dashed", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1098930470, + "versionNonce": 1334450943, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + 66 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t1221609862", + "type": "text", + "x": 130.0, + "y": 259.0, + "width": 96.80000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1113598261, + "versionNonce": 1614062564, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "writeback, later", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "writeback, later", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a931590773", + "type": "arrow", + "x": 265, + "y": 240, + "width": 0, + "height": 66, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 247087539, + "versionNonce": 1243101347, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + 66 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t114187948", + "type": "text", + "x": 265.0, + "y": 259.0, + "width": 114.95, + "height": 13.75, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 134674188, + "versionNonce": 356266068, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "fdatasync(2) at end", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "fdatasync(2) at end", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a1279015857", + "type": "arrow", + "x": 470, + "y": 120, + "width": 0, + "height": 56, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 86166119, + "versionNonce": 1072258702, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + 56 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t74960658", + "type": "text", + "x": 470.0, + "y": 134.0, + "width": 48.400000000000006, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1895684165, + "versionNonce": 1303443037, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "write(2)", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "write(2)", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a444070995", + "type": "arrow", + "x": 470, + "y": 240, + "width": 0, + "height": 66, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 146490694, + "versionNonce": 282870661, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + 66 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t924747083", + "type": "text", + "x": 470.0, + "y": 259.0, + "width": 163.35000000000002, + "height": 13.75, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 733005081, + "versionNonce": 692068291, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "fdatasync(2) every N blocks", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "fdatasync(2) every N blocks", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a435245620", + "type": "arrow", + "x": 740, + "y": 120, + "width": 0, + "height": 56, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1474842193, + "versionNonce": 1656314736, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + 56 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t548330904", + "type": "text", + "x": 740.0, + "y": 134.0, + "width": 102.85000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 221068341, + "versionNonce": 180639280, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "O_DIRECT write(2)", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "O_DIRECT write(2)", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a1271654242", + "type": "arrow", + "x": 740, + "y": 240, + "width": 0, + "height": 66, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1226552082, + "versionNonce": 1310643746, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + 66 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t1036811032", + "type": "text", + "x": 740.0, + "y": 259.0, + "width": 121.00000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 586751197, + "versionNonce": 885222908, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "waits for the device", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "waits for the device", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t1937086015", + "type": "text", + "x": 200, + "y": 22, + "width": 107.25000000000001, + "height": 16.25, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 290123688, + "versionNonce": 904938698, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "--mode buffered", + "fontSize": 13, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "--mode buffered", + "lineHeight": 1.25, + "baseline": 13 + }, + { + "id": "t1969209015", + "type": "text", + "x": 470, + "y": 22, + "width": 128.70000000000002, + "height": 16.25, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 313453101, + "versionNonce": 237131008, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "--mode fsync-every", + "fontSize": 13, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "--mode fsync-every", + "lineHeight": 1.25, + "baseline": 13 + }, + { + "id": "t557358928", + "type": "text", + "x": 740, + "y": 22, + "width": 92.95, + "height": 16.25, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 566632950, + "versionNonce": 326913068, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "--mode direct", + "fontSize": 13, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "--mode direct", + "lineHeight": 1.25, + "baseline": 13 + }, + { + "id": "t1337227401", + "type": "text", + "x": 450, + "y": 394, + "width": 721.0500000000001, + "height": 14.375, + "angle": 0, + "strokeColor": "#1d1d1d", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1062001345, + "versionNonce": 1216911399, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "measured on this host (btrfs on NVMe, 64 MiB): 3013 MiB/s + 17 ms fdatasync | 463 MiB/s end to end | 785 MiB/s", + "fontSize": 11.5, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "measured on this host (btrfs on NVMe, 64 MiB): 3013 MiB/s + 17 ms fdatasync | 463 MiB/s end to end | 785 MiB/s", + "lineHeight": 1.25, + "baseline": 11.5 + } + ], + "appState": { + "viewBackgroundColor": "#ffffff", + "gridSize": null + }, + "files": {} +} \ No newline at end of file diff --git a/assets/diagrams/08-write-path-page-cache.svg b/assets/diagrams/08-write-path-page-cache.svg new file mode 100644 index 0000000..1e66396 --- /dev/null +++ b/assets/diagrams/08-write-path-page-cache.svg @@ -0,0 +1,47 @@ + + + +user space + +kernel + +storage + +iobench user buffer +one 64 KiB block, 4096-aligned, written M x 16 times + +page cache +write(2) copies in and dirties pages — Dirty: in /proc/meminfo + +no cache copy +kernel DMAs straight +from the user buffer + +NVMe device +bytes are durable only after they land here + +write(2) — a memcpy +write(2) — a memcpy + +writeback, later +writeback, later + +fdatasync(2) at end +fdatasync(2) at end + +write(2) +write(2) + +fdatasync(2) every N blocks +fdatasync(2) every N blocks + +O_DIRECT write(2) +O_DIRECT write(2) + +waits for the device +waits for the device +--mode buffered +--mode fsync-every +--mode direct +measured on this host (btrfs on NVMe, 64 MiB): 3013 MiB/s + 17 ms fdatasync | 463 MiB/s end to end | 785 MiB/s + \ No newline at end of file diff --git a/assets/diagrams/09-event-loop-anatomy.excalidraw b/assets/diagrams/09-event-loop-anatomy.excalidraw new file mode 100644 index 0000000..492db42 --- /dev/null +++ b/assets/diagrams/09-event-loop-anatomy.excalidraw @@ -0,0 +1,1955 @@ +{ + "type": "excalidraw", + "version": 2, + "source": "https://excalidraw.com", + "elements": [ + { + "id": "r808018521", + "type": "rectangle", + "x": 20, + "y": 36, + "width": 840, + "height": 152, + "angle": 0, + "strokeColor": "#dddddd", + "backgroundColor": "#f7faff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 932157005, + "versionNonce": 582964823, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1893568615", + "type": "text", + "x": 32, + "y": 42, + "width": 396.00000000000006, + "height": 15.0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 36866070, + "versionNonce": 1515742201, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "user space \u2014 one thread; nothing blocks except epoll_wait(2)", + "fontSize": 12, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "user space \u2014 one thread; nothing blocks except epoll_wait(2)", + "lineHeight": 1.25, + "baseline": 12 + }, + { + "id": "r203451411", + "type": "rectangle", + "x": 20, + "y": 228, + "width": 840, + "height": 168, + "angle": 0, + "strokeColor": "#dddddd", + "backgroundColor": "#f5f5f5", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1120390496, + "versionNonce": 174206165, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t464802381", + "type": "text", + "x": 32, + "y": 234, + "width": 40, + "height": 15.0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1887117140, + "versionNonce": 1077331265, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "kernel", + "fontSize": 12, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "kernel", + "lineHeight": 1.25, + "baseline": 12 + }, + { + "id": "r1938453118", + "type": "rectangle", + "x": 40, + "y": 76, + "width": 180, + "height": 64, + "angle": 0, + "strokeColor": "#2f6db5", + "backgroundColor": "#eef4fb", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 871326623, + "versionNonce": 969403089, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t761060268", + "type": "text", + "x": 50, + "y": 84, + "width": 115.50000000000001, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 266244179, + "versionNonce": 1512453260, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "1 \u00b7 arm timerfd", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "1 \u00b7 arm timerfd", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t800002695", + "type": "text", + "x": 50, + "y": 104, + "width": 139.15, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1663614222, + "versionNonce": 1042926100, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "one-shot: min(deadline,", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "one-shot: min(deadline,", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t1181120790", + "type": "text", + "x": 50, + "y": 119, + "width": 133.10000000000002, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1920056455, + "versionNonce": 1796117249, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "earliest debounce due)", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "earliest debounce due)", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1420888629", + "type": "rectangle", + "x": 260, + "y": 76, + "width": 170, + "height": 64, + "angle": 0, + "strokeColor": "#e8870c", + "backgroundColor": "#fff8ef", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1315046195, + "versionNonce": 528358680, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t731877477", + "type": "text", + "x": 270, + "y": 84, + "width": 107.80000000000001, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1960610047, + "versionNonce": 295496719, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "2 \u00b7 epoll_wait", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "2 \u00b7 epoll_wait", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1379595940", + "type": "text", + "x": 270, + "y": 104, + "width": 66.55000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 631650535, + "versionNonce": 366319438, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "timeout \u22121:", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "timeout \u22121:", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t1541998771", + "type": "text", + "x": 270, + "y": 119, + "width": 127.05000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1346778461, + "versionNonce": 25819124, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "sleep until readiness", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "sleep until readiness", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r101812860", + "type": "rectangle", + "x": 470, + "y": 76, + "width": 170, + "height": 64, + "angle": 0, + "strokeColor": "#2f6db5", + "backgroundColor": "#eef4fb", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1786301721, + "versionNonce": 1981846113, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t186317011", + "type": "text", + "x": 480, + "y": 84, + "width": 92.4, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1190486719, + "versionNonce": 185962768, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "3 \u00b7 dispatch", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "3 \u00b7 dispatch", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t260422870", + "type": "text", + "x": 480, + "y": 104, + "width": 78.65, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1738958213, + "versionNonce": 800275759, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "switch on the", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "switch on the", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t923480864", + "type": "text", + "x": 480, + "y": 119, + "width": 84.7, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 480013180, + "versionNonce": 353224236, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "data.u32 token", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "data.u32 token", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1504692636", + "type": "rectangle", + "x": 680, + "y": 76, + "width": 160, + "height": 64, + "angle": 0, + "strokeColor": "#2f6db5", + "backgroundColor": "#eef4fb", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 75996508, + "versionNonce": 760452687, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1860108232", + "type": "text", + "x": 690, + "y": 84, + "width": 115.50000000000001, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1676236386, + "versionNonce": 581603466, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "4 \u00b7 drain + act", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "4 \u00b7 drain + act", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1531842936", + "type": "text", + "x": 690, + "y": 104, + "width": 108.9, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1453874988, + "versionNonce": 1897205340, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "read until EAGAIN,", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "read until EAGAIN,", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t1552251731", + "type": "text", + "x": 690, + "y": 119, + "width": 114.95, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1087956681, + "versionNonce": 1659596604, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "coalesce, flush due", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "coalesce, flush due", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r188939858", + "type": "rectangle", + "x": 300, + "y": 240, + "width": 280, + "height": 52, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#111111", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1169509617, + "versionNonce": 1666426205, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1264738513", + "type": "text", + "x": 310, + "y": 248, + "width": 192.50000000000003, + "height": 17.5, + "angle": 0, + "strokeColor": "#ffffff", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1588161009, + "versionNonce": 1508716519, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "eventpoll \u2014 interest list", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "eventpoll \u2014 interest list", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1836846748", + "type": "text", + "x": 310, + "y": 268, + "width": 205.70000000000002, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 489473243, + "versionNonce": 227895655, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "tfd 3\u00b74\u00b75, level-triggered EPOLLIN", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "tfd 3\u00b74\u00b75, level-triggered EPOLLIN", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1193309", + "type": "rectangle", + "x": 60, + "y": 322, + "width": 200, + "height": 60, + "angle": 0, + "strokeColor": "#888888", + "backgroundColor": "#f4f4f4", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 616715366, + "versionNonce": 929043886, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t75636561", + "type": "text", + "x": 70, + "y": 330, + "width": 154.0, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1258153429, + "versionNonce": 1747983346, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "inotify fd \u2014 token 0", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "inotify fd \u2014 token 0", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1646270224", + "type": "text", + "x": 70, + "y": 350, + "width": 139.15, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1115674672, + "versionNonce": 1006649251, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "wd 1 on DIR, mask 0x3c6", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "wd 1 on DIR, mask 0x3c6", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t1722497270", + "type": "text", + "x": 70, + "y": 365, + "width": 127.05000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 508746218, + "versionNonce": 912991583, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "per-watch event queue", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "per-watch event queue", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r301981497", + "type": "rectangle", + "x": 340, + "y": 322, + "width": 200, + "height": 60, + "angle": 0, + "strokeColor": "#888888", + "backgroundColor": "#f4f4f4", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1896257264, + "versionNonce": 1810811132, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t599484553", + "type": "text", + "x": 350, + "y": 330, + "width": 130.9, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1645801143, + "versionNonce": 1954657275, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "timerfd \u2014 token 1", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "timerfd \u2014 token 1", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t590645428", + "type": "text", + "x": 350, + "y": 350, + "width": 96.80000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 541513389, + "versionNonce": 332389952, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "CLOCK_MONOTONIC,", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "CLOCK_MONOTONIC,", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t853793055", + "type": "text", + "x": 350, + "y": 365, + "width": 121.00000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 177906859, + "versionNonce": 325910985, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "one-shot expirations", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "one-shot expirations", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1825775718", + "type": "rectangle", + "x": 620, + "y": 322, + "width": 200, + "height": 60, + "angle": 0, + "strokeColor": "#888888", + "backgroundColor": "#f4f4f4", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 900261201, + "versionNonce": 90937400, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t63484729", + "type": "text", + "x": 630, + "y": 330, + "width": 138.60000000000002, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 833625289, + "versionNonce": 1311869405, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "signalfd \u2014 token 2", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "signalfd \u2014 token 2", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t690502333", + "type": "text", + "x": 630, + "y": 350, + "width": 133.10000000000002, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 474026948, + "versionNonce": 1196834092, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "blocked SIGINT/SIGTERM", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "blocked SIGINT/SIGTERM", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t1185748185", + "type": "text", + "x": 630, + "y": 365, + "width": 96.80000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 832546725, + "versionNonce": 508286065, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "as readable data", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "as readable data", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a1029346997", + "type": "arrow", + "x": 220, + "y": 108, + "width": 40, + "height": 0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 808277688, + "versionNonce": 1891149855, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 40, + 0 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "a1924626827", + "type": "arrow", + "x": 430, + "y": 108, + "width": 40, + "height": 0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 409565919, + "versionNonce": 1925954446, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 40, + 0 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t1846192184", + "type": "text", + "x": 450.0, + "y": 94.0, + "width": 40, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1634694937, + "versionNonce": 802278205, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "wakeup", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "wakeup", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a908709472", + "type": "arrow", + "x": 640, + "y": 108, + "width": 40, + "height": 0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1547105379, + "versionNonce": 401208271, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 40, + 0 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "a260589400", + "type": "arrow", + "x": 760, + "y": 140, + "width": 0, + "height": 30, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 30141005, + "versionNonce": 1216621032, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + 30 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "a1246772654", + "type": "arrow", + "x": 760, + "y": 170, + "width": 630, + "height": 0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 91700958, + "versionNonce": 1208513345, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + -630, + 0 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t1225920851", + "type": "text", + "x": 445.0, + "y": 156.0, + "width": 145.20000000000002, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 35393282, + "versionNonce": 694359752, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "loop: re-arm, wait again", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "loop: re-arm, wait again", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a678535308", + "type": "arrow", + "x": 130, + "y": 170, + "width": 0, + "height": 30, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1259271988, + "versionNonce": 855364726, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + -30 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "a1486754027", + "type": "arrow", + "x": 190, + "y": 140, + "width": 170, + "height": 182, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "dashed", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1195557563, + "versionNonce": 198733381, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 170, + 182 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t1599457903", + "type": "text", + "x": 275.0, + "y": 217.0, + "width": 90.75000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1647865170, + "versionNonce": 813018574, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "timerfd_settime", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "timerfd_settime", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a350037784", + "type": "arrow", + "x": 160, + "y": 322, + "width": 200, + "height": 30, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1755437432, + "versionNonce": 837793468, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 200, + -30 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t1920195064", + "type": "text", + "x": 260.0, + "y": 293.0, + "width": 66.55000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1119361603, + "versionNonce": 929478423, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "file events", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "file events", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a1518392022", + "type": "arrow", + "x": 440, + "y": 322, + "width": 0, + "height": 30, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 967205403, + "versionNonce": 435145336, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + -30 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t1296755299", + "type": "text", + "x": 440.0, + "y": 293.0, + "width": 60.50000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 339595744, + "versionNonce": 1436875393, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "expiration", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "expiration", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a1718751654", + "type": "arrow", + "x": 720, + "y": 322, + "width": 200, + "height": 30, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1074682726, + "versionNonce": 1681111665, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + -200, + -30 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t857297535", + "type": "text", + "x": 620.0, + "y": 293.0, + "width": 40, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 854739877, + "versionNonce": 987677958, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "signal", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "signal", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a1506038510", + "type": "arrow", + "x": 440, + "y": 240, + "width": 95, + "height": 100, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1710860368, + "versionNonce": 1189094881, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + -95, + -100 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t1874080312", + "type": "text", + "x": 392.5, + "y": 176.0, + "width": 139.15, + "height": 13.75, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 130114106, + "versionNonce": 262017937, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "ready: {EPOLLIN, token}", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "ready: {EPOLLIN, token}", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t1851222363", + "type": "text", + "x": 440, + "y": 16, + "width": 302.5, + "height": 13.75, + "angle": 0, + "strokeColor": "#1d1d1d", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 231394158, + "versionNonce": 1231753942, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "fwatch watch \u2014 one epoll loop, three event sources", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "fwatch watch \u2014 one epoll loop, three event sources", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t390779445", + "type": "text", + "x": 440, + "y": 414, + "width": 739.2, + "height": 15.0, + "angle": 0, + "strokeColor": "#666666", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 722797912, + "versionNonce": 1866322957, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "Go inverts this picture: the runtime netpoller owns the loop (epoll_pwait) and surfaces each source as a channel", + "fontSize": 12, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "Go inverts this picture: the runtime netpoller owns the loop (epoll_pwait) and surfaces each source as a channel", + "lineHeight": 1.25, + "baseline": 12 + } + ], + "appState": { + "viewBackgroundColor": "#ffffff", + "gridSize": null + }, + "files": {} +} \ No newline at end of file diff --git a/assets/diagrams/09-event-loop-anatomy.svg b/assets/diagrams/09-event-loop-anatomy.svg new file mode 100644 index 0000000..ac7b27d --- /dev/null +++ b/assets/diagrams/09-event-loop-anatomy.svg @@ -0,0 +1,65 @@ + + + +user space — one thread; nothing blocks except epoll_wait(2) + +kernel + +1 · arm timerfd +one-shot: min(deadline, +earliest debounce due) + +2 · epoll_wait +timeout −1: +sleep until readiness + +3 · dispatch +switch on the +data.u32 token + +4 · drain + act +read until EAGAIN, +coalesce, flush due + +eventpoll — interest list +tfd 3·4·5, level-triggered EPOLLIN + +inotify fd — token 0 +wd 1 on DIR, mask 0x3c6 +per-watch event queue + +timerfd — token 1 +CLOCK_MONOTONIC, +one-shot expirations + +signalfd — token 2 +blocked SIGINT/SIGTERM +as readable data + + +wakeup +wakeup + + + +loop: re-arm, wait again +loop: re-arm, wait again + + +timerfd_settime +timerfd_settime + +file events +file events + +expiration +expiration + +signal +signal + +ready: {EPOLLIN, token} +ready: {EPOLLIN, token} +fwatch watch — one epoll loop, three event sources +Go inverts this picture: the runtime netpoller owns the loop (epoll_pwait) and surfaces each source as a channel + \ No newline at end of file diff --git a/assets/diagrams/10-readiness-vs-completion.excalidraw b/assets/diagrams/10-readiness-vs-completion.excalidraw new file mode 100644 index 0000000..83a7fac --- /dev/null +++ b/assets/diagrams/10-readiness-vs-completion.excalidraw @@ -0,0 +1,1390 @@ +{ + "type": "excalidraw", + "version": 2, + "source": "https://excalidraw.com", + "elements": [ + { + "id": "r33065972", + "type": "rectangle", + "x": 20, + "y": 40, + "width": 460, + "height": 440, + "angle": 0, + "strokeColor": "#dddddd", + "backgroundColor": "#eef4fb", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1672684975, + "versionNonce": 196985044, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1318589425", + "type": "text", + "x": 32, + "y": 46, + "width": 191.4, + "height": 15.0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 449750217, + "versionNonce": 282354467, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "readiness \u2014 epoll (chapter 9)", + "fontSize": 12, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "readiness \u2014 epoll (chapter 9)", + "lineHeight": 1.25, + "baseline": 12 + }, + { + "id": "r771634928", + "type": "rectangle", + "x": 500, + "y": 40, + "width": 460, + "height": 440, + "angle": 0, + "strokeColor": "#dddddd", + "backgroundColor": "#fff8ef", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1658121823, + "versionNonce": 412361628, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t438999970", + "type": "text", + "x": 512, + "y": 46, + "width": 237.60000000000002, + "height": 15.0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 450692923, + "versionNonce": 1963055295, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "completion \u2014 io_uring (this chapter)", + "fontSize": 12, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "completion \u2014 io_uring (this chapter)", + "lineHeight": 1.25, + "baseline": 12 + }, + { + "id": "r1919846616", + "type": "rectangle", + "x": 60, + "y": 80, + "width": 380, + "height": 56, + "angle": 0, + "strokeColor": "#2f6db5", + "backgroundColor": "#eef4fb", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1334384000, + "versionNonce": 380988267, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1727331113", + "type": "text", + "x": 70, + "y": 88, + "width": 161.70000000000002, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1176842367, + "versionNonce": 1952061950, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "epoll_wait(epfd, ...)", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "epoll_wait(epfd, ...)", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t309066185", + "type": "text", + "x": 70, + "y": 108, + "width": 205.70000000000002, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 650490825, + "versionNonce": 878467583, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "\u201ctell me when these fds are ready\u201d", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "\u201ctell me when these fds are ready\u201d", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1367554015", + "type": "rectangle", + "x": 60, + "y": 176, + "width": 380, + "height": 56, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 997092332, + "versionNonce": 838704390, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1938473920", + "type": "text", + "x": 70, + "y": 184, + "width": 184.8, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 302696236, + "versionNonce": 1804080016, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "kernel: readiness events", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "kernel: readiness events", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t2010713", + "type": "text", + "x": 70, + "y": 204, + "width": 199.65, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 794083039, + "versionNonce": 858131360, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "fd 7 readable, fd 12 writable ...", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "fd 7 readable, fd 12 writable ...", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1971273124", + "type": "rectangle", + "x": 60, + "y": 272, + "width": 380, + "height": 56, + "angle": 0, + "strokeColor": "#2f6db5", + "backgroundColor": "#eef4fb", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1981120200, + "versionNonce": 644913586, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t187978706", + "type": "text", + "x": 70, + "y": 280, + "width": 223.3, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 645327470, + "versionNonce": 1986121165, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "you still do the I/O yourself", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "you still do the I/O yourself", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1740499328", + "type": "text", + "x": 70, + "y": 300, + "width": 248.05, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1769673446, + "versionNonce": 1832329304, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "read(fd) / write(fd) \u2014 one syscall per op", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "read(fd) / write(fd) \u2014 one syscall per op", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r741905621", + "type": "rectangle", + "x": 60, + "y": 368, + "width": 380, + "height": 56, + "angle": 0, + "strokeColor": "#999999", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "dashed", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1403408713, + "versionNonce": 1019750585, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1804846894", + "type": "text", + "x": 70, + "y": 376, + "width": 254.10000000000002, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1982952644, + "versionNonce": 1007739687, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "and it can still fail with EAGAIN", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "and it can still fail with EAGAIN", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t988952869", + "type": "text", + "x": 70, + "y": 396, + "width": 199.65, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1067237111, + "versionNonce": 740942329, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "readiness is a hint, not a result", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "readiness is a hint, not a result", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1253144830", + "type": "rectangle", + "x": 540, + "y": 80, + "width": 380, + "height": 56, + "angle": 0, + "strokeColor": "#2f6db5", + "backgroundColor": "#eef4fb", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 928484917, + "versionNonce": 316158416, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1884749643", + "type": "text", + "x": 550, + "y": 88, + "width": 223.3, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 742888701, + "versionNonce": 1006237755, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "queue SQEs describing the I/O", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "queue SQEs describing the I/O", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t759418367", + "type": "text", + "x": 550, + "y": 108, + "width": 260.15000000000003, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1242315980, + "versionNonce": 250042840, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "\u201cread 128 KiB at offset 0 into this buffer\u201d", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "\u201cread 128 KiB at offset 0 into this buffer\u201d", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1507667372", + "type": "rectangle", + "x": 540, + "y": 176, + "width": 380, + "height": 56, + "angle": 0, + "strokeColor": "#e8870c", + "backgroundColor": "#fff8ef", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 202581492, + "versionNonce": 1961279555, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1424733054", + "type": "text", + "x": 550, + "y": 184, + "width": 254.10000000000002, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 122616603, + "versionNonce": 1443272391, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "io_uring_enter \u2014 submit N, wait M", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "io_uring_enter \u2014 submit N, wait M", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t9328183", + "type": "text", + "x": 550, + "y": 204, + "width": 211.75000000000003, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1049376781, + "versionNonce": 1770098097, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "one syscall carries the whole batch", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "one syscall carries the whole batch", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r95601264", + "type": "rectangle", + "x": 540, + "y": 272, + "width": 380, + "height": 56, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1633955903, + "versionNonce": 1526038808, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t849757185", + "type": "text", + "x": 550, + "y": 280, + "width": 146.3, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1604120516, + "versionNonce": 607377769, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "kernel does the I/O", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "kernel does the I/O", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1547151558", + "type": "text", + "x": 550, + "y": 300, + "width": 296.45000000000005, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 729491323, + "versionNonce": 1897958637, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "inline if it can, io-wq workers if it would block", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "inline if it can, io-wq workers if it would block", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1767937136", + "type": "rectangle", + "x": 540, + "y": 368, + "width": 380, + "height": 56, + "angle": 0, + "strokeColor": "#2f6db5", + "backgroundColor": "#eef4fb", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 138944644, + "versionNonce": 1205131035, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t755736042", + "type": "text", + "x": 550, + "y": 376, + "width": 207.9, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1421276688, + "versionNonce": 803486117, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "CQEs carry finished results", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "CQEs carry finished results", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t54683949", + "type": "text", + "x": 550, + "y": 396, + "width": 242.00000000000003, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 33043266, + "versionNonce": 1272576505, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "cqe.res = bytes done \u2014 no second syscall", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "cqe.res = bytes done \u2014 no second syscall", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a104127072", + "type": "arrow", + "x": 250, + "y": 136, + "width": 0, + "height": 40, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 466217375, + "versionNonce": 66749844, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + 40 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "a1279252924", + "type": "arrow", + "x": 250, + "y": 232, + "width": 0, + "height": 40, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 147938089, + "versionNonce": 1784297906, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + 40 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t457262854", + "type": "text", + "x": 250.0, + "y": 238.0, + "width": 40, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1932256284, + "versionNonce": 66325576, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "wakeup", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "wakeup", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a358838350", + "type": "arrow", + "x": 250, + "y": 328, + "width": 0, + "height": 40, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "dashed", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 251009784, + "versionNonce": 1353160413, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + 40 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "a1514869305", + "type": "arrow", + "x": 730, + "y": 136, + "width": 0, + "height": 40, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 153770145, + "versionNonce": 142771292, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + 40 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "a1372968071", + "type": "arrow", + "x": 730, + "y": 232, + "width": 0, + "height": 40, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1626850937, + "versionNonce": 1918220603, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + 40 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t1414720222", + "type": "text", + "x": 730.0, + "y": 238.0, + "width": 40, + "height": 13.75, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 997950383, + "versionNonce": 186349422, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "submit", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "submit", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a1086038598", + "type": "arrow", + "x": 730, + "y": 328, + "width": 0, + "height": 40, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 763991811, + "versionNonce": 653235902, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + 40 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t1578321558", + "type": "text", + "x": 730.0, + "y": 334.0, + "width": 48.400000000000006, + "height": 13.75, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 687250798, + "versionNonce": 646220508, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "complete", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "complete", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t372392314", + "type": "text", + "x": 20, + "y": 16, + "width": 564.85, + "height": 16.25, + "angle": 0, + "strokeColor": "#1d1d1d", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 475170814, + "versionNonce": 1443847434, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "Readiness answers \u201cmay I?\u201d per fd; completion answers \u201cit's done\u201d per operation", + "fontSize": 13, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "Readiness answers \u201cmay I?\u201d per fd; completion answers \u201cit's done\u201d per operation", + "lineHeight": 1.25, + "baseline": 13 + }, + { + "id": "t459008290", + "type": "text", + "x": 60, + "y": 450, + "width": 344.85, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 30852995, + "versionNonce": 1932125645, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "fwatch rw engine on 403 files: 850 read(2) + 444 write(2)", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "fwatch rw engine on 403 files: 850 read(2) + 444 write(2)", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t1437876313", + "type": "text", + "x": 540, + "y": 450, + "width": 453.75000000000006, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 980183658, + "versionNonce": 1224274744, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "fwatch uring engine, same tree: 407 io_uring_enter(2), zero data read/write", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "fwatch uring engine, same tree: 407 io_uring_enter(2), zero data read/write", + "lineHeight": 1.25, + "baseline": 11 + } + ], + "appState": { + "viewBackgroundColor": "#ffffff", + "gridSize": null + }, + "files": {} +} \ No newline at end of file diff --git a/assets/diagrams/10-readiness-vs-completion.svg b/assets/diagrams/10-readiness-vs-completion.svg new file mode 100644 index 0000000..b680489 --- /dev/null +++ b/assets/diagrams/10-readiness-vs-completion.svg @@ -0,0 +1,46 @@ + + + +readiness — epoll (chapter 9) + +completion — io_uring (this chapter) + +epoll_wait(epfd, ...) +“tell me when these fds are ready” + +kernel: readiness events +fd 7 readable, fd 12 writable ... + +you still do the I/O yourself +read(fd) / write(fd) — one syscall per op + +and it can still fail with EAGAIN +readiness is a hint, not a result + +queue SQEs describing the I/O +“read 128 KiB at offset 0 into this buffer” + +io_uring_enter — submit N, wait M +one syscall carries the whole batch + +kernel does the I/O +inline if it can, io-wq workers if it would block + +CQEs carry finished results +cqe.res = bytes done — no second syscall + + +wakeup +wakeup + + + +submit +submit + +complete +complete +Readiness answers “may I?” per fd; completion answers “it's done” per operation +fwatch rw engine on 403 files: 850 read(2) + 444 write(2) +fwatch uring engine, same tree: 407 io_uring_enter(2), zero data read/write + \ No newline at end of file diff --git a/assets/diagrams/10-uring-rings.excalidraw b/assets/diagrams/10-uring-rings.excalidraw new file mode 100644 index 0000000..6667858 --- /dev/null +++ b/assets/diagrams/10-uring-rings.excalidraw @@ -0,0 +1,1952 @@ +{ + "type": "excalidraw", + "version": 2, + "source": "https://excalidraw.com", + "elements": [ + { + "id": "r1194578067", + "type": "rectangle", + "x": 20, + "y": 40, + "width": 280, + "height": 460, + "angle": 0, + "strokeColor": "#dddddd", + "backgroundColor": "#eef4fb", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1432745933, + "versionNonce": 1403368646, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t187777083", + "type": "text", + "x": 32, + "y": 46, + "width": 125.4, + "height": 15.0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1990875976, + "versionNonce": 1056454590, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "user space \u2014 fwatch", + "fontSize": 12, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "user space \u2014 fwatch", + "lineHeight": 1.25, + "baseline": 12 + }, + { + "id": "r746341140", + "type": "rectangle", + "x": 320, + "y": 40, + "width": 320, + "height": 460, + "angle": 0, + "strokeColor": "#dddddd", + "backgroundColor": "#fff8ef", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 674110981, + "versionNonce": 85874828, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1516908603", + "type": "text", + "x": 332, + "y": 46, + "width": 257.40000000000003, + "height": 15.0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1016879099, + "versionNonce": 305041173, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "shared memory \u2014 mmap'd from the ring fd", + "fontSize": 12, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "shared memory \u2014 mmap'd from the ring fd", + "lineHeight": 1.25, + "baseline": 12 + }, + { + "id": "r1403825575", + "type": "rectangle", + "x": 660, + "y": 40, + "width": 300, + "height": 460, + "angle": 0, + "strokeColor": "#dddddd", + "backgroundColor": "#f4f4f4", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1038770864, + "versionNonce": 1389051428, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1258133635", + "type": "text", + "x": 672, + "y": 46, + "width": 40, + "height": 15.0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 67454884, + "versionNonce": 579417868, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "kernel", + "fontSize": 12, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "kernel", + "lineHeight": 1.25, + "baseline": 12 + }, + { + "id": "r1115070571", + "type": "rectangle", + "x": 40, + "y": 90, + "width": 240, + "height": 74, + "angle": 0, + "strokeColor": "#2f6db5", + "backgroundColor": "#eef4fb", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 522550058, + "versionNonce": 245775918, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t390103154", + "type": "text", + "x": 50, + "y": 98, + "width": 130.9, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1182127178, + "versionNonce": 100899594, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "copy_file_uring()", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "copy_file_uring()", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t713312823", + "type": "text", + "x": 50, + "y": 118, + "width": 175.45000000000002, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1094408566, + "versionNonce": 390150582, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "fills up to 16 SQEs per batch", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "fills up to 16 SQEs per batch", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t1521541744", + "type": "text", + "x": 50, + "y": 133, + "width": 151.25, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 347872402, + "versionNonce": 469284224, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "8 linked READ\u2192WRITE pairs", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "8 linked READ\u2192WRITE pairs", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r993885577", + "type": "rectangle", + "x": 40, + "y": 210, + "width": 240, + "height": 64, + "angle": 0, + "strokeColor": "#2f6db5", + "backgroundColor": "#eef4fb", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1902705855, + "versionNonce": 848027236, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1896692317", + "type": "text", + "x": 50, + "y": 218, + "width": 184.8, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 135434233, + "versionNonce": 882691380, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "submit_and_wait(2\u00b7pairs)", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "submit_and_wait(2\u00b7pairs)", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t30499810", + "type": "text", + "x": 50, + "y": 238, + "width": 199.65, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 688213591, + "versionNonce": 1774431271, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "store-release SQ tail, then enter", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "store-release SQ tail, then enter", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r363658018", + "type": "rectangle", + "x": 40, + "y": 390, + "width": 240, + "height": 64, + "angle": 0, + "strokeColor": "#2f6db5", + "backgroundColor": "#eef4fb", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1401181858, + "versionNonce": 1119156208, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t500428054", + "type": "text", + "x": 50, + "y": 398, + "width": 138.60000000000002, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 452044760, + "versionNonce": 845113061, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "drain() \u2014 pop CQEs", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "drain() \u2014 pop CQEs", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1608696458", + "type": "text", + "x": 50, + "y": 418, + "width": 187.55, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 85287148, + "versionNonce": 1753391175, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "load-acquire CQ tail, bump head", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "load-acquire CQ tail, bump head", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r277586773", + "type": "rectangle", + "x": 345, + "y": 90, + "width": 270, + "height": 74, + "angle": 0, + "strokeColor": "#e8870c", + "backgroundColor": "#fff8ef", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1973187866, + "versionNonce": 103338748, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1278364784", + "type": "text", + "x": 355, + "y": 98, + "width": 53.900000000000006, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1559906621, + "versionNonce": 834154648, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "SQ ring", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "SQ ring", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1081524036", + "type": "text", + "x": 355, + "y": 118, + "width": 199.65, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1781182211, + "versionNonce": 1886621727, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "head \u00b7 tail \u00b7 ring_mask \u00b7 array[]", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "head \u00b7 tail \u00b7 ring_mask \u00b7 array[]", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t1436125137", + "type": "text", + "x": 355, + "y": 133, + "width": 157.3, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1976606470, + "versionNonce": 1485882909, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "indices into the SQE array", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "indices into the SQE array", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1242656111", + "type": "rectangle", + "x": 345, + "y": 200, + "width": 270, + "height": 74, + "angle": 0, + "strokeColor": "#e8870c", + "backgroundColor": "#fff8ef", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1174833788, + "versionNonce": 1084282264, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1613668180", + "type": "text", + "x": 355, + "y": 208, + "width": 223.3, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1288380161, + "versionNonce": 118866444, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "SQE array (64 \u00d7 io_uring_sqe)", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "SQE array (64 \u00d7 io_uring_sqe)", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1319504212", + "type": "text", + "x": 355, + "y": 228, + "width": 163.35000000000002, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 560037814, + "versionNonce": 1094256464, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "opcode, fd, addr, len, off,", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "opcode, fd, addr, len, off,", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t1373778980", + "type": "text", + "x": 355, + "y": 243, + "width": 193.60000000000002, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1154402047, + "versionNonce": 1902427406, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "flags (IOSQE_IO_LINK), user_data", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "flags (IOSQE_IO_LINK), user_data", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1970156935", + "type": "rectangle", + "x": 345, + "y": 390, + "width": 270, + "height": 74, + "angle": 0, + "strokeColor": "#e8870c", + "backgroundColor": "#fff8ef", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 227261676, + "versionNonce": 935481525, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t531876114", + "type": "text", + "x": 355, + "y": 398, + "width": 215.60000000000002, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1147849245, + "versionNonce": 791194409, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "CQ ring (128 \u00d7 io_uring_cqe)", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "CQ ring (128 \u00d7 io_uring_cqe)", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1893238273", + "type": "text", + "x": 355, + "y": 418, + "width": 121.00000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1091822007, + "versionNonce": 341663172, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "head \u00b7 tail \u00b7 cqes[]", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "head \u00b7 tail \u00b7 cqes[]", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t1471888620", + "type": "text", + "x": 355, + "y": 433, + "width": 187.55, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 607481768, + "versionNonce": 227882214, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "cqe.res = result, cqe.user_data", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "cqe.res = result, cqe.user_data", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1230566763", + "type": "rectangle", + "x": 680, + "y": 90, + "width": 260, + "height": 64, + "angle": 0, + "strokeColor": "#888888", + "backgroundColor": "#f4f4f4", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1974884892, + "versionNonce": 1612502096, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1140723003", + "type": "text", + "x": 690, + "y": 98, + "width": 207.9, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 449537102, + "versionNonce": 1763722270, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "io_uring_setup(64, ¶ms)", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "io_uring_setup(64, ¶ms)", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t558366777", + "type": "text", + "x": 690, + "y": 118, + "width": 169.4, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 356951889, + "versionNonce": 265195155, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "one fd; rings mmap'd from it", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "one fd; rings mmap'd from it", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1277517154", + "type": "rectangle", + "x": 680, + "y": 210, + "width": 260, + "height": 74, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#111111", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 877421535, + "versionNonce": 923637406, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t225673231", + "type": "text", + "x": 690, + "y": 218, + "width": 200.20000000000002, + "height": 17.5, + "angle": 0, + "strokeColor": "#ffffff", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 253798637, + "versionNonce": 1070967983, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "io_uring_enter(fd, 16, 16,", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "io_uring_enter(fd, 16, 16,", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t730994382", + "type": "text", + "x": 690, + "y": 238, + "width": 60.50000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1236698679, + "versionNonce": 1092382801, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "GETEVENTS)", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "GETEVENTS)", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t819900026", + "type": "text", + "x": 690, + "y": 253, + "width": 169.4, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 735050112, + "versionNonce": 324373377, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "submit N + wait, one syscall", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "submit N + wait, one syscall", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1432660474", + "type": "rectangle", + "x": 680, + "y": 390, + "width": 260, + "height": 64, + "angle": 0, + "strokeColor": "#888888", + "backgroundColor": "#f4f4f4", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 219795957, + "versionNonce": 985683994, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t577761114", + "type": "text", + "x": 690, + "y": 398, + "width": 192.50000000000003, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1726733739, + "versionNonce": 1598751140, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "executes SQEs, posts CQEs", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "executes SQEs, posts CQEs", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1387746443", + "type": "text", + "x": 690, + "y": 418, + "width": 127.05000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 496253271, + "versionNonce": 911561063, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "store-release CQ tail", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "store-release CQ tail", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a851087830", + "type": "arrow", + "x": 280, + "y": 127, + "width": 65, + "height": 0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 774528917, + "versionNonce": 1895589357, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 65, + 0 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t1664237923", + "type": "text", + "x": 312.5, + "y": 113.0, + "width": 60.50000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1834904520, + "versionNonce": 663628515, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "write SQEs", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "write SQEs", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a400443605", + "type": "arrow", + "x": 280, + "y": 242, + "width": 65, + "height": 5, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1907292294, + "versionNonce": 1876069224, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 65, + -5 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t893302524", + "type": "text", + "x": 312.5, + "y": 225.5, + "width": 72.60000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1222612127, + "versionNonce": 1282120, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "publish tail", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "publish tail", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a1003424266", + "type": "arrow", + "x": 160, + "y": 274, + "width": 0, + "height": 116, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1614388107, + "versionNonce": 1408214289, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + 116 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t554295883", + "type": "text", + "x": 160.0, + "y": 318.0, + "width": 114.95, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1547728145, + "versionNonce": 1789096872, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "after enter returns", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "after enter returns", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a69301964", + "type": "arrow", + "x": 345, + "y": 422, + "width": 65, + "height": 0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 946706598, + "versionNonce": 840643814, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + -65, + 0 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t1381064832", + "type": "text", + "x": 312.5, + "y": 408.0, + "width": 48.400000000000006, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1056591568, + "versionNonce": 1025836980, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "pop CQEs", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "pop CQEs", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a437326388", + "type": "arrow", + "x": 810, + "y": 154, + "width": 0, + "height": 56, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1755131259, + "versionNonce": 487995553, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + 56 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t1768910332", + "type": "text", + "x": 810.0, + "y": 168.0, + "width": 42.35, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 445397520, + "versionNonce": 1841696793, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "ring fd", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "ring fd", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a1354060795", + "type": "arrow", + "x": 680, + "y": 247, + "width": 65, + "height": 10, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "dashed", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 7724102, + "versionNonce": 1020655574, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + -65, + -10 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t1732496750", + "type": "text", + "x": 647.5, + "y": 228.0, + "width": 60.50000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 72902619, + "versionNonce": 1819981876, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "consume SQ", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "consume SQ", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a1733210176", + "type": "arrow", + "x": 810, + "y": 284, + "width": 0, + "height": 106, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 19086355, + "versionNonce": 1469660311, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + 106 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "a648991244", + "type": "arrow", + "x": 680, + "y": 422, + "width": 65, + "height": 0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "dashed", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 254082016, + "versionNonce": 891904203, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + -65, + 0 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t1876098735", + "type": "text", + "x": 647.5, + "y": 408.0, + "width": 72.60000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 91954189, + "versionNonce": 1094609859, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "post results", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "post results", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t1795232151", + "type": "text", + "x": 20, + "y": 16, + "width": 650.6500000000001, + "height": 16.25, + "angle": 0, + "strokeColor": "#1d1d1d", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1504704290, + "versionNonce": 1885547459, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "io_uring: two lock-free rings shared with the kernel \u2014 describe operations, don't call them", + "fontSize": 13, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "io_uring: two lock-free rings shared with the kernel \u2014 describe operations, don't call them", + "lineHeight": 1.25, + "baseline": 13 + }, + { + "id": "t1249113351", + "type": "text", + "x": 320, + "y": 520, + "width": 629.2, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1503251653, + "versionNonce": 1163734440, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "on this host (kernel 7.1.3): IORING_FEAT_SINGLE_MMAP \u2014 SQ+CQ rings in one 2368-byte mapping at offset 0,", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "on this host (kernel 7.1.3): IORING_FEAT_SINGLE_MMAP \u2014 SQ+CQ rings in one 2368-byte mapping at offset 0,", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t30621459", + "type": "text", + "x": 320, + "y": 536, + "width": 369.05, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 491804051, + "versionNonce": 1378984362, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "SQE array in a second mapping at IORING_OFF_SQES (0x10000000)", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "SQE array in a second mapping at IORING_OFF_SQES (0x10000000)", + "lineHeight": 1.25, + "baseline": 11 + } + ], + "appState": { + "viewBackgroundColor": "#ffffff", + "gridSize": null + }, + "files": {} +} \ No newline at end of file diff --git a/assets/diagrams/10-uring-rings.svg b/assets/diagrams/10-uring-rings.svg new file mode 100644 index 0000000..ba903c7 --- /dev/null +++ b/assets/diagrams/10-uring-rings.svg @@ -0,0 +1,66 @@ + + + +user space — fwatch + +shared memory — mmap'd from the ring fd + +kernel + +copy_file_uring() +fills up to 16 SQEs per batch +8 linked READ→WRITE pairs + +submit_and_wait(2·pairs) +store-release SQ tail, then enter + +drain() — pop CQEs +load-acquire CQ tail, bump head + +SQ ring +head · tail · ring_mask · array[] +indices into the SQE array + +SQE array (64 × io_uring_sqe) +opcode, fd, addr, len, off, +flags (IOSQE_IO_LINK), user_data + +CQ ring (128 × io_uring_cqe) +head · tail · cqes[] +cqe.res = result, cqe.user_data + +io_uring_setup(64, &params) +one fd; rings mmap'd from it + +io_uring_enter(fd, 16, 16, +GETEVENTS) +submit N + wait, one syscall + +executes SQEs, posts CQEs +store-release CQ tail + +write SQEs +write SQEs + +publish tail +publish tail + +after enter returns +after enter returns + +pop CQEs +pop CQEs + +ring fd +ring fd + +consume SQ +consume SQ + + +post results +post results +io_uring: two lock-free rings shared with the kernel — describe operations, don't call them +on this host (kernel 7.1.3): IORING_FEAT_SINGLE_MMAP — SQ+CQ rings in one 2368-byte mapping at offset 0, +SQE array in a second mapping at IORING_OFF_SQES (0x10000000) + \ No newline at end of file diff --git a/assets/diagrams/README.md b/assets/diagrams/README.md index 7e37603..48bdabe 100644 --- a/assets/diagrams/README.md +++ b/assets/diagrams/README.md @@ -11,3 +11,13 @@ here as you add them: | `01-toolchain-landscape` | 1 | Three language toolchain stacks converging on the shared demo.sh contract | | `02-lab-topology` | 2 | Host, libvirt NAT network, and the systems-target/systems-peer guests | | `03-telemetry-pipeline` | 3 | Demos exporting OTLP into the lsp-lgtm container and out to Grafana | +| `07-fd-table-vfs` | 7 | fd table → open file description → inode, annotated with the four dirfds of a paused fwatch walk | +| `05-error-flow-three-ways` | 5 | One ENOSPC's journey from write(2) through errno into each language's error type, converging on identical output and exit 3 | +| `08-write-path-page-cache` | 8 | The write path user buffer → page cache → device, with each iobench mode's bypass and flush points | +| `04-user-kernel-path` | 4 | One getrandom call crossing the user/kernel boundary, and the vDSO fast path that never traps | +| `04-runtime-syscall-layers` | 4 | The C++/Go/Rust layer stacks converging on the one register-level kernel ABI, with real strace -c totals | +| `10-uring-rings` | 10 | The SQ ring, SQE array, and CQ ring shared via mmap, the acquire/release handoffs around them, and io_uring_enter as the only syscall | +| `10-readiness-vs-completion` | 10 | Readiness (epoll answers "may I?") beside completion (io_uring answers "it is done"), with the real per-engine syscall counts from fwatch | +| `09-event-loop-anatomy` | 9 | One epoll loop over three event sources — inotify, timerfd, signalfd — with the kernel eventpoll interest list and the readiness path back into epoll_wait | +| `06-three-concurrency-models` | 6 | parhash's producer/queue/workers/cancellation shape in all three vocabularies, with observed /proc thread counts | +| `06-cancellation-drain` | 6 | The SIGINT drain sequence — one flag flips, walker and workers wind down, in-flight hashes finish, exit 130 | diff --git a/examples/04-syscalls-and-the-abi/README.md b/examples/04-syscalls-and-the-abi/README.md new file mode 100644 index 0000000..32f194d --- /dev/null +++ b/examples/04-syscalls-and-the-abi/README.md @@ -0,0 +1,64 @@ +# 04-syscalls-and-the-abi — sysprobe + +A syscall specimen: the same small program three times — C++23, Go, and +Rust — performing a fixed, labeled sequence of system calls so you can watch +each language's runtime hit the kernel ABI under `strace(1)`: + +1. **open** — `openat(2)` of a scratch file in `$TMPDIR` (default `/tmp`), + preferring an anonymous `O_TMPFILE` inode, with a `mkstemp`-style + create+unlink fallback on filesystems without `O_TMPFILE` support +2. **write** — `write(2)` of a 25-byte payload to that fd, then `close(2)` + (RAII scope exit in C++, checked `unix.Close` in Go, `OwnedFd` drop in Rust) +3. **sleep** — `nanosleep(2)` for 10 ms, resuming on `EINTR` +4. **random** — `getrandom(2)` for 16 bytes, looping over partial reads + +## CLI and output + +The binary is named `app` and takes no arguments (`app ` prints +`usage: app` to stderr and exits 2). On success it prints exactly: + +``` +step=open ok +step=write ok +step=sleep ok +step=random ok +sysprobe: 4 steps ok +``` + +and exits 0. On failure it prints `sysprobe: :
` to stderr and +exits 1. All three implementations expose identical observable behavior, so +one `verify.lua` covers them all. + +## Watching it under strace + +The point of the program. For example: + +``` +strace -e trace=openat,write,close,nanosleep,getrandom ./cpp/build/release/app +``` + +shows the labeled sequence — `openat(... O_TMPFILE|O_RDWR|O_CLOEXEC)`, the +25-byte `write`, the `close`, `nanosleep({tv_sec=0, tv_nsec=10000000})`, and +`getrandom(..., 16, 0) = 16` — interleaved with each runtime's own startup +syscalls, which is exactly the contrast the chapter is about. + +## The demo contract + +Every language directory has a `demo.sh` with the same interface: + +- `./demo.sh` — build then run +- `./demo.sh build` — build only +- `./demo.sh run [args]` — run the built binary +- With env `TARGET` set, `run` deploys the binary to that lab VM via + `scripts/lab/deploy-to-vm.sh "$TARGET" -- [args]` instead of + running locally. + +The top-level `demo.sh` is the dispatcher: `./demo.sh cpp run` execs +`cpp/demo.sh run`; bare `./demo.sh` (or `all`) builds and runs all three. + +## Verification + +`verify.lua` runs `./demo.sh $LSP_LANG run` via `scripts/lib/checks.lua` and +asserts exit 0, all four `step= ok` lines, the `sysprobe: 4 steps ok` +summary, and that the five lines appear in order as one contiguous block. +Exit 0 = pass, 1 = fail, 77 = skip. diff --git a/examples/04-syscalls-and-the-abi/cpp/.clang-format b/examples/04-syscalls-and-the-abi/cpp/.clang-format new file mode 100644 index 0000000..2c8640f --- /dev/null +++ b/examples/04-syscalls-and-the-abi/cpp/.clang-format @@ -0,0 +1,8 @@ +--- +BasedOnStyle: LLVM +Language: Cpp +Standard: c++20 +IndentWidth: 4 +ColumnLimit: 100 +PointerAlignment: Left +AllowShortFunctionsOnASingleLine: Inline diff --git a/examples/04-syscalls-and-the-abi/cpp/.clang-tidy b/examples/04-syscalls-and-the-abi/cpp/.clang-tidy new file mode 100644 index 0000000..2f275ba --- /dev/null +++ b/examples/04-syscalls-and-the-abi/cpp/.clang-tidy @@ -0,0 +1,11 @@ +--- +Checks: > + bugprone-*, + modernize-*, + performance-*, + readability-*, + -modernize-use-trailing-return-type, + -readability-identifier-length +WarningsAsErrors: '' +HeaderFilterRegex: '.*' +FormatStyle: file diff --git a/examples/04-syscalls-and-the-abi/cpp/CMakeLists.txt b/examples/04-syscalls-and-the-abi/cpp/CMakeLists.txt new file mode 100644 index 0000000..2de38a8 --- /dev/null +++ b/examples/04-syscalls-and-the-abi/cpp/CMakeLists.txt @@ -0,0 +1,8 @@ +cmake_minimum_required(VERSION 3.25) +project(sysprobe CXX) + +set(CMAKE_CXX_STANDARD 23) +set(CMAKE_CXX_STANDARD_REQUIRED ON) +set(CMAKE_CXX_EXTENSIONS OFF) + +add_executable(app src/main.cpp) diff --git a/examples/04-syscalls-and-the-abi/cpp/CMakePresets.json b/examples/04-syscalls-and-the-abi/cpp/CMakePresets.json new file mode 100644 index 0000000..e31bd2f --- /dev/null +++ b/examples/04-syscalls-and-the-abi/cpp/CMakePresets.json @@ -0,0 +1,62 @@ +{ + "version": 6, + "cmakeMinimumRequired": { "major": 3, "minor": 25, "patch": 0 }, + "configurePresets": [ + { + "name": "_base", + "hidden": true, + "generator": "Ninja", + "binaryDir": "${sourceDir}/build/${presetName}", + "cacheVariables": { + "CMAKE_EXPORT_COMPILE_COMMANDS": "ON" + } + }, + { + "name": "release", + "inherits": "_base", + "displayName": "Release (-O2 with debug info)", + "cacheVariables": { "CMAKE_BUILD_TYPE": "RelWithDebInfo" } + }, + { + "name": "debug", + "inherits": "_base", + "displayName": "Debug", + "cacheVariables": { "CMAKE_BUILD_TYPE": "Debug" } + }, + { + "name": "asan", + "inherits": "_base", + "displayName": "ASan + UBSan instrumented build", + "cacheVariables": { + "CMAKE_BUILD_TYPE": "Debug", + "CMAKE_CXX_FLAGS": "-O1 -g -fno-omit-frame-pointer -fsanitize=address,undefined", + "CMAKE_EXE_LINKER_FLAGS": "-fsanitize=address,undefined" + } + }, + { + "name": "release-clang", + "inherits": "release", + "displayName": "Release (clang)", + "cacheVariables": { + "CMAKE_C_COMPILER": "clang", + "CMAKE_CXX_COMPILER": "clang++" + } + }, + { + "name": "debug-clang", + "inherits": "debug", + "displayName": "Debug (clang)", + "cacheVariables": { + "CMAKE_C_COMPILER": "clang", + "CMAKE_CXX_COMPILER": "clang++" + } + } + ], + "buildPresets": [ + { "name": "release", "configurePreset": "release" }, + { "name": "debug", "configurePreset": "debug" }, + { "name": "asan", "configurePreset": "asan" }, + { "name": "release-clang", "configurePreset": "release-clang" }, + { "name": "debug-clang", "configurePreset": "debug-clang" } + ] +} diff --git a/examples/04-syscalls-and-the-abi/cpp/demo.sh b/examples/04-syscalls-and-the-abi/cpp/demo.sh new file mode 100755 index 0000000..6af6509 --- /dev/null +++ b/examples/04-syscalls-and-the-abi/cpp/demo.sh @@ -0,0 +1,40 @@ +#!/usr/bin/env bash +set -euo pipefail + +cd "$(dirname "${BASH_SOURCE[0]}")" +REPO_ROOT="$(cd ../../.. && pwd)" +BIN=build/release/app + +build() { + cmake --preset release + cmake --build --preset release +} + +run() { + if [[ ! -x "$BIN" ]]; then + build + fi + if [[ -n "${TARGET:-}" ]]; then + "$REPO_ROOT/scripts/lab/deploy-to-vm.sh" "$TARGET" "$BIN" -- "$@" + else + "./$BIN" "$@" + fi +} + +case "${1:-}" in + build) + build + ;; + run) + shift + run "$@" + ;; + "") + build + run + ;; + *) + echo "usage: $0 [build|run [args]]" >&2 + exit 2 + ;; +esac diff --git a/examples/04-syscalls-and-the-abi/cpp/src/main.cpp b/examples/04-syscalls-and-the-abi/cpp/src/main.cpp new file mode 100644 index 0000000..1f53973 --- /dev/null +++ b/examples/04-syscalls-and-the-abi/cpp/src/main.cpp @@ -0,0 +1,170 @@ +// sysprobe: a labeled syscall specimen — openat+write+close of an unlinked +// temp file, a 10 ms nanosleep, and 16 bytes of getrandom — built to be +// watched under strace(1). Prints "step= ok" per step, then a summary. + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include + +namespace { + +[[nodiscard]] std::error_code last_error() { + return {errno, std::system_category()}; +} + +// Owning RAII wrapper for a file descriptor; the destructor issues close(2). +class unique_fd { +public: + unique_fd() = default; + explicit unique_fd(int fd) : fd_(fd) {} + unique_fd(const unique_fd&) = delete; + unique_fd& operator=(const unique_fd&) = delete; + unique_fd(unique_fd&& other) noexcept : fd_(std::exchange(other.fd_, -1)) {} + unique_fd& operator=(unique_fd&& other) noexcept { + if (this != &other) { + reset(); + fd_ = std::exchange(other.fd_, -1); + } + return *this; + } + ~unique_fd() { reset(); } + + [[nodiscard]] int get() const { return fd_; } + void reset() { + if (fd_ >= 0) { + ::close(fd_); + fd_ = -1; + } + } + +private: + int fd_ = -1; +}; + +// Step 1: openat(2). Prefer an anonymous O_TMPFILE inode (no name ever +// appears in the directory); fall back to mkstemp(3) + unlink(2) on +// filesystems that lack O_TMPFILE. +[[nodiscard]] std::expected +open_scratch(const std::string& dir) { + if (const int fd = + ::openat(AT_FDCWD, dir.c_str(), O_TMPFILE | O_RDWR | O_CLOEXEC, 0600); + fd >= 0) { + return unique_fd{fd}; + } + if (errno != EOPNOTSUPP && errno != EISDIR && errno != EINVAL) { + return std::unexpected(last_error()); + } + std::string tmpl = dir + "/sysprobe.XXXXXX"; + const int fd = ::mkstemp(tmpl.data()); + if (fd < 0) { + return std::unexpected(last_error()); + } + unique_fd owned{fd}; + if (::unlink(tmpl.c_str()) != 0) { + return std::unexpected(last_error()); + } + return owned; +} + +// Step 2: write(2), retrying on EINTR and short writes. +[[nodiscard]] std::expected +write_all(const unique_fd& fd, std::span buf) { + while (!buf.empty()) { + const ssize_t n = ::write(fd.get(), buf.data(), buf.size()); + if (n < 0) { + if (errno == EINTR) { + continue; + } + return std::unexpected(last_error()); + } + buf = buf.subspan(static_cast(n)); + } + return {}; +} + +// Step 3: nanosleep(2), resuming with the remaining time on EINTR. +[[nodiscard]] std::expected sleep_ms(long ms) { + timespec req{.tv_sec = 0, .tv_nsec = ms * 1'000'000L}; + timespec rem{}; + while (::nanosleep(&req, &rem) != 0) { + if (errno != EINTR) { + return std::unexpected(last_error()); + } + req = rem; + } + return {}; +} + +// Step 4: getrandom(2), looping over partial reads. +[[nodiscard]] std::expected +fill_random(std::span buf) { + while (!buf.empty()) { + const ssize_t n = ::getrandom(buf.data(), buf.size(), 0); + if (n < 0) { + if (errno == EINTR) { + continue; + } + return std::unexpected(last_error()); + } + buf = buf.subspan(static_cast(n)); + } + return {}; +} + +} // namespace + +int main(int argc, char**) { + if (argc > 1) { + std::println(stderr, "usage: app"); + return 2; + } + + const char* env = std::getenv("TMPDIR"); + const std::string dir = (env != nullptr && *env != '\0') ? env : "/tmp"; + + { + auto fd = open_scratch(dir); + if (!fd) { + std::println(stderr, "sysprobe: open: {}", fd.error().message()); + return 1; + } + std::println("step=open ok"); + + constexpr std::string_view payload = "sysprobe scratch payload\n"; + if (auto w = write_all(*fd, std::span{payload.data(), payload.size()}); !w) { + std::println(stderr, "sysprobe: write: {}", w.error().message()); + return 1; + } + std::println("step=write ok"); + } // unique_fd destructor issues close(2) here, before the sleep + + if (auto s = sleep_ms(10); !s) { + std::println(stderr, "sysprobe: sleep: {}", s.error().message()); + return 1; + } + std::println("step=sleep ok"); + + std::array entropy{}; + if (auto r = fill_random(entropy); !r) { + std::println(stderr, "sysprobe: random: {}", r.error().message()); + return 1; + } + std::println("step=random ok"); + + std::println("sysprobe: 4 steps ok"); + return 0; +} diff --git a/examples/04-syscalls-and-the-abi/demo.sh b/examples/04-syscalls-and-the-abi/demo.sh new file mode 100755 index 0000000..afbbe4c --- /dev/null +++ b/examples/04-syscalls-and-the-abi/demo.sh @@ -0,0 +1,23 @@ +#!/usr/bin/env bash +set -euo pipefail +cd "$(dirname "$0")" + +langs=(cpp go rust) + +case "${1:-all}" in + cpp|go|rust) + lang="$1" + shift + exec "./$lang/demo.sh" "$@" + ;; + all) + for lang in "${langs[@]}"; do "./$lang/demo.sh"; done + ;; + build) + for lang in "${langs[@]}"; do "./$lang/demo.sh" build; done + ;; + *) + echo "usage: $0 [cpp|go|rust|all|build] [args...]" >&2 + exit 2 + ;; +esac diff --git a/examples/04-syscalls-and-the-abi/go/demo.sh b/examples/04-syscalls-and-the-abi/go/demo.sh new file mode 100755 index 0000000..28606df --- /dev/null +++ b/examples/04-syscalls-and-the-abi/go/demo.sh @@ -0,0 +1,23 @@ +#!/usr/bin/env bash +set -euo pipefail +cd "$(dirname "$0")" +REPO_ROOT="$(cd ../../.. && pwd)" +BIN=bin/app + +build() { + go build -o "$BIN" . +} + +run() { + if [[ -n "${TARGET:-}" ]]; then + exec "$REPO_ROOT/scripts/lab/deploy-to-vm.sh" "$TARGET" "$BIN" -- "$@" + fi + exec "./$BIN" "$@" +} + +case "${1:-}" in + build) build ;; + run) shift; run "$@" ;; + "") build; run ;; + *) echo "usage: $0 [build|run [args...]]" >&2; exit 2 ;; +esac diff --git a/examples/04-syscalls-and-the-abi/go/go.mod b/examples/04-syscalls-and-the-abi/go/go.mod new file mode 100644 index 0000000..98bba97 --- /dev/null +++ b/examples/04-syscalls-and-the-abi/go/go.mod @@ -0,0 +1,7 @@ +module github.com/patterncatalyst/linux-systems-programming/examples/04-syscalls-and-the-abi/go + +go 1.26 + +toolchain go1.26.5 + +require golang.org/x/sys v0.47.0 diff --git a/examples/04-syscalls-and-the-abi/go/go.sum b/examples/04-syscalls-and-the-abi/go/go.sum new file mode 100644 index 0000000..37ee2d4 --- /dev/null +++ b/examples/04-syscalls-and-the-abi/go/go.sum @@ -0,0 +1,2 @@ +golang.org/x/sys v0.47.0 h1:o7XGOvZQCADBQQ4Y7VNq2dRWQR7JmOUW8Kxx4ZsNgWs= +golang.org/x/sys v0.47.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw= diff --git a/examples/04-syscalls-and-the-abi/go/main.go b/examples/04-syscalls-and-the-abi/go/main.go new file mode 100644 index 0000000..1a0ad67 --- /dev/null +++ b/examples/04-syscalls-and-the-abi/go/main.go @@ -0,0 +1,126 @@ +// sysprobe: a labeled syscall specimen — openat+write+close of an unlinked +// temp file, a 10 ms nanosleep, and 16 bytes of getrandom — built to be +// watched under strace(1). Prints "step= ok" per step, then a summary. +package main + +import ( + "errors" + "fmt" + "os" + + "golang.org/x/sys/unix" +) + +const payload = "sysprobe scratch payload\n" + +// openScratch is step 1: openat(2). Prefer an anonymous O_TMPFILE inode (no +// name ever appears in the directory); fall back to a named create+unlink on +// filesystems that lack O_TMPFILE. +func openScratch(dir string) (int, error) { + fd, err := unix.Openat(unix.AT_FDCWD, dir, + unix.O_TMPFILE|unix.O_RDWR|unix.O_CLOEXEC, 0o600) + if err == nil { + return fd, nil + } + if !errors.Is(err, unix.EOPNOTSUPP) && !errors.Is(err, unix.EISDIR) && + !errors.Is(err, unix.EINVAL) { + return -1, fmt.Errorf("openat %s (O_TMPFILE): %w", dir, err) + } + path := fmt.Sprintf("%s/sysprobe.%d", dir, os.Getpid()) + fd, err = unix.Openat(unix.AT_FDCWD, path, + unix.O_CREAT|unix.O_EXCL|unix.O_RDWR|unix.O_CLOEXEC, 0o600) + if err != nil { + return -1, fmt.Errorf("openat %s: %w", path, err) + } + if err := unix.Unlink(path); err != nil { + unix.Close(fd) + return -1, fmt.Errorf("unlink %s: %w", path, err) + } + return fd, nil +} + +// writeAll is step 2: write(2), retrying on EINTR and short writes. +func writeAll(fd int, buf []byte) error { + for len(buf) > 0 { + n, err := unix.Write(fd, buf) + if err != nil { + if errors.Is(err, unix.EINTR) { + continue + } + return fmt.Errorf("write: %w", err) + } + buf = buf[n:] + } + return nil +} + +// sleepMS is step 3: nanosleep(2), resuming with the remaining time on EINTR. +func sleepMS(ms int64) error { + req := unix.Timespec{Nsec: ms * 1_000_000} + var rem unix.Timespec + for { + err := unix.Nanosleep(&req, &rem) + if err == nil { + return nil + } + if !errors.Is(err, unix.EINTR) { + return fmt.Errorf("nanosleep: %w", err) + } + req = rem + } +} + +// fillRandom is step 4: getrandom(2), looping over partial reads. +func fillRandom(buf []byte) error { + for len(buf) > 0 { + n, err := unix.Getrandom(buf, 0) + if err != nil { + if errors.Is(err, unix.EINTR) { + continue + } + return fmt.Errorf("getrandom: %w", err) + } + buf = buf[n:] + } + return nil +} + +func fail(step string, err error) { + fmt.Fprintf(os.Stderr, "sysprobe: %s: %v\n", step, err) + os.Exit(1) +} + +func main() { + if len(os.Args) > 1 { + fmt.Fprintln(os.Stderr, "usage: app") + os.Exit(2) + } + + fd, err := openScratch(os.TempDir()) + if err != nil { + fail("open", err) + } + fmt.Println("step=open ok") + + if err := writeAll(fd, []byte(payload)); err != nil { + fail("write", err) + } + fmt.Println("step=write ok") + + if err := unix.Close(fd); err != nil { // close(2) before the sleep + fail("close", err) + } + + if err := sleepMS(10); err != nil { + fail("sleep", err) + } + fmt.Println("step=sleep ok") + + entropy := make([]byte, 16) + if err := fillRandom(entropy); err != nil { + fail("random", err) + } + fmt.Println("step=random ok") + + fmt.Println("sysprobe: 4 steps ok") +} diff --git a/examples/04-syscalls-and-the-abi/rust/Cargo.lock b/examples/04-syscalls-and-the-abi/rust/Cargo.lock new file mode 100644 index 0000000..bd2d4ed --- /dev/null +++ b/examples/04-syscalls-and-the-abi/rust/Cargo.lock @@ -0,0 +1,73 @@ +# This file is automatically @generated by Cargo. +# It is not intended for manual editing. +version = 4 + +[[package]] +name = "anyhow" +version = "1.0.104" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "330a5ed07fa54e4702c9d6c4174f74427fc0ef6e214bbd677ae50a5099946470" + +[[package]] +name = "bitflags" +version = "2.13.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b588b76d00fde79687d7646a9b5bdf3cc0f655e0bbd080335a95d7e96f3587da" + +[[package]] +name = "errno" +version = "0.3.14" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "39cab71617ae0d63f51a36d69f866391735b51691dbda63cf6f96d042b63efeb" +dependencies = [ + "libc", + "windows-sys", +] + +[[package]] +name = "libc" +version = "0.2.186" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "68ab91017fe16c622486840e4c83c9a37afeff978bd239b5293d61ece587de66" + +[[package]] +name = "linux-raw-sys" +version = "0.12.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "32a66949e030da00e8c7d4434b251670a91556f4144941d37452769c25d58a53" + +[[package]] +name = "rustix" +version = "1.1.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b6fe4565b9518b83ef4f91bb47ce29620ca828bd32cb7e408f0062e9930ba190" +dependencies = [ + "bitflags", + "errno", + "libc", + "linux-raw-sys", + "windows-sys", +] + +[[package]] +name = "sysprobe" +version = "0.1.0" +dependencies = [ + "anyhow", + "rustix", +] + +[[package]] +name = "windows-link" +version = "0.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5" + +[[package]] +name = "windows-sys" +version = "0.61.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ae137229bcbd6cdf0f7b80a31df61766145077ddf49416a728b02cb3921ff3fc" +dependencies = [ + "windows-link", +] diff --git a/examples/04-syscalls-and-the-abi/rust/Cargo.toml b/examples/04-syscalls-and-the-abi/rust/Cargo.toml new file mode 100644 index 0000000..1362a47 --- /dev/null +++ b/examples/04-syscalls-and-the-abi/rust/Cargo.toml @@ -0,0 +1,12 @@ +[package] +name = "sysprobe" +version = "0.1.0" +edition = "2024" + +[[bin]] +name = "app" +path = "src/main.rs" + +[dependencies] +anyhow = "1" +rustix = { version = "1", features = ["fs", "rand", "thread"] } diff --git a/examples/04-syscalls-and-the-abi/rust/demo.sh b/examples/04-syscalls-and-the-abi/rust/demo.sh new file mode 100755 index 0000000..220feea --- /dev/null +++ b/examples/04-syscalls-and-the-abi/rust/demo.sh @@ -0,0 +1,26 @@ +#!/usr/bin/env bash +set -euo pipefail +cd "$(dirname "${BASH_SOURCE[0]}")" + +BIN=target/release/app + +build() { + cargo build --release +} + +run() { + if [[ -n "${TARGET:-}" ]]; then + local repo_root + repo_root=$(cd ../../.. && pwd) + "$repo_root/scripts/lab/deploy-to-vm.sh" "$TARGET" "$BIN" -- "$@" + else + "./$BIN" "$@" + fi +} + +case "${1:-}" in + build) build ;; + run) shift; run "$@" ;; + "") build; run ;; + *) echo "usage: $0 [build|run [args...]]" >&2; exit 2 ;; +esac diff --git a/examples/04-syscalls-and-the-abi/rust/rust-toolchain.toml b/examples/04-syscalls-and-the-abi/rust/rust-toolchain.toml new file mode 100644 index 0000000..4099f1a --- /dev/null +++ b/examples/04-syscalls-and-the-abi/rust/rust-toolchain.toml @@ -0,0 +1,3 @@ +[toolchain] +channel = "1.97.1" +components = ["rustfmt", "clippy", "rust-src"] diff --git a/examples/04-syscalls-and-the-abi/rust/src/main.rs b/examples/04-syscalls-and-the-abi/rust/src/main.rs new file mode 100644 index 0000000..00bedf9 --- /dev/null +++ b/examples/04-syscalls-and-the-abi/rust/src/main.rs @@ -0,0 +1,117 @@ +//! sysprobe: a labeled syscall specimen — openat+write+close of an unlinked +//! temp file, a 10 ms nanosleep, and 16 bytes of getrandom — built to be +//! watched under strace(1). Prints "step= ok" per step, then a summary. + +use std::os::fd::{AsFd, BorrowedFd, OwnedFd}; +use std::process::ExitCode; + +use anyhow::{Context, Result, bail}; +use rustix::fs::{self, AtFlags, CWD, Mode, OFlags, Timespec}; +use rustix::io::Errno; +use rustix::rand::{GetRandomFlags, getrandom}; +use rustix::thread::{NanosleepRelativeResult, nanosleep}; + +const PAYLOAD: &[u8] = b"sysprobe scratch payload\n"; + +/// Step 1: openat(2). Prefer an anonymous O_TMPFILE inode (no name ever +/// appears in the directory); fall back to a named create+unlink on +/// filesystems that lack O_TMPFILE. +fn open_scratch(dir: &str) -> Result { + let mode = Mode::RUSR | Mode::WUSR; + match fs::openat( + CWD, + dir, + OFlags::TMPFILE | OFlags::RDWR | OFlags::CLOEXEC, + mode, + ) { + Ok(fd) => return Ok(fd), + Err(Errno::OPNOTSUPP | Errno::ISDIR | Errno::INVAL) => {} + Err(e) => return Err(e).with_context(|| format!("openat {dir} (O_TMPFILE)")), + } + let path = format!("{dir}/sysprobe.{}", std::process::id()); + let fd = fs::openat( + CWD, + &path, + OFlags::CREATE | OFlags::EXCL | OFlags::RDWR | OFlags::CLOEXEC, + mode, + ) + .with_context(|| format!("openat {path}"))?; + fs::unlinkat(CWD, &path, AtFlags::empty()).with_context(|| format!("unlink {path}"))?; + Ok(fd) +} + +/// Step 2: write(2), retrying on EINTR and short writes. +fn write_all(fd: BorrowedFd<'_>, mut buf: &[u8]) -> Result<()> { + while !buf.is_empty() { + match rustix::io::write(fd, buf) { + Ok(n) => buf = &buf[n..], + Err(Errno::INTR) => continue, + Err(e) => return Err(e).context("write"), + } + } + Ok(()) +} + +/// Step 3: nanosleep(2), resuming with the remaining time on EINTR. +fn sleep_ms(ms: i64) -> Result<()> { + let mut req = Timespec { + tv_sec: 0, + tv_nsec: ms * 1_000_000, + }; + loop { + match nanosleep(&req) { + NanosleepRelativeResult::Ok => return Ok(()), + NanosleepRelativeResult::Interrupted(rem) => req = rem, + NanosleepRelativeResult::Err(e) => bail!("nanosleep: {e}"), + } + } +} + +/// Step 4: getrandom(2), looping over partial reads. +fn fill_random(mut buf: &mut [u8]) -> Result<()> { + while !buf.is_empty() { + match getrandom(&mut *buf, GetRandomFlags::empty()) { + Ok(n) => buf = &mut buf[n..], + Err(Errno::INTR) => continue, + Err(e) => return Err(e).context("getrandom"), + } + } + Ok(()) +} + +fn run() -> Result<()> { + let dir = std::env::var("TMPDIR") + .ok() + .filter(|d| !d.is_empty()) + .unwrap_or_else(|| "/tmp".to_owned()); + + let fd = open_scratch(&dir).context("open")?; + println!("step=open ok"); + + write_all(fd.as_fd(), PAYLOAD).context("write")?; + println!("step=write ok"); + + drop(fd); // OwnedFd drop issues close(2) here, before the sleep + + sleep_ms(10).context("sleep")?; + println!("step=sleep ok"); + + let mut entropy = [0u8; 16]; + fill_random(&mut entropy).context("random")?; + println!("step=random ok"); + + println!("sysprobe: 4 steps ok"); + Ok(()) +} + +fn main() -> ExitCode { + if std::env::args_os().len() > 1 { + eprintln!("usage: app"); + return ExitCode::from(2); + } + if let Err(e) = run() { + eprintln!("sysprobe: {e:#}"); + return ExitCode::from(1); + } + ExitCode::SUCCESS +} diff --git a/examples/04-syscalls-and-the-abi/verify.lua b/examples/04-syscalls-and-the-abi/verify.lua new file mode 100644 index 0000000..0da0f28 --- /dev/null +++ b/examples/04-syscalls-and-the-abi/verify.lua @@ -0,0 +1,29 @@ +-- Verify sysprobe: run the demo for LSP_LANG and assert the labeled syscall +-- sequence — four "step= ok" lines plus the summary, in order. +-- Invoked from the example directory with LSP_LANG (cpp|go|rust) and EXAMPLE_DIR set. + +local script_dir = arg[0]:match("(.*/)") or "./" +local repo_root = os.getenv("REPO_ROOT") or (script_dir .. "../..") +package.path = package.path .. ";" .. repo_root .. "/scripts/lib/?.lua" +local checks = require("checks") + +local lang = os.getenv("LSP_LANG") +if lang ~= "cpp" and lang ~= "go" and lang ~= "rust" then + checks.skip("LSP_LANG must be cpp|go|rust (got " .. tostring(lang) .. ")") +end + +local r = checks.run("./demo.sh " .. lang .. " run") +checks.expect_exit(r, 0, lang .. ": demo exits 0") + +-- Every step line and the summary must be present... +for _, step in ipairs({ "open", "write", "sleep", "random" }) do + checks.expect_match(r.out, "step=" .. step .. " ok", lang .. ": step=" .. step .. " ok") +end +checks.expect_match(r.out, "sysprobe: 4 steps ok", lang .. ": summary line") + +-- ...and in order, as one contiguous five-line block. +checks.expect_match(r.out, + "step=open ok\nstep=write ok\nstep=sleep ok\nstep=random ok\nsysprobe: 4 steps ok", + lang .. ": steps appear in order") + +checks.finish() diff --git a/examples/05-errors-three-ways/README.md b/examples/05-errors-three-ways/README.md new file mode 100644 index 0000000..7c29939 --- /dev/null +++ b/examples/05-errors-three-ways/README.md @@ -0,0 +1,89 @@ +# 05-errors-three-ways — copyx + +A file copier, written three times — C++23, Go, and Rust — to establish the +book's error taxonomy: every failure is classified by *which side of the copy +broke*, the reason is printed in one fixed shape, and the exit code carries +the classification for scripts and callers. + +``` +copyx SRC DST +``` + +## Behavior contract (identical in all three languages) + +| Outcome | stdout / stderr | Exit | +| --- | --- | --- | +| Success | stdout: `copied bytes` | 0 | +| Bad usage (argc != 3) | stderr: `usage: copyx SRC DST` | 2 | +| Source-side failure (open/read SRC) | stderr: `copyx: ` | 2 | +| Destination-side failure (open/write/close DST) | stderr: `copyx: ` | 3 | + +`` names the path (and the syscall when it is not `open(2)`), then the +kernel's explanation in `strerror(3)` form: + +``` +copyx: nope.bin: No such file or directory # exit 2 +copyx: read src.fifo: Input/output error # exit 2 +copyx: write /dev/full: No space left on device # exit 3 +copyx: close /dev/full: No space left on device # exit 3 +``` + +Two policies are deliberately visible in each implementation: + +- **EINTR is retried.** `read(2)`/`write(2)` interrupted by a signal + transferred nothing; each language has an explicit retry loop + (`read_some`/`readSome`) rather than treating EINTR as an error. +- **Short writes are resumed.** `write_all`/`writeAll` advances past the + bytes the kernel accepted and reissues the rest until the buffer is gone. + The write side also *observes* `close(2)`, because that is where deferred + IO errors surface — the read side's close result carries nothing actionable + and is released by RAII/defer. + +The copy loop uses a 64 KiB buffer; `DST` is created `0644` and truncated. + +### How each language spells the same taxonomy + +- **C++23** (`cpp/`): an RAII `Fd` owner (move-only, closes in the + destructor, explicit `close()` returning `std::expected` for the write side); every fallible helper returns + `std::expected`; `copy_file` maps those into a + `Failure{message, exit_code}`. +- **Go** (`go/`): raw `unix.Open/Read/Write/Close` from + `golang.org/x/sys/unix`; a custom `*opError` carrying phase, path, errno, + and exit code, with `Unwrap()` so `errors.Is(err, unix.ENOENT)` still works; + EINTR detected with `errors.Is(err, unix.EINTR)`. Go's own errno strings + are lowercase, so the first letter is capitalized to match `strerror(3)`. +- **Rust 2024** (`rust/`): `rustix::fs::open` returning `OwnedFd`, + `rustix::io::{read, write}` over `AsFd`, a `thiserror` enum with one variant + per phase and an `exit_code()` mapping; the checked close hands the fd out + of the `OwnedFd` with `into_raw_fd()` into `rustix::io::try_close` (the + `try_close` crate feature) so the close result is observed exactly once. + +## The demo contract + +Each language directory has a `demo.sh` with the book-wide interface: + +- `./demo.sh build` — build only +- `./demo.sh run [args]` — run the built binary with `args` + (with env `TARGET` set, deploys to that lab VM via + `scripts/lab/deploy-to-vm.sh` instead of running locally) +- `./demo.sh` — build, then run a local walkthrough of the taxonomy: + a real 200 000-byte copy verified with `cmp`, a missing source (exit 2), + and a write to `/dev/full` (exit 3) + +The top-level `demo.sh` dispatches: `./demo.sh cpp run a b`, `./demo.sh all`, +`./demo.sh build`. + +## Verification + +`verify.lua` (run per language with `LSP_LANG` set) asserts observable +behavior, not exit-0: + +- a 200 000-byte copy (size chosen so the final read is short) exits 0, + prints `copied 200000 bytes`, and `cmp` proves the destination is + byte-identical to the source; +- an empty source copies to an existing, empty destination with + `copied 0 bytes`; +- a missing source exits 2 with `copyx: … No such file or directory`; +- writing to `/dev/full` exits 3 with `copyx: write /dev/full: …` and a + reason mentioning space. diff --git a/examples/05-errors-three-ways/cpp/.clang-format b/examples/05-errors-three-ways/cpp/.clang-format new file mode 100644 index 0000000..2c8640f --- /dev/null +++ b/examples/05-errors-three-ways/cpp/.clang-format @@ -0,0 +1,8 @@ +--- +BasedOnStyle: LLVM +Language: Cpp +Standard: c++20 +IndentWidth: 4 +ColumnLimit: 100 +PointerAlignment: Left +AllowShortFunctionsOnASingleLine: Inline diff --git a/examples/05-errors-three-ways/cpp/.clang-tidy b/examples/05-errors-three-ways/cpp/.clang-tidy new file mode 100644 index 0000000..2f275ba --- /dev/null +++ b/examples/05-errors-three-ways/cpp/.clang-tidy @@ -0,0 +1,11 @@ +--- +Checks: > + bugprone-*, + modernize-*, + performance-*, + readability-*, + -modernize-use-trailing-return-type, + -readability-identifier-length +WarningsAsErrors: '' +HeaderFilterRegex: '.*' +FormatStyle: file diff --git a/examples/05-errors-three-ways/cpp/CMakeLists.txt b/examples/05-errors-three-ways/cpp/CMakeLists.txt new file mode 100644 index 0000000..6bdb445 --- /dev/null +++ b/examples/05-errors-three-ways/cpp/CMakeLists.txt @@ -0,0 +1,8 @@ +cmake_minimum_required(VERSION 3.25) +project(template_hello CXX) + +set(CMAKE_CXX_STANDARD 23) +set(CMAKE_CXX_STANDARD_REQUIRED ON) +set(CMAKE_CXX_EXTENSIONS OFF) + +add_executable(app src/main.cpp) diff --git a/examples/05-errors-three-ways/cpp/CMakePresets.json b/examples/05-errors-three-ways/cpp/CMakePresets.json new file mode 100644 index 0000000..e31bd2f --- /dev/null +++ b/examples/05-errors-three-ways/cpp/CMakePresets.json @@ -0,0 +1,62 @@ +{ + "version": 6, + "cmakeMinimumRequired": { "major": 3, "minor": 25, "patch": 0 }, + "configurePresets": [ + { + "name": "_base", + "hidden": true, + "generator": "Ninja", + "binaryDir": "${sourceDir}/build/${presetName}", + "cacheVariables": { + "CMAKE_EXPORT_COMPILE_COMMANDS": "ON" + } + }, + { + "name": "release", + "inherits": "_base", + "displayName": "Release (-O2 with debug info)", + "cacheVariables": { "CMAKE_BUILD_TYPE": "RelWithDebInfo" } + }, + { + "name": "debug", + "inherits": "_base", + "displayName": "Debug", + "cacheVariables": { "CMAKE_BUILD_TYPE": "Debug" } + }, + { + "name": "asan", + "inherits": "_base", + "displayName": "ASan + UBSan instrumented build", + "cacheVariables": { + "CMAKE_BUILD_TYPE": "Debug", + "CMAKE_CXX_FLAGS": "-O1 -g -fno-omit-frame-pointer -fsanitize=address,undefined", + "CMAKE_EXE_LINKER_FLAGS": "-fsanitize=address,undefined" + } + }, + { + "name": "release-clang", + "inherits": "release", + "displayName": "Release (clang)", + "cacheVariables": { + "CMAKE_C_COMPILER": "clang", + "CMAKE_CXX_COMPILER": "clang++" + } + }, + { + "name": "debug-clang", + "inherits": "debug", + "displayName": "Debug (clang)", + "cacheVariables": { + "CMAKE_C_COMPILER": "clang", + "CMAKE_CXX_COMPILER": "clang++" + } + } + ], + "buildPresets": [ + { "name": "release", "configurePreset": "release" }, + { "name": "debug", "configurePreset": "debug" }, + { "name": "asan", "configurePreset": "asan" }, + { "name": "release-clang", "configurePreset": "release-clang" }, + { "name": "debug-clang", "configurePreset": "debug-clang" } + ] +} diff --git a/examples/05-errors-three-ways/cpp/demo.sh b/examples/05-errors-three-ways/cpp/demo.sh new file mode 100755 index 0000000..1013225 --- /dev/null +++ b/examples/05-errors-three-ways/cpp/demo.sh @@ -0,0 +1,64 @@ +#!/usr/bin/env bash +set -euo pipefail + +cd "$(dirname "${BASH_SOURCE[0]}")" +REPO_ROOT="$(cd ../../.. && pwd)" +BIN=build/release/app + +build() { + cmake --preset release + cmake --build --preset release +} + +run() { + if [[ ! -x "$BIN" ]]; then + build + fi + if [[ -n "${TARGET:-}" ]]; then + "$REPO_ROOT/scripts/lab/deploy-to-vm.sh" "$TARGET" "$BIN" -- "$@" + else + "./$BIN" "$@" + fi +} + +# Local walkthrough of the error taxonomy: one success, one exit-2 path, +# one exit-3 path. Runs the binary directly (not via TARGET deploy) because +# the scratch files live on this host. +demo() { + local tmp + tmp="$(mktemp -d)" + trap "rm -rf '$tmp'" EXIT + head -c 200000 /dev/urandom > "$tmp/src.bin" + + echo "== copy 200000 random bytes ==" + "./$BIN" "$tmp/src.bin" "$tmp/dst.bin" + cmp "$tmp/src.bin" "$tmp/dst.bin" && echo "cmp: src and dst identical" + + echo "== missing source (expect exit 2) ==" + local rc=0 + "./$BIN" "$tmp/missing.bin" "$tmp/x.bin" || rc=$? + echo "exit=$rc" + + echo "== write to /dev/full (expect exit 3) ==" + rc=0 + "./$BIN" "$tmp/src.bin" /dev/full || rc=$? + echo "exit=$rc" +} + +case "${1:-}" in + build) + build + ;; + run) + shift + run "$@" + ;; + "") + build + demo + ;; + *) + echo "usage: $0 [build|run [args]]" >&2 + exit 2 + ;; +esac diff --git a/examples/05-errors-three-ways/cpp/src/main.cpp b/examples/05-errors-three-ways/cpp/src/main.cpp new file mode 100644 index 0000000..abfba7f --- /dev/null +++ b/examples/05-errors-three-ways/cpp/src/main.cpp @@ -0,0 +1,172 @@ +// copyx SRC DST — file copier that establishes the book's error taxonomy. +// +// exit 0 — success; "copied bytes" on stdout +// exit 2 — source-side failure (open/read SRC); "copyx: " on stderr +// exit 3 — destination-side failure (open/write/close DST); same shape +// +// EINTR on read(2)/write(2) is retried (the syscall was interrupted before +// completing; reissuing it is the only correct policy), and short writes are +// resumed until the whole buffer is on its way. + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include + +namespace { + +[[nodiscard]] std::error_code last_error() noexcept { + return {errno, std::generic_category()}; +} + +// RAII owner of a file descriptor: closes on scope exit, move-only. +class Fd { +public: + explicit Fd(int fd) noexcept : fd_{fd} {} + Fd(const Fd&) = delete; + Fd& operator=(const Fd&) = delete; + Fd(Fd&& other) noexcept : fd_{std::exchange(other.fd_, -1)} {} + Fd& operator=(Fd&& other) noexcept { + if (this != &other) { + reset(); + fd_ = std::exchange(other.fd_, -1); + } + return *this; + } + ~Fd() { reset(); } + + [[nodiscard]] int get() const noexcept { return fd_; } + + // Explicit close for callers that must observe the result: on the write + // side, close(2) is where deferred IO errors surface. + [[nodiscard]] std::expected close() noexcept { + if (fd_ < 0) { + return {}; + } + if (::close(std::exchange(fd_, -1)) != 0) { + return std::unexpected(last_error()); + } + return {}; + } + +private: + void reset() noexcept { + if (fd_ >= 0) { + ::close(std::exchange(fd_, -1)); + } + } + int fd_ = -1; +}; + +[[nodiscard]] std::expected +open_fd(const char* path, int flags, mode_t mode = 0) { + const int fd = ::open(path, flags, mode); // NOLINT(cppcoreguidelines-pro-type-vararg) + if (fd < 0) { + return std::unexpected(last_error()); + } + return Fd{fd}; +} + +// read(2) once, retrying EINTR: interrupted means nothing was consumed. +[[nodiscard]] std::expected +read_some(const Fd& fd, std::span buf) { + for (;;) { + const ssize_t n = ::read(fd.get(), buf.data(), buf.size()); + if (n >= 0) { + return static_cast(n); + } + if (errno == EINTR) { + continue; // interrupted before transferring anything: retry + } + return std::unexpected(last_error()); + } +} + +// write(2) until the whole span is written: EINTR restarts the call, a short +// write resumes from where the kernel stopped. +[[nodiscard]] std::expected +write_all(const Fd& fd, std::span buf) { + while (!buf.empty()) { + const ssize_t n = ::write(fd.get(), buf.data(), buf.size()); + if (n > 0) { + buf = buf.subspan(static_cast(n)); + continue; + } + if (n < 0 && errno == EINTR) { + continue; // interrupted: reissue the same span + } + return std::unexpected(last_error()); + } + return {}; +} + +struct Failure { + std::string message; // printed after the "copyx: " prefix + int exit_code; +}; + +[[nodiscard]] std::expected +copy_file(const char* src_path, const char* dst_path) { + auto src = open_fd(src_path, O_RDONLY | O_CLOEXEC); + if (!src) { + return std::unexpected( + Failure{std::format("{}: {}", src_path, src.error().message()), 2}); + } + + auto dst = open_fd(dst_path, O_WRONLY | O_CREAT | O_TRUNC | O_CLOEXEC, 0644); + if (!dst) { + return std::unexpected( + Failure{std::format("{}: {}", dst_path, dst.error().message()), 3}); + } + + std::uint64_t total = 0; + std::array buf{}; + for (;;) { + const auto n = read_some(*src, buf); + if (!n) { + return std::unexpected( + Failure{std::format("read {}: {}", src_path, n.error().message()), 2}); + } + if (*n == 0) { + break; // EOF + } + if (auto w = write_all(*dst, std::span{buf}.first(*n)); !w) { + return std::unexpected( + Failure{std::format("write {}: {}", dst_path, w.error().message()), 3}); + } + total += *n; + } + + // The write side must observe close(2): deferred IO errors land here. + if (auto c = dst->close(); !c) { + return std::unexpected( + Failure{std::format("close {}: {}", dst_path, c.error().message()), 3}); + } + return total; +} + +} // namespace + +int main(int argc, char** argv) { + if (argc != 3) { + std::println(stderr, "usage: copyx SRC DST"); + return 2; + } + const auto copied = copy_file(argv[1], argv[2]); + if (!copied) { + std::println(stderr, "copyx: {}", copied.error().message); + return copied.error().exit_code; + } + std::println("copied {} bytes", *copied); + return 0; +} diff --git a/examples/05-errors-three-ways/demo.sh b/examples/05-errors-three-ways/demo.sh new file mode 100755 index 0000000..afbbe4c --- /dev/null +++ b/examples/05-errors-three-ways/demo.sh @@ -0,0 +1,23 @@ +#!/usr/bin/env bash +set -euo pipefail +cd "$(dirname "$0")" + +langs=(cpp go rust) + +case "${1:-all}" in + cpp|go|rust) + lang="$1" + shift + exec "./$lang/demo.sh" "$@" + ;; + all) + for lang in "${langs[@]}"; do "./$lang/demo.sh"; done + ;; + build) + for lang in "${langs[@]}"; do "./$lang/demo.sh" build; done + ;; + *) + echo "usage: $0 [cpp|go|rust|all|build] [args...]" >&2 + exit 2 + ;; +esac diff --git a/examples/05-errors-three-ways/go/demo.sh b/examples/05-errors-three-ways/go/demo.sh new file mode 100755 index 0000000..5514759 --- /dev/null +++ b/examples/05-errors-three-ways/go/demo.sh @@ -0,0 +1,47 @@ +#!/usr/bin/env bash +set -euo pipefail +cd "$(dirname "$0")" +REPO_ROOT="$(cd ../../.. && pwd)" +BIN=bin/app + +build() { + go build -o "$BIN" . +} + +run() { + if [[ -n "${TARGET:-}" ]]; then + exec "$REPO_ROOT/scripts/lab/deploy-to-vm.sh" "$TARGET" "$BIN" -- "$@" + fi + exec "./$BIN" "$@" +} + +# Local walkthrough of the error taxonomy: one success, one exit-2 path, +# one exit-3 path. Runs the binary directly (not via TARGET deploy) because +# the scratch files live on this host. +demo() { + local tmp + tmp="$(mktemp -d)" + trap "rm -rf '$tmp'" EXIT + head -c 200000 /dev/urandom > "$tmp/src.bin" + + echo "== copy 200000 random bytes ==" + "./$BIN" "$tmp/src.bin" "$tmp/dst.bin" + cmp "$tmp/src.bin" "$tmp/dst.bin" && echo "cmp: src and dst identical" + + echo "== missing source (expect exit 2) ==" + local rc=0 + "./$BIN" "$tmp/missing.bin" "$tmp/x.bin" || rc=$? + echo "exit=$rc" + + echo "== write to /dev/full (expect exit 3) ==" + rc=0 + "./$BIN" "$tmp/src.bin" /dev/full || rc=$? + echo "exit=$rc" +} + +case "${1:-}" in + build) build ;; + run) shift; run "$@" ;; + "") build; demo ;; + *) echo "usage: $0 [build|run [args...]]" >&2; exit 2 ;; +esac diff --git a/examples/05-errors-three-ways/go/go.mod b/examples/05-errors-three-ways/go/go.mod new file mode 100644 index 0000000..8bf98df --- /dev/null +++ b/examples/05-errors-three-ways/go/go.mod @@ -0,0 +1,7 @@ +module github.com/patterncatalyst/linux-systems-programming/examples/05-errors-three-ways/go + +go 1.26 + +toolchain go1.26.5 + +require golang.org/x/sys v0.47.0 diff --git a/examples/05-errors-three-ways/go/go.sum b/examples/05-errors-three-ways/go/go.sum new file mode 100644 index 0000000..37ee2d4 --- /dev/null +++ b/examples/05-errors-three-ways/go/go.sum @@ -0,0 +1,2 @@ +golang.org/x/sys v0.47.0 h1:o7XGOvZQCADBQQ4Y7VNq2dRWQR7JmOUW8Kxx4ZsNgWs= +golang.org/x/sys v0.47.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw= diff --git a/examples/05-errors-three-ways/go/main.go b/examples/05-errors-three-ways/go/main.go new file mode 100644 index 0000000..55c8006 --- /dev/null +++ b/examples/05-errors-three-ways/go/main.go @@ -0,0 +1,139 @@ +// copyx SRC DST — file copier that establishes the book's error taxonomy. +// +// exit 0 — success; "copied bytes" on stdout +// exit 2 — source-side failure (open/read SRC); "copyx: " on stderr +// exit 3 — destination-side failure (open/write/close DST); same shape +// +// EINTR on read(2)/write(2) is retried (the syscall was interrupted before +// completing; reissuing it is the only correct policy), and short writes are +// resumed until the whole buffer is on its way. +package main + +import ( + "errors" + "fmt" + "os" + "unicode" + + "golang.org/x/sys/unix" +) + +// opError pins a failure to a syscall phase so main can map it onto the +// taxonomy: source-side failures exit 2, destination-side failures exit 3. +type opError struct { + op string // "read", "write", "close"; "" means open(2) + path string + err unix.Errno // reachable via errors.Is / errors.As through Unwrap + exit int +} + +func (e *opError) Error() string { + if e.op == "" { + return fmt.Sprintf("%s: %s", e.path, strerror(e.err)) + } + return fmt.Sprintf("%s %s: %s", e.op, e.path, strerror(e.err)) +} + +func (e *opError) Unwrap() error { return e.err } + +// newOpError wraps whatever the unix package returned; the raw errno is +// recovered with errors.As so the chain stays inspectable. +func newOpError(op, path string, err error, exit int) *opError { + var errno unix.Errno + if !errors.As(err, &errno) { + errno = unix.EIO + } + return &opError{op: op, path: path, err: errno, exit: exit} +} + +// strerror renders an errno the way glibc's strerror(3) does — first letter +// capitalized — so all three implementations print identical reason text. +func strerror(errno unix.Errno) string { + r := []rune(errno.Error()) + if len(r) > 0 { + r[0] = unicode.ToUpper(r[0]) + } + return string(r) +} + +// readSome issues read(2) once, retrying EINTR: interrupted means nothing +// was consumed, so reissue the call. +func readSome(fd int, buf []byte) (int, error) { + for { + n, err := unix.Read(fd, buf) + if errors.Is(err, unix.EINTR) { + continue // interrupted before transferring anything: retry + } + return n, err + } +} + +// writeAll pushes the whole buffer through write(2): EINTR restarts the +// call, a short write resumes from where the kernel stopped. +func writeAll(fd int, buf []byte) error { + for len(buf) > 0 { + n, err := unix.Write(fd, buf) + if errors.Is(err, unix.EINTR) { + continue // interrupted: reissue the same span + } + if err != nil { + return err + } + buf = buf[n:] + } + return nil +} + +func copyFile(srcPath, dstPath string) (int64, error) { + src, err := unix.Open(srcPath, unix.O_RDONLY|unix.O_CLOEXEC, 0) + if err != nil { + return 0, newOpError("", srcPath, fmt.Errorf("open %s: %w", srcPath, err), 2) + } + defer unix.Close(src) // read side: nothing actionable in the close result + + dst, err := unix.Open(dstPath, unix.O_WRONLY|unix.O_CREAT|unix.O_TRUNC|unix.O_CLOEXEC, 0o644) + if err != nil { + return 0, newOpError("", dstPath, fmt.Errorf("open %s: %w", dstPath, err), 3) + } + + var total int64 + buf := make([]byte, 64*1024) + for { + n, err := readSome(src, buf) + if err != nil { + unix.Close(dst) + return 0, newOpError("read", srcPath, err, 2) + } + if n == 0 { + break // EOF + } + if err := writeAll(dst, buf[:n]); err != nil { + unix.Close(dst) + return 0, newOpError("write", dstPath, err, 3) + } + total += int64(n) + } + + // The write side must observe close(2): deferred IO errors land here. + if err := unix.Close(dst); err != nil { + return 0, newOpError("close", dstPath, err, 3) + } + return total, nil +} + +func main() { + if len(os.Args) != 3 { + fmt.Fprintln(os.Stderr, "usage: copyx SRC DST") + os.Exit(2) + } + n, err := copyFile(os.Args[1], os.Args[2]) + if err != nil { + fmt.Fprintf(os.Stderr, "copyx: %s\n", err) + var op *opError + if errors.As(err, &op) { + os.Exit(op.exit) + } + os.Exit(3) + } + fmt.Printf("copied %d bytes\n", n) +} diff --git a/examples/05-errors-three-ways/rust/Cargo.lock b/examples/05-errors-three-ways/rust/Cargo.lock new file mode 100644 index 0000000..fe689d4 --- /dev/null +++ b/examples/05-errors-three-ways/rust/Cargo.lock @@ -0,0 +1,122 @@ +# This file is automatically @generated by Cargo. +# It is not intended for manual editing. +version = 4 + +[[package]] +name = "bitflags" +version = "2.13.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b588b76d00fde79687d7646a9b5bdf3cc0f655e0bbd080335a95d7e96f3587da" + +[[package]] +name = "copyx" +version = "0.1.0" +dependencies = [ + "rustix", + "thiserror", +] + +[[package]] +name = "errno" +version = "0.3.14" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "39cab71617ae0d63f51a36d69f866391735b51691dbda63cf6f96d042b63efeb" +dependencies = [ + "libc", + "windows-sys", +] + +[[package]] +name = "libc" +version = "0.2.186" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "68ab91017fe16c622486840e4c83c9a37afeff978bd239b5293d61ece587de66" + +[[package]] +name = "linux-raw-sys" +version = "0.12.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "32a66949e030da00e8c7d4434b251670a91556f4144941d37452769c25d58a53" + +[[package]] +name = "proc-macro2" +version = "1.0.107" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "985e7ec9bb745e6ce6535b544d84d6cd6f7ad8bd711c398938ae983b91a766d9" +dependencies = [ + "unicode-ident", +] + +[[package]] +name = "quote" +version = "1.0.47" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1fbf4db142a473a8d80c26bbf18454ed458bf8d26c8219c331daecfdbd079001" +dependencies = [ + "proc-macro2", +] + +[[package]] +name = "rustix" +version = "1.1.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b6fe4565b9518b83ef4f91bb47ce29620ca828bd32cb7e408f0062e9930ba190" +dependencies = [ + "bitflags", + "errno", + "libc", + "linux-raw-sys", + "windows-sys", +] + +[[package]] +name = "syn" +version = "3.0.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f2fac314a64dc9a36e61a9eb4261a5e9bbfbc922b27e518af97bc32b926cf967" +dependencies = [ + "proc-macro2", + "quote", + "unicode-ident", +] + +[[package]] +name = "thiserror" +version = "2.0.19" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "09a43598840e33d5b0331f38c5e30d13bb11c11210a4b58f0d9b18a5a5eefcd9" +dependencies = [ + "thiserror-impl", +] + +[[package]] +name = "thiserror-impl" +version = "2.0.19" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "43cbfe0cf76104d42a574802844187e84a305e531ed54455f11fbde0f10541cd" +dependencies = [ + "proc-macro2", + "quote", + "syn", +] + +[[package]] +name = "unicode-ident" +version = "1.0.24" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e6e4313cd5fcd3dad5cafa179702e2b244f760991f45397d14d4ebf38247da75" + +[[package]] +name = "windows-link" +version = "0.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5" + +[[package]] +name = "windows-sys" +version = "0.61.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ae137229bcbd6cdf0f7b80a31df61766145077ddf49416a728b02cb3921ff3fc" +dependencies = [ + "windows-link", +] diff --git a/examples/05-errors-three-ways/rust/Cargo.toml b/examples/05-errors-three-ways/rust/Cargo.toml new file mode 100644 index 0000000..6625db2 --- /dev/null +++ b/examples/05-errors-three-ways/rust/Cargo.toml @@ -0,0 +1,12 @@ +[package] +name = "copyx" +version = "0.1.0" +edition = "2024" + +[[bin]] +name = "app" +path = "src/main.rs" + +[dependencies] +rustix = { version = "1", features = ["fs", "try_close"] } +thiserror = "2" diff --git a/examples/05-errors-three-ways/rust/demo.sh b/examples/05-errors-three-ways/rust/demo.sh new file mode 100755 index 0000000..901fc40 --- /dev/null +++ b/examples/05-errors-three-ways/rust/demo.sh @@ -0,0 +1,50 @@ +#!/usr/bin/env bash +set -euo pipefail +cd "$(dirname "${BASH_SOURCE[0]}")" + +BIN=target/release/app + +build() { + cargo build --release +} + +run() { + if [[ -n "${TARGET:-}" ]]; then + local repo_root + repo_root=$(cd ../../.. && pwd) + "$repo_root/scripts/lab/deploy-to-vm.sh" "$TARGET" "$BIN" -- "$@" + else + "./$BIN" "$@" + fi +} + +# Local walkthrough of the error taxonomy: one success, one exit-2 path, +# one exit-3 path. Runs the binary directly (not via TARGET deploy) because +# the scratch files live on this host. +demo() { + local tmp + tmp="$(mktemp -d)" + trap "rm -rf '$tmp'" EXIT + head -c 200000 /dev/urandom > "$tmp/src.bin" + + echo "== copy 200000 random bytes ==" + "./$BIN" "$tmp/src.bin" "$tmp/dst.bin" + cmp "$tmp/src.bin" "$tmp/dst.bin" && echo "cmp: src and dst identical" + + echo "== missing source (expect exit 2) ==" + local rc=0 + "./$BIN" "$tmp/missing.bin" "$tmp/x.bin" || rc=$? + echo "exit=$rc" + + echo "== write to /dev/full (expect exit 3) ==" + rc=0 + "./$BIN" "$tmp/src.bin" /dev/full || rc=$? + echo "exit=$rc" +} + +case "${1:-}" in + build) build ;; + run) shift; run "$@" ;; + "") build; demo ;; + *) echo "usage: $0 [build|run [args...]]" >&2; exit 2 ;; +esac diff --git a/examples/05-errors-three-ways/rust/rust-toolchain.toml b/examples/05-errors-three-ways/rust/rust-toolchain.toml new file mode 100644 index 0000000..4099f1a --- /dev/null +++ b/examples/05-errors-three-ways/rust/rust-toolchain.toml @@ -0,0 +1,3 @@ +[toolchain] +channel = "1.97.1" +components = ["rustfmt", "clippy", "rust-src"] diff --git a/examples/05-errors-three-ways/rust/src/main.rs b/examples/05-errors-three-ways/rust/src/main.rs new file mode 100644 index 0000000..b1f9658 --- /dev/null +++ b/examples/05-errors-three-ways/rust/src/main.rs @@ -0,0 +1,136 @@ +//! copyx SRC DST — file copier that establishes the book's error taxonomy. +//! +//! exit 0 — success; "copied bytes" on stdout +//! exit 2 — source-side failure (open/read SRC); "copyx: " on stderr +//! exit 3 — destination-side failure (open/write/close DST); same shape +//! +//! EINTR on read(2)/write(2) is retried (the syscall was interrupted before +//! completing; reissuing it is the only correct policy), and short writes are +//! resumed until the whole buffer is on its way. + +use std::os::fd::{AsFd, IntoRawFd, OwnedFd}; +use std::process::ExitCode; + +use rustix::fs::{Mode, OFlags, open}; +use rustix::io::{Errno, read, try_close, write}; +use thiserror::Error; + +/// Each variant pins a failure to a syscall phase so `main` can map it onto +/// the taxonomy: source-side failures exit 2, destination-side failures 3. +#[derive(Debug, Error)] +enum CopyError { + #[error("{path}: {reason}")] + SrcOpen { path: String, reason: String }, + #[error("read {path}: {reason}")] + Read { path: String, reason: String }, + #[error("{path}: {reason}")] + DstOpen { path: String, reason: String }, + #[error("write {path}: {reason}")] + Write { path: String, reason: String }, + #[error("close {path}: {reason}")] + Close { path: String, reason: String }, +} + +impl CopyError { + fn exit_code(&self) -> u8 { + match self { + CopyError::SrcOpen { .. } | CopyError::Read { .. } => 2, + CopyError::DstOpen { .. } | CopyError::Write { .. } | CopyError::Close { .. } => 3, + } + } +} + +/// Render an errno the way glibc's strerror(3) does, so all three +/// implementations print identical reason text. `Errno`'s own Display goes +/// through `std::io::Error`, which appends " (os error N)" — strip it. +fn strerror(errno: Errno) -> String { + let s = std::io::Error::from(errno).to_string(); + match s.rfind(" (os error ") { + Some(idx) => s[..idx].to_owned(), + None => s, + } +} + +/// read(2) once, retrying EINTR: interrupted means nothing was consumed. +fn read_some(fd: impl AsFd, buf: &mut [u8]) -> Result { + loop { + match read(&fd, &mut *buf) { + Err(Errno::INTR) => continue, // interrupted before transferring anything: retry + other => return other, + } + } +} + +/// write(2) until the whole slice is written: EINTR restarts the call, a +/// short write resumes from where the kernel stopped. +fn write_all(fd: impl AsFd, mut buf: &[u8]) -> Result<(), Errno> { + while !buf.is_empty() { + match write(&fd, buf) { + Ok(n) => buf = &buf[n..], + Err(Errno::INTR) => continue, // interrupted: reissue the same span + Err(e) => return Err(e), + } + } + Ok(()) +} + +/// close(2) with the result observed: on the write side this is where +/// deferred IO errors surface. `into_raw_fd` transfers ownership out of the +/// `OwnedFd`, so the fd is closed exactly once. +fn close_checked(fd: OwnedFd) -> Result<(), Errno> { + unsafe { try_close(fd.into_raw_fd()) } +} + +fn copy_file(src_path: &str, dst_path: &str) -> Result { + let src = open(src_path, OFlags::RDONLY | OFlags::CLOEXEC, Mode::empty()).map_err(|e| { + CopyError::SrcOpen { path: src_path.to_owned(), reason: strerror(e) } + })?; // OwnedFd: dropped (and closed) on every path below + + let dst = open( + dst_path, + OFlags::WRONLY | OFlags::CREATE | OFlags::TRUNC | OFlags::CLOEXEC, + Mode::from_bits_truncate(0o644), + ) + .map_err(|e| CopyError::DstOpen { path: dst_path.to_owned(), reason: strerror(e) })?; + + let mut total: u64 = 0; + let mut buf = vec![0u8; 64 * 1024]; + loop { + let n = read_some(&src, &mut buf).map_err(|e| CopyError::Read { + path: src_path.to_owned(), + reason: strerror(e), + })?; + if n == 0 { + break; // EOF + } + write_all(&dst, &buf[..n]).map_err(|e| CopyError::Write { + path: dst_path.to_owned(), + reason: strerror(e), + })?; + total += n as u64; + } + + close_checked(dst).map_err(|e| CopyError::Close { + path: dst_path.to_owned(), + reason: strerror(e), + })?; + Ok(total) +} + +fn main() -> ExitCode { + let args: Vec = std::env::args().collect(); + if args.len() != 3 { + eprintln!("usage: copyx SRC DST"); + return ExitCode::from(2); + } + match copy_file(&args[1], &args[2]) { + Ok(n) => { + println!("copied {n} bytes"); + ExitCode::SUCCESS + } + Err(e) => { + eprintln!("copyx: {e}"); + ExitCode::from(e.exit_code()) + } + } +} diff --git a/examples/05-errors-three-ways/verify.lua b/examples/05-errors-three-ways/verify.lua new file mode 100644 index 0000000..d34d333 --- /dev/null +++ b/examples/05-errors-three-ways/verify.lua @@ -0,0 +1,59 @@ +-- Verify copyx (05-errors-three-ways): behavioral checks for the error taxonomy. +-- Invoked from the example directory with LSP_LANG (cpp|go|rust) and EXAMPLE_DIR set. +-- +-- What a passing run proves: a real copy produced a byte-identical file and +-- reported the exact byte count; a missing source exited 2 with a reason on +-- stderr; ENOSPC via /dev/full exited 3 with a reason naming the space problem. + +local script_dir = arg[0]:match("(.*/)") or "./" +local repo_root = os.getenv("REPO_ROOT") or (script_dir .. "../..") +package.path = package.path .. ";" .. repo_root .. "/scripts/lib/?.lua" +local checks = require("checks") + +local lang = os.getenv("LSP_LANG") +if lang ~= "cpp" and lang ~= "go" and lang ~= "rust" then + checks.skip("LSP_LANG must be cpp|go|rust (got " .. tostring(lang) .. ")") +end + +local demo = "./demo.sh " .. lang .. " run" + +local m = checks.run("mktemp -d") +local tmp = m.out:gsub("%s+$", "") +if m.exit ~= 0 or tmp == "" then + checks.skip("mktemp -d failed") +end +local function p(name) return "'" .. tmp .. "/" .. name .. "'" end + +-- Seed a source whose size is not a multiple of the 64 KiB copy buffer, +-- so the loop exercises a short final read. +checks.run("head -c 200000 /dev/urandom > " .. p("src.bin")) + +-- 1. Real copy: exact byte count on stdout, exit 0, dst byte-identical to src. +local r = checks.run(demo .. " " .. p("src.bin") .. " " .. p("dst.bin")) +checks.expect_exit(r, 0, lang .. ": successful copy exits 0") +checks.expect_match(r.out, "copied 200000 bytes", lang .. ": reports exact byte count") +r = checks.run("cmp " .. p("src.bin") .. " " .. p("dst.bin")) +checks.expect_exit(r, 0, lang .. ": cmp: dst is byte-identical to src") + +-- 2. Empty source: still success, zero bytes, empty dst created. +checks.run(": > " .. p("empty.bin")) +r = checks.run(demo .. " " .. p("empty.bin") .. " " .. p("empty-out.bin")) +checks.expect_exit(r, 0, lang .. ": empty-file copy exits 0") +checks.expect_match(r.out, "copied 0 bytes", lang .. ": empty file reports 0 bytes") +r = checks.run("test -f " .. p("empty-out.bin") .. " && test ! -s " .. p("empty-out.bin")) +checks.expect_exit(r, 0, lang .. ": empty dst exists and is empty") + +-- 3. Missing source: taxonomy exit 2, "copyx: " on stderr. +r = checks.run(demo .. " " .. p("no-such-file") .. " " .. p("x.bin")) +checks.expect_exit(r, 2, lang .. ": missing source exits 2") +checks.expect_match(r.out, "copyx: .*No such file or directory", + lang .. ": missing source prints copyx: reason") + +-- 4. Write failure (/dev/full -> ENOSPC): taxonomy exit 3, reason mentions space. +r = checks.run(demo .. " " .. p("src.bin") .. " /dev/full") +checks.expect_exit(r, 3, lang .. ": ENOSPC write exits 3") +checks.expect_match(r.out, "copyx: write /dev/full", lang .. ": names the failing op and path") +checks.expect_match(r.out, "space", lang .. ": reason mentions space") + +checks.run("rm -rf '" .. tmp .. "'") +checks.finish() diff --git a/examples/06-concurrency-three-ways/README.md b/examples/06-concurrency-three-ways/README.md new file mode 100644 index 0000000..d9e9b9e --- /dev/null +++ b/examples/06-concurrency-three-ways/README.md @@ -0,0 +1,90 @@ +# 06-concurrency-three-ways + +Chapter 6's example: `parhash`, a parallel checksummer implemented three times +— C++23, Go, and Rust — with each language's native concurrency toolkit, but +one shared observable contract so a single `verify.lua` covers all three. + +## The program + +``` +parhash DIR +``` + +Walks every regular file under `DIR` (symlinks are never followed), hashes +each with FNV-1a 64 (implemented inline in all three sources — no hash +libraries), using a bounded pool of **4 workers**, then prints one line per +file, **sorted by relative path**, followed by a summary: + +``` +bbd23ea491ed9813 a.txt +4d038ef62a7d9c0b b.txt +f839d20567c0a911 sub/c.txt +51d88627df287325 sub/deep/d.bin +cbf29ce484222325 zz-empty.dat +parhash: 5 files, 4 workers +``` + +Given the same tree, the three binaries produce byte-identical stdout. + +**Errors:** missing/extra argument → `usage: parhash DIR` on stderr, exit 2; +`DIR` not a directory → `parhash: cannot walk DIR` on stderr, exit 1; +an unreadable file mid-walk → `parhash: skipping ` on stderr, and the +walk continues. + +**SIGINT:** the program stops accepting work (the walker quits enumerating, +workers refuse queued-but-unstarted paths), drains in-flight hashes (the file +each worker is on always finishes), prints the sorted lines for everything +that completed, prints `parhash: interrupted` on stderr, and exits **130**. + +## Three concurrency shapes, one contract + +| | work distribution | cancellation | +|---|---|---| +| **C++23** | 4 `std::jthread` workers popping a mutex-guarded `std::deque` via a `condition_variable_any` whose wait is `stop_token`-aware | SIGINT is blocked in every thread; a watcher `jthread` `sigtimedwait()`s for it, flips an `std::atomic`, and `request_stop()`s a shared `std::stop_source` | +| **Go** | 4 worker goroutines receiving from an unbuffered `chan string` fed by a walker goroutine; results return on a second channel | `signal.NotifyContext` cancels a `context.Context`; workers `select` `ctx.Done()` ahead of the next path | +| **Rust** | 4 scoped threads (`std::thread::scope`) sharing one `mpsc::Receiver` behind a `Mutex`; results return on a second `mpsc` channel | `signal-hook` sets a shared `AtomicBool` that walker and workers poll between files | + +The Go build runs with `-race` (see `go/demo.sh`) — this is the concurrency +chapter, so the race detector referees every verification run. + +## Layout + +``` +06-concurrency-three-ways/ +├── demo.sh # dispatcher: ./demo.sh [cpp|go|rust|all|build] [args...] +├── verify.lua # automated check driven by CI +├── cpp/ # CMake preset build (GCC or clang), demo.sh +├── go/ # go build -race (toolchain go1.26.5), demo.sh +└── rust/ # cargo build (rustup pin 1.97.1; anyhow + signal-hook), demo.sh +``` + +## The demo contract + +Every language directory has a `demo.sh` with the book-wide interface: + +- `./demo.sh` — build then run +- `./demo.sh build` — build only +- `./demo.sh run [args]` — run the built binary (e.g. `./demo.sh run /etc`) +- With env `TARGET` set, `run` deploys the binary to that lab VM via + `scripts/lab/deploy-to-vm.sh "$TARGET" -- [args]` instead of + running locally. + +All three `run` paths `exec` the binary, so a signal sent to the `demo.sh` +PID lands on `parhash` itself — which is exactly how `verify.lua` delivers +its `SIGINT`. + +## Verification + +`verify.lua` runs from this directory with `LSP_LANG` set to one language at +a time and asserts, per language: + +1. a 5-file fixture (nested dirs, a binary file, an empty file) produces + stdout **byte-identical** (`diff -u`) to the precomputed expected output — + the same expected bytes for all three languages; +2. a missing argument prints the usage line and exits 2; +3. on a big sparse tree (~6 GiB logical, ~0 on disk, seconds of hashing), + `kill -INT` after 100 ms yields only well-formed partial hash lines on + stdout, no summary line, `parhash: interrupted` on stderr, and exit 130. + +Verified on the Fedora 44 reference host (kernel 7.1.3-200.fc44): all three +languages PASS all 8 checks, Go under the race detector. diff --git a/examples/06-concurrency-three-ways/cpp/.clang-format b/examples/06-concurrency-three-ways/cpp/.clang-format new file mode 100644 index 0000000..2c8640f --- /dev/null +++ b/examples/06-concurrency-three-ways/cpp/.clang-format @@ -0,0 +1,8 @@ +--- +BasedOnStyle: LLVM +Language: Cpp +Standard: c++20 +IndentWidth: 4 +ColumnLimit: 100 +PointerAlignment: Left +AllowShortFunctionsOnASingleLine: Inline diff --git a/examples/06-concurrency-three-ways/cpp/.clang-tidy b/examples/06-concurrency-three-ways/cpp/.clang-tidy new file mode 100644 index 0000000..2f275ba --- /dev/null +++ b/examples/06-concurrency-three-ways/cpp/.clang-tidy @@ -0,0 +1,11 @@ +--- +Checks: > + bugprone-*, + modernize-*, + performance-*, + readability-*, + -modernize-use-trailing-return-type, + -readability-identifier-length +WarningsAsErrors: '' +HeaderFilterRegex: '.*' +FormatStyle: file diff --git a/examples/06-concurrency-three-ways/cpp/CMakeLists.txt b/examples/06-concurrency-three-ways/cpp/CMakeLists.txt new file mode 100644 index 0000000..1fb111a --- /dev/null +++ b/examples/06-concurrency-three-ways/cpp/CMakeLists.txt @@ -0,0 +1,8 @@ +cmake_minimum_required(VERSION 3.25) +project(parhash CXX) + +set(CMAKE_CXX_STANDARD 23) +set(CMAKE_CXX_STANDARD_REQUIRED ON) +set(CMAKE_CXX_EXTENSIONS OFF) + +add_executable(app src/main.cpp) diff --git a/examples/06-concurrency-three-ways/cpp/CMakePresets.json b/examples/06-concurrency-three-ways/cpp/CMakePresets.json new file mode 100644 index 0000000..e31bd2f --- /dev/null +++ b/examples/06-concurrency-three-ways/cpp/CMakePresets.json @@ -0,0 +1,62 @@ +{ + "version": 6, + "cmakeMinimumRequired": { "major": 3, "minor": 25, "patch": 0 }, + "configurePresets": [ + { + "name": "_base", + "hidden": true, + "generator": "Ninja", + "binaryDir": "${sourceDir}/build/${presetName}", + "cacheVariables": { + "CMAKE_EXPORT_COMPILE_COMMANDS": "ON" + } + }, + { + "name": "release", + "inherits": "_base", + "displayName": "Release (-O2 with debug info)", + "cacheVariables": { "CMAKE_BUILD_TYPE": "RelWithDebInfo" } + }, + { + "name": "debug", + "inherits": "_base", + "displayName": "Debug", + "cacheVariables": { "CMAKE_BUILD_TYPE": "Debug" } + }, + { + "name": "asan", + "inherits": "_base", + "displayName": "ASan + UBSan instrumented build", + "cacheVariables": { + "CMAKE_BUILD_TYPE": "Debug", + "CMAKE_CXX_FLAGS": "-O1 -g -fno-omit-frame-pointer -fsanitize=address,undefined", + "CMAKE_EXE_LINKER_FLAGS": "-fsanitize=address,undefined" + } + }, + { + "name": "release-clang", + "inherits": "release", + "displayName": "Release (clang)", + "cacheVariables": { + "CMAKE_C_COMPILER": "clang", + "CMAKE_CXX_COMPILER": "clang++" + } + }, + { + "name": "debug-clang", + "inherits": "debug", + "displayName": "Debug (clang)", + "cacheVariables": { + "CMAKE_C_COMPILER": "clang", + "CMAKE_CXX_COMPILER": "clang++" + } + } + ], + "buildPresets": [ + { "name": "release", "configurePreset": "release" }, + { "name": "debug", "configurePreset": "debug" }, + { "name": "asan", "configurePreset": "asan" }, + { "name": "release-clang", "configurePreset": "release-clang" }, + { "name": "debug-clang", "configurePreset": "debug-clang" } + ] +} diff --git a/examples/06-concurrency-three-ways/cpp/demo.sh b/examples/06-concurrency-three-ways/cpp/demo.sh new file mode 100755 index 0000000..6d53e45 --- /dev/null +++ b/examples/06-concurrency-three-ways/cpp/demo.sh @@ -0,0 +1,40 @@ +#!/usr/bin/env bash +set -euo pipefail + +cd "$(dirname "${BASH_SOURCE[0]}")" +REPO_ROOT="$(cd ../../.. && pwd)" +BIN=build/release/app + +build() { + cmake --preset release + cmake --build --preset release +} + +run() { + if [[ ! -x "$BIN" ]]; then + build + fi + if [[ -n "${TARGET:-}" ]]; then + "$REPO_ROOT/scripts/lab/deploy-to-vm.sh" "$TARGET" "$BIN" -- "$@" + else + exec "./$BIN" "$@" # exec so signals sent to this PID reach the app + fi +} + +case "${1:-}" in + build) + build + ;; + run) + shift + run "$@" + ;; + "") + build + run + ;; + *) + echo "usage: $0 [build|run [args]]" >&2 + exit 2 + ;; +esac diff --git a/examples/06-concurrency-three-ways/cpp/src/main.cpp b/examples/06-concurrency-three-ways/cpp/src/main.cpp new file mode 100644 index 0000000..6d6e02e --- /dev/null +++ b/examples/06-concurrency-three-ways/cpp/src/main.cpp @@ -0,0 +1,230 @@ +// parhash: parallel FNV-1a 64 checksummer — C++23 flavor. +// +// Concurrency shape: 4 std::jthread workers pull paths from a mutex-guarded +// queue (condition_variable_any so waits are stop_token-aware). A dedicated +// jthread sigtimedwait()s for SIGINT (blocked in every other thread) and, on +// delivery, flips an atomic flag and request_stop()s the pool: the producer +// stops enqueuing, workers finish the file they are on and exit, and main +// prints whatever completed plus "parhash: interrupted" (exit 130). + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include + +namespace fs = std::filesystem; + +namespace { + +constexpr int kWorkers = 4; +constexpr std::uint64_t kFnvOffset = 0xcbf29ce484222325ULL; +constexpr std::uint64_t kFnvPrime = 0x00000100000001b3ULL; + +// RAII wrapper: an owned file descriptor, closed exactly once. +class OwnedFd { +public: + explicit OwnedFd(int fd) noexcept : fd_(fd) {} + ~OwnedFd() { + if (fd_ >= 0) { + ::close(fd_); + } + } + OwnedFd(const OwnedFd&) = delete; + OwnedFd& operator=(const OwnedFd&) = delete; + OwnedFd(OwnedFd&& other) noexcept : fd_(std::exchange(other.fd_, -1)) {} + OwnedFd& operator=(OwnedFd&&) = delete; + + [[nodiscard]] bool valid() const noexcept { return fd_ >= 0; } + [[nodiscard]] int get() const noexcept { return fd_; } + +private: + int fd_; +}; + +[[nodiscard]] std::error_code last_error() { + return {errno, std::system_category()}; +} + +// Stream a file through FNV-1a 64. Never throws; errors come back typed. +[[nodiscard]] std::expected hash_file(const fs::path& path) { + OwnedFd fd(::open(path.c_str(), O_RDONLY | O_CLOEXEC)); + if (!fd.valid()) { + return std::unexpected(last_error()); + } + std::uint64_t hash = kFnvOffset; + std::array buf{}; + for (;;) { + const ssize_t n = ::read(fd.get(), buf.data(), buf.size()); + if (n < 0) { + if (errno == EINTR) { + continue; + } + return std::unexpected(last_error()); + } + if (n == 0) { + break; + } + for (ssize_t i = 0; i < n; ++i) { + hash ^= buf[static_cast(i)]; + hash *= kFnvPrime; + } + } + return hash; +} + +// Work item: path relative to the root (display + sort key) plus the real path. +struct Task { + std::string rel; + fs::path full; +}; + +// Mutex-guarded FIFO. pop() blocks with a stop_token-aware wait, so a +// request_stop() wakes idle workers immediately and makes them refuse +// queued-but-unstarted items; a file a worker is mid-hash on always finishes. +class TaskQueue { +public: + void push(Task task) { + { + std::lock_guard lock(mutex_); + queue_.push_back(std::move(task)); + } + cv_.notify_one(); + } + + void close() { + { + std::lock_guard lock(mutex_); + closed_ = true; + } + cv_.notify_all(); + } + + [[nodiscard]] std::optional pop(std::stop_token st) { + std::unique_lock lock(mutex_); + cv_.wait(lock, st, [this] { return closed_ || !queue_.empty(); }); + if (st.stop_requested() || queue_.empty()) { + return std::nullopt; + } + Task task = std::move(queue_.front()); + queue_.pop_front(); + return task; + } + +private: + std::mutex mutex_; + std::condition_variable_any cv_; + std::deque queue_; + bool closed_ = false; +}; + +struct Entry { + std::string rel; + std::uint64_t hash; +}; + +} // namespace + +int main(int argc, char** argv) { + if (argc != 2) { + std::println(stderr, "usage: parhash DIR"); + return 2; + } + const fs::path root = argv[1]; + std::error_code ec; + if (!fs::is_directory(root, ec)) { + std::println(stderr, "parhash: cannot walk {}", root.string()); + return 1; + } + + // Block SIGINT process-wide before any thread exists; only the watcher + // thread consumes it, via sigtimedwait, so no async handler ever runs. + sigset_t sigint_set{}; + sigemptyset(&sigint_set); + sigaddset(&sigint_set, SIGINT); + pthread_sigmask(SIG_BLOCK, &sigint_set, nullptr); + + std::atomic interrupted{false}; + std::stop_source cancel; + TaskQueue queue; + + std::mutex results_mutex; + std::vector results; + + std::jthread watcher([&](std::stop_token st) { + const timespec tick{.tv_sec = 0, .tv_nsec = 20'000'000}; // 20 ms poll + while (!st.stop_requested()) { + if (::sigtimedwait(&sigint_set, nullptr, &tick) == SIGINT) { + interrupted.store(true, std::memory_order_relaxed); + cancel.request_stop(); // stop accepting work; drain in-flight + return; + } + } + }); + + { + std::vector workers; + workers.reserve(kWorkers); + for (int i = 0; i < kWorkers; ++i) { + workers.emplace_back([&, st = cancel.get_token()] { + while (auto task = queue.pop(st)) { + const auto hash = hash_file(task->full); + if (!hash) { + std::println(stderr, "parhash: skipping {}", task->rel); + continue; + } + std::lock_guard lock(results_mutex); + results.push_back({std::move(task->rel), *hash}); + } + }); + } + + // Producer: stream the walk into the queue, checking for cancellation + // between entries so a big tree stops enumerating promptly. + for (auto it = fs::recursive_directory_iterator(root, ec); + !ec && it != fs::recursive_directory_iterator(); it.increment(ec)) { + if (interrupted.load(std::memory_order_relaxed)) { + break; + } + std::error_code type_ec; + // symlink_status: classify the entry itself, never what it links to + // (matches Go's DirEntry.Type and Rust's DirEntry::file_type). + if (!fs::is_regular_file(it->symlink_status(type_ec)) || type_ec) { + continue; + } + queue.push({it->path().lexically_relative(root).generic_string(), it->path()}); + } + queue.close(); + } // jthread dtors join all four workers here + + watcher.request_stop(); + watcher.join(); + + std::ranges::sort(results, {}, &Entry::rel); + for (const auto& entry : results) { + std::println("{:016x} {}", entry.hash, entry.rel); + } + if (interrupted.load(std::memory_order_relaxed)) { + std::println(stderr, "parhash: interrupted"); + return 130; + } + std::println("parhash: {} files, {} workers", results.size(), kWorkers); + return 0; +} diff --git a/examples/06-concurrency-three-ways/demo.sh b/examples/06-concurrency-three-ways/demo.sh new file mode 100755 index 0000000..afbbe4c --- /dev/null +++ b/examples/06-concurrency-three-ways/demo.sh @@ -0,0 +1,23 @@ +#!/usr/bin/env bash +set -euo pipefail +cd "$(dirname "$0")" + +langs=(cpp go rust) + +case "${1:-all}" in + cpp|go|rust) + lang="$1" + shift + exec "./$lang/demo.sh" "$@" + ;; + all) + for lang in "${langs[@]}"; do "./$lang/demo.sh"; done + ;; + build) + for lang in "${langs[@]}"; do "./$lang/demo.sh" build; done + ;; + *) + echo "usage: $0 [cpp|go|rust|all|build] [args...]" >&2 + exit 2 + ;; +esac diff --git a/examples/06-concurrency-three-ways/go/demo.sh b/examples/06-concurrency-three-ways/go/demo.sh new file mode 100755 index 0000000..113996e --- /dev/null +++ b/examples/06-concurrency-three-ways/go/demo.sh @@ -0,0 +1,23 @@ +#!/usr/bin/env bash +set -euo pipefail +cd "$(dirname "$0")" +REPO_ROOT="$(cd ../../.. && pwd)" +BIN=bin/app + +build() { + go build -race -o "$BIN" . # race detector on: this is the concurrency chapter +} + +run() { + if [[ -n "${TARGET:-}" ]]; then + exec "$REPO_ROOT/scripts/lab/deploy-to-vm.sh" "$TARGET" "$BIN" -- "$@" + fi + exec "./$BIN" "$@" +} + +case "${1:-}" in + build) build ;; + run) shift; run "$@" ;; + "") build; run ;; + *) echo "usage: $0 [build|run [args...]]" >&2; exit 2 ;; +esac diff --git a/examples/06-concurrency-three-ways/go/go.mod b/examples/06-concurrency-three-ways/go/go.mod new file mode 100644 index 0000000..40fa2f7 --- /dev/null +++ b/examples/06-concurrency-three-ways/go/go.mod @@ -0,0 +1,5 @@ +module github.com/patterncatalyst/linux-systems-programming/examples/06-concurrency-three-ways/go + +go 1.26 + +toolchain go1.26.5 diff --git a/examples/06-concurrency-three-ways/go/main.go b/examples/06-concurrency-three-ways/go/main.go new file mode 100644 index 0000000..a978486 --- /dev/null +++ b/examples/06-concurrency-three-ways/go/main.go @@ -0,0 +1,156 @@ +// parhash: parallel FNV-1a 64 checksummer — Go flavor. +// +// Concurrency shape: 4 worker goroutines receive relative paths from an +// unbuffered channel fed by a walker goroutine; results flow back on a second +// channel that main drains. signal.NotifyContext turns SIGINT into context +// cancellation: the walker stops sending, workers finish the file they are on +// (select prefers Done over the next path), and main prints whatever +// completed plus "parhash: interrupted" (exit 130). +package main + +import ( + "context" + "errors" + "fmt" + "io" + "io/fs" + "os" + "os/signal" + "path/filepath" + "slices" + "strings" + "sync" +) + +const workers = 4 + +const ( + fnvOffset = 0xcbf29ce484222325 + fnvPrime = 0x00000100000001b3 +) + +type result struct { + rel string + sum uint64 +} + +// hashFile streams path through FNV-1a 64 in 64 KiB chunks. +func hashFile(path string) (uint64, error) { + f, err := os.Open(path) + if err != nil { + return 0, fmt.Errorf("open %s: %w", path, err) + } + defer f.Close() + + hash := uint64(fnvOffset) + buf := make([]byte, 64*1024) + for { + n, err := f.Read(buf) + for _, b := range buf[:n] { + hash ^= uint64(b) + hash *= fnvPrime + } + if errors.Is(err, io.EOF) { + return hash, nil + } + if err != nil { + return 0, fmt.Errorf("read %s: %w", path, err) + } + } +} + +// walk streams the relative path of every regular file under root into paths, +// stopping early once ctx is cancelled. +func walk(ctx context.Context, root string, paths chan<- string) { + defer close(paths) + _ = filepath.WalkDir(root, func(p string, d fs.DirEntry, err error) error { + if ctx.Err() != nil { + return filepath.SkipAll // interrupted: stop accepting work + } + if err != nil || !d.Type().IsRegular() { + return nil // unreadable subtree or non-file: skip, keep walking + } + rel, err := filepath.Rel(root, p) + if err != nil { + return nil + } + select { + case paths <- rel: + case <-ctx.Done(): + return filepath.SkipAll + } + return nil + }) +} + +func worker(ctx context.Context, root string, paths <-chan string, results chan<- result) { + for { + // Done first: once cancelled, refuse queued-but-unstarted work. + select { + case <-ctx.Done(): + return + default: + } + select { + case <-ctx.Done(): + return + case rel, ok := <-paths: + if !ok { + return + } + sum, err := hashFile(filepath.Join(root, rel)) + if err != nil { + fmt.Fprintf(os.Stderr, "parhash: skipping %s\n", rel) + continue + } + results <- result{rel: rel, sum: sum} + } + } +} + +func main() { + if len(os.Args) != 2 { + fmt.Fprintln(os.Stderr, "usage: parhash DIR") + os.Exit(2) + } + root := os.Args[1] + if info, err := os.Stat(root); err != nil || !info.IsDir() { + fmt.Fprintf(os.Stderr, "parhash: cannot walk %s\n", root) + os.Exit(1) + } + + ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt) + defer stop() + + paths := make(chan string) + results := make(chan result) + + var wg sync.WaitGroup + for range workers { + wg.Add(1) + go func() { + defer wg.Done() + worker(ctx, root, paths, results) + }() + } + go walk(ctx, root, paths) + go func() { + wg.Wait() // all in-flight hashes drained + close(results) + }() + + var out []result + for r := range results { + out = append(out, r) + } + + slices.SortFunc(out, func(a, b result) int { return strings.Compare(a.rel, b.rel) }) + for _, r := range out { + fmt.Printf("%016x %s\n", r.sum, r.rel) + } + if ctx.Err() != nil { + fmt.Fprintln(os.Stderr, "parhash: interrupted") + os.Exit(130) + } + fmt.Printf("parhash: %d files, %d workers\n", len(out), workers) +} diff --git a/examples/06-concurrency-three-ways/rust/Cargo.lock b/examples/06-concurrency-three-ways/rust/Cargo.lock new file mode 100644 index 0000000..20e06a8 --- /dev/null +++ b/examples/06-concurrency-three-ways/rust/Cargo.lock @@ -0,0 +1,68 @@ +# This file is automatically @generated by Cargo. +# It is not intended for manual editing. +version = 4 + +[[package]] +name = "anyhow" +version = "1.0.104" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "330a5ed07fa54e4702c9d6c4174f74427fc0ef6e214bbd677ae50a5099946470" + +[[package]] +name = "errno" +version = "0.3.14" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "39cab71617ae0d63f51a36d69f866391735b51691dbda63cf6f96d042b63efeb" +dependencies = [ + "libc", + "windows-sys", +] + +[[package]] +name = "libc" +version = "0.2.186" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "68ab91017fe16c622486840e4c83c9a37afeff978bd239b5293d61ece587de66" + +[[package]] +name = "parhash" +version = "0.1.0" +dependencies = [ + "anyhow", + "signal-hook", +] + +[[package]] +name = "signal-hook" +version = "0.4.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b2a0c28ca5908dbdbcd52e6fdaa00358ab88637f8ab33e1f188dd510eb44b53d" +dependencies = [ + "libc", + "signal-hook-registry", +] + +[[package]] +name = "signal-hook-registry" +version = "1.4.8" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c4db69cba1110affc0e9f7bcd48bbf87b3f4fc7c61fc9155afd4c469eb3d6c1b" +dependencies = [ + "errno", + "libc", +] + +[[package]] +name = "windows-link" +version = "0.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5" + +[[package]] +name = "windows-sys" +version = "0.61.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ae137229bcbd6cdf0f7b80a31df61766145077ddf49416a728b02cb3921ff3fc" +dependencies = [ + "windows-link", +] diff --git a/examples/06-concurrency-three-ways/rust/Cargo.toml b/examples/06-concurrency-three-ways/rust/Cargo.toml new file mode 100644 index 0000000..3fcacf4 --- /dev/null +++ b/examples/06-concurrency-three-ways/rust/Cargo.toml @@ -0,0 +1,12 @@ +[package] +name = "parhash" +version = "0.1.0" +edition = "2024" + +[[bin]] +name = "app" +path = "src/main.rs" + +[dependencies] +anyhow = "1.0.104" +signal-hook = "0.4.4" diff --git a/examples/06-concurrency-three-ways/rust/demo.sh b/examples/06-concurrency-three-ways/rust/demo.sh new file mode 100755 index 0000000..f799724 --- /dev/null +++ b/examples/06-concurrency-three-ways/rust/demo.sh @@ -0,0 +1,26 @@ +#!/usr/bin/env bash +set -euo pipefail +cd "$(dirname "${BASH_SOURCE[0]}")" + +BIN=target/release/app + +build() { + cargo build --release +} + +run() { + if [[ -n "${TARGET:-}" ]]; then + local repo_root + repo_root=$(cd ../../.. && pwd) + "$repo_root/scripts/lab/deploy-to-vm.sh" "$TARGET" "$BIN" -- "$@" + else + exec "./$BIN" "$@" # exec so signals sent to this PID reach the app + fi +} + +case "${1:-}" in + build) build ;; + run) shift; run "$@" ;; + "") build; run ;; + *) echo "usage: $0 [build|run [args...]]" >&2; exit 2 ;; +esac diff --git a/examples/06-concurrency-three-ways/rust/rust-toolchain.toml b/examples/06-concurrency-three-ways/rust/rust-toolchain.toml new file mode 100644 index 0000000..4099f1a --- /dev/null +++ b/examples/06-concurrency-three-ways/rust/rust-toolchain.toml @@ -0,0 +1,3 @@ +[toolchain] +channel = "1.97.1" +components = ["rustfmt", "clippy", "rust-src"] diff --git a/examples/06-concurrency-three-ways/rust/src/main.rs b/examples/06-concurrency-three-ways/rust/src/main.rs new file mode 100644 index 0000000..bab4771 --- /dev/null +++ b/examples/06-concurrency-three-ways/rust/src/main.rs @@ -0,0 +1,130 @@ +// parhash: parallel FNV-1a 64 checksummer — Rust flavor. +// +// Concurrency shape: 4 scoped threads share one mpsc Receiver behind a Mutex +// and pull relative paths from it; results flow back to the main thread on a +// second mpsc channel. SIGINT sets a shared AtomicBool (signal-hook's flag +// registry): the walker stops sending, each worker finishes the file it is on +// and refuses further work, and main prints whatever completed plus +// "parhash: interrupted" (exit 130). + +use std::fs::File; +use std::io::Read; +use std::path::{Path, PathBuf}; +use std::process::ExitCode; +use std::sync::atomic::{AtomicBool, Ordering}; +use std::sync::{mpsc, Arc, Mutex}; + +use anyhow::{Context, Result}; + +const WORKERS: usize = 4; +const FNV_OFFSET: u64 = 0xcbf2_9ce4_8422_2325; +const FNV_PRIME: u64 = 0x0000_0100_0000_01b3; + +/// Stream a file through FNV-1a 64 in 64 KiB chunks. +fn hash_file(path: &Path) -> Result { + let mut file = File::open(path).with_context(|| format!("open {}", path.display()))?; + let mut buf = vec![0u8; 64 * 1024]; + let mut hash = FNV_OFFSET; + loop { + let n = file + .read(&mut buf) + .with_context(|| format!("read {}", path.display()))?; + if n == 0 { + return Ok(hash); + } + for &byte in &buf[..n] { + hash ^= u64::from(byte); + hash = hash.wrapping_mul(FNV_PRIME); + } + } +} + +/// Recursively send the relative path of every regular file under `dir`, +/// stopping early once `interrupted` is set. Unreadable subtrees are skipped. +fn walk(dir: &Path, rel: &Path, interrupted: &AtomicBool, tx: &mpsc::Sender) { + let Ok(entries) = std::fs::read_dir(dir) else { + return; + }; + for entry in entries.flatten() { + if interrupted.load(Ordering::Relaxed) { + return; // interrupted: stop accepting work + } + let Ok(file_type) = entry.file_type() else { + continue; + }; + let child_rel = rel.join(entry.file_name()); + if file_type.is_dir() { + walk(&entry.path(), &child_rel, interrupted, tx); + } else if file_type.is_file() { + let _ = tx.send(child_rel.to_string_lossy().into_owned()); + } + } +} + +fn main() -> ExitCode { + let args: Vec = std::env::args().collect(); + let [_, ref root] = args[..] else { + eprintln!("usage: parhash DIR"); + return ExitCode::from(2); + }; + let root = PathBuf::from(root); + if !root.is_dir() { + eprintln!("parhash: cannot walk {}", root.display()); + return ExitCode::from(1); + } + + let interrupted = Arc::new(AtomicBool::new(false)); + signal_hook::flag::register(signal_hook::consts::SIGINT, Arc::clone(&interrupted)) + .expect("register SIGINT flag"); + + let (path_tx, path_rx) = mpsc::channel::(); + let (result_tx, result_rx) = mpsc::channel::<(String, u64)>(); + let path_rx = Arc::new(Mutex::new(path_rx)); + + let mut results: Vec<(String, u64)> = Vec::new(); + std::thread::scope(|scope| { + for _ in 0..WORKERS { + let path_rx = Arc::clone(&path_rx); + let result_tx = result_tx.clone(); + let interrupted = Arc::clone(&interrupted); + let root = &root; + scope.spawn(move || { + loop { + if interrupted.load(Ordering::Relaxed) { + return; // refuse queued-but-unstarted work + } + // Lock only around recv so dequeues serialize but hashing + // runs in parallel across all four workers. + let Ok(rel) = path_rx.lock().expect("receiver lock").recv() else { + return; // walker done and queue drained + }; + match hash_file(&root.join(&rel)) { + Ok(sum) => { + let _ = result_tx.send((rel, sum)); + } + Err(_) => eprintln!("parhash: skipping {rel}"), + } + } + }); + } + drop(result_tx); // main keeps no sender: result_rx ends when workers do + + walk(&root, Path::new(""), &interrupted, &path_tx); + drop(path_tx); // walker done: idle workers see the channel close + + for entry in result_rx { + results.push(entry); // drains until every in-flight hash lands + } + }); + + results.sort_by(|a, b| a.0.cmp(&b.0)); + for (rel, sum) in &results { + println!("{sum:016x} {rel}"); + } + if interrupted.load(Ordering::Relaxed) { + eprintln!("parhash: interrupted"); + return ExitCode::from(130); + } + println!("parhash: {} files, {WORKERS} workers", results.len()); + ExitCode::SUCCESS +} diff --git a/examples/06-concurrency-three-ways/verify.lua b/examples/06-concurrency-three-ways/verify.lua new file mode 100644 index 0000000..a577871 --- /dev/null +++ b/examples/06-concurrency-three-ways/verify.lua @@ -0,0 +1,99 @@ +-- Verify parhash for LSP_LANG: fixed fixture -> byte-identical output across +-- all three languages, wrong usage -> exit 2, and SIGINT on a big tree -> +-- partial results + "parhash: interrupted" + exit 130. +-- Invoked from the example directory with LSP_LANG (cpp|go|rust) and EXAMPLE_DIR set. + +local script_dir = arg[0]:match("(.*/)") or "./" +local repo_root = os.getenv("REPO_ROOT") or (script_dir .. "../..") +package.path = package.path .. ";" .. repo_root .. "/scripts/lib/?.lua" +local checks = require("checks") + +local lang = os.getenv("LSP_LANG") +if lang ~= "cpp" and lang ~= "go" and lang ~= "rust" then + checks.skip("LSP_LANG must be cpp|go|rust (got " .. tostring(lang) .. ")") +end + +local demo = "./demo.sh " .. lang .. " run" + +-- Scratch area (fixture, big tree, captured streams); removed at the end. +local tmp = checks.run("mktemp -d").out:gsub("%s+$", "") +if tmp == "" or not tmp:match("^/") then + checks.skip("mktemp -d failed") +end +local function cleanup() + os.execute("rm -rf '" .. tmp .. "'") +end + +-- --------------------------------------------------------------------------- +-- 1. Known fixture: 5 files (nested dirs, binary, empty) with precomputed +-- FNV-1a 64 sums. The output must be byte-identical for every language. +-- --------------------------------------------------------------------------- +local fix = tmp .. "/fixture" +checks.run(table.concat({ + "mkdir -p " .. fix .. "/sub/deep", + "printf 'alpha\\n' > " .. fix .. "/a.txt", + "printf 'bravo\\n' > " .. fix .. "/b.txt", + "printf 'charlie\\n' > " .. fix .. "/sub/c.txt", + "head -c 1024 /dev/zero > " .. fix .. "/sub/deep/d.bin", + ": > " .. fix .. "/zz-empty.dat", +}, " && ")) + +local expected_file = tmp .. "/expected.txt" +do + local f = assert(io.open(expected_file, "w")) + f:write("bbd23ea491ed9813 a.txt\n") + f:write("4d038ef62a7d9c0b b.txt\n") + f:write("f839d20567c0a911 sub/c.txt\n") + f:write("51d88627df287325 sub/deep/d.bin\n") + f:write("cbf29ce484222325 zz-empty.dat\n") + f:write("parhash: 5 files, 4 workers\n") + f:close() +end + +local actual = tmp .. "/actual.txt" +local r = checks.run(demo .. " " .. fix .. " > " .. actual) +checks.expect_exit(r, 0, lang .. ": fixture run exits 0") + +local d = checks.run("diff -u " .. expected_file .. " " .. actual) +checks.expect_exit(d, 0, lang .. ": stdout is byte-identical to the expected 5 sorted sums + summary") +if d.exit ~= 0 then + print(d.out) -- show the diff on failure +end + +-- --------------------------------------------------------------------------- +-- 2. CLI contract: no argument -> usage on stderr, exit 2. +-- --------------------------------------------------------------------------- +local u = checks.run(demo) +checks.expect_exit(u, 2, lang .. ": missing DIR exits 2") +checks.expect_match(u.out, "usage: parhash DIR", lang .. ": missing DIR prints usage") + +-- --------------------------------------------------------------------------- +-- 3. SIGINT drain: a big sparse tree (16 small + 96 x 64 MiB files, ~6 GiB +-- logical, ~0 on disk) takes seconds to hash; INT at 100 ms must yield +-- only completed hash lines, "parhash: interrupted" on stderr, exit 130. +-- --------------------------------------------------------------------------- +local big = tmp .. "/big" +checks.run(table.concat({ + "mkdir -p " .. big, + "for i in $(seq -w 1 16); do printf 'small %s\\n' \"$i\" > " .. big .. "/aa-$i.txt; done", + "for i in $(seq -w 1 96); do truncate -s 64M " .. big .. "/big-$i.bin; done", +}, " && ")) + +local int_out = tmp .. "/int-out.txt" +local int_err = tmp .. "/int-err.txt" +local i = checks.run(demo .. " " .. big .. " > " .. int_out .. " 2> " .. int_err .. + " & pid=$!; sleep 0.1; kill -INT $pid; wait $pid") +checks.expect_exit(i, 130, lang .. ": SIGINT exits 130") + +local err_text = checks.run("cat " .. int_err).out +checks.expect_match(err_text, "parhash: interrupted", lang .. ": SIGINT prints 'parhash: interrupted' on stderr") + +-- Partial stdout: every line is a completed "<16 hex> " entry and the +-- normal-completion summary never appears. +local bad_lines = checks.run("grep -cEv '^[0-9a-f]{16} ' " .. int_out .. " || true").out +checks.expect_match(bad_lines, "^0%s*$", lang .. ": interrupted stdout holds only hash lines (no summary)") +local summary = checks.run("grep -c 'files, 4 workers' " .. int_out .. " || true").out +checks.expect_match(summary, "^0%s*$", lang .. ": interrupted run prints no summary line") + +cleanup() +checks.finish() diff --git a/examples/07-fds-raii-and-the-vfs/README.md b/examples/07-fds-raii-and-the-vfs/README.md new file mode 100644 index 0000000..6c97097 --- /dev/null +++ b/examples/07-fds-raii-and-the-vfs/README.md @@ -0,0 +1,112 @@ +# 07 — fds, RAII, and the VFS: `fwatch` v0 + +The file-watcher artifact makes its first appearance. `fwatch` v0 has no +events and no timing — it is a purely deterministic snapshot/diff tool built +on directory file descriptors, implemented three times (C++23, Go, Rust) with +identical observable behavior. + +The point of the chapter: a directory is just another fd. Every implementation +walks the tree with *dirfd-relative* syscalls — `openat(2)` to descend, +`fstatat(2)` with `AT_SYMLINK_NOFOLLOW` to classify — and every fd lives +inside an owning wrapper so it is closed exactly once on every path, +including errors: + +- **C++**: a move-only `Fd` class (RAII around `close(2)`) and a `DirStream` + class around `DIR*` — note that `fdopendir(3)` *takes ownership* of the fd, + so `DirStream::adopt(Fd)` releases the fd into the stream and `closedir(3)` + is the single cleanup. Fallible operations return + `std::expected` (or a small `Failure` carrying the + formatted path context). +- **Go**: `unix.Openat` / `unix.Fstatat` from `golang.org/x/sys/unix`, with + `os.NewFile` adopting each directory fd so one deferred `Close` owns it. + Errors are wrapped with `%w`; `errors.Is(err, unix.ENOENT)` skips entries + that vanish mid-walk and `errors.As` digs the raw errno back out for the + common message format. +- **Rust**: `rustix::fs::{open, openat, statat}` returning `OwnedFd` — drop is + the close. `rustix::fs::Dir::read_from` dups the fd for its own cursor, so + the `OwnedFd` stays usable as the anchor for the `*at()` calls. `anyhow` + carries the error context; `?` does the plumbing. + +## CLI + +The binary is named `app` (as everywhere in the book) but speaks as `fwatch`: + +``` +usage: fwatch snapshot DIR + fwatch diff DIR SNAPSHOT_FILE +``` + +**`fwatch snapshot DIR`** walks `DIR` recursively and prints one line per +regular file to stdout, sorted bytewise by path: + +``` + +``` + +Directories are descended into but not listed; symlinks, sockets, pipes, and +devices are ignored in v0. Entries that disappear between `readdir` and +`fstatat` are silently skipped. + +**`fwatch diff DIR SNAPSHOT_FILE`** re-scans `DIR`, compares against the +saved snapshot, and prints — in sorted path order — + +``` ++ new file +- file is gone +~ size or mtime changed +fwatch: added, removed, modified +``` + +then exits 0 (a non-empty diff is not an error). The summary line is always +printed, even when all counts are zero. + +Snapshot lines are parsed from the right (the last two space-separated fields +are size and mtime), so paths containing spaces round-trip. + +**Exit codes**: 0 success, 1 runtime error (`fwatch: error: ... (errno N)` on +stderr, identical text in all three languages), 2 usage error (usage text on +stderr). + +## Layout + +``` +07-fds-raii-and-the-vfs/ +├── demo.sh # dispatcher: ./demo.sh [cpp|go|rust|all|build] [args...] +├── verify.lua # automated check driven by CI +├── cpp/ # CMake preset build, demo.sh +├── go/ # go build, demo.sh +└── rust/ # cargo build, demo.sh +``` + +## The demo contract + +Every language directory has a `demo.sh` with the book-wide interface: + +- `./demo.sh` — build, then run a self-contained fixture demo: create a small + tree in a temp dir, `snapshot` it, mutate it (append / remove / create), + and `diff` it against the saved snapshot. +- `./demo.sh build` — build only +- `./demo.sh run [args]` — run the built binary with `args` + (e.g. `./demo.sh run snapshot /etc`) +- With env `TARGET` set, `run` deploys the binary to that lab VM via + `scripts/lab/deploy-to-vm.sh "$TARGET" -- [args]` instead of + running locally. + +The top-level `demo.sh` is only a dispatcher: `./demo.sh cpp run snapshot .` +execs `cpp/demo.sh run snapshot .`. + +## Verification + +`verify.lua` (run per language with `LSP_LANG` set) builds a fixture tree +whose mtimes are pinned with `touch -d @EPOCH`, then asserts observable +behavior end to end: + +1. `snapshot` output is byte-for-byte the four expected sorted entry lines. +2. `diff` against the unchanged tree prints only + `fwatch: 0 added, 0 removed, 0 modified`. +3. After adding `new.txt`, deleting `gone.txt`, and appending to + retouching + `a.txt`, `diff` prints exactly `~ a.txt`, `- gone.txt`, `+ new.txt`, and + the `1 added, 1 removed, 1 modified` summary — nothing else. +4. A fresh `snapshot` reflects the new size and mtime of `a.txt`. +5. Error paths: missing directory → exit 1 with `(errno 2)`; missing snapshot + file → exit 1; missing arguments → exit 2 with the usage text. diff --git a/examples/07-fds-raii-and-the-vfs/cpp/.clang-format b/examples/07-fds-raii-and-the-vfs/cpp/.clang-format new file mode 100644 index 0000000..2c8640f --- /dev/null +++ b/examples/07-fds-raii-and-the-vfs/cpp/.clang-format @@ -0,0 +1,8 @@ +--- +BasedOnStyle: LLVM +Language: Cpp +Standard: c++20 +IndentWidth: 4 +ColumnLimit: 100 +PointerAlignment: Left +AllowShortFunctionsOnASingleLine: Inline diff --git a/examples/07-fds-raii-and-the-vfs/cpp/.clang-tidy b/examples/07-fds-raii-and-the-vfs/cpp/.clang-tidy new file mode 100644 index 0000000..2f275ba --- /dev/null +++ b/examples/07-fds-raii-and-the-vfs/cpp/.clang-tidy @@ -0,0 +1,11 @@ +--- +Checks: > + bugprone-*, + modernize-*, + performance-*, + readability-*, + -modernize-use-trailing-return-type, + -readability-identifier-length +WarningsAsErrors: '' +HeaderFilterRegex: '.*' +FormatStyle: file diff --git a/examples/07-fds-raii-and-the-vfs/cpp/CMakeLists.txt b/examples/07-fds-raii-and-the-vfs/cpp/CMakeLists.txt new file mode 100644 index 0000000..ff62606 --- /dev/null +++ b/examples/07-fds-raii-and-the-vfs/cpp/CMakeLists.txt @@ -0,0 +1,8 @@ +cmake_minimum_required(VERSION 3.25) +project(fwatch CXX) + +set(CMAKE_CXX_STANDARD 23) +set(CMAKE_CXX_STANDARD_REQUIRED ON) +set(CMAKE_CXX_EXTENSIONS OFF) + +add_executable(app src/main.cpp) diff --git a/examples/07-fds-raii-and-the-vfs/cpp/CMakePresets.json b/examples/07-fds-raii-and-the-vfs/cpp/CMakePresets.json new file mode 100644 index 0000000..e31bd2f --- /dev/null +++ b/examples/07-fds-raii-and-the-vfs/cpp/CMakePresets.json @@ -0,0 +1,62 @@ +{ + "version": 6, + "cmakeMinimumRequired": { "major": 3, "minor": 25, "patch": 0 }, + "configurePresets": [ + { + "name": "_base", + "hidden": true, + "generator": "Ninja", + "binaryDir": "${sourceDir}/build/${presetName}", + "cacheVariables": { + "CMAKE_EXPORT_COMPILE_COMMANDS": "ON" + } + }, + { + "name": "release", + "inherits": "_base", + "displayName": "Release (-O2 with debug info)", + "cacheVariables": { "CMAKE_BUILD_TYPE": "RelWithDebInfo" } + }, + { + "name": "debug", + "inherits": "_base", + "displayName": "Debug", + "cacheVariables": { "CMAKE_BUILD_TYPE": "Debug" } + }, + { + "name": "asan", + "inherits": "_base", + "displayName": "ASan + UBSan instrumented build", + "cacheVariables": { + "CMAKE_BUILD_TYPE": "Debug", + "CMAKE_CXX_FLAGS": "-O1 -g -fno-omit-frame-pointer -fsanitize=address,undefined", + "CMAKE_EXE_LINKER_FLAGS": "-fsanitize=address,undefined" + } + }, + { + "name": "release-clang", + "inherits": "release", + "displayName": "Release (clang)", + "cacheVariables": { + "CMAKE_C_COMPILER": "clang", + "CMAKE_CXX_COMPILER": "clang++" + } + }, + { + "name": "debug-clang", + "inherits": "debug", + "displayName": "Debug (clang)", + "cacheVariables": { + "CMAKE_C_COMPILER": "clang", + "CMAKE_CXX_COMPILER": "clang++" + } + } + ], + "buildPresets": [ + { "name": "release", "configurePreset": "release" }, + { "name": "debug", "configurePreset": "debug" }, + { "name": "asan", "configurePreset": "asan" }, + { "name": "release-clang", "configurePreset": "release-clang" }, + { "name": "debug-clang", "configurePreset": "debug-clang" } + ] +} diff --git a/examples/07-fds-raii-and-the-vfs/cpp/demo.sh b/examples/07-fds-raii-and-the-vfs/cpp/demo.sh new file mode 100755 index 0000000..73e69d3 --- /dev/null +++ b/examples/07-fds-raii-and-the-vfs/cpp/demo.sh @@ -0,0 +1,58 @@ +#!/usr/bin/env bash +set -euo pipefail + +cd "$(dirname "${BASH_SOURCE[0]}")" +REPO_ROOT="$(cd ../../.. && pwd)" +BIN=build/release/app + +build() { + cmake --preset release + cmake --build --preset release +} + +run() { + if [[ ! -x "$BIN" ]]; then + build + fi + if [[ -n "${TARGET:-}" ]]; then + "$REPO_ROOT/scripts/lab/deploy-to-vm.sh" "$TARGET" "$BIN" -- "$@" + else + "./$BIN" "$@" + fi +} + +# Self-contained fixture demo: fwatch needs a directory to watch, so the bare +# invocation builds one, snapshots it, mutates it, and diffs. +demo() { + local tmp + tmp=$(mktemp -d) + trap "rm -rf '$tmp'" EXIT + mkdir -p "$tmp/tree/sub" + printf 'one\n' >"$tmp/tree/a.txt" + printf 'nested\n' >"$tmp/tree/sub/b.txt" + echo "# fwatch snapshot" + "./$BIN" snapshot "$tmp/tree" | tee "$tmp/snap.txt" + printf 'grown\n' >>"$tmp/tree/a.txt" + rm "$tmp/tree/sub/b.txt" + printf 'new\n' >"$tmp/tree/c.txt" + echo "# fwatch diff (after append/remove/create)" + "./$BIN" diff "$tmp/tree" "$tmp/snap.txt" +} + +case "${1:-}" in + build) + build + ;; + run) + shift + run "$@" + ;; + "") + build + demo + ;; + *) + echo "usage: $0 [build|run [args]]" >&2 + exit 2 + ;; +esac diff --git a/examples/07-fds-raii-and-the-vfs/cpp/src/main.cpp b/examples/07-fds-raii-and-the-vfs/cpp/src/main.cpp new file mode 100644 index 0000000..53bfa80 --- /dev/null +++ b/examples/07-fds-raii-and-the-vfs/cpp/src/main.cpp @@ -0,0 +1,348 @@ +// fwatch v0 — snapshot/diff a directory tree with dirfd-relative syscalls. +// +// fwatch snapshot DIR one " " line per +// regular file, sorted by path +// fwatch diff DIR SNAPSHOT_FILE re-scan and print +/-/~ lines plus a +// "fwatch: A added, R removed, M modified" +// summary +// +// The walk is deliberately explicit: openat(2) relative to the parent +// directory fd, fdopendir(3) to enumerate, fstatat(2) with +// AT_SYMLINK_NOFOLLOW to classify. Every fd and DIR* lives inside an RAII +// wrapper; fallible operations return std::expected. + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include + +namespace { + +// --- RAII wrappers ----------------------------------------------------------- + +// Owns a file descriptor; closes it exactly once. +class Fd { +public: + Fd() = default; + explicit Fd(int fd) noexcept : fd_(fd) {} + ~Fd() { reset(); } + + Fd(const Fd&) = delete; + Fd& operator=(const Fd&) = delete; + + Fd(Fd&& other) noexcept : fd_(std::exchange(other.fd_, -1)) {} + Fd& operator=(Fd&& other) noexcept { + if (this != &other) { + reset(); + fd_ = std::exchange(other.fd_, -1); + } + return *this; + } + + [[nodiscard]] int get() const noexcept { return fd_; } + [[nodiscard]] int release() noexcept { return std::exchange(fd_, -1); } + explicit operator bool() const noexcept { return fd_ >= 0; } + +private: + void reset() noexcept { + if (fd_ >= 0) { + ::close(fd_); + fd_ = -1; + } + } + int fd_ = -1; +}; + +// Owns a DIR*. fdopendir(3) transfers ownership of the underlying fd to the +// DIR stream, so closedir(3) is the one and only cleanup. +class DirStream { +public: + // Adopt an already-open directory fd. On success the Fd is released into + // the DIR*; on failure the Fd destructor still closes it. + [[nodiscard]] static std::expected adopt(Fd fd) { + DIR* dir = ::fdopendir(fd.get()); + if (dir == nullptr) { + return std::unexpected(std::error_code{errno, std::system_category()}); + } + (void)fd.release(); // the DIR* owns the fd now + return DirStream{dir}; + } + + ~DirStream() { + if (dir_ != nullptr) ::closedir(dir_); + } + + DirStream(const DirStream&) = delete; + DirStream& operator=(const DirStream&) = delete; + + DirStream(DirStream&& other) noexcept : dir_(std::exchange(other.dir_, nullptr)) {} + DirStream& operator=(DirStream&& other) noexcept { + if (this != &other) { + if (dir_ != nullptr) ::closedir(dir_); + dir_ = std::exchange(other.dir_, nullptr); + } + return *this; + } + + [[nodiscard]] DIR* get() const noexcept { return dir_; } + [[nodiscard]] int fd() const noexcept { return ::dirfd(dir_); } + +private: + explicit DirStream(DIR* dir) noexcept : dir_(dir) {} + DIR* dir_ = nullptr; +}; + +// --- Scanning ---------------------------------------------------------------- + +struct Info { + std::int64_t size; + std::int64_t mtime; + bool operator==(const Info&) const = default; +}; + +using Tree = std::map; // sorted by path, bytewise + +// A failed operation, already formatted with the path it failed on. +struct Failure { + std::string message; // e.g. "cannot open directory 'x' (errno 2)" +}; + +constexpr int kDirFlags = O_RDONLY | O_DIRECTORY | O_CLOEXEC | O_NOFOLLOW; + +[[nodiscard]] std::expected open_dir_at(int parent_fd, + const char* name) { + const int raw = ::openat(parent_fd, name, kDirFlags); + if (raw < 0) { + return std::unexpected(std::error_code{errno, std::system_category()}); + } + return Fd{raw}; +} + +[[nodiscard]] std::expected stat_at(int dirfd, + const char* name) { + struct stat st{}; + if (::fstatat(dirfd, name, &st, AT_SYMLINK_NOFOLLOW) != 0) { + return std::unexpected(std::error_code{errno, std::system_category()}); + } + return st; +} + +[[nodiscard]] std::string join(const std::string& base, const std::string& name) { + if (base.empty()) return name; + return base + "/" + name; +} + +[[nodiscard]] Failure fail(const std::string& what, const std::string& path, + std::error_code ec) { + return Failure{std::format("cannot {} '{}' (errno {})", what, path, ec.value())}; +} + +// Walk the directory opened as `fd`, recording every regular file into `tree`. +// `display` is the path for error messages, `rel` the DIR-relative prefix. +[[nodiscard]] std::expected walk(Fd fd, const std::string& display, + const std::string& rel, Tree& tree) { + auto stream = DirStream::adopt(std::move(fd)); + if (!stream) { + return std::unexpected(fail("open directory", display, stream.error())); + } + + while (const dirent* entry = ::readdir(stream->get())) { + const std::string name = entry->d_name; + if (name == "." || name == "..") continue; + + const auto st = stat_at(stream->fd(), name.c_str()); + if (!st) { + if (st.error().value() == ENOENT) continue; // vanished mid-walk + return std::unexpected(fail("stat", join(display, name), st.error())); + } + + if (S_ISREG(st->st_mode)) { + tree[join(rel, name)] = Info{static_cast(st->st_size), + static_cast(st->st_mtim.tv_sec)}; + } else if (S_ISDIR(st->st_mode)) { + auto sub = open_dir_at(stream->fd(), name.c_str()); + if (!sub) { + if (sub.error().value() == ENOENT) continue; + return std::unexpected( + fail("open directory", join(display, name), sub.error())); + } + if (auto walked = + walk(std::move(*sub), join(display, name), join(rel, name), tree); + !walked) { + return walked; + } + } + // Symlinks, sockets, pipes, devices: not part of the v0 snapshot. + } + return {}; +} + +[[nodiscard]] std::expected scan(const std::string& dir) { + auto fd = open_dir_at(AT_FDCWD, dir.c_str()); + if (!fd) { + return std::unexpected(fail("open directory", dir, fd.error())); + } + Tree tree; + if (auto walked = walk(std::move(*fd), dir, "", tree); !walked) { + return std::unexpected(walked.error()); + } + return tree; +} + +// --- Snapshot file I/O ------------------------------------------------------- + +[[nodiscard]] std::expected read_file( + const std::string& path) { + const int raw = ::open(path.c_str(), O_RDONLY | O_CLOEXEC); + if (raw < 0) { + return std::unexpected(std::error_code{errno, std::system_category()}); + } + const Fd fd{raw}; + std::string content; + char buf[65536]; + for (;;) { + const ssize_t n = ::read(fd.get(), buf, sizeof buf); + if (n < 0) { + if (errno == EINTR) continue; + return std::unexpected(std::error_code{errno, std::system_category()}); + } + if (n == 0) break; + content.append(buf, static_cast(n)); + } + return content; +} + +// Parse "path size mtime" lines. The path may contain spaces: size and mtime +// are the last two space-separated fields. +[[nodiscard]] std::expected parse_snapshot(const std::string& text) { + Tree tree; + std::size_t pos = 0; + int lineno = 0; + while (pos < text.size()) { + std::size_t end = text.find('\n', pos); + if (end == std::string::npos) end = text.size(); + const std::string line = text.substr(pos, end - pos); + pos = end + 1; + ++lineno; + if (line.empty()) continue; + + const auto malformed = [lineno] { + return Failure{std::format("malformed snapshot line {}", lineno)}; + }; + + const std::size_t sp2 = line.rfind(' '); + if (sp2 == std::string::npos || sp2 == 0) return std::unexpected(malformed()); + const std::size_t sp1 = line.rfind(' ', sp2 - 1); + if (sp1 == std::string::npos || sp1 == 0) return std::unexpected(malformed()); + + const std::string path = line.substr(0, sp1); + const std::string size_s = line.substr(sp1 + 1, sp2 - sp1 - 1); + const std::string mtime_s = line.substr(sp2 + 1); + + Info info{}; + const auto [p1, e1] = + std::from_chars(size_s.data(), size_s.data() + size_s.size(), info.size); + const auto [p2, e2] = std::from_chars(mtime_s.data(), + mtime_s.data() + mtime_s.size(), info.mtime); + if (e1 != std::errc{} || p1 != size_s.data() + size_s.size() || + e2 != std::errc{} || p2 != mtime_s.data() + mtime_s.size()) { + return std::unexpected(malformed()); + } + tree[path] = info; + } + return tree; +} + +// --- Subcommands ------------------------------------------------------------- + +int cmd_snapshot(const std::string& dir) { + const auto tree = scan(dir); + if (!tree) { + std::println(stderr, "fwatch: error: {}", tree.error().message); + return 1; + } + for (const auto& [path, info] : *tree) { + std::println("{} {} {}", path, info.size, info.mtime); + } + return 0; +} + +int cmd_diff(const std::string& dir, const std::string& snapshot_path) { + const auto text = read_file(snapshot_path); + if (!text) { + std::println(stderr, "fwatch: error: cannot read snapshot '{}' (errno {})", + snapshot_path, text.error().value()); + return 1; + } + const auto before = parse_snapshot(*text); + if (!before) { + std::println(stderr, "fwatch: error: {}", before.error().message); + return 1; + } + const auto after = scan(dir); + if (!after) { + std::println(stderr, "fwatch: error: {}", after.error().message); + return 1; + } + + // Sorted union of both key sets; each map is already ordered. + std::vector paths; + paths.reserve(before->size() + after->size()); + for (const auto& [path, info] : *before) paths.push_back(path); + for (const auto& [path, info] : *after) paths.push_back(path); + std::ranges::sort(paths); + const auto dead = std::ranges::unique(paths); + paths.erase(dead.begin(), dead.end()); + + int added = 0; + int removed = 0; + int modified = 0; + for (const auto& path : paths) { + const auto old_it = before->find(path); + const auto new_it = after->find(path); + if (old_it == before->end()) { + std::println("+ {}", path); + ++added; + } else if (new_it == after->end()) { + std::println("- {}", path); + ++removed; + } else if (old_it->second != new_it->second) { + std::println("~ {}", path); + ++modified; + } + } + std::println("fwatch: {} added, {} removed, {} modified", added, removed, modified); + return 0; +} + +int usage() { + std::println(stderr, "usage: fwatch snapshot DIR"); + std::println(stderr, " fwatch diff DIR SNAPSHOT_FILE"); + return 2; +} + +} // namespace + +int main(int argc, char** argv) { + const std::vector args(argv + 1, argv + argc); + if (args.size() == 2 && args[0] == "snapshot") { + return cmd_snapshot(args[1]); + } + if (args.size() == 3 && args[0] == "diff") { + return cmd_diff(args[1], args[2]); + } + return usage(); +} diff --git a/examples/07-fds-raii-and-the-vfs/demo.sh b/examples/07-fds-raii-and-the-vfs/demo.sh new file mode 100755 index 0000000..afbbe4c --- /dev/null +++ b/examples/07-fds-raii-and-the-vfs/demo.sh @@ -0,0 +1,23 @@ +#!/usr/bin/env bash +set -euo pipefail +cd "$(dirname "$0")" + +langs=(cpp go rust) + +case "${1:-all}" in + cpp|go|rust) + lang="$1" + shift + exec "./$lang/demo.sh" "$@" + ;; + all) + for lang in "${langs[@]}"; do "./$lang/demo.sh"; done + ;; + build) + for lang in "${langs[@]}"; do "./$lang/demo.sh" build; done + ;; + *) + echo "usage: $0 [cpp|go|rust|all|build] [args...]" >&2 + exit 2 + ;; +esac diff --git a/examples/07-fds-raii-and-the-vfs/go/demo.sh b/examples/07-fds-raii-and-the-vfs/go/demo.sh new file mode 100755 index 0000000..f99dc4d --- /dev/null +++ b/examples/07-fds-raii-and-the-vfs/go/demo.sh @@ -0,0 +1,41 @@ +#!/usr/bin/env bash +set -euo pipefail +cd "$(dirname "$0")" +REPO_ROOT="$(cd ../../.. && pwd)" +BIN=bin/app + +build() { + go build -o "$BIN" . +} + +run() { + if [[ -n "${TARGET:-}" ]]; then + exec "$REPO_ROOT/scripts/lab/deploy-to-vm.sh" "$TARGET" "$BIN" -- "$@" + fi + exec "./$BIN" "$@" +} + +# Self-contained fixture demo: fwatch needs a directory to watch, so the bare +# invocation builds one, snapshots it, mutates it, and diffs. +demo() { + local tmp + tmp=$(mktemp -d) + trap "rm -rf '$tmp'" EXIT + mkdir -p "$tmp/tree/sub" + printf 'one\n' >"$tmp/tree/a.txt" + printf 'nested\n' >"$tmp/tree/sub/b.txt" + echo "# fwatch snapshot" + "./$BIN" snapshot "$tmp/tree" | tee "$tmp/snap.txt" + printf 'grown\n' >>"$tmp/tree/a.txt" + rm "$tmp/tree/sub/b.txt" + printf 'new\n' >"$tmp/tree/c.txt" + echo "# fwatch diff (after append/remove/create)" + "./$BIN" diff "$tmp/tree" "$tmp/snap.txt" +} + +case "${1:-}" in + build) build ;; + run) shift; run "$@" ;; + "") build; demo ;; + *) echo "usage: $0 [build|run [args...]]" >&2; exit 2 ;; +esac diff --git a/examples/07-fds-raii-and-the-vfs/go/go.mod b/examples/07-fds-raii-and-the-vfs/go/go.mod new file mode 100644 index 0000000..859c00a --- /dev/null +++ b/examples/07-fds-raii-and-the-vfs/go/go.mod @@ -0,0 +1,7 @@ +module github.com/patterncatalyst/linux-systems-programming/examples/07-fds-raii-and-the-vfs/go + +go 1.26 + +toolchain go1.26.5 + +require golang.org/x/sys v0.47.0 diff --git a/examples/07-fds-raii-and-the-vfs/go/go.sum b/examples/07-fds-raii-and-the-vfs/go/go.sum new file mode 100644 index 0000000..37ee2d4 --- /dev/null +++ b/examples/07-fds-raii-and-the-vfs/go/go.sum @@ -0,0 +1,2 @@ +golang.org/x/sys v0.47.0 h1:o7XGOvZQCADBQQ4Y7VNq2dRWQR7JmOUW8Kxx4ZsNgWs= +golang.org/x/sys v0.47.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw= diff --git a/examples/07-fds-raii-and-the-vfs/go/main.go b/examples/07-fds-raii-and-the-vfs/go/main.go new file mode 100644 index 0000000..8040b5d --- /dev/null +++ b/examples/07-fds-raii-and-the-vfs/go/main.go @@ -0,0 +1,228 @@ +// fwatch v0 — snapshot/diff a directory tree with dirfd-relative syscalls. +// +// fwatch snapshot DIR one " " line per +// regular file, sorted by path +// fwatch diff DIR SNAPSHOT_FILE re-scan and print +/-/~ lines plus a +// "fwatch: A added, R removed, M modified" +// summary +// +// The walk is deliberately explicit: unix.Openat relative to the parent +// directory fd, Readdirnames on an os.File wrapping that fd, and +// unix.Fstatat with AT_SYMLINK_NOFOLLOW to classify entries. os.File is the +// fd's owner: one deferred Close per directory, no leaks on any error path. +package main + +import ( + "errors" + "fmt" + "os" + "sort" + "strconv" + "strings" + + "golang.org/x/sys/unix" +) + +type info struct { + size int64 + mtime int64 +} + +type tree map[string]info + +const dirFlags = unix.O_RDONLY | unix.O_DIRECTORY | unix.O_CLOEXEC | unix.O_NOFOLLOW + +// errnoOf digs the raw errno out of a wrapped error chain so every language +// implementation reports the same "(errno N)" suffix. +func errnoOf(err error) int { + var errno unix.Errno + if errors.As(err, &errno) { + return int(errno) + } + return 0 +} + +func failf(what, path string, err error) error { + return fmt.Errorf("cannot %s '%s' (errno %d)", what, path, errnoOf(err)) +} + +func join(base, name string) string { + if base == "" { + return name + } + return base + "/" + name +} + +// walk enumerates the directory owned by f, recording every regular file +// into out. display is the path for error messages, rel the DIR-relative +// prefix. f is closed before walk returns. +func walk(f *os.File, display, rel string, out tree) error { + defer f.Close() + + names, err := f.Readdirnames(-1) + if err != nil { + return fmt.Errorf("reading '%s': %w", display, err) + } + + dirfd := int(f.Fd()) + for _, name := range names { + var st unix.Stat_t + if err := unix.Fstatat(dirfd, name, &st, unix.AT_SYMLINK_NOFOLLOW); err != nil { + if errors.Is(err, unix.ENOENT) { + continue // vanished mid-walk + } + return failf("stat", join(display, name), err) + } + + switch st.Mode & unix.S_IFMT { + case unix.S_IFREG: + out[join(rel, name)] = info{size: st.Size, mtime: st.Mtim.Sec} + case unix.S_IFDIR: + childFd, err := unix.Openat(dirfd, name, dirFlags, 0) + if err != nil { + if errors.Is(err, unix.ENOENT) { + continue + } + return failf("open directory", join(display, name), err) + } + child := os.NewFile(uintptr(childFd), join(display, name)) + if err := walk(child, join(display, name), join(rel, name), out); err != nil { + return err + } + } + // Symlinks, sockets, pipes, devices: not part of the v0 snapshot. + } + return nil +} + +func scan(dir string) (tree, error) { + fd, err := unix.Open(dir, dirFlags, 0) + if err != nil { + return nil, failf("open directory", dir, err) + } + out := tree{} + if err := walk(os.NewFile(uintptr(fd), dir), dir, "", out); err != nil { + return nil, err + } + return out, nil +} + +func sortedPaths(t tree) []string { + paths := make([]string, 0, len(t)) + for path := range t { + paths = append(paths, path) + } + sort.Strings(paths) + return paths +} + +// parseSnapshot reads "path size mtime" lines. The path may contain spaces: +// size and mtime are the last two space-separated fields. +func parseSnapshot(text string) (tree, error) { + out := tree{} + for i, line := range strings.Split(text, "\n") { + if line == "" { + continue + } + malformed := fmt.Errorf("malformed snapshot line %d", i+1) + + sp2 := strings.LastIndexByte(line, ' ') + if sp2 <= 0 { + return nil, malformed + } + sp1 := strings.LastIndexByte(line[:sp2], ' ') + if sp1 <= 0 { + return nil, malformed + } + + size, err1 := strconv.ParseInt(line[sp1+1:sp2], 10, 64) + mtime, err2 := strconv.ParseInt(line[sp2+1:], 10, 64) + if err1 != nil || err2 != nil { + return nil, malformed + } + out[line[:sp1]] = info{size: size, mtime: mtime} + } + return out, nil +} + +func cmdSnapshot(dir string) error { + t, err := scan(dir) + if err != nil { + return err + } + for _, path := range sortedPaths(t) { + fmt.Printf("%s %d %d\n", path, t[path].size, t[path].mtime) + } + return nil +} + +func cmdDiff(dir, snapshotPath string) error { + text, err := os.ReadFile(snapshotPath) + if err != nil { + return failf("read snapshot", snapshotPath, err) + } + before, err := parseSnapshot(string(text)) + if err != nil { + return err + } + after, err := scan(dir) + if err != nil { + return err + } + + // Sorted union of both key sets. + union := make(map[string]struct{}, len(before)+len(after)) + for path := range before { + union[path] = struct{}{} + } + for path := range after { + union[path] = struct{}{} + } + paths := make([]string, 0, len(union)) + for path := range union { + paths = append(paths, path) + } + sort.Strings(paths) + + added, removed, modified := 0, 0, 0 + for _, path := range paths { + old, wasThere := before[path] + now, isThere := after[path] + switch { + case !wasThere: + fmt.Printf("+ %s\n", path) + added++ + case !isThere: + fmt.Printf("- %s\n", path) + removed++ + case old != now: + fmt.Printf("~ %s\n", path) + modified++ + } + } + fmt.Printf("fwatch: %d added, %d removed, %d modified\n", added, removed, modified) + return nil +} + +func usage() { + fmt.Fprintln(os.Stderr, "usage: fwatch snapshot DIR") + fmt.Fprintln(os.Stderr, " fwatch diff DIR SNAPSHOT_FILE") + os.Exit(2) +} + +func main() { + args := os.Args[1:] + var err error + switch { + case len(args) == 2 && args[0] == "snapshot": + err = cmdSnapshot(args[1]) + case len(args) == 3 && args[0] == "diff": + err = cmdDiff(args[1], args[2]) + default: + usage() + } + if err != nil { + fmt.Fprintf(os.Stderr, "fwatch: error: %v\n", err) + os.Exit(1) + } +} diff --git a/examples/07-fds-raii-and-the-vfs/rust/Cargo.lock b/examples/07-fds-raii-and-the-vfs/rust/Cargo.lock new file mode 100644 index 0000000..5d100e9 --- /dev/null +++ b/examples/07-fds-raii-and-the-vfs/rust/Cargo.lock @@ -0,0 +1,73 @@ +# This file is automatically @generated by Cargo. +# It is not intended for manual editing. +version = 4 + +[[package]] +name = "anyhow" +version = "1.0.104" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "330a5ed07fa54e4702c9d6c4174f74427fc0ef6e214bbd677ae50a5099946470" + +[[package]] +name = "bitflags" +version = "2.13.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b588b76d00fde79687d7646a9b5bdf3cc0f655e0bbd080335a95d7e96f3587da" + +[[package]] +name = "errno" +version = "0.3.14" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "39cab71617ae0d63f51a36d69f866391735b51691dbda63cf6f96d042b63efeb" +dependencies = [ + "libc", + "windows-sys", +] + +[[package]] +name = "fwatch" +version = "0.1.0" +dependencies = [ + "anyhow", + "rustix", +] + +[[package]] +name = "libc" +version = "0.2.186" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "68ab91017fe16c622486840e4c83c9a37afeff978bd239b5293d61ece587de66" + +[[package]] +name = "linux-raw-sys" +version = "0.12.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "32a66949e030da00e8c7d4434b251670a91556f4144941d37452769c25d58a53" + +[[package]] +name = "rustix" +version = "1.1.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b6fe4565b9518b83ef4f91bb47ce29620ca828bd32cb7e408f0062e9930ba190" +dependencies = [ + "bitflags", + "errno", + "libc", + "linux-raw-sys", + "windows-sys", +] + +[[package]] +name = "windows-link" +version = "0.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5" + +[[package]] +name = "windows-sys" +version = "0.61.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ae137229bcbd6cdf0f7b80a31df61766145077ddf49416a728b02cb3921ff3fc" +dependencies = [ + "windows-link", +] diff --git a/examples/07-fds-raii-and-the-vfs/rust/Cargo.toml b/examples/07-fds-raii-and-the-vfs/rust/Cargo.toml new file mode 100644 index 0000000..9255e52 --- /dev/null +++ b/examples/07-fds-raii-and-the-vfs/rust/Cargo.toml @@ -0,0 +1,12 @@ +[package] +name = "fwatch" +version = "0.1.0" +edition = "2024" + +[[bin]] +name = "app" +path = "src/main.rs" + +[dependencies] +anyhow = "1" +rustix = { version = "1", features = ["fs"] } diff --git a/examples/07-fds-raii-and-the-vfs/rust/demo.sh b/examples/07-fds-raii-and-the-vfs/rust/demo.sh new file mode 100755 index 0000000..7f6948e --- /dev/null +++ b/examples/07-fds-raii-and-the-vfs/rust/demo.sh @@ -0,0 +1,44 @@ +#!/usr/bin/env bash +set -euo pipefail +cd "$(dirname "${BASH_SOURCE[0]}")" + +BIN=target/release/app + +build() { + cargo build --release +} + +run() { + if [[ -n "${TARGET:-}" ]]; then + local repo_root + repo_root=$(cd ../../.. && pwd) + "$repo_root/scripts/lab/deploy-to-vm.sh" "$TARGET" "$BIN" -- "$@" + else + "./$BIN" "$@" + fi +} + +# Self-contained fixture demo: fwatch needs a directory to watch, so the bare +# invocation builds one, snapshots it, mutates it, and diffs. +demo() { + local tmp + tmp=$(mktemp -d) + trap "rm -rf '$tmp'" EXIT + mkdir -p "$tmp/tree/sub" + printf 'one\n' >"$tmp/tree/a.txt" + printf 'nested\n' >"$tmp/tree/sub/b.txt" + echo "# fwatch snapshot" + "./$BIN" snapshot "$tmp/tree" | tee "$tmp/snap.txt" + printf 'grown\n' >>"$tmp/tree/a.txt" + rm "$tmp/tree/sub/b.txt" + printf 'new\n' >"$tmp/tree/c.txt" + echo "# fwatch diff (after append/remove/create)" + "./$BIN" diff "$tmp/tree" "$tmp/snap.txt" +} + +case "${1:-}" in + build) build ;; + run) shift; run "$@" ;; + "") build; demo ;; + *) echo "usage: $0 [build|run [args...]]" >&2; exit 2 ;; +esac diff --git a/examples/07-fds-raii-and-the-vfs/rust/rust-toolchain.toml b/examples/07-fds-raii-and-the-vfs/rust/rust-toolchain.toml new file mode 100644 index 0000000..4099f1a --- /dev/null +++ b/examples/07-fds-raii-and-the-vfs/rust/rust-toolchain.toml @@ -0,0 +1,3 @@ +[toolchain] +channel = "1.97.1" +components = ["rustfmt", "clippy", "rust-src"] diff --git a/examples/07-fds-raii-and-the-vfs/rust/src/main.rs b/examples/07-fds-raii-and-the-vfs/rust/src/main.rs new file mode 100644 index 0000000..e35edc1 --- /dev/null +++ b/examples/07-fds-raii-and-the-vfs/rust/src/main.rs @@ -0,0 +1,183 @@ +// fwatch v0 — snapshot/diff a directory tree with dirfd-relative syscalls. +// +// fwatch snapshot DIR one " " line per +// regular file, sorted by path +// fwatch diff DIR SNAPSHOT_FILE re-scan and print +/-/~ lines plus a +// "fwatch: A added, R removed, M modified" +// summary +// +// The walk is deliberately explicit: rustix::fs::openat relative to the +// parent directory fd (an OwnedFd — closed exactly once, on drop), +// rustix::fs::Dir to enumerate, and rustix::fs::statat with +// SYMLINK_NOFOLLOW to classify entries. + +use std::collections::BTreeMap; +use std::os::fd::OwnedFd; +use std::process::ExitCode; + +use anyhow::{Result, anyhow, bail}; +use rustix::fs::{AtFlags, Dir, FileType, Mode, OFlags, openat, statat}; + +#[derive(Clone, Copy, PartialEq, Eq)] +struct Info { + size: i64, + mtime: i64, +} + +/// Sorted by path, bytewise — BTreeMap keeps snapshot output deterministic. +type Tree = BTreeMap; + +const DIR_FLAGS: OFlags = OFlags::RDONLY + .union(OFlags::DIRECTORY) + .union(OFlags::CLOEXEC) + .union(OFlags::NOFOLLOW); + +fn fail(what: &str, path: &str, errno: rustix::io::Errno) -> anyhow::Error { + anyhow!("cannot {what} '{path}' (errno {})", errno.raw_os_error()) +} + +fn join(base: &str, name: &str) -> String { + if base.is_empty() { name.to_string() } else { format!("{base}/{name}") } +} + +/// Walk the directory owned by `fd`, recording every regular file into +/// `tree`. `display` is the path for error messages, `rel` the DIR-relative +/// prefix. `fd` is dropped (closed) when the walk of this level finishes. +fn walk(fd: OwnedFd, display: &str, rel: &str, tree: &mut Tree) -> Result<()> { + // Dir::read_from dups the fd for its own cursor; `fd` stays usable as the + // anchor for the *at() calls below. + let dir = Dir::read_from(&fd).map_err(|e| fail("open directory", display, e))?; + + for entry in dir { + let entry = entry.map_err(|e| fail("read directory", display, e))?; + let name = entry.file_name().to_string_lossy().into_owned(); + if name == "." || name == ".." { + continue; + } + + let st = match statat(&fd, &name, AtFlags::SYMLINK_NOFOLLOW) { + Ok(st) => st, + Err(rustix::io::Errno::NOENT) => continue, // vanished mid-walk + Err(e) => return Err(fail("stat", &join(display, &name), e)), + }; + + match FileType::from_raw_mode(st.st_mode) { + FileType::RegularFile => { + tree.insert( + join(rel, &name), + Info { size: st.st_size as i64, mtime: st.st_mtime as i64 }, + ); + } + FileType::Directory => { + let sub = match openat(&fd, &name, DIR_FLAGS, Mode::empty()) { + Ok(sub) => sub, + Err(rustix::io::Errno::NOENT) => continue, + Err(e) => return Err(fail("open directory", &join(display, &name), e)), + }; + walk(sub, &join(display, &name), &join(rel, &name), tree)?; + } + // Symlinks, sockets, pipes, devices: not part of the v0 snapshot. + _ => {} + } + } + Ok(()) +} + +fn scan(dir: &str) -> Result { + let fd = rustix::fs::open(dir, DIR_FLAGS, Mode::empty()) + .map_err(|e| fail("open directory", dir, e))?; + let mut tree = Tree::new(); + walk(fd, dir, "", &mut tree)?; + Ok(tree) +} + +/// Parse "path size mtime" lines. The path may contain spaces: size and +/// mtime are the last two space-separated fields. +fn parse_snapshot(text: &str) -> Result { + let mut tree = Tree::new(); + for (i, line) in text.split('\n').enumerate() { + if line.is_empty() { + continue; + } + let malformed = || anyhow!("malformed snapshot line {}", i + 1); + + let mut fields = line.rsplitn(3, ' '); + let mtime_s = fields.next().ok_or_else(malformed)?; + let size_s = fields.next().ok_or_else(malformed)?; + let path = fields.next().ok_or_else(malformed)?; + if path.is_empty() { + bail!(malformed()); + } + + let size: i64 = size_s.parse().map_err(|_| malformed())?; + let mtime: i64 = mtime_s.parse().map_err(|_| malformed())?; + tree.insert(path.to_string(), Info { size, mtime }); + } + Ok(tree) +} + +fn cmd_snapshot(dir: &str) -> Result<()> { + for (path, info) in scan(dir)? { + println!("{} {} {}", path, info.size, info.mtime); + } + Ok(()) +} + +fn cmd_diff(dir: &str, snapshot_path: &str) -> Result<()> { + let text = std::fs::read_to_string(snapshot_path).map_err(|e| { + anyhow!( + "cannot read snapshot '{snapshot_path}' (errno {})", + e.raw_os_error().unwrap_or(0) + ) + })?; + let before = parse_snapshot(&text)?; + let after = scan(dir)?; + + // Sorted union of both key sets. + let mut paths: Vec<&String> = before.keys().chain(after.keys()).collect(); + paths.sort(); + paths.dedup(); + + let (mut added, mut removed, mut modified) = (0, 0, 0); + for path in paths { + match (before.get(path), after.get(path)) { + (None, Some(_)) => { + println!("+ {path}"); + added += 1; + } + (Some(_), None) => { + println!("- {path}"); + removed += 1; + } + (Some(old), Some(now)) if old != now => { + println!("~ {path}"); + modified += 1; + } + _ => {} + } + } + println!("fwatch: {added} added, {removed} removed, {modified} modified"); + Ok(()) +} + +fn usage() -> ExitCode { + eprintln!("usage: fwatch snapshot DIR"); + eprintln!(" fwatch diff DIR SNAPSHOT_FILE"); + ExitCode::from(2) +} + +fn main() -> ExitCode { + let args: Vec = std::env::args().skip(1).collect(); + let result = match args.iter().map(String::as_str).collect::>()[..] { + ["snapshot", dir] => cmd_snapshot(dir), + ["diff", dir, snap] => cmd_diff(dir, snap), + _ => return usage(), + }; + match result { + Ok(()) => ExitCode::SUCCESS, + Err(e) => { + eprintln!("fwatch: error: {e}"); + ExitCode::from(1) + } + } +} diff --git a/examples/07-fds-raii-and-the-vfs/verify.lua b/examples/07-fds-raii-and-the-vfs/verify.lua new file mode 100644 index 0000000..7c14a43 --- /dev/null +++ b/examples/07-fds-raii-and-the-vfs/verify.lua @@ -0,0 +1,102 @@ +-- Verify fwatch v0: build a fixture tree with pinned mtimes, assert the exact +-- snapshot lines, mutate the tree (add/remove/append+retouch), and assert the +-- exact diff lines and summary. Also checks usage and error-path behavior. +-- Invoked from the example directory with LSP_LANG (cpp|go|rust) and EXAMPLE_DIR set. + +local script_dir = arg[0]:match("(.*/)") or "./" +local repo_root = os.getenv("REPO_ROOT") or (script_dir .. "../..") +package.path = package.path .. ";" .. repo_root .. "/scripts/lib/?.lua" +local checks = require("checks") + +local lang = os.getenv("LSP_LANG") +if lang ~= "cpp" and lang ~= "go" and lang ~= "rust" then + checks.skip("LSP_LANG must be cpp|go|rust (got " .. tostring(lang) .. ")") +end + +local demo = "./demo.sh " .. lang .. " run " + +-- Make sure the binary exists so `run` output is only the program's output. +local build = checks.run("./demo.sh " .. lang .. " build") +checks.expect_exit(build, 0, lang .. ": builds") + +-- Fixture: four regular files across nested directories, mtimes pinned with +-- touch -d @EPOCH so the snapshot bytes are fully deterministic. +local tmp = checks.run("mktemp -d").out:gsub("%s+$", "") +if tmp == "" then checks.skip("mktemp failed") end +local tree = tmp .. "/tree" +local snap = tmp .. "/snap.txt" + +local setup = checks.run(table.concat({ + "mkdir -p '" .. tree .. "/sub/deep'", + "printf hello > '" .. tree .. "/a.txt'", -- 5 bytes + "printf bye > '" .. tree .. "/gone.txt'", -- 3 bytes + "printf worldwide > '" .. tree .. "/sub/b.txt'", -- 9 bytes + "printf abc > '" .. tree .. "/sub/deep/c.bin'", -- 3 bytes + "touch -d @1000000000 '" .. tree .. "/a.txt'", + "touch -d @1000000300 '" .. tree .. "/gone.txt'", + "touch -d @1000000100 '" .. tree .. "/sub/b.txt'", + "touch -d @1000000200 '" .. tree .. "/sub/deep/c.bin'", +}, " && ")) +checks.expect_exit(setup, 0, lang .. ": fixture tree created") + +-- snapshot: exact sorted " " lines. +local r = checks.run(demo .. "snapshot '" .. tree .. "' > '" .. snap .. "'") +checks.expect_exit(r, 0, lang .. ": snapshot exits 0") +local written = checks.run("cat '" .. snap .. "'") +checks.expect_match(written.out, + "^a%.txt 5 1000000000\n" .. + "gone%.txt 3 1000000300\n" .. + "sub/b%.txt 9 1000000100\n" .. + "sub/deep/c%.bin 3 1000000200\n$", + lang .. ": snapshot is exactly the four sorted entry lines") + +-- diff against an unchanged tree: nothing but the zero summary. +r = checks.run(demo .. "diff '" .. tree .. "' '" .. snap .. "'") +checks.expect_exit(r, 0, lang .. ": no-change diff exits 0") +checks.expect_match(r.out, "^fwatch: 0 added, 0 removed, 0 modified\n$", + lang .. ": no-change diff prints only the zero summary") + +-- Mutate: create new.txt, delete gone.txt, grow a.txt and move its mtime. +local mutate = checks.run(table.concat({ + "printf newfile > '" .. tree .. "/new.txt'", + "rm '" .. tree .. "/gone.txt'", + "printf ' again' >> '" .. tree .. "/a.txt'", -- 5 -> 11 bytes + "touch -d @1000000500 '" .. tree .. "/a.txt'", +}, " && ")) +checks.expect_exit(mutate, 0, lang .. ": tree mutated") + +-- diff: exactly one ~, one -, one + (in sorted path order) plus the summary. +r = checks.run(demo .. "diff '" .. tree .. "' '" .. snap .. "'") +checks.expect_exit(r, 0, lang .. ": diff exits 0") +checks.expect_match(r.out, + "^~ a%.txt\n" .. + "%- gone%.txt\n" .. + "%+ new%.txt\n" .. + "fwatch: 1 added, 1 removed, 1 modified\n$", + lang .. ": diff shows ~/-/+ lines and the 1/1/1 summary, nothing else") + +-- The modified file keeps its identity in a fresh snapshot (size 11, new mtime). +r = checks.run(demo .. "snapshot '" .. tree .. "'") +checks.expect_exit(r, 0, lang .. ": re-snapshot exits 0") +checks.expect_match(r.out, "^a%.txt 11 1000000500\n", + lang .. ": re-snapshot sees the new size and mtime") + +-- Error paths: missing directory (errno 2), missing snapshot file, bad usage. +local tmp_pat = tmp:gsub("%-", "%%-"):gsub("%.", "%%.") +r = checks.run(demo .. "snapshot '" .. tmp .. "/nope'") +checks.expect_exit(r, 1, lang .. ": snapshot of missing dir exits 1") +checks.expect_match(r.out, + "^fwatch: error: cannot open directory '" .. tmp_pat .. "/nope' %(errno 2%)\n$", + lang .. ": missing dir reports errno 2 on stderr") + +r = checks.run(demo .. "diff '" .. tree .. "' '" .. tmp .. "/nosnap'") +checks.expect_exit(r, 1, lang .. ": diff with missing snapshot exits 1") +checks.expect_match(r.out, "cannot read snapshot '" .. tmp_pat .. "/nosnap' %(errno 2%)", + lang .. ": missing snapshot reports errno 2") + +r = checks.run(demo .. "snapshot") +checks.expect_exit(r, 2, lang .. ": missing args exit 2") +checks.expect_match(r.out, "usage: fwatch snapshot DIR", lang .. ": usage on stderr") + +checks.run("rm -rf '" .. tmp .. "'") +checks.finish() diff --git a/examples/08-page-cache-and-durability/README.md b/examples/08-page-cache-and-durability/README.md new file mode 100644 index 0000000..8c7857d --- /dev/null +++ b/examples/08-page-cache-and-durability/README.md @@ -0,0 +1,99 @@ +# 08-page-cache-and-durability — iobench + +Chapter 8's example: the same small write benchmark implemented three times — +C++23, Go, and Rust — to show what the page cache gives you (throughput) and +what it defers (durability). One binary, three sync strategies: + +``` +iobench --mode buffered|fsync-every|direct [--every N] [--size-mb M] FILE +``` + +It writes M MiB (default 64) to FILE in 64 KiB blocks: + +- `buffered` — plain `write(2)`s land in the page cache. The report line times + the writes alone; a second line `fsync_ms=` times the closing + `fdatasync(2)` that actually makes the data durable. The gap between the two + is the chapter's point. +- `fsync-every` — `fdatasync(2)` every N blocks (default 8, plus a trailing + sync if the block count is not a multiple of N), all inside the timed + region. This is what "durable as you go" costs. +- `direct` — `O_DIRECT` with 4096-aligned buffers, bypassing the page cache. + If the filesystem rejects `O_DIRECT` at `open(2)` with `EINVAL`, the program + prints `direct: unsupported on this filesystem` and exits 4. + +## Output and exit codes + +``` +mode= bytes= ms= MiB/s= # always, on stdout +fsync_ms= # buffered mode only +``` + +`bytes` is exact (M × 1048576), `ms` is integer milliseconds, `MiB/s` has one +decimal. Exit codes: `0` success, `2` usage error (usage line on stderr), `4` +`O_DIRECT` unsupported, `1` any other I/O failure. + +Representative run on the reference host (Fedora 44, btrfs, NVMe): + +``` +$ ./demo.sh go run --mode buffered bench.bin +mode=buffered bytes=67108864 ms=21 MiB/s=2975.5 +fsync_ms=17 +$ ./demo.sh go run --mode fsync-every bench.bin +mode=fsync-every bytes=67108864 ms=143 MiB/s=445.7 +$ ./demo.sh go run --mode direct bench.bin +mode=direct bytes=67108864 ms=81 MiB/s=786.2 +``` + +Buffered "throughput" is mostly the memcpy into the cache; the durable-as-you-go +and cache-bypass modes pay the real device cost inside the timed region. On +tmpfs (`/tmp`), `fdatasync` is nearly free and the three modes converge — +worth showing in class, and why `verify.lua` asserts shapes, not timings. + +## Implementations + +All three expose identical observable behavior (CLI, output, exit codes): + +- `cpp/` — C++23: RAII `Fd` wrapper around the descriptor, + `std::expected` for every fallible op, `std::println`, + `std::aligned_alloc` under a `unique_ptr` for the O_DIRECT buffer. +- `go/` — Go 1.26: raw syscalls via `golang.org/x/sys/unix` + (`Open`/`Write`/`Fdatasync`), wrapped errors with `%w`, + `errors.Is(err, unix.EINVAL)` for the O_DIRECT probe, manual 4096 alignment + of the block. +- `rust/` — Rust 2024: `rustix` (`open`/`write`/`fdatasync` over `OwnedFd`), + `anyhow` with context for error paths, `Errno::INVAL` match for the + O_DIRECT probe, aligned block carved from a stable `Vec`. + +Every write loop retries short writes and `EINTR` — a 64 KiB `write(2)` is not +guaranteed to complete in one call. + +## The demo contract + +Each language directory has a `demo.sh` with the book-wide interface: + +- `./demo.sh` — build then run +- `./demo.sh build` — build only +- `./demo.sh run [args]` — run the built binary +- With env `TARGET` set, `run` deploys the binary to that lab VM via + `scripts/lab/deploy-to-vm.sh` instead of running locally. + +The top-level `demo.sh` is the dispatcher: `./demo.sh cpp run --mode buffered +out.bin` execs `cpp/demo.sh run --mode buffered out.bin`. + +## Verification + +`verify.lua` (run per language with `LSP_LANG` set) asserts behavior, not just +exit status: + +- no args → usage line, exit 2; unknown mode → exit 2 and no file created +- buffered 8 MiB → report line with `bytes=8388608`, `fsync_ms` line directly + after it, and the file is exactly 8388608 bytes on disk (`stat`) +- fsync-every 4 MiB (`--every 4`) → single report line with `bytes=4194304`, + no `fsync_ms` line, file exactly 4194304 bytes +- direct 4 MiB → either the report line and correct size, or the documented + `EINVAL` message with exit 4 — both are valid outcomes per filesystem + +Timing fields are asserted for shape only (`ms=%d+`, `MiB/s=%d+%.%d`); +asserting magnitudes would flake. An `strace -c` cross-check of +`--mode fsync-every --every 4 --size-mb 4` shows exactly 64 data writes and +16 `fdatasync` calls — the syscall pattern the chapter describes. diff --git a/examples/08-page-cache-and-durability/cpp/.clang-format b/examples/08-page-cache-and-durability/cpp/.clang-format new file mode 100644 index 0000000..2c8640f --- /dev/null +++ b/examples/08-page-cache-and-durability/cpp/.clang-format @@ -0,0 +1,8 @@ +--- +BasedOnStyle: LLVM +Language: Cpp +Standard: c++20 +IndentWidth: 4 +ColumnLimit: 100 +PointerAlignment: Left +AllowShortFunctionsOnASingleLine: Inline diff --git a/examples/08-page-cache-and-durability/cpp/.clang-tidy b/examples/08-page-cache-and-durability/cpp/.clang-tidy new file mode 100644 index 0000000..2f275ba --- /dev/null +++ b/examples/08-page-cache-and-durability/cpp/.clang-tidy @@ -0,0 +1,11 @@ +--- +Checks: > + bugprone-*, + modernize-*, + performance-*, + readability-*, + -modernize-use-trailing-return-type, + -readability-identifier-length +WarningsAsErrors: '' +HeaderFilterRegex: '.*' +FormatStyle: file diff --git a/examples/08-page-cache-and-durability/cpp/CMakeLists.txt b/examples/08-page-cache-and-durability/cpp/CMakeLists.txt new file mode 100644 index 0000000..a0599b3 --- /dev/null +++ b/examples/08-page-cache-and-durability/cpp/CMakeLists.txt @@ -0,0 +1,8 @@ +cmake_minimum_required(VERSION 3.25) +project(iobench CXX) + +set(CMAKE_CXX_STANDARD 23) +set(CMAKE_CXX_STANDARD_REQUIRED ON) +set(CMAKE_CXX_EXTENSIONS OFF) + +add_executable(app src/main.cpp) diff --git a/examples/08-page-cache-and-durability/cpp/CMakePresets.json b/examples/08-page-cache-and-durability/cpp/CMakePresets.json new file mode 100644 index 0000000..e31bd2f --- /dev/null +++ b/examples/08-page-cache-and-durability/cpp/CMakePresets.json @@ -0,0 +1,62 @@ +{ + "version": 6, + "cmakeMinimumRequired": { "major": 3, "minor": 25, "patch": 0 }, + "configurePresets": [ + { + "name": "_base", + "hidden": true, + "generator": "Ninja", + "binaryDir": "${sourceDir}/build/${presetName}", + "cacheVariables": { + "CMAKE_EXPORT_COMPILE_COMMANDS": "ON" + } + }, + { + "name": "release", + "inherits": "_base", + "displayName": "Release (-O2 with debug info)", + "cacheVariables": { "CMAKE_BUILD_TYPE": "RelWithDebInfo" } + }, + { + "name": "debug", + "inherits": "_base", + "displayName": "Debug", + "cacheVariables": { "CMAKE_BUILD_TYPE": "Debug" } + }, + { + "name": "asan", + "inherits": "_base", + "displayName": "ASan + UBSan instrumented build", + "cacheVariables": { + "CMAKE_BUILD_TYPE": "Debug", + "CMAKE_CXX_FLAGS": "-O1 -g -fno-omit-frame-pointer -fsanitize=address,undefined", + "CMAKE_EXE_LINKER_FLAGS": "-fsanitize=address,undefined" + } + }, + { + "name": "release-clang", + "inherits": "release", + "displayName": "Release (clang)", + "cacheVariables": { + "CMAKE_C_COMPILER": "clang", + "CMAKE_CXX_COMPILER": "clang++" + } + }, + { + "name": "debug-clang", + "inherits": "debug", + "displayName": "Debug (clang)", + "cacheVariables": { + "CMAKE_C_COMPILER": "clang", + "CMAKE_CXX_COMPILER": "clang++" + } + } + ], + "buildPresets": [ + { "name": "release", "configurePreset": "release" }, + { "name": "debug", "configurePreset": "debug" }, + { "name": "asan", "configurePreset": "asan" }, + { "name": "release-clang", "configurePreset": "release-clang" }, + { "name": "debug-clang", "configurePreset": "debug-clang" } + ] +} diff --git a/examples/08-page-cache-and-durability/cpp/demo.sh b/examples/08-page-cache-and-durability/cpp/demo.sh new file mode 100755 index 0000000..6af6509 --- /dev/null +++ b/examples/08-page-cache-and-durability/cpp/demo.sh @@ -0,0 +1,40 @@ +#!/usr/bin/env bash +set -euo pipefail + +cd "$(dirname "${BASH_SOURCE[0]}")" +REPO_ROOT="$(cd ../../.. && pwd)" +BIN=build/release/app + +build() { + cmake --preset release + cmake --build --preset release +} + +run() { + if [[ ! -x "$BIN" ]]; then + build + fi + if [[ -n "${TARGET:-}" ]]; then + "$REPO_ROOT/scripts/lab/deploy-to-vm.sh" "$TARGET" "$BIN" -- "$@" + else + "./$BIN" "$@" + fi +} + +case "${1:-}" in + build) + build + ;; + run) + shift + run "$@" + ;; + "") + build + run + ;; + *) + echo "usage: $0 [build|run [args]]" >&2 + exit 2 + ;; +esac diff --git a/examples/08-page-cache-and-durability/cpp/src/main.cpp b/examples/08-page-cache-and-durability/cpp/src/main.cpp new file mode 100644 index 0000000..38d16f9 --- /dev/null +++ b/examples/08-page-cache-and-durability/cpp/src/main.cpp @@ -0,0 +1,269 @@ +// iobench: page cache vs durability — buffered writes, periodic fdatasync(2), +// and O_DIRECT, with identical CLI and output across the three languages. +// +// iobench --mode buffered|fsync-every|direct [--every N] [--size-mb M] FILE +// +// Writes M MiB (default 64) in 64 KiB blocks. +// buffered plain write(2)s; reports write time, then a second line +// "fsync_ms=" for the closing fdatasync(2) +// fsync-every fdatasync(2) every N blocks (default 8), timed end to end +// direct O_DIRECT with 4096-aligned buffers; if open(2) fails with +// EINVAL, prints "direct: unsupported on this filesystem" +// and exits 4 + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include + +namespace { + +constexpr std::size_t kBlockSize = 64 * 1024; +constexpr std::size_t kAlignment = 4096; +constexpr std::size_t kMiB = 1024 * 1024; + +enum class Mode { buffered, fsync_every, direct }; + +[[nodiscard]] constexpr std::string_view mode_name(Mode m) { + switch (m) { + case Mode::buffered: return "buffered"; + case Mode::fsync_every: return "fsync-every"; + case Mode::direct: return "direct"; + } + return "?"; +} + +struct Options { + Mode mode = Mode::buffered; + std::size_t every = 8; + std::size_t size_mb = 64; + std::string file; +}; + +// Owning RAII wrapper around a file descriptor. +class Fd { +public: + explicit Fd(int fd) noexcept : fd_{fd} {} + Fd(const Fd&) = delete; + Fd& operator=(const Fd&) = delete; + Fd(Fd&& other) noexcept : fd_{std::exchange(other.fd_, -1)} {} + Fd& operator=(Fd&& other) noexcept { + if (this != &other) { + reset(); + fd_ = std::exchange(other.fd_, -1); + } + return *this; + } + ~Fd() { reset(); } + + [[nodiscard]] int get() const noexcept { return fd_; } + +private: + void reset() noexcept { + if (fd_ >= 0) { + ::close(fd_); + fd_ = -1; + } + } + int fd_ = -1; +}; + +[[nodiscard]] std::error_code last_error() { + return {errno, std::system_category()}; +} + +[[nodiscard]] std::expected open_output(const std::string& path, bool direct) { + int flags = O_WRONLY | O_CREAT | O_TRUNC; + if (direct) { + flags |= O_DIRECT; + } + const int fd = ::open(path.c_str(), flags, 0644); + if (fd < 0) { + return std::unexpected(last_error()); + } + return Fd{fd}; +} + +[[nodiscard]] std::expected write_all(const Fd& fd, + std::span data) { + while (!data.empty()) { + const ssize_t n = ::write(fd.get(), data.data(), data.size()); + if (n < 0) { + if (errno == EINTR) { + continue; + } + return std::unexpected(last_error()); + } + data = data.subspan(static_cast(n)); + } + return {}; +} + +[[nodiscard]] std::expected datasync(const Fd& fd) { + if (::fdatasync(fd.get()) != 0) { + return std::unexpected(last_error()); + } + return {}; +} + +struct FreeDeleter { + void operator()(void* p) const noexcept { std::free(p); } +}; +using AlignedBlock = std::unique_ptr; + +// One 64 KiB block, 4096-aligned (required by O_DIRECT, harmless otherwise). +[[nodiscard]] AlignedBlock make_block() { + auto* raw = static_cast(std::aligned_alloc(kAlignment, kBlockSize)); + AlignedBlock block{raw}; + if (block) { + for (std::size_t i = 0; i < kBlockSize; ++i) { + block[i] = static_cast(i & 0xFF); + } + } + return block; +} + +[[nodiscard]] std::optional parse_positive(std::string_view text) { + std::size_t value = 0; + const auto [ptr, ec] = std::from_chars(text.data(), text.data() + text.size(), value); + if (ec != std::errc{} || ptr != text.data() + text.size() || value == 0) { + return std::nullopt; + } + return value; +} + +[[nodiscard]] std::optional parse_args(std::span args) { + Options opt; + bool have_mode = false; + for (std::size_t i = 0; i < args.size(); ++i) { + const std::string_view a = args[i]; + if (a == "--mode" || a == "--every" || a == "--size-mb") { + if (i + 1 >= args.size()) { + return std::nullopt; + } + const std::string_view v = args[++i]; + if (a == "--mode") { + if (v == "buffered") { + opt.mode = Mode::buffered; + } else if (v == "fsync-every") { + opt.mode = Mode::fsync_every; + } else if (v == "direct") { + opt.mode = Mode::direct; + } else { + return std::nullopt; + } + have_mode = true; + } else { + const auto n = parse_positive(v); + if (!n) { + return std::nullopt; + } + (a == "--every" ? opt.every : opt.size_mb) = *n; + } + } else if (a.starts_with("--")) { + return std::nullopt; + } else if (opt.file.empty()) { + opt.file = a; + } else { + return std::nullopt; + } + } + if (!have_mode || opt.file.empty()) { + return std::nullopt; + } + return opt; +} + +// Elapsed nanoseconds, clamped to >= 1 so the throughput division is defined. +[[nodiscard]] std::int64_t elapsed_ns(std::chrono::steady_clock::time_point from, + std::chrono::steady_clock::time_point to) { + const auto ns = std::chrono::duration_cast(to - from).count(); + return ns > 0 ? ns : 1; +} + +[[nodiscard]] int run(const Options& opt) { + auto fd = open_output(opt.file, opt.mode == Mode::direct); + if (!fd) { + if (opt.mode == Mode::direct && fd.error() == std::errc::invalid_argument) { + std::println(stderr, "direct: unsupported on this filesystem"); + return 4; + } + std::println(stderr, "error: open {}: {}", opt.file, fd.error().message()); + return 1; + } + + const auto block = make_block(); + if (!block) { + std::println(stderr, "error: aligned_alloc failed"); + return 1; + } + const std::span data{block.get(), kBlockSize}; + + const std::size_t nblocks = opt.size_mb * (kMiB / kBlockSize); + const auto bytes = static_cast(opt.size_mb) * kMiB; + + const auto t0 = std::chrono::steady_clock::now(); + for (std::size_t i = 0; i < nblocks; ++i) { + if (const auto w = write_all(*fd, data); !w) { + std::println(stderr, "error: write {}: {}", opt.file, w.error().message()); + return 1; + } + if (opt.mode == Mode::fsync_every && (i + 1) % opt.every == 0) { + if (const auto s = datasync(*fd); !s) { + std::println(stderr, "error: fdatasync {}: {}", opt.file, s.error().message()); + return 1; + } + } + } + if (opt.mode == Mode::fsync_every && nblocks % opt.every != 0) { + if (const auto s = datasync(*fd); !s) { + std::println(stderr, "error: fdatasync {}: {}", opt.file, s.error().message()); + return 1; + } + } + const auto t1 = std::chrono::steady_clock::now(); + + const std::int64_t ns = elapsed_ns(t0, t1); + const double mib_per_s = + (static_cast(bytes) / static_cast(kMiB)) / (static_cast(ns) / 1e9); + std::println("mode={} bytes={} ms={} MiB/s={:.1f}", mode_name(opt.mode), bytes, + ns / 1'000'000, mib_per_s); + + if (opt.mode == Mode::buffered) { + const auto t2 = std::chrono::steady_clock::now(); + if (const auto s = datasync(*fd); !s) { + std::println(stderr, "error: fdatasync {}: {}", opt.file, s.error().message()); + return 1; + } + const auto t3 = std::chrono::steady_clock::now(); + std::println("fsync_ms={}", elapsed_ns(t2, t3) / 1'000'000); + } + return 0; +} + +} // namespace + +int main(int argc, char** argv) { + const auto opt = parse_args(std::span{argv + 1, static_cast(argc - 1)}); + if (!opt) { + std::println(stderr, + "usage: iobench --mode buffered|fsync-every|direct [--every N] [--size-mb M] FILE"); + return 2; + } + return run(*opt); +} diff --git a/examples/08-page-cache-and-durability/demo.sh b/examples/08-page-cache-and-durability/demo.sh new file mode 100755 index 0000000..afbbe4c --- /dev/null +++ b/examples/08-page-cache-and-durability/demo.sh @@ -0,0 +1,23 @@ +#!/usr/bin/env bash +set -euo pipefail +cd "$(dirname "$0")" + +langs=(cpp go rust) + +case "${1:-all}" in + cpp|go|rust) + lang="$1" + shift + exec "./$lang/demo.sh" "$@" + ;; + all) + for lang in "${langs[@]}"; do "./$lang/demo.sh"; done + ;; + build) + for lang in "${langs[@]}"; do "./$lang/demo.sh" build; done + ;; + *) + echo "usage: $0 [cpp|go|rust|all|build] [args...]" >&2 + exit 2 + ;; +esac diff --git a/examples/08-page-cache-and-durability/go/demo.sh b/examples/08-page-cache-and-durability/go/demo.sh new file mode 100755 index 0000000..28606df --- /dev/null +++ b/examples/08-page-cache-and-durability/go/demo.sh @@ -0,0 +1,23 @@ +#!/usr/bin/env bash +set -euo pipefail +cd "$(dirname "$0")" +REPO_ROOT="$(cd ../../.. && pwd)" +BIN=bin/app + +build() { + go build -o "$BIN" . +} + +run() { + if [[ -n "${TARGET:-}" ]]; then + exec "$REPO_ROOT/scripts/lab/deploy-to-vm.sh" "$TARGET" "$BIN" -- "$@" + fi + exec "./$BIN" "$@" +} + +case "${1:-}" in + build) build ;; + run) shift; run "$@" ;; + "") build; run ;; + *) echo "usage: $0 [build|run [args...]]" >&2; exit 2 ;; +esac diff --git a/examples/08-page-cache-and-durability/go/go.mod b/examples/08-page-cache-and-durability/go/go.mod new file mode 100644 index 0000000..5dbb0b5 --- /dev/null +++ b/examples/08-page-cache-and-durability/go/go.mod @@ -0,0 +1,7 @@ +module github.com/patterncatalyst/linux-systems-programming/examples/08-page-cache-and-durability/go + +go 1.26 + +toolchain go1.26.5 + +require golang.org/x/sys v0.47.0 diff --git a/examples/08-page-cache-and-durability/go/go.sum b/examples/08-page-cache-and-durability/go/go.sum new file mode 100644 index 0000000..37ee2d4 --- /dev/null +++ b/examples/08-page-cache-and-durability/go/go.sum @@ -0,0 +1,2 @@ +golang.org/x/sys v0.47.0 h1:o7XGOvZQCADBQQ4Y7VNq2dRWQR7JmOUW8Kxx4ZsNgWs= +golang.org/x/sys v0.47.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw= diff --git a/examples/08-page-cache-and-durability/go/main.go b/examples/08-page-cache-and-durability/go/main.go new file mode 100644 index 0000000..6697b2d --- /dev/null +++ b/examples/08-page-cache-and-durability/go/main.go @@ -0,0 +1,202 @@ +// iobench: page cache vs durability — buffered writes, periodic fdatasync(2), +// and O_DIRECT, with identical CLI and output across the three languages. +// +// iobench --mode buffered|fsync-every|direct [--every N] [--size-mb M] FILE +// +// Writes M MiB (default 64) in 64 KiB blocks. +// +// buffered plain write(2)s; reports write time, then a second line +// "fsync_ms=" for the closing fdatasync(2) +// fsync-every fdatasync(2) every N blocks (default 8), timed end to end +// direct O_DIRECT with 4096-aligned buffers; if open(2) fails with +// EINVAL, prints "direct: unsupported on this filesystem" +// and exits 4 +package main + +import ( + "errors" + "fmt" + "os" + "strconv" + "strings" + "time" + "unsafe" + + "golang.org/x/sys/unix" +) + +const ( + blockSize = 64 * 1024 + alignment = 4096 + mib = 1024 * 1024 + usageLine = "usage: iobench --mode buffered|fsync-every|direct [--every N] [--size-mb M] FILE" +) + +type options struct { + mode string + every uint64 + sizeMB uint64 + file string +} + +var errUsage = errors.New("usage") + +func parsePositive(s string) (uint64, error) { + n, err := strconv.ParseUint(s, 10, 64) + if err != nil || n == 0 { + return 0, errUsage + } + return n, nil +} + +func parseArgs(args []string) (options, error) { + opt := options{every: 8, sizeMB: 64} + haveMode := false + for i := 0; i < len(args); i++ { + a := args[i] + switch a { + case "--mode", "--every", "--size-mb": + if i+1 >= len(args) { + return opt, errUsage + } + i++ + v := args[i] + switch a { + case "--mode": + if v != "buffered" && v != "fsync-every" && v != "direct" { + return opt, errUsage + } + opt.mode = v + haveMode = true + case "--every": + n, err := parsePositive(v) + if err != nil { + return opt, err + } + opt.every = n + case "--size-mb": + n, err := parsePositive(v) + if err != nil { + return opt, err + } + opt.sizeMB = n + } + default: + if strings.HasPrefix(a, "--") || opt.file != "" { + return opt, errUsage + } + opt.file = a + } + } + if !haveMode || opt.file == "" { + return opt, errUsage + } + return opt, nil +} + +// alignedBlock returns one 64 KiB block whose base address is 4096-aligned +// (required by O_DIRECT, harmless otherwise). The Go heap does not move +// allocations, so the alignment computed here stays valid. +func alignedBlock() []byte { + raw := make([]byte, blockSize+alignment) + addr := uintptr(unsafe.Pointer(&raw[0])) + off := int((alignment - addr%alignment) % alignment) + block := raw[off : off+blockSize] + for i := range block { + block[i] = byte(i) + } + return block +} + +// writeAll retries write(2) across short writes and EINTR. +func writeAll(fd int, data []byte) error { + for len(data) > 0 { + n, err := unix.Write(fd, data) + if err != nil { + if errors.Is(err, unix.EINTR) { + continue + } + return fmt.Errorf("write: %w", err) + } + data = data[n:] + } + return nil +} + +func datasync(fd int) error { + if err := unix.Fdatasync(fd); err != nil { + return fmt.Errorf("fdatasync: %w", err) + } + return nil +} + +// clampNS keeps the throughput division defined on a sub-nanosecond clock read. +func clampNS(d time.Duration) int64 { + if ns := d.Nanoseconds(); ns > 0 { + return ns + } + return 1 +} + +func run(opt options) (int, error) { + flags := unix.O_WRONLY | unix.O_CREAT | unix.O_TRUNC + if opt.mode == "direct" { + flags |= unix.O_DIRECT + } + fd, err := unix.Open(opt.file, flags, 0o644) + if err != nil { + if opt.mode == "direct" && errors.Is(err, unix.EINVAL) { + fmt.Fprintln(os.Stderr, "direct: unsupported on this filesystem") + return 4, nil + } + return 1, fmt.Errorf("open %s: %w", opt.file, err) + } + defer unix.Close(fd) + + block := alignedBlock() + nblocks := opt.sizeMB * (mib / blockSize) + bytes := opt.sizeMB * mib + + t0 := time.Now() + for i := uint64(0); i < nblocks; i++ { + if err := writeAll(fd, block); err != nil { + return 1, fmt.Errorf("%s: %w", opt.file, err) + } + if opt.mode == "fsync-every" && (i+1)%opt.every == 0 { + if err := datasync(fd); err != nil { + return 1, fmt.Errorf("%s: %w", opt.file, err) + } + } + } + if opt.mode == "fsync-every" && nblocks%opt.every != 0 { + if err := datasync(fd); err != nil { + return 1, fmt.Errorf("%s: %w", opt.file, err) + } + } + ns := clampNS(time.Since(t0)) + + mibPerS := (float64(bytes) / float64(mib)) / (float64(ns) / 1e9) + fmt.Printf("mode=%s bytes=%d ms=%d MiB/s=%.1f\n", opt.mode, bytes, ns/1e6, mibPerS) + + if opt.mode == "buffered" { + t2 := time.Now() + if err := datasync(fd); err != nil { + return 1, fmt.Errorf("%s: %w", opt.file, err) + } + fmt.Printf("fsync_ms=%d\n", clampNS(time.Since(t2))/1e6) + } + return 0, nil +} + +func main() { + opt, err := parseArgs(os.Args[1:]) + if err != nil { + fmt.Fprintln(os.Stderr, usageLine) + os.Exit(2) + } + code, err := run(opt) + if err != nil { + fmt.Fprintln(os.Stderr, "error:", err) + } + os.Exit(code) +} diff --git a/examples/08-page-cache-and-durability/rust/Cargo.lock b/examples/08-page-cache-and-durability/rust/Cargo.lock new file mode 100644 index 0000000..301b73a --- /dev/null +++ b/examples/08-page-cache-and-durability/rust/Cargo.lock @@ -0,0 +1,73 @@ +# This file is automatically @generated by Cargo. +# It is not intended for manual editing. +version = 4 + +[[package]] +name = "anyhow" +version = "1.0.104" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "330a5ed07fa54e4702c9d6c4174f74427fc0ef6e214bbd677ae50a5099946470" + +[[package]] +name = "bitflags" +version = "2.13.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b588b76d00fde79687d7646a9b5bdf3cc0f655e0bbd080335a95d7e96f3587da" + +[[package]] +name = "errno" +version = "0.3.14" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "39cab71617ae0d63f51a36d69f866391735b51691dbda63cf6f96d042b63efeb" +dependencies = [ + "libc", + "windows-sys", +] + +[[package]] +name = "iobench" +version = "0.1.0" +dependencies = [ + "anyhow", + "rustix", +] + +[[package]] +name = "libc" +version = "0.2.186" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "68ab91017fe16c622486840e4c83c9a37afeff978bd239b5293d61ece587de66" + +[[package]] +name = "linux-raw-sys" +version = "0.12.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "32a66949e030da00e8c7d4434b251670a91556f4144941d37452769c25d58a53" + +[[package]] +name = "rustix" +version = "1.1.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b6fe4565b9518b83ef4f91bb47ce29620ca828bd32cb7e408f0062e9930ba190" +dependencies = [ + "bitflags", + "errno", + "libc", + "linux-raw-sys", + "windows-sys", +] + +[[package]] +name = "windows-link" +version = "0.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5" + +[[package]] +name = "windows-sys" +version = "0.61.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ae137229bcbd6cdf0f7b80a31df61766145077ddf49416a728b02cb3921ff3fc" +dependencies = [ + "windows-link", +] diff --git a/examples/08-page-cache-and-durability/rust/Cargo.toml b/examples/08-page-cache-and-durability/rust/Cargo.toml new file mode 100644 index 0000000..81b3e21 --- /dev/null +++ b/examples/08-page-cache-and-durability/rust/Cargo.toml @@ -0,0 +1,12 @@ +[package] +name = "iobench" +version = "0.1.0" +edition = "2024" + +[[bin]] +name = "app" +path = "src/main.rs" + +[dependencies] +anyhow = "1.0.100" +rustix = { version = "1", features = ["fs"] } diff --git a/examples/08-page-cache-and-durability/rust/demo.sh b/examples/08-page-cache-and-durability/rust/demo.sh new file mode 100755 index 0000000..220feea --- /dev/null +++ b/examples/08-page-cache-and-durability/rust/demo.sh @@ -0,0 +1,26 @@ +#!/usr/bin/env bash +set -euo pipefail +cd "$(dirname "${BASH_SOURCE[0]}")" + +BIN=target/release/app + +build() { + cargo build --release +} + +run() { + if [[ -n "${TARGET:-}" ]]; then + local repo_root + repo_root=$(cd ../../.. && pwd) + "$repo_root/scripts/lab/deploy-to-vm.sh" "$TARGET" "$BIN" -- "$@" + else + "./$BIN" "$@" + fi +} + +case "${1:-}" in + build) build ;; + run) shift; run "$@" ;; + "") build; run ;; + *) echo "usage: $0 [build|run [args...]]" >&2; exit 2 ;; +esac diff --git a/examples/08-page-cache-and-durability/rust/rust-toolchain.toml b/examples/08-page-cache-and-durability/rust/rust-toolchain.toml new file mode 100644 index 0000000..4099f1a --- /dev/null +++ b/examples/08-page-cache-and-durability/rust/rust-toolchain.toml @@ -0,0 +1,3 @@ +[toolchain] +channel = "1.97.1" +components = ["rustfmt", "clippy", "rust-src"] diff --git a/examples/08-page-cache-and-durability/rust/src/main.rs b/examples/08-page-cache-and-durability/rust/src/main.rs new file mode 100644 index 0000000..5f10bb3 --- /dev/null +++ b/examples/08-page-cache-and-durability/rust/src/main.rs @@ -0,0 +1,196 @@ +// iobench: page cache vs durability — buffered writes, periodic fdatasync(2), +// and O_DIRECT, with identical CLI and output across the three languages. +// +// iobench --mode buffered|fsync-every|direct [--every N] [--size-mb M] FILE +// +// Writes M MiB (default 64) in 64 KiB blocks. +// buffered plain write(2)s; reports write time, then a second line +// "fsync_ms=" for the closing fdatasync(2) +// fsync-every fdatasync(2) every N blocks (default 8), timed end to end +// direct O_DIRECT with 4096-aligned buffers; if open(2) fails with +// EINVAL, prints "direct: unsupported on this filesystem" +// and exits 4 + +use std::os::fd::OwnedFd; +use std::process::ExitCode; +use std::time::Instant; + +use anyhow::{Context, Result}; +use rustix::fs::{Mode, OFlags, fdatasync, open}; +use rustix::io::Errno; + +const BLOCK_SIZE: usize = 64 * 1024; +const ALIGNMENT: usize = 4096; +const MIB: u64 = 1024 * 1024; +const USAGE: &str = + "usage: iobench --mode buffered|fsync-every|direct [--every N] [--size-mb M] FILE"; + +#[derive(Clone, Copy, PartialEq, Eq)] +enum SyncMode { + Buffered, + FsyncEvery, + Direct, +} + +impl SyncMode { + fn name(self) -> &'static str { + match self { + SyncMode::Buffered => "buffered", + SyncMode::FsyncEvery => "fsync-every", + SyncMode::Direct => "direct", + } + } +} + +struct Options { + mode: SyncMode, + every: u64, + size_mb: u64, + file: String, +} + +fn parse_positive(text: &str) -> Option { + match text.parse::() { + Ok(n) if n > 0 => Some(n), + _ => None, + } +} + +fn parse_args(args: &[String]) -> Option { + let mut mode = None; + let mut every = 8u64; + let mut size_mb = 64u64; + let mut file: Option = None; + + let mut it = args.iter(); + while let Some(a) = it.next() { + match a.as_str() { + "--mode" => { + mode = Some(match it.next()?.as_str() { + "buffered" => SyncMode::Buffered, + "fsync-every" => SyncMode::FsyncEvery, + "direct" => SyncMode::Direct, + _ => return None, + }); + } + "--every" => every = parse_positive(it.next()?)?, + "--size-mb" => size_mb = parse_positive(it.next()?)?, + other if other.starts_with("--") => return None, + other => { + if file.is_some() { + return None; + } + file = Some(other.to_string()); + } + } + } + Some(Options { mode: mode?, every, size_mb, file: file? }) +} + +/// One 64 KiB block whose base address is 4096-aligned (required by O_DIRECT, +/// harmless otherwise). The backing Vec never reallocates, so the aligned +/// range stays valid. +struct AlignedBlock { + backing: Vec, + offset: usize, +} + +impl AlignedBlock { + fn new() -> Self { + let mut backing = vec![0u8; BLOCK_SIZE + ALIGNMENT]; + let addr = backing.as_ptr() as usize; + let offset = (ALIGNMENT - addr % ALIGNMENT) % ALIGNMENT; + for (i, b) in backing[offset..offset + BLOCK_SIZE].iter_mut().enumerate() { + *b = i as u8; + } + Self { backing, offset } + } + + fn as_slice(&self) -> &[u8] { + &self.backing[self.offset..self.offset + BLOCK_SIZE] + } +} + +fn open_output(path: &str, direct: bool) -> rustix::io::Result { + let mut flags = OFlags::WRONLY | OFlags::CREATE | OFlags::TRUNC; + if direct { + flags |= OFlags::DIRECT; + } + open(path, flags, Mode::from_raw_mode(0o644)) +} + +/// write(2) the whole block, retrying across short writes and EINTR. +fn write_all(fd: &OwnedFd, mut data: &[u8]) -> rustix::io::Result<()> { + while !data.is_empty() { + match rustix::io::write(fd, data) { + Ok(n) => data = &data[n..], + Err(Errno::INTR) => continue, + Err(e) => return Err(e), + } + } + Ok(()) +} + +/// Elapsed nanoseconds since `from`, clamped to >= 1 so the throughput +/// division is defined. +fn elapsed_ns(from: Instant) -> u128 { + from.elapsed().as_nanos().max(1) +} + +fn run(opt: &Options) -> Result { + let fd = match open_output(&opt.file, opt.mode == SyncMode::Direct) { + Ok(fd) => fd, + Err(Errno::INVAL) if opt.mode == SyncMode::Direct => { + eprintln!("direct: unsupported on this filesystem"); + return Ok(ExitCode::from(4)); + } + Err(e) => return Err(e).context(format!("open {}", opt.file)), + }; + + let block = AlignedBlock::new(); + let nblocks = opt.size_mb * (MIB / BLOCK_SIZE as u64); + let bytes = opt.size_mb * MIB; + + let t0 = Instant::now(); + for i in 0..nblocks { + write_all(&fd, block.as_slice()).with_context(|| format!("write {}", opt.file))?; + if opt.mode == SyncMode::FsyncEvery && (i + 1) % opt.every == 0 { + fdatasync(&fd).with_context(|| format!("fdatasync {}", opt.file))?; + } + } + if opt.mode == SyncMode::FsyncEvery && nblocks % opt.every != 0 { + fdatasync(&fd).with_context(|| format!("fdatasync {}", opt.file))?; + } + let ns = elapsed_ns(t0); + + let mib_per_s = (bytes as f64 / MIB as f64) / (ns as f64 / 1e9); + println!( + "mode={} bytes={} ms={} MiB/s={:.1}", + opt.mode.name(), + bytes, + ns / 1_000_000, + mib_per_s + ); + + if opt.mode == SyncMode::Buffered { + let t2 = Instant::now(); + fdatasync(&fd).with_context(|| format!("fdatasync {}", opt.file))?; + println!("fsync_ms={}", elapsed_ns(t2) / 1_000_000); + } + Ok(ExitCode::SUCCESS) +} + +fn main() -> ExitCode { + let args: Vec = std::env::args().skip(1).collect(); + let Some(opt) = parse_args(&args) else { + eprintln!("{USAGE}"); + return ExitCode::from(2); + }; + match run(&opt) { + Ok(code) => code, + Err(err) => { + eprintln!("error: {err:#}"); + ExitCode::from(1) + } + } +} diff --git a/examples/08-page-cache-and-durability/verify.lua b/examples/08-page-cache-and-durability/verify.lua new file mode 100644 index 0000000..b0389dd --- /dev/null +++ b/examples/08-page-cache-and-durability/verify.lua @@ -0,0 +1,73 @@ +-- Verify 08-page-cache-and-durability: run iobench in buffered, fsync-every, +-- and direct modes and assert the observable behavior — report line shapes, +-- exact byte counts, the on-disk file size (stat), and exit codes. Timing +-- values are asserted for shape only (numbers flake). +-- Invoked from the example directory with LSP_LANG (cpp|go|rust) and EXAMPLE_DIR set. + +local script_dir = arg[0]:match("(.*/)") or "./" +local repo_root = os.getenv("REPO_ROOT") or (script_dir .. "../..") +package.path = package.path .. ";" .. repo_root .. "/scripts/lib/?.lua" +local checks = require("checks") + +local lang = os.getenv("LSP_LANG") +if lang ~= "cpp" and lang ~= "go" and lang ~= "rust" then + checks.skip("LSP_LANG must be cpp|go|rust (got " .. tostring(lang) .. ")") +end + +local demo = "./demo.sh " .. lang .. " run" +-- Absolute path: each language's demo.sh runs the binary from its own +-- subdirectory, so a relative FILE would land there instead of here. +local example_dir = os.getenv("EXAMPLE_DIR") or "." +local out_file = example_dir .. "/verify-" .. lang .. ".bin" +os.remove(out_file) + +local function expect_file_size(path, size, label) + local r = checks.run("stat -c %s " .. path) + checks.expect_match(r.out, "^" .. size .. "%s*$", label) +end + +-- 1. No arguments: usage on stderr, exit 2. +local r = checks.run(demo) +checks.expect_exit(r, 2, lang .. ": no args exits 2") +checks.expect_match(r.out, "usage: iobench %-%-mode buffered|fsync%-every|direct", + lang .. ": no args prints usage") + +-- 2. Unknown mode: usage, exit 2, and no file gets created. +r = checks.run(demo .. " --mode sideways " .. out_file) +checks.expect_exit(r, 2, lang .. ": bad mode exits 2") +local stat = checks.run("stat -c %s " .. out_file) +checks.expect_match(tostring(stat.exit ~= 0), "true", lang .. ": bad mode creates no file") + +-- 3. Buffered: 8 MiB on disk, report line + separate fsync_ms line, in order. +r = checks.run(demo .. " --mode buffered --size-mb 8 " .. out_file) +checks.expect_exit(r, 0, lang .. ": buffered exits 0") +checks.expect_match(r.out, "mode=buffered bytes=8388608 ms=%d+ MiB/s=%d+%.%d", + lang .. ": buffered report line") +checks.expect_match(r.out, "MiB/s=%d+%.%d\nfsync_ms=%d+", + lang .. ": fsync_ms line follows the report line") +expect_file_size(out_file, 8388608, lang .. ": buffered file is 8 MiB on disk") + +-- 4. fsync-every: 4 MiB, sync every 4 blocks, single report line. +r = checks.run(demo .. " --mode fsync-every --every 4 --size-mb 4 " .. out_file) +checks.expect_exit(r, 0, lang .. ": fsync-every exits 0") +checks.expect_match(r.out, "mode=fsync%-every bytes=4194304 ms=%d+ MiB/s=%d+%.%d", + lang .. ": fsync-every report line") +checks.expect_match(tostring(r.out:find("fsync_ms") == nil), "true", + lang .. ": fsync-every prints no fsync_ms line") +expect_file_size(out_file, 4194304, lang .. ": fsync-every file is 4 MiB on disk") + +-- 5. Direct: either O_DIRECT works here (report + size) or the filesystem +-- rejects it with the documented message and exit 4. +r = checks.run(demo .. " --mode direct --size-mb 4 " .. out_file) +if r.exit == 4 then + checks.expect_match(r.out, "direct: unsupported on this filesystem", + lang .. ": direct EINVAL message with exit 4") +else + checks.expect_exit(r, 0, lang .. ": direct exits 0") + checks.expect_match(r.out, "mode=direct bytes=4194304 ms=%d+ MiB/s=%d+%.%d", + lang .. ": direct report line") + expect_file_size(out_file, 4194304, lang .. ": direct file is 4 MiB on disk") +end + +os.remove(out_file) +checks.finish() diff --git a/examples/09-event-loops-epoll-inotify/README.md b/examples/09-event-loops-epoll-inotify/README.md new file mode 100644 index 0000000..f437e55 --- /dev/null +++ b/examples/09-event-loops-epoll-inotify/README.md @@ -0,0 +1,77 @@ +# 09 — Event loops: epoll + inotify (`fwatch` v1) + +`fwatch` is the file watcher that grows across Part II. Chapter 07/08 gave it +polling-era subcommands (`snapshot`, `diff`); this chapter adds `watch`, a +real-time watcher — and the three implementations deliberately showcase the +two ways Linux programs consume readiness: + +| Language | Architecture | +|---|---| +| **C++23** | one explicit `epoll` loop over three fds — the inotify fd, a `timerfd` (drives both the 100 ms per-path debounce and the overall timeout), and a `signalfd` (SIGINT/SIGTERM as readable data). Single thread; every fd owned by an RAII wrapper; fallible setup returns `std::expected`. | +| **Rust (edition 2024)** | the same single-threaded epoll architecture via the `nix` crate (`Epoll`, `Inotify`, `TimerFd`, `SignalFd` — all `OwnedFd`-backed, closed on drop), with `anyhow` for error context. | +| **Go 1.26** | the idiomatic inversion: the runtime owns the poller. A goroutine reads the `O_NONBLOCK` inotify fd through `os.File` (parking on the netpoller, which is itself an epoll loop), and feeds a channel; `signal.Notify` and `time.After` are channels too; one `select` loop multiplexes all of it. | + +All three expose identical observable behavior, so one `verify.lua` covers +them. + +## CLI + +``` +usage: fwatch + snapshot DIR one line per regular file: namesizemtime_ns + diff OLD NEW compare two snapshots: created|modified|deleted + watch DIR [--timeout-ms T] watch DIR (default 2000 ms) until timeout or SIGINT/SIGTERM +``` + +- `snapshot DIR` — top-level regular files only, sorted by name, printed to + stdout as `name\tsize\tmtime_ns`. +- `diff OLD NEW` — reads two snapshot files and prints one line per changed + name, in name order: `created `, `deleted `, `modified ` + (size or mtime changed). Exit 0. +- `watch DIR` — prints `fwatch: watching DIR` to stderr, then one line per + filesystem event to stdout: + + ``` + event: created|modified|deleted + ``` + + Events are debounced 100 ms per path (a create followed immediately by + writes collapses to one `created`; delete wins within a window; a fast + delete+recreate reads as `modified`). On timeout the last line is + `fwatch: exiting (timeout)`; on SIGINT/SIGTERM it is + `fwatch: exiting (signal)`. Both exit 0. +- Exit codes: `0` success, `1` runtime error (e.g. missing directory), `2` + usage error. + +## Demo contract + +Each language directory has the standard `demo.sh`: + +- `./demo.sh build` — build only +- `./demo.sh run [args]` — run the built binary (`app`); the local path + `exec`s the binary so a backgrounded `run watch ...` can be signalled + directly +- `./demo.sh` — build, then run a 1-second self-demo (`watch .`) +- With env `TARGET` set, `run` deploys to that lab VM via + `scripts/lab/deploy-to-vm.sh` + +The top-level `./demo.sh [cpp|go|rust|all|build]` dispatches per language. + +## Try it + +```sh +./demo.sh cpp build +mkdir -p /tmp/fw && ./cpp/build/release/app watch /tmp/fw --timeout-ms 5000 & +sleep 0.3; echo hi > /tmp/fw/x.txt; echo more >> /tmp/fw/x.txt; rm /tmp/fw/x.txt +wait # one debounced "event: ..." stream, then "fwatch: exiting (timeout)" +``` + +## Verification + +`verify.lua` (run per language with `LSP_LANG` set) asserts observable +behavior: the v0 snapshot format and diff regression +(`modified a.txt` / `deleted b.txt` / `created c.txt` in name order), a +backgrounded `watch` that sees created/modified/deleted events for files +touched after startup, the debounce collapsing create+write into exactly one +event line, the clean `(timeout)` and `(signal)` exit lines with exit 0, and +the exit-code contract (2 for usage, 1 for a missing directory). diff --git a/examples/09-event-loops-epoll-inotify/cpp/.clang-format b/examples/09-event-loops-epoll-inotify/cpp/.clang-format new file mode 100644 index 0000000..2c8640f --- /dev/null +++ b/examples/09-event-loops-epoll-inotify/cpp/.clang-format @@ -0,0 +1,8 @@ +--- +BasedOnStyle: LLVM +Language: Cpp +Standard: c++20 +IndentWidth: 4 +ColumnLimit: 100 +PointerAlignment: Left +AllowShortFunctionsOnASingleLine: Inline diff --git a/examples/09-event-loops-epoll-inotify/cpp/.clang-tidy b/examples/09-event-loops-epoll-inotify/cpp/.clang-tidy new file mode 100644 index 0000000..2f275ba --- /dev/null +++ b/examples/09-event-loops-epoll-inotify/cpp/.clang-tidy @@ -0,0 +1,11 @@ +--- +Checks: > + bugprone-*, + modernize-*, + performance-*, + readability-*, + -modernize-use-trailing-return-type, + -readability-identifier-length +WarningsAsErrors: '' +HeaderFilterRegex: '.*' +FormatStyle: file diff --git a/examples/09-event-loops-epoll-inotify/cpp/CMakeLists.txt b/examples/09-event-loops-epoll-inotify/cpp/CMakeLists.txt new file mode 100644 index 0000000..ff62606 --- /dev/null +++ b/examples/09-event-loops-epoll-inotify/cpp/CMakeLists.txt @@ -0,0 +1,8 @@ +cmake_minimum_required(VERSION 3.25) +project(fwatch CXX) + +set(CMAKE_CXX_STANDARD 23) +set(CMAKE_CXX_STANDARD_REQUIRED ON) +set(CMAKE_CXX_EXTENSIONS OFF) + +add_executable(app src/main.cpp) diff --git a/examples/09-event-loops-epoll-inotify/cpp/CMakePresets.json b/examples/09-event-loops-epoll-inotify/cpp/CMakePresets.json new file mode 100644 index 0000000..e31bd2f --- /dev/null +++ b/examples/09-event-loops-epoll-inotify/cpp/CMakePresets.json @@ -0,0 +1,62 @@ +{ + "version": 6, + "cmakeMinimumRequired": { "major": 3, "minor": 25, "patch": 0 }, + "configurePresets": [ + { + "name": "_base", + "hidden": true, + "generator": "Ninja", + "binaryDir": "${sourceDir}/build/${presetName}", + "cacheVariables": { + "CMAKE_EXPORT_COMPILE_COMMANDS": "ON" + } + }, + { + "name": "release", + "inherits": "_base", + "displayName": "Release (-O2 with debug info)", + "cacheVariables": { "CMAKE_BUILD_TYPE": "RelWithDebInfo" } + }, + { + "name": "debug", + "inherits": "_base", + "displayName": "Debug", + "cacheVariables": { "CMAKE_BUILD_TYPE": "Debug" } + }, + { + "name": "asan", + "inherits": "_base", + "displayName": "ASan + UBSan instrumented build", + "cacheVariables": { + "CMAKE_BUILD_TYPE": "Debug", + "CMAKE_CXX_FLAGS": "-O1 -g -fno-omit-frame-pointer -fsanitize=address,undefined", + "CMAKE_EXE_LINKER_FLAGS": "-fsanitize=address,undefined" + } + }, + { + "name": "release-clang", + "inherits": "release", + "displayName": "Release (clang)", + "cacheVariables": { + "CMAKE_C_COMPILER": "clang", + "CMAKE_CXX_COMPILER": "clang++" + } + }, + { + "name": "debug-clang", + "inherits": "debug", + "displayName": "Debug (clang)", + "cacheVariables": { + "CMAKE_C_COMPILER": "clang", + "CMAKE_CXX_COMPILER": "clang++" + } + } + ], + "buildPresets": [ + { "name": "release", "configurePreset": "release" }, + { "name": "debug", "configurePreset": "debug" }, + { "name": "asan", "configurePreset": "asan" }, + { "name": "release-clang", "configurePreset": "release-clang" }, + { "name": "debug-clang", "configurePreset": "debug-clang" } + ] +} diff --git a/examples/09-event-loops-epoll-inotify/cpp/demo.sh b/examples/09-event-loops-epoll-inotify/cpp/demo.sh new file mode 100755 index 0000000..d47954d --- /dev/null +++ b/examples/09-event-loops-epoll-inotify/cpp/demo.sh @@ -0,0 +1,41 @@ +#!/usr/bin/env bash +set -euo pipefail + +cd "$(dirname "${BASH_SOURCE[0]}")" +REPO_ROOT="$(cd ../../.. && pwd)" +BIN=build/release/app + +build() { + cmake --preset release + cmake --build --preset release +} + +run() { + if [[ ! -x "$BIN" ]]; then + build + fi + if [[ -n "${TARGET:-}" ]]; then + "$REPO_ROOT/scripts/lab/deploy-to-vm.sh" "$TARGET" "$BIN" -- "$@" + else + # exec so a backgrounded `run watch ...` can be signalled directly + exec "./$BIN" "$@" + fi +} + +case "${1:-}" in + build) + build + ;; + run) + shift + run "$@" + ;; + "") + build + run watch . --timeout-ms 1000 + ;; + *) + echo "usage: $0 [build|run [args]]" >&2 + exit 2 + ;; +esac diff --git a/examples/09-event-loops-epoll-inotify/cpp/src/main.cpp b/examples/09-event-loops-epoll-inotify/cpp/src/main.cpp new file mode 100644 index 0000000..f8ca463 --- /dev/null +++ b/examples/09-event-loops-epoll-inotify/cpp/src/main.cpp @@ -0,0 +1,459 @@ +// fwatch v1 — a file watcher growing chapter by chapter. +// +// v0 subcommands (snapshot/diff) still work; v1 adds `watch`, built as ONE +// epoll loop over three kernel file descriptors on a single thread: +// +// inotify fd — filesystem events for the watched directory +// timerfd — the debounce (100 ms per path) AND the overall timeout +// signalfd — SIGINT/SIGTERM delivered as ordinary readable data +// +// The readiness model is explicit: nothing here blocks except epoll_wait(2). + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include +#include +#include +#include + +namespace { + +namespace fs = std::filesystem; +using std::chrono::milliseconds; +using std::chrono::nanoseconds; +using std::chrono::steady_clock; + +constexpr auto kDebounce = milliseconds{100}; + +[[nodiscard]] std::error_code last_error() { + return {errno, std::system_category()}; +} + +// --------------------------------------------------------------------------- +// RAII file descriptor — the only owner of every fd in this program. +// --------------------------------------------------------------------------- + +class Fd { +public: + Fd() = default; + explicit Fd(int fd) : fd_{fd} {} + Fd(const Fd&) = delete; + Fd& operator=(const Fd&) = delete; + Fd(Fd&& other) noexcept : fd_{std::exchange(other.fd_, -1)} {} + Fd& operator=(Fd&& other) noexcept { + if (this != &other) { + reset(); + fd_ = std::exchange(other.fd_, -1); + } + return *this; + } + ~Fd() { reset(); } + + void reset() noexcept { + if (fd_ >= 0) { + ::close(fd_); + } + fd_ = -1; + } + [[nodiscard]] int get() const noexcept { return fd_; } + +private: + int fd_ = -1; +}; + +[[nodiscard]] std::expected checked(int fd) { + if (fd < 0) { + return std::unexpected(last_error()); + } + return Fd{fd}; +} + +// --------------------------------------------------------------------------- +// v0: snapshot + diff +// --------------------------------------------------------------------------- + +struct FileState { + std::string name; + std::uint64_t size; + std::int64_t mtime_ns; +}; + +[[nodiscard]] std::expected, std::error_code> +scan_dir(const std::string& dir) { + std::error_code ec; + std::vector out; + for (const auto& entry : fs::directory_iterator{dir, ec}) { + struct stat st{}; + if (::fstatat(AT_FDCWD, entry.path().c_str(), &st, AT_SYMLINK_NOFOLLOW) != 0) { + continue; // raced with a concurrent delete + } + if (!S_ISREG(st.st_mode)) { + continue; + } + out.push_back(FileState{ + .name = entry.path().filename().string(), + .size = static_cast(st.st_size), + .mtime_ns = static_cast(st.st_mtim.tv_sec) * 1'000'000'000 + + st.st_mtim.tv_nsec, + }); + } + if (ec) { + return std::unexpected(ec); + } + std::ranges::sort(out, {}, &FileState::name); + return out; +} + +int cmd_snapshot(const std::string& dir) { + const auto files = scan_dir(dir); + if (!files) { + std::println(stderr, "fwatch: snapshot: {}: {}", dir, files.error().message()); + return 1; + } + for (const auto& f : *files) { + std::println("{}\t{}\t{}", f.name, f.size, f.mtime_ns); + } + return 0; +} + +// Snapshot line -> (name, "size\tmtime_ns"); malformed lines are skipped. +[[nodiscard]] std::expected, std::error_code> +load_snapshot(const std::string& path) { + std::ifstream in{path}; + if (!in) { + return std::unexpected(std::make_error_code(std::errc::no_such_file_or_directory)); + } + std::map out; + std::string line; + while (std::getline(in, line)) { + const auto tab2 = line.rfind('\t'); + if (tab2 == std::string::npos || tab2 == 0) { + continue; + } + const auto tab1 = line.rfind('\t', tab2 - 1); + if (tab1 == std::string::npos) { + continue; + } + out.insert_or_assign(line.substr(0, tab1), line.substr(tab1 + 1)); + } + return out; +} + +int cmd_diff(const std::string& old_path, const std::string& new_path) { + const auto old_snap = load_snapshot(old_path); + if (!old_snap) { + std::println(stderr, "fwatch: diff: {}: {}", old_path, old_snap.error().message()); + return 1; + } + const auto new_snap = load_snapshot(new_path); + if (!new_snap) { + std::println(stderr, "fwatch: diff: {}: {}", new_path, new_snap.error().message()); + return 1; + } + std::set names; + for (const auto& [name, _] : *old_snap) { + names.insert(name); + } + for (const auto& [name, _] : *new_snap) { + names.insert(name); + } + for (const auto& name : names) { + const auto o = old_snap->find(name); + const auto n = new_snap->find(name); + if (o == old_snap->end()) { + std::println("created {}", name); + } else if (n == new_snap->end()) { + std::println("deleted {}", name); + } else if (o->second != n->second) { + std::println("modified {}", name); + } + } + return 0; +} + +// --------------------------------------------------------------------------- +// v1: watch — one epoll loop, three fds, zero threads beyond main. +// --------------------------------------------------------------------------- + +enum class Kind : std::uint8_t { created, modified, deleted }; + +[[nodiscard]] std::string_view kind_name(Kind k) { + switch (k) { + case Kind::created: return "created"; + case Kind::modified: return "modified"; + case Kind::deleted: return "deleted"; + } + return "?"; +} + +[[nodiscard]] std::optional classify(std::uint32_t mask) { + if ((mask & (IN_CREATE | IN_MOVED_TO)) != 0) { + return Kind::created; + } + if ((mask & (IN_DELETE | IN_MOVED_FROM)) != 0) { + return Kind::deleted; + } + if ((mask & (IN_MODIFY | IN_ATTRIB)) != 0) { + return Kind::modified; + } + return std::nullopt; +} + +// Coalescing rule shared by all three implementations: within one debounce +// window, delete wins, a fresh creation stays "created" through later writes, +// and a delete+recreate pair reads as "modified". +[[nodiscard]] Kind merge(std::optional old_kind, Kind new_kind) { + if (!old_kind) { + return new_kind; + } + if (new_kind == Kind::deleted) { + return Kind::deleted; + } + if (*old_kind == Kind::created) { + return Kind::created; + } + return Kind::modified; +} + +struct Pending { + Kind kind; + steady_clock::time_point due; +}; + +[[nodiscard]] std::expected make_signalfd() { + sigset_t mask; + sigemptyset(&mask); + sigaddset(&mask, SIGINT); + sigaddset(&mask, SIGTERM); + if (::sigprocmask(SIG_BLOCK, &mask, nullptr) != 0) { + return std::unexpected(last_error()); + } + return checked(::signalfd(-1, &mask, SFD_NONBLOCK | SFD_CLOEXEC)); +} + +[[nodiscard]] std::expected arm_timer(const Fd& tfd, nanoseconds rel) { + if (rel <= nanoseconds::zero()) { + rel = nanoseconds{1}; // 0 would disarm; fire "immediately" instead + } + itimerspec spec{}; + spec.it_value.tv_sec = rel.count() / 1'000'000'000; + spec.it_value.tv_nsec = rel.count() % 1'000'000'000; + if (::timerfd_settime(tfd.get(), 0, &spec, nullptr) != 0) { + return std::unexpected(last_error()); + } + return {}; +} + +void flush(std::map& pending, steady_clock::time_point now, bool all) { + // std::map iterates in name order, so a batch flush is deterministic. + for (auto it = pending.begin(); it != pending.end();) { + if (all || it->second.due <= now) { + std::println("event: {} {}", kind_name(it->second.kind), it->first); + it = pending.erase(it); + } else { + ++it; + } + } + std::fflush(stdout); +} + +int cmd_watch(const std::string& dir, int timeout_ms) { + auto inotify = checked(::inotify_init1(IN_NONBLOCK | IN_CLOEXEC)); + if (!inotify) { + std::println(stderr, "fwatch: watch: inotify_init1: {}", inotify.error().message()); + return 1; + } + constexpr std::uint32_t watch_mask = + IN_CREATE | IN_MODIFY | IN_ATTRIB | IN_DELETE | IN_MOVED_FROM | IN_MOVED_TO; + if (::inotify_add_watch(inotify->get(), dir.c_str(), watch_mask) < 0) { + std::println(stderr, "fwatch: watch: {}: {}", dir, last_error().message()); + return 1; + } + auto timer = checked(::timerfd_create(CLOCK_MONOTONIC, TFD_NONBLOCK | TFD_CLOEXEC)); + if (!timer) { + std::println(stderr, "fwatch: watch: timerfd_create: {}", timer.error().message()); + return 1; + } + auto sigfd = make_signalfd(); + if (!sigfd) { + std::println(stderr, "fwatch: watch: signalfd: {}", sigfd.error().message()); + return 1; + } + auto epoll = checked(::epoll_create1(EPOLL_CLOEXEC)); + if (!epoll) { + std::println(stderr, "fwatch: watch: epoll_create1: {}", epoll.error().message()); + return 1; + } + + enum : std::uint32_t { tok_inotify, tok_timer, tok_signal }; + const auto add = [&](const Fd& fd, + std::uint32_t token) -> std::expected { + epoll_event ev{.events = EPOLLIN, .data = {.u32 = token}}; + if (::epoll_ctl(epoll->get(), EPOLL_CTL_ADD, fd.get(), &ev) != 0) { + return std::unexpected(last_error()); + } + return {}; + }; + auto registered = add(*inotify, tok_inotify) + .and_then([&] { return add(*timer, tok_timer); }) + .and_then([&] { return add(*sigfd, tok_signal); }); + if (!registered) { + std::println(stderr, "fwatch: watch: epoll_ctl: {}", registered.error().message()); + return 1; + } + + std::println(stderr, "fwatch: watching {}", dir); + const auto deadline = steady_clock::now() + milliseconds{timeout_ms}; + std::map pending; + + for (;;) { + // One timer covers both jobs: it is always armed to whichever comes + // first — the overall timeout or the earliest per-path debounce. + auto next = deadline; + for (const auto& [_, p] : pending) { + next = std::min(next, p.due); + } + if (auto armed = arm_timer(*timer, next - steady_clock::now()); !armed) { + std::println(stderr, "fwatch: watch: timerfd_settime: {}", armed.error().message()); + return 1; + } + + std::array events{}; + const int n = ::epoll_wait(epoll->get(), events.data(), events.size(), -1); + if (n < 0) { + if (errno == EINTR) { + continue; + } + std::println(stderr, "fwatch: watch: epoll_wait: {}", last_error().message()); + return 1; + } + + for (int i = 0; i < n; ++i) { + switch (events[i].data.u32) { + case tok_inotify: { + alignas(inotify_event) char buf[4096]; + for (;;) { + const ssize_t got = ::read(inotify->get(), buf, sizeof buf); + if (got <= 0) { + break; // EAGAIN: queue drained + } + const auto now = steady_clock::now(); + for (const char* p = buf; p < buf + got;) { + const auto* ev = reinterpret_cast(p); + p += sizeof(inotify_event) + ev->len; + if (ev->len == 0) { + continue; // event on the directory itself + } + const auto kind = classify(ev->mask); + if (!kind) { + continue; + } + const std::string name{ev->name}; + const auto it = pending.find(name); + const auto old_kind = it == pending.end() + ? std::nullopt + : std::optional{it->second.kind}; + pending.insert_or_assign( + name, Pending{merge(old_kind, *kind), now + kDebounce}); + } + } + break; + } + case tok_timer: { + std::uint64_t expirations = 0; + std::ignore = ::read(timer->get(), &expirations, sizeof expirations); + const auto now = steady_clock::now(); + if (now >= deadline) { + flush(pending, now, /*all=*/true); + std::println("fwatch: exiting (timeout)"); + return 0; + } + flush(pending, now, /*all=*/false); + break; + } + case tok_signal: { + signalfd_siginfo info{}; + std::ignore = ::read(sigfd->get(), &info, sizeof info); + flush(pending, steady_clock::now(), /*all=*/true); + std::println("fwatch: exiting (signal)"); + return 0; + } + default: + break; + } + } + } +} + +// --------------------------------------------------------------------------- +// CLI +// --------------------------------------------------------------------------- + +int usage() { + std::println(stderr, "usage: fwatch "); + std::println(stderr, + " snapshot DIR one line per regular file: " + "namesizemtime_ns"); + std::println(stderr, + " diff OLD NEW compare two snapshots: " + "created|modified|deleted "); + std::println(stderr, + " watch DIR [--timeout-ms T] watch DIR (default 2000 ms) until timeout " + "or SIGINT/SIGTERM"); + return 2; +} + +} // namespace + +int main(int argc, char** argv) { + const std::vector args{argv + 1, argv + argc}; + if (args.empty()) { + return usage(); + } + const auto cmd = args[0]; + if (cmd == "snapshot" && args.size() == 2) { + return cmd_snapshot(std::string{args[1]}); + } + if (cmd == "diff" && args.size() == 3) { + return cmd_diff(std::string{args[1]}, std::string{args[2]}); + } + if (cmd == "watch" && (args.size() == 2 || args.size() == 4)) { + int timeout_ms = 2000; + if (args.size() == 4) { + if (args[2] != "--timeout-ms") { + return usage(); + } + try { + timeout_ms = std::stoi(std::string{args[3]}); + } catch (const std::exception&) { + return usage(); + } + if (timeout_ms <= 0) { + return usage(); + } + } + return cmd_watch(std::string{args[1]}, timeout_ms); + } + return usage(); +} diff --git a/examples/09-event-loops-epoll-inotify/demo.sh b/examples/09-event-loops-epoll-inotify/demo.sh new file mode 100755 index 0000000..afbbe4c --- /dev/null +++ b/examples/09-event-loops-epoll-inotify/demo.sh @@ -0,0 +1,23 @@ +#!/usr/bin/env bash +set -euo pipefail +cd "$(dirname "$0")" + +langs=(cpp go rust) + +case "${1:-all}" in + cpp|go|rust) + lang="$1" + shift + exec "./$lang/demo.sh" "$@" + ;; + all) + for lang in "${langs[@]}"; do "./$lang/demo.sh"; done + ;; + build) + for lang in "${langs[@]}"; do "./$lang/demo.sh" build; done + ;; + *) + echo "usage: $0 [cpp|go|rust|all|build] [args...]" >&2 + exit 2 + ;; +esac diff --git a/examples/09-event-loops-epoll-inotify/go/demo.sh b/examples/09-event-loops-epoll-inotify/go/demo.sh new file mode 100755 index 0000000..8fd965b --- /dev/null +++ b/examples/09-event-loops-epoll-inotify/go/demo.sh @@ -0,0 +1,23 @@ +#!/usr/bin/env bash +set -euo pipefail +cd "$(dirname "$0")" +REPO_ROOT="$(cd ../../.. && pwd)" +BIN=bin/app + +build() { + go build -o "$BIN" . +} + +run() { + if [[ -n "${TARGET:-}" ]]; then + exec "$REPO_ROOT/scripts/lab/deploy-to-vm.sh" "$TARGET" "$BIN" -- "$@" + fi + exec "./$BIN" "$@" +} + +case "${1:-}" in + build) build ;; + run) shift; run "$@" ;; + "") build; run watch . --timeout-ms 1000 ;; + *) echo "usage: $0 [build|run [args...]]" >&2; exit 2 ;; +esac diff --git a/examples/09-event-loops-epoll-inotify/go/go.mod b/examples/09-event-loops-epoll-inotify/go/go.mod new file mode 100644 index 0000000..d275fdb --- /dev/null +++ b/examples/09-event-loops-epoll-inotify/go/go.mod @@ -0,0 +1,7 @@ +module github.com/patterncatalyst/linux-systems-programming/examples/09-event-loops-epoll-inotify/go + +go 1.26 + +toolchain go1.26.5 + +require golang.org/x/sys v0.47.0 diff --git a/examples/09-event-loops-epoll-inotify/go/go.sum b/examples/09-event-loops-epoll-inotify/go/go.sum new file mode 100644 index 0000000..37ee2d4 --- /dev/null +++ b/examples/09-event-loops-epoll-inotify/go/go.sum @@ -0,0 +1,2 @@ +golang.org/x/sys v0.47.0 h1:o7XGOvZQCADBQQ4Y7VNq2dRWQR7JmOUW8Kxx4ZsNgWs= +golang.org/x/sys v0.47.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw= diff --git a/examples/09-event-loops-epoll-inotify/go/main.go b/examples/09-event-loops-epoll-inotify/go/main.go new file mode 100644 index 0000000..d87fa83 --- /dev/null +++ b/examples/09-event-loops-epoll-inotify/go/main.go @@ -0,0 +1,321 @@ +// fwatch v1 — a file watcher growing chapter by chapter. +// +// v0 subcommands (snapshot/diff) still work; v1 adds `watch`. Where the C++ +// and Rust versions run one explicit epoll loop over inotify + timerfd + +// signalfd, the Go version leans on the runtime: a goroutine reads the +// inotify fd through os.File (the netpoller does the epoll_wait for us) and +// feeds a channel, signals arrive on a channel via signal.Notify, and the +// main select loop multiplexes events, the debounce timer, and the overall +// timeout. Same observable behavior, no visible poller. +package main + +import ( + "errors" + "fmt" + "io/fs" + "os" + "os/signal" + "path/filepath" + "sort" + "strconv" + "strings" + "syscall" + "time" + "unsafe" + + "golang.org/x/sys/unix" +) + +const debounce = 100 * time.Millisecond + +// --------------------------------------------------------------------------- +// v0: snapshot + diff +// --------------------------------------------------------------------------- + +func cmdSnapshot(dir string) error { + entries, err := os.ReadDir(dir) // sorted by filename + if err != nil { + if errors.Is(err, fs.ErrNotExist) { + return fmt.Errorf("%s: no such directory", dir) + } + return fmt.Errorf("%s: %w", dir, err) + } + for _, e := range entries { + info, err := os.Lstat(filepath.Join(dir, e.Name())) + if err != nil { + continue // raced with a concurrent delete + } + if !info.Mode().IsRegular() { + continue + } + st := info.Sys().(*syscall.Stat_t) + mtimeNs := int64(st.Mtim.Sec)*1_000_000_000 + int64(st.Mtim.Nsec) + fmt.Printf("%s\t%d\t%d\n", e.Name(), info.Size(), mtimeNs) + } + return nil +} + +// loadSnapshot parses "namesizemtime_ns" lines into +// name -> "sizemtime_ns"; malformed lines are skipped. +func loadSnapshot(path string) (map[string]string, error) { + data, err := os.ReadFile(path) + if err != nil { + return nil, fmt.Errorf("%s: %w", path, err) + } + out := make(map[string]string) + for _, line := range strings.Split(string(data), "\n") { + tab2 := strings.LastIndexByte(line, '\t') + if tab2 <= 0 { + continue + } + tab1 := strings.LastIndexByte(line[:tab2], '\t') + if tab1 < 0 { + continue + } + out[line[:tab1]] = line[tab1+1:] + } + return out, nil +} + +func cmdDiff(oldPath, newPath string) error { + oldSnap, err := loadSnapshot(oldPath) + if err != nil { + return err + } + newSnap, err := loadSnapshot(newPath) + if err != nil { + return err + } + seen := make(map[string]bool, len(oldSnap)+len(newSnap)) + names := make([]string, 0, len(oldSnap)+len(newSnap)) + for name := range oldSnap { + seen[name] = true + names = append(names, name) + } + for name := range newSnap { + if !seen[name] { + names = append(names, name) + } + } + sort.Strings(names) + for _, name := range names { + o, inOld := oldSnap[name] + n, inNew := newSnap[name] + switch { + case !inOld: + fmt.Printf("created %s\n", name) + case !inNew: + fmt.Printf("deleted %s\n", name) + case o != n: + fmt.Printf("modified %s\n", name) + } + } + return nil +} + +// --------------------------------------------------------------------------- +// v1: watch — goroutine + channels; the runtime owns the epoll loop. +// --------------------------------------------------------------------------- + +type fsEvent struct { + kind string // "created" | "modified" | "deleted" + name string +} + +func classify(mask uint32) (string, bool) { + switch { + case mask&(unix.IN_CREATE|unix.IN_MOVED_TO) != 0: + return "created", true + case mask&(unix.IN_DELETE|unix.IN_MOVED_FROM) != 0: + return "deleted", true + case mask&(unix.IN_MODIFY|unix.IN_ATTRIB) != 0: + return "modified", true + } + return "", false +} + +// mergeKinds is the coalescing rule shared by all three implementations: +// within one debounce window, delete wins, a fresh creation stays "created" +// through later writes, and a delete+recreate pair reads as "modified". +func mergeKinds(oldKind, newKind string) string { + switch { + case oldKind == "": + return newKind + case newKind == "deleted": + return "deleted" + case oldKind == "created": + return "created" + default: + return "modified" + } +} + +// readEvents pumps parsed inotify events into ch. The fd is O_NONBLOCK, so +// os.File hands it to the runtime netpoller: Read parks this goroutine, not +// an OS thread, until the kernel marks the fd readable. +func readEvents(f *os.File, ch chan<- fsEvent) { + defer close(ch) + buf := make([]byte, 4096) + for { + n, err := f.Read(buf) + if err != nil { + return // fd closed at exit, or watcher torn down + } + for off := 0; off+unix.SizeofInotifyEvent <= n; { + raw := (*unix.InotifyEvent)(unsafe.Pointer(&buf[off])) + nameBytes := buf[off+unix.SizeofInotifyEvent : off+unix.SizeofInotifyEvent+int(raw.Len)] + off += unix.SizeofInotifyEvent + int(raw.Len) + name := strings.TrimRight(string(nameBytes), "\x00") + if name == "" { + continue // event on the directory itself + } + if kind, ok := classify(raw.Mask); ok { + ch <- fsEvent{kind: kind, name: name} + } + } + } +} + +type pendingEvent struct { + kind string + due time.Time +} + +func cmdWatch(dir string, timeoutMs int) error { + ifd, err := unix.InotifyInit1(unix.IN_NONBLOCK | unix.IN_CLOEXEC) + if err != nil { + return fmt.Errorf("inotify_init1: %w", err) + } + watchMask := uint32(unix.IN_CREATE | unix.IN_MODIFY | unix.IN_ATTRIB | + unix.IN_DELETE | unix.IN_MOVED_FROM | unix.IN_MOVED_TO) + if _, err := unix.InotifyAddWatch(ifd, dir, watchMask); err != nil { + unix.Close(ifd) + if errors.Is(err, unix.ENOENT) { + return fmt.Errorf("%s: no such directory", dir) + } + return fmt.Errorf("%s: %w", dir, err) + } + inotifyFile := os.NewFile(uintptr(ifd), "inotify") // registers with the netpoller + defer inotifyFile.Close() + + events := make(chan fsEvent, 64) + go readEvents(inotifyFile, events) + + sigs := make(chan os.Signal, 1) + signal.Notify(sigs, syscall.SIGINT, syscall.SIGTERM) + defer signal.Stop(sigs) + + fmt.Fprintf(os.Stderr, "fwatch: watching %s\n", dir) + timeout := time.After(time.Duration(timeoutMs) * time.Millisecond) + + pending := make(map[string]*pendingEvent) + flushTimer := time.NewTimer(time.Hour) + flushTimer.Stop() + defer flushTimer.Stop() + + flush := func(all bool) { + now := time.Now() + names := make([]string, 0, len(pending)) + for name, p := range pending { + if all || !p.due.After(now) { + names = append(names, name) + } + } + sort.Strings(names) // deterministic batch order + for _, name := range names { + fmt.Printf("event: %s %s\n", pending[name].kind, name) + delete(pending, name) + } + } + rearm := func() { + if len(pending) == 0 { + flushTimer.Stop() + return + } + earliest := time.Time{} + for _, p := range pending { + if earliest.IsZero() || p.due.Before(earliest) { + earliest = p.due + } + } + flushTimer.Reset(time.Until(earliest)) + } + + for { + select { + case ev, ok := <-events: + if !ok { + return errors.New("inotify reader stopped unexpectedly") + } + oldKind := "" + if p, exists := pending[ev.name]; exists { + oldKind = p.kind + } + pending[ev.name] = &pendingEvent{ + kind: mergeKinds(oldKind, ev.kind), + due: time.Now().Add(debounce), + } + rearm() + case <-flushTimer.C: + flush(false) + rearm() + case <-timeout: + flush(true) + fmt.Println("fwatch: exiting (timeout)") + return nil + case <-sigs: + flush(true) + fmt.Println("fwatch: exiting (signal)") + return nil + } + } +} + +// --------------------------------------------------------------------------- +// CLI +// --------------------------------------------------------------------------- + +func usage() { + fmt.Fprintln(os.Stderr, "usage: fwatch ") + fmt.Fprintln(os.Stderr, " snapshot DIR one line per regular file: namesizemtime_ns") + fmt.Fprintln(os.Stderr, " diff OLD NEW compare two snapshots: created|modified|deleted ") + fmt.Fprintln(os.Stderr, " watch DIR [--timeout-ms T] watch DIR (default 2000 ms) until timeout or SIGINT/SIGTERM") + os.Exit(2) +} + +func main() { + args := os.Args[1:] + if len(args) == 0 { + usage() + } + var err error + var cmd string + switch { + case args[0] == "snapshot" && len(args) == 2: + cmd = "snapshot" + err = cmdSnapshot(args[1]) + case args[0] == "diff" && len(args) == 3: + cmd = "diff" + err = cmdDiff(args[1], args[2]) + case args[0] == "watch" && (len(args) == 2 || len(args) == 4): + cmd = "watch" + timeoutMs := 2000 + if len(args) == 4 { + if args[2] != "--timeout-ms" { + usage() + } + var perr error + timeoutMs, perr = strconv.Atoi(args[3]) + if perr != nil || timeoutMs <= 0 { + usage() + } + } + err = cmdWatch(args[1], timeoutMs) + default: + usage() + } + if err != nil { + fmt.Fprintf(os.Stderr, "fwatch: %s: %v\n", cmd, err) + os.Exit(1) + } +} diff --git a/examples/09-event-loops-epoll-inotify/rust/Cargo.lock b/examples/09-event-loops-epoll-inotify/rust/Cargo.lock new file mode 100644 index 0000000..a14f48f --- /dev/null +++ b/examples/09-event-loops-epoll-inotify/rust/Cargo.lock @@ -0,0 +1,53 @@ +# This file is automatically @generated by Cargo. +# It is not intended for manual editing. +version = 4 + +[[package]] +name = "anyhow" +version = "1.0.104" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "330a5ed07fa54e4702c9d6c4174f74427fc0ef6e214bbd677ae50a5099946470" + +[[package]] +name = "bitflags" +version = "2.13.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b588b76d00fde79687d7646a9b5bdf3cc0f655e0bbd080335a95d7e96f3587da" + +[[package]] +name = "cfg-if" +version = "1.0.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9330f8b2ff13f34540b44e946ef35111825727b38d33286ef986142615121801" + +[[package]] +name = "cfg_aliases" +version = "0.2.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f079e83a288787bcd14a6aea84cee5c87a67c5a3e660c30f557a3d24761b3527" + +[[package]] +name = "fwatch" +version = "0.1.0" +dependencies = [ + "anyhow", + "nix", +] + +[[package]] +name = "libc" +version = "0.2.186" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "68ab91017fe16c622486840e4c83c9a37afeff978bd239b5293d61ece587de66" + +[[package]] +name = "nix" +version = "0.30.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "74523f3a35e05aba87a1d978330aef40f67b0304ac79c1c00b294c9830543db6" +dependencies = [ + "bitflags", + "cfg-if", + "cfg_aliases", + "libc", +] diff --git a/examples/09-event-loops-epoll-inotify/rust/Cargo.toml b/examples/09-event-loops-epoll-inotify/rust/Cargo.toml new file mode 100644 index 0000000..c8b6e53 --- /dev/null +++ b/examples/09-event-loops-epoll-inotify/rust/Cargo.toml @@ -0,0 +1,12 @@ +[package] +name = "fwatch" +version = "0.1.0" +edition = "2024" + +[[bin]] +name = "app" +path = "src/main.rs" + +[dependencies] +anyhow = "1" +nix = { version = "0.30", features = ["event", "inotify", "signal", "time"] } diff --git a/examples/09-event-loops-epoll-inotify/rust/demo.sh b/examples/09-event-loops-epoll-inotify/rust/demo.sh new file mode 100755 index 0000000..f01488f --- /dev/null +++ b/examples/09-event-loops-epoll-inotify/rust/demo.sh @@ -0,0 +1,27 @@ +#!/usr/bin/env bash +set -euo pipefail +cd "$(dirname "${BASH_SOURCE[0]}")" + +BIN=target/release/app + +build() { + cargo build --release +} + +run() { + if [[ -n "${TARGET:-}" ]]; then + local repo_root + repo_root=$(cd ../../.. && pwd) + "$repo_root/scripts/lab/deploy-to-vm.sh" "$TARGET" "$BIN" -- "$@" + else + # exec so a backgrounded `run watch ...` can be signalled directly + exec "./$BIN" "$@" + fi +} + +case "${1:-}" in + build) build ;; + run) shift; run "$@" ;; + "") build; run watch . --timeout-ms 1000 ;; + *) echo "usage: $0 [build|run [args...]]" >&2; exit 2 ;; +esac diff --git a/examples/09-event-loops-epoll-inotify/rust/rust-toolchain.toml b/examples/09-event-loops-epoll-inotify/rust/rust-toolchain.toml new file mode 100644 index 0000000..4099f1a --- /dev/null +++ b/examples/09-event-loops-epoll-inotify/rust/rust-toolchain.toml @@ -0,0 +1,3 @@ +[toolchain] +channel = "1.97.1" +components = ["rustfmt", "clippy", "rust-src"] diff --git a/examples/09-event-loops-epoll-inotify/rust/src/main.rs b/examples/09-event-loops-epoll-inotify/rust/src/main.rs new file mode 100644 index 0000000..dd800d4 --- /dev/null +++ b/examples/09-event-loops-epoll-inotify/rust/src/main.rs @@ -0,0 +1,288 @@ +// fwatch v1 — a file watcher growing chapter by chapter. +// +// v0 subcommands (snapshot/diff) still work; v1 adds `watch`, built exactly +// like the C++ version: ONE epoll loop over three kernel file descriptors on +// a single thread — +// +// inotify fd — filesystem events for the watched directory +// timerfd — the debounce (100 ms per path) AND the overall timeout +// signalfd — SIGINT/SIGTERM delivered as ordinary readable data +// +// nix wraps each fd in an owning type (OwnedFd underneath), so every resource +// closes on drop; the readiness model stays explicit. + +use std::collections::{BTreeMap, BTreeSet}; +use std::os::linux::fs::MetadataExt; +use std::process::ExitCode; +use std::time::{Duration, Instant}; + +use anyhow::{Context, Result}; +use nix::errno::Errno; +use nix::sys::epoll::{Epoll, EpollCreateFlags, EpollEvent, EpollFlags, EpollTimeout}; +use nix::sys::inotify::{AddWatchFlags, InitFlags, Inotify}; +use nix::sys::signal::{SigSet, Signal}; +use nix::sys::signalfd::{SfdFlags, SignalFd}; +use nix::sys::time::TimeSpec; +use nix::sys::timerfd::{ClockId, Expiration, TimerFd, TimerFlags, TimerSetTimeFlags}; + +const DEBOUNCE: Duration = Duration::from_millis(100); + +// --------------------------------------------------------------------------- +// v0: snapshot + diff +// --------------------------------------------------------------------------- + +fn cmd_snapshot(dir: &str) -> Result<()> { + let mut files: Vec<(String, u64, i64)> = Vec::new(); + let entries = std::fs::read_dir(dir).with_context(|| dir.to_string())?; + for entry in entries { + let entry = entry.with_context(|| dir.to_string())?; + let Ok(meta) = entry.metadata() else { + continue; // raced with a concurrent delete + }; + if !meta.file_type().is_file() { + continue; + } + let mtime_ns = meta.st_mtime() * 1_000_000_000 + meta.st_mtime_nsec(); + files.push((entry.file_name().to_string_lossy().into_owned(), meta.st_size(), mtime_ns)); + } + files.sort(); + for (name, size, mtime_ns) in files { + println!("{name}\t{size}\t{mtime_ns}"); + } + Ok(()) +} + +/// Parse "namesizemtime_ns" lines into name -> "sizemtime_ns"; +/// malformed lines are skipped. +fn load_snapshot(path: &str) -> Result> { + let text = std::fs::read_to_string(path).with_context(|| path.to_string())?; + let mut out = BTreeMap::new(); + for line in text.lines() { + let Some((rest, mtime)) = line.rsplit_once('\t') else { + continue; + }; + let Some((name, size)) = rest.rsplit_once('\t') else { + continue; + }; + if name.is_empty() { + continue; + } + out.insert(name.to_string(), format!("{size}\t{mtime}")); + } + Ok(out) +} + +fn cmd_diff(old_path: &str, new_path: &str) -> Result<()> { + let old_snap = load_snapshot(old_path)?; + let new_snap = load_snapshot(new_path)?; + let names: BTreeSet<&String> = old_snap.keys().chain(new_snap.keys()).collect(); + for name in names { + match (old_snap.get(name), new_snap.get(name)) { + (None, Some(_)) => println!("created {name}"), + (Some(_), None) => println!("deleted {name}"), + (Some(o), Some(n)) if o != n => println!("modified {name}"), + _ => {} + } + } + Ok(()) +} + +// --------------------------------------------------------------------------- +// v1: watch — one epoll loop, three fds, zero threads beyond main. +// --------------------------------------------------------------------------- + +#[derive(Clone, Copy, PartialEq, Eq)] +enum Kind { + Created, + Modified, + Deleted, +} + +impl Kind { + fn name(self) -> &'static str { + match self { + Kind::Created => "created", + Kind::Modified => "modified", + Kind::Deleted => "deleted", + } + } +} + +fn classify(mask: AddWatchFlags) -> Option { + if mask.intersects(AddWatchFlags::IN_CREATE | AddWatchFlags::IN_MOVED_TO) { + Some(Kind::Created) + } else if mask.intersects(AddWatchFlags::IN_DELETE | AddWatchFlags::IN_MOVED_FROM) { + Some(Kind::Deleted) + } else if mask.intersects(AddWatchFlags::IN_MODIFY | AddWatchFlags::IN_ATTRIB) { + Some(Kind::Modified) + } else { + None + } +} + +/// Coalescing rule shared by all three implementations: within one debounce +/// window, delete wins, a fresh creation stays "created" through later +/// writes, and a delete+recreate pair reads as "modified". +fn merge(old: Option, new: Kind) -> Kind { + match (old, new) { + (None, k) => k, + (_, Kind::Deleted) => Kind::Deleted, + (Some(Kind::Created), _) => Kind::Created, + _ => Kind::Modified, + } +} + +struct Pending { + kind: Kind, + due: Instant, +} + +const TOK_INOTIFY: u64 = 0; +const TOK_TIMER: u64 = 1; +const TOK_SIGNAL: u64 = 2; + +fn arm_timer(timer: &TimerFd, rel: Duration) -> Result<()> { + // A zero TimeSpec would disarm the timer; fire "immediately" instead. + let rel = rel.max(Duration::from_nanos(1)); + timer + .set(Expiration::OneShot(TimeSpec::from_duration(rel)), TimerSetTimeFlags::empty()) + .context("timerfd_settime")?; + Ok(()) +} + +fn flush(pending: &mut BTreeMap, now: Instant, all: bool) { + // BTreeMap iterates in name order, so a batch flush is deterministic. + let due: Vec = pending + .iter() + .filter(|(_, p)| all || p.due <= now) + .map(|(name, _)| name.clone()) + .collect(); + for name in due { + let p = pending.remove(&name).expect("collected from this map"); + println!("event: {} {}", p.kind.name(), name); + } +} + +fn cmd_watch(dir: &str, timeout_ms: u64) -> Result<()> { + let inotify = + Inotify::init(InitFlags::IN_NONBLOCK | InitFlags::IN_CLOEXEC).context("inotify_init1")?; + let watch_mask = AddWatchFlags::IN_CREATE + | AddWatchFlags::IN_MODIFY + | AddWatchFlags::IN_ATTRIB + | AddWatchFlags::IN_DELETE + | AddWatchFlags::IN_MOVED_FROM + | AddWatchFlags::IN_MOVED_TO; + inotify.add_watch(dir, watch_mask).with_context(|| dir.to_string())?; + + let timer = TimerFd::new( + ClockId::CLOCK_MONOTONIC, + TimerFlags::TFD_NONBLOCK | TimerFlags::TFD_CLOEXEC, + ) + .context("timerfd_create")?; + + let mut sigset = SigSet::empty(); + sigset.add(Signal::SIGINT); + sigset.add(Signal::SIGTERM); + sigset.thread_block().context("sigprocmask")?; + let sigfd = SignalFd::with_flags(&sigset, SfdFlags::SFD_NONBLOCK | SfdFlags::SFD_CLOEXEC) + .context("signalfd")?; + + let epoll = Epoll::new(EpollCreateFlags::EPOLL_CLOEXEC).context("epoll_create1")?; + epoll.add(&inotify, EpollEvent::new(EpollFlags::EPOLLIN, TOK_INOTIFY)).context("epoll_ctl")?; + epoll.add(&timer, EpollEvent::new(EpollFlags::EPOLLIN, TOK_TIMER)).context("epoll_ctl")?; + epoll.add(&sigfd, EpollEvent::new(EpollFlags::EPOLLIN, TOK_SIGNAL)).context("epoll_ctl")?; + + eprintln!("fwatch: watching {dir}"); + let deadline = Instant::now() + Duration::from_millis(timeout_ms); + let mut pending: BTreeMap = BTreeMap::new(); + + loop { + // One timer covers both jobs: it is always armed to whichever comes + // first — the overall timeout or the earliest per-path debounce. + let next = pending.values().map(|p| p.due).min().map_or(deadline, |d| d.min(deadline)); + arm_timer(&timer, next.saturating_duration_since(Instant::now()))?; + + let mut events = [EpollEvent::empty(); 8]; + let n = match epoll.wait(&mut events, EpollTimeout::NONE) { + Ok(n) => n, + Err(Errno::EINTR) => continue, + Err(e) => return Err(e).context("epoll_wait"), + }; + + for event in &events[..n] { + match event.data() { + TOK_INOTIFY => loop { + let batch = match inotify.read_events() { + Ok(batch) => batch, + Err(Errno::EAGAIN) => break, // queue drained + Err(e) => return Err(e).context("inotify read"), + }; + let now = Instant::now(); + for ev in batch { + let Some(name) = ev.name else { + continue; // event on the directory itself + }; + let Some(kind) = classify(ev.mask) else { + continue; + }; + let name = name.to_string_lossy().into_owned(); + let old = pending.get(&name).map(|p| p.kind); + pending + .insert(name, Pending { kind: merge(old, kind), due: now + DEBOUNCE }); + } + }, + TOK_TIMER => { + let mut buf = [0u8; 8]; + let _ = nix::unistd::read(&timer, &mut buf); + let now = Instant::now(); + if now >= deadline { + flush(&mut pending, now, true); + println!("fwatch: exiting (timeout)"); + return Ok(()); + } + flush(&mut pending, now, false); + } + TOK_SIGNAL => { + let _ = sigfd.read_signal(); + flush(&mut pending, Instant::now(), true); + println!("fwatch: exiting (signal)"); + return Ok(()); + } + _ => {} + } + } + } +} + +// --------------------------------------------------------------------------- +// CLI +// --------------------------------------------------------------------------- + +fn usage() -> ExitCode { + eprintln!("usage: fwatch "); + eprintln!(" snapshot DIR one line per regular file: namesizemtime_ns"); + eprintln!(" diff OLD NEW compare two snapshots: created|modified|deleted "); + eprintln!(" watch DIR [--timeout-ms T] watch DIR (default 2000 ms) until timeout or SIGINT/SIGTERM"); + ExitCode::from(2) +} + +fn main() -> ExitCode { + let args: Vec = std::env::args().skip(1).collect(); + let (cmd, result) = match args.iter().map(String::as_str).collect::>()[..] { + ["snapshot", dir] => ("snapshot", cmd_snapshot(dir)), + ["diff", old, new] => ("diff", cmd_diff(old, new)), + ["watch", dir] => ("watch", cmd_watch(dir, 2000)), + ["watch", dir, "--timeout-ms", t] => match t.parse::() { + Ok(ms) if ms > 0 => ("watch", cmd_watch(dir, ms)), + _ => return usage(), + }, + _ => return usage(), + }; + match result { + Ok(()) => ExitCode::SUCCESS, + Err(err) => { + eprintln!("fwatch: {cmd}: {err:#}"); + ExitCode::from(1) + } + } +} diff --git a/examples/09-event-loops-epoll-inotify/verify.lua b/examples/09-event-loops-epoll-inotify/verify.lua new file mode 100644 index 0000000..f183ffd --- /dev/null +++ b/examples/09-event-loops-epoll-inotify/verify.lua @@ -0,0 +1,119 @@ +-- Verify fwatch v1 (09-event-loops-epoll-inotify) for LSP_LANG. +-- +-- Asserts observable behavior, identical across cpp/go/rust: +-- 1. v0 regression — snapshot format and diff created/modified/deleted lines +-- 2. watch: background run, mutate the directory after ~400 ms, expect one +-- debounced event line per path and a clean "(timeout)" exit, exit 0 +-- 3. watch: SIGTERM produces the "(signal)" exit line, exit 0 +-- 4. usage errors exit 2; watching a missing directory exits 1 +-- +-- Invoked from the example directory with LSP_LANG (cpp|go|rust) and +-- EXAMPLE_DIR set; the interpreter may be lua or luajit. + +local script_dir = arg[0]:match("(.*/)") or "./" +local repo_root = os.getenv("REPO_ROOT") or (script_dir .. "../..") +package.path = package.path .. ";" .. repo_root .. "/scripts/lib/?.lua" +local checks = require("checks") + +local lang = os.getenv("LSP_LANG") +if lang ~= "cpp" and lang ~= "go" and lang ~= "rust" then + checks.skip("LSP_LANG must be cpp|go|rust (got " .. tostring(lang) .. ")") +end + +local demo = "./demo.sh " .. lang .. " run" + +-- Scratch area for watched dirs and snapshot files. +local wd = checks.run("mktemp -d").out:gsub("%s+$", "") +if wd == "" or not wd:match("^/") then + checks.skip("mktemp -d did not yield a directory") +end + +local function sh(script) + return checks.run(script:gsub("@WD@", wd)) +end + +-- --------------------------------------------------------------------------- +-- 1. v0 regression: snapshot + diff +-- --------------------------------------------------------------------------- + +local snap = sh([[ +set -e +mkdir @WD@/w +printf 'one\n' > @WD@/w/a.txt +printf 'two\n' > @WD@/w/b.txt +]] .. demo .. [[ snapshot @WD@/w | tee @WD@/s1.txt +]]) +checks.expect_exit(snap, 0, lang .. ": snapshot exits 0") +checks.expect_match(snap.out, "a%.txt\t4\t%d+", lang .. ": snapshot lists a.txt with size 4 and mtime_ns") +checks.expect_match(snap.out, "b%.txt\t4\t%d+", lang .. ": snapshot lists b.txt") + +local diff = sh([[ +set -e +sleep 0.05 +printf 'more\n' >> @WD@/w/a.txt +rm @WD@/w/b.txt +printf 'new\n' > @WD@/w/c.txt +]] .. demo .. [[ snapshot @WD@/w > @WD@/s2.txt +]] .. demo .. [[ diff @WD@/s1.txt @WD@/s2.txt +]]) +checks.expect_exit(diff, 0, lang .. ": diff exits 0") +checks.expect_match(diff.out, "modified a%.txt\ndeleted b%.txt\ncreated c%.txt", + lang .. ": diff reports modified/deleted/created in name order") + +-- --------------------------------------------------------------------------- +-- 2. watch: debounced events, then a clean timeout exit +-- --------------------------------------------------------------------------- + +local watch = sh([[ +set -e +]] .. demo .. [[ watch @WD@/w --timeout-ms 2000 > @WD@/watch.out 2> @WD@/watch.err & +pid=$! +sleep 0.4 +printf 'hello\n' > @WD@/w/n1.txt +printf 'again\n' >> @WD@/w/a.txt +rm @WD@/w/c.txt +wait $pid +cat @WD@/watch.err @WD@/watch.out +]]) +checks.expect_exit(watch, 0, lang .. ": watch exits 0 after timeout") +checks.expect_match(watch.out, "fwatch: watching ", lang .. ": watch announces the directory on stderr") +checks.expect_match(watch.out, "event: created n1%.txt", lang .. ": created event for new file") +checks.expect_match(watch.out, "event: modified a%.txt", lang .. ": modified event for appended file") +checks.expect_match(watch.out, "event: deleted c%.txt", lang .. ": deleted event for removed file") +checks.expect_match(watch.out, "fwatch: exiting %(timeout%)", lang .. ": clean timeout exit line") + +-- Debounce check: creating n1.txt fires IN_CREATE + IN_MODIFY back to back, +-- but the 100 ms window must coalesce them into exactly ONE event line. +local _, n1_lines = watch.out:gsub("event: %a+ n1%.txt", "") +checks.expect_match("n1-event-lines=" .. n1_lines, "^n1%-event%-lines=1$", + lang .. ": create+write within 100 ms debounce to one event line") + +-- --------------------------------------------------------------------------- +-- 3. watch: SIGTERM -> "(signal)" exit line, still exit 0 +-- --------------------------------------------------------------------------- + +local sig = sh([[ +set -e +]] .. demo .. [[ watch @WD@/w --timeout-ms 10000 > @WD@/sig.out 2>/dev/null & +pid=$! +sleep 0.5 +kill -TERM $pid +wait $pid +cat @WD@/sig.out +]]) +checks.expect_exit(sig, 0, lang .. ": watch exits 0 on SIGTERM") +checks.expect_match(sig.out, "fwatch: exiting %(signal%)", lang .. ": signal exit line") + +-- --------------------------------------------------------------------------- +-- 4. error shapes +-- --------------------------------------------------------------------------- + +local bogus = checks.run(demo .. " bogus") +checks.expect_exit(bogus, 2, lang .. ": unknown subcommand exits 2") +checks.expect_match(bogus.out, "usage: fwatch", lang .. ": unknown subcommand prints usage") + +local missing = sh(demo .. " watch @WD@/no-such-dir") +checks.expect_exit(missing, 1, lang .. ": watching a missing directory exits 1") + +checks.run("rm -rf " .. wd) +checks.finish() diff --git a/examples/10-io-uring/README.md b/examples/10-io-uring/README.md new file mode 100644 index 0000000..758c704 --- /dev/null +++ b/examples/10-io-uring/README.md @@ -0,0 +1,81 @@ +# 10-io-uring — fwatch v2, the batched sync engine + +Chapter 10 grows `fwatch` into a small directory sync tool with two copy +engines, and uses the difference between them to teach io_uring: submission +and completion rings shared with the kernel, batches of SQEs per syscall, and +linked operations. + +``` +fwatch scan DIR # v0: polling snapshot +fwatch watch DIR --timeout MS # v1: inotify events +fwatch sync SRCDIR DSTDIR [--engine rw|uring] # v2: batched copy (default rw) +``` + +## Behavior (identical across C++, Go, and Rust) + +- `scan DIR` walks the tree and prints one line: + `scanned files bytes` (regular files only; symlinks skipped). +- `watch DIR --timeout MS` watches `DIR` (non-recursive) via inotify for + `MS` milliseconds, printing `event CREATE|MODIFY|DELETE ` per event + and a final `watched events`. +- `sync SRCDIR DSTDIR` recreates the directory tree (including empty + directories) and copies every regular file, then prints + `synced files bytes engine= ms=`. +- No/bad arguments print a usage block on stderr and exit 2; runtime errors + print `fwatch: ...` on stderr and exit 1. + +## The two engines + +- `--engine rw` (all three languages): a plain `read(2)`/`write(2)` loop, + 128 KiB at a time — the baseline from earlier chapters. +- `--engine uring`: + - **C++**: a self-contained educational io_uring layer built on raw + `syscall(2)` wrappers — `io_uring_setup(2)`, `io_uring_enter(2)`, and the + `mmap(2)`'d SQ ring, CQ ring, and SQE array (no liburing; the host lacks + `liburing-devel`, and seeing the rings by hand is the point). Each file is + copied as batches of linked READ→WRITE SQE pairs (`IOSQE_IO_LINK`), one + pair per 128 KiB chunk, up to 8 pairs per `io_uring_enter`. Under + `strace -c`, all data movement collapses into a handful of + `io_uring_enter` calls. + - **Rust**: the same batched linked-pair design on the `io-uring` crate + (0.7), which owns the ring setup/mmap details the C++ version spells out. + - **Go**: prints `engine=uring: unsupported in Go (see chapter 10)` and + exits **64**. That is deliberate, not a gap: the Go runtime already + multiplexes goroutines over epoll in the netpoller, and io_uring support + (golang/go#31908) remains on hold. The chapter uses this asymmetry to + discuss where each language's I/O story comes from. + +## Demo contract + +Each language directory has a `demo.sh` with the book-wide interface: + +- `./demo.sh build` — build only +- `./demo.sh run [args]` — run the built binary (builds first if needed) +- With env `TARGET` set, `run` deploys the binary to that lab VM via + `scripts/lab/deploy-to-vm.sh` instead of running locally. + +The top-level `demo.sh` dispatches: `./demo.sh cpp run sync SRC DST` execs +`cpp/demo.sh run sync SRC DST`. The binary is always named `app`. + +Try it: + +```sh +./demo.sh build +mkdir -p /tmp/fw/src && echo hi > /tmp/fw/src/a.txt +./demo.sh cpp run sync /tmp/fw/src /tmp/fw/dst --engine uring +./demo.sh rust run sync /tmp/fw/src /tmp/fw/dst --engine uring +./demo.sh go run sync /tmp/fw/src /tmp/fw/dst --engine uring # exit 64 +``` + +## Verification + +`verify.lua` (run per language with `LSP_LANG` set) builds a fixture tree — +five files totalling 1,348,634 bytes, sized to straddle the 128 KiB chunk and +the 8-pair batch limit — then asserts: + +- exact `scanned`/`synced` counter lines, +- `diff -r` byte-equality of the rw copy (all languages) and the uring copy + (C++ and Rust), +- the Go uring refusal message with exit 64, +- live `event CREATE ...` lines from `watch` while a file is created, and +- usage + exit 2 for a bare invocation. diff --git a/examples/10-io-uring/cpp/.clang-format b/examples/10-io-uring/cpp/.clang-format new file mode 100644 index 0000000..2c8640f --- /dev/null +++ b/examples/10-io-uring/cpp/.clang-format @@ -0,0 +1,8 @@ +--- +BasedOnStyle: LLVM +Language: Cpp +Standard: c++20 +IndentWidth: 4 +ColumnLimit: 100 +PointerAlignment: Left +AllowShortFunctionsOnASingleLine: Inline diff --git a/examples/10-io-uring/cpp/.clang-tidy b/examples/10-io-uring/cpp/.clang-tidy new file mode 100644 index 0000000..2f275ba --- /dev/null +++ b/examples/10-io-uring/cpp/.clang-tidy @@ -0,0 +1,11 @@ +--- +Checks: > + bugprone-*, + modernize-*, + performance-*, + readability-*, + -modernize-use-trailing-return-type, + -readability-identifier-length +WarningsAsErrors: '' +HeaderFilterRegex: '.*' +FormatStyle: file diff --git a/examples/10-io-uring/cpp/CMakeLists.txt b/examples/10-io-uring/cpp/CMakeLists.txt new file mode 100644 index 0000000..ff62606 --- /dev/null +++ b/examples/10-io-uring/cpp/CMakeLists.txt @@ -0,0 +1,8 @@ +cmake_minimum_required(VERSION 3.25) +project(fwatch CXX) + +set(CMAKE_CXX_STANDARD 23) +set(CMAKE_CXX_STANDARD_REQUIRED ON) +set(CMAKE_CXX_EXTENSIONS OFF) + +add_executable(app src/main.cpp) diff --git a/examples/10-io-uring/cpp/CMakePresets.json b/examples/10-io-uring/cpp/CMakePresets.json new file mode 100644 index 0000000..e31bd2f --- /dev/null +++ b/examples/10-io-uring/cpp/CMakePresets.json @@ -0,0 +1,62 @@ +{ + "version": 6, + "cmakeMinimumRequired": { "major": 3, "minor": 25, "patch": 0 }, + "configurePresets": [ + { + "name": "_base", + "hidden": true, + "generator": "Ninja", + "binaryDir": "${sourceDir}/build/${presetName}", + "cacheVariables": { + "CMAKE_EXPORT_COMPILE_COMMANDS": "ON" + } + }, + { + "name": "release", + "inherits": "_base", + "displayName": "Release (-O2 with debug info)", + "cacheVariables": { "CMAKE_BUILD_TYPE": "RelWithDebInfo" } + }, + { + "name": "debug", + "inherits": "_base", + "displayName": "Debug", + "cacheVariables": { "CMAKE_BUILD_TYPE": "Debug" } + }, + { + "name": "asan", + "inherits": "_base", + "displayName": "ASan + UBSan instrumented build", + "cacheVariables": { + "CMAKE_BUILD_TYPE": "Debug", + "CMAKE_CXX_FLAGS": "-O1 -g -fno-omit-frame-pointer -fsanitize=address,undefined", + "CMAKE_EXE_LINKER_FLAGS": "-fsanitize=address,undefined" + } + }, + { + "name": "release-clang", + "inherits": "release", + "displayName": "Release (clang)", + "cacheVariables": { + "CMAKE_C_COMPILER": "clang", + "CMAKE_CXX_COMPILER": "clang++" + } + }, + { + "name": "debug-clang", + "inherits": "debug", + "displayName": "Debug (clang)", + "cacheVariables": { + "CMAKE_C_COMPILER": "clang", + "CMAKE_CXX_COMPILER": "clang++" + } + } + ], + "buildPresets": [ + { "name": "release", "configurePreset": "release" }, + { "name": "debug", "configurePreset": "debug" }, + { "name": "asan", "configurePreset": "asan" }, + { "name": "release-clang", "configurePreset": "release-clang" }, + { "name": "debug-clang", "configurePreset": "debug-clang" } + ] +} diff --git a/examples/10-io-uring/cpp/demo.sh b/examples/10-io-uring/cpp/demo.sh new file mode 100755 index 0000000..6af6509 --- /dev/null +++ b/examples/10-io-uring/cpp/demo.sh @@ -0,0 +1,40 @@ +#!/usr/bin/env bash +set -euo pipefail + +cd "$(dirname "${BASH_SOURCE[0]}")" +REPO_ROOT="$(cd ../../.. && pwd)" +BIN=build/release/app + +build() { + cmake --preset release + cmake --build --preset release +} + +run() { + if [[ ! -x "$BIN" ]]; then + build + fi + if [[ -n "${TARGET:-}" ]]; then + "$REPO_ROOT/scripts/lab/deploy-to-vm.sh" "$TARGET" "$BIN" -- "$@" + else + "./$BIN" "$@" + fi +} + +case "${1:-}" in + build) + build + ;; + run) + shift + run "$@" + ;; + "") + build + run + ;; + *) + echo "usage: $0 [build|run [args]]" >&2 + exit 2 + ;; +esac diff --git a/examples/10-io-uring/cpp/src/main.cpp b/examples/10-io-uring/cpp/src/main.cpp new file mode 100644 index 0000000..61465e3 --- /dev/null +++ b/examples/10-io-uring/cpp/src/main.cpp @@ -0,0 +1,595 @@ +// fwatch v2 — batched sync engine (chapter 10: io_uring). +// +// Subcommands: +// fwatch scan DIR (v0: polling snapshot) +// fwatch watch DIR --timeout MS (v1: inotify events) +// fwatch sync SRCDIR DSTDIR [--engine rw|uring] (v2: batched copy) +// +// The uring engine is a deliberately small, self-contained io_uring layer +// built on raw syscall(2) wrappers: io_uring_setup(2) + io_uring_enter(2), +// with the SQ/CQ rings and the SQE array mapped via mmap(2). No liburing. +// Each file is copied by batching linked READ->WRITE SQE pairs (one pair per +// 128 KiB chunk, up to 8 pairs per io_uring_enter) at explicit offsets. + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include +#include +#include +#include + +namespace fs = std::filesystem; + +namespace { + +using std::expected; +using std::unexpected; + +[[nodiscard]] std::error_code errno_ec() { + return std::error_code{errno, std::system_category()}; +} + +// --- RAII wrappers --------------------------------------------------------- + +class Fd { +public: + Fd() = default; + explicit Fd(int fd) : fd_(fd) {} + ~Fd() { reset(); } + Fd(Fd&& other) noexcept : fd_(std::exchange(other.fd_, -1)) {} + Fd& operator=(Fd&& other) noexcept { + if (this != &other) { + reset(); + fd_ = std::exchange(other.fd_, -1); + } + return *this; + } + Fd(const Fd&) = delete; + Fd& operator=(const Fd&) = delete; + + [[nodiscard]] int get() const { return fd_; } + + [[nodiscard]] static expected open(const fs::path& path, int flags, + mode_t mode = 0) { + const int fd = ::open(path.c_str(), flags, mode); + if (fd < 0) { + return unexpected(errno_ec()); + } + return Fd{fd}; + } + +private: + void reset() { + if (fd_ >= 0) { + ::close(fd_); + fd_ = -1; + } + } + int fd_ = -1; +}; + +class Mmap { +public: + Mmap() = default; + ~Mmap() { reset(); } + Mmap(Mmap&& other) noexcept + : ptr_(std::exchange(other.ptr_, nullptr)), len_(std::exchange(other.len_, 0)) {} + Mmap& operator=(Mmap&& other) noexcept { + if (this != &other) { + reset(); + ptr_ = std::exchange(other.ptr_, nullptr); + len_ = std::exchange(other.len_, 0); + } + return *this; + } + Mmap(const Mmap&) = delete; + Mmap& operator=(const Mmap&) = delete; + + [[nodiscard]] static expected map(size_t len, int fd, + uint64_t offset) { + void* p = ::mmap(nullptr, len, PROT_READ | PROT_WRITE, MAP_SHARED | MAP_POPULATE, fd, + static_cast(offset)); + if (p == MAP_FAILED) { + return unexpected(errno_ec()); + } + Mmap m; + m.ptr_ = p; + m.len_ = len; + return m; + } + + template + [[nodiscard]] T* at(size_t off) const { + return reinterpret_cast(static_cast(ptr_) + off); + } + +private: + void reset() { + if (ptr_ != nullptr) { + ::munmap(ptr_, len_); + ptr_ = nullptr; + } + } + void* ptr_ = nullptr; + size_t len_ = 0; +}; + +// --- a small educational io_uring layer (no liburing) ---------------------- + +class Ring { +public: + [[nodiscard]] static expected, std::error_code> create( + unsigned entries) { + io_uring_params params{}; + const long fd = ::syscall(__NR_io_uring_setup, entries, ¶ms); // io_uring_setup(2) + if (fd < 0) { + return unexpected(errno_ec()); + } + auto ring = std::unique_ptr(new Ring{}); + ring->fd_ = Fd{static_cast(fd)}; + + // The SQ ring holds indices into the separately-mapped SQE array; the + // CQ ring holds the CQEs themselves. Modern kernels expose both rings + // through a single mapping (IORING_FEAT_SINGLE_MMAP). + size_t sq_len = params.sq_off.array + params.sq_entries * sizeof(unsigned); + size_t cq_len = params.cq_off.cqes + params.cq_entries * sizeof(io_uring_cqe); + const bool single_mmap = (params.features & IORING_FEAT_SINGLE_MMAP) != 0; + if (single_mmap) { + sq_len = cq_len = std::max(sq_len, cq_len); + } + + auto sq_map = Mmap::map(sq_len, ring->fd_.get(), IORING_OFF_SQ_RING); + if (!sq_map) { + return unexpected(sq_map.error()); + } + ring->sq_map_ = std::move(*sq_map); + + const Mmap* cq_view = &ring->sq_map_; + if (!single_mmap) { + auto cq_map = Mmap::map(cq_len, ring->fd_.get(), IORING_OFF_CQ_RING); + if (!cq_map) { + return unexpected(cq_map.error()); + } + ring->cq_map_ = std::move(*cq_map); + cq_view = &ring->cq_map_; + } + + auto sqe_map = Mmap::map(params.sq_entries * sizeof(io_uring_sqe), ring->fd_.get(), + IORING_OFF_SQES); + if (!sqe_map) { + return unexpected(sqe_map.error()); + } + ring->sqe_map_ = std::move(*sqe_map); + + ring->sq_head_ = ring->sq_map_.at(params.sq_off.head); + ring->sq_tail_ = ring->sq_map_.at(params.sq_off.tail); + ring->sq_mask_ = *ring->sq_map_.at(params.sq_off.ring_mask); + ring->sq_array_ = ring->sq_map_.at(params.sq_off.array); + ring->cq_head_ = cq_view->at(params.cq_off.head); + ring->cq_tail_ = cq_view->at(params.cq_off.tail); + ring->cq_mask_ = *cq_view->at(params.cq_off.ring_mask); + ring->cqes_ = cq_view->at(params.cq_off.cqes); + ring->sqes_ = ring->sqe_map_.at(0); + ring->local_tail_ = *ring->sq_tail_; + return ring; + } + + // Claim the next SQE slot (zeroed); nullptr when the SQ ring is full. + [[nodiscard]] io_uring_sqe* try_get_sqe() { + const unsigned head = std::atomic_ref(*sq_head_).load(std::memory_order_acquire); + if (local_tail_ - head > sq_mask_) { + return nullptr; + } + const unsigned idx = local_tail_ & sq_mask_; + io_uring_sqe* sqe = &sqes_[idx]; + std::memset(sqe, 0, sizeof(*sqe)); + sq_array_[idx] = idx; + ++local_tail_; + ++to_submit_; + return sqe; + } + + // Publish queued SQEs and wait for wait_nr completions in one syscall. + [[nodiscard]] expected submit_and_wait(unsigned wait_nr) { + std::atomic_ref(*sq_tail_).store(local_tail_, std::memory_order_release); + return enter(std::exchange(to_submit_, 0), wait_nr); + } + + // Pop one CQE if available. + [[nodiscard]] bool pop(io_uring_cqe& out) { + const unsigned head = *cq_head_; + const unsigned tail = std::atomic_ref(*cq_tail_).load(std::memory_order_acquire); + if (head == tail) { + return false; + } + out = cqes_[head & cq_mask_]; + std::atomic_ref(*cq_head_).store(head + 1, std::memory_order_release); + return true; + } + + // Consume exactly `need` CQEs, re-entering the kernel if a wait returned + // before all completions had arrived. + template + [[nodiscard]] expected drain(unsigned need, OnCqe&& on_cqe) { + io_uring_cqe cqe{}; + while (need > 0) { + while (need > 0 && pop(cqe)) { + on_cqe(cqe); + --need; + } + if (need == 0) { + break; + } + if (auto r = enter(0, need); !r) { + return unexpected(r.error()); + } + } + return {}; + } + +private: + Ring() = default; + + [[nodiscard]] expected enter(unsigned to_submit, unsigned wait_nr) { + while (true) { + const long r = ::syscall(__NR_io_uring_enter, fd_.get(), to_submit, wait_nr, + IORING_ENTER_GETEVENTS, nullptr, 0); // io_uring_enter(2) + if (r >= 0) { + return {}; + } + if (errno == EINTR) { + continue; // safe: the kernel caps submission at what is queued + } + return unexpected(errno_ec()); + } + } + + Fd fd_; + Mmap sq_map_; + Mmap cq_map_; // unused when IORING_FEAT_SINGLE_MMAP + Mmap sqe_map_; + unsigned* sq_head_ = nullptr; + unsigned* sq_tail_ = nullptr; + unsigned* sq_array_ = nullptr; + unsigned* cq_head_ = nullptr; + unsigned* cq_tail_ = nullptr; + io_uring_sqe* sqes_ = nullptr; + io_uring_cqe* cqes_ = nullptr; + unsigned sq_mask_ = 0; + unsigned cq_mask_ = 0; + unsigned local_tail_ = 0; + unsigned to_submit_ = 0; +}; + +// --- copy engines ---------------------------------------------------------- + +constexpr size_t kChunk = 128 * 1024; +constexpr unsigned kPairs = 8; // READ->WRITE pairs batched per submit + +[[nodiscard]] expected copy_file_rw(const fs::path& src, + const fs::path& dst) { + auto in = Fd::open(src, O_RDONLY | O_CLOEXEC); + if (!in) { + return unexpected(in.error()); + } + auto out = Fd::open(dst, O_WRONLY | O_CREAT | O_TRUNC | O_CLOEXEC, 0644); + if (!out) { + return unexpected(out.error()); + } + std::vector buf(kChunk); + uint64_t total = 0; + while (true) { + const ssize_t n = ::read(in->get(), buf.data(), buf.size()); + if (n < 0) { + if (errno == EINTR) { + continue; + } + return unexpected(errno_ec()); + } + if (n == 0) { + break; + } + size_t written = 0; + while (written < static_cast(n)) { + const ssize_t w = ::write(out->get(), buf.data() + written, + static_cast(n) - written); + if (w < 0) { + if (errno == EINTR) { + continue; + } + return unexpected(errno_ec()); + } + written += static_cast(w); + } + total += static_cast(n); + } + return total; +} + +[[nodiscard]] expected copy_file_uring(Ring& ring, const fs::path& src, + const fs::path& dst) { + auto in = Fd::open(src, O_RDONLY | O_CLOEXEC); + if (!in) { + return unexpected(in.error()); + } + auto out = Fd::open(dst, O_WRONLY | O_CREAT | O_TRUNC | O_CLOEXEC, 0644); + if (!out) { + return unexpected(out.error()); + } + struct stat st{}; + if (::fstat(in->get(), &st) != 0) { + return unexpected(errno_ec()); + } + const auto size = static_cast(st.st_size); + + std::array, kPairs> bufs; + for (auto& b : bufs) { + b.resize(kChunk); + } + + uint64_t off = 0; + while (off < size) { + std::array lens{}; + unsigned pairs = 0; + while (pairs < kPairs && off < size) { + const auto len = static_cast(std::min(kChunk, size - off)); + lens[pairs] = len; + io_uring_sqe* rd = ring.try_get_sqe(); + io_uring_sqe* wr = rd != nullptr ? ring.try_get_sqe() : nullptr; + if (rd == nullptr || wr == nullptr) { + return unexpected(std::make_error_code(std::errc::no_buffer_space)); + } + // READ this chunk into the pair's buffer... + rd->opcode = IORING_OP_READ; + rd->fd = in->get(); + rd->addr = reinterpret_cast(bufs[pairs].data()); + rd->len = len; + rd->off = off; + rd->flags = IOSQE_IO_LINK; // ...then, only if it fully succeeded, + rd->user_data = uint64_t{pairs} << 1U; + // ...WRITE the same buffer at the same offset in dst. + wr->opcode = IORING_OP_WRITE; + wr->fd = out->get(); + wr->addr = reinterpret_cast(bufs[pairs].data()); + wr->len = len; + wr->off = off; + wr->user_data = (uint64_t{pairs} << 1U) | 1U; + off += len; + ++pairs; + } + if (auto s = ring.submit_and_wait(2 * pairs); !s) { + return unexpected(s.error()); + } + bool short_io = false; + auto d = ring.drain(2 * pairs, [&](const io_uring_cqe& cqe) { + const auto slot = static_cast(cqe.user_data >> 1U); + if (cqe.res < 0 || static_cast(cqe.res) != lens[slot]) { + short_io = true; + } + }); + if (!d) { + return unexpected(d.error()); + } + if (short_io) { + return unexpected(std::make_error_code(std::errc::io_error)); + } + } + return size; +} + +// --- subcommands ----------------------------------------------------------- + +int usage() { + std::println(stderr, "usage: fwatch "); + std::println(stderr, " fwatch scan DIR"); + std::println(stderr, " fwatch watch DIR --timeout MS"); + std::println(stderr, " fwatch sync SRCDIR DSTDIR [--engine rw|uring]"); + return 2; +} + +int fail(std::string_view what, std::error_code ec) { + std::println(stderr, "fwatch: {}: {}", what, ec.message()); + return 1; +} + +int cmd_scan(const fs::path& dir) { + std::error_code ec; + uint64_t files = 0; + uint64_t bytes = 0; + fs::recursive_directory_iterator it(dir, fs::directory_options::none, ec); + if (ec) { + return fail(dir.string(), ec); + } + for (const fs::recursive_directory_iterator end; it != end; it.increment(ec)) { + if (ec) { + return fail(dir.string(), ec); + } + const auto st = it->symlink_status(ec); + if (ec) { + return fail(it->path().string(), ec); + } + if (fs::is_regular_file(st)) { + const auto sz = fs::file_size(it->path(), ec); + if (ec) { + return fail(it->path().string(), ec); + } + ++files; + bytes += sz; + } + } + std::println("scanned {} files {} bytes", files, bytes); + return 0; +} + +int cmd_watch(const fs::path& dir, long timeout_ms) { + Fd ifd{::inotify_init1(IN_NONBLOCK | IN_CLOEXEC)}; + if (ifd.get() < 0) { + return fail("inotify_init1", errno_ec()); + } + if (::inotify_add_watch(ifd.get(), dir.c_str(), IN_CREATE | IN_MODIFY | IN_DELETE) < 0) { + return fail(dir.string(), errno_ec()); + } + using clock = std::chrono::steady_clock; + const auto deadline = clock::now() + std::chrono::milliseconds(timeout_ms); + uint64_t events = 0; + alignas(inotify_event) std::array buf{}; + while (true) { + const auto remaining = + std::chrono::duration_cast(deadline - clock::now()) + .count(); + if (remaining <= 0) { + break; + } + pollfd pfd{.fd = ifd.get(), .events = POLLIN, .revents = 0}; + const int rc = ::poll(&pfd, 1, static_cast(remaining)); + if (rc < 0) { + if (errno == EINTR) { + continue; + } + return fail("poll", errno_ec()); + } + if (rc == 0) { + continue; // deadline check at the loop top ends the watch + } + const ssize_t n = ::read(ifd.get(), buf.data(), buf.size()); + if (n < 0) { + if (errno == EAGAIN || errno == EINTR) { + continue; + } + return fail("inotify read", errno_ec()); + } + for (ssize_t pos = 0; pos < n;) { + const auto* ev = reinterpret_cast(buf.data() + pos); + std::string_view kind; + if ((ev->mask & IN_CREATE) != 0) { + kind = "CREATE"; + } else if ((ev->mask & IN_MODIFY) != 0) { + kind = "MODIFY"; + } else if ((ev->mask & IN_DELETE) != 0) { + kind = "DELETE"; + } + if (!kind.empty() && ev->len > 0) { + std::println("event {} {}", kind, static_cast(ev->name)); + ++events; + } + pos += static_cast(sizeof(inotify_event) + ev->len); + } + } + std::println("watched {} events", events); + return 0; +} + +int cmd_sync(const fs::path& src, const fs::path& dst, std::string_view engine) { + using clock = std::chrono::steady_clock; + const auto t0 = clock::now(); + + std::error_code ec; + if (!fs::is_directory(src, ec) || ec) { + return fail(src.string(), std::make_error_code(std::errc::not_a_directory)); + } + fs::create_directories(dst, ec); + if (ec) { + return fail(dst.string(), ec); + } + + std::unique_ptr ring; + if (engine == "uring") { + auto r = Ring::create(64); + if (!r) { + return fail("io_uring_setup", r.error()); + } + ring = std::move(*r); + } + + uint64_t files = 0; + uint64_t bytes = 0; + fs::recursive_directory_iterator it(src, fs::directory_options::none, ec); + if (ec) { + return fail(src.string(), ec); + } + for (const fs::recursive_directory_iterator end; it != end; it.increment(ec)) { + if (ec) { + return fail(src.string(), ec); + } + const fs::path target = dst / it->path().lexically_relative(src); + const auto st = it->symlink_status(ec); + if (ec) { + return fail(it->path().string(), ec); + } + if (fs::is_directory(st)) { + fs::create_directories(target, ec); + if (ec) { + return fail(target.string(), ec); + } + } else if (fs::is_regular_file(st)) { + const auto copied = ring != nullptr ? copy_file_uring(*ring, it->path(), target) + : copy_file_rw(it->path(), target); + if (!copied) { + return fail(it->path().string(), copied.error()); + } + ++files; + bytes += *copied; + } + } + + const auto ms = + std::chrono::duration_cast(clock::now() - t0).count(); + std::println("synced {} files {} bytes engine={} ms={}", files, bytes, engine, ms); + return 0; +} + +} // namespace + +int main(int argc, char** argv) { + const std::vector args(argv + 1, argv + argc); + if (args.empty()) { + return usage(); + } + const std::string_view cmd = args[0]; + + if (cmd == "scan" && args.size() == 2) { + return cmd_scan(args[1]); + } + if (cmd == "watch" && args.size() == 4 && args[2] == "--timeout") { + long timeout_ms = 0; + const auto [ptr, err] = + std::from_chars(args[3].data(), args[3].data() + args[3].size(), timeout_ms); + if (err != std::errc{} || ptr != args[3].data() + args[3].size() || timeout_ms < 0) { + return usage(); + } + return cmd_watch(args[1], timeout_ms); + } + if (cmd == "sync" && (args.size() == 3 || args.size() == 5)) { + std::string_view engine = "rw"; + if (args.size() == 5) { + if (args[3] != "--engine") { + return usage(); + } + engine = args[4]; + } + if (engine != "rw" && engine != "uring") { + return usage(); + } + return cmd_sync(args[1], args[2], engine); + } + return usage(); +} diff --git a/examples/10-io-uring/demo.sh b/examples/10-io-uring/demo.sh new file mode 100755 index 0000000..afbbe4c --- /dev/null +++ b/examples/10-io-uring/demo.sh @@ -0,0 +1,23 @@ +#!/usr/bin/env bash +set -euo pipefail +cd "$(dirname "$0")" + +langs=(cpp go rust) + +case "${1:-all}" in + cpp|go|rust) + lang="$1" + shift + exec "./$lang/demo.sh" "$@" + ;; + all) + for lang in "${langs[@]}"; do "./$lang/demo.sh"; done + ;; + build) + for lang in "${langs[@]}"; do "./$lang/demo.sh" build; done + ;; + *) + echo "usage: $0 [cpp|go|rust|all|build] [args...]" >&2 + exit 2 + ;; +esac diff --git a/examples/10-io-uring/go/demo.sh b/examples/10-io-uring/go/demo.sh new file mode 100755 index 0000000..28606df --- /dev/null +++ b/examples/10-io-uring/go/demo.sh @@ -0,0 +1,23 @@ +#!/usr/bin/env bash +set -euo pipefail +cd "$(dirname "$0")" +REPO_ROOT="$(cd ../../.. && pwd)" +BIN=bin/app + +build() { + go build -o "$BIN" . +} + +run() { + if [[ -n "${TARGET:-}" ]]; then + exec "$REPO_ROOT/scripts/lab/deploy-to-vm.sh" "$TARGET" "$BIN" -- "$@" + fi + exec "./$BIN" "$@" +} + +case "${1:-}" in + build) build ;; + run) shift; run "$@" ;; + "") build; run ;; + *) echo "usage: $0 [build|run [args...]]" >&2; exit 2 ;; +esac diff --git a/examples/10-io-uring/go/go.mod b/examples/10-io-uring/go/go.mod new file mode 100644 index 0000000..3f18c95 --- /dev/null +++ b/examples/10-io-uring/go/go.mod @@ -0,0 +1,7 @@ +module github.com/patterncatalyst/linux-systems-programming/examples/10-io-uring/go + +go 1.26 + +toolchain go1.26.5 + +require golang.org/x/sys v0.47.0 diff --git a/examples/10-io-uring/go/go.sum b/examples/10-io-uring/go/go.sum new file mode 100644 index 0000000..37ee2d4 --- /dev/null +++ b/examples/10-io-uring/go/go.sum @@ -0,0 +1,2 @@ +golang.org/x/sys v0.47.0 h1:o7XGOvZQCADBQQ4Y7VNq2dRWQR7JmOUW8Kxx4ZsNgWs= +golang.org/x/sys v0.47.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw= diff --git a/examples/10-io-uring/go/main.go b/examples/10-io-uring/go/main.go new file mode 100644 index 0000000..25bb54b --- /dev/null +++ b/examples/10-io-uring/go/main.go @@ -0,0 +1,269 @@ +// fwatch v2 — batched sync engine (chapter 10: io_uring). +// +// Subcommands: +// +// fwatch scan DIR (v0: polling snapshot) +// fwatch watch DIR --timeout MS (v1: inotify events) +// fwatch sync SRCDIR DSTDIR [--engine rw|uring] (v2: batched copy) +// +// Go has no supported io_uring story: the runtime already multiplexes +// goroutines over epoll in the netpoller, and the io_uring proposal +// (golang/go#31908) has been on hold for years. So the uring engine is +// deliberately absent here — "fwatch sync ... --engine uring" reports it +// unsupported and exits 64. That asymmetry IS the chapter's point. +package main + +import ( + "errors" + "fmt" + "io" + "io/fs" + "os" + "path/filepath" + "strconv" + "strings" + "time" + "unsafe" + + "golang.org/x/sys/unix" +) + +const chunk = 128 * 1024 + +func usage() int { + fmt.Fprintln(os.Stderr, "usage: fwatch ") + fmt.Fprintln(os.Stderr, " fwatch scan DIR") + fmt.Fprintln(os.Stderr, " fwatch watch DIR --timeout MS") + fmt.Fprintln(os.Stderr, " fwatch sync SRCDIR DSTDIR [--engine rw|uring]") + return 2 +} + +// walk visits every entry under src depth-first and reports its path and type. +func walk(src string, visit func(path string, d fs.DirEntry) error) error { + return filepath.WalkDir(src, func(path string, d fs.DirEntry, err error) error { + if err != nil { + return fmt.Errorf("walk %s: %w", path, err) + } + if path == src { + return nil + } + return visit(path, d) + }) +} + +func cmdScan(dir string) int { + var files, bytes uint64 + err := walk(dir, func(path string, d fs.DirEntry) error { + if !d.Type().IsRegular() { + return nil + } + info, err := d.Info() + if err != nil { + return fmt.Errorf("stat %s: %w", path, err) + } + files++ + bytes += uint64(info.Size()) + return nil + }) + if err != nil { + fmt.Fprintf(os.Stderr, "fwatch: %v\n", err) + return 1 + } + fmt.Printf("scanned %d files %d bytes\n", files, bytes) + return 0 +} + +// copyFileRW copies src to dst with a plain read/write loop. +func copyFileRW(src, dst string) (uint64, error) { + in, err := os.Open(src) + if err != nil { + return 0, fmt.Errorf("open %s: %w", src, err) + } + defer in.Close() + out, err := os.OpenFile(dst, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0o644) + if err != nil { + return 0, fmt.Errorf("create %s: %w", dst, err) + } + defer out.Close() + + buf := make([]byte, chunk) + var total uint64 + for { + n, err := in.Read(buf) + if n > 0 { + if _, werr := out.Write(buf[:n]); werr != nil { + return total, fmt.Errorf("write %s: %w", dst, werr) + } + total += uint64(n) + } + if errors.Is(err, io.EOF) { + break + } + if err != nil { + return total, fmt.Errorf("read %s: %w", src, err) + } + } + if err := out.Close(); err != nil { + return total, fmt.Errorf("close %s: %w", dst, err) + } + return total, nil +} + +func cmdSync(src, dst, engine string) int { + if engine == "uring" { + fmt.Fprintln(os.Stderr, "engine=uring: unsupported in Go (see chapter 10)") + return 64 + } + start := time.Now() + if err := os.MkdirAll(dst, 0o755); err != nil { + fmt.Fprintf(os.Stderr, "fwatch: %v\n", err) + return 1 + } + var files, bytes uint64 + err := walk(src, func(path string, d fs.DirEntry) error { + rel, err := filepath.Rel(src, path) + if err != nil { + return fmt.Errorf("rel %s: %w", path, err) + } + target := filepath.Join(dst, rel) + switch { + case d.IsDir(): + return os.MkdirAll(target, 0o755) + case d.Type().IsRegular(): + n, err := copyFileRW(path, target) + if err != nil { + return err + } + files++ + bytes += n + } + return nil + }) + if err != nil { + fmt.Fprintf(os.Stderr, "fwatch: %v\n", err) + return 1 + } + ms := time.Since(start).Milliseconds() + fmt.Printf("synced %d files %d bytes engine=%s ms=%d\n", files, bytes, engine, ms) + return 0 +} + +type inotifyEvent struct { + kind string + name string +} + +func kindOf(mask uint32) string { + switch { + case mask&unix.IN_CREATE != 0: + return "CREATE" + case mask&unix.IN_MODIFY != 0: + return "MODIFY" + case mask&unix.IN_DELETE != 0: + return "DELETE" + } + return "" +} + +// readEvents parses one inotify read(2) buffer into events with names. +func readEvents(buf []byte) []inotifyEvent { + var out []inotifyEvent + for off := 0; off+unix.SizeofInotifyEvent <= len(buf); { + raw := (*unix.InotifyEvent)(unsafe.Pointer(&buf[off])) + nameLen := int(raw.Len) + name := "" + if nameLen > 0 { + b := buf[off+unix.SizeofInotifyEvent : off+unix.SizeofInotifyEvent+nameLen] + name = strings.TrimRight(string(b), "\x00") + } + if kind := kindOf(raw.Mask); kind != "" && name != "" { + out = append(out, inotifyEvent{kind: kind, name: name}) + } + off += unix.SizeofInotifyEvent + nameLen + } + return out +} + +func cmdWatch(dir string, timeoutMS int) int { + fd, err := unix.InotifyInit1(unix.IN_CLOEXEC) + if err != nil { + fmt.Fprintf(os.Stderr, "fwatch: inotify_init1: %v\n", err) + return 1 + } + defer unix.Close(fd) + if _, err := unix.InotifyAddWatch(fd, dir, + unix.IN_CREATE|unix.IN_MODIFY|unix.IN_DELETE); err != nil { + fmt.Fprintf(os.Stderr, "fwatch: %s: %v\n", dir, err) + return 1 + } + + events := make(chan inotifyEvent, 64) + errs := make(chan error, 1) + // Reader goroutine: blocking read(2) on the inotify fd, events flow out + // over a channel; process exit reclaims it once the deadline fires. + go func() { + buf := make([]byte, 16*1024) + for { + n, err := unix.Read(fd, buf) + if err != nil { + if errors.Is(err, unix.EINTR) { + continue + } + errs <- fmt.Errorf("inotify read: %w", err) + return + } + for _, ev := range readEvents(buf[:n]) { + events <- ev + } + } + }() + + deadline := time.After(time.Duration(timeoutMS) * time.Millisecond) + var count uint64 + for { + select { + case ev := <-events: + fmt.Printf("event %s %s\n", ev.kind, ev.name) + count++ + case err := <-errs: + fmt.Fprintf(os.Stderr, "fwatch: %v\n", err) + return 1 + case <-deadline: + fmt.Printf("watched %d events\n", count) + return 0 + } + } +} + +func run(args []string) int { + if len(args) == 0 { + return usage() + } + switch cmd := args[0]; { + case cmd == "scan" && len(args) == 2: + return cmdScan(args[1]) + case cmd == "watch" && len(args) == 4 && args[2] == "--timeout": + timeoutMS, err := strconv.Atoi(args[3]) + if err != nil || timeoutMS < 0 { + return usage() + } + return cmdWatch(args[1], timeoutMS) + case cmd == "sync" && (len(args) == 3 || len(args) == 5): + engine := "rw" + if len(args) == 5 { + if args[3] != "--engine" { + return usage() + } + engine = args[4] + } + if engine != "rw" && engine != "uring" { + return usage() + } + return cmdSync(args[1], args[2], engine) + } + return usage() +} + +func main() { + os.Exit(run(os.Args[1:])) +} diff --git a/examples/10-io-uring/rust/Cargo.lock b/examples/10-io-uring/rust/Cargo.lock new file mode 100644 index 0000000..0b1ced6 --- /dev/null +++ b/examples/10-io-uring/rust/Cargo.lock @@ -0,0 +1,65 @@ +# This file is automatically @generated by Cargo. +# It is not intended for manual editing. +version = 4 + +[[package]] +name = "anyhow" +version = "1.0.104" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "330a5ed07fa54e4702c9d6c4174f74427fc0ef6e214bbd677ae50a5099946470" + +[[package]] +name = "bitflags" +version = "2.13.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b588b76d00fde79687d7646a9b5bdf3cc0f655e0bbd080335a95d7e96f3587da" + +[[package]] +name = "cfg-if" +version = "1.0.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9330f8b2ff13f34540b44e946ef35111825727b38d33286ef986142615121801" + +[[package]] +name = "cfg_aliases" +version = "0.2.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f079e83a288787bcd14a6aea84cee5c87a67c5a3e660c30f557a3d24761b3527" + +[[package]] +name = "fwatch" +version = "2.0.0" +dependencies = [ + "anyhow", + "io-uring", + "nix", +] + +[[package]] +name = "io-uring" +version = "0.7.13" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9080b15e63775b9a2ac7dca720f7050a8b955e092ea0f6020a4a80f69998cdc0" +dependencies = [ + "bitflags", + "cfg-if", + "libc", +] + +[[package]] +name = "libc" +version = "0.2.186" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "68ab91017fe16c622486840e4c83c9a37afeff978bd239b5293d61ece587de66" + +[[package]] +name = "nix" +version = "0.31.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cf20d2fde8ff38632c426f1165ed7436270b44f199fc55284c38276f9db47c3d" +dependencies = [ + "bitflags", + "cfg-if", + "cfg_aliases", + "libc", +] diff --git a/examples/10-io-uring/rust/Cargo.toml b/examples/10-io-uring/rust/Cargo.toml new file mode 100644 index 0000000..7c0268c --- /dev/null +++ b/examples/10-io-uring/rust/Cargo.toml @@ -0,0 +1,13 @@ +[package] +name = "fwatch" +version = "2.0.0" +edition = "2024" + +[[bin]] +name = "app" +path = "src/main.rs" + +[dependencies] +anyhow = "1.0.104" +io-uring = "0.7.13" +nix = { version = "0.31.3", features = ["inotify", "poll"] } diff --git a/examples/10-io-uring/rust/demo.sh b/examples/10-io-uring/rust/demo.sh new file mode 100755 index 0000000..220feea --- /dev/null +++ b/examples/10-io-uring/rust/demo.sh @@ -0,0 +1,26 @@ +#!/usr/bin/env bash +set -euo pipefail +cd "$(dirname "${BASH_SOURCE[0]}")" + +BIN=target/release/app + +build() { + cargo build --release +} + +run() { + if [[ -n "${TARGET:-}" ]]; then + local repo_root + repo_root=$(cd ../../.. && pwd) + "$repo_root/scripts/lab/deploy-to-vm.sh" "$TARGET" "$BIN" -- "$@" + else + "./$BIN" "$@" + fi +} + +case "${1:-}" in + build) build ;; + run) shift; run "$@" ;; + "") build; run ;; + *) echo "usage: $0 [build|run [args...]]" >&2; exit 2 ;; +esac diff --git a/examples/10-io-uring/rust/rust-toolchain.toml b/examples/10-io-uring/rust/rust-toolchain.toml new file mode 100644 index 0000000..4099f1a --- /dev/null +++ b/examples/10-io-uring/rust/rust-toolchain.toml @@ -0,0 +1,3 @@ +[toolchain] +channel = "1.97.1" +components = ["rustfmt", "clippy", "rust-src"] diff --git a/examples/10-io-uring/rust/src/main.rs b/examples/10-io-uring/rust/src/main.rs new file mode 100644 index 0000000..c0a66f3 --- /dev/null +++ b/examples/10-io-uring/rust/src/main.rs @@ -0,0 +1,275 @@ +// fwatch v2 — batched sync engine (chapter 10: io_uring). +// +// Subcommands: +// fwatch scan DIR (v0: polling snapshot) +// fwatch watch DIR --timeout MS (v1: inotify events) +// fwatch sync SRCDIR DSTDIR [--engine rw|uring] (v2: batched copy) +// +// The uring engine uses the io-uring crate: each file is copied by batching +// linked READ->WRITE SQE pairs (one pair per 128 KiB chunk, up to 8 pairs per +// submit_and_wait) at explicit offsets. + +use std::fs::{self, File}; +use std::io::{Read, Write}; +use std::os::fd::{AsFd, AsRawFd}; +use std::path::{Path, PathBuf}; +use std::process::ExitCode; +use std::time::{Duration, Instant}; + +use anyhow::{Context, Result, anyhow, bail}; +use io_uring::{IoUring, opcode, squeue, types}; +use nix::errno::Errno; +use nix::poll::{PollFd, PollFlags, PollTimeout, poll}; +use nix::sys::inotify::{AddWatchFlags, InitFlags, Inotify}; + +const CHUNK: u64 = 128 * 1024; +const PAIRS: usize = 8; // READ->WRITE pairs batched per submit + +fn usage() -> ExitCode { + eprintln!("usage: fwatch "); + eprintln!(" fwatch scan DIR"); + eprintln!(" fwatch watch DIR --timeout MS"); + eprintln!(" fwatch sync SRCDIR DSTDIR [--engine rw|uring]"); + ExitCode::from(2) +} + +/// Recursively collect entries below `root` (the root itself excluded). +/// Symlinks and other special files are recorded as neither dir nor regular. +fn collect(root: &Path, dir: &Path, out: &mut Vec<(PathBuf, fs::Metadata)>) -> Result<()> { + for entry in fs::read_dir(dir).with_context(|| format!("read_dir {}", dir.display()))? { + let entry = entry.with_context(|| format!("read_dir {}", dir.display()))?; + let path = entry.path(); + let meta = fs::symlink_metadata(&path) + .with_context(|| format!("stat {}", path.display()))?; + let is_dir = meta.is_dir(); + out.push((path.clone(), meta)); + if is_dir { + collect(root, &path, out)?; + } + } + Ok(()) +} + +fn cmd_scan(dir: &Path) -> Result<()> { + let mut entries = Vec::new(); + collect(dir, dir, &mut entries)?; + let (mut files, mut bytes) = (0u64, 0u64); + for (_, meta) in &entries { + if meta.is_file() { + files += 1; + bytes += meta.len(); + } + } + println!("scanned {files} files {bytes} bytes"); + Ok(()) +} + +/// Plain read/write loop copy. +fn copy_file_rw(src: &Path, dst: &Path) -> Result { + let mut input = File::open(src).with_context(|| format!("open {}", src.display()))?; + let mut output = File::create(dst).with_context(|| format!("create {}", dst.display()))?; + let mut buf = vec![0u8; CHUNK as usize]; + let mut total = 0u64; + loop { + let n = input + .read(&mut buf) + .with_context(|| format!("read {}", src.display()))?; + if n == 0 { + break; + } + output + .write_all(&buf[..n]) + .with_context(|| format!("write {}", dst.display()))?; + total += n as u64; + } + Ok(total) +} + +/// io_uring copy: batches of linked READ->WRITE pairs at explicit offsets. +fn copy_file_uring(ring: &mut IoUring, src: &Path, dst: &Path) -> Result { + let input = File::open(src).with_context(|| format!("open {}", src.display()))?; + let output = File::create(dst).with_context(|| format!("create {}", dst.display()))?; + let size = input.metadata()?.len(); + + let mut bufs = vec![vec![0u8; CHUNK as usize]; PAIRS]; + let mut off = 0u64; + while off < size { + let mut lens = [0u32; PAIRS]; + let mut pairs = 0usize; + { + let mut sq = ring.submission(); + while pairs < PAIRS && off < size { + let len = CHUNK.min(size - off) as u32; + lens[pairs] = len; + let buf = bufs[pairs].as_mut_ptr(); + // READ this chunk into the pair's buffer, then — only if it + // fully succeeded (IO_LINK) — WRITE it at the same offset. + let read = opcode::Read::new(types::Fd(input.as_raw_fd()), buf, len) + .offset(off) + .build() + .flags(squeue::Flags::IO_LINK) + .user_data((pairs as u64) << 1); + let write = opcode::Write::new(types::Fd(output.as_raw_fd()), buf, len) + .offset(off) + .build() + .user_data(((pairs as u64) << 1) | 1); + // SAFETY: the buffers and fds outlive the submissions; we wait + // for all completions before the next loop iteration. + unsafe { + sq.push(&read).map_err(|_| anyhow!("submission queue full"))?; + sq.push(&write).map_err(|_| anyhow!("submission queue full"))?; + } + off += u64::from(len); + pairs += 1; + } + } // drop the SQ view: syncs the tail before entering the kernel + ring.submit_and_wait(2 * pairs) + .context("io_uring_enter")?; + let mut seen = 0usize; + while seen < 2 * pairs { + for cqe in ring.completion() { + let slot = (cqe.user_data() >> 1) as usize; + if cqe.result() < 0 || cqe.result() as u32 != lens[slot] { + bail!( + "io_uring short op on {}: got {} want {}", + src.display(), + cqe.result(), + lens[slot] + ); + } + seen += 1; + } + if seen < 2 * pairs { + ring.submit_and_wait(2 * pairs - seen).context("io_uring_enter")?; + } + } + } + Ok(size) +} + +fn cmd_sync(src: &Path, dst: &Path, engine: &str) -> Result<()> { + let start = Instant::now(); + if !src.is_dir() { + bail!("{}: not a directory", src.display()); + } + fs::create_dir_all(dst).with_context(|| format!("mkdir {}", dst.display()))?; + + let mut ring = if engine == "uring" { + Some(IoUring::new(64).context("io_uring_setup")?) + } else { + None + }; + + let mut entries = Vec::new(); + collect(src, src, &mut entries)?; + let (mut files, mut bytes) = (0u64, 0u64); + for (path, meta) in &entries { + let target = dst.join(path.strip_prefix(src)?); + if meta.is_dir() { + fs::create_dir_all(&target) + .with_context(|| format!("mkdir {}", target.display()))?; + } else if meta.is_file() { + let copied = match ring.as_mut() { + Some(ring) => copy_file_uring(ring, path, &target)?, + None => copy_file_rw(path, &target)?, + }; + files += 1; + bytes += copied; + } + } + + let ms = start.elapsed().as_millis(); + println!("synced {files} files {bytes} bytes engine={engine} ms={ms}"); + Ok(()) +} + +fn cmd_watch(dir: &Path, timeout_ms: u64) -> Result<()> { + let inotify = Inotify::init(InitFlags::IN_NONBLOCK | InitFlags::IN_CLOEXEC) + .context("inotify_init1")?; + inotify + .add_watch( + dir, + AddWatchFlags::IN_CREATE | AddWatchFlags::IN_MODIFY | AddWatchFlags::IN_DELETE, + ) + .with_context(|| format!("watch {}", dir.display()))?; + + let deadline = Instant::now() + Duration::from_millis(timeout_ms); + let mut count = 0u64; + loop { + let now = Instant::now(); + if now >= deadline { + break; + } + let remaining = (deadline - now).as_millis().min(i32::MAX as u128) as i32; + let mut fds = [PollFd::new(inotify.as_fd(), PollFlags::POLLIN)]; + match poll( + &mut fds, + PollTimeout::try_from(remaining).unwrap_or(PollTimeout::MAX), + ) { + Ok(0) => continue, // deadline check at the loop top ends the watch + Ok(_) => {} + Err(Errno::EINTR) => continue, + Err(e) => return Err(e).context("poll"), + } + match inotify.read_events() { + Ok(events) => { + for ev in events { + let kind = if ev.mask.contains(AddWatchFlags::IN_CREATE) { + "CREATE" + } else if ev.mask.contains(AddWatchFlags::IN_MODIFY) { + "MODIFY" + } else if ev.mask.contains(AddWatchFlags::IN_DELETE) { + "DELETE" + } else { + continue; + }; + if let Some(name) = ev.name { + println!("event {} {}", kind, name.to_string_lossy()); + count += 1; + } + } + } + Err(Errno::EAGAIN) => continue, + Err(e) => return Err(e).context("inotify read"), + } + } + println!("watched {count} events"); + Ok(()) +} + +fn dispatch(args: &[String]) -> Option> { + match args.first().map(String::as_str) { + Some("scan") if args.len() == 2 => Some(cmd_scan(Path::new(&args[1]))), + Some("watch") if args.len() == 4 && args[2] == "--timeout" => { + let timeout_ms: u64 = args[3].parse().ok()?; + Some(cmd_watch(Path::new(&args[1]), timeout_ms)) + } + Some("sync") if args.len() == 3 || args.len() == 5 => { + let engine = if args.len() == 5 { + if args[3] != "--engine" { + return None; + } + args[4].as_str() + } else { + "rw" + }; + if engine != "rw" && engine != "uring" { + return None; + } + Some(cmd_sync(Path::new(&args[1]), Path::new(&args[2]), engine)) + } + _ => None, + } +} + +fn main() -> ExitCode { + let args: Vec = std::env::args().skip(1).collect(); + match dispatch(&args) { + None => usage(), + Some(Ok(())) => ExitCode::SUCCESS, + Some(Err(err)) => { + eprintln!("fwatch: {err:#}"); + ExitCode::FAILURE + } + } +} diff --git a/examples/10-io-uring/verify.lua b/examples/10-io-uring/verify.lua new file mode 100644 index 0000000..da9be6f --- /dev/null +++ b/examples/10-io-uring/verify.lua @@ -0,0 +1,82 @@ +-- Verify fwatch v2: build a fixture tree, then assert observable behavior of +-- every subcommand — exact scan/sync counters, byte-equal copies (diff -r), +-- inotify event lines from watch, and the Go uring refusal (exit 64). +-- Invoked from the example directory with LSP_LANG (cpp|go|rust) and EXAMPLE_DIR set. + +local script_dir = arg[0]:match("(.*/)") or "./" +local repo_root = os.getenv("REPO_ROOT") or (script_dir .. "../..") +package.path = package.path .. ";" .. repo_root .. "/scripts/lib/?.lua" +local checks = require("checks") + +local lang = os.getenv("LSP_LANG") +if lang ~= "cpp" and lang ~= "go" and lang ~= "rust" then + checks.skip("LSP_LANG must be cpp|go|rust (got " .. tostring(lang) .. ")") +end + +local function demo(args) + return checks.run("./demo.sh " .. lang .. " run " .. args) +end + +-- Fixture: 5 regular files (1348634 bytes total) across nested dirs, plus an +-- empty directory and an empty file. Sizes straddle the engines' 128 KiB +-- chunk: b.bin needs 3 chunks, d.bin needs 9 (i.e. two 8-pair uring batches). +local ws = checks.run("mktemp -d").out:gsub("%s+$", "") +assert(ws:match("^/"), "mktemp -d must yield an absolute path") +local src = ws .. "/src" +checks.run(string.format([[ + mkdir -p %s/sub/deep %s/emptydir + printf 'hello fwatch\n' > %s/a.txt + head -c 300000 /dev/urandom > %s/sub/b.bin + head -c 1048581 /dev/urandom > %s/sub/deep/d.bin + printf '0123456789012345678901234567890123456789' > %s/sub/c.txt + : > %s/empty.dat +]], src, src, src, src, src, src, src)) + +-- No arguments: usage on stderr, exit 2. +local r = demo("") +checks.expect_exit(r, 2, lang .. ": bare invocation exits 2") +checks.expect_match(r.out, "usage: fwatch", lang .. ": prints usage") + +-- v0: scan reports the exact file and byte totals. +r = demo("scan " .. src) +checks.expect_exit(r, 0, lang .. ": scan exits 0") +checks.expect_match(r.out, "scanned 5 files 1348634 bytes", lang .. ": scan totals") + +-- v2: sync with the rw engine — counters plus a byte-identical tree. +local dst_rw = ws .. "/dst-rw" +r = demo("sync " .. src .. " " .. dst_rw .. " --engine rw") +checks.expect_exit(r, 0, lang .. ": sync rw exits 0") +checks.expect_match(r.out, "synced 5 files 1348634 bytes engine=rw ms=%d+", + lang .. ": sync rw summary") +r = checks.run("diff -r " .. src .. " " .. dst_rw) +checks.expect_exit(r, 0, lang .. ": rw copy is byte-equal (diff -r)") + +-- v2: the uring engine. Go refuses by design (exit 64); C++ (raw syscalls + +-- mmap'd rings) and Rust (io-uring crate) must produce an identical tree. +local dst_uring = ws .. "/dst-uring" +r = demo("sync " .. src .. " " .. dst_uring .. " --engine uring") +if lang == "go" then + checks.expect_exit(r, 64, "go: engine=uring exits 64") + checks.expect_match(r.out, "engine=uring: unsupported in Go %(see chapter 10%)", + "go: uring refusal message") +else + checks.expect_exit(r, 0, lang .. ": sync uring exits 0") + checks.expect_match(r.out, "synced 5 files 1348634 bytes engine=uring ms=%d+", + lang .. ": sync uring summary") + r = checks.run("diff -r " .. src .. " " .. dst_uring) + checks.expect_exit(r, 0, lang .. ": uring copy is byte-equal (diff -r)") +end + +-- v1: watch emits inotify event lines. Create a file 0.4 s into the watch; +-- creating-then-writing yields CREATE (and usually MODIFY) events. +local watched = ws .. "/watched" +checks.run("mkdir -p " .. watched) +r = checks.run(string.format( + "( sleep 0.4; printf x > %s/newfile.txt ) & ./demo.sh %s run watch %s --timeout 1500", + watched, lang, watched)) +checks.expect_exit(r, 0, lang .. ": watch exits 0 after its timeout") +checks.expect_match(r.out, "event CREATE newfile%.txt", lang .. ": CREATE event line") +checks.expect_match(r.out, "watched %d+ events", lang .. ": watch summary line") + +checks.run("rm -rf " .. ws) +checks.finish() diff --git a/examples/manifest.yaml b/examples/manifest.yaml index 109bb1c..bb84d69 100644 --- a/examples/manifest.yaml +++ b/examples/manifest.yaml @@ -17,3 +17,31 @@ examples: langs: [cpp, go, rust] mode: local timeout: 180 + - id: 04-syscalls-and-the-abi + langs: [cpp, go, rust] + mode: local + timeout: 180 + - id: 05-errors-three-ways + langs: [cpp, go, rust] + mode: local + timeout: 180 + - id: 06-concurrency-three-ways + langs: [cpp, go, rust] + mode: local + timeout: 240 + - id: 07-fds-raii-and-the-vfs + langs: [cpp, go, rust] + mode: local + timeout: 180 + - id: 08-page-cache-and-durability + langs: [cpp, go, rust] + mode: local + timeout: 300 + - id: 09-event-loops-epoll-inotify + langs: [cpp, go, rust] + mode: local + timeout: 240 + - id: 10-io-uring + langs: [cpp, go, rust] + mode: local + timeout: 300 diff --git a/scripts/check-host.sh b/scripts/check-host.sh index b57e7a6..1d1aa6f 100755 --- a/scripts/check-host.sh +++ b/scripts/check-host.sh @@ -213,6 +213,28 @@ for tool in git gh; do fi done +# ── Debug + observation tooling (Parts 1-2 cross-checks, Part 8 debugging) ─ +for pair in "gdb:sudo dnf install -y gdb" \ + "valgrind:sudo dnf install -y valgrind" \ + "perf:sudo dnf install -y perf" \ + "strace:sudo dnf install -y strace" \ + "ltrace:sudo dnf install -y ltrace"; do + tool="${pair%%:*}"; hint="${pair#*:}" + if command -v "$tool" >/dev/null 2>&1; then + record ok "$tool" "$("$tool" --version 2>/dev/null | head -1 | grep -oE '[0-9]+\.[0-9]+(\.[0-9]+)?' | head -1)" + else + record fail "$tool" "not installed" "$hint" + fi +done + +# Sanitizer runtimes — the asan/tsan CMake presets cannot link without them. +if ls /usr/lib64/libasan.so.* >/dev/null 2>&1 && ls /usr/lib64/libtsan.so.* >/dev/null 2>&1; then + record ok "sanitizer runtimes" "libasan + libtsan present" +else + record fail "sanitizer runtimes" "libasan/libtsan missing" \ + "sudo dnf install -y libasan libubsan libtsan liblsan" +fi + # ── Soft requirements (warn only) ──────────────────────────────────────── # Conan 2.x — only some C++ demos use it. if command -v conan >/dev/null 2>&1; then @@ -227,14 +249,6 @@ else record warn "conan 2.x" "not installed" "pip install --user 'conan>=2.0,<3.0'" fi -# ruby + bundler — only for previewing the Jekyll site locally. -if command -v ruby >/dev/null 2>&1 && command -v bundler >/dev/null 2>&1; then - record ok "ruby + bundler" "$(ruby --version 2>/dev/null | grep -oE '[0-9]+\.[0-9]+\.[0-9]+' | head -1)" -else - record warn "ruby + bundler" "not installed" \ - "sudo dnf install -y ruby rubygem-bundler (local Jekyll preview only)" -fi - # Go linters / debugger — used by CI; nice to have locally. for pair in "golangci-lint:see golangci-lint.run/docs/welcome/install" \ "staticcheck:go install honnef.co/go/tools/cmd/staticcheck@latest" \