From 1d4f5bc49ad4687c13468f2df1bae8a568cd5897 Mon Sep 17 00:00:00 2001 From: PatternCatalyst Date: Sun, 26 Jul 2026 11:37:24 -0400 Subject: [PATCH] =?UTF-8?q?docs(r12):=20Rust=20macros=20for=20systems=20?= =?UTF-8?q?=E2=80=94=20example=2043=20+=20chapter=2043=20(Deep=20Dives)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The book's first single-language deep dive (Rust only): a cargo workspace with a proc-macro crate `sysmacros` and the `app` binary that uses it. Example 43-rust-macros-for-systems (mode: local, langs: [rust]): - #[derive(SysError)] — a miniature thiserror: Display + std::error::Error generated from #[error("...")] on named/tuple enum variants via syn 2.0 + quote; malformed input becomes a compile_error!, not a panic. - #[instrument] — an attribute macro that logs a fn's entry (name+args), exit (return value), and Instant-measured duration. - a sound unsafe unaligned read of a #[repr(C)] record from a byte slice, validated clean by miri; a defect-named --oob-read path reads 4 bytes past the 8-byte buffer, which miri catches as out-of-bounds UB. - verify.lua asserts behaviour on the pinned stable toolchain (PASS 8/FAIL 0); miri (nightly) is exercised by the chapter, not the CI gate. First single-language example: demo.sh dispatches rust only, verify.lua skips non-rust; the runner renders the cpp/go columns as "-". Chapter 43 to the spine (Part "Deep Dives"): Tools-used box, verbatim rust excerpts (no tri-language codetabs), Figures 43.1 (proc-macro pipeline) + 43.2 (miri bounds check), verified footer with the real miri diagnostic. Gate: --only 43 → 1 passed/0 failed; full --mode local → 97 passed/0 failed/ 3 skipped (skips are 38-otel, lgtm down). validate: OK. Crates: syn 2.0, quote 1.0, proc-macro2 1.0; stable 1.97.1 + nightly miri. --- _docs/43-rust-macros-for-systems.md | 528 ++++++ .../diagrams/43-miri-bounds-check.excalidraw | 1535 +++++++++++++++++ assets/diagrams/43-miri-bounds-check.svg | 49 + .../43-proc-macro-pipeline.excalidraw | 1431 +++++++++++++++ assets/diagrams/43-proc-macro-pipeline.svg | 47 + assets/diagrams/README.md | 2 + examples/43-rust-macros-for-systems/README.md | 49 + examples/43-rust-macros-for-systems/demo.sh | 24 + .../rust/Cargo.lock | 54 + .../rust/Cargo.toml | 14 + .../43-rust-macros-for-systems/rust/demo.sh | 29 + .../rust/rust-toolchain.toml | 3 + .../rust/src/main.rs | 78 + .../rust/sysmacros/Cargo.toml | 12 + .../rust/sysmacros/src/lib.rs | 164 ++ .../43-rust-macros-for-systems/verify.lua | 44 + examples/manifest.yaml | 4 + 17 files changed, 4067 insertions(+) create mode 100644 _docs/43-rust-macros-for-systems.md create mode 100644 assets/diagrams/43-miri-bounds-check.excalidraw create mode 100644 assets/diagrams/43-miri-bounds-check.svg create mode 100644 assets/diagrams/43-proc-macro-pipeline.excalidraw create mode 100644 assets/diagrams/43-proc-macro-pipeline.svg create mode 100644 examples/43-rust-macros-for-systems/README.md create mode 100755 examples/43-rust-macros-for-systems/demo.sh create mode 100644 examples/43-rust-macros-for-systems/rust/Cargo.lock create mode 100644 examples/43-rust-macros-for-systems/rust/Cargo.toml create mode 100755 examples/43-rust-macros-for-systems/rust/demo.sh create mode 100644 examples/43-rust-macros-for-systems/rust/rust-toolchain.toml create mode 100644 examples/43-rust-macros-for-systems/rust/src/main.rs create mode 100644 examples/43-rust-macros-for-systems/rust/sysmacros/Cargo.toml create mode 100644 examples/43-rust-macros-for-systems/rust/sysmacros/src/lib.rs create mode 100644 examples/43-rust-macros-for-systems/verify.lua diff --git a/_docs/43-rust-macros-for-systems.md b/_docs/43-rust-macros-for-systems.md new file mode 100644 index 0000000..2768725 --- /dev/null +++ b/_docs/43-rust-macros-for-systems.md @@ -0,0 +1,528 @@ +--- +title: "Rust macros for systems code: proc-macros, syn/quote, and what miri proves about unsafe" +order: 43 +part: "Deep Dives" +description: "sysmacros is chapter 43's proc-macro crate, powering two Rust-only techniques in the app binary -- #[derive(SysError)], a miniature thiserror built with syn 2.0's AST and quote's token emission, and #[instrument], a compile-time tracing attribute that times a function with Instant -- plus a sound unsafe unaligned read that cargo +nightly miri validates clean and, under the deliberately-named --oob-read path, catches as undefined behaviour. The book's first single-language deep dive, verified on the Fedora 44 host with verify.lua PASS 8/FAIL 0, the exact six-line stable stdout, and miri's own out-of-bounds diagnostic reproduced byte for byte." +duration: "35 minutes" +--- + +Every chapter so far has mirrored the same behavior across C++23, Go, and +Rust, because the point of those chapters was a Linux mechanism — a +syscall, a signal, a namespace — that all three languages sit on top of +equally. This one breaks that pattern on purpose. Proc-macros are a Rust +compiler feature with no C++ or Go equivalent worth building a codetab +around, and `miri`, the interpreter that catches undefined behaviour a +normal build would run right past, only exists because Rust's `unsafe` +carries obligations the type system stops checking. Chapter 12 gave +`pmon` a `PmonError`-shaped hierarchy by hand, written out field by field; +this chapter asks how much of that boilerplate a macro can write for +you, and how you prove the `unsafe` block sitting a few lines below it is +still doing exactly what its comment claims. + +`sysmacros` is a small cargo workspace: a `proc-macro = true` crate holding +two macros, and an `app` binary that uses them. `#[derive(SysError)]` +turns an errno-style enum into a `Display` + `std::error::Error` impl from +one `#[error("...")]` attribute per variant — a miniature `thiserror`, +built from scratch with `syn 2.0` and `quote`. `#[instrument]` wraps a +function to print its entry, its exit, and its wall-clock duration, +without a tracing runtime. And `read_record` is one `unsafe` unaligned +read of a `repr(C)` struct out of a byte buffer — sound on the path that +checks its bounds, and undefined behaviour on the `--oob-read` path that +skips the check, which is exactly the distinction `miri` exists to draw. + +{% include excalidraw.html + file="43-proc-macro-pipeline" + alt="Two horizontal bands. Top band: the app crate, main.rs's annotated source (#[derive(SysError)] enum PmonError, #[instrument] fn hash_chunk) feeding rustc, which packages a TokenStream and hands it off; on the right, rustc splices tokens back into the AST, typechecks, and compiles the result into the app binary (generated Display + Error impls, the instrumented hash_chunk). Bottom band: sysmacros, a separate proc-macro crate compiled FIRST, containing two nodes -- syn::parse_macro_input! turning the TokenStream into a real DeriveInput/ItemFn AST, and quote! emitting a new TokenStream2 (the Display impl or the wrapped function). Arrows cross the band boundary labeled TokenStream (macro input) going down and TokenStream (macro output) coming back up. A note at the bottom reads: sysmacros never runs at app runtime -- it is pure tokens-in/tokens-out, executed only while rustc is compiling app." + caption="Figure 43.1 — the compile-time-only pipeline: annotated source becomes a TokenStream, the separately-compiled sysmacros crate parses it with syn and re-emits new tokens with quote, and rustc splices the result into app" %} + +> **Tools used** — `cargo` (host, build/run the pinned stable toolchain — +> `rustc 1.97.1` per `rust-toolchain.toml`, in `scripts/check-host.sh`), +> `cargo +nightly miri` / the `miri` component (host — a nightly toolchain +> plus `miri` installed via `rustup`, exercised throughout this chapter but +> **not** part of the stable CI gate, so it is intentionally not in +> `scripts/check-host.sh`), `lua` (host, drives `verify.lua` — in +> `scripts/check-host.sh`). + +## Two macros, one crate: `sysmacros` + +`sysmacros/Cargo.toml` sets `proc-macro = true` in its `[lib]` section, +which is what makes it a fundamentally different kind of dependency: +rustc compiles it as a *compiler plugin* — a program that runs during +`app`'s own compilation, not inside `app` itself — before it ever touches +`main.rs`. That is why this is a two-crate cargo workspace rather than one +package: a proc-macro can only live in its own crate, compiled first, and +`app`'s `Cargo.toml` depends on it the ordinary way (`sysmacros = { path = +"sysmacros" }`). Nothing in `sysmacros/src/lib.rs` ever runs when you +execute the finished `app` binary; every line of it runs once, at build +time, and produces tokens that rustc splices into the program in its +place. + +The macro `PmonError` actually uses is the plainer of the two ideas — +each variant just needs a message: + +```rust +#[derive(SysError, Debug)] +enum PmonError { + #[error("open {path} failed: {errno}")] + Open { path: String, errno: i32 }, + #[error("child {pid} exited with status {code}")] + ChildExit { pid: u32, code: i32 }, + #[error("policy rejected: {0}")] + PolicyRejected(String), +} +``` + +`#[derive(SysError)]` reads that `#[error("...")]` string per variant and +writes the `Display` arm that formats it, plus a blanket `impl +std::error::Error`. The named-field variants (`Open`, `ChildExit`) format +their own fields by name; the tuple variant (`PolicyRejected`) formats its +one field positionally as `{0}`. `#[instrument]` sits on `hash_chunk` +below it the same way `#[error(...)]` sits on an enum variant: an +annotation the macro reads, not code that runs before the function does. +Both macros end at the same place — a `syn::Result` that is +either the new code or a `syn::Error` describing exactly what was wrong +with the input — which "Errors, three ways" returns to below. + +## A sound unsafe read, and what miri proves + +`read_record` is the one piece of this chapter that is not macro-generated, +and it earns its own section because "sound" is a specific claim, not a +synonym for "compiles": + +```rust +fn read_record(buf: &[u8], oob: bool) -> Record { + let off = if oob { 4 } else { 0 }; + // oob: deliberately reads past the 8-byte buffer. + // SOUND path checks the bounds; the --oob-read path skips the check (the + // bug miri catches). + if !oob { + assert!(buf.len() >= off + core::mem::size_of::()); + } + // SAFETY (sound path): buf has >= 8 bytes, Record is repr(C) plain-old-data, + // read_unaligned tolerates any alignment. Under --oob-read this reads 4 bytes + // past the end -> undefined behaviour, which is exactly what miri reports. + unsafe { core::ptr::read_unaligned(buf.as_ptr().add(off) as *const Record) } +} +``` + +An `unsafe` block is a promise to the compiler: "I checked something the +type system can't, and this operation is valid anyway." `read_unaligned` +needs three things to be true — the pointer is readable for +`size_of::()` bytes, those bytes form a valid `Record` (satisfied +trivially here, since `Record` is `repr(C)` and made only of integers), +and the read doesn't race another write. The `assert!` right above the +`unsafe` block is what makes the sound path's promise true: it checks the +buffer really does have 8 bytes before offset 0 is read from. `--oob-read` +sets `off = 4` and, by design, skips that `assert!` — the read now reaches +for offset `4..12` against an 8-byte `buf`, and nothing in a normal +`cargo build`/`cargo run` will tell you that happened. That silence is the +whole reason `miri` exists: it is not a linter looking at your source, it +is an interpreter that tracks every allocation's real bounds and aborts +the instant a read or write steps outside them. + +{% include excalidraw.html + file="43-miri-bounds-check" + alt="An 8-byte buffer BUF: [u8; 8] drawn as eight numbered byte boxes 0 through 7, labeled above as magic: u32 (bytes 0..4) and slots: u32 (bytes 4..8), followed by four dashed ghost boxes numbered 8 through 11 marked past the end (never allocated). Below the real bytes, a blue SOUND window box spans bytes 0 through 7, labeled: read_record(buf, oob=false), read_unaligned::(ptr.add(0)), window [0..8) fully inside the 8-byte alloc, cargo +nightly miri run: exit 0, no UB. Below and offset right, an amber window box spans bytes 4 through 11 (crossing from the real buffer into the ghost bytes), labeled: --oob-read: read_record(buf, oob=true), read_unaligned::(ptr.add(4)), window [4..12) -- 4 bytes past the 8-byte alloc, miri: only 4 bytes from the end of the allocation (main.rs:57). Dashed amber boundary markers point from the OOB window up into the byte row at the offset-4 and offset-12 boundaries." + caption="Figure 43.2 — the same 8-byte Record read from two offsets: the SOUND window [0..8) miri accepts, and the --oob-read window [4..12) that runs 4 bytes past the allocation, exactly what miri's diagnostic names" %} + +## How the code works + +Both macros are one function each, taking real syntax in and giving real +syntax back, and both follow the same three-step shape: `parse_macro_input!` +turns the raw `TokenStream` into a typed `syn` AST, an `expand_*` function +walks that AST and builds a `quote!` block, and the top-level macro +function unwraps the `Result`, turning `Ok` into the new tokens and `Err` +into a `compile_error!`. `expand_sys_error` is the derive side of that +shape: + +```rust +fn expand_sys_error(input: &DeriveInput) -> syn::Result { + let name = &input.ident; + + let data_enum = match &input.data { + Data::Enum(data_enum) => data_enum, + _ => { + return Err(syn::Error::new_spanned( + input, + "SysError can only be derived for enums", + )) + } + }; + + let mut arms = Vec::with_capacity(data_enum.variants.len()); + for variant in &data_enum.variants { + let vident = &variant.ident; + + let error_attr = variant + .attrs + .iter() + .find(|attr| attr.path().is_ident("error")) + .ok_or_else(|| { + syn::Error::new_spanned( + variant, + format!("variant `{vident}` is missing #[error(\"...\")]"), + ) + })?; + let message: LitStr = error_attr.parse_args()?; + + let arm = match &variant.fields { + Fields::Named(fields) => { + let idents: Vec<_> = fields + .named + .iter() + .map(|f| f.ident.clone().expect("named field always has an ident")) + .collect(); + // Edition-2024 inline format-arg capture: `path` and `errno` + // in the literal resolve directly to the bindings destructured + // in this match arm's pattern. + quote! { + Self::#vident { #(#idents),* } => write!(f, #message), + } + } + Fields::Unnamed(fields) => { + let idents: Vec<_> = (0..fields.unnamed.len()) + .map(|i| format_ident!("f{}", i)) + .collect(); + quote! { + Self::#vident(#(#idents),*) => write!(f, #message, #(#idents),*), + } + } + Fields::Unit => { + quote! { + Self::#vident => write!(f, #message), + } + } + }; + arms.push(arm); + } + + Ok(quote! { + impl ::std::fmt::Display for #name { + fn fmt(&self, f: &mut ::std::fmt::Formatter<'_>) -> ::std::fmt::Result { + match self { + #(#arms)* + } + } + } + + impl ::std::error::Error for #name {} + }) +} +``` + +`attr.path().is_ident("error")` is syn 2.x's API for "is this attribute +named `error`" — `path()` is a method, not a field, one of the surface +changes from syn 1.x. `#[proc_macro_derive(SysError, attributes(error))]` +(not shown above) declares `error` as an *inert helper attribute*: it +tells rustc that `#[error("...")]` on a variant is this derive's business, +not an unknown-attribute error. `error_attr.parse_args::()` +(spelled `parse_args()?` above, with the type driven by the `LitStr` +annotation on `message`) parses whatever is inside the parentheses as a +single string literal — exactly the `"open {path} failed: {errno}"` from +`PmonError::Open`. The three `Fields` arms are why `PolicyRejected`'s tuple +variant formats differently from the two named-field variants: for +`Fields::Named`, the destructured pattern `Self::#vident { #(#idents),* }` +binds `path` and `errno` as local names, so `write!(f, #message)` alone is +enough — edition 2024's inline format-arg capture resolves `{path}` and +`{errno}` in the literal directly against those bindings, with no +positional arguments at all. For `Fields::Unnamed`, there is no name to +capture, so the arm binds synthetic identifiers (`f0`, `f1`, ...) and +passes them to `write!` explicitly as `#(#idents),*` after the message. +The whole function returns one `quote!` block containing both the +`Display` impl and the `Error` impl, and that block — not `PmonError`'s +own source — is what rustc actually type-checks and compiles into `app`. + +`expand_instrument` takes the same shape and applies it to a function +instead of an enum: + +```rust +fn expand_instrument(func: ItemFn) -> syn::Result { + let ItemFn { + attrs, + vis, + sig, + block, + } = func; + + let fn_name = sig.ident.to_string(); + + let mut arg_idents = Vec::with_capacity(sig.inputs.len()); + for input in &sig.inputs { + match input { + syn::FnArg::Typed(pat_type) => match &*pat_type.pat { + syn::Pat::Ident(pat_ident) => arg_idents.push(pat_ident.ident.clone()), + other => { + return Err(syn::Error::new_spanned( + other, + "#[instrument] requires simple identifier parameters", + )) + } + }, + syn::FnArg::Receiver(receiver) => { + return Err(syn::Error::new_spanned( + receiver, + "#[instrument] does not support methods with `self`", + )) + } + } + } + + let entry_fmt = { + let args_fmt = arg_idents + .iter() + .map(|id| format!("{id}={{}}")) + .collect::>() + .join(", "); + format!("-> {fn_name}({args_fmt})") + }; + let exit_fmt = format!("<- {fn_name} = {{}} ({{}}us)"); + + Ok(quote! { + #(#attrs)* + #vis #sig { + println!(#entry_fmt, #(#arg_idents),*); + let __instrument_start = ::std::time::Instant::now(); + let __instrument_result = (move || #block)(); + let __instrument_elapsed = __instrument_start.elapsed(); + println!(#exit_fmt, __instrument_result, __instrument_elapsed.as_micros()); + __instrument_result + } + }) +} +``` + +`parse_macro_input!(item as ItemFn)` (in the caller, not shown) gives +`expand_instrument` a full parsed function — signature, visibility, +attributes, and body — as one `syn::ItemFn`. The loop over `sig.inputs` +rejects anything that isn't a plain identifier parameter (`self` receivers, +destructuring patterns) with a `syn::Error` rather than silently mangling +it. The generated function keeps the original `#(#attrs)*`, `#vis`, and +`#sig` untouched — same name, same signature, same visibility — and +replaces only `#block`, the body, with one that prints an entry line, +runs the original body inside a closure so `#block`'s own `return`s and +`?` still work as written, times it with `Instant`, prints the exit line +with the return value and the elapsed microseconds, and returns the +result. `hash_chunk(len: usize) -> u64` goes in as ordinary Rust and comes +out as a function with the identical signature that also happens to log +itself — the caller never sees a difference except the printed lines. + +## Errors, three ways + +Every earlier chapter's "errors, three ways" contrasted three languages; +this one has only Rust, so the three ways are the three moments an error +in this program can actually surface. **Compile time**: a malformed macro +input — `#[derive(SysError)]` on a struct, or a variant missing +`#[error("...")]` — becomes a `syn::Error`, and `derive_sys_error` turns +that into real tokens with `err.to_compile_error().into()`, so a bad +macro invocation is a compiler error at the call site, never a panic +inside `sysmacros` itself. **Run time**: once compiled, `PmonError`'s +three variants are ordinary values — `println!("error: {open_err}")` +drives the generated `Display` impl the same way any hand-written error +type would, producing `error: open /etc/shadow failed: 13` and the two +lines after it. **A checked run**: `read_record`'s `--oob-read` path +produces no error at all under a normal `cargo run` — the read succeeds, +reads whatever bytes happen to sit past the allocation, and the program +keeps going, which is precisely what makes undefined behaviour dangerous +rather than merely broken. `cargo +nightly miri run` is what turns that +silence into a real, located diagnostic: not a `Result`, not a panic, but +an interpreter-level Undefined Behavior report naming the exact +instruction and the exact number of bytes it stepped past the end of the +allocation. + +## Concurrency lens + +There is a genuine concurrency angle here, and it is not the hot loop — +there isn't one. `expand_sys_error` and `expand_instrument` run inside +rustc's own build process, once per macro invocation, as ordinary +synchronous function calls; nothing in `sysmacros` spawns a thread or +shares mutable state across invocations, because a proc-macro never needs +to — it is handed one `TokenStream`, returns one `TokenStream`, and is +done. `#[instrument]`'s `Instant::now()`/`.elapsed()` pair is just as +thread-agnostic at the *other* end of the pipeline: the generated wrapper +times whatever thread calls the instrumented function, on that thread's +own clock, with no global state to synchronize. Call `hash_chunk` from ten +threads and you get ten independent entry/exit pairs, interleaved on +stdout by nothing more than `println!`'s own internal lock. Contrast a +real tracing framework like `tracing`: `#[tracing::instrument]` has to +enter a span on a thread- or task-local subscriber stack, so concurrent +spans compose into a tree a `Registry` maintains. This chapter's +`#[instrument]` composes with concurrency by having nothing to compose — +it wraps whatever the function does and gets out of the way, at the cost +of never answering "which call is this exit line for" once two threads +print at once. + +## Build, run, observe + +```bash +[host]$ cd examples/43-rust-macros-for-systems && ./demo.sh build +``` + +`./demo.sh rust run` builds (if needed) and runs the release binary on the +pinned stable toolchain — the exact path `verify.lua` exercises: + +```console +[host]$ ./demo.sh rust run +error: open /etc/shadow failed: 13 +error: child 4242 exited with status 1 +error: policy rejected: sandbox blocked os.execute +-> hash_chunk(len=4096) +<- hash_chunk = 10911624183873311525 (4us) +record: magic=0x4b565053 slots=256 +``` + +Six lines, three techniques: lines 1–3 are `#[derive(SysError)]`'s +generated `Display`, one line per variant shape (named-field, multi-named-field, +tuple); lines 4–5 are `#[instrument]` around `hash_chunk(len=4096)` — the +hash itself (`10911624183873311525`) is a deterministic FNV-1a over +`0..4096`, so it is identical on every run, and only the microsecond +figure (`4us` here) varies; line 6 is the sound `read_record` call +printing the `Record` it read. + +`./demo.sh rust miri` runs the identical scenario under +`cargo +nightly miri run --quiet` — the interpreter, not the compiled +binary. It prints the same six lines and exits 0 (no UB reported), with +one visible difference: the instrument line reads +`<- hash_chunk = 10911624183873311525 (491785us)` instead of `(4us)`. The +hash value does not change — determinism holds whether the code runs +natively or under an interpreter — but the wall-clock duration jumps from +4 microseconds to roughly 491,785 microseconds, a slowdown of about five +orders of magnitude. That is not a regression to worry about: `miri` is a +UB checker, not a profiler, and it re-executes every memory operation +through its own bookkeeping instead of the CPU directly. The lesson is in +the contrast, not the absolute number — a wall-clock measurement taken +under `miri` tells you nothing about the compiled program's real +performance, only about whether its memory operations are sound. + +`./demo.sh rust miri -- --oob-read` runs the same binary with the +deliberately out-of-bounds path enabled, and this time `miri` stops +before printing the sixth line at all: + +```console +[host]$ cargo +nightly miri run --quiet -- --oob-read +error: Undefined Behavior: memory access failed: attempting to access 8 bytes, but got alloc25+0x4 which is only 4 bytes from the end of the allocation + --> src/main.rs:57:14 + | +57 | unsafe { core::ptr::read_unaligned(buf.as_ptr().add(off) as *const Record) } + | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Undefined Behavior occurred here + | + = help: this indicates a bug in the program: it performed an invalid operation, and caused Undefined Behavior + = help: see https://doc.rust-lang.org/nightly/reference/behavior-considered-undefined.html for further information + = note: stack backtrace: 0: read_record (src/main.rs:57) 1: main (src/main.rs:76) +``` + +`alloc25` is `miri`'s own name for the specific allocation `BUF` occupies +in its interpreted heap; `+0x4` is the offset the failing read started at +(`off = 4` on the `--oob-read` path); "only 4 bytes from the end of the +allocation" is `miri` doing arithmetic a native run never does — `BUF` is +8 bytes, the read needs 8 more bytes starting at offset 4, and offset 4 +plus 8 is 12, four bytes past where the allocation actually ends. The +line and column (`src/main.rs:57:14`) point at the exact `unsafe` block +quoted above. On stable Rust this same `--oob-read` invocation runs to +completion, silently reading whatever garbage bytes happen to sit past +`BUF` in memory — no error, no crash, no signal that anything was wrong. + +The gate the CI-equivalent runner checks is the stable path only: + +```console +[host]$ python3 scripts/test-all-examples.py --only 43-rust-macros-for-systems +... +1 passed, 0 failed, 0 skipped +``` + +```console +[host]$ cd examples/43-rust-macros-for-systems && LSP_LANG=rust lua verify.lua +... +PASS 8 / FAIL 0 +``` + +## Cross-check: the generated code is exactly what a systems error type needs, and the unsafe read only ever touches in-bounds bytes + +Two independent claims, two independent proofs. The first is that +`#[derive(SysError)]`'s generated `Display` impl is not an approximation +of a hand-written systems error type — it *is* one: `verify.lua` asserts +the exact three lines `PmonError`'s variants must produce +(`open /etc/shadow failed: 13`, `child 4242 exited with status 1`, +`policy rejected: sandbox blocked os.execute`), plus the `#[instrument]` +entry/exit shape and the parsed `Record` fields, for `PASS 8 / FAIL 0` — +eight assertions against behaviour, not "the binary exited 0." The second +is that the sound `read_record` path really is sound and the +`--oob-read` path really is a bug, and that claim is checked by an +interpreter rather than taken on faith: the identical scenario run under +`cargo +nightly miri run` exits 0 with no UB reported on the default path, +and reports the precise out-of-bounds access — 4 bytes past an 8-byte +allocation, at `src/main.rs:57` — the moment `--oob-read` is passed. The +491785us-versus-4us timing gap is the same proof from a different angle: +it shows `miri` is genuinely re-executing every memory access through its +own interpreter rather than fast-pathing to the native instruction, which +is exactly the mechanism that lets it catch a bounds violation a native +`cargo run` sails straight through. + +## What you learned + +- **A proc-macro crate is a second, separate compilation**: `sysmacros` + sets `proc-macro = true` and is compiled by rustc as a compiler plugin + before `main.rs` is ever touched — every proc-macro is `TokenStream` in, + `TokenStream` out, and none of that code exists at `app`'s runtime. +- **`#[derive(SysError)]` earns the name "miniature `thiserror`"**: one + `syn::Result` function parses each variant's + `#[error("...")]` string with `syn 2.0`, builds a `Display` match arm + per `Fields` shape (named, tuple, unit) with `quote!`, and edition 2024's + inline format-arg capture resolves `{path}`/`{errno}` straight against + the destructured match bindings — no positional arguments needed. +- **Malformed macro input is a `compile_error!`, never a panic**: both + macros convert a `syn::Error` into real tokens via + `err.to_compile_error()`, so a missing `#[error(...)]` attribute or a + non-enum target fails the build with a located diagnostic, the same + discipline a hand-written derive would need to earn. +- **`unsafe` is a promise the type system stopped checking, and `miri` + checks it anyway**: the sound `read_record` path's `assert!` makes its + `SAFETY` comment true; the `--oob-read` path skips that assert and reads + 4 bytes past an 8-byte allocation — silent on stable, a named Undefined + Behavior report under `cargo +nightly miri run`. +- **`miri` is a correctness checker, not a profiler**: the identical + scenario took 4 microseconds natively and roughly 491,785 microseconds + under `miri` — a real, expected, five-order-of-magnitude interpreter + slowdown that says nothing about `app`'s real performance and everything + about whether its memory operations are sound. +- **`#[instrument]` is thread-agnostic because it holds no state at all**: + it wraps whatever function it's given in a `println!`/`Instant`/`println!` + sandwich with no shared subscriber or span stack, unlike a runtime + tracing crate that has to maintain one — composable with any threading + model precisely because it has nothing to synchronize. + +Two deep dives remain in this part: this chapter's Rust-only look at +compile-time metaprogramming, and the next — the Go runtime itself, from +a systems programmer's seat. + +--- + +

verified — on the Fedora +44 reference host this session: python3 scripts/test-all-examples.py +--only 43-rust-macros-for-systems reported 1 passed, 0 +failed (rust column only; this is the book's first single-language +example), and LSP_LANG=rust lua verify.lua reported +PASS 8 / FAIL 0. ./demo.sh rust run on the pinned +stable rustc 1.97.1 toolchain produced the exact six-line +transcript quoted above: the three #[derive(SysError)] +Display lines, the #[instrument] entry/exit pair +around hash_chunk(len=4096) (hash +10911624183873311525, deterministic; 4us measured +this run), and the parsed record: magic=0x4b565053 slots=256 +line. cargo +nightly miri run --quiet (the sound path) exited 0 +with no Undefined Behavior reported, printing the same six lines with the +instrument line's duration at 491785us instead of +4us — a real ~5-order-of-magnitude interpreter slowdown, not a +regression. cargo +nightly miri run --quiet -- --oob-read +reproduced the exact Undefined Behavior diagnostic quoted above: memory +access failed at alloc25+0x4, "only 4 bytes from the end of +the allocation," located at src/main.rs:57:14, the +read_unaligned call inside read_record. Crate +pins: syn 2.0 (features full), quote +1.0, proc-macro2 1.0, on stable rustc +1.97.1 (rust-toolchain.toml) with a separately +installed nightly toolchain plus the miri component via +rustup. Not exercised: this example is mode: local +per examples/manifest.yaml — there is no VM or LGTM path for +it; and the --oob-read path is only ever run under +miri in this book, never asserted on stable Rust, where the +same call is undefined behaviour rather than a checkable outcome.

diff --git a/assets/diagrams/43-miri-bounds-check.excalidraw b/assets/diagrams/43-miri-bounds-check.excalidraw new file mode 100644 index 0000000..f69cbf5 --- /dev/null +++ b/assets/diagrams/43-miri-bounds-check.excalidraw @@ -0,0 +1,1535 @@ +{ + "type": "excalidraw", + "version": 2, + "source": "https://excalidraw.com", + "elements": [ + { + "id": "r1795308423", + "type": "rectangle", + "x": 20, + "y": 20, + "width": 900, + "height": 400, + "angle": 0, + "strokeColor": "#dddddd", + "backgroundColor": "#fafafa", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1265389281, + "versionNonce": 1621905117, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "r1377278247", + "type": "rectangle", + "x": 140, + "y": 130, + "width": 50, + "height": 50, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1806350969, + "versionNonce": 37132363, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1892278996", + "type": "text", + "x": 150, + "y": 138, + "width": 40, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 615930997, + "versionNonce": 591322110, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "0", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "0", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "r1844006356", + "type": "rectangle", + "x": 190, + "y": 130, + "width": 50, + "height": 50, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1466011700, + "versionNonce": 645900241, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1921964798", + "type": "text", + "x": 200, + "y": 138, + "width": 40, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1037467018, + "versionNonce": 1178255260, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "1", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "1", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "r763640696", + "type": "rectangle", + "x": 240, + "y": 130, + "width": 50, + "height": 50, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1958458373, + "versionNonce": 155270866, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t207759467", + "type": "text", + "x": 250, + "y": 138, + "width": 40, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1943171300, + "versionNonce": 430481774, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "2", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "2", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "r277266720", + "type": "rectangle", + "x": 290, + "y": 130, + "width": 50, + "height": 50, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1007156244, + "versionNonce": 897810309, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t89081383", + "type": "text", + "x": 300, + "y": 138, + "width": 40, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1692717358, + "versionNonce": 319264135, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "3", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "3", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "r244492372", + "type": "rectangle", + "x": 340, + "y": 130, + "width": 50, + "height": 50, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1817075569, + "versionNonce": 1849699423, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t917455385", + "type": "text", + "x": 350, + "y": 138, + "width": 40, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1430645237, + "versionNonce": 883510614, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "4", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "4", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "r1724432945", + "type": "rectangle", + "x": 390, + "y": 130, + "width": 50, + "height": 50, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 12297048, + "versionNonce": 1721627207, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t308328436", + "type": "text", + "x": 400, + "y": 138, + "width": 40, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1281565448, + "versionNonce": 1790224455, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "5", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "5", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "r1733761343", + "type": "rectangle", + "x": 440, + "y": 130, + "width": 50, + "height": 50, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1095781619, + "versionNonce": 924652655, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t107798609", + "type": "text", + "x": 450, + "y": 138, + "width": 40, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 128642948, + "versionNonce": 394970373, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "6", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "6", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "r1331077401", + "type": "rectangle", + "x": 490, + "y": 130, + "width": 50, + "height": 50, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 516923497, + "versionNonce": 1477282503, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1188596979", + "type": "text", + "x": 500, + "y": 138, + "width": 40, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 657595380, + "versionNonce": 1967263558, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "7", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "7", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "r1129695651", + "type": "rectangle", + "x": 540, + "y": 130, + "width": 50, + "height": 50, + "angle": 0, + "strokeColor": "#999999", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "dashed", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1451819205, + "versionNonce": 1089043755, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1204462556", + "type": "text", + "x": 550, + "y": 138, + "width": 40, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 703259918, + "versionNonce": 693725210, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "8", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "8", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "r1544135756", + "type": "rectangle", + "x": 590, + "y": 130, + "width": 50, + "height": 50, + "angle": 0, + "strokeColor": "#999999", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "dashed", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1236781604, + "versionNonce": 86919221, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1924864518", + "type": "text", + "x": 600, + "y": 138, + "width": 40, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1111029466, + "versionNonce": 851861170, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "9", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "9", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "r761989265", + "type": "rectangle", + "x": 640, + "y": 130, + "width": 50, + "height": 50, + "angle": 0, + "strokeColor": "#999999", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "dashed", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 28767753, + "versionNonce": 37269439, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1734160921", + "type": "text", + "x": 650, + "y": 138, + "width": 40, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1105797078, + "versionNonce": 241404891, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "10", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "10", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "r895790713", + "type": "rectangle", + "x": 690, + "y": 130, + "width": 50, + "height": 50, + "angle": 0, + "strokeColor": "#999999", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "dashed", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 626145693, + "versionNonce": 33978623, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t771836645", + "type": "text", + "x": 700, + "y": 138, + "width": 40, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 322155064, + "versionNonce": 1024333659, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "11", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "11", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "r502657901", + "type": "rectangle", + "x": 140, + "y": 80, + "width": 200, + "height": 32, + "angle": 0, + "strokeColor": "#999999", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1330668796, + "versionNonce": 920136295, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1236190982", + "type": "text", + "x": 150, + "y": 88, + "width": 177.10000000000002, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 924910085, + "versionNonce": 553504706, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "magic: u32 (bytes 0..4)", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "magic: u32 (bytes 0..4)", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "r1496332201", + "type": "rectangle", + "x": 340, + "y": 80, + "width": 200, + "height": 32, + "angle": 0, + "strokeColor": "#999999", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 132156821, + "versionNonce": 1653677319, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t272413532", + "type": "text", + "x": 350, + "y": 88, + "width": 177.10000000000002, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1274065329, + "versionNonce": 909361679, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "slots: u32 (bytes 4..8)", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "slots: u32 (bytes 4..8)", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "r935732448", + "type": "rectangle", + "x": 140, + "y": 220, + "width": 400, + "height": 100, + "angle": 0, + "strokeColor": "#2f6db5", + "backgroundColor": "#eef4fb", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1951770563, + "versionNonce": 790058571, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1416852542", + "type": "text", + "x": 150, + "y": 228, + "width": 261.8, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 322949680, + "versionNonce": 809538539, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "SOUND: read_record(buf, oob=false)", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "SOUND: read_record(buf, oob=false)", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t205910875", + "type": "text", + "x": 150, + "y": 248, + "width": 217.8, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 243456976, + "versionNonce": 1927654983, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "read_unaligned::(ptr.add(0))", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "read_unaligned::(ptr.add(0))", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t1361350683", + "type": "text", + "x": 150, + "y": 263, + "width": 278.3, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 178050525, + "versionNonce": 1426135186, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "window [0..8) -- fully inside the 8-byte alloc", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "window [0..8) -- fully inside the 8-byte alloc", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t159416715", + "type": "text", + "x": 150, + "y": 278, + "width": 229.9, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1168130719, + "versionNonce": 1540086448, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "cargo +nightly miri run: exit 0, no UB", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "cargo +nightly miri run: exit 0, no UB", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1381695497", + "type": "rectangle", + "x": 340, + "y": 340, + "width": 400, + "height": 100, + "angle": 0, + "strokeColor": "#e8870c", + "backgroundColor": "#fff8ef", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 294638709, + "versionNonce": 1134377662, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1038069525", + "type": "text", + "x": 350, + "y": 348, + "width": 292.6, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1214450770, + "versionNonce": 108258568, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "--oob-read: read_record(buf, oob=true)", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "--oob-read: read_record(buf, oob=true)", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t38770828", + "type": "text", + "x": 350, + "y": 368, + "width": 217.8, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1642667589, + "versionNonce": 674302995, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "read_unaligned::(ptr.add(4))", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "read_unaligned::(ptr.add(4))", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t1032456740", + "type": "text", + "x": 350, + "y": 383, + "width": 284.35, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 522013982, + "versionNonce": 48437328, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "window [4..12) -- 4 bytes past the 8-byte alloc", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "window [4..12) -- 4 bytes past the 8-byte alloc", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t1013315576", + "type": "text", + "x": 350, + "y": 398, + "width": 405.35, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1726761586, + "versionNonce": 571366818, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "miri: \"...only 4 bytes from the end of the allocation\" (main.rs:57)", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "miri: \"...only 4 bytes from the end of the allocation\" (main.rs:57)", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a900229483", + "type": "arrow", + "x": 140, + "y": 220, + "width": 0, + "height": 40, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "dashed", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 596671473, + "versionNonce": 1546929851, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + -40 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "a818092179", + "type": "arrow", + "x": 540, + "y": 220, + "width": 0, + "height": 40, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "dashed", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1064955016, + "versionNonce": 1661940375, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + -40 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "a1836087373", + "type": "arrow", + "x": 340, + "y": 340, + "width": 0, + "height": 160, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "dashed", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 721564994, + "versionNonce": 675698167, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + -160 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "a1849952495", + "type": "arrow", + "x": 740, + "y": 340, + "width": 0, + "height": 160, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "dashed", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1014756658, + "versionNonce": 1659613510, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + -160 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t1636156380", + "type": "text", + "x": 140, + "y": 50, + "width": 402.6, + "height": 15.0, + "angle": 0, + "strokeColor": "#1d1d1d", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1465098689, + "versionNonce": 1806989027, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "BUF: [u8; 8] -- one 8-byte stack allocation (miri: \"alloc25\")", + "fontSize": 12, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "BUF: [u8; 8] -- one 8-byte stack allocation (miri: \"alloc25\")", + "lineHeight": 1.25, + "baseline": 12 + }, + { + "id": "t130616436", + "type": "text", + "x": 540, + "y": 135, + "width": 115.50000000000001, + "height": 12.5, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 592885482, + "versionNonce": 702506145, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "end of the allocation", + "fontSize": 10, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "end of the allocation", + "lineHeight": 1.25, + "baseline": 10 + }, + { + "id": "t987660289", + "type": "text", + "x": 540, + "y": 190, + "width": 181.50000000000003, + "height": 13.75, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1800387609, + "versionNonce": 1292936478, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "past the end (never allocated)", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "past the end (never allocated)", + "lineHeight": 1.25, + "baseline": 11 + } + ], + "appState": { + "viewBackgroundColor": "#ffffff", + "gridSize": null + }, + "files": {} +} \ No newline at end of file diff --git a/assets/diagrams/43-miri-bounds-check.svg b/assets/diagrams/43-miri-bounds-check.svg new file mode 100644 index 0000000..3c33094 --- /dev/null +++ b/assets/diagrams/43-miri-bounds-check.svg @@ -0,0 +1,49 @@ + + + + +0 + +1 + +2 + +3 + +4 + +5 + +6 + +7 + +8 + +9 + +10 + +11 + +magic: u32 (bytes 0..4) + +slots: u32 (bytes 4..8) + +SOUND: read_record(buf, oob=false) +read_unaligned::<Record>(ptr.add(0)) +window [0..8) -- fully inside the 8-byte alloc +cargo +nightly miri run: exit 0, no UB + +--oob-read: read_record(buf, oob=true) +read_unaligned::<Record>(ptr.add(4)) +window [4..12) -- 4 bytes past the 8-byte alloc +miri: "...only 4 bytes from the end of the allocation" (main.rs:57) + + + + +BUF: [u8; 8] -- one 8-byte stack allocation (miri: "alloc25") +end of the allocation +past the end (never allocated) + \ No newline at end of file diff --git a/assets/diagrams/43-proc-macro-pipeline.excalidraw b/assets/diagrams/43-proc-macro-pipeline.excalidraw new file mode 100644 index 0000000..bd5accf --- /dev/null +++ b/assets/diagrams/43-proc-macro-pipeline.excalidraw @@ -0,0 +1,1431 @@ +{ + "type": "excalidraw", + "version": 2, + "source": "https://excalidraw.com", + "elements": [ + { + "id": "r1491378567", + "type": "rectangle", + "x": 20, + "y": 30, + "width": 960, + "height": 190, + "angle": 0, + "strokeColor": "#dddddd", + "backgroundColor": "#eef4fb", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 540303644, + "versionNonce": 1327785481, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t89695293", + "type": "text", + "x": 32, + "y": 36, + "width": 389.40000000000003, + "height": 15.0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1513669385, + "versionNonce": 1674132968, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "app crate -- main.rs (built around the sysmacros expansion)", + "fontSize": 12, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "app crate -- main.rs (built around the sysmacros expansion)", + "lineHeight": 1.25, + "baseline": 12 + }, + { + "id": "r20346891", + "type": "rectangle", + "x": 20, + "y": 250, + "width": 960, + "height": 190, + "angle": 0, + "strokeColor": "#dddddd", + "backgroundColor": "#fff8ef", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 588186635, + "versionNonce": 968845467, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t448276024", + "type": "text", + "x": 32, + "y": 256, + "width": 369.6, + "height": 15.0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1827138831, + "versionNonce": 529625371, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "sysmacros -- a separate proc-macro crate, compiled FIRST", + "fontSize": 12, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "sysmacros -- a separate proc-macro crate, compiled FIRST", + "lineHeight": 1.25, + "baseline": 12 + }, + { + "id": "r525271524", + "type": "rectangle", + "x": 50, + "y": 80, + "width": 210, + "height": 110, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#ffffff", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1776782065, + "versionNonce": 459871177, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t473931985", + "type": "text", + "x": 60, + "y": 88, + "width": 200.20000000000002, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 261860784, + "versionNonce": 1470378136, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "main.rs (annotated source)", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "main.rs (annotated source)", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1289872303", + "type": "text", + "x": 60, + "y": 108, + "width": 114.95, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1052617643, + "versionNonce": 801916330, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "#[derive(SysError)]", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "#[derive(SysError)]", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t690308202", + "type": "text", + "x": 60, + "y": 123, + "width": 127.05000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 119127496, + "versionNonce": 465247379, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "enum PmonError { .. }", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "enum PmonError { .. }", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t916553704", + "type": "text", + "x": 60, + "y": 138, + "width": 78.65, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 283913569, + "versionNonce": 193087341, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "#[instrument]", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "#[instrument]", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t6406124", + "type": "text", + "x": 60, + "y": 153, + "width": 151.25, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 592946435, + "versionNonce": 1674438729, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "fn hash_chunk(len) -> u64", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "fn hash_chunk(len) -> u64", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1828419316", + "type": "rectangle", + "x": 310, + "y": 95, + "width": 190, + "height": 80, + "angle": 0, + "strokeColor": "#2f6db5", + "backgroundColor": "#eef4fb", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1607951397, + "versionNonce": 1579709120, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t57111495", + "type": "text", + "x": 320, + "y": 103, + "width": 177.10000000000002, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1915134551, + "versionNonce": 668746264, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "rustc: macro invocation", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "rustc: macro invocation", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t261821804", + "type": "text", + "x": 320, + "y": 123, + "width": 151.25, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 273827931, + "versionNonce": 892660011, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "sees the attribute/derive", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "sees the attribute/derive", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t1193116387", + "type": "text", + "x": 320, + "y": 138, + "width": 133.10000000000002, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1312355213, + "versionNonce": 1408994175, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "packages a TokenStream", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "packages a TokenStream", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r138905362", + "type": "rectangle", + "x": 560, + "y": 95, + "width": 190, + "height": 80, + "angle": 0, + "strokeColor": "#2f6db5", + "backgroundColor": "#eef4fb", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1091845090, + "versionNonce": 626504325, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1756320177", + "type": "text", + "x": 570, + "y": 103, + "width": 192.50000000000003, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 555170263, + "versionNonce": 354234096, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "rustc: expand & typecheck", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "rustc: expand & typecheck", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t12559674", + "type": "text", + "x": 570, + "y": 123, + "width": 163.35000000000002, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1457054830, + "versionNonce": 1586311840, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "splices tokens into the AST", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "splices tokens into the AST", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t496178320", + "type": "text", + "x": 570, + "y": 138, + "width": 127.05000000000001, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1354476729, + "versionNonce": 717466056, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "typechecks the result", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "typechecks the result", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1419798083", + "type": "rectangle", + "x": 800, + "y": 80, + "width": 170, + "height": 110, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "#111111", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1071521409, + "versionNonce": 469546626, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t435622209", + "type": "text", + "x": 810, + "y": 88, + "width": 77.0, + "height": 17.5, + "angle": 0, + "strokeColor": "#ffffff", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1764057774, + "versionNonce": 1312277285, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "app binary", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "app binary", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1671714830", + "type": "text", + "x": 810, + "y": 108, + "width": 151.25, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 989226598, + "versionNonce": 1955167364, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "generated Display + Error", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "generated Display + Error", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t1668647391", + "type": "text", + "x": 810, + "y": 123, + "width": 139.15, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1844968728, + "versionNonce": 86619953, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "instrumented hash_chunk", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "instrumented hash_chunk", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t424421068", + "type": "text", + "x": 810, + "y": 138, + "width": 139.15, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 507196951, + "versionNonce": 1671974299, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "(tokens, now real code)", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "(tokens, now real code)", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r1048556342", + "type": "rectangle", + "x": 310, + "y": 290, + "width": 190, + "height": 90, + "angle": 0, + "strokeColor": "#e8870c", + "backgroundColor": "#fff8ef", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 1226978182, + "versionNonce": 212403940, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t1109957766", + "type": "text", + "x": 320, + "y": 298, + "width": 177.10000000000002, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 246532162, + "versionNonce": 2531598, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "syn::parse_macro_input!", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "syn::parse_macro_input!", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t124897302", + "type": "text", + "x": 320, + "y": 318, + "width": 157.3, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 361473971, + "versionNonce": 1898240428, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "TokenStream -> DeriveInput", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "TokenStream -> DeriveInput", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t736979789", + "type": "text", + "x": 320, + "y": 333, + "width": 175.45000000000002, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1347439491, + "versionNonce": 245640129, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "/ ItemFn -- a real, typed AST", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "/ ItemFn -- a real, typed AST", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "r487792219", + "type": "rectangle", + "x": 560, + "y": 290, + "width": 190, + "height": 90, + "angle": 0, + "strokeColor": "#e8870c", + "backgroundColor": "#fff8ef", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 3 + }, + "seed": 161885644, + "versionNonce": 1178167070, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false + }, + { + "id": "t929396851", + "type": "text", + "x": 570, + "y": 298, + "width": 100.10000000000001, + "height": 17.5, + "angle": 0, + "strokeColor": "#111111", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1704341804, + "versionNonce": 1663362525, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "quote! { .. }", + "fontSize": 14, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "quote! { .. }", + "lineHeight": 1.25, + "baseline": 14 + }, + { + "id": "t1256021640", + "type": "text", + "x": 570, + "y": 318, + "width": 139.15, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1135140644, + "versionNonce": 1455751339, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "AST -> new TokenStream2", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "AST -> new TokenStream2", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "t1356892780", + "type": "text", + "x": 570, + "y": 333, + "width": 151.25, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 425734136, + "versionNonce": 1506573702, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "Display impl / wrapped fn", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "Display impl / wrapped fn", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a1513857226", + "type": "arrow", + "x": 260, + "y": 130, + "width": 48, + "height": 0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 891764247, + "versionNonce": 1800359410, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 48, + 0 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "a1170563204", + "type": "arrow", + "x": 405, + "y": 176, + "width": 0, + "height": 112, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 488380344, + "versionNonce": 1475130449, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + 112 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t1813632360", + "type": "text", + "x": 405.0, + "y": 218.0, + "width": 151.25, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 561477644, + "versionNonce": 1750719696, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "TokenStream (macro input)", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "TokenStream (macro input)", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a332532366", + "type": "arrow", + "x": 502, + "y": 335, + "width": 56, + "height": 0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 787156461, + "versionNonce": 257533462, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 56, + 0 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t1659566718", + "type": "text", + "x": 530.0, + "y": 321.0, + "width": 40, + "height": 13.75, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 1001840862, + "versionNonce": 457408669, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "AST", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "AST", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a1767571393", + "type": "arrow", + "x": 655, + "y": 288, + "width": 0, + "height": 112, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 62801607, + "versionNonce": 86357591, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 0, + -112 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t385732070", + "type": "text", + "x": 655.0, + "y": 218.0, + "width": 157.3, + "height": 13.75, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 362219043, + "versionNonce": 1192993035, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "TokenStream (macro output)", + "fontSize": 11, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "TokenStream (macro output)", + "lineHeight": 1.25, + "baseline": 11 + }, + { + "id": "a408968307", + "type": "arrow", + "x": 752, + "y": 130, + "width": 46, + "height": 0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 2, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": { + "type": 2 + }, + "seed": 1358391641, + "versionNonce": 995060172, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "points": [ + [ + 0, + 0 + ], + [ + 46, + 0 + ] + ], + "lastCommittedPoint": null, + "startBinding": null, + "endBinding": null, + "startArrowhead": null, + "endArrowhead": "arrow" + }, + { + "id": "t1152460330", + "type": "text", + "x": 40, + "y": 390, + "width": 759.0000000000001, + "height": 15.0, + "angle": 0, + "strokeColor": "#b8650a", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 845546475, + "versionNonce": 699634936, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "sysmacros never runs at app runtime: it is pure tokens-in / tokens-out, executed only while rustc is compiling app.", + "fontSize": 12, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "sysmacros never runs at app runtime: it is pure tokens-in / tokens-out, executed only while rustc is compiling app.", + "lineHeight": 1.25, + "baseline": 12 + }, + { + "id": "t1385858308", + "type": "text", + "x": 40, + "y": 410, + "width": 732.6, + "height": 15.0, + "angle": 0, + "strokeColor": "#555555", + "backgroundColor": "transparent", + "fillStyle": "solid", + "strokeWidth": 1, + "strokeStyle": "solid", + "roughness": 1, + "opacity": 100, + "groupIds": [], + "frameId": null, + "roundness": null, + "seed": 532613454, + "versionNonce": 259652276, + "isDeleted": false, + "boundElements": [], + "updated": 1, + "link": null, + "locked": false, + "text": "Two crates, one build: cargo compiles sysmacros to a host-side compiler plugin before it ever compiles main.rs.", + "fontSize": 12, + "fontFamily": 1, + "textAlign": "left", + "verticalAlign": "top", + "containerId": null, + "originalText": "Two crates, one build: cargo compiles sysmacros to a host-side compiler plugin before it ever compiles main.rs.", + "lineHeight": 1.25, + "baseline": 12 + } + ], + "appState": { + "viewBackgroundColor": "#ffffff", + "gridSize": null + }, + "files": {} +} \ No newline at end of file diff --git a/assets/diagrams/43-proc-macro-pipeline.svg b/assets/diagrams/43-proc-macro-pipeline.svg new file mode 100644 index 0000000..112c6b4 --- /dev/null +++ b/assets/diagrams/43-proc-macro-pipeline.svg @@ -0,0 +1,47 @@ + + + +app crate -- main.rs (built around the sysmacros expansion) + +sysmacros -- a separate proc-macro crate, compiled FIRST + +main.rs (annotated source) +#[derive(SysError)] +enum PmonError { .. } +#[instrument] +fn hash_chunk(len) -> u64 + +rustc: macro invocation +sees the attribute/derive +packages a TokenStream + +rustc: expand & typecheck +splices tokens into the AST +typechecks the result + +app binary +generated Display + Error +instrumented hash_chunk +(tokens, now real code) + +syn::parse_macro_input! +TokenStream -> DeriveInput +/ ItemFn -- a real, typed AST + +quote! { .. } +AST -> new TokenStream2 +Display impl / wrapped fn + + +TokenStream (macro input) +TokenStream (macro input) + +AST +AST + +TokenStream (macro output) +TokenStream (macro output) + +sysmacros never runs at app runtime: it is pure tokens-in / tokens-out, executed only while rustc is compiling app. +Two crates, one build: cargo compiles sysmacros to a host-side compiler plugin before it ever compiles main.rs. + \ No newline at end of file diff --git a/assets/diagrams/README.md b/assets/diagrams/README.md index 37e29b1..0f08fcf 100644 --- a/assets/diagrams/README.md +++ b/assets/diagrams/README.md @@ -83,3 +83,5 @@ here as you add them: | `36-agent-architecture` | 36 | sysagent's `sample`/`serve` subcommands sharing one procfs reader layer, against the row of cross-check tools run at the same moment (vmstat, mpstat, free -m, ss -s, a direct `/proc/meminfo` read, `kill -INT`), with forward references to Chapter 37 (USE method), 38 (OTLP export), and 45 (PCP pmrep) | | `42-host-script-boundary` | 42 | the three host runtimes (C++/sol2, Go/gopher-lua, Rust/mlua) each calling into one shared `policy.lua` with `on_exit(info)`/`decision{action,delay_ms,reason}`, the `host.log`/`host.now_ms` callbacks bound in, the sandbox boundary blocking os/io/package/debug and nilling the loader globals, and the Lua 5.1-vs-5.4 gap that forces the shared subset | | `42-sighup-reload-lifecycle` | 42 | the hot-reload sequence -- load v1, phase-1 decisions (500/1000ms), parking on SIGHUP, an operator editing the policy file and sending `kill -HUP`, reload with a fresh Lua state and `on_load()` logging v1.1.0, phase-2 decisions (100ms restart, escalate), exit -- with the three per-language SIGHUP-consumption mechanisms shown side by side and the process's PID never changing | +| `43-proc-macro-pipeline` | 43 | the compile-time-only pipeline behind `#[derive(SysError)]`/`#[instrument]` -- annotated source handed to rustc as a TokenStream, the `sysmacros` proc-macro crate (compiled first, a separate crate) parsing it with `syn::parse_macro_input!` into a real AST and re-emitting new tokens with `quote!`, spliced back into `app` by rustc -- pure tokens-in/tokens-out, nothing here runs at app runtime | +| `43-miri-bounds-check` | 43 | the 8-byte `Record` buffer (`magic: u32` + `slots: u32`) with the SOUND `read_unaligned` window `[0..8)` fully inside the allocation beside the `--oob-read` window `[4..12)` running 4 bytes past the end -- the exact contrast `cargo +nightly miri run` catches (exit 0, no UB) versus reports (`only 4 bytes from the end of the allocation`, main.rs:57) | diff --git a/examples/43-rust-macros-for-systems/README.md b/examples/43-rust-macros-for-systems/README.md new file mode 100644 index 0000000..95816ad --- /dev/null +++ b/examples/43-rust-macros-for-systems/README.md @@ -0,0 +1,49 @@ +# 43 — rust-macros-for-systems + +The book's first **single-language** example (Rust only): the deep dives don't +mirror across three languages. It shows three metaprogramming techniques a +systems programmer actually reaches for, as a small cargo **workspace** — a +proc-macro crate plus the binary that uses it: + +- **`#[derive(SysError)]`** — a derive macro that generates `Display` + `std::error::Error` + from `#[error("...")]` attributes (a miniature `thiserror`), for errno-style + error enums. +- **`#[instrument]`** — an attribute macro that wraps a function to log its + entry (name + args), exit (return value), and wall-clock duration — tracing + without a runtime. +- **A sound `unsafe` unaligned read**, validated with **miri** — plus a + defect-named `--oob-read` path that miri flags as undefined behaviour. + +## Layout + +``` +43-rust-macros-for-systems/ +├── demo.sh # dispatcher (rust only) +├── verify.lua # rust-only behavioural check +└── rust/ + ├── Cargo.toml # workspace root + `app` bin package + ├── src/main.rs # the app: uses the macros + the unsafe module + └── sysmacros/ # the proc-macro crate (proc-macro = true) + └── src/lib.rs # SysError derive + instrument attribute macro +``` + +## Run it + +```bash +./demo.sh rust run # the scenario on stable 1.97.1 (what CI verifies) +./demo.sh rust miri # cargo +nightly miri run — the sound path, no UB +./demo.sh rust miri -- --oob-read # miri catches the out-of-bounds read +``` + +## Verify + +`verify.lua` (Rust only; skips other langs) asserts the generated `Display` +strings, the `#[instrument]` entry/exit/timing lines, and the parsed record — +behaviour, not exit codes. The gate runs on the **pinned stable** toolchain; +miri (nightly) is exercised by the chapter, not the gate. + +```bash +LSP_LANG=rust lua verify.lua +``` + +Mode: `local`. Crates: `syn 2.0`, `quote 1.0`, `proc-macro2 1.0`. diff --git a/examples/43-rust-macros-for-systems/demo.sh b/examples/43-rust-macros-for-systems/demo.sh new file mode 100755 index 0000000..fe517ed --- /dev/null +++ b/examples/43-rust-macros-for-systems/demo.sh @@ -0,0 +1,24 @@ +#!/usr/bin/env bash +set -euo pipefail +cd "$(dirname "$0")" + +# Single-language example (Rust only): proc-macros need their own crate, so this +# is a cargo workspace. The dispatcher still follows the book's demo contract. +langs=(rust) + +case "${1:-all}" in + rust) + shift + exec "./rust/demo.sh" "$@" + ;; + all) + for lang in "${langs[@]}"; do "./$lang/demo.sh"; done + ;; + build) + for lang in "${langs[@]}"; do "./$lang/demo.sh" build; done + ;; + *) + echo "usage: $0 [rust|all|build] [args...]" >&2 + exit 2 + ;; +esac diff --git a/examples/43-rust-macros-for-systems/rust/Cargo.lock b/examples/43-rust-macros-for-systems/rust/Cargo.lock new file mode 100644 index 0000000..e896cf4 --- /dev/null +++ b/examples/43-rust-macros-for-systems/rust/Cargo.lock @@ -0,0 +1,54 @@ +# This file is automatically @generated by Cargo. +# It is not intended for manual editing. +version = 4 + +[[package]] +name = "app" +version = "0.1.0" +dependencies = [ + "sysmacros", +] + +[[package]] +name = "proc-macro2" +version = "1.0.107" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "985e7ec9bb745e6ce6535b544d84d6cd6f7ad8bd711c398938ae983b91a766d9" +dependencies = [ + "unicode-ident", +] + +[[package]] +name = "quote" +version = "1.0.47" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1fbf4db142a473a8d80c26bbf18454ed458bf8d26c8219c331daecfdbd079001" +dependencies = [ + "proc-macro2", +] + +[[package]] +name = "syn" +version = "2.0.119" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "872831b642d1a07999a962a351ed35b955ea2cfc8f3862091e2a240a84f17297" +dependencies = [ + "proc-macro2", + "quote", + "unicode-ident", +] + +[[package]] +name = "sysmacros" +version = "0.1.0" +dependencies = [ + "proc-macro2", + "quote", + "syn", +] + +[[package]] +name = "unicode-ident" +version = "1.0.24" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e6e4313cd5fcd3dad5cafa179702e2b244f760991f45397d14d4ebf38247da75" diff --git a/examples/43-rust-macros-for-systems/rust/Cargo.toml b/examples/43-rust-macros-for-systems/rust/Cargo.toml new file mode 100644 index 0000000..9563102 --- /dev/null +++ b/examples/43-rust-macros-for-systems/rust/Cargo.toml @@ -0,0 +1,14 @@ +[package] +name = "app" +version = "0.1.0" +edition = "2024" + +[[bin]] +name = "app" +path = "src/main.rs" + +[dependencies] +sysmacros = { path = "sysmacros" } + +[workspace] +members = ["sysmacros"] diff --git a/examples/43-rust-macros-for-systems/rust/demo.sh b/examples/43-rust-macros-for-systems/rust/demo.sh new file mode 100755 index 0000000..c67ab70 --- /dev/null +++ b/examples/43-rust-macros-for-systems/rust/demo.sh @@ -0,0 +1,29 @@ +#!/usr/bin/env bash +set -euo pipefail +cd "$(dirname "${BASH_SOURCE[0]}")" + +BIN=target/release/app + +build() { + cargo build --release +} + +run() { + if [[ ! -x "$BIN" ]]; then build; fi + "./$BIN" "$@" +} + +# miri runs the app under the interpreter that checks for undefined behaviour. +# It needs nightly + the miri component; the CI gate does NOT depend on it +# (verify.lua exercises the stable build only). Used by the chapter. +miri() { + cargo +nightly miri run --quiet "$@" +} + +case "${1:-}" in + build) build ;; + run) shift; run "$@" ;; + miri) shift; miri "$@" ;; + "") build; run ;; + *) echo "usage: $0 [build|run [args]|miri [args]]" >&2; exit 2 ;; +esac diff --git a/examples/43-rust-macros-for-systems/rust/rust-toolchain.toml b/examples/43-rust-macros-for-systems/rust/rust-toolchain.toml new file mode 100644 index 0000000..4099f1a --- /dev/null +++ b/examples/43-rust-macros-for-systems/rust/rust-toolchain.toml @@ -0,0 +1,3 @@ +[toolchain] +channel = "1.97.1" +components = ["rustfmt", "clippy", "rust-src"] diff --git a/examples/43-rust-macros-for-systems/rust/src/main.rs b/examples/43-rust-macros-for-systems/rust/src/main.rs new file mode 100644 index 0000000..67800a1 --- /dev/null +++ b/examples/43-rust-macros-for-systems/rust/src/main.rs @@ -0,0 +1,78 @@ +//! Chapter 43's app: three metaprogramming techniques a systems programmer +//! actually reaches for. +//! +//! - `#[derive(SysError)]` (from the `sysmacros` proc-macro crate) generates +//! `Display` + `std::error::Error` for an errno-style error enum from +//! `#[error("...")]` attributes — a miniature `thiserror`. +//! - `#[instrument]` wraps `hash_chunk` to log its entry, exit, and +//! wall-clock duration — tracing without a runtime. +//! - `read_record` is a sound `unsafe` unaligned read of a `repr(C)` struct +//! out of a byte buffer, validated with miri; `--oob-read` deliberately +//! reads 4 bytes past the buffer so miri reports the undefined behaviour. + +use sysmacros::{SysError, instrument}; + +#[derive(SysError, Debug)] +enum PmonError { + #[error("open {path} failed: {errno}")] + Open { path: String, errno: i32 }, + #[error("child {pid} exited with status {code}")] + ChildExit { pid: u32, code: i32 }, + #[error("policy rejected: {0}")] + PolicyRejected(String), +} + +#[instrument] +fn hash_chunk(len: usize) -> u64 { + // Deterministic FNV-1a over 0..len — the exact value doesn't matter, only + // that it's reproducible across runs. + let mut h = 0xcbf29ce484222325u64; + for i in 0..len { + h = (h ^ i as u64).wrapping_mul(0x100000001b3); + } + h +} + +#[repr(C)] +#[derive(Clone, Copy)] +struct Record { + magic: u32, + slots: u32, +} + +// magic 0x4b565053 (LE bytes "SPVK"), slots 256 (LE). +const BUF: [u8; 8] = [0x53, 0x50, 0x56, 0x4B, 0x00, 0x01, 0x00, 0x00]; + +fn read_record(buf: &[u8], oob: bool) -> Record { + let off = if oob { 4 } else { 0 }; + // oob: deliberately reads past the 8-byte buffer. + // SOUND path checks the bounds; the --oob-read path skips the check (the + // bug miri catches). + if !oob { + assert!(buf.len() >= off + core::mem::size_of::()); + } + // SAFETY (sound path): buf has >= 8 bytes, Record is repr(C) plain-old-data, + // read_unaligned tolerates any alignment. Under --oob-read this reads 4 bytes + // past the end -> undefined behaviour, which is exactly what miri reports. + unsafe { core::ptr::read_unaligned(buf.as_ptr().add(off) as *const Record) } +} + +fn main() { + let open_err = PmonError::Open { + path: "/etc/shadow".to_string(), + errno: 13, + }; + println!("error: {open_err}"); + + let child_err = PmonError::ChildExit { pid: 4242, code: 1 }; + println!("error: {child_err}"); + + let policy_err = PmonError::PolicyRejected("sandbox blocked os.execute".to_string()); + println!("error: {policy_err}"); + + hash_chunk(4096); + + let oob = std::env::args().any(|a| a == "--oob-read"); + let rec = read_record(&BUF, oob); + println!("record: magic={:#x} slots={}", rec.magic, rec.slots); +} diff --git a/examples/43-rust-macros-for-systems/rust/sysmacros/Cargo.toml b/examples/43-rust-macros-for-systems/rust/sysmacros/Cargo.toml new file mode 100644 index 0000000..633bfed --- /dev/null +++ b/examples/43-rust-macros-for-systems/rust/sysmacros/Cargo.toml @@ -0,0 +1,12 @@ +[package] +name = "sysmacros" +version = "0.1.0" +edition = "2024" + +[lib] +proc-macro = true + +[dependencies] +syn = { version = "2.0", features = ["full"] } +quote = "1.0" +proc-macro2 = "1.0" diff --git a/examples/43-rust-macros-for-systems/rust/sysmacros/src/lib.rs b/examples/43-rust-macros-for-systems/rust/sysmacros/src/lib.rs new file mode 100644 index 0000000..027ca89 --- /dev/null +++ b/examples/43-rust-macros-for-systems/rust/sysmacros/src/lib.rs @@ -0,0 +1,164 @@ +//! `sysmacros` — the proc-macro crate backing chapter 43's two techniques: +//! +//! - `#[derive(SysError)]`: a miniature `thiserror`. Each enum variant carries +//! `#[error("...")]`; this derive generates `Display` (interpolating the +//! variant's own fields via edition-2024 inline format-arg capture) and a +//! blanket `std::error::Error` impl. +//! - `#[instrument]`: an attribute macro that wraps a free function to print +//! an entry line (name + args), time the body with `Instant`, print an exit +//! line (return value + elapsed microseconds), and return the value +//! unchanged. +//! +//! Malformed input (a `#[derive(SysError)]` on a non-enum, or a variant +//! missing `#[error(...)]`) is reported as a real `compile_error!` via +//! `syn::Error::to_compile_error()` — never a panic. + +use proc_macro::TokenStream; +use proc_macro2::TokenStream as TokenStream2; +use quote::{format_ident, quote}; +use syn::{parse_macro_input, Data, DeriveInput, Fields, ItemFn, LitStr}; + +#[proc_macro_derive(SysError, attributes(error))] +pub fn derive_sys_error(input: TokenStream) -> TokenStream { + let input = parse_macro_input!(input as DeriveInput); + match expand_sys_error(&input) { + Ok(tokens) => tokens.into(), + Err(err) => err.to_compile_error().into(), + } +} + +fn expand_sys_error(input: &DeriveInput) -> syn::Result { + let name = &input.ident; + + let data_enum = match &input.data { + Data::Enum(data_enum) => data_enum, + _ => { + return Err(syn::Error::new_spanned( + input, + "SysError can only be derived for enums", + )) + } + }; + + let mut arms = Vec::with_capacity(data_enum.variants.len()); + for variant in &data_enum.variants { + let vident = &variant.ident; + + let error_attr = variant + .attrs + .iter() + .find(|attr| attr.path().is_ident("error")) + .ok_or_else(|| { + syn::Error::new_spanned( + variant, + format!("variant `{vident}` is missing #[error(\"...\")]"), + ) + })?; + let message: LitStr = error_attr.parse_args()?; + + let arm = match &variant.fields { + Fields::Named(fields) => { + let idents: Vec<_> = fields + .named + .iter() + .map(|f| f.ident.clone().expect("named field always has an ident")) + .collect(); + // Edition-2024 inline format-arg capture: `path` and `errno` + // in the literal resolve directly to the bindings destructured + // in this match arm's pattern. + quote! { + Self::#vident { #(#idents),* } => write!(f, #message), + } + } + Fields::Unnamed(fields) => { + let idents: Vec<_> = (0..fields.unnamed.len()) + .map(|i| format_ident!("f{}", i)) + .collect(); + quote! { + Self::#vident(#(#idents),*) => write!(f, #message, #(#idents),*), + } + } + Fields::Unit => { + quote! { + Self::#vident => write!(f, #message), + } + } + }; + arms.push(arm); + } + + Ok(quote! { + impl ::std::fmt::Display for #name { + fn fmt(&self, f: &mut ::std::fmt::Formatter<'_>) -> ::std::fmt::Result { + match self { + #(#arms)* + } + } + } + + impl ::std::error::Error for #name {} + }) +} + +#[proc_macro_attribute] +pub fn instrument(_attr: TokenStream, item: TokenStream) -> TokenStream { + let func = parse_macro_input!(item as ItemFn); + match expand_instrument(func) { + Ok(tokens) => tokens.into(), + Err(err) => err.to_compile_error().into(), + } +} + +fn expand_instrument(func: ItemFn) -> syn::Result { + let ItemFn { + attrs, + vis, + sig, + block, + } = func; + + let fn_name = sig.ident.to_string(); + + let mut arg_idents = Vec::with_capacity(sig.inputs.len()); + for input in &sig.inputs { + match input { + syn::FnArg::Typed(pat_type) => match &*pat_type.pat { + syn::Pat::Ident(pat_ident) => arg_idents.push(pat_ident.ident.clone()), + other => { + return Err(syn::Error::new_spanned( + other, + "#[instrument] requires simple identifier parameters", + )) + } + }, + syn::FnArg::Receiver(receiver) => { + return Err(syn::Error::new_spanned( + receiver, + "#[instrument] does not support methods with `self`", + )) + } + } + } + + let entry_fmt = { + let args_fmt = arg_idents + .iter() + .map(|id| format!("{id}={{}}")) + .collect::>() + .join(", "); + format!("-> {fn_name}({args_fmt})") + }; + let exit_fmt = format!("<- {fn_name} = {{}} ({{}}us)"); + + Ok(quote! { + #(#attrs)* + #vis #sig { + println!(#entry_fmt, #(#arg_idents),*); + let __instrument_start = ::std::time::Instant::now(); + let __instrument_result = (move || #block)(); + let __instrument_elapsed = __instrument_start.elapsed(); + println!(#exit_fmt, __instrument_result, __instrument_elapsed.as_micros()); + __instrument_result + } + }) +} diff --git a/examples/43-rust-macros-for-systems/verify.lua b/examples/43-rust-macros-for-systems/verify.lua new file mode 100644 index 0000000..2d8ab6b --- /dev/null +++ b/examples/43-rust-macros-for-systems/verify.lua @@ -0,0 +1,44 @@ +-- Verify 43-rust-macros-for-systems (Rust only). +-- +-- Asserts the observable output of the three techniques the example teaches: +-- a #[derive(SysError)] that generates Display from #[error("...")] strings, an +-- #[instrument] attribute macro that logs a function's entry/exit/timing, and a +-- sound unsafe unaligned read. The gate runs on the pinned STABLE toolchain; +-- miri (nightly) is exercised by the chapter, not here. + +local script_dir = arg[0]:match("(.*/)") or "./" +local repo_root = os.getenv("REPO_ROOT") or (script_dir .. "../..") +package.path = package.path .. ";" .. repo_root .. "/scripts/lib/?.lua" +local checks = require("checks") + +local lang = os.getenv("LSP_LANG") +if lang ~= "rust" then + checks.skip("43-rust-macros-for-systems is Rust-only (LSP_LANG=" .. tostring(lang) .. ")") +end + +local b = checks.run("./demo.sh rust build") +checks.expect_exit(b, 0, "rust: builds (workspace: proc-macro crate + app)") + +local r = checks.run("./demo.sh rust run") +checks.expect_exit(r, 0, "rust: run exits 0") + +-- #[derive(SysError)]: Display generated from #[error("...")] with field interpolation. +checks.expect_match(r.out, "error: open /etc/shadow failed: 13", + "derive: named-field error Display") +checks.expect_match(r.out, "error: child 4242 exited with status 1", + "derive: multi-field error Display") +checks.expect_match(r.out, "error: policy rejected: sandbox blocked os%.execute", + "derive: tuple-field error Display") + +-- #[instrument]: entry line names the fn + args, exit line carries the return +-- value and a measured duration. +checks.expect_match(r.out, "%-> hash_chunk%(len=4096%)", + "instrument: entry logs fn + args") +checks.expect_match(r.out, "<%- hash_chunk = %d+ %(%d+us%)", + "instrument: exit logs return value + duration") + +-- Sound unsafe: an unaligned read of a repr(C) record out of a byte slice. +checks.expect_match(r.out, "record: magic=0x4b565053 slots=256", + "unsafe: unaligned read parses the record") + +checks.finish() diff --git a/examples/manifest.yaml b/examples/manifest.yaml index e5c15cd..dbbb7b0 100644 --- a/examples/manifest.yaml +++ b/examples/manifest.yaml @@ -176,3 +176,7 @@ examples: langs: [cpp, go, rust] mode: local timeout: 360 + - id: 43-rust-macros-for-systems + langs: [rust] + mode: local + timeout: 300