docs: publish the 2026-07-29 review, the ADR log, and retire ISSUES.md

ARCHITECTURE.md

@@ -1,6 +1,8 @@
 # Chisel — Architecture and On-Disk Format
 
-This document is the cold-start reference for someone (human or AI) picking up the Chisel codebase in a context-free session. It is a compressed map of the layers, the exact on-disk byte format, the load-bearing invariants, and the landmines. Read it whole at the start of a session. For *what Chisel does* and how to use it, see [`README.md`](README.md). For the running decision log — open issues, closed issues, every design tradeoff with date-stamped rationale — see [`ISSUES.md`](ISSUES.md).
+This document is the cold-start reference for someone (human or AI) picking up the Chisel codebase in a context-free session. It is a compressed map of the layers, the exact on-disk byte format, the load-bearing invariants, and the landmines. Read it whole at the start of a session. For *what Chisel does* and how to use it, see [`README.md`](README.md). For the design decisions and their rejected alternatives, see [`docs/adr/`](docs/adr/); for the open work, see [GitHub issues](https://github.com/pgexperts/chisel/issues).
+
+> **On `I<number>` markers.** Comments throughout the source cite issue ids like `I61 (ISSUES.md, 2026-05-22)`. Until 2026-08-03 the project tracked its issues in a git-tracked `ISSUES.md`; that file was retired in favour of GitHub issues, and its open entries were migrated there. The markers were deliberately left in place rather than rewritten across ~167 sites — they are dated provenance, and the retired file is still readable in git history (`git show 0ffe3bc:ISSUES.md`). Treat an `I<number>` as "see the decision log as it stood on that date", not as a live link.
 
 For the theory of operation and the rationale behind these decisions — why shadow-paging over WAL, the rejected alternatives, the implementation history — see [`THEORY.md`](THEORY.md).
 

README.md

@@ -293,7 +293,7 @@ On create, Chisel generates a random data-encryption key (DEK), encrypts every p
 
 The wrapped DEK lives in an **8-slot key table**. Because the DEK itself never changes, credential rotation only re-wraps the DEK in a slot — it is O(1), independent of database size. `add_key` stages a second credential (both open the DB), `rotate_key` replaces one credential in place, and `remove_key` retires one (refusing the last remaining slot with `LastKeySlot`). A full table returns `NoFreeKeySlot`.
 
-See [ARCHITECTURE.md#on-disk-encryption](ARCHITECTURE.md#on-disk-encryption) for the on-disk layout (crypto header, key slots, per-page nonce stride) and [THEORY.md](THEORY.md) for the rationale behind the envelope scheme and the shadow-paging nonce discipline (with [ISSUES.md](ISSUES.md) as the dated decision log).
+See [ARCHITECTURE.md#on-disk-encryption](ARCHITECTURE.md#on-disk-encryption) for the on-disk layout (crypto header, key slots, per-page nonce stride) and [THEORY.md](THEORY.md) for the rationale behind the envelope scheme and the shadow-paging nonce discipline (with [`docs/adr/`](docs/adr/) as the dated decision log).
 
 ## API reference
 
@@ -473,7 +473,8 @@ db.remove_key("correct horse battery staple")                          # retire
 
 - [`ARCHITECTURE.md`](ARCHITECTURE.md) — living architecture overview: layer model, commit protocol, recovery, full on-disk format byte-by-byte, and cross-cutting concepts. Start here if you're reading the codebase to *act* on it.
 - [`THEORY.md`](THEORY.md) — theory of operation: *why* the design is what it is — the load-bearing decisions, the rejected alternatives, and the implementation history. Read this to build a durable model before changing the engine.
-- [`ISSUES.md`](ISSUES.md) — running decision log: open issues, closed issues, and every design tradeoff with date-stamped rationale.
+- [`docs/adr/`](docs/adr/) — architecture decision records: one file per decision, with the context and the alternatives that were rejected.
+- [GitHub issues](https://github.com/pgexperts/chisel/issues) — the running issue log. This replaced a tracked `ISSUES.md` on 2026-08-03; `I<number>` markers in code comments refer to entries in that retired file, which remains readable in git history.
 
 ## License
 

THEORY.md

@@ -161,7 +161,7 @@ There is one **critical, revised** sub-decision that is easy to get wrong and wo
 
 **Why.** Envelope encryption makes credential rotation O(1) — you re-wrap the DEK — instead of O(database size). The per-slot KDF choice matches input entropy: HKDF is fast and correct for high-entropy keys, while Argon2id is memory-hard to resist brute-forcing low-entropy passphrases (its params are recorded per slot). And every rotation op is an ordinary superblock A/B + fsync commit, so it reuses the existing crash-safe protocol wholesale: a metadata-only `rewrite_crypto_header` commit persists a rotated slot table atomically (write the inactive slot, fsync, promote), so a crash mid-rotation leaves the old table intact.
 
-Two threat-model boundaries are documented rather than solved, and you should know them before you rely on this: there is **no rollback/replay resistance** (an attacker who substitutes a wholly older, validly-signed image is undetectable without an external trust anchor like a TPM), and the DEK sits in plaintext in process memory during a session (mitigated by zeroize-on-drop, not by encryption). See spec `2026-06-29` §3/§5/§9 and ISSUES.md I142.
+Two threat-model boundaries are documented rather than solved, and you should know them before you rely on this: there is **no rollback/replay resistance** (an attacker who substitutes a wholly older, validly-signed image is undetectable without an external trust anchor like a TPM), and the DEK sits in plaintext in process memory during a session (mitigated by zeroize-on-drop, not by encryption). See spec `2026-06-29` §3/§5/§9 and [issue #140](https://github.com/pgexperts/chisel/issues/140) (the deferred bulk DEK rotation, formerly I142).
 
 ### Encryption page format: 8232-byte stride, logical page stays 8192, MAJOR 1→2 (ADR-15)
 

docs/adr/0000-decision-register-overview.md

@@ -0,0 +1,34 @@
+---
+id: 0000
+title: Decision register (overview)
+date: 2026-05-04
+status: Accepted
+---
+
+# 0000. Decision register (overview)
+
+Chisel is a single-writer embedded transactional storage engine in Rust. The decisions below are the ones that, if reversed, would require rewriting substantial parts of the engine. Smaller decisions (specific bit layouts, error message wording, individual issue resolutions) are tracked as [GitHub issues](https://github.com/pgexperts/chisel/issues); before 2026-08-03 they lived in a tracked `ISSUES.md`, which ADR [0017](0017-github-issues-replace-tracked-issues-md.md) retired.
+
+**Reading `ISSUES.md` citations.** Records 0006, 0007, 0012, 0013 and 0015 predate that retirement and cite `ISSUES.md` entries by `I<number>`. Those bodies are left as written — an Accepted record is superseded, not edited, and the citations are accurate as dated provenance. Resolve one with `git show 0ffe3bc:ISSUES.md`, the last commit carrying the file.
+
+| # | Decision | Status | Reversibility |
+|---|---|---|---|
+| 1 | Shadow paging, not WAL | Accepted | Hard — touches commit protocol, recovery, every page-mutation path |
+| 2 | Single-writer enforced by `&mut self` | Accepted | Hard — every API signature would change |
+| 3 | Per-module COW (no centralized abstraction) | Accepted | Medium — affects 5 modules |
+| 4 | N rotating superblocks (configurable 2..=16) | Accepted | Hard — recovery and commit both depend |
+| 5 | Spillway sidecar file over hard ceiling | Accepted (2026-05-04) | Medium — supersedes `HARD_CEILING_MULTIPLIER` |
+| 6 | Poison model on fatal errors | Accepted | Easy — could be relaxed, but Linux fsyncgate semantics make retry unsafe regardless |
+| 7 | Two-tier format versioning (file MAJOR/MINOR + per-page byte) | Accepted | Hard — affects every page header |
+| 8 | In-memory mode via `Vec<u8>`-backed PageIo | Accepted | Easy — additive; could be removed |
+| 9 | Counter instrumentation via `Chisel::counters()` | Accepted (PR 1, bench-suite) | Easy — additive, `#[non_exhaustive]` |
+| 10 | Bench-suite series (cross-engine comparison + dedicated machine foundation) | Accepted (PRs 1-8 shipped 2026-04-30 → 2026-05-04) | Easy — bench/ is a sibling crate; no engine impact |
+| 11 | macOS-fsync fairness via `PRAGMA fullfsync=ON` on SqliteEngine | Accepted (PR 8, 2026-05-04) | Easy — bench-side only |
+| 12 | Chunk tags + reverse membership index | Accepted (2026-06-02) | Medium — new on-disk subsystem; additive format (MINOR) |
+| 13 | Within-session iteration-stability contract | Accepted (2026-06-04) | Easy — documents existing behavior; public API contract |
+| 14 | Client byte — opaque per-chunk u8 in the last reserved entry byte | Accepted (2026-06-05) | Easy — additive; reuses reserved byte [15], no format change |
+| 15 | On-disk encryption (XChaCha20-Poly1305, envelope DEK/KEK, MAJOR=2) | Accepted (2026-06-30) | Hard — first MAJOR format bump; encrypted-stride + sealed superblock + page-I/O seal seam |
+
+The body of this ADR walks each decision in turn.
+
+---

docs/adr/0001-shadow-paging-not-wal.md

@@ -0,0 +1,27 @@
+---
+id: 0001
+title: Shadow paging, not WAL
+date: unknown
+status: Accepted
+---
+
+# 0001. Shadow paging, not WAL
+
+**Context:** Two dominant approaches to ACID durability exist for embedded engines. Write-ahead log (WAL) writes intent records to a sequential journal first, then applies changes to the data file in place; recovery replays unfinished log entries. Shadow paging writes new versions of mutated pages to fresh page slots, leaves old pages intact, and atomically swaps a root pointer to make the new state visible; recovery picks the most recent valid root.
+
+**Decision:** Shadow paging. Every mutation allocates a fresh page via `PageCache::new_page`; the previously-committed page stays intact at its original position. Commit writes the new pages' bytes, fsyncs, then writes a new superblock to a different slot than the currently-active one and fsyncs again. Recovery on open is `Superblock::select` over the N candidate slots, picking the one with the highest valid `txn_counter`.
+
+**Alternatives considered:**
+
+- *WAL with in-place updates.* Standard for production-grade DBs (PostgreSQL, SQLite). Rejected for v1: WAL recovery is a substantial subsystem (replay state machine, checkpoint handling, log truncation) that adds risk surface comparable to the entire rest of Chisel. Shadow paging trades disk space (live + previous version of every mutated page until commit) for code simplicity.
+- *Hybrid (WAL for small writes, shadow for large).* Considered briefly, rejected as combining the worst of both — recovery code paths multiply and the boundary between modes becomes another correctness obligation.
+
+**Consequences:**
+
+- *Positive:* No log replay; recovery is one read of N superblock slots plus checksum validation. The "is this database open" check is the same code path as crash recovery. Crash safety is provable by inspection: any state where the previous superblock is intact remains recoverable, and `fsync` ordering ensures the new superblock isn't durable until its referenced data pages are.
+- *Positive:* COW is a natural fit. Every mutation produces a new page; transactions are simply "the set of new pages plus a candidate new superblock." Rollback is "discard the new pages and the new superblock."
+- *Negative:* Disk space cost. Updating a single byte of a page costs an entire new page (8 KB) until the next commit, when the old page becomes freeable. Workloads that write small deltas to many pages have high write amplification.
+- *Negative:* Defragmentation becomes necessary over time. `defrag.rs` exists for this.
+- *Locked-in:* Reverting to WAL would require rewriting `transaction.rs`, `page_cache.rs` (no more "fresh page per mutation"), and the recovery path in `lib.rs`.
+
+---

docs/adr/0002-single-writer-enforced-by-mut-self.md

@@ -0,0 +1,27 @@
+---
+id: 0002
+title: Single-writer enforced by `&mut self`
+date: unknown
+status: Accepted
+---
+
+# 0002. Single-writer enforced by `&mut self`
+
+**Context:** Embedded databases face a choice: single-writer (one mutator at a time, often with multiple concurrent readers) vs. multi-writer (transactions interleave, requiring locking, MVCC, or both). The choice affects the API surface, the storage format (MVCC needs version chains), the recovery model, and the testing burden.
+
+**Decision:** Single-writer, single-process. Enforced at three levels: (a) the OS via exclusive `flock` in `page_io.rs`, (b) the type system via `&mut self` on every mutating Chisel API, (c) explicit project-memory note that this is *philosophical*, not a v1 simplification.
+
+**Alternatives considered:**
+
+- *Multi-writer with internal locking.* Would require RwLock or Mutex around `PageCache`, transaction-conflict detection, deadlock handling. Roughly doubles the engine's complexity.
+- *MVCC.* Adds version chains to every page, garbage-collection responsibilities, snapshot-isolation semantics. Out of scope for an embedded single-process engine.
+- *Single-writer at v1, multi-writer at v2.* Rejected because the `&mut self` API is load-bearing — relaxing it later would be a breaking change for every consumer, and the type system encodes the invariant in a way internal locking cannot.
+
+**Consequences:**
+
+- *Positive:* No internal locking. `RefCell<PageCache>` (not `Mutex`) inside `TransactionManager` lets `read()` / `handles()` / `stats()` take `&self` without external wrapping; the borrow checker handles the rest.
+- *Positive:* The type system makes "two concurrent transactions" impossible to express. There is no test for it because there is no API for it.
+- *Negative:* Workloads that need concurrent writers must serialize at a higher layer (e.g., Exilis does this via its own `RefCell<Chisel>` inside the storage backend).
+- *Locked-in:* See above. Multi-writer would be a v2.0 breaking change, not a minor.
+
+---

docs/adr/0003-per-module-cow-no-centralized-abstraction.md

@@ -0,0 +1,26 @@
+---
+id: 0003
+title: Per-module COW, no centralized abstraction
+date: unknown
+status: Accepted
+---
+
+# 0003. Per-module COW, no centralized abstraction
+
+**Context:** Multiple modules need copy-on-write semantics: the handle table (radix tree) must clone the path from root to a modified leaf, the freemap rewrites itself on every commit's `persist_freemap`, data pages reuse the same page across commits via `claim_page`. A centralized COW abstraction (a trait, a generic page-mutation type) would seem to factor out repeated logic.
+
+**Decision:** Each module implements its own COW. `handle_table.rs` clones the root-to-leaf path. `freemap.rs` allocates a new freemap page in `persist_freemap`. `data_page.rs` mutates in place via `claim_page` (which takes `&mut PageBytes` from `PageCache::write_page`). No `trait Cow` or `enum CowStrategy`.
+
+**Alternatives considered:**
+
+- *Centralized `trait Cow` over all page-type modules.* Would require uniform interface (e.g., `fn cow_root(&mut self, cache, root_id) -> Result<u64>`). Rejected because the modules' actual COW shapes differ enough that the trait would either be too generic (lose useful information) or too specific (have variants that work for only one module).
+- *Generic page-mutation type that wraps a strategy.* Same problem as above plus the ergonomic cost of generics in the public API.
+
+**Consequences:**
+
+- *Positive:* Each module's COW logic is co-located with the page-type logic it serves. Reading `handle_table.rs` shows you both the radix tree algorithm and the COW it implements.
+- *Positive:* Freedom to evolve. The handle table's COW grew several optimizations (`grow()`, short-circuit at depth boundaries) without affecting other modules.
+- *Negative:* Repeated boilerplate across 3-4 modules. Each writes its own "allocate a new page, write the new state, return the new page ID."
+- *Negative:* Onboarding cost. A new contributor wonders "where is the COW abstraction?" and the answer is "there isn't one, and that's deliberate."
+
+---

docs/adr/0004-n-rotating-superblocks-for-atomic-commit.md

@@ -0,0 +1,27 @@
+---
+id: 0004
+title: N rotating superblocks for atomic commit
+date: unknown
+status: Accepted
+---
+
+# 0004. N rotating superblocks for atomic commit
+
+**Context:** The commit protocol's atomicity hinges on swapping a root pointer in a single durable write. The simplest implementation is a single superblock at offset 0, overwritten on every commit. But a single superblock is vulnerable: a torn write (kernel buffered the new bytes but crashed before all of them reached disk) leaves the file unrecoverable.
+
+**Decision:** N superblocks (configurable at create time via `Options::superblock_count`, range 2..=16, default 2) occupy file offsets 0..N. Commit writes to slot `txn_counter % N` — always the slot with the lowest `txn_counter` among the surviving N. Recovery (`Superblock::select`) reads all N slots, validates each (magic + checksum + `superblock_count` in range), and picks the highest valid `txn_counter`.
+
+**Alternatives considered:**
+
+- *Single superblock with double-write buffer.* PostgreSQL-style approach (every page is written twice, once to a buffer area and once in place). Rejected because shadow paging already provides the same guarantee for data pages — the only page that needs the double-write is the superblock itself, and N rotating slots is conceptually simpler than maintaining a separate buffer area.
+- *Write-side journal for the superblock.* Mini-WAL just for the superblock. Rejected: same complexity argument as ADR-1.
+
+**Consequences:**
+
+- *Positive:* Trivial torn-write recovery. A torn slot fails the checksum; `Superblock::select` ignores it and picks the previous slot. Higher N (3..16) survives consecutive torn writes.
+- *Positive:* No separate journal. The superblock's own slots ARE the journal.
+- *Positive:* Configurable space/durability tradeoff. The user picks N at create time based on their crash tolerance.
+- *Negative:* N pages of overhead at the start of every file. With default N=2 and 8 KB pages, that's 16 KB minimum file size before any data.
+- *Locked-in:* The on-disk layout reserves the first N pages for superblocks; changing N for an existing file would require migration.
+
+---