diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index a37f5ea..4c69f37 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -712,9 +712,11 @@ The DEK wrapping uses detached XChaCha20-Poly1305 with AAD bound to the slot's K - `add_key(old_key, new_key)`: derives a new KEK, wraps the same DEK into a free slot, then commits. - `rotate_key(old_key, new_key)`: `add_key` followed by clearing the old slot in the same commit. +- Every key-slot rewrite also scrubs the key-slot region of the OTHER superblock slots (`CryptoHeader::overwrite_slot_table`). Without that, a revoked credential's wrapped DEK survived verbatim in a sibling slot of the live file — cleartext at bytes 332..1356, wrapping a DEK that never changes — so read access plus the revoked credential was enough to recover the current DEK. Only the table region is patched, so each sibling keeps its own counter, roots and sealed body; the body's AAD covers magic/format_version/txn_counter/superblock_count, not the crypto header. +- `Chisel::rekey(path, key, argon2)` (ADR 0018) is the bulk alternative: a fresh DEK, every page re-sealed, the file replaced by atomic rename. It is an associated function on a path rather than a method because the rename invalidates any descriptor opened beforehand, and it collapses the key-slot table to the supplied credential because a slot's KEK cannot be re-derived without its own credential. - `remove_key(key)`: clears the matching slot, refusing to clear the last active slot (which would make the database permanently unreadable). -Bulk DEK rotation (re-encrypting every page under a fresh DEK) is deferred; see I142. +Bulk DEK rotation is implemented as `Chisel::rekey` (ADR 0018) — the operation for a compromised DEK, as opposed to a compromised credential. It is offline and takes a path, builds the rotated database in a scratch file, and publishes it with an atomic rename; it collapses the key-slot table to the single supplied credential, because a slot's KEK cannot be re-derived without its own credential. **Spillway.** For encrypted databases the in-memory spillway carries sealed blobs: pages are encrypted exactly once on eviction (`seal` on evict-to-spillway) and copied verbatim — no decryption or re-encryption — on drain to the main file. Rehydration decrypts the blob back into the cache. No plaintext page content is ever written to disk by an encrypted database, even during spill. diff --git a/README.md b/README.md index d643d84..6b6e793 100644 --- a/README.md +++ b/README.md @@ -25,7 +25,7 @@ Chisel is designed for single-writer embedded use: one process holds the file vi - **Named roots** — a small fixed table in the superblock mapping string names to handles. Survives commit / rollback transactionally. - **Defragmentation** — explicit `defrag()` consolidates sparse pages and returns a count-based stats record. - **In-memory mode** — same engine, `Vec`-backed I/O, no file and no lock. For tests, benchmarks, and ephemeral work. -- **At-rest encryption** — optional, off by default. Every page is sealed with XChaCha20-Poly1305 AEAD under a client-supplied key (raw 32-byte or Argon2id passphrase). An 8-slot envelope table wraps the data-encryption key, so credential rotation is O(1) — no bulk re-encryption. +- **At-rest encryption** — optional, off by default. Every page is sealed with XChaCha20-Poly1305 AEAD under a client-supplied key (raw 32-byte or Argon2id passphrase). An 8-slot envelope table wraps the data-encryption key, so credential rotation is O(1) — no bulk re-encryption. `rekey()` performs the heavy alternative when the data key itself is compromised: a crash-safe whole-file re-encryption under a fresh key. - **Poison model** — any fatal error (I/O failure, checksum mismatch, commit-protocol failure) poisons the handle; recovery is drop-and-reopen. Mirrors `std::sync::Mutex` poisoning. - **Single-writer** — exclusive `flock` at the filesystem level; `&mut self` on every mutating method. @@ -293,6 +293,12 @@ On create, Chisel generates a random data-encryption key (DEK), encrypts every p The wrapped DEK lives in an **8-slot key table**. Because the DEK itself never changes, credential rotation only re-wraps the DEK in a slot — it is O(1), independent of database size. `add_key` stages a second credential (both open the DB), `rotate_key` replaces one credential in place, and `remove_key` retires one (refusing the last remaining slot with `LastKeySlot`). A full table returns `NoFreeKeySlot`. +Rotating a *credential* is not the same as rotating the *data key*. Because the DEK never changes, `rotate_key` and `remove_key` deny a credential a way in — they do not cut it off from data it has already seen, and they cannot help if the DEK itself leaked (a process memory dump, a core file). `Chisel::rekey(path, key, argon2_params)` is the operation for that case: it generates a fresh DEK, re-encrypts every page, and replaces the file atomically. + +`rekey` takes a **path, not an open handle**, and that is deliberate: it replaces the file by `rename`, so a descriptor opened beforehand would afterwards refer to the original, now-unlinked inode. Close the database, rekey, reopen. It is crash-safe — the rotated database is built in a scratch file beside the original and published atomically, so the path only ever names a complete file — and it costs O(total_pages) I/O plus a transient second copy on disk. + +It also **collapses the key-slot table to the single credential you supply**. That is forced rather than chosen: each slot's KEK is derived from its own credential, so wrapping the new DEK for a slot requires the credential that slot belongs to. Re-add the others with `add_key` afterwards. It is the safer default anyway — if you are rotating because the DEK leaked, preserving every credential that could reach it is not the goal. + See [ARCHITECTURE.md#on-disk-encryption](ARCHITECTURE.md#on-disk-encryption) for the on-disk layout (crypto header, key slots, per-page nonce stride) and [THEORY.md](THEORY.md) for the rationale behind the envelope scheme and the shadow-paging nonce discipline (with [`docs/adr/`](docs/adr/) as the dated decision log). ## API reference @@ -332,6 +338,7 @@ See [ARCHITECTURE.md#on-disk-encryption](ARCHITECTURE.md#on-disk-encryption) for | `defrag(options)` | Consolidate sparse pages | | `add_key(existing, new)` | Stage a second credential; both `existing` and `new` then open the DB. `Result<()>` | | `rotate_key(old, new)` | Replace credential `old` with `new` in place. `Result<()>` | +| `Chisel::rekey(path, key, argon2)` | Re-encrypt the whole database under a fresh DEK. Associated function, not a method — takes a path. `Result<()>` | | `remove_key(key)` | Retire credential `key`; `LastKeySlot` if it is the only one. `Result<()>` | ## Options @@ -469,6 +476,12 @@ db.remove_key("correct horse battery staple") # retire `add_key` / `rotate_key` raise `NoFreeKeySlotError` when the 8-slot table is full; `remove_key` raises `LastKeySlotError` rather than leaving the DB with no usable credential. +```python +chisel.rekey("my.db", key=b"\x00" * 32) # whole-file re-encryption under a fresh data key +``` + +`chisel.rekey` is a module-level function, like `chisel.open`, because it names a database by path rather than assuming one is open — it replaces the file, so any handle held across the call would be stale. Every credential other than the one supplied is revoked; re-add them with `add_key`. + ## Design documents - [`ARCHITECTURE.md`](ARCHITECTURE.md) — living architecture overview: layer model, commit protocol, recovery, full on-disk format byte-by-byte, and cross-cutting concepts. Start here if you're reading the codebase to *act* on it. diff --git a/THEORY.md b/THEORY.md index 08676db..0c82217 100644 --- a/THEORY.md +++ b/THEORY.md @@ -155,13 +155,13 @@ There is one **critical, revised** sub-decision that is easy to get wrong and wo ### Encryption keys: envelope DEK/KEK with an 8-slot table, O(1) rotation (ADR-15) -**Chosen.** A random per-database 256-bit DEK (from `OsRng` at create) seals every page and the sensitive superblock body. The DEK is never stored bare — it is wrapped under a KEK derived from the client key (HKDF-SHA256 for raw keys, Argon2id for passphrases) and held in an 8-slot key-slot table in the superblock's plaintext reserved region. `add_key` / `rotate_key` / `remove_key` re-wrap the *stable* DEK — O(1), no data re-encryption. `rotate_key` stages the new slot before revoking the old (no zero-key window); `remove_key` refuses to clear the last active slot (brick prevention). A successful unwrap *is* proof the client key is correct — there is no separate password verifier. +**Chosen.** A random per-database 256-bit DEK (from `OsRng` at create) seals every page and the sensitive superblock body. The DEK is never stored bare — it is wrapped under a KEK derived from the client key (HKDF-SHA256 for raw keys, Argon2id for passphrases) and held in an 8-slot key-slot table in the superblock's plaintext reserved region. `add_key` / `rotate_key` / `remove_key` re-wrap the *stable* DEK — O(1), no data re-encryption. `rotate_key` stages the new slot before revoking the old (no zero-key window); `remove_key` refuses to clear the last active slot (brick prevention). A successful unwrap *is* proof the client key is correct — there is no separate password verifier. Rotating a credential is not rotating the data key: the DEK is unchanged, so revocation denies entry rather than re-keying, and anyone who captured the DEK while the credential was valid keeps reading. `rekey` (ADR 0018) is the answer when the DEK itself is what leaked — a whole-file re-encryption published by atomic rename, since shadow paging cannot make an in-place rewrite crash-safe. -**Rejected.** Encrypting only the data pages and leaving the superblock plaintext — rejected because `named_roots` holds user-chosen names, which are real user data a plaintext body would leak. Full DEK rotation (re-encrypting every page under a fresh DEK) — deferred (I142) as a heavy O(total_pages) whole-file operation reserved for "the DEK itself is compromised"; credential rotation is the far more common need and is O(1). +**Rejected.** Encrypting only the data pages and leaving the superblock plaintext — rejected because `named_roots` holds user-chosen names, which are real user data a plaintext body would leak. Full DEK rotation (re-encrypting every page under a fresh DEK) — implemented separately as `rekey` (ADR 0018) rather than folded into the credential path, because the two answer different questions: credential rotation is the far more common need and is O(1), while a whole-file re-encryption is reserved for "the DEK itself is compromised" and is O(total_pages). **Why.** Envelope encryption makes credential rotation O(1) — you re-wrap the DEK — instead of O(database size). The per-slot KDF choice matches input entropy: HKDF is fast and correct for high-entropy keys, while Argon2id is memory-hard to resist brute-forcing low-entropy passphrases (its params are recorded per slot). And every rotation op is an ordinary superblock A/B + fsync commit, so it reuses the existing crash-safe protocol wholesale: a metadata-only `rewrite_crypto_header` commit persists a rotated slot table atomically (write the inactive slot, fsync, promote), so a crash mid-rotation leaves the old table intact. -Two threat-model boundaries are documented rather than solved, and you should know them before you rely on this: there is **no rollback/replay resistance** (an attacker who substitutes a wholly older, validly-signed image is undetectable without an external trust anchor like a TPM), and the DEK sits in plaintext in process memory during a session (mitigated by zeroize-on-drop, not by encryption). See spec `2026-06-29` §3/§5/§9 and [issue #140](https://github.com/pgexperts/chisel/issues/140) (the deferred bulk DEK rotation, formerly I142). +Two threat-model boundaries are documented rather than solved, and you should know them before you rely on this: there is **no rollback/replay resistance** (an attacker who substitutes a wholly older, validly-signed image is undetectable without an external trust anchor like a TPM), and the DEK sits in plaintext in process memory during a session (mitigated by zeroize-on-drop, not by encryption). See spec `2026-06-29` §3/§5/§9. Note that `rekey` (ADR 0018) does bear on the first boundary: it invalidates every page image sealed under the old DEK, so it ends an attacker's ability to splice in stale pages captured beforehand — it does not, however, make the engine detect such splicing, which remains open as [issue #142](https://github.com/pgexperts/chisel/issues/142). ### Encryption page format: 8232-byte stride, logical page stays 8192, MAJOR 1→2 (ADR-15) diff --git a/docs/adr/0018-bulk-dek-rotation-via-copy-and-rename.md b/docs/adr/0018-bulk-dek-rotation-via-copy-and-rename.md new file mode 100644 index 0000000..7bb3604 --- /dev/null +++ b/docs/adr/0018-bulk-dek-rotation-via-copy-and-rename.md @@ -0,0 +1,109 @@ +--- +id: 0018 +title: Bulk DEK rotation as an offline copy-then-rename operation on a path +date: 2026-08-04 +status: Accepted +summary: rekey() re-encrypts the whole database under a fresh DEK by building a replacement file and renaming it into place; it takes a path rather than a handle, and collapses the key-slot table to the single supplied credential. +--- + +# 0018. Bulk DEK rotation as an offline copy-then-rename operation on a path + +## Context + +ADR [0015](0015-on-disk-encryption-xchacha20-poly1305-envelope-keys.md) chose an +envelope scheme: one per-database DEK seals every page, and up to eight key +slots each wrap that DEK under a KEK derived from a client credential. That +makes *credential* rotation O(1) — `add_key`/`rotate_key`/`remove_key` touch +only the superblock, and no page is re-encrypted. It was recorded at the time +that full DEK rotation was deferred. + +Deferred is not the same as unnecessary. The two operations answer different +questions, and only one of them was answerable: + +- a credential leaked → deny that credential a way in → credential rotation +- **the DEK itself leaked** (process memory dump, core file, attached debugger) + → the data must be re-sealed under a key the attacker does not have + +Credential rotation cannot help with the second case at all, because the DEK it +re-wraps is the very thing that leaked. The fix for CRYPTO-1 sharpened the point +by documenting it in `rotate_key`: revocation denies entry, it does not +re-key. Without a bulk rotation there was no operation in the crate that did. + +Two things made the design non-obvious. + +**Crash safety cannot lean on shadow paging.** Shadow paging protects writes +that go to *new* pages; a DEK rotation rewrites every page where it already is. +A crash halfway through leaves some pages sealed under the new DEK and some +under the old, with the surviving superblock naming one of them — an +unrecoverable mix, and precisely the kind of half-state the engine otherwise +never produces. + +**The operation invalidates its own handle.** Any strategy that replaces the +file by rename leaves a previously-opened file descriptor pointing at the +original, now-unlinked inode. Reads and writes through it would silently target +a deleted file. + +## Decision + +We will implement `Chisel::rekey(path, key, argon2_params)` as an **offline +operation on a path**, which builds the rotated database in a scratch file +beside the original and publishes it with an atomic `rename`. + +Concretely: open normally (validating the key and taking the exclusive flock), +generate a fresh DEK, write every page into `.rekey-tmp` — superblock slots +rebuilt from the winning superblock under a new crypto header, every other page +opened under the old DEK and re-sealed under the new one *at the same page id* — +`fsync`, `rename`, `fsync` the directory, then drop the handle. + +The rotated database carries **exactly one key slot**: the credential supplied. + +## Alternatives considered + +- **In-place two-pass rotation.** Rejected: not crash-safe for the reason + above, and making it so would need a journal — a second durability mechanism + in an engine whose entire premise (ADR + [0001](0001-shadow-paging-not-wal.md)) is that it does not have one. + +- **`rekey(&mut self)` on a live handle.** Rejected: it would hand back a + handle whose file descriptor names a deleted inode. Making that safe means + swapping the descriptor and re-acquiring the flock underneath a live cache — + real complexity to preserve an ergonomic nicety on an operation that rewrites + the entire file and is not something anyone runs in a loop. + +- **`rekey(self)` consuming the handle.** Closer, and it does dispose of the + stale-descriptor hazard, but `Chisel` does not retain its path, so the caller + would have to pass it back in and could pass the wrong one. A path-taking + associated function has one source of truth and reads as what it is. + +- **Re-wrapping the new DEK into all currently-active slots.** Not + implementable, not merely declined: each slot's KEK is derived from *its own* + credential, and only one was supplied. There is no way to produce a valid wrap + for a credential you do not hold. It is also the safer default — if you are + rotating because the DEK leaked, silently preserving every credential that + could reach it is not what you want. `add_key` restores the others. + +- **Keeping the older superblock slots' historical roots.** Declined. Their + sealed bodies are under the old DEK, so preserving them means re-sealing + states that are already unreachable. Every slot in the new file carries the + winning superblock's roots at staggered counters, exactly as `create_new` + seeds a fresh bank. + +## Consequences + +- Rotation costs O(total_pages) I/O and, transiently, a second copy of the + database on disk. It is maintenance, not routine operation. +- The scratch file is created `O_EXCL | O_NOFOLLOW` mode 0600, matching the + hardening the database and spillway received, so a planted file at the + predictable scratch path cannot be adopted or followed. +- The directory `fsync` after the rename is best-effort: the file contents are + already durable, so a failure costs rename durability across a crash rather + than correctness. +- `rekey` must live outside `transaction/`, since it reaches across an open + handle and the filesystem at once. It is its own module. +- Because it is a free function on a path, the PyO3 binding exposes it as a + module-level `chisel.rekey(path, key)` mirroring `chisel.open()` rather than + as a method — the same reasoning carried across the boundary. +- Rotation does not defend against per-page temporal replay + ([#142](https://github.com/pgexperts/chisel/issues/142)); it does, however, + invalidate every previously-sealed page image, so it is the operation that + ends an attacker's ability to splice stale pages sealed under the old DEK. diff --git a/docs/adr/README.md b/docs/adr/README.md index 058ebca..6361ac8 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -22,3 +22,4 @@ | [0015](0015-on-disk-encryption-xchacha20-poly1305-envelope-keys.md) | On-disk encryption (XChaCha20-Poly1305, envelope keys) | Accepted | 2026-06-30 | | | [0016](0016-swift-binding-via-uniffi.md) | Swift binding via UniFFI | Accepted | 2026-07-19 | The iOS/macOS Swift binding is a UniFFI-generated FFI over an Arc> wrapper crate, with the engine crate left unchanged. | | [0017](0017-github-issues-replace-tracked-issues-md.md) | Track issues in GitHub, not a tracked ISSUES.md | Accepted | 2026-08-03 | The 1868-line ISSUES.md decision log was retired; open entries were migrated to GitHub issues and the file deleted. | +| [0018](0018-bulk-dek-rotation-via-copy-and-rename.md) | Bulk DEK rotation as an offline copy-then-rename operation on a path | Accepted | 2026-08-04 | rekey() re-encrypts the whole database under a fresh DEK by building a replacement file and renaming it into place; it takes a path rather than a handle, and collapses the key-slot table to the single supplied credential. | diff --git a/python/chisel/__init__.py b/python/chisel/__init__.py index cb61a4f..a51fb1e 100644 --- a/python/chisel/__init__.py +++ b/python/chisel/__init__.py @@ -17,6 +17,7 @@ Savepoint, DrainInsertion, open, + rekey, ChiselError, OperationalError, FatalError, @@ -130,7 +131,7 @@ class DefragStats: __all__ = [ "__version__", - "Chisel", "Transaction", "Savepoint", "DrainInsertion", "open", + "Chisel", "Transaction", "Savepoint", "DrainInsertion", "open", "rekey", "Stats", "Counters", "DefragOptions", "DefragStats", "ChiselError", "OperationalError", "FatalError", "InvalidHandleError", "NoActiveTransactionError", diff --git a/python/chisel/__init__.pyi b/python/chisel/__init__.pyi index a3ea61c..4d98448 100644 --- a/python/chisel/__init__.pyi +++ b/python/chisel/__init__.pyi @@ -142,6 +142,36 @@ def open( ) -> Chisel: ... +def rekey(path: str | os.PathLike[str], key: bytes | str) -> None: + """Re-encrypt an entire database under a freshly generated data key. + + Takes a path rather than an open handle, and mirrors :func:`open` for that + reason: it rewrites the whole file and replaces it by rename, so a handle + opened beforehand would afterwards refer to the original, now-unlinked + inode. Close any open database first, call this, then reopen. + + Use this only when the DATA key itself is believed compromised. If a + credential leaked, :meth:`Chisel.rotate_key` is the right tool -- it is + O(1) and touches only the superblock. + + Every credential other than ``key`` is revoked, because each key slot's + wrapping key is derived from its own credential and only ``key`` was + supplied. Add the others back with :meth:`Chisel.add_key` afterwards. + + For a passphrase credential the Argon2 cost parameters are INHERITED from + the slot ``key`` currently unlocks, so a deliberately hardened database is + not silently downgraded to the defaults by the rotation. + + Crash-safe: the rotated database is built beside the original and published + with an atomic rename, so the path only ever names a complete file. + + Raises: + DatabaseFileNotFoundError: no database at ``path``. + EncryptionNotSupportedError: the database is not encrypted. + InvalidEncryptionKeyError: ``key`` unlocks no key slot. + """ + + class Chisel: @property def is_poisoned(self) -> bool: ... diff --git a/python/src/db.rs b/python/src/db.rs index 15ddc02..a23dc5d 100644 --- a/python/src/db.rs +++ b/python/src/db.rs @@ -681,9 +681,45 @@ fn closed_err() -> PyErr { crate::errors::ClosedError::new_err("database handle has been closed") } +/// Bulk DEK rotation — re-encrypt an entire database under a fresh data key. +/// +/// A module-level function rather than a `Chisel` method because it operates on +/// a PATH, not an open handle: it rewrites the whole file and replaces it by +/// rename, so any handle opened beforehand would afterwards refer to the +/// original, now-unlinked inode. Mirrors `chisel.open()` in shape for that same +/// reason — both name a database by path rather than assuming one is open. +/// +/// I156: the GIL is released for the duration. This is O(total_pages) I/O plus +/// an Argon2id derivation for a passphrase credential; holding the GIL across +/// it would freeze every other Python thread for the length of a whole-file +/// rewrite, which is the worst case of exactly the problem I156 describes. +#[pyfunction] +#[pyo3(signature = (path, key))] +pub fn rekey(py: Python<'_>, path: Py, key: Py) -> PyResult<()> { + // Coerce both arguments under the GIL, before detaching, so a bad type + // raises a synchronous Python TypeError — same discipline as open(). + let path_buf: PathBuf = { + let bound = path.bind(py); + let s: String = if let Ok(py_str) = bound.cast::() { + py_str.to_str()?.to_owned() + } else { + let os = py.import("os")?; + os.call_method1("fspath", (path,))?.extract()? + }; + PathBuf::from(s) + }; + // Zeroizing on coercion, before any engine call, so key material never + // reaches an error message, traceback, or repr. + let key = py_key(key.bind(py))?; + + py.detach(|| chisel::Chisel::rekey(&path_buf, &key, None)) + .map_err(to_py_err) +} + pub fn register(m: &Bound<'_, PyModule>) -> PyResult<()> { m.add_class::()?; m.add_class::()?; m.add_function(wrap_pyfunction!(open, m)?)?; + m.add_function(wrap_pyfunction!(rekey, m)?)?; Ok(()) } diff --git a/python/tests/test_rekey.py b/python/tests/test_rekey.py new file mode 100644 index 0000000..defadf2 --- /dev/null +++ b/python/tests/test_rekey.py @@ -0,0 +1,155 @@ +"""Tests for chisel.rekey() — bulk DEK rotation through the binding. + +`rekey` is the heavy sibling of rotate_key: rotate_key re-wraps the SAME data +key under a different credential (O(1), superblock only), while rekey generates +a fresh data key and re-encrypts every page. + +It is a module-level function rather than a Chisel method because it operates +on a PATH: it rewrites the whole file and replaces it by rename, so a handle +opened beforehand would afterwards refer to the original, now-unlinked inode. + +Covers: + - values, named roots and large (overflow-chained) values survive + - the on-disk bytes actually change (a silent no-op would pass everything else) + - the same credential still opens the database afterwards + - credentials NOT supplied to rekey are revoked, and can be re-added + - a wrong key is refused without modifying the file + - a plaintext database is refused + - a missing file is refused + - the database is still writable afterwards + - no scratch file is left beside the database + - bad argument types raise TypeError before anything is touched +""" + +import pathlib + +import chisel +import pytest + +KEY = bytes([0xA1]) * 32 +OTHER = bytes([0xB2]) * 32 +BIG = b"\xc7" * (8192 * 3) + + +def _seed(path: pathlib.Path, key: bytes | str) -> tuple[int, int]: + """Create a database with enough structure that a rotation has real work: + many small values, one overflow-chained value, and a named root.""" + with chisel.open(path, encryption_key=key) as db: + with db.transaction() as txn: + small = txn.allocate(b"small value") + big = txn.allocate(BIG) + db.set_root_name("primary", small) + return small, big + + +def test_rekey_preserves_data_and_changes_the_ciphertext(tmp_path): + path = tmp_path / "rk.db" + small, big = _seed(path, KEY) + before = path.read_bytes() + + chisel.rekey(path, KEY) + + after = path.read_bytes() + assert len(before) == len(after), "the page count must not change" + assert before != after, "every page is sealed under a new data key" + + # Same credential, same data. + with chisel.open(path, encryption_key=KEY) as db: + assert db.read(small) == b"small value" + assert db.read(big) == BIG + assert db.get_root_name("primary") == small + + +def test_rekey_revokes_credentials_it_was_not_given(tmp_path): + # The documented consequence: each key slot's wrapping key is derived from + # its own credential, so a credential that was not supplied cannot have the + # new data key wrapped for it. Asserting it keeps the doc honest. + path = tmp_path / "rk.db" + small, _ = _seed(path, KEY) + with chisel.open(path, encryption_key=KEY) as db: + db.add_key(KEY, OTHER) + + # Both work beforehand. + with chisel.open(path, encryption_key=OTHER) as db: + assert db.read(small) == b"small value" + + chisel.rekey(path, KEY) + + with chisel.open(path, encryption_key=KEY) as db: + assert db.read(small) == b"small value" + with pytest.raises(chisel.InvalidEncryptionKeyError): + chisel.open(path, encryption_key=OTHER) + + # The documented remedy. + with chisel.open(path, encryption_key=KEY) as db: + db.add_key(KEY, OTHER) + with chisel.open(path, encryption_key=OTHER) as db: + assert db.read(small) == b"small value" + + +def test_rekey_with_a_passphrase_credential(tmp_path): + path = tmp_path / "rk.db" + small, _ = _seed(path, "correct horse battery staple") + chisel.rekey(path, "correct horse battery staple") + with chisel.open(path, encryption_key="correct horse battery staple") as db: + assert db.read(small) == b"small value" + + +def test_rekey_refuses_a_wrong_key_without_touching_the_file(tmp_path): + path = tmp_path / "rk.db" + small, _ = _seed(path, KEY) + before = path.read_bytes() + + with pytest.raises(chisel.InvalidEncryptionKeyError): + chisel.rekey(path, OTHER) + + assert path.read_bytes() == before, "a refused rekey must not modify a byte" + with chisel.open(path, encryption_key=KEY) as db: + assert db.read(small) == b"small value" + + +def test_rekey_refuses_a_plaintext_database(tmp_path): + path = tmp_path / "plain.db" + with chisel.open(path) as db: + with db.transaction() as txn: + txn.allocate(b"cleartext") + with pytest.raises(chisel.EncryptionNotSupportedError): + chisel.rekey(path, KEY) + + +def test_rekey_refuses_a_missing_file(tmp_path): + with pytest.raises(chisel.DatabaseFileNotFoundError): + chisel.rekey(tmp_path / "nope.db", KEY) + + +def test_database_is_writable_after_a_rotation(tmp_path): + # The freemap, handle table and next_handle all come through the superblock + # body that was re-sealed under the new key, so a rotated database that + # reads but cannot write would be a real (and quiet) failure. + path = tmp_path / "rk.db" + small, _ = _seed(path, KEY) + + chisel.rekey(path, KEY) + + with chisel.open(path, encryption_key=KEY) as db: + with db.transaction() as txn: + fresh = txn.allocate(b"written after the rotation") + with chisel.open(path, encryption_key=KEY) as db: + assert db.read(fresh) == b"written after the rotation" + assert db.read(small) == b"small value" + + +def test_rekey_leaves_no_scratch_file(tmp_path): + path = tmp_path / "rk.db" + _seed(path, KEY) + chisel.rekey(path, KEY) + assert not (tmp_path / "rk.db.rekey-tmp").exists() + + +def test_rekey_rejects_bad_argument_types(tmp_path): + path = tmp_path / "rk.db" + _seed(path, KEY) + # Coercion happens under the GIL before any engine call, so this is a + # synchronous TypeError and the file is never opened. + with pytest.raises(TypeError): + chisel.rekey(path, 12345) diff --git a/src/lib.rs b/src/lib.rs index 928b04f..6899bd9 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -60,6 +60,7 @@ pub(crate) mod overflow; pub(crate) mod page; pub(crate) mod page_cache; pub(crate) mod page_io; +pub(crate) mod rekey; mod spillway; pub(crate) mod stats; pub(crate) mod superblock; @@ -1108,6 +1109,70 @@ impl Chisel { pub fn remove_key(&mut self, key: &crypto::Key) -> Result<()> { self.txm.remove_key(key) } + + /// Re-encrypt the entire database under a freshly generated data-encryption + /// key. The heavy sibling of [`Chisel::rotate_key`]. + /// + /// Reach for this only when the DEK ITSELF is believed compromised — a + /// process memory dump, a core file, a debugger session. If what leaked was + /// a *credential*, [`Chisel::rotate_key`] is the right tool: it is O(1), + /// touches only the superblock, and is the common operational need + /// (password change, key rollover, adding a second credential). + /// + /// # Why this is an offline operation on a path + /// + /// It takes a path rather than `&mut self` because it rewrites the whole + /// file and replaces it by `rename`. After a successful rotation, any file + /// descriptor opened before the call refers to the ORIGINAL, now-unlinked + /// inode — reads and writes through it would silently target a deleted + /// file. Handing back a live handle would be handing back that hazard, so + /// there is no handle to misuse: close any open `Chisel` first, call this, + /// then [`Chisel::open`] again. + /// + /// The exclusive `flock` is held for the whole rotation, so no other + /// process can open the database while it is in flight. + /// + /// # Crash safety + /// + /// The rotated database is built in a scratch file beside the original and + /// published with an atomic `rename`. The database path therefore only ever + /// names a complete file — the fully-rotated one, or the untouched + /// original. A crash at any point leaves one or the other, never a mix. + /// + /// In-place rotation would NOT be safe, and shadow paging does not rescue + /// it: shadow paging protects writes that go to *new* pages, while this + /// rewrites pages where they are. A crash halfway would leave some pages + /// under the new DEK and some under the old, with no way to tell which. + /// + /// # This collapses the key-slot table to `key` alone + /// + /// Every other credential is revoked. That is not a shortcut — it is + /// forced: each slot's KEK is derived from its own credential, so re-wrapping + /// the new DEK for a slot requires the credential that slot belongs to, and + /// only `key` was supplied. Add the others back with [`Chisel::add_key`] + /// afterwards. + /// + /// It is also the safer default for the situation this exists for. If you + /// are rotating because the DEK leaked, quietly preserving every credential + /// that could reach it is not what you want. + /// + /// # Cost + /// + /// O(total_pages) I/O and, transiently, a second copy of the database on + /// disk. Not something to schedule routinely. + /// + /// # Errors + /// `FileNotFound` if `path` does not exist; `EncryptionNotSupported` if the + /// database is plaintext; `InvalidEncryptionKey` if `key` unlocks no slot; + /// `IoError` for any failure building or publishing the replacement — in + /// which case the original file is untouched and the scratch file removed. + pub fn rekey( + path: &Path, + key: &crypto::Key, + argon2_params: Option, + ) -> Result<()> { + rekey::rekey(path, key, argon2_params) + } } #[cfg(test)] diff --git a/src/rekey.rs b/src/rekey.rs new file mode 100644 index 0000000..e5ff070 --- /dev/null +++ b/src/rekey.rs @@ -0,0 +1,719 @@ +// rekey.rs — bulk DEK rotation: re-encrypt an entire database under a fresh +// data-encryption key. +// +// Role in the system: this is the heavy sibling of the O(1) credential rotation +// in `transaction::keys`. Those operations (`add_key`/`rotate_key`/`remove_key`) +// re-wrap the SAME DEK under a different KEK — they change who can get in, and +// touch only the superblock. This one changes the key the data itself is sealed +// with, so it must rewrite every page in the file. +// +// Which one you want depends on what leaked: +// +// * a credential (passphrase, raw key) -> credential rotation, O(1) +// * the DEK itself (process memory dump) -> this, O(total_pages) +// +// Layer: sits above the engine rather than inside it. It opens a normal +// `Chisel` handle to validate the key and take the exclusive flock, reads +// through that handle's `PageIo`, and writes a whole new file — so it depends +// on the public open path plus crate-internal access to the live cipher. +// +// Crash safety is copy-then-atomic-rename, and it has to be. The obvious +// in-place alternative is NOT safe under shadow paging: shadow paging protects +// writes that go to NEW pages, and this rewrites pages in place. A crash +// halfway through leaves some pages sealed under the new DEK and some under the +// old, with the surviving superblock naming whichever DEK it names — an +// unrecoverable mix. `rename(2)` is atomic within a filesystem, so the database +// path only ever names a complete file: the fully-rotated one, or the original. + +use std::path::Path; + +use crate::crypto::{Key, PageCipher}; +use crate::error::{ChiselError, Result}; +use crate::page::PAGE_SIZE; +use crate::superblock::{ + CryptoHeader, KeySlot, Superblock, ALGO_XCHACHA20POLY1305, KEY_SLOT_COUNT, +}; +use crate::{Chisel, Options}; + +/// Suffix for the scratch file the rotated database is built in. It lives +/// beside the database so `rename` stays within one filesystem — a rename +/// across filesystems is not atomic and would defeat the whole strategy. +const TMP_SUFFIX: &str = ".rekey-tmp"; + +/// Re-encrypt every page of the database at `path` under a freshly generated +/// DEK, then atomically replace the original. +/// +/// See [`Chisel::rekey`] for the caller-facing contract; this is its body. +pub(crate) fn rekey( + path: &Path, + key: &Key, + argon2_params: Option, +) -> Result<()> { + // Open normally. This does all the validation we would otherwise duplicate + // — file exists, superblock selects, format version and page size accepted, + // algorithm known, key unlocks a slot — and takes the exclusive flock for + // the duration. `create_if_missing(false)` because rotating the key of a + // database that does not exist is a caller error, not a create. + let mut db = Chisel::open( + path, + Options::default() + .encryption_key(key.clone()) + .create_if_missing(false), + )?; + + let old_cipher = match db.txm.cipher.clone() { + Some(c) => c, + // A plaintext database has no DEK to rotate. Operational: the caller + // asked for something that does not apply, nothing is damaged. + None => return Err(ChiselError::EncryptionNotSupported), + }; + + // Snapshot the geometry we are about to reproduce. `page_count` is in + // stride units, and the stride is ENC_PAGE_SIZE for every encrypted file. + let (page_count, superblock_count, stride) = { + let mut cache = db.txm.cache.borrow_mut(); + let io = cache.io_mut(); + (io.page_count()?, db.txm.superblock_count, io.stride()) + }; + debug_assert_eq!(stride, crate::crypto::ENC_PAGE_SIZE); + + // The winning superblock, which every slot of the new file will carry. Its + // roots are the committed state; older slots' roots are history we + // deliberately do not preserve, because their sealed bodies are under the + // OLD DEK and re-sealing states we cannot reach buys nothing. + let winner = current_superblock(&mut db)?; + + // Fresh DEK, and a crypto header holding exactly ONE credential: the one + // supplied. See `Chisel::rekey`'s doc — re-wrapping the new DEK for the + // other active slots is impossible, because each of those slots' KEK is + // derived from a credential we were not given. + // Cost parameters: an explicit override wins, otherwise INHERIT what the + // slot this key just unlocked was using. + // + // Defaulting to `Argon2Params::default()` here would silently DOWNGRADE a + // database whose owner had deliberately hardened its KDF cost — during the + // operation you run because you were compromised, with no signal. The old + // slot's parameters are right there in the header we are replacing, and + // "keep what you had" is the only defensible default for a key rotation. + let effective_params = match argon2_params { + Some(p) => Some(p), + None => inherited_argon2(&db, key), + }; + + let new_dek = crate::crypto::random_dek(); + let mut new_header = CryptoHeader { + algorithm: ALGO_XCHACHA20POLY1305, + stride: crate::crypto::ENC_PAGE_SIZE as u32, + slots: [KeySlot::EMPTY; KEY_SLOT_COUNT], + }; + new_header.wrap_into(0, key, &new_dek, effective_params)?; + let new_cipher = PageCipher::new(new_dek); + + let tmp_path = { + let mut p = path.as_os_str().to_owned(); + p.push(TMP_SUFFIX); + std::path::PathBuf::from(p) + }; + + // Build the replacement, cleaning up the scratch file on any failure so a + // botched rotation does not leave debris that the next attempt trips over + // (the create is O_EXCL — see `create_scratch`). + let build = build_rotated_file( + &mut db, + &tmp_path, + page_count, + superblock_count, + &winner, + &new_header, + &old_cipher, + &new_cipher, + ); + if let Err(e) = build { + let _ = std::fs::remove_file(&tmp_path); + return Err(e); + } + + // The linearization point. Before this the database path names the + // original; after it, the rotated file. There is no in-between state a + // reader can observe, which is the entire reason for the copy. + std::fs::rename(&tmp_path, path).map_err(ChiselError::IoError)?; + + // A rename is durable only once the DIRECTORY entry is durable. Without + // this, a crash right after a successful rename can resurrect the old name + // binding on some filesystems — the file contents are safe (they were + // fsynced) but the path could still point at the original inode. + if let Some(dir) = path.parent() { + // An empty parent means `path` was relative with no directory + // component; the current directory is the right target then. + let dir = if dir.as_os_str().is_empty() { + Path::new(".") + } else { + dir + }; + if let Ok(f) = std::fs::File::open(dir) { + // Best-effort: some platforms refuse to fsync a directory handle. + // The data itself is already durable, so a failure here costs + // rename durability across a crash, not correctness of the file. + let _ = f.sync_all(); + } + } + + // Drop the handle only now, so the flock is held across the rename and no + // other process can open the database mid-rotation. The handle's file + // descriptor refers to the ORIGINAL (now unlinked) inode, which is exactly + // why `rekey` does not hand it back: any further use would silently read + // and write a deleted file. + drop(db); + Ok(()) +} + +/// The Argon2 cost parameters the supplied key's CURRENT slot uses, if any. +/// +/// `None` when the database has no crypto header, when the key unlocks no slot +/// (unreachable — `Chisel::open` already proved it does), or when the slot is +/// an HKDF slot, whose params are zero by format definition and meaningless to +/// carry forward. In every one of those cases `wrap_into` falls back to its own +/// default, which is correct: a raw key never reads these fields at all. +fn inherited_argon2(db: &Chisel, key: &Key) -> Option { + let header = db.txm.crypto_header.as_ref()?; + let (idx, _dek) = header.unlock(key).ok()?; + let slot = &header.slots[idx]; + if slot.kdf_id == crate::crypto::KdfId::Argon2id as u8 { + Some(slot.argon2) + } else { + None + } +} + +/// Re-read the winning superblock through the live handle. +/// +/// `open_existing` already selected and decrypted it, but the decrypted result +/// lives inside the manager rather than being handed back, so read slot +/// `txn_counter % superblock_count` — the slot the last commit wrote — and +/// decrypt it with the session cipher. +fn current_superblock(db: &mut Chisel) -> Result { + let txn_counter = db.txm.txn_counter; + let n = db.txm.superblock_count as u64; + let mut cache = db.txm.cache.borrow_mut(); + let cipher = db + .txm + .cipher + .as_ref() + .ok_or(ChiselError::EncryptionNotSupported)?; + + // Try the expected slot first, then every other, so a torn slot that + // `select` already routed around does not defeat this. + let mut unit = [0u8; crate::crypto::ENC_PAGE_SIZE]; + let expected = txn_counter % n; + let order = std::iter::once(expected).chain((0..n).filter(|s| *s != expected)); + for slot in order { + cache.io_mut().read_page_unit_into(slot, &mut unit)?; + let mut image = [0u8; PAGE_SIZE]; + image.copy_from_slice(&unit[..PAGE_SIZE]); + if let Some(mut sb) = Superblock::deserialize(&image) { + if sb.txn_counter == txn_counter && sb.decrypt_body(cipher, &image).is_ok() { + return Ok(sb); + } + } + } + // The manager opened from one of these slots moments ago, so failing here + // means the file changed underneath a held flock. + Err(ChiselError::CorruptSuperblock { + defects: Vec::new(), + }) +} + +/// Create the scratch file with the same hardening the database itself gets: +/// mode 0600, `O_EXCL` so we never adopt a file we did not create, and +/// `O_NOFOLLOW` so a planted symlink at the predictable scratch path cannot +/// redirect the write. +fn create_scratch(tmp_path: &Path) -> Result { + let mut opts = std::fs::OpenOptions::new(); + opts.read(true).write(true).create_new(true); + #[cfg(unix)] + { + use std::os::unix::fs::OpenOptionsExt; + opts.mode(0o600).custom_flags(libc::O_NOFOLLOW); + } + match opts.open(tmp_path) { + Ok(f) => Ok(f), + Err(e) if e.kind() == std::io::ErrorKind::AlreadyExists => { + // Debris from a crashed earlier attempt, or someone squatting on a + // predictable path. Same discipline as the spillway sidecar: remove + // it ONLY if it is a plain file this uid owns with a single link, + // and never unlink another user's entry. Then retry, still + // exclusively, so a re-plant between the two syscalls loses. + reclaim_scratch(tmp_path)?; + opts.open(tmp_path).map_err(ChiselError::IoError) + } + Err(e) => Err(ChiselError::IoError(e)), + } +} + +/// Remove a pre-existing scratch entry, but only when it is plausibly our own +/// debris. Mirrors `spillway::reclaim_stale_sidecar` — same threat (a +/// predictable path in a directory another local user may write), same answer. +#[cfg(unix)] +fn reclaim_scratch(path: &Path) -> Result<()> { + use std::os::unix::fs::MetadataExt; + let md = std::fs::symlink_metadata(path).map_err(ChiselError::IoError)?; + // SAFETY: geteuid() is a pure read of process credentials; it cannot fail + // and touches no memory we own. + let ours = + md.file_type().is_file() && md.nlink() == 1 && md.uid() == unsafe { libc::geteuid() }; + if !ours { + return Err(ChiselError::IoError(std::io::Error::new( + std::io::ErrorKind::PermissionDenied, + "rekey scratch path is occupied by an entry this process did not create \ + (symlink, foreign owner, or extra hard link) — refusing to remove it", + ))); + } + std::fs::remove_file(path).map_err(ChiselError::IoError) +} + +#[cfg(not(unix))] +fn reclaim_scratch(path: &Path) -> Result<()> { + std::fs::remove_file(path).map_err(ChiselError::IoError) +} + +/// Write the fully-rotated database into `tmp_path` and fsync it. +/// +/// Page classes are handled differently, and the distinction is the crux of +/// the whole operation: +/// +/// * superblock slots are NOT `PageCipher`-sealed — their sensitive body is +/// sealed by `serialize_encrypted`, and their crypto header is cleartext. +/// They are rebuilt from `winner` under the new header and new DEK. +/// * every other page IS `PageCipher`-sealed with AAD = page_id, so it is +/// opened under the old DEK and re-sealed under the new one at the same +/// page id. Keeping the page id fixed is what lets the roots stay valid: +/// no pointer in the tree changes, only the bytes under each pointer. +#[allow(clippy::too_many_arguments)] +fn build_rotated_file( + db: &mut Chisel, + tmp_path: &Path, + page_count: u64, + superblock_count: u32, + winner: &Superblock, + new_header: &CryptoHeader, + old_cipher: &PageCipher, + new_cipher: &PageCipher, +) -> Result<()> { + use std::io::Write; + + let mut out = create_scratch(tmp_path)?; + let mut cache = db.txm.cache.borrow_mut(); + + // Superblock bank. Every slot carries the same roots; counters descend from + // the winner's in rotation order, so the winner still wins selection and + // the next commit's `(counter + 1) % N` target is still the slot it would + // have been. Staggering rather than writing one counter N times keeps + // `select`'s tie-break from having to choose between identical slots. + let n = superblock_count as u64; + let top = winner.txn_counter; + for k in 0..n { + let slot = (top + n - k) % n; + let mut sb = winner.clone(); + sb.txn_counter = top.saturating_sub(k); + sb.encryption = Some(*new_header); + let image = sb.serialize_encrypted(new_cipher); + let mut unit = [0u8; crate::crypto::ENC_PAGE_SIZE]; + unit[..image.len()].copy_from_slice(&image); + out.seek_write_unit(slot, &unit)?; + } + + // Data pages. + let mut unit = [0u8; crate::crypto::ENC_PAGE_SIZE]; + for page_id in n..page_count { + cache.io_mut().read_page_unit_into(page_id, &mut unit)?; + // A unit that does not decrypt under the current DEK is copied through + // verbatim rather than failing the rotation, and the distinction + // matters more than it looks. + // + // Two kinds of unit legitimately do not decrypt in a HEALTHY database: + // + // * never-written space — the file grew past a page id that was never + // filled, which POSIX zero-fills; and + // * crash debris — a process killed mid-commit can leave a TORN unit + // (a partial write of the 8232-byte blob) at a page id the winning + // superblock's freemap marks free. Shadow paging recovers by simply + // never reading those pages, which is why the database opens and + // works perfectly afterwards. + // + // Refusing to rotate on either would make this operation fail + // deterministically on a database that is completely fine — and fail at + // precisely the moment it exists for, since you reach for a DEK + // rotation when something has already gone wrong. That is the worst + // possible time to discover an availability trap. + // + // Copying verbatim cannot lose committed data: a page that is part of + // the committed state MUST decrypt under the current DEK, or the + // database could not be read at all. So anything landing here is by + // construction not live. What it does give up is using rekey as a + // corruption detector — a genuinely damaged LIVE page would be carried + // across rather than reported. That is the right trade: rekey is a key + // rotation, not an fsck, and a damaged live page already surfaces as + // DecryptionFailed on the read path where it belongs. + // + // Preserving the bytes (rather than zeroing) also keeps the file + // byte-length and page geometry identical, which the tests assert. + match old_cipher.open(page_id, &unit) { + Ok(plaintext) => { + let sealed = new_cipher.seal(page_id, &plaintext); + out.seek_write_unit(page_id, &sealed)?; + } + Err(_) => out.seek_write_unit(page_id, &unit)?, + } + } + + out.flush().map_err(ChiselError::IoError)?; + // Durable before the rename, not after: the rename publishes this file, and + // publishing bytes that are not yet on disk is exactly the crash window the + // copy strategy exists to avoid. + out.sync_all().map_err(ChiselError::IoError)?; + Ok(()) +} + +/// Small helper so the two write sites above read as "put this unit at this +/// page id" rather than repeating the offset arithmetic. +trait SeekWriteUnit { + fn seek_write_unit( + &mut self, + page_id: u64, + unit: &[u8; crate::crypto::ENC_PAGE_SIZE], + ) -> Result<()>; +} + +impl SeekWriteUnit for std::fs::File { + fn seek_write_unit( + &mut self, + page_id: u64, + unit: &[u8; crate::crypto::ENC_PAGE_SIZE], + ) -> Result<()> { + use std::io::{Seek, SeekFrom, Write}; + self.seek(SeekFrom::Start( + page_id * crate::crypto::ENC_PAGE_SIZE as u64, + )) + .map_err(ChiselError::IoError)?; + self.write_all(unit).map_err(ChiselError::IoError) + } +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::Handle; + use tempfile::TempDir; + use zeroize::Zeroizing; + + fn raw(b: u8) -> Key { + Key::Raw(Zeroizing::new(vec![b; 32])) + } + + /// Build a database with enough content to span several pages, including an + /// overflow chain and a tagged chunk, so the rotation has real structure to + /// preserve rather than one page of nothing. + fn seed(path: &Path, key: &Key) -> Vec<(Handle, Vec)> { + let mut db = Chisel::open(path, Options::default().encryption_key(key.clone())).unwrap(); + let mut out = Vec::new(); + db.begin().unwrap(); + for i in 0..40u8 { + let v = vec![i; 300]; + out.push((db.allocate(&v).unwrap(), v)); + } + // Larger than a page: exercises the overflow chain, whose pages are + // sealed exactly like any other and must survive the rotation. + let big = vec![0xC7u8; PAGE_SIZE * 3]; + out.push((db.allocate(&big).unwrap(), big)); + db.set_root_name("primary", out[0].0).unwrap(); + db.commit().unwrap(); + drop(db); + out + } + + fn assert_intact(path: &Path, key: &Key, expected: &[(Handle, Vec)]) { + let db = Chisel::open(path, Options::default().encryption_key(key.clone())).unwrap(); + for (h, v) in expected { + assert_eq!(&db.read(*h).unwrap(), v, "handle {h:?} did not survive"); + } + assert_eq!(db.get_root_name("primary").unwrap(), Some(expected[0].0)); + } + + #[test] + fn rekey_preserves_every_value_under_a_new_dek() { + let dir = TempDir::new().unwrap(); + let path = dir.path().join("rk.db"); + let k = raw(0x31); + let expected = seed(&path, &k); + + // Capture the on-disk bytes of a data page so we can prove they really + // changed — a rotation that silently no-ops would otherwise pass every + // behavioural assertion here. + let before = std::fs::read(&path).unwrap(); + + Chisel::rekey(&path, &k, None).unwrap(); + + let after = std::fs::read(&path).unwrap(); + assert_eq!(before.len(), after.len(), "page count must not change"); + assert_ne!( + before, after, + "every page is sealed under a new DEK, so the ciphertext must differ" + ); + + // The SAME credential still opens it: rekey rotates the data key, not + // the credential. + assert_intact(&path, &k, &expected); + } + + #[test] + fn rekey_actually_changes_the_data_key() { + // The byte-comparison in the test above does NOT prove this, and it is + // worth being explicit about why: `PageCipher::seal` draws a fresh + // random nonce on every call, so re-sealing under the SAME DEK also + // changes every byte in the file. A bug that reused `old_cipher` for + // sealing, or wrapped the OLD DEK into the new header, would sail past + // every other assertion here. + // + // So assert the property directly: after the rotation, the old cipher + // must no longer be able to open a data page. + let dir = TempDir::new().unwrap(); + let path = dir.path().join("rk.db"); + let k = raw(0x91); + seed(&path, &k); + + // Recover the pre-rotation DEK the same way an attacker holding the + // credential would: unwrap it from the on-disk key slot. + let old_cipher = { + let db = Chisel::open(&path, Options::default().encryption_key(k.clone())).unwrap(); + db.txm.cipher.clone().expect("encrypted fixture") + }; + // A page that is definitely a sealed data page (past the superblock bank). + let probe = crate::DEFAULT_SUPERBLOCK_COUNT as u64; + let read_unit = |p: &Path, id: u64| { + use std::io::{Read, Seek, SeekFrom}; + let mut f = std::fs::File::open(p).unwrap(); + let mut u = [0u8; crate::crypto::ENC_PAGE_SIZE]; + f.seek(SeekFrom::Start(id * crate::crypto::ENC_PAGE_SIZE as u64)) + .unwrap(); + f.read_exact(&mut u).unwrap(); + u + }; + assert!( + old_cipher.open(probe, &read_unit(&path, probe)).is_ok(), + "fixture must have a decryptable data page at {probe} before the rotation" + ); + + Chisel::rekey(&path, &k, None).unwrap(); + + assert!( + old_cipher.open(probe, &read_unit(&path, probe)).is_err(), + "the old DEK must no longer open any page — if it does, the data key \ + was not actually rotated and only the nonces changed" + ); + } + + #[test] + fn rekey_inherits_the_argon2_cost_of_the_slot_it_unlocks() { + // Silently re-wrapping at Argon2Params::default() would DOWNGRADE a + // database whose owner hardened its KDF — during the operation you run + // because you were compromised, with no signal at all. + use crate::Argon2Params; + let dir = TempDir::new().unwrap(); + let path = dir.path().join("rk.db"); + let pass = || Key::Passphrase(Zeroizing::new("a passphrase".to_string())); + // Distinct from the OWASP default (19456/2/1) in every field, and cheap + // enough to keep the test fast. + let hardened = Argon2Params { + m_cost: 8192, + t_cost: 3, + p_cost: 2, + }; + { + let mut db = Chisel::open( + &path, + Options::default() + .encryption_key(pass()) + .argon2_params(hardened), + ) + .unwrap(); + db.begin().unwrap(); + db.allocate(b"payload").unwrap(); + db.commit().unwrap(); + } + + Chisel::rekey(&path, &pass(), None).unwrap(); + + let db = Chisel::open(&path, Options::default().encryption_key(pass())).unwrap(); + let header = db.txm.crypto_header.expect("encrypted"); + let (idx, _) = header.unlock(&pass()).unwrap(); + assert_eq!( + header.slots[idx].argon2, hardened, + "rekey must carry the existing cost parameters forward, not reset them" + ); + } + + #[test] + fn rekey_survives_an_undecryptable_page() { + // A crash mid-commit can leave a TORN unit at a page id the winning + // superblock's freemap marks free. Shadow paging recovers by never + // reading it, so the database is healthy — but a rekey that insisted + // every unit decrypt would fail deterministically on that healthy + // database, at exactly the moment the operation is needed. + use std::io::{Seek, SeekFrom, Write}; + let dir = TempDir::new().unwrap(); + let path = dir.path().join("rk.db"); + let k = raw(0xA5); + let expected = seed(&path, &k); + + // Grow the file by one unit of garbage: a page id past everything the + // committed state references, holding bytes that are neither zeros nor + // valid ciphertext. This is the shape crash debris takes. + let stray = { + let mut f = std::fs::OpenOptions::new().write(true).open(&path).unwrap(); + let len = f.metadata().unwrap().len(); + let id = len / crate::crypto::ENC_PAGE_SIZE as u64; + f.seek(SeekFrom::Start(id * crate::crypto::ENC_PAGE_SIZE as u64)) + .unwrap(); + f.write_all(&[0x5Au8; crate::crypto::ENC_PAGE_SIZE]) + .unwrap(); + f.sync_all().unwrap(); + id + }; + + Chisel::rekey(&path, &k, None).expect("crash debris must not block a rotation"); + + assert_intact(&path, &k, &expected); + // The debris is carried across verbatim, so the file geometry is + // unchanged — nothing was dropped or zero-filled behind our back. + let len = std::fs::metadata(&path).unwrap().len(); + assert_eq!(len / crate::crypto::ENC_PAGE_SIZE as u64, stray + 1); + } + + #[test] + fn rekey_leaves_no_scratch_file_behind() { + let dir = TempDir::new().unwrap(); + let path = dir.path().join("rk.db"); + let k = raw(0x32); + seed(&path, &k); + Chisel::rekey(&path, &k, None).unwrap(); + + let scratch = dir.path().join("rk.db.rekey-tmp"); + assert!( + !scratch.exists(), + "the scratch file must be renamed away, not left beside the database" + ); + } + + #[test] + fn rekey_collapses_the_key_slot_table_to_the_supplied_credential() { + // The documented consequence, and the one most likely to surprise: the + // other credentials cannot be carried across, because each slot's KEK + // is derived from a credential we were not given. Asserting it here + // makes the doc a tested claim rather than a promise. + let dir = TempDir::new().unwrap(); + let path = dir.path().join("rk.db"); + let first = raw(0x41); + let second = raw(0x42); + let expected = seed(&path, &first); + { + let mut db = + Chisel::open(&path, Options::default().encryption_key(first.clone())).unwrap(); + db.add_key(&first, &second).unwrap(); + } + // Both work before. + assert_intact(&path, &first, &expected); + assert_intact(&path, &second, &expected); + + Chisel::rekey(&path, &first, None).unwrap(); + + assert_intact(&path, &first, &expected); + assert!( + Chisel::open(&path, Options::default().encryption_key(second.clone())).is_err(), + "a credential not supplied to rekey cannot survive it" + ); + // ...and can be added back, which is the documented remedy. + { + let mut db = + Chisel::open(&path, Options::default().encryption_key(first.clone())).unwrap(); + db.add_key(&first, &second).unwrap(); + } + assert_intact(&path, &second, &expected); + } + + #[test] + fn rekey_refuses_a_wrong_key_without_touching_the_file() { + let dir = TempDir::new().unwrap(); + let path = dir.path().join("rk.db"); + let k = raw(0x51); + let expected = seed(&path, &k); + let before = std::fs::read(&path).unwrap(); + + match Chisel::rekey(&path, &raw(0x52), None) { + Err(ChiselError::InvalidEncryptionKey) => {} + other => panic!("expected InvalidEncryptionKey, got {other:?}"), + } + + assert_eq!( + before, + std::fs::read(&path).unwrap(), + "a refused rekey must not modify a single byte" + ); + assert_intact(&path, &k, &expected); + } + + #[test] + fn rekey_refuses_a_plaintext_database() { + let dir = TempDir::new().unwrap(); + let path = dir.path().join("plain.db"); + { + let mut db = Chisel::open(&path, Options::default()).unwrap(); + db.begin().unwrap(); + db.allocate(b"cleartext").unwrap(); + db.commit().unwrap(); + } + // The open inside rekey supplies a key to a plaintext DB, which the + // open path itself rejects — the error the caller sees names the real + // mismatch either way. + match Chisel::rekey(&path, &raw(0x61), None) { + Err(ChiselError::EncryptionNotSupported) => {} + other => panic!("expected EncryptionNotSupported, got {other:?}"), + } + } + + #[test] + fn rekey_refuses_a_missing_file() { + let dir = TempDir::new().unwrap(); + match Chisel::rekey(&dir.path().join("nope.db"), &raw(0x71), None) { + Err(ChiselError::FileNotFound) => {} + other => panic!("expected FileNotFound, got {other:?}"), + } + } + + #[test] + fn the_database_stays_writable_after_a_rotation() { + // The rotated file must be a fully functional database, not just a + // readable one: the freemap, handle table and next_handle all come + // through the superblock body that was re-sealed under the new DEK. + let dir = TempDir::new().unwrap(); + let path = dir.path().join("rk.db"); + let k = raw(0x81); + let expected = seed(&path, &k); + + Chisel::rekey(&path, &k, None).unwrap(); + + let mut db = Chisel::open(&path, Options::default().encryption_key(k.clone())).unwrap(); + db.begin().unwrap(); + let fresh = db.allocate(b"written after the rotation").unwrap(); + db.delete(expected[1].0).unwrap(); + db.commit().unwrap(); + drop(db); + + let db = Chisel::open(&path, Options::default().encryption_key(k)).unwrap(); + assert_eq!(db.read(fresh).unwrap(), b"written after the rotation"); + assert!( + db.read(expected[1].0).is_err(), + "the delete must have stuck" + ); + assert_eq!(db.read(expected[0].0).unwrap(), expected[0].1); + } +} diff --git a/src/transaction/mod.rs b/src/transaction/mod.rs index 5f0e875..c3560dc 100644 --- a/src/transaction/mod.rs +++ b/src/transaction/mod.rs @@ -154,7 +154,7 @@ pub struct TransactionManager { // `borrow_mut()`; reborrowing for downstream `&mut PageCache` parameters // (e.g., handle_table methods) is done via `&mut *cache` on a single // RefMut held for the duration of the operation. - cache: RefCell, + pub(crate) cache: RefCell, // Roots that match the superblock currently on disk. Safe to read at any time. committed_roots: Roots, // Roots under construction. Equals committed_roots when no txn is active; @@ -167,14 +167,14 @@ pub struct TransactionManager { // Monotonically increasing. Written into each new superblock; the higher value // wins on recovery. Also used to pick the inactive slot on commit via // `txn_counter % superblock_count`. - txn_counter: u64, + pub(crate) txn_counter: u64, // Number of superblock slots occupying pages 0..superblock_count // (ISSUES.md R4). Set at open time from the winning superblock's // own `superblock_count` field; cached here so commit doesn't have // to re-fetch it. Must equal every slot's self-reported value in a // healthy database; divergence would indicate mid-flight reconfig // or corruption. - superblock_count: u32, + pub(crate) superblock_count: u32, active_txn: bool, savepoints: Vec, // Pages whose contents are no longer reachable from the new roots. @@ -221,12 +221,12 @@ pub struct TransactionManager { /// seal/open. The manager's copy is also threaded through CommitCtx for the /// superblock body seal on every commit. The DEK inside PageCipher is /// zeroizing and is cleared on drop. - cipher: Option, + pub(crate) cipher: Option, /// The crypto-header (algorithm id + key-slot table) for an encrypted database. /// Written verbatim into every committed superblock. `None` for plaintext DBs. /// The key-slot contents never change after create or open: the slots hold the /// DEK wrapped under KEKs from each user key and are opaque to the commit path. - crypto_header: Option, + pub(crate) crypto_header: Option, // Test-only fault injection consolidated off the production type (review // 2026-06-22 SMELL #4): the four BUG#2 atomic-staging arming flags live in