From 968f4765096e4e7bdb909eaeaeea0fcb52429c0a Mon Sep 17 00:00:00 2001 From: Cursor Agent Date: Thu, 15 Oct 2026 01:41:10 +0000 Subject: [PATCH 1/2] Add the V1 A CPU contract for the G3.5 KV tier. Introduce core-types, protocol, and MemoryStore with a versioned identity, length-delimited keys, and a fenced block lifecycle. Tests cover hits, quotas, timeout, and cancel on x86_64 and aarch64 qemu-user. Real vLLM and DPU paths stay out of the tree. Co-authored-by: RustFS --- .github/workflows/ci.yml | 57 + .gitignore | 7 + AGENTS.md | 34 + COMPATIBILITY.md | 24 + Cargo.lock | 431 +++++++ Cargo.toml | 34 + README.md | 95 +- THIRD_PARTY_NOTICES.md | 12 + crates/core-types/Cargo.toml | 18 + crates/core-types/src/checksum.rs | 47 + crates/core-types/src/codec.rs | 159 +++ crates/core-types/src/error.rs | 117 ++ crates/core-types/src/identity.rs | 669 +++++++++++ crates/core-types/src/key.rs | 97 ++ crates/core-types/src/lib.rs | 129 ++ crates/core-types/src/lifecycle.rs | 553 +++++++++ crates/core-types/src/limits.rs | 90 ++ crates/core-types/src/props.rs | 179 +++ crates/protocol/Cargo.toml | 18 + crates/protocol/src/api.rs | 164 +++ crates/protocol/src/auth.rs | 62 + crates/protocol/src/codec.rs | 593 ++++++++++ crates/protocol/src/lib.rs | 27 + crates/protocol/src/negotiate.rs | 52 + crates/storage/Cargo.toml | 23 + crates/storage/src/bin/kv_block_xfer.rs | 84 ++ crates/storage/src/lib.rs | 19 + crates/storage/src/memory.rs | 1438 +++++++++++++++++++++++ crates/storage/src/props.rs | 176 +++ crates/storage/src/tests.rs | 755 ++++++++++++ crates/storage/src/transfer.rs | 174 +++ crates/storage/tests/cross_process.rs | 105 ++ docs/ARCHITECTURE.md | 52 + docs/BENCHMARKS.md | 14 + docs/OPERATIONS.md | 25 + docs/PROTOCOL.md | 53 + docs/REUSE_AUDIT.md | 13 + docs/RISKS.md | 16 + docs/SECURITY.md | 24 + docs/adr/0001-integration-route.md | 36 + rust-toolchain.toml | 4 + 41 files changed, 6632 insertions(+), 47 deletions(-) create mode 100644 .github/workflows/ci.yml create mode 100644 .gitignore create mode 100644 AGENTS.md create mode 100644 COMPATIBILITY.md create mode 100644 Cargo.lock create mode 100644 Cargo.toml create mode 100644 THIRD_PARTY_NOTICES.md create mode 100644 crates/core-types/Cargo.toml create mode 100644 crates/core-types/src/checksum.rs create mode 100644 crates/core-types/src/codec.rs create mode 100644 crates/core-types/src/error.rs create mode 100644 crates/core-types/src/identity.rs create mode 100644 crates/core-types/src/key.rs create mode 100644 crates/core-types/src/lib.rs create mode 100644 crates/core-types/src/lifecycle.rs create mode 100644 crates/core-types/src/limits.rs create mode 100644 crates/core-types/src/props.rs create mode 100644 crates/protocol/Cargo.toml create mode 100644 crates/protocol/src/api.rs create mode 100644 crates/protocol/src/auth.rs create mode 100644 crates/protocol/src/codec.rs create mode 100644 crates/protocol/src/lib.rs create mode 100644 crates/protocol/src/negotiate.rs create mode 100644 crates/storage/Cargo.toml create mode 100644 crates/storage/src/bin/kv_block_xfer.rs create mode 100644 crates/storage/src/lib.rs create mode 100644 crates/storage/src/memory.rs create mode 100644 crates/storage/src/props.rs create mode 100644 crates/storage/src/tests.rs create mode 100644 crates/storage/src/transfer.rs create mode 100644 crates/storage/tests/cross_process.rs create mode 100644 docs/ARCHITECTURE.md create mode 100644 docs/BENCHMARKS.md create mode 100644 docs/OPERATIONS.md create mode 100644 docs/PROTOCOL.md create mode 100644 docs/REUSE_AUDIT.md create mode 100644 docs/RISKS.md create mode 100644 docs/SECURITY.md create mode 100644 docs/adr/0001-integration-route.md create mode 100644 rust-toolchain.toml diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml new file mode 100644 index 0000000..6c7995d --- /dev/null +++ b/.github/workflows/ci.yml @@ -0,0 +1,57 @@ +name: ci + +on: + push: + pull_request: + +jobs: + test-x86_64: + name: test (x86_64) + runs-on: ubuntu-latest + timeout-minutes: 30 + steps: + - uses: actions/checkout@v4 + - uses: dtolnay/rust-toolchain@stable + with: + components: clippy, rustfmt + - name: Format + run: cargo fmt --all -- --check + - name: Clippy + run: cargo clippy --workspace --all-targets -- -D warnings + - name: Test + run: cargo test --workspace --all-targets + - name: Reject accelerator crates in V1 A + run: | + if grep -RInE 'cuda|doca|nixl|rdma|spdk' crates/*/Cargo.toml; then + echo "V1 A crates must not depend on GPU or DPU stacks" + exit 1 + fi + + test-aarch64: + name: test (aarch64 runner) + runs-on: ubuntu-24.04-arm + timeout-minutes: 30 + steps: + - uses: actions/checkout@v4 + - uses: dtolnay/rust-toolchain@stable + - name: Test + run: cargo test --workspace --all-targets + + test-aarch64-qemu: + name: test (aarch64 qemu-user) + runs-on: ubuntu-latest + timeout-minutes: 45 + steps: + - uses: actions/checkout@v4 + - name: Cross toolchain + run: | + sudo apt-get update + sudo apt-get install -y gcc-aarch64-linux-gnu qemu-user-static libc6-dev-arm64-cross + - uses: dtolnay/rust-toolchain@stable + with: + targets: aarch64-unknown-linux-gnu + - name: Test under qemu-aarch64 + env: + CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER: aarch64-linux-gnu-gcc + CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_RUNNER: qemu-aarch64-static -L /usr/aarch64-linux-gnu + run: cargo test --workspace --all-targets --target aarch64-unknown-linux-gnu diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..3fe0ed2 --- /dev/null +++ b/.gitignore @@ -0,0 +1,7 @@ +/target +**/*.rs.bk +.DS_Store +*.log +/tmp +.cargo/registry +.cargo/git diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..f5408df --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,34 @@ +# Agent notes + +GPUCache is the G3.5 shared KV tier. RustFS is the colder G4 backend. Read [README.md](README.md) and [docs/adr/0001-integration-route.md](docs/adr/0001-integration-route.md) before changing scope. + +## Phase + +The active phase is **V1 milestone A** unless a human explicitly starts a later milestone. Do not implement V1 B, V2 RDMA, or V3 DPU work in a V1 A change. Do not publish crates, cut releases, deploy, or edit a website unless asked. + +## Honesty rules + +- Do not claim DPU, DOCA, RDMA, SPDK, or CUDA acceleration. V1 is CPU only. An aarch64 build is not a BlueField test. +- Do not invent vLLM, LMCache, or NIXL results. A real engine KV export/import needs a GPU and stays `NOT_RUN_HARDWARE` until that run exists. +- Do not treat a KV block as model-agnostic long-term memory. A miss falls back to recompute. A wrong-tenant or wrong-layout hit is a defect. +- Docs in this repo are English. + +## Crates + +| Crate | May depend on | +| --- | --- | +| `gpucache-core-types` | `sha2` only | +| `gpucache-protocol` | `gpucache-core-types` | +| `gpucache-storage` | the two crates above | + +No crate may add CUDA, DOCA, NIXL, RDMA, or SPDK. `unsafe` is forbidden. Registered buffers whose completion is unknown stay quarantined until drain or a newer epoch; do not free them only because a lease timer fired. + +## Checks + +```bash +cargo fmt --all -- --check +cargo clippy --workspace --all-targets -- -D warnings +cargo test --workspace --all-targets +``` + +`cargo test --workspace --all-targets` is the one-command CPU repro. diff --git a/COMPATIBILITY.md b/COMPATIBILITY.md new file mode 100644 index 0000000..a40b87a --- /dev/null +++ b/COMPATIBILITY.md @@ -0,0 +1,24 @@ +# Compatibility manifest + +Recorded for V1 milestone A. This file is not a performance acceptance report and not a hardware inventory of a BlueField node. + +| Component | Value | Result | +| --- | --- | --- | +| OS | Ubuntu 24.04.5 LTS | Recorded on the V1 A build host | +| Kernel | Linux 6.12.94+ | Recorded on the V1 A build host | +| Architecture exercised locally | x86_64 native, and aarch64 via qemu-user (`qemu-aarch64-static`) | `PASS` for `cargo test --workspace --all-targets` on both. This is not a BlueField run. | +| Rust | stable 1.99.0 (b940084d7 2026-09-28); MSRV declared as 1.85 | `PASS` | +| Python | 3.12.3 present on the build host | Not used by V1 A | +| CUDA | Not installed | `NOT_RUN_HARDWARE` | +| GPU / driver | None in this environment | `NOT_RUN_HARDWARE` | +| vLLM | Not pinned; no engine commit | `NOT_RUN_HARDWARE` | +| LMCache | Not a dependency | `NOT_RUN_HARDWARE` (route undecided) | +| NIXL / UCX / RDMA | Not a dependency | Not in V1 | +| DOCA / BlueField BFB / firmware | Not present | `NOT_RUN_HARDWARE` | +| RustFS server / client | No endpoint authorized | `NOT_RUN_HARDWARE` | +| Direct Rust deps | `sha2` 0.10.8; dev `proptest` 1.4.0 | Apache-2.0 OR MIT; see `Cargo.lock` | +| License of this repo | Apache-2.0 | See `LICENSE` | + +CI (see `.github/workflows/ci.yml`) builds and runs the same tests on x86_64 and on aarch64 (GitHub ARM runner, plus qemu-user cross execution). An aarch64 result is a CPU result. It does not certify BlueField-3, DOCA, or NVMe-on-DPU. + +The integration decision between a native vLLM KV connector and LMCache is pending. No engine commit is pinned because the spike did not run. diff --git a/Cargo.lock b/Cargo.lock new file mode 100644 index 0000000..3c87f09 --- /dev/null +++ b/Cargo.lock @@ -0,0 +1,431 @@ +# This file is automatically @generated by Cargo. +# It is not intended for manual editing. +version = 3 + +[[package]] +name = "autocfg" +version = "1.5.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f2032f911046de80f0a198e0901378627c33f59ea0ac00e363d481118bd70a53" + +[[package]] +name = "bit-set" +version = "0.5.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0700ddab506f33b20a03b13996eccd309a48e5ff77d0d95926aa0210fb4e95f1" +dependencies = [ + "bit-vec", +] + +[[package]] +name = "bit-vec" +version = "0.6.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "349f9b6a179ed607305526ca489b34ad0a41aed5f7980fa90eb03160b69598fb" + +[[package]] +name = "bitflags" +version = "2.13.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3ded4057c258ba199e2d26386d3af3780957ecaee6c4ef4041c6b4b8b97c0b06" + +[[package]] +name = "block-buffer" +version = "0.10.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3078c7629b62d3f0439517fa394996acacc5cbc91c5a20d8c658e77abd503a71" +dependencies = [ + "generic-array", +] + +[[package]] +name = "cfg-if" +version = "1.0.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4e7648175b45a9a48536d676f68d918270699102aa8dab5496df06904c914600" + +[[package]] +name = "cpufeatures" +version = "0.2.17" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "59ed5838eebb26a2bb2e58f6d5b5316989ae9d08bab10e0e6d103e656d1b0280" +dependencies = [ + "libc", +] + +[[package]] +name = "crypto-common" +version = "0.1.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "78c8292055d1c1df0cce5d180393dc8cce0abec0a7102adb6c7b1eef6016d60a" +dependencies = [ + "generic-array", + "typenum", +] + +[[package]] +name = "digest" +version = "0.10.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9ed9a281f7bc9b7576e61468ba615a66a5c8cfdff42420a70aa82701a3b1e292" +dependencies = [ + "block-buffer", + "crypto-common", +] + +[[package]] +name = "errno" +version = "0.3.14" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "39cab71617ae0d63f51a36d69f866391735b51691dbda63cf6f96d042b63efeb" +dependencies = [ + "libc", + "windows-sys", +] + +[[package]] +name = "fastrand" +version = "2.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "da7c62ceae207dd37ea5b845da6a0696c799f85e97da1ab5b7910be3c1c80223" + +[[package]] +name = "fnv" +version = "1.0.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3f9eec918d3f24069decb9af1554cad7c880e2da24a9afd88aca000531ab82c1" + +[[package]] +name = "generic-array" +version = "0.14.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "85649ca51fd72272d7821adaf274ad91c288277713d9c18820d8499a7ff69e9a" +dependencies = [ + "typenum", + "version_check", +] + +[[package]] +name = "getrandom" +version = "0.2.17" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ff2abc00be7fca6ebc474524697ae276ad847ad0a6b3faa4bcb027e9a4614ad0" +dependencies = [ + "cfg-if", + "libc", + "wasi", +] + +[[package]] +name = "getrandom" +version = "0.4.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "300e883d756b2e4ec94e02791f39b04b522276138852cfc41d9fb7e904106099" +dependencies = [ + "cfg-if", + "libc", + "r-efi", +] + +[[package]] +name = "gpucache-core-types" +version = "0.1.0" +dependencies = [ + "proptest", + "sha2", +] + +[[package]] +name = "gpucache-protocol" +version = "0.1.0" +dependencies = [ + "gpucache-core-types", + "proptest", +] + +[[package]] +name = "gpucache-storage" +version = "0.1.0" +dependencies = [ + "gpucache-core-types", + "gpucache-protocol", + "proptest", +] + +[[package]] +name = "lazy_static" +version = "1.5.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "20870f649af7073d53e38067b2a84312175d56ea15217e1b15bc83506ec50afb" + +[[package]] +name = "libc" +version = "0.2.190" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ce5d3ddc6d3fa000eb1536d85e147bfe31aacaba692ed6a876f95cb7c855be78" + +[[package]] +name = "libm" +version = "0.2.16" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b6d2cec3eae94f9f509c767b45932f1ada8350c4bdb85af2fcab4a3c14807981" + +[[package]] +name = "linux-raw-sys" +version = "0.12.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "32a66949e030da00e8c7d4434b251670a91556f4144941d37452769c25d58a53" + +[[package]] +name = "num-traits" +version = "0.2.19" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "071dfc062690e90b734c0b2273ce72ad0ffa95f0c74596bc250dcfd960262841" +dependencies = [ + "autocfg", + "libm", +] + +[[package]] +name = "once_cell" +version = "1.21.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9f7c3e4beb33f85d45ae3e3a1792185706c8e16d043238c593331cc7cd313b50" + +[[package]] +name = "ppv-lite86" +version = "0.2.21" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "85eae3c4ed2f50dcfe72643da4befc30deadb458a9b590d720cde2f2b1e97da9" +dependencies = [ + "zerocopy", +] + +[[package]] +name = "proc-macro2" +version = "1.0.107" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "985e7ec9bb745e6ce6535b544d84d6cd6f7ad8bd711c398938ae983b91a766d9" +dependencies = [ + "unicode-ident", +] + +[[package]] +name = "proptest" +version = "1.4.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "31b476131c3c86cb68032fdc5cb6d5a1045e3e42d96b69fa599fd77701e1f5bf" +dependencies = [ + "bit-set", + "bit-vec", + "bitflags", + "lazy_static", + "num-traits", + "rand", + "rand_chacha", + "rand_xorshift", + "regex-syntax", + "rusty-fork", + "tempfile", + "unarray", +] + +[[package]] +name = "quick-error" +version = "1.2.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a1d01941d82fa2ab50be1e79e6714289dd7cde78eba4c074bc5a4374f650dfe0" + +[[package]] +name = "quote" +version = "1.0.47" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1fbf4db142a473a8d80c26bbf18454ed458bf8d26c8219c331daecfdbd079001" +dependencies = [ + "proc-macro2", +] + +[[package]] +name = "r-efi" +version = "6.0.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f8dcc9c7d52a811697d2151c701e0d08956f92b0e24136cf4cf27b57a6a0d9bf" + +[[package]] +name = "rand" +version = "0.8.8" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e058c7de0b26af77780c769414d6257830bb240f3c38477dbc2c16e5f54d6d4c" +dependencies = [ + "libc", + "rand_chacha", + "rand_core", +] + +[[package]] +name = "rand_chacha" +version = "0.3.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e6c10a63a0fa32252be49d21e7709d4d4baf8d231c2dbce1eaa8141b9b127d88" +dependencies = [ + "ppv-lite86", + "rand_core", +] + +[[package]] +name = "rand_core" +version = "0.6.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ec0be4795e2f6a28069bec0b5ff3e2ac9bafc99e6a9a7dc3547996c5c816922c" +dependencies = [ + "getrandom 0.2.17", +] + +[[package]] +name = "rand_xorshift" +version = "0.3.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d25bf25ec5ae4a3f1b92f929810509a2f53d7dca2f50b794ff57e3face536c8f" +dependencies = [ + "rand_core", +] + +[[package]] +name = "regex-syntax" +version = "0.8.11" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d6f6ff9a378485b298a5286656da665ba74413d36db0979633275d2e708145d4" + +[[package]] +name = "rustix" +version = "1.1.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "891efababe418670775f199f0d233d84843c227a0949a883ce15b37c78d6629d" +dependencies = [ + "bitflags", + "errno", + "libc", + "linux-raw-sys", + "windows-sys", +] + +[[package]] +name = "rusty-fork" +version = "0.3.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cc6bf79ff24e648f6da1f8d1f011e9cac26491b619e6b9280f2b47f1774e6ee2" +dependencies = [ + "fnv", + "quick-error", + "tempfile", + "wait-timeout", +] + +[[package]] +name = "sha2" +version = "0.10.8" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "793db75ad2bcafc3ffa7c68b215fee268f537982cd901d132f89c6343f3a3dc8" +dependencies = [ + "cfg-if", + "cpufeatures", + "digest", +] + +[[package]] +name = "syn" +version = "2.0.119" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "872831b642d1a07999a962a351ed35b955ea2cfc8f3862091e2a240a84f17297" +dependencies = [ + "proc-macro2", + "quote", + "unicode-ident", +] + +[[package]] +name = "tempfile" +version = "3.27.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "32497e9a4c7b38532efcdebeef879707aa9f794296a4f0244f6f69e9bc8574bd" +dependencies = [ + "fastrand", + "getrandom 0.4.3", + "once_cell", + "rustix", + "windows-sys", +] + +[[package]] +name = "typenum" +version = "1.20.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b6f5e870be6c3b371b77fe0ee0bafb859fa4964b4404c27de1d380043c4dda20" + +[[package]] +name = "unarray" +version = "0.1.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "eaea85b334db583fe3274d12b4cd1880032beab409c0d774be044d4480ab9a94" + +[[package]] +name = "unicode-ident" +version = "1.0.27" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a2c754d6c33795a1c324727428e5a7dedb5b06195f9890bdbcba760d3e246563" + +[[package]] +name = "version_check" +version = "0.9.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0b928f33d975fc6ad9f86c8f283853ad26bdd5b10b7f1542aa2fa15e2289105a" + +[[package]] +name = "wait-timeout" +version = "0.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "09ac3b126d3914f9849036f826e054cbabdc8519970b8998ddaf3b5bd3c65f11" +dependencies = [ + "libc", +] + +[[package]] +name = "wasi" +version = "0.11.1+wasi-snapshot-preview1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ccf3ec651a847eb01de73ccad15eb7d99f80485de043efb2f370cd654f4ea44b" + +[[package]] +name = "windows-link" +version = "0.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5" + +[[package]] +name = "windows-sys" +version = "0.61.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ae137229bcbd6cdf0f7b80a31df61766145077ddf49416a728b02cb3921ff3fc" +dependencies = [ + "windows-link", +] + +[[package]] +name = "zerocopy" +version = "0.8.63" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e5fe1f8f1b06191a00962174c61aa5005e0bb391a6d80d07e24d115c01a92ed8" +dependencies = [ + "zerocopy-derive", +] + +[[package]] +name = "zerocopy-derive" +version = "0.8.63" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "863ad3ac83293fb4d740aedbfdc9240dd8d1a50c1099acd76ce80ce7c7230c7f" +dependencies = [ + "proc-macro2", + "quote", + "syn", +] diff --git a/Cargo.toml b/Cargo.toml new file mode 100644 index 0000000..d1af7bd --- /dev/null +++ b/Cargo.toml @@ -0,0 +1,34 @@ +[workspace] +resolver = "2" +members = [ + "crates/core-types", + "crates/protocol", + "crates/storage", +] + +[workspace.package] +version = "0.1.0" +edition = "2021" +license = "Apache-2.0" +rust-version = "1.85" +authors = ["RustFS, Inc."] +repository = "https://github.com/rustfs/gpucache" + +[workspace.dependencies] +sha2 = "=0.10.8" +gpucache-core-types = { path = "crates/core-types" } +gpucache-protocol = { path = "crates/protocol" } +gpucache-storage = { path = "crates/storage" } + +[workspace.lints.rust] +unsafe_code = "forbid" +unused_must_use = "deny" + +[workspace.lints.clippy] +all = { level = "warn", priority = -1 } +doc_markdown = "allow" +module_name_repetitions = "allow" +must_use_candidate = "allow" +missing_errors_doc = "allow" +missing_panics_doc = "allow" +struct_excessive_bools = "allow" diff --git a/README.md b/README.md index 45095a0..ea89caa 100644 --- a/README.md +++ b/README.md @@ -1,69 +1,70 @@ -# RustFS GPUCache 🚀 +# GPUCache -[![Rust](https://img.shields.io/badge/Language-Rust-orange.svg)](https://www.rust-lang.org/) -[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0) -[![Status](https://img.shields.io/badge/Status-Active_Development-brightgreen.svg)]() +GPUCache is an open shared KV-cache tier for disaggregated inference. In NVIDIA CMX terminology this tier is **G3.5**: a shared cache in front of GPU HBM, reused across requests that have the same tenant, model, and layout. [RustFS](https://github.com/rustfs/rustfs) is the colder **G4** capacity backend. G4 is not on the prefill-to-decode hot path. -**GPUCache** is an AI-native, highly optimized KV Cache offloading layer designed specifically for LLM inference and Agentic AI workloads. Built entirely in Rust and engineered for the NVIDIA BlueField DPU ecosystem, it transforms standard EBOF (Ethernet Bunch of Flash) NVMe nodes into a shared, persistent, and hyper-fast L4 context memory pool for GPU clusters. +A KV block is recomputable accelerator state. It is not the user's only copy of a prompt, and it is not model-agnostic long-term memory. A miss, timeout, checksum failure, version mismatch, or store outage must fall back to the inference engine and recompute. A wrong hit is a failure. -We are redefining the **NVIDIA G3.5 Memory Tier**. +## Status ---- +**V1 milestone A is implemented. DPU acceleration is not.** -## đź’ˇ The Vision: Breaking the "Memory Wall" +V1 runs the cache contract on CPU. The long-term deployment target is a storage-side BlueField-3, but this repository does not link DOCA, CUDA, RDMA, or SPDK, and it does not claim that any of those paths exist. An aarch64 build is a CPU build. It is not a BlueField functional test. -In large language model (LLM) inference—especially with long contexts and complex Agent frameworks—GPU HBM (High Bandwidth Memory) capacity is the primary bottleneck. When the KV Cache exceeds VRAM limits, it gets evicted, forcing the GPU to endure the massive "Recompute Tax" when the same context is needed again. +| Milestone | Status | +| --- | --- | +| V1 A: identity, protocol, `MemoryStore`, error/timeout/cancel tests | Implemented in this tree | +| V1 A: real vLLM KV block export/import | `NOT_RUN_HARDWARE` (no GPU in this environment) | +| V1 B: TCP service, file/NVMe, admission, RustFS smoke | Not started | +| V1 C: fuzz, fault injection, engine output comparison, paired baselines | Not started | +| V2: NIC RDMA (NIXL or Mooncake) plus a real RustFS cold path | Not started; blocked on V1 | +| V3: BlueField-3 offload and production hardening | Not started; not purchased and not measured | -Traditional storage is too slow (software overhead, CPU bottlenecks, TCP/IP stack delays). +The integration route (native vLLM KV connector or LMCache) is **not chosen**. See [docs/adr/0001-integration-route.md](docs/adr/0001-integration-route.md). -**GPUCache** solves this by pushing the intelligence directly into the network edge. By offloading KV Cache to an EBOF node powered by an ARM-based DPU, we eliminate the x86 host CPU and RAM entirely. GPUs can read/write context directly to NVMe flash over the network as if it were local memory. +## What V1 A does -## 🏗️ Technical Architecture (The RustFS Edge) +Three library crates, none of which depend on a GPU or DPU: -GPUCache is designed with a radical "Hardware-Software Co-design" philosophy: +- `gpucache-core-types` — versioned `CacheIdentity`, length-delimited physical and logical keys, the block lifecycle, checksums, quotas, and errors. +- `gpucache-protocol` — tenant auth context, batch messages, deadlines, and wire-version negotiation. +- `gpucache-storage` — in-process `MemoryStore`. -* **Zero-GC, 100% Rust:** Unlike other emerging solutions (e.g., MinIO MemKV) that rely on garbage-collected languages, GPUCache is written in Rust. This guarantees predictable, microsecond-level tail latencies and minimal memory footprint—crucial when running on the highly constrained ARM cores of a DPU. -* **Serverless EBOF Architecture:** We completely bypass the traditional x86 storage node architecture. GPUCache runs directly on the NVIDIA BlueField-3/4 DPU, utilizing SPDK to directly manage the underlying NVMe SSDs via a PCIe switch. -* **End-to-End RDMA (RoCEv2):** Context data travels directly from the GPU VRAM over the Spectrum-X network to the DPU, and straight into flash. Zero host-CPU intervention on both the compute and storage sides. -* **Hardware Erasure Coding (EC) Offload:** Instead of wasteful multi-replica setups, we utilize Erasure Coding to maximize storage density. To maintain line-rate performance (400G/800G), the EC parity calculations are offloaded directly to the DPU's hardware acceleration engines via DOCA. -* **Smart Pre-fetching & Eviction:** GPUCache is not just a block device; it is a KV-aware engine. It understands the access patterns of LLM frameworks (integrating with vLLM/TensorRT-LLM) to intelligently route and pre-fetch PageAttention blocks. +The batch API is Lookup, Reserve/Put, Commit, Get, Release/Cancel, and Invalidate. Every item carries `request_id`, epoch, deadline, and, where it applies, generation, length, and checksum. Only a `COMMITTED` block whose identity is compatible is a hit. `ABSENT`, `RESERVED`, `WRITING`, `EVICTING`, and `QUARANTINED` blocks are not readable. -## ⚔️ GPUCache vs. The Status Quo +Lifecycle: -| Feature | Traditional Storage | MinIO MemKV (Go) | RustFS GPUCache (Rust) | -| :--- | :--- | :--- | :--- | -| **Language** | C / C++ / Java | Go (Subject to GC pauses) | **Rust (Zero-cost abstraction, deterministic)** | -| **Data Path** | CPU -> Memory -> NIC | RDMA direct to DPU | **RDMA direct to DPU** | -| **Storage Engine** | File / Object (S3) | Proprietary NixL | **KV-Optimized NVMe-oF** | -| **Resilience** | 3x Replication (Expensive) | Unknown/WIP | **Hardware-Offloaded Erasure Coding** | +```text +ABSENT -> RESERVED -> WRITING -> COMMITTED -> EVICTING -> ABSENT +``` -## 🗺️ Roadmap & Milestones +A failed or timed-out registered region goes to `QUARANTINED`. Lease expiry does not prove that a copy has stopped, so that buffer stays allocated until the owner attests drain or a newer store epoch fences it. Same tokens do not imply the same bytes: the logical prefix key and the physical byte key are different encodings. V1 reuses only within one tenant, one model, and one layout. A reshard, quant, or layout mismatch is a miss. There is no cross-tenant dedup and no partial-prefix hit in this milestone. -- [ ] Core Rust KV engine implementation. -- [ ] Basic RDMA / RoCEv2 communication layer. -- [ ] Integration with SPDK for direct NVMe addressing. -- [ ] NVIDIA DOCA API integration for ARM cross-compilation. -- [ ] DPU Hardware Erasure Coding (EC) offload module. +Resource limits cover object size, in-flight requests, memory registrations, per-tenant in-flight bytes, and retained bytes. The idempotency table is bounded. -## 🤝 Call for Contributors +## What you can run -We are at the frontier of AI infrastructure. Building a CPU-less, DPU-driven storage layer is incredibly challenging and requires expertise across the entire stack. We are actively seeking brilliant minds to join us in this open-source journey. +From a clean checkout, with Rust stable (MSRV 1.85): -We are specifically looking for contributors with expertise in: -1. **Rust Systems Programming:** Low-level memory management, async runtimes, and lock-free data structures. -2. **Network Engineering:** RDMA, RoCEv2, and Infiniband verbs. -3. **Hardware Acceleration:** NVIDIA DOCA, BlueField DPUs, and ARM64 cross-compilation. -4. **AI Inference Frameworks:** Developers familiar with vLLM, TensorRT-LLM, and PageAttention mechanisms to build the client-side interceptors. +```bash +cargo test --workspace --all-targets +``` -### How to get involved: -* **Star the repo** to show your support! -* Check the [Issues](https://github.com/rustfs/gpucache/issues) tab for `good first issue` or `help wanted` tags. -* Join our community discussions in the [Discussions](https://github.com/rustfs/gpucache/discussions) tab to propose architectural improvements. -* Fork the repository and submit your PRs! +That command is the CPU contract repro. It includes unit tests, property tests, timeout and cancel tests, a single-process import of a synthetic block, and a cross-process import through `kv-block-xfer`. The bytes in those tests are synthetic. They are not a vLLM KV tensor. -## đź“„ License +`kv-block-xfer import ` reads one export frame on stdin and imports it into a fresh `MemoryStore`. It does not talk to an engine. -This project is licensed under the Apache 2.0 License - see the [LICENSE](LICENSE) file for details. +## Layout ---- -*Built with passion by the RustFS community. Let's make the Memory Wall a thing of the past.* +```text +crates/core-types identity, keys, lifecycle, errors +crates/protocol batch messages and version negotiation +crates/storage MemoryStore and kv-block-xfer +docs/adr integration-route decision record +COMPATIBILITY.md environment manifest +``` + +Design notes: [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md), [docs/PROTOCOL.md](docs/PROTOCOL.md), [docs/SECURITY.md](docs/SECURITY.md), [docs/OPERATIONS.md](docs/OPERATIONS.md), [docs/BENCHMARKS.md](docs/BENCHMARKS.md), [docs/RISKS.md](docs/RISKS.md). + +## License + +Apache-2.0. See [LICENSE](LICENSE) and [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md). diff --git a/THIRD_PARTY_NOTICES.md b/THIRD_PARTY_NOTICES.md new file mode 100644 index 0000000..01386d0 --- /dev/null +++ b/THIRD_PARTY_NOTICES.md @@ -0,0 +1,12 @@ +# Third-party notices + +GPUCache V1 A is Apache-2.0. See [LICENSE](LICENSE). + +Direct dependencies: + +| Crate | Version | License | Role | +| --- | --- | --- | --- | +| sha2 | 0.10.8 | Apache-2.0 OR MIT | Payload and fingerprint SHA-256 | +| proptest | 1.4.0 | Apache-2.0 OR MIT | Dev-only property tests | + +Transitive crates are pinned in [Cargo.lock](Cargo.lock). They are build or test dependencies (including `rand`, `tempfile`, `regex-syntax`, and cryptographic helpers used by `sha2`). No CUDA, DOCA, NIXL, or other accelerator binary is vendored. This milestone does not publish a release and does not attach a generated SBOM; `cargo metadata --format-version 1` on this lockfile is the input for one when a release is authorized. diff --git a/crates/core-types/Cargo.toml b/crates/core-types/Cargo.toml new file mode 100644 index 0000000..ff3ba5a --- /dev/null +++ b/crates/core-types/Cargo.toml @@ -0,0 +1,18 @@ +[package] +name = "gpucache-core-types" +version.workspace = true +edition.workspace = true +license.workspace = true +rust-version.workspace = true +authors.workspace = true +description = "Versioned KV-cache identity, lifecycle, and error model. No GPU or DPU dependencies." +publish = false + +[dependencies] +sha2.workspace = true + +[dev-dependencies] +proptest = "=1.4.0" + +[lints] +workspace = true diff --git a/crates/core-types/src/checksum.rs b/crates/core-types/src/checksum.rs new file mode 100644 index 0000000..9ca8cae --- /dev/null +++ b/crates/core-types/src/checksum.rs @@ -0,0 +1,47 @@ +// Copyright 2024-2026 RustFS, Inc. +// SPDX-License-Identifier: Apache-2.0 + +use sha2::{Digest as _, Sha256}; + +use crate::error::CacheError; + +/// SHA-256 of payload bytes. Not a substitute for a transport MAC. +#[derive(Clone, Copy, PartialEq, Eq, Hash, Debug)] +pub struct Checksum(pub [u8; 32]); + +impl Checksum { + pub fn as_bytes(&self) -> &[u8; 32] { + &self.0 + } +} + +pub fn content_checksum(payload: &[u8]) -> Checksum { + let dig = Sha256::digest(payload); + let mut out = [0u8; 32]; + out.copy_from_slice(&dig); + Checksum(out) +} + +pub fn verify_checksum(payload: &[u8], expected: Checksum) -> Result<(), CacheError> { + let actual = content_checksum(payload); + if actual != expected { + return Err(CacheError::ChecksumMismatch); + } + Ok(()) +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn checksum_is_stable_and_detects_a_flip() { + let a = content_checksum(b"kv-block"); + assert_eq!(a, content_checksum(b"kv-block")); + assert!(verify_checksum(b"kv-block", a).is_ok()); + assert_eq!( + verify_checksum(b"kv-blokk", a), + Err(CacheError::ChecksumMismatch) + ); + } +} diff --git a/crates/core-types/src/codec.rs b/crates/core-types/src/codec.rs new file mode 100644 index 0000000..ddc7397 --- /dev/null +++ b/crates/core-types/src/codec.rs @@ -0,0 +1,159 @@ +// Copyright 2024-2026 RustFS, Inc. +// SPDX-License-Identifier: Apache-2.0 + +use crate::error::CacheError; + +/// Absolute decoder cap. Store quotas are tighter; this only stops hostile lengths +/// from allocating before validation. +pub const HARD_BYTES_CAP: usize = 64 * 1024 * 1024; + +pub struct ByteWriter { + buf: Vec, +} + +impl ByteWriter { + pub fn new() -> Self { + Self { buf: Vec::new() } + } + + pub fn with_capacity(cap: usize) -> Self { + Self { + buf: Vec::with_capacity(cap), + } + } + + pub fn u8(&mut self, v: u8) { + self.buf.push(v); + } + + pub fn u16(&mut self, v: u16) { + self.buf.extend_from_slice(&v.to_le_bytes()); + } + + pub fn u32(&mut self, v: u32) { + self.buf.extend_from_slice(&v.to_le_bytes()); + } + + pub fn u64(&mut self, v: u64) { + self.buf.extend_from_slice(&v.to_le_bytes()); + } + + pub fn raw(&mut self, bytes: &[u8]) { + self.buf.extend_from_slice(bytes); + } + + /// Length-delimited bytes. The length is part of the encoding so two + /// concatenations of different splits cannot collide. + pub fn bytes(&mut self, bytes: &[u8]) -> Result<(), CacheError> { + if bytes.len() > HARD_BYTES_CAP { + return Err(CacheError::ObjectTooLarge { + limit: HARD_BYTES_CAP as u64, + actual: bytes.len() as u64, + }); + } + self.u32( + u32::try_from(bytes.len()).map_err(|_| CacheError::Malformed { + reason: "field length exceeds u32", + })?, + ); + self.raw(bytes); + Ok(()) + } + + pub fn finish(self) -> Vec { + self.buf + } +} + +impl Default for ByteWriter { + fn default() -> Self { + Self::new() + } +} + +pub struct ByteReader<'a> { + buf: &'a [u8], + i: usize, +} + +impl<'a> ByteReader<'a> { + pub fn new(buf: &'a [u8]) -> Self { + Self { buf, i: 0 } + } + + pub fn remaining(&self) -> usize { + self.buf.len().saturating_sub(self.i) + } + + pub fn finish(self) -> Result<(), CacheError> { + if self.i != self.buf.len() { + return Err(CacheError::Malformed { + reason: "trailing bytes", + }); + } + Ok(()) + } + + fn take(&mut self, n: usize) -> Result<&'a [u8], CacheError> { + let end = self.i.checked_add(n).ok_or(CacheError::Malformed { + reason: "length overflow", + })?; + if end > self.buf.len() { + return Err(CacheError::Malformed { + reason: "truncated buffer", + }); + } + let out = &self.buf[self.i..end]; + self.i = end; + Ok(out) + } + + pub fn u8(&mut self) -> Result { + Ok(self.take(1)?[0]) + } + + pub fn u16(&mut self) -> Result { + let b = self.take(2)?; + Ok(u16::from_le_bytes([b[0], b[1]])) + } + + pub fn u32(&mut self) -> Result { + let b = self.take(4)?; + Ok(u32::from_le_bytes([b[0], b[1], b[2], b[3]])) + } + + pub fn u64(&mut self) -> Result { + let b = self.take(8)?; + Ok(u64::from_le_bytes([ + b[0], b[1], b[2], b[3], b[4], b[5], b[6], b[7], + ])) + } + + pub fn bytes(&mut self, max: usize) -> Result<&'a [u8], CacheError> { + let len = self.u32()? as usize; + if len > max || len > HARD_BYTES_CAP { + return Err(CacheError::ObjectTooLarge { + limit: max.min(HARD_BYTES_CAP) as u64, + actual: len as u64, + }); + } + self.take(len) + } + + pub fn array32(&mut self) -> Result<[u8; 32], CacheError> { + let b = self.take(32)?; + let mut out = [0u8; 32]; + out.copy_from_slice(b); + Ok(out) + } + + pub fn raw(&mut self, n: usize) -> Result<&'a [u8], CacheError> { + if n > HARD_BYTES_CAP { + return Err(CacheError::ObjectTooLarge { + limit: HARD_BYTES_CAP as u64, + actual: n as u64, + }); + } + self.take(n) + } +} diff --git a/crates/core-types/src/error.rs b/crates/core-types/src/error.rs new file mode 100644 index 0000000..aaa360b --- /dev/null +++ b/crates/core-types/src/error.rs @@ -0,0 +1,117 @@ +// Copyright 2024-2026 RustFS, Inc. +// SPDX-License-Identifier: Apache-2.0 + +use core::fmt; + +use crate::lifecycle::BlockState; + +/// Which bounded resource rejected an admission. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum QuotaKind { + InFlightRequests, + Registrations, + InFlightBytes, + RetainedBytes, + TenantRetainedBytes, + BatchItems, + IdempotencyTable, + PrefixLinks, +} + +/// Recoverable contract failure. Payloads and prompts are never embedded. +#[derive(Clone, Debug, PartialEq, Eq)] +pub enum CacheError { + InvalidTransition { + from: BlockState, + event: &'static str, + }, + NotReclaimable { + from: BlockState, + }, + InUse { + refs: u32, + }, + FencingRejected { + reason: &'static str, + }, + ChecksumMismatch, + LengthMismatch { + expected: u64, + actual: u64, + }, + GenerationMismatch { + expected: u64, + actual: u64, + }, + UnknownGeneration, + IdempotencyConflict, + QuotaExceeded { + kind: QuotaKind, + }, + ObjectTooLarge { + limit: u64, + actual: u64, + }, + Unauthorized, + TimedOut, + Cancelled, + Malformed { + reason: &'static str, + }, + WireVersionUnsupported { + local: u16, + remote: u16, + }, + NotExposed { + state: BlockState, + }, + Incompatible, + EpochMismatch { + store: u64, + request: u64, + }, + IdentityInvalid { + reason: &'static str, + }, +} + +impl fmt::Display for CacheError { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + match self { + Self::InvalidTransition { from, event } => { + write!(f, "invalid transition from {from:?} on {event}") + } + Self::NotReclaimable { from } => write!(f, "buffer not reclaimable from {from:?}"), + Self::InUse { refs } => write!(f, "buffer still referenced ({refs})"), + Self::FencingRejected { reason } => write!(f, "fencing rejected: {reason}"), + Self::ChecksumMismatch => write!(f, "checksum mismatch"), + Self::LengthMismatch { expected, actual } => { + write!(f, "length mismatch expected {expected} actual {actual}") + } + Self::GenerationMismatch { expected, actual } => { + write!(f, "generation mismatch expected {expected} actual {actual}") + } + Self::UnknownGeneration => write!(f, "unknown generation"), + Self::IdempotencyConflict => write!(f, "idempotency conflict"), + Self::QuotaExceeded { kind } => write!(f, "quota exceeded: {kind:?}"), + Self::ObjectTooLarge { limit, actual } => { + write!(f, "object too large limit {limit} actual {actual}") + } + Self::Unauthorized => write!(f, "unauthorized"), + Self::TimedOut => write!(f, "timed out"), + Self::Cancelled => write!(f, "cancelled"), + Self::Malformed { reason } => write!(f, "malformed: {reason}"), + Self::WireVersionUnsupported { local, remote } => { + write!(f, "wire version unsupported local {local} remote {remote}") + } + Self::NotExposed { state } => write!(f, "block not exposed in state {state:?}"), + Self::Incompatible => write!(f, "incompatible cache identity"), + Self::EpochMismatch { store, request } => { + write!(f, "epoch mismatch store {store} request {request}") + } + Self::IdentityInvalid { reason } => write!(f, "identity invalid: {reason}"), + } + } +} + +impl std::error::Error for CacheError {} diff --git a/crates/core-types/src/identity.rs b/crates/core-types/src/identity.rs new file mode 100644 index 0000000..72b6623 --- /dev/null +++ b/crates/core-types/src/identity.rs @@ -0,0 +1,669 @@ +// Copyright 2024-2026 RustFS, Inc. +// SPDX-License-Identifier: Apache-2.0 + +use crate::codec::{ByteReader, ByteWriter}; +use crate::error::CacheError; +use crate::limits::IdentityLimits; +use crate::TenantId; + +pub const IDENTITY_VERSION: u16 = 1; + +const TAG_TENANT: u16 = 1; +const TAG_TRUST: u16 = 2; +const TAG_MODEL_REV: u16 = 3; +const TAG_MODEL_CFG: u16 = 4; +const TAG_TOKENIZER: u16 = 5; +const TAG_ADAPTER: u16 = 6; +const TAG_DTYPE: u16 = 7; +const TAG_QUANT: u16 = 8; +const TAG_LAYOUT: u16 = 9; +const TAG_SHARD: u16 = 10; +const TAG_POSITION: u16 = 11; +const TAG_ROPE: u16 = 12; +const TAG_PREFIX: u16 = 13; +const TAG_BLOCK_TOKENS: u16 = 14; +const TAG_VALID_LEN: u16 = 15; +const TAG_MULTIMODAL: u16 = 16; + +/// 32-byte digest. Callers hash weights, configs, and token blocks outside this crate. +#[derive(Clone, Copy, PartialEq, Eq, Hash, Debug)] +pub struct Digest(pub [u8; 32]); + +#[derive(Clone, PartialEq, Eq, Hash, Debug)] +pub struct TrustDomain(Vec); + +impl TrustDomain { + pub fn new(bytes: impl Into>) -> Result { + let bytes = bytes.into(); + if bytes.is_empty() || bytes.len() > crate::limits::DEFAULT_MAX_FIELD_BYTES as usize { + return Err(CacheError::IdentityInvalid { + reason: "trust domain length", + }); + } + Ok(Self(bytes)) + } + + pub fn as_bytes(&self) -> &[u8] { + &self.0 + } +} + +#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)] +#[repr(u16)] +pub enum KvDtype { + F16 = 1, + Bf16 = 2, + F32 = 3, + F8E4M3 = 4, + F8E5M2 = 5, + I8 = 6, +} + +impl KvDtype { + fn from_u16(v: u16) -> Result { + match v { + 1 => Ok(Self::F16), + 2 => Ok(Self::Bf16), + 3 => Ok(Self::F32), + 4 => Ok(Self::F8E4M3), + 5 => Ok(Self::F8E5M2), + 6 => Ok(Self::I8), + _ => Err(CacheError::IdentityInvalid { + reason: "unknown kv dtype", + }), + } + } +} + +#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)] +#[repr(u16)] +pub enum QuantFormat { + None = 1, + Fp8 = 2, + Int8 = 3, +} + +impl QuantFormat { + fn from_u16(v: u16) -> Result { + match v { + 1 => Ok(Self::None), + 2 => Ok(Self::Fp8), + 3 => Ok(Self::Int8), + _ => Err(CacheError::IdentityInvalid { + reason: "unknown quant format", + }), + } + } +} + +#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)] +#[repr(u16)] +pub enum PositionEncoding { + Absolute = 1, + Rope = 2, + Alibi = 3, +} + +impl PositionEncoding { + fn from_u16(v: u16) -> Result { + match v { + 1 => Ok(Self::Absolute), + 2 => Ok(Self::Rope), + 3 => Ok(Self::Alibi), + _ => Err(CacheError::IdentityInvalid { + reason: "unknown position encoding", + }), + } + } +} + +#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)] +#[repr(u16)] +pub enum RopeScaling { + None = 1, + Linear = 2, + Dynamic = 3, +} + +impl RopeScaling { + fn from_u16(v: u16) -> Result { + match v { + 1 => Ok(Self::None), + 2 => Ok(Self::Linear), + 3 => Ok(Self::Dynamic), + _ => Err(CacheError::IdentityInvalid { + reason: "unknown rope scaling", + }), + } + } +} + +/// RoPE parameters stored as raw IEEE-754 bits so the key does not depend on +/// float formatting. +#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)] +pub struct RopeConfig { + pub theta_bits: u64, + pub scaling: RopeScaling, + pub scaling_factor_bits: u64, + pub max_position_embeddings: u32, +} + +#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)] +pub struct LayoutAbi { + pub family: u16, + pub major: u16, + pub minor: u16, +} + +#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)] +pub struct ShardPlacement { + pub tp_size: u16, + pub tp_rank: u16, + pub pp_size: u16, + pub pp_rank: u16, +} + +#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)] +pub enum AdapterIdentity { + None, + Lora { revision: Digest }, +} + +/// Hash of multimodal inputs plus the processor revision. Raw media is not stored. +#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)] +pub struct MultimodalIdentity { + pub processor_revision: Digest, + pub input_digest: Digest, +} + +/// Full identity of one KV block. Fields are hashes and layout parameters. +/// Plaintext prompts and token ids do not belong here. +#[derive(Clone, Debug, PartialEq, Eq, Hash)] +pub struct CacheIdentity { + pub version: u16, + pub tenant: TenantId, + pub trust_domain: TrustDomain, + pub model_revision: Digest, + pub model_config_hash: Digest, + pub tokenizer_revision: Digest, + pub adapter: AdapterIdentity, + pub kv_dtype: KvDtype, + pub quant_format: QuantFormat, + pub layout: LayoutAbi, + pub shard: ShardPlacement, + pub position: PositionEncoding, + pub rope: RopeConfig, + pub prefix_chain: Vec, + pub block_token_count: u32, + pub valid_length: u32, + pub multimodal: Option, +} + +impl CacheIdentity { + /// Canonical identity body. The store still has to call [`Self::validate`]. + pub fn to_bytes(&self) -> Result, CacheError> { + self.encode_body(false) + } + + pub fn from_bytes(bytes: &[u8]) -> Result { + Self::decode_body(bytes, false) + } + + pub fn validate(&self, limits: &IdentityLimits) -> Result<(), CacheError> { + if self.version != IDENTITY_VERSION { + return Err(CacheError::IdentityInvalid { + reason: "identity version", + }); + } + check_field(self.tenant.as_bytes(), limits.max_field_bytes, "tenant")?; + check_field( + self.trust_domain.as_bytes(), + limits.max_field_bytes, + "trust domain", + )?; + if self.layout.family == 0 || self.layout.major == 0 { + return Err(CacheError::IdentityInvalid { + reason: "layout abi", + }); + } + if self.shard.tp_size == 0 + || self.shard.pp_size == 0 + || self.shard.tp_rank >= self.shard.tp_size + || self.shard.pp_rank >= self.shard.pp_size + { + return Err(CacheError::IdentityInvalid { reason: "shard" }); + } + if self.block_token_count == 0 || self.block_token_count > limits.max_block_tokens { + return Err(CacheError::IdentityInvalid { + reason: "block token count", + }); + } + if self.valid_length == 0 || self.valid_length > self.block_token_count { + return Err(CacheError::IdentityInvalid { + reason: "valid length", + }); + } + if self.prefix_chain.is_empty() + || self.prefix_chain.len() > limits.max_prefix_links as usize + { + return Err(CacheError::QuotaExceeded { + kind: crate::QuotaKind::PrefixLinks, + }); + } + if self.rope.scaling == RopeScaling::None && self.rope.scaling_factor_bits != 0 { + return Err(CacheError::IdentityInvalid { + reason: "rope scaling factor", + }); + } + Ok(()) + } + + pub(crate) fn encode_body(&self, logical_only: bool) -> Result, CacheError> { + let mut w = ByteWriter::with_capacity(256); + w.u16(self.version); + write_tlv(&mut w, TAG_TENANT, self.tenant.as_bytes())?; + write_tlv(&mut w, TAG_TRUST, self.trust_domain.as_bytes())?; + write_tlv(&mut w, TAG_MODEL_REV, &self.model_revision.0)?; + write_tlv(&mut w, TAG_MODEL_CFG, &self.model_config_hash.0)?; + write_tlv(&mut w, TAG_TOKENIZER, &self.tokenizer_revision.0)?; + write_tlv(&mut w, TAG_ADAPTER, &encode_adapter(&self.adapter))?; + if !logical_only { + write_tlv(&mut w, TAG_DTYPE, &u16::to_le_bytes(self.kv_dtype as u16))?; + write_tlv( + &mut w, + TAG_QUANT, + &u16::to_le_bytes(self.quant_format as u16), + )?; + write_tlv(&mut w, TAG_LAYOUT, &encode_layout(&self.layout))?; + write_tlv(&mut w, TAG_SHARD, &encode_shard(&self.shard))?; + write_tlv( + &mut w, + TAG_POSITION, + &u16::to_le_bytes(self.position as u16), + )?; + write_tlv(&mut w, TAG_ROPE, &encode_rope(&self.rope))?; + } + write_tlv(&mut w, TAG_PREFIX, &encode_prefix(&self.prefix_chain)?)?; + if !logical_only { + write_tlv( + &mut w, + TAG_BLOCK_TOKENS, + &self.block_token_count.to_le_bytes(), + )?; + write_tlv(&mut w, TAG_VALID_LEN, &self.valid_length.to_le_bytes())?; + } + if let Some(mm) = &self.multimodal { + let mut mm_bytes = Vec::with_capacity(64); + mm_bytes.extend_from_slice(&mm.processor_revision.0); + mm_bytes.extend_from_slice(&mm.input_digest.0); + write_tlv(&mut w, TAG_MULTIMODAL, &mm_bytes)?; + } + Ok(w.finish()) + } + + pub(crate) fn decode_body(bytes: &[u8], logical_only: bool) -> Result { + let mut r = ByteReader::new(bytes); + let version = r.u16()?; + if version != IDENTITY_VERSION { + return Err(CacheError::IdentityInvalid { + reason: "identity version", + }); + } + let mut tenant = None; + let mut trust = None; + let mut model_revision = None; + let mut model_config_hash = None; + let mut tokenizer_revision = None; + let mut adapter = None; + let mut kv_dtype = None; + let mut quant_format = None; + let mut layout = None; + let mut shard = None; + let mut position = None; + let mut rope = None; + let mut prefix_chain = None; + let mut block_token_count = None; + let mut valid_length = None; + let mut multimodal = None; + let mut last_tag = 0u16; + while r.remaining() > 0 { + let tag = r.u16()?; + if tag <= last_tag { + return Err(CacheError::Malformed { + reason: "tlv tags must be strictly increasing", + }); + } + last_tag = tag; + let value = r.bytes(1024 * 1024)?; + match tag { + TAG_TENANT => tenant = Some(TenantId::new(value.to_vec())?), + TAG_TRUST => trust = Some(TrustDomain::new(value.to_vec())?), + TAG_MODEL_REV => model_revision = Some(Digest(expect_len(value, 32)?)), + TAG_MODEL_CFG => model_config_hash = Some(Digest(expect_len(value, 32)?)), + TAG_TOKENIZER => tokenizer_revision = Some(Digest(expect_len(value, 32)?)), + TAG_ADAPTER => adapter = Some(decode_adapter(value)?), + TAG_DTYPE => kv_dtype = Some(KvDtype::from_u16(expect_u16(value)?)?), + TAG_QUANT => quant_format = Some(QuantFormat::from_u16(expect_u16(value)?)?), + TAG_LAYOUT => layout = Some(decode_layout(value)?), + TAG_SHARD => shard = Some(decode_shard(value)?), + TAG_POSITION => position = Some(PositionEncoding::from_u16(expect_u16(value)?)?), + TAG_ROPE => rope = Some(decode_rope(value)?), + TAG_PREFIX => prefix_chain = Some(decode_prefix(value)?), + TAG_BLOCK_TOKENS => block_token_count = Some(expect_u32(value)?), + TAG_VALID_LEN => valid_length = Some(expect_u32(value)?), + TAG_MULTIMODAL => { + if value.len() != 64 { + return Err(CacheError::Malformed { + reason: "multimodal identity", + }); + } + let mut processor = [0u8; 32]; + let mut input = [0u8; 32]; + processor.copy_from_slice(&value[..32]); + input.copy_from_slice(&value[32..]); + multimodal = Some(MultimodalIdentity { + processor_revision: Digest(processor), + input_digest: Digest(input), + }); + } + _ => { + return Err(CacheError::Malformed { + reason: "unknown identity tag", + }) + } + } + } + r.finish()?; + let id = Self { + version, + tenant: tenant.ok_or(CacheError::Malformed { + reason: "missing tenant", + })?, + trust_domain: trust.ok_or(CacheError::Malformed { + reason: "missing trust domain", + })?, + model_revision: model_revision.ok_or(CacheError::Malformed { + reason: "missing model revision", + })?, + model_config_hash: model_config_hash.ok_or(CacheError::Malformed { + reason: "missing model config", + })?, + tokenizer_revision: tokenizer_revision.ok_or(CacheError::Malformed { + reason: "missing tokenizer", + })?, + adapter: adapter.ok_or(CacheError::Malformed { + reason: "missing adapter", + })?, + kv_dtype: if logical_only { + KvDtype::Bf16 + } else { + kv_dtype.ok_or(CacheError::Malformed { + reason: "missing dtype", + })? + }, + quant_format: if logical_only { + QuantFormat::None + } else { + quant_format.ok_or(CacheError::Malformed { + reason: "missing quant", + })? + }, + layout: if logical_only { + LayoutAbi { + family: 1, + major: 1, + minor: 0, + } + } else { + layout.ok_or(CacheError::Malformed { + reason: "missing layout", + })? + }, + shard: if logical_only { + ShardPlacement { + tp_size: 1, + tp_rank: 0, + pp_size: 1, + pp_rank: 0, + } + } else { + shard.ok_or(CacheError::Malformed { + reason: "missing shard", + })? + }, + position: if logical_only { + PositionEncoding::Rope + } else { + position.ok_or(CacheError::Malformed { + reason: "missing position", + })? + }, + rope: if logical_only { + RopeConfig { + theta_bits: 0, + scaling: RopeScaling::None, + scaling_factor_bits: 0, + max_position_embeddings: 1, + } + } else { + rope.ok_or(CacheError::Malformed { + reason: "missing rope", + })? + }, + prefix_chain: prefix_chain.ok_or(CacheError::Malformed { + reason: "missing prefix", + })?, + block_token_count: if logical_only { + 1 + } else { + block_token_count.ok_or(CacheError::Malformed { + reason: "missing block tokens", + })? + }, + valid_length: if logical_only { + 1 + } else { + valid_length.ok_or(CacheError::Malformed { + reason: "missing valid length", + })? + }, + multimodal, + }; + if logical_only + && (kv_dtype.is_some() + || quant_format.is_some() + || layout.is_some() + || shard.is_some() + || position.is_some() + || rope.is_some() + || block_token_count.is_some() + || valid_length.is_some()) + { + return Err(CacheError::Malformed { + reason: "physical field on logical key", + }); + } + Ok(id) + } +} + +fn check_field(bytes: &[u8], max: u32, reason: &'static str) -> Result<(), CacheError> { + if bytes.is_empty() || bytes.len() > max as usize { + return Err(CacheError::IdentityInvalid { reason }); + } + Ok(()) +} + +fn write_tlv(w: &mut ByteWriter, tag: u16, value: &[u8]) -> Result<(), CacheError> { + w.u16(tag); + w.bytes(value) +} + +fn expect_len(value: &[u8], n: usize) -> Result<[u8; 32], CacheError> { + if value.len() != n || n != 32 { + return Err(CacheError::Malformed { + reason: "digest length", + }); + } + let mut out = [0u8; 32]; + out.copy_from_slice(value); + Ok(out) +} + +fn expect_u16(value: &[u8]) -> Result { + if value.len() != 2 { + return Err(CacheError::Malformed { + reason: "u16 field", + }); + } + Ok(u16::from_le_bytes([value[0], value[1]])) +} + +fn expect_u32(value: &[u8]) -> Result { + if value.len() != 4 { + return Err(CacheError::Malformed { + reason: "u32 field", + }); + } + Ok(u32::from_le_bytes([value[0], value[1], value[2], value[3]])) +} + +fn encode_adapter(adapter: &AdapterIdentity) -> Vec { + match adapter { + AdapterIdentity::None => vec![0], + AdapterIdentity::Lora { revision } => { + let mut v = Vec::with_capacity(33); + v.push(1); + v.extend_from_slice(&revision.0); + v + } + } +} + +fn decode_adapter(value: &[u8]) -> Result { + match value.first().copied() { + Some(0) if value.len() == 1 => Ok(AdapterIdentity::None), + Some(1) if value.len() == 33 => { + let mut d = [0u8; 32]; + d.copy_from_slice(&value[1..]); + Ok(AdapterIdentity::Lora { + revision: Digest(d), + }) + } + _ => Err(CacheError::Malformed { reason: "adapter" }), + } +} + +fn encode_layout(layout: &LayoutAbi) -> [u8; 6] { + let mut o = [0u8; 6]; + o[0..2].copy_from_slice(&layout.family.to_le_bytes()); + o[2..4].copy_from_slice(&layout.major.to_le_bytes()); + o[4..6].copy_from_slice(&layout.minor.to_le_bytes()); + o +} + +fn decode_layout(value: &[u8]) -> Result { + if value.len() != 6 { + return Err(CacheError::Malformed { reason: "layout" }); + } + Ok(LayoutAbi { + family: u16::from_le_bytes([value[0], value[1]]), + major: u16::from_le_bytes([value[2], value[3]]), + minor: u16::from_le_bytes([value[4], value[5]]), + }) +} + +fn encode_shard(shard: &ShardPlacement) -> [u8; 8] { + let mut o = [0u8; 8]; + o[0..2].copy_from_slice(&shard.tp_size.to_le_bytes()); + o[2..4].copy_from_slice(&shard.tp_rank.to_le_bytes()); + o[4..6].copy_from_slice(&shard.pp_size.to_le_bytes()); + o[6..8].copy_from_slice(&shard.pp_rank.to_le_bytes()); + o +} + +fn decode_shard(value: &[u8]) -> Result { + if value.len() != 8 { + return Err(CacheError::Malformed { reason: "shard" }); + } + Ok(ShardPlacement { + tp_size: u16::from_le_bytes([value[0], value[1]]), + tp_rank: u16::from_le_bytes([value[2], value[3]]), + pp_size: u16::from_le_bytes([value[4], value[5]]), + pp_rank: u16::from_le_bytes([value[6], value[7]]), + }) +} + +fn encode_rope(rope: &RopeConfig) -> [u8; 22] { + let mut o = [0u8; 22]; + o[0..8].copy_from_slice(&rope.theta_bits.to_le_bytes()); + o[8..10].copy_from_slice(&(rope.scaling as u16).to_le_bytes()); + o[10..18].copy_from_slice(&rope.scaling_factor_bits.to_le_bytes()); + o[18..22].copy_from_slice(&rope.max_position_embeddings.to_le_bytes()); + o +} + +fn decode_rope(value: &[u8]) -> Result { + if value.len() != 22 { + return Err(CacheError::Malformed { reason: "rope" }); + } + Ok(RopeConfig { + theta_bits: u64::from_le_bytes(copy8(&value[0..8])), + scaling: RopeScaling::from_u16(u16::from_le_bytes([value[8], value[9]]))?, + scaling_factor_bits: u64::from_le_bytes(copy8(&value[10..18])), + max_position_embeddings: u32::from_le_bytes(copy4(&value[18..22])), + }) +} + +fn copy8(value: &[u8]) -> [u8; 8] { + let mut out = [0u8; 8]; + out.copy_from_slice(value); + out +} + +fn copy4(value: &[u8]) -> [u8; 4] { + let mut out = [0u8; 4]; + out.copy_from_slice(value); + out +} + +fn encode_prefix(chain: &[Digest]) -> Result, CacheError> { + let mut w = ByteWriter::with_capacity(4 + chain.len() * 32); + let n = u32::try_from(chain.len()).map_err(|_| CacheError::Malformed { + reason: "prefix chain", + })?; + w.u32(n); + for link in chain { + w.raw(&link.0); + } + Ok(w.finish()) +} + +fn decode_prefix(value: &[u8]) -> Result, CacheError> { + let mut r = ByteReader::new(value); + let n = r.u32()? as usize; + if n == 0 || n > 4096 { + return Err(CacheError::Malformed { + reason: "prefix chain", + }); + } + let mut out = Vec::with_capacity(n); + for _ in 0..n { + out.push(Digest(r.array32()?)); + } + r.finish()?; + Ok(out) +} + +/// True when two identities share logical content fields but differ in a +/// byte-layout field. Used by tests; the store compares encoded keys. +pub fn logical_fields_diverge(a: &CacheIdentity, b: &CacheIdentity) -> bool { + a.tenant != b.tenant + || a.trust_domain != b.trust_domain + || a.model_revision != b.model_revision + || a.model_config_hash != b.model_config_hash + || a.tokenizer_revision != b.tokenizer_revision + || a.adapter != b.adapter + || a.prefix_chain != b.prefix_chain + || a.multimodal != b.multimodal +} diff --git a/crates/core-types/src/key.rs b/crates/core-types/src/key.rs new file mode 100644 index 0000000..1d9bb25 --- /dev/null +++ b/crates/core-types/src/key.rs @@ -0,0 +1,97 @@ +// Copyright 2024-2026 RustFS, Inc. +// SPDX-License-Identifier: Apache-2.0 + +use crate::codec::{ByteReader, ByteWriter}; +use crate::error::CacheError; +use crate::identity::{AdapterIdentity, CacheIdentity, Digest, MultimodalIdentity}; +use crate::limits::IdentityLimits; +use crate::{TenantId, TrustDomain}; + +pub const KEY_VERSION: u16 = 1; +pub const PHYSICAL_MAGIC: &[u8; 4] = b"GPCK"; +pub const LOGICAL_MAGIC: &[u8; 4] = b"GPCL"; + +/// Logical prefix key. It identifies content, not the bytes of a layout. +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct LogicalPrefix { + pub version: u16, + pub tenant: TenantId, + pub trust_domain: TrustDomain, + pub model_revision: Digest, + pub model_config_hash: Digest, + pub tokenizer_revision: Digest, + pub adapter: AdapterIdentity, + pub prefix_chain: Vec, + pub multimodal: Option, +} + +impl LogicalPrefix { + pub fn from_identity(id: &CacheIdentity) -> Self { + Self { + version: id.version, + tenant: id.tenant.clone(), + trust_domain: id.trust_domain.clone(), + model_revision: id.model_revision, + model_config_hash: id.model_config_hash, + tokenizer_revision: id.tokenizer_revision, + adapter: id.adapter, + prefix_chain: id.prefix_chain.clone(), + multimodal: id.multimodal, + } + } +} + +/// Physical byte key. Same tokens do not imply the same bytes. +pub fn encode_physical_key( + id: &CacheIdentity, + limits: &IdentityLimits, +) -> Result, CacheError> { + id.validate(limits)?; + encode_key(PHYSICAL_MAGIC, &id.encode_body(false)?) +} + +pub fn encode_logical_key( + id: &CacheIdentity, + limits: &IdentityLimits, +) -> Result, CacheError> { + id.validate(limits)?; + encode_key(LOGICAL_MAGIC, &id.encode_body(true)?) +} + +pub fn decode_physical_key(bytes: &[u8]) -> Result { + let body = strip_key(bytes, PHYSICAL_MAGIC)?; + CacheIdentity::decode_body(&body, false) +} + +pub fn decode_logical_key(bytes: &[u8]) -> Result { + let body = strip_key(bytes, LOGICAL_MAGIC)?; + let id = CacheIdentity::decode_body(&body, true)?; + Ok(LogicalPrefix::from_identity(&id)) +} + +fn encode_key(magic: &[u8; 4], body: &[u8]) -> Result, CacheError> { + let mut w = ByteWriter::with_capacity(8 + body.len()); + w.raw(magic); + w.u16(KEY_VERSION); + w.bytes(body)?; + Ok(w.finish()) +} + +fn strip_key(bytes: &[u8], magic: &[u8; 4]) -> Result, CacheError> { + let mut r = ByteReader::new(bytes); + let got = r.raw(4)?; + if got != magic { + return Err(CacheError::Malformed { + reason: "key magic", + }); + } + let version = r.u16()?; + if version != KEY_VERSION { + return Err(CacheError::IdentityInvalid { + reason: "key version", + }); + } + let body = r.bytes(1024 * 1024)?.to_vec(); + r.finish()?; + Ok(body) +} diff --git a/crates/core-types/src/lib.rs b/crates/core-types/src/lib.rs new file mode 100644 index 0000000..95dde00 --- /dev/null +++ b/crates/core-types/src/lib.rs @@ -0,0 +1,129 @@ +// Copyright 2024-2026 RustFS, Inc. +// SPDX-License-Identifier: Apache-2.0 +//! Versioned cache identity, key codec, block lifecycle, and errors. +//! +//! This crate is the CPU contract for the G3.5 shared KV tier. It does not +//! link CUDA, DOCA, RDMA, or a DPU SDK, and it does not treat a KV block as +//! model-agnostic long-term memory. + +#![forbid(unsafe_code)] + +mod checksum; +mod codec; +mod error; +mod identity; +mod key; +mod lifecycle; +mod limits; + +#[cfg(test)] +mod props; + +pub use checksum::{content_checksum, verify_checksum, Checksum}; +pub use codec::{ByteReader, ByteWriter, HARD_BYTES_CAP}; +pub use error::{CacheError, QuotaKind}; +pub use identity::{ + logical_fields_diverge, AdapterIdentity, CacheIdentity, Digest, KvDtype, LayoutAbi, + MultimodalIdentity, PositionEncoding, QuantFormat, RopeConfig, RopeScaling, ShardPlacement, + TrustDomain, IDENTITY_VERSION, +}; +pub use key::{ + decode_logical_key, decode_physical_key, encode_logical_key, encode_physical_key, + LogicalPrefix, KEY_VERSION, LOGICAL_MAGIC, PHYSICAL_MAGIC, +}; +pub use lifecycle::{Attestation, BlockState, Completion, Effect, Event, Lifecycle}; +pub use limits::{IdentityLimits, ResourceLimits}; + +/// Opaque tenant identifier. Callers must pass an id, never a prompt. +#[derive(Clone, PartialEq, Eq, Hash, Debug)] +pub struct TenantId(Vec); + +impl TenantId { + pub fn new(bytes: impl Into>) -> Result { + let bytes = bytes.into(); + if bytes.is_empty() || bytes.len() > limits::DEFAULT_MAX_FIELD_BYTES as usize { + return Err(CacheError::IdentityInvalid { + reason: "tenant id length", + }); + } + Ok(Self(bytes)) + } + + pub fn as_bytes(&self) -> &[u8] { + &self.0 + } +} + +impl AsRef<[u8]> for TenantId { + fn as_ref(&self) -> &[u8] { + &self.0 + } +} + +/// Client-generated idempotency key. All-zero is rejected. +#[derive(Clone, Copy, PartialEq, Eq, Hash, Debug)] +pub struct RequestId(pub [u8; 16]); + +impl RequestId { + pub fn new(bytes: [u8; 16]) -> Result { + if bytes == [0u8; 16] { + return Err(CacheError::Malformed { + reason: "request_id must not be all zeros", + }); + } + Ok(Self(bytes)) + } + + pub fn from_counter(n: u128) -> Result { + Self::new(n.to_le_bytes()) + } +} + +/// Fixed synthetic identity used by unit tests and the CPU transfer tool. +/// +/// These bytes are not a vLLM KV block. A real engine export remains +/// `NOT_RUN_HARDWARE` until it is executed on a GPU. +pub fn synthetic_identity(nonce: u8, layout_family: u16) -> CacheIdentity { + let mut model = [0x11u8; 32]; + model[0] = nonce; + let mut tokenizer = [0x22u8; 32]; + tokenizer[0] = nonce; + let mut config = [0x33u8; 32]; + config[0] = nonce; + let mut link = [0x44u8; 32]; + link[0] = nonce; + link[1] = layout_family as u8; + CacheIdentity { + version: IDENTITY_VERSION, + tenant: TenantId::new(b"tenant-a".to_vec()).expect("fixture tenant"), + trust_domain: TrustDomain::new(b"td-local".to_vec()).expect("fixture trust"), + model_revision: Digest(model), + model_config_hash: Digest(config), + tokenizer_revision: Digest(tokenizer), + adapter: AdapterIdentity::None, + kv_dtype: KvDtype::Bf16, + quant_format: QuantFormat::None, + layout: LayoutAbi { + family: layout_family, + major: 1, + minor: 0, + }, + shard: ShardPlacement { + tp_size: 1, + tp_rank: 0, + pp_size: 1, + pp_rank: 0, + }, + position: PositionEncoding::Rope, + rope: RopeConfig { + theta_bits: 1_000_000.0f64.to_bits(), + scaling: RopeScaling::None, + scaling_factor_bits: 0, + max_position_embeddings: 4096, + }, + prefix_chain: vec![Digest(link)], + block_token_count: 16, + valid_length: 16, + multimodal: None, + } +} diff --git a/crates/core-types/src/lifecycle.rs b/crates/core-types/src/lifecycle.rs new file mode 100644 index 0000000..5b78005 --- /dev/null +++ b/crates/core-types/src/lifecycle.rs @@ -0,0 +1,553 @@ +// Copyright 2024-2026 RustFS, Inc. +// SPDX-License-Identifier: Apache-2.0 + +use crate::error::CacheError; + +/// Block lifecycle. `Quarantined` is the failure holding state: the buffer +/// stays allocated until drain or fencing proves the transfer has stopped. +#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)] +pub enum BlockState { + Absent, + Reserved, + Writing, + Committed, + Evicting, + Quarantined, +} + +impl BlockState { + pub const fn is_externally_readable(self) -> bool { + matches!(self, Self::Committed) + } +} + +#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)] +pub enum Completion { + NotStarted, + InFlight, + Completed, + Unknown, +} + +/// What the caller claims about in-flight DMA or a copy. +/// +/// `KnownIdle` is not accepted as proof for a registered region. A lease +/// expiry is not proof either. Only `Drained` from the owning request, or a +/// later fenced reclaim, may reuse the buffer. +#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)] +pub enum Attestation { + KnownIdle, + Unknown, + Drained, +} + +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum Event { + Reserve, + BeginWrite, + Commit, + BeginEvict, + FinishEvict, + WriterCancel(Attestation), + WriterTimeout, + ReaderPin, + ReaderUnpin(Attestation), + ReaderLeaseExpire, + ChecksumRejected, + /// Stored bytes failed a read-time checksum. Hide them and hold the buffer. + CorruptHold, + Fence, + /// Drop `count` stuck reader refs after the store epoch has moved. + DropFencedRefs { + count: u32, + }, + ReclaimFenced, +} + +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum Effect { + None, + /// The buffer may be dropped and its registration freed. + ReleaseBuffer, + /// The buffer must stay allocated and must not be handed to a new request. + HoldBuffer, +} + +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct Lifecycle { + pub state: BlockState, + pub registered: bool, + pub refs: u32, + pub completion: Completion, + pub reuse_blocked: bool, + pub fenced: bool, +} + +impl Lifecycle { + pub fn absent() -> Self { + Self { + state: BlockState::Absent, + registered: false, + refs: 0, + completion: Completion::NotStarted, + reuse_blocked: false, + fenced: false, + } + } + + /// Hit visibility. Quarantine, eviction, and partial writes are never hits. + pub fn is_hit_visible(&self) -> bool { + self.state == BlockState::Committed && self.completion == Completion::Completed + } + + pub fn apply(&mut self, event: Event) -> Result { + let name = event_name(event); + match event { + Event::Reserve => self.reserve(name), + Event::BeginWrite => self.begin_write(name), + Event::Commit => self.commit(name), + Event::BeginEvict => self.begin_evict(name), + Event::FinishEvict => self.finish_evict(), + Event::WriterCancel(att) => self.writer_cancel(att), + Event::WriterTimeout => self.writer_timeout(), + Event::ReaderPin => self.reader_pin(name), + Event::ReaderUnpin(att) => self.reader_unpin(att), + Event::ReaderLeaseExpire => self.reader_lease_expire(), + Event::ChecksumRejected => self.checksum_rejected(), + Event::CorruptHold => self.corrupt_hold(), + Event::Fence => self.fence(), + Event::DropFencedRefs { count } => self.drop_fenced_refs(count), + Event::ReclaimFenced => self.reclaim_fenced(), + } + } + + fn reserve(&mut self, event: &'static str) -> Result { + self.require(BlockState::Absent, event)?; + if self.refs != 0 || self.registered { + return Err(CacheError::NotReclaimable { from: self.state }); + } + self.state = BlockState::Reserved; + self.completion = Completion::NotStarted; + self.reuse_blocked = false; + self.fenced = false; + Ok(Effect::None) + } + + fn begin_write(&mut self, event: &'static str) -> Result { + self.require(BlockState::Reserved, event)?; + if self.registered { + return Err(CacheError::InvalidTransition { + from: self.state, + event, + }); + } + self.state = BlockState::Writing; + self.registered = true; + self.completion = Completion::InFlight; + Ok(Effect::HoldBuffer) + } + + fn commit(&mut self, event: &'static str) -> Result { + self.require(BlockState::Writing, event)?; + if !self.registered || self.completion != Completion::InFlight { + return Err(CacheError::InvalidTransition { + from: self.state, + event, + }); + } + self.state = BlockState::Committed; + self.completion = Completion::Completed; + Ok(Effect::None) + } + + fn corrupt_hold(&mut self) -> Result { + if !matches!(self.state, BlockState::Committed | BlockState::Evicting) { + return Err(CacheError::InvalidTransition { + from: self.state, + event: "corrupt_hold", + }); + } + self.enter_quarantine(Completion::Unknown, self.fenced) + } + + fn checksum_rejected(&mut self) -> Result { + self.require(BlockState::Writing, "checksum_rejected")?; + // The bytes were fully observed and rejected. Completion is known, + // so the buffer is reclaimable and must not become readable. + self.clear_to_absent(Completion::Completed); + Ok(Effect::ReleaseBuffer) + } + + fn begin_evict(&mut self, event: &'static str) -> Result { + self.require(BlockState::Committed, event)?; + self.state = BlockState::Evicting; + Ok(Effect::None) + } + + fn finish_evict(&mut self) -> Result { + if self.state != BlockState::Evicting { + return Err(CacheError::InvalidTransition { + from: self.state, + event: "finish_evict", + }); + } + if self.refs != 0 { + return Err(CacheError::InUse { refs: self.refs }); + } + if self.reuse_blocked || self.completion == Completion::Unknown { + return Err(CacheError::NotReclaimable { from: self.state }); + } + self.clear_to_absent(Completion::Completed); + Ok(Effect::ReleaseBuffer) + } + + fn writer_timeout(&mut self) -> Result { + match self.state { + BlockState::Reserved if !self.registered => { + self.clear_to_absent(Completion::NotStarted); + Ok(Effect::ReleaseBuffer) + } + BlockState::Writing => self.enter_quarantine(Completion::Unknown, false), + other => Err(CacheError::InvalidTransition { + from: other, + event: "writer_timeout", + }), + } + } + + fn writer_cancel(&mut self, attestation: Attestation) -> Result { + match (self.state, attestation) { + (BlockState::Reserved, _) if !self.registered => { + self.clear_to_absent(Completion::NotStarted); + Ok(Effect::ReleaseBuffer) + } + (BlockState::Writing, Attestation::Drained) => { + self.clear_to_absent(Completion::Completed); + Ok(Effect::ReleaseBuffer) + } + (BlockState::Writing, Attestation::Unknown | Attestation::KnownIdle) => { + self.enter_quarantine(Completion::Unknown, false) + } + (BlockState::Quarantined, Attestation::Drained) if self.refs == 0 => { + self.clear_to_absent(Completion::Completed); + Ok(Effect::ReleaseBuffer) + } + (state, _) => Err(CacheError::InvalidTransition { + from: state, + event: "writer_cancel", + }), + } + } + + fn reader_pin(&mut self, event: &'static str) -> Result { + self.require(BlockState::Committed, event)?; + self.refs = self + .refs + .checked_add(1) + .ok_or(CacheError::InUse { refs: u32::MAX })?; + Ok(Effect::HoldBuffer) + } + + fn reader_unpin(&mut self, attestation: Attestation) -> Result { + if !matches!(self.state, BlockState::Committed | BlockState::Evicting) || self.refs == 0 { + return Err(CacheError::InvalidTransition { + from: self.state, + event: "reader_unpin", + }); + } + match attestation { + Attestation::Drained => { + self.refs -= 1; + if self.refs == 0 && !self.reuse_blocked { + Ok(Effect::None) + } else { + Ok(Effect::HoldBuffer) + } + } + Attestation::Unknown | Attestation::KnownIdle => { + // Losing track of a reader is not permission to recycle the buffer. + // Checksummed committed bytes stay readable to other callers. + self.reuse_blocked = true; + Ok(Effect::HoldBuffer) + } + } + } + + fn reader_lease_expire(&mut self) -> Result { + if !matches!(self.state, BlockState::Committed | BlockState::Evicting) { + return Err(CacheError::InvalidTransition { + from: self.state, + event: "reader_lease_expire", + }); + } + if self.refs == 0 { + return Ok(Effect::None); + } + // Expiry does not drop the ref and does not prove DMA stopped. + self.reuse_blocked = true; + Ok(Effect::HoldBuffer) + } + + fn fence(&mut self) -> Result { + match self.state { + BlockState::Reserved if !self.registered => { + self.clear_to_absent(Completion::NotStarted); + Ok(Effect::ReleaseBuffer) + } + BlockState::Writing => self.enter_quarantine(Completion::Unknown, true), + BlockState::Quarantined => { + self.fenced = true; + self.reuse_blocked = true; + Ok(Effect::HoldBuffer) + } + BlockState::Evicting => { + self.fenced = true; + Ok(Effect::HoldBuffer) + } + BlockState::Committed | BlockState::Absent => { + self.fenced = true; + Ok(Effect::None) + } + BlockState::Reserved => self.enter_quarantine(Completion::Unknown, true), + } + } + + fn drop_fenced_refs(&mut self, count: u32) -> Result { + if !self.fenced { + return Err(CacheError::FencingRejected { + reason: "epoch has not moved", + }); + } + if count > self.refs { + return Err(CacheError::Malformed { + reason: "stuck ref count", + }); + } + self.refs -= count; + if self.refs == 0 { + self.reuse_blocked = false; + if self.state == BlockState::Committed { + self.completion = Completion::Completed; + } + } + Ok(Effect::None) + } + + fn reclaim_fenced(&mut self) -> Result { + if !self.fenced { + return Err(CacheError::FencingRejected { + reason: "reclaim requires a newer epoch", + }); + } + if self.refs != 0 { + return Err(CacheError::InUse { refs: self.refs }); + } + match self.state { + BlockState::Quarantined | BlockState::Evicting => { + self.clear_to_absent(Completion::Completed); + Ok(Effect::ReleaseBuffer) + } + other => Err(CacheError::NotReclaimable { from: other }), + } + } + + fn enter_quarantine( + &mut self, + completion: Completion, + fenced: bool, + ) -> Result { + self.state = BlockState::Quarantined; + self.completion = completion; + self.reuse_blocked = true; + self.fenced = fenced; + self.registered = true; + Ok(Effect::HoldBuffer) + } + + fn clear_to_absent(&mut self, completion: Completion) { + self.state = BlockState::Absent; + self.registered = false; + self.refs = 0; + self.completion = completion; + self.reuse_blocked = false; + self.fenced = false; + } + + fn require(&self, expected: BlockState, event: &'static str) -> Result<(), CacheError> { + if self.state != expected { + return Err(CacheError::InvalidTransition { + from: self.state, + event, + }); + } + Ok(()) + } +} + +fn event_name(event: Event) -> &'static str { + match event { + Event::Reserve => "reserve", + Event::BeginWrite => "begin_write", + Event::Commit => "commit", + Event::BeginEvict => "begin_evict", + Event::FinishEvict => "finish_evict", + Event::WriterCancel(_) => "writer_cancel", + Event::WriterTimeout => "writer_timeout", + Event::ReaderPin => "reader_pin", + Event::ReaderUnpin(_) => "reader_unpin", + Event::ReaderLeaseExpire => "reader_lease_expire", + Event::ChecksumRejected => "checksum_rejected", + Event::CorruptHold => "corrupt_hold", + Event::Fence => "fence", + Event::DropFencedRefs { .. } => "drop_fenced_refs", + Event::ReclaimFenced => "reclaim_fenced", + } +} + +#[cfg(test)] +mod tests { + use super::*; + + fn drive(events: &[Event]) -> Result { + let mut life = Lifecycle::absent(); + for ev in events { + life.apply(*ev)?; + } + Ok(life) + } + + #[test] + fn happy_path_passes_through_every_published_state() { + let mut life = Lifecycle::absent(); + assert_eq!(life.apply(Event::Reserve).unwrap(), Effect::None); + assert_eq!(life.state, BlockState::Reserved); + assert!(!life.is_hit_visible()); + assert_eq!(life.apply(Event::BeginWrite).unwrap(), Effect::HoldBuffer); + assert_eq!(life.state, BlockState::Writing); + assert!(!life.is_hit_visible()); + life.apply(Event::Commit).unwrap(); + assert!(life.is_hit_visible()); + life.apply(Event::BeginEvict).unwrap(); + assert_eq!(life.state, BlockState::Evicting); + assert!(!life.is_hit_visible()); + assert_eq!( + life.apply(Event::FinishEvict).unwrap(), + Effect::ReleaseBuffer + ); + assert_eq!(life.state, BlockState::Absent); + } + + #[test] + fn commit_is_rejected_before_write_and_after_commit() { + let mut life = Lifecycle::absent(); + assert!(life.apply(Event::Commit).is_err()); + life.apply(Event::Reserve).unwrap(); + assert!(matches!( + life.apply(Event::Commit), + Err(CacheError::InvalidTransition { .. }) + )); + life.apply(Event::BeginWrite).unwrap(); + life.apply(Event::Commit).unwrap(); + assert!(life.apply(Event::Commit).is_err()); + assert!(life.apply(Event::Reserve).is_err()); + } + + #[test] + fn timeout_on_a_write_quarantines_instead_of_freeing() { + let mut life = drive(&[Event::Reserve, Event::BeginWrite, Event::WriterTimeout]).unwrap(); + assert_eq!(life.state, BlockState::Quarantined); + assert!(life.reuse_blocked); + assert_eq!(life.completion, Completion::Unknown); + assert!(!life.is_hit_visible()); + assert!(life.apply(Event::ReclaimFenced).is_err()); + } + + #[test] + fn known_idle_does_not_reclaim_a_registered_write() { + let life = drive(&[ + Event::Reserve, + Event::BeginWrite, + Event::WriterCancel(Attestation::KnownIdle), + ]) + .unwrap(); + assert_eq!(life.state, BlockState::Quarantined); + assert_eq!( + drive(&[ + Event::Reserve, + Event::BeginWrite, + Event::WriterCancel(Attestation::Unknown), + ]) + .unwrap() + .state, + BlockState::Quarantined + ); + } + + #[test] + fn drained_owner_can_reclaim_and_unregistered_timeout_frees() { + let drained = drive(&[ + Event::Reserve, + Event::BeginWrite, + Event::WriterCancel(Attestation::Drained), + ]) + .unwrap(); + assert_eq!(drained.state, BlockState::Absent); + assert!(!drained.registered); + let reserved = drive(&[Event::Reserve, Event::WriterTimeout]).unwrap(); + assert_eq!(reserved.state, BlockState::Absent); + } + + #[test] + fn fenced_reclaim_requires_the_fence_bit_and_zero_refs() { + let mut life = drive(&[Event::Reserve, Event::BeginWrite, Event::Fence]).unwrap(); + assert!(life.fenced); + assert_eq!(life.state, BlockState::Quarantined); + assert_eq!( + life.apply(Event::ReclaimFenced).unwrap(), + Effect::ReleaseBuffer + ); + assert_eq!(life.state, BlockState::Absent); + } + + #[test] + fn reader_lease_expiry_blocks_reuse_until_fence_drops_the_ref() { + let mut life = drive(&[ + Event::Reserve, + Event::BeginWrite, + Event::Commit, + Event::ReaderPin, + ]) + .unwrap(); + life.apply(Event::ReaderLeaseExpire).unwrap(); + life.apply(Event::BeginEvict).unwrap(); + assert!(matches!( + life.apply(Event::FinishEvict), + Err(CacheError::InUse { .. } | CacheError::NotReclaimable { .. }) + )); + assert!(life.apply(Event::DropFencedRefs { count: 1 }).is_err()); + life.apply(Event::Fence).unwrap(); + life.apply(Event::DropFencedRefs { count: 1 }).unwrap(); + assert_eq!(life.refs, 0); + assert_eq!( + life.apply(Event::ReclaimFenced).unwrap(), + Effect::ReleaseBuffer + ); + assert_eq!(life.state, BlockState::Absent); + } + + #[test] + fn checksum_rejection_frees_without_exposing_the_block() { + let life = drive(&[Event::Reserve, Event::BeginWrite, Event::ChecksumRejected]).unwrap(); + assert_eq!(life.state, BlockState::Absent); + assert!(!life.is_hit_visible()); + } + + #[test] + fn absent_to_committed_is_impossible_in_one_event() { + let mut life = Lifecycle::absent(); + assert!(life.apply(Event::Commit).is_err()); + assert!(life.apply(Event::BeginEvict).is_err()); + assert!(life.apply(Event::FinishEvict).is_err()); + assert_eq!(life.state, BlockState::Absent); + } +} diff --git a/crates/core-types/src/limits.rs b/crates/core-types/src/limits.rs new file mode 100644 index 0000000..0b04270 --- /dev/null +++ b/crates/core-types/src/limits.rs @@ -0,0 +1,90 @@ +// Copyright 2024-2026 RustFS, Inc. +// SPDX-License-Identifier: Apache-2.0 + +use crate::error::{CacheError, QuotaKind}; + +pub const DEFAULT_MAX_FIELD_BYTES: u32 = 128; + +/// Bounds checked while validating a [`crate::CacheIdentity`]. +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct IdentityLimits { + pub max_field_bytes: u32, + pub max_prefix_links: u32, + pub max_block_tokens: u32, +} + +impl IdentityLimits { + pub const fn from_resources(limits: &ResourceLimits) -> Self { + Self { + max_field_bytes: limits.max_field_bytes, + max_prefix_links: limits.max_prefix_links, + max_block_tokens: limits.max_block_tokens, + } + } +} + +/// Hard caps enforced by MemoryStore. Every counter is a ceiling, not a hint. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub struct ResourceLimits { + pub max_object_bytes: u64, + pub max_in_flight_requests: u32, + pub max_registrations: u32, + pub max_in_flight_bytes_per_tenant: u64, + pub max_retained_bytes: u64, + pub max_retained_bytes_per_tenant: u64, + pub max_prefix_links: u32, + pub max_batch_items: u32, + pub max_idempotency_entries: u32, + pub idempotency_ttl_ms: u64, + pub max_field_bytes: u32, + pub max_block_tokens: u32, +} + +impl ResourceLimits { + /// Local development ceilings. Not a capacity plan and not a benchmark. + pub const fn dev_defaults() -> Self { + Self { + max_object_bytes: 16 * 1024 * 1024, + max_in_flight_requests: 128, + max_registrations: 1024, + max_in_flight_bytes_per_tenant: 64 * 1024 * 1024, + max_retained_bytes: 256 * 1024 * 1024, + max_retained_bytes_per_tenant: 128 * 1024 * 1024, + max_prefix_links: 1024, + max_batch_items: 64, + max_idempotency_entries: 4096, + idempotency_ttl_ms: 60_000, + max_field_bytes: DEFAULT_MAX_FIELD_BYTES, + max_block_tokens: 1_048_576, + } + } + + pub fn validate(&self) -> Result<(), CacheError> { + if self.max_object_bytes == 0 + || self.max_in_flight_requests == 0 + || self.max_registrations == 0 + || self.max_in_flight_bytes_per_tenant == 0 + || self.max_retained_bytes == 0 + || self.max_retained_bytes_per_tenant == 0 + || self.max_prefix_links == 0 + || self.max_batch_items == 0 + || self.max_idempotency_entries == 0 + || self.idempotency_ttl_ms == 0 + || self.max_field_bytes == 0 + || self.max_block_tokens == 0 + { + return Err(CacheError::Malformed { + reason: "resource limit must be non-zero", + }); + } + if self.max_object_bytes > self.max_retained_bytes + || self.max_in_flight_bytes_per_tenant > self.max_retained_bytes + || self.max_retained_bytes_per_tenant > self.max_retained_bytes + { + return Err(CacheError::QuotaExceeded { + kind: QuotaKind::RetainedBytes, + }); + } + Ok(()) + } +} diff --git a/crates/core-types/src/props.rs b/crates/core-types/src/props.rs new file mode 100644 index 0000000..b49c67b --- /dev/null +++ b/crates/core-types/src/props.rs @@ -0,0 +1,179 @@ +// Copyright 2024-2026 RustFS, Inc. +// SPDX-License-Identifier: Apache-2.0 + +use crate::identity::{AdapterIdentity, Digest, KvDtype, QuantFormat}; +use crate::key::{decode_physical_key, encode_logical_key, encode_physical_key}; +use crate::lifecycle::{BlockState, Event, Lifecycle}; +use crate::limits::IdentityLimits; +use crate::{synthetic_identity, TenantId, TrustDomain}; +use proptest::prelude::*; + +fn limits() -> IdentityLimits { + IdentityLimits { + max_field_bytes: 128, + max_prefix_links: 32, + max_block_tokens: 4096, + } +} + +fn arb_bytes(max: usize) -> impl Strategy> { + prop::collection::vec(any::(), 1..=max) +} + +fn arb_identity() -> impl Strategy { + ( + arb_bytes(16), + arb_bytes(16), + any::(), + 1u16..=4, + 1u16..=4, + 0u16..=3, + any::(), + any::(), + ) + .prop_map( + |(tenant, trust, nonce, family, tp, rank_off, link_n, lora)| { + let mut id = synthetic_identity(nonce, family.max(1)); + id.tenant = TenantId::new(tenant).expect("tenant"); + id.trust_domain = TrustDomain::new(trust).expect("trust"); + let tp = tp.max(1); + id.shard.tp_size = tp; + id.shard.tp_rank = rank_off % tp; + id.kv_dtype = match nonce % 3 { + 0 => KvDtype::Bf16, + 1 => KvDtype::F16, + _ => KvDtype::F32, + }; + id.quant_format = if nonce % 2 == 0 { + QuantFormat::None + } else { + QuantFormat::Fp8 + }; + let mut link = [0u8; 32]; + link[0] = link_n; + id.prefix_chain = vec![Digest(link)]; + if lora { + id.adapter = AdapterIdentity::Lora { + revision: Digest([nonce; 32]), + }; + } + id + }, + ) +} + +proptest! { + #![proptest_config(ProptestConfig::with_cases(64))] + + #[test] + fn physical_key_roundtrip_is_deterministic(id in arb_identity()) { + let limits = limits(); + let once = encode_physical_key(&id, &limits).unwrap(); + let twice = encode_physical_key(&id, &limits).unwrap(); + prop_assert_eq!(once.clone(), twice); + let back = decode_physical_key(&once).unwrap(); + prop_assert_eq!(back.clone(), id); + let again = encode_physical_key(&back, &limits).unwrap(); + prop_assert_eq!(again, once); + } + + #[test] + fn layout_change_keeps_logical_key_and_changes_physical(id in arb_identity(), family in 5u16..=9) { + let limits = limits(); + let mut other = id.clone(); + other.layout.family = family; + let logical_a = encode_logical_key(&id, &limits).unwrap(); + let logical_b = encode_logical_key(&other, &limits).unwrap(); + let physical_a = encode_physical_key(&id, &limits).unwrap(); + let physical_b = encode_physical_key(&other, &limits).unwrap(); + prop_assert_eq!(logical_a, logical_b); + prop_assert_ne!(physical_a, physical_b); + } + + #[test] + fn length_delimited_fields_do_not_alias(a in arb_bytes(12), b in arb_bytes(12)) { + let mut left = synthetic_identity(1, 1); + let mut right = synthetic_identity(1, 1); + let mut tenant_l = a.clone(); + tenant_l.extend_from_slice(&b); + // Split the same byte string at a different boundary. + let split = (a.len() / 2).max(1).min(a.len()); + left.tenant = TenantId::new(a.clone()).unwrap(); + left.trust_domain = TrustDomain::new(b.clone()).unwrap(); + right.tenant = TenantId::new(a[..split].to_vec()).unwrap(); + right.trust_domain = TrustDomain::new({ + let mut t = a[split..].to_vec(); + t.extend_from_slice(&b); + t + }).unwrap(); + let limits = limits(); + if left.tenant.as_bytes() != right.tenant.as_bytes() + || left.trust_domain.as_bytes() != right.trust_domain.as_bytes() + { + let lk = encode_physical_key(&left, &limits).unwrap(); + let rk = encode_physical_key(&right, &limits).unwrap(); + prop_assert_ne!(lk, rk); + } + } +} + +#[test] +fn naive_concatenation_collision_is_rejected_by_lengths() { + let limits = limits(); + let mut ab_c = synthetic_identity(3, 1); + let mut a_bc = synthetic_identity(3, 1); + ab_c.tenant = TenantId::new(b"ab".to_vec()).unwrap(); + ab_c.trust_domain = TrustDomain::new(b"c".to_vec()).unwrap(); + a_bc.tenant = TenantId::new(b"a".to_vec()).unwrap(); + a_bc.trust_domain = TrustDomain::new(b"bc".to_vec()).unwrap(); + let left = encode_physical_key(&ab_c, &limits).unwrap(); + let right = encode_physical_key(&a_bc, &limits).unwrap(); + assert_ne!(left, right); + assert!(!left.windows(12).any(|w| w == b"hello prompt")); +} + +#[test] +fn readable_flag_is_only_set_in_committed() { + let mut life = Lifecycle::absent(); + let events = [ + Event::Reserve, + Event::BeginWrite, + Event::Commit, + Event::ReaderPin, + Event::BeginEvict, + ]; + for ev in events { + let before = life.state; + life.apply(ev).unwrap(); + if life.is_hit_visible() { + assert_eq!(life.state, BlockState::Committed); + } + let _ = before; + } + assert!(!life.is_hit_visible()); +} + +#[test] +fn random_illegal_events_do_not_invent_a_hit() { + let mut lcg = 0x1234_5678_u64; + let mut life = Lifecycle::absent(); + for _ in 0..200 { + lcg = lcg.wrapping_mul(6364136223846793005).wrapping_add(1); + let ev = match lcg % 8 { + 0 => Event::Commit, + 1 => Event::BeginEvict, + 2 => Event::FinishEvict, + 3 => Event::ReclaimFenced, + 4 => Event::ChecksumRejected, + 5 => Event::CorruptHold, + 6 => Event::ReaderPin, + _ => Event::BeginWrite, + }; + let _ = life.apply(ev); + if life.is_hit_visible() { + assert_eq!(life.state, BlockState::Committed); + } else { + assert_ne!(life.state, BlockState::Committed); + } + } +} diff --git a/crates/protocol/Cargo.toml b/crates/protocol/Cargo.toml new file mode 100644 index 0000000..f6f1a28 --- /dev/null +++ b/crates/protocol/Cargo.toml @@ -0,0 +1,18 @@ +[package] +name = "gpucache-protocol" +version.workspace = true +edition.workspace = true +license.workspace = true +rust-version.workspace = true +authors.workspace = true +description = "Versioned batch protocol for the GPUCache shared KV tier. No GPU or DPU dependencies." +publish = false + +[dependencies] +gpucache-core-types.workspace = true + +[dev-dependencies] +proptest = "=1.4.0" + +[lints] +workspace = true diff --git a/crates/protocol/src/api.rs b/crates/protocol/src/api.rs new file mode 100644 index 0000000..94a0812 --- /dev/null +++ b/crates/protocol/src/api.rs @@ -0,0 +1,164 @@ +// Copyright 2024-2026 RustFS, Inc. +// SPDX-License-Identifier: Apache-2.0 + +use gpucache_core_types::{ + Attestation, BlockState, CacheError, CacheIdentity, Checksum, RequestId, +}; + +/// Every mutating and read item carries the fencing epoch and a deadline. +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct ItemHeader { + pub request_id: RequestId, + pub epoch: u64, + pub deadline_ms: u64, +} + +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct LookupItem { + pub header: ItemHeader, + pub identity: CacheIdentity, +} + +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct ReservePutItem { + pub header: ItemHeader, + pub identity: CacheIdentity, + /// Zero allocates a generation. Any other value continues that reservation. + pub generation: u64, + pub length: u64, + pub checksum: Checksum, + pub payload: Option>, + pub lease_ttl_ms: u64, +} + +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct CommitItem { + pub header: ItemHeader, + pub generation: u64, + pub length: u64, + pub checksum: Checksum, +} + +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct GetItem { + pub header: ItemHeader, + pub generation: u64, + pub identity: CacheIdentity, + pub pin_ttl_ms: u64, +} + +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct ReleaseCancelItem { + pub header: ItemHeader, + pub generation: u64, + pub region_id: Option, + /// Set when releasing a reader pin created by `Get`. + pub pin_owner: Option, + pub attestation: Attestation, +} + +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct InvalidateItem { + pub header: ItemHeader, + pub generation: u64, +} + +#[derive(Clone, Debug, PartialEq, Eq)] +pub enum RequestBody { + Lookup(Vec), + ReservePut(Vec), + Commit(Vec), + Get(Vec), + ReleaseCancel(Vec), + Invalidate(Vec), +} + +impl RequestBody { + pub fn len(&self) -> usize { + match self { + Self::Lookup(v) => v.len(), + Self::ReservePut(v) => v.len(), + Self::Commit(v) => v.len(), + Self::Get(v) => v.len(), + Self::ReleaseCancel(v) => v.len(), + Self::Invalidate(v) => v.len(), + } + } + + pub fn is_empty(&self) -> bool { + self.len() == 0 + } +} + +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum MissReason { + Absent, + NotCommitted, + Incompatible, + Evicting, + Quarantined, + Expired, +} + +#[derive(Clone, Debug, PartialEq, Eq)] +pub enum LookupOutcome { + Hit { + generation: u64, + length: u64, + checksum: Checksum, + valid_length: u32, + }, + Miss { + reason: MissReason, + }, +} + +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct ReserveOutcome { + pub generation: u64, + pub state: BlockState, + pub region_id: Option, + pub lease_deadline_ms: u64, +} + +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct CommitOutcome { + pub generation: u64, + pub length: u64, + pub checksum: Checksum, +} + +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct GetHit { + pub generation: u64, + pub length: u64, + pub checksum: Checksum, + pub region_id: u64, + pub lease_deadline_ms: u64, + pub payload: Vec, +} + +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct ReleaseOutcome { + pub generation: u64, + pub state: BlockState, + pub quarantined: bool, +} + +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct InvalidateOutcome { + pub generation: u64, + pub state: BlockState, +} + +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct ItemResult { + pub request_id: RequestId, + pub body: Result, +} + +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct BatchReply { + pub store_epoch: u64, + pub items: Vec>, +} diff --git a/crates/protocol/src/auth.rs b/crates/protocol/src/auth.rs new file mode 100644 index 0000000..2c41feb --- /dev/null +++ b/crates/protocol/src/auth.rs @@ -0,0 +1,62 @@ +// Copyright 2024-2026 RustFS, Inc. +// SPDX-License-Identifier: Apache-2.0 + +use gpucache_core_types::{CacheError, TenantId, TrustDomain}; + +/// Minimum tenant gate. This is not mTLS and not a production authenticator. +/// The fingerprint is a hash of a credential, never the credential itself. +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct AuthContext { + pub tenant: TenantId, + pub trust_domain: TrustDomain, + pub credential_fingerprint: [u8; 32], +} + +impl AuthContext { + pub fn new( + tenant: TenantId, + trust_domain: TrustDomain, + credential_fingerprint: [u8; 32], + ) -> Result { + if credential_fingerprint == [0u8; 32] { + return Err(CacheError::Unauthorized); + } + Ok(Self { + tenant, + trust_domain, + credential_fingerprint, + }) + } +} + +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct TenantPolicy { + pub tenant: TenantId, + pub trust_domain: TrustDomain, + pub credential_fingerprint: [u8; 32], +} + +impl TenantPolicy { + pub fn new( + tenant: TenantId, + trust_domain: TrustDomain, + credential_fingerprint: [u8; 32], + ) -> Result { + if credential_fingerprint == [0u8; 32] { + return Err(CacheError::Unauthorized); + } + Ok(Self { + tenant, + trust_domain, + credential_fingerprint, + }) + } +} + +pub fn fingerprints_equal(a: &[u8; 32], b: &[u8; 32]) -> bool { + let mut diff = 0u8; + for (x, y) in a.iter().zip(b.iter()) { + diff |= x ^ y; + } + diff == 0 +} diff --git a/crates/protocol/src/codec.rs b/crates/protocol/src/codec.rs new file mode 100644 index 0000000..a876378 --- /dev/null +++ b/crates/protocol/src/codec.rs @@ -0,0 +1,593 @@ +// Copyright 2024-2026 RustFS, Inc. +// SPDX-License-Identifier: Apache-2.0 + +use gpucache_core_types::{ + Attestation, ByteReader, ByteWriter, CacheError, CacheIdentity, Checksum, RequestId, + HARD_BYTES_CAP, +}; + +use crate::api::{ + CommitItem, GetItem, InvalidateItem, ItemHeader, LookupItem, ReleaseCancelItem, RequestBody, + ReservePutItem, +}; +use crate::auth::AuthContext; +use crate::negotiate::WIRE_VERSION; + +pub const WIRE_MAGIC: &[u8; 4] = b"GCK1"; +pub const EXPORT_MAGIC: &[u8; 4] = b"GEXP"; +pub const EXPORT_VERSION: u16 = 1; + +const OP_LOOKUP: u16 = 1; +const OP_RESERVE: u16 = 2; +const OP_COMMIT: u16 = 3; +const OP_GET: u16 = 4; +const OP_RELEASE: u16 = 5; +const OP_INVALIDATE: u16 = 6; +const OP_HELLO: u16 = 7; +const OP_HELLO_ACK: u16 = 8; + +const MAX_ITEMS: usize = 1024; + +/// Deadlines are absolute monotonic milliseconds. Zero is rejected. +pub fn ensure_deadline(now_ms: u64, deadline_ms: u64) -> Result<(), CacheError> { + if deadline_ms == 0 { + return Err(CacheError::Malformed { + reason: "deadline required", + }); + } + if now_ms >= deadline_ms { + return Err(CacheError::TimedOut); + } + Ok(()) +} + +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct CommittedExport { + pub identity: CacheIdentity, + pub source_generation: u64, + pub checksum: Checksum, + pub payload: Vec, +} + +pub fn encode_request(auth: &AuthContext, body: &RequestBody) -> Result, CacheError> { + let mut w = ByteWriter::with_capacity(128); + w.raw(WIRE_MAGIC); + w.u16(WIRE_VERSION); + w.u16(op_of(body)); + write_auth(&mut w, auth)?; + w.u32( + u32::try_from(body.len()).map_err(|_| CacheError::Malformed { + reason: "batch length", + })?, + ); + match body { + RequestBody::Lookup(items) => { + for item in items { + write_header(&mut w, &item.header)?; + write_identity(&mut w, &item.identity)?; + } + } + RequestBody::ReservePut(items) => { + for item in items { + write_header(&mut w, &item.header)?; + write_identity(&mut w, &item.identity)?; + w.u64(item.generation); + w.u64(item.length); + w.raw(&item.checksum.0); + write_payload(&mut w, item.payload.as_deref())?; + w.u64(item.lease_ttl_ms); + } + } + RequestBody::Commit(items) => { + for item in items { + write_header(&mut w, &item.header)?; + w.u64(item.generation); + w.u64(item.length); + w.raw(&item.checksum.0); + } + } + RequestBody::Get(items) => { + for item in items { + write_header(&mut w, &item.header)?; + w.u64(item.generation); + write_identity(&mut w, &item.identity)?; + w.u64(item.pin_ttl_ms); + } + } + RequestBody::ReleaseCancel(items) => { + for item in items { + write_header(&mut w, &item.header)?; + w.u64(item.generation); + write_opt_u64(&mut w, item.region_id); + write_opt_request(&mut w, item.pin_owner)?; + w.u8(attestation_byte(item.attestation)); + } + } + RequestBody::Invalidate(items) => { + for item in items { + write_header(&mut w, &item.header)?; + w.u64(item.generation); + } + } + } + Ok(w.finish()) +} + +pub fn decode_request(bytes: &[u8]) -> Result<(AuthContext, RequestBody), CacheError> { + let mut r = ByteReader::new(bytes); + expect_magic(&mut r, WIRE_MAGIC)?; + let version = r.u16()?; + if version != WIRE_VERSION { + return Err(CacheError::WireVersionUnsupported { + local: WIRE_VERSION, + remote: version, + }); + } + let op = r.u16()?; + let auth = read_auth(&mut r)?; + let n = r.u32()? as usize; + if n > MAX_ITEMS { + return Err(CacheError::QuotaExceeded { + kind: gpucache_core_types::QuotaKind::BatchItems, + }); + } + let body = match op { + OP_LOOKUP => { + let mut items = Vec::with_capacity(n); + for _ in 0..n { + items.push(LookupItem { + header: read_header(&mut r)?, + identity: read_identity(&mut r)?, + }); + } + RequestBody::Lookup(items) + } + OP_RESERVE => { + let mut items = Vec::with_capacity(n); + for _ in 0..n { + items.push(ReservePutItem { + header: read_header(&mut r)?, + identity: read_identity(&mut r)?, + generation: r.u64()?, + length: r.u64()?, + checksum: Checksum(r.array32()?), + payload: read_payload(&mut r)?, + lease_ttl_ms: r.u64()?, + }); + } + RequestBody::ReservePut(items) + } + OP_COMMIT => { + let mut items = Vec::with_capacity(n); + for _ in 0..n { + items.push(CommitItem { + header: read_header(&mut r)?, + generation: r.u64()?, + length: r.u64()?, + checksum: Checksum(r.array32()?), + }); + } + RequestBody::Commit(items) + } + OP_GET => { + let mut items = Vec::with_capacity(n); + for _ in 0..n { + items.push(GetItem { + header: read_header(&mut r)?, + generation: r.u64()?, + identity: read_identity(&mut r)?, + pin_ttl_ms: r.u64()?, + }); + } + RequestBody::Get(items) + } + OP_RELEASE => { + let mut items = Vec::with_capacity(n); + for _ in 0..n { + items.push(ReleaseCancelItem { + header: read_header(&mut r)?, + generation: r.u64()?, + region_id: read_opt_u64(&mut r)?, + pin_owner: read_opt_request(&mut r)?, + attestation: read_attestation(r.u8()?)?, + }); + } + RequestBody::ReleaseCancel(items) + } + OP_INVALIDATE => { + let mut items = Vec::with_capacity(n); + for _ in 0..n { + items.push(InvalidateItem { + header: read_header(&mut r)?, + generation: r.u64()?, + }); + } + RequestBody::Invalidate(items) + } + _ => { + return Err(CacheError::Malformed { + reason: "unknown op", + }) + } + }; + r.finish()?; + Ok((auth, body)) +} + +pub fn encode_hello(versions: &[u16]) -> Result, CacheError> { + if versions.len() > 32 { + return Err(CacheError::Malformed { + reason: "hello versions", + }); + } + let mut w = ByteWriter::new(); + w.raw(WIRE_MAGIC); + w.u16(WIRE_VERSION); + w.u16(OP_HELLO); + w.u32(versions.len() as u32); + for v in versions { + w.u16(*v); + } + Ok(w.finish()) +} + +pub fn decode_hello(bytes: &[u8]) -> Result, CacheError> { + let mut r = ByteReader::new(bytes); + expect_magic(&mut r, WIRE_MAGIC)?; + let version = r.u16()?; + if version != WIRE_VERSION { + return Err(CacheError::WireVersionUnsupported { + local: WIRE_VERSION, + remote: version, + }); + } + if r.u16()? != OP_HELLO { + return Err(CacheError::Malformed { reason: "hello op" }); + } + let n = r.u32()? as usize; + if n == 0 || n > 32 { + return Err(CacheError::Malformed { + reason: "hello versions", + }); + } + let mut out = Vec::with_capacity(n); + for _ in 0..n { + out.push(r.u16()?); + } + r.finish()?; + Ok(out) +} + +pub fn encode_hello_ack(selected: u16, epoch: u64) -> Vec { + let mut w = ByteWriter::new(); + w.raw(WIRE_MAGIC); + w.u16(WIRE_VERSION); + w.u16(OP_HELLO_ACK); + w.u16(selected); + w.u64(epoch); + w.finish() +} + +pub fn decode_hello_ack(bytes: &[u8]) -> Result<(u16, u64), CacheError> { + let mut r = ByteReader::new(bytes); + expect_magic(&mut r, WIRE_MAGIC)?; + let version = r.u16()?; + if version != WIRE_VERSION { + return Err(CacheError::WireVersionUnsupported { + local: WIRE_VERSION, + remote: version, + }); + } + if r.u16()? != OP_HELLO_ACK { + return Err(CacheError::Malformed { + reason: "hello ack op", + }); + } + let selected = r.u16()?; + let epoch = r.u64()?; + r.finish()?; + Ok((selected, epoch)) +} + +pub fn encode_export(frame: &CommittedExport) -> Result, CacheError> { + if frame.payload.len() as u64 > HARD_BYTES_CAP as u64 { + return Err(CacheError::ObjectTooLarge { + limit: HARD_BYTES_CAP as u64, + actual: frame.payload.len() as u64, + }); + } + gpucache_core_types::verify_checksum(&frame.payload, frame.checksum)?; + let mut w = ByteWriter::with_capacity(64 + frame.payload.len()); + w.raw(EXPORT_MAGIC); + w.u16(EXPORT_VERSION); + write_identity(&mut w, &frame.identity)?; + w.u64(frame.source_generation); + w.raw(&frame.checksum.0); + w.bytes(&frame.payload)?; + Ok(w.finish()) +} + +pub fn decode_export(bytes: &[u8]) -> Result { + let mut r = ByteReader::new(bytes); + expect_magic(&mut r, EXPORT_MAGIC)?; + let version = r.u16()?; + if version != EXPORT_VERSION { + return Err(CacheError::WireVersionUnsupported { + local: EXPORT_VERSION, + remote: version, + }); + } + let identity = read_identity(&mut r)?; + let source_generation = r.u64()?; + let checksum = Checksum(r.array32()?); + let payload = r.bytes(HARD_BYTES_CAP)?.to_vec(); + r.finish()?; + gpucache_core_types::verify_checksum(&payload, checksum)?; + Ok(CommittedExport { + identity, + source_generation, + checksum, + payload, + }) +} + +fn op_of(body: &RequestBody) -> u16 { + match body { + RequestBody::Lookup(_) => OP_LOOKUP, + RequestBody::ReservePut(_) => OP_RESERVE, + RequestBody::Commit(_) => OP_COMMIT, + RequestBody::Get(_) => OP_GET, + RequestBody::ReleaseCancel(_) => OP_RELEASE, + RequestBody::Invalidate(_) => OP_INVALIDATE, + } +} + +fn write_auth(w: &mut ByteWriter, auth: &AuthContext) -> Result<(), CacheError> { + w.bytes(auth.tenant.as_bytes())?; + w.bytes(auth.trust_domain.as_bytes())?; + w.raw(&auth.credential_fingerprint); + Ok(()) +} + +fn read_auth(r: &mut ByteReader<'_>) -> Result { + let tenant = gpucache_core_types::TenantId::new(r.bytes(4096)?.to_vec())?; + let trust = gpucache_core_types::TrustDomain::new(r.bytes(4096)?.to_vec())?; + let credential_fingerprint = r.array32()?; + AuthContext::new(tenant, trust, credential_fingerprint) +} + +fn write_header(w: &mut ByteWriter, header: &ItemHeader) -> Result<(), CacheError> { + w.raw(&header.request_id.0); + w.u64(header.epoch); + w.u64(header.deadline_ms); + Ok(()) +} + +fn read_header(r: &mut ByteReader<'_>) -> Result { + let mut id = [0u8; 16]; + id.copy_from_slice(r.raw(16)?); + Ok(ItemHeader { + request_id: RequestId::new(id)?, + epoch: r.u64()?, + deadline_ms: r.u64()?, + }) +} + +fn write_identity(w: &mut ByteWriter, id: &CacheIdentity) -> Result<(), CacheError> { + w.bytes(&id.to_bytes()?) +} + +fn read_identity(r: &mut ByteReader<'_>) -> Result { + let bytes = r.bytes(1024 * 1024)?; + CacheIdentity::from_bytes(bytes) +} + +fn write_payload(w: &mut ByteWriter, payload: Option<&[u8]>) -> Result<(), CacheError> { + match payload { + None => { + w.u8(0); + Ok(()) + } + Some(bytes) => { + w.u8(1); + w.bytes(bytes) + } + } +} + +fn read_payload(r: &mut ByteReader<'_>) -> Result>, CacheError> { + match r.u8()? { + 0 => Ok(None), + 1 => Ok(Some(r.bytes(HARD_BYTES_CAP)?.to_vec())), + _ => Err(CacheError::Malformed { + reason: "payload tag", + }), + } +} + +fn write_opt_u64(w: &mut ByteWriter, value: Option) { + match value { + None => w.u8(0), + Some(v) => { + w.u8(1); + w.u64(v); + } + } +} + +fn read_opt_u64(r: &mut ByteReader<'_>) -> Result, CacheError> { + match r.u8()? { + 0 => Ok(None), + 1 => Ok(Some(r.u64()?)), + _ => Err(CacheError::Malformed { + reason: "optional u64", + }), + } +} + +fn write_opt_request(w: &mut ByteWriter, value: Option) -> Result<(), CacheError> { + match value { + None => w.u8(0), + Some(id) => { + w.u8(1); + w.raw(&id.0); + } + } + Ok(()) +} + +fn read_opt_request(r: &mut ByteReader<'_>) -> Result, CacheError> { + match r.u8()? { + 0 => Ok(None), + 1 => { + let mut id = [0u8; 16]; + id.copy_from_slice(r.raw(16)?); + Ok(Some(RequestId::new(id)?)) + } + _ => Err(CacheError::Malformed { + reason: "optional request", + }), + } +} + +fn attestation_byte(value: Attestation) -> u8 { + match value { + Attestation::KnownIdle => 1, + Attestation::Unknown => 2, + Attestation::Drained => 3, + } +} + +fn read_attestation(value: u8) -> Result { + match value { + 1 => Ok(Attestation::KnownIdle), + 2 => Ok(Attestation::Unknown), + 3 => Ok(Attestation::Drained), + _ => Err(CacheError::Malformed { + reason: "attestation", + }), + } +} + +fn expect_magic(r: &mut ByteReader<'_>, magic: &[u8; 4]) -> Result<(), CacheError> { + let got = r.raw(4)?; + if got != magic { + return Err(CacheError::Malformed { reason: "magic" }); + } + Ok(()) +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::negotiate::negotiate; + use gpucache_core_types::{content_checksum, synthetic_identity, TenantId, TrustDomain}; + + fn auth() -> AuthContext { + AuthContext::new( + TenantId::new(b"tenant-a".to_vec()).unwrap(), + TrustDomain::new(b"td-local".to_vec()).unwrap(), + [7u8; 32], + ) + .unwrap() + } + + fn header(n: u128) -> ItemHeader { + ItemHeader { + request_id: RequestId::from_counter(n).unwrap(), + epoch: 1, + deadline_ms: 50, + } + } + + #[test] + fn request_roundtrip_and_truncated_frame_fail() { + let payload = b"synthetic-kv".to_vec(); + let body = RequestBody::ReservePut(vec![ReservePutItem { + header: header(1), + identity: synthetic_identity(1, 1), + generation: 0, + length: payload.len() as u64, + checksum: content_checksum(&payload), + payload: Some(payload.clone()), + lease_ttl_ms: 20, + }]); + let bytes = encode_request(&auth(), &body).unwrap(); + let (got_auth, got_body) = decode_request(&bytes).unwrap(); + assert_eq!(got_auth, auth()); + assert_eq!(got_body, body); + assert!(decode_request(&bytes[..bytes.len() - 1]).is_err()); + let mut flipped = bytes.clone(); + flipped[0] ^= 0xff; + assert!(decode_request(&flipped).is_err()); + } + + #[test] + fn export_rejects_a_bad_checksum_without_returning_bytes() { + let payload = b"block-bytes".to_vec(); + let frame = CommittedExport { + identity: synthetic_identity(2, 1), + source_generation: 4, + checksum: content_checksum(&payload), + payload: payload.clone(), + }; + let bytes = encode_export(&frame).unwrap(); + let back = decode_export(&bytes).unwrap(); + assert_eq!(back.payload, payload); + let mut bad = bytes.clone(); + let idx = bad.len() - 1; + bad[idx] ^= 0x01; + assert_eq!(decode_export(&bad), Err(CacheError::ChecksumMismatch)); + } + + #[test] + fn hello_negotiates_and_deadlines_fail_closed() { + let hello = encode_hello(&[1]).unwrap(); + let remote = decode_hello(&hello).unwrap(); + assert_eq!(negotiate(&[1], &remote).unwrap(), 1); + let ack = encode_hello_ack(1, 1); + assert_eq!(decode_hello_ack(&ack).unwrap(), (1, 1)); + assert_eq!(ensure_deadline(10, 10), Err(CacheError::TimedOut)); + assert!(ensure_deadline(10, 0).is_err()); + assert!(ensure_deadline(10, 11).is_ok()); + } + + #[test] + fn every_batch_op_roundtrips() { + let id = synthetic_identity(4, 2); + let cases = vec![ + RequestBody::Lookup(vec![LookupItem { + header: header(2), + identity: id.clone(), + }]), + RequestBody::Commit(vec![CommitItem { + header: header(3), + generation: 9, + length: 4, + checksum: content_checksum(b"abcd"), + }]), + RequestBody::Get(vec![GetItem { + header: header(4), + generation: 9, + identity: id.clone(), + pin_ttl_ms: 5, + }]), + RequestBody::ReleaseCancel(vec![ReleaseCancelItem { + header: header(5), + generation: 9, + region_id: Some(3), + pin_owner: Some(RequestId::from_counter(4).unwrap()), + attestation: Attestation::Drained, + }]), + RequestBody::Invalidate(vec![InvalidateItem { + header: header(6), + generation: 9, + }]), + ]; + for body in cases { + let bytes = encode_request(&auth(), &body).unwrap(); + let (_, back) = decode_request(&bytes).unwrap(); + assert_eq!(back, body); + } + } +} diff --git a/crates/protocol/src/lib.rs b/crates/protocol/src/lib.rs new file mode 100644 index 0000000..78b39ce --- /dev/null +++ b/crates/protocol/src/lib.rs @@ -0,0 +1,27 @@ +// Copyright 2024-2026 RustFS, Inc. +// SPDX-License-Identifier: Apache-2.0 +//! Batch messages, deadlines, and wire-version negotiation for GPUCache. +//! +//! V1 speaks this contract over an in-process API. A TCP listener is a later +//! milestone. Nothing here depends on a GPU, RDMA stack, or DPU. + +#![forbid(unsafe_code)] + +mod api; +mod auth; +mod codec; +mod negotiate; + +pub use api::{ + BatchReply, CommitItem, CommitOutcome, GetHit, GetItem, InvalidateItem, InvalidateOutcome, + ItemHeader, ItemResult, LookupItem, LookupOutcome, MissReason, ReleaseCancelItem, + ReleaseOutcome, RequestBody, ReserveOutcome, ReservePutItem, +}; +pub use auth::{fingerprints_equal, AuthContext, TenantPolicy}; +pub use codec::{ + decode_export, decode_hello, decode_hello_ack, decode_request, encode_export, encode_hello, + encode_hello_ack, encode_request, ensure_deadline, CommittedExport, EXPORT_VERSION, WIRE_MAGIC, +}; +pub use negotiate::{negotiate, HELLO_OP, WIRE_VERSION}; + +pub use gpucache_core_types::Attestation; diff --git a/crates/protocol/src/negotiate.rs b/crates/protocol/src/negotiate.rs new file mode 100644 index 0000000..6ff48fd --- /dev/null +++ b/crates/protocol/src/negotiate.rs @@ -0,0 +1,52 @@ +// Copyright 2024-2026 RustFS, Inc. +// SPDX-License-Identifier: Apache-2.0 + +use gpucache_core_types::CacheError; + +/// Current store wire version. Unknown versions fail closed. +pub const WIRE_VERSION: u16 = 1; +pub const HELLO_OP: u16 = 7; + +/// Highest version present on both sides. Empty overlap is an error. +pub fn negotiate(local: &[u16], remote: &[u16]) -> Result { + if local.is_empty() || remote.is_empty() { + return Err(CacheError::WireVersionUnsupported { + local: 0, + remote: 0, + }); + } + let mut best: Option = None; + for version in remote { + if local.contains(version) { + best = Some(best.map_or(*version, |b| b.max(*version))); + } + } + match best { + Some(v) => Ok(v), + None => Err(CacheError::WireVersionUnsupported { + local: local.iter().copied().max().unwrap_or(0), + remote: remote.iter().copied().max().unwrap_or(0), + }), + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn picks_the_highest_shared_version() { + assert_eq!(negotiate(&[1], &[1, 2]).unwrap(), 1); + assert_eq!(negotiate(&[1, 2], &[2]).unwrap(), 2); + assert_eq!(negotiate(&[1, 3], &[2, 3, 4]).unwrap(), 3); + } + + #[test] + fn rejects_an_empty_overlap() { + assert!(matches!( + negotiate(&[1], &[2]), + Err(CacheError::WireVersionUnsupported { .. }) + )); + assert!(negotiate(&[], &[1]).is_err()); + } +} diff --git a/crates/storage/Cargo.toml b/crates/storage/Cargo.toml new file mode 100644 index 0000000..07b281b --- /dev/null +++ b/crates/storage/Cargo.toml @@ -0,0 +1,23 @@ +[package] +name = "gpucache-storage" +version.workspace = true +edition.workspace = true +license.workspace = true +rust-version.workspace = true +authors.workspace = true +description = "In-process MemoryStore for the GPUCache V1 A contract. No GPU or DPU dependencies." +publish = false + +[dependencies] +gpucache-core-types.workspace = true +gpucache-protocol.workspace = true + +[dev-dependencies] +proptest = "=1.4.0" + +[[bin]] +name = "kv-block-xfer" +path = "src/bin/kv_block_xfer.rs" + +[lints] +workspace = true diff --git a/crates/storage/src/bin/kv_block_xfer.rs b/crates/storage/src/bin/kv_block_xfer.rs new file mode 100644 index 0000000..866120f --- /dev/null +++ b/crates/storage/src/bin/kv_block_xfer.rs @@ -0,0 +1,84 @@ +// Copyright 2024-2026 RustFS, Inc. +// SPDX-License-Identifier: Apache-2.0 +//! Import one length-delimited KV block frame from stdin into a fresh MemoryStore. +//! +//! The frame is a CPU contract object. This tool does not talk to vLLM and does +//! not claim a GPU or DPU path. + +use std::io::{self, Read}; +use std::process::ExitCode; + +use gpucache_protocol::decode_export; +use gpucache_storage::import_committed_block; + +fn main() -> ExitCode { + let mut args = std::env::args().skip(1); + let Some(cmd) = args.next() else { + eprintln!("usage: kv-block-xfer import "); + return ExitCode::from(1); + }; + if cmd != "import" { + eprintln!("usage: kv-block-xfer import "); + return ExitCode::from(1); + } + let Some(hex_arg) = args.next() else { + eprintln!("error=missing credential fingerprint"); + return ExitCode::from(1); + }; + let Ok(fingerprint) = parse_hex32(&hex_arg) else { + eprintln!("error=credential fingerprint must be 32 bytes of hex"); + return ExitCode::from(1); + }; + let mut buf = Vec::new(); + if io::stdin().read_to_end(&mut buf).is_err() { + eprintln!("error=stdin"); + return ExitCode::from(1); + } + let frame = match decode_export(&buf) { + Ok(frame) => frame, + Err(err) => { + eprintln!("error={err}"); + return ExitCode::from(2); + } + }; + match import_committed_block(&frame, fingerprint, 0) { + Ok(report) => { + println!("status=IMPORTED"); + println!("physical_key={}", report.physical_key_hex); + println!("logical_key={}", report.logical_key_hex); + println!("checksum={}", report.checksum_hex); + println!("length={}", report.length); + println!("source_generation={}", report.source_generation); + println!("local_generation={}", report.local_generation); + println!("state=COMMITTED"); + ExitCode::SUCCESS + } + Err(err) => { + eprintln!("error={err}"); + ExitCode::from(2) + } + } +} + +fn parse_hex32(text: &str) -> Result<[u8; 32], ()> { + if text.len() != 64 { + return Err(()); + } + let mut out = [0u8; 32]; + let bytes = text.as_bytes(); + for i in 0..32 { + let hi = hex_val(bytes[i * 2])?; + let lo = hex_val(bytes[i * 2 + 1])?; + out[i] = (hi << 4) | lo; + } + Ok(out) +} + +fn hex_val(byte: u8) -> Result { + match byte { + b'0'..=b'9' => Ok(byte - b'0'), + b'a'..=b'f' => Ok(byte - b'a' + 10), + b'A'..=b'F' => Ok(byte - b'A' + 10), + _ => Err(()), + } +} diff --git a/crates/storage/src/lib.rs b/crates/storage/src/lib.rs new file mode 100644 index 0000000..20a3079 --- /dev/null +++ b/crates/storage/src/lib.rs @@ -0,0 +1,19 @@ +// Copyright 2024-2026 RustFS, Inc. +// SPDX-License-Identifier: Apache-2.0 +//! In-process MemoryStore for the V1 A cache contract. +//! +//! File, NVMe, and RustFS backends are not implemented here. This crate does +//! not link a GPU, RDMA, or DPU library. + +#![forbid(unsafe_code)] + +mod memory; +mod transfer; + +pub use memory::{MemoryStore, ReclaimRequest, StoreStats}; +pub use transfer::{import_committed_block, ImportReport}; + +#[cfg(test)] +mod props; +#[cfg(test)] +mod tests; diff --git a/crates/storage/src/memory.rs b/crates/storage/src/memory.rs new file mode 100644 index 0000000..7d199b0 --- /dev/null +++ b/crates/storage/src/memory.rs @@ -0,0 +1,1438 @@ +// Copyright 2024-2026 RustFS, Inc. +// SPDX-License-Identifier: Apache-2.0 + +use std::collections::HashMap; + +use gpucache_core_types::{ + content_checksum, encode_logical_key, encode_physical_key, verify_checksum, Attestation, + BlockState, CacheError, CacheIdentity, Checksum, Event, IdentityLimits, Lifecycle, QuotaKind, + RequestId, ResourceLimits, +}; +use gpucache_protocol::{ + ensure_deadline, fingerprints_equal, AuthContext, BatchReply, CommitItem, CommitOutcome, + GetHit, GetItem, InvalidateItem, InvalidateOutcome, ItemResult, LookupItem, LookupOutcome, + MissReason, ReleaseCancelItem, ReleaseOutcome, ReserveOutcome, ReservePutItem, TenantPolicy, +}; + +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +struct Charges { + retained: bool, + inflight: bool, + registration: bool, +} + +impl Charges { + const NONE: Self = Self { + retained: false, + inflight: false, + registration: false, + }; +} + +#[derive(Clone, Copy, Debug)] +struct ChargeTarget { + retained: bool, + inflight: bool, + registration: bool, +} + +#[derive(Clone, Debug)] +struct Pin { + request_id: RequestId, + deadline_ms: u64, + stuck: bool, +} + +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +enum FailKind { + Timeout, + Cancelled, + Fenced, + Corrupt, +} + +struct Slot { + identity: CacheIdentity, + physical: Vec, + logical: Vec, + life: Lifecycle, + length: u64, + checksum: Checksum, + bytes: Option>, + reserve_epoch: u64, + lease_deadline_ms: u64, + region_id: Option, + pins: Vec, + charges: Charges, + fail: Option, +} + +#[derive(Clone, Debug)] +enum IdemKind { + Reserve(Result), + Commit(Result), + Release(Result), + Invalidate(Result), +} + +struct IdemRecord { + fingerprint: Checksum, + inserted_ms: u64, + kind: IdemKind, +} + +/// Explicit fenced reclaim. `observed_epoch` must equal the current store epoch. +pub struct ReclaimRequest { + pub request_id: RequestId, + pub epoch: u64, + pub deadline_ms: u64, + pub generation: u64, + pub observed_epoch: u64, +} + +/// Process-local shared KV store. Only `COMMITTED` compatible blocks are hits. +pub struct MemoryStore { + limits: ResourceLimits, + identity_limits: IdentityLimits, + policies: HashMap, TenantPolicy>, + slots: HashMap, + by_physical: HashMap, u64>, + logical: HashMap, Vec>>, + idem: HashMap, + next_generation: u64, + next_region: u64, + epoch: u64, + retained_bytes: u64, + inflight_requests: u32, + registrations: u32, + tenant_retained: HashMap, u64>, + tenant_inflight: HashMap, u64>, +} + +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub struct StoreStats { + pub retained_bytes: u64, + pub registrations: u32, + pub in_flight_requests: u32, + pub quarantined: u32, + pub committed: u32, + pub idempotency_entries: u32, +} + +impl MemoryStore { + pub fn new(limits: ResourceLimits, policies: Vec) -> Result { + limits.validate()?; + let mut map = HashMap::with_capacity(policies.len()); + for policy in policies { + if map + .insert(policy.tenant.as_bytes().to_vec(), policy) + .is_some() + { + return Err(CacheError::Malformed { + reason: "duplicate tenant policy", + }); + } + } + Ok(Self { + identity_limits: IdentityLimits::from_resources(&limits), + limits, + policies: map, + slots: HashMap::new(), + by_physical: HashMap::new(), + logical: HashMap::new(), + idem: HashMap::new(), + next_generation: 1, + next_region: 1, + epoch: 1, + retained_bytes: 0, + inflight_requests: 0, + registrations: 0, + tenant_retained: HashMap::new(), + tenant_inflight: HashMap::new(), + }) + } + + pub fn epoch(&self) -> u64 { + self.epoch + } + + pub fn stats(&self) -> StoreStats { + let mut quarantined = 0; + let mut committed = 0; + for slot in self.slots.values() { + match slot.life.state { + BlockState::Quarantined => quarantined += 1, + BlockState::Committed => committed += 1, + _ => {} + } + } + StoreStats { + retained_bytes: self.retained_bytes, + registrations: self.registrations, + in_flight_requests: self.inflight_requests, + quarantined, + committed, + idempotency_entries: u32::try_from(self.idem.len()).unwrap_or(u32::MAX), + } + } + + pub fn state_of( + &mut self, + auth: &AuthContext, + generation: u64, + now_ms: u64, + ) -> Result { + self.poll(now_ms); + self.authorize(auth, None)?; + let slot = self + .slots + .get(&generation) + .ok_or(CacheError::UnknownGeneration)?; + self.same_tenant(auth, &slot.identity)?; + Ok(slot.life.state) + } + + /// Advance the store epoch and fence in-flight writers. + /// + /// Committed blocks stay readable. Registered regions whose completion is + /// unknown stay allocated until [`Self::reclaim_fenced`]. + pub fn bump_epoch(&mut self, now_ms: u64) -> u64 { + self.poll(now_ms); + self.epoch = self.epoch.saturating_add(1); + let generations: Vec = self.slots.keys().copied().collect(); + for generation in generations { + let applied = self + .slots + .get_mut(&generation) + .and_then(|slot| slot.life.apply(Event::Fence).ok()); + if applied.is_some() { + if let Some(slot) = self.slots.get_mut(&generation) { + if slot.life.state == BlockState::Quarantined && slot.fail.is_none() { + slot.fail = Some(FailKind::Fenced); + } + } + let _ = self.reconcile(generation); + } + } + self.epoch + } + + pub fn reclaim_fenced( + &mut self, + auth: &AuthContext, + now_ms: u64, + request: ReclaimRequest, + ) -> Result { + self.poll(now_ms); + self.authorize(auth, None)?; + let ReclaimRequest { + request_id, + epoch, + deadline_ms, + generation, + observed_epoch, + } = request; + let fp = fingerprint(&[ + &[9u8][..], + &epoch.to_le_bytes(), + &generation.to_le_bytes(), + &observed_epoch.to_le_bytes(), + ]); + if let Some(prev) = self.replay(request_id, &fp)? { + return match prev { + IdemKind::Release(result) => result, + _ => Err(CacheError::IdempotencyConflict), + }; + } + self.preflight( + &gpucache_protocol::ItemHeader { + request_id, + epoch, + deadline_ms, + }, + now_ms, + )?; + self.ensure_idem_room(now_ms)?; + let result = + self.reclaim_fenced_inner(auth, epoch, deadline_ms, generation, observed_epoch, now_ms); + self.remember(request_id, fp, now_ms, IdemKind::Release(result.clone()))?; + result + } + + pub fn lookup( + &mut self, + auth: &AuthContext, + now_ms: u64, + items: Vec, + ) -> Result, CacheError> { + self.prepare( + auth, + now_ms, + &request_ids(items.iter().map(|i| i.header.request_id)), + )?; + let mut out = Vec::with_capacity(items.len()); + for item in items { + let result = self.lookup_one(auth, now_ms, &item); + out.push(ItemResult { + request_id: item.header.request_id, + body: result, + }); + } + Ok(self.reply(out)) + } + + pub fn reserve_put( + &mut self, + auth: &AuthContext, + now_ms: u64, + items: Vec, + ) -> Result, CacheError> { + self.prepare( + auth, + now_ms, + &request_ids(items.iter().map(|i| i.header.request_id)), + )?; + let mut out = Vec::with_capacity(items.len()); + for item in items { + let id = item.header.request_id; + let result = self.reserve_one(auth, now_ms, &item); + out.push(ItemResult { + request_id: id, + body: result, + }); + } + Ok(self.reply(out)) + } + + pub fn commit( + &mut self, + auth: &AuthContext, + now_ms: u64, + items: Vec, + ) -> Result, CacheError> { + self.prepare( + auth, + now_ms, + &request_ids(items.iter().map(|i| i.header.request_id)), + )?; + let mut out = Vec::with_capacity(items.len()); + for item in items { + let id = item.header.request_id; + let result = self.commit_one(auth, now_ms, &item); + out.push(ItemResult { + request_id: id, + body: result, + }); + } + Ok(self.reply(out)) + } + + pub fn get( + &mut self, + auth: &AuthContext, + now_ms: u64, + items: Vec, + ) -> Result, CacheError> { + self.prepare( + auth, + now_ms, + &request_ids(items.iter().map(|i| i.header.request_id)), + )?; + let mut out = Vec::with_capacity(items.len()); + for item in items { + let id = item.header.request_id; + let result = self.get_one(auth, now_ms, &item); + out.push(ItemResult { + request_id: id, + body: result, + }); + } + Ok(self.reply(out)) + } + + pub fn release_cancel( + &mut self, + auth: &AuthContext, + now_ms: u64, + items: Vec, + ) -> Result, CacheError> { + self.prepare( + auth, + now_ms, + &request_ids(items.iter().map(|i| i.header.request_id)), + )?; + let mut out = Vec::with_capacity(items.len()); + for item in items { + let id = item.header.request_id; + let result = self.release_one(auth, now_ms, &item); + out.push(ItemResult { + request_id: id, + body: result, + }); + } + Ok(self.reply(out)) + } + + pub fn invalidate( + &mut self, + auth: &AuthContext, + now_ms: u64, + items: Vec, + ) -> Result, CacheError> { + self.prepare( + auth, + now_ms, + &request_ids(items.iter().map(|i| i.header.request_id)), + )?; + let mut out = Vec::with_capacity(items.len()); + for item in items { + let id = item.header.request_id; + let result = self.invalidate_one(auth, now_ms, &item); + out.push(ItemResult { + request_id: id, + body: result, + }); + } + Ok(self.reply(out)) + } + + fn reply(&self, items: Vec>) -> BatchReply { + BatchReply { + store_epoch: self.epoch, + items, + } + } + + fn prepare( + &mut self, + auth: &AuthContext, + now_ms: u64, + ids: &[RequestId], + ) -> Result<(), CacheError> { + self.authorize(auth, None)?; + if ids.is_empty() { + return Err(CacheError::Malformed { + reason: "empty batch", + }); + } + if ids.len() > self.limits.max_batch_items as usize { + return Err(CacheError::QuotaExceeded { + kind: QuotaKind::BatchItems, + }); + } + let mut seen = HashMap::::new(); + for id in ids { + if seen.insert(*id, ()).is_some() { + return Err(CacheError::Malformed { + reason: "duplicate request_id", + }); + } + } + self.poll(now_ms); + Ok(()) + } + + fn lookup_one( + &mut self, + auth: &AuthContext, + now_ms: u64, + item: &LookupItem, + ) -> Result { + self.check_header(&item.header, now_ms)?; + self.authorize(auth, Some(&item.identity))?; + item.identity.validate(&self.identity_limits)?; + let physical = encode_physical_key(&item.identity, &self.identity_limits)?; + let logical = encode_logical_key(&item.identity, &self.identity_limits)?; + if let Some(generation) = self.by_physical.get(&physical).copied() { + let slot = self + .slots + .get(&generation) + .ok_or(CacheError::UnknownGeneration)?; + if slot.identity != item.identity { + return Ok(LookupOutcome::Miss { + reason: MissReason::Incompatible, + }); + } + if slot.life.is_hit_visible() { + return Ok(LookupOutcome::Hit { + generation, + length: slot.length, + checksum: slot.checksum, + valid_length: slot.identity.valid_length, + }); + } + return Ok(LookupOutcome::Miss { + reason: miss_reason(slot.life.state), + }); + } + if self + .logical + .get(&logical) + .map(|entries| !entries.is_empty()) + .unwrap_or(false) + { + return Ok(LookupOutcome::Miss { + reason: MissReason::Incompatible, + }); + } + Ok(LookupOutcome::Miss { + reason: MissReason::Absent, + }) + } + + fn reserve_one( + &mut self, + auth: &AuthContext, + now_ms: u64, + item: &ReservePutItem, + ) -> Result { + let fp = reserve_fingerprint(item)?; + if let Some(prev) = self.replay(item.header.request_id, &fp)? { + return match prev { + IdemKind::Reserve(result) => result, + _ => Err(CacheError::IdempotencyConflict), + }; + } + self.preflight(&item.header, now_ms)?; + self.ensure_idem_room(now_ms)?; + let result = self.reserve_new(auth, now_ms, item); + self.remember( + item.header.request_id, + fp, + now_ms, + IdemKind::Reserve(result.clone()), + )?; + result + } + + fn reserve_new( + &mut self, + auth: &AuthContext, + now_ms: u64, + item: &ReservePutItem, + ) -> Result { + self.check_header(&item.header, now_ms)?; + self.authorize(auth, Some(&item.identity))?; + item.identity.validate(&self.identity_limits)?; + if item.length == 0 || item.lease_ttl_ms == 0 { + return Err(CacheError::Malformed { + reason: "length and lease ttl are required", + }); + } + if item.length > self.limits.max_object_bytes { + return Err(CacheError::ObjectTooLarge { + limit: self.limits.max_object_bytes, + actual: item.length, + }); + } + if let Some(payload) = &item.payload { + if payload.len() as u64 != item.length { + return Err(CacheError::LengthMismatch { + expected: item.length, + actual: payload.len() as u64, + }); + } + } + if item.generation != 0 { + return self.continue_put(auth, now_ms, item); + } + let physical = encode_physical_key(&item.identity, &self.identity_limits)?; + let logical = encode_logical_key(&item.identity, &self.identity_limits)?; + if let Some(existing) = self.by_physical.get(&physical).copied() { + let state = self.slots[&existing].life.state; + return Err(CacheError::InvalidTransition { + from: state, + event: "reserve", + }); + } + let lease_deadline_ms = now_ms.saturating_add(item.lease_ttl_ms); + let generation = self.next_generation; + self.next_generation = self.next_generation.saturating_add(1); + let mut life = Lifecycle::absent(); + life.apply(Event::Reserve)?; + let mut region_id = None; + if item.payload.is_some() { + life.apply(Event::BeginWrite)?; + region_id = Some(self.alloc_region()); + } + let slot = Slot { + identity: item.identity.clone(), + physical: physical.clone(), + logical: logical.clone(), + life, + length: item.length, + checksum: item.checksum, + bytes: item.payload.clone(), + reserve_epoch: self.epoch, + lease_deadline_ms, + region_id, + pins: Vec::new(), + charges: Charges::NONE, + fail: None, + }; + self.slots.insert(generation, slot); + self.index_add(physical, logical, generation); + if let Err(err) = self.reconcile(generation) { + self.remove_slot(generation); + return Err(err); + } + let state = self.slots[&generation].life.state; + Ok(ReserveOutcome { + generation, + state, + region_id, + lease_deadline_ms, + }) + } + + fn continue_put( + &mut self, + auth: &AuthContext, + now_ms: u64, + item: &ReservePutItem, + ) -> Result { + let payload = item.payload.clone().ok_or(CacheError::Malformed { + reason: "put requires payload", + })?; + let physical = encode_physical_key(&item.identity, &self.identity_limits)?; + let generation = item.generation; + let (state, reserve_epoch, matches) = { + let slot = self + .slots + .get(&generation) + .ok_or(CacheError::UnknownGeneration)?; + self.same_tenant(auth, &slot.identity)?; + ( + slot.life.state, + slot.reserve_epoch, + slot.physical == physical + && slot.checksum == item.checksum + && slot.length == item.length, + ) + }; + if !matches { + return Err(CacheError::IdempotencyConflict); + } + if state != BlockState::Reserved { + return Err(CacheError::InvalidTransition { + from: state, + event: "begin_write", + }); + } + if reserve_epoch != self.epoch { + return Err(CacheError::FencingRejected { + reason: "reservation epoch is stale", + }); + } + if self.registrations >= self.limits.max_registrations { + return Err(CacheError::QuotaExceeded { + kind: QuotaKind::Registrations, + }); + } + let region_id = self.alloc_region(); + { + let slot = self + .slots + .get_mut(&generation) + .ok_or(CacheError::UnknownGeneration)?; + slot.life.apply(Event::BeginWrite)?; + slot.bytes = Some(payload); + slot.region_id = Some(region_id); + slot.lease_deadline_ms = now_ms.saturating_add(item.lease_ttl_ms); + } + self.reconcile(generation)?; + Ok(ReserveOutcome { + generation, + state: self.slots[&generation].life.state, + region_id: Some(region_id), + lease_deadline_ms: self.slots[&generation].lease_deadline_ms, + }) + } + + fn commit_one( + &mut self, + auth: &AuthContext, + now_ms: u64, + item: &CommitItem, + ) -> Result { + let fp = fingerprint(&[ + &[3u8][..], + &item.header.epoch.to_le_bytes(), + &item.generation.to_le_bytes(), + &item.length.to_le_bytes(), + &item.checksum.0, + ]); + if let Some(prev) = self.replay(item.header.request_id, &fp)? { + return match prev { + IdemKind::Commit(result) => result, + _ => Err(CacheError::IdempotencyConflict), + }; + } + self.preflight(&item.header, now_ms)?; + self.ensure_idem_room(now_ms)?; + let result = self.commit_new(auth, now_ms, item); + self.remember( + item.header.request_id, + fp, + now_ms, + IdemKind::Commit(result.clone()), + )?; + result + } + + fn commit_new( + &mut self, + auth: &AuthContext, + now_ms: u64, + item: &CommitItem, + ) -> Result { + self.check_header(&item.header, now_ms)?; + let generation = item.generation; + let (reserve_epoch, state, fail, declared_len, declared_sum, payload) = { + let slot = self + .slots + .get(&generation) + .ok_or(CacheError::UnknownGeneration)?; + self.same_tenant(auth, &slot.identity)?; + ( + slot.reserve_epoch, + slot.life.state, + slot.fail, + slot.length, + slot.checksum, + slot.bytes.clone().unwrap_or_default(), + ) + }; + if reserve_epoch != self.epoch { + return Err(CacheError::FencingRejected { + reason: "commit epoch is stale", + }); + } + if state == BlockState::Quarantined { + return Err(fail_error(fail, state)); + } + if state != BlockState::Writing { + return Err(CacheError::NotExposed { state }); + } + if declared_len != item.length { + self.reject_write(generation)?; + return Err(CacheError::LengthMismatch { + expected: declared_len, + actual: item.length, + }); + } + if item.checksum != declared_sum || verify_checksum(&payload, declared_sum).is_err() { + self.reject_write(generation)?; + return Err(CacheError::ChecksumMismatch); + } + self.apply_event(generation, Event::Commit, None)?; + Ok(CommitOutcome { + generation, + length: item.length, + checksum: declared_sum, + }) + } + + fn reject_write(&mut self, generation: u64) -> Result<(), CacheError> { + // Completion is known: the bytes were inspected and rejected. + self.apply_event(generation, Event::ChecksumRejected, None) + } + + fn get_one( + &mut self, + auth: &AuthContext, + now_ms: u64, + item: &GetItem, + ) -> Result { + self.check_header(&item.header, now_ms)?; + self.authorize(auth, Some(&item.identity))?; + if item.pin_ttl_ms == 0 { + return Err(CacheError::Malformed { + reason: "pin ttl required", + }); + } + let generation = item.generation; + let (visible, fail, state, region_id, payload, checksum, length, same_identity) = { + let slot = self + .slots + .get(&generation) + .ok_or(CacheError::UnknownGeneration)?; + self.same_tenant(auth, &slot.identity)?; + ( + slot.life.is_hit_visible(), + slot.fail, + slot.life.state, + slot.region_id, + slot.bytes.clone().unwrap_or_default(), + slot.checksum, + slot.length, + slot.identity == item.identity, + ) + }; + if !same_identity { + return Err(CacheError::Incompatible); + } + if !visible { + return Err(fail_error(fail, state)); + } + let region_id = region_id.ok_or(CacheError::Malformed { + reason: "missing region", + })?; + if payload.len() as u64 != length || verify_checksum(&payload, checksum).is_err() { + self.apply_event(generation, Event::CorruptHold, Some(FailKind::Corrupt))?; + return Err(CacheError::ChecksumMismatch); + } + let lease_deadline_ms = now_ms.saturating_add(item.pin_ttl_ms); + { + let slot = self + .slots + .get_mut(&generation) + .ok_or(CacheError::UnknownGeneration)?; + slot.life.apply(Event::ReaderPin)?; + slot.pins.push(Pin { + request_id: item.header.request_id, + deadline_ms: lease_deadline_ms, + stuck: false, + }); + } + Ok(GetHit { + generation, + length, + checksum, + region_id, + lease_deadline_ms, + payload, + }) + } + + fn release_one( + &mut self, + auth: &AuthContext, + now_ms: u64, + item: &ReleaseCancelItem, + ) -> Result { + let fp = release_fingerprint(item); + if let Some(prev) = self.replay(item.header.request_id, &fp)? { + return match prev { + IdemKind::Release(result) => result, + _ => Err(CacheError::IdempotencyConflict), + }; + } + self.preflight(&item.header, now_ms)?; + self.ensure_idem_room(now_ms)?; + let result = self.release_new(auth, now_ms, item); + self.remember( + item.header.request_id, + fp, + now_ms, + IdemKind::Release(result.clone()), + )?; + result + } + + fn release_new( + &mut self, + auth: &AuthContext, + now_ms: u64, + item: &ReleaseCancelItem, + ) -> Result { + self.check_header(&item.header, now_ms)?; + let generation = item.generation; + let slot = self + .slots + .get(&generation) + .ok_or(CacheError::UnknownGeneration)?; + self.same_tenant(auth, &slot.identity)?; + if let Some(pin_owner) = item.pin_owner { + return self.release_pin(item, generation, pin_owner); + } + let fail = match item.attestation { + Attestation::Drained => None, + Attestation::Unknown | Attestation::KnownIdle => Some(FailKind::Cancelled), + }; + let event = Event::WriterCancel(item.attestation); + self.apply_event(generation, event, fail)?; + self.outcome_for(generation, item.generation) + } + + fn release_pin( + &mut self, + item: &ReleaseCancelItem, + generation: u64, + pin_owner: RequestId, + ) -> Result { + let region = item.region_id.ok_or(CacheError::Malformed { + reason: "pin region required", + })?; + { + let slot = self + .slots + .get_mut(&generation) + .ok_or(CacheError::UnknownGeneration)?; + if slot.region_id != Some(region) { + return Err(CacheError::Malformed { + reason: "region mismatch", + }); + } + let pos = slot + .pins + .iter() + .position(|pin| pin.request_id == pin_owner) + .ok_or(CacheError::UnknownGeneration)?; + match item.attestation { + Attestation::Drained => { + slot.pins.remove(pos); + slot.life.apply(Event::ReaderUnpin(Attestation::Drained))?; + } + Attestation::Unknown | Attestation::KnownIdle => { + slot.pins[pos].stuck = true; + slot.life.apply(Event::ReaderUnpin(item.attestation))?; + } + } + } + self.maybe_finish(generation)?; + self.outcome_for(generation, generation) + } + + fn invalidate_one( + &mut self, + auth: &AuthContext, + now_ms: u64, + item: &InvalidateItem, + ) -> Result { + let fp = fingerprint(&[ + &[6u8][..], + &item.header.epoch.to_le_bytes(), + &item.generation.to_le_bytes(), + ]); + if let Some(prev) = self.replay(item.header.request_id, &fp)? { + return match prev { + IdemKind::Invalidate(result) => result, + _ => Err(CacheError::IdempotencyConflict), + }; + } + self.preflight(&item.header, now_ms)?; + self.ensure_idem_room(now_ms)?; + let result = self.invalidate_new(auth, now_ms, item); + self.remember( + item.header.request_id, + fp, + now_ms, + IdemKind::Invalidate(result.clone()), + )?; + result + } + + fn invalidate_new( + &mut self, + auth: &AuthContext, + now_ms: u64, + item: &InvalidateItem, + ) -> Result { + self.check_header(&item.header, now_ms)?; + let generation = item.generation; + let slot = self + .slots + .get(&generation) + .ok_or(CacheError::UnknownGeneration)?; + self.same_tenant(auth, &slot.identity)?; + if !slot.life.is_hit_visible() { + return Err(CacheError::NotExposed { + state: slot.life.state, + }); + } + self.apply_event(generation, Event::BeginEvict, None)?; + self.maybe_finish(generation)?; + let state = self + .slots + .get(&generation) + .map(|s| s.life.state) + .unwrap_or(BlockState::Absent); + Ok(InvalidateOutcome { generation, state }) + } + + fn reclaim_fenced_inner( + &mut self, + auth: &AuthContext, + epoch: u64, + deadline_ms: u64, + generation: u64, + observed_epoch: u64, + now_ms: u64, + ) -> Result { + self.check_header( + &gpucache_protocol::ItemHeader { + request_id: RequestId::from_counter(1).map_err(|_| CacheError::Malformed { + reason: "request_id", + })?, + epoch, + deadline_ms, + }, + now_ms, + )?; + if observed_epoch != self.epoch { + return Err(CacheError::FencingRejected { + reason: "observed epoch is stale", + }); + } + let slot = self + .slots + .get(&generation) + .ok_or(CacheError::UnknownGeneration)?; + self.same_tenant(auth, &slot.identity)?; + if !slot.life.fenced { + return Err(CacheError::FencingRejected { + reason: "reclaim requires a newer epoch", + }); + } + let stuck = slot.pins.iter().filter(|pin| pin.stuck).count() as u32; + if stuck > 0 { + self.apply_event(generation, Event::DropFencedRefs { count: stuck }, None)?; + if let Some(slot) = self.slots.get_mut(&generation) { + slot.pins.retain(|pin| !pin.stuck); + } + } + let state = self + .slots + .get(&generation) + .map(|s| s.life.state) + .unwrap_or(BlockState::Absent); + if matches!(state, BlockState::Quarantined | BlockState::Evicting) { + self.apply_event(generation, Event::ReclaimFenced, None)?; + } + self.outcome_for(generation, generation) + } + + fn maybe_finish(&mut self, generation: u64) -> Result<(), CacheError> { + let ready = self.slots.get(&generation).map(|slot| { + slot.life.state == BlockState::Evicting + && slot.life.refs == 0 + && !slot.life.reuse_blocked + }); + if ready.unwrap_or(false) { + self.apply_event(generation, Event::FinishEvict, None)?; + } + Ok(()) + } + + fn outcome_for(&self, generation: u64, reported: u64) -> Result { + let (state, quarantined) = match self.slots.get(&generation) { + None => (BlockState::Absent, false), + Some(slot) => ( + slot.life.state, + slot.life.state == BlockState::Quarantined || slot.life.reuse_blocked, + ), + }; + Ok(ReleaseOutcome { + generation: reported, + state, + quarantined, + }) + } + + fn apply_event( + &mut self, + generation: u64, + event: Event, + fail: Option, + ) -> Result<(), CacheError> { + { + let slot = self + .slots + .get_mut(&generation) + .ok_or(CacheError::UnknownGeneration)?; + slot.life.apply(event)?; + if slot.life.state == BlockState::Quarantined { + if let Some(kind) = fail { + slot.fail = Some(kind); + } + } + } + self.reconcile(generation) + } + + fn poll(&mut self, now_ms: u64) { + self.expire_idem(now_ms); + let expired: Vec = self + .slots + .iter() + .filter(|(_, slot)| { + matches!(slot.life.state, BlockState::Reserved | BlockState::Writing) + && now_ms >= slot.lease_deadline_ms + }) + .map(|(generation, _)| *generation) + .collect(); + for generation in expired { + let _ = self.apply_event(generation, Event::WriterTimeout, Some(FailKind::Timeout)); + } + let generations: Vec = self.slots.keys().copied().collect(); + for generation in generations { + if let Some(slot) = self.slots.get_mut(&generation) { + let mut newly_stuck = false; + for pin in &mut slot.pins { + if !pin.stuck && now_ms >= pin.deadline_ms { + pin.stuck = true; + newly_stuck = true; + } + } + if newly_stuck { + let _ = slot.life.apply(Event::ReaderLeaseExpire); + } + } + } + } + + fn authorize( + &self, + auth: &AuthContext, + identity: Option<&CacheIdentity>, + ) -> Result<(), CacheError> { + let policy = self + .policies + .get(auth.tenant.as_bytes()) + .ok_or(CacheError::Unauthorized)?; + if !fingerprints_equal(&policy.credential_fingerprint, &auth.credential_fingerprint) { + return Err(CacheError::Unauthorized); + } + if policy.trust_domain != auth.trust_domain { + return Err(CacheError::Unauthorized); + } + if let Some(identity) = identity { + self.same_tenant(auth, identity)?; + } + Ok(()) + } + + fn same_tenant(&self, auth: &AuthContext, identity: &CacheIdentity) -> Result<(), CacheError> { + if identity.tenant != auth.tenant || identity.trust_domain != auth.trust_domain { + return Err(CacheError::Unauthorized); + } + Ok(()) + } + + fn preflight( + &self, + header: &gpucache_protocol::ItemHeader, + now_ms: u64, + ) -> Result<(), CacheError> { + self.check_header(header, now_ms) + } + + fn check_header( + &self, + header: &gpucache_protocol::ItemHeader, + now_ms: u64, + ) -> Result<(), CacheError> { + if header.epoch != self.epoch { + return Err(CacheError::EpochMismatch { + store: self.epoch, + request: header.epoch, + }); + } + ensure_deadline(now_ms, header.deadline_ms) + } + + fn alloc_region(&mut self) -> u64 { + let id = self.next_region; + self.next_region = self.next_region.saturating_add(1); + id + } + + fn index_add(&mut self, physical: Vec, logical: Vec, generation: u64) { + self.by_physical.insert(physical.clone(), generation); + let entries = self.logical.entry(logical).or_default(); + if !entries.contains(&physical) { + entries.push(physical); + } + } + + fn remove_slot(&mut self, generation: u64) { + let Some(slot) = self.slots.remove(&generation) else { + return; + }; + self.by_physical.remove(&slot.physical); + if let Some(entries) = self.logical.get_mut(&slot.logical) { + entries.retain(|key| key != &slot.physical); + if entries.is_empty() { + self.logical.remove(&slot.logical); + } + } + } + + fn reconcile(&mut self, generation: u64) -> Result<(), CacheError> { + let state = self + .slots + .get(&generation) + .ok_or(CacheError::UnknownGeneration)? + .life + .state; + self.apply_charge_delta(generation, charge_target(state))?; + if state == BlockState::Absent { + self.remove_slot(generation); + } + Ok(()) + } + + fn apply_charge_delta( + &mut self, + generation: u64, + target: ChargeTarget, + ) -> Result<(), CacheError> { + let slot = self + .slots + .get(&generation) + .ok_or(CacheError::UnknownGeneration)?; + let tenant = slot.identity.tenant.as_bytes().to_vec(); + let len = slot.length; + let current = slot.charges; + let retained_delta = delta(current.retained, target.retained, len); + let inflight_delta = delta(current.inflight, target.inflight, len); + let request_delta: i32 = match (current.inflight, target.inflight) { + (false, true) => 1, + (true, false) => -1, + _ => 0, + }; + let registration_delta: i32 = match (current.registration, target.registration) { + (false, true) => 1, + (true, false) => -1, + _ => 0, + }; + let new_retained = add_i128(self.retained_bytes, retained_delta)?; + let new_requests = add_i32(self.inflight_requests, request_delta)?; + let new_regs = add_i32(self.registrations, registration_delta)?; + let new_tenant_retained = add_i128( + self.tenant_retained.get(&tenant).copied().unwrap_or(0), + retained_delta, + )?; + let new_tenant_inflight = add_i128( + self.tenant_inflight.get(&tenant).copied().unwrap_or(0), + inflight_delta, + )?; + if target.retained && !current.retained { + if new_retained > self.limits.max_retained_bytes { + return Err(CacheError::QuotaExceeded { + kind: QuotaKind::RetainedBytes, + }); + } + if new_tenant_retained > self.limits.max_retained_bytes_per_tenant { + return Err(CacheError::QuotaExceeded { + kind: QuotaKind::TenantRetainedBytes, + }); + } + } + if target.inflight && !current.inflight { + if new_requests > u64::from(self.limits.max_in_flight_requests) { + return Err(CacheError::QuotaExceeded { + kind: QuotaKind::InFlightRequests, + }); + } + if new_tenant_inflight > self.limits.max_in_flight_bytes_per_tenant { + return Err(CacheError::QuotaExceeded { + kind: QuotaKind::InFlightBytes, + }); + } + } + if target.registration + && !current.registration + && new_regs > u64::from(self.limits.max_registrations) + { + return Err(CacheError::QuotaExceeded { + kind: QuotaKind::Registrations, + }); + } + self.retained_bytes = new_retained; + self.inflight_requests = u32::try_from(new_requests).unwrap_or(u32::MAX); + self.registrations = u32::try_from(new_regs).unwrap_or(u32::MAX); + set_counter( + &mut self.tenant_retained, + tenant.clone(), + new_tenant_retained, + ); + set_counter(&mut self.tenant_inflight, tenant, new_tenant_inflight); + if let Some(slot) = self.slots.get_mut(&generation) { + slot.charges = Charges { + retained: target.retained, + inflight: target.inflight, + registration: target.registration, + }; + } + Ok(()) + } + + fn replay( + &self, + id: RequestId, + fingerprint: &Checksum, + ) -> Result, CacheError> { + match self.idem.get(&id) { + None => Ok(None), + Some(record) if &record.fingerprint == fingerprint => Ok(Some(record.kind.clone())), + Some(_) => Err(CacheError::IdempotencyConflict), + } + } + + fn ensure_idem_room(&mut self, now_ms: u64) -> Result<(), CacheError> { + self.expire_idem(now_ms); + if self.idem.len() >= self.limits.max_idempotency_entries as usize { + return Err(CacheError::QuotaExceeded { + kind: QuotaKind::IdempotencyTable, + }); + } + Ok(()) + } + + fn remember( + &mut self, + id: RequestId, + fingerprint: Checksum, + now_ms: u64, + kind: IdemKind, + ) -> Result<(), CacheError> { + self.idem.insert( + id, + IdemRecord { + fingerprint, + inserted_ms: now_ms, + kind, + }, + ); + Ok(()) + } + + fn expire_idem(&mut self, now_ms: u64) { + let ttl = self.limits.idempotency_ttl_ms; + self.idem + .retain(|_, record| now_ms.saturating_sub(record.inserted_ms) < ttl); + } +} + +fn charge_target(state: BlockState) -> ChargeTarget { + match state { + BlockState::Absent => ChargeTarget { + retained: false, + inflight: false, + registration: false, + }, + BlockState::Reserved => ChargeTarget { + retained: true, + inflight: true, + registration: false, + }, + BlockState::Writing => ChargeTarget { + retained: true, + inflight: true, + registration: true, + }, + BlockState::Committed | BlockState::Evicting | BlockState::Quarantined => ChargeTarget { + retained: true, + inflight: false, + registration: true, + }, + } +} + +fn delta(from: bool, to: bool, len: u64) -> i128 { + match (from, to) { + (false, true) => i128::from(len), + (true, false) => -i128::from(len), + _ => 0, + } +} + +fn add_i128(base: u64, delta_value: i128) -> Result { + let next = i128::from(base) + delta_value; + u64::try_from(next).map_err(|_| CacheError::Malformed { + reason: "accounting underflow", + }) +} + +fn add_i32(base: u32, delta_value: i32) -> Result { + let next = i64::from(base) + i64::from(delta_value); + u64::try_from(next).map_err(|_| CacheError::Malformed { + reason: "accounting underflow", + }) +} + +fn set_counter(map: &mut HashMap, u64>, key: Vec, value: u64) { + if value == 0 { + map.remove(&key); + } else { + map.insert(key, value); + } +} + +fn miss_reason(state: BlockState) -> MissReason { + match state { + BlockState::Absent => MissReason::Absent, + BlockState::Reserved | BlockState::Writing => MissReason::NotCommitted, + BlockState::Committed => MissReason::NotCommitted, + BlockState::Evicting => MissReason::Evicting, + BlockState::Quarantined => MissReason::Quarantined, + } +} + +fn fail_error(fail: Option, state: BlockState) -> CacheError { + match fail { + Some(FailKind::Timeout) => CacheError::TimedOut, + Some(FailKind::Cancelled) => CacheError::Cancelled, + Some(FailKind::Fenced) => CacheError::FencingRejected { + reason: "writer fenced", + }, + Some(FailKind::Corrupt) => CacheError::ChecksumMismatch, + None => CacheError::NotExposed { state }, + } +} + +fn request_ids(ids: I) -> Vec +where + I: Iterator, +{ + ids.collect() +} + +fn fingerprint(parts: &[&[u8]]) -> Checksum { + let mut buf = Vec::new(); + for part in parts { + let len = u32::try_from(part.len()).unwrap_or(u32::MAX); + buf.extend_from_slice(&len.to_le_bytes()); + buf.extend_from_slice(part); + } + content_checksum(&buf) +} + +fn reserve_fingerprint(item: &ReservePutItem) -> Result { + let identity = item.identity.to_bytes()?; + let payload_sum = item + .payload + .as_ref() + .map(|bytes| content_checksum(bytes).0) + .unwrap_or([0u8; 32]); + Ok(fingerprint(&[ + &[2u8][..], + &item.header.epoch.to_le_bytes(), + &item.generation.to_le_bytes(), + &item.length.to_le_bytes(), + &item.checksum.0, + &identity, + &payload_sum, + &item.lease_ttl_ms.to_le_bytes(), + ])) +} + +fn release_fingerprint(item: &ReleaseCancelItem) -> Checksum { + let region = item.region_id.unwrap_or(0).to_le_bytes(); + let pin = item.pin_owner.map(|id| id.0).unwrap_or([0u8; 16]); + let att = match item.attestation { + Attestation::KnownIdle => 1u8, + Attestation::Unknown => 2, + Attestation::Drained => 3, + }; + fingerprint(&[ + &[5u8][..], + &item.header.epoch.to_le_bytes(), + &item.generation.to_le_bytes(), + ®ion, + &pin, + &[att], + ]) +} diff --git a/crates/storage/src/props.rs b/crates/storage/src/props.rs new file mode 100644 index 0000000..129cfed --- /dev/null +++ b/crates/storage/src/props.rs @@ -0,0 +1,176 @@ +// Copyright 2024-2026 RustFS, Inc. +// SPDX-License-Identifier: Apache-2.0 + +use gpucache_core_types::{ + content_checksum, synthetic_identity, CacheError, RequestId, ResourceLimits, +}; +use gpucache_protocol::{ + AuthContext, CommitItem, GetItem, InvalidateItem, ItemHeader, LookupItem, LookupOutcome, + ReservePutItem, TenantPolicy, +}; + +use crate::MemoryStore; + +fn lcg(state: &mut u64) -> u64 { + *state = state.wrapping_mul(6364136223846793005).wrapping_add(1); + *state +} + +#[test] +fn random_operations_never_return_uncommitted_or_foreign_bytes() { + let mut limits = ResourceLimits::dev_defaults(); + limits.max_object_bytes = 32; + limits.max_retained_bytes = 256; + limits.max_retained_bytes_per_tenant = 256; + limits.max_in_flight_bytes_per_tenant = 128; + limits.max_in_flight_requests = 4; + limits.max_registrations = 4; + limits.max_idempotency_entries = 128; + let identity = synthetic_identity(1, 1); + let fp = [9u8; 32]; + let policy = + TenantPolicy::new(identity.tenant.clone(), identity.trust_domain.clone(), fp).unwrap(); + let auth = AuthContext::new(identity.tenant, identity.trust_domain, fp).unwrap(); + let mut store = MemoryStore::new(limits, vec![policy]).unwrap(); + let mut shadow: Vec<(u64, Vec)> = Vec::new(); + let mut seed = 0xdec0_u64; + let mut next_id = 100u128; + for _ in 0..80 { + let roll = lcg(&mut seed) % 10; + let epoch = store.epoch(); + let header = |n: u128| ItemHeader { + request_id: RequestId::from_counter(n).unwrap(), + epoch, + deadline_ms: 10_000, + }; + if roll < 4 { + let len = 1 + (lcg(&mut seed) % 8) as usize; + let mut payload = vec![0u8; len]; + for byte in &mut payload { + *byte = lcg(&mut seed) as u8; + } + let bad = lcg(&mut seed) % 5 == 0; + let checksum = if bad { + content_checksum(b"not-the-payload") + } else { + content_checksum(&payload) + }; + let id_reserve = next_id; + next_id += 1; + let reserved = store.reserve_put( + &auth, + 0, + vec![ReservePutItem { + header: header(id_reserve), + identity: synthetic_identity(1, 1), + generation: 0, + length: payload.len() as u64, + checksum, + payload: Some(payload.clone()), + lease_ttl_ms: 5_000, + }], + ); + if let Ok(reply) = reserved { + if let Ok(outcome) = &reply.items[0].body { + let id_commit = next_id; + next_id += 1; + let committed = store.commit( + &auth, + 0, + vec![CommitItem { + header: header(id_commit), + generation: outcome.generation, + length: payload.len() as u64, + checksum, + }], + ); + if let Ok(done) = committed { + if done.items[0].body.is_ok() && !bad { + shadow.push((outcome.generation, payload)); + } else { + assert!(done.items[0].body.is_err() || bad); + if bad { + assert!(done.items[0].body.is_err()); + } + } + } + } + } + } else if roll == 4 { + if let Some((generation, _)) = shadow.pop() { + let _ = store.invalidate( + &auth, + 0, + vec![InvalidateItem { + header: header(next_id), + generation, + }], + ); + next_id += 1; + } + } else if roll == 5 { + let mut other = synthetic_identity(1, 1); + other.layout.family = 7; + other.shard.tp_size = 2; + let looked = store + .lookup( + &auth, + 0, + vec![LookupItem { + header: header(next_id), + identity: other, + }], + ) + .unwrap(); + next_id += 1; + assert!(!matches!( + looked.items[0].body.as_ref().unwrap(), + LookupOutcome::Hit { .. } + )); + } + for (generation, payload) in &shadow { + let looked = store + .lookup( + &auth, + 0, + vec![LookupItem { + header: header(next_id), + identity: synthetic_identity(1, 1), + }], + ) + .unwrap(); + next_id += 1; + match &looked.items[0].body { + Ok(LookupOutcome::Hit { + checksum, length, .. + }) => { + assert_eq!(*length, payload.len() as u64); + assert_eq!(*checksum, content_checksum(payload)); + } + other => panic!("committed block disappeared: {other:?}"), + } + let got = store + .get( + &auth, + 0, + vec![GetItem { + header: header(next_id), + generation: *generation, + identity: synthetic_identity(1, 1), + pin_ttl_ms: 100, + }], + ) + .unwrap(); + next_id += 1; + match &got.items[0].body { + Ok(hit) => assert_eq!(hit.payload, *payload), + Err(err) => panic!("get of committed block failed: {err}"), + } + } + let stats = store.stats(); + assert!(stats.retained_bytes <= limits.max_retained_bytes); + assert!(stats.registrations <= limits.max_registrations); + assert!(stats.in_flight_requests <= limits.max_in_flight_requests); + } + let _ = CacheError::TimedOut; +} diff --git a/crates/storage/src/tests.rs b/crates/storage/src/tests.rs new file mode 100644 index 0000000..c226ac4 --- /dev/null +++ b/crates/storage/src/tests.rs @@ -0,0 +1,755 @@ +// Copyright 2024-2026 RustFS, Inc. +// SPDX-License-Identifier: Apache-2.0 + +use gpucache_core_types::{ + content_checksum, synthetic_identity, Attestation, BlockState, CacheError, RequestId, + ResourceLimits, TenantId, TrustDomain, +}; +use gpucache_protocol::{ + encode_export, AuthContext, CommitItem, CommittedExport, GetItem, InvalidateItem, ItemHeader, + LookupItem, LookupOutcome, ReleaseCancelItem, ReservePutItem, TenantPolicy, +}; + +use crate::import_committed_block; +use crate::{MemoryStore, ReclaimRequest}; + +const FP: [u8; 32] = [9u8; 32]; + +fn limits() -> ResourceLimits { + ResourceLimits::dev_defaults() +} + +fn open(extra: Option) -> (MemoryStore, AuthContext) { + let id = synthetic_identity(1, 1); + let policy = TenantPolicy::new(id.tenant.clone(), id.trust_domain.clone(), FP).unwrap(); + let auth = AuthContext::new(id.tenant, id.trust_domain, FP).unwrap(); + let mut policies = vec![policy]; + if let Some(extra) = extra { + policies.push(extra); + } + let store = MemoryStore::new(limits(), policies).unwrap(); + (store, auth) +} + +fn hdr(store: &MemoryStore, n: u128, now: u64) -> ItemHeader { + ItemHeader { + request_id: RequestId::from_counter(n).unwrap(), + epoch: store.epoch(), + deadline_ms: now + 1_000, + } +} + +fn put_item(store: &MemoryStore, n: u128, now: u64, bytes: &[u8], family: u16) -> ReservePutItem { + ReservePutItem { + header: hdr(store, n, now), + identity: synthetic_identity(1, family), + generation: 0, + length: bytes.len() as u64, + checksum: content_checksum(bytes), + payload: Some(bytes.to_vec()), + lease_ttl_ms: 500, + } +} + +fn commit_bytes(store: &mut MemoryStore, auth: &AuthContext, n: u128, bytes: &[u8]) -> u64 { + let now = 0; + let reserved = store + .reserve_put(auth, now, vec![put_item(store, n, now, bytes, 1)]) + .unwrap(); + let generation = reserved.items[0].body.as_ref().unwrap().generation; + let committed = store + .commit( + auth, + now, + vec![CommitItem { + header: hdr(store, n + 1, now), + generation, + length: bytes.len() as u64, + checksum: content_checksum(bytes), + }], + ) + .unwrap(); + committed.items[0].body.as_ref().unwrap().generation +} + +#[test] +fn single_process_export_import_roundtrip() { + let (mut store, auth) = open(None); + let payload = b"synthetic-kv-block-not-vllm".to_vec(); + let generation = commit_bytes(&mut store, &auth, 10, &payload); + let hit = store + .get( + &auth, + 0, + vec![GetItem { + header: hdr(&store, 30, 0), + generation, + identity: synthetic_identity(1, 1), + pin_ttl_ms: 100, + }], + ) + .unwrap(); + let got = hit.items[0].body.as_ref().unwrap(); + assert_eq!(got.payload, payload); + let frame = CommittedExport { + identity: synthetic_identity(1, 1), + source_generation: generation, + checksum: content_checksum(&payload), + payload: payload.clone(), + }; + let encoded = encode_export(&frame).unwrap(); + let decoded = gpucache_protocol::decode_export(&encoded).unwrap(); + let report = import_committed_block(&decoded, FP, 0).unwrap(); + assert_eq!(report.length, payload.len() as u64); + assert_eq!(report.checksum_hex, hex(&content_checksum(&payload).0)); + assert_eq!(report.source_generation, generation); + let _ = store.release_cancel( + &auth, + 0, + vec![ReleaseCancelItem { + header: hdr(&store, 31, 0), + generation, + region_id: Some(got.region_id), + pin_owner: Some(RequestId::from_counter(30).unwrap()), + attestation: Attestation::Drained, + }], + ); +} + +#[test] +fn only_committed_compatible_blocks_hit() { + let (mut store, auth) = open(None); + let payload = b"visible-only-after-commit".to_vec(); + let now = 0; + let missing = store + .lookup( + &auth, + now, + vec![LookupItem { + header: hdr(&store, 1, now), + identity: synthetic_identity(1, 1), + }], + ) + .unwrap(); + assert!(matches!( + missing.items[0].body.as_ref().unwrap(), + LookupOutcome::Miss { .. } + )); + let reserved = store + .reserve_put(&auth, now, vec![put_item(&store, 2, now, &payload, 1)]) + .unwrap(); + let generation = reserved.items[0].body.as_ref().unwrap().generation; + let hidden = store + .lookup( + &auth, + now, + vec![LookupItem { + header: hdr(&store, 3, now), + identity: synthetic_identity(1, 1), + }], + ) + .unwrap(); + assert!(matches!( + hidden.items[0].body.as_ref().unwrap(), + LookupOutcome::Miss { + reason: gpucache_protocol::MissReason::NotCommitted + } + )); + let premature = store + .get( + &auth, + now, + vec![GetItem { + header: hdr(&store, 4, now), + generation, + identity: synthetic_identity(1, 1), + pin_ttl_ms: 10, + }], + ) + .unwrap(); + assert!(matches!( + premature.items[0].body, + Err(CacheError::NotExposed { + state: BlockState::Writing + }) + )); + store + .commit( + &auth, + now, + vec![CommitItem { + header: hdr(&store, 5, now), + generation, + length: payload.len() as u64, + checksum: content_checksum(&payload), + }], + ) + .unwrap(); + let found = store + .lookup( + &auth, + now, + vec![LookupItem { + header: hdr(&store, 6, now), + identity: synthetic_identity(1, 1), + }], + ) + .unwrap(); + assert!(matches!( + found.items[0].body.as_ref().unwrap(), + LookupOutcome::Hit { .. } + )); + let mut other = synthetic_identity(1, 1); + other.layout.family = 9; + let incompatible = store + .lookup( + &auth, + now, + vec![LookupItem { + header: hdr(&store, 7, now), + identity: other.clone(), + }], + ) + .unwrap(); + match incompatible.items[0].body.clone().unwrap() { + LookupOutcome::Miss { + reason: gpucache_protocol::MissReason::Incompatible, + } => {} + other => panic!("layout mismatch must miss, got {other:?}"), + } + let wrong_get = store + .get( + &auth, + now, + vec![GetItem { + header: hdr(&store, 8, now), + generation, + identity: other, + pin_ttl_ms: 10, + }], + ) + .unwrap(); + assert!(matches!( + wrong_get.items[0].body, + Err(CacheError::Incompatible) + )); +} + +#[test] +fn checksum_and_generation_failures_do_not_publish_bytes() { + let (mut store, auth) = open(None); + let payload = b"checksum-me".to_vec(); + let now = 0; + let reserved = store + .reserve_put(&auth, now, vec![put_item(&store, 1, now, &payload, 1)]) + .unwrap(); + let generation = reserved.items[0].body.as_ref().unwrap().generation; + let bad = store + .commit( + &auth, + now, + vec![CommitItem { + header: hdr(&store, 2, now), + generation, + length: payload.len() as u64, + checksum: content_checksum(b"other"), + }], + ) + .unwrap(); + assert_eq!(bad.items[0].body, Err(CacheError::ChecksumMismatch)); + assert_eq!( + store.state_of(&auth, generation, now), + Err(CacheError::UnknownGeneration) + ); + assert_eq!(store.stats().committed, 0); + let generation = commit_bytes(&mut store, &auth, 20, &payload); + let wrong_gen = store + .get( + &auth, + now, + vec![GetItem { + header: hdr(&store, 40, now), + generation: generation + 50, + identity: synthetic_identity(1, 1), + pin_ttl_ms: 10, + }], + ) + .unwrap(); + assert_eq!(wrong_gen.items[0].body, Err(CacheError::UnknownGeneration)); +} + +#[test] +fn timeout_quarantines_a_registered_region_until_fence() { + let (mut store, auth) = open(None); + let payload = b"inflight".to_vec(); + let reserved = store + .reserve_put(&auth, 0, vec![put_item(&store, 1, 0, &payload, 1)]) + .unwrap(); + let outcome = reserved.items[0].body.as_ref().unwrap(); + let generation = outcome.generation; + let region = outcome.region_id.unwrap(); + assert_eq!( + store.state_of(&auth, generation, 0).unwrap(), + BlockState::Writing + ); + assert_eq!( + store.state_of(&auth, generation, 500).unwrap(), + BlockState::Quarantined + ); + assert!(store.stats().retained_bytes >= payload.len() as u64); + assert_eq!(store.stats().registrations, 1); + let hidden = store + .get( + &auth, + 500, + vec![GetItem { + header: hdr(&store, 2, 500), + generation, + identity: synthetic_identity(1, 1), + pin_ttl_ms: 10, + }], + ) + .unwrap(); + assert_eq!(hidden.items[0].body, Err(CacheError::TimedOut)); + let early = store.reclaim_fenced( + &auth, + 500, + ReclaimRequest { + request_id: RequestId::from_counter(3).unwrap(), + epoch: store.epoch(), + deadline_ms: 1_500, + generation, + observed_epoch: store.epoch(), + }, + ); + assert!(matches!(early, Err(CacheError::FencingRejected { .. }))); + let epoch = store.bump_epoch(500); + let reclaimed = store + .reclaim_fenced( + &auth, + 500, + ReclaimRequest { + request_id: RequestId::from_counter(4).unwrap(), + epoch, + deadline_ms: 1_500, + generation, + observed_epoch: epoch, + }, + ) + .unwrap(); + assert_eq!(reclaimed.state, BlockState::Absent); + assert_eq!(store.stats().retained_bytes, 0); + assert_eq!(store.stats().registrations, 0); + let again = store + .reserve_put(&auth, 500, vec![put_item(&store, 5, 500, &payload, 1)]) + .unwrap(); + let new_region = again.items[0].body.as_ref().unwrap().region_id.unwrap(); + assert_ne!(new_region, region); +} + +#[test] +fn cancel_unknown_quarantines_and_drained_reclaims() { + let (mut store, auth) = open(None); + let payload = b"cancel-me".to_vec(); + let generation = store + .reserve_put(&auth, 0, vec![put_item(&store, 1, 0, &payload, 1)]) + .unwrap() + .items[0] + .body + .as_ref() + .unwrap() + .generation; + let cancelled = store + .release_cancel( + &auth, + 0, + vec![ReleaseCancelItem { + header: hdr(&store, 2, 0), + generation, + region_id: None, + pin_owner: None, + attestation: Attestation::Unknown, + }], + ) + .unwrap(); + assert_eq!( + cancelled.items[0].body.as_ref().unwrap().state, + BlockState::Quarantined + ); + let late = store + .commit( + &auth, + 0, + vec![CommitItem { + header: hdr(&store, 3, 0), + generation, + length: payload.len() as u64, + checksum: content_checksum(&payload), + }], + ) + .unwrap(); + assert_eq!(late.items[0].body, Err(CacheError::Cancelled)); + let (mut store, auth) = open(None); + let generation = store + .reserve_put(&auth, 0, vec![put_item(&store, 1, 0, &payload, 1)]) + .unwrap() + .items[0] + .body + .as_ref() + .unwrap() + .generation; + let drained = store + .release_cancel( + &auth, + 0, + vec![ReleaseCancelItem { + header: hdr(&store, 2, 0), + generation, + region_id: None, + pin_owner: None, + attestation: Attestation::Drained, + }], + ) + .unwrap(); + assert_eq!( + drained.items[0].body.as_ref().unwrap().state, + BlockState::Absent + ); + assert_eq!(store.stats().retained_bytes, 0); +} + +#[test] +fn unregistered_reservation_timeout_frees_immediately() { + let (mut store, auth) = open(None); + let item = ReservePutItem { + payload: None, + ..put_item(&store, 1, 0, b"reserved-only", 1) + }; + let generation = store.reserve_put(&auth, 0, vec![item]).unwrap().items[0] + .body + .as_ref() + .unwrap() + .generation; + assert_eq!( + store.state_of(&auth, generation, 0).unwrap(), + BlockState::Reserved + ); + assert_eq!(store.stats().registrations, 0); + assert_eq!( + store.state_of(&auth, generation, 500).unwrap_err(), + CacheError::UnknownGeneration + ); + assert_eq!(store.stats().retained_bytes, 0); +} + +#[test] +fn invalidate_hides_bytes_and_waits_for_reader_pins() { + let (mut store, auth) = open(None); + let payload = b"pinned".to_vec(); + let generation = commit_bytes(&mut store, &auth, 1, &payload); + let hit = store + .get( + &auth, + 0, + vec![GetItem { + header: hdr(&store, 10, 0), + generation, + identity: synthetic_identity(1, 1), + pin_ttl_ms: 50, + }], + ) + .unwrap(); + let region = hit.items[0].body.as_ref().unwrap().region_id; + let invalidated = store + .invalidate( + &auth, + 0, + vec![InvalidateItem { + header: hdr(&store, 11, 0), + generation, + }], + ) + .unwrap(); + assert_eq!( + invalidated.items[0].body.as_ref().unwrap().state, + BlockState::Evicting + ); + let hidden = store + .get( + &auth, + 0, + vec![GetItem { + header: hdr(&store, 12, 0), + generation, + identity: synthetic_identity(1, 1), + pin_ttl_ms: 10, + }], + ) + .unwrap(); + assert!(hidden.items[0].body.is_err()); + assert!(store.stats().retained_bytes >= payload.len() as u64); + let released = store + .release_cancel( + &auth, + 0, + vec![ReleaseCancelItem { + header: hdr(&store, 13, 0), + generation, + region_id: Some(region), + pin_owner: Some(RequestId::from_counter(10).unwrap()), + attestation: Attestation::Drained, + }], + ) + .unwrap(); + assert_eq!( + released.items[0].body.as_ref().unwrap().state, + BlockState::Absent + ); + assert_eq!(store.stats().retained_bytes, 0); +} + +#[test] +fn reader_lease_expiry_does_not_recycle_the_buffer() { + let (mut store, auth) = open(None); + let payload = b"lease".to_vec(); + let generation = commit_bytes(&mut store, &auth, 1, &payload); + let hit = store + .get( + &auth, + 0, + vec![GetItem { + header: hdr(&store, 8, 0), + generation, + identity: synthetic_identity(1, 1), + pin_ttl_ms: 10, + }], + ) + .unwrap(); + let region = hit.items[0].body.as_ref().unwrap().region_id; + store + .invalidate( + &auth, + 20, + vec![InvalidateItem { + header: hdr(&store, 9, 20), + generation, + }], + ) + .unwrap(); + assert_eq!( + store.state_of(&auth, generation, 20).unwrap(), + BlockState::Evicting + ); + assert!(store.stats().registrations >= 1); + let epoch = store.bump_epoch(20); + let reclaimed = store + .reclaim_fenced( + &auth, + 20, + ReclaimRequest { + request_id: RequestId::from_counter(15).unwrap(), + epoch, + deadline_ms: 2_000, + generation, + observed_epoch: epoch, + }, + ) + .unwrap(); + assert_eq!(reclaimed.state, BlockState::Absent); + let _ = region; +} + +#[test] +fn quotas_reject_without_growing_past_the_ceiling() { + let mut tight = limits(); + tight.max_object_bytes = 4; + tight.max_in_flight_requests = 1; + tight.max_registrations = 1; + tight.max_retained_bytes = 8; + tight.max_retained_bytes_per_tenant = 8; + tight.max_in_flight_bytes_per_tenant = 8; + let id = synthetic_identity(1, 1); + let policy = TenantPolicy::new(id.tenant.clone(), id.trust_domain.clone(), FP).unwrap(); + let auth = AuthContext::new(id.tenant, id.trust_domain, FP).unwrap(); + let mut store = MemoryStore::new(tight, vec![policy]).unwrap(); + let too_big = store.reserve_put(&auth, 0, vec![put_item(&store, 1, 0, b"12345", 1)]); + assert!(matches!( + too_big.unwrap().items[0].body, + Err(CacheError::ObjectTooLarge { .. }) + )); + let first = store + .reserve_put(&auth, 0, vec![put_item(&store, 2, 0, b"abcd", 1)]) + .unwrap(); + assert!(first.items[0].body.is_ok()); + let second = store + .reserve_put(&auth, 0, vec![put_item(&store, 3, 0, b"wxyz", 2)]) + .unwrap(); + assert!(matches!( + second.items[0].body, + Err(CacheError::QuotaExceeded { .. }) + )); + assert!(store.stats().retained_bytes <= 8); + assert!(store.stats().in_flight_requests <= 1); + assert!(store.stats().registrations <= 1); +} + +#[test] +fn idempotent_replay_and_conflict() { + let (mut store, auth) = open(None); + let payload = b"once".to_vec(); + let item = put_item(&store, 7, 0, &payload, 1); + let first = store.reserve_put(&auth, 0, vec![item.clone()]).unwrap(); + let second = store.reserve_put(&auth, 0, vec![item.clone()]).unwrap(); + assert_eq!(first.items[0].body, second.items[0].body); + assert_eq!(store.stats().in_flight_requests, 1); + let mut changed = item.clone(); + changed.length = 3; + changed.payload = Some(b"no!".to_vec()); + changed.checksum = content_checksum(b"no!"); + let conflict = store.reserve_put(&auth, 0, vec![changed]).unwrap(); + assert_eq!(conflict.items[0].body, Err(CacheError::IdempotencyConflict)); +} + +#[test] +fn auth_fails_closed_and_blocks_cross_tenant_reads() { + let (store, _) = { + let store = MemoryStore::new(limits(), vec![]).unwrap(); + let id = synthetic_identity(1, 1); + let auth = AuthContext::new(id.tenant, id.trust_domain, FP).unwrap(); + (store, auth) + }; + let id = synthetic_identity(1, 1); + let auth = AuthContext::new(id.tenant.clone(), id.trust_domain.clone(), FP).unwrap(); + let mut store = store; + let denied = store.lookup( + &auth, + 0, + vec![LookupItem { + header: hdr(&store, 1, 0), + identity: id.clone(), + }], + ); + assert_eq!(denied, Err(CacheError::Unauthorized)); + + let other_fp = [4u8; 32]; + let other_policy = TenantPolicy::new( + TenantId::new(b"tenant-b".to_vec()).unwrap(), + TrustDomain::new(b"td-local".to_vec()).unwrap(), + other_fp, + ) + .unwrap(); + let (mut store, auth_a) = open(Some(other_policy)); + let generation = commit_bytes(&mut store, &auth_a, 1, b"secret-bytes"); + let auth_b = AuthContext::new( + TenantId::new(b"tenant-b".to_vec()).unwrap(), + TrustDomain::new(b"td-local".to_vec()).unwrap(), + other_fp, + ) + .unwrap(); + let stolen = store + .get( + &auth_b, + 0, + vec![GetItem { + header: hdr(&store, 9, 0), + generation, + identity: synthetic_identity(1, 1), + pin_ttl_ms: 10, + }], + ) + .unwrap(); + assert_eq!(stolen.items[0].body, Err(CacheError::Unauthorized)); + assert_eq!(store.stats().committed, 1); +} + +#[test] +fn batch_is_item_scoped_and_duplicate_ids_fail_closed() { + let (mut store, auth) = open(None); + let ok = put_item(&store, 1, 0, b"ok-block", 1); + let mut huge = put_item(&store, 2, 0, b"x", 2); + huge.length = limits().max_object_bytes + 1; + huge.payload = None; + let mixed = store.reserve_put(&auth, 0, vec![ok, huge]).unwrap(); + assert!(mixed.items[0].body.is_ok()); + assert!(matches!( + mixed.items[1].body, + Err(CacheError::ObjectTooLarge { .. }) + )); + let dup = put_item(&store, 3, 0, b"dup", 3); + let denied = store.reserve_put(&auth, 0, vec![dup.clone(), dup]); + assert_eq!( + denied, + Err(CacheError::Malformed { + reason: "duplicate request_id" + }) + ); +} + +#[test] +fn protocol_frame_drives_the_store() { + let (mut store, auth) = open(None); + let payload = b"from-wire".to_vec(); + let item = put_item(&store, 1, 0, &payload, 1); + let encoded = gpucache_protocol::encode_request( + &auth, + &gpucache_protocol::RequestBody::ReservePut(vec![item]), + ) + .unwrap(); + let (wire_auth, body) = gpucache_protocol::decode_request(&encoded).unwrap(); + let gpucache_protocol::RequestBody::ReservePut(items) = body else { + panic!("op"); + }; + let reserved = store.reserve_put(&wire_auth, 0, items).unwrap(); + let generation = reserved.items[0].body.as_ref().unwrap().generation; + store + .commit( + &auth, + 0, + vec![CommitItem { + header: hdr(&store, 2, 0), + generation, + length: payload.len() as u64, + checksum: content_checksum(&payload), + }], + ) + .unwrap(); + let looked = store + .lookup( + &auth, + 0, + vec![LookupItem { + header: hdr(&store, 3, 0), + identity: synthetic_identity(1, 1), + }], + ) + .unwrap(); + match looked.items[0].body.clone().unwrap() { + LookupOutcome::Hit { + length, checksum, .. + } => { + assert_eq!(length, payload.len() as u64); + assert_eq!(checksum, content_checksum(&payload)); + } + other => panic!("expected hit, got {other:?}"), + } +} + +#[test] +fn request_deadline_is_enforced() { + let (mut store, auth) = open(None); + let mut item = put_item(&store, 1, 0, b"late", 1); + item.header.deadline_ms = 5; + let timed = store.reserve_put(&auth, 5, vec![item]).unwrap(); + assert_eq!(timed.items[0].body, Err(CacheError::TimedOut)); + assert_eq!(store.stats().retained_bytes, 0); +} + +fn hex(bytes: &[u8]) -> String { + const DIGITS: &[u8; 16] = b"0123456789abcdef"; + let mut out = String::with_capacity(bytes.len() * 2); + for byte in bytes { + out.push(DIGITS[(byte >> 4) as usize] as char); + out.push(DIGITS[(byte & 0x0f) as usize] as char); + } + out +} diff --git a/crates/storage/src/transfer.rs b/crates/storage/src/transfer.rs new file mode 100644 index 0000000..cb83acb --- /dev/null +++ b/crates/storage/src/transfer.rs @@ -0,0 +1,174 @@ +// Copyright 2024-2026 RustFS, Inc. +// SPDX-License-Identifier: Apache-2.0 + +use gpucache_core_types::{ + content_checksum, encode_logical_key, encode_physical_key, verify_checksum, CacheError, + IdentityLimits, RequestId, ResourceLimits, +}; +use gpucache_protocol::{ + Attestation, AuthContext, CommitItem, CommittedExport, GetItem, ItemHeader, LookupItem, + LookupOutcome, ReleaseCancelItem, ReservePutItem, TenantPolicy, +}; + +use crate::memory::MemoryStore; + +/// Result of importing one synthetic or engine-produced block into a fresh store. +/// +/// V1 A tests use synthetic bytes. A real vLLM export is `NOT_RUN_HARDWARE`. +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct ImportReport { + pub physical_key_hex: String, + pub logical_key_hex: String, + pub checksum_hex: String, + pub length: u64, + pub source_generation: u64, + pub local_generation: u64, +} + +pub fn import_committed_block( + frame: &CommittedExport, + credential_fingerprint: [u8; 32], + now_ms: u64, +) -> Result { + verify_checksum(&frame.payload, frame.checksum)?; + if content_checksum(&frame.payload) != frame.checksum { + return Err(CacheError::ChecksumMismatch); + } + let limits = ResourceLimits::dev_defaults(); + let identity_limits = IdentityLimits::from_resources(&limits); + frame.identity.validate(&identity_limits)?; + let policy = TenantPolicy::new( + frame.identity.tenant.clone(), + frame.identity.trust_domain.clone(), + credential_fingerprint, + )?; + let auth = AuthContext::new( + frame.identity.tenant.clone(), + frame.identity.trust_domain.clone(), + credential_fingerprint, + )?; + let mut store = MemoryStore::new(limits, vec![policy])?; + let epoch = store.epoch(); + let deadline = now_ms.saturating_add(60_000); + let reserved = store.reserve_put( + &auth, + now_ms, + vec![ReservePutItem { + header: ItemHeader { + request_id: RequestId::from_counter(1)?, + epoch, + deadline_ms: deadline, + }, + identity: frame.identity.clone(), + generation: 0, + length: frame.payload.len() as u64, + checksum: frame.checksum, + payload: Some(frame.payload.clone()), + lease_ttl_ms: 30_000, + }], + )?; + let reserved = one(reserved.items)?; + let committed = store.commit( + &auth, + now_ms, + vec![CommitItem { + header: ItemHeader { + request_id: RequestId::from_counter(2)?, + epoch: store.epoch(), + deadline_ms: deadline, + }, + generation: reserved.generation, + length: frame.payload.len() as u64, + checksum: frame.checksum, + }], + )?; + let committed = one(committed.items)?; + let looked = store.lookup( + &auth, + now_ms, + vec![LookupItem { + header: ItemHeader { + request_id: RequestId::from_counter(3)?, + epoch: store.epoch(), + deadline_ms: deadline, + }, + identity: frame.identity.clone(), + }], + )?; + match one(looked.items)? { + LookupOutcome::Hit { checksum, .. } if checksum == frame.checksum => {} + LookupOutcome::Hit { .. } => return Err(CacheError::ChecksumMismatch), + LookupOutcome::Miss { .. } => { + return Err(CacheError::NotExposed { + state: gpucache_core_types::BlockState::Absent, + }) + } + } + let got = store.get( + &auth, + now_ms, + vec![GetItem { + header: ItemHeader { + request_id: RequestId::from_counter(4)?, + epoch: store.epoch(), + deadline_ms: deadline, + }, + generation: committed.generation, + identity: frame.identity.clone(), + pin_ttl_ms: 1_000, + }], + )?; + let hit = one(got.items)?; + if hit.payload != frame.payload { + return Err(CacheError::ChecksumMismatch); + } + let _ = store.release_cancel( + &auth, + now_ms, + vec![ReleaseCancelItem { + header: ItemHeader { + request_id: RequestId::from_counter(5)?, + epoch: store.epoch(), + deadline_ms: deadline, + }, + generation: committed.generation, + region_id: Some(hit.region_id), + pin_owner: Some(RequestId::from_counter(4)?), + attestation: Attestation::Drained, + }], + )?; + let physical = encode_physical_key(&frame.identity, &identity_limits)?; + let logical = encode_logical_key(&frame.identity, &identity_limits)?; + Ok(ImportReport { + physical_key_hex: hex(&physical), + logical_key_hex: hex(&logical), + checksum_hex: hex(&frame.checksum.0), + length: frame.payload.len() as u64, + source_generation: frame.source_generation, + local_generation: committed.generation, + }) +} + +fn one(mut items: Vec>) -> Result { + if items.len() != 1 { + return Err(CacheError::Malformed { + reason: "import batch", + }); + } + match items.pop() { + Some(item) => item.body, + None => Err(CacheError::Malformed { + reason: "import batch", + }), + } +} + +fn hex(bytes: &[u8]) -> String { + const DIGITS: &[u8; 16] = b"0123456789abcdef"; + let mut out = String::with_capacity(bytes.len() * 2); + for byte in bytes { + out.push(DIGITS[(byte >> 4) as usize] as char); + out.push(DIGITS[(byte & 0x0f) as usize] as char); + } + out +} diff --git a/crates/storage/tests/cross_process.rs b/crates/storage/tests/cross_process.rs new file mode 100644 index 0000000..6b10217 --- /dev/null +++ b/crates/storage/tests/cross_process.rs @@ -0,0 +1,105 @@ +// Copyright 2024-2026 RustFS, Inc. +// SPDX-License-Identifier: Apache-2.0 + +use std::io::Write; +use std::process::{Command, Stdio}; + +use gpucache_core_types::{ + content_checksum, encode_physical_key, synthetic_identity, IdentityLimits, +}; +use gpucache_protocol::{encode_export, CommittedExport}; + +const FP_HEX: &str = "0909090909090909090909090909090909090909090909090909090909090909"; + +#[test] +fn cross_process_export_import_roundtrip() { + let payload = b"cross-process-synthetic-kv".to_vec(); + let identity = synthetic_identity(1, 1); + let frame = encode_export(&CommittedExport { + identity: identity.clone(), + source_generation: 11, + checksum: content_checksum(&payload), + payload: payload.clone(), + }) + .unwrap(); + let physical = encode_physical_key( + &identity, + &IdentityLimits { + max_field_bytes: 128, + max_prefix_links: 1024, + max_block_tokens: 1_048_576, + }, + ) + .unwrap(); + let output = run_import(&frame); + assert!( + output.status.success(), + "stderr={}", + String::from_utf8_lossy(&output.stderr) + ); + let stdout = String::from_utf8(output.stdout).unwrap(); + assert!(stdout.contains("status=IMPORTED")); + assert!(stdout.contains("state=COMMITTED")); + assert!(stdout.contains(&format!("length={}", payload.len()))); + assert!(stdout.contains(&format!("checksum={}", hex(&content_checksum(&payload).0)))); + assert!(stdout.contains(&format!("physical_key={}", hex(&physical)))); + assert!(stdout.contains("source_generation=11")); +} + +#[test] +fn cross_process_corrupt_frame_is_rejected() { + let payload = b"cross-process-synthetic-kv".to_vec(); + let mut frame = encode_export(&CommittedExport { + identity: synthetic_identity(1, 1), + source_generation: 11, + checksum: content_checksum(&payload), + payload, + }) + .unwrap(); + let last = frame.len() - 1; + frame[last] ^= 0x5a; + let output = run_import(&frame); + assert!(!output.status.success()); + let stderr = String::from_utf8(output.stderr).unwrap(); + assert!(stderr.contains("checksum"), "stderr={stderr}"); +} + +fn xfer_bin() -> String { + std::env::var("CARGO_BIN_EXE_kv_block_xfer").unwrap_or_else(|_| { + let mut path = std::path::PathBuf::from(env!("CARGO_MANIFEST_DIR")); + path.pop(); + path.pop(); + path.push("target"); + path.push("debug"); + path.push("kv-block-xfer"); + path.to_string_lossy().into_owned() + }) +} + +fn run_import(frame: &[u8]) -> std::process::Output { + let mut child = Command::new(xfer_bin()) + .arg("import") + .arg(FP_HEX) + .stdin(Stdio::piped()) + .stdout(Stdio::piped()) + .stderr(Stdio::piped()) + .spawn() + .expect("spawn kv-block-xfer"); + child + .stdin + .as_mut() + .expect("stdin") + .write_all(frame) + .expect("write frame"); + child.wait_with_output().expect("wait") +} + +fn hex(bytes: &[u8]) -> String { + const DIGITS: &[u8; 16] = b"0123456789abcdef"; + let mut out = String::with_capacity(bytes.len() * 2); + for byte in bytes { + out.push(DIGITS[(byte >> 4) as usize] as char); + out.push(DIGITS[(byte & 0x0f) as usize] as char); + } + out +} diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md new file mode 100644 index 0000000..7696687 --- /dev/null +++ b/docs/ARCHITECTURE.md @@ -0,0 +1,52 @@ +# Architecture (V1 A) + +GPUCache is a shared KV data tier. The hot tier is G3.5. RustFS is an optional colder G4 capacity tier and is not implemented in this milestone. The inference engine still owns tokenization, attention, and scheduling. This repo does not replace them. + +```text +engine (not in tree; spike NOT_RUN_HARDWARE) + | + | CacheIdentity + batch API + v +gpucache-protocol + | + v +gpucache-storage::MemoryStore + | + v +gpucache-core-types (identity, keys, lifecycle, errors) +``` + +Planned crates that are **not** in V1 A: `metadata` as a standalone service, `transport` (TCP), file/NVMe store, RustFS object store, engine adapter, DPU agent, bench harness, metrics. Their contracts are reserved so later milestones do not push GPU or DPU types into `core-types`. + +## Identity + +`CacheIdentity` is version 1. It carries tenant and trust domain, model revision, model-config digest, tokenizer revision, adapter/LoRA identity, KV dtype, quant format, layout ABI, TP/PP shard, position encoding, RoPE parameters (IEEE-754 bits), a token-prefix **hash** chain, block token capacity, valid length, and an optional multimodal processor/input digest. + +Raw prompts, token ids, and media bytes are not fields. Callers hash those outside the store. + +Keys: + +- Logical prefix key (`GPCL`): tenant, model, tokenizer, adapter, prefix hashes, multimodal digests. Layout and shard are absent on purpose. +- Physical byte key (`GPCK`): the logical fields plus dtype, quant, layout, shard, RoPE, and lengths. + +Both encodings are versioned and length-delimited. V1 hits only when the physical key matches a `COMMITTED` block. A different physical key with the same logical key is an incompatible miss. Partial-prefix reuse is not implemented; a shorter chain is a different key. + +## Lifecycle + +```text +ABSENT -> RESERVED -> WRITING -> COMMITTED -> EVICTING -> ABSENT + \ \-> QUARANTINED -> ABSENT + \-> ABSENT (no buffer was registered) +``` + +`COMMITTED` is the only readable state, and only when the stored checksum still matches the bytes. `Get` of anything else returns an error and no payload. `Lookup` returns a miss reason and no payload. + +`QUARANTINED` means a registered region might still be in use. Timeout, lease expiry, and `KnownIdle` do not free it. Reclaim requires either `Attestation::Drained` from an authorized tenant caller, or `bump_epoch` plus `reclaim_fenced` with the new epoch. Region ids are monotonic and are not recycled. + +## Quotas + +`ResourceLimits` bounds object size, in-flight requests, registrations (including committed and quarantined buffers), per-tenant in-flight bytes, global retained bytes, and per-tenant retained bytes. The idempotency table is capped and entries expire. Over-capacity admission fails; V1 A does not silently evict a committed block to make room. `Invalidate` is the explicit eviction path. + +## Process model + +`MemoryStore` is single-threaded and in-process. The cross-process test exports a committed frame and runs `kv-block-xfer` in a child, which imports through the same public API. That is a wire-format check, not a network service. TCP is V1 B. diff --git a/docs/BENCHMARKS.md b/docs/BENCHMARKS.md new file mode 100644 index 0000000..f96bae9 --- /dev/null +++ b/docs/BENCHMARKS.md @@ -0,0 +1,14 @@ +# Benchmarks + +No performance run was executed for V1 A. + +| Comparison the V1 exit criteria ask for | Result | +| --- | --- | +| Engine-native prefix cache | `NOT_RUN_HARDWARE` | +| Same-box local offload | `NOT_RUN_HARDWARE` | +| GPUCache CPU/TCP | `NOT_RUN` (TCP service is V1 B; no tokens were served) | +| Cold / warm / no reuse / multi-turn / HBM cliff | `NOT_RUN_HARDWARE` | + +Memcpy or a synthetic store bandwidth number is not token goodput and is not reported. There is no `FAIL_PERFORMANCE` figure because there is no paired measurement to fail. The CPU contract tests in `cargo test --workspace --all-targets` check correctness only. + +Do not add a benchmark that prints a success banner without raw samples, a manifest, and the hardware it ran on. diff --git a/docs/OPERATIONS.md b/docs/OPERATIONS.md new file mode 100644 index 0000000..25b2e45 --- /dev/null +++ b/docs/OPERATIONS.md @@ -0,0 +1,25 @@ +# Operations (V1 A) + +V1 A is a library plus one import helper. There is no daemon, no package, and no production config. + +## Reproduce the CPU tests + +```bash +cargo test --workspace --all-targets +``` + +Rust stable is selected by `rust-toolchain.toml`. The declared MSRV is 1.85. CI also runs `cargo fmt --all -- --check` and `cargo clippy --workspace --all-targets -- -D warnings`. + +## Import one frame + +`kv-block-xfer` is built by the test suite. It reads a `GEXP` frame on stdin: + +```bash +cargo run -p gpucache-storage --bin kv-block-xfer -- import <64-hex-char fingerprint> +``` + +The fingerprint is the tenant credential hash, not a prompt. The tool creates a one-tenant `MemoryStore`, commits the block, and prints `status=IMPORTED` only after lookup and get succeed. A checksum failure exits 2 and prints `error=` on stderr. This path uses synthetic or caller-supplied bytes. It does not contact vLLM. + +## Not operated here + +No NVMe format, no DPU mode change, no firmware, no IOMMU or BIOS change, no production network change, no RustFS bucket. Those need a separate authorization. There is no website deploy and no crates.io publish in this milestone. diff --git a/docs/PROTOCOL.md b/docs/PROTOCOL.md new file mode 100644 index 0000000..02e69c6 --- /dev/null +++ b/docs/PROTOCOL.md @@ -0,0 +1,53 @@ +# Protocol (wire version 1) + +V1 A defines the messages. It does not open a socket. A later TCP listener must speak this framing unchanged. + +## Framing + +Multi-byte integers are little-endian. Byte strings are `u32` length plus bytes. Decoders reject truncated buffers, trailing bytes, unknown versions, and unknown TLV tags. Declared lengths above 64 MiB are rejected before allocation. Store quotas are tighter than that cap. + +Request frame: + +| Field | Contents | +| --- | --- | +| magic | `GCK1` | +| version | `u16` = 1 | +| op | `u16` (1 lookup, 2 reserve/put, 3 commit, 4 get, 5 release/cancel, 6 invalidate) | +| auth | tenant, trust domain, 32-byte credential fingerprint | +| items | `u32` count, then items | + +Hello (`op` 7) carries the versions the peer supports and no tenant data. HelloAck (`op` 8) returns the selected version and the store epoch. `negotiate` picks the highest version present on both sides and errors on an empty overlap. + +Export frame (`GEXP`, version 1) carries the identity body, source generation, SHA-256 checksum, and payload. Decode recomputes the checksum and fails closed on mismatch. The credential is not in the frame; the importer supplies it out of band. + +## Item headers + +Every item has: + +- `request_id` — 16 bytes, not all zeros +- `epoch` — must equal the store epoch +- `deadline_ms` — absolute monotonic deadline; `0` is malformed; `now >= deadline` is `TimedOut` + +Mutating ops are idempotent inside a bounded window. The same `request_id` and the same canonical arguments replay the stored result. The same id with different arguments is `IdempotencyConflict`. Lookup and Get are not logged; they are safe to repeat. A deadline miss before any mutation is not logged, so the caller can retry that id with a later deadline. + +## Operations + +**Lookup.** Identity in, hit metadata or a miss out. No payload. Miss reasons include absent, not committed, incompatible, evicting, and quarantined. + +**Reserve/Put.** `generation == 0` creates a reservation and, when a payload is present, moves it to `WRITING`. A non-zero generation continues an existing `RESERVED` slot. Checksum is declared here and checked at commit. Bytes are not readable yet. + +**Commit.** Generation, length, and checksum must match the stored write. The checksum is recomputed. Mismatch deletes the write and returns an error. Success moves the block to `COMMITTED`. + +**Get.** Returns payload only for a visible committed block whose identity matches and whose bytes still hash to the stored checksum. A bad checksum quarantines the region and returns no bytes. A successful get pins the region until release or a fenced drop. + +**Release/Cancel.** With no pin owner, cancels an in-flight reserve/write. `Drained` reclaims. `Unknown` and `KnownIdle` quarantine a registered region. With a pin owner, releases that reader pin under the same rules. + +**Invalidate.** `COMMITTED` becomes `EVICTING` and is immediately unreadable. The buffer is freed only after reader pins drain. If a pin's completion is unknown, freeing waits for a newer epoch. + +## Deadlines and leases + +Request deadlines abort the call. Writer leases move `RESERVED` with no buffer back to `ABSENT`, and move `WRITING` to `QUARANTINED`. A reader lease expiry keeps the pin and blocks buffer reuse. It does not, by itself, hide a checksum-valid committed block from a new reader, and it does not free memory. + +## Compatibility + +A hit requires the physical key and a `COMMITTED` block. Logical-key overlap with a different layout, dtype, quant, or shard is a miss. V1 A does not assemble a partial prefix. Engines that need a shorter hit must ask again at V1 B, which must either implement a safe partial-prefix rule or keep whole-block hits. diff --git a/docs/REUSE_AUDIT.md b/docs/REUSE_AUDIT.md new file mode 100644 index 0000000..e5d864e --- /dev/null +++ b/docs/REUSE_AUDIT.md @@ -0,0 +1,13 @@ +# Reuse audit (V1 A) + +The execution spec asks for a look at LMCache, Mooncake, Dynamo KVBM, NIXL, OpenLake, ContextStore, and Fluxon before adding dependencies. V1 A adds none of them. + +| Candidate | Why it is not in the tree | +| --- | --- | +| LMCache | Plausible vLLM bridge. Route not selected; GPU spike `NOT_RUN_HARDWARE`. | +| Mooncake | RDMA / KV transport. Out of scope until V2, and only after V1 correctness. | +| Dynamo KVBM | Another KV manager. Not required to define the CPU contract. | +| NIXL | Candidate V2 transport. No Rust binding is vendored. Ordinary NIC RDMA is not a DPU result. | +| OpenLake, ContextStore, Fluxon | Not required for identity, protocol, or `MemoryStore`. No version is pinned because no code is imported. | + +This is a scope decision, not a line-by-line license audit of those upstream trees. Nothing from them was copied. Direct dependencies are only `sha2` (runtime) and `proptest` (tests). See [THIRD_PARTY_NOTICES.md](../THIRD_PARTY_NOTICES.md). diff --git a/docs/RISKS.md b/docs/RISKS.md new file mode 100644 index 0000000..abd1e04 --- /dev/null +++ b/docs/RISKS.md @@ -0,0 +1,16 @@ +# Risks (V1 A) + +| Risk | Status | Note | +| --- | --- | --- | +| Wrong-tenant or wrong-layout hit | Mitigated in `MemoryStore` tests | Physical key includes tenant and layout. Incompatible lookup returns a miss and no bytes. | +| Reclaim of a buffer that might still be copied | Mitigated by quarantine | Timeout and `KnownIdle` do not free a registered region. | +| Unbounded memory | Mitigated by quotas | Admission fails instead of evicting. Quarantine still holds bytes until fence or drain, which is intentional and counted against the quota. | +| Idempotency window expiry | Accepted | After TTL, a request id may run again. Callers must not reuse ids they still care about. | +| `Drained` is only tenant-authenticated | Accepted for V1 A | Not a device proof. Documented in SECURITY.md. | +| No real engine bytes | `NOT_RUN_HARDWARE` | Synthetic frames can pass while a real vLLM layout still mismatches. | +| No TCP, NVMe, or RustFS | Not started | V1 B. A mock must not be described as a measured RustFS run. | +| No RDMA / DPU measurement | `NOT_RUN_HARDWARE` | Do not infer a speedup from this milestone. | +| License of a future DOCA or model stack | `NOT_RUN_LICENSE` | No DOCA binary and no model weights are in the tree. | +| GPU rental or DPU purchase | `NOT_RUN_BUDGET` | No paid API, GPU, or DPU was used. | + +Minimum reproduction of the CPU contract is `cargo test --workspace --all-targets`. There is no hardware failure log because the hardware was not present. diff --git a/docs/SECURITY.md b/docs/SECURITY.md new file mode 100644 index 0000000..f267fea --- /dev/null +++ b/docs/SECURITY.md @@ -0,0 +1,24 @@ +# Security (V1 A) + +V1 A is a library. It does not listen on a network. Binding it to a public address without authentication would violate the product rule; nothing in this milestone opens a socket. + +## What is enforced + +- Every store call requires an `AuthContext`: tenant, trust domain, and a 32-byte credential fingerprint. The fingerprint is compared in constant time. An all-zero fingerprint is rejected. +- Empty policy configuration fails closed. Unknown tenants fail closed. +- An identity whose tenant or trust domain does not match the caller is `Unauthorized`. The store does not return that block's bytes. +- Physical keys include the tenant. There is no cross-tenant dedup and no list-all-keys API. +- Logs and errors do not embed payloads, prompts, or credentials. Checksums in errors are not included; the error is `ChecksumMismatch`. +- Only `COMMITTED`, checksum-valid, same-identity blocks are returned. +- Quarantine plus fencing is the rule for buffers whose completion is unknown. A timer is not treated as proof that a copy has stopped. + +## What is not enforced + +- mTLS, replay protection beyond the idempotency window, secret rotation, and audit shipping are later work (the product note puts production hardening in V3; V1 still requires this minimum gate before any network exposure). +- `Attestation::Drained` is accepted from any caller who already passed the tenant gate. It is not a device measurement. A buggy client inside the tenant can reclaim its own in-flight buffer early. +- The idempotency table forgets entries after `idempotency_ttl_ms` or when it is full (new ids are then rejected until expiry). Reusing an id after expiry can run the operation again. +- This crate does not encrypt KV bytes at rest. MemoryStore keeps them in process memory. + +## Fallback + +Callers must treat every error and every miss as "recompute in the engine". They must not stitch a partial or quarantined block into a decode. diff --git a/docs/adr/0001-integration-route.md b/docs/adr/0001-integration-route.md new file mode 100644 index 0000000..e85ee94 --- /dev/null +++ b/docs/adr/0001-integration-route.md @@ -0,0 +1,36 @@ +# ADR 0001: Inference integration route + +- Status: **Pending** +- Date: 2026-10-15 +- Milestone: V1 A + +## Context + +GPUCache has to sit beside one real inference engine. The engine for the product is vLLM. Two integration routes are plausible: + +1. **Native vLLM KV connector.** Use the connector hook that the pinned vLLM revision already exposes, and move KV blocks through that API. The cache then sees the engine's real block layout, dtype, and TP/PP sharding. +2. **LMCache bridge.** Reuse LMCache's existing vLLM integration and treat GPUCache as a storage backend behind that bridge. This avoids writing a second connector, at the cost of another project's release cadence, license, and layout assumptions. + +V1 must pick one primary path. Maintaining both half-implemented is how layout bugs ship. The choice depends on a spike that exports one real KV block from vLLM, moves it, imports it, and checks the engine output. A compile of either project, or a fake tensor, is not that spike. + +## Decision + +**No route is selected.** The spike was not run. + +This environment has no GPU, no approved model weights, and no budget for a rented accelerator. The spike is `NOT_RUN_HARDWARE`. V1 A therefore stops at the CPU contract: versioned identity, the batch protocol, and `MemoryStore`. Neither vLLM nor LMCache is a crate dependency. + +When a GPU is authorized, the spike should: + +- pin one vLLM commit and one model revision whose license is already approved; +- export one real KV block for a fixed TP/PP layout and dtype; +- round-trip it through the V1 identity and checksum rules; +- import it and compare engine output under the engine's own determinism rules; +- record which hook already exists in that vLLM revision. + +Prefer the hook that already exists in the pinned revision. Switch only if that spike fails. Do not start the second path in the same change. + +## Consequences + +- Callers must not assume GPUCache can read an arbitrary engine's KV bytes. +- Resharding, a different quant format, or a different layout ABI is a miss, not a conversion. +- A later ADR will replace this status with Accepted and name the one route. Until then, engine adapters stay out of the tree. diff --git a/rust-toolchain.toml b/rust-toolchain.toml new file mode 100644 index 0000000..33893d4 --- /dev/null +++ b/rust-toolchain.toml @@ -0,0 +1,4 @@ +[toolchain] +channel = "stable" +profile = "minimal" +components = ["clippy", "rustfmt"] From f09e817ac573308c0b1d3c36f17d27587c41e445 Mon Sep 17 00:00:00 2001 From: Cursor Agent Date: Thu, 15 Oct 2026 01:58:05 +0000 Subject: [PATCH 2/2] Record that the hot KV tier does not use erasure coding. A lost or failed G3.5 block is a clean miss, then recompute and repopulate. Cold durability stays with RustFS G4. No parity and no durability replicas. Co-authored-by: RustFS --- AGENTS.md | 5 +++-- README.md | 9 ++++++--- docs/ARCHITECTURE.md | 4 ++++ docs/RISKS.md | 1 + docs/adr/0002-no-erasure-coding.md | 28 ++++++++++++++++++++++++++++ 5 files changed, 42 insertions(+), 5 deletions(-) create mode 100644 docs/adr/0002-no-erasure-coding.md diff --git a/AGENTS.md b/AGENTS.md index f5408df..a60b27e 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,6 +1,6 @@ # Agent notes -GPUCache is the G3.5 shared KV tier. RustFS is the colder G4 backend. Read [README.md](README.md) and [docs/adr/0001-integration-route.md](docs/adr/0001-integration-route.md) before changing scope. +GPUCache is the G3.5 shared KV tier. RustFS is the colder G4 backend. Read [README.md](README.md), [docs/adr/0001-integration-route.md](docs/adr/0001-integration-route.md), and [docs/adr/0002-no-erasure-coding.md](docs/adr/0002-no-erasure-coding.md) before changing scope. ## Phase @@ -11,6 +11,7 @@ The active phase is **V1 milestone A** unless a human explicitly starts a later - Do not claim DPU, DOCA, RDMA, SPDK, or CUDA acceleration. V1 is CPU only. An aarch64 build is not a BlueField test. - Do not invent vLLM, LMCache, or NIXL results. A real engine KV export/import needs a GPU and stays `NOT_RUN_HARDWARE` until that run exists. - Do not treat a KV block as model-agnostic long-term memory. A miss falls back to recompute. A wrong-tenant or wrong-layout hit is a defect. +- Do not add erasure coding, parity, or durability replicas. A lost or failed hot block is a clean miss, then recompute and repopulate. Cold durability belongs to RustFS. An extra hot-prefix copy, if it is ever added, is a latency hint and not a durability promise. - Docs in this repo are English. ## Crates @@ -21,7 +22,7 @@ The active phase is **V1 milestone A** unless a human explicitly starts a later | `gpucache-protocol` | `gpucache-core-types` | | `gpucache-storage` | the two crates above | -No crate may add CUDA, DOCA, NIXL, RDMA, or SPDK. `unsafe` is forbidden. Registered buffers whose completion is unknown stay quarantined until drain or a newer epoch; do not free them only because a lease timer fired. +No crate may add CUDA, DOCA, NIXL, RDMA, SPDK, erasure coding, or replica placement. `unsafe` is forbidden. Registered buffers whose completion is unknown stay quarantined until drain or a newer epoch; do not free them only because a lease timer fired. ## Checks diff --git a/README.md b/README.md index ea89caa..b38ed3b 100644 --- a/README.md +++ b/README.md @@ -4,6 +4,8 @@ GPUCache is an open shared KV-cache tier for disaggregated inference. In NVIDIA A KV block is recomputable accelerator state. It is not the user's only copy of a prompt, and it is not model-agnostic long-term memory. A miss, timeout, checksum failure, version mismatch, or store outage must fall back to the inference engine and recompute. A wrong hit is a failure. +GPUCache does not do erasure coding and does not keep durability replicas. A lost or failed hot block is a clean miss: the engine recomputes and may repopulate the cache. Durability for cold data is RustFS G4's job. An extra copy of a hot shared prefix may be added later as a latency hint; that copy is not erasure coding and it is not a durability promise. See [docs/adr/0002-no-erasure-coding.md](docs/adr/0002-no-erasure-coding.md). + ## Status **V1 milestone A is implemented. DPU acceleration is not.** @@ -17,9 +19,10 @@ V1 runs the cache contract on CPU. The long-term deployment target is a storage- | V1 B: TCP service, file/NVMe, admission, RustFS smoke | Not started | | V1 C: fuzz, fault injection, engine output comparison, paired baselines | Not started | | V2: NIC RDMA (NIXL or Mooncake) plus a real RustFS cold path | Not started; blocked on V1 | -| V3: BlueField-3 offload and production hardening | Not started; not purchased and not measured | +| V3: BlueField-3 data-path offload and production hardening | Not started; not purchased and not measured. Not an erasure-coding milestone | +| Erasure coding and durability replicas | Will not be built. A lost or failed hot block is a clean miss, then recompute and repopulate. Cold durability is RustFS G4 | -The integration route (native vLLM KV connector or LMCache) is **not chosen**. See [docs/adr/0001-integration-route.md](docs/adr/0001-integration-route.md). +The integration route (native vLLM KV connector or LMCache) is **not chosen**. See [docs/adr/0001-integration-route.md](docs/adr/0001-integration-route.md). Hot-tier failure policy is [docs/adr/0002-no-erasure-coding.md](docs/adr/0002-no-erasure-coding.md). ## What V1 A does @@ -59,7 +62,7 @@ That command is the CPU contract repro. It includes unit tests, property tests, crates/core-types identity, keys, lifecycle, errors crates/protocol batch messages and version negotiation crates/storage MemoryStore and kv-block-xfer -docs/adr integration-route decision record +docs/adr decision records (integration route, no erasure coding) COMPATIBILITY.md environment manifest ``` diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 7696687..4b81f6f 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -18,6 +18,10 @@ gpucache-core-types (identity, keys, lifecycle, errors) Planned crates that are **not** in V1 A: `metadata` as a standalone service, `transport` (TCP), file/NVMe store, RustFS object store, engine adapter, DPU agent, bench harness, metrics. Their contracts are reserved so later milestones do not push GPU or DPU types into `core-types`. +## Durability + +The hot tier does not repair lost blocks. There is no erasure coding and no replica set. Loss, corruption, eviction, and a failed check are clean misses. The engine recomputes and may write the block again. An extra copy of a hot shared prefix is a possible later latency optimization, not a durability feature, and it is not in this tree. Cold durability is RustFS's responsibility. See [adr/0002-no-erasure-coding.md](adr/0002-no-erasure-coding.md). + ## Identity `CacheIdentity` is version 1. It carries tenant and trust domain, model revision, model-config digest, tokenizer revision, adapter/LoRA identity, KV dtype, quant format, layout ABI, TP/PP shard, position encoding, RoPE parameters (IEEE-754 bits), a token-prefix **hash** chain, block token capacity, valid length, and an optional multimodal processor/input digest. diff --git a/docs/RISKS.md b/docs/RISKS.md index abd1e04..a1bddd1 100644 --- a/docs/RISKS.md +++ b/docs/RISKS.md @@ -12,5 +12,6 @@ | No RDMA / DPU measurement | `NOT_RUN_HARDWARE` | Do not infer a speedup from this milestone. | | License of a future DOCA or model stack | `NOT_RUN_LICENSE` | No DOCA binary and no model weights are in the tree. | | GPU rental or DPU purchase | `NOT_RUN_BUDGET` | No paid API, GPU, or DPU was used. | +| Hot block loss | Accepted | No erasure coding and no durability replicas. The outcome is a clean miss, then recompute and repopulate. Cold durability is RustFS G4. See ADR 0002. | Minimum reproduction of the CPU contract is `cargo test --workspace --all-targets`. There is no hardware failure log because the hardware was not present. diff --git a/docs/adr/0002-no-erasure-coding.md b/docs/adr/0002-no-erasure-coding.md new file mode 100644 index 0000000..b4e1b4a --- /dev/null +++ b/docs/adr/0002-no-erasure-coding.md @@ -0,0 +1,28 @@ +# ADR 0002: The hot tier does not use erasure coding or replicas + +- Status: **Accepted** +- Date: 2026-10-15 +- Milestone: V1 A + +## Context + +GPUCache is a shared KV-cache tier (G3.5). A block is recomputable accelerator state. RustFS is the colder G4 capacity backend, and durability for cold data belongs there. + +Hardware erasure coding on a DPU was considered because other storage systems use it. It is not part of this cache. Parity and replica maintenance would sit on the hit path of data the inference engine can rebuild. That is the wrong cost for a miss-tolerant tier. + +## Decision + +GPUCache does not implement erasure coding. It does not implement durability replicas. + +If a hot block is lost, corrupt, evicted, or fails a check, the result is a clean miss. The engine recomputes the block and may populate the cache again. There is no rebuild from parity. + +An extra copy of a hot shared prefix may be considered later as a latency optimization. That copy is not erasure coding, it is not a durability promise, and it is not implemented in V1 A. + +DOCA EC, SPDK RAID targets, and similar offloads are not GPUCache components. They must not reappear on the roadmap as cache features. A later BlueField data-path milestone is not an erasure-coding milestone. + +## Consequences + +- The README and roadmap do not list an EC milestone. +- The V1 A crates have no EC types, no parity buffers, and no replica placement. +- RustFS remains responsible for any durable cold object it is later asked to store. That client is not in V1 A. +- A cache miss after a node loss is correct behavior. The KV block is not the user's only copy.