From 21c3b100952a54f7480273a8c38f01f508b56835 Mon Sep 17 00:00:00 2001
From: Yao Yao <lowsfer@users.noreply.github.com>
Date: Tue, 21 Jul 2026 08:45:44 +0000
Subject: [PATCH 01/12] [None][docs] add KVCacheManagerV2 C++ development guide

Replace the temporary Python-to-C++ migration plan with durable guidance
colocated with the C++ implementation.

Document the architecture, ownership and lifetime model, cache and page
state machines, hashing and partial-coverage invariants, storage and CUDA
constraints, nanobind boundaries, high-risk changes, and the fast
direct-Python test workflow. This preserves useful migration knowledge
after the Python backend and its reference documents are removed.

Signed-off-by: Yao Yao <lowsfer@users.noreply.github.com>
---
 .../kv_cache_manager_v2/AGENTS.md             | 301 ++++++++++++++++++
 1 file changed, 301 insertions(+)
 create mode 100644 cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/AGENTS.md

diff --git a/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/AGENTS.md b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/AGENTS.md
new file mode 100644
index 000000000000..ceb8e5e55e97
--- /dev/null
+++ b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/AGENTS.md
@@ -0,0 +1,301 @@
+<!--
+SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+SPDX-License-Identifier: Apache-2.0
+-->
+
+# KVCacheManagerV2 C++ Guide
+
+This directory contains the C++ implementation of KVCacheManagerV2: allocation,
+prefix reuse, eviction, GPU/host/disk movement, page locking, and KV-cache event
+generation. Python accesses it through the nanobind bindings in
+`cpp/tensorrt_llm/nanobind/batch_manager/kvCacheManagerV2.cpp`.
+
+This guide is self-contained. Do not make changes here depend on the Python
+KVCacheManagerV2 implementation or its migration documents; those files are
+temporary and will be removed after the migration.
+
+The current C++ headers and tests are the source of truth. Historical Python
+behavior remains useful for compatibility testing, but migration-era proposed
+layouts or ownership models must not override the implementation.
+
+## Layout
+
+- `common.h`, `config.h`, `tokenIdExt.h`, and `exceptions.h`: shared types,
+  configuration, token encoding, and error types.
+- `blockRadixTree.*`: the shared prefix-reuse tree and SHA-256 block keys.
+- `page.*`, `kvCache.*`, and `kvCacheManager.*`: page lifecycle, per-request
+  cache state, and the top-level manager.
+- `storage/`, `storageManager.*`, `evictionController.*`, and `copyEngine.*`:
+  pools, eviction ownership, migration, and data movement.
+- `lifeCycleRegistry.*`: layer-group/lifecycle mapping, including attention and
+  SSM behavior.
+- `eventManager.*` and `eventSink.h`: KV-cache event derivation and delivery.
+- `utils/`: typed indices, ownership helpers, CUDA events, host memory, and
+  math utilities.
+
+## Design principles
+
+- Prefer composition when it expresses ownership or containment. Use
+  inheritance only for a real IS-A relationship or required virtual dispatch,
+  such as `CommittedPage : Page` and `EventManager : EventSink`.
+- Preserve strongly typed indices. `LayerGroupId` is a public lifecycle
+  semantic, `PoolGroupIndex` is a storage-layout index, `BlockOrdinal` is a
+  sequence position, and `SlotId`/page indices address storage. Do not convert
+  or compare them implicitly.
+- Keep hot-path checks under `TLLM_CHECK_DEBUG*` or `gDebug` when they are too
+  expensive for release builds. Do not remove debug invariants simply because
+  production does not execute them.
+- Keep the core independent of Python. C++ storage, copying, CUDA, hashing, and
+  lifecycle operations call C++ APIs directly; Python adaptation belongs in
+  nanobind.
+
+## Architecture and request flow
+
+`KvCacheManager` owns the global services: a `LifeCycleRegistry`,
+`BlockRadixTree`, and `StorageManager`. It creates a `KvCache` per request.
+The request cache starts `SUSPENDED`, becomes `ACTIVE` through `resume()`, and
+must eventually be `close()`d. Its normal flow is:
+
+1. Match the input `TokenSpan` against `BlockRadixTree` within a `ReuseScope`.
+2. Allocate request-local pages for unmatched blocks through `StorageManager`.
+3. Lock or migrate required committed pages to GPU before model execution.
+4. Commit completed blocks to the tree, making their immutable pages available
+   for later requests; `stopCommitting()` finalizes this process.
+5. Suspend or close the request, returning pages to holding/eviction ownership.
+
+`LifeCycleRegistry` maps model layers to lifecycle groups. Attention lifecycles
+may have sliding-window and sink-token rules; SSM lifecycles represent a
+recurrent-state checkpoint. A pool-group index is a storage-layout index and is
+not interchangeable with a layer ID or lifecycle ID.
+
+`StorageManager` coordinates GPU, host, and disk cache levels. It allocates
+slots, schedules pages for eviction, migrates pages between levels, and resizes
+pools. `CopyEngine` performs the actual batched transfers; C++ code calls it
+directly and must not round-trip through Python bindings.
+
+The dependency direction is broadly:
+
+```text
+types/config/exceptions
+  -> lifecycle + memory/CUDA utilities
+  -> storage pools + eviction + copy engine + radix tree
+  -> pages + storage manager
+  -> per-request KvCache
+  -> KvCacheManager
+  -> nanobind API
+```
+
+## State machines
+
+### Per-request cache
+
+- `SUSPENDED`: no active CUDA-stream use; committed pages can be held or
+  evicted.
+- `ACTIVE`: pages required by the request are locked to GPU and use the cache's
+  CUDA stream.
+- `CLOSED`: resources are released; further use is invalid.
+
+`commit()` finalizes full blocks. Its `isEnd=true` form is a terminal-memory
+contract: later writes to the request's KV memory are invalid, because final
+live pages may be moved into the radix tree rather than copied.
+
+`stopCommitting()` is a distinct transition and must not call `commit()`:
+doing so would append the same tokens twice. It also releases stale held SWA
+pages and performs final commit-state bookkeeping.
+
+### Page status
+
+- `LOCKED`: required on GPU; neither eviction nor dropping is permitted.
+- `HELD`: eviction is allowed, but dropping is not.
+- `DROPPABLE`: both eviction and dropping are allowed.
+
+The `PageHolder`, `UniqPageLock`, and `SharedPageLock` types implement these
+transitions. CUDA ready/finish events are part of their correctness contract:
+they establish write completion, migration ordering, and safe reuse across
+streams. A stream change for an active cache intentionally synchronizes the
+new stream with the old one.
+
+## Ownership and lifetime
+
+The high-level ownership shape is:
+
+```text
+KvCacheManager
+|- LifeCycleRegistry (value)
+|- StorageManager (shared)
+|- BlockRadixTree (shared)
+`- living KvCache registry (non-owning pointers)
+
+KvCache
+|- KvCacheManager (shared; cache keeps manager alive)
+`- per-beam/per-block page holders and locks
+
+BlockRadixTree
+`- roots -> child Blocks (strong ownership through next maps)
+              `- lifecycle page entries (raw observer links)
+
+Eviction controller
+`- prioritized LRU lists (strong ownership of droppable Pages)
+```
+
+- Follow the existing `SharedPtr`/`WeakPtr` conventions in `utils/sharedPtr.h`.
+  Treat every ownership edge as intentional; do not replace weak edges with
+  strong ones merely to simplify access.
+- `KvCache` keeps its `KvCacheManager` alive. The manager's registry of living
+  caches must not create the reverse strong-reference cycle.
+- A committed page is referenced by the radix tree without making the tree its
+  permanent owner. Eviction queues may be the only strong owner of a droppable
+  page, so never store a raw pointer past the operation that obtained it.
+- The eviction queue stores strong page ownership and its `NodeRef` is valid
+  only for the eviction policy that created it. Exclude a page from eviction
+  before moving it to another cache level; only then schedule it in the new
+  level's policy.
+- Destructors can trigger tree detachment, page unlinking, or eviction updates.
+  Keep teardown order explicit, make cleanup idempotent where needed, and audit
+  re-entrancy before changing a destructor or `close`/`shutdown` path.
+- `CommittedPage::numTokensInBlock` can be smaller than its block's token span.
+  For attention it describes a reusable prefix; for SSM it is an exact state
+  checkpoint. Do not assume every page covers its whole block.
+- `Block::prev`, `Block::storage`, `CommittedPage::block`, and page manager
+  pointers are observer/back-reference links with lifetime invariants, not
+  ownership. Explicit unlinking and teardown order keep them valid.
+- `Block::releasePages()` must run while `StorageManager` is alive. Do not defer
+  page reclamation solely to `Block` destruction: external block references can
+  outlive tree membership.
+
+## Correctness invariants
+
+- Block keys are SHA-256 digests over the reuse scope and the token sequence.
+  They are a security boundary for cross-request reuse. Do not replace,
+  truncate, or use a non-cryptographic hash unless prefix matching also gains a
+  token-content equality check.
+- `knownNoDigest=true` is an external guarantee (for example `text_only`), not
+  a hint derived by scanning tokens. Passing it incorrectly corrupts key hashes.
+- The radix tree owns child blocks and a child never outlives its parent. Preserve
+  parent/child attachment order when replacing or removing blocks.
+- Root removal is deferred through `proposeToEraseEmptyRoot()` and drained only
+  at tree safe points. Do not erase roots directly from a destructor chain.
+- Pages are immutable after commit. Locking and CUDA events establish when their
+  data is safe to read or migrate; preserve those synchronization boundaries.
+- Event payloads describe complete blocks. A lifecycle with partial page coverage
+  must not emit an event for the full block.
+- Hash bytes and token encoding must stay compatible across all callers. Normal
+  token IDs use the little-endian `TokenIdExt` representation; digest tokens and
+  `ReuseScope` values participate in the same chained block-key protocol.
+- Partial attention coverage uses a `>=` prefix check. SSM coverage names an
+  exact recurrent-state checkpoint and must be truncated to that boundary.
+- Multi-beam block arrays, sliding-window stale ranges, sink tokens, partial
+  block reuse, SSM snapshots, intra-batch rebasing, and rollback after OOM are
+  coupled inside `KvCache`; changes there require broad invariant testing.
+
+## Storage, eviction, and memory
+
+- Every cache level has storage and a per-level eviction controller; each pool
+  group has a priority-sorted set of LRU queues. Lower priority is evicted first,
+  then least-recently-used within that priority.
+- `NodeRef` is a stable `std::list` iterator, but only within the exact
+  `LRUEvictionPolicy` that issued it. Iterators from different lists are not
+  comparable or interchangeable.
+- Eviction failure must preserve queue consistency. If a multi-pool eviction
+  cannot satisfy all requested slots, restore pages already removed before
+  propagating `OutOfPagesError`.
+- Host and disk memory code directly uses `mmap`, `munmap`, `mremap`,
+  `madvise`, `posix_fallocate`, and CUDA host registration. Preserve cleanup on
+  partial failure, CUDA unregister/register ordering across resize, and the
+  distinction between host and disk OOM errors.
+- CUDA virtual memory and copy operations use the driver/runtime APIs directly.
+  Keep allocation granularity, stream ordering, and source/destination lifetime
+  valid through asynchronous copies.
+
+## Interfaces, bindings, and build
+
+- `TokenSpan` is non-owning. The caller retains its backing storage for the full
+  call; the manager reads tokens but never stores the span itself. This enables
+  the int32 zero-copy matching path.
+- Keep C++ implementation sources co-located here and add every compiled source
+  to this directory's `CMakeLists.txt`. The parent target consumes its source
+  list; do not add a separate shared library for this subsystem.
+- SHA-256 support is vendored under `cpp/tensorrt_llm/batch_manager/common/sha256`
+  and configured by this directory's `CMakeLists.txt`. Preserve the
+  architecture-specific SHA extension flags when changing the hash integration.
+  Do not add OpenSSL/libcrypto merely for block hashing; avoiding that dependency
+  is intentional for wheel portability.
+- Nanobind bindings belong in
+  `cpp/tensorrt_llm/nanobind/batch_manager/kvCacheManagerV2.cpp`. Keep the
+  public Python surface compatible with the runtime package; use the
+  introspection API only for white-box tests and diagnostics.
+
+## Nanobind and concurrency
+
+- C++ public APIs are called under the executor's single-threaded KV-cache
+  access model. Do not add mutexes or relax that model without auditing every
+  manager, cache, page, and callback path.
+- Binding code may release the GIL only while it touches no Python objects. Keep
+  Python conversions and callbacks under the GIL, and keep non-owning token
+  buffers alive for the entire C++ call.
+- Reacquire the GIL before invoking a Python callback, wrapping a C++ result,
+  creating an `nb::object`, or changing Python reference counts. If the
+  single-threaded access precondition is ever relaxed, concurrency protection
+  must be designed for the whole manager rather than added piecemeal.
+- Exceptions crossing the binding boundary need an explicit nanobind mapping.
+  Preserve Python exception type and attributes when adding or changing a C++
+  exception.
+
+## High-risk changes
+
+Use extra review and tests for changes involving:
+
+- destructor, `shutdown()`, `close()`, tree detachment, or page unlink order;
+- eviction ownership, `NodeRef`, migration, pool resizing, or OOM rollback;
+- block hashes, token encoding, salts/LoRA reuse scopes, or `knownNoDigest`;
+- partial attention coverage, SSM checkpoints, SWA windows/sinks, or final
+  snapshots;
+- CUDA stream/event lifetime or host-memory registration;
+- `KvCache` commit state, beam forks, reuse rebasing, or page-index buffers;
+- nanobind GIL release, callbacks, non-owning buffers, or exception translation.
+
+## Development and tests
+
+- Focused C++ unit tests are in `cpp/tests/unit_tests/batch_manager/`, notably
+  `radixBlockTreeTest.cpp`, `kvCacheManagerTest.cpp`,
+  `kvCacheManagerV2DigestPoolTest.cpp`, `kvCacheManagerV2HostMemTest.cpp`,
+  `kvCacheManagerV2StatsTest.cpp`, and `kvCacheManagerV2TypedIndexTest.cpp`.
+- Python behavior and backend-parity tests are in
+  `tests/unittest/kv_cache_manager_v2_tests/`. During development, prefer the
+  fast path below: set `PYTHONPATH` to `tensorrt_llm/runtime/` and execute the
+  test file directly with `python`. Do not use `pytest` for this fast path; the
+  file's test runner avoids importing the full `tensorrt_llm` package.
+
+  ```bash
+  REPO_ROOT="$(git rev-parse --show-toplevel)"
+  PYTHONPATH="$REPO_ROOT/tensorrt_llm/runtime/" \
+      python "$REPO_ROOT/tests/unittest/kv_cache_manager_v2_tests/test_kv_cache_manager_v2.py" -v
+  ```
+
+- Run one test class or method by passing its unittest name:
+
+  ```bash
+  REPO_ROOT="$(git rev-parse --show-toplevel)"
+  PYTHONPATH="$REPO_ROOT/tensorrt_llm/runtime/" \
+      python "$REPO_ROOT/tests/unittest/kv_cache_manager_v2_tests/test_kv_cache_manager_v2.py" \
+      TestNoBatching.test_basic -v
+  ```
+
+- Before final validation, also exercise the production import path:
+
+  ```bash
+  REPO_ROOT="$(git rev-parse --show-toplevel)"
+  PYTHONPATH="$REPO_ROOT/" \
+      python "$REPO_ROOT/tests/unittest/kv_cache_manager_v2_tests/test_kv_cache_manager_v2.py" -v
+  ```
+
+- Run `test_kv_cache_event_manager.py` after event changes,
+  `test_kv_cache_salting.py` after hashing/reuse-scope changes, and the stats
+  tests after allocation or event-accounting changes. Run both available
+  backends for changes shared by the C++ and Python surfaces.
+- Run focused tests when possible, then the affected KVCacheManagerV2 Python
+  suite on both available backends. Use `TLLM_DEBUG_MODE=1` when diagnosing an
+  invariant failure.
+- Follow the repository C++ standards in `CODING_GUIDELINES.md`: Allman braces,
+  east-const, explicit ownership, and clang-format. Do not make unrelated
+  formatting changes in this high-churn subsystem.

From 8860441965b2e2d29124a177805772ad3cb5d804 Mon Sep 17 00:00:00 2001
From: Yao Yao <lowsfer@users.noreply.github.com>
Date: Sun, 26 Jul 2026 12:23:24 +0000
Subject: [PATCH 02/12] [None][refactor] simplify KVCacheManagerV2 token and
 key handling

Replace the variant-based TokenIdExt with a four-byte tagged value. Store
normal token IDs inline and keep rare multimodal digests in an address-stable
process-global pool with value semantics.

Pass digest-free knowledge explicitly to hashing and add manager- and
request-level text_only configuration so eligible workloads use bulk SHA-256.
Validate the contract at the nanobind boundary to prevent digest tokens from
silently corrupting block keys.

Move blockchain-key and multimodal-token builders into the native C++ backend
while retaining equivalent Python behavior and byte-compatible keys. Remove
the obsolete Python cache-key module and cover digest ownership, salting,
routing, and statistics with the updated tests.

Signed-off-by: Yao Yao <lowsfer@users.noreply.github.com>
---
 .../kv_cache_manager_v2/CMakeLists.txt        |   1 +
 .../kv_cache_manager_v2/blockRadixTree.cpp    | 234 +++++++--------
 .../kv_cache_manager_v2/blockRadixTree.h      |  99 ++++--
 .../kv_cache_manager_v2/common.h              | 100 +-----
 .../kv_cache_manager_v2/config.h              |   6 +
 .../kv_cache_manager_v2/eventManager.cpp      |  12 +-
 .../kv_cache_manager_v2/kvCache.cpp           |  38 ++-
 .../kv_cache_manager_v2/kvCache.h             |  15 +-
 .../kv_cache_manager_v2/kvCacheManager.cpp    |  15 +-
 .../kv_cache_manager_v2/kvCacheManager.h      |  25 +-
 .../batch_manager/kv_cache_manager_v2/page.h  |   1 -
 .../kv_cache_manager_v2/tokenIdExt.cpp        | 284 ++++++++++++++++++
 .../kv_cache_manager_v2/tokenIdExt.h          | 183 +++++++++++
 .../kv_cache_manager_v2/utils/math.h          |   9 +
 .../batch_manager/kvCacheManagerV2.cpp        | 147 +++++++--
 .../unit_tests/batch_manager/CMakeLists.txt   |   5 +
 .../kvCacheManagerV2DigestPoolTest.cpp        | 215 +++++++++++++
 .../kvCacheManagerV2StatsTest.cpp             |   4 +-
 .../runtime/kv_cache_manager_v2/AGENTS.md     |   2 +-
 .../runtime/kv_cache_manager_v2/__init__.py   |  11 +-
 .../runtime/kv_cache_manager_v2/__init__.pyi  |  10 +-
 .../kv_cache_manager_v2/_block_radix_tree.py  | 125 +++++++-
 .../runtime/kv_cache_manager_v2/_cache_key.py | 134 ---------
 .../runtime/kv_cache_manager_v2/_config.py    |  10 +
 .../kv_cache_manager_v2/_core/_kv_cache.py    |  26 ++
 .../_core/_kv_cache_manager.py                |  11 +-
 .../runtime/kv_cache_manager_v2/_page.py      |   3 -
 .../test_kv_cache_manager_v2.py               |   6 +-
 .../test_kv_cache_salting.py                  |  12 +-
 29 files changed, 1294 insertions(+), 449 deletions(-)
 create mode 100644 cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/tokenIdExt.cpp
 create mode 100644 cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/tokenIdExt.h
 create mode 100644 cpp/tests/unit_tests/batch_manager/kvCacheManagerV2DigestPoolTest.cpp
 delete mode 100644 tensorrt_llm/runtime/kv_cache_manager_v2/_cache_key.py

diff --git a/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/CMakeLists.txt b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/CMakeLists.txt
index 163ad82ffd3f..9a7d3af13671 100644
--- a/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/CMakeLists.txt
+++ b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/CMakeLists.txt
@@ -49,6 +49,7 @@ set(KV_CACHE_MANAGER_V2_SRCS
     kv_cache_manager_v2/storage/core.cpp
     kv_cache_manager_v2/evictionController.cpp
     kv_cache_manager_v2/copyEngine.cpp
+    kv_cache_manager_v2/tokenIdExt.cpp
     kv_cache_manager_v2/blockRadixTree.cpp
     kv_cache_manager_v2/eventManager.cpp
     kv_cache_manager_v2/page.cpp
diff --git a/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/blockRadixTree.cpp b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/blockRadixTree.cpp
index 5d0d4bc314c0..9df8c42ba690 100644
--- a/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/blockRadixTree.cpp
+++ b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/blockRadixTree.cpp
@@ -25,11 +25,19 @@
 
 #include "tensorrt_llm/common/assert.h"
 #include <algorithm>
+#include <cassert>
 #include <cstring>
 #include <stdexcept>
 #include <utility>
 #include <variant>
 
+// Token hashing reinterprets TokenIdExt bytes as a raw little-endian stream (both
+// the per-element and the bulk paths), so a normal token's 4 bytes equal its
+// integer id. Guard the assumption at compile time (std::endian is C++20; C++17 here).
+#if defined(__BYTE_ORDER__) && (__BYTE_ORDER__ != __ORDER_LITTLE_ENDIAN__)
+#error "kv_cache_manager_v2 block hashing requires a little-endian target"
+#endif
+
 namespace tensorrt_llm::batch_manager::kv_cache_manager_v2
 {
 
@@ -82,17 +90,6 @@ void ensureSha256Detected()
 }
 } // namespace
 
-static void hashInt64(CSHA256& h, int64_t v)
-{
-    unsigned char buf[8];
-    auto const unsignedValue = static_cast<uint64_t>(v);
-    for (int i = 0; i < 8; ++i)
-    {
-        buf[i] = static_cast<unsigned char>((unsignedValue >> (8 * i)) & 0xFFU);
-    }
-    h.Write(buf, sizeof(buf));
-}
-
 Hasher::Hasher()
 {
     ensureSha256Detected();
@@ -114,13 +111,15 @@ Hasher& Hasher::update(ReuseScope const& scope)
 
 Hasher& Hasher::update(TokenId token)
 {
-    hashInt64(mState, static_cast<int64_t>(token));
+    static_assert(sizeof(TokenId) == sizeof(TokenIdExt));
+    assert(TokenIdExt(token).tokenId() == token);
+    mState.Write(reinterpret_cast<unsigned char const*>(&token), sizeof(token));
     return *this;
 }
 
-Hasher& Hasher::update(BlockKey const& key)
+Hasher& Hasher::update(Digest const& digest)
 {
-    mState.Write(reinterpret_cast<unsigned char const*>(key.data()), key.size());
+    mState.Write(reinterpret_cast<unsigned char const*>(digest.data()), digest.size());
     return *this;
 }
 
@@ -132,26 +131,50 @@ Hasher& Hasher::update(std::vector<uint8_t> const& bytes)
 
 Hasher& Hasher::update(TokenIdExt const& tokenExt)
 {
-    std::visit(
-        [this](auto const& v)
-        {
-            using T = std::decay_t<decltype(v)>;
-            if constexpr (std::is_same_v<T, TokenId>)
-                hashInt64(mState, static_cast<int64_t>(v));
-            else
-                mState.Write(reinterpret_cast<unsigned char const*>(v.data()), v.size());
-        },
-        tokenExt);
-    return *this;
+    if (tokenExt.isDigest())
+    {
+        return update(tokenExt.digest());
+    }
+    return update(tokenExt.tokenId());
 }
 
-Hasher& Hasher::update(TokenIdExt const* tokens, size_t count)
+Hasher& Hasher::update(TokenIdExt const* tokens, size_t count, bool knownNoDigest)
 {
-    // Python uses array("Q", data).tobytes() to reduce per-token interpreter
-    // overhead.  In C++ the compiler inlines each update() call, so the loop
-    // is already optimal; batching would only add a heap allocation.
-    for (size_t i = 0; i < count; ++i)
-        update(tokens[i]);
+    TokenIdExt const* const end = tokens + count;
+
+    // Bulk-write a contiguous run of normal tokens [begin, stop) as one raw
+    // little-endian uint32 block — the whole point of the 4-byte layout. A single
+    // Write of k tokens is byte-identical to k per-token writes.
+    auto const writeNormalRun = [this](TokenIdExt const* begin, TokenIdExt const* stop)
+    {
+        if (begin != stop)
+        {
+            mState.Write(
+                reinterpret_cast<unsigned char const*>(begin), static_cast<size_t>(stop - begin) * sizeof(TokenIdExt));
+        }
+    };
+
+    if (knownNoDigest)
+    {
+        TLLM_CHECK_DEBUG(std::none_of(tokens, end, [](TokenIdExt const& t) { return t.isDigest(); }));
+        writeNormalRun(tokens, end);
+        return *this;
+    }
+
+    // Unknown/digest-bearing: bulk-write each maximal run of normal tokens and
+    // hash each (rare, multi-modal) digest on its own. All-normal collapses to a
+    // single Write.
+    for (TokenIdExt const* pos = tokens; pos != end;)
+    {
+        TokenIdExt const* const digestIt = std::find_if(pos, end, [](TokenIdExt const& t) { return t.isDigest(); });
+        writeNormalRun(pos, digestIt);
+        if (digestIt == end)
+        {
+            break;
+        }
+        update(digestIt->digest()); // 32 digest bytes
+        pos = digestIt + 1;
+    }
     return *this;
 }
 
@@ -166,10 +189,10 @@ BlockKey Hasher::digest() const
 }
 
 // ---------------------------------------------------------------------------
-// genMultiModalTokens
+// genMultimodalCacheKeyTokens
 // ---------------------------------------------------------------------------
 
-std::vector<TokenIdExt> genMultiModalTokens(
+std::vector<TokenIdExt> genMultimodalCacheKeyTokens(
     int idOffset, std::vector<uint8_t> const& multiModalDataDigest, int numTokens, int tokenOffset)
 {
     TLLM_CHECK_DEBUG(numTokens > 0);
@@ -181,68 +204,18 @@ std::vector<TokenIdExt> genMultiModalTokens(
     {
         if (tokenOffset + i == 0)
         {
-            Digest d;
-            std::memcpy(d.data(), multiModalDataDigest.data(), kDIGEST_LEN);
-            result.emplace_back(DigestToken(d));
+            Digest digest;
+            std::memcpy(digest.data(), multiModalDataDigest.data(), kDIGEST_LEN);
+            result.emplace_back(digest);
         }
         else
         {
-            result.emplace_back(TokenId(idOffset + tokenOffset + i));
+            result.emplace_back(TokenId{idOffset + tokenOffset + i});
         }
     }
     return result;
 }
 
-// ---------------------------------------------------------------------------
-// makeBlockchainKeyGenerator — lazy key generator.
-// Returns a callable that yields one BlockKey per call (nullopt when done).
-// First call yields root entry (empty token block). Mirrors Python's generator.
-// ---------------------------------------------------------------------------
-
-static auto makeBlockchainKeyGenerator(
-    int tokensPerBlock, ReuseScope reuseScope, TokenIdExt const* tokens, size_t numTokens)
-{
-    // digest carries the running hash from the previous block.
-    BlockKey digest = Hasher(reuseScope).digest();
-    // ordinal = -1: next call yields root (reuseScope digest).
-    // ordinal >= 0: next call yields key for tokens[ordinal*tpb .. (ordinal+1)*tpb).
-    int ordinal = -1;
-
-    return [=]() mutable -> std::optional<BlockKey>
-    {
-        if (ordinal == -1)
-        {
-            ordinal++;
-            return digest; // root key
-        }
-
-        size_t beg = static_cast<size_t>(ordinal) * static_cast<size_t>(tokensPerBlock);
-        if (beg >= numTokens)
-            return std::nullopt;
-
-        size_t end = std::min(beg + static_cast<size_t>(tokensPerBlock), numTokens);
-
-        Hasher h;
-        h.update(digest);
-        h.update(tokens + beg, end - beg);
-        digest = h.digest();
-
-        ordinal++;
-        return digest;
-    };
-}
-
-// Eager wrapper for callers that need all keys at once.
-std::vector<BlockKey> sequenceToBlockchainKeys(
-    int tokensPerBlock, ReuseScope const& reuseScope, std::vector<TokenIdExt> const& tokens)
-{
-    std::vector<BlockKey> result;
-    auto gen = makeBlockchainKeyGenerator(tokensPerBlock, reuseScope, tokens.data(), tokens.size());
-    while (auto key = gen())
-        result.push_back(*key);
-    return result;
-}
-
 // ---------------------------------------------------------------------------
 // RootBlock
 // ---------------------------------------------------------------------------
@@ -309,11 +282,13 @@ SharedPtr<Block> NodeBase::detachNext(BlockKey const& blockKey)
 namespace
 {
 
-static bool isPrefix(std::vector<TokenIdExt> const& prefix, std::vector<TokenIdExt> const& full)
+// Takes raw (ptr, size) so it works uniformly over any TokenIdExt buffer
+// (the query vector and the std::vector<TokenIdExt> that backs Block::tokens).
+static bool isPrefix(TokenIdExt const* prefix, size_t prefixLen, TokenIdExt const* full, size_t fullLen)
 {
-    if (prefix.size() > full.size())
+    if (prefixLen > fullLen)
         return false;
-    for (size_t i = 0; i < prefix.size(); ++i)
+    for (size_t i = 0; i < prefixLen; ++i)
     {
         if (prefix[i] != full[i])
             return false;
@@ -323,21 +298,35 @@ static bool isPrefix(std::vector<TokenIdExt> const& prefix, std::vector<TokenIdE
 
 } // anonymous namespace
 
-BlockKey Block::makeKey(BlockKey const& prevKey, TokenIdExt const* tokens, size_t count)
+BlockKey Block::makeKey(BlockKey const& prevKey, TokenIdExt const* tokens, size_t count, bool knownNoDigest)
 {
     Hasher h;
     h.update(prevKey);
-    h.update(tokens, count);
+    h.update(tokens, count, knownNoDigest);
     return h.digest();
 }
 
-Block::Block(BlockKey k, std::vector<TokenIdExt> toks, NodeBase* prevNode, LifeCycleId numLifeCycles)
+Block::Block(BlockKey k, std::vector<TokenIdExt> toks, NodeBase* prevNode)
     : NodeBase(k, prevNode->eventSink)
     , tokens(std::move(toks))
     , prev(prevNode)
-    , storage(numLifeCycles, nullptr)
+    , storage(prevNode->numLifeCycles(), nullptr) // tree-wide count, derived from prev
     , mOrdinal(prevNode->ordinal() + 1)
 {
+    // key is a caller-supplied second source of truth (precomputed for the
+    // pre-construction dedup lookup). Verify it matches what we'd derive from
+    // prev + tokens so the two can never silently drift. Debug-only and fully
+    // compiled out in release: this re-hashes (exactly the recomputation the param
+    // exists to avoid). knownNoDigest=false lets makeKey's update() scan for digests
+    // itself — correct regardless of content, so no separate scan is needed here.
+    TLLM_CHECK_DEBUG(k == Block::makeKey(prevNode->key, tokens.data(), tokens.size(), /*knownNoDigest=*/false));
+}
+
+// Delegates to the tree, mirroring Python's RootBlock.num_life_cycles. Defined
+// out-of-line so BlockRadixTree is complete at the point of use.
+LifeCycleId RootBlock::numLifeCycles() const noexcept
+{
+    return tree->numLifeCycles();
 }
 
 int Block::tokensPerBlock() const noexcept
@@ -508,8 +497,7 @@ std::vector<SharedPtr<Block>> Block::clearStaleBlocksAfterPageUnlink(
 // addOrGetExistingBlock
 // ---------------------------------------------------------------------------
 
-SharedPtr<Block> addOrGetExistingBlock(
-    NodeBase* prev, LifeCycleId numLifeCycles, std::vector<TokenIdExt> tokens, bool* isNew)
+SharedPtr<Block> addOrGetExistingBlock(NodeBase* prev, std::vector<TokenIdExt> tokens, bool knownNoDigest, bool* isNew)
 {
     TLLM_CHECK_DEBUG_WITH_INFO(prev, "prev must not be null");
 
@@ -521,7 +509,7 @@ SharedPtr<Block> addOrGetExistingBlock(
 
     auto& prevNext = prev->next;
     int const tpb = prev->tokensPerBlock();
-    BlockKey newKey = Block::makeKey(prev->key, tokens.data(), tokens.size());
+    BlockKey newKey = Block::makeKey(prev->key, tokens.data(), tokens.size(), knownNoDigest);
 
     // Exact match: return existing block (not new — mirrors Python's UselessBlockError path).
     auto it = prevNext.find(newKey);
@@ -538,7 +526,8 @@ SharedPtr<Block> addOrGetExistingBlock(
     {
         for (auto const& [k, sibling] : prevNext)
         {
-            if (sibling->tokens.size() >= tokens.size() && isPrefix(tokens, sibling->tokens))
+            if (sibling->tokens.size() >= tokens.size()
+                && isPrefix(tokens.data(), tokens.size(), sibling->tokens.data(), sibling->tokens.size()))
                 throw UselessBlockError(sibling);
         }
     }
@@ -552,7 +541,8 @@ SharedPtr<Block> addOrGetExistingBlock(
     std::vector<BlockKey> toRemove;
     for (auto const& [k, sibling] : prevNext)
     {
-        if (sibling->tokens.size() < tokens.size() && isPrefix(sibling->tokens, tokens))
+        if (sibling->tokens.size() < tokens.size()
+            && isPrefix(sibling->tokens.data(), sibling->tokens.size(), tokens.data(), tokens.size()))
         {
             TLLM_CHECK_DEBUG(!sibling->isFull() && sibling->key == k && sibling->next.empty());
             toRemove.push_back(k);
@@ -562,8 +552,9 @@ SharedPtr<Block> addOrGetExistingBlock(
     // would already have replaced the shorter one.
     TLLM_CHECK_DEBUG(toRemove.size() <= 1);
 
-    // Create the new block. ordinal and tokensPerBlock are derived from prev inside the Block ctor.
-    auto block = makeShared<Block>(newKey, std::move(tokens), prev, numLifeCycles);
+    // Create the new block. ordinal, tokensPerBlock, and numLifeCycles are all
+    // derived from prev. Block stores the tokens as a plain vector (moved in).
+    auto block = makeShared<Block>(newKey, std::move(tokens), prev);
 
     // Keep the parent attached while covered children are replaced. Adding the replacement
     // first prevents detachNext() from pruning an emptied RootBlock out of the tree.
@@ -727,52 +718,51 @@ int numMatchedTokens(std::vector<BlockRadixTree::MatchResult> const& matched, in
 
 } // anonymous namespace
 
-std::vector<BlockRadixTree::MatchResult> BlockRadixTree::matchTokenPath(
-    ReuseScope const& reuseScope, std::vector<TokenIdExt> const& tokens, bool enablePartialMatch) const
+std::vector<BlockRadixTree::MatchResult> BlockRadixTree::matchTokenPath(ReuseScope const& reuseScope,
+    std::vector<TokenIdExt> const& tokens, bool knownNoDigest, bool enablePartialMatch) const
 {
     drainPendingRootErases();
 
     std::vector<MatchResult> results;
 
     // Lazily compute one key per iteration — no wasted hashing on early miss.
-    auto gen = makeBlockchainKeyGenerator(mTokensPerBlock, reuseScope, tokens.data(), tokens.size());
+    auto gen = sequenceToBlockchainKeys(mTokensPerBlock, reuseScope, tokens.data(), tokens.size(), knownNoDigest);
 
-    // First key is the root key.
-    auto rootKey = gen();
-    if (!rootKey)
+    // First step is the root key (empty token range).
+    auto rootStep = gen();
+    if (!rootStep)
         return results;
-    auto rootIt = mRoots.find(*rootKey);
+    auto rootIt = mRoots.find(rootStep->key);
     if (rootIt == mRoots.end())
         return results;
 
     RootBlock const& root = *rootIt->second;
     std::unordered_map<BlockKey, SharedPtr<Block>> const* currentNext = &root.next;
-    // ordinal tracks which block we're on (0-based, after root).
-    BlockOrdinal ordinal{0};
+    // Token range of the first unmatched block, captured on miss for the partial pass.
+    HalfOpenRange<size_t> missedRange;
     bool missed = false;
 
-    while (auto key = gen())
+    // Each step carries the block's key and its token range — no need to re-derive
+    // block boundaries here.
+    while (auto step = gen())
     {
-        auto blockIt = currentNext->find(*key);
+        auto blockIt = currentNext->find(step->key);
         if (blockIt == currentNext->end())
         {
+            missedRange = step->tokens;
             missed = true;
             break;
         }
-        size_t beg = toSizeT(ordinal) * static_cast<size_t>(mTokensPerBlock);
-        int numTokens = static_cast<int>(std::min(static_cast<size_t>(mTokensPerBlock), tokens.size() - beg));
         Block* block = blockIt->second.get();
-        results.push_back({block, numTokens});
+        results.push_back({block, static_cast<int>(step->tokens.length())});
         currentNext = &block->next;
-        ordinal++;
     }
 
     // Partial match in children of current node.
     if (missed && enablePartialMatch)
     {
-        size_t beg = toSizeT(ordinal) * static_cast<size_t>(mTokensPerBlock);
-        size_t missedCount = std::min(static_cast<size_t>(mTokensPerBlock), tokens.size() - beg);
-        auto [best, bestMatch] = findBestPartialMatchInNextNodes(*currentNext, tokens.data() + beg, missedCount);
+        auto [best, bestMatch]
+            = findBestPartialMatchInNextNodes(*currentNext, tokens.data() + missedRange.beg, missedRange.length());
         if (best)
             results.push_back({best, bestMatch});
     }
@@ -873,10 +863,10 @@ std::vector<BlockRadixTree::MatchResult> BlockRadixTree::pruneMatch(std::vector<
     return matched;
 }
 
-BlockRadixTree::ReuseMatch BlockRadixTree::match(
-    ReuseScope const& reuseScope, std::vector<TokenIdExt> const& tokens, bool enablePartialMatch) const
+BlockRadixTree::ReuseMatch BlockRadixTree::match(ReuseScope const& reuseScope, std::vector<TokenIdExt> const& tokens,
+    bool knownNoDigest, bool enablePartialMatch) const
 {
-    auto const matched = pruneMatch(matchTokenPath(reuseScope, tokens, enablePartialMatch));
+    auto const matched = pruneMatch(matchTokenPath(reuseScope, tokens, knownNoDigest, enablePartialMatch));
     ReuseMatch result{};
     result.numTokens = numMatchedTokens(matched, mTokensPerBlock);
     result.numLookupTokens = static_cast<int>(tokens.size());
diff --git a/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/blockRadixTree.h b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/blockRadixTree.h
index c4a1debcb093..7b674db5f038 100644
--- a/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/blockRadixTree.h
+++ b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/blockRadixTree.h
@@ -20,10 +20,13 @@
 #include "kv_cache_manager_v2/common.h"
 #include "kv_cache_manager_v2/eventSink.h"
 #include "kv_cache_manager_v2/lifeCycleRegistry.h"
+#include "kv_cache_manager_v2/tokenIdExt.h"
+#include "kv_cache_manager_v2/utils/math.h" // HalfOpenRange
 #include "kv_cache_manager_v2/utils/sharedPtr.h"
 
 #include "sha256.h"
 
+#include <algorithm>
 #include <array>
 #include <cstdint>
 #include <iterator>
@@ -88,11 +91,14 @@ class Hasher
     explicit Hasher(ReuseScope const& seed);
 
     Hasher& update(TokenId token);
-    Hasher& update(BlockKey const& key);
+    Hasher& update(Digest const& digest); // 32 raw bytes (BlockKey is a Digest alias)
     Hasher& update(ReuseScope const& scope);
     Hasher& update(std::vector<uint8_t> const& bytes);
     Hasher& update(TokenIdExt const& tokenExt);
-    Hasher& update(TokenIdExt const* tokens, size_t count);
+    // knownNoDigest: caller guarantees the range holds no digest, enabling the bulk
+    // fast path. Only pass true from external knowledge (request/model text_only) — never
+    // from scanning the tokens, since false already makes update() scan internally.
+    Hasher& update(TokenIdExt const* tokens, size_t count, bool knownNoDigest = false);
 
     BlockKey digest() const;
 
@@ -100,16 +106,61 @@ class Hasher
     CSHA256 mState;
 };
 
+// One step of the blockchain-key generator: a block's key plus the half-open token
+// index range [beg, end) it covers ([0, 0) for the root). Carrying the range lets
+// callers slice the tokens without re-deriving block boundaries, mirroring Python's
+// (token_block, key) pairs.
+struct BlockchainKeyStep
+{
+    BlockKey key;
+    HalfOpenRange<size_t> tokens;
+};
+
 // ---------------------------------------------------------------------------
-// Utility: convert a token sequence → list of BlockKeys.
-// First key is the root (reuseScope digest), then one per token block.
-// Mirrors Python's sequence_to_blockchain_keys().
+// sequenceToBlockchainKeys — lazy per-block key generator.
+// Returns a callable yielding one BlockchainKeyStep per call (nullopt when done):
+// the first call yields the root (reuseScope digest, empty [0,0) range), then one
+// step per tokensPerBlock chunk chained on the previous digest. Mirrors Python's
+// sequence_to_blockchain_keys(). Lazy so a caller that stops early (e.g. on the
+// first mismatch) skips the remaining hashing. Inline so both the tree
+// (matchTokenPath) and the nanobind layer drive the same generator.
+// knownNoDigest: from external text_only knowledge, never a scan (see Hasher::update).
 // ---------------------------------------------------------------------------
-std::vector<BlockKey> sequenceToBlockchainKeys(
-    int tokensPerBlock, ReuseScope const& reuseScope, std::vector<TokenIdExt> const& tokens);
+inline auto sequenceToBlockchainKeys(
+    int tokensPerBlock, ReuseScope reuseScope, TokenIdExt const* tokens, size_t numTokens, bool knownNoDigest = false)
+{
+    // digest carries the running hash from the previous block.
+    BlockKey digest = Hasher(reuseScope).digest();
+    // ordinal = -1: next call yields root (reuseScope digest).
+    // ordinal >= 0: next call yields key for tokens[ordinal*tpb .. (ordinal+1)*tpb).
+    int ordinal = -1;
+
+    return [=]() mutable -> std::optional<BlockchainKeyStep>
+    {
+        if (ordinal == -1)
+        {
+            ordinal++;
+            return BlockchainKeyStep{digest, {}}; // root key, empty [0,0) token range
+        }
 
-// Generate multi-modal token IDs (mirrors gen_multi_modal_tokens in Python).
-std::vector<TokenIdExt> genMultiModalTokens(
+        size_t beg = static_cast<size_t>(ordinal) * static_cast<size_t>(tokensPerBlock);
+        if (beg >= numTokens)
+            return std::nullopt;
+
+        size_t end = std::min(beg + static_cast<size_t>(tokensPerBlock), numTokens);
+
+        Hasher h;
+        h.update(digest);
+        h.update(tokens + beg, end - beg, knownNoDigest);
+        digest = h.digest();
+
+        ordinal++;
+        return BlockchainKeyStep{digest, {beg, end}};
+    };
+}
+
+// Generate multi-modal token IDs (mirrors gen_multimodal_cache_key_tokens in Python).
+std::vector<TokenIdExt> genMultimodalCacheKeyTokens(
     int idOffset, std::vector<uint8_t> const& multiModalDataDigest, int numTokens, int tokenOffset = 0);
 
 // ---------------------------------------------------------------------------
@@ -139,6 +190,10 @@ struct NodeBase
     /// RootBlock: delegates to tree. Block: len(prev->tokens) or prev->tokensPerBlock().
     virtual int tokensPerBlock() const noexcept = 0;
 
+    /// Tree-wide life-cycle count. RootBlock: delegates to tree. Block: storage.size().
+    /// Mirrors Python's num_life_cycles property.
+    virtual LifeCycleId numLifeCycles() const noexcept = 0;
+
 protected:
     NodeBase(BlockKey k, EventSink* sink)
         : key(k)
@@ -172,6 +227,7 @@ struct RootBlock : NodeBase
     }
 
     int tokensPerBlock() const noexcept override;
+    LifeCycleId numLifeCycles() const noexcept override; // delegates to tree
 };
 
 // ---------------------------------------------------------------------------
@@ -181,6 +237,8 @@ struct RootBlock : NodeBase
 // ---------------------------------------------------------------------------
 struct Block : NodeBase, EnableSharedFromThis<Block>
 {
+    // A block's tokens are written once and never re-hashed because its key is
+    // computed before construction, so store them in a plain vector.
     std::vector<TokenIdExt> tokens;
 
     // Previous node in the chain (RootBlock or Block). Null after detaching from the tree.
@@ -189,10 +247,14 @@ struct Block : NodeBase, EnableSharedFromThis<Block>
 
     TypedVec<LifeCycleId, CommittedPage*> storage;
 
-    Block(BlockKey key, std::vector<TokenIdExt> tokens, NodeBase* prev, LifeCycleId numLifeCycles);
+    // key is precomputed by the caller (for the pre-construction dedup lookup);
+    // numLifeCycles is derived from prev. tokens is moved in as a plain vector.
+    Block(BlockKey key, std::vector<TokenIdExt> tokens, NodeBase* prev);
     ~Block() override;
 
-    static BlockKey makeKey(BlockKey const& prevKey, TokenIdExt const* tokens, size_t count);
+    // knownNoDigest: from external text_only knowledge, never a scan (see Hasher::update).
+    static BlockKey makeKey(
+        BlockKey const& prevKey, TokenIdExt const* tokens, size_t count, bool knownNoDigest = false);
 
     Type type() const noexcept override
     {
@@ -206,7 +268,7 @@ struct Block : NodeBase, EnableSharedFromThis<Block>
 
     int tokensPerBlock() const noexcept override;
 
-    LifeCycleId numLifeCycles() const noexcept
+    LifeCycleId numLifeCycles() const noexcept override
     {
         return storage.size();
     }
@@ -309,8 +371,9 @@ class BlockRadixTree
         int numLookupTokens;
     };
 
-    ReuseMatch match(
-        ReuseScope const& reuseScope, std::vector<TokenIdExt> const& tokens, bool enablePartialMatch = false) const;
+    // knownNoDigest: from external text_only knowledge, never a scan (see Hasher::update).
+    ReuseMatch match(ReuseScope const& reuseScope, std::vector<TokenIdExt> const& tokens, bool knownNoDigest = false,
+        bool enablePartialMatch = false) const;
 
     // Clear all cached pages. ~Block() handles excludeFromEviction for DROPPABLE pages.
     void clear();
@@ -346,8 +409,9 @@ class BlockRadixTree
     }
 
 private:
-    std::vector<MatchResult> matchTokenPath(
-        ReuseScope const& reuseScope, std::vector<TokenIdExt> const& tokens, bool enablePartialMatch) const;
+    // knownNoDigest: from external text_only knowledge, never a scan (see Hasher::update).
+    std::vector<MatchResult> matchTokenPath(ReuseScope const& reuseScope, std::vector<TokenIdExt> const& tokens,
+        bool knownNoDigest, bool enablePartialMatch) const;
     std::vector<MatchResult> pruneMatch(std::vector<MatchResult> matched) const;
 
     // Erase any pending empty root blocks from mRoots.
@@ -371,8 +435,9 @@ class BlockRadixTree
 // prefix of an existing sibling — mirrors Python's UselessBlockError.
 // If isNew is non-null, *isNew is set to true if a new block was created, false
 // if an existing block was returned.
+// knownNoDigest: from external text_only knowledge, never a scan (see Hasher::update).
 SharedPtr<Block> addOrGetExistingBlock(
-    NodeBase* prev, LifeCycleId numLifeCycles, std::vector<TokenIdExt> tokens, bool* isNew = nullptr);
+    NodeBase* prev, std::vector<TokenIdExt> tokens, bool knownNoDigest, bool* isNew = nullptr);
 
 // Post-order traversal: remove a subtree rooted at `root` from its parent's
 // next map. ~Block() handles page cleanup. Mirrors Python's remove_subtree().
diff --git a/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/common.h b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/common.h
index 0f53938b95e2..8bf1eec60c27 100644
--- a/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/common.h
+++ b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/common.h
@@ -17,16 +17,17 @@
 
 #pragma once
 
+#include "kv_cache_manager_v2/tokenIdExt.h" // TokenId, Digest, TokenIdExt
 #include "kv_cache_manager_v2/utils/typedIndex.h"
 #include "tensorrt_llm/batch_manager/common.h"
 
 #include <array>
 #include <cstdint>
 #include <cstring>
-#include <memory>
 #include <optional>
 #include <string>
 #include <sys/types.h>
+#include <type_traits>
 #include <variant>
 
 namespace tensorrt_llm::batch_manager::kv_cache_manager_v2
@@ -76,96 +77,12 @@ enum class PageIndexMode : int
 using CacheLevel = StrongIndex<int, struct CacheLevelTag, 0>;
 inline constexpr CacheLevel kGpuLevel{0};
 
-// Vocabulary token identifier (normal tokens only).
-using TokenId = int64_t;
-
 // Opaque request identifier shared with the rest of the batch manager.
 using RequestIdType = tensorrt_llm::batch_manager::RequestIdType;
 
 // Opaque LoRA task identifier shared with the rest of the batch manager.
 using LoraTaskIdType = tensorrt_llm::runtime::LoraTaskIdType;
 
-// 32-byte aligned to enable SIMD.
-inline constexpr int kDIGEST_LEN = 32;
-
-struct alignas(kDIGEST_LEN) Digest : std::array<std::byte, kDIGEST_LEN>
-{
-    // Custom operator== needed to emit SIMD code
-    bool operator==(Digest const& o) const noexcept
-    {
-        return std::memcmp(this, &o, kDIGEST_LEN) == 0;
-    }
-
-    bool operator!=(Digest const& o) const noexcept
-    {
-        return !(*this == o);
-    }
-};
-
-// Heap-allocated digest token for multi-modal tokens.
-// Copyable (deep-copies the digest) with value-based equality.
-// Digest tokens are rare, so unique_ptr keeps sizeof(TokenIdExt) small.
-class DigestToken
-{
-public:
-    explicit DigestToken(Digest const& d)
-        : mData(std::make_unique<Digest>(d))
-    {
-    }
-
-    explicit DigestToken(std::unique_ptr<Digest> d)
-        : mData(std::move(d))
-    {
-    }
-
-    DigestToken(DigestToken const& o)
-        : mData(std::make_unique<Digest>(*o.mData))
-    {
-    }
-
-    DigestToken(DigestToken&&) noexcept = default;
-
-    DigestToken& operator=(DigestToken const& o)
-    {
-        if (this != &o)
-            mData = std::make_unique<Digest>(*o.mData);
-        return *this;
-    }
-
-    DigestToken& operator=(DigestToken&&) noexcept = default;
-
-    bool operator==(DigestToken const& o) const
-    {
-        return *mData == *o.mData;
-    }
-
-    bool operator!=(DigestToken const& o) const
-    {
-        return !(*this == o);
-    }
-
-    std::byte const* data() const noexcept
-    {
-        return mData->data();
-    }
-
-    size_t size() const noexcept
-    {
-        return mData->size();
-    }
-
-    Digest const& digest() const noexcept
-    {
-        return *mData;
-    }
-
-private:
-    std::unique_ptr<Digest> mData;
-};
-
-// Extended token id: normal TokenId or a heap-allocated digest for multi-modal tokens.
-using TokenIdExt = std::variant<TokenId, DigestToken>;
-
 // Ordinal index of a KV cache block (sequence of tokens).
 using BlockOrdinal = StrongIndex<int, struct BlockOrdinalTag, -1>;
 inline constexpr BlockOrdinal kBadBlockOrdinal{-1};
@@ -228,16 +145,3 @@ using Address = std::variant<MemAddress, DiskAddress>;
 using DataRole = std::string;
 
 } // namespace tensorrt_llm::batch_manager::kv_cache_manager_v2
-
-// std::hash specialization for Digest/BlockKey so unordered_map works without a custom hasher.
-template <>
-struct std::hash<tensorrt_llm::batch_manager::kv_cache_manager_v2::Digest>
-{
-    size_t operator()(tensorrt_llm::batch_manager::kv_cache_manager_v2::Digest const& k) const noexcept
-    {
-        // First 8 bytes of a SHA-256 digest are already well-distributed.
-        uint64_t v;
-        std::memcpy(&v, k.data(), sizeof(v));
-        return static_cast<size_t>(v);
-    }
-};
diff --git a/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/config.h b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/config.h
index 6852ed51f5a1..0471a7f0f3ea 100644
--- a/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/config.h
+++ b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/config.h
@@ -293,6 +293,12 @@ struct KVCacheManagerConfig
     // Collect V2 KV cache allocation, reuse, and transfer statistics.
     bool enableStats = true;
 
+    // Deployment-level guarantee that no request carries multi-modal content, so token
+    // sequences never contain digests. Lets block-key hashing take the digest-free fast
+    // path without scanning. A per-KvCache text_only override may only tighten this
+    // (a text-only deployment forbids a request claiming otherwise). Default false.
+    bool textOnly = false;
+
     bool enableSwaScratchReuse() const noexcept
     {
         return swaScratchReuse.has_value();
diff --git a/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/eventManager.cpp b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/eventManager.cpp
index 301d941d1cfc..e8fce2216564 100644
--- a/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/eventManager.cpp
+++ b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/eventManager.cpp
@@ -564,17 +564,16 @@ std::optional<KVCacheStoredBlockData> EventManager::storedBlockFromBlock(
     tokens.reserve(block.tokens.size());
     for (auto const& token : block.tokens)
     {
-        if (auto const* tokenId = std::get_if<TokenId>(&token))
+        if (!token.isDigest())
         {
             UniqueToken uniqueToken;
-            uniqueToken.tokenId = EventTokenId{std::in_place_index<0>, *tokenId};
+            uniqueToken.tokenId = EventTokenId{std::in_place_index<0>, token.tokenId()};
             tokens.push_back(std::move(uniqueToken));
         }
         else
         {
             UniqueToken uniqueToken;
-            uniqueToken.tokenId
-                = EventTokenId{std::in_place_index<1>, digestToHex(std::get<DigestToken>(token).digest())};
+            uniqueToken.tokenId = EventTokenId{std::in_place_index<1>, digestToHex(token.digest())};
             tokens.push_back(std::move(uniqueToken));
         }
     }
@@ -645,13 +644,12 @@ uint64_t EventManager::v1HashFromBlock(Block const& block)
         {
             for (auto const& token : currentBlock.tokens)
             {
-                auto const* tokenId = std::get_if<TokenId>(&token);
-                if (tokenId == nullptr)
+                if (token.isDigest())
                 {
                     parentIsV1Compatible = false;
                     break;
                 }
-                textTokens.push_back(*tokenId);
+                textTokens.push_back(token.tokenId());
             }
         }
         if (parentIsV1Compatible)
diff --git a/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/kvCache.cpp b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/kvCache.cpp
index 6beb2bd4f9eb..652fa3f633e6 100644
--- a/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/kvCache.cpp
+++ b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/kvCache.cpp
@@ -58,7 +58,8 @@ void copySlotData(StorageManager& storageMgr, CacheLevel dstLevel, CacheLevel sr
 // ---------------------------------------------------------------------------
 
 KvCache::KvCache(KvCacheManager& manager, ReuseScope reuseScope, std::optional<BlockRadixTree::ReuseMatch> reuseMatch,
-    std::optional<RequestIdType> mId, PriorityCb priorityCb, std::optional<int> expectedPromptLength)
+    std::optional<RequestIdType> mId, PriorityCb priorityCb, std::optional<int> expectedPromptLength,
+    std::optional<bool> textOnly)
     : id(mId)
     , mManager(manager.shared_from_this())
     , mReuseScope(std::move(reuseScope))
@@ -92,6 +93,10 @@ KvCache::KvCache(KvCacheManager& manager, ReuseScope reuseScope, std::optional<B
     mEnableSwaScratchReuse = manager.isSwaScratchReuseEnabled();
     mScratchSlots.resize(manager.storage().numLifeCycles());
 
+    TLLM_CHECK_WITH_INFO(!(manager.textOnly() && textOnly == std::optional<bool>{false}),
+        "text_only=false is not allowed when the manager is configured text_only=true");
+    mTextOnly = textOnly.value_or(manager.textOnly());
+
     if (reuseMatch.has_value())
     {
         _setupForReuse(*reuseMatch);
@@ -873,7 +878,6 @@ void KvCache::_snapshotPartialBlockToTree(BlockOrdinal ordinal, bool commitSsm)
     int const numTokens = static_cast<int>(tokens.size());
     TLLM_CHECK_DEBUG(0 < numTokens && numTokens < mTokensPerBlock);
 
-    LifeCycleId numLc = mManager->storage().numLifeCycles();
     NodeBase* prevNode = nullptr;
     RootBlock& root = mManager->radixTree().addOrGetExisting(mReuseScope);
     if (ordinal == BlockOrdinal{0})
@@ -889,7 +893,7 @@ void KvCache::_snapshotPartialBlockToTree(BlockOrdinal ordinal, bool commitSsm)
     SharedPtr<Block> treeBlock;
     try
     {
-        treeBlock = addOrGetExistingBlock(prevNode, numLc, tokens, &isNew);
+        treeBlock = addOrGetExistingBlock(prevNode, tokens, textOnly(), &isNew);
     }
     catch (UselessBlockError const& e)
     {
@@ -1535,7 +1539,7 @@ void KvCache::_commitBlock(int ord, bool isLast, bool commitSsm, bool moveSsm)
     SharedPtr<Block> newBlock;
     try
     {
-        newBlock = addOrGetExistingBlock(prevNode, numLc, tokenBlock, &blockIsNew);
+        newBlock = addOrGetExistingBlock(prevNode, tokenBlock, textOnly(), &blockIsNew);
     }
     catch (UselessBlockError const& e)
     {
@@ -2519,6 +2523,32 @@ void KvCache::setEnableSwaScratchReuse(bool enable)
     mEnableSwaScratchReuse = false;
 }
 
+bool KvCache::textOnly() const noexcept
+{
+    return mTextOnly;
+}
+
+void KvCache::setTextOnly(bool textOnly)
+{
+    // A text-only deployment is a hard guarantee: a request may not opt out.
+    if (!textOnly && mManager->textOnly())
+    {
+        throw std::invalid_argument(
+            "Cannot set text_only=false for a request when the KV cache manager is configured text_only=true");
+    }
+    // Claiming text-only is a fast-path claim; verify the committed tokens are digest-free.
+    if (textOnly)
+    {
+        bool const hasDigest = std::any_of(
+            mCommittedTokens.begin(), mCommittedTokens.end(), [](TokenIdExt const& t) { return t.isDigest(); });
+        if (hasDigest)
+        {
+            throw std::invalid_argument("Cannot set text_only=true: this sequence has already committed digest tokens");
+        }
+    }
+    mTextOnly = textOnly;
+}
+
 KvCache::DeltaScratchSlots KvCache::_takeExcessScratchSlots(int capacity, int historyLength)
 {
     LifeCycleId numLc = mManager->storage().numLifeCycles();
diff --git a/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/kvCache.h b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/kvCache.h
index 2fca81c8d17e..67da97a6b58a 100644
--- a/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/kvCache.h
+++ b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/kvCache.h
@@ -211,7 +211,8 @@ class KvCache : public std::enable_shared_from_this<KvCache>
     using PriorityCb = std::function<Priority(BlockOrdinal, LifeCycleId)>;
 
     KvCache(KvCacheManager& manager, ReuseScope reuseScope, std::optional<BlockRadixTree::ReuseMatch> reuseMatch,
-        std::optional<RequestIdType> id, PriorityCb priorityCb, std::optional<int> expectedPromptLength = std::nullopt);
+        std::optional<RequestIdType> id, PriorityCb priorityCb, std::optional<int> expectedPromptLength = std::nullopt,
+        std::optional<bool> textOnly = std::nullopt);
 
     ~KvCache();
 
@@ -431,6 +432,16 @@ class KvCache : public std::enable_shared_from_this<KvCache>
     // Enable or disable SWA scratch reuse. Throws if the transition is invalid.
     void setEnableSwaScratchReuse(bool enable);
 
+    // Resolved text-only status: the per-sequence override, else the manager config
+    // default. When true, this sequence's tokens are known to be digest-free, letting
+    // block-key hashing skip the digest scan.
+    bool textOnly() const noexcept;
+
+    // Set the per-sequence text-only override. Throws if the transition is invalid:
+    // a text-only deployment (config.textOnly) forbids setting false, and setting true
+    // requires the already-committed tokens to be digest-free (verified by a scan).
+    void setTextOnly(bool textOnly);
+
     // Whether the given page index mode is supported (SHARED requires no scratch slots).
     bool supportsIndexMode(PageIndexMode mode) const;
 
@@ -609,6 +620,8 @@ class KvCache : public std::enable_shared_from_this<KvCache>
     TypedVec<BlockOrdinal, SeqBlock> mBlocks;
 
     std::vector<TokenIdExt> mCommittedTokens;
+    // Per-sequence text-only override; nullopt means inherit the manager config default.
+    bool mTextOnly = false;
     int mNumCommittedBlocks;
     std::optional<CachedCudaEvent> mFinishEvent;
     int mTokensPerBlock;
diff --git a/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/kvCacheManager.cpp b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/kvCacheManager.cpp
index 617dc86e233b..056cb870a5c9 100644
--- a/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/kvCacheManager.cpp
+++ b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/kvCacheManager.cpp
@@ -172,7 +172,7 @@ void KvCacheManager::clearReusableBlocks()
 
 std::shared_ptr<KvCache> KvCacheManager::createKvCache(ReuseScope reuseScope,
     std::vector<TokenIdExt> const& inputTokens, std::optional<RequestIdType> id, KvCache::PriorityCb priorityCb,
-    std::optional<int> expectedPromptLength)
+    std::optional<int> expectedPromptLength, std::optional<bool> textOnly)
 {
     if (!priorityCb)
     {
@@ -190,22 +190,23 @@ std::shared_ptr<KvCache> KvCacheManager::createKvCache(ReuseScope reuseScope,
     std::optional<BlockRadixTree::ReuseMatch> reuseMatch;
     if (!inputTokens.empty())
     {
-        reuseMatch = matchReuse(reuseScope, inputTokens);
+        reuseMatch = matchReuse(reuseScope, inputTokens, textOnly.value_or(this->textOnly()));
     }
 
     return std::make_shared<KvCache>(*this, std::move(reuseScope), std::move(reuseMatch), std::move(id),
-        std::move(priorityCb), expectedPromptLength);
+        std::move(priorityCb), expectedPromptLength, textOnly);
 }
 
 BlockRadixTree::ReuseMatch KvCacheManager::matchReuse(
-    ReuseScope const& reuseScope, std::vector<TokenIdExt> const& inputTokens) const
+    ReuseScope const& reuseScope, std::vector<TokenIdExt> const& inputTokens, bool knownNoDigest) const
 {
-    return mRadixTree->match(reuseScope, inputTokens, enablePartialMatch());
+    return mRadixTree->match(reuseScope, inputTokens, knownNoDigest, enablePartialMatch());
 }
 
-int KvCacheManager::probeReuse(ReuseScope reuseScope, std::vector<TokenIdExt> const& inputTokens) const
+int KvCacheManager::probeReuse(
+    ReuseScope reuseScope, std::vector<TokenIdExt> const& inputTokens, bool knownNoDigest) const
 {
-    return matchReuse(reuseScope, inputTokens).numTokens;
+    return matchReuse(reuseScope, inputTokens, knownNoDigest).numTokens;
 }
 
 // ---- Memory pool queries --------------------------------------------------
diff --git a/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/kvCacheManager.h b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/kvCacheManager.h
index e51269f51284..4c868adaa457 100644
--- a/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/kvCacheManager.h
+++ b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/kvCacheManager.h
@@ -133,15 +133,24 @@ class KvCacheManager : public std::enable_shared_from_this<KvCacheManager>
     // ---- KvCache creation -------------------------------------------------
 
     // Create a new KvCache. Returned cache is SUSPENDED; call activate() with a stream.
-    // input_tokens: optional sequence to match against existing cached blocks.
-    // priorityCb:   optional priority override per block.
+    // input_tokens:         optional sequence to match against existing cached blocks.
+    // priorityCb:           optional priority override per block.
+    // expectedPromptLength: token count marking the prefill->generation boundary; once
+    //                       historyLength reaches it, later capacity growth is recorded as
+    //                       generation-phase allocation stats (defaults to inputTokens.size()).
+    //                       Stats-only: no effect on allocation, reuse, or correctness.
+    // textOnly:             per-sequence override of the text-only (digest-free) guarantee;
+    //                       nullopt inherits the manager config default.
     std::shared_ptr<KvCache> createKvCache(ReuseScope reuseScope = {}, std::vector<TokenIdExt> const& inputTokens = {},
         std::optional<RequestIdType> id = std::nullopt, KvCache::PriorityCb priorityCb = {},
-        std::optional<int> expectedPromptLength = std::nullopt);
+        std::optional<int> expectedPromptLength = std::nullopt, std::optional<bool> textOnly = std::nullopt);
 
+    // knownNoDigest: from external text_only knowledge, never a scan (see Hasher::update).
+    // Defaults false (safe: the scanning path is taken).
     BlockRadixTree::ReuseMatch matchReuse(
-        ReuseScope const& reuseScope, std::vector<TokenIdExt> const& inputTokens) const;
-    int probeReuse(ReuseScope reuseScope = {}, std::vector<TokenIdExt> const& inputTokens = {}) const;
+        ReuseScope const& reuseScope, std::vector<TokenIdExt> const& inputTokens, bool knownNoDigest = false) const;
+    int probeReuse(
+        ReuseScope reuseScope = {}, std::vector<TokenIdExt> const& inputTokens = {}, bool knownNoDigest = false) const;
 
     // ---- Memory pool queries -----------------------------------------------
 
@@ -175,6 +184,12 @@ class KvCacheManager : public std::enable_shared_from_this<KvCacheManager>
         return mConfig.commitMinSnapshot;
     }
 
+    // Deployment-level text-only guarantee (see KVCacheManagerConfig::textOnly).
+    bool textOnly() const noexcept
+    {
+        return mConfig.textOnly;
+    }
+
     bool isSwaScratchReuseEnabled() const noexcept
     {
         return mConfig.enableSwaScratchReuse();
diff --git a/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/page.h b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/page.h
index fc3a02252471..a446196177cd 100644
--- a/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/page.h
+++ b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/page.h
@@ -130,7 +130,6 @@ class UncommittedPage : public Page
     KvCache* kvCache;
     BlockOrdinal ordinal;
     BeamIndex beamIndex;
-    std::vector<TokenIdExt> tokens;
 
     UncommittedPage(KvCache& kvc, BlockOrdinal ord, LifeCycleId lc, CacheLevel level, BeamIndex bi = kDefaultBeamIndex);
 
diff --git a/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/tokenIdExt.cpp b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/tokenIdExt.cpp
new file mode 100644
index 000000000000..965b0a44b107
--- /dev/null
+++ b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/tokenIdExt.cpp
@@ -0,0 +1,284 @@
+/*
+ * SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+ * SPDX-License-Identifier: Apache-2.0
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include "kv_cache_manager_v2/tokenIdExt.h"
+
+#include "kv_cache_manager_v2/utils/math.h" // DynamicBitset
+#include "tensorrt_llm/common/assert.h"
+
+#include <cstddef>
+#include <cstdint>
+#include <cstdio>
+#include <deque>
+#include <mutex>
+
+namespace tensorrt_llm::batch_manager::kv_cache_manager_v2
+{
+
+// ---------------------------------------------------------------------------
+// DigestPool — process-global, address-stable store of 32-byte multi-modal
+// Digests, referenced by a 31-bit slot index packed into a TokenIdExt. It is a
+// pure implementation detail of TokenIdExt, so it lives here (anonymous
+// namespace) rather than in the header.
+//
+// Digests are rare (multi-modal only), so all access is guarded by a single
+// mutex; the all-normal-token hashing fast path never touches the pool. Storage
+// is a std::deque (element references stay valid across push_back — unlike a
+// reallocating std::vector). Occupancy is a DynamicBitset; alloc() takes the
+// LOWEST free slot (front-packing via a rolling minFreeHint) so live digests
+// cluster at the front and free() can pop trailing free slots off the deque
+// (never remapping a live index → outstanding borrows stay valid). The index is
+// unobservable (equality/hashing use the 32 bytes, never the index), so a single
+// static singleton is safe.
+// ---------------------------------------------------------------------------
+namespace
+{
+
+class DigestPool
+{
+public:
+    static DigestPool& instance()
+    {
+        static DigestPool pool;
+        return pool;
+    }
+
+    ~DigestPool()
+    {
+        // Runs at process exit (single-threaded → no lock). Every digest
+        // TokenIdExt should have freed its slot by now; a nonzero count means one
+        // outlived the pool — a lifetime bug. Use fprintf, not the logger, whose
+        // lifetime at static destruction is not guaranteed.
+        if (size_t const inUse = mInUse.numSetBits(); inUse != 0)
+        {
+            (void) std::fprintf(stderr, "[ERROR] DigestPool destroyed with %zu digest slot(s) still in use\n", inUse);
+        }
+    }
+
+    DigestPool(DigestPool const&) = delete;
+    DigestPool& operator=(DigestPool const&) = delete;
+    DigestPool(DigestPool&&) = delete;
+    DigestPool& operator=(DigestPool&&) = delete;
+
+    // Store a copy of `digest` in the lowest free slot; return its index.
+    uint32_t alloc(Digest const& digest)
+    {
+        std::lock_guard<std::mutex> const lock(mMutex);
+        return allocLocked(digest);
+    }
+
+    // Duplicate the digest at slot `idx` into a fresh slot; return the new index.
+    uint32_t duplicate(uint32_t idx)
+    {
+        std::lock_guard<std::mutex> const lock(mMutex);
+        return allocLocked(mStore[idx]); // Safe for deque
+    }
+
+    // The digest at slot `idx`. The reference stays valid after the lock is
+    // released and across later shrinks (which only pop free tail slots). Uses
+    // at() so a bad index (e.g. the sentinel of a moved-from handle) throws
+    // rather than reading out of bounds; digests are rare so the check is cheap.
+    [[nodiscard]] Digest const& get(uint32_t idx) const
+    {
+        std::lock_guard<std::mutex> const lock(mMutex);
+        return mStore.at(idx);
+    }
+
+    // Clear slot `idx` and reclaim trailing free slots.
+    void free(uint32_t idx)
+    {
+        if (idx == TokenIdExt::kValueMask)
+        {
+            return; // the default / moved-from sentinel index — nothing to free
+        }
+        std::lock_guard<std::mutex> const lock(mMutex);
+        mInUse.clear(idx);
+        if (idx < mMinFreeHint)
+        {
+            mMinFreeHint = idx; // a lower slot is now free
+        }
+        shrinkTailLocked();
+    }
+
+    [[nodiscard]] size_t liveCount() const
+    {
+        std::lock_guard<std::mutex> const lock(mMutex);
+        return mInUse.numSetBits();
+    }
+
+private:
+    DigestPool() = default;
+
+    // Slot count. Also checks the occupancy bitset stays sized to the store.
+    // Precondition: caller holds mMutex and mStore/mInUse are in sync (i.e. not
+    // called between growing/shrinking one and resizing the other).
+    [[nodiscard]] size_t capacity() const
+    {
+        TLLM_CHECK_DEBUG(mInUse.size() == mStore.size());
+        return mStore.size();
+    }
+
+    // Precondition: caller holds mMutex. Store `digest` in the lowest free slot
+    // (front-packing), growing the deque only when no free slot exists.
+    uint32_t allocLocked(Digest const& digest)
+    {
+        size_t const cap = capacity();
+        size_t idx = mMinFreeHint;
+        while (idx < cap && mInUse.get(idx))
+        {
+            ++idx;
+        }
+        if (idx == cap)
+        {
+            // No free slot below the high-water mark — grow by one. Indices stay
+            // strictly below kValueMask, which is reserved as the bad-handle sentinel.
+            TLLM_CHECK_WITH_INFO(cap < TokenIdExt::kValueMask, "DigestPool exhausted the 31-bit index space");
+            mStore.push_back(digest);
+            mInUse.resize(mStore.size()); // re-sync the bitset; new bit is clear
+        }
+        else
+        {
+            mStore[idx] = digest;
+        }
+        mInUse.set(idx);
+        mMinFreeHint = idx + 1; // everything below is now occupied
+        return static_cast<uint32_t>(idx);
+    }
+
+    // Precondition: caller holds mMutex. Pop free tail slots off the deque when
+    // the trailing-free run reaches kSlackHigh, leaving ~kSlackLow of buffer.
+    void shrinkTailLocked()
+    {
+        size_t const cap = capacity();
+        // Highest in-use index + 1 (scan down over the trailing free run only).
+        size_t liveEnd = cap;
+        while (liveEnd > 0 && !mInUse.get(liveEnd - 1))
+        {
+            --liveEnd;
+        }
+        size_t const trailingFree = cap - liveEnd;
+        if (trailingFree < kSlackHigh)
+        {
+            return;
+        }
+        size_t const newCapacity = liveEnd + kSlackLow; // leave a small buffer
+        // Only free tail slots are dropped, so no live index is remapped.
+        mStore.resize(newCapacity);
+        mInUse.resize(newCapacity);
+        if (mMinFreeHint > liveEnd)
+        {
+            mMinFreeHint = liveEnd; // lowest free slot is now at the reclaimed tail
+        }
+    }
+
+    // Absolute slack (in slots), NOT a ratio: a deque grows/shrinks one fixed
+    // chunk at a time, so we only damp boundary-block churn.
+    static constexpr size_t kSlackHigh = 256;
+    static constexpr size_t kSlackLow = 64;
+
+    mutable std::mutex mMutex;
+    std::deque<Digest> mStore; // slot storage; mStore.size() == the slot count (== bitset capacity)
+    DynamicBitset mInUse{0};   // bit i set == slot i occupied
+    size_t mMinFreeHint{0};    // lower bound on the lowest free slot index
+};
+
+} // namespace
+
+// ---------------------------------------------------------------------------
+// TokenIdExt — RAII members that touch the pool (construct/copy=alloc, dtor=free).
+// ---------------------------------------------------------------------------
+
+TokenIdExt::TokenIdExt(Digest const& digestValue)
+    : mBits(DigestPool::instance().alloc(digestValue) | kTagMask)
+{
+}
+
+TokenIdExt::~TokenIdExt()
+{
+    if (isDigest())
+    {
+        DigestPool::instance().free(digestIndex());
+    }
+}
+
+TokenIdExt::TokenIdExt(TokenIdExt const& other)
+    // Clone a digest into a fresh slot; a normal token just copies its bits.
+    : mBits(other.isDigest() ? (DigestPool::instance().duplicate(other.digestIndex()) | kTagMask) : other.mBits)
+{
+}
+
+TokenIdExt& TokenIdExt::operator=(TokenIdExt const& other)
+{
+    if (this != &other)
+    {
+        // Allocate the clone before releasing our slot (self-safe, exception-safe).
+        uint32_t const newBits
+            = other.isDigest() ? (DigestPool::instance().duplicate(other.digestIndex()) | kTagMask) : other.mBits;
+        if (isDigest())
+        {
+            DigestPool::instance().free(digestIndex());
+        }
+        mBits = newBits;
+    }
+    return *this;
+}
+
+TokenIdExt& TokenIdExt::operator=(TokenIdExt&& other) noexcept
+{
+    if (this != &other)
+    {
+        if (isDigest())
+        {
+            DigestPool::instance().free(digestIndex());
+        }
+        mBits = other.mBits;
+        other.mBits = kBadToken;
+    }
+    return *this;
+}
+
+Digest const& TokenIdExt::digest() const
+{
+    return DigestPool::instance().get(digestIndex());
+}
+
+bool TokenIdExt::operator==(TokenIdExt const& other) const noexcept
+{
+    if (mBits == other.mBits)
+    {
+        return true;
+    }
+    // Distinct bits are unequal unless both are digests whose 32-byte contents
+    // match (equal digests may live in different pool slots — no dedup).
+    if (isDigest() && other.isDigest())
+    {
+        return digest() == other.digest();
+    }
+    return false;
+}
+
+namespace detail
+{
+
+size_t digestPoolLiveCount()
+{
+    return DigestPool::instance().liveCount();
+}
+
+} // namespace detail
+
+} // namespace tensorrt_llm::batch_manager::kv_cache_manager_v2
diff --git a/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/tokenIdExt.h b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/tokenIdExt.h
new file mode 100644
index 000000000000..832641733d6e
--- /dev/null
+++ b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/tokenIdExt.h
@@ -0,0 +1,183 @@
+/*
+ * SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+ * SPDX-License-Identifier: Apache-2.0
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#pragma once
+
+#include "tensorrt_llm/common/assert.h"
+
+#include <array>
+#include <cstddef>
+#include <cstdint>
+#include <cstring>
+#include <type_traits>
+#include <vector>
+
+namespace tensorrt_llm::batch_manager::kv_cache_manager_v2
+{
+
+// Vocabulary token identifier (normal tokens only). 32-bit, matching the
+// runtime-wide tensorrt_llm::runtime::TokenIdType — real token ids are < vocab
+// size (~10^5-10^6), well within int32.
+using TokenId = std::int32_t;
+
+// 32-byte aligned to enable SIMD.
+inline constexpr int kDIGEST_LEN = 32;
+
+struct alignas(kDIGEST_LEN) Digest : std::array<std::byte, kDIGEST_LEN>
+{
+    // Custom operator== needed to emit SIMD code
+    bool operator==(Digest const& o) const noexcept
+    {
+        return std::memcmp(this, &o, kDIGEST_LEN) == 0;
+    }
+
+    bool operator!=(Digest const& o) const noexcept
+    {
+        return !(*this == o);
+    }
+};
+
+// ---------------------------------------------------------------------------
+// TokenIdExt — 4-byte self-describing token handle (RAII value type).
+//
+// One uint32_t; the high bit tags the low 31 bits:
+//   - tag 0: normal token id (stored verbatim). An all-normal array is a
+//     contiguous little-endian int32 array, hashed in one CSHA256::Write(N*4).
+//   - tag 1: multi-modal digest; low bits index a slot in an internal pool that
+//     holds the 32-byte Digest.
+//
+// A digest handle owns its pool slot: construct from a Digest to allocate, copy
+// to clone into a fresh slot, and destroy to free. Normal handles own nothing.
+// Pool-touching members are defined out-of-line in tokenIdExt.cpp.
+// ---------------------------------------------------------------------------
+class TokenIdExt
+{
+public:
+    static constexpr uint32_t kTagMask = 0x80000000U;
+    static constexpr uint32_t kValueMask = 0x7FFFFFFFU;
+    // Sentinel for default / moved-from handles: a *digest* (tag 1) whose index is
+    // the reserved value kValueMask — never a real slot. Tagging it as a digest
+    // leaves the whole tag-0 space to real token ids, and its destructor is a
+    // no-op because DigestPool::free ignores that index. Copying or dereferencing
+    // a moved-from handle is a bug and fails loudly (duplicate/get hit a bad slot).
+    static constexpr uint32_t kBadToken = kTagMask | kValueMask; // 0xFFFFFFFF
+    // Maximum normal token id — the full 31-bit range. (Digest slot indices stay
+    // strictly below kValueMask, which is reserved for the sentinel above.)
+    static constexpr uint32_t kMaxValue = kValueMask;
+
+    TokenIdExt() noexcept = default; // kBadToken (see mBits initializer)
+
+    // Normal token id (tag 0). Precondition: 0 <= id <= kMaxValue (so the value
+    // fits the 31-bit field and leaves the digest tag bit clear).
+    explicit TokenIdExt(TokenId id)
+        : mBits(static_cast<uint32_t>(id))
+    {
+        TLLM_CHECK_DEBUG(id >= 0 && id <= static_cast<TokenId>(kMaxValue));
+    }
+
+    // Multi-modal digest (tag 1): copies `digest` into a fresh pool slot.
+    explicit TokenIdExt(Digest const& digest);
+
+    ~TokenIdExt();
+    TokenIdExt(TokenIdExt const& other);            // clones a digest slot
+    TokenIdExt& operator=(TokenIdExt const& other); // clones a digest slot
+
+    TokenIdExt(TokenIdExt&& other) noexcept
+        : mBits(other.mBits)
+    {
+        other.mBits = kBadToken; // steal the slot; leave source empty
+    }
+
+    TokenIdExt& operator=(TokenIdExt&& other) noexcept;
+
+    [[nodiscard]] bool isDigest() const noexcept
+    {
+        return (mBits & kTagMask) != 0;
+    }
+
+    // Valid iff !isDigest(). A normal handle's tag bit is clear by construction
+    // (the ctor range-checks id <= kMaxValue), so the raw bits are already the id.
+    [[nodiscard]] TokenId tokenId() const noexcept
+    {
+        TLLM_CHECK_DEBUG(!isDigest());
+        return static_cast<TokenId>(mBits);
+    }
+
+    // The pooled 32-byte digest. Precondition: isDigest().
+    [[nodiscard]] Digest const& digest() const;
+
+    // Raw 4-byte payload — consumed by the bulk-hash fast path.
+    [[nodiscard]] uint32_t raw() const noexcept
+    {
+        return mBits;
+    }
+
+    // Value equality: normal/tag-mismatch compare raw bits; digest-vs-digest
+    // compares the pooled 32 bytes (equal content in different slots is equal).
+    bool operator==(TokenIdExt const& other) const noexcept;
+
+    bool operator!=(TokenIdExt const& other) const noexcept
+    {
+        return !(*this == other);
+    }
+
+private:
+    [[nodiscard]] uint32_t digestIndex() const noexcept
+    {
+        TLLM_CHECK_DEBUG(isDigest());
+        return mBits & kValueMask;
+    }
+
+    uint32_t mBits{kBadToken};
+};
+
+static_assert(sizeof(TokenIdExt) == 4, "TokenIdExt must be exactly 4 bytes for bulk hashing");
+static_assert(std::is_standard_layout_v<TokenIdExt>, "TokenIdExt must be standard-layout for byte-stream hashing");
+
+// The digest pool that backs digest-tagged TokenIdExt is an implementation
+// detail hidden entirely in tokenIdExt.cpp (anonymous namespace). Only this
+// introspection hook is exposed, for tests.
+namespace detail
+{
+
+// Live digest-pool slot count. Compare as a delta vs a captured baseline because
+// the pool is a process-global singleton.
+[[nodiscard]] size_t digestPoolLiveCount();
+
+} // namespace detail
+
+// Token sequences are held directly as std::vector<TokenIdExt>. TokenIdExt is an
+// RAII value type (copy = clone the digest slot, destroy = free it), so a plain
+// vector copies/destroys correctly with no wrapper. The "is this sequence
+// digest-free" summary that once lived here is now an explicit knownNoDigest /
+// text_only flag threaded from the request/model level (see blockRadixTree Hasher
+// and KvCache::textOnly).
+
+} // namespace tensorrt_llm::batch_manager::kv_cache_manager_v2
+
+// std::hash specialization for Digest/BlockKey so unordered_map works without a custom hasher.
+template <>
+struct std::hash<tensorrt_llm::batch_manager::kv_cache_manager_v2::Digest>
+{
+    size_t operator()(tensorrt_llm::batch_manager::kv_cache_manager_v2::Digest const& k) const noexcept
+    {
+        // First 8 bytes of a SHA-256 digest are already well-distributed.
+        uint64_t v;
+        std::memcpy(&v, k.data(), sizeof(v));
+        return static_cast<size_t>(v);
+    }
+};
diff --git a/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/utils/math.h b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/utils/math.h
index bdd1fd503708..c946a6810607 100644
--- a/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/utils/math.h
+++ b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/utils/math.h
@@ -231,9 +231,16 @@ class DynamicBitset
     explicit DynamicBitset(size_t capacity)
         : mWords(divUp(capacity, size_t{64}), uint64_t{0})
         , mNumSetBits(0)
+        , mSize(capacity)
     {
     }
 
+    // Logical bit capacity (as last set by the constructor / resize()).
+    [[nodiscard]] size_t size() const noexcept
+    {
+        return mSize;
+    }
+
     void set(size_t index)
     {
         if (!get(index))
@@ -289,6 +296,7 @@ class DynamicBitset
 
         // Grow (zero-filled) or shrink storage to the new word count.
         mWords.resize(newWords, uint64_t{0});
+        mSize = newCapacity;
     }
 
     // Returns true if any bit in [start, end) is set.
@@ -329,6 +337,7 @@ class DynamicBitset
 private:
     std::vector<uint64_t> mWords;
     size_t mNumSetBits;
+    size_t mSize; // logical bit capacity (mWords holds divUp(mSize, 64) words)
 };
 
 // ---------------------------------------------------------------------------
diff --git a/cpp/tensorrt_llm/nanobind/batch_manager/kvCacheManagerV2.cpp b/cpp/tensorrt_llm/nanobind/batch_manager/kvCacheManagerV2.cpp
index 62faf997cf1b..767db2b08c4c 100644
--- a/cpp/tensorrt_llm/nanobind/batch_manager/kvCacheManagerV2.cpp
+++ b/cpp/tensorrt_llm/nanobind/batch_manager/kvCacheManagerV2.cpp
@@ -51,6 +51,7 @@
 #include <sstream>
 #include <stdexcept>
 #include <string>
+#include <type_traits>
 #include <unordered_map>
 #include <unordered_set>
 #include <utility>
@@ -61,11 +62,14 @@ namespace kv = tensorrt_llm::batch_manager::kv_cache_manager_v2;
 namespace tensorrt_llm::nanobind::batch_manager
 {
 
-// Helper: convert a Python iterable of int|bytes to vector<TokenIdExt>.
-// nanobind's variant caster can't auto-convert bytes → DigestToken.
-static std::vector<kv::TokenIdExt> castTokenIterable(nb::handle tokens)
+// Helper: convert a Python iterable of int|bytes to a token vector, and report
+// whether it is digest-free (knownNoDigest). A normal int becomes a 31-bit token id
+// (range-checked); 32 bytes become a digest. The digest-free flag is computed for free
+// here while iterating, feeding the hashing fast path. Python contract: `int | bytes(32)`.
+static std::pair<std::vector<kv::TokenIdExt>, bool> castTokenIterable(nb::handle tokens)
 {
     std::vector<kv::TokenIdExt> vec;
+    bool knownNoDigest = true;
     for (auto item : nb::cast<nb::iterable>(tokens))
     {
         if (nb::isinstance<nb::bytes>(item))
@@ -75,16 +79,22 @@ static std::vector<kv::TokenIdExt> castTokenIterable(nb::handle tokens)
             {
                 throw std::invalid_argument("Token bytes must have length kDIGEST_LEN");
             }
-            kv::Digest d;
-            std::memcpy(d.data(), b.c_str(), kv::kDIGEST_LEN);
-            vec.emplace_back(kv::DigestToken(d));
+            kv::Digest digest;
+            std::memcpy(digest.data(), b.c_str(), kv::kDIGEST_LEN);
+            vec.emplace_back(digest);
+            knownNoDigest = false;
         }
         else
         {
-            vec.emplace_back(nb::cast<kv::TokenId>(item));
+            auto const tokenId = nb::cast<int64_t>(item);
+            if (tokenId < 0 || tokenId > static_cast<int64_t>(kv::TokenIdExt::kMaxValue))
+            {
+                throw std::invalid_argument("Token id out of range [0, 2^31)");
+            }
+            vec.emplace_back(static_cast<kv::TokenId>(tokenId));
         }
     }
-    return vec;
+    return {std::move(vec), knownNoDigest};
 }
 
 static kv::TypedVec<kv::PoolIndex, size_t> typedPoolSizeList(std::vector<size_t> const& slotSizeList)
@@ -165,14 +175,14 @@ static nb::list tokenList(std::vector<kv::TokenIdExt> const& tokens)
     nb::list result;
     for (auto const& tok : tokens)
     {
-        if (auto* id = std::get_if<kv::TokenId>(&tok))
+        if (!tok.isDigest())
         {
-            result.append(*id);
+            result.append(tok.tokenId());
         }
         else
         {
-            auto const& d = std::get<kv::DigestToken>(tok);
-            result.append(nb::bytes(reinterpret_cast<char const*>(d.data()), d.size()));
+            auto const& digest = tok.digest();
+            result.append(nb::bytes(reinterpret_cast<char const*>(digest.data()), digest.size()));
         }
     }
     return result;
@@ -499,6 +509,52 @@ static kv::ReuseScope castReuseScope(nb::object reuseScope)
         "reuse_scope must be None, ReuseScope, an int lora_task_id, or an object with lora_id and salt");
 }
 
+// Lazy Python iterator over a token sequence's blockchain keys. Each __next__ pulls
+// one key from the C++ generator (one SHA-256 hash), so a caller that stops early
+// (e.g. on the first cache-key mismatch) skips the remaining hashing. Yields
+// (token_block, key) pairs matching Python's sequence_to_blockchain_keys: root
+// ([], reuseScope digest) first, then one per tokensPerBlock chunk.
+class BlockchainKeyIterator
+{
+public:
+    BlockchainKeyIterator(
+        int tokensPerBlock, kv::ReuseScope const& reuseScope, std::vector<kv::TokenIdExt> tokens, bool knownNoDigest)
+        : mTokens(std::move(tokens))
+        , mGen(kv::sequenceToBlockchainKeys(tokensPerBlock, reuseScope, mTokens.data(), mTokens.size(), knownNoDigest))
+    {
+    }
+
+    nb::object next()
+    {
+        std::optional<kv::BlockchainKeyStep> const step = mGen();
+        if (!step)
+        {
+            throw nb::stop_iteration();
+        }
+        nb::bytes keyBytes(reinterpret_cast<char const*>(step->key.data()), step->key.size());
+        // The step carries its token range, so the root naturally yields the empty [0,0) block.
+        std::vector<kv::TokenIdExt> block(mTokens.begin() + static_cast<ptrdiff_t>(step->tokens.beg),
+            mTokens.begin() + static_cast<ptrdiff_t>(step->tokens.end));
+        return nb::make_tuple(tokenList(block), std::move(keyBytes));
+    }
+
+private:
+    // Concrete generator (closure) type — sequenceToBlockchainKeys's return type is
+    // fixed, so decltype deduces it directly, avoiding std::function's type erasure and
+    // per-iterator heap allocation (the closure captures a 32-byte key, over the SBO limit).
+    using KeyGen = decltype(kv::sequenceToBlockchainKeys(std::declval<int>(), std::declval<kv::ReuseScope>(),
+        std::declval<kv::TokenIdExt const*>(), std::declval<size_t>(), std::declval<bool>()));
+
+    std::vector<kv::TokenIdExt> mTokens; // owned; mGen holds a stable pointer into it
+    KeyGen mGen;                         // declared after mTokens (init order)
+};
+
+// nanobind move-constructs the iterator from the factory return; that must stay valid,
+// since mGen captures mTokens.data() and vector-move preserves the buffer address. Adding
+// a non-movable member would silently force a copy and dangle that pointer — forbid it.
+static_assert(std::is_move_constructible_v<BlockchainKeyIterator>,
+    "BlockchainKeyIterator must be move-constructible (mGen captures mTokens.data())");
+
 void KvCacheManagerV2Bindings::initBindings(nb::module_& m)
 {
     // Export the C++ debug mode as an immutable Python bool snapshot.
@@ -1337,7 +1393,8 @@ void KvCacheManagerV2Bindings::initBindings(nb::module_& m)
                 nb::list layers, float maxUtilForResume, bool enablePartialReuse,
                 std::optional<kv::BatchDesc> typicalStep, std::vector<kv::BatchDesc> constraints,
                 std::optional<std::vector<float>> initialPoolRatio,
-                std::optional<kv::SwaScratchReuseConfig> swaScratchReuse, bool commitMinSnapshot, bool enableStats)
+                std::optional<kv::SwaScratchReuseConfig> swaScratchReuse, bool commitMinSnapshot, bool enableStats,
+                bool textOnly)
             {
                 new (cfg) kv::KVCacheManagerConfig();
                 cfg->tokensPerBlock = tokensPerBlock;
@@ -1358,6 +1415,7 @@ void KvCacheManagerV2Bindings::initBindings(nb::module_& m)
                 cfg->swaScratchReuse = std::move(swaScratchReuse);
                 cfg->commitMinSnapshot = commitMinSnapshot;
                 cfg->enableStats = enableStats;
+                cfg->textOnly = textOnly;
                 // Mirror Python's __post_init__: validate at construction. Config-integrity
                 // failures raise AssertionError (translated below).
                 cfg->validate();
@@ -1366,7 +1424,7 @@ void KvCacheManagerV2Bindings::initBindings(nb::module_& m)
             nb::arg("max_util_for_resume") = 0.97f, nb::arg("enable_partial_reuse") = true,
             nb::arg("typical_step") = std::nullopt, nb::arg("constraints") = std::vector<kv::BatchDesc>{},
             nb::arg("initial_pool_ratio").none() = std::nullopt, nb::arg("swa_scratch_reuse").none() = std::nullopt,
-            nb::arg("commit_min_snapshot") = false, nb::arg("enable_stats") = true)
+            nb::arg("commit_min_snapshot") = false, nb::arg("enable_stats") = true, nb::arg("text_only") = false)
         .def_rw("tokens_per_block", &kv::KVCacheManagerConfig::tokensPerBlock)
         .def_rw("cache_tiers", &kv::KVCacheManagerConfig::cacheTiers)
         .def_rw("layers", &kv::KVCacheManagerConfig::layers)
@@ -1378,6 +1436,7 @@ void KvCacheManagerV2Bindings::initBindings(nb::module_& m)
         .def_rw("swa_scratch_reuse", &kv::KVCacheManagerConfig::swaScratchReuse)
         .def_rw("commit_min_snapshot", &kv::KVCacheManagerConfig::commitMinSnapshot)
         .def_rw("enable_stats", &kv::KVCacheManagerConfig::enableStats)
+        .def_rw("text_only", &kv::KVCacheManagerConfig::textOnly)
         .def_prop_ro("enable_swa_scratch_reuse", &kv::KVCacheManagerConfig::enableSwaScratchReuse)
         .def("validate", &kv::KVCacheManagerConfig::validate) DEF_COPY(kv::KVCacheManagerConfig);
 
@@ -1417,7 +1476,14 @@ void KvCacheManagerV2Bindings::initBindings(nb::module_& m)
             "commit",
             [](kv::KvCache& self, nb::object acceptedInputTokens, nb::object beamSearchIndices, bool isEnd)
             {
-                auto vec = castTokenIterable(acceptedInputTokens);
+                // commit() sources knownNoDigest from the KvCache's text_only flag. Guard
+                // that claim against the actual tokens: a text_only sequence committing a
+                // digest would silently corrupt the block-key hash. Free today (the flag is
+                // a byproduct of building the vector); revisit if C++/Python conversion
+                // becomes zero-copy and this becomes a dedicated scan.
+                auto [vec, knownNoDigest] = castTokenIterable(acceptedInputTokens);
+                TLLM_CHECK_WITH_INFO(!(self.textOnly() && !knownNoDigest),
+                    "commit() received a digest token on a text_only sequence — hashing would be corrupted");
                 if (!beamSearchIndices.is_none())
                 {
                     PyErr_SetString(PyExc_AssertionError, "beam_search_indices must be None");
@@ -1464,6 +1530,8 @@ void KvCacheManagerV2Bindings::initBindings(nb::module_& m)
         .def_prop_ro("has_scratch_slots", &kv::KvCache::hasScratchSlots)
         .def_prop_rw("enable_swa_scratch_reuse", &kv::KvCache::isSwaScratchReuseEnabled,
             [](kv::KvCache& self, bool enable) { self.setEnableSwaScratchReuse(enable); })
+        .def_prop_rw(
+            "text_only", &kv::KvCache::textOnly, [](kv::KvCache& self, bool textOnly) { self.setTextOnly(textOnly); })
         .def("supports_index_mode", &kv::KvCache::supportsIndexMode, nb::arg("mode"))
         .def_prop_ro("status", [](kv::KvCache const& kvc) { return kvc.status(); })
         .def_prop_ro("is_active", &kv::KvCache::isActive)
@@ -1656,12 +1724,12 @@ void KvCacheManagerV2Bindings::initBindings(nb::module_& m)
         [](kv::KvCacheManager& manager, nb::object reuseScope, nb::object tokens, int lcId, bool enablePartial)
         {
             auto rs = castReuseScope(reuseScope);
-            auto vec = castTokenIterable(tokens);
+            auto [vec, knownNoDigest] = castTokenIterable(tokens);
             int numTokens = 0;
             std::vector<std::optional<std::pair<int, int>>> pages;
             {
                 nb::gil_scoped_release release;
-                auto matchResult = manager.radixTree().match(rs, vec, enablePartial);
+                auto matchResult = manager.radixTree().match(rs, vec, knownNoDigest, enablePartial);
                 numTokens = matchResult.numTokens;
                 kv::LifeCycleId lc{lcId};
                 pages.reserve(matchResult.blocks.stdSize());
@@ -1792,14 +1860,20 @@ void KvCacheManagerV2Bindings::initBindings(nb::module_& m)
             "create_kv_cache",
             [](std::shared_ptr<kv::KvCacheManager> self, nb::object reuseScopeObj, nb::object inputTokens,
                 std::optional<kv::RequestIdType> id, nb::object customPriorityCallback,
-                std::optional<int> expectedPromptLength)
+                std::optional<int> expectedPromptLength, std::optional<bool> textOnly)
             {
                 kv::ReuseScope reuseScope = castReuseScope(std::move(reuseScopeObj));
                 std::vector<kv::TokenIdExt> tokens;
                 bool const hasInputTokens = !inputTokens.is_none();
-                if (!inputTokens.is_none())
+                if (hasInputTokens)
                 {
-                    tokens = castTokenIterable(inputTokens);
+                    bool knownNoDigest = true;
+                    std::tie(tokens, knownNoDigest) = castTokenIterable(inputTokens);
+                    // Guard the text_only claim against the actual input tokens (see commit()).
+                    bool const resolvedTextOnly = textOnly.value_or(self->textOnly());
+                    TLLM_CHECK_WITH_INFO(!(resolvedTextOnly && !knownNoDigest),
+                        "create_kv_cache received digest input_tokens on a text_only sequence — hashing would be "
+                        "corrupted");
                 }
                 if (!expectedPromptLength.has_value() && hasInputTokens)
                 {
@@ -1808,22 +1882,24 @@ void KvCacheManagerV2Bindings::initBindings(nb::module_& m)
                 kv::KvCache::PriorityCb priorityCb = castPriorityCallback(*self, std::move(customPriorityCallback));
                 nb::gil_scoped_release release;
                 return self->createKvCache(
-                    std::move(reuseScope), tokens, id, std::move(priorityCb), expectedPromptLength);
+                    std::move(reuseScope), tokens, id, std::move(priorityCb), expectedPromptLength, textOnly);
             },
             nb::arg("reuse_scope") = nb::none(), nb::arg("input_tokens") = nb::none(), nb::arg("id") = std::nullopt,
-            nb::arg("custom_priority_callback") = nb::none(), nb::arg("expected_prompt_length") = std::nullopt)
+            nb::arg("custom_priority_callback") = nb::none(), nb::arg("expected_prompt_length") = std::nullopt,
+            nb::arg("text_only") = std::nullopt)
         .def(
             "probe_reuse",
             [](std::shared_ptr<kv::KvCacheManager> self, nb::object reuseScopeObj, nb::object inputTokens)
             {
                 kv::ReuseScope reuseScope = castReuseScope(std::move(reuseScopeObj));
                 std::vector<kv::TokenIdExt> tokens;
+                bool knownNoDigest = true;
                 if (!inputTokens.is_none())
                 {
-                    tokens = castTokenIterable(inputTokens);
+                    std::tie(tokens, knownNoDigest) = castTokenIterable(inputTokens);
                 }
                 nb::gil_scoped_release release;
-                return self->probeReuse(std::move(reuseScope), tokens);
+                return self->probeReuse(std::move(reuseScope), tokens, knownNoDigest);
             },
             nb::arg("reuse_scope") = nb::none(), nb::arg("input_tokens") = nb::none())
         .def("get_mem_pool_base_address", &kv::KvCacheManager::getMemPoolBaseAddress, nb::arg("layer_id"),
@@ -1934,6 +2010,29 @@ void KvCacheManagerV2Bindings::initBindings(nb::module_& m)
                 return self(baseIndices, indexMode, scratch);
             },
             nb::arg("base_indices"), nb::arg("index_mode") = nb::none(), nb::arg("scratch") = nb::none());
+
+    m.def(
+        "gen_multimodal_cache_key_tokens",
+        [](int idOffset, nb::bytes multiModalDataDigest, int numTokens, int tokenOffset)
+        {
+            auto const* first = reinterpret_cast<uint8_t const*>(multiModalDataDigest.c_str());
+            std::vector<uint8_t> digest(first, first + nb::len(multiModalDataDigest));
+            return tokenList(kv::genMultimodalCacheKeyTokens(idOffset, digest, numTokens, tokenOffset));
+        },
+        nb::arg("id_offset"), nb::arg("multi_modal_data_digest"), nb::arg("num_tokens"), nb::arg("token_offset") = 0);
+    // Lazy iterator yielding (token_block, key) pairs; hashes one block per __next__.
+    nb::class_<BlockchainKeyIterator>(m, "_BlockchainKeyIterator")
+        .def("__iter__", [](nb::handle self) { return self; })
+        .def("__next__", &BlockchainKeyIterator::next);
+    m.def(
+        "sequence_to_blockchain_keys",
+        [](int tokensPerBlock, nb::object reuseScopeObj, nb::object tokensObj)
+        {
+            auto const rs = castReuseScope(std::move(reuseScopeObj));
+            auto [vec, knownNoDigest] = castTokenIterable(tokensObj);
+            return BlockchainKeyIterator(tokensPerBlock, rs, std::move(vec), knownNoDigest);
+        },
+        nb::arg("tokens_per_block"), nb::arg("reuse_scope"), nb::arg("tokens"));
 }
 
 } // namespace tensorrt_llm::nanobind::batch_manager
diff --git a/cpp/tests/unit_tests/batch_manager/CMakeLists.txt b/cpp/tests/unit_tests/batch_manager/CMakeLists.txt
index 02f214986712..420d07dd543e 100644
--- a/cpp/tests/unit_tests/batch_manager/CMakeLists.txt
+++ b/cpp/tests/unit_tests/batch_manager/CMakeLists.txt
@@ -37,6 +37,11 @@ target_include_directories(
   kvCacheManagerV2StatsTest
   PRIVATE ${PROJECT_SOURCE_DIR}/tensorrt_llm/batch_manager
           ${PROJECT_SOURCE_DIR}/tensorrt_llm/common/sha256)
+add_gtest(kvCacheManagerV2DigestPoolTest kvCacheManagerV2DigestPoolTest.cpp)
+target_include_directories(
+  kvCacheManagerV2DigestPoolTest
+  PRIVATE ${PROJECT_SOURCE_DIR}/tensorrt_llm/batch_manager
+          ${PROJECT_SOURCE_DIR}/tensorrt_llm/common/sha256)
 add_gtest(kvCacheUtilsTest kvCacheUtilsTest.cpp)
 add_gtest(llmRequestTest llmRequestTest.cpp)
 add_gtest(microBatchSchedulerTest microBatchSchedulerTest.cpp)
diff --git a/cpp/tests/unit_tests/batch_manager/kvCacheManagerV2DigestPoolTest.cpp b/cpp/tests/unit_tests/batch_manager/kvCacheManagerV2DigestPoolTest.cpp
new file mode 100644
index 000000000000..9b5ac0fbfda7
--- /dev/null
+++ b/cpp/tests/unit_tests/batch_manager/kvCacheManagerV2DigestPoolTest.cpp
@@ -0,0 +1,215 @@
+/*
+ * SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+ * SPDX-License-Identifier: Apache-2.0
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include "tensorrt_llm/batch_manager/kv_cache_manager_v2/blockRadixTree.h"
+#include "tensorrt_llm/batch_manager/kv_cache_manager_v2/common.h"
+#include "tensorrt_llm/batch_manager/kv_cache_manager_v2/tokenIdExt.h"
+
+#include <gtest/gtest.h>
+
+#include <algorithm>
+#include <array>
+#include <cstddef>
+#include <cstdint>
+#include <cstring>
+#include <vector>
+
+namespace
+{
+using namespace tensorrt_llm::batch_manager::kv_cache_manager_v2;
+
+// Build a deterministic 32-byte digest from a seed byte.
+Digest makeDigest(std::byte seed)
+{
+    Digest digest;
+    for (size_t i = 0; i < kDIGEST_LEN; ++i)
+    {
+        digest[i] = static_cast<std::byte>(static_cast<uint8_t>(seed) + static_cast<uint8_t>(i));
+    }
+    return digest;
+}
+
+// Core cache-key-stability guarantee: hashing an all-normal token block in one
+// bulk Write must be bit-identical to the per-element path.
+TEST(DigestPoolTest, BulkHashEqualsPerElementForNormalTokens)
+{
+    std::vector<TokenIdExt> tokens;
+    for (int i = 0; i < 37; ++i) // odd, non-power-of-two count
+    {
+        tokens.emplace_back(TokenId{i * 7 + 1});
+    }
+    ASSERT_FALSE(std::any_of(tokens.begin(), tokens.end(), [](TokenIdExt const& t) { return t.isDigest(); }));
+
+    // Per-element (slow) path.
+    Hasher slow;
+    for (auto const& tok : tokens)
+    {
+        slow.update(tok);
+    }
+
+    // Bulk (fast) path — knownNoDigest=true does one bulk Write, no per-element scan.
+    Hasher fast;
+    fast.update(tokens.data(), tokens.size(), /*knownNoDigest=*/true);
+
+    EXPECT_EQ(slow.digest(), fast.digest());
+}
+
+// Two digests with identical bytes in DISTINCT pool slots must compare equal
+// (equality dereferences the pool) and hash identically. Slots are released when
+// the owning tokens go out of scope.
+TEST(DigestPoolTest, DigestValueEqualityAcrossDistinctSlots)
+{
+    size_t const baseline = detail::digestPoolLiveCount();
+    Digest const bytes = makeDigest(std::byte{0x42});
+
+    TokenIdExt const tokA(bytes);
+    TokenIdExt const tokB(bytes);
+    EXPECT_EQ(detail::digestPoolLiveCount(), baseline + 2); // two distinct slots
+    ASSERT_TRUE(tokA.isDigest());
+    EXPECT_EQ(tokA, tokB);                                  // by-value (pooled) equality
+    EXPECT_NE(tokA, TokenIdExt(TokenId{5}));                // digest != normal
+
+    // Hashing the two distinct-slot digests yields the same contribution.
+    Hasher hashA;
+    hashA.update(tokA);
+    Hasher hashB;
+    hashB.update(tokB);
+    EXPECT_EQ(hashA.digest(), hashB.digest());
+}
+
+// A copied digest token clones its slot; both are freed on destruction.
+TEST(DigestPoolTest, CopyDigestTokenClonesSlot)
+{
+    size_t const baseline = detail::digestPoolLiveCount();
+    Digest const bytes = makeDigest(std::byte{0x5A});
+    {
+        TokenIdExt const original(bytes);
+        EXPECT_EQ(detail::digestPoolLiveCount(), baseline + 1);
+        TokenIdExt const copy = original; // clone → second slot
+        EXPECT_EQ(detail::digestPoolLiveCount(), baseline + 2);
+        EXPECT_EQ(original, copy);
+        EXPECT_EQ(copy.digest(), bytes);
+    }
+    EXPECT_EQ(detail::digestPoolLiveCount(), baseline); // both slots freed
+}
+
+// clone-on-copy: the copy owns an independent slot; destroying the original
+// leaves the copy valid, and slots return to the free-list (liveCount delta 0).
+TEST(DigestPoolTest, CloneIndependenceAndFreeReuse)
+{
+    size_t const baseline = detail::digestPoolLiveCount();
+    Digest const bytes = makeDigest(std::byte{0x11});
+
+    {
+        std::vector<TokenIdExt> original;
+        original.emplace_back(TokenId{1});
+        original.emplace_back(bytes);
+        original.emplace_back(TokenId{2});
+        ASSERT_TRUE(std::any_of(original.begin(), original.end(), [](TokenIdExt const& t) { return t.isDigest(); }));
+        EXPECT_EQ(detail::digestPoolLiveCount(), baseline + 1);
+
+        std::vector<TokenIdExt> copy(original); // deep clone → a second slot
+        EXPECT_EQ(detail::digestPoolLiveCount(), baseline + 2);
+
+        // Destroy the original; the copy's digest bytes must remain valid.
+        {
+            std::vector<TokenIdExt> dying(std::move(original));
+        } // dying (holding original's slot) destructs here
+        EXPECT_EQ(detail::digestPoolLiveCount(), baseline + 1);
+
+        ASSERT_EQ(copy.size(), 3U);
+        ASSERT_TRUE(copy[1].isDigest());
+        EXPECT_EQ(copy[1].digest(), bytes);
+    }                                                   // copy destructs
+
+    EXPECT_EQ(detail::digestPoolLiveCount(), baseline); // all slots freed / reused
+}
+
+// Exercise front-packed allocation + tail-shrink reclamation: allocate well past
+// the shrink threshold, release, and confirm content integrity survives the
+// intervening deque shrinks and that every slot is reclaimed.
+TEST(DigestPoolTest, FrontPackAndTailShrinkChurn)
+{
+    size_t const baseline = detail::digestPoolLiveCount();
+
+    auto distinctDigest = [](int seed)
+    {
+        Digest digest;
+        for (size_t i = 0; i < kDIGEST_LEN; ++i)
+        {
+            digest[i] = static_cast<std::byte>((seed + static_cast<int>(i)) & 0xFF);
+        }
+        // Embed the full seed so digests are distinct beyond 256 entries.
+        std::memcpy(digest.data(), &seed, sizeof(seed));
+        return digest;
+    };
+
+    int const count = 600; // > kSlackHigh (256), forces growth then shrink
+    std::vector<TokenIdExt> tokens;
+    tokens.reserve(count);
+    for (int i = 0; i < count; ++i)
+    {
+        tokens.emplace_back(distinctDigest(i));
+    }
+    EXPECT_EQ(detail::digestPoolLiveCount(), baseline + static_cast<size_t>(count));
+    for (int i = 0; i < count; ++i)
+    {
+        EXPECT_EQ(tokens[i].digest(), distinctDigest(i)) << "content at " << i;
+    }
+
+    // Release the first half (front slots), then confirm the survivors are intact
+    // after the shrink churn triggered by freeing.
+    tokens.erase(tokens.begin(), tokens.begin() + count / 2);
+    EXPECT_EQ(detail::digestPoolLiveCount(), baseline + static_cast<size_t>(count - count / 2));
+    for (size_t i = 0; i < tokens.size(); ++i)
+    {
+        EXPECT_EQ(tokens[i].digest(), distinctDigest(static_cast<int>(i) + count / 2)) << "survivor at " << i;
+    }
+
+    // A fresh allocation must front-pack into a reclaimed low slot.
+    TokenIdExt const refill(distinctDigest(9999));
+    EXPECT_EQ(refill.digest(), distinctDigest(9999));
+
+    tokens.clear();
+    // `refill` still alive here.
+    EXPECT_EQ(detail::digestPoolLiveCount(), baseline + 1);
+}
+
+// A block containing a sparse digest still hashes deterministically; the
+// per-slice fast path applies to the all-text portion.
+TEST(DigestPoolTest, MixedBlockHashesDeterministically)
+{
+    auto build = [](Digest const& mm)
+    {
+        std::vector<TokenIdExt> tokens;
+        tokens.emplace_back(TokenId{10});
+        tokens.emplace_back(mm);
+        tokens.emplace_back(TokenId{20});
+        tokens.emplace_back(TokenId{30});
+        return tokens;
+    };
+    Digest const mm = makeDigest(std::byte{0x7E});
+    std::vector<TokenIdExt> const a = build(mm);
+    std::vector<TokenIdExt> const b = build(mm); // distinct slots, identical content
+
+    Hasher ha;
+    ha.update(a.data(), a.size());
+    Hasher hb;
+    hb.update(b.data(), b.size());
+    EXPECT_EQ(ha.digest(), hb.digest());
+}
+} // namespace
diff --git a/cpp/tests/unit_tests/batch_manager/kvCacheManagerV2StatsTest.cpp b/cpp/tests/unit_tests/batch_manager/kvCacheManagerV2StatsTest.cpp
index 2f7169638d29..116697230bc3 100644
--- a/cpp/tests/unit_tests/batch_manager/kvCacheManagerV2StatsTest.cpp
+++ b/cpp/tests/unit_tests/batch_manager/kvCacheManagerV2StatsTest.cpp
@@ -219,7 +219,7 @@ TEST(KvCacheManagerV2StatsTest, PeakBlockStatsResetStartsNextIntervalFromCurrent
         {
             tokens.emplace_back(TokenId{token++});
         }
-        auto block = addOrGetExistingBlock(previous, LifeCycleId{1}, std::move(tokens));
+        auto block = addOrGetExistingBlock(previous, std::move(tokens), /*knownNoDigest=*/true);
         auto page = makeShared<CommittedPage>(
             &storage, block, lifeCycle, kGpuLevel, static_cast<int>(block->tokens.size()), kPriorityDefault);
         page->setSlot(slot);
@@ -311,7 +311,7 @@ TEST(KvCacheManagerV2StatsTest, MigrationAndLastTierDropRecordersReceiveExactPag
             {
                 tokens.emplace_back(TokenId{tokenBase++});
             }
-            auto block = addOrGetExistingBlock(previous, LifeCycleId{1}, std::move(tokens));
+            auto block = addOrGetExistingBlock(previous, std::move(tokens), /*knownNoDigest=*/true);
             auto page = makeShared<CommittedPage>(
                 &storage, block, lifeCycle, kGpuLevel, static_cast<int>(block->tokens.size()), kPriorityDefault);
             page->setSlot(slot);
diff --git a/tensorrt_llm/runtime/kv_cache_manager_v2/AGENTS.md b/tensorrt_llm/runtime/kv_cache_manager_v2/AGENTS.md
index c9a096a03f5a..278fc3ee4872 100644
--- a/tensorrt_llm/runtime/kv_cache_manager_v2/AGENTS.md
+++ b/tensorrt_llm/runtime/kv_cache_manager_v2/AGENTS.md
@@ -71,7 +71,7 @@ The test file uses `find_spec("kv_cache_manager_v2")` to detect whether the pack
 
 6. **`_life_cycle_registry.py`** — Maps `LayerGroupId`→`LifeCycleId`. Each layer group has either `AttnLifeCycle` (with optional sliding window + sink tokens) or `SsmLifeCycle`. Controls which blocks are "stale" and eligible for eviction.
 
-7. **`_block_radix_tree.py`** — Radix tree for prefix sharing across sequences. Blocks store pages and token IDs. Supports multi-modal tokens via `gen_multi_modal_tokens`.
+7. **`_block_radix_tree.py`** — Radix tree for prefix sharing across sequences. Blocks store pages and token IDs. Supports multi-modal tokens via `gen_multimodal_cache_key_tokens`.
 
 8. **`_eviction_controller/`** — Decides which pages to evict when memory is low, per cache level.
 
diff --git a/tensorrt_llm/runtime/kv_cache_manager_v2/__init__.py b/tensorrt_llm/runtime/kv_cache_manager_v2/__init__.py
index 7b9322b1f63c..51d5ea079a48 100644
--- a/tensorrt_llm/runtime/kv_cache_manager_v2/__init__.py
+++ b/tensorrt_llm/runtime/kv_cache_manager_v2/__init__.py
@@ -24,8 +24,8 @@
 
 if _BACKEND == "python":
     from . import rawref  # noqa: F401
-    from ._block_radix_tree import ReuseScope  # noqa: F401
-    from ._cache_key import (  # noqa: F401
+    from ._block_radix_tree import (  # noqa: F401
+        ReuseScope,
         gen_multimodal_cache_key_tokens,
         sequence_to_blockchain_keys,
     )
@@ -194,6 +194,7 @@ class _KVCacheManagerConfigFieldSpec:
         swa_scratch_reuse: object = None
         commit_min_snapshot: bool = False
         enable_stats: bool = True
+        text_only: bool = False
 
     KVCacheManagerConfig.__dataclass_fields__ = _KVCacheManagerConfigFieldSpec.__dataclass_fields__
     del _KVCacheManagerConfigFieldSpec, _dataclasses
@@ -275,10 +276,8 @@ class PageIndexMode(int):
         SHARED = 0
         PER_LAYER = 1
 
-    from ._cache_key import (  # noqa: F401
-        gen_multimodal_cache_key_tokens,
-        sequence_to_blockchain_keys,
-    )
+    gen_multimodal_cache_key_tokens = _cpp.gen_multimodal_cache_key_tokens
+    sequence_to_blockchain_keys = _cpp.sequence_to_blockchain_keys
 
     def exact_div(x: int, y: int) -> int:
         assert x % y == 0
diff --git a/tensorrt_llm/runtime/kv_cache_manager_v2/__init__.pyi b/tensorrt_llm/runtime/kv_cache_manager_v2/__init__.pyi
index 506290e52586..c0d1a989c09b 100644
--- a/tensorrt_llm/runtime/kv_cache_manager_v2/__init__.pyi
+++ b/tensorrt_llm/runtime/kv_cache_manager_v2/__init__.pyi
@@ -191,6 +191,7 @@ class KVCacheManagerConfig:
     swa_scratch_reuse: SwaScratchReuseConfig | None = None
     commit_min_snapshot: bool = False
     enable_stats: bool = True
+    text_only: bool = False
     @property
     def enable_swa_scratch_reuse(self) -> bool: ...
 
@@ -287,7 +288,7 @@ class KVCacheEventManager:
     def flush_iteration_events(self) -> None: ...
     def get_latest_events(self, timeout_ms: float | None = None) -> list[KVCacheEvent]: ...
 
-# From _cache_key.py
+# Backend-neutral key builders (native C++ under the C++ backend, pure-Python otherwise).
 def gen_multimodal_cache_key_tokens(
     id_offset: int,
     multi_modal_data_digest: bytes,
@@ -318,6 +319,8 @@ class _KVCache:
         reuse_match: Any | None,
         id: Any,
         custom_priority_callback: Callable[[int, Any], Priority],
+        expected_prompt_length: int | None = None,
+        text_only: bool | None = None,
     ) -> None: ...
     def set_base_page_index_buf(
         self, beam_idx: BeamIndex, layer_group_id: LayerGroupId, buf: memoryview | None
@@ -381,6 +384,10 @@ class _KVCache:
     def enable_swa_scratch_reuse(self) -> bool: ...
     @enable_swa_scratch_reuse.setter
     def enable_swa_scratch_reuse(self, enable: bool) -> None: ...
+    @property
+    def text_only(self) -> bool: ...
+    @text_only.setter
+    def text_only(self, text_only: bool) -> None: ...
     def supports_index_mode(self, mode: PageIndexMode) -> bool: ...
     @property
     def status(self) -> _Status: ...
@@ -492,6 +499,7 @@ class KVCacheManager:
         id: Any = None,
         custom_priority_callback: Callable[[int, Any], Priority] = ...,
         expected_prompt_length: int | None = None,
+        text_only: bool | None = None,
     ) -> _KVCache: ...
     def probe_reuse(
         self,
diff --git a/tensorrt_llm/runtime/kv_cache_manager_v2/_block_radix_tree.py b/tensorrt_llm/runtime/kv_cache_manager_v2/_block_radix_tree.py
index 66e225ee63ba..e1a42ed372bb 100644
--- a/tensorrt_llm/runtime/kv_cache_manager_v2/_block_radix_tree.py
+++ b/tensorrt_llm/runtime/kv_cache_manager_v2/_block_radix_tree.py
@@ -14,18 +14,13 @@
 # limitations under the License.
 
 from itertools import chain
-from typing import TYPE_CHECKING, Iterator, NamedTuple, Sequence, TypeVar, cast
+import hashlib
+import itertools
+from array import array
+from typing import TYPE_CHECKING, Iterable, Iterator, NamedTuple, Sequence, TypeVar, cast
 
 from . import rawref
-from ._cache_key import (  # noqa: F401
-    BlockKey,
-    Hasher,
-    TokenBlock,
-    gen_multimodal_cache_key_tokens,
-    reuse_scope_to_bytes,
-    sequence_to_blockchain_keys,
-)
-from ._common import NDEBUG, BlockOrdinal, PageStatus, TokenIdExt
+from ._common import NDEBUG, BlockOrdinal, PageStatus, TokenId, TokenIdExt
 from ._life_cycle_registry import AttnLifeCycle, LifeCycle, LifeCycleId, LifeCycleRegistry
 from ._utils import TypedIndexList, filled_list, map_optional, typed_range, unwrap_rawref
 
@@ -34,6 +29,116 @@
     from ._page import CommittedPage
 
 
+BlockKey = bytes
+TokenBlock = list[TokenIdExt]
+
+
+# id_offset is usually vocab_size. Backend-neutral (depends only on _common); the
+# C++ backend exposes a native gen_multimodal_cache_key_tokens via nanobind instead.
+def gen_multimodal_cache_key_tokens(
+    id_offset: int, multi_modal_data_digest: bytes, num_tokens: int, token_offset: int = 0
+) -> list[TokenIdExt]:
+    """Create synthetic tokens used only when building multimodal KV-cache keys.
+
+    Item-local token 0 carries the content digest; later offsets use deterministic IDs above the vocab.
+    """
+    assert num_tokens > 0
+    assert token_offset >= 0
+    return [
+        multi_modal_data_digest if token_offset + i == 0 else TokenId(id_offset + token_offset + i)
+        for i in range(num_tokens)
+    ]
+
+
+class Hasher:
+    # SECURITY INVARIANT: the block-key hash MUST stay cryptographically
+    # collision-resistant and >= 256-bit. The radix tree is a globally shared,
+    # cross-request/cross-tenant cache index; prefix matches are decided purely by
+    # digest equality with NO re-check of the underlying tokens; and the hashed
+    # input (tokens, the user-supplied cache_salt, multimodal content bytes) is
+    # attacker-influenceable. A collision therefore silently reuses another
+    # request's KV blocks (cross-request corruption / data leak), and cache_salt
+    # tenant isolation relies entirely on this hash's collision resistance. Do NOT
+    # swap in a non-cryptographic hash (xxHash, HighwayHash, ...) or truncate below
+    # 256 bits without first adding a token-content equality check on match. The
+    # C++ backend (blockRadixTree) mirrors this with SHA-256 (CSHA256).
+    __slots__ = "_hasher"
+    _hasher: "hashlib._Hash"
+
+    def __init__(self, data: int | bytes | None | Sequence[int | bytes] = None) -> None:
+        self._hasher = hashlib.sha256()
+        if data is not None:
+            self.update(data)
+
+    # This function is perf-critical. Expect compromised code quality.
+    def update(self, data: int | bytes | Sequence[int | bytes]) -> "Hasher":
+        if type(data) is int:
+            assert NDEBUG or (data >= 0 and data < (1 << 31))
+            self._hasher.update(data.to_bytes(4, "little"))
+        elif type(data) is bytes:
+            self._hasher.update(data)
+        else:
+            # Hash the whole token block in one C call instead of one per token.
+            # array("I", data).tobytes() packs each int as 4 native-endian bytes
+            # (unsigned int); all NVIDIA GPU host platforms (x86_64, aarch64/Grace)
+            # are little-endian so this is byte-identical to the per-token
+            # to_bytes(4, "little") loop AND to the C++ backend's 4-byte TokenIdExt
+            # layout (normal token = little-endian id, high tag bit clear). Falls
+            # back to that loop for multimodal blocks (which contain bytes items).
+            try:
+                self._hasher.update(array("I", data).tobytes())  # type: ignore
+            except (TypeError, OverflowError):
+                for item in data:  # type: ignore
+                    assert (
+                        NDEBUG
+                        or (type(item) is int and (0 <= item < (1 << 31)))
+                        or type(item) is bytes
+                    )
+                    self._hasher.update(item.to_bytes(4, "little") if (type(item) is int) else item)  # type: ignore
+        return self
+
+    @property
+    def digest(self) -> bytes:
+        return self._hasher.digest()
+
+
+def reuse_scope_to_bytes(reuse_scope: Iterable[int | None]) -> bytes:
+    """Serialize a reuse scope to its reuse-namespace bytes.
+
+    Backend-neutral: reads the scope's fields by iteration, so it works for both
+    the pure-Python ``ReuseScope`` NamedTuple and the C++ binding without relying
+    on a ``to_bytes()`` method. The layout mirrors the C++ ``emitReuseScopeBytes``:
+    a mask byte (one bit per field, set when the field is present) followed by one
+    little-endian ``uint64`` per present field (``signed=False``).
+    """
+    values = list(reuse_scope)
+    mask = sum((value is not None) << i for i, value in enumerate(values))
+    ret = mask.to_bytes((len(values) + 7) // 8, "little", signed=False)
+    for value in values:
+        if value is not None:
+            ret += int(value).to_bytes(8, "little", signed=False)
+    return ret
+
+
+def sequence_to_blockchain_keys(
+    tokens_per_block: int, reuse_scope: Iterable[int | None], tokens: Sequence[TokenIdExt]
+) -> Iterator[tuple[TokenBlock, BlockKey]]:
+    """Yield ``(token_block, key)`` pairs seeding a blockchain of KV-cache keys.
+
+    The first pair is the root (``[]``, reuse-scope digest); each subsequent pair
+    hashes one ``tokens_per_block`` chunk on top of the previous digest.
+    """
+    digest = Hasher(reuse_scope_to_bytes(reuse_scope)).digest
+    yield [], digest
+    iterator = iter(tokens)
+    while True:
+        token_block = list(itertools.islice(iterator, tokens_per_block))
+        if not token_block:
+            break
+        digest = Hasher(digest).update(token_block).digest
+        yield token_block, digest
+
+
 class ReuseScope(NamedTuple):
     """Per-request namespace for prefix reuse."""
 
diff --git a/tensorrt_llm/runtime/kv_cache_manager_v2/_cache_key.py b/tensorrt_llm/runtime/kv_cache_manager_v2/_cache_key.py
deleted file mode 100644
index c418c4a9708a..000000000000
--- a/tensorrt_llm/runtime/kv_cache_manager_v2/_cache_key.py
+++ /dev/null
@@ -1,134 +0,0 @@
-# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
-# SPDX-License-Identifier: Apache-2.0
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-"""Backend-neutral helpers for deriving KV-cache reuse keys.
-
-These are the pure-Python reference implementation shared by both the Python
-and C++ backends. They depend only on the light-weight, backend-neutral
-``_common`` module (no CUDA / bindings), so they can be imported and re-exported
-as public API regardless of the active backend.
-"""
-
-import hashlib
-import itertools
-from array import array
-from typing import Iterable, Iterator, Sequence
-
-from ._common import NDEBUG, TokenId, TokenIdExt
-
-BlockKey = bytes
-TokenBlock = list[TokenIdExt]
-
-
-# id_offset is usually vocab_size
-def gen_multimodal_cache_key_tokens(
-    id_offset: int, multi_modal_data_digest: bytes, num_tokens: int, token_offset: int = 0
-) -> list[TokenIdExt]:
-    """Create synthetic tokens used only when building multimodal KV-cache keys.
-
-    Item-local token 0 carries the content digest; later offsets use deterministic IDs above the vocab.
-    """
-    assert num_tokens > 0
-    assert token_offset >= 0
-    return [
-        multi_modal_data_digest if token_offset + i == 0 else TokenId(id_offset + token_offset + i)
-        for i in range(num_tokens)
-    ]
-
-
-class Hasher:
-    # SECURITY INVARIANT: the block-key hash MUST stay cryptographically
-    # collision-resistant and >= 256-bit. The radix tree is a globally shared,
-    # cross-request/cross-tenant cache index; prefix matches are decided purely by
-    # digest equality with NO re-check of the underlying tokens; and the hashed
-    # input (tokens, the user-supplied cache_salt, multimodal content bytes) is
-    # attacker-influenceable. A collision therefore silently reuses another
-    # request's KV blocks (cross-request corruption / data leak), and cache_salt
-    # tenant isolation relies entirely on this hash's collision resistance. Do NOT
-    # swap in a non-cryptographic hash (xxHash, HighwayHash, ...) or truncate below
-    # 256 bits without first adding a token-content equality check on match. The
-    # C++ backend (blockRadixTree) mirrors this with SHA-256 (CSHA256).
-    __slots__ = "_hasher"
-    _hasher: "hashlib._Hash"
-
-    def __init__(self, data: int | bytes | None | Sequence[int | bytes] = None) -> None:
-        self._hasher = hashlib.sha256()
-        if data is not None:
-            self.update(data)
-
-    # This function is perf-critical. Expect compromised code quality.
-    def update(self, data: int | bytes | Sequence[int | bytes]) -> "Hasher":
-        if type(data) is int:
-            assert NDEBUG or (data >= 0 and data < (1 << 64))
-            self._hasher.update(data.to_bytes(8, "little"))
-        elif type(data) is bytes:
-            self._hasher.update(data)
-        else:
-            # Hash the whole token block in one C call instead of one per token.
-            # array("Q", data).tobytes() packs each int as 8 native-endian bytes;
-            # all NVIDIA GPU host platforms (x86_64, aarch64/Grace) are little-endian
-            # so this is byte-identical to the per-token to_bytes(8, "little") loop.
-            # Falls back to that loop for multimodal blocks (which contain bytes items).
-            try:
-                self._hasher.update(array("Q", data).tobytes())  # type: ignore
-            except (TypeError, OverflowError):
-                for item in data:  # type: ignore
-                    assert (
-                        NDEBUG
-                        or (type(item) is int and (0 <= item < (1 << 64)))
-                        or type(item) is bytes
-                    )
-                    self._hasher.update(item.to_bytes(8, "little") if (type(item) is int) else item)  # type: ignore
-        return self
-
-    @property
-    def digest(self) -> bytes:
-        return self._hasher.digest()
-
-
-def reuse_scope_to_bytes(reuse_scope: Iterable[int | None]) -> bytes:
-    """Serialize a reuse scope to its reuse-namespace bytes.
-
-    Backend-neutral: reads the scope's fields by iteration, so it works for both
-    the pure-Python ``ReuseScope`` NamedTuple and the C++ binding without relying
-    on a ``to_bytes()`` method. The layout mirrors the C++ ``emitReuseScopeBytes``:
-    a mask byte (one bit per field, set when the field is present) followed by one
-    little-endian ``uint64`` per present field (``signed=False``).
-    """
-    values = list(reuse_scope)
-    mask = sum((value is not None) << i for i, value in enumerate(values))
-    ret = mask.to_bytes((len(values) + 7) // 8, "little", signed=False)
-    for value in values:
-        if value is not None:
-            ret += int(value).to_bytes(8, "little", signed=False)
-    return ret
-
-
-def sequence_to_blockchain_keys(
-    tokens_per_block: int, reuse_scope: Iterable[int | None], tokens: Sequence[TokenIdExt]
-) -> Iterator[tuple[TokenBlock, BlockKey]]:
-    """Yield ``(token_block, key)`` pairs seeding a blockchain of KV-cache keys.
-
-    The first pair is the root (``[]``, reuse-scope digest); each subsequent pair
-    hashes one ``tokens_per_block`` chunk on top of the previous digest.
-    """
-    digest = Hasher(reuse_scope_to_bytes(reuse_scope)).digest
-    yield [], digest
-    iterator = iter(tokens)
-    while True:
-        token_block = list(itertools.islice(iterator, tokens_per_block))
-        if not token_block:
-            break
-        digest = Hasher(digest).update(token_block).digest
-        yield token_block, digest
diff --git a/tensorrt_llm/runtime/kv_cache_manager_v2/_config.py b/tensorrt_llm/runtime/kv_cache_manager_v2/_config.py
index 8f374e7a37f7..1dde1d8e8cdd 100644
--- a/tensorrt_llm/runtime/kv_cache_manager_v2/_config.py
+++ b/tensorrt_llm/runtime/kv_cache_manager_v2/_config.py
@@ -245,6 +245,16 @@ class KVCacheManagerConfig:
     Collect V2 KV cache allocation, reuse, and transfer statistics.
     """
 
+    text_only: bool = False
+    """
+    Deployment-level guarantee that no request carries multi-modal content, so token
+    sequences never contain digests. A per-_KVCache text_only override may only tighten
+    this (a text-only deployment forbids a request claiming otherwise). Default False.
+
+    (In this pure-Python backend the block hasher has no digest-free fast path, so this
+    flag is carried for API/behavior parity with the C++ backend but changes no hashing.)
+    """
+
     @property
     def enable_swa_scratch_reuse(self) -> bool:
         return self.swa_scratch_reuse is not None
diff --git a/tensorrt_llm/runtime/kv_cache_manager_v2/_core/_kv_cache.py b/tensorrt_llm/runtime/kv_cache_manager_v2/_core/_kv_cache.py
index d71bf3db6acb..8d52a86a1132 100644
--- a/tensorrt_llm/runtime/kv_cache_manager_v2/_core/_kv_cache.py
+++ b/tensorrt_llm/runtime/kv_cache_manager_v2/_core/_kv_cache.py
@@ -246,6 +246,7 @@ class _KVCache:
         "_ssm_blocks",
         "_never_resumed",
         "_enable_swa_scratch_reuse",
+        "_text_only",
         "_scratch_slots",
         "_pending_stats",
         "__rawref__",
@@ -305,6 +306,7 @@ def __init__(
         id: int | None,
         custom_priority_callback: Callable[[BlockOrdinal, LifeCycle], Priority],
         expected_prompt_length: int | None = None,
+        text_only: bool | None = None,
     ):
         self.id = id
         self._manager = manager
@@ -335,6 +337,11 @@ def __init__(
         )
         self._never_resumed = True
         self._enable_swa_scratch_reuse = manager.enable_swa_scratch_reuse
+        if text_only is False and manager.text_only:
+            raise ValueError(
+                "text_only=False is not allowed when the manager is configured text_only=True"
+            )
+        self._text_only = manager.text_only if text_only is None else text_only
         self._scratch_slots = make_typed(
             lambda _: list[ScratchSlotLock](), manager._storage.num_life_cycles
         )
@@ -697,6 +704,25 @@ def enable_swa_scratch_reuse(self, enable: bool) -> None:
         assert not self.has_scratch_slots
         self._enable_swa_scratch_reuse = False
 
+    @property
+    def text_only(self) -> bool:
+        return self._text_only
+
+    @text_only.setter
+    def text_only(self, text_only: bool) -> None:
+        # A text-only deployment is a hard guarantee: a request may not opt out.
+        if not text_only and self.manager.text_only:
+            raise ValueError(
+                "Cannot set text_only=False for a request when the KV cache manager is "
+                "configured text_only=True"
+            )
+        # Claiming text-only is a fast-path claim; verify committed tokens are digest-free.
+        if text_only and any(isinstance(t, bytes) for t in self._committed_tokens):
+            raise ValueError(
+                "Cannot set text_only=True: this sequence has already committed digest tokens"
+            )
+        self._text_only = text_only
+
     def supports_index_mode(self, mode: PageIndexMode) -> bool:
         match mode:
             case PageIndexMode.PER_LAYER:
diff --git a/tensorrt_llm/runtime/kv_cache_manager_v2/_core/_kv_cache_manager.py b/tensorrt_llm/runtime/kv_cache_manager_v2/_core/_kv_cache_manager.py
index 633c2f56c2e2..1a8925c1da75 100644
--- a/tensorrt_llm/runtime/kv_cache_manager_v2/_core/_kv_cache_manager.py
+++ b/tensorrt_llm/runtime/kv_cache_manager_v2/_core/_kv_cache_manager.py
@@ -409,12 +409,16 @@ def create_kv_cache(
         custom_priority_callback: Callable[[BlockOrdinal, LifeCycle], Priority] = lambda _,
         __: PRIORITY_DEFAULT,
         expected_prompt_length: int | None = None,
+        text_only: bool | None = None,
     ) -> _KVCache:
         """
         reuse_scope: namespace to match before matching any tokens.
         custom_priority_callback: takes block index and layer sliding window size, returns priority.
         If priority returned is higher than existing priority for reused blocks, the block priority is updated.
-        expected_prompt_length: optional prompt length hint used to size SWA scratch slots.
+        expected_prompt_length: optional token count marking the prefill->generation
+            boundary; once history_length reaches it, subsequent capacity growth is
+            recorded as generation-phase allocation stats (defaults to len(input_tokens)).
+            Stats-only: does not affect allocation, reuse, or correctness.
         Newly created KV cache is suspended. You need to call resume() with a cuda stream to make it active
         & ready in that stream.
         Returns None if suspended=False and we don't have enough resource.
@@ -437,6 +441,7 @@ def create_kv_cache(
             id,
             custom_priority_callback,
             expected_prompt_length,
+            text_only,
         )
 
     def _match_reuse(
@@ -958,3 +963,7 @@ def init_config(self) -> KVCacheManagerConfig:
     @property
     def commit_min_snapshot(self) -> bool:
         return self.init_config.commit_min_snapshot
+
+    @property
+    def text_only(self) -> bool:
+        return self.init_config.text_only
diff --git a/tensorrt_llm/runtime/kv_cache_manager_v2/_page.py b/tensorrt_llm/runtime/kv_cache_manager_v2/_page.py
index c3cee03270f5..0ace68f53278 100644
--- a/tensorrt_llm/runtime/kv_cache_manager_v2/_page.py
+++ b/tensorrt_llm/runtime/kv_cache_manager_v2/_page.py
@@ -31,7 +31,6 @@
     PageIndex,
     PageStatus,
     Priority,
-    TokenIdExt,
 )
 
 if TYPE_CHECKING:
@@ -132,8 +131,6 @@ class UncommittedPage(Page):
     ordinal: BlockOrdinal
     beam_index: BeamIndex
 
-    tokens: list[TokenIdExt] = field(default_factory=list)
-
     def is_committed(self) -> bool:
         return False
 
diff --git a/tests/unittest/kv_cache_manager_v2_tests/test_kv_cache_manager_v2.py b/tests/unittest/kv_cache_manager_v2_tests/test_kv_cache_manager_v2.py
index 48c10a006fc0..341a34523227 100755
--- a/tests/unittest/kv_cache_manager_v2_tests/test_kv_cache_manager_v2.py
+++ b/tests/unittest/kv_cache_manager_v2_tests/test_kv_cache_manager_v2.py
@@ -4310,14 +4310,16 @@ def _ref_update(seed: bytes, block: "list[int | bytes]") -> bytes:
         h = hashlib.sha256()
         h.update(seed)
         for item in block:
-            h.update(item.to_bytes(8, "little") if type(item) is int else item)
+            # Normal token ids are packed as 4 little-endian bytes (31-bit range),
+            # matching the C++ backend's 4-byte TokenIdExt layout.
+            h.update(item.to_bytes(4, "little") if type(item) is int else item)
         return h.digest()
 
     def test_update_int_block_matches_reference(self) -> None:
         rng = random.Random(123)
         seed = b"\xaa\xbb\xcc"
         for n in (0, 1, 7, 32, 33, 257):
-            block = [rng.randint(0, (1 << 60)) for _ in range(n)]
+            block = [rng.randint(0, (1 << 31) - 1) for _ in range(n)]
             self.assertEqual(
                 Hasher(seed).update(block).digest,
                 self._ref_update(seed, block),
diff --git a/tests/unittest/kv_cache_manager_v2_tests/test_kv_cache_salting.py b/tests/unittest/kv_cache_manager_v2_tests/test_kv_cache_salting.py
index 4c2dc9615686..6d0316f26bbb 100644
--- a/tests/unittest/kv_cache_manager_v2_tests/test_kv_cache_salting.py
+++ b/tests/unittest/kv_cache_manager_v2_tests/test_kv_cache_salting.py
@@ -20,15 +20,21 @@
 from typing import TYPE_CHECKING, cast
 
 if not TYPE_CHECKING and find_spec("kv_cache_manager_v2") is not None:
-    from kv_cache_manager_v2 import TokenId, sequence_to_blockchain_keys
-    from kv_cache_manager_v2._block_radix_tree import Block, BlockRadixTree, ReuseScope
+    from kv_cache_manager_v2 import TokenId
+    from kv_cache_manager_v2._block_radix_tree import (
+        Block,
+        BlockRadixTree,
+        ReuseScope,
+        sequence_to_blockchain_keys,
+    )
     from kv_cache_manager_v2._life_cycle_registry import LifeCycleRegistry
 else:
-    from tensorrt_llm.runtime.kv_cache_manager_v2 import TokenId, sequence_to_blockchain_keys
+    from tensorrt_llm.runtime.kv_cache_manager_v2 import TokenId
     from tensorrt_llm.runtime.kv_cache_manager_v2._block_radix_tree import (
         Block,
         BlockRadixTree,
         ReuseScope,
+        sequence_to_blockchain_keys,
     )
     from tensorrt_llm.runtime.kv_cache_manager_v2._life_cycle_registry import LifeCycleRegistry
 

From 7e87d3372d5e705adc51d1b055ca08a9ce7cae19 Mon Sep 17 00:00:00 2001
From: Yao Yao <lowsfer@users.noreply.github.com>
Date: Mon, 27 Jul 2026 02:30:26 +0000
Subject: [PATCH 03/12] [None][perf] KVv2: zero-copy int32 token ingest on the
 block-reuse hot path

Avoid the per-token Python round-trip when feeding request tokens to the
KVCacheManagerV2 C++ backend for prefix matching and commit. Tokens were copied
to a Python list (get_tokens) then converted back element-by-element (nb::cast
per token) into std::vector<TokenIdExt>.

A digest-free token is bit-identical across int32_t, the 4-byte TokenIdExt, and
the hashed byte stream, so a contiguous int32 buffer is reinterpret_cast to
TokenIdExt const* with no copy and no per-token boxing.

- Core: TokenSpan (= Span<TokenIdExt const>, moved into common.h) + toSpan()
  helper; match/createKvCache/probeReuse/matchReuse/KvCache ctor/_setupForReuse/
  commit take a TokenSpan (dead match(vector) overload removed).
- nanobind: LlmRequest.get_tokens_view(beam) returns a zero-copy read-only int32
  ndarray (reference_internal); withTokens() ingest helper takes the int32 fast
  path (nb::try_cast, convert=false) or the existing per-element fallback for
  multimodal/digest tokens.
- Dispatcher: backend-aware _reuse_token_source() -- the C++ backend gets the
  zero-copy view, the pure-Python backend keeps a plain list (cannot consume
  numpy arrays).
- Test: int32-ndarray ingest hashes identically to the list path (both
  directions); skipped on the Python backend.

Full KVCacheManagerV2 unit suite green on both backends. Egress get_tokens_view
is ~44x faster than get_tokens on a 4K-token prompt (30 us -> 0.7 us), and O(1)
in sequence length.

Signed-off-by: Yao Yao <lowsfer@users.noreply.github.com>
---
 .../kv_cache_manager_v2/blockRadixTree.cpp    |  12 +-
 .../kv_cache_manager_v2/blockRadixTree.h      |   8 +-
 .../kv_cache_manager_v2/common.h              |  61 ++++++
 .../kv_cache_manager_v2/kvCache.cpp           |   6 +-
 .../kv_cache_manager_v2/kvCache.h             |  43 +----
 .../kv_cache_manager_v2/kvCacheManager.cpp    |  15 +-
 .../kv_cache_manager_v2/kvCacheManager.h      |   9 +-
 .../nanobind/batch_manager/bindings.cpp       |  15 ++
 .../batch_manager/kvCacheManagerV2.cpp        | 174 +++++++++++-------
 .../_torch/pyexecutor/kv_cache_manager_v2.py  |  24 ++-
 .../test_kv_cache_manager_v2.py               |  55 ++++++
 11 files changed, 288 insertions(+), 134 deletions(-)

diff --git a/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/blockRadixTree.cpp b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/blockRadixTree.cpp
index 9df8c42ba690..6122f8f3d333 100644
--- a/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/blockRadixTree.cpp
+++ b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/blockRadixTree.cpp
@@ -718,15 +718,15 @@ int numMatchedTokens(std::vector<BlockRadixTree::MatchResult> const& matched, in
 
 } // anonymous namespace
 
-std::vector<BlockRadixTree::MatchResult> BlockRadixTree::matchTokenPath(ReuseScope const& reuseScope,
-    std::vector<TokenIdExt> const& tokens, bool knownNoDigest, bool enablePartialMatch) const
+std::vector<BlockRadixTree::MatchResult> BlockRadixTree::matchTokenPath(
+    ReuseScope const& reuseScope, TokenSpan tokens, bool knownNoDigest, bool enablePartialMatch) const
 {
     drainPendingRootErases();
 
     std::vector<MatchResult> results;
 
     // Lazily compute one key per iteration — no wasted hashing on early miss.
-    auto gen = sequenceToBlockchainKeys(mTokensPerBlock, reuseScope, tokens.data(), tokens.size(), knownNoDigest);
+    auto gen = sequenceToBlockchainKeys(mTokensPerBlock, reuseScope, tokens.begin(), tokens.size(), knownNoDigest);
 
     // First step is the root key (empty token range).
     auto rootStep = gen();
@@ -762,7 +762,7 @@ std::vector<BlockRadixTree::MatchResult> BlockRadixTree::matchTokenPath(ReuseSco
     if (missed && enablePartialMatch)
     {
         auto [best, bestMatch]
-            = findBestPartialMatchInNextNodes(*currentNext, tokens.data() + missedRange.beg, missedRange.length());
+            = findBestPartialMatchInNextNodes(*currentNext, tokens.begin() + missedRange.beg, missedRange.length());
         if (best)
             results.push_back({best, bestMatch});
     }
@@ -863,8 +863,8 @@ std::vector<BlockRadixTree::MatchResult> BlockRadixTree::pruneMatch(std::vector<
     return matched;
 }
 
-BlockRadixTree::ReuseMatch BlockRadixTree::match(ReuseScope const& reuseScope, std::vector<TokenIdExt> const& tokens,
-    bool knownNoDigest, bool enablePartialMatch) const
+BlockRadixTree::ReuseMatch BlockRadixTree::match(
+    ReuseScope const& reuseScope, TokenSpan tokens, bool knownNoDigest, bool enablePartialMatch) const
 {
     auto const matched = pruneMatch(matchTokenPath(reuseScope, tokens, knownNoDigest, enablePartialMatch));
     ReuseMatch result{};
diff --git a/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/blockRadixTree.h b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/blockRadixTree.h
index 7b674db5f038..11f7455fe7c3 100644
--- a/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/blockRadixTree.h
+++ b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/blockRadixTree.h
@@ -372,7 +372,9 @@ class BlockRadixTree
     };
 
     // knownNoDigest: from external text_only knowledge, never a scan (see Hasher::update).
-    ReuseMatch match(ReuseScope const& reuseScope, std::vector<TokenIdExt> const& tokens, bool knownNoDigest = false,
+    // Takes a non-owning TokenSpan so a zero-copy int32 token buffer can be matched without
+    // allocating/copying (the hot path). Callers holding a std::vector pass toSpan(vec).
+    ReuseMatch match(ReuseScope const& reuseScope, TokenSpan tokens, bool knownNoDigest = false,
         bool enablePartialMatch = false) const;
 
     // Clear all cached pages. ~Block() handles excludeFromEviction for DROPPABLE pages.
@@ -410,8 +412,8 @@ class BlockRadixTree
 
 private:
     // knownNoDigest: from external text_only knowledge, never a scan (see Hasher::update).
-    std::vector<MatchResult> matchTokenPath(ReuseScope const& reuseScope, std::vector<TokenIdExt> const& tokens,
-        bool knownNoDigest, bool enablePartialMatch) const;
+    std::vector<MatchResult> matchTokenPath(
+        ReuseScope const& reuseScope, TokenSpan tokens, bool knownNoDigest, bool enablePartialMatch) const;
     std::vector<MatchResult> pruneMatch(std::vector<MatchResult> matched) const;
 
     // Erase any pending empty root blocks from mRoots.
diff --git a/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/common.h b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/common.h
index 8bf1eec60c27..2fc12c04137e 100644
--- a/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/common.h
+++ b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/common.h
@@ -20,15 +20,18 @@
 #include "kv_cache_manager_v2/tokenIdExt.h" // TokenId, Digest, TokenIdExt
 #include "kv_cache_manager_v2/utils/typedIndex.h"
 #include "tensorrt_llm/batch_manager/common.h"
+#include "tensorrt_llm/common/assert.h"
 
 #include <array>
 #include <cstdint>
 #include <cstring>
+#include <limits>
 #include <optional>
 #include <string>
 #include <sys/types.h>
 #include <type_traits>
 #include <variant>
+#include <vector>
 
 namespace tensorrt_llm::batch_manager::kv_cache_manager_v2
 {
@@ -120,6 +123,64 @@ inline constexpr Priority kPriorityDefault = 35;
 // Optional sliding window size (nullopt = no sliding window).
 using SlidingWindowSize = std::optional<int>;
 
+// ---------------------------------------------------------------------------
+// Span<T> — non-owning view into a contiguous buffer (a C++17 stand-in for
+// std::span). Aggregate, so `Span<T>{}` value-initializes to an empty view and
+// `Span<T>{ptr, len}` is a plain brace-init. Supports operator[] for uniform
+// access with std::vector<T>.
+// ---------------------------------------------------------------------------
+template <typename T>
+struct Span
+{
+    T* ptr;
+    int len;
+
+    T& operator[](int idx)
+    {
+        return ptr[idx];
+    }
+
+    T const& operator[](int idx) const
+    {
+        return ptr[idx];
+    }
+
+    int size() const noexcept
+    {
+        return len;
+    }
+
+    T* data() const noexcept
+    {
+        return ptr;
+    }
+
+    T* begin() const noexcept
+    {
+        return ptr;
+    }
+
+    T* end() const noexcept
+    {
+        return ptr + len;
+    }
+};
+
+// Non-owning const Span over a std::vector — for call sites that hold an owning vector but
+// need a Span (e.g. the per-element/multimodal fallback).
+template <typename T>
+inline Span<T const> toSpan(std::vector<T> const& vec) noexcept
+{
+    TLLM_CHECK_DEBUG(vec.size() <= static_cast<size_t>(std::numeric_limits<int>::max()));
+    return Span<T const>{vec.data(), static_cast<int>(vec.size())};
+}
+
+// Non-owning view of a token sequence — a C++17 stand-in for std::span<TokenIdExt const>.
+// Used on the hot ingest path: a digest-free int32 token buffer can be reinterpret_cast to
+// TokenIdExt const* and matched/hashed with no per-token copy. TokenIdExt is 4 bytes and
+// bit-identical to a normal int32 token (see tokenIdExt.h).
+using TokenSpan = Span<TokenIdExt const>;
+
 // ---------------------------------------------------------------------------
 // Address types
 // ---------------------------------------------------------------------------
diff --git a/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/kvCache.cpp b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/kvCache.cpp
index 652fa3f633e6..4fe8b6c41d62 100644
--- a/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/kvCache.cpp
+++ b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/kvCache.cpp
@@ -1717,12 +1717,12 @@ void KvCache::_commitBlock(int ord, bool isLast, bool commitSsm, bool moveSsm)
 // commit
 // ---------------------------------------------------------------------------
 
-void KvCache::commit(std::vector<TokenIdExt> const& tokens, bool isEnd)
+void KvCache::commit(TokenSpan tokens, bool isEnd)
 {
     TLLM_CHECK_DEBUG(mStatus == Status::ACTIVE);
     if (mBeamWidth != BeamIndex{1})
         throw LogicError("Not implemented yet for beam search");
-    if (tokens.empty())
+    if (tokens.size() == 0)
     {
         if (isEnd)
             stopCommitting();
@@ -1906,7 +1906,7 @@ std::unique_ptr<PlannedDropHandle> KvCache::planCommittedBlockDrop()
     if (numCommittedTokens() == 0)
         return nullptr;
 
-    auto const match = mManager->matchReuse(mReuseScope, mCommittedTokens);
+    auto const match = mManager->matchReuse(mReuseScope, toSpan(mCommittedTokens));
     if (match.numTokens != numCommittedTokens() || match.blocks.empty())
         return nullptr;
 
diff --git a/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/kvCache.h b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/kvCache.h
index 67da97a6b58a..89e0aebe93c5 100644
--- a/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/kvCache.h
+++ b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/kvCache.h
@@ -113,47 +113,6 @@ struct SeqBlock
     }
 };
 
-// ---------------------------------------------------------------------------
-// Span<T> — non-owning view into a contiguous buffer.
-// Supports operator[] for uniform access with std::vector<int>.
-// ---------------------------------------------------------------------------
-template <typename T>
-struct Span
-{
-    T* ptr;
-    int32_t len;
-
-    T& operator[](int idx)
-    {
-        return ptr[idx];
-    }
-
-    T operator[](int idx) const
-    {
-        return ptr[idx];
-    }
-
-    int size() const noexcept
-    {
-        return len;
-    }
-
-    T* data() const noexcept
-    {
-        return ptr;
-    }
-
-    T* begin() const noexcept
-    {
-        return ptr;
-    }
-
-    T* end() const noexcept
-    {
-        return ptr + len;
-    }
-};
-
 // ---------------------------------------------------------------------------
 // PlannedDropHandle — tracks committed pages planned for dropping without
 // owning them. Mirrors Python's PlannedDropHandle in _core/_kv_cache.py.
@@ -258,7 +217,7 @@ class KvCache : public std::enable_shared_from_this<KvCache>
     // This is a terminal-memory contract: callers must not perform later writes
     // to this KvCache's memory. The final live pages may be moved into the radix
     // tree instead of copied (SSM state and the last partial block).
-    void commit(std::vector<TokenIdExt> const& tokens, bool isEnd = false);
+    void commit(TokenSpan tokens, bool isEnd = false);
 
     // Stop committing (called by close() automatically).
     void stopCommitting();
diff --git a/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/kvCacheManager.cpp b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/kvCacheManager.cpp
index 056cb870a5c9..ba61fcfbb04a 100644
--- a/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/kvCacheManager.cpp
+++ b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/kvCacheManager.cpp
@@ -170,16 +170,16 @@ void KvCacheManager::clearReusableBlocks()
     mRadixTree->clear();
 }
 
-std::shared_ptr<KvCache> KvCacheManager::createKvCache(ReuseScope reuseScope,
-    std::vector<TokenIdExt> const& inputTokens, std::optional<RequestIdType> id, KvCache::PriorityCb priorityCb,
-    std::optional<int> expectedPromptLength, std::optional<bool> textOnly)
+std::shared_ptr<KvCache> KvCacheManager::createKvCache(ReuseScope reuseScope, TokenSpan inputTokens,
+    std::optional<RequestIdType> id, KvCache::PriorityCb priorityCb, std::optional<int> expectedPromptLength,
+    std::optional<bool> textOnly)
 {
     if (!priorityCb)
     {
         priorityCb = [](BlockOrdinal, LifeCycleId) { return kPriorityDefault; };
     }
 
-    if (!expectedPromptLength.has_value() && !inputTokens.empty())
+    if (!expectedPromptLength.has_value() && inputTokens.size() != 0)
     {
         expectedPromptLength = static_cast<int>(inputTokens.size());
     }
@@ -188,7 +188,7 @@ std::shared_ptr<KvCache> KvCacheManager::createKvCache(ReuseScope reuseScope,
     // hand it to the KvCache, rather than having the cache re-walk the radix tree.
     // Mirrors Python KVCacheManager.allocate() passing a ReuseMatch into _KVCache.
     std::optional<BlockRadixTree::ReuseMatch> reuseMatch;
-    if (!inputTokens.empty())
+    if (inputTokens.size() != 0)
     {
         reuseMatch = matchReuse(reuseScope, inputTokens, textOnly.value_or(this->textOnly()));
     }
@@ -198,13 +198,12 @@ std::shared_ptr<KvCache> KvCacheManager::createKvCache(ReuseScope reuseScope,
 }
 
 BlockRadixTree::ReuseMatch KvCacheManager::matchReuse(
-    ReuseScope const& reuseScope, std::vector<TokenIdExt> const& inputTokens, bool knownNoDigest) const
+    ReuseScope const& reuseScope, TokenSpan inputTokens, bool knownNoDigest) const
 {
     return mRadixTree->match(reuseScope, inputTokens, knownNoDigest, enablePartialMatch());
 }
 
-int KvCacheManager::probeReuse(
-    ReuseScope reuseScope, std::vector<TokenIdExt> const& inputTokens, bool knownNoDigest) const
+int KvCacheManager::probeReuse(ReuseScope reuseScope, TokenSpan inputTokens, bool knownNoDigest) const
 {
     return matchReuse(reuseScope, inputTokens, knownNoDigest).numTokens;
 }
diff --git a/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/kvCacheManager.h b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/kvCacheManager.h
index 4c868adaa457..fd2addf9d3af 100644
--- a/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/kvCacheManager.h
+++ b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/kvCacheManager.h
@@ -141,16 +141,17 @@ class KvCacheManager : public std::enable_shared_from_this<KvCacheManager>
     //                       Stats-only: no effect on allocation, reuse, or correctness.
     // textOnly:             per-sequence override of the text-only (digest-free) guarantee;
     //                       nullopt inherits the manager config default.
-    std::shared_ptr<KvCache> createKvCache(ReuseScope reuseScope = {}, std::vector<TokenIdExt> const& inputTokens = {},
+    // inputTokens is a non-owning view; the caller must keep the underlying buffer alive for the
+    // duration of the call (matching reads it but never stores it).
+    std::shared_ptr<KvCache> createKvCache(ReuseScope reuseScope = {}, TokenSpan inputTokens = {},
         std::optional<RequestIdType> id = std::nullopt, KvCache::PriorityCb priorityCb = {},
         std::optional<int> expectedPromptLength = std::nullopt, std::optional<bool> textOnly = std::nullopt);
 
     // knownNoDigest: from external text_only knowledge, never a scan (see Hasher::update).
     // Defaults false (safe: the scanning path is taken).
     BlockRadixTree::ReuseMatch matchReuse(
-        ReuseScope const& reuseScope, std::vector<TokenIdExt> const& inputTokens, bool knownNoDigest = false) const;
-    int probeReuse(
-        ReuseScope reuseScope = {}, std::vector<TokenIdExt> const& inputTokens = {}, bool knownNoDigest = false) const;
+        ReuseScope const& reuseScope, TokenSpan inputTokens, bool knownNoDigest = false) const;
+    int probeReuse(ReuseScope reuseScope = {}, TokenSpan inputTokens = {}, bool knownNoDigest = false) const;
 
     // ---- Memory pool queries -----------------------------------------------
 
diff --git a/cpp/tensorrt_llm/nanobind/batch_manager/bindings.cpp b/cpp/tensorrt_llm/nanobind/batch_manager/bindings.cpp
index 02355e9f7507..6da2739ed681 100644
--- a/cpp/tensorrt_llm/nanobind/batch_manager/bindings.cpp
+++ b/cpp/tensorrt_llm/nanobind/batch_manager/bindings.cpp
@@ -34,6 +34,7 @@
 #include <ATen/ATen.h>
 #include <algorithm>
 #include <nanobind/nanobind.h>
+#include <nanobind/ndarray.h>
 #include <nanobind/stl/chrono.h>
 #include <nanobind/stl/optional.h>
 #include <nanobind/stl/shared_ptr.h>
@@ -131,6 +132,20 @@ void initBindings(nb::module_& m)
                 return GenLlmReq::VecTokens(tokens.begin() + begin, tokens.begin() + end);
             },
             nb::arg("beam"), nb::arg("begin"), nb::arg("end"))
+        // Zero-copy read-only int32 view of a beam's tokens (aliases the internal buffer; no
+        // per-token PyLong allocation, unlike get_tokens). reference_internal keeps this request
+        // alive for the view's lifetime. Consume synchronously — the view is invalidated if the
+        // token buffer is mutated/reallocated (e.g. by add_new_token). Used on the KV-cache-v2
+        // block-reuse hot path where it feeds an int32 fast path with no Python round-trip.
+        .def(
+            "get_tokens_view",
+            [](GenLlmReq const& self, GenLlmReq::SizeType32 beam)
+            {
+                auto const& tokens = self.getTokens(beam);
+                return nb::ndarray<nb::numpy, GenLlmReq::TokenIdType const, nb::ndim<1>, nb::c_contig>(
+                    tokens.data(), {tokens.size()});
+            },
+            nb::arg("beam"), nb::rv_policy::reference_internal)
         .def("get_last_tokens", nb::overload_cast<GenLlmReq::SizeType32>(&GenLlmReq::getLastTokens), nb::arg("beam"))
         .def("get_last_tokens", nb::overload_cast<>(&GenLlmReq::getLastTokens))
         .def("get_beam_width_by_iter", &GenLlmReq::getBeamWidthByIter, nb::arg("for_next_iteration") = false)
diff --git a/cpp/tensorrt_llm/nanobind/batch_manager/kvCacheManagerV2.cpp b/cpp/tensorrt_llm/nanobind/batch_manager/kvCacheManagerV2.cpp
index 767db2b08c4c..a1d892cfabde 100644
--- a/cpp/tensorrt_llm/nanobind/batch_manager/kvCacheManagerV2.cpp
+++ b/cpp/tensorrt_llm/nanobind/batch_manager/kvCacheManagerV2.cpp
@@ -97,6 +97,38 @@ static std::pair<std::vector<kv::TokenIdExt>, bool> castTokenIterable(nb::handle
     return {std::move(vec), knownNoDigest};
 }
 
+// Zero-copy-friendly token ingestion. Invokes fn(TokenSpan, knownNoDigest) with a non-owning
+// view of the tokens; fn must consume it synchronously (it may release the GIL — the backing
+// buffer must outlive the call).
+//
+// Fast path: a contiguous 1-D int32 buffer (numpy view, memoryview, array.array, torch tensor) is
+// reinterpret_cast to TokenIdExt const* — no copy, no per-token boxing — since a normal token id
+// is bit-identical to a 4-byte TokenIdExt (see tokenIdExt.h). convert=false keeps it strictly
+// zero-copy; a non-int32 / non-contiguous input falls through.
+//
+// Fallback: any int|bytes(32) iterable via castTokenIterable — the multimodal/digest path.
+template <class Fn>
+static auto withTokens(nb::handle tokens, Fn&& fn)
+{
+    nb::ndarray<int32_t const, nb::ndim<1>, nb::c_contig> arr;
+    if (nb::try_cast(tokens, arr, /*convert=*/false))
+    {
+        auto const count = static_cast<int32_t>(arr.shape(0));
+        auto const* raw = arr.data();
+        // Every element must be a normal id (high/digest bit clear); a negative int32 would look
+        // like a pooled digest (see tokenIdExt.h) and corrupt the hash. Digests never reach here —
+        // they take the fallback — so knownNoDigest=true holds; this asserts it in debug builds.
+        TLLM_CHECK_DEBUG_WITH_INFO(std::all_of(raw, raw + count, [](int32_t id) { return id >= 0; }),
+            "token id must be in [0, 2^31) for the zero-copy path");
+        // Sound reinterpret: TokenIdExt is bit-identical to its int32 id (tokenIdExt.h) and this
+        // buffer is only ever read const (never written through the alias).
+        auto const* data = reinterpret_cast<kv::TokenIdExt const*>(raw);
+        return fn(kv::TokenSpan{data, count}, /*knownNoDigest=*/true);
+    }
+    auto [vec, knownNoDigest] = castTokenIterable(tokens);
+    return fn(kv::toSpan(vec), knownNoDigest);
+}
+
 static kv::TypedVec<kv::PoolIndex, size_t> typedPoolSizeList(std::vector<size_t> const& slotSizeList)
 {
     return kv::TypedVec<kv::PoolIndex, size_t>{slotSizeList};
@@ -1476,14 +1508,6 @@ void KvCacheManagerV2Bindings::initBindings(nb::module_& m)
             "commit",
             [](kv::KvCache& self, nb::object acceptedInputTokens, nb::object beamSearchIndices, bool isEnd)
             {
-                // commit() sources knownNoDigest from the KvCache's text_only flag. Guard
-                // that claim against the actual tokens: a text_only sequence committing a
-                // digest would silently corrupt the block-key hash. Free today (the flag is
-                // a byproduct of building the vector); revisit if C++/Python conversion
-                // becomes zero-copy and this becomes a dedicated scan.
-                auto [vec, knownNoDigest] = castTokenIterable(acceptedInputTokens);
-                TLLM_CHECK_WITH_INFO(!(self.textOnly() && !knownNoDigest),
-                    "commit() received a digest token on a text_only sequence — hashing would be corrupted");
                 if (!beamSearchIndices.is_none())
                 {
                     PyErr_SetString(PyExc_AssertionError, "beam_search_indices must be None");
@@ -1491,8 +1515,17 @@ void KvCacheManagerV2Bindings::initBindings(nb::module_& m)
                 }
                 // Note: an empty token list with is_end=True must still stop committing,
                 // so we do not early-return on empty; commit() handles it.
-                nb::gil_scoped_release release;
-                self.commit(vec, isEnd);
+                withTokens(acceptedInputTokens,
+                    [&](kv::TokenSpan view, bool knownNoDigest)
+                    {
+                        // commit() sources knownNoDigest from the KvCache's text_only flag. Guard
+                        // that claim against the actual tokens: a text_only sequence committing a
+                        // digest would silently corrupt the block-key hash.
+                        TLLM_CHECK_WITH_INFO(!(self.textOnly() && !knownNoDigest),
+                            "commit() received a digest token on a text_only sequence — hashing would be corrupted");
+                        nb::gil_scoped_release release;
+                        self.commit(view, isEnd);
+                    });
             },
             nb::arg("accepted_input_tokens"), nb::arg("beam_search_indices").none() = nb::none(),
             nb::arg("is_end") = false)
@@ -1724,26 +1757,31 @@ void KvCacheManagerV2Bindings::initBindings(nb::module_& m)
         [](kv::KvCacheManager& manager, nb::object reuseScope, nb::object tokens, int lcId, bool enablePartial)
         {
             auto rs = castReuseScope(reuseScope);
-            auto [vec, knownNoDigest] = castTokenIterable(tokens);
             int numTokens = 0;
             std::vector<std::optional<std::pair<int, int>>> pages;
-            {
-                nb::gil_scoped_release release;
-                auto matchResult = manager.radixTree().match(rs, vec, knownNoDigest, enablePartial);
-                numTokens = matchResult.numTokens;
-                kv::LifeCycleId lc{lcId};
-                pages.reserve(matchResult.blocks.stdSize());
-                for (auto* block : matchResult.blocks)
+            withTokens(tokens,
+                [&](kv::TokenSpan view, bool knownNoDigest)
                 {
-                    auto* page = block->getPage(lc);
-                    if (page == nullptr)
+                    nb::gil_scoped_release release;
+                    auto matchResult = manager.radixTree().match(rs, view, knownNoDigest, enablePartial);
+                    numTokens = matchResult.numTokens;
+                    kv::LifeCycleId lc{lcId};
+                    pages.reserve(matchResult.blocks.stdSize());
+                    for (auto* block : matchResult.blocks)
                     {
-                        pages.emplace_back(std::nullopt);
-                        continue;
+                        auto* page = block->storage.at(lc);
+                        if (page == nullptr)
+                        {
+                            pages.emplace_back(std::nullopt);
+                            continue;
+                        }
+                        int const slotId = page->slotId().value();
+                        int numTokensInBlock = -1;
+                        if (auto* ssm = dynamic_cast<kv::SsmCommittedPage*>(page))
+                            numTokensInBlock = ssm->numTokensInBlock;
+                        pages.emplace_back(std::make_pair(slotId, numTokensInBlock));
                     }
-                    pages.emplace_back(std::make_pair(page->slotId().value(), page->numTokensInBlock));
-                }
-            }
+                });
             return std::make_tuple(numTokens, std::move(pages));
         },
         nb::arg("manager"), nb::arg("reuse_scope"), nb::arg("tokens"), nb::arg("lc_id"),
@@ -1755,24 +1793,25 @@ void KvCacheManagerV2Bindings::initBindings(nb::module_& m)
         [](kv::KvCacheManager& manager, nb::object reuseScope, nb::object tokens, int lcId, bool enablePartial)
         {
             auto rs = castReuseScope(reuseScope);
-            auto vec = castTokenIterable(tokens);
             int numTokens = 0;
             std::vector<std::optional<int>> counts;
-            {
-                nb::gil_scoped_release release;
-                auto matchResult = manager.radixTree().match(rs, vec, enablePartial);
-                numTokens = matchResult.numTokens;
-                kv::LifeCycleId lc{lcId};
-                counts.reserve(matchResult.blocks.stdSize());
-                for (auto* block : matchResult.blocks)
+            withTokens(tokens,
+                [&](kv::TokenSpan view, bool knownNoDigest)
                 {
-                    auto* page = block->storage.at(lc);
-                    if (page == nullptr)
-                        counts.emplace_back(std::nullopt);
-                    else
-                        counts.emplace_back(page->plannedDropCount);
-                }
-            }
+                    nb::gil_scoped_release release;
+                    auto matchResult = manager.radixTree().match(rs, view, knownNoDigest, enablePartial);
+                    numTokens = matchResult.numTokens;
+                    kv::LifeCycleId lc{lcId};
+                    counts.reserve(matchResult.blocks.stdSize());
+                    for (auto* block : matchResult.blocks)
+                    {
+                        auto* page = block->storage.at(lc);
+                        if (page == nullptr)
+                            counts.emplace_back(std::nullopt);
+                        else
+                            counts.emplace_back(page->plannedDropCount);
+                    }
+                });
             return std::make_tuple(numTokens, std::move(counts));
         },
         nb::arg("manager"), nb::arg("reuse_scope"), nb::arg("tokens"), nb::arg("lc_id"),
@@ -1863,26 +1902,30 @@ void KvCacheManagerV2Bindings::initBindings(nb::module_& m)
                 std::optional<int> expectedPromptLength, std::optional<bool> textOnly)
             {
                 kv::ReuseScope reuseScope = castReuseScope(std::move(reuseScopeObj));
-                std::vector<kv::TokenIdExt> tokens;
-                bool const hasInputTokens = !inputTokens.is_none();
-                if (hasInputTokens)
-                {
-                    bool knownNoDigest = true;
-                    std::tie(tokens, knownNoDigest) = castTokenIterable(inputTokens);
-                    // Guard the text_only claim against the actual input tokens (see commit()).
-                    bool const resolvedTextOnly = textOnly.value_or(self->textOnly());
-                    TLLM_CHECK_WITH_INFO(!(resolvedTextOnly && !knownNoDigest),
-                        "create_kv_cache received digest input_tokens on a text_only sequence — hashing would be "
-                        "corrupted");
-                }
-                if (!expectedPromptLength.has_value() && hasInputTokens)
+                kv::KvCache::PriorityCb priorityCb = castPriorityCallback(*self, std::move(customPriorityCallback));
+                if (inputTokens.is_none())
                 {
-                    expectedPromptLength = static_cast<int>(tokens.size());
+                    nb::gil_scoped_release release;
+                    return self->createKvCache(std::move(reuseScope), kv::TokenSpan{}, id, std::move(priorityCb),
+                        expectedPromptLength, textOnly);
                 }
-                kv::KvCache::PriorityCb priorityCb = castPriorityCallback(*self, std::move(customPriorityCallback));
-                nb::gil_scoped_release release;
-                return self->createKvCache(
-                    std::move(reuseScope), tokens, id, std::move(priorityCb), expectedPromptLength, textOnly);
+                return withTokens(inputTokens,
+                    [&](kv::TokenSpan view, bool knownNoDigest)
+                    {
+                        // Guard the text_only claim against the actual input tokens (see commit()).
+                        bool const resolvedTextOnly = textOnly.value_or(self->textOnly());
+                        TLLM_CHECK_WITH_INFO(!(resolvedTextOnly && !knownNoDigest),
+                            "create_kv_cache received digest input_tokens on a text_only sequence — hashing would be "
+                            "corrupted");
+                        std::optional<int> promptLen = expectedPromptLength;
+                        if (!promptLen.has_value() && view.size() != 0)
+                        {
+                            promptLen = static_cast<int>(view.size());
+                        }
+                        nb::gil_scoped_release release;
+                        return self->createKvCache(
+                            std::move(reuseScope), view, id, std::move(priorityCb), promptLen, textOnly);
+                    });
             },
             nb::arg("reuse_scope") = nb::none(), nb::arg("input_tokens") = nb::none(), nb::arg("id") = std::nullopt,
             nb::arg("custom_priority_callback") = nb::none(), nb::arg("expected_prompt_length") = std::nullopt,
@@ -1892,14 +1935,17 @@ void KvCacheManagerV2Bindings::initBindings(nb::module_& m)
             [](std::shared_ptr<kv::KvCacheManager> self, nb::object reuseScopeObj, nb::object inputTokens)
             {
                 kv::ReuseScope reuseScope = castReuseScope(std::move(reuseScopeObj));
-                std::vector<kv::TokenIdExt> tokens;
-                bool knownNoDigest = true;
-                if (!inputTokens.is_none())
+                if (inputTokens.is_none())
                 {
-                    std::tie(tokens, knownNoDigest) = castTokenIterable(inputTokens);
+                    nb::gil_scoped_release release;
+                    return self->probeReuse(std::move(reuseScope), kv::TokenSpan{}, /*knownNoDigest=*/true);
                 }
-                nb::gil_scoped_release release;
-                return self->probeReuse(std::move(reuseScope), tokens, knownNoDigest);
+                return withTokens(inputTokens,
+                    [&](kv::TokenSpan view, bool knownNoDigest)
+                    {
+                        nb::gil_scoped_release release;
+                        return self->probeReuse(std::move(reuseScope), view, knownNoDigest);
+                    });
             },
             nb::arg("reuse_scope") = nb::none(), nb::arg("input_tokens") = nb::none())
         .def("get_mem_pool_base_address", &kv::KvCacheManager::getMemPoolBaseAddress, nb::arg("layer_id"),
diff --git a/tensorrt_llm/_torch/pyexecutor/kv_cache_manager_v2.py b/tensorrt_llm/_torch/pyexecutor/kv_cache_manager_v2.py
index 2c567824a8cb..47b964348646 100644
--- a/tensorrt_llm/_torch/pyexecutor/kv_cache_manager_v2.py
+++ b/tensorrt_llm/_torch/pyexecutor/kv_cache_manager_v2.py
@@ -2313,7 +2313,7 @@ def _prepare_context_impl(self, req: LlmRequest) -> bool:
                 self.conversation_manager.prepare_request(req)
             kv_cache = self.kv_cache_map.get(req.py_request_id)
             if kv_cache is None:
-                all_tokens = req.get_tokens(DEFAULT_BEAM_INDEX)
+                all_tokens = self._reuse_token_source(req)
                 # Last token cannot be recovered, so we don't include it in
                 # the input tokens to look up for the block that can be reused.
                 if self.enable_block_reuse:
@@ -2539,6 +2539,18 @@ def _prepare_draft_resources(self, scheduled_batch: ScheduledRequests):
                         f"{req.py_request_id}: could not resize to {new_cap} tokens"
                     )
 
+    def _reuse_token_source(self, req: LlmRequest) -> Sequence[int]:
+        """Beam-0 tokens for block reuse, in the form the active backend consumes.
+
+        The C++ backend ingests a zero-copy int32 view (get_tokens_view) — no per-token
+        PyLong allocation. The pure-Python backend cannot consume a numpy array, so it gets
+        a plain list (get_tokens), preserving pre-optimization behavior. ``_cpp_introspection``
+        is None exactly when the Python backend is active.
+        """
+        if _cpp_introspection is not None:
+            return req.get_tokens_view(DEFAULT_BEAM_INDEX)
+        return req.get_tokens(DEFAULT_BEAM_INDEX)
+
     def _augment_tokens_for_block_reuse(
         self, tokens: Sequence[int], req: LlmRequest, start: int = 0, end: int | None = None
     ) -> Sequence[TokenIdExt]:
@@ -2570,7 +2582,11 @@ def _augment_tokens_for_block_reuse(
         ):
             return tokens[chunk_start:chunk_end] if is_sliced else tokens
 
-        result: list[TokenIdExt] = list(tokens[chunk_start:chunk_end])
+        # Multimodal path: materialize a Python-int list (digest bytes get spliced in below),
+        # which flows through the per-element binding fallback. tokens may be a zero-copy numpy
+        # int32 view (get_tokens_view) — use tolist() so elements are Python ints, not np.int32.
+        chunk = tokens[chunk_start:chunk_end]
+        result: list[TokenIdExt] = chunk.tolist() if hasattr(chunk, "tolist") else list(chunk)
         run_metadata = _resolve_multimodal_run_metadata(req)
         if run_metadata is not None:
             return _augment_tokens_with_mm_run_metadata(
@@ -3193,7 +3209,7 @@ def try_commit_blocks(self, request: LlmRequest) -> None:
 
         if request.context_current_position > kv_cache.num_committed_tokens:
             tokens = self._augment_tokens_for_block_reuse(
-                request.get_tokens(DEFAULT_BEAM_INDEX),
+                self._reuse_token_source(request),
                 request,
                 start=kv_cache.num_committed_tokens,
                 end=request.context_current_position,
@@ -3729,7 +3745,7 @@ def prefetch_for_context_tokens(self, requests: list) -> bool:
         # is NOT registered in kv_cache_map / IndexMapper.
         success = True
         for req in requests:
-            all_tokens = req.get_tokens(DEFAULT_BEAM_INDEX)
+            all_tokens = self._reuse_token_source(req)
             tokens = self._augment_tokens_for_block_reuse(all_tokens, req, end=len(all_tokens) - 1)
             # Use the same salt derivation as _create_kv_cache so the transient
             # cache hits the same radix-tree blocks.
diff --git a/tests/unittest/kv_cache_manager_v2_tests/test_kv_cache_manager_v2.py b/tests/unittest/kv_cache_manager_v2_tests/test_kv_cache_manager_v2.py
index 341a34523227..c093ff586674 100755
--- a/tests/unittest/kv_cache_manager_v2_tests/test_kv_cache_manager_v2.py
+++ b/tests/unittest/kv_cache_manager_v2_tests/test_kv_cache_manager_v2.py
@@ -793,6 +793,61 @@ def test_planned_drop_handle_rejects_partial_coverage(self) -> None:
                 kv_cache.close()
         stream_holder.take_finish_event().synchronize()
 
+    def test_int32_ndarray_ingest_matches_list(self) -> None:
+        """Zero-copy int32-ndarray ingest must hash identically to the list path.
+
+        The int32-ndarray path must agree with the per-element list path across
+        create_kv_cache / commit / probe_reuse.
+
+        A digest-free int32 token is bit-identical to a normal 4-byte TokenIdExt,
+        so the C++ binding reinterprets a contiguous int32 buffer to TokenIdExt*
+        with no copy. If that reinterpret disagreed with the list path by even one
+        bit, blocks committed via the ndarray path would not be found by a list
+        probe (and vice versa), so the equalities below would fail.
+        """
+        # The int32-ndarray ingest fast path lives in the C++ binding; the pure-Python
+        # backend consumes plain lists (the dispatcher hands it get_tokens, not a view).
+        if os.environ.get("TLLM_KV_CACHE_MANAGER_V2_BACKEND", "cpp").lower() == "python":
+            self.skipTest("int32-ndarray ingest is a C++-backend fast path")
+
+        import numpy as np
+
+        tokens_per_block = 8
+        self.prepare(16 << 20, 0, 0, 2, None, 0, tokens_per_block=tokens_per_block)
+        # Pure-int prompt (no randbytes/digest tokens) so the int32 fast path applies.
+        prompt = [TokenId(i) for i in range(tokens_per_block * 3)]
+        prompt_np = np.asarray(prompt, dtype=np.int32)
+        assert prompt_np.dtype == np.int32 and prompt_np.flags["C_CONTIGUOUS"]
+
+        def commit_prompt(tokens) -> None:
+            kv_cache = self.manager.create_kv_cache(None, tokens)
+            with TemporaryCudaStream([]) as stream_holder:
+                stream = cast(CudaStream, stream_holder.handle)
+                self.assertTrue(kv_cache.resume(stream))
+                self.assertTrue(kv_cache.resize(len(prompt)))
+                committed = kv_cache.num_committed_tokens
+                if committed < len(prompt):
+                    kv_cache.commit(tokens[committed:])
+                kv_cache.stop_committing()
+            _ = stream_holder.take_finish_event()
+            kv_cache.close()
+
+        # Commit via the int32-ndarray fast path (both create_kv_cache and commit).
+        commit_prompt(prompt_np)
+
+        # Probing with a list and with an int32 ndarray must both fully match —
+        # proving the ndarray commit hashes like the list path, and the ndarray
+        # probe hashes like the list probe.
+        self.assertEqual(self.manager.probe_reuse(None, prompt), len(prompt))
+        self.assertEqual(self.manager.probe_reuse(None, prompt_np), len(prompt))
+
+        # Reverse direction: commit via the list path in a fresh tree, probe via
+        # the int32-ndarray path → full match.
+        self.manager.clear_reusable_blocks()
+        self.assertEqual(self.manager.probe_reuse(None, prompt_np), 0)
+        commit_prompt(prompt)
+        self.assertEqual(self.manager.probe_reuse(None, prompt_np), len(prompt))
+
     def test_reuse_scope_isolates_reuse(self) -> None:
         self.prepare(16 << 20, 0, 0, 2, None, 0, tokens_per_block=8)
         tokens = [TokenId(i) for i in range(64)]

From ebd6115f94a7dca3176ddd2949cd3fa3d8bcaa5a Mon Sep 17 00:00:00 2001
From: Yao Yao <lowsfer@users.noreply.github.com>
Date: Fri, 31 Jul 2026 04:23:38 +0000
Subject: [PATCH 04/12] [None][fix] filter partial KV cache event coverage

Signed-off-by: Yao Yao <lowsfer@users.noreply.github.com>
---
 .../runtime/kv_cache_manager_v2/_event_manager.py    | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/tensorrt_llm/runtime/kv_cache_manager_v2/_event_manager.py b/tensorrt_llm/runtime/kv_cache_manager_v2/_event_manager.py
index c493918ca367..e9584638f921 100644
--- a/tensorrt_llm/runtime/kv_cache_manager_v2/_event_manager.py
+++ b/tensorrt_llm/runtime/kv_cache_manager_v2/_event_manager.py
@@ -560,12 +560,12 @@ def _stored_block_from_radix_block(
 
     @staticmethod
     def _life_cycle_ids_from_radix_block(block: Any) -> set[int]:
-        life_cycle_ids = set[int]()
-        for life_cycle_id in range(len(block.storage)):
-            page = block.get_page(life_cycle_id)
-            if page is not None and page.num_tokens_in_block >= len(block.tokens):
-                life_cycle_ids.add(life_cycle_id)
-        return life_cycle_ids
+        return {
+            life_cycle_id
+            for life_cycle_id in range(len(block.storage))
+            if (page := block.get_page(life_cycle_id)) is not None
+            and page.num_tokens_in_block >= len(block.tokens)
+        }
 
     def _parent_hash_from_radix_block(self, block: Any) -> EventBlockHash | None:
         parent = block.prev

From aece44e35ac5a307205b892f5e90b3dcfd8cdd4b Mon Sep 17 00:00:00 2001
From: Yao Yao <lowsfer@users.noreply.github.com>
Date: Fri, 31 Jul 2026 04:23:54 +0000
Subject: [PATCH 05/12] [None][test] cover KV cache event derivation with real
 blocks

Signed-off-by: Yao Yao <lowsfer@users.noreply.github.com>
---
 .../batch_manager/kvCacheManagerV2.cpp        | 109 +++++
 .../kv_cache_manager_v2/_introspection.py     | 116 +++++
 .../test_kv_cache_event_manager.py            | 444 +++++++++---------
 3 files changed, 453 insertions(+), 216 deletions(-)

diff --git a/cpp/tensorrt_llm/nanobind/batch_manager/kvCacheManagerV2.cpp b/cpp/tensorrt_llm/nanobind/batch_manager/kvCacheManagerV2.cpp
index a1d892cfabde..216d20cb7887 100644
--- a/cpp/tensorrt_llm/nanobind/batch_manager/kvCacheManagerV2.cpp
+++ b/cpp/tensorrt_llm/nanobind/batch_manager/kvCacheManagerV2.cpp
@@ -541,6 +541,33 @@ static kv::ReuseScope castReuseScope(nb::object reuseScope)
         "reuse_scope must be None, ReuseScope, an int lora_task_id, or an object with lora_id and salt");
 }
 
+class EventManagerTestBlock
+{
+public:
+    EventManagerTestBlock(kv::SharedPtr<kv::Block> block_, std::vector<kv::SharedPtr<kv::CommittedPage>> pages_)
+        : block(std::move(block_))
+        , pages(std::move(pages_))
+    {
+    }
+
+    ~EventManagerTestBlock()
+    {
+        close();
+    }
+
+    void close()
+    {
+        for (auto const& page : pages)
+        {
+            block->unlinkPage(page->lifeCycle, page.get());
+        }
+        pages.clear();
+    }
+
+    kv::SharedPtr<kv::Block> block;
+    std::vector<kv::SharedPtr<kv::CommittedPage>> pages;
+};
+
 // Lazy Python iterator over a token sequence's blockchain keys. Each __next__ pulls
 // one key from the C++ generator (one SHA-256 hash), so a caller that stops early
 // (e.g. on the first cache-key mismatch) skips the remaining hashing. Yields
@@ -1646,6 +1673,88 @@ void KvCacheManagerV2Bindings::initBindings(nb::module_& m)
 
     // ---- Introspection -------------------------------------------------------
     auto mIntrospection = m.def_submodule("_introspection", "KV cache manager v2 introspection helpers");
+
+    nb::class_<EventManagerTestBlock>(mIntrospection, "TestBlock").def("close", &EventManagerTestBlock::close);
+    mIntrospection.def(
+        "make_test_block",
+        [](kv::KvCacheManager& manager, nb::object tokenObject, std::vector<int> coveragePerLc, nb::object parentObject,
+            nb::object reuseScopeObject)
+        {
+            auto [tokens, knownNoDigest] = castTokenIterable(tokenObject);
+            if (tokens.empty())
+            {
+                throw std::invalid_argument("make_test_block requires at least one token");
+            }
+            if (coveragePerLc.size() != static_cast<size_t>(manager.lifeCycles().size().value()))
+            {
+                throw std::invalid_argument("coverage_per_lc length must match the manager lifecycle count");
+            }
+            for (int coverage : coveragePerLc)
+            {
+                if (coverage < 0 || coverage > static_cast<int>(tokens.size()))
+                {
+                    throw std::invalid_argument("lifecycle coverage must be between zero and the block token count");
+                }
+            }
+
+            kv::NodeBase* parent = nullptr;
+            if (parentObject.is_none())
+            {
+                parent = &manager.radixTree().addOrGetExisting(castReuseScope(std::move(reuseScopeObject)));
+            }
+            else
+            {
+                parent = nb::cast<EventManagerTestBlock&>(parentObject).block.get();
+            }
+            auto block = kv::addOrGetExistingBlock(parent, std::move(tokens), knownNoDigest);
+
+            kv::TypedVec<kv::LifeCycleId, kv::SlotCount> counts(manager.lifeCycles().size(), 0);
+            for (kv::LifeCycleId lifeCycle{0}; lifeCycle < manager.lifeCycles().size(); ++lifeCycle)
+            {
+                counts[lifeCycle] = coveragePerLc.at(lifeCycle.value()) > 0 ? 1 : 0;
+            }
+            auto slots = manager.storage().newGpuSlots(counts);
+            std::vector<kv::SharedPtr<kv::CommittedPage>> pages;
+            pages.reserve(coveragePerLc.size());
+            for (kv::LifeCycleId lifeCycle{0}; lifeCycle < manager.lifeCycles().size(); ++lifeCycle)
+            {
+                int const coverage = coveragePerLc.at(lifeCycle.value());
+                if (coverage == 0)
+                {
+                    continue;
+                }
+                kv::SharedPtr<kv::CommittedPage> page;
+                if (coverage < static_cast<int>(block->tokens.size()))
+                {
+                    page = kv::makeShared<kv::SsmCommittedPage>(
+                        &manager.storage(), block, lifeCycle, kv::kGpuLevel, kv::kPriorityDefault, coverage);
+                }
+                else
+                {
+                    page = kv::makeShared<kv::CommittedPage>(
+                        &manager.storage(), block, lifeCycle, kv::kGpuLevel, kv::kPriorityDefault);
+                }
+                page->setSlot(slots[lifeCycle].front());
+                block->storage[lifeCycle] = page.get();
+                pages.push_back(std::move(page));
+            }
+            return std::make_unique<EventManagerTestBlock>(std::move(block), std::move(pages));
+        },
+        nb::arg("manager"), nb::arg("tokens"), nb::arg("coverage_per_lc"), nb::arg("parent").none() = nb::none(),
+        nb::arg("reuse_scope").none() = nb::none(), nb::keep_alive<0, 1>(), nb::keep_alive<0, 4>());
+    mIntrospection.def(
+        "test_block_key", [](EventManagerTestBlock const& block) { return digestBytes(block.block->key); },
+        nb::arg("block"));
+    mIntrospection.def(
+        "event_manager_add_stored_block",
+        [](kv::EventManager& eventManager, EventManagerTestBlock const& block)
+        { eventManager.addStoredBlock(*block.block); },
+        nb::arg("event_manager"), nb::arg("block"), nb::call_guard<nb::gil_scoped_release>());
+    mIntrospection.def(
+        "event_manager_add_stored_life_cycle",
+        [](kv::EventManager& eventManager, EventManagerTestBlock const& block, int lifeCycleId)
+        { eventManager.addStoredLifeCycle(*block.block, kv::LifeCycleId{lifeCycleId}); },
+        nb::arg("event_manager"), nb::arg("block"), nb::arg("life_cycle_id"), nb::call_guard<nb::gil_scoped_release>());
     nb::class_<kv::StorageStatistics>(mIntrospection, "StorageStatistics")
         .def_prop_ro("slot_sizes", [](kv::StorageStatistics const& self) { return self.slotSizes.raw(); })
         .def_ro("total", &kv::StorageStatistics::total)
diff --git a/tensorrt_llm/runtime/kv_cache_manager_v2/_introspection.py b/tensorrt_llm/runtime/kv_cache_manager_v2/_introspection.py
index f8f5e3214f6e..b8b28f786835 100644
--- a/tensorrt_llm/runtime/kv_cache_manager_v2/_introspection.py
+++ b/tensorrt_llm/runtime/kv_cache_manager_v2/_introspection.py
@@ -19,6 +19,24 @@
 from typing import Any
 
 
+class _TestBlock:
+    """Own a real radix-tree block and its real committed pages for a test."""
+
+    __slots__ = ("block", "pages")
+
+    def __init__(self, block: Any, pages: list[Any]) -> None:
+        self.block = block
+        self.pages = pages
+
+    def close(self) -> None:
+        for page in self.pages:
+            self.block.unlink_page(page.life_cycle, page)
+        self.pages.clear()
+
+    def __del__(self) -> None:
+        self.close()
+
+
 def _cpp_introspection_module() -> Any | None:
     package = sys.modules.get(__package__)
     if package is None:
@@ -26,6 +44,104 @@ def _cpp_introspection_module() -> Any | None:
     return getattr(package, "_cpp_introspection", None)
 
 
+def make_test_block(
+    manager: Any,
+    tokens: Any,
+    coverage_per_lc: list[int],
+    parent: Any = None,
+    reuse_scope: Any = None,
+) -> Any:
+    """Build a real block with real GPU pages at the requested lifecycle coverage."""
+    cpp_introspection = _cpp_introspection_module()
+    if cpp_introspection is not None:
+        return cpp_introspection.make_test_block(
+            manager, list(tokens), coverage_per_lc, parent, reuse_scope
+        )
+
+    from . import rawref
+    from ._block_radix_tree import ReuseScope, _add_or_get_existing
+    from ._common import GPU_LEVEL, PRIORITY_DEFAULT
+    from ._page import CommittedPage, SsmCommittedPage
+
+    token_list = list(tokens)
+    if not token_list:
+        raise ValueError("make_test_block requires at least one token")
+    if len(coverage_per_lc) != manager._life_cycles.size:
+        raise ValueError(
+            "coverage_per_lc length must match the manager lifecycle count "
+            f"({manager._life_cycles.size})"
+        )
+    if any(coverage < 0 or coverage > len(token_list) for coverage in coverage_per_lc):
+        raise ValueError("lifecycle coverage must be between zero and the block token count")
+
+    if parent is None:
+        parent_block = manager._radix_tree.add_or_get_existing(reuse_scope or ReuseScope())
+    else:
+        parent_block = parent.block
+    block = _add_or_get_existing(parent_block, token_list)
+    if block is None:
+        raise ValueError("make_test_block could not add the requested block")
+
+    counts = [int(coverage > 0) for coverage in coverage_per_lc]
+    slots = manager._storage.new_gpu_slots(counts)
+    pages = []
+    for life_cycle, coverage in enumerate(coverage_per_lc):
+        if coverage == 0:
+            continue
+        page_type = SsmCommittedPage if coverage < len(token_list) else CommittedPage
+        page_args = (
+            manager._storage,
+            block,
+            life_cycle,
+            GPU_LEVEL,
+            slots[life_cycle].pop(),
+            PRIORITY_DEFAULT,
+        )
+        page = (
+            page_type(*page_args, coverage)
+            if page_type is SsmCommittedPage
+            else page_type(*page_args)
+        )
+        block.storage[life_cycle] = rawref.ref(page)
+        pages.append(page)
+    return _TestBlock(block, pages)
+
+
+def close_test_block(block: Any) -> None:
+    """Unlink and release a test block pages before its manager shuts down."""
+    cpp_introspection = _cpp_introspection_module()
+    if cpp_introspection is not None:
+        block.close()
+        return
+    block.close()
+
+
+def test_block_key(block: Any) -> bytes:
+    """Return a test block real radix-tree key."""
+    cpp_introspection = _cpp_introspection_module()
+    if cpp_introspection is not None:
+        return bytes(cpp_introspection.test_block_key(block))
+    return bytes(block.block.key)
+
+
+def event_manager_add_stored_block(event_manager: Any, block: Any) -> None:
+    """Derive and enqueue stored events from a real test block."""
+    cpp_introspection = _cpp_introspection_module()
+    if cpp_introspection is not None:
+        cpp_introspection.event_manager_add_stored_block(event_manager, block)
+        return
+    event_manager.add_stored_block_event_from_block(block.block)
+
+
+def event_manager_add_stored_life_cycle(event_manager: Any, block: Any, life_cycle_id: int) -> None:
+    """Derive and enqueue one lifecycle stored event from a real test block."""
+    cpp_introspection = _cpp_introspection_module()
+    if cpp_introspection is not None:
+        cpp_introspection.event_manager_add_stored_life_cycle(event_manager, block, life_cycle_id)
+        return
+    event_manager.add_stored_life_cycle_event_from_block(block.block, life_cycle_id)
+
+
 def active_page_stats(kv_cache: Any) -> tuple[list[int], list[int]]:
     """Return active pages and unscheduled evictable active pages by cache level."""
     cpp_introspection = _cpp_introspection_module()
diff --git a/tests/unittest/kv_cache_manager_v2_tests/test_kv_cache_event_manager.py b/tests/unittest/kv_cache_manager_v2_tests/test_kv_cache_event_manager.py
index 576d400451b0..13160a140016 100644
--- a/tests/unittest/kv_cache_manager_v2_tests/test_kv_cache_event_manager.py
+++ b/tests/unittest/kv_cache_manager_v2_tests/test_kv_cache_event_manager.py
@@ -55,7 +55,7 @@
 )
 
 if not TYPE_CHECKING and find_spec("kv_cache_manager_v2") is not None:
-    from kv_cache_manager_v2 import CacheLevel, CudaStream, KVCacheManager, TokenId
+    from kv_cache_manager_v2 import CacheLevel, CudaStream, KVCacheManager, TokenId, _introspection
     from kv_cache_manager_v2._block_radix_tree import Block, ReuseScope, RootBlock
     from kv_cache_manager_v2._utils import CachedCudaStream, init_cuda_once, temporary_sys_path
 else:
@@ -64,6 +64,7 @@
         CudaStream,
         KVCacheManager,
         TokenId,
+        _introspection,
     )
     from tensorrt_llm.runtime.kv_cache_manager_v2._block_radix_tree import (
         Block,
@@ -82,50 +83,9 @@
     torch = None
 
 
-_DEFAULT_CACHE_LEVEL = CacheLevel(0)
 _USING_CPP_BACKEND = os.environ.get("TLLM_KV_CACHE_MANAGER_V2_BACKEND", "cpp").lower() != "python"
 
 
-class _FakePage:
-    def __init__(self, num_tokens_in_block, cache_level=_DEFAULT_CACHE_LEVEL, priority=0):
-        self.num_tokens_in_block = num_tokens_in_block
-        self.cache_level = cache_level
-        self.priority = priority
-
-
-class _FakePageRef:
-    def __init__(self, page):
-        self._page = page
-
-    def __call__(self):
-        return self._page
-
-
-class _FakeRootBlock:
-    ordinal = -1
-
-    def __init__(self, lora_task_id=None, cache_salt_id=None, reuse_scope=None):
-        # Support both the new ReuseScope-based shape and the legacy flat
-        # (lora_task_id, cache_salt_id) shape so tests can exercise either.
-        self.lora_task_id = lora_task_id
-        self.cache_salt_id = cache_salt_id
-        if reuse_scope is not None:
-            self.reuse_scope = reuse_scope
-
-
-class _FakeBlock:
-    def __init__(self, key, tokens, num_life_cycles=1, prev=None):
-        self.key = key
-        self.tokens = tokens
-        self.prev = prev or _FakeRootBlock()
-        self.ordinal = getattr(self.prev, "ordinal", -1) + 1
-        self.storage = [_FakePageRef(_FakePage(len(tokens))) for _ in range(num_life_cycles)]
-
-    def get_page(self, lc_idx):
-        page_ref = self.storage[lc_idx]
-        return None if page_ref is None else page_ref()
-
-
 with temporary_sys_path(os.path.dirname(os.path.abspath(__file__))):
     from test_kv_cache_manager_v2 import create_config
 
@@ -154,6 +114,54 @@ def _create_test_manager(
     )
 
 
+@pytest.fixture
+def real_block_factory():
+    init_cuda_once()
+    gc.collect()
+    gc.disable()
+    managers = []
+    blocks = []
+
+    def create(event_manager, *, num_life_cycles=1, tokens_per_block=4):
+        manager = _create_test_manager(
+            event_manager,
+            tokens_per_block=tokens_per_block,
+            window_size=tokens_per_block if num_life_cycles == 2 else None,
+        )
+        managers.append(manager)
+
+        def make(tokens, coverage_per_lc, *, parent=None, reuse_scope=None):
+            block = _introspection.make_test_block(
+                manager, tokens, coverage_per_lc, parent, reuse_scope
+            )
+            blocks.append(block)
+            return block
+
+        return make
+
+    yield create
+
+    for block in blocks:
+        _introspection.close_test_block(block)
+    blocks.clear()
+    gc.collect()
+    for manager in reversed(managers):
+        manager.shutdown()
+    gc.enable()
+
+
+def _block_key(block):
+    return _introspection.test_block_key(block)
+
+
+def _add_stored_block(event_manager, block):
+    _introspection.event_manager_add_stored_block(event_manager, block)
+
+
+def _add_stored_life_cycle(event_manager, block, life_cycle_id):
+    _introspection.event_manager_add_stored_life_cycle(event_manager, block, life_cycle_id)
+
+
 def _token_ids(start, end):
     return [TokenId(token_id) for token_id in range(start, end)]
 
@@ -579,26 +587,23 @@ def test_v2_kv_cache_event_manager_accepts_removed_iterables():
     }
 
 
-def test_v2_kv_cache_event_manager_coalesces_contiguous_stored_events():
-    event_manager = KVCacheEventManager(max_kv_event_entries=8, window_size=128)
-    block0 = _FakeBlock(b"\xab\xcd", [1, 2], num_life_cycles=2)
-    block1 = _FakeBlock(b"\xab\xce", [3, 4], num_life_cycles=2, prev=block0)
-
-    event_manager.add_stored_block_event_from_block(block0)
-    event_manager.add_stored_block_event_from_block(block1)
+def test_v2_kv_cache_event_manager_coalesces_contiguous_stored_events(
+    real_block_factory,
+):
+    event_manager = NativeKVCacheEventManager(max_kv_event_entries=8, window_size=128)
+    make_block = real_block_factory(event_manager, num_life_cycles=2, tokens_per_block=2)
+    block0 = make_block([1, 2], [2, 2])
+    block1 = make_block([3, 4], [2, 2], parent=block0)
 
+    _add_stored_block(event_manager, block0)
+    _add_stored_block(event_manager, block1)
     events = _flush_serialized_events(event_manager)
+    expected_hashes = [_block_key(block0).hex(), _block_key(block1).hex()]
 
     assert [event["data"]["type"] for event in events] == ["stored", "stored"]
     assert [event["layer_group_id"] for event in events] == [0, 1]
-    assert [block["block_hash"] for block in events[0]["data"]["blocks"]] == [
-        "abcd",
-        "abce",
-    ]
-    assert [block["block_hash"] for block in events[1]["data"]["blocks"]] == [
-        "abcd",
-        "abce",
-    ]
+    assert [block["block_hash"] for block in events[0]["data"]["blocks"]] == expected_hashes
+    assert [block["block_hash"] for block in events[1]["data"]["blocks"]] == expected_hashes
     assert events[0]["data"]["parent_hash"] is None
     assert events[1]["data"]["parent_hash"] is None
 
@@ -611,33 +616,28 @@ def test_v2_kv_cache_event_manager_serializes_layer_group_id():
 
     assert [event["layer_group_id"] for event in events] == [0, 1]
     assert [event["data"] for event in events] == [
-        {
-            "type": "created",
-            "num_blocks_per_cache_level": [2, 3],
-        },
-        {
-            "type": "created",
-            "num_blocks_per_cache_level": [2, 3],
-        },
+        {"type": "created", "num_blocks_per_cache_level": [2, 3]},
+        {"type": "created", "num_blocks_per_cache_level": [2, 3]},
     ]
 
 
-def test_v2_kv_cache_event_manager_sha256_64_compatibility_mode():
-    event_manager = KVCacheEventManager(
+def test_v2_kv_cache_event_manager_sha256_64_compatibility_mode(real_block_factory):
+    event_manager = NativeKVCacheEventManager(
         max_kv_event_entries=8,
         window_size=128,
         hash_algo=KV_CACHE_HASH_ALGO_V2_SHA256_64,
     )
-    block0 = _FakeBlock(bytes.fromhex("8000000000000001" + "00" * 24), [1, 2])
-    block1 = _FakeBlock(bytes.fromhex("0102030405060708" + "00" * 24), [3, 4], prev=block0)
+    make_block = real_block_factory(event_manager, tokens_per_block=2)
+    block0 = make_block([1, 2], [2])
+    block1 = make_block([3, 4], [2], parent=block0)
 
-    event_manager.add_stored_block_event_from_block(block0)
-    event_manager.add_stored_block_event_from_block(block1)
-    event_manager.add_removed_event(block0.key)
+    _add_stored_block(event_manager, block0)
+    _add_stored_block(event_manager, block1)
+    event_manager.add_removed_event(_block_key(block0))
     events = _flush_serialized_events(event_manager)
 
-    expected_block0_hash = truncate_sha256_hash_to_int64(block0.key)
-    expected_block1_hash = truncate_sha256_hash_to_int64(block1.key)
+    expected_block0_hash = truncate_sha256_hash_to_int64(_block_key(block0))
+    expected_block1_hash = truncate_sha256_hash_to_int64(_block_key(block1))
     assert [event["hash_algo"] for event in events] == [
         KV_CACHE_HASH_ALGO_V2_SHA256_64,
         KV_CACHE_HASH_ALGO_V2_SHA256_64,
@@ -655,18 +655,20 @@ def test_v2_kv_cache_event_manager_sha256_64_compatibility_mode():
     assert all(isinstance(block_hash, int) for block_hash in events[1]["data"]["block_hashes"])
 
 
-def test_v2_kv_cache_event_manager_v1_hash_algo_matches_v1_block_key_hash():
-    event_manager = KVCacheEventManager(
+def test_v2_kv_cache_event_manager_v1_hash_algo_matches_v1_block_key_hash(
+    real_block_factory,
+):
+    event_manager = NativeKVCacheEventManager(
         max_kv_event_entries=8,
         window_size=128,
         hash_algo=KV_CACHE_HASH_ALGO_V1,
     )
-    root = _FakeRootBlock()
-    block0 = _FakeBlock(b"block0", [1, 2, 3, 4], prev=root)
-    block1 = _FakeBlock(b"block1", [5, 6], prev=block0)
+    make_block = real_block_factory(event_manager)
+    block0 = make_block([1, 2, 3, 4], [4])
+    block1 = make_block([5, 6], [2], parent=block0)
 
-    event_manager.add_stored_block_event_from_block(block0)
-    event_manager.add_stored_block_event_from_block(block1)
+    _add_stored_block(event_manager, block0)
+    _add_stored_block(event_manager, block1)
     events = _flush_serialized_events(event_manager)
 
     assert events[0]["hash_algo"] == KV_CACHE_HASH_ALGO_V1
@@ -687,18 +689,18 @@ def test_v2_root_key_distinguishes_lora_from_cache_salt_id():
     )
 
 
-def test_v2_kv_cache_event_manager_v1_hash_algo_mixes_cache_salt_id():
-    event_manager = KVCacheEventManager(
+def test_v2_kv_cache_event_manager_v1_hash_algo_mixes_cache_salt_id(real_block_factory):
+    event_manager = NativeKVCacheEventManager(
         max_kv_event_entries=8,
         window_size=128,
         hash_algo=KV_CACHE_HASH_ALGO_V1,
     )
-    root = _FakeRootBlock(cache_salt_id=123)
-    block0 = _FakeBlock(b"block0", [1, 2, 3, 4], prev=root)
-    block1 = _FakeBlock(b"block1", [5, 6], prev=block0)
+    make_block = real_block_factory(event_manager)
+    block0 = make_block([1, 2, 3, 4], [4], reuse_scope=ReuseScope(salt=123))
+    block1 = make_block([5, 6], [2], parent=block0)
 
-    event_manager.add_stored_block_event_from_block(block0)
-    event_manager.add_stored_block_event_from_block(block1)
+    _add_stored_block(event_manager, block0)
+    _add_stored_block(event_manager, block1)
     events = _flush_serialized_events(event_manager)
 
     assert events[0]["hash_algo"] == KV_CACHE_HASH_ALGO_V1
@@ -708,61 +710,41 @@ def test_v2_kv_cache_event_manager_v1_hash_algo_mixes_cache_salt_id():
     ]
 
 
-def test_v2_kv_cache_event_manager_v1_hash_reads_root_reuse_scope():
-    # Regression test: when ``RootBlock`` exposes its scope via a ReuseScope
-    # NamedTuple rather than direct ``lora_task_id`` / ``cache_salt_id``
-    # attributes, ``_root_attrs_from_root_block`` must still recover the same
-    # (lora_id, salt) — otherwise V1-compat event hashes silently collapse to
-    # (None, None) for every LoRA/salt request and Dynamo routing degrades.
-    tokens0 = [1, 2, 3, 4]
-    tokens1 = [5, 6]
-
-    def hashes_for(root):
-        event_manager = KVCacheEventManager(
+def test_v2_kv_cache_event_manager_v1_hash_reads_root_reuse_scope(real_block_factory):
+    def hashes_for(reuse_scope):
+        event_manager = NativeKVCacheEventManager(
             max_kv_event_entries=8,
             window_size=128,
             hash_algo=KV_CACHE_HASH_ALGO_V1,
         )
-        block0 = _FakeBlock(b"block0", tokens0, prev=root)
-        block1 = _FakeBlock(b"block1", tokens1, prev=block0)
-        event_manager.add_stored_block_event_from_block(block0)
-        event_manager.add_stored_block_event_from_block(block1)
+        make_block = real_block_factory(event_manager)
+        block0 = make_block([1, 2, 3, 4], [4], reuse_scope=reuse_scope)
+        block1 = make_block([5, 6], [2], parent=block0)
+        _add_stored_block(event_manager, block0)
+        _add_stored_block(event_manager, block1)
         return _stored_block_hashes(_flush_serialized_events(event_manager))
 
-    # ReuseScope-shaped RootBlock and legacy-shape RootBlock must produce
-    # identical event hashes for the same scope.
-    scope_root = _FakeRootBlock(reuse_scope=ReuseScope(lora_id=11, salt=22))
-    legacy_root = _FakeRootBlock(lora_task_id=11, cache_salt_id=22)
-    assert hashes_for(scope_root) == hashes_for(legacy_root)
-
-    # Different scopes must still produce different hashes (no silent collapse).
-    other_scope_root = _FakeRootBlock(reuse_scope=ReuseScope(lora_id=99, salt=22))
-    assert hashes_for(scope_root) != hashes_for(other_scope_root)
-
-    # An unsalted ReuseScope must match an unsalted legacy root.
-    empty_scope_root = _FakeRootBlock(reuse_scope=ReuseScope())
-    unsalted_legacy_root = _FakeRootBlock()
-    assert hashes_for(empty_scope_root) == hashes_for(unsalted_legacy_root)
+    scope_hashes = hashes_for(ReuseScope(lora_id=11, salt=22))
+    assert scope_hashes != hashes_for(ReuseScope(lora_id=99, salt=22))
+    assert scope_hashes != hashes_for(ReuseScope())
 
 
-def test_v2_kv_cache_event_manager_v1_hash_recomputes_removed_parent():
-    event_manager = KVCacheEventManager(
+def test_v2_kv_cache_event_manager_v1_hash_recomputes_removed_parent(
+    real_block_factory,
+):
+    event_manager = NativeKVCacheEventManager(
         max_kv_event_entries=8,
         window_size=128,
         hash_algo=KV_CACHE_HASH_ALGO_V1,
     )
-    root = _FakeRootBlock()
-    block0 = _FakeBlock(b"block0", [1, 2, 3, 4], prev=root)
-    block1 = _FakeBlock(b"block1", [5, 6], prev=block0)
-
-    event_manager.add_stored_block_event_from_block(block0)
-    event_manager.add_removed_event(block0.key)
+    make_block = real_block_factory(event_manager)
+    block0 = make_block([1, 2, 3, 4], [4])
+    block1 = make_block([5, 6], [2], parent=block0)
 
-    assert event_manager._v1_hash_from_radix_block(block1) == 6875034662206558884
-
-    event_manager.add_stored_block_event_from_block(block1)
-    events = _flush_serialized_events(event_manager)
-    stored_events = _stored_events(events)
+    _add_stored_block(event_manager, block0)
+    event_manager.add_removed_event(_block_key(block0))
+    _add_stored_block(event_manager, block1)
+    stored_events = _stored_events(_flush_serialized_events(event_manager))
 
     assert stored_events[-1]["data"]["parent_hash"] == 924206229973855
     assert stored_events[-1]["data"]["blocks"][0]["block_hash"] == 6875034662206558884
@@ -782,21 +764,21 @@ def test_v2_kv_cache_event_manager_v1_hash_algo_matches_cpp_hasher():
     assert KVCacheEventManager._hash_block_key([1, 2, 3, 4], 0, 123, None) == lora_hash
 
 
-def test_v2_kv_cache_event_manager_v1_hash_events_match_cpp_hasher():
+def test_v2_kv_cache_event_manager_v1_hash_events_match_cpp_hasher(real_block_factory):
     _tb = pytest.importorskip("tensorrt_llm.bindings")
     block_key = _tb.internal.batch_manager.BlockKey
     block_key_hasher = _tb.internal.batch_manager.BlockKeyHasher
-    event_manager = KVCacheEventManager(
+    event_manager = NativeKVCacheEventManager(
         max_kv_event_entries=8,
         window_size=128,
         hash_algo=KV_CACHE_HASH_ALGO_V1,
     )
-    root = _FakeRootBlock()
-    block0 = _FakeBlock(b"block0", [1, 2, 3, 4], prev=root)
-    block1 = _FakeBlock(b"block1", [5, 6], prev=block0)
+    make_block = real_block_factory(event_manager)
+    block0 = make_block([1, 2, 3, 4], [4])
+    block1 = make_block([5, 6], [2], parent=block0)
 
-    event_manager.add_stored_block_event_from_block(block0)
-    event_manager.add_stored_block_event_from_block(block1)
+    _add_stored_block(event_manager, block0)
+    _add_stored_block(event_manager, block1)
     events = _flush_serialized_events(event_manager)
 
     parent_hash = block_key_hasher.hash(block_key([1, 2, 3, 4]))
@@ -1031,32 +1013,54 @@ def test_v2_kv_cache_event_manager_serializes_updated_event():
     }
 
 
-def test_v2_kv_cache_event_manager_uses_stored_registry_for_removed_event():
-    event_manager = KVCacheEventManager(max_kv_event_entries=8, window_size=128)
-    block = _FakeBlock(b"\xab\xcd", [1, 2])
-
-    event_manager.add_stored_block_event_from_block(block)
-    block.storage = []
-    event_manager.add_removed_event(block.key)
-    event_manager.add_removed_event(block.key)
-
+def test_v2_kv_cache_event_manager_uses_stored_registry_for_removed_event(
+    real_block_factory,
+):
+    event_manager = NativeKVCacheEventManager(max_kv_event_entries=8, window_size=128)
+    make_block = real_block_factory(event_manager)
+    block = make_block([1, 2], [2])
+    block_hash = _block_key(block).hex()
+
+    _add_stored_block(event_manager, block)
+    event_manager.add_removed_event(_block_key(block))
+    event_manager.add_removed_event(_block_key(block))
     events = _flush_serialized_events(event_manager)
 
     assert [event["data"]["type"] for event in events] == ["stored", "removed"]
     assert [event["layer_group_id"] for event in events] == [0, 0]
-    assert events[0]["data"]["blocks"][0]["block_hash"] == "abcd"
+    assert events[0]["data"]["blocks"][0]["block_hash"] == block_hash
     assert "layer_groups" not in events[0]["data"]["blocks"][0]
-    assert events[1]["data"]["block_hashes"] == ["abcd"]
+    assert events[1]["data"]["block_hashes"] == [block_hash]
     assert "layer_groups" not in events[1]["data"]
 
 
-def test_v2_kv_cache_event_manager_emits_partial_life_cycle_removed_events():
-    event_manager = KVCacheEventManager(max_kv_event_entries=8, window_size=128)
-    block = _FakeBlock(b"\xab\xcd", [1, 2], num_life_cycles=2)
+def test_v2_kv_cache_event_manager_omits_partial_life_cycle_coverage(
+    real_block_factory,
+):
+    event_manager = NativeKVCacheEventManager(max_kv_event_entries=8, window_size=128)
+    make_block = real_block_factory(event_manager, num_life_cycles=2)
+    block = make_block([1, 2], [2, 1])
+
+    _add_stored_block(event_manager, block)
+    events = _flush_serialized_events(event_manager)
+
+    assert [event["layer_group_id"] for event in events] == [0]
+    assert _stored_block_hashes(events) == [_block_key(block).hex()]
+
+    _add_stored_life_cycle(event_manager, block, 1)
+    assert _flush_serialized_events(event_manager) == []
 
-    event_manager.add_stored_block_event_from_block(block)
-    event_manager.add_removed_life_cycle_event(block.key, 0)
 
+def test_v2_kv_cache_event_manager_emits_partial_life_cycle_removed_events(
+    real_block_factory,
+):
+    event_manager = NativeKVCacheEventManager(max_kv_event_entries=8, window_size=128)
+    make_block = real_block_factory(event_manager, num_life_cycles=2)
+    block = make_block([1, 2], [2, 2])
+    block_hash = _block_key(block).hex()
+
+    _add_stored_block(event_manager, block)
+    event_manager.add_removed_life_cycle_event(_block_key(block), 0)
     events = _flush_serialized_events(event_manager)
 
     assert [event["data"]["type"] for event in events] == [
@@ -1065,33 +1069,31 @@ def test_v2_kv_cache_event_manager_emits_partial_life_cycle_removed_events():
         "removed",
     ]
     assert [event["layer_group_id"] for event in events] == [0, 1, 0]
-    assert events[0]["data"]["blocks"][0]["block_hash"] == "abcd"
-    assert events[1]["data"]["blocks"][0]["block_hash"] == "abcd"
-    assert "layer_groups" not in events[0]["data"]["blocks"][0]
-    assert "layer_groups" not in events[1]["data"]["blocks"][0]
-    assert events[2]["data"]["block_hashes"] == ["abcd"]
-    assert "layer_groups" not in events[2]["data"]
-
-    event_manager.add_removed_life_cycle_event(block.key, 1)
-    event_manager.add_removed_life_cycle_event(block.key, 1)
+    assert events[0]["data"]["blocks"][0]["block_hash"] == block_hash
+    assert events[1]["data"]["blocks"][0]["block_hash"] == block_hash
+    assert events[2]["data"]["block_hashes"] == [block_hash]
 
+    event_manager.add_removed_life_cycle_event(_block_key(block), 1)
+    event_manager.add_removed_life_cycle_event(_block_key(block), 1)
     events = _flush_serialized_events(event_manager)
 
     assert [event["data"]["type"] for event in events] == ["removed"]
     assert events[0]["layer_group_id"] == 1
-    assert events[0]["data"]["block_hashes"] == ["abcd"]
-    assert "layer_groups" not in events[0]["data"]
+    assert events[0]["data"]["block_hashes"] == [block_hash]
 
 
-def test_v2_kv_cache_event_manager_whole_block_removal_clears_life_cycle_state():
-    event_manager = KVCacheEventManager(max_kv_event_entries=8, window_size=128)
-    block = _FakeBlock(b"\xab\xce", [1, 2], num_life_cycles=2)
-
-    event_manager.add_stored_block_event_from_block(block)
-    event_manager.add_removed_life_cycle_event(block.key, 0)
-    event_manager.add_removed_event(block.key)
-    event_manager.add_removed_event(block.key)
-
+def test_v2_kv_cache_event_manager_whole_block_removal_clears_life_cycle_state(
+    real_block_factory,
+):
+    event_manager = NativeKVCacheEventManager(max_kv_event_entries=8, window_size=128)
+    make_block = real_block_factory(event_manager, num_life_cycles=2)
+    block = make_block([1, 2], [2, 2])
+    block_hash = _block_key(block).hex()
+
+    _add_stored_block(event_manager, block)
+    event_manager.add_removed_life_cycle_event(_block_key(block), 0)
+    event_manager.add_removed_event(_block_key(block))
+    event_manager.add_removed_event(_block_key(block))
     events = _flush_serialized_events(event_manager)
 
     assert [event["data"]["type"] for event in events] == [
@@ -1101,21 +1103,26 @@ def test_v2_kv_cache_event_manager_whole_block_removal_clears_life_cycle_state()
         "removed",
     ]
     assert [event["layer_group_id"] for event in events] == [0, 1, 0, 1]
-    assert events[2]["data"]["block_hashes"] == ["abce"]
-    assert events[3]["data"]["block_hashes"] == ["abce"]
+    assert events[2]["data"]["block_hashes"] == [block_hash]
+    assert events[3]["data"]["block_hashes"] == [block_hash]
 
 
-def test_v2_kv_cache_event_manager_readds_life_cycle_emits_stored_event():
-    event_manager = KVCacheEventManager(max_kv_event_entries=8, window_size=128)
-    block = _FakeBlock(b"\xab\xcf", [1, 2], num_life_cycles=2)
+def test_v2_kv_cache_event_manager_readds_life_cycle_emits_stored_event(
+    real_block_factory,
+):
+    event_manager = NativeKVCacheEventManager(max_kv_event_entries=8, window_size=128)
+    make_block = real_block_factory(event_manager, num_life_cycles=2)
+    block = make_block([1, 2], [2, 2])
+    block_key = _block_key(block)
+    block_hash = block_key.hex()
 
-    event_manager.add_stored_block_event_from_block(block)
+    _add_stored_block(event_manager, block)
     event_types = [event["data"]["type"] for event in _flush_serialized_events(event_manager)]
     assert event_types == ["stored", "stored"]
 
-    event_manager.add_removed_life_cycle_event(block.key, 0)
-    event_manager.add_stored_life_cycle_event_from_block(block, 0)
-    event_manager.add_removed_life_cycle_event(block.key, 1)
+    event_manager.add_removed_life_cycle_event(block_key, 0)
+    _add_stored_life_cycle(event_manager, block, 0)
+    event_manager.add_removed_life_cycle_event(block_key, 1)
     events = _flush_serialized_events(event_manager)
 
     assert [event["data"]["type"] for event in events] == [
@@ -1124,73 +1131,78 @@ def test_v2_kv_cache_event_manager_readds_life_cycle_emits_stored_event():
         "removed",
     ]
     assert [event["layer_group_id"] for event in events] == [0, 0, 1]
-    assert events[0]["data"]["block_hashes"] == ["abcf"]
-    assert events[1]["data"]["blocks"][0]["block_hash"] == "abcf"
-    assert "layer_groups" not in events[1]["data"]["blocks"][0]
-    assert events[2]["data"]["block_hashes"] == ["abcf"]
+    assert events[0]["data"]["block_hashes"] == [block_hash]
+    assert events[1]["data"]["blocks"][0]["block_hash"] == block_hash
+    assert events[2]["data"]["block_hashes"] == [block_hash]
 
-    event_manager.add_removed_life_cycle_event(block.key, 0)
+    event_manager.add_removed_life_cycle_event(block_key, 0)
     events = _flush_serialized_events(event_manager)
-
     assert [event["data"]["type"] for event in events] == ["removed"]
     assert events[0]["layer_group_id"] == 0
-    assert events[0]["data"]["block_hashes"] == ["abcf"]
-    assert "layer_groups" not in events[0]["data"]
+    assert events[0]["data"]["block_hashes"] == [block_hash]
 
 
-def test_v2_kv_cache_event_manager_omits_partial_life_cycle_coverage():
-    event_manager = KVCacheEventManager(max_kv_event_entries=8, window_size=128)
-    block = _FakeBlock(b"\xab\xd3", [1, 2], num_life_cycles=2)
-    block.storage[1] = _FakePageRef(_FakePage(num_tokens_in_block=1))
+def test_v2_kv_cache_event_manager_omits_partial_life_cycle_coverage(real_block_factory):
+    event_manager = NativeKVCacheEventManager(max_kv_event_entries=8, window_size=128)
+    make_block = real_block_factory(event_manager, num_life_cycles=2)
+    block = make_block([1, 2], [2, 1])
 
-    event_manager.add_stored_block_event_from_block(block)
+    _add_stored_block(event_manager, block)
     events = _flush_serialized_events(event_manager)
 
     assert [event["layer_group_id"] for event in events] == [0]
     assert [token["token_id"] for token in events[0]["data"]["blocks"][0]["tokens"]] == [1, 2]
 
-    event_manager.add_stored_life_cycle_event_from_block(block, 1)
+    _add_stored_life_cycle(event_manager, block, 1)
     assert _flush_serialized_events(event_manager) == []
 
 
-def test_v2_kv_cache_event_manager_reemits_stored_after_all_life_cycles_were_removed():
-    event_manager = KVCacheEventManager(max_kv_event_entries=8, window_size=128)
-    block = _FakeBlock(b"\xab\xd0", [1, 2])
+def test_v2_kv_cache_event_manager_reemits_stored_after_all_life_cycles_were_removed(
+    real_block_factory,
+):
+    event_manager = NativeKVCacheEventManager(max_kv_event_entries=8, window_size=128)
+    make_block = real_block_factory(event_manager)
+    block = make_block([1, 2], [2])
+    block_key = _block_key(block)
 
-    event_manager.add_stored_block_event_from_block(block)
-    event_manager.add_removed_life_cycle_event(block.key, 0)
+    _add_stored_block(event_manager, block)
+    event_manager.add_removed_life_cycle_event(block_key, 0)
     event_types = [event["data"]["type"] for event in _flush_serialized_events(event_manager)]
     assert event_types == ["stored", "removed"]
 
-    event_manager.add_stored_life_cycle_event_from_block(block, 0)
+    _add_stored_life_cycle(event_manager, block, 0)
     events = _flush_serialized_events(event_manager)
 
     assert [event["data"]["type"] for event in events] == ["stored"]
     assert events[0]["layer_group_id"] == 0
-    assert events[0]["data"]["blocks"][0]["block_hash"] == "abd0"
-    assert "layer_groups" not in events[0]["data"]["blocks"][0]
-
+    assert events[0]["data"]["blocks"][0]["block_hash"] == block_key.hex()
 
-def test_v2_kv_cache_event_manager_flushes_removed_before_updated_event():
-    event_manager = KVCacheEventManager(max_kv_event_entries=8, window_size=128)
-    removed_block = _FakeBlock(b"\xab\xd1", [1, 2])
-    updated_block = _FakeBlock(b"\xab\xd2", [3, 4])
 
-    event_manager.add_stored_block_event_from_block(removed_block)
-    event_manager.add_stored_block_event_from_block(updated_block)
+def test_v2_kv_cache_event_manager_flushes_removed_before_updated_event(
+    real_block_factory,
+):
+    event_manager = NativeKVCacheEventManager(max_kv_event_entries=8, window_size=128)
+    make_block = real_block_factory(event_manager)
+    removed_block = make_block([1, 2], [2])
+    updated_block = make_block([3, 4], [2])
+    removed_key = _block_key(removed_block)
+    updated_key = _block_key(updated_block)
+
+    _add_stored_block(event_manager, removed_block)
+    _add_stored_block(event_manager, updated_block)
     _flush_serialized_events(event_manager)
 
-    event_manager.add_removed_event(removed_block.key)
+    event_manager.add_removed_event(removed_key)
     event_manager.add_updated_event(
-        updated_block.key,
-        cache_level=KVCacheEventDiff(old_value=0, new_value=1),
+        updated_key,
+        cache_level=NativeKVCacheEventDiff(old_value=0, new_value=1),
         layer_group_id=0,
     )
     events = _flush_serialized_events(event_manager)
 
     assert [event["data"]["type"] for event in events] == ["removed", "updated"]
-    assert events[0]["data"]["block_hashes"] == ["abd1"]
-    assert events[1]["data"]["block_hash"] == "abd2"
+    assert events[0]["data"]["block_hashes"] == [removed_key.hex()]
+    assert events[1]["data"]["block_hash"] == updated_key.hex()
 
 
 @pytest.mark.skipif(torch is None or not torch.cuda.is_available(), reason="requires CUDA")

From c792b3e8946138bd5696ebc8e2a4938879a7ddfa Mon Sep 17 00:00:00 2001
From: Yao Yao <lowsfer@users.noreply.github.com>
Date: Fri, 31 Jul 2026 05:36:28 +0000
Subject: [PATCH 06/12] [None][fix] order KV cache tree teardown before storage

Signed-off-by: Yao Yao <lowsfer@users.noreply.github.com>
---
 .../batch_manager/kv_cache_manager_v2/blockRadixTree.cpp | 9 +--------
 .../batch_manager/kv_cache_manager_v2/kvCacheManager.h   | 2 +-
 .../runtime/kv_cache_manager_v2/_block_radix_tree.py     | 5 -----
 .../kv_cache_manager_v2/_core/_kv_cache_manager.py       | 6 +++---
 4 files changed, 5 insertions(+), 17 deletions(-)

diff --git a/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/blockRadixTree.cpp b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/blockRadixTree.cpp
index 6122f8f3d333..33ec64d238ba 100644
--- a/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/blockRadixTree.cpp
+++ b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/blockRadixTree.cpp
@@ -584,10 +584,6 @@ SharedPtr<Block> removeSubtree(Block& root)
 
     // Post-order traversal using prev/next links — O(1) extra space.
     // Descend to leaves first, remove on the way back up.
-    // Each block's pages are reclaimed eagerly via releasePages() while the
-    // StorageManager is still alive, rather than deferring to ~Block(): an external
-    // reference can keep a Block alive past StorageManager teardown, after which
-    // page->manager would be dangling. Mirrors Python's remove_subtree().
     while (true)
     {
         // Descend: if the current block has children, go to the first child.
@@ -597,10 +593,7 @@ SharedPtr<Block> removeSubtree(Block& root)
         }
         else
         {
-            current->releasePages();
-            // Remove this block from its parent's next map.
-            // Null prev to detach — the block may outlive the tree if held
-            // externally (e.g., by nanobind/Python shared_ptr).
+            // Remove this block from its parent.s next map and null prev to detach it.
             NodeBase* parent = current->prev;
             BlockKey const currentKey = current->key;
             auto detached = parent->detachNext(currentKey);
diff --git a/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/kvCacheManager.h b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/kvCacheManager.h
index fd2addf9d3af..0f5990862524 100644
--- a/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/kvCacheManager.h
+++ b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/kvCacheManager.h
@@ -334,8 +334,8 @@ class KvCacheManager : public std::enable_shared_from_this<KvCacheManager>
     KVCacheManagerConfig mConfig;
     LifeCycleRegistry mLifeCycles;
     std::shared_ptr<EventSink> mEventSink;
-    std::shared_ptr<BlockRadixTree> mRadixTree;
     std::shared_ptr<StorageManager> mStorage;
+    std::shared_ptr<BlockRadixTree> mRadixTree;
 
     // Weak references to all living KvCaches.
     std::set<KvCache*> mLivingKvCaches;
diff --git a/tensorrt_llm/runtime/kv_cache_manager_v2/_block_radix_tree.py b/tensorrt_llm/runtime/kv_cache_manager_v2/_block_radix_tree.py
index e1a42ed372bb..ffa2461ad285 100644
--- a/tensorrt_llm/runtime/kv_cache_manager_v2/_block_radix_tree.py
+++ b/tensorrt_llm/runtime/kv_cache_manager_v2/_block_radix_tree.py
@@ -195,10 +195,6 @@ def detach_next(parent: "Block | RootBlock", key: BlockKey) -> "Block | None":
 def remove_subtree(root: "Block") -> None:
     # taking O(1) space
     # remove leaf blocks one by one, in post-order
-    # Each block's pages are reclaimed eagerly via _release_pages() while the
-    # StorageManager is still alive, rather than deferring to ~Block()/__del__().
-    # An external reference (e.g. a caller holding a matched Block) can keep a Block
-    # alive past StorageManager teardown, after which page.manager would be dangling.
     removed_block_hashes: list[BlockKey] = []
     tree = try_get_tree(root)
     event_manager = tree.event_manager if tree is not None else None
@@ -207,7 +203,6 @@ def remove_subtree(root: "Block") -> None:
         if block.next:
             block = next(iter(block.next.values()))
         else:
-            block._release_pages()
             removed_block_hashes.append(block.key)
             if block._prev() is None:
                 assert block is root
diff --git a/tensorrt_llm/runtime/kv_cache_manager_v2/_core/_kv_cache_manager.py b/tensorrt_llm/runtime/kv_cache_manager_v2/_core/_kv_cache_manager.py
index 1a8925c1da75..c14e86a871fd 100644
--- a/tensorrt_llm/runtime/kv_cache_manager_v2/_core/_kv_cache_manager.py
+++ b/tensorrt_llm/runtime/kv_cache_manager_v2/_core/_kv_cache_manager.py
@@ -198,8 +198,8 @@ class KVCacheManager:
     __slots__ = (
         "_init_config",
         "_life_cycles",
-        "_radix_tree",
         "_storage",
+        "_radix_tree",
         "_living_kv_caches",
         "_avg_reused_length",
         "_avg_sqr_capacity",
@@ -221,8 +221,8 @@ class KVCacheManager:
     )
     _init_config: KVCacheManagerConfig
     _life_cycles: LifeCycleRegistry
-    _radix_tree: BlockRadixTree
     _storage: StorageManager
+    _radix_tree: BlockRadixTree
     _living_kv_caches: set[rawref.ref[_KVCache]]
     # Eventually we should let the eviction controller evict associated pages together, i.e.
     # when a page eviction makes other pages in the same cache level useless, it should also
@@ -260,7 +260,6 @@ def __init__(
         config = deepcopy(config)
         self._init_config = config
         self._life_cycles = LifeCycleRegistry(config)
-        self._radix_tree = BlockRadixTree(self._life_cycles, config.tokens_per_block, event_manager)
         storage_config = create_storage_config(config)
         self._storage = StorageManager(
             self._life_cycles,
@@ -273,6 +272,7 @@ def __init__(
             event_manager=event_manager,
             max_util_for_resume=config.max_util_for_resume,
         )
+        self._radix_tree = BlockRadixTree(self._life_cycles, config.tokens_per_block, event_manager)
         self._living_kv_caches = set[rawref.ref[_KVCache]]()
         decay = 0.9999
         self._avg_reused_length = MovingAverage(decay)

From 66f6b35b5f3ff9056060718f63a13d4906c0ad7e Mon Sep 17 00:00:00 2001
From: Yao Yao <lowsfer@users.noreply.github.com>
Date: Wed, 5 Aug 2026 12:32:48 +0000
Subject: [PATCH 07/12] [None][fix] harden KVCacheManagerV2 edge cases

Validate multimodal digest lengths at the backend boundary and cover digest-token move ownership. Preserve the text-only fast path when matching committed blocks for planned drops.

Make partial construction and test-helper teardown safe, clarify orphan block page lifetime, and apply the remaining review-driven naming and documentation cleanups.

Signed-off-by: Yao Yao <lowsfer@users.noreply.github.com>
---
 .../kv_cache_manager_v2/AGENTS.md             |  9 +++---
 .../kv_cache_manager_v2/blockRadixTree.cpp    |  6 ++--
 .../kv_cache_manager_v2/blockRadixTree.h      | 11 ++++----
 .../kv_cache_manager_v2/kvCache.cpp           |  2 +-
 .../kv_cache_manager_v2/kvCache.h             |  2 +-
 .../kv_cache_manager_v2/tokenIdExt.cpp        |  2 +-
 .../kv_cache_manager_v2/tokenIdExt.h          |  4 +--
 .../batch_manager/kvCacheManagerV2.cpp        | 15 ++++++++--
 .../kvCacheManagerV2DigestPoolTest.cpp        | 28 +++++++++++++++++++
 .../_torch/pyexecutor/kv_cache_manager_v2.py  |  1 +
 .../kv_cache_manager_v2/_block_radix_tree.py  | 21 ++++++++------
 .../kv_cache_manager_v2/_core/_kv_cache.py    |  8 ++++--
 .../_core/_kv_cache_manager.py                | 16 ++++++++---
 .../test_kv_cache_event_manager.py            |  4 ++-
 .../test_kv_cache_manager_v2.py               |  7 +++++
 15 files changed, 100 insertions(+), 36 deletions(-)

diff --git a/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/AGENTS.md b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/AGENTS.md
index ceb8e5e55e97..001d3296a829 100644
--- a/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/AGENTS.md
+++ b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/AGENTS.md
@@ -159,9 +159,10 @@ Eviction controller
 - `Block::prev`, `Block::storage`, `CommittedPage::block`, and page manager
   pointers are observer/back-reference links with lifetime invariants, not
   ownership. Explicit unlinking and teardown order keep them valid.
-- `Block::releasePages()` must run while `StorageManager` is alive. Do not defer
-  page reclamation solely to `Block` destruction: external block references can
-  outlive tree membership.
+- Orphan blocks may retain pages while a live `KvCache` still references them;
+  `Block::~Block()` reclaims those pages when the last block owner releases them.
+  Every `KvCache` must be closed before manager shutdown so this deferred cleanup
+  runs while `StorageManager` is still alive.
 
 ## Correctness invariants
 
@@ -215,7 +216,7 @@ Eviction controller
 - Keep C++ implementation sources co-located here and add every compiled source
   to this directory's `CMakeLists.txt`. The parent target consumes its source
   list; do not add a separate shared library for this subsystem.
-- SHA-256 support is vendored under `cpp/tensorrt_llm/batch_manager/common/sha256`
+- SHA-256 support is vendored under `cpp/tensorrt_llm/common/sha256`
   and configured by this directory's `CMakeLists.txt`. Preserve the
   architecture-specific SHA extension flags when changing the hash integration.
   Do not add OpenSSL/libcrypto merely for block hashing; avoiding that dependency
diff --git a/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/blockRadixTree.cpp b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/blockRadixTree.cpp
index 33ec64d238ba..77d898f38e44 100644
--- a/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/blockRadixTree.cpp
+++ b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/blockRadixTree.cpp
@@ -284,7 +284,7 @@ namespace
 
 // Takes raw (ptr, size) so it works uniformly over any TokenIdExt buffer
 // (the query vector and the std::vector<TokenIdExt> that backs Block::tokens).
-static bool isPrefix(TokenIdExt const* prefix, size_t prefixLen, TokenIdExt const* full, size_t fullLen)
+bool isPrefix(TokenIdExt const* prefix, size_t prefixLen, TokenIdExt const* full, size_t fullLen)
 {
     if (prefixLen > fullLen)
         return false;
@@ -593,7 +593,7 @@ SharedPtr<Block> removeSubtree(Block& root)
         }
         else
         {
-            // Remove this block from its parent.s next map and null prev to detach it.
+            // Remove this block from its parent's next map and null prev to detach it.
             NodeBase* parent = current->prev;
             BlockKey const currentKey = current->key;
             auto detached = parent->detachNext(currentKey);
@@ -628,7 +628,7 @@ BlockRadixTree::BlockRadixTree(
 
 BlockRadixTree::~BlockRadixTree()
 {
-    // Clear all roots (which will drop all blocks).
+    // Clear all roots (which will drop all blocks without external owners).
     mRoots.clear();
 }
 
diff --git a/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/blockRadixTree.h b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/blockRadixTree.h
index 11f7455fe7c3..aff76497dc40 100644
--- a/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/blockRadixTree.h
+++ b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/blockRadixTree.h
@@ -328,11 +328,10 @@ struct Block : NodeBase, EnableSharedFromThis<Block>
 
     // Reclaim every page held by this block: null each page's back-pointer and, for
     // DROPPABLE pages still scheduled for eviction, remove them from the eviction
-    // controller (releasing their storage slots). Idempotent. Must run during tree
-    // teardown (removeSubtree) rather than being deferred to ~Block(), so page
-    // reclamation does not depend on this Block's destruction timing — an external
-    // reference can keep a Block alive past StorageManager teardown, after which
-    // page->manager would be dangling. Mirrors Python's Block._release_pages().
+    // controller (releasing their storage slots). Idempotent. Cleanup is normally
+    // deferred to ~Block(): an orphan block may remain referenced by a live KvCache,
+    // and every KvCache must close before StorageManager teardown. Mirrors Python's
+    // Block._release_pages().
     void releasePages();
 
 private:
@@ -377,7 +376,7 @@ class BlockRadixTree
     ReuseMatch match(ReuseScope const& reuseScope, TokenSpan tokens, bool knownNoDigest = false,
         bool enablePartialMatch = false) const;
 
-    // Clear all cached pages. ~Block() handles excludeFromEviction for DROPPABLE pages.
+    // Detach all cached blocks. ~Block() releases pages when the last owner drops a block.
     void clear();
 
     int tokensPerBlock() const noexcept
diff --git a/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/kvCache.cpp b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/kvCache.cpp
index 4fe8b6c41d62..94da257033b1 100644
--- a/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/kvCache.cpp
+++ b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/kvCache.cpp
@@ -1906,7 +1906,7 @@ std::unique_ptr<PlannedDropHandle> KvCache::planCommittedBlockDrop()
     if (numCommittedTokens() == 0)
         return nullptr;
 
-    auto const match = mManager->matchReuse(mReuseScope, toSpan(mCommittedTokens));
+    auto const match = mManager->matchReuse(mReuseScope, toSpan(mCommittedTokens), textOnly());
     if (match.numTokens != numCommittedTokens() || match.blocks.empty())
         return nullptr;
 
diff --git a/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/kvCache.h b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/kvCache.h
index 89e0aebe93c5..afb6e3cce152 100644
--- a/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/kvCache.h
+++ b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/kvCache.h
@@ -579,7 +579,7 @@ class KvCache : public std::enable_shared_from_this<KvCache>
     TypedVec<BlockOrdinal, SeqBlock> mBlocks;
 
     std::vector<TokenIdExt> mCommittedTokens;
-    // Per-sequence text-only override; nullopt means inherit the manager config default.
+    // Resolved per-sequence text-only state after applying the manager default.
     bool mTextOnly = false;
     int mNumCommittedBlocks;
     std::optional<CachedCudaEvent> mFinishEvent;
diff --git a/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/tokenIdExt.cpp b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/tokenIdExt.cpp
index 965b0a44b107..90336bfe1113 100644
--- a/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/tokenIdExt.cpp
+++ b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/tokenIdExt.cpp
@@ -256,7 +256,7 @@ Digest const& TokenIdExt::digest() const
     return DigestPool::instance().get(digestIndex());
 }
 
-bool TokenIdExt::operator==(TokenIdExt const& other) const noexcept
+bool TokenIdExt::operator==(TokenIdExt const& other) const
 {
     if (mBits == other.mBits)
     {
diff --git a/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/tokenIdExt.h b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/tokenIdExt.h
index 832641733d6e..888ffc24b9c9 100644
--- a/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/tokenIdExt.h
+++ b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/tokenIdExt.h
@@ -128,9 +128,9 @@ class TokenIdExt
 
     // Value equality: normal/tag-mismatch compare raw bits; digest-vs-digest
     // compares the pooled 32 bytes (equal content in different slots is equal).
-    bool operator==(TokenIdExt const& other) const noexcept;
+    bool operator==(TokenIdExt const& other) const;
 
-    bool operator!=(TokenIdExt const& other) const noexcept
+    bool operator!=(TokenIdExt const& other) const
     {
         return !(*this == other);
     }
diff --git a/cpp/tensorrt_llm/nanobind/batch_manager/kvCacheManagerV2.cpp b/cpp/tensorrt_llm/nanobind/batch_manager/kvCacheManagerV2.cpp
index 216d20cb7887..b54f85c4b51d 100644
--- a/cpp/tensorrt_llm/nanobind/batch_manager/kvCacheManagerV2.cpp
+++ b/cpp/tensorrt_llm/nanobind/batch_manager/kvCacheManagerV2.cpp
@@ -552,7 +552,13 @@ class EventManagerTestBlock
 
     ~EventManagerTestBlock()
     {
-        close();
+        try
+        {
+            close();
+        }
+        catch (...)
+        {
+        }
     }
 
     void close()
@@ -2170,8 +2176,13 @@ void KvCacheManagerV2Bindings::initBindings(nb::module_& m)
         "gen_multimodal_cache_key_tokens",
         [](int idOffset, nb::bytes multiModalDataDigest, int numTokens, int tokenOffset)
         {
+            auto const digestSize = nb::len(multiModalDataDigest);
+            if (digestSize != kv::kDIGEST_LEN)
+            {
+                throw std::invalid_argument("multi_modal_data_digest must have length kDIGEST_LEN");
+            }
             auto const* first = reinterpret_cast<uint8_t const*>(multiModalDataDigest.c_str());
-            std::vector<uint8_t> digest(first, first + nb::len(multiModalDataDigest));
+            std::vector<uint8_t> digest(first, first + digestSize);
             return tokenList(kv::genMultimodalCacheKeyTokens(idOffset, digest, numTokens, tokenOffset));
         },
         nb::arg("id_offset"), nb::arg("multi_modal_data_digest"), nb::arg("num_tokens"), nb::arg("token_offset") = 0);
diff --git a/cpp/tests/unit_tests/batch_manager/kvCacheManagerV2DigestPoolTest.cpp b/cpp/tests/unit_tests/batch_manager/kvCacheManagerV2DigestPoolTest.cpp
index 9b5ac0fbfda7..310f96499860 100644
--- a/cpp/tests/unit_tests/batch_manager/kvCacheManagerV2DigestPoolTest.cpp
+++ b/cpp/tests/unit_tests/batch_manager/kvCacheManagerV2DigestPoolTest.cpp
@@ -26,6 +26,8 @@
 #include <cstddef>
 #include <cstdint>
 #include <cstring>
+#include <stdexcept>
+#include <utility>
 #include <vector>
 
 namespace
@@ -107,6 +109,32 @@ TEST(DigestPoolTest, CopyDigestTokenClonesSlot)
     EXPECT_EQ(detail::digestPoolLiveCount(), baseline); // both slots freed
 }
 
+// Moving transfers ownership without cloning or double-freeing a digest slot.
+TEST(DigestPoolTest, MoveTransfersSlotWithoutCloning)
+{
+    size_t const baseline = detail::digestPoolLiveCount();
+    Digest const bytes = makeDigest(std::byte{0x33});
+    Digest const other = makeDigest(std::byte{0x77});
+    {
+        TokenIdExt source(bytes);
+        EXPECT_EQ(detail::digestPoolLiveCount(), baseline + 1);
+
+        TokenIdExt moved(std::move(source));
+        EXPECT_EQ(detail::digestPoolLiveCount(), baseline + 1);
+        EXPECT_EQ(source.raw(), TokenIdExt::kBadToken);
+        EXPECT_EQ(moved.digest(), bytes);
+        EXPECT_THROW((void) (source == moved), std::out_of_range);
+
+        TokenIdExt target(other);
+        EXPECT_EQ(detail::digestPoolLiveCount(), baseline + 2);
+        target = std::move(moved);
+        EXPECT_EQ(detail::digestPoolLiveCount(), baseline + 1);
+        EXPECT_EQ(moved.raw(), TokenIdExt::kBadToken);
+        EXPECT_EQ(target.digest(), bytes);
+    }
+    EXPECT_EQ(detail::digestPoolLiveCount(), baseline);
+}
+
 // clone-on-copy: the copy owns an independent slot; destroying the original
 // leaves the copy valid, and slots return to the free-list (liveCount delta 0).
 TEST(DigestPoolTest, CloneIndependenceAndFreeReuse)
diff --git a/tensorrt_llm/_torch/pyexecutor/kv_cache_manager_v2.py b/tensorrt_llm/_torch/pyexecutor/kv_cache_manager_v2.py
index 47b964348646..b5aa5225e29d 100644
--- a/tensorrt_llm/_torch/pyexecutor/kv_cache_manager_v2.py
+++ b/tensorrt_llm/_torch/pyexecutor/kv_cache_manager_v2.py
@@ -66,6 +66,7 @@
     ReuseScope,
     SwaScratchReuseConfig,
     TokenIdExt,
+    _cpp_introspection,
     _introspection,
     _KVCache,
     exact_div,
diff --git a/tensorrt_llm/runtime/kv_cache_manager_v2/_block_radix_tree.py b/tensorrt_llm/runtime/kv_cache_manager_v2/_block_radix_tree.py
index ffa2461ad285..f42bb5b8fb6d 100644
--- a/tensorrt_llm/runtime/kv_cache_manager_v2/_block_radix_tree.py
+++ b/tensorrt_llm/runtime/kv_cache_manager_v2/_block_radix_tree.py
@@ -13,10 +13,10 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-from itertools import chain
 import hashlib
 import itertools
 from array import array
+from itertools import chain
 from typing import TYPE_CHECKING, Iterable, Iterator, NamedTuple, Sequence, TypeVar, cast
 
 from . import rawref
@@ -32,6 +32,11 @@
 BlockKey = bytes
 TokenBlock = list[TokenIdExt]
 
+_SHA256_DIGEST_SIZE = hashlib.sha256().digest_size
+_UINT_ITEM_SIZE = array("I").itemsize
+if _UINT_ITEM_SIZE != 4:
+    raise RuntimeError("Hasher requires a platform with 4-byte unsigned ints")
+
 
 # id_offset is usually vocab_size. Backend-neutral (depends only on _common); the
 # C++ backend exposes a native gen_multimodal_cache_key_tokens via nanobind instead.
@@ -42,6 +47,8 @@ def gen_multimodal_cache_key_tokens(
 
     Item-local token 0 carries the content digest; later offsets use deterministic IDs above the vocab.
     """
+    if len(multi_modal_data_digest) != _SHA256_DIGEST_SIZE:
+        raise ValueError(f"multi_modal_data_digest must have length {_SHA256_DIGEST_SIZE}")
     assert num_tokens > 0
     assert token_offset >= 0
     return [
@@ -65,7 +72,7 @@ class Hasher:
     __slots__ = "_hasher"
     _hasher: "hashlib._Hash"
 
-    def __init__(self, data: int | bytes | None | Sequence[int | bytes] = None) -> None:
+    def __init__(self, data: int | bytes | Sequence[int | bytes] | None = None) -> None:
         self._hasher = hashlib.sha256()
         if data is not None:
             self.update(data)
@@ -455,11 +462,9 @@ def _release_pages(self) -> None:
         Idempotent: afterwards ``storage`` holds no pages, so it is safe to call again
         from ``__del__``.
 
-        This must run during radix-tree teardown (``remove_subtree``/``clear``) rather
-        than being deferred to ``__del__``, so that page reclamation does not depend on
-        this ``Block`` object's destruction timing. An external reference can keep the
-        ``Block`` alive past ``StorageManager`` teardown, after which ``page.manager``
-        would be a dangling reference.
+        Cleanup is normally deferred to ``__del__``. An orphan block may remain
+        referenced by a live ``_KVCache`` and retain its pages until that cache closes;
+        every cache must close before ``StorageManager`` teardown.
         """
         for lc_idx in typed_range(self.num_life_cycles):
             page = self.get_page(lc_idx)
@@ -618,7 +623,7 @@ def num_life_cycles(self) -> LifeCycleId:
     def clear(self) -> None:
         # taking O(1) space
         # remove leaf blocks one by one, in post-order
-        # ~Block() / __del__() handles page cleanup.
+        # Block.__del__() handles page cleanup when the last owner releases each block.
         # detach_next() auto-prunes empty RootBlocks from the tree.
         while self.next:
             root = next(iter(self.next.values()))
diff --git a/tensorrt_llm/runtime/kv_cache_manager_v2/_core/_kv_cache.py b/tensorrt_llm/runtime/kv_cache_manager_v2/_core/_kv_cache.py
index 8d52a86a1132..4061af81297f 100644
--- a/tensorrt_llm/runtime/kv_cache_manager_v2/_core/_kv_cache.py
+++ b/tensorrt_llm/runtime/kv_cache_manager_v2/_core/_kv_cache.py
@@ -307,13 +307,15 @@ def __init__(
         custom_priority_callback: Callable[[BlockOrdinal, LifeCycle], Priority],
         expected_prompt_length: int | None = None,
         text_only: bool | None = None,
-    ):
+    ) -> None:
+        # Keep a partially constructed cache inert if validation fails.
+        self.__rawref__ = rawref.NULL
+        self._status = self.Status.CLOSED
         self.id = id
         self._manager = manager
         self._reuse_scope = reuse_scope
         self._get_priority = custom_priority_callback
         self._cuda_stream = None
-        self._status = self.Status.SUSPENDED
         self._beam_width = BeamIndex(1)
         self._expected_prompt_length = (
             max(expected_prompt_length, 0) if expected_prompt_length is not None else None
@@ -346,7 +348,7 @@ def __init__(
             lambda _: list[ScratchSlotLock](), manager._storage.num_life_cycles
         )
         self._pending_stats = _PendingStats()
-        self.__rawref__ = rawref.NULL
+        self._status = self.Status.SUSPENDED
         if reuse_match is not None:
             self._setup_for_reuse(reuse_match)
         self._refresh_generation_alloc_ready()
diff --git a/tensorrt_llm/runtime/kv_cache_manager_v2/_core/_kv_cache_manager.py b/tensorrt_llm/runtime/kv_cache_manager_v2/_core/_kv_cache_manager.py
index c14e86a871fd..a7ec42bec54b 100644
--- a/tensorrt_llm/runtime/kv_cache_manager_v2/_core/_kv_cache_manager.py
+++ b/tensorrt_llm/runtime/kv_cache_manager_v2/_core/_kv_cache_manager.py
@@ -261,7 +261,7 @@ def __init__(
         self._init_config = config
         self._life_cycles = LifeCycleRegistry(config)
         storage_config = create_storage_config(config)
-        self._storage = StorageManager(
+        storage = StorageManager(
             self._life_cycles,
             storage_config,
             config.tokens_per_block,
@@ -272,7 +272,9 @@ def __init__(
             event_manager=event_manager,
             max_util_for_resume=config.max_util_for_resume,
         )
-        self._radix_tree = BlockRadixTree(self._life_cycles, config.tokens_per_block, event_manager)
+        radix_tree = BlockRadixTree(self._life_cycles, config.tokens_per_block, event_manager)
+        self._storage = storage
+        self._radix_tree = radix_tree
         self._living_kv_caches = set[rawref.ref[_KVCache]]()
         decay = 0.9999
         self._avg_reused_length = MovingAverage(decay)
@@ -314,8 +316,14 @@ def _check_no_living_kv_caches(self, api: str) -> None:
 
     def shutdown(self) -> None:
         self._check_no_living_kv_caches("shutdown()")
-        self.clear_reusable_blocks()
-        self._storage.destroy()
+        # A failed constructor may leave either owner unset. Release tree pages
+        # before destroying storage whenever the corresponding objects exist.
+        radix_tree = getattr(self, "_radix_tree", None)
+        if radix_tree is not None:
+            radix_tree.clear()
+        storage = getattr(self, "_storage", None)
+        if storage is not None:
+            storage.destroy()
 
     def clear_reusable_blocks(self) -> None:
         self._check_no_living_kv_caches("clear_reusable_blocks()")
diff --git a/tests/unittest/kv_cache_manager_v2_tests/test_kv_cache_event_manager.py b/tests/unittest/kv_cache_manager_v2_tests/test_kv_cache_event_manager.py
index 13160a140016..5ac4b2de9865 100644
--- a/tests/unittest/kv_cache_manager_v2_tests/test_kv_cache_event_manager.py
+++ b/tests/unittest/kv_cache_manager_v2_tests/test_kv_cache_event_manager.py
@@ -1142,7 +1142,9 @@ def test_v2_kv_cache_event_manager_readds_life_cycle_emits_stored_event(
     assert events[0]["data"]["block_hashes"] == [block_hash]
 
 
-def test_v2_kv_cache_event_manager_omits_partial_life_cycle_coverage(real_block_factory):
+def test_v2_kv_cache_event_manager_serializes_only_fully_covered_life_cycle_tokens(
+    real_block_factory,
+):
     event_manager = NativeKVCacheEventManager(max_kv_event_entries=8, window_size=128)
     make_block = real_block_factory(event_manager, num_life_cycles=2)
     block = make_block([1, 2], [2, 1])
diff --git a/tests/unittest/kv_cache_manager_v2_tests/test_kv_cache_manager_v2.py b/tests/unittest/kv_cache_manager_v2_tests/test_kv_cache_manager_v2.py
index c093ff586674..4939dc35d52c 100755
--- a/tests/unittest/kv_cache_manager_v2_tests/test_kv_cache_manager_v2.py
+++ b/tests/unittest/kv_cache_manager_v2_tests/test_kv_cache_manager_v2.py
@@ -54,6 +54,7 @@
         TokenIdExt,
         _introspection,
         _KVCache,
+        gen_multimodal_cache_key_tokens,
     )
     from kv_cache_manager_v2._block_radix_tree import Hasher
     from kv_cache_manager_v2._common import (
@@ -107,6 +108,7 @@
         TokenIdExt,
         _introspection,
         _KVCache,
+        gen_multimodal_cache_key_tokens,
     )
     from tensorrt_llm.runtime.kv_cache_manager_v2._block_radix_tree import Hasher
     from tensorrt_llm.runtime.kv_cache_manager_v2._common import (
@@ -4386,6 +4388,11 @@ def test_update_mixed_multimodal_block(self) -> None:
         seed = b"\x01"
         self.assertEqual(Hasher(seed).update(block).digest, self._ref_update(seed, block))
 
+    def test_multimodal_digest_requires_sha256_length(self) -> None:
+        for digest_size in (31, 33):
+            with self.subTest(digest_size=digest_size), self.assertRaises(ValueError):
+                gen_multimodal_cache_key_tokens(100, bytes(digest_size), 1)
+
 
 if __name__ == "__main__":
     unittest.main()

From 27693ca0124dcc1bdea08590b2a54936fa81dee3 Mon Sep 17 00:00:00 2001
From: Yao Yao <lowsfer@users.noreply.github.com>
Date: Wed, 5 Aug 2026 12:57:29 +0000
Subject: [PATCH 08/12] [None][fix] address KVCacheManagerV2 review follow-ups

Initialize default Span views safely and document their non-owning lifetime contract. Bring isPrefix in line with control-flow style requirements.

Document the per-cache text_only override and strengthen digest-pool coverage to verify reclaimed-slot reuse.

Signed-off-by: Yao Yao <lowsfer@users.noreply.github.com>
---
 .../kv_cache_manager_v2/blockRadixTree.cpp    |  4 +++
 .../kv_cache_manager_v2/common.h              | 29 +++++++++++--------
 .../kvCacheManagerV2DigestPoolTest.cpp        |  3 ++
 .../_core/_kv_cache_manager.py                | 25 +++++++++++-----
 4 files changed, 42 insertions(+), 19 deletions(-)

diff --git a/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/blockRadixTree.cpp b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/blockRadixTree.cpp
index 77d898f38e44..a6db17976f30 100644
--- a/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/blockRadixTree.cpp
+++ b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/blockRadixTree.cpp
@@ -287,11 +287,15 @@ namespace
 bool isPrefix(TokenIdExt const* prefix, size_t prefixLen, TokenIdExt const* full, size_t fullLen)
 {
     if (prefixLen > fullLen)
+    {
         return false;
+    }
     for (size_t i = 0; i < prefixLen; ++i)
     {
         if (prefix[i] != full[i])
+        {
             return false;
+        }
     }
     return true;
 }
diff --git a/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/common.h b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/common.h
index 2fc12c04137e..459997a042e3 100644
--- a/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/common.h
+++ b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/common.h
@@ -124,16 +124,17 @@ inline constexpr Priority kPriorityDefault = 35;
 using SlidingWindowSize = std::optional<int>;
 
 // ---------------------------------------------------------------------------
-// Span<T> — non-owning view into a contiguous buffer (a C++17 stand-in for
-// std::span). Aggregate, so `Span<T>{}` value-initializes to an empty view and
-// `Span<T>{ptr, len}` is a plain brace-init. Supports operator[] for uniform
-// access with std::vector<T>.
+//! Non-owning view into a contiguous buffer (a C++17 stand-in for std::span).
+//!
+//! The referenced buffer must outlive the view. This remains an aggregate, so
+//! `Span<T>{}` creates an empty view and `Span<T>{ptr, len}` is plain brace-init.
+//! Supports operator[] for uniform access with std::vector<T>.
 // ---------------------------------------------------------------------------
 template <typename T>
 struct Span
 {
-    T* ptr;
-    int len;
+    T* ptr = nullptr;
+    int len = 0;
 
     T& operator[](int idx)
     {
@@ -166,8 +167,10 @@ struct Span
     }
 };
 
-// Non-owning const Span over a std::vector — for call sites that hold an owning vector but
-// need a Span (e.g. the per-element/multimodal fallback).
+//! Create a non-owning const Span over a std::vector.
+//!
+//! The source vector must outlive the returned view and must not reallocate while
+//! the view is in use.
 template <typename T>
 inline Span<T const> toSpan(std::vector<T> const& vec) noexcept
 {
@@ -175,10 +178,12 @@ inline Span<T const> toSpan(std::vector<T> const& vec) noexcept
     return Span<T const>{vec.data(), static_cast<int>(vec.size())};
 }
 
-// Non-owning view of a token sequence — a C++17 stand-in for std::span<TokenIdExt const>.
-// Used on the hot ingest path: a digest-free int32 token buffer can be reinterpret_cast to
-// TokenIdExt const* and matched/hashed with no per-token copy. TokenIdExt is 4 bytes and
-// bit-identical to a normal int32 token (see tokenIdExt.h).
+//! Non-owning view of a token sequence; the source buffer must outlive the view.
+//!
+//! Used on the hot ingest path: a digest-free int32 token buffer can be
+//! reinterpret_cast to TokenIdExt const* and matched/hashed with no per-token
+//! copy. TokenIdExt is 4 bytes and bit-identical to a normal int32 token (see
+//! tokenIdExt.h).
 using TokenSpan = Span<TokenIdExt const>;
 
 // ---------------------------------------------------------------------------
diff --git a/cpp/tests/unit_tests/batch_manager/kvCacheManagerV2DigestPoolTest.cpp b/cpp/tests/unit_tests/batch_manager/kvCacheManagerV2DigestPoolTest.cpp
index 310f96499860..6f08ba5878f6 100644
--- a/cpp/tests/unit_tests/batch_manager/kvCacheManagerV2DigestPoolTest.cpp
+++ b/cpp/tests/unit_tests/batch_manager/kvCacheManagerV2DigestPoolTest.cpp
@@ -199,6 +199,8 @@ TEST(DigestPoolTest, FrontPackAndTailShrinkChurn)
         EXPECT_EQ(tokens[i].digest(), distinctDigest(i)) << "content at " << i;
     }
 
+    uint32_t const reclaimedSlot = tokens.front().raw();
+
     // Release the first half (front slots), then confirm the survivors are intact
     // after the shrink churn triggered by freeing.
     tokens.erase(tokens.begin(), tokens.begin() + count / 2);
@@ -210,6 +212,7 @@ TEST(DigestPoolTest, FrontPackAndTailShrinkChurn)
 
     // A fresh allocation must front-pack into a reclaimed low slot.
     TokenIdExt const refill(distinctDigest(9999));
+    EXPECT_EQ(refill.raw(), reclaimedSlot);
     EXPECT_EQ(refill.digest(), distinctDigest(9999));
 
     tokens.clear();
diff --git a/tensorrt_llm/runtime/kv_cache_manager_v2/_core/_kv_cache_manager.py b/tensorrt_llm/runtime/kv_cache_manager_v2/_core/_kv_cache_manager.py
index a7ec42bec54b..d6d96eb75079 100644
--- a/tensorrt_llm/runtime/kv_cache_manager_v2/_core/_kv_cache_manager.py
+++ b/tensorrt_llm/runtime/kv_cache_manager_v2/_core/_kv_cache_manager.py
@@ -420,13 +420,24 @@ def create_kv_cache(
         text_only: bool | None = None,
     ) -> _KVCache:
         """
-        reuse_scope: namespace to match before matching any tokens.
-        custom_priority_callback: takes block index and layer sliding window size, returns priority.
-        If priority returned is higher than existing priority for reused blocks, the block priority is updated.
-        expected_prompt_length: optional token count marking the prefill->generation
-            boundary; once history_length reaches it, subsequent capacity growth is
-            recorded as generation-phase allocation stats (defaults to len(input_tokens)).
-            Stats-only: does not affect allocation, reuse, or correctness.
+        Args:
+            reuse_scope: Namespace to match before matching any tokens.
+            input_tokens: Optional initial tokens used for reuse matching.
+            id: Optional cache identifier.
+            custom_priority_callback: Takes a block index and layer sliding-window
+                size and returns a priority. Reused blocks are updated when the
+                returned priority is higher than their existing priority.
+            expected_prompt_length: Optional token count marking the
+                prefill-to-generation boundary. Once history length reaches it,
+                subsequent capacity growth is recorded as generation-phase
+                allocation statistics. Defaults to the length of ``input_tokens``
+                and does not affect allocation, reuse, or correctness.
+            text_only: Optional per-cache override for the manager setting. ``True``
+                enables digest-free fast paths and requires all tokens to be text
+                token IDs; ``False`` permits digest tokens but is invalid when the
+                manager is configured with ``text_only=True``; ``None`` inherits
+                the manager setting.
+
         Newly created KV cache is suspended. You need to call resume() with a cuda stream to make it active
         & ready in that stream.
         Returns None if suspended=False and we don't have enough resource.

From ab865ac8155abdbdbf93dddff27afbc6c610f511 Mon Sep 17 00:00:00 2001
From: Yao Yao <lowsfer@users.noreply.github.com>
Date: Wed, 5 Aug 2026 15:06:47 +0000
Subject: [PATCH 09/12] [None][fix] drop stale SsmCommittedPage references in
 KVCacheManagerV2 introspection

SsmCommittedPage was folded into CommittedPage, whose num_tokens_in_block now
carries the token count for both attention and SSM life cycles. Two introspection
helpers still referenced the removed class, breaking the build.

- nanobind make_test_block: collapse the two-branch construction into a single
  CommittedPage. This also fixes the argument order, which passed Priority where
  the constructor expects numTokensInBlock.
- nanobind reuse_match_pages: read page->numTokensInBlock directly instead of
  down-casting; block storage already holds CommittedPage pointers. Attention
  pages now report their real token count rather than -1, matching the Python
  backend, which has always returned it.
- Python make_test_block: drop the SsmCommittedPage import, which would have
  raised ImportError on the pure-Python backend.

Signed-off-by: Yao Yao <lowsfer@users.noreply.github.com>
---
 .../batch_manager/kvCacheManagerV2.cpp         | 18 +++---------------
 .../kv_cache_manager_v2/_introspection.py      | 11 +++--------
 2 files changed, 6 insertions(+), 23 deletions(-)

diff --git a/cpp/tensorrt_llm/nanobind/batch_manager/kvCacheManagerV2.cpp b/cpp/tensorrt_llm/nanobind/batch_manager/kvCacheManagerV2.cpp
index b54f85c4b51d..0ef06a19c9ce 100644
--- a/cpp/tensorrt_llm/nanobind/batch_manager/kvCacheManagerV2.cpp
+++ b/cpp/tensorrt_llm/nanobind/batch_manager/kvCacheManagerV2.cpp
@@ -1729,17 +1729,8 @@ void KvCacheManagerV2Bindings::initBindings(nb::module_& m)
                 {
                     continue;
                 }
-                kv::SharedPtr<kv::CommittedPage> page;
-                if (coverage < static_cast<int>(block->tokens.size()))
-                {
-                    page = kv::makeShared<kv::SsmCommittedPage>(
-                        &manager.storage(), block, lifeCycle, kv::kGpuLevel, kv::kPriorityDefault, coverage);
-                }
-                else
-                {
-                    page = kv::makeShared<kv::CommittedPage>(
-                        &manager.storage(), block, lifeCycle, kv::kGpuLevel, kv::kPriorityDefault);
-                }
+                auto page = kv::makeShared<kv::CommittedPage>(
+                    &manager.storage(), block, lifeCycle, kv::kGpuLevel, coverage, kv::kPriorityDefault);
                 page->setSlot(slots[lifeCycle].front());
                 block->storage[lifeCycle] = page.get();
                 pages.push_back(std::move(page));
@@ -1891,10 +1882,7 @@ void KvCacheManagerV2Bindings::initBindings(nb::module_& m)
                             continue;
                         }
                         int const slotId = page->slotId().value();
-                        int numTokensInBlock = -1;
-                        if (auto* ssm = dynamic_cast<kv::SsmCommittedPage*>(page))
-                            numTokensInBlock = ssm->numTokensInBlock;
-                        pages.emplace_back(std::make_pair(slotId, numTokensInBlock));
+                        pages.emplace_back(std::make_pair(slotId, page->numTokensInBlock));
                     }
                 });
             return std::make_tuple(numTokens, std::move(pages));
diff --git a/tensorrt_llm/runtime/kv_cache_manager_v2/_introspection.py b/tensorrt_llm/runtime/kv_cache_manager_v2/_introspection.py
index b8b28f786835..d8e535d93aab 100644
--- a/tensorrt_llm/runtime/kv_cache_manager_v2/_introspection.py
+++ b/tensorrt_llm/runtime/kv_cache_manager_v2/_introspection.py
@@ -61,7 +61,7 @@ def make_test_block(
     from . import rawref
     from ._block_radix_tree import ReuseScope, _add_or_get_existing
     from ._common import GPU_LEVEL, PRIORITY_DEFAULT
-    from ._page import CommittedPage, SsmCommittedPage
+    from ._page import CommittedPage
 
     token_list = list(tokens)
     if not token_list:
@@ -88,20 +88,15 @@ def make_test_block(
     for life_cycle, coverage in enumerate(coverage_per_lc):
         if coverage == 0:
             continue
-        page_type = SsmCommittedPage if coverage < len(token_list) else CommittedPage
-        page_args = (
+        page = CommittedPage(
             manager._storage,
             block,
             life_cycle,
             GPU_LEVEL,
             slots[life_cycle].pop(),
+            coverage,
             PRIORITY_DEFAULT,
         )
-        page = (
-            page_type(*page_args, coverage)
-            if page_type is SsmCommittedPage
-            else page_type(*page_args)
-        )
         block.storage[life_cycle] = rawref.ref(page)
         pages.append(page)
     return _TestBlock(block, pages)

From 44eb9fa7f209a51159ed6091a9e25a0502e3d2f2 Mon Sep 17 00:00:00 2001
From: Yao Yao <lowsfer@users.noreply.github.com>
Date: Wed, 5 Aug 2026 18:12:26 +0000
Subject: [PATCH 10/12] [None][fix] add get_tokens_view to KVCacheManagerV2
 test request fakes

The zero-copy token ingest path calls req.get_tokens_view() whenever the C++
backend is active, but the duck-typed request fakes used by the KVCacheManagerV2
tests only implement get_tokens, so every test driving the block-reuse path
failed with AttributeError.

Mirror the binding on all three fakes: return a contiguous 1-D int32 ndarray,
whose dtype is what selects the C++ int32 ingest fast path. Two are classes
(_StatsRequest, _ContextRequest); the third is an inline SimpleNamespace in
test_try_commit_blocks_commits_partial_block_at_context_end.

That last test also compared the committed tokens against a list. Once the
token source is a view, commit() receives an ndarray slice and the comparison
becomes elementwise, so assert on list(...) to keep it valid under either
backend without weakening it.

Signed-off-by: Yao Yao <lowsfer@users.noreply.github.com>
---
 .../_torch/executor/test_kv_cache_manager_v2.py | 17 ++++++++++++++++-
 .../test_kv_cache_stats_behavior.py             | 10 ++++++++++
 2 files changed, 26 insertions(+), 1 deletion(-)

diff --git a/tests/unittest/_torch/executor/test_kv_cache_manager_v2.py b/tests/unittest/_torch/executor/test_kv_cache_manager_v2.py
index 89a8b81fdc99..f90bced3c025 100644
--- a/tests/unittest/_torch/executor/test_kv_cache_manager_v2.py
+++ b/tests/unittest/_torch/executor/test_kv_cache_manager_v2.py
@@ -17,6 +17,7 @@
 from types import SimpleNamespace
 from unittest.mock import patch
 
+import numpy as np
 import pytest
 import torch
 
@@ -219,6 +220,9 @@ def test_try_commit_blocks_commits_partial_block_at_context_end() -> None:
         context_current_position=10,
         context_remaining_length=0,
         get_tokens=lambda beam_id: list(range(10)),
+        # The C++ backend takes get_tokens_view on this path; it yields a contiguous
+        # 1-D int32 view, so commit() sees an ndarray slice rather than a list.
+        get_tokens_view=lambda beam_id: np.arange(10, dtype=np.int32),
     )
     kv_cache = _FakeKVCache(num_committed_tokens=4)
     manager = object.__new__(KVCacheManagerV2)
@@ -229,7 +233,9 @@ def test_try_commit_blocks_commits_partial_block_at_context_end() -> None:
 
     manager.try_commit_blocks(request)
 
-    assert kv_cache.committed_tokens == [4, 5, 6, 7, 8, 9]
+    # list() so the assertion holds whichever token source the active backend used:
+    # a plain list (Python backend) or an int32 ndarray slice (C++ backend).
+    assert list(kv_cache.committed_tokens) == [4, 5, 6, 7, 8, 9]
     assert kv_cache.num_committed_tokens == 10
     assert kv_cache.stopped_committing
 
@@ -280,6 +286,15 @@ def get_tokens(self, beam_id: int = DEFAULT_BEAM_INDEX) -> list[int]:
         assert beam_id == DEFAULT_BEAM_INDEX
         return self.tokens
 
+    def get_tokens_view(self, beam_id: int = DEFAULT_BEAM_INDEX) -> np.ndarray:
+        """Mirror LlmRequest.get_tokens_view, which the C++ backend takes on the reuse path.
+
+        The real binding returns a zero-copy contiguous 1-D int32 view of the token buffer;
+        the dtype matters because it selects the C++ int32 ingest fast path.
+        """
+        assert beam_id == DEFAULT_BEAM_INDEX
+        return np.asarray(self.tokens, dtype=np.int32)
+
     def set_prepopulated_prompt_len(self, length: int, tokens_per_block: int) -> None:
         self.prepopulated_prompt = (length, tokens_per_block)
 
diff --git a/tests/unittest/kv_cache_manager_v2_tests/test_kv_cache_stats_behavior.py b/tests/unittest/kv_cache_manager_v2_tests/test_kv_cache_stats_behavior.py
index d848244d16f5..12d5ae5a256d 100644
--- a/tests/unittest/kv_cache_manager_v2_tests/test_kv_cache_stats_behavior.py
+++ b/tests/unittest/kv_cache_manager_v2_tests/test_kv_cache_stats_behavior.py
@@ -16,6 +16,7 @@
 from dataclasses import dataclass, field
 from types import SimpleNamespace
 
+import numpy as np
 import pytest
 import torch
 
@@ -84,6 +85,15 @@ def get_tokens(self, beam_id: int = DEFAULT_BEAM_INDEX) -> list[int]:
         assert beam_id == DEFAULT_BEAM_INDEX
         return self.tokens
 
+    def get_tokens_view(self, beam_id: int = DEFAULT_BEAM_INDEX) -> np.ndarray:
+        """Mirror LlmRequest.get_tokens_view, which the C++ backend takes on the reuse path.
+
+        The real binding returns a zero-copy contiguous 1-D int32 view of the token buffer;
+        the dtype matters because it selects the C++ int32 ingest fast path.
+        """
+        assert beam_id == DEFAULT_BEAM_INDEX
+        return np.asarray(self.tokens, dtype=np.int32)
+
     def set_prepopulated_prompt_len(self, length: int, tokens_per_block: int) -> None:
         self.prepopulated_prompt = (length, tokens_per_block)
 

From 4fbd212c50d7066916fe6682e7e53b97077a7ca2 Mon Sep 17 00:00:00 2001
From: Yao Yao <lowsfer@users.noreply.github.com>
Date: Thu, 6 Aug 2026 04:46:28 +0000
Subject: [PATCH 11/12] [None][fix] address KVCacheManagerV2 review feedback

- commit(): debug-check that a text-only KV cache never ingests digest tokens.
  text_only lets block-key hashing skip the digest scan, so a digest slipping in
  would be keyed by its DigestPool slot index instead of its content. The check
  is debug-only because avoiding that O(n) scan on the commit hot path is the
  point of the flag; setTextOnly() already verifies the claim eagerly.
- gen_multimodal_cache_key_tokens(): raise ValueError instead of asserting, so
  the checks survive python -O, matching the adjacent digest-length validation.
- Document Hasher, Hasher.update and gen_multimodal_cache_key_tokens, including
  the integer encoding that both backends must agree on.
- Annotate the commit_prompt test helper, which takes a token sequence or a
  contiguous int32 ndarray.
- DigestPool test: scope the refill token so the test also proves its slot
  returns to the pool, instead of ending with the allocation still alive.

Signed-off-by: Yao Yao <lowsfer@users.noreply.github.com>
---
 .../kv_cache_manager_v2/kvCache.cpp           |  4 ++
 .../kvCacheManagerV2DigestPoolTest.cpp        | 17 ++++---
 .../kv_cache_manager_v2/_block_radix_tree.py  | 46 +++++++++++++++++--
 .../test_kv_cache_manager_v2.py               |  4 +-
 4 files changed, 60 insertions(+), 11 deletions(-)

diff --git a/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/kvCache.cpp b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/kvCache.cpp
index 94da257033b1..c6a1c51c5388 100644
--- a/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/kvCache.cpp
+++ b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/kvCache.cpp
@@ -1731,6 +1731,10 @@ void KvCache::commit(TokenSpan tokens, bool isEnd)
     if (mCommitState == CommitState::USER_STOP)
         throw LogicError("Cannot commit tokens after stop_committing()");
 
+    TLLM_CHECK_DEBUG_WITH_INFO(
+        !mTextOnly || std::none_of(tokens.begin(), tokens.end(), [](TokenIdExt const& t) { return t.isDigest(); }),
+        "Cannot commit digest tokens to a text-only KV cache");
+
     bool const commitMinSnapshot = mManager->commitMinSnapshot();
     auto ssmLcId = mManager->lifeCycles().ssmLifeCycleId();
     if (commitMinSnapshot)
diff --git a/cpp/tests/unit_tests/batch_manager/kvCacheManagerV2DigestPoolTest.cpp b/cpp/tests/unit_tests/batch_manager/kvCacheManagerV2DigestPoolTest.cpp
index 6f08ba5878f6..77f03b74e490 100644
--- a/cpp/tests/unit_tests/batch_manager/kvCacheManagerV2DigestPoolTest.cpp
+++ b/cpp/tests/unit_tests/batch_manager/kvCacheManagerV2DigestPoolTest.cpp
@@ -211,13 +211,18 @@ TEST(DigestPoolTest, FrontPackAndTailShrinkChurn)
     }
 
     // A fresh allocation must front-pack into a reclaimed low slot.
-    TokenIdExt const refill(distinctDigest(9999));
-    EXPECT_EQ(refill.raw(), reclaimedSlot);
-    EXPECT_EQ(refill.digest(), distinctDigest(9999));
+    {
+        TokenIdExt const refill(distinctDigest(9999));
+        EXPECT_EQ(refill.raw(), reclaimedSlot);
+        EXPECT_EQ(refill.digest(), distinctDigest(9999));
 
-    tokens.clear();
-    // `refill` still alive here.
-    EXPECT_EQ(detail::digestPoolLiveCount(), baseline + 1);
+        tokens.clear();
+        // `refill` still alive here.
+        EXPECT_EQ(detail::digestPoolLiveCount(), baseline + 1);
+    }
+    // `refill` destroyed: its slot must return to the pool, or a leak in the refill
+    // allocation path would go unnoticed and silently raise the baseline for later tests.
+    EXPECT_EQ(detail::digestPoolLiveCount(), baseline);
 }
 
 // A block containing a sparse digest still hashes deterministically; the
diff --git a/tensorrt_llm/runtime/kv_cache_manager_v2/_block_radix_tree.py b/tensorrt_llm/runtime/kv_cache_manager_v2/_block_radix_tree.py
index f42bb5b8fb6d..0adbb847e783 100644
--- a/tensorrt_llm/runtime/kv_cache_manager_v2/_block_radix_tree.py
+++ b/tensorrt_llm/runtime/kv_cache_manager_v2/_block_radix_tree.py
@@ -46,11 +46,29 @@ def gen_multimodal_cache_key_tokens(
     """Create synthetic tokens used only when building multimodal KV-cache keys.
 
     Item-local token 0 carries the content digest; later offsets use deterministic IDs above the vocab.
+
+    Args:
+        id_offset: First synthetic id, usually ``vocab_size``, so generated ids cannot
+            collide with real token ids.
+        multi_modal_data_digest: Content digest of the multimodal item; must be exactly
+            ``_SHA256_DIGEST_SIZE`` bytes.
+        num_tokens: Number of synthetic tokens to generate. Must be positive.
+        token_offset: Item-local index of the first generated token. Must be non-negative;
+            only offset 0 carries the digest.
+
+    Returns:
+        The generated tokens, digest first when ``token_offset`` is 0.
+
+    Raises:
+        ValueError: If the digest length is wrong, ``num_tokens`` is not positive, or
+            ``token_offset`` is negative.
     """
     if len(multi_modal_data_digest) != _SHA256_DIGEST_SIZE:
         raise ValueError(f"multi_modal_data_digest must have length {_SHA256_DIGEST_SIZE}")
-    assert num_tokens > 0
-    assert token_offset >= 0
+    if num_tokens <= 0:
+        raise ValueError("num_tokens must be positive")
+    if token_offset < 0:
+        raise ValueError("token_offset must be non-negative")
     return [
         multi_modal_data_digest if token_offset + i == 0 else TokenId(id_offset + token_offset + i)
         for i in range(num_tokens)
@@ -58,6 +76,18 @@ def gen_multimodal_cache_key_tokens(
 
 
 class Hasher:
+    """Incremental SHA-256 hasher used to derive block keys for the radix tree.
+
+    Accepts ints (encoded as 4 little-endian bytes each, matching the C++ backend's
+    4-byte ``TokenIdExt`` layout), raw ``bytes`` (multimodal content digests and
+    reuse-scope fields), or a sequence mixing the two. Both backends must produce
+    identical digests for the same logical input, so the encoding is part of the
+    on-disk/cross-process contract and cannot change unilaterally.
+
+    Args:
+        data: Optional initial value, hashed immediately as if passed to ``update``.
+    """
+
     # SECURITY INVARIANT: the block-key hash MUST stay cryptographically
     # collision-resistant and >= 256-bit. The radix tree is a globally shared,
     # cross-request/cross-tenant cache index; prefix matches are decided purely by
@@ -77,8 +107,18 @@ def __init__(self, data: int | bytes | Sequence[int | bytes] | None = None) -> N
         if data is not None:
             self.update(data)
 
-    # This function is perf-critical. Expect compromised code quality.
     def update(self, data: int | bytes | Sequence[int | bytes]) -> "Hasher":
+        """Fold ``data`` into the running digest.
+
+        Args:
+            data: An int token id (0 <= id < 2**31), raw ``bytes``, or a sequence of
+                either. An all-int sequence takes a single-call fast path; a sequence
+                containing ``bytes`` (multimodal blocks) falls back to per-item hashing.
+
+        Returns:
+            This ``Hasher``, to allow chaining.
+        """
+        # This function is perf-critical. Expect compromised code quality.
         if type(data) is int:
             assert NDEBUG or (data >= 0 and data < (1 << 31))
             self._hasher.update(data.to_bytes(4, "little"))
diff --git a/tests/unittest/kv_cache_manager_v2_tests/test_kv_cache_manager_v2.py b/tests/unittest/kv_cache_manager_v2_tests/test_kv_cache_manager_v2.py
index 4939dc35d52c..41d1d9bcd81e 100755
--- a/tests/unittest/kv_cache_manager_v2_tests/test_kv_cache_manager_v2.py
+++ b/tests/unittest/kv_cache_manager_v2_tests/test_kv_cache_manager_v2.py
@@ -26,7 +26,7 @@
 from importlib.util import find_spec
 from random import randbytes
 from statistics import median
-from typing import TYPE_CHECKING, Any, Iterator, NamedTuple, cast, get_type_hints
+from typing import TYPE_CHECKING, Any, Iterator, NamedTuple, Sequence, cast, get_type_hints
 
 if not TYPE_CHECKING and find_spec("kv_cache_manager_v2") is not None:
     from kv_cache_manager_v2 import (
@@ -821,7 +821,7 @@ def test_int32_ndarray_ingest_matches_list(self) -> None:
         prompt_np = np.asarray(prompt, dtype=np.int32)
         assert prompt_np.dtype == np.int32 and prompt_np.flags["C_CONTIGUOUS"]
 
-        def commit_prompt(tokens) -> None:
+        def commit_prompt(tokens: "Sequence[TokenIdExt] | np.ndarray") -> None:
             kv_cache = self.manager.create_kv_cache(None, tokens)
             with TemporaryCudaStream([]) as stream_holder:
                 stream = cast(CudaStream, stream_holder.handle)

From cb56bff17d746a2466e469dab98a789d23ad9840 Mon Sep 17 00:00:00 2001
From: Yao Yao <lowsfer@users.noreply.github.com>
Date: Mon, 10 Aug 2026 10:58:26 +0000
Subject: [PATCH 12/12] [None][perf] inline the TokenIdExt hot path and
 constrain zero-copy ingest to CPU

The radix tree holds a large number of TokenIdExt, so building, copying and
tearing down those containers is hot. Every special member was already a tag
test plus a pool call, and only the pool call needs DigestPool, which is
private to tokenIdExt.cpp. Move the tag test and the normal-token path inline
and leave just the digest slow paths out-of-line as freeSlot/duplicateSlot.
Destroying or copying a container of normal tokens is now a predictable branch
per element instead of an opaque call. Verified on the nanobind translation
unit: the undefined references to ~TokenIdExt() and the copy constructor are
gone, replaced by references to the two digest-only helpers.

Also address review feedback on the zero-copy token path:

- Constrain the ingest ndarray to nb::device::cpu. This is correctness, not
  documentation: the digest scan and the hasher dereference arr.data() on the
  host, so without it a contiguous int32 CUDA tensor would bind here and we
  would read a device pointer from host code. Such input now falls through to
  the castTokenIterable path instead.
- Assert alignof(TokenIdExt) == alignof(int32_t) next to the existing size and
  standard-layout asserts, and record why the binary compatibility is a design
  contract. Misalignment is the drift that would make the zero-copy view of an
  int32 token buffer unsafe.

Signed-off-by: Yao Yao <lowsfer@users.noreply.github.com>
---
 .../kv_cache_manager_v2/tokenIdExt.cpp        | 57 ++-------------
 .../kv_cache_manager_v2/tokenIdExt.h          | 70 +++++++++++++++++--
 .../batch_manager/kvCacheManagerV2.cpp        | 10 +--
 3 files changed, 74 insertions(+), 63 deletions(-)

diff --git a/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/tokenIdExt.cpp b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/tokenIdExt.cpp
index 90336bfe1113..b6ab10dab190 100644
--- a/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/tokenIdExt.cpp
+++ b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/tokenIdExt.cpp
@@ -207,48 +207,14 @@ TokenIdExt::TokenIdExt(Digest const& digestValue)
 {
 }
 
-TokenIdExt::~TokenIdExt()
-{
-    if (isDigest())
-    {
-        DigestPool::instance().free(digestIndex());
-    }
-}
-
-TokenIdExt::TokenIdExt(TokenIdExt const& other)
-    // Clone a digest into a fresh slot; a normal token just copies its bits.
-    : mBits(other.isDigest() ? (DigestPool::instance().duplicate(other.digestIndex()) | kTagMask) : other.mBits)
+void TokenIdExt::freeSlot(uint32_t index) noexcept
 {
+    DigestPool::instance().free(index);
 }
 
-TokenIdExt& TokenIdExt::operator=(TokenIdExt const& other)
+uint32_t TokenIdExt::duplicateSlot(uint32_t index)
 {
-    if (this != &other)
-    {
-        // Allocate the clone before releasing our slot (self-safe, exception-safe).
-        uint32_t const newBits
-            = other.isDigest() ? (DigestPool::instance().duplicate(other.digestIndex()) | kTagMask) : other.mBits;
-        if (isDigest())
-        {
-            DigestPool::instance().free(digestIndex());
-        }
-        mBits = newBits;
-    }
-    return *this;
-}
-
-TokenIdExt& TokenIdExt::operator=(TokenIdExt&& other) noexcept
-{
-    if (this != &other)
-    {
-        if (isDigest())
-        {
-            DigestPool::instance().free(digestIndex());
-        }
-        mBits = other.mBits;
-        other.mBits = kBadToken;
-    }
-    return *this;
+    return DigestPool::instance().duplicate(index);
 }
 
 Digest const& TokenIdExt::digest() const
@@ -256,21 +222,6 @@ Digest const& TokenIdExt::digest() const
     return DigestPool::instance().get(digestIndex());
 }
 
-bool TokenIdExt::operator==(TokenIdExt const& other) const
-{
-    if (mBits == other.mBits)
-    {
-        return true;
-    }
-    // Distinct bits are unequal unless both are digests whose 32-byte contents
-    // match (equal digests may live in different pool slots — no dedup).
-    if (isDigest() && other.isDigest())
-    {
-        return digest() == other.digest();
-    }
-    return false;
-}
-
 namespace detail
 {
 
diff --git a/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/tokenIdExt.h b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/tokenIdExt.h
index 888ffc24b9c9..c0248361dd9e 100644
--- a/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/tokenIdExt.h
+++ b/cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/tokenIdExt.h
@@ -92,9 +92,35 @@ class TokenIdExt
     // Multi-modal digest (tag 1): copies `digest` into a fresh pool slot.
     explicit TokenIdExt(Digest const& digest);
 
-    ~TokenIdExt();
-    TokenIdExt(TokenIdExt const& other);            // clones a digest slot
-    TokenIdExt& operator=(TokenIdExt const& other); // clones a digest slot
+    ~TokenIdExt()
+    {
+        if (isDigest())
+        {
+            freeSlot(digestIndex());
+        }
+    }
+
+    // Clones a digest slot; a normal token just copies its bits.
+    TokenIdExt(TokenIdExt const& other)
+        : mBits(other.isDigest() ? (duplicateSlot(other.digestIndex()) | kTagMask) : other.mBits)
+    {
+    }
+
+    // Clones a digest slot; a normal token just copies its bits.
+    TokenIdExt& operator=(TokenIdExt const& other)
+    {
+        if (this != &other)
+        {
+            // Allocate the clone before releasing our slot (self-safe, exception-safe).
+            uint32_t const newBits = other.isDigest() ? (duplicateSlot(other.digestIndex()) | kTagMask) : other.mBits;
+            if (isDigest())
+            {
+                freeSlot(digestIndex());
+            }
+            mBits = newBits;
+        }
+        return *this;
+    }
 
     TokenIdExt(TokenIdExt&& other) noexcept
         : mBits(other.mBits)
@@ -102,7 +128,19 @@ class TokenIdExt
         other.mBits = kBadToken; // steal the slot; leave source empty
     }
 
-    TokenIdExt& operator=(TokenIdExt&& other) noexcept;
+    TokenIdExt& operator=(TokenIdExt&& other) noexcept
+    {
+        if (this != &other)
+        {
+            if (isDigest())
+            {
+                freeSlot(digestIndex());
+            }
+            mBits = other.mBits;
+            other.mBits = kBadToken;
+        }
+        return *this;
+    }
 
     [[nodiscard]] bool isDigest() const noexcept
     {
@@ -128,7 +166,20 @@ class TokenIdExt
 
     // Value equality: normal/tag-mismatch compare raw bits; digest-vs-digest
     // compares the pooled 32 bytes (equal content in different slots is equal).
-    bool operator==(TokenIdExt const& other) const;
+    bool operator==(TokenIdExt const& other) const
+    {
+        if (mBits == other.mBits)
+        {
+            return true;
+        }
+        // Distinct bits are unequal unless both are digests whose 32-byte contents
+        // match (equal digests may live in different pool slots — no dedup).
+        if (isDigest() && other.isDigest())
+        {
+            return digest() == other.digest();
+        }
+        return false;
+    }
 
     bool operator!=(TokenIdExt const& other) const
     {
@@ -142,11 +193,20 @@ class TokenIdExt
         return mBits & kValueMask;
     }
 
+    static void freeSlot(uint32_t index) noexcept;
+    [[nodiscard]] static uint32_t duplicateSlot(uint32_t index);
+
     uint32_t mBits{kBadToken};
 };
 
+// Binary compatibility with int32_t is a deliberate design contract, not a coincidence: a
+// non-negative token id has the same 4-byte little-endian image as the TokenIdExt holding it.
+// The nanobind zero-copy ingest path relies on it to view a contiguous int32 token buffer as
+// TokenIdExt without copying, and Hasher::update relies on it to bulk-hash an all-normal run in
+// one Write.
 static_assert(sizeof(TokenIdExt) == 4, "TokenIdExt must be exactly 4 bytes for bulk hashing");
 static_assert(std::is_standard_layout_v<TokenIdExt>, "TokenIdExt must be standard-layout for byte-stream hashing");
+static_assert(alignof(TokenIdExt) == alignof(int32_t), "TokenIdExt must align like int32_t for zero-copy token views");
 
 // The digest pool that backs digest-tagged TokenIdExt is an implementation
 // detail hidden entirely in tokenIdExt.cpp (anonymous namespace). Only this
diff --git a/cpp/tensorrt_llm/nanobind/batch_manager/kvCacheManagerV2.cpp b/cpp/tensorrt_llm/nanobind/batch_manager/kvCacheManagerV2.cpp
index 0ef06a19c9ce..dcebcc4e6446 100644
--- a/cpp/tensorrt_llm/nanobind/batch_manager/kvCacheManagerV2.cpp
+++ b/cpp/tensorrt_llm/nanobind/batch_manager/kvCacheManagerV2.cpp
@@ -101,16 +101,16 @@ static std::pair<std::vector<kv::TokenIdExt>, bool> castTokenIterable(nb::handle
 // view of the tokens; fn must consume it synchronously (it may release the GIL — the backing
 // buffer must outlive the call).
 //
-// Fast path: a contiguous 1-D int32 buffer (numpy view, memoryview, array.array, torch tensor) is
-// reinterpret_cast to TokenIdExt const* — no copy, no per-token boxing — since a normal token id
-// is bit-identical to a 4-byte TokenIdExt (see tokenIdExt.h). convert=false keeps it strictly
-// zero-copy; a non-int32 / non-contiguous input falls through.
+// Fast path: a contiguous 1-D int32 CPU buffer (numpy view, memoryview, array.array, torch CPU
+// tensor) is reinterpret_cast to TokenIdExt const* — no copy, no per-token boxing — since a normal
+// token id is bit-identical to a 4-byte TokenIdExt (see tokenIdExt.h). convert=false keeps it
+// strictly zero-copy; a non-int32 / non-contiguous input falls through.
 //
 // Fallback: any int|bytes(32) iterable via castTokenIterable — the multimodal/digest path.
 template <class Fn>
 static auto withTokens(nb::handle tokens, Fn&& fn)
 {
-    nb::ndarray<int32_t const, nb::ndim<1>, nb::c_contig> arr;
+    nb::ndarray<int32_t const, nb::ndim<1>, nb::c_contig, nb::device::cpu> arr;
     if (nb::try_cast(tokens, arr, /*convert=*/false))
     {
         auto const count = static_cast<int32_t>(arr.shape(0));