[None][chore] Remove unused token_range and is_last_slice parameters from _create_kv_slice by Tabrizian · Pull Request #16083 · NVIDIA/TensorRT-LLM

Tabrizian · 2026-07-07T21:21:01Z

Description

Part of the Python CacheTransceiver refactoring/cleanup.

The token_range and is_last_slice parameters of KvCacheTransceiverV2._create_kv_slice are dead code: no caller ever passes them — all call sites invoke _create_kv_slice(req) and rely on the defaults (computed range TokenRange(0, prompt_len + num_extra_kv_tokens) and is_last_slice=True). This PR removes both parameters, inlines the default range computation, and hardcodes is_last_slice=True in the constructed KVSlice.

Notes:

The KVSlice.token_range and KVSlice.is_last_slice fields are unchanged; only the never-used function parameters are removed. Downstream consumers (e.g. the native transfer protocol in native/transfer.py) are unaffected.
The chunked/pipelined prefill transfer work ([TRTLLM-12499][feat] Pipelined KVCache transfer for disaggregated serving in Python Cache Transceiver #15727) does not use these parameters either — it constructs chunked KVSlice objects directly via the dataclass constructor.
The unit test test_respects_explicit_token_range, which existed solely to exercise the removed passthrough, is deleted. The tests covering the default range computation are kept.

Test Coverage

tests/unittest/disaggregated/test_cache_reuse_adapter.py::TestCreateKvSliceTokenRange (default-range tests retained)
tests/unittest/disaggregated/test_cache_transceiver_single_process.py

PR Checklist

Please review the following before submitting your PR:

PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.
PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.
Test cases are provided for new code paths (see test instructions)
If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.
Any new dependencies have been scanned for license and vulnerabilities
CODEOWNERS updated if ownership changes
Documentation updated as needed
Update tava architecture diagram if there is a significant design change in PR.
The reviewers assigned automatically/manually are appropriate for the PR.
Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

Tabrizian · 2026-07-07T21:21:11Z

/bot run --disable-fail-fast

coderabbitai · 2026-07-07T21:23:11Z

📝 Walkthrough

Walkthrough

The token_range parameter was removed from KvCacheTransceiverV2._create_kv_slice, which now computes the token range internally based solely on req.prompt_len. The corresponding test verifying explicit token_range propagation was removed.

Changes

Token Range Simplification

Layer / File(s)	Summary
Simplify _create_kv_slice token range handling `tensorrt_llm/_torch/disaggregation/transceiver.py`, `tests/unittest/disaggregated/test_cache_reuse_adapter.py`	Removed the `token_range` parameter from `_create_kv_slice`; the method now always computes `token_range` internally based on `req.prompt_len`, and the test covering explicit `token_range` propagation was deleted.

Estimated code review effort: 1 (Trivial) | ~3 minutes

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title matches the refactor/cleanup change and clearly states the unused parameters removed from `_create_kv_slice`.
Description check	✅ Passed	The description follows the template with filled Description, Test Coverage, and PR Checklist sections and explains the change and rationale.

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

_{Comment @coderabbitai help to get the list of available commands.}

Tabrizian · 2026-07-07T21:24:14Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-07-07T21:26:53Z

PR_Github #58081 [ run ] triggered by Bot. Commit: 390cb3c Link to invocation

tensorrt-cicd · 2026-07-07T21:30:15Z

PR_Github #58082 [ run ] triggered by Bot. Commit: 390cb3c Link to invocation

tensorrt-cicd · 2026-07-07T21:33:54Z

PR_Github #58081 [ run ] completed with state ABORTED. Commit: 390cb3c

Link to invocation

tensorrt-cicd · 2026-07-08T03:00:19Z

PR_Github #58082 [ run ] completed with state SUCCESS. Commit: 390cb3c
/LLM/main/L0_MergeRequest_PR pipeline #46746 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

Tabrizian · 2026-07-08T16:00:44Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-07-08T16:06:31Z

PR_Github #58260 [ run ] triggered by Bot. Commit: 9607bfa Link to invocation

tensorrt-cicd · 2026-07-09T06:22:20Z

PR_Github #58260 [ run ] completed with state FAILURE. Commit: 9607bfa
/LLM/main/L0_MergeRequest_PR pipeline #46900 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

Tabrizian · 2026-07-09T16:15:16Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-07-09T16:21:48Z

PR_Github #58489 [ run ] triggered by Bot. Commit: 6230e6e Link to invocation

tensorrt-cicd · 2026-07-10T09:45:17Z

PR_Github #58489 [ run ] completed with state FAILURE. Commit: 6230e6e
/LLM/main/L0_MergeRequest_PR pipeline #47097 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

The token_range parameter of KvCacheTransceiverV2._create_kv_slice is never passed by any caller; every call site relies on the default computed range TokenRange(0, prompt_len + num_extra_kv_tokens). Drop the parameter and the test case that existed only to exercise the explicit-range passthrough. The KVSlice.token_range field and its default computation are unchanged. Signed-off-by: Iman Tabrizian <10105175+tabrizian@users.noreply.github.com>

…lice Like token_range, the is_last_slice parameter is never passed by any caller; every call site relies on the True default. Hardcode is_last_slice=True in the constructed KVSlice. The KVSlice.is_last_slice field and the native transfer protocol are unchanged. Signed-off-by: Iman Tabrizian <10105175+tabrizian@users.noreply.github.com>

Tabrizian · 2026-07-10T22:22:41Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-07-10T22:29:15Z

PR_Github #58712 [ run ] triggered by Bot. Commit: ce9b56b Link to invocation

tensorrt-cicd · 2026-07-11T02:54:29Z

PR_Github #58712 [ run ] completed with state SUCCESS. Commit: ce9b56b
/LLM/main/L0_MergeRequest_PR pipeline #47295 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

Tabrizian · 2026-07-12T07:28:37Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-07-12T07:34:56Z

PR_Github #58798 [ run ] triggered by Bot. Commit: ce9b56b Link to invocation

tensorrt-cicd · 2026-07-12T07:34:57Z

PR_Github #58798 [ run ] completed with state DISABLED
Pipeline is freezed and top-1 instance is under maintenance.

Link to invocation

Tabrizian · 2026-07-12T08:27:03Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-07-12T08:33:44Z

PR_Github #58801 [ run ] triggered by Bot. Commit: ce9b56b Link to invocation

tensorrt-cicd · 2026-07-12T08:33:45Z

PR_Github #58801 [ run ] completed with state DISABLED
Pipeline is freezed and top-1 instance is under maintenance.

Link to invocation

Tabrizian · 2026-07-12T17:24:22Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-07-12T17:33:05Z

PR_Github #58819 [ run ] triggered by Bot. Commit: ce9b56b Link to invocation

tensorrt-cicd · 2026-07-12T23:41:32Z

PR_Github #58819 [ run ] completed with state SUCCESS. Commit: ce9b56b
/LLM/main/L0_MergeRequest_PR pipeline #47372 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

Tabrizian · 2026-07-13T03:07:56Z

/bot skip --comment "Unrelated CI failure"

tensorrt-cicd · 2026-07-13T03:14:29Z

PR_Github #58873 [ skip ] triggered by Bot. Commit: ce9b56b Link to invocation

tensorrt-cicd · 2026-07-13T03:20:14Z

PR_Github #58873 [ skip ] completed with state SUCCESS. Commit: ce9b56b
Skipping testing for commit ce9b56b

Link to invocation

Tabrizian · 2026-07-13T04:35:23Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-07-13T04:41:39Z

PR_Github #58886 [ run ] triggered by Bot. Commit: ce9b56b Link to invocation

Tabrizian · 2026-07-13T04:59:27Z

/bot kill

tensorrt-cicd · 2026-07-13T05:05:36Z

PR_Github #58893 [ kill ] triggered by Bot. Commit: ce9b56b Link to invocation

tensorrt-cicd · 2026-07-13T05:11:49Z

PR_Github/16083-ce9b56b #58886 was force-killed by a newer pipeline run.
L0 job information not available (job may not have been triggered yet).

Link to superseding invocation

tensorrt-cicd · 2026-07-13T05:11:50Z

PR_Github #58893 [ kill ] completed with state SUCCESS. Commit: ce9b56b
Successfully killed previous jobs for commit ce9b56b

Link to invocation

Tabrizian · 2026-07-13T18:21:36Z

/bot skip --comment "CI has passed"

tensorrt-cicd · 2026-07-13T18:28:38Z

PR_Github #59015 [ skip ] triggered by Bot. Commit: ce9b56b Link to invocation

tensorrt-cicd · 2026-07-13T18:37:51Z

PR_Github #59015 [ skip ] completed with state SUCCESS. Commit: ce9b56b
Skipping testing for commit ce9b56b

Link to invocation

Tabrizian requested a review from a team as a code owner July 7, 2026 21:21

Tabrizian requested review from byshiue and zhaoyangwang-nvidia July 7, 2026 21:21

github-actions Bot assigned Tabrizian Jul 7, 2026

Tabrizian changed the title ~~[None][chore] Remove unused token_range parameter from _create_kv_slice~~ [None][chore] Remove unused token_range and is_last_slice parameters from _create_kv_slice Jul 7, 2026

Tabrizian force-pushed the user/imant/remove-token-range-param branch from 390cb3c to eafac5e Compare July 7, 2026 21:53

Tabrizian requested a review from Shixiaowei02 July 7, 2026 22:02

Shixiaowei02 approved these changes Jul 8, 2026

View reviewed changes

Tabrizian force-pushed the user/imant/remove-token-range-param branch from eafac5e to 9607bfa Compare July 8, 2026 16:00

Tabrizian force-pushed the user/imant/remove-token-range-param branch from 9607bfa to 6230e6e Compare July 9, 2026 16:15

Tabrizian added 2 commits July 10, 2026 15:22

Tabrizian force-pushed the user/imant/remove-token-range-param branch from 6230e6e to ce9b56b Compare July 10, 2026 22:22

Tabrizian enabled auto-merge (squash) July 13, 2026 18:21

Tabrizian requested a review from pcastonguay July 13, 2026 18:57

pcastonguay approved these changes Jul 13, 2026

View reviewed changes

Tabrizian merged commit e8321d2 into NVIDIA:main Jul 13, 2026
9 checks passed

Uh oh!

Conversation

Tabrizian commented Jul 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Test Coverage

PR Checklist

GitHub Bot Help

Uh oh!

Tabrizian commented Jul 7, 2026

Uh oh!

coderabbitai Bot commented Jul 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Uh oh!

Tabrizian commented Jul 7, 2026

Uh oh!

tensorrt-cicd commented Jul 7, 2026

Uh oh!

tensorrt-cicd commented Jul 7, 2026

Uh oh!

tensorrt-cicd commented Jul 7, 2026

Uh oh!

tensorrt-cicd commented Jul 8, 2026

Uh oh!

Tabrizian commented Jul 8, 2026

Uh oh!

tensorrt-cicd commented Jul 8, 2026

Uh oh!

tensorrt-cicd commented Jul 9, 2026

Uh oh!

Tabrizian commented Jul 9, 2026

Uh oh!

tensorrt-cicd commented Jul 9, 2026

Uh oh!

tensorrt-cicd commented Jul 10, 2026

Uh oh!

Tabrizian commented Jul 10, 2026

Uh oh!

tensorrt-cicd commented Jul 10, 2026

Uh oh!

tensorrt-cicd commented Jul 11, 2026

Uh oh!

Tabrizian commented Jul 12, 2026

Uh oh!

tensorrt-cicd commented Jul 12, 2026

Uh oh!

tensorrt-cicd commented Jul 12, 2026

Uh oh!

Tabrizian commented Jul 12, 2026

Uh oh!

tensorrt-cicd commented Jul 12, 2026

Uh oh!

tensorrt-cicd commented Jul 12, 2026

Uh oh!

Tabrizian commented Jul 12, 2026

Uh oh!

tensorrt-cicd commented Jul 12, 2026

Uh oh!

tensorrt-cicd commented Jul 12, 2026

Uh oh!

Tabrizian commented Jul 13, 2026

Uh oh!

tensorrt-cicd commented Jul 13, 2026

Uh oh!

tensorrt-cicd commented Jul 13, 2026

Uh oh!

Tabrizian commented Jul 13, 2026

Uh oh!

tensorrt-cicd commented Jul 13, 2026

Uh oh!

Tabrizian commented Jul 13, 2026

Uh oh!

tensorrt-cicd commented Jul 13, 2026

Uh oh!

tensorrt-cicd commented Jul 13, 2026

Uh oh!

tensorrt-cicd commented Jul 13, 2026

Uh oh!

Tabrizian commented Jul 7, 2026 •

edited

Loading

coderabbitai Bot commented Jul 7, 2026 •

edited

Loading