[https://nvbugs/6566772][fix] Predicate idle on not self.is_shutdown so a non-blocking timedelta(0) is…

[trtllm-agent]

Summary

• Root cause: An idle fetch iteration used an unbounded request_queue.get() even after the shutdown sentinel had been drained, so no item could ever wake it and the executor loop never reached should_stop_processing, deadlocking teardown.
• Fix: Predicate idle on not self.is_shutdown so a non-blocking timedelta(0) is selected once shutdown is set, letting the loop reach its exit check while in-flight requests still drain normally.
• Original test: pytest tests/unittest/llmapi/test_llm.py::test_generate_with_detokenization_stop_words_streaming -v
• Automated fix generated by repair-bot

Test plan

• Verify fix on the same GPU type as the original failure
• Check for regressions in related tests

Links

• Bug: https://nvbugs/6566772

Dev Engineer Review

• Updated PyExecutor request fetching to avoid an unbounded request_queue.get() after shutdown.
• The executor now uses a non-blocking timeout after shutdown detection.
• This allows the loop to reach its shutdown check while in-flight requests continue to drain.
• The change fixes the teardown deadlock without changing public APIs.
• No configuration or test-list files changed.

QA Engineer Review

No test changes.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: e9d1da25-923c-45ca-b832-24a5e7306047

📥 Commits

Reviewing files that changed from the base of the PR and between 81db688 and 7a289c7.

📒 Files selected for processing (1)

• tensorrt_llm/_torch/pyexecutor/py_executor.py

---

Walkthrough

The executor request-fetch condition now checks self.is_shutdown. After shutdown detection, the executor can reach its shutdown check without waiting for another queue item.

Changes

Executor shutdown handling

Layer / File(s)	Summary
Shutdown-aware request fetching tensorrt_llm/_torch/pyexecutor/py_executor.py	The idle request-fetch condition no longer blocks when self.is_shutdown is true. The executor can complete shutdown after consuming the sentinel.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related PRs

• NVIDIA/TensorRT-LLM#17107: Modifies the same executor request and idle-shutdown handling, but addresses async KV-transfer polling.

Suggested reviewers: asfiyab-nvidia

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title identifies the bug, fix type, and shutdown predicate change that resolves the PyExecutor teardown deadlock.
Description check	✅ Passed	The description explains the root cause, fix, affected behavior, tests, and bug link, but it does not include the template checklist.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}