Skip to content

fix false negative healthy status check for turbomind #4745

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
irexyc wants to merge 2 commits into
base: main
Choose a base branch
from
+5 −1
Open

fix false negative healthy status check for turbomind #4745

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
0889292
fix false negative for healthy status check
irexyc Jul 10, 2026
ad62ba2
update
irexyc Jul 10, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion lmdeploy/turbomind/turbomind.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -173,6 +173,7 @@ def __init__(self,
self._create_engine()

self.session_len = _engine_config.session_len
self.health_executor = ThreadPoolExecutor(max_workers=1)
Comment on lines 175 to +176

def _process_weights(self):
"""Process weight."""
Expand Down Expand Up @@ -419,7 +420,10 @@ def _get_health_status(self) -> dict:

async def get_health_status(self) -> dict:
"""Get backend health status without blocking the event loop."""
return await asyncio.to_thread(self._get_health_status)
# MultimodalProcessor may submit a large number of tasks to the default thread pool, causing
# the _get_health_status task to be selected after the DEFAULT_PROBE_TIMEOUT has already elapsed.
loop = asyncio.get_running_loop()
return await loop.run_in_executor(self.health_executor, self._get_health_status)
Comment on lines +423 to +426
Comment on lines +425 to +426


def _get_logits(outputs, offset: int):
Expand Down
Loading