feat: release thunderagent program on session end via trajectory_final#5
Merged
Conversation
Adds a deterministic program-close for the experimental dynamo
thunderagent_router. On true session teardown the provider fires a throwaway
max_tokens=1 request carrying nvext.agent_context.trajectory_final=true; the
router short-circuits it, releasing the program from its table + paused set so
its tokens stop counting against worker utilization. Best-effort — the router's
idle/decay reaper is the backstop.
Fires on session_shutdown{reason:"quit"}, the single true teardown signal, NOT
on agent_end: pi emits agent_end once per user prompt, so for multiturn agents
(the norm) hooking it would close the program after the first turn and leak it
for the rest of the session, dropping router worker/KV affinity mid-conversation.
Continuation reasons (reload/fork/new/resume) keep the same trajectory_id, so the
program persists. print-mode (batch) also emits "quit" on dispose and awaits the
handler, so one-shot runs still close exactly once.
Separate from the subagent session_control close: that frees SGLang KV; this
frees scheduler bookkeeping.
Validated end-to-end on GB200: a 3-turn pi RPC session releases the program once
at quit with no leak. Unit tests cover the event routing.
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds a deterministic program close for the experimental
dynamo.thunderagent_routerscheduler. When a pi agent session ends, the provider fires a throwawaymax_tokens=1request carryingnvext.agent_context.trajectory_final=true. The router short-circuits it and releases the program from its table + paused set, so its tokens stop counting against worker utilization. Best-effort — the router's idle/decay reaper is the backstop.When it fires (multiturn-correct)
On
session_shutdown{reason:"quit"}— the single true teardown signal — not onagent_end.pi emits
agent_endonce per user prompt. Agents here are multiturn, so the whole session is one trajectory/program (sametrajectory_idacross every prompt). Hookingagent_end(with a one-shot guard) would:reasonvaluesreload/fork/new/resumekeep the sametrajectory_id, so the program legitimately persists.print-mode(batch) also emitsquiton dispose and awaits the handler, so one-shot runs still close exactly once.Separate from the subagent
session_controlclose: that frees SGLang KV; this frees scheduler bookkeeping.Validation
test/program-close.test.ts):agent_endis not hooked;reload/forkdon't close;quitcloses exactly once carryingtrajectory_final, idempotent.thunderagent_router):request_endtrace spans withtrajectory_final: true,output_tokens: 0.agent_endversion released after turn 1 and leaked runs 2-3 (confirmed via a post-exit close probe that reaped the orphaned program).Related
AgentContext.trajectory_final+end_programshort-circuit).