Skip to content

fix(models): preserve live tools in retry request snapshots - #1087

Open
zakahan wants to merge 1 commit into
volcengine:mainfrom
zakahan:fix/retry-snapshot-tool-state
Open

fix(models): preserve live tools in retry request snapshots#1087
zakahan wants to merge 1 commit into
volcengine:mainfrom
zakahan:fix/retry-snapshot-tool-state

Conversation

@zakahan

@zakahan zakahan commented Sep 9, 2026

Copy link
Copy Markdown
Collaborator

RetryingLiteLlm can fail before making a model request with TypeError: cannot pickle '_asyncio.Future' object when a tool in LlmRequest.tools_dict holds asynchronous runtime state. This was observed in a Skill sandbox after tool responses were returned, when the agent tried to start its next model turn. The unconditional retry snapshot deep-copies live tool instances even when no HTTP 429 occurs.

Preserve tool instance identity through the deepcopy memo while continuing to deep-copy request contents, configuration, and the tool mapping. This keeps the retry payload isolated from first-attempt mutations without cloning tool sessions or asynchronous state. Retry eligibility and fallback behavior remain unchanged.

Validation:

  • Added four regression cases covering streaming/non-streaming requests with and without a pre-output 429. All four reproduce the Future error before the fix and pass afterward.
  • Regression checks verify live tool identity and isolation of nested contents, config, and tool-map mutations on retry.
  • pytest tests/models -q: 16 passed, including existing no-replay-after-output and non-429 tests.
  • Pre-commit on both changed files: Ruff lint/format and Gitleaks passed.

No live model calls were used for these tests.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant