fix(server): replay function_call items when translating Responses input - #226
fix(server): replay function_call items when translating Responses input#226Carleon wants to merge 1 commit into
Conversation
responsesInputToMessages skipped function_call items while still translating their matching function_call_output into a role:"tool" message. That left the tool reply orphaned: a tool_call_id with no preceding assistant tool_calls entry. Chat backends drop orphaned tool replies, so the model never saw the tool result and re-issued the identical call every turn, making agentic clients loop forever. Emit the assistant tool_calls message instead of skipping it. Items that cannot form a valid tool_calls entry (missing call_id or name) are still skipped, as are reasoning and item_reference, which carry no state the upstream can use. Adds regression tests covering the call/output pair, default arguments, skipped items, and parallel tool calls. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Replay assistant function_call items as tool_calls messages so the matching function_call_output tool message is not orphaned; chat backends drop orphaned tool replies, making agentic clients loop forever on a re-issued tool call. reasoning and item_reference stay skipped. Ports and adapts PR #226 by @Carleon to the post-split tree.
|
Reviewed and merged manually as c4ac39d (this PR could not merge directly: it was based on e0f134c, before the repo split moved all Go code under backend/). Verdict: approved. The diff matches the report exactly — function_call replayed as an assistant tool_calls message immediately before its function_call_output -> tool translation, reasoning/item_reference stay skipped, malformed calls are dropped rather than emitted, absent arguments default to "{}", and ordering is preserved for parallel calls. All four regression tests pass as-is. The port also fixed two stale doc comments that still claimed function_call is non-replayable/skipped, so the documented translation guarantees now match the wire behavior. Verified end-to-end: hermetic Go suite, golangci-lint (0 issues), and the -race CI job all pass on main. Thank you for the report and reproduction — this unblocked Codex-style agentic clients on /v1/responses. |
Problem
Agentic clients that use
/v1/responsesnever make progress: the model re-issues the same tool call on every turn and the conversation loops forever.I hit this with Codex CLI 0.150.1, which is a realistic trigger because Codex removed
wire_api = "chat"support, so it can only speak the Responses format. A one-step task (cat a.txt) executed the same command ~100 times in five minutes without finishing.Root cause
In
responsesInputToMessages(internal/server/responses.go),function_call_outputis translated into arole: "tool"message, butfunction_callis skipped:The result is an orphaned tool reply: a
tool_call_idwith no preceding assistanttool_callsentry. The translated conversation reaching upstream looks likeChat backends drop a tool message that answers no call, so the model never sees the result and asks again.
The existing comment's reasoning is about execution, but replaying a
function_calldoes not ask the upstream to run anything. It only records that the call happened, which is exactly what makes the following tool message well-formed.Reproduction
Same conversation, both wire formats, against v1.6.0. Chat is correct, Responses is not.
/v1/chat/completions:{"messages":[ {"role":"user","content":"What is the weather in Berlin?"}, {"role":"assistant","tool_calls":[{"id":"call_w1","type":"function","function":{"name":"get_weather","arguments":"{\"city\":\"Berlin\"}"}}]}, {"role":"tool","tool_call_id":"call_w1","content":"{\"temp_c\": 19, \"sky\": \"clear\"}"} ]}→
"The weather in Berlin is currently 19°C with clear skies"✅/v1/responses:{"input":[ {"type":"message","role":"user","content":[{"type":"input_text","text":"What is the weather in Berlin?"}]}, {"type":"function_call","call_id":"call_w1","name":"get_weather","arguments":"{\"city\":\"Berlin\"}"}, {"type":"function_call_output","call_id":"call_w1","output":"{\"temp_c\": 19, \"sky\": \"clear\"}"} ]}→ returns the identical
function_callforget_weatheragain ❌Fix
Emit the assistant
tool_callsmessage rather than skipping it, so each call is replayed immediately before the tool message that answers it.reasoninganditem_referencekeep their existing skip behaviour, since they carry no state the upstream can use. Afunction_callmissingcall_idornameis also still skipped rather than emitted malformed, and an absentargumentsbecomes"{}"because the Chat format requires a JSON string there.Tests
New
internal/server/responses_function_call_replay_test.gocovers:argumentsdefaulting to"{}"function_callmissingcall_idorname, plusreasoninganditem_reference, still being skippedVerified red/green: with the fix reverted, three of the four fail with the orphaned-tool-reply output shown above.
go build ./...andgo test ./...are clean across all 21 packages.Verification against a live upstream
Built and deployed against real traffic. After the fix, the same Codex task completes in a single tool call, and a multi-step task (count lines in one file, read a number from another, sum them) also completes correctly in one pass.
🤖 Generated with Claude Code