Skip to content

fix(server): replay function_call items when translating Responses input - #226

Closed
Carleon wants to merge 1 commit into
trefeon:mainfrom
Carleon:fix/responses-function-call-roundtrip
Closed

fix(server): replay function_call items when translating Responses input#226
Carleon wants to merge 1 commit into
trefeon:mainfrom
Carleon:fix/responses-function-call-roundtrip

Conversation

@Carleon

@Carleon Carleon commented Aug 30, 2026

Copy link
Copy Markdown

Problem

Agentic clients that use /v1/responses never make progress: the model re-issues the same tool call on every turn and the conversation loops forever.

I hit this with Codex CLI 0.150.1, which is a realistic trigger because Codex removed wire_api = "chat" support, so it can only speak the Responses format. A one-step task (cat a.txt) executed the same command ~100 times in five minutes without finishing.

Root cause

In responsesInputToMessages (internal/server/responses.go), function_call_output is translated into a role: "tool" message, but function_call is skipped:

case "function_call", "reasoning", "item_reference":
    // Locally unexecutable or non-replayable items are skipped
    // (the reference does the same — the upstream cannot run
    // them).
    continue

The result is an orphaned tool reply: a tool_call_id with no preceding assistant tool_calls entry. The translated conversation reaching upstream looks like

user:  What is the weather in Berlin?
tool:  {"temp_c": 19, "sky": "clear"}     <- answers nothing

Chat backends drop a tool message that answers no call, so the model never sees the result and asks again.

The existing comment's reasoning is about execution, but replaying a function_call does not ask the upstream to run anything. It only records that the call happened, which is exactly what makes the following tool message well-formed.

Reproduction

Same conversation, both wire formats, against v1.6.0. Chat is correct, Responses is not.

/v1/chat/completions:

{"messages":[
  {"role":"user","content":"What is the weather in Berlin?"},
  {"role":"assistant","tool_calls":[{"id":"call_w1","type":"function","function":{"name":"get_weather","arguments":"{\"city\":\"Berlin\"}"}}]},
  {"role":"tool","tool_call_id":"call_w1","content":"{\"temp_c\": 19, \"sky\": \"clear\"}"}
]}

"The weather in Berlin is currently 19°C with clear skies"

/v1/responses:

{"input":[
  {"type":"message","role":"user","content":[{"type":"input_text","text":"What is the weather in Berlin?"}]},
  {"type":"function_call","call_id":"call_w1","name":"get_weather","arguments":"{\"city\":\"Berlin\"}"},
  {"type":"function_call_output","call_id":"call_w1","output":"{\"temp_c\": 19, \"sky\": \"clear\"}"}
]}

→ returns the identical function_call for get_weather again ❌

Fix

Emit the assistant tool_calls message rather than skipping it, so each call is replayed immediately before the tool message that answers it.

reasoning and item_reference keep their existing skip behaviour, since they carry no state the upstream can use. A function_call missing call_id or name is also still skipped rather than emitted malformed, and an absent arguments becomes "{}" because the Chat format requires a JSON string there.

Tests

New internal/server/responses_function_call_replay_test.go covers:

  • the call/output pair surviving translation in the right order (the regression)
  • omitted arguments defaulting to "{}"
  • function_call missing call_id or name, plus reasoning and item_reference, still being skipped
  • parallel tool calls each being replayed before their own result

Verified red/green: with the fix reverted, three of the four fail with the orphaned-tool-reply output shown above. go build ./... and go test ./... are clean across all 21 packages.

Verification against a live upstream

Built and deployed against real traffic. After the fix, the same Codex task completes in a single tool call, and a multi-step task (count lines in one file, read a number from another, sum them) also completes correctly in one pass.

🤖 Generated with Claude Code

responsesInputToMessages skipped function_call items while still translating
their matching function_call_output into a role:"tool" message. That left the
tool reply orphaned: a tool_call_id with no preceding assistant tool_calls
entry. Chat backends drop orphaned tool replies, so the model never saw the
tool result and re-issued the identical call every turn, making agentic
clients loop forever.

Emit the assistant tool_calls message instead of skipping it. Items that
cannot form a valid tool_calls entry (missing call_id or name) are still
skipped, as are reasoning and item_reference, which carry no state the
upstream can use.

Adds regression tests covering the call/output pair, default arguments,
skipped items, and parallel tool calls.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
trefeon added a commit that referenced this pull request Aug 30, 2026
Replay assistant function_call items as tool_calls messages so the
matching function_call_output tool message is not orphaned; chat backends
drop orphaned tool replies, making agentic clients loop forever on a
re-issued tool call. reasoning and item_reference stay skipped.

Ports and adapts PR #226 by @Carleon to the post-split tree.
@trefeon

trefeon commented Aug 30, 2026

Copy link
Copy Markdown
Owner

Reviewed and merged manually as c4ac39d (this PR could not merge directly: it was based on e0f134c, before the repo split moved all Go code under backend/).

Verdict: approved. The diff matches the report exactly — function_call replayed as an assistant tool_calls message immediately before its function_call_output -> tool translation, reasoning/item_reference stay skipped, malformed calls are dropped rather than emitted, absent arguments default to "{}", and ordering is preserved for parallel calls. All four regression tests pass as-is.

The port also fixed two stale doc comments that still claimed function_call is non-replayable/skipped, so the documented translation guarantees now match the wire behavior.

Verified end-to-end: hermetic Go suite, golangci-lint (0 issues), and the -race CI job all pass on main. Thank you for the report and reproduction — this unblocked Codex-style agentic clients on /v1/responses.

@trefeon trefeon closed this Aug 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants