-
Notifications
You must be signed in to change notification settings - Fork 295
fix: clear nativeArgs when tool-call finalize fails (#1221) #1634
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
f99422e
79b28e7
02286d8
d5ba3bd
08b4082
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,113 @@ | ||
| /** | ||
| * Regression test for issue #1221: truncated tool-call arguments can be silently | ||
| * written to disk. | ||
| * | ||
| * When a streamed native tool call's arguments are cut off mid-value (e.g. the | ||
| * model hits max_tokens while still writing write_to_file's `content` string), | ||
| * NativeToolCallParser.finalizeStreamingToolCall() returns null. Task.ts | ||
| * (~line 3748) reuses the same tool-use object the streaming phase was mutating | ||
| * in place and only sets `partial = false` - before the fix it left `nativeArgs` | ||
| * (built from the incomplete partial parse) untouched. | ||
| * | ||
| * presentAssistantMessage.ts (~line 443) is supposed to short-circuit exactly | ||
| * this case with a structured tool_result instead of executing the tool - but | ||
| * its guard is `isKnownTool && !block.nativeArgs && !customTool`. With | ||
| * nativeArgs still populated, the guard never fired and the truncated content | ||
| * would be passed straight to write_to_file's execution path. | ||
| * | ||
| * The fix clears `existingToolUse.nativeArgs` alongside `partial = false` at | ||
| * the finalize-null site, so the pre-existing guard actually does what its own | ||
| * comment already claimed. | ||
| */ | ||
|
|
||
| import { isValidToolName } from "../../tools/validateToolUse" | ||
| import type { ToolUse, WriteToFileToolUse } from "../../../shared/tools" | ||
|
|
||
| describe("Truncated native tool-call args on finalize failure (issue #1221)", () => { | ||
| /** | ||
| * Simulates the finalize-null branch from Task.ts (~line 3748) as it exists | ||
| * after the fix: on finalizeStreamingToolCall() returning null, mark the | ||
| * tool non-partial and clear nativeArgs. | ||
| */ | ||
| function finalizeNullBranch(existingToolUse: ToolUse): ToolUse { | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is there a reason to keep these hand-written copies of the finalize-null branch and guard, given they can drift from the real logic that
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Thought about pulling this file honestly. The drift risk is real, but I kept both - the Task.spec.ts test is the one that actually proves the fix works (drove it through a fix revert to confirm it fails for the right reason), so it'd catch drift in the finalize-null branch even if this file went stale. This one's just cheap and pins the exact guard condition down precisely, which is handy if someone's trying to understand what the bug was without reading through a mocked stream setup. Not attached to it though - if you'd rather it go, say so and I'll pull it.
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm fine with leaving it, I think it's ok to have it here in case the other spec changes
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Thanks for taking a look. |
||
| existingToolUse.partial = false | ||
| existingToolUse.nativeArgs = undefined | ||
| return existingToolUse | ||
|
Comment on lines
+32
to
+35
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy lift Test the production finalization path.
Drive a truncated 🤖 Prompt for AI Agents |
||
| } | ||
|
|
||
| /** | ||
| * Simulates the finalize-null branch as it existed *before* the fix, for a | ||
| * companion test proving the old behavior really was the bug (not just an | ||
| * assumption). | ||
| */ | ||
| function finalizeNullBranchBeforeFix(existingToolUse: ToolUse): ToolUse { | ||
| existingToolUse.partial = false | ||
| return existingToolUse | ||
| } | ||
|
|
||
| /** | ||
| * Simulates the short-circuit guard from presentAssistantMessage.ts (~line | ||
| * 443): `isKnownTool && !block.nativeArgs && !customTool`. Returns true when | ||
| * the tool call would be blocked (a structured tool_result emitted, no | ||
| * execution), false when it would proceed to execution. | ||
| */ | ||
| function wouldBeBlocked(block: ToolUse, customTool: unknown = undefined): boolean { | ||
| const isKnownTool = isValidToolName(String(block.name)) | ||
| return Boolean(isKnownTool && !block.nativeArgs && !customTool) | ||
| } | ||
|
|
||
| it("clears nativeArgs so a truncated write_to_file call is blocked instead of executed", () => { | ||
| // A write_to_file call whose `content` was cut off mid-stream - exactly | ||
| // the scenario in #1221. The streaming phase already populated nativeArgs | ||
| // from the incomplete partial-json parse before finalize failed. | ||
| const truncated: WriteToFileToolUse = { | ||
| type: "tool_use", | ||
| name: "write_to_file", | ||
| params: {}, | ||
| partial: true, | ||
| nativeArgs: { path: "src/config.json", content: '{"apiKey": "sk-live-abc123' /* cut off mid-string */ }, | ||
| } | ||
|
|
||
| finalizeNullBranch(truncated) | ||
|
|
||
| expect(truncated.partial).toBe(false) | ||
| expect(truncated.nativeArgs).toBeUndefined() | ||
| expect(wouldBeBlocked(truncated)).toBe(true) | ||
| }) | ||
|
|
||
| it("companion: without the fix, the same truncated call would NOT have been blocked", () => { | ||
| const truncated: WriteToFileToolUse = { | ||
| type: "tool_use", | ||
| name: "write_to_file", | ||
| params: {}, | ||
| partial: true, | ||
| nativeArgs: { path: "src/config.json", content: '{"apiKey": "sk-live-abc123' }, | ||
| } | ||
|
|
||
| finalizeNullBranchBeforeFix(truncated) | ||
|
|
||
| // This is the bug: partial is false (presented as "complete"), but | ||
| // nativeArgs still carries the truncated value, so the guard's | ||
| // `!block.nativeArgs` never becomes true and the call would proceed to | ||
| // execution with the truncated content. | ||
| expect(truncated.partial).toBe(false) | ||
| expect(truncated.nativeArgs).toEqual({ path: "src/config.json", content: '{"apiKey": "sk-live-abc123' }) | ||
| expect(wouldBeBlocked(truncated)).toBe(false) | ||
| }) | ||
|
|
||
| it("does not affect a normally-finalized (non-null) tool call", () => { | ||
| // When finalizeStreamingToolCall() succeeds, Task.ts replaces the block | ||
| // with the freshly-finalized one instead of taking this branch at all - | ||
| // this test just confirms a complete, valid nativeArgs is never touched | ||
| // by wouldBeBlocked's guard simulation. | ||
| const complete: WriteToFileToolUse = { | ||
| type: "tool_use", | ||
| name: "write_to_file", | ||
| params: {}, | ||
| partial: false, | ||
| nativeArgs: { path: "src/config.json", content: '{"apiKey": "sk-live-abc123xyz"}' }, | ||
| } | ||
|
|
||
| expect(wouldBeBlocked(complete)).toBe(false) | ||
| }) | ||
| }) | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would it also be worth clearing
paramshere, since the streaming partial parse still leaves the truncated values inparamsand they get echoed into API history via thetoolUse.nativeArgs || toolUse.paramsfallback?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good catch, and you're actually right about the mechanism - I'd assumed params stayed {} for native calls based on how it's built at finalize time (Task.ts:713 area), but missed that the streaming partial-update path (NativeToolCallParser.ts:391) populates it too, for handlePartial's UI hooks.
Checked whether that made it an execution risk though: BaseTool.handle only ever reads nativeArgs to build execute()'s params, and throws instead of falling back to block.params when nativeArgs is undefined - so it was never actually exploitable. The real effect was just that the truncated content kept ending up in conversation history via the nativeArgs || params fallback, under a different name than before.
Cleared params to {} too in the latest commit and added an assertion that the recorded history entry for a truncated call doesn't contain the leaked content.