Skip to content

complete_chat raises an opaque pydantic ValidationError when the model smollm3-3b returns a malformed tool call #864

Description

@ambs19

Summary

When a loaded model emits a malformed tool_call in its response (e.g. a tool call
with a missing function.name and arguments set to the string "null"),
ChatClient.complete_chat() crashes with a raw pydantic ValidationError instead of a
clear, catchable SDK error. The same unguarded model_validate call exists in the
streaming path. A single misbehaving model therefore aborts the whole application with an
error that gives no hint about the actual cause (the model's output).

Environment

  • foundry-local-sdk 1.2.1 (also present on main)
  • Python 3.13, macOS
  • Reproduced with model smollm3-3b while passing tools=[...]

What happens

smollm3-3b, during a tool-calling conversation, returned a second tool call with no
function name and arguments == "null":

choices.0.message.tool_calls.1.function.name
  Field required [type=missing, input_value={'arguments': 'null'}, input_type=dict]

which surfaces as:

pydantic_core._pydantic_core.ValidationError: 3 validation errors for ChatCompletion
choices.0.message.tool_calls.1.ChatCompletionMessageFunctionToolCall.function.name
  Field required [type=missing, input_value={'arguments': 'null'}, input_type=dict]
choices.0.message.tool_calls.1.ChatCompletionMessageCustomToolCall.custom
  Field required [type=missing, ...]
choices.0.message.tool_calls.1.ChatCompletionMessageCustomToolCall.type
  Input should be 'custom' [type=literal_error, input_value='function', ...]

Stack (abridged) — the crash is at chat_client.py complete_chat
ChatCompletion.model_validate_json(response_json):

File ".../foundry_local_sdk/openai/chat_client.py", line 219, in complete_chat
    completion = ChatCompletion.model_validate_json(response.data)
File ".../pydantic/main.py", line 782, in model_validate_json
    return cls.__pydantic_validator__.validate_json(...)
pydantic_core._pydantic_core.ValidationError: 3 validation errors for ChatCompletion

Minimal, model-free repro (shows the SDK has no guard around response validation):

from openai.types.chat import ChatCompletion

# A response containing a malformed second tool call (missing function.name,
# arguments == "null") — exactly what smollm3-3b emitted through Foundry Local.
malformed = '''{
  "id": "chatcmpl-x", "object": "chat.completion", "created": 0, "model": "smollm3-3b",
  "choices": [{
    "index": 0, "finish_reason": "tool_calls",
    "message": {
      "role": "assistant", "content": null,
      "tool_calls": [
        {"index": 0, "id": "a", "type": "function",
         "function": {"name": "get_weather", "arguments": "{}"}},
        {"index": 1, "id": "b", "type": "function",
         "function": {"arguments": "null"}}
      ]
    }
  }]
}'''

ChatCompletion.model_validate_json(malformed)   # -> raw pydantic ValidationError

Expected behavior

The docstring already promises FoundryLocalException on failure. The SDK should either:

  1. (minimum) wrap the parse and re-raise a clear FoundryLocalException that includes
    the raw response payload, so the error is catchable and actionable; and/or
  2. (nicer) tolerate a malformed tool call — drop/skip the invalid entry and log a
    warning — so one bad tool call doesn't abort an otherwise-usable response.

Why it matters

Smaller models are exactly the ones Foundry Local targets, and they routinely emit
imperfect tool calls. The SDK crashing with an internal pydantic traceback (rather than a
clear SDK error) makes tool-calling apps brittle and hard to debug.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions