Skip to content

fix: content-filter responses raise instead of normalizing #358

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
andrewwhitecdw wants to merge 1 commit into
base: main
Choose a base branch
from
+23 −1
Open

fix: content-filter responses raise instead of normalizing #358

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion examples/experimental/litellm/src/switchyard_litellm/client.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -290,7 +290,7 @@ def _response(response: ModelResponse) -> dict[str, object]:
"name": tool_call.function.name,
"arguments": arguments,
})
if not content:
if not content and choice.finish_reason != "content_filter":
raise ValueError("LiteLLM returned no text content")
return {
"id": response.id,
Expand Down
22 changes: 22 additions & 0 deletions examples/experimental/litellm/tests/test_client.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -552,3 +552,25 @@ async def test_cached_token_count_preserves_explicit_zero() -> None:
await client.aclose()

assert response["usage"]["cached_input_tokens"] == 0


def test_response_content_filter_empty_content() -> None:
"""Empty content with a content_filter finish reason must be normalized."""
from types import SimpleNamespace

from switchyard_litellm.client import _response

response = SimpleNamespace(
id="chatcmpl-test",
model="openai/strong",
choices=[
SimpleNamespace(
message=SimpleNamespace(content=None, tool_calls=None),
finish_reason="content_filter",
)
],
usage=None,
)
result = _response(response)
assert result["outputs"][0]["content"] == []
assert result["outputs"][0]["stop_reason"] == "content_filter"
Comment on lines +563 to +576

@coderabbitai coderabbitai Bot Aug 11, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail
uv run mypy --strict \
  examples/experimental/litellm/src/switchyard_litellm/client.py \
  examples/experimental/litellm/tests/test_client.py

Repository: NVIDIA-NeMo/Switchyard

Length of output: 200

🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '%s\n' '--- target test ---'
sed -n '1,120p' examples/experimental/litellm/tests/test_client.py
sed -n '520,610p' examples/experimental/litellm/tests/test_client.py

printf '%s\n' '--- client response definitions and call sites ---'
rg -n -C 5 'def _response|ModelResponse|content_filter|stop_reason|outputs' \
  examples/experimental/litellm/src examples/experimental/litellm/tests

printf '%s\n' '--- project typing configuration ---'
fd -HI 'pyproject.toml|mypy.ini|setup.cfg|tox.ini' . -x sh -c 'echo "--- $1"; sed -n "1,240p" "$1"' sh {}

Repository: NVIDIA-NeMo/Switchyard

Length of output: 34012

🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '%s\n' '--- available type-checking tools ---'
command -v mypy || true
command -v pyright || true
python3 - <<'PY'
import importlib.util
for name in ("mypy", "pyright", "litellm"):
    print(f"{name}: {importlib.util.find_spec(name)}")
PY

printf '%s\n' '--- LiteLLM dependency and lock references ---'
rg -n -C 3 'litellm|1\.92\.0|ModelResponse' \
  examples/experimental/litellm uv.lock requirements*.txt 2>/dev/null || true

printf '%s\n' '--- test module declarations and type-check exclusions ---'
rg -n -C 4 'mypy|strict|ignore_errors|test_client|examples/experimental/litellm' \
  pyproject.toml examples/experimental/litellm

Repository: NVIDIA-NeMo/Switchyard

Length of output: 48555

🏁 Script executed:

#!/bin/bash
set -euo pipefail

python3 - <<'PY'
import ast
from pathlib import Path

client_path = Path("examples/experimental/litellm/src/switchyard_litellm/client.py")
test_path = Path("examples/experimental/litellm/tests/test_client.py")

client = ast.parse(client_path.read_text())
test = ast.parse(test_path.read_text())

def find_function(tree: ast.AST, name: str) -> ast.FunctionDef:
    for node in ast.walk(tree):
        if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)) and node.name == name:
            return node
    raise AssertionError(f"missing function: {name}")

response_fn = find_function(client, "_response")
arg = response_fn.args.args[0]
print(f"_response parameter annotation: {ast.unparse(arg.annotation)}")
print(f"_response return annotation: {ast.unparse(response_fn.returns)}")

test_fn = find_function(test, "test_response_content_filter_empty_content")
for node in ast.walk(test_fn):
    if isinstance(node, ast.Call) and isinstance(node.func, ast.Name):
        if node.func.id in {"SimpleNamespace", "_response"}:
            print(f"{node.func.id} call at line {node.lineno}: {ast.unparse(node)}")
    if isinstance(node, ast.Subscript):
        print(f"subscript at line {node.lineno}: {ast.unparse(node)}")

config = ast.parse(Path("pyproject.toml").read_text()) if False else None
print("SimpleNamespace fixture present: " + str(
    any(
        isinstance(node, ast.Call)
        and isinstance(node.func, ast.Name)
        and node.func.id == "SimpleNamespace"
        for node in ast.walk(test_fn)
    )
))
print("result output is indexed twice: " + str(sum(
    isinstance(node, ast.Subscript)
    and isinstance(node.value, ast.Subscript)
    and isinstance(node.value.value, ast.Name)
    and node.value.value.id == "result"
    for node in ast.walk(test_fn)
) == 2))
PY

Repository: NVIDIA-NeMo/Switchyard

Length of output: 1099

Make the regression fixture type-safe under strict mypy.

_response requires ModelResponse, but this test passes a SimpleNamespace. Its dict[str, object] result is also indexed without narrowing. Use a typed ModelResponse fixture and narrow outputs before indexing.

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@examples/experimental/litellm/tests/test_client.py` around lines 563 - 576,
Update the _response regression test fixture to construct a typed ModelResponse
instead of SimpleNamespace, preserving the content_filter response values.
Narrow or validate the returned outputs collection before indexing it so the
dict[str, object] result passes strict mypy checks.

Source: Coding guidelines