fix(handlers): SAO-18193 Responses API and multimodal messages

[etserend]

Summary

Follow-up to #259 (merged in 0.5.0). Fixes SplunkAOTracingProcessor (OpenAI Agents SDK) on the paths #259 left unchanged:

• Responses API LLM spans (the SDK default). Input was one user message holding the whole item list as JSON, and output was one JSON blob. Input items now become chat messages in the order sent: tool calls as assistant tool_calls, tool results as tool messages, developer as system, and role-less items such as reasoning are skipped. Output items become one assistant message with text and tool calls.
• Workflow/agent spans on the Responses path. They showed "Workflow Step" as input and a raw {"type": "output_text", ...} part as output. They now show the latest user message and the answer text.
• Multimodal input on both paths. Text, image, and file parts become content blocks (reusing the LangChain converter) instead of a JSON dump of the whole message. The ingestion-hook path builds LoggedLlmSpan, so the blocks survive there too.
• Streamed Chat Completions (Runner.run_streamed). The SDK records a Responses-shaped response object as the generation output; it is now normalized like Responses output instead of being dumped as JSON.

Not changed (by design)

• A file_id-only image or file has no data or URL to attach, so it stays a small text part.
• Non-streamed Chat Completions workflow output still carries extra provider fields from the raw message, as on fix(handlers): SAO-17638 LLM span double-encoding and workflow input #259; left for a follow-up.

Test plan

• poetry run pytest tests/test_openai_agents.py tests/test_openai_agents_utils.py -n 0 — 72 passed, 1 skipped, 2 failed (both pre-existing network-dependent VCR tests, identical failure on main, unrelated to this change)
• ruff check and ruff format --check on changed files — clean
• poetry run invoke type-check — clean, no issues

🤖 Generated with Claude Code

[etserend]

🤖 This review was generated by the Astra agent (claude-opus-5-5). It may contain mistakes.

Verdict: approve — The Responses, multimodal, and streamed paths work as described. One gap remains for input_file parts that carry a URL or raw base64, and some tests are missing.

General Comments

• 🟡 minor (testing): The new tests do not cover these paths:
• A Responses API LLM span on the ingestion-hook path. LoggedLlmSpan now receives tool messages and tool_calls dicts there, and only Chat Completions input is tested on that path.
• An input_file part on either path, with file_data or with file_url.
• End-to-end export of a tool message whose content is a list of content blocks. Only the util-level normalizer is tested for this case.
  Add one focused test for each path. The input_file gap in the line comment shows that this path needs a test.

Follow-ups

Suggested follow-up work that could be tracked as Jira tickets:

• src/splunk_ao/utils/openai_agents.py:94-100: An assistant text message and the function calls after it are one model turn, but they become two assistant messages in the input. _normalize_responses_output merges them into one message for the same turn. Consider merging function calls into a preceding assistant text message too, so input and output agree.
• src/splunk_ao/utils/openai_agents.py:423-433: A reasoning-only response, or an unset response, records the literal string "null" as the LLM output. The workflow can then inherit it as its output. An empty assistant message is probably a better fallback than "null".

[opikalova]

🤖 This review was generated by the Astra agent (claude-opus-5-5). It may contain mistakes.

Verdict: approve — The Responses, multimodal, and streamed paths are correct. The remaining issues are minor: None handling has gaps, and the open input_file thread is still valid.

General Comments

• 🟡 minor (documentation): Astra found no Jira ticket in the branch name, the title or the description.

Follow-ups

Suggested follow-up work that could be tracked as Jira tickets:

• src/splunk_ao/utils/openai_agents.py:407-409: On the Responses path, the system prompt goes in response.instructions. It is not in the input items. The LLM span stores it only in metadata["instructions"], but the Chat Completions path records it as a system input message. Record it as a leading system message or as gen_ai.system_instructions, so that the two paths agree.

[etserend]

🤖 This review was generated by the Astra agent (claude-opus-5-5). It may contain mistakes.

Verdict: approve — The Responses, multimodal and streamed paths work correctly. One minor gap remains: raw base64 file parts on the Chat Completions path still leak.

Follow-ups

Suggested follow-up work that could be tracked as Jira tickets:

• src/splunk_ao/utils/openai_agents.py:61-65: _content_block returns None for a content part that is not a str or dict, and _message_content silently drops it. _as_dict converts only top-level items. Apply _as_dict to each part too, so that a pydantic part from a custom model keeps its content.

[etserend]

🟡 minor (bug): The raw-base64 guard applies to Responses input_file parts only. The Chat Completions path has the same gap.

The Agents SDK Chat Completions converter copies input_file.file_data unchanged into a {"type": "file", "file": {...}} part. That part reaches _convert_langchain_content_block directly. When file_data has no data: prefix, the converter returns json.dumps(block). That text contains the full base64 payload.

Failure scenario:

• A user sends {"type": "input_file", "filename": "a.pdf", "file_data": "<raw b64>"} through OpenAIChatCompletionsModel.
• The LLM span input holds a text block with the whole file payload.

The ticket requires "no JSON dump" on both paths. Apply the same guard to kind == "file". Add a TestGenerationMultimodalInput case for a raw-base64 file part.

Suggested change

	return _convert_langchain_content_block(part)
	elif kind == "file":
	inner = part.get("file")
	file_data = inner.get("file_data") if isinstance(inner, dict) else None
	if isinstance(file_data, str) and file_data and not file_data.startswith("data:"):
	# Raw base64 without a data URI: do not dump the payload as text.
	meta = {k: v for k, v in inner.items() if k != "file_data"}
	return {"type": "text", "text": serialize_to_str({"type": "file", "file": meta})}
	return _convert_langchain_content_block(part)

🤖 Generated by the Astra agent

[opikalova]

🤖 This review was generated by the Astra agent (claude-opus-5-5). It may contain mistakes.

Verdict: approve — The latest commit fixes the None, "null", and input_file findings correctly. Two minor issues remain: one test is missing, and the open Chat Completions raw-base64 thread is still valid.

General Comments

• 🟡 minor (documentation): Astra found no Jira ticket in the branch name, the title or the description.

Follow-ups

Suggested follow-up work that could be tracked as Jira tickets:

• src/splunk_ao/utils/serialization.py:49-59: _convert_langchain_content_block returns json.dumps(block) for a file part whose file_data is not a data URI. That text holds the full raw base64 payload. The open thread on openai_agents.py:58 proposes a guard only in _content_block. Put the guard in the shared converter instead. Then the LangChain handler path and both OpenAI Agents paths drop the payload in the same way.
• src/splunk_ao/converter/attribute_mapping.py:194-198: For a tool message, the exporter puts the content into tool_call_response.response without changes. A tool result that contains content blocks thus exports the SDK-internal shape {"type": "data", "base64": ...}. User messages get the OTel blob/uri part shape instead. test_responses_api_tool_output_with_image_exports_content_blocks asserts this internal data type. Consider mapping content-block lists through _content_part, so that tool results use the same wire shape as messages.

[opikalova]

🟡 minor (testing): The commit message says that this line fixes a whole-trace crash on the ingestion-hook path when trace_include_sensitive_data=False. No test covers that case.

test_ingestion_hook_llm_span_with_no_output_does_not_crash_trace sets input=[...]. Thus it would still pass if this line and line 442 went back to passing None through.

Add one hook-path test for each span type, with no data set:

• generation_span(model="gpt-4", parent=root_span) with no input and no output.
• response_span(parent=root_span) with no input and no response.

For both, assert that the trace commits, that llm_span.input == [], and that llm_span.output.content == "". Also assert that the workflow output is not "null". This locks in the change from serialize_to_str(None) to "".

🤖 Generated by the Astra agent