langchain: preserve dict-form output messages - #417
Conversation
Assisted-by: Codex
Assisted-by: Codex
Pull request dashboard statusWaiting on the author · refreshed 2026-08-21 01:40 UTC Respond to 6 review items (e.g. link a commit, explain why not, ask a follow-up): Status above doesn't look right?
|
There was a problem hiding this comment.
Pull request overview
This PR fixes LangChain workflow/agent output telemetry losing assistant messages when outputs are provided in LangChain-supported dict form ({"role": ..., "content": ...}), by normalizing output messages with LangChain’s convert_to_messages before building gen_ai.output.messages.
Changes:
- Normalize output-side
messagesviaconvert_to_messagesinto_output_messages, with a safe fallback when normalization fails. - Add tests covering dict-form assistant outputs through both the converter (
make_output_message) and the workflowon_chain_endpath. - Add a changelog fragment documenting the fix.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| instrumentation/opentelemetry-instrumentation-genai-langchain/src/opentelemetry/instrumentation/genai/langchain/utils.py | Normalizes output messages (including role/content dicts) before converting to OutputMessage. |
| instrumentation/opentelemetry-instrumentation-genai-langchain/tests/test_callback_handler.py | Adds regression tests ensuring dict-form assistant outputs are preserved in telemetry. |
| instrumentation/opentelemetry-instrumentation-genai-langchain/.changelog/417.fixed | Documents the bug fix in the package changelog fragments. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Assisted-by: Codex
| *input* side of the next inference call, not the output side of the | ||
| previous one. | ||
| LangChain-supported message representations, such as role/content dicts, | ||
| are normalized first. Non-``AIMessage`` entries are skipped: only |
There was a problem hiding this comment.
The docstring needs to be fixed, the non-"ai messages" entries are skipped should be removed.
| @@ -0,0 +1 @@ | |||
| preserve dict-form assistant messages in workflow and agent output telemetry | |||
eternalcuriouslearner
left a comment
eternalcuriouslearner
left a comment
There was a problem hiding this comment.
LGTM!! requested few changes to address few loopholes.
|
|
||
| def to_output_messages( | ||
| messages: Iterable[BaseMessage], | ||
| messages: Iterable[Any], |
There was a problem hiding this comment.
nit(blocking): Can we avoid using Any here?
| normalized_messages: Iterable[BaseMessage] = convert_to_messages( | ||
| materialized_messages | ||
| ) | ||
| except Exception: # pylint: disable=broad-except |
There was a problem hiding this comment.
nit(blocking): Do we have a test for this? If not can you please add it?
| except Exception: # pylint: disable=broad-except | ||
| normalized_messages = [ | ||
| m for m in materialized_messages if isinstance(m, BaseMessage) | ||
| ] |
There was a problem hiding this comment.
question(blocking): Can we extract this logic into a function like say
def _normalize_messages(
messages: Iterable[MessageLikeRepresentation],
) -> list[BaseMessage]:
materialized = list(messages)
try:
return convert_to_messages(materialized)
except Exception: # pylint: disable=broad-except
normalized: list[BaseMessage] = []
for message in materialized:
try:
normalized.extend(convert_to_messages([message]))
except Exception: # pylint: disable=broad-except
continue
return normalizedand use this logic for both to_output_messages and to_input_messages functions?
Description
LangChain accepts role/content dicts as messages. The input path already normalizes them, but the output path only kept
AIMessageobjects, so workflow and agent spans silently lost dict-form assistant output.Normalize output messages with LangChain's
convert_to_messagesbefore building telemetry, and cover both the converter andon_chain_endpath.Fixes #388
Type of change
How has this been tested?
uv run tox -e py312-test-instrumentation-genai-langchain-latest(237 passed)uv run tox -e py312-test-instrumentation-genai-langchain-oldest(236 passed, 1 skipped)uv run tox -e typecheckuv run tox -e precommituv run tox -e py314-test-instrumentation-genai-langchain-conformance(5 existing scenario skips)Checklist
AI-assisted implementation; I reproduced the missing output on current
main, reviewed the diff, and ran the checks above locally.