fix(langchain): emit invoke_agent for create_agent, including nested agents - #391
fix(langchain): emit invoke_agent for create_agent, including nested agents#391AgentGymLeader wants to merge 13 commits into
Conversation
Assisted-by: OpenAI GPT-5
Runtime testing exposed that nested agents could be missed when LangGraph propagated the parent node name into langgraph_node. Because agent run-name resolution falls back to langgraph_node, restrict the lc_agent_name comparison to the explicit kwargs name. This keeps inherited node metadata from contaminating the comparison while allowing nested agents without an explicit run name.
Pull request dashboard statusWaiting on the author · refreshed 2026-08-22 15:51 UTC Respond to 6 review items (e.g. link a commit, explain why not, ask a follow-up): Status above doesn't look right?
|
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
This PR refines agent span classification for LangChain/LangGraph runs by using LangChain’s lc_agent_name signal while avoiding false positives for internal LangGraph nodes.
Changes:
- Add
lc_agent_name-aware agent detection logic inclassify_chain_runvia_has_agent_signals. - Avoid misclassifying internal LangGraph nodes as agents by comparing against the callback’s explicit run name.
- Add unit tests covering top-level agents, nested agents, internal nodes, and suppression behavior.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| instrumentation/opentelemetry-instrumentation-genai-langchain/src/opentelemetry/instrumentation/genai/langchain/operation_mapping.py | Updates agent-signal detection to incorporate lc_agent_name with LangGraph-aware heuristics. |
| instrumentation/opentelemetry-instrumentation-genai-langchain/tests/test_operation_mapping.py | Adds tests validating new classification behavior and suppression rules. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Assisted-by: OpenAI Codex
Assisted-by: GPT-5
Assisted-by: OpenAI Codex
Assisted-by: ChatGPT 5.2
Distinguish confirmed agent ancestors from confirmed non-agent ancestors and unknown ancestry when classifying nested runs. Trim the classification corpus to focused three-state coverage. Assisted-by: ChatGPT 5.2
Assisted-by: OpenAI Codex
Not fixed: agents nested inside another agent I couldn't do from callback metadata; the measurement and three options are in the PR body. I resolved your four threads and the two Copilot ones. Reopen anything I closed too early. |
Compiled create_agent graphs carry their own unshadowed marker in graph.config, so patching Pregel.stream/astream lets a graph announce itself on a context stack that the callback handler claims when the root run starts. Nested agents then layer correctly regardless of what the enclosing agent's config merged over their callback metadata. Replaces the metadata/ancestry inference: create_agent_ancestry, the has_create_agent_marker plumbing, and the ls_integration / lc_agent_name branches in _has_agent_signals and resolve_agent_name are removed. Also covers langgraph's deprecated create_react_agent, which compiles its own graph and carries no marker, by tagging what the factory returns. Verified: three-level nesting, same-named nested agents, async, streaming, unnamed agents, agents invoked without config forwarding, and nested create_react_agent. No false positives for a named RunnableLambda or a plain subgraph invoked from a tool. User overrides (agent_name, otel_trace, otel_agent_span, run_name) behave as they do without this change. Prototype only - the existing tests for the removed inference still need to be dropped and the corpus tests reworked to drive real invocations. Assisted-by: Claude Opus 5
|
@lmolkova Folding your prototype in, and reworking the corpus tests through Hadn't clocked that the mock made those tests pass with no span at all. |
Ruff and the formatter over the modules added by the previous two commits. Applied here rather than by amending them so authorship stays intact.
| @@ -0,0 +1 @@ | |||
| Emit ``invoke_agent`` spans for top-level LangChain ``create_agent`` graph roots without misclassifying known descendants; nested agents remain unsupported. | |||
| try: | ||
| import langgraph.prebuilt | ||
| import langgraph.pregel | ||
| except ImportError: | ||
| pass | ||
| else: | ||
| for method in ("stream", "astream"): | ||
| unwrap(langgraph.pregel.Pregel, method) | ||
| for symbol in ( | ||
| "create_react_agent", | ||
| "create_tool_calling_executor", | ||
| ): | ||
| unwrap(langgraph.prebuilt, symbol) |
lmolkova
left a comment
There was a problem hiding this comment.
thanks! a few more minor comments. Please update PR description to describe what PR does, it does not need to describe a history of changes it went through
| # parent's name, unless the user renamed it - then the rename | ||
| # already opened the layer and this run would duplicate it. | ||
| if ( | ||
| suggested_agent_name == declared_agent_name |
There was a problem hiding this comment.
A nested agent still gets no span when the outer agent is named via metadata["agent_name"] - the inner run inherits that name, so it is neither equal to declared_agent_name nor different from the parent's, and this branch drops the layer.
outer = create_agent(model, [delegate], name="outer").with_config(
metadata={"agent_name": "math_agent"}
)
outer.invoke({"messages": [("user", "start")]})
# execute_tool delegate, invoke_agent math_agent - inner agent missing| suggested_agent_name == declared_agent_name | |
| declared_agent_name |
That restores the layer with all existing tests still passing, but the inner span is then named math_agent. The name precedence in resolve_agent_name needs the same look: inherited metadata outranks the graph's own declared name.
| telemetry.invoke_local_agent.assert_not_called() | ||
|
|
||
|
|
||
| @pytest.mark.xfail( |
There was a problem hiding this comment.
This strict xfail asserts nested agents do not work, while test_nested_agent_gets_its_own_span_under_the_calling_tool asserts they do. It only fails because it never starts the instrumentor - add start_instrumentation and it XPASSes. Drop it.
| return _announce_at_stream_start(wrapped(*args, **kwargs), name) | ||
|
|
||
|
|
||
| def wrap_astream( |
There was a problem hiding this comment.
The async path has no tests - every new test drives invoke(). wrap_astream is the trickier half (contextvars across __anext__), so it needs at least the root, nested and concurrent-gather cases the sync side has.
What
Classifies
create_agentruns by having the compiled graph announce itself, instead of inferring from callback metadata.How this got here
The first version of this PR read
ls_integration/lc_agent_nameoff the callback metadata. That does not work: when an agent is invoked from inside another agent's tool with the config forwarded, LangChain merges the caller's config over the inner agent's own, so those fields describe the outer agent. We could not find any callback field that separates the two —serializedisNone,langgraph_checkpoint_nsdoes not deepen at the nested root, andtags/kwargs["name"]do not distinguish an agent from any other named runnable.@lmolkova reached the same conclusion independently and supplied the mechanism that does work: a compiled graph's own bound config is never shadowed, so patching
Pregel.stream/astreamlets the graph announce itself on a context stack that the handler claims when the root run starts. That commit is hers, cherry-picked here with authorship preserved.What changed
create_agent_ancestry, thehas_create_agent_markerplumbing, and thels_integration/lc_agent_nameclassification branches are gone.agent_context.pyholds the announcement stack;claim_agent()takes only the innermost unclaimed entry, so nesting layers without any ancestry walk.Behaviour change worth noting: a nested agent now gets its own
invoke_agentspan carrying its owngen_ai.agent.name, falling back toLangGraphwhen unnamed. The metadata version suppressed it, because the marker it saw was inherited and therefore untrustworthy. With ground truth there is nothing to suppress.Deprecated
create_react_agentandcreate_tool_calling_executorcompile their own graphs and carry no bound-config marker, so they are tagged at the factory instead.Tests
The classification tests used to construct the callback handler directly against a
MagicMocktelemetry handler. They asserted the shape ofinvoke_local_agentcalls, which meant they passed for classifications that emitted no usable span at all. They now drive real invocations throughinstrument()against a real tracer provider and assert on exported spans — names, parent/child nesting, andgen_ai.agent.name. The nesting cases assert the actual parent chain, which is the only form in which they mean anything.Seven tests that existed only to check the removed inference are deleted.
Verified on both the latest and oldest supported LangChain:
246 passedand235 passed, 5 skippedrespectively.langgraph.graphis imported behind the version guard because it is not importable on the oldest.Pregel.invokeandainvokedrive their runs throughself.stream/self.astream, so the patch covers non-streaming invocation too.