Skip to content

fix(langchain): emit invoke_agent for create_agent, including nested agents - #391

Open
AgentGymLeader wants to merge 13 commits into
open-telemetry:mainfrom
AgentGymLeader:langchain-nested-agent-classification
Open

fix(langchain): emit invoke_agent for create_agent, including nested agents#391
AgentGymLeader wants to merge 13 commits into
open-telemetry:mainfrom
AgentGymLeader:langchain-nested-agent-classification

Conversation

@AgentGymLeader

@AgentGymLeader AgentGymLeader commented Aug 15, 2026

Copy link
Copy Markdown
Contributor

What

Classifies create_agent runs by having the compiled graph announce itself, instead of inferring from callback metadata.

How this got here

The first version of this PR read ls_integration / lc_agent_name off the callback metadata. That does not work: when an agent is invoked from inside another agent's tool with the config forwarded, LangChain merges the caller's config over the inner agent's own, so those fields describe the outer agent. We could not find any callback field that separates the two — serialized is None, langgraph_checkpoint_ns does not deepen at the nested root, and tags / kwargs["name"] do not distinguish an agent from any other named runnable.

@lmolkova reached the same conclusion independently and supplied the mechanism that does work: a compiled graph's own bound config is never shadowed, so patching Pregel.stream / astream lets the graph announce itself on a context stack that the handler claims when the root run starts. That commit is hers, cherry-picked here with authorship preserved.

What changed

create_agent_ancestry, the has_create_agent_marker plumbing, and the ls_integration / lc_agent_name classification branches are gone. agent_context.py holds the announcement stack; claim_agent() takes only the innermost unclaimed entry, so nesting layers without any ancestry walk.

Behaviour change worth noting: a nested agent now gets its own invoke_agent span carrying its own gen_ai.agent.name, falling back to LangGraph when unnamed. The metadata version suppressed it, because the marker it saw was inherited and therefore untrustworthy. With ground truth there is nothing to suppress.

Deprecated create_react_agent and create_tool_calling_executor compile their own graphs and carry no bound-config marker, so they are tagged at the factory instead.

Tests

The classification tests used to construct the callback handler directly against a MagicMock telemetry handler. They asserted the shape of invoke_local_agent calls, which meant they passed for classifications that emitted no usable span at all. They now drive real invocations through instrument() against a real tracer provider and assert on exported spans — names, parent/child nesting, and gen_ai.agent.name. The nesting cases assert the actual parent chain, which is the only form in which they mean anything.

Seven tests that existed only to check the removed inference are deleted.

Verified on both the latest and oldest supported LangChain: 246 passed and 235 passed, 5 skipped respectively. langgraph.graph is imported behind the version guard because it is not importable on the oldest.

Pregel.invoke and ainvoke drive their runs through self.stream / self.astream, so the patch covers non-streaming invocation too.

Runtime testing exposed that nested agents could be missed when LangGraph propagated the parent node name into langgraph_node.

Because agent run-name resolution falls back to langgraph_node, restrict the lc_agent_name comparison to the explicit kwargs name. This keeps inherited node metadata from contaminating the comparison while allowing nested agents without an explicit run name.
Copilot AI lite review requested due to automatic review settings August 15, 2026 13:53
@AgentGymLeader
AgentGymLeader requested a review from a team as a code owner August 15, 2026 13:53
@opentelemetry-pr-dashboard

opentelemetry-pr-dashboard Bot commented Aug 15, 2026

Copy link
Copy Markdown

Pull request dashboard status

Waiting on the author · refreshed 2026-08-22 15:51 UTC

Respond to 6 review items (e.g. link a commit, explain why not, ask a follow-up):

  • Inline threads: 1, 2, 3, 4, 5
  • Top-level threads: 6
Status above doesn't look right?
  • Just replied or pushed? Anything around or after the refresh time above may not be picked up yet — give it a few minutes.
  • Should this be with reviewers? Comment /dashboard route:reviewers to route it to them.
  • Anything wrong — including the routing? Report it with what you expected; it helps us improve the dashboard.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

This PR refines agent span classification for LangChain/LangGraph runs by using LangChain’s lc_agent_name signal while avoiding false positives for internal LangGraph nodes.

Changes:

  • Add lc_agent_name-aware agent detection logic in classify_chain_run via _has_agent_signals.
  • Avoid misclassifying internal LangGraph nodes as agents by comparing against the callback’s explicit run name.
  • Add unit tests covering top-level agents, nested agents, internal nodes, and suppression behavior.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
instrumentation/opentelemetry-instrumentation-genai-langchain/src/opentelemetry/instrumentation/genai/langchain/operation_mapping.py Updates agent-signal detection to incorporate lc_agent_name with LangGraph-aware heuristics.
instrumentation/opentelemetry-instrumentation-genai-langchain/tests/test_operation_mapping.py Adds tests validating new classification behavior and suppression rules.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Distinguish confirmed agent ancestors from confirmed non-agent ancestors and unknown ancestry when classifying nested runs. Trim the classification corpus to focused three-state coverage.

Assisted-by: ChatGPT 5.2
@AgentGymLeader

AgentGymLeader commented Aug 16, 2026

Copy link
Copy Markdown
Contributor Author

@lmolkova Pushed 42aaaf4.

  • Conformance: updated the existing scenario. It uses create_agent now and expects invoke_agent, green under weaver 0.25.1.
  • ls_integration: agreed, that's the marker. A run is a root when it carries it and no ancestor does, so the default form emits invoke_agent now.
  • run_name override: dropped the name comparison. It compares against the nearest marked ancestor now, using the parent runs the handler already tracks.
  • resolve_agent_name: checks lc_agent_name before the run name now.

Not fixed: agents nested inside another agent I couldn't do from callback metadata; the measurement and three options are in the PR body.

I resolved your four threads and the two Copilot ones. Reopen anything I closed too early.

Compiled create_agent graphs carry their own unshadowed marker in
graph.config, so patching Pregel.stream/astream lets a graph announce
itself on a context stack that the callback handler claims when the root
run starts. Nested agents then layer correctly regardless of what the
enclosing agent's config merged over their callback metadata.

Replaces the metadata/ancestry inference: create_agent_ancestry, the
has_create_agent_marker plumbing, and the ls_integration / lc_agent_name
branches in _has_agent_signals and resolve_agent_name are removed.

Also covers langgraph's deprecated create_react_agent, which compiles its
own graph and carries no marker, by tagging what the factory returns.

Verified: three-level nesting, same-named nested agents, async, streaming,
unnamed agents, agents invoked without config forwarding, and nested
create_react_agent. No false positives for a named RunnableLambda or a
plain subgraph invoked from a tool. User overrides (agent_name, otel_trace,
otel_agent_span, run_name) behave as they do without this change.

Prototype only - the existing tests for the removed inference still need to
be dropped and the corpus tests reworked to drive real invocations.

Assisted-by: Claude Opus 5
@AgentGymLeader

Copy link
Copy Markdown
Contributor Author

@lmolkova Folding your prototype in, and reworking the corpus tests through instrument() against exported spans.

Hadn't clocked that the mock made those tests pass with no span at all.

Ruff and the formatter over the modules added by the previous two commits.
Applied here rather than by amending them so authorship stays intact.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 2 comments.

@@ -0,0 +1 @@
Emit ``invoke_agent`` spans for top-level LangChain ``create_agent`` graph roots without misclassifying known descendants; nested agents remain unsupported.
Comment on lines +119 to +131
try:
import langgraph.prebuilt
import langgraph.pregel
except ImportError:
pass
else:
for method in ("stream", "astream"):
unwrap(langgraph.pregel.Pregel, method)
for symbol in (
"create_react_agent",
"create_tool_calling_executor",
):
unwrap(langgraph.prebuilt, symbol)

@lmolkova lmolkova left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks! a few more minor comments. Please update PR description to describe what PR does, it does not need to describe a history of changes it went through

# parent's name, unless the user renamed it - then the rename
# already opened the layer and this run would duplicate it.
if (
suggested_agent_name == declared_agent_name

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A nested agent still gets no span when the outer agent is named via metadata["agent_name"] - the inner run inherits that name, so it is neither equal to declared_agent_name nor different from the parent's, and this branch drops the layer.

outer = create_agent(model, [delegate], name="outer").with_config(
    metadata={"agent_name": "math_agent"}
)
outer.invoke({"messages": [("user", "start")]})
# execute_tool delegate, invoke_agent math_agent - inner agent missing
Suggested change
suggested_agent_name == declared_agent_name
declared_agent_name

That restores the layer with all existing tests still passing, but the inner span is then named math_agent. The name precedence in resolve_agent_name needs the same look: inherited metadata outranks the graph's own declared name.

telemetry.invoke_local_agent.assert_not_called()


@pytest.mark.xfail(

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This strict xfail asserts nested agents do not work, while test_nested_agent_gets_its_own_span_under_the_calling_tool asserts they do. It only fails because it never starts the instrumentor - add start_instrumentation and it XPASSes. Drop it.

return _announce_at_stream_start(wrapped(*args, **kwargs), name)


def wrap_astream(

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The async path has no tests - every new test drives invoke(). wrap_astream is the trickier half (contextvars across __anext__), so it needs at least the root, nested and concurrent-gather cases the sync side has.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

3 participants