Gate sensitive attributes in gen_ai.tool.definitions per semconv spec update - #377
Gate sensitive attributes in gen_ai.tool.definitions per semconv spec update#377rads-1996 wants to merge 15 commits into
gen_ai.tool.definitions per semconv spec update#377Conversation
b21b2e2 to
d079031
Compare
gen_ai.tool.definitions and gen_ai.tool.description as sensitive attributes per semconv spec updategen_ai.tool.definitions and gen_ai.tool.description as sensitive attributes per semconv spec update
b7bf74e to
04d1fb9
Compare
Pull request dashboard statusWaiting on the author · refreshed 2026-08-22 00:21 UTC Two things need attention:
Status above doesn't look right?
|
There was a problem hiding this comment.
Pull request overview
This PR updates GenAI content-capture behavior to align with the updated semantic conventions by treating gen_ai.tool.definitions and gen_ai.tool.description as sensitive and omitting them from spans when span content capture is disabled, with corresponding cross-package test updates.
Changes:
- Gate
gen_ai.tool.descriptionon tool spans behind span content-capture (should_capture_content_on_span) at span-creation time. - Gate
gen_ai.tool.definitionsemission viaget_content_attributes(..., for_span=...)so it is only recorded on the relevant signal (span vs event) when content capture targets that signal. - Update unit/integration tests across util + multiple instrumentations to assert tool description/definitions are omitted under
NO_CONTENTand present when capture is enabled; add towncrier fragments.
Reviewed changes
Copilot reviewed 20 out of 20 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| util/opentelemetry-util-genai/tests/test_toolcall.py | Ensures tool span start attributes (including description) are only asserted under span content capture. |
| util/opentelemetry-util-genai/tests/test_handler_agent.py | Adds coverage that tool definitions are omitted from spans when content capture is disabled. |
| util/opentelemetry-util-genai/src/opentelemetry/util/genai/_tool_invocation.py | Gates gen_ai.tool.description at span creation based on should_capture_content_on_span. |
| util/opentelemetry-util-genai/src/opentelemetry/util/genai/_invocation.py | Changes content attribute serialization to omit tool definitions when content capture does not target the signal. |
| util/opentelemetry-util-genai/.changelog/377.added | Adds release note fragment for sensitive tool attributes. |
| instrumentation/opentelemetry-instrumentation-google-genai/tests/utils/test_tool_call_wrapper.py | Updates wrapper tests to assert tool description is omitted without content capture. |
| instrumentation/opentelemetry-instrumentation-google-genai/tests/interactions/base.py | Adds test asserting tool definitions are omitted on spans when content capture is off. |
| instrumentation/opentelemetry-instrumentation-google-genai/tests/generate_content/nonstreaming_base.py | Updates no-content expectations to ensure tool definitions are absent from events/spans. |
| instrumentation/opentelemetry-instrumentation-google-genai/.changelog/377.added | Adds release note fragment for sensitive tool attributes. |
| instrumentation/opentelemetry-instrumentation-genai-qwen-agent/tests/test_tool.py | Extends no-content test assertions to include tool description omission. |
| instrumentation/opentelemetry-instrumentation-genai-qwen-agent/.changelog/377.added | Adds release note fragment for sensitive tool attributes. |
| instrumentation/opentelemetry-instrumentation-genai-openai/tests/test_chat_completions.py | Updates tests to assert tool definitions are not present when content capture is off. |
| instrumentation/opentelemetry-instrumentation-genai-openai/tests/test_async_chat_completions.py | Same as sync: asserts tool definitions omitted when content capture is off. |
| instrumentation/opentelemetry-instrumentation-genai-openai/.changelog/377.added | Adds release note fragment for sensitive tool attributes. |
| instrumentation/opentelemetry-instrumentation-genai-langchain/tests/test_tools.py | Adds tests around tool description/definitions behavior under different content capture modes. |
| instrumentation/opentelemetry-instrumentation-genai-langchain/tests/test_llm_call.py | Adds test asserting legacy OpenAI tool definitions are omitted when content capture is disabled. |
| instrumentation/opentelemetry-instrumentation-genai-langchain/.changelog/377.added | Adds release note fragment for sensitive tool attributes. |
| instrumentation/opentelemetry-instrumentation-genai-agno/tests/test_tools.py | Updates/extends tests to cover tool definitions omission without span capture; adds needed imports/mode coverage. |
| instrumentation/opentelemetry-instrumentation-genai-agno/tests/requirements.oldest.txt | Ensures oldest tests install workspace util dependency as needed. |
| instrumentation/opentelemetry-instrumentation-genai-agno/.changelog/377.added | Adds release note fragment for sensitive tool attributes. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
|
|
||
| def _get_start_attributes(self) -> dict[str, AttributeValue]: | ||
| """Return sampling-relevant attributes available at span creation time.""" | ||
| tool_description = ( |
There was a problem hiding this comment.
tool description may be sensitive, but it's still recommended -https://github.com/open-telemetry/semantic-conventions-genai/blob/main/docs/registry/attributes/gen-ai.md.
Only opt-in attributes should be behind the flag
There was a problem hiding this comment.
Further down in that doc we state: "Since this attribute could be large, it's NOT RECOMMENDED to populate non-required properties by default. Instrumentations MAY provide a way to enable populating optional properties." -- not sure which properties are "non-required" but I think its tool description and tool parameters
As an aside why do we have a sem conv for tool description at all ? That seems like it's already in tool defintion and can/should be removed
There was a problem hiding this comment.
@lmolkova, @DylanRussell So does that mean the tool description should be emitted, but be gated inside tool definitions? That does not tally well. Also, for sensitive attributes that are not opt in, we should provide a way to the users to hide them behind some sort of config, if they want to, right? Otherwise, it might end up leaking sensitive information. What do you think? Please ignore the updated code. I will refactor the code after the direction is decided.
| patch( | ||
| "opentelemetry.util.genai._invocation.get_content_capturing_mode", | ||
| return_value=ContentCapturingMode.SPAN_ONLY, | ||
| ), |
There was a problem hiding this comment.
Patching the private opentelemetry.util.genai._invocation.get_content_capturing_mode reaches into util-genai internals and only flips half the state: TelemetryHandler snapshots content capture at construction (the instrument_agno fixture already built it), so the test runs a combination that can't occur in production and wouldn't catch a regression in the instrumentation-side gating.
The shared instrument helper already covers this — it sets OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT before instrumenting:
# conftest.py
@pytest.fixture
def instrument_agno_with_content(tracer_provider, logger_provider, meter_provider):
with instrument(
AgnoInstrumentor(),
tracer_provider=tracer_provider,
logger_provider=logger_provider,
meter_provider=meter_provider,
content_capture="SPAN_ONLY",
) as instrumentor:
yield instrumentor# test_tools.py
def test_agent_run_with_tools(instrument_agno_with_content, span_exporter) -> None:
...
with (
patch.object(Agent, "run", wraps=agent.run),
patch("agno.models.base.Model.response", return_value=mock_output),
):Same for the arun test, and for the no-capture test below — that one can use the plain instrument_agno fixture (or content_capture="NO_CONTENT") with no patch at all.
| # every oldest env installs. Pin here only test-only deps that nothing else already provides. | ||
| # | ||
|
|
||
| -e ./util/opentelemetry-util-genai |
There was a problem hiding this comment.
question(blocking): Is this needed and what's the use of this?
There was a problem hiding this comment.
I will remove it.
|
|
||
| # Tool definitions are always captured, the sem conv recommends adding params / description only | ||
| # when the content capture mode is set.. | ||
| # Tool definitions carry sensitive content (parameters / description) and |
There was a problem hiding this comment.
I dont think this is right, that isn't my reading of the sem conv,, i think tool definitions should ways be present but some parts of them should be omitted if content capture flag isn't set..
703bc1a to
6e0fba8
Compare
gen_ai.tool.definitions and gen_ai.tool.description as sensitive attributes per semconv spec updategen_ai.tool.definitions per semconv spec update
|
Hi @rads-1996 — just a friendly reminder that this pull request is waiting on you. The dashboard status comment has the open items and is kept current.
|
a3ce5ad to
6a1fab1
Compare
… attributes per [GenAI Spec](open-telemetry/semantic-conventions-genai#431)
bae82e8 to
1cef84c
Compare
Description
Fixes #360
Type of change
Please delete options that are not relevant.
How has this been tested?
Please describe the tests that you ran to verify your changes. Provide
instructions so we can reproduce. List any relevant details for your test
configuration.
Checklist
See CONTRIBUTING.md
for the style guide, changelog guidance, and more.