feat: trace every request with prompts redacted by privacy class - #30
Merged
Merged
Conversation
Section 7.1 has the gateway attach trace and routing metadata and section 14 requires sensitive prompts to be redacted from traces. There was no tracing at all. Each chat completion is now one OpenTelemetry span carrying what explains the route: tenant, effective and declared privacy, task, model, revision, adapter, cache result, score, reason, and token usage. The response quotes it as X-Trace-Id. Only the API is a runtime dependency, so tracing is a no-op until ROUTER_OTLP_ENDPOINT points at a collector. Redaction is evaluated after the tenant floor. Restricted prompts record their length only, with no digest, because a digest of a short or templated prompt can be reversed by guessing. Private prompts add a digest. Content is recorded only for public prompts, only on operator opt-in, and only as a bounded prefix. Completions are never recorded, and a failure records its error type rather than its message because an engine error can echo the request it rejected. A streamed response owns its span until the stream ends, so usage and failures after the handler returns are still recorded. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
github-actions
Bot
added
documentation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Third PR closing gaps between the design spec and the implementation.
Gap this closes
§7.1 "Attach trace and routing metadata" and §14 "Redact sensitive prompts from traces". OpenTelemetry is in the §5 architecture diagram and appeared nowhere in the code.
What changed
tracing.py: one span per chat completion with tenant, effective and declared privacy, task, model, revision, adapter, cache result, score, candidate count, route reason, and token usage (GenAI semantic-convention names where they exist). The response quotes the trace asX-Trace-Id.opentelemetry-apiis a runtime dependency; the SDK and OTLP exporter are in a newtracingextra. With noROUTER_OTLP_ENDPOINT, tracing is a no-op and no header is sent.restrictedrecords length only (no digest),privateadds a SHA-256 digest,publicrecords a bounded content prefix only withROUTER_TRACE_PROMPT_CONTENT=true. Completions are never recorded.429is attributed to its tenant in the trace.Test plan
publicwith content recording on is still fully redacted; an engine error echoing the prompt does not leak it; streamed spans carry usage and failure statusruff format --check .,ruff check .,mypycleanpytest tests/unit tests/integration: 205 passed, coverage 98%; the 3 errors are the local Windows temp-dir permission issue ontmp_path, which does not occur on CItracingextra and a collector, and is excluded from coverage🤖 Generated with Claude Code