Skip to content

Anthropic prompt caching: cache tokens not emitted on the Anthropic paths — is session total_tokens systematically low? #1445

Description

@roy-tong

A semantics question found while auditing token accounting across agent-telemetry SDKs.

The observation

On the Anthropic paths, usage extraction reads only usage.input_tokens / usage.output_tokens and computes total = input + output:

  • non-streaming: agentops/instrumentation/providers/anthropic/attributes/message.py (~L319-332)
  • streaming: .../anthropic/stream_wrapper.py (~L167-177 and ~L406-418)

Anthropic's cache_read_input_tokens / cache_creation_input_tokens are separate fields (not included in input_tokens), and I don't find them read anywhere on these paths — LLM_USAGE_CACHE_READ_INPUT_TOKENS / CACHE_CREATION_INPUT_TOKENS are defined in agentops/semconv/span_attributes.py but have no setter calls in the Anthropic instrumentation.

Meanwhile the OpenAI Responses path (providers/openai/attributes/response.py, ~L296-307) does emit cached_tokens → cache_read and reasoning_tokens — so the asymmetry is within one SDK.

Why it matters

The OTel GenAI semantic conventions for Anthropic (semantic-conventions-genai, docs/gen-ai/anthropic.md) are explicit: cache read/write tokens MUST be added to the native input_tokens to compute gen_ai.usage.input_tokens. If they're dropped instead, a prompt-caching Claude session reports total_tokens systematically below what the provider actually meters (cache reads are billed, just at a lower rate — they're not free tokens).

The question

Is this asymmetry intentional (e.g. a deliberate "native usage only" convention on the Anthropic path)? If not: for sessions with cache_control enabled, does any downstream token/cost figure derived from these spans undercount? Happy to send a minimal repro — same prompt called twice with caching on, asserting the cache attributes are absent and the total vs the Anthropic console's billed tokens diverge — or a small PR adding the cache fields per the semconv rule.

Context: I maintain AgentMeasure, a measurement-conformance project running a public 30-projects/30-days check of agent-telemetry semantics; this is the finding worth asking about directly. Not claiming a bug without your read on the intended convention.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions