Problem (one or two sentences)
With the OpenAI Compatible provider, cache reads are always recorded as 0 even when the endpoint's response reports a cache hit, so the task cost shown is inflated by roughly the cache hit ratio — measured at ~2.3× on real usage.
Context (who is affected and when)
Anyone using the OpenAI Compatible provider against an endpoint that returns usage in the OpenAI shape (prompt_tokens_details.cached_tokens) rather than the Anthropic shape — e.g. a local LiteLLM proxy in front of TensorX or DeepInfra. Affects every task, since the cost display, task history, allowedMaxCost, and the profile's "Cache reads price" field all derive from the same (wrong) number.
Reproduction steps
- Configure an OpenAI Compatible profile pointed at an OpenAI-compatible endpoint that supports prompt caching and reports it via usage.prompt_tokens_details.cached_tokens (confirmed with TensorX and DeepInfra behind a LiteLLM proxy). Enable "Supports prompt caching" and set a cache-reads price.
- Send a request with a long, stable prefix (~4600+ tokens) as the system/first message, plus a short user message.
- Send a second request with the same prefix and a different short user message.
- Inspect the raw HTTP response usage for both calls (e.g. via the endpoint's own log, or curl directly against the endpoint outside Zoo).
- Compare that to what Zoo records for the task (tasks//history_item.json → cacheReads, and the cost shown in the UI).
Expected result
Zoo reflects the cache hit from call 2: the task's cacheReads is greater than 0, and those cached tokens are priced at the profile's cache-reads price instead of the full input rate.
Actual result
cacheReads is 0 for every task, regardless of real cache hits. Measured directly:
tensorx/qwen/qwen3.8-flash-next
call 1: usage = {"prompt_tokens": 4621, "completion_tokens": 16}
call 2: usage = {"prompt_tokens": 4621, "prompt_tokens_details": {"cached_tokens": 4608}}
deepinfra/deepseek-ai/DeepSeek-V4-Flash-0731
call 1: usage = {"prompt_tokens": 5053, "prompt_tokens_details": {"cached_tokens": 0}, "estimated_cost": 0.000304}
call 2: usage = {"prompt_tokens": 5053, "prompt_tokens_details": {"cached_tokens": 4864}, "estimated_cost": 0.0000847}
The endpoint applies the discount itself (3.6× cheaper on call 2, by its own estimated_cost), but Zoo never reads prompt_tokens_details.cached_tokens, so it prices call 2 as if nothing were cached. Over one day of real tasks this made Zoo report $5.48 where the provider dashboards billed $2.36 — roughly 2.3× over.
Looking at the bundled source, the OpenAI Compatible handler builds usage from Anthropic's field names only:
cacheWriteTokens: e?.cache_creation_input_tokens || void 0,
cacheReadTokens: e?.cache_read_input_tokens || void 0
It never looks at prompt_tokens_details.cached_tokens. A generic usage normalizer already exists elsewhere in the bundle that resolves exactly this path (it tries cacheReadTokens, cache_read_input_tokens, cached_tokens, prompt_tokens_details.cached_tokens, input_tokens_details.cached_tokens), and the OpenAI-native handler's normalizeUsage already reads input_tokens_details ?? prompt_tokens_details. The OpenAI Compatible handler is the one path that doesn't.
Related: #1277 reports OpenRouter cache misses due to missing session affinity — a different root cause (the cache never gets hit there). This report is about a cache that is hit but never counted. PR #1132 touched usage/cost normalization for OpenAI-compatible providers ("normalizes conditional fields, cached tokens, missing/zero values") but was closed unmerged on 2026-08-10, so this behavior remains.
Variations tried (optional)
Reproduced identically on two different providers behind the same proxy (TensorX and DeepInfra), so it's not endpoint-specific. Both report the OpenAI usage shape; neither is read.
App Version
v3.82.0
API Provider (optional)
OpenAI Compatible
Model Used (optional)
reproduced on tensorx/qwen/qwen3.8-flash-next and deepinfra/deepseek-ai/DeepSeek-V4-Flash-0731
Zoo Code Task Links (optional)
No response
Relevant logs or errors (optional)
Problem (one or two sentences)
With the OpenAI Compatible provider, cache reads are always recorded as 0 even when the endpoint's response reports a cache hit, so the task cost shown is inflated by roughly the cache hit ratio — measured at ~2.3× on real usage.
Context (who is affected and when)
Anyone using the OpenAI Compatible provider against an endpoint that returns usage in the OpenAI shape (prompt_tokens_details.cached_tokens) rather than the Anthropic shape — e.g. a local LiteLLM proxy in front of TensorX or DeepInfra. Affects every task, since the cost display, task history, allowedMaxCost, and the profile's "Cache reads price" field all derive from the same (wrong) number.
Reproduction steps
Expected result
Zoo reflects the cache hit from call 2: the task's cacheReads is greater than 0, and those cached tokens are priced at the profile's cache-reads price instead of the full input rate.
Actual result
cacheReads is 0 for every task, regardless of real cache hits. Measured directly:
The endpoint applies the discount itself (3.6× cheaper on call 2, by its own estimated_cost), but Zoo never reads prompt_tokens_details.cached_tokens, so it prices call 2 as if nothing were cached. Over one day of real tasks this made Zoo report $5.48 where the provider dashboards billed $2.36 — roughly 2.3× over.
Looking at the bundled source, the OpenAI Compatible handler builds usage from Anthropic's field names only:
It never looks at prompt_tokens_details.cached_tokens. A generic usage normalizer already exists elsewhere in the bundle that resolves exactly this path (it tries cacheReadTokens, cache_read_input_tokens, cached_tokens, prompt_tokens_details.cached_tokens, input_tokens_details.cached_tokens), and the OpenAI-native handler's normalizeUsage already reads input_tokens_details ?? prompt_tokens_details. The OpenAI Compatible handler is the one path that doesn't.
Related: #1277 reports OpenRouter cache misses due to missing session affinity — a different root cause (the cache never gets hit there). This report is about a cache that is hit but never counted. PR #1132 touched usage/cost normalization for OpenAI-compatible providers ("normalizes conditional fields, cached tokens, missing/zero values") but was closed unmerged on 2026-08-10, so this behavior remains.
Variations tried (optional)
Reproduced identically on two different providers behind the same proxy (TensorX and DeepInfra), so it's not endpoint-specific. Both report the OpenAI usage shape; neither is read.
App Version
v3.82.0
API Provider (optional)
OpenAI Compatible
Model Used (optional)
reproduced on tensorx/qwen/qwen3.8-flash-next and deepinfra/deepseek-ai/DeepSeek-V4-Flash-0731
Zoo Code Task Links (optional)
No response
Relevant logs or errors (optional)