Skip to content
Merged
15 changes: 15 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,21 @@ All notable changes to this project will be documented here. Format follows [Kee

## [Unreleased]

### Changed

- **README restructured.** The gateway deep-dives (Cloudflare, Databricks, Snowflake, Ramp Router) and the provider coverage/token-semantics tables moved to `docs/`, verbatim; the README keeps a short quickstart per provider and per gateway with a link to each full guide.

### Added

- **Ramp Router support: a customer pointing a client at Router billed nothing at all.** Router is an OpenAI-Responses-compatible gateway in front of OpenAI, Anthropic, Google Vertex, Fireworks and xAI, so a wrapped OpenAI client aimed at it made real, paid calls that produced **zero events** — the SDK had no way to tell a Router-routed call from a direct OpenAI one. The trick that identifies Cloudflare Workers AI cannot work here: Router's model ids are account-specific and opaque (its docs: "Never invent one or reuse a provider's public model name"), and an Anthropic-served response arrives in OpenAI's schema, byte-indistinguishable from a real OpenAI one. `base_url` is the only signal, read once at wrap time so the hot path gains no per-call work — and matched on the parsed HOST, because `"api.router.com" in base_url` also matches `https://evil.example.com/api.router.com/v1` and would stamp an unrelated endpoint's traffic as Router-served.
- **The model that answered is billed, not the one requested.** Two Router features make those diverge: a `models` fallback list sends no `model` field at all, and Switchyard routing can serve a different model than the one asked for. Router answers with a resolved bare vendor snapshot (`openai:gpt-5.4-nano` in, `gpt-5.4-nano-2026-03-17` out), which is what bills; the served tier is read from the response's own top-level `service_tier` field. As a fallback for an id that reaches the adapter unresolved, an explicit `provider:provider-model[:service-tier]` candidate is split on the **first** colon only — Fireworks candidates carry a path as their model segment (`fireworks:accounts/fireworks/models/kimi-k2p7-code`), so a naive split silently loses everything after the second separator. The service tier is matched against Router's documented set rather than read as "text after the last colon": a model segment can contain a colon of its own, and mistaking one for a tier renames the model and splits it into a second row in Lago against the live path's own name.
- **Price mode deliberately misses, and falls back to exact token events.** Router is stamped as its own provider, matching no vendor in the price tables, for three separate money reasons rather than as a shortcut. A **BYOK-served request costs $0** through Router ("your provider bills you directly and Ramp Router does not charge for that usage") and nothing in the response says which key served it, so a list-price lookup bills the full amount for usage Router never charged for. A **non-default service tier** does not bill at the published base rate — Router's own catalog says tiers "may use different rates". And the **overlap semantics belong to Router, not the served vendor** — measured live on an Anthropic-served model: Router normalizes the _numbers_ to OpenAI's convention, not just the schema (a warm `cache_control` call reports the cached block inside `input_tokens`; reasoning reports inside `output_tokens`), so `ramp_router` carries its own `OPENAI_SHAPED_APIS` entry, and stamping the served vendor would de-overlap with the wrong convention whenever that vendor's native one differs. Token mode, the default, is exact either way. The served tier is recorded in `extras` (diagnostic only — `extras` is not serialized into Lago events), and the catalog's own `router.pricing` rates make a Router price mode buildable later — but the response still cannot say whether a BYOK key served the call for $0, so token counts stay the honest default.
- **No backfill adapter, stated rather than invented.** Router is the first gateway here with no programmatic usage surface: its only routes are `GET /v1/models`, `POST /v1/responses`, `POST /v1/messages` and `POST /v1/messages/count_tokens`, and usage lives in the dashboard. An "analytics API" appears exactly once across all 20 documentation pages, in a limits table, with no path, auth or record shape. An adapter over guessed field names would have tests proving only that it matches the guess, so `src/lago_agent_sdk/gateway/__init__.py` carries the seam and the reason instead.
- **Nothing is skipped as a gateway cache hit**, because Router has no response cache — "Self-service Router response caching ... is not currently configurable". There is no `cf-aig-cache-status` analogue to read, and `_is_cache_hit` is untouched in both wrappers. Provider _prompt_ caching does pass through and those tokens bill normally.
- **Rebased onto the provider-hint table, the token-semantics roster and the REST dedup rather than around them.** Router's detection is the one hint that must match a HOST instead of a path, so `_provider_hint_for` keeps the path table's first say and gains a parsed-host arm after it. `ramp_router` joins `TOKEN_BILLED_PROVIDERS` — a structurally unpriceable provider must route to token events without a per-call price-miss report, the same decision Databricks and Snowflake carry — and joins `KNOWN_PROVIDERS` with its convention recorded in `OPENAI_SHAPED_APIS` — first entered conservatively as additive, then settled by measurement against a live account: Router normalizes the numbers to OpenAI's convention on cache and reasoning alike, so the guard reads Router counts as OpenAI-shaped. Both decisions are pinned by tests that fail if either set entry is reverted.
- **The `total_tokens` guard reads Router's stamped `api`, not the surface the call arrived on.** The adapter reassigns `api` to `ramp_router` after parsing the model, and the guard originally ran before that — so it asked `token_semantics("ramp_router", "responses")`, which matches no subset set and answers all-additive, while `compute_cost` and `deoverlapped_token_total` read the stamped `api="ramp_router"` and de-overlapped the same row as subset: the exact divergence `token_semantics.py` exists to make impossible. Unreachable on every captured payload — all ten report `total == input + output`, streamed included — but the guard exists for the day Router misreports, and on that day it under-folds a genuine remainder by `cache_read + reasoning`; where that over-count exceeds the declared total it suppresses the fold entirely, dropping generated tokens with no `extras` key and no `on_error`. The stamp now lands above the guard.
- 57 new tests. The price-mode ones load a real price table and bill the same model both directly and through Router, because the first version of them passed against a table that had **silently failed to load** — everything misses when nothing is priced, so asserting a miss on its own proved nothing. Captured fixtures from a live Router account back the seven wire-behaviour questions (catalog with `router.pricing`, buffered + fallback + streamed calls, the cache pair that settled the overlap question, a reasoning call, and three error families), written by a capture script that scrubs in the same step it writes and swept for credentials, emails and routable addresses before committing. The unit-test shapes stay hand-built to the documented schema where only the SDK's own decisions are under test.

### Fixed

- **A `views=("rest",)` backfill of a window the live path already billed is now rejected by Lago instead of billing every call twice.** The wrapped OpenAI client stamps each Cortex call's events with the id the reader derives from that call's row — `sfc_rest_<subscription>_<REQUEST_ID>` — read from the response's `x-snowflake-request-id` header, which is byte-identical to the REST view's `REQUEST_ID` (measured live). Both paths build the key through one shared `snowflake_event_id` helper, so the two spellings cannot drift apart silently. The dedup holds only when the backfill runs with the default `event_id_prefix` and resolves the same subscription the live path billed; both constraints are documented on `backfill_snowflake`. Streamed calls carry the key too — the header is readable off the stream's own response before the body is consumed (verified live). A missing header, or a client without `.with_raw_response`, keeps today's per-event UUID; a real-OpenAI call's ids are unchanged even if a proxy injects the header, because the keying is gated on the Snowflake provider hint rather than on the header's presence. One residual is documented rather than closed: a cache-creation call's cached block still double-counts on a backfilled live window — the wire reports creations as reads, so the two sides bill that component under different metrics and different ids; the call's input and output are deduplicated.
Expand Down
1 change: 1 addition & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,7 @@ uv lock --upgrade-package X # bump a single package
- `src/lago_agent_sdk/gateway/` — second front door: gateway usage logs → `CanonicalUsage`, for backfill
- `tests/unit/` — unit tests, organized to mirror `src/`
- `tests/unit/adapters/fixtures/` — captured real provider responses, used by adapter tests
- `tests/unit/adapters/fixtures/capture_*.{py,ts}` — capture scripts; each reads its credential from the environment and scrubs secrets in the same step that writes a fixture into the tree

## Adding a provider

Expand Down
Loading
Loading