diff --git a/CHANGELOG.md b/CHANGELOG.md index ac73501..555a8f6 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,26 @@ All notable changes to this project will be documented here. Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); versions follow [SemVer](https://semver.org). +## [Unreleased] + +### Added + +- **Ramp Router price mode: a Router call is now priced from Router's own catalog.** `GET /v1/models` publishes a per-model rate block, and it is the rate Router bills — measured against a live account's dashboard export on 2026-09-04 and 2026-09-07: every default-tier call whose counts the response fully reports reconciled at exactly the catalog rate across all five served vendors, including the cache split (xAI, 194 in / 192 cached, to the last digit). `ramp_router` leaves `TOKEN_BILLED_PROVIDERS`: its miss is no longer structural, so a Router call that cannot be priced reports through `on_error` like any other provider's. + - **The Router key is learned from the wrapped client.** The catalog is account-scoped, and `OpenAI(api_key=...)` already carries the key that unlocks it, so `wrap()` reads it (same mechanism as the Mistral alias source) and primes the catalog before the first call. `LagoConfig.ramp_router_api_key` exists for pricing Router usage without `wrap()` and always wins over a learned key. + - **Every name a response can report is indexed.** Router answers with the vendor's own dated snapshot (`gpt-5.4-nano-2026-03-17`, version-stripped to the catalog id) or, for Fireworks and Baseten, the vendor's own path (`accounts/fireworks/models/…`), which is the entry's `provider_model` or an alias — never its id. All nine distinct served names across every capture resolve. A name two entries claim at different rates is left unpriced rather than guessed; a zero cache rate is stored as "no separate rate" so those tokens stay inside `input` at the input rate instead of billing a cached block at $0. + - **Only the default tier is priced.** `flex` measured 0.5x the catalog rate and `priority` 2.0x on two vendors, and neither factor is published anywhere machine-readable, so an explicitly reported non-default tier is a price miss: token events plus a `PricingUnavailableError` whose new `detail` names the tier. A response with **no** tier bills at the base rate — a 237-call sweep showed Router omits `service_tier` only on responses that stopped with zero output, and bills those at standard. + - **Eight OpenAI models bill off Router's own catalog; the SDK bills the catalog and documents the mismatch.** Reconciled row by row against the dashboard export of that sweep: `gpt-5.4-mini`, `gpt-5.4-nano`, `gpt-5.5`, `gpt-5.5-pro`, `gpt-5.6-luna`, `gpt-5.6-terra` and `gpt-6-astra` bill exactly 1.1x their published rate on every field; `gpt-5.6-sol` bills 0.55x — a 50%-off promotion on Router's model page, times the same 1.1. Every other model on every vendor billed the catalog rate exactly. No factor is applied in code — the day Router corrects its catalog a baked-in factor would be the thing out of sync — and the docs give the measured factors, dated, as a `markup` a customer can apply to those models and drop when the catalog catches up. + - **A catalog entry served through a different backend is a miss, not a price.** Ten Fireworks-owned entries carry a Baseten alias; when Baseten serves, Router bills Baseten's unpublished rate (1.11x to 2.4x off the catalog's) and names the model by that alias. Those aliases are not indexed, so the call takes the honest token-event path. + - **OpenAI-served cache writes are billed at the catalog write rate.** `/v1/responses` reports `cache_write_tokens` inside `input_tokens` for OpenAI models, and Router bills it at `cache_write_input` (gpt-5.6-luna, 4493 in / 4490 written: `3 × input + 4490 × cache_write + output`, exact). The adapter maps the count for the Router surface only; OpenAI-native keeps it unmapped, where Databricks metered the same field at the plain input rate. Anthropic-served writes stay unpriceable on this surface — the count is never reported — and bill at the input rate, documented as a shortfall of up to 20% (5m TTL) or 50% (1h TTL) on a cold call, in proportion to how much of the prompt was the cached prefix. + - Two dashboard-reconciled rows join `money_golden.json` in both repos. + - **An Anthropic client pointed at Router is detected too, on Router's `/v1/messages` surface.** Both wrappers now read one shared host helper, so they cannot disagree about what counts as Router. The Anthropic adapter takes the wrapper's hint and stamps `provider=ramp_router api=ramp_router_messages` — a stamp distinct from the Responses surface's, because the two report the same vendor under different conventions: `/v1/messages` keeps Anthropic's additive shape for every vendor (measured on an Anthropic and an xAI model), where `/v1/responses` folds cached tokens inside `input_tokens`. The Router key is learned from the Anthropic client the same way. Five new captured fixtures (11-15) back it, including a cold and warm `cache_control` pair and a streamed call. + - **Anthropic cache writes through Router are billed at their TTL rate, exactly.** `/v1/messages` reports `cache_creation_input_tokens` with the 5-minute / 1-hour split, and Router's catalog publishes both rates, so `ModelPrice` carries `cache_write_5m` / `cache_write_1h` and `compute_cost` bills each part at its own rate, consuming the lump `cache_write` count so nothing is billed twice. Reconciled against the dashboard: 16 in + 20,113 written (5m) + 5 out on `claude-haiku-4-5` = $0.02518225. Inert everywhere else: OpenRouter publishes a single Anthropic write rate, and that path is byte-for-byte unchanged. Two more dashboard-reconciled rows join `money_golden.json`. + +### Fixed + +- **Streamed Router calls now carry the served `service_tier`.** The stream wrapper forwarded usage and model only, so every streamed Router call reached price mode tier-less — which is a miss. +- **The catalog fixture's input rates were blanked by our own capture scrub.** The scrub treated any key named `input` as prompt content, including `router.pricing.input`, and the resulting empty column was cited as Router's data. Content keys are no longer blanked under `pricing`; the fixture is recaptured with all 68 rates. + ## [0.3.0] - 2026-09-02 ### Changed diff --git a/README.md b/README.md index 4ed0bb7..df6b0f8 100644 --- a/README.md +++ b/README.md @@ -181,7 +181,7 @@ Full guide, including cache semantics, dedup, attribution via `QUERY_TAG` and ac ### Ramp Router -An OpenAI-Responses-compatible gateway in front of OpenAI, Anthropic, Google Vertex, Fireworks and xAI. The model that answered is the one billed — Router can serve a different model than the one requested. +A gateway in front of OpenAI, Anthropic, Google Vertex, Fireworks and xAI, with an OpenAI-shaped surface and an Anthropic-shaped one. Point either client at it. The model that answered is the one billed — Router can serve a different model than the one requested. ```python client = sdk.wrap( @@ -191,7 +191,7 @@ client = sdk.wrap( client.responses.create(model=os.environ["RAMP_ROUTER_MODEL"], input="Summarize this invoice.") ``` -Full guide, including why price mode falls back to token events for Router traffic: [docs/ramp-router.md](docs/ramp-router.md). +Full guide, including how price mode bills Router's own catalog rate: [docs/ramp-router.md](docs/ramp-router.md). ## Multi-tenant — pick a subscription per call @@ -226,7 +226,7 @@ sdk = LagoSDK(api_key="...", config=LagoConfig( )) ``` -Price mode emits one `llm_cost` event per priced field (input, output, cache, ...), each carrying `precise_total_amount_cents` for Lago's **dynamic charge model** plus a `token_type` property so a single billable metric can be grouped by both `model` and `token_type`. Prices come from public sources (OpenRouter for native providers, the AWS Bedrock price list for Bedrock), fetched and cached in the background — your LLM call is never blocked on pricing. If a price isn't available yet, the SDK falls back to token-count events and reports via `on_error` rather than under-billing. Per-call override: `extra_lago={"mode": "price", "markup": 1.5}`. +Price mode emits one `llm_cost` event per priced field (input, output, cache, ...), each carrying `precise_total_amount_cents` for Lago's **dynamic charge model** plus a `token_type` property so a single billable metric can be grouped by both `model` and `token_type`. Prices come from public sources (OpenRouter for native providers, the AWS Bedrock price list for Bedrock) and from a gateway's own catalog where it publishes one (Cloudflare Workers AI, Ramp Router), fetched and cached in the background — your LLM call is never blocked on pricing. If a price isn't available yet, the SDK falls back to token-count events and reports via `on_error` rather than under-billing. Per-call override: `extra_lago={"mode": "price", "markup": 1.5}`. ## Error policy diff --git a/docs/ramp-router.md b/docs/ramp-router.md index 04ff1e8..7b0b2b7 100644 --- a/docs/ramp-router.md +++ b/docs/ramp-router.md @@ -1,6 +1,6 @@ # Ramp Router -[Ramp Router](https://router.com) is an OpenAI-Responses-compatible gateway in front of OpenAI, Anthropic, Google Vertex, Fireworks and xAI. Point an OpenAI client at it and `wrap()` detects it from the `base_url` — no other code change: +[Ramp Router](https://router.com) is an OpenAI-Responses-compatible gateway in front of OpenAI, Anthropic, Google Vertex, Fireworks and xAI. Point an OpenAI client at it and `wrap()` detects it from the `base_url` — no other code change. An Anthropic client pointed at Router is detected the same way; see [Two surfaces](#two-surfaces-responses-and-messages) for when to use which: ```python from openai import OpenAI @@ -22,7 +22,7 @@ sdk.flush() - **The model that answered is the one billed.** Router diverges from what you asked for in two ways: a `models` fallback list sends no `model` field at all, and Switchyard routing can serve a different model than the one requested. The SDK bills the model the response reports. - **Router answers with a resolved vendor snapshot, so nothing needs stripping.** Ask for `openai:gpt-5.4-nano` and the response says `gpt-5.4-nano-2026-03-17`; that bare name is what bills, so a Router-served model rolls up in Lago against the same row a direct call to it reports. (A `provider:model[:tier]` candidate is still split if one ever reaches the adapter unresolved, but no captured response carries that shape.) -- **The served service tier is recorded** in `usage.extras["service_tier"]`, read from the response's own `service_tier` field — `flex`, `default`, and so on. Note `extras` is diagnostic: it is **not** sent to Lago, so the tier is visible to an `on_error`/debug hook but does not reach your events or split a charge. Pricing a tier differently needs the Lago plan to do it. +- **The served service tier is recorded** in `usage.extras["service_tier"]`, read from the response's own `service_tier` field — `flex`, `default`, and so on. Note `extras` is diagnostic: it is **not** sent to Lago, so the tier is visible to an `on_error`/debug hook but does not reach your events or split a charge. In price mode a non-default tier is a reported miss rather than a multiplied rate — see below. - **Streaming bills once**, from the terminal usage event. `models` fallback, buffered and streamed calls all work unmodified — note the typed Python client rejects the non-standard `models` kwarg, so a fallback list goes through `extra_body={"models": [...]}` (verified live; the served model bills either way). Attribution works the same way as anywhere else in this SDK — `subscription` at wrap time or `extra_lago={"subscription": ...}` per call. Separately, it is worth putting the same id in Router's own `metadata` field, which Router stores with its usage record and shows in the request detail: @@ -38,21 +38,65 @@ client.responses.create( That costs nothing today and is what a backfill would key off later. -## Price mode falls back to token events +## Price mode bills Router's own catalog -In `pricing_mode="price"`, a Router call currently emits **token events rather than `llm_cost`**. That is deliberate, not a gap, and it is worth understanding before turning price mode on for Router traffic: +In `pricing_mode="price"`, a Router call is priced from **Router's own `GET /v1/models` catalog** — the rate Router bills, not OpenRouter's listing for the same model. Measured against a live account's dashboard export on 2026-09-04 and 2026-09-07: every default-tier call whose token counts the response fully reports reconciled at exactly the catalog rate, across all five served vendors (OpenAI, Anthropic, xAI, Fireworks, Baseten), including cache reads (xAI, 194 in / 192 cached: `2 × input + 192 × cache_read + output`, to the last digit) and OpenAI-served cache writes (`gpt-5.6-luna`, 4,490 written tokens at the `cache_write_input` rate). One model is the exception, noted below. -- **A BYOK-served request costs $0 through Router.** Router's own words: "When a request is served with your provider key, your provider bills you directly and Ramp Router does not charge for that usage." Nothing in the response says which key served it, so pricing at list rate would bill a customer the full amount for usage Router never charged for. -- **A non-default service tier does not bill at the published base rate.** Router's catalog says outright that "service tiers, long contexts, caching, and other features may use different rates", and that a Fast tier's "pricing may differ from the base rates shown here". Billing flex at the standard rate over-bills. -- **The token overlap semantics are Router's own, and they are measured.** OpenAI counts cached tokens inside `input_tokens` and reasoning inside `output_tokens`; Anthropic counts both additively. Router normalizes the _numbers_ to OpenAI's convention, not just the schema — verified live on an Anthropic-served model, where a warm `cache_control` call reported the cached block inside an unchanged `input_tokens`. But that convention is Router's, not the served vendor's, which is exactly why the served vendor must not be stamped as the provider: it would de-overlap with the wrong rules whenever its native convention differs. +- **No extra configuration.** The catalog is account-specific and needs your Router key, and `wrap()` learns it from the client you pass in. Set `LagoConfig.ramp_router_api_key` only to price Router usage without calling `wrap()`; an explicit value always wins over a learned one. +- **The served model is what is looked up.** Router answers with the vendor's own snapshot (`gpt-5.4-nano-2026-03-17`) or path (`accounts/fireworks/models/…`), and the lookup resolves each of those back to its catalog entry — by exact name, alias, or version-strip. Cost events carry `price_source: "ramp_router"`. +- **Only the default tier is priced.** A call Router served at `flex` (measured 0.5x the catalog rate) or `priority` (measured 2x) emits token events plus an `on_error` report that names the tier. The SDK applies no tier multiplier: the factors are Router's policy, published nowhere machine-readable. Pin the default tier in the request if every call must price (Anthropic models reject a pinned tier and serve the default unpinned). A response that reports **no** tier bills at the base rate: in a 237-call sweep Router omitted `service_tier` only on responses that stopped with zero output, and billed all of them at standard. +- **A BYOK-served request is billed at the catalog rate.** Router charges $0 for it and your vendor bills you directly; the response is byte-identical either way, so nothing in the SDK can tell. The catalog rate equals what the vendor charges, so the amount is right even though the payee differs. Router also falls back to its shared key when yours fails, and bills normally then. +- **Eight OpenAI models bill off Router's own catalog. The SDK bills the catalog anyway; correct it with `markup` if you want to match Router's dashboard.** Reconciled row by row against the dashboard export of a 237-call sweep on 2026-09-07, these models billed a constant multiple of their published rate on every field (input, output, cache read, cache write), at both served tiers, with no exceptions; every other model on every vendor billed exactly the catalog rate. -So Router is treated as a provider of its own, matching no vendor in the price tables. Token mode — the default — is exact and unaffected: it emits the counts Router reported, the same per-field `llm_*` events a direct provider call produces. Price mode takes a clean miss and falls back to those same token events, with no error on your call path. Bill Router traffic in token mode and price it with a Lago plan. + | model | Router billed, as a multiple of its own catalog rate | markup that matched the dashboard on 2026-09-07 | + |---|---|---| + | `gpt-5.4-mini`, `gpt-5.4-nano`, `gpt-5.5`, `gpt-5.5-pro`, `gpt-5.6-luna`, `gpt-5.6-terra`, `gpt-6-astra` | 1.1x | `1.1` | + | `gpt-5.6-sol` | 0.55x: a **50% off** promotion shown on Router's model page ($2.00 / $10.00 against the catalog's $4 / $20), times the same 1.1 | `0.55` while the promotion lasts | + + The SDK does not apply these factors, on purpose. Sol's is a time-limited promotion and the 1.1 looks like a catalog that lags Router's newest models; either can change on a day of Router's choosing, and a factor baked into the SDK would become the thing that is wrong, for every customer, until a release ships. A markup is yours: apply it per call with `extra_lago={"markup": 1.1}` on those models (or globally with `LagoConfig.markup` if your traffic is all one model), reconcile against Router's dashboard from time to time, and drop it when the catalog catches up. Router's documentation names no fee. Treat the table as a measurement with a date, not a rate card. +- **A catalog entry served through a different backend is a miss, not a price.** Ten Fireworks-owned entries also carry a Baseten alias (`deepseek-ai/…`, `zai-org/…`, `moonshotai/…`, `nvidia/…`, `openai/gpt-oss-120b`). When Router serves one through Baseten it bills Baseten's rate, which the catalog does not publish (measured 1.11x to 2.4x away from the Fireworks rate), and the response names the model by that alias. The SDK does not index those aliases, so such a call emits token events plus an `on_error` report rather than a wrong price. The cost is that Baseten-served calls whose rate happened to match also miss. +- **Anthropic-served cache writes bill at the input rate.** `/v1/responses` reports no write count for Anthropic models — a cold `cache_control` write is byte-identical to an uncached call of the same size — so the written prefix is priced as `input` rather than at the `cache_write_input_5m`/`_1h` rate. Measured shortfall on such a call: up to 20% with the 5-minute TTL and up to 50% with the 1-hour TTL, in proportion to how much of the prompt was the cached prefix. Reads are exact. Router's `/v1/messages` surface does report the write count, and an Anthropic client pointed at Router bills it exactly — see [Two surfaces](#two-surfaces-responses-and-messages). + +Any other miss — the table still cold on the very first call, a model the catalog does not list, no key learned — falls back to token events and reports through `on_error`, like every other provider. Token mode, the default, is unaffected: it emits the counts Router reported. + +## Two surfaces: Responses and Messages + +Router exposes every model in your catalog on two API surfaces. The SDK detects Router on both from the client's base URL, prices both from the same catalog with the same key, and stamps them differently because they report the same vendor under different conventions. + +| | `POST /v1/responses` | `POST /v1/messages` | +|---|---|---| +| Client | `OpenAI(base_url="https://api.router.com/v1")` | `Anthropic(base_url="https://api.router.com")` | +| Schema | OpenAI Responses, for every vendor | Anthropic Messages, for every vendor | +| Token convention (measured) | cached tokens **inside** `input_tokens` | cached tokens **beside** `input_tokens` — Anthropic's additive convention, whichever vendor served | +| Anthropic cache **write** reported | no | yes: `cache_creation_input_tokens`, with the 5-minute / 1-hour split | +| Price mode | catalog rate, default tier only | same, and each cache write billed at `cache_write_input_5m` or `_1h` | +| `service_tier` | top level of the response | inside `usage`, buffered and streamed | +| Events stamped | `provider=ramp_router api=ramp_router` | `provider=ramp_router api=ramp_router_messages` | + +```python +from anthropic import Anthropic + +client = sdk.wrap( + Anthropic(api_key=os.environ["RAMP_ROUTER_API_KEY"], base_url="https://api.router.com"), + subscription="sub_acme", +) +client.messages.create( + model=os.environ["RAMP_ROUTER_MODEL"], + max_tokens=256, + system=[{"type": "text", "text": LONG_SYSTEM_PROMPT, "cache_control": {"type": "ephemeral"}}], + messages=[{"role": "user", "content": "Summarize this invoice."}], +) +``` + +**Which to use.** If your Anthropic-model traffic uses prompt caching, reach Router with an Anthropic client. It is the only surface on which a cold cache write is priced exactly — reconciled to the cent against Router's dashboard on 2026-09-04: 16 input + 20,113 written tokens at the 5-minute rate + 5 output on `claude-haiku-4-5` = $0.02518225, and the warm repeat $0.0020523. On the Responses surface the same write is invisible and bills at the input rate (see above). Everything else bills identically on both surfaces: reads, plain calls, OpenAI-served cache writes, and the default-tier rule. + +Both surfaces were measured with the same models on the same account. `/v1/messages` accepts non-Anthropic models too (an OpenAI and an xAI model were captured), and renders their usage in Anthropic's shape with thinking tokens inside `output_tokens`; the adapter treats them exactly like a native Anthropic response. The two `api` stamps exist so the token-overlap rules can never be applied to the wrong surface: the Responses stamp subtracts cached tokens from `input`, the Messages stamp does not, and the SDK pins that distinction in tests. ## Measured behaviours and limitations - **There is no backfill path**, because Router exposes no programmatic usage surface. Its only routes are `GET /v1/models`, `POST /v1/responses`, `POST /v1/messages` and `POST /v1/messages/count_tokens`; usage lives in the dashboard's Logs view. An "analytics API" is mentioned once in Router's limits table with no path, auth or record shape. Unlike the [Cloudflare connector](cloudflare.md), there is no Logs API loop to show you. - **Nothing is skipped as a gateway cache hit**, because Router has no response cache: "Self-service Router response caching, which would reuse an entire previous response without calling a model provider, is a separate optimization and is not currently configurable." Provider _prompt_ caching does pass through, and those cache-read and cache-write tokens are billed like any other. -- **Do not point a wrapped `Anthropic` client at Router.** `POST /v1/messages` exists and routes to the same providers, and the call will _work_ — but the Anthropic wrapper has no `base_url` detection, so the SDK bills it as though it came from Anthropic directly. Events are stamped `provider=anthropic api=native`, priced at Anthropic's list rate even though Router may have served the call BYOK at $0, and de-overlapped with Anthropic's additive cache convention rather than Router's inside-`input` one. That last one double-counts every cached token if Router normalizes `/v1/messages` the way it demonstrably does `/v1/responses` (fixture `06b`: the cached block sits inside an unchanged `input_tokens`) — a warm 18,825-token call bills 37,650. Not yet measured on `/v1/messages`, which is the reason to stay off it. Use the Responses surface until the Anthropic wrapper learns the host. +- **Two deprecated OpenAI models never price.** Router serves `gpt-3.5-turbo` and `gpt-4` as `gpt-3.5-turbo-0125` and `gpt-4-0613`, a 4-digit month-day snapshot suffix the SDK's version strip does not remove, so they miss the catalog and emit token events. A blind 4-digit strip would be wrong: `deepseek-v4-pro-0813` is its own catalog entry, and stripping it lands on a different model's rate. - **A proxy in front of Router is not detected.** Detection matches the `api.router.com` host (and `*.router.com`). Reaching Router through your own hostname bills as plain OpenAI, with the wrong provider and an unparsed model id. - **`api.router.com` sits behind bot management.** A rejected client can get an HTML challenge page rather than Router's documented JSON error envelope. The SDK degrades to zero usage rather than throwing, either way. - **Failures never bill.** Every documented status — including 402 `insufficient_credits`, 429, and 502 `all_candidates_failed` — emits nothing, as does a response reporting zero usage. diff --git a/src/lago_agent_sdk/adapters/anthropic_native.py b/src/lago_agent_sdk/adapters/anthropic_native.py index b0b0e57..38c7c38 100644 --- a/src/lago_agent_sdk/adapters/anthropic_native.py +++ b/src/lago_agent_sdk/adapters/anthropic_native.py @@ -13,6 +13,10 @@ reasoning_tokens — even with extended thinking enabled Unknown usage fields (service_tier, inference_geo, server_tool_use, …) land in extras. + +Ramp Router's `/v1/messages` surface answers in this exact shape for EVERY vendor it +fronts, so the same extractor serves it — with a `provider_hint` from the wrapper, since +nothing in the body says Router was in the path (see RAMP_ROUTER_MESSAGES_API). """ from __future__ import annotations @@ -20,6 +24,26 @@ from typing import Any, cast from ..canonical import CanonicalUsage +from .openai_native import RAMP_ROUTER_PROVIDER + +#: `api` stamped on a Router call that arrived through `/v1/messages`. +#: +#: Distinct from the Responses surface's stamp ("ramp_router", which sits in +#: OPENAI_SHAPED_APIS) because the two surfaces report the SAME vendor's numbers under +#: DIFFERENT conventions. Measured 2026-09-04 and 2026-09-07 against a live account: +#: `/v1/messages` keeps Anthropic's additive shape for every vendor — haiku reports +#: `input_tokens: 16` beside `cache_read_input_tokens: 20113`; an xAI model reports +#: `input_tokens: 65` beside `cache_read_input_tokens: 128` and `thinking_tokens: 200` +#: INSIDE `output_tokens: 201` — while `/v1/responses` folds the cached block inside +#: `input_tokens`. Token semantics key on the surface, so this stamp must stay OUT of +#: OPENAI_SHAPED_APIS: the provider-keyed sets do not name "ramp_router", which leaves +#: the all-additive default, the measured answer here. Putting the Responses stamp on +#: this surface would subtract a cached block that was never inside `input`. +#: +#: The write count Router's Responses surface cannot report for Anthropic models IS +#: reported here (`cache_creation_input_tokens`, with the 5m/1h split), and reconciled +#: exactly against the dashboard — the reason this surface is worth detecting at all. +RAMP_ROUTER_MESSAGES_API = "ramp_router_messages" _KNOWN_USAGE_FIELDS = { "input_tokens", @@ -67,13 +91,24 @@ def _resolve_model(response_model: Any, requested_model: str) -> str: return requested_model or "" -def extract_anthropic_native(response: Any, model_id: str = "") -> CanonicalUsage: +def extract_anthropic_native(response: Any, model_id: str = "", provider_hint: str = "") -> CanonicalUsage: """Translate an Anthropic native response (Message or dict) → CanonicalUsage. Accepts the SDK's pydantic Message object, a dict (e.g. captured fixture), or a synthetic `{"usage": {...}}` blob produced by the streaming wrapper. + + `provider_hint` is the wrapper's word that the client was pointed at a gateway; only + the wrapper can know, because the body never says. Today the one value it takes is + RAMP_ROUTER_PROVIDER, which stamps the call as Router traffic on the Messages + surface. The served tier needs no special handling: Router puts `service_tier` + INSIDE `usage` on this surface (buffered, and on both `message_start` and + `message_delta` when streamed — measured), so the drift sweep below already lands it + in `extras["service_tier"]`, where the price-mode tier gate reads it. """ resp = _to_dict(response) if not isinstance(response, dict) else response + provider, api = "anthropic", "native" + if provider_hint == RAMP_ROUTER_PROVIDER: + provider, api = RAMP_ROUTER_PROVIDER, RAMP_ROUTER_MESSAGES_API usage = _safe_dict(resp.get("usage")) cache_creation = _safe_dict(usage.get("cache_creation")) @@ -99,7 +134,7 @@ def extract_anthropic_native(response: Any, model_id: str = "") -> CanonicalUsag cache_write_1h=_safe_int(cache_creation.get("ephemeral_1h_input_tokens")), tool_calls=tool_calls, model=_resolve_model(resp.get("model"), model_id), - provider="anthropic", - api="native", + provider=provider, + api=api, extras=extras, ) diff --git a/src/lago_agent_sdk/adapters/openai_native.py b/src/lago_agent_sdk/adapters/openai_native.py index b03d20b..e6817a0 100644 --- a/src/lago_agent_sdk/adapters/openai_native.py +++ b/src/lago_agent_sdk/adapters/openai_native.py @@ -64,12 +64,14 @@ #: `provider:provider-model` candidate, nothing in the response says who served it, and #: the model-string rule `_infer_provider` uses cannot see it. #: -#: "ramp_router" is in `TOKEN_BILLED_PROVIDERS` and deliberately absent from -#: `_VENDOR_MAP`. Two distinct things would otherwise go wrong at once: +#: "ramp_router" is deliberately absent from `_VENDOR_MAP`; it prices against Router's +#: OWN catalog instead (see the Ramp Router section of pricing.py). Two distinct things +#: would otherwise go wrong at once: #: -#: * A price lookup under a guessed vendor can be flatly wrong. Router bills at list -#: price on its shared key but $0 for a BYOK-served request, and a non-default -#: service tier bills at a rate its own catalog says "may differ" from the base one. +#: * A price lookup under a guessed vendor can be flatly wrong. Router's catalog rate +#: is the billed rate (measured exact against the dashboard across five vendors); +#: OpenRouter's listing for the "same" model is a different company's price, and +#: Router serves models literally named `claude-haiku-4-5` and `o4-mini`. #: * The overlap semantics belong to ROUTER, not to the served vendor. Measured #: 2026-08-28 on an Anthropic-served model — the case that would diverge if anything #: did: Router normalizes the NUMBERS to OpenAI's convention, not just the schema @@ -79,13 +81,13 @@ #: one differs — "ramp_router" carries its own OPENAI_SHAPED_APIS entry instead. #: #: Token mode is unaffected and exact either way: it emits the counts Router reported. -#: Price mode routes to those same token events via `TOKEN_BILLED_PROVIDERS`, with no -#: per-call price-miss report — a structural, permanent miss must not cry wolf on the -#: error hook (the same decision Databricks and Snowflake got). The catalog DOES publish -#: per-model rates (`router.pricing`, 01_real_models_catalog.json), so a Router price -#: mode is buildable — but the response still cannot say whether a BYOK key served the -#: call ($0) or which tier rate applied, and every observed catalog entry carries an -#: EMPTY input rate, so token counts stay the honest default. +#: Price mode bills the catalog rate for a call served at the default tier; any other +#: served tier is a reported miss (see `ramp_router_unpriced_tier`). Two things the +#: response cannot say are decided, not guessed: a BYOK-served call ($0 through Router) +#: is billed at the catalog rate regardless — the response is byte-identical either way +#: and the rate equals what the vendor bills directly — and no factor is ever applied on top +#: of the published rate, even for the models measured to bill off it (documented, with +#: a `markup` recommendation, in docs/ramp-router.md). RAMP_ROUTER_PROVIDER = "ramp_router" # Router's documented service tiers, appearing as the third segment of a REQUESTED @@ -351,7 +353,24 @@ def extract_openai_native(response: Any, model_id: str = "", provider_hint: str # sends no `model` at all, and Switchyard routing can serve a different model than # the one asked for, so the response is the only place the SERVED model appears. model = resolved_model + # The cache-write count that reaches CanonicalUsage. Zero for every surface but + # Router — see _MAPPED_DETAIL_FIELDS for why OpenAI-native keeps it out. + mapped_cache_write = 0 if provider_hint == RAMP_ROUTER_PROVIDER: + # Router bills an OpenAI-served cache WRITE at its catalog's `cache_write_input` + # rate — measured 2026-09-07 against the dashboard: gpt-5.6-luna, input_tokens + # 4493 with input_tokens_details.cache_write_tokens 4490, charged as 3 x input + + # 4490 x cache_write + out, exactly, at both served tiers. So on THIS surface the + # count is mapped, where for OpenAI-native it deliberately is not (Databricks + # metered the same field at the plain input rate — see _MAPPED_DETAIL_FIELDS). + # Same wire shape, two measured billing conventions; the hint is what tells + # them apart. The count stays INSIDE `input` (OPENAI_SHAPED_APIS), so + # compute_cost moves it out before pricing and the total_tokens guard leaves it + # alone. Anthropic-served Router calls never report the key — their cold write + # is invisible here and bills at the input rate, a documented limitation. + mapped_cache_write = cache_write + extras.pop("input_tokens_details.cache_write_tokens", None) + extras.pop("prompt_tokens_details.cache_write_tokens", None) router_provider, parsed_model, tier = _parse_router_model(resolved_model) if router_provider: model = parsed_model @@ -466,6 +485,7 @@ def extract_openai_native(response: Any, model_id: str = "", provider_hint: str input=input_tokens, output=output_tokens, cache_read=cache_read, + cache_write=mapped_cache_write, reasoning=reasoning, audio_input=audio_input, audio_output=audio_output, diff --git a/src/lago_agent_sdk/config.py b/src/lago_agent_sdk/config.py index 26a66c1..3dc063d 100644 --- a/src/lago_agent_sdk/config.py +++ b/src/lago_agent_sdk/config.py @@ -80,6 +80,12 @@ class LagoConfig: # pricing Mistral usage without ever calling wrap() (e.g. a log-backfill # path); an explicit value here always wins over an auto-detected one. mistral_api_key: str | None = field(default=None, repr=False) + # Usually NOT needed either — wrap()-ing an OpenAI client pointed at Ramp Router + # learns the key that client already carries, and Router's catalog (the price + # source for "ramp_router") is account-scoped, so no other key would do. Set it + # explicitly only when pricing Router usage without ever calling wrap(); an + # explicit value here always wins over a learned one. + ramp_router_api_key: str | None = field(default=None, repr=False) # Optional injected PricingProvider (or a stub) — primarily for tests/overrides. # Typed Any to avoid a config→pricing import cycle. pricing_provider: Any | None = field(default=None, repr=False) diff --git a/src/lago_agent_sdk/exceptions.py b/src/lago_agent_sdk/exceptions.py index 1a2004d..f3a55aa 100644 --- a/src/lago_agent_sdk/exceptions.py +++ b/src/lago_agent_sdk/exceptions.py @@ -28,8 +28,15 @@ class PricingUnavailableError(LagoSDKError): """Price mode could not resolve a price (table not warm yet, or model not matched). Surfaced via on_error; the SDK falls back to emitting token events.""" - def __init__(self, provider: str, model: str, api: str) -> None: - super().__init__(f"no price for provider={provider!r} model={model!r} api={api!r}") + def __init__(self, provider: str, model: str, api: str, detail: str | None = None) -> None: + # `detail` is for the miss that is NOT "table cold / model unknown" — a Ramp + # Router call served at a non-default tier is unpriced by decision, and the + # customer needs to read that off the error rather than chase a model name. + message = f"no price for provider={provider!r} model={model!r} api={api!r}" + if detail: + message = f"{message}: {detail}" + super().__init__(message) self.provider = provider self.model = model self.api = api + self.detail = detail diff --git a/src/lago_agent_sdk/pricing.py b/src/lago_agent_sdk/pricing.py index c4bc022..cf7ecc7 100644 --- a/src/lago_agent_sdk/pricing.py +++ b/src/lago_agent_sdk/pricing.py @@ -25,6 +25,12 @@ *does* list the resolved id (e.g. ``mistralai/mistral-small-2603``) with real pricing. ``/v1/models`` exposes the resolution directly via each model's ``aliases`` array; needs the customer's own Mistral API key. + - Ramp Router's own ``GET /v1/models`` for ``ramp_router`` — like Cloudflare's + catalog, the rate the gateway actually bills at (measured exact against a live + account's dashboard export across five served vendors), and like Cloudflare's + it is account-scoped and needs the customer's Router key. The key is learned + from the wrapped client at ``wrap()`` time, or set via + ``LagoConfig.ramp_router_api_key``; without either the source is simply empty. Design constraints (mirror the queue's non-blocking guarantee): - ``lookup()`` is pure in-memory and O(1); it NEVER does network I/O, so the @@ -66,6 +72,7 @@ AWS_BEDROCK_REGION_INDEX = f"{AWS_PRICING_HOST}/offers/v1.0/aws/AmazonBedrock/current/region_index.json" CLOUDFLARE_MODELS_URL_TEMPLATE = "https://api.cloudflare.com/client/v4/accounts/{account_id}/ai/models/search" MISTRAL_MODELS_URL = "https://api.mistral.ai/v1/models" +RAMP_ROUTER_MODELS_URL = "https://api.router.com/v1/models" # Canonical usage fields we know how to price. PRICED_FIELDS = ("input", "output", "cache_read", "cache_write", "reasoning") @@ -107,7 +114,12 @@ # a real vendor prefix would let a near-miss model string match Anthropic's or # OpenAI's own OpenRouter rate — a silent mispricing of a call Snowflake charged in # credits. The absence is the guard; do not "fix" it. -TOKEN_BILLED_PROVIDERS = frozenset({"databricks", "snowflake", "ramp_router"}) +# +# "ramp_router" was here until its catalog became a price source (see the Ramp Router +# section below). Its miss is no longer structural: a Router call that cannot be priced +# now reports through on_error like any other provider's, because the customer CAN act +# on it — a missing Router key, a cold table, or a non-default service tier. +TOKEN_BILLED_PROVIDERS = frozenset({"databricks", "snowflake"}) # Canonical field -> OpenRouter pricing key. _OPENROUTER_FIELD_MAP = { @@ -140,6 +152,36 @@ "per M cached input tokens": "cache_read", } +# Ramp Router's `router.pricing` key -> canonical field. Every one of the live catalog's +# entries carries all six keys as STRINGS in USD per 1M tokens (measured 2026-09-07, 68 +# of 68). `cache_write_input_5m` / `_1h` are Anthropic's TTL-split write rates; the +# count they price is reported only on Router's `/v1/messages` surface (the Anthropic +# wrapper), never on `/v1/responses` — see adapters/anthropic_native.py. +_RAMP_ROUTER_FIELD_MAP = { + "input": "input", + "output": "output", + "cache_read": "cache_read_input", + "cache_write": "cache_write_input", + "cache_write_5m": "cache_write_input_5m", + "cache_write_1h": "cache_write_input_1h", +} + +# Served service tiers that bill at the catalog's published rate. Any OTHER reported +# tier — `flex` (measured 0.5x), `priority` (measured 2.0x on two vendors), or a tier +# Router adds later — is a price MISS: token events plus an on_error report, never a +# multiplied rate. The tier multipliers are Router's policy, published nowhere +# machine-readable. `standard` is the dashboard's spelling of the tier the API reports +# as `default`; accepted so a vocabulary change on the wire stays a base-rate call. +# +# A response with NO tier at all is priced at the base rate (decided 2026-09-07 on +# data): in a 237-call sweep Router omitted `service_tier` on exactly the responses +# that stopped with zero output (`incomplete`, both surfaces, six calls) and billed +# every one of them at the standard rate; flex and priority were reported explicitly +# whenever they applied. So absence has only ever meant standard, and treating it as +# a miss turned $0.50 of real usage into token events for no gain. +RAMP_ROUTER_BASE_RATE_TIERS = frozenset({"default", "standard"}) + + # Cloudflare's catalog page size, and a hard bound on the paging loop. The loop runs # on the queue's flush tick ahead of the drain, so it must terminate even if the # endpoint keeps returning full pages. 40 pages covers ~2000 models against a real @@ -311,6 +353,14 @@ class ModelPrice: cache_read: Decimal | None = None cache_write: Decimal | None = None reasoning: Decimal | None = None + # Anthropic prices a cache write by its TTL: 1.25x input for the 5-minute cache, 2x + # for the 1-hour one. Only a source that publishes both can fill these (Ramp Router's + # catalog does; OpenRouter publishes one `input_cache_write`, the 5m rate, so native + # Anthropic bills every write at it). When they are set AND the usage carries the + # matching `cache_write_5m`/`cache_write_1h` split, `compute_cost` bills each part at + # its own rate instead of the lump `cache_write` rate — see `_split_cache_write`. + cache_write_5m: Decimal | None = None + cache_write_1h: Decimal | None = None def get(self, field_name: str) -> Decimal | None: return getattr(self, field_name, None) @@ -365,15 +415,12 @@ def compute_cost(usage: CanonicalUsage, price: ModelPrice, markup: Decimal) -> C counts["input"] = max(0, counts["input"] - counts["cache_read"]) if inc_cache_write and price.get("cache_write") is not None: counts["input"] = max(0, counts["input"] - counts["cache_write"]) + split = _split_cache_write(usage, price, counts) base = Decimal(0) fields: dict[str, dict[str, str]] = {} - for f in PRICED_FIELDS: - count = counts[f] - if not count: - continue - unit = price.get(f) - if unit is None: + for f, count, unit in [(f, counts[f], price.get(f)) for f in PRICED_FIELDS] + split: + if not count or unit is None: continue cost = unit * count base += cost @@ -385,6 +432,46 @@ def compute_cost(usage: CanonicalUsage, price: ModelPrice, markup: Decimal) -> C return _finalize_breakdown(base, markup, price.source, fields) +_CACHE_WRITE_TTL_FIELDS = ("cache_write_5m", "cache_write_1h") + + +def _split_cache_write( + usage: Any, price: ModelPrice, counts: dict[str, int] +) -> list[tuple[str, int, Decimal | None]]: + """Move the TTL-split part of `cache_write` onto its own rates, when both sides carry + the split. + + `cache_write_5m` / `cache_write_1h` are a breakdown OF `cache_write`, not additions to + it (Anthropic: `cache_creation_input_tokens == ephemeral_5m + ephemeral_1h`, measured + on every capture). So each part priced here is REMOVED from the lump count, and only + a remainder — a surface reporting a lump with no split — still bills at the lump + rate. Engages only when the price publishes a rate for that TTL: on OpenRouter's + single-rate Anthropic listing nothing moves and the lump path is unchanged. + + The 1h rate is 2x input where the 5m rate is 1.25x; billing a 1h write at the 5m + rate under-bills it by 37.5%, which is what this exists to prevent on the one + source (Ramp Router) that publishes both and the one surface (`/v1/messages`) that + reports the split. Reconciled exactly against Router's dashboard on 2026-09-04: + 20,113 tokens at the 5m rate + 16 input + 5 output = $0.02518225. + + Mutates `counts["cache_write"]`; returns (field, count, unit) triples to price. + """ + split: list[tuple[str, int, Decimal | None]] = [] + for f in _CACHE_WRITE_TTL_FIELDS: + unit = price.get(f) + n = int(getattr(usage, f, 0) or 0) + if unit is None or n <= 0: + continue + # Never bill more split tokens than the lump reports: a surface whose split + # exceeds its total is misreporting, and the lump is the authoritative count. + n = min(n, counts["cache_write"]) + if n <= 0: + continue + counts["cache_write"] -= n + split.append((f, n, unit)) + return split + + def _finalize_breakdown( base: Decimal, markup: Decimal, source: str, fields: dict[str, dict[str, str]] ) -> CostBreakdown: @@ -759,6 +846,189 @@ def lookup_cloudflare_workers_ai(table: dict[str, ModelPrice], model: str) -> Mo return None +# ---------------------------------------------------------------------- +# Ramp Router parsing + matching +# +# Router's own `GET /v1/models` is the price source for the same reason Cloudflare's +# catalog is Workers AI's: it is the rate the gateway actually bills, not a third +# party's listing for the same model hosted elsewhere. Measured against a live +# account's dashboard export: every default-tier row whose counts the response fully +# reports reconciled at exactly 1.000000x the catalog rate — 28 rows across five +# served vendors on 2026-09-04, including the cache split (grok, 194 in / 192 cached: +# 2 x input + 192 x cache_read + out, to the last digit), and an OpenAI cache WRITE +# billed at `cache_write_input` on 2026-09-07 (gpt-5.6-luna, 4493 in / 4490 written). +# The five default-tier rows that did NOT reconcile were Anthropic cold cache writes, +# whose write count this surface never reports — see the adapter. +# +# Where Router bills OFF its own catalog (measured 2026-09-07: eight OpenAI models at a +# constant 1.1x or 0.55x of their published rate), the SDK still bills the PUBLISHED rate +# and documents the mismatch with its date, recommending `markup` on those models. A +# factor baked into the SDK would be the thing out of sync the day Router corrects its +# catalog — a customer's markup can be dropped the same day, an SDK release cannot. +# Where Router serves an entry through a backend other than the one the rate belongs +# to, the served name is refused rather than mispriced — see `_is_foreign_backend_alias`. +# ---------------------------------------------------------------------- +def _is_foreign_backend_alias(alias: str, provider_model: Any) -> bool: + """True when an alias names the SAME model on a DIFFERENT backend than the entry's own. + + Router serves some catalog entries through more than one hosting provider and bills + the rate of whichever served — but publishes ONE rate per entry, the entry's own + provider's. Measured 2026-09-07: ten Fireworks-owned entries carry a Baseten alias + (`deepseek-ai/DeepSeek-V4-Flash-0731`, `zai-org/GLM-5.2`, `moonshotai/Kimi-K2.7-Code`, + …); when Baseten served, Router billed Baseten's rate, 1.11x to 2.4x away from the + catalog's. The served model name is that alias, so it is the one signal that the + published rate does not apply — and a name the SDK refuses to index is an honest + miss (token events + on_error) instead of a wrong price. Decided by the user, + 2026-09-07, knowing it also turns the Baseten-served rows that happened to match + (kimi-k3, glm-5p3-flash, deepseek-v4-pro) into misses. + + "Different backend" is read off the path prefix: `provider_model` says where the + entry's rate comes from (`accounts/fireworks/models/…`), and an alias whose leading + path segment differs (`deepseek-ai/…`) is another host's spelling. A bare alias with + no path (`zai-org/GLM-5.3-Flash` has one; `gpt-5-chat-latest` would not) is a plain + synonym and stays indexed. + """ + if "/" not in alias or not isinstance(provider_model, str) or "/" not in provider_model: + return False + return alias.split("/", 1)[0] != provider_model.split("/", 1)[0] + + +def parse_ramp_router(data: Any) -> dict[str, ModelPrice]: + """Parse Router's `/v1/models` into {name: ModelPrice}, keyed on every name a + served response can report for the entry. + + Router answers with a RESOLVED vendor snapshot, not the catalog id: `gpt-5.4-nano` + in the catalog, `gpt-5.4-nano-2026-03-17` in the response — `lookup_ramp_router` + strips that. But Fireworks- and Baseten-served responses report the vendor's own + path (`accounts/fireworks/models/…`, `thinkingmachines/inkling-small`), which is the + entry's `router.provider_model` or one of its `router.aliases`, never its `id`. So + every one of `id`, `router.request_name`, `router.provider_model` and + `router.aliases[]` is indexed (measured: all 9 distinct served names across every + capture resolve, 5 by version-strip and 4 by exact name). + + Two rules keep that widening honest: + + * A name claimed by two entries with DIFFERENT rates is unpriced — removed and + pinned so no later entry can re-add it. Guessing between two rates is a + mispricing, not a miss. The live catalog has exactly one shared name today + (`…/nemotron-3-ultra-nvfp4`, the provider_model of two entries) and both + carry identical rates, so it prices; the rule is for the day they diverge. + * A ZERO cache rate means "no separate rate", not "free": `cache_write_input` + is "0" on every Anthropic entry because their write price lives in the + `_5m`/`_1h` keys, and `cache_read_input` is "0" on the pro and legacy OpenAI + models that do not cache at all. Stored as None so `compute_cost` leaves those + tokens inside `input` at the input rate — the floor — rather than billing a + cached block at $0. Zero `input`/`output` is kept as a genuine published zero. + + One more rule, measured against the dashboard on 2026-09-07: an alias that names the + entry on a DIFFERENT backend is NOT indexed — see `_is_foreign_backend_alias`. A call + served there misses rather than misprices. The published rate is otherwise stored + as-is, even for the models measured to bill off it (see the section comment). + + An entry with no token rate at all is simply absent, the same safe miss as + everywhere else. + """ + table: dict[str, ModelPrice] = {} + conflicts: set[str] = set() + models = data.get("data") if isinstance(data, dict) else None + if not isinstance(models, list): + return table + for m in models: + if not isinstance(m, dict): + continue + mid = m.get("id") + router = m.get("router") + pricing = router.get("pricing") if isinstance(router, dict) else None + if ( + not isinstance(mid, str) + or not mid + or not isinstance(router, dict) + or not isinstance(pricing, dict) + ): + continue + fields: dict[str, Decimal] = {} + for field, key in _RAMP_ROUTER_FIELD_MAP.items(): + per_million = _parse_price(pricing.get(key)) + if per_million is None: + continue + if per_million == 0 and field.startswith("cache_"): + continue + fields[field] = (per_million / Decimal(1_000_000)).quantize(_Q, rounding=ROUND_DOWN) + if not fields: + continue + mp = ModelPrice(source="ramp_router", **fields) + provider_model = router.get("provider_model") + names = {mid} + for key in ("request_name", "provider_model"): + v = router.get(key) + if isinstance(v, str) and v: + names.add(v) + aliases = router.get("aliases") + if isinstance(aliases, list): + names.update( + a + for a in aliases + if isinstance(a, str) and a and not _is_foreign_backend_alias(a, provider_model) + ) + for name in names: + if name in conflicts: + continue + prior = table.get(name) + if prior is None: + table[name] = mp + elif prior != mp: + del table[name] + conflicts.add(name) + if conflicts: + # Once per fetch, not per call: a customer can act on it (the name is unpriced + # until Router's catalog stops disagreeing with itself), so it must be visible. + logger.warning( + "lago: ramp router catalog lists %d name(s) under more than one rate; left unpriced: %s", + len(conflicts), + sorted(conflicts), + ) + return table + + +def lookup_ramp_router(table: dict[str, ModelPrice], model: str) -> ModelPrice | None: + """Exact served name first, then the version-stripped form. + + The strip is the same `_strip_version` the OpenRouter path uses, because Router + reports the vendor's own dated snapshot for OpenAI- and Anthropic-served calls + (`o3-2025-04-16`, `claude-haiku-4-5-20251001`) while its catalog lists the bare id. + Verified collision-free against the live catalog: no stripped served name lands on + a different entry than the exact one would. + """ + hit = table.get(model) + if hit is not None: + return hit + return table.get(_strip_version(model)) + + +def ramp_router_unpriced_tier(usage: Any) -> str | None: + """The served tier that keeps a Router call OUT of price mode, or None. + + None means "bill the catalog rate": either this is not a Router call at all, or + Router served it at a base-rate tier (see RAMP_ROUTER_BASE_RATE_TIERS). Otherwise + the offending tier is returned so the miss report can say WHY — a customer seeing + "no price" for a model that priced a second ago needs to know it was the tier. + The tier is read from `extras["service_tier"]`, where the adapter records the + response's own field (top level on `/v1/responses`, inside `usage` on + `/v1/messages`). A Router call with NO tier bills at the base rate — see + RAMP_ROUTER_BASE_RATE_TIERS for the measurement behind that. Only an explicitly + reported non-base tier is a miss. + """ + if (getattr(usage, "provider", "") or "").lower() != "ramp_router": + return None + extras = getattr(usage, "extras", None) or {} + tier = extras.get("service_tier") + if tier is None or tier == "": + return None + if isinstance(tier, str) and tier.lower() in RAMP_ROUTER_BASE_RATE_TIERS: + return None + return tier if isinstance(tier, str) else repr(tier) + + # ---------------------------------------------------------------------- # Bedrock parsing + matching # @@ -899,6 +1169,7 @@ def fetch_openrouter(self) -> dict[str, Any]: ... def fetch_bedrock(self, region: str) -> dict[str, ModelPrice]: ... def fetch_cloudflare_workers_ai(self) -> dict[str, ModelPrice]: ... def fetch_mistral_aliases(self, api_key: str | None = None) -> dict[str, str]: ... + def fetch_ramp_router(self, api_key: str | None = None) -> dict[str, ModelPrice]: ... class HttpPricingFetcher: @@ -915,6 +1186,12 @@ class HttpPricingFetcher: customer's own key. Without it, ``fetch_mistral_aliases`` returns an empty map, so alias resolution is simply skipped and lookups fall back to whatever the request already spelled out (safe miss, not a break). + + ``ramp_router_api_key``: Router's catalog is account-scoped too. Without it (and + without one learned from a wrapped client), ``fetch_ramp_router`` returns an + empty table, so Router pricing is unavailable and every Router call in price + mode reports a miss and bills token events — loudly, because unlike the two + above this is a source the customer almost always has the key for. """ def __init__( @@ -923,11 +1200,13 @@ def __init__( cloudflare_account_id: str | None = None, cloudflare_api_token: str | None = None, mistral_api_key: str | None = None, + ramp_router_api_key: str | None = None, ) -> None: self._timeout = timeout self._cf_account_id = cloudflare_account_id self._cf_api_token = cloudflare_api_token self._mistral_api_key = mistral_api_key + self._ramp_router_api_key = ramp_router_api_key def fetch_openrouter(self) -> dict[str, Any]: import requests @@ -1006,6 +1285,23 @@ def fetch_mistral_aliases(self, api_key: str | None = None) -> dict[str, str]: resp.raise_for_status() return parse_mistral_aliases(resp.json()) + def fetch_ramp_router(self, api_key: str | None = None) -> dict[str, ModelPrice]: + import requests + + # Same precedence as Mistral: an explicitly configured key always wins over + # one learned from a wrapped client. + key = self._ramp_router_api_key or api_key + if not key: + return {} + # `api.router.com` sits behind Cloudflare bot management, which rejects + # urllib's default User-Agent outright (403). `requests`' own default passes + # (measured 2026-09-07), so nothing is overridden here — noted so nobody + # "simplifies" this onto urllib. + headers = {"Authorization": f"Bearer {key}"} + resp = requests.get(RAMP_ROUTER_MODELS_URL, headers=headers, timeout=self._timeout) + resp.raise_for_status() + return parse_ramp_router(resp.json()) + # ---------------------------------------------------------------------- # PricingProvider — cache + background refresh + non-blocking lookup @@ -1020,11 +1316,13 @@ def __init__( cloudflare_account_id: str | None = None, cloudflare_api_token: str | None = None, mistral_api_key: str | None = None, + ramp_router_api_key: str | None = None, ) -> None: self._fetcher: PricingFetcher = fetcher or HttpPricingFetcher( cloudflare_account_id=cloudflare_account_id, cloudflare_api_token=cloudflare_api_token, mistral_api_key=mistral_api_key, + ramp_router_api_key=ramp_router_api_key, ) self._ttl = ttl_seconds self._default_region = default_region @@ -1050,6 +1348,12 @@ def __init__( # for making real calls, so alias resolution can reuse it without # ever requiring a separate LagoConfig.mistral_api_key. self._mistral_api_key_override: str | None = None + self._ramp_router: dict[str, ModelPrice] | None = None + self._ramp_router_fetched = 0.0 + self._ramp_router_stale = False + # Learned from a wrapped OpenAI client pointed at Router, same mechanism and + # same precedence as the Mistral key above. + self._ramp_router_api_key_override: str | None = None self._refreshing: set[str] = set() # Per-source post-failure backoff — see `_in_backoff`. self._failure_backoff_until: dict[str, float] = {} @@ -1068,6 +1372,7 @@ def _heal_fork(self) -> None: self._bedrock_stale = set(self._bedrock.keys()) self._cloudflare_stale = self._cloudflare_workers_ai is not None or self._cloudflare_stale self._mistral_stale = self._mistral_aliases is not None or self._mistral_stale + self._ramp_router_stale = self._ramp_router is not None or self._ramp_router_stale self._refreshing = set() def prime(self, providers: Iterable[str] = ()) -> None: @@ -1088,7 +1393,7 @@ def prime(self, providers: Iterable[str] = ()) -> None: call for a given provider can race a cold cache; every provider that session never calls costs nothing. - Pass `providers=["mistral"]` and/or `["workers-ai"]` when you already + Pass `providers=["mistral"]`, `["workers-ai"]` and/or `["ramp_router"]` when you already know, in advance, which of these two you're about to call this session — this eagerly warms exactly that source too, so even ITS first call prices correctly instead of paying the one-time lazy @@ -1116,6 +1421,9 @@ def prime(self, providers: Iterable[str] = ()) -> None: elif key == "mistral": if self._is_cold(self._mistral_aliases, self._mistral_fetched): self._mistral_stale = True + elif key == "ramp_router": + if self._is_cold(self._ramp_router, self._ramp_router_fetched): + self._ramp_router_stale = True def _is_cold(self, table: Any, fetched_at: float) -> bool: """True when a table needs fetching: absent, or older than the TTL. @@ -1165,6 +1473,17 @@ def learn_mistral_api_key(self, api_key: str) -> None: if not self._mistral_api_key_override: self._mistral_api_key_override = api_key + def learn_ramp_router_api_key(self, api_key: str) -> None: + """Adopt the Router key a wrapped OpenAI client already carries, so the + catalog can be fetched without a separate `LagoConfig.ramp_router_api_key`. + Pure in-memory, no I/O. Same precedence as the Mistral key: an explicit + config value wins, and the first learned key is kept.""" + if not api_key: + return + with self._lock: + if not self._ramp_router_api_key_override: + self._ramp_router_api_key_override = api_key + # ---- non-blocking lookup (customer thread) ---- def lookup(self, provider: str, model: str, api: str) -> ModelPrice | None: try: @@ -1180,6 +1499,13 @@ def lookup(self, provider: str, model: str, api: str) -> ModelPrice | None: if not fresh: self._bedrock_stale.add(region) return lookup_bedrock(table, model) if table is not None else None + if (provider or "").lower() == "ramp_router": + with self._lock: + table_rr = self._ramp_router + fresh_rr = table_rr is not None and (time.time() - self._ramp_router_fetched) < self._ttl + if not fresh_rr: + self._ramp_router_stale = True + return lookup_ramp_router(table_rr, model) if table_rr is not None else None if (provider or "").lower() == "workers-ai": with self._lock: table_cf = self._cloudflare_workers_ai @@ -1222,6 +1548,7 @@ def maybe_refresh(self) -> None: and not self._bedrock_stale and not self._cloudflare_stale and not self._mistral_stale + and not self._ramp_router_stale ): return with self._lock: @@ -1250,6 +1577,11 @@ def _ready(source: str) -> bool: ) if do_mistral: self._refreshing.add("mistral_aliases") + do_ramp_router = ( + self._ramp_router_stale and "ramp_router" not in self._refreshing and _ready("ramp_router") + ) + if do_ramp_router: + self._refreshing.add("ramp_router") regions = [ r for r in self._bedrock_stale @@ -1305,6 +1637,23 @@ def _ready(source: str) -> bool: with self._lock: self._refreshing.discard("mistral_aliases") + if do_ramp_router: + try: + with self._lock: + learned_rr_key = self._ramp_router_api_key_override + table_rr = self._fetcher.fetch_ramp_router(learned_rr_key) + with self._lock: + self._ramp_router = table_rr + self._ramp_router_fetched = time.time() + self._ramp_router_stale = False + self._note_success("ramp_router") + except Exception as exc: # noqa: BLE001 + self._note_failure("ramp_router") + self._report(exc, "pricing.fetch_ramp_router") + finally: + with self._lock: + self._refreshing.discard("ramp_router") + for r in regions: try: table = self._fetcher.fetch_bedrock(r) diff --git a/src/lago_agent_sdk/sdk.py b/src/lago_agent_sdk/sdk.py index 4298033..0c395fc 100644 --- a/src/lago_agent_sdk/sdk.py +++ b/src/lago_agent_sdk/sdk.py @@ -26,6 +26,7 @@ compute_precomputed_cost, deoverlapped_token_total, money_str_to_cents, + ramp_router_unpriced_tier, ) from .queue import EventQueue @@ -153,6 +154,7 @@ def __init__( cloudflare_account_id=self.config.cloudflare_account_id, cloudflare_api_token=self.config.cloudflare_api_token, mistral_api_key=self.config.mistral_api_key, + ramp_router_api_key=self.config.ramp_router_api_key, ) if self.config.pricing_mode == "price": self._pricing.prime() # eager warm when price mode is the global default @@ -220,10 +222,47 @@ def _auto_prime_pricing_for(self, kind: str, client: Any) -> None: base_url = "" if "gateway.ai.cloudflare.com" in base_url: provider = "workers-ai" + else: + provider = self._learn_ramp_router_key_if_pointed_there(client) + elif kind == "anthropic": + # Router's second surface, `/v1/messages`, is reached with an Anthropic + # client. Same catalog, same key, same warm-up. + provider = self._learn_ramp_router_key_if_pointed_there(client) if provider: self._pricing.prime([provider]) self._queue.wake() + def _learn_ramp_router_key_if_pointed_there(self, client: Any) -> str | None: + """Ramp Router's catalog needs the customer's Router key, and the client being + wrapped already carries it — both `openai.OpenAI` and `anthropic.Anthropic` expose + the constructor's key as `.api_key` (verified on openai 2.38 and anthropic 0.103, + sync and async). Same shape as the Mistral arm: learn it here, so the very first + Router call has a warm table instead of a cold miss. Detection reuses the + wrappers' own host match so the three cannot disagree about what counts as + Router. Returns the provider to prime, or None when the client is not Router's.""" + from .adapters.openai_native import RAMP_ROUTER_PROVIDER + from .wrappers.ramp_router import client_points_at_ramp_router + + if not client_points_at_ramp_router(client): + return None + key = self._extract_client_api_key(client) + if key: + self._pricing.learn_ramp_router_api_key(key) + return RAMP_ROUTER_PROVIDER + + @staticmethod + def _extract_client_api_key(client: Any) -> str | None: + """The constructor's key at `client.api_key`, as the openai and anthropic SDKs + both expose it (verified against real instances). Defensive for the same reason + as the Mistral reader: a client variant without it degrades to "no key learned" — + then `LagoConfig.ramp_router_api_key`, then a reported miss — rather than + raising.""" + try: + key = client.api_key + except Exception: # noqa: BLE001 + return None + return key if isinstance(key, str) and key else None + @staticmethod def _extract_mistral_api_key(client: Any) -> str | None: """The mistralai SDK stores the constructor's `api_key=...` at @@ -418,11 +457,23 @@ def emit( self._emit_token_events(usage, sub, dimensions, event_id, at) return else: - price = self._pricing.lookup(usage.provider, usage.model, usage.api) + # A Ramp Router call served at a non-default tier bills at a rate the + # catalog does not publish (flex measured 0.5x, priority 2.0x), so it is + # a miss BEFORE the table is consulted — and a miss that names the tier, + # because the same model priced fine a moment ago and a bare "no price" + # would send the customer looking at the wrong thing. + unpriced_tier = ramp_router_unpriced_tier(usage) + if unpriced_tier is None: + price = self._pricing.lookup(usage.provider, usage.model, usage.api) + detail = None + else: + price = None + detail = f"served service_tier {unpriced_tier!r} bills at a rate Router does not publish" if price is None: # Don't silently under-bill: fall back to token events + report. self._report_error( - PricingUnavailableError(usage.provider, usage.model, usage.api), "pricing" + PricingUnavailableError(usage.provider, usage.model, usage.api, detail=detail), + "pricing", ) self._emit_token_events(usage, sub, dimensions, event_id, at) return diff --git a/src/lago_agent_sdk/token_semantics.py b/src/lago_agent_sdk/token_semantics.py index c425e1e..97f9ea7 100644 --- a/src/lago_agent_sdk/token_semantics.py +++ b/src/lago_agent_sdk/token_semantics.py @@ -157,6 +157,14 @@ # rather than measured — every observed write reported cache_write_tokens: 0 # while the warm read proved the cache existed, and the arithmetic keeps the # write inside input — re-verify the day a nonzero write appears in a capture. + # + # Router's OTHER surface, `/v1/messages` (stamped "ramp_router_messages" by the + # Anthropic adapter), is deliberately ABSENT. It keeps Anthropic's additive + # convention for every vendor: measured 2026-09-04 (haiku, `input_tokens: 16` + # beside `cache_read_input_tokens: 20113`, reconciled exactly against the + # dashboard) and 2026-09-07 (an xAI model, `input_tokens: 65` beside + # `cache_read_input_tokens: 128`, thinking inside output). Same gateway, same + # model, two conventions — one per surface, which is why the stamp differs. "ramp_router", } ) diff --git a/src/lago_agent_sdk/wrappers/anthropic.py b/src/lago_agent_sdk/wrappers/anthropic.py index 2c0a6a6..c309532 100644 --- a/src/lago_agent_sdk/wrappers/anthropic.py +++ b/src/lago_agent_sdk/wrappers/anthropic.py @@ -32,6 +32,8 @@ from typing import Any from ..adapters import extract_anthropic_native +from ..adapters.openai_native import RAMP_ROUTER_PROVIDER +from .ramp_router import client_points_at_ramp_router logger = logging.getLogger("lago_agent_sdk.wrappers.anthropic") @@ -135,6 +137,13 @@ def wrap_anthropic_client( raw_create = getattr(getattr(messages, "with_raw_response", None), "create", None) original_stream = getattr(messages, "stream", None) is_async = type(client).__name__.startswith("Async") + # Resolved once, here, and threaded through every emit path below — including the + # stream manager, which builds its own adapter call. An Anthropic client pointed at + # Ramp Router's `/v1/messages` answers in Anthropic's exact schema, so the base URL + # is the only thing that says Router was in the path. Without it the call billed as + # native Anthropic: OpenRouter's rate instead of Router's catalog, no Router key + # learned, and events stamped provider=anthropic. + provider_hint = RAMP_ROUTER_PROVIDER if client_points_at_ramp_router(client) else "" def _resolve_opts(lago_opts: dict[str, Any]) -> dict[str, Any]: return { @@ -146,7 +155,7 @@ def _resolve_opts(lago_opts: dict[str, Any]) -> dict[str, Any]: def _emit_from(payload: Any, model_id: str, opts: dict[str, Any]) -> None: try: - usage = extract_anthropic_native(payload, model_id=model_id) + usage = extract_anthropic_native(payload, model_id=model_id, provider_hint=provider_hint) sdk.emit(usage, **opts) except Exception as exc: # noqa: BLE001 logger.warning("lago: anthropic emit failed: %s", exc) @@ -243,7 +252,7 @@ def _wrap_stream_manager(*args: Any, **kwargs: Any) -> Any: model_id = kwargs.get("model", "") opts = _resolve_opts(lago_opts) inner = original_stream(*args, **kwargs) - return _LagoStreamManager(inner, sdk, model_id, opts, is_async=is_async) + return _LagoStreamManager(inner, sdk, model_id, opts, is_async=is_async, provider_hint=provider_hint) if original_create is not None: messages.create = _create_async if is_async else _create @@ -269,6 +278,7 @@ def __init__( opts: dict[str, Any], *, is_async: bool, + provider_hint: str = "", ) -> None: self._inner = inner self._sdk = sdk @@ -276,6 +286,7 @@ def __init__( self._opts = opts self._stream: Any = None self._is_async = is_async + self._provider_hint = provider_hint # ----- sync ----- def __enter__(self) -> Any: @@ -310,7 +321,9 @@ def _emit_final(self) -> None: if final is not None: from ..adapters import extract_anthropic_native - usage = extract_anthropic_native(final, model_id=self._model_id) + usage = extract_anthropic_native( + final, model_id=self._model_id, provider_hint=self._provider_hint + ) self._sdk.emit(usage, **self._opts) except Exception as exc: # noqa: BLE001 logger.warning("lago: anthropic stream-manager emit failed: %s", exc) @@ -330,7 +343,9 @@ async def _emit_final_async(self) -> None: if final is not None: from ..adapters import extract_anthropic_native - usage = extract_anthropic_native(final, model_id=self._model_id) + usage = extract_anthropic_native( + final, model_id=self._model_id, provider_hint=self._provider_hint + ) self._sdk.emit(usage, **self._opts) except Exception as exc: # noqa: BLE001 logger.warning("lago: anthropic async stream-manager emit failed: %s", exc) diff --git a/src/lago_agent_sdk/wrappers/openai.py b/src/lago_agent_sdk/wrappers/openai.py index 50cec40..7dd20e3 100644 --- a/src/lago_agent_sdk/wrappers/openai.py +++ b/src/lago_agent_sdk/wrappers/openai.py @@ -33,7 +33,6 @@ from __future__ import annotations import logging -import urllib.parse from collections.abc import AsyncIterator, Iterator from typing import Any @@ -46,6 +45,7 @@ # format that would drift without an error. The module is pure (canonical-only imports, # no I/O), so this pulls nothing heavy into the wrap() path. from ..gateway.adapters.snowflake_cortex import SNOWFLAKE_EVENT_ID_PREFIX, snowflake_event_id +from .ramp_router import is_ramp_router_base_url logger = logging.getLogger("lago_agent_sdk.wrappers.openai") @@ -170,14 +170,10 @@ def _snowflake_request_id(header_owner: Any) -> str: # Ramp Router cannot be a row in the path table above: it serves every provider it # fronts through one dedicated host with no distinguishing path, so the HOST is the -# signal — and it must be the PARSED host, never a substring test. A substring row -# ("api.router.com") also matches `https://evil.example.com/api.router.com/v1`, which -# would stamp an unrelated endpoint's traffic as Router-served. The `.router.com` -# suffix arm covers a regional or staging host without widening to arbitrary domains — -# `evilrouter.com` does not end in `.router.com`. The path table keeps first say: its -# rows are more specific, and no Snowflake or Databricks URL lives under router.com. -_RAMP_ROUTER_HOST = "api.router.com" -_RAMP_ROUTER_DOMAIN = ".router.com" +# signal. The match itself lives in `wrappers/ramp_router.py`, because the Anthropic +# wrapper needs the identical answer for Router's `/v1/messages` surface. The path table +# keeps first say: its rows are more specific, and no Snowflake or Databricks URL lives +# under router.com. def _provider_hint_for(client: Any) -> str: @@ -205,12 +201,7 @@ def _provider_hint_for(client: Any) -> str: for path, provider in _PROVIDER_BY_BASE_URL_PATH: if path in base_url: return provider - try: - host = urllib.parse.urlsplit(base_url).hostname or "" - except ValueError: - # A relative or malformed base_url is not a gateway. Never throw out of wrap(). - return "" - if host == _RAMP_ROUTER_HOST or host.endswith(_RAMP_ROUTER_DOMAIN): + if is_ramp_router_base_url(base_url): return RAMP_ROUTER_PROVIDER return "" @@ -300,19 +291,34 @@ def _extract_stream_usage(payload: Any) -> dict[str, Any] | None: lists, so price mode missed and silently degraded to token events. It matters most on a gateway, where the resolved name is what decides which price table the call is even looked up in. + + `service_tier` rides along for the same reason. The adapter reads it off the + response's top level to record which tier SERVED the call, and on Ramp Router + that decides whether the call prices at all — a non-default tier is a reported + miss. The terminal `response.completed` event carries it (fixture + 04_real_streamed.json: `flex`), but a usage-and-model-only payload dropped it, + so every streamed Router call reached price mode tier-less and missed. """ if not isinstance(payload, dict): return None usage = payload.get("usage") if isinstance(usage, dict) and usage: - return {"usage": usage, "model": payload.get("model")} + return { + "usage": usage, + "model": payload.get("model"), + "service_tier": payload.get("service_tier"), + } # Responses API stream events nest usage under `.response.usage` — and the # resolved model under `.response.model`, not at the event's top level. response = payload.get("response") if isinstance(response, dict): nested = response.get("usage") if isinstance(nested, dict) and nested: - return {"usage": nested, "model": response.get("model")} + return { + "usage": nested, + "model": response.get("model"), + "service_tier": response.get("service_tier"), + } return None def _make_sync_create(original: Any, raw_create: Any | None, is_responses_api: bool = False) -> Any: diff --git a/src/lago_agent_sdk/wrappers/ramp_router.py b/src/lago_agent_sdk/wrappers/ramp_router.py new file mode 100644 index 0000000..3ee665a --- /dev/null +++ b/src/lago_agent_sdk/wrappers/ramp_router.py @@ -0,0 +1,46 @@ +"""Ramp Router host detection, shared by every wrapper a customer can point at Router. + +Router serves every provider it fronts through one dedicated host on two surfaces — +`/v1/responses` (reached with an OpenAI client) and `/v1/messages` (reached with an +Anthropic client). Neither surface leaves a mark in the response body: an +Anthropic-served answer on the first is byte-indistinguishable from real OpenAI, and any +vendor's answer on the second is rendered in Anthropic's schema. So the client's base URL +is the ONLY signal, and both wrappers must read it the same way — a copy in each would +drift, and the Anthropic wrapper going without one billed Router traffic as native +Anthropic (wrong price table, no Router key learned). + +It must be the PARSED host, never a substring test. A substring row ("api.router.com") +also matches `https://evil.example.com/api.router.com/v1`, which would stamp an unrelated +endpoint's traffic as Router-served. The `.router.com` suffix arm covers a regional or +staging host without widening to arbitrary domains — `evilrouter.com` does not end in +`.router.com`. +""" + +from __future__ import annotations + +import urllib.parse +from typing import Any + +RAMP_ROUTER_HOST = "api.router.com" +RAMP_ROUTER_DOMAIN = ".router.com" + + +def is_ramp_router_base_url(base_url: Any) -> bool: + """True when `base_url` names Router's host. A relative, malformed or non-string value + is not a gateway — and never raises, because this runs inside `wrap()`.""" + try: + host = urllib.parse.urlsplit(str(base_url or "")).hostname or "" + except ValueError: + return False + return host == RAMP_ROUTER_HOST or host.endswith(RAMP_ROUTER_DOMAIN) + + +def client_points_at_ramp_router(client: Any) -> bool: + """Read the client's `base_url` defensively and test it. Both the openai and + anthropic SDKs expose the constructor's URL as `.base_url` (an httpx.URL); some + client variants may not, and a property that raises must not break `wrap()`.""" + try: + base_url = getattr(client, "base_url", "") + except Exception: # noqa: BLE001 — a custom client's property may raise + return False + return is_ramp_router_base_url(base_url) diff --git a/tests/unit/adapters/fixtures/capture_ramp_router.py b/tests/unit/adapters/fixtures/capture_ramp_router.py index 0e53053..9b0803d 100644 --- a/tests/unit/adapters/fixtures/capture_ramp_router.py +++ b/tests/unit/adapters/fixtures/capture_ramp_router.py @@ -113,20 +113,25 @@ def _scrub_string(s: str) -> str: ) -def _scrub(value: Any, key: str = "") -> Any: +def _scrub(value: Any, key: str = "", parent: str = "") -> Any: """Deep-scrub a captured payload. Content keys are blanked to "" rather than deleted, so the shape a test reads is the shape Router really sent — deleting entries would change what the fixture proves. + + `parent` exists because a content key is only content in a content position. The + catalog's `router.pricing.input` is a RATE, and blanking it shipped a fixture whose + every input rate read "" — which was then taken for Router's own data and cited as + a reason price mode could not be built. Nothing under `pricing` is ever content. """ if isinstance(value, str): - if key in _CONTENT_KEYS: + if key in _CONTENT_KEYS and parent != "pricing": return "" return _scrub_string(value) if isinstance(value, list): - return [_scrub(v, key) for v in value] + return [_scrub(v, key, parent) for v in value] if isinstance(value, dict): - return {k: _scrub(v, k) for k, v in value.items() if k.lower() not in _DROP_HEADERS} + return {k: _scrub(v, k, key) for k, v in value.items() if k.lower() not in _DROP_HEADERS} return value @@ -163,7 +168,9 @@ def save(name: str, probe: str, question: str, payload: dict[str, Any]) -> None: # --------------------------------------------------------------------------- -def call(method: str, path: str, body: Any | None = None) -> dict[str, Any]: +def call( + method: str, path: str, body: Any | None = None, extra_headers: dict[str, str] | None = None +) -> dict[str, Any]: """One request, captured whole. The body is parsed as JSON when it is JSON and kept as text when it is not: @@ -178,6 +185,7 @@ def call(method: str, path: str, body: Any | None = None) -> dict[str, Any]: "Authorization": f"Bearer {API_KEY}", "Content-Type": "application/json", "User-Agent": "lago-agent-sdk-capture/0.2.0", + **(extra_headers or {}), }, data=None if body is None else json.dumps(body).encode(), ) @@ -195,9 +203,13 @@ def call(method: str, path: str, body: Any | None = None) -> dict[str, Any]: return {"_status": status, "_headers": header_map, "_body": parsed, "_body_was_json": was_json} -def call_stream(body: Any) -> dict[str, Any]: - """A streamed request, captured as the ordered list of SSE events.""" - captured = call("POST", "/responses", body) +def call_stream( + body: Any, path: str = "/responses", extra_headers: dict[str, str] | None = None +) -> dict[str, Any]: + """A streamed request, captured as the ordered list of SSE events. Anthropic's SSE + prefixes each `data:` line with an `event:` line; only the data lines are kept, which + is also all the wrapper reads.""" + captured = call("POST", path, body, extra_headers=extra_headers) raw = captured["_body"] if isinstance(captured["_body"], str) else "" events: list[Any] = [] for line in raw.split("\n"): @@ -241,6 +253,81 @@ def candidate_id(models: list[dict[str, Any]], model_id: str | None) -> str | No return None +# --------------------------------------------------------------------------- +# P11-P15: Router's SECOND surface, `POST /v1/messages` (Anthropic-shaped), reached with an +# Anthropic client. Captured because it is the only surface that reports an +# Anthropic-served cache WRITE — `/v1/responses` has no field for it — and because nothing +# in these bodies says Router was in the path, so these fixtures are what the Anthropic +# wrapper's provider hint is tested against. Run alone with `--messages-only`. +# --------------------------------------------------------------------------- +# Required on this surface (a request without it is rejected, measured 2026-09-04). +_MESSAGES_HEADERS = {"anthropic-version": "2023-06-01"} + + +def _cache_prefix(n_lines: int = 220) -> str: + """~2.6k tokens of synthetic ledger lines: above Haiku's 2,048-token minimum cacheable + prefix, and cheap to write (a few tenths of a cent).""" + return " ".join( + f"Ledger entry {i}: account ACC-{(i * 7) % 9973:05d}, debit {(i * 11) % 503}.{(i * 13) % 100:02d} EUR, " + f"memo 'batch {i // 12} settlement', region {chr(65 + (i * 5) % 26)}." + for i in range(n_lines) + ) + + +def _probe_messages(cheap: str | None, anthropic: str | None) -> None: + def messages_call(body: dict[str, Any]) -> dict[str, Any]: + return call("POST", "/messages", body, extra_headers=_MESSAGES_HEADERS) + + user = [{"role": "user", "content": PROMPT}] + if anthropic: + print("[P11] /v1/messages, plain call, Anthropic-served") + save( + "11_real_messages_plain.json", + "P11", + "does the Anthropic-shaped surface report usage in Anthropic's native shape, service_tier inside usage?", + messages_call({"model": anthropic, "max_tokens": MAX_OUTPUT_TOKENS, "messages": user}), + ) + cached = { + "model": anthropic, + "max_tokens": MAX_OUTPUT_TOKENS, + "system": [{"type": "text", "text": _cache_prefix(), "cache_control": {"type": "ephemeral"}}], + "messages": user, + } + print("[P12] /v1/messages, cache_control cold — is the WRITE count reported here?") + save( + "12_real_messages_cache_control_cold.json", + "P12", + "does a cold cache_control write report cache_creation_input_tokens, with the 5m/1h split?", + messages_call(cached), + ) + print("[P13] /v1/messages, cache_control warm") + save( + "13_real_messages_cache_control_warm.json", + "P13", + "warm repeat of P12 — cache_read_input_tokens beside input_tokens (additive)?", + messages_call(cached), + ) + print("[P14] /v1/messages, streamed") + save( + "14_real_messages_streamed.json", + "P14", + "where do usage and service_tier land across message_start / message_delta?", + call_stream( + {"model": anthropic, "max_tokens": MAX_OUTPUT_TOKENS, "stream": True, "messages": user}, + path="/messages", + extra_headers=_MESSAGES_HEADERS, + ), + ) + if cheap: + print("[P15] /v1/messages, OpenAI-served model") + save( + "15_real_messages_openai_served.json", + "P15", + "does a non-Anthropic vendor's usage arrive in Anthropic's additive shape on this surface?", + messages_call({"model": cheap, "max_tokens": MAX_OUTPUT_TOKENS, "messages": user}), + ) + + def main() -> None: # ---- P1: the catalog. Does it publish prices? ---------------------------------- print("[P1] GET /v1/models") @@ -280,6 +367,10 @@ def main() -> None: ) print(f" using: cheap={cheap} anthropic={anthropic} reasoning={reasoning}") + if "--messages-only" in sys.argv: + _probe_messages(cheap, anthropic) + return + # ---- P2: requested alias, or served candidate? ---------------------------------- if cheap: print("[P2] buffered call, plain `model`") diff --git a/tests/unit/adapters/fixtures/ramp_router/01_real_models_catalog.json b/tests/unit/adapters/fixtures/ramp_router/01_real_models_catalog.json index d5e3e1c..a206675 100644 --- a/tests/unit/adapters/fixtures/ramp_router/01_real_models_catalog.json +++ b/tests/unit/adapters/fixtures/ramp_router/01_real_models_catalog.json @@ -5,26 +5,430 @@ "_scrubbed": "credentials, emails, IPs, dashboard URLs; prompt and completion text blanked to \"\"", "_status": 200, "_headers": { - "alt-svc": "h3=\":443\"; ma=86400", - "cf-cache-status": "DYNAMIC", - "cf-ray": "a3238328fd280956-EWR", - "connection": "keep-alive", - "content-encoding": "br", + "date": "Mon, 07 Sep 2026 10:46:47 GMT", "content-type": "application/json", - "date": "Fri, 28 Aug 2026 13:02:47 GMT", - "server": "cloudflare", - "strict-transport-security": "max-age=31536000; includeSubDomains; preload", "transfer-encoding": "chunked", + "connection": "close", "vary": "Origin", + "x-request-id": "604917a0-b621-4541-a254-57bda6cb43c8", + "x-trace-id": "604917a0-b621-4541-a254-57bda6cb43c8", + "cf-cache-status": "DYNAMIC", + "strict-transport-security": "max-age=31536000; includeSubDomains; preload", "x-content-type-options": "nosniff", - "x-request-id": "52ee6546-2898-4565-8832-8913ef8c4eb1", - "x-trace-id": "52ee6546-2898-4565-8832-8913ef8c4eb1" + "server": "cloudflare", + "cf-ray": "a37521b50dfc6ba3-DFW", + "alt-svc": "h3=\":443\"; ma=86400" }, "_body": { "object": "list", "data": [ { - "id": "accounts/fireworks/models/deepseek-v4-pro-0813", + "id": "gpt-6-astra", + "object": "model", + "created": 1788480000, + "owned_by": "openai", + "display_name": "GPT-6 Astra", + "context_length": 1050000, + "router": { + "schema_version": 1, + "request_name": "gpt-6-astra", + "catalog_id": "openai:gpt-6-astra", + "provider_model": "gpt-6-astra", + "display_name": "GPT-6 Astra", + "provider_display_name": "OpenAI", + "description": "GPT-6 Astra is our most capable model, built for the hardest end-to-end work. Use it for complex reasoning, coding, computer use, research, and document creation. reasoning.effort supports low , medium , high , xhigh , and max .", + "status": "active", + "listing": { + "order": 0, + "featured": true, + "snapshot": false + }, + "limits": { + "context_window": 1050000, + "max_input_tokens": null, + "max_output_tokens": 128000 + }, + "capabilities": { + "modalities": { + "input": [ + "", + "" + ], + "output": [ + "text" + ] + }, + "tools": { + "supported": true, + "parallel": null + }, + "structured_outputs": true, + "temperature": false, + "prompt_caching": true, + "reasoning": { + "supported": true, + "efforts": [ + { + "value": "low", + "description": "Fast responses with lighter reasoning" + }, + { + "value": "medium", + "description": "Balances speed and reasoning depth" + }, + { + "value": "high", + "description": "Greater reasoning depth" + }, + { + "value": "xhigh", + "description": "Reasons longer than high" + }, + { + "value": "max", + "description": "Reasons as long as the model allows" + } + ], + "default_effort": "medium", + "summary": { + "output_supported": true, + "request_parameter_supported": true, + "values": [ + "auto", + "concise", + "detailed" + ], + "default": "auto" + }, + "continuation": { + "request_include": [ + "reasoning.encrypted_content" + ], + "required": true + } + }, + "verbosity": { + "supported": true, + "values": [ + "low", + "medium", + "high" + ], + "default": "medium" + } + }, + "pricing": { + "input": "10", + "output": "50", + "cache_read_input": "1", + "cache_write_input": "12.5", + "cache_write_input_5m": "0", + "cache_write_input_1h": "0" + } + } + }, + { + "id": "claude-fable-5-1", + "object": "model", + "created": 1788048000, + "owned_by": "anthropic", + "display_name": "Claude Fable 5.1", + "context_length": 1000000, + "router": { + "schema_version": 1, + "request_name": "claude-fable-5-1", + "catalog_id": "anthropic:claude-fable-5-1", + "provider_model": "claude-fable-5-1", + "display_name": "Claude Fable 5.1", + "provider_display_name": "Anthropic", + "description": "Frontier intelligence for long-running agents, agentic coding, and knowledge work", + "status": "active", + "listing": { + "order": 1, + "featured": false, + "snapshot": false + }, + "limits": { + "context_window": 1000000, + "max_input_tokens": 1000000, + "max_output_tokens": 128000 + }, + "capabilities": { + "modalities": { + "input": [ + "", + "" + ], + "output": [ + "text" + ] + }, + "tools": { + "supported": true, + "parallel": null + }, + "structured_outputs": true, + "temperature": true, + "prompt_caching": true, + "reasoning": { + "supported": true, + "efforts": [ + { + "value": "minimal", + "description": "Reasons briefly before answering" + }, + { + "value": "low", + "description": "Fast responses with lighter reasoning" + }, + { + "value": "medium", + "description": "Balances speed and reasoning depth" + }, + { + "value": "high", + "description": "Greater reasoning depth" + }, + { + "value": "xhigh", + "description": "Reasons longer than high" + }, + { + "value": "max", + "description": "Reasons as long as the model allows" + } + ], + "default_effort": "high", + "summary": { + "output_supported": true, + "request_parameter_supported": false, + "values": [] + }, + "continuation": { + "request_include": [], + "required": false + } + }, + "verbosity": { + "supported": false, + "values": [] + } + }, + "pricing": { + "input": "10", + "output": "50", + "cache_read_input": "0.25", + "cache_write_input": "0", + "cache_write_input_5m": "12.5", + "cache_write_input_1h": "20" + } + } + }, + { + "id": "glm-5p3", + "object": "model", + "created": 1787875200, + "owned_by": "fireworks", + "display_name": "GLM 5.3", + "context_length": 1048576, + "router": { + "schema_version": 1, + "request_name": "glm-5p3", + "catalog_id": "fireworks:accounts/fireworks/models/glm-5p3", + "provider_model": "accounts/fireworks/models/glm-5p3", + "display_name": "GLM 5.3", + "provider_display_name": "Fireworks", + "aliases": [ + "accounts/fireworks/models/glm-5p3" + ], + "description": "GLM-5.3 uses the same base model as GLM-5.2 \u2014 every gain comes from post-training. Compared with GLM-5.2, it is much better at complex coding and long-horizon tasks", + "status": "active", + "listing": { + "order": 2, + "featured": true, + "snapshot": false + }, + "limits": { + "context_window": 1048576, + "max_input_tokens": null, + "max_output_tokens": 1048576 + }, + "capabilities": { + "modalities": { + "input": [ + "" + ], + "output": [ + "text" + ] + }, + "tools": { + "supported": true, + "parallel": null + }, + "structured_outputs": true, + "temperature": true, + "prompt_caching": true, + "reasoning": { + "supported": true, + "efforts": [ + { + "value": "none", + "description": "Answers without reasoning first" + }, + { + "value": "minimal", + "description": "Reasons briefly before answering" + }, + { + "value": "low", + "description": "Fast responses with lighter reasoning" + }, + { + "value": "medium", + "description": "Balances speed and reasoning depth" + }, + { + "value": "high", + "description": "Greater reasoning depth" + }, + { + "value": "xhigh", + "description": "Reasons longer than high" + }, + { + "value": "max", + "description": "Reasons as long as the model allows" + } + ], + "default_effort": "xhigh", + "summary": { + "output_supported": true, + "request_parameter_supported": false, + "values": [] + }, + "continuation": { + "request_include": [], + "required": false + } + }, + "verbosity": { + "supported": false, + "values": [] + } + }, + "pricing": { + "input": "1.4", + "output": "4.4", + "cache_read_input": "0.26", + "cache_write_input": "0", + "cache_write_input_5m": "0", + "cache_write_input_1h": "0" + } + } + }, + { + "id": "glm-5p3-flash", + "object": "model", + "created": 1787702400, + "owned_by": "fireworks", + "display_name": "GLM 5.3 Flash", + "context_length": 1048576, + "router": { + "schema_version": 1, + "request_name": "glm-5p3-flash", + "catalog_id": "fireworks:accounts/fireworks/models/glm-5p3-flash", + "provider_model": "accounts/fireworks/models/glm-5p3-flash", + "display_name": "GLM 5.3 Flash", + "provider_display_name": "Fireworks", + "aliases": [ + "accounts/fireworks/models/glm-5p3-flash", + "zai-org/GLM-5.3-Flash" + ], + "status": "active", + "listing": { + "order": 3, + "featured": true, + "snapshot": false + }, + "limits": { + "context_window": 1048576, + "max_input_tokens": null, + "max_output_tokens": 1048576 + }, + "capabilities": { + "modalities": { + "input": [ + "", + "" + ], + "output": [ + "text" + ] + }, + "tools": { + "supported": true, + "parallel": null + }, + "structured_outputs": true, + "temperature": true, + "prompt_caching": true, + "reasoning": { + "supported": true, + "efforts": [ + { + "value": "minimal", + "description": "Reasons briefly before answering" + }, + { + "value": "low", + "description": "Fast responses with lighter reasoning" + }, + { + "value": "medium", + "description": "Balances speed and reasoning depth" + }, + { + "value": "high", + "description": "Greater reasoning depth" + }, + { + "value": "xhigh", + "description": "Reasons longer than high" + }, + { + "value": "max", + "description": "Reasons as long as the model allows" + } + ], + "default_effort": "high", + "summary": { + "output_supported": true, + "request_parameter_supported": false, + "values": [] + }, + "continuation": { + "request_include": [], + "required": false + } + }, + "verbosity": { + "supported": false, + "values": [] + } + }, + "pricing": { + "input": "0.15", + "output": "0.5", + "cache_read_input": "0.03", + "cache_write_input": "0", + "cache_write_input_5m": "0", + "cache_write_input_1h": "0" + } + } + }, + { + "id": "deepseek-v4-pro-0813", "object": "model", "created": 1786579200, "owned_by": "fireworks", @@ -32,15 +436,19 @@ "context_length": 1048576, "router": { "schema_version": 1, - "request_name": "accounts/fireworks/models/deepseek-v4-pro-0813", + "request_name": "deepseek-v4-pro-0813", "catalog_id": "fireworks:accounts/fireworks/models/deepseek-v4-pro-0813", "provider_model": "accounts/fireworks/models/deepseek-v4-pro-0813", "display_name": "DeepSeek-V4-Pro-0813", "provider_display_name": "Fireworks", + "aliases": [ + "accounts/fireworks/models/deepseek-v4-pro-0813", + "deepseek-ai/DeepSeek-V4-Pro-0813" + ], "description": "DeepSeek-V4-Pro-0813 is the official release of DeepSeek-V4-Pro, superseding the preview version, with greatly enhanced agentic capabilities and performance improvements that are especially pronounced in production environments. It is built on the DeepSeek-V4-Pro (Preview) model structure, with a DSpark speculative decoding module attached.", "status": "active", "listing": { - "order": 0, + "order": 5, "featured": false, "snapshot": false }, @@ -114,7 +522,7 @@ } }, "pricing": { - "input": "", + "input": "1.32", "output": "3.96", "cache_read_input": "0.044", "cache_write_input": "0", @@ -139,7 +547,7 @@ "provider_display_name": "xAI", "status": "active", "listing": { - "order": 1, + "order": 7, "featured": false, "snapshot": false }, @@ -206,7 +614,7 @@ } }, "pricing": { - "input": "", + "input": "2", "output": "6", "cache_read_input": "0.5", "cache_write_input": "0", @@ -216,7 +624,7 @@ } }, { - "id": "accounts/fireworks/models/nemotron-lightning-3p5-30b-a3b", + "id": "nemotron-lightning-3p5-30b-a3b", "object": "model", "created": 1786060800, "owned_by": "fireworks", @@ -224,15 +632,18 @@ "context_length": 262144, "router": { "schema_version": 1, - "request_name": "accounts/fireworks/models/nemotron-lightning-3p5-30b-a3b", + "request_name": "nemotron-lightning-3p5-30b-a3b", "catalog_id": "fireworks:accounts/fireworks/models/nemotron-lightning-3p5-30b-a3b", "provider_model": "accounts/fireworks/models/nemotron-lightning-3p5-30b-a3b", "display_name": "Nemotron Lightning 3.5 30B A3B", "provider_display_name": "Fireworks", + "aliases": [ + "accounts/fireworks/models/nemotron-lightning-3p5-30b-a3b" + ], "description": "Nemotron-Lightning-3.5-30B-A3B is a 30B-parameter Mixture-of-Experts language model (3B active) from NVIDIA's Nemotron-H family, built on a hybrid Mamba-Transformer architecture for efficient long-context inference. Like other models in the family, it responds to queries by first generating a reasoning trace and then concluding with a final response, with reasoning behavior configurable through a flag in the chat template. It includes a multi-token prediction (MTP) speculative decoding head for low-latency serving.", "status": "active", "listing": { - "order": 2, + "order": 8, "featured": false, "snapshot": false }, @@ -276,7 +687,7 @@ } }, "pricing": { - "input": "", + "input": "0.05", "output": "0.2", "cache_read_input": "0.01", "cache_write_input": "0", @@ -286,7 +697,7 @@ } }, { - "id": "accounts/fireworks/models/qwen3p8-max", + "id": "qwen3p8-max", "object": "model", "created": 1785888000, "owned_by": "fireworks", @@ -294,15 +705,18 @@ "context_length": 1000000, "router": { "schema_version": 1, - "request_name": "accounts/fireworks/models/qwen3p8-max", + "request_name": "qwen3p8-max", "catalog_id": "fireworks:accounts/fireworks/models/qwen3p8-max", "provider_model": "accounts/fireworks/models/qwen3p8-max", "display_name": "Qwen 3.8 Max", "provider_display_name": "Fireworks", + "aliases": [ + "accounts/fireworks/models/qwen3p8-max" + ], "description": "Built on the architectural foundation of Qwen3.5, Qwen3.8 delivers substantial gains across coding, professional work, research, and long-horizon agentic tasks. Beyond answering harder questions, Qwen3.8 is designed to carry complex, multi-step tasks through to completion with greater reliability.", "status": "active", "listing": { - "order": 3, + "order": 9, "featured": false, "snapshot": false }, @@ -373,7 +787,7 @@ } }, "pricing": { - "input": "", + "input": "2", "output": "6", "cache_read_input": "0.25", "cache_write_input": "0", @@ -383,7 +797,7 @@ } }, { - "id": "accounts/fireworks/models/deepseek-v4-flash-0731", + "id": "deepseek-v4-flash-0731", "object": "model", "created": 1785456000, "owned_by": "fireworks", @@ -391,15 +805,19 @@ "context_length": 1048576, "router": { "schema_version": 1, - "request_name": "accounts/fireworks/models/deepseek-v4-flash-0731", + "request_name": "deepseek-v4-flash-0731", "catalog_id": "fireworks:accounts/fireworks/models/deepseek-v4-flash-0731", "provider_model": "accounts/fireworks/models/deepseek-v4-flash-0731", "display_name": "DeepSeek-V4-Flash-0731", "provider_display_name": "Fireworks", + "aliases": [ + "accounts/fireworks/models/deepseek-v4-flash-0731", + "deepseek-ai/DeepSeek-V4-Flash-0731" + ], "description": "DeepSeek-V4-Flash-0731 is the official release of DeepSeek-V4-Flash, superseding the preview version, with substantially enhanced agentic capabilities. It has the same model structure as DeepSeek-V4-Flash-DSpark, i.e. it comes with a speculative decoding module attached.", "status": "active", "listing": { - "order": 4, + "order": 10, "featured": false, "snapshot": false }, @@ -473,7 +891,7 @@ } }, "pricing": { - "input": "", + "input": "0.22", "output": "0.66", "cache_read_input": "0.007", "cache_write_input": "0", @@ -482,6 +900,85 @@ } } }, + { + "id": "inkling-small", + "object": "model", + "created": 1785369600, + "owned_by": "baseten", + "display_name": "Inkling Small (Baseten)", + "context_length": 1048576, + "router": { + "schema_version": 1, + "request_name": "inkling-small", + "catalog_id": "baseten:thinkingmachines/inkling-small", + "provider_model": "thinkingmachines/inkling-small", + "display_name": "Inkling Small (Baseten)", + "provider_display_name": "Baseten", + "aliases": [ + "thinkingmachines/inkling-small" + ], + "description": "Inkling Small served through Baseten Model APIs.", + "status": "active", + "listing": { + "order": 11, + "featured": false, + "snapshot": false + }, + "limits": { + "context_window": 1048576, + "max_input_tokens": null, + "max_output_tokens": 1048576 + }, + "capabilities": { + "modalities": { + "input": [ + "" + ], + "output": [ + "text" + ] + }, + "tools": { + "supported": true, + "parallel": null + }, + "structured_outputs": true, + "temperature": true, + "prompt_caching": true, + "reasoning": { + "supported": true, + "efforts": [ + { + "value": "high", + "description": "Greater reasoning depth" + } + ], + "default_effort": "high", + "summary": { + "output_supported": true, + "request_parameter_supported": false, + "values": [] + }, + "continuation": { + "request_include": [], + "required": false + } + }, + "verbosity": { + "supported": false, + "values": [] + } + }, + "pricing": { + "input": "0.5", + "output": "1.2", + "cache_read_input": "0.1", + "cache_write_input": "0", + "cache_write_input_5m": "0", + "cache_write_input_1h": "0" + } + } + }, { "id": "claude-opus-5", "object": "model", @@ -499,7 +996,7 @@ "description": "For complex agentic coding and enterprise work", "status": "active", "listing": { - "order": 5, + "order": 13, "featured": true, "snapshot": false }, @@ -574,7 +1071,7 @@ } }, "pricing": { - "input": "", + "input": "5", "output": "25", "cache_read_input": "0.5", "cache_write_input": "0", @@ -584,7 +1081,7 @@ } }, { - "id": "accounts/fireworks/models/kimi-k3", + "id": "kimi-k3", "object": "model", "created": 1784419200, "owned_by": "fireworks", @@ -592,15 +1089,19 @@ "context_length": 1048576, "router": { "schema_version": 1, - "request_name": "accounts/fireworks/models/kimi-k3", + "request_name": "kimi-k3", "catalog_id": "fireworks:accounts/fireworks/models/kimi-k3", "provider_model": "accounts/fireworks/models/kimi-k3", "display_name": "Kimi K3", "provider_display_name": "Fireworks", - "description": "Kimi K3 is Kimi’s most capable flagship model to date, with 2.8 trillion parameters. It is built on Kimi Delta Attention (KDA), a hybrid linear attention mechanism, and Attention Residuals, with native visual understanding and a 1M-token context window. It is the world’s first open-source model in the 3-trillion-parameter class, designed for frontier intelligence scenarios including long-horizon coding, knowledge work, and reasoning.", + "aliases": [ + "moonshotai/Kimi-K3", + "accounts/fireworks/models/kimi-k3" + ], + "description": "Kimi K3 is Kimi\u2019s most capable flagship model to date, with 2.8 trillion parameters. It is built on Kimi Delta Attention (KDA), a hybrid linear attention mechanism, and Attention Residuals, with native visual understanding and a 1M-token context window. It is the world\u2019s first open-source model in the 3-trillion-parameter class, designed for frontier intelligence scenarios including long-horizon coding, knowledge work, and reasoning.", "status": "active", "listing": { - "order": 8, + "order": 17, "featured": true, "snapshot": false }, @@ -671,7 +1172,7 @@ } }, "pricing": { - "input": "", + "input": "3", "output": "15", "cache_read_input": "0.3", "cache_write_input": "0", @@ -681,7 +1182,7 @@ } }, { - "id": "accounts/fireworks/routers/kimi-k3-fast", + "id": "kimi-k3-fast", "object": "model", "created": 1784419200, "owned_by": "fireworks", @@ -689,15 +1190,18 @@ "context_length": 1048576, "router": { "schema_version": 1, - "request_name": "accounts/fireworks/routers/kimi-k3-fast", + "request_name": "kimi-k3-fast", "catalog_id": "fireworks:accounts/fireworks/routers/kimi-k3-fast", "provider_model": "accounts/fireworks/routers/kimi-k3-fast", "display_name": "Kimi K3 Fast", "provider_display_name": "Fireworks", - "description": "Kimi K3 is Kimi’s most capable flagship model to date, with 2.8 trillion parameters. It is built on Kimi Delta Attention (KDA), a hybrid linear attention mechanism, and Attention Residuals, with native visual understanding and a 1M-token context window. It is the world’s first open-source model in the 3-trillion-parameter class, designed for frontier intelligence scenarios including long-horizon coding, knowledge work, and reasoning.", + "aliases": [ + "accounts/fireworks/routers/kimi-k3-fast" + ], + "description": "Kimi K3 is Kimi\u2019s most capable flagship model to date, with 2.8 trillion parameters. It is built on Kimi Delta Attention (KDA), a hybrid linear attention mechanism, and Attention Residuals, with native visual understanding and a 1M-token context window. It is the world\u2019s first open-source model in the 3-trillion-parameter class, designed for frontier intelligence scenarios including long-horizon coding, knowledge work, and reasoning.", "status": "active", "listing": { - "order": 9, + "order": 18, "featured": false, "snapshot": false }, @@ -768,7 +1272,7 @@ } }, "pricing": { - "input": "", + "input": "4.5", "output": "22.5", "cache_read_input": "0.45", "cache_write_input": "0", @@ -777,6 +1281,85 @@ } } }, + { + "id": "inkling", + "object": "model", + "created": 1784073600, + "owned_by": "baseten", + "display_name": "Inkling (Baseten)", + "context_length": 1048576, + "router": { + "schema_version": 1, + "request_name": "inkling", + "catalog_id": "baseten:thinkingmachines/inkling", + "provider_model": "thinkingmachines/inkling", + "display_name": "Inkling (Baseten)", + "provider_display_name": "Baseten", + "aliases": [ + "thinkingmachines/inkling" + ], + "description": "Inkling served through Baseten Model APIs.", + "status": "active", + "listing": { + "order": 19, + "featured": false, + "snapshot": false + }, + "limits": { + "context_window": 1048576, + "max_input_tokens": null, + "max_output_tokens": 1048576 + }, + "capabilities": { + "modalities": { + "input": [ + "" + ], + "output": [ + "text" + ] + }, + "tools": { + "supported": true, + "parallel": null + }, + "structured_outputs": true, + "temperature": true, + "prompt_caching": true, + "reasoning": { + "supported": true, + "efforts": [ + { + "value": "high", + "description": "Greater reasoning depth" + } + ], + "default_effort": "high", + "summary": { + "output_supported": true, + "request_parameter_supported": false, + "values": [] + }, + "continuation": { + "request_include": [], + "required": false + } + }, + "verbosity": { + "supported": false, + "values": [] + } + }, + "pricing": { + "input": "1", + "output": "4.05", + "cache_read_input": "0.17", + "cache_write_input": "0", + "cache_write_input_5m": "0", + "cache_write_input_1h": "0" + } + } + }, { "id": "claude-sonnet-5", "object": "model", @@ -794,7 +1377,7 @@ "description": "The best combination of speed and intelligence", "status": "active", "listing": { - "order": 10, + "order": 21, "featured": false, "snapshot": false }, @@ -869,7 +1452,7 @@ } }, "pricing": { - "input": "", + "input": "2", "output": "10", "cache_read_input": "0.2", "cache_write_input": "0", @@ -895,7 +1478,7 @@ "description": "SpaceXAI's intelligent coding model for agentic software, engineering, and workflow tasks. Trained in SpaceXAI's data centers in Memphis with new datasets spanning science, engineering, and math.", "status": "active", "listing": { - "order": 11, + "order": 22, "featured": false, "snapshot": false }, @@ -962,7 +1545,7 @@ } }, "pricing": { - "input": "", + "input": "2", "output": "6", "cache_read_input": "0.3", "cache_write_input": "0", @@ -985,10 +1568,10 @@ "provider_model": "gpt-5.6-luna", "display_name": "GPT-5.6 Luna", "provider_display_name": "OpenAI", - "description": "GPT-5.6 Luna is designed for cost-sensitive, high-volume workloads. It roughly corresponds to the nano model tier used in earlier GPT-5 families.", + "description": "GPT-5.6 Luna is designed for cost-sensitive, high-volume workloads. It roughly corresponds to the nano model tier used in earlier GPT-5 families. Reasoning.effort supports: none, low, medium (default), high, xhigh, and max.", "status": "active", "listing": { - "order": 12, + "order": 23, "featured": true, "snapshot": false }, @@ -1071,7 +1654,7 @@ } }, "pricing": { - "input": "", + "input": "0.2", "output": "1.2", "cache_read_input": "0.02", "cache_write_input": "0.25", @@ -1094,10 +1677,10 @@ "provider_model": "gpt-5.6-terra", "display_name": "GPT-5.6 Terra", "provider_display_name": "OpenAI", - "description": "GPT-5.6 Terra is designed for workloads that balance intelligence and cost. It roughly corresponds to the mini model tier used in earlier GPT-5 families.", + "description": "GPT-5.6 Terra is designed for workloads that balance intelligence and cost. It roughly corresponds to the mini model tier used in earlier GPT-5 families. Reasoning.effort supports: none, low, medium (default), high, xhigh, and max.", "status": "active", "listing": { - "order": 13, + "order": 24, "featured": false, "snapshot": false }, @@ -1180,7 +1763,7 @@ } }, "pricing": { - "input": "", + "input": "2", "output": "12", "cache_read_input": "0.2", "cache_write_input": "2.5", @@ -1203,10 +1786,10 @@ "provider_model": "gpt-5.6-sol", "display_name": "GPT-5.6 Sol", "provider_display_name": "OpenAI", - "description": "GPT-5.6 Sol is the frontier model in the GPT-5.6 family. It roughly corresponds to the unsuffixed model tier used in earlier GPT-5 families. The gpt-5.6 alias routes requests to GPT-5.6 Sol.", + "description": "GPT-5.6 Sol is a flagship model in the GPT-5.6 family. It roughly corresponds to the unsuffixed model tier used in earlier GPT-5 families. The gpt-5.6 alias routes requests to GPT-5.6 Sol. Reasoning.effort supports: none, low, medium (default), high, xhigh, and max.", "status": "active", "listing": { - "order": 14, + "order": 25, "featured": true, "snapshot": false }, @@ -1289,7 +1872,7 @@ } }, "pricing": { - "input": "", + "input": "4", "output": "20", "cache_read_input": "0.4", "cache_write_input": "5", @@ -1299,7 +1882,7 @@ } }, { - "id": "accounts/fireworks/models/glm-5p2", + "id": "glm-5p2", "object": "model", "created": 1781568000, "owned_by": "fireworks", @@ -1307,18 +1890,19 @@ "context_length": 1048576, "router": { "schema_version": 1, - "request_name": "accounts/fireworks/models/glm-5p2", + "request_name": "glm-5p2", "catalog_id": "fireworks:accounts/fireworks/models/glm-5p2", "provider_model": "accounts/fireworks/models/glm-5p2", "display_name": "GLM 5.2", "provider_display_name": "Fireworks", "aliases": [ - "glm-5p2" + "accounts/fireworks/models/glm-5p2", + "zai-org/GLM-5.2" ], "description": "GLM-5.2 introduces a robust 1M-token context and advanced, multi-effort coding capabilities to significantly enhance performance on long-horizon tasks. Its new IndexShare architecture and improved MTP layer simultaneously boost efficiency by reducing per-token FLOPs and increasing speculative decoding lengths.", "status": "active", "listing": { - "order": 15, + "order": 26, "featured": true, "snapshot": false }, @@ -1392,7 +1976,7 @@ } }, "pricing": { - "input": "", + "input": "1.4", "output": "4.4", "cache_read_input": "0.14", "cache_write_input": "0", @@ -1402,7 +1986,7 @@ } }, { - "id": "accounts/fireworks/routers/glm-5p2-fast", + "id": "glm-5p2-fast", "object": "model", "created": 1781568000, "owned_by": "fireworks", @@ -1410,15 +1994,19 @@ "context_length": 1048576, "router": { "schema_version": 1, - "request_name": "accounts/fireworks/routers/glm-5p2-fast", + "request_name": "glm-5p2-fast", "catalog_id": "fireworks:accounts/fireworks/routers/glm-5p2-fast", "provider_model": "accounts/fireworks/routers/glm-5p2-fast", "display_name": "GLM 5.2 Fast", "provider_display_name": "Fireworks", + "aliases": [ + "zai-org/GLM-5.2-Fast", + "accounts/fireworks/routers/glm-5p2-fast" + ], "description": "GLM-5.2 introduces a robust 1M-token context and advanced, multi-effort coding capabilities to significantly enhance performance on long-horizon tasks. Its new IndexShare architecture and improved MTP layer simultaneously boost efficiency by reducing per-token FLOPs and increasing speculative decoding lengths.", "status": "active", "listing": { - "order": 17, + "order": 27, "featured": false, "snapshot": false }, @@ -1492,7 +2080,7 @@ } }, "pricing": { - "input": "", + "input": "2.1", "output": "6.6", "cache_read_input": "0.21", "cache_write_input": "0", @@ -1502,7 +2090,7 @@ } }, { - "id": "accounts/fireworks/models/kimi-k2p7-code", + "id": "kimi-k2p7-code", "object": "model", "created": 1781222400, "owned_by": "fireworks", @@ -1510,15 +2098,19 @@ "context_length": 262144, "router": { "schema_version": 1, - "request_name": "accounts/fireworks/models/kimi-k2p7-code", + "request_name": "kimi-k2p7-code", "catalog_id": "fireworks:accounts/fireworks/models/kimi-k2p7-code", "provider_model": "accounts/fireworks/models/kimi-k2p7-code", "display_name": "Kimi K2.7 Code", "provider_display_name": "Fireworks", + "aliases": [ + "moonshotai/Kimi-K2.7-Code", + "accounts/fireworks/models/kimi-k2p7-code" + ], "description": "Kimi K2.7 Code is a coding-focused agentic model built upon Kimi K2.6. With substantial improvements on real-world long-horizon coding tasks, it strengthens end-to-end task completion across complex software engineering workflows while improving token efficiency, reducing thinking-token usage by approximately 30% compared with Kimi K2.6.", "status": "active", "listing": { - "order": 18, + "order": 30, "featured": false, "snapshot": false }, @@ -1589,7 +2181,7 @@ } }, "pricing": { - "input": "", + "input": "0.95", "output": "4", "cache_read_input": "0.19", "cache_write_input": "0", @@ -1599,7 +2191,7 @@ } }, { - "id": "accounts/fireworks/models/minimax-m3", + "id": "minimax-m3", "object": "model", "created": 1781136000, "owned_by": "fireworks", @@ -1607,26 +2199,126 @@ "context_length": 512000, "router": { "schema_version": 1, - "request_name": "accounts/fireworks/models/minimax-m3", - "catalog_id": "fireworks:accounts/fireworks/models/minimax-m3", - "provider_model": "accounts/fireworks/models/minimax-m3", - "display_name": "Minimax M3", - "provider_display_name": "Fireworks", - "description": "MiniMax-M3 is a native multimodal model with 1M context running ~428B parameters and ~23B activated parameters. It brings native multimodality. enabling deeper semantic fusion across text, image, and video. M3 also introduces MiniMax Sparse Attention (MSA) to improve long context efficiency, achieving frontier-level performance across long-horizon agentic benchmarks, excelling in both coding and cowork.", + "request_name": "minimax-m3", + "catalog_id": "fireworks:accounts/fireworks/models/minimax-m3", + "provider_model": "accounts/fireworks/models/minimax-m3", + "display_name": "Minimax M3", + "provider_display_name": "Fireworks", + "aliases": [ + "accounts/fireworks/models/minimax-m3" + ], + "description": "MiniMax-M3 is a native multimodal model with 1M context running ~428B parameters and ~23B activated parameters. It brings native multimodality. enabling deeper semantic fusion across text, image, and video. M3 also introduces MiniMax Sparse Attention (MSA) to improve long context efficiency, achieving frontier-level performance across long-horizon agentic benchmarks, excelling in both coding and cowork.", + "status": "active", + "listing": { + "order": 32, + "featured": false, + "snapshot": false + }, + "limits": { + "context_window": 512000, + "max_input_tokens": null, + "max_output_tokens": 512000 + }, + "capabilities": { + "modalities": { + "input": [ + "" + ], + "output": [ + "text" + ] + }, + "tools": { + "supported": true, + "parallel": null + }, + "structured_outputs": true, + "temperature": true, + "prompt_caching": true, + "reasoning": { + "supported": true, + "efforts": [ + { + "value": "none", + "description": "Answers without reasoning first" + }, + { + "value": "minimal", + "description": "Reasons briefly before answering" + }, + { + "value": "low", + "description": "Fast responses with lighter reasoning" + }, + { + "value": "medium", + "description": "Balances speed and reasoning depth" + }, + { + "value": "high", + "description": "Greater reasoning depth" + }, + { + "value": "xhigh", + "description": "Reasons longer than high" + } + ], + "default_effort": "none", + "summary": { + "output_supported": true, + "request_parameter_supported": false, + "values": [] + }, + "continuation": { + "request_include": [], + "required": false + } + }, + "verbosity": { + "supported": false, + "values": [] + } + }, + "pricing": { + "input": "0.3", + "output": "1.2", + "cache_read_input": "0.06", + "cache_write_input": "0", + "cache_write_input_5m": "0", + "cache_write_input_1h": "0" + } + } + }, + { + "id": "claude-fable-5", + "object": "model", + "created": 1780790400, + "owned_by": "anthropic", + "display_name": "Claude Fable 5", + "context_length": 1000000, + "router": { + "schema_version": 1, + "request_name": "claude-fable-5", + "catalog_id": "anthropic:claude-fable-5", + "provider_model": "claude-fable-5", + "display_name": "Claude Fable 5", + "provider_display_name": "Anthropic", + "description": "Next-generation intelligence for long-running agents", "status": "active", "listing": { - "order": 20, + "order": 34, "featured": false, "snapshot": false }, "limits": { - "context_window": 512000, - "max_input_tokens": null, - "max_output_tokens": 512000 + "context_window": 1000000, + "max_input_tokens": 1000000, + "max_output_tokens": 128000 }, "capabilities": { "modalities": { "input": [ + "", "" ], "output": [ @@ -1643,10 +2335,6 @@ "reasoning": { "supported": true, "efforts": [ - { - "value": "none", - "description": "Answers without reasoning first" - }, { "value": "minimal", "description": "Reasons briefly before answering" @@ -1666,9 +2354,13 @@ { "value": "xhigh", "description": "Reasons longer than high" + }, + { + "value": "max", + "description": "Reasons as long as the model allows" } ], - "default_effort": "none", + "default_effort": "high", "summary": { "output_supported": true, "request_parameter_supported": false, @@ -1685,45 +2377,48 @@ } }, "pricing": { - "input": "", - "output": "1.2", - "cache_read_input": "0.06", + "input": "10", + "output": "50", + "cache_read_input": "1", "cache_write_input": "0", - "cache_write_input_5m": "0", - "cache_write_input_1h": "0" + "cache_write_input_5m": "12.5", + "cache_write_input_1h": "20" } } }, { - "id": "claude-fable-5", + "id": "nemotron-3-ultra", "object": "model", - "created": 1780790400, - "owned_by": "anthropic", - "display_name": "Claude Fable 5", - "context_length": 1000000, + "created": 1780358400, + "owned_by": "fireworks", + "display_name": "NVIDIA Nemotron 3 Ultra NVFP4", + "context_length": 262144, "router": { "schema_version": 1, - "request_name": "claude-fable-5", - "catalog_id": "anthropic:claude-fable-5", - "provider_model": "claude-fable-5", - "display_name": "Claude Fable 5", - "provider_display_name": "Anthropic", - "description": "Next-generation intelligence for long-running agents", + "request_name": "nemotron-3-ultra", + "catalog_id": "fireworks:accounts/fireworks/models/nemotron-3-ultra-nvfp4", + "provider_model": "accounts/fireworks/models/nemotron-3-ultra-nvfp4", + "display_name": "NVIDIA Nemotron 3 Ultra NVFP4", + "provider_display_name": "Fireworks", + "aliases": [ + "nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B", + "accounts/fireworks/models/nemotron-3-ultra-nvfp4" + ], + "description": "Nemotron-3-Ultra-550B-A55B-NVFP4 is a frontier-scale large language model (LLM) trained by NVIDIA, designed to deliver strong agentic, reasoning, and conversational capabilities. It is optimized for the most demanding workloads, including complex multi-step agents, long-context analysis, and high-accuracy reasoning over code, math, and science. The model employs a hybrid Latent Mixture-of-Experts (LatentMoE) architecture, utilizing interleaved Mamba-2 and MoE layers, along with select Attention layers. Like the Super model, the Ultra model incorporates Multi-Token Prediction (MTP) layers for faster text generation and improved quality, and it is trained using an NVFP4 pre-training recipe to maximize compute efficiency. The model has 55B active parameters and 550B parameters in total.", "status": "active", "listing": { - "order": 22, + "order": 36, "featured": false, "snapshot": false }, "limits": { - "context_window": 1000000, - "max_input_tokens": 1000000, - "max_output_tokens": 128000 + "context_window": 262144, + "max_input_tokens": null, + "max_output_tokens": 262144 }, "capabilities": { "modalities": { "input": [ - "", "" ], "output": [ @@ -1740,6 +2435,10 @@ "reasoning": { "supported": true, "efforts": [ + { + "value": "none", + "description": "Answers without reasoning first" + }, { "value": "minimal", "description": "Reasons briefly before answering" @@ -1759,13 +2458,9 @@ { "value": "xhigh", "description": "Reasons longer than high" - }, - { - "value": "max", - "description": "Reasons as long as the model allows" } ], - "default_effort": "high", + "default_effort": "none", "summary": { "output_supported": true, "request_parameter_supported": false, @@ -1782,17 +2477,17 @@ } }, "pricing": { - "input": "", - "output": "50", - "cache_read_input": "1", + "input": "0.6", + "output": "2.4", + "cache_read_input": "0.12", "cache_write_input": "0", - "cache_write_input_5m": "12.5", - "cache_write_input_1h": "20" + "cache_write_input_5m": "0", + "cache_write_input_1h": "0" } } }, { - "id": "accounts/fireworks/models/nemotron-3-ultra-nvfp4", + "id": "nemotron-3-ultra-nvfp4", "object": "model", "created": 1780358400, "owned_by": "fireworks", @@ -1800,7 +2495,7 @@ "context_length": 262144, "router": { "schema_version": 1, - "request_name": "accounts/fireworks/models/nemotron-3-ultra-nvfp4", + "request_name": "nemotron-3-ultra-nvfp4", "catalog_id": "fireworks:accounts/fireworks/models/nemotron-3-ultra-nvfp4", "provider_model": "accounts/fireworks/models/nemotron-3-ultra-nvfp4", "display_name": "NVIDIA Nemotron 3 Ultra NVFP4", @@ -1808,7 +2503,7 @@ "description": "Nemotron-3-Ultra-550B-A55B-NVFP4 is a frontier-scale large language model (LLM) trained by NVIDIA, designed to deliver strong agentic, reasoning, and conversational capabilities. It is optimized for the most demanding workloads, including complex multi-step agents, long-context analysis, and high-accuracy reasoning over code, math, and science. The model employs a hybrid Latent Mixture-of-Experts (LatentMoE) architecture, utilizing interleaved Mamba-2 and MoE layers, along with select Attention layers. Like the Super model, the Ultra model incorporates Multi-Token Prediction (MTP) layers for faster text generation and improved quality, and it is trained using an NVFP4 pre-training recipe to maximize compute efficiency. The model has 55B active parameters and 550B parameters in total.", "status": "active", "listing": { - "order": 23, + "order": 36, "featured": false, "snapshot": false }, @@ -1878,7 +2573,7 @@ } }, "pricing": { - "input": "", + "input": "0.6", "output": "2.4", "cache_read_input": "0.12", "cache_write_input": "0", @@ -1906,7 +2601,7 @@ ], "status": "active", "listing": { - "order": 24, + "order": 37, "featured": false, "snapshot": false }, @@ -1981,7 +2676,7 @@ } }, "pricing": { - "input": "", + "input": "5", "output": "25", "cache_read_input": "0.5", "cache_write_input": "0", @@ -1991,7 +2686,86 @@ } }, { - "id": "accounts/fireworks/models/deepseek-v4-flash", + "id": "deepseek-v4-pro", + "object": "model", + "created": 1777075200, + "owned_by": "baseten", + "display_name": "DeepSeek V4 Pro (Baseten)", + "context_length": 1048576, + "router": { + "schema_version": 1, + "request_name": "deepseek-v4-pro", + "catalog_id": "baseten:deepseek-ai/DeepSeek-V4-Pro", + "provider_model": "deepseek-ai/DeepSeek-V4-Pro", + "display_name": "DeepSeek V4 Pro (Baseten)", + "provider_display_name": "Baseten", + "aliases": [ + "deepseek-ai/DeepSeek-V4-Pro" + ], + "description": "DeepSeek V4 Pro served through Baseten Model APIs.", + "status": "active", + "listing": { + "order": 41, + "featured": false, + "snapshot": false + }, + "limits": { + "context_window": 1048576, + "max_input_tokens": null, + "max_output_tokens": 384000 + }, + "capabilities": { + "modalities": { + "input": [ + "" + ], + "output": [ + "text" + ] + }, + "tools": { + "supported": true, + "parallel": null + }, + "structured_outputs": true, + "temperature": true, + "prompt_caching": true, + "reasoning": { + "supported": true, + "efforts": [ + { + "value": "high", + "description": "Greater reasoning depth" + } + ], + "default_effort": "high", + "summary": { + "output_supported": true, + "request_parameter_supported": false, + "values": [] + }, + "continuation": { + "request_include": [], + "required": false + } + }, + "verbosity": { + "supported": false, + "values": [] + } + }, + "pricing": { + "input": "1.74", + "output": "3.48", + "cache_read_input": "0.145", + "cache_write_input": "0", + "cache_write_input_5m": "0", + "cache_write_input_1h": "0" + } + } + }, + { + "id": "deepseek-v4-flash", "object": "model", "created": 1776988800, "owned_by": "fireworks", @@ -1999,15 +2773,18 @@ "context_length": 1048576, "router": { "schema_version": 1, - "request_name": "accounts/fireworks/models/deepseek-v4-flash", + "request_name": "deepseek-v4-flash", "catalog_id": "fireworks:accounts/fireworks/models/deepseek-v4-flash", "provider_model": "accounts/fireworks/models/deepseek-v4-flash", "display_name": "DeepSeek-V4-Flash", "provider_display_name": "Fireworks", + "aliases": [ + "accounts/fireworks/models/deepseek-v4-flash" + ], "description": "DeepSeek-V4-Flash is a streamlined open-source Mixture-of-Experts model optimized for fast, cost-efficient inference while preserving strong reasoning and coding performance at 1M token context scale. It leverages the same hybrid attention innovations as Pro but is tuned for lower latency and higher throughput in real-time applications. It delivers near-Pro reasoning quality under sufficient compute budget, making it ideal for interactive agents and high-volume production workloads.", "status": "active", "listing": { - "order": 28, + "order": 42, "featured": true, "snapshot": false }, @@ -2081,7 +2858,7 @@ } }, "pricing": { - "input": "", + "input": "0.14", "output": "0.28", "cache_read_input": "0.028", "cache_write_input": "0", @@ -2107,7 +2884,7 @@ "description": "GPT-5.5 Pro uses more compute to think harder and provide consistently better answers.", "status": "active", "listing": { - "order": 31, + "order": 45, "featured": false, "snapshot": false }, @@ -2178,7 +2955,7 @@ } }, "pricing": { - "input": "", + "input": "30", "output": "180", "cache_read_input": "0", "cache_write_input": "0", @@ -2201,10 +2978,10 @@ "provider_model": "gpt-5.5", "display_name": "GPT-5.5", "provider_display_name": "OpenAI", - "description": "GPT-5.5 is our newest frontier model for the most complex professional work. Learn more in our GPT-5.5 model guidance . Reasoning.effort supports: none, low, medium (default), high and xhigh.", + "description": "GPT-5.5 is a flagship model for the most complex professional work. Learn more in our GPT-5.5 model guidance . Reasoning.effort supports: none, low, medium (default), high and xhigh.", "status": "active", "listing": { - "order": 33, + "order": 47, "featured": false, "snapshot": false }, @@ -2283,7 +3060,7 @@ } }, "pricing": { - "input": "", + "input": "5", "output": "30", "cache_read_input": "0.5", "cache_write_input": "0", @@ -2293,7 +3070,7 @@ } }, { - "id": "accounts/fireworks/models/kimi-k2p6", + "id": "kimi-k2p6", "object": "model", "created": 1776384000, "owned_by": "fireworks", @@ -2301,15 +3078,19 @@ "context_length": 262144, "router": { "schema_version": 1, - "request_name": "accounts/fireworks/models/kimi-k2p6", + "request_name": "kimi-k2p6", "catalog_id": "fireworks:accounts/fireworks/models/kimi-k2p6", "provider_model": "accounts/fireworks/models/kimi-k2p6", "display_name": "Kimi K2.6", "provider_display_name": "Fireworks", + "aliases": [ + "moonshotai/Kimi-K2.6", + "accounts/fireworks/models/kimi-k2p6" + ], "description": "Kimi K2.6 is an open-source, native multimodal agentic model that advances practical capabilities in long-horizon coding, coding-driven design, proactive autonomous execution, and swarm-based task orchestration.", "status": "active", "listing": { - "order": 34, + "order": 49, "featured": false, "snapshot": false }, @@ -2380,7 +3161,7 @@ } }, "pricing": { - "input": "", + "input": "0.95", "output": "4", "cache_read_input": "0.16", "cache_write_input": "0", @@ -2406,7 +3187,7 @@ "description": "Fast, reliable model with strong tool calling and instruction following capabilities.", "status": "active", "listing": { - "order": 36, + "order": 51, "featured": false, "snapshot": false }, @@ -2477,7 +3258,7 @@ } }, "pricing": { - "input": "", + "input": "1.25", "output": "2.5", "cache_read_input": "0.2", "cache_write_input": "0", @@ -2503,7 +3284,7 @@ "description": "SpaceXAI's intelligent coding model for agentic software, engineering, and workflow tasks.", "status": "active", "listing": { - "order": 37, + "order": 52, "featured": false, "snapshot": false }, @@ -2548,7 +3329,7 @@ } }, "pricing": { - "input": "", + "input": "1", "output": "2", "cache_read_input": "0.2", "cache_write_input": "0", @@ -2573,7 +3354,7 @@ "provider_display_name": "Anthropic", "status": "active", "listing": { - "order": 38, + "order": 53, "featured": false, "snapshot": false }, @@ -2648,7 +3429,7 @@ } }, "pricing": { - "input": "", + "input": "5", "output": "25", "cache_read_input": "0.5", "cache_write_input": "0", @@ -2662,22 +3443,22 @@ "object": "model", "created": 1773451123, "owned_by": "openai", - "display_name": "GPT-5.4 mini", + "display_name": "GPT-5.4 Mini", "context_length": 400000, "router": { "schema_version": 1, "request_name": "gpt-5.4-mini", "catalog_id": "openai:gpt-5.4-mini", "provider_model": "gpt-5.4-mini", - "display_name": "GPT-5.4 mini", + "display_name": "GPT-5.4 Mini", "provider_display_name": "OpenAI", "aliases": [ "fast-default" ], - "description": "GPT-5.4 mini brings the strengths of GPT-5.4 to a faster, more efficient model designed for high-volume workloads. Learn more in our Model guidance page. Reasoning.effort supports: none (default), low, medium, high and xhigh.", + "description": "GPT-5.4 Mini brings the strengths of GPT-5.4 to a faster, more efficient model designed for high-volume workloads. Learn more in our Model guidance page. Reasoning.effort supports: none (default), low, medium, high and xhigh.", "status": "active", "listing": { - "order": 40, + "order": 55, "featured": false, "snapshot": false }, @@ -2756,7 +3537,7 @@ } }, "pricing": { - "input": "", + "input": "0.75", "output": "4.5", "cache_read_input": "0.075", "cache_write_input": "0", @@ -2782,7 +3563,7 @@ "description": "GPT-5.4 nano is designed for tasks where speed and cost matter most like classification, data extraction, ranking, and sub-agents. Learn more in our Model guidance page. Reasoning.effort supports: none (default), low, medium, high and xhigh.", "status": "active", "listing": { - "order": 43, + "order": 58, "featured": false, "snapshot": false }, @@ -2861,7 +3642,7 @@ } }, "pricing": { - "input": "", + "input": "0.2", "output": "1.25", "cache_read_input": "0.02", "cache_write_input": "0", @@ -2887,7 +3668,7 @@ "description": "Grok 4.20 is a high-performance model with industry-leading speed and agentic tool calling capabilities. It combines the lowest hallucination rate on the market with strict prompt adherence, delivering consistently precise and truthful responses.", "status": "active", "listing": { - "order": 45, + "order": 60, "featured": false, "snapshot": false }, @@ -2932,7 +3713,7 @@ } }, "pricing": { - "input": "", + "input": "1.25", "output": "2.5", "cache_read_input": "0.2", "cache_write_input": "0", @@ -2958,7 +3739,7 @@ "description": "Grok 4.20 is a high-performance model with industry-leading speed and agentic tool calling capabilities. It combines the lowest hallucination rate on the market with strict prompt adherence, delivering consistently precise and truthful responses.", "status": "active", "listing": { - "order": 46, + "order": 61, "featured": false, "snapshot": false }, @@ -3003,7 +3784,7 @@ } }, "pricing": { - "input": "", + "input": "1.25", "output": "2.5", "cache_read_input": "0.2", "cache_write_input": "0", @@ -3029,7 +3810,7 @@ "description": "Multiple agents collaborate in parallel to perform deep research tasks.", "status": "active", "listing": { - "order": 47, + "order": 62, "featured": false, "snapshot": false }, @@ -3104,7 +3885,7 @@ } }, "pricing": { - "input": "", + "input": "1.25", "output": "2.5", "cache_read_input": "0.2", "cache_write_input": "0", @@ -3127,10 +3908,10 @@ "provider_model": "gpt-5.4", "display_name": "GPT-5.4", "provider_display_name": "OpenAI", - "description": "GPT-5.4 is our frontier model for complex professional work. Learn more in our GPT-5.4 model guidance . Reasoning.effort supports: none (default), low, medium, high and xhigh.", + "description": "GPT-5.4 is a flagship model for complex professional work. Learn more in our GPT-5.4 model guidance . Reasoning.effort supports: none (default), low, medium, high and xhigh.", "status": "active", "listing": { - "order": 48, + "order": 63, "featured": false, "snapshot": false }, @@ -3209,7 +3990,7 @@ } }, "pricing": { - "input": "", + "input": "2.5", "output": "15", "cache_read_input": "0.25", "cache_write_input": "0", @@ -3235,7 +4016,7 @@ "description": "GPT-5.4 Pro uses more compute to think harder and provide consistently better answers.", "status": "active", "listing": { - "order": 50, + "order": 65, "featured": false, "snapshot": false }, @@ -3306,7 +4087,7 @@ } }, "pricing": { - "input": "", + "input": "30", "output": "180", "cache_read_input": "0", "cache_write_input": "0", @@ -3334,7 +4115,7 @@ ], "status": "active", "listing": { - "order": 52, + "order": 67, "featured": false, "snapshot": false }, @@ -3409,7 +4190,7 @@ } }, "pricing": { - "input": "", + "input": "3", "output": "15", "cache_read_input": "0.3", "cache_write_input": "0", @@ -3435,7 +4216,7 @@ "description": "GPT-5.3-Codex is optimized for agentic coding tasks in Codex or similar environments. GPT-5.3-Codex supports low , medium , high , and xhigh reasoning effort settings. If you want to learn more about prompting GPT-5.3-Codex, refer to our dedicated guide .", "status": "active", "listing": { - "order": 54, + "order": 69, "featured": false, "snapshot": false }, @@ -3514,7 +4295,7 @@ } }, "pricing": { - "input": "", + "input": "1.75", "output": "14", "cache_read_input": "0.175", "cache_write_input": "0", @@ -3539,7 +4320,7 @@ "provider_display_name": "Anthropic", "status": "active", "listing": { - "order": 55, + "order": 70, "featured": false, "snapshot": false }, @@ -3614,7 +4395,7 @@ } }, "pricing": { - "input": "", + "input": "5", "output": "25", "cache_read_input": "0.5", "cache_write_input": "0", @@ -3623,6 +4404,79 @@ } } }, + { + "id": "glm-4p7", + "object": "model", + "created": 1766448000, + "owned_by": "baseten", + "display_name": "GLM 4.7 (Baseten)", + "context_length": 200000, + "router": { + "schema_version": 1, + "request_name": "glm-4p7", + "catalog_id": "baseten:zai-org/GLM-4.7", + "provider_model": "zai-org/GLM-4.7", + "display_name": "GLM 4.7 (Baseten)", + "provider_display_name": "Baseten", + "aliases": [ + "zai-org/GLM-4.7" + ], + "description": "GLM 4.7 served through Baseten Model APIs.", + "status": "active", + "listing": { + "order": 71, + "featured": false, + "snapshot": false + }, + "limits": { + "context_window": 200000, + "max_input_tokens": null, + "max_output_tokens": 200000 + }, + "capabilities": { + "modalities": { + "input": [ + "" + ], + "output": [ + "text" + ] + }, + "tools": { + "supported": true, + "parallel": null + }, + "structured_outputs": true, + "temperature": true, + "prompt_caching": false, + "reasoning": { + "supported": false, + "efforts": [], + "summary": { + "output_supported": null, + "request_parameter_supported": null, + "values": [] + }, + "continuation": { + "request_include": [], + "required": null + } + }, + "verbosity": { + "supported": false, + "values": [] + } + }, + "pricing": { + "input": "0.6", + "output": "2.2", + "cache_read_input": "0.6", + "cache_write_input": "0", + "cache_write_input_5m": "0", + "cache_write_input_1h": "0" + } + } + }, { "id": "gpt-5.2-pro", "object": "model", @@ -3640,7 +4494,7 @@ "description": "GPT-5.2 Pro is our previous pro model for complex professional work. We recommend using GPT-5.5 Pro for the latest pro model. GPT-5.2 Pro is available in the Responses API only to enable support for multi-turn model interactions before responding to API requests, and other advanced API features in the future. Since GPT-5.2 Pro is designed to tackle tough problems, some requests may take several minutes to finish. To avoid timeouts, try using background mode. GPT-5.2 Pro supports reasoning.effort: medium, high, xhigh.", "status": "active", "listing": { - "order": 56, + "order": 72, "featured": false, "snapshot": false }, @@ -3711,7 +4565,7 @@ } }, "pricing": { - "input": "", + "input": "21", "output": "168", "cache_read_input": "0", "cache_write_input": "0", @@ -3734,10 +4588,10 @@ "provider_model": "gpt-5.2", "display_name": "GPT-5.2", "provider_display_name": "OpenAI", - "description": "GPT-5.2 is our previous frontier model for complex professional work. We recommend using the latest GPT-5.6 . Learn more on the Model guidance page. Reasoning.effort supports: none (default), low, medium, high and xhigh.", + "description": "GPT-5.2 is our previous flagship model for complex professional work. We recommend using the latest GPT-6 Astra . Learn more on the Model guidance page. Reasoning.effort supports: none (default), low, medium, high and xhigh.", "status": "active", "listing": { - "order": 58, + "order": 74, "featured": false, "snapshot": false }, @@ -3816,7 +4670,7 @@ } }, "pricing": { - "input": "", + "input": "1.75", "output": "14", "cache_read_input": "0.175", "cache_write_input": "0", @@ -3842,7 +4696,7 @@ "description": "GPT-5.1 is our flagship model for coding and agentic tasks with configurable reasoning and non-reasoning effort. Learn more in our GPT-5.1 model guidance . Reasoning.effort supports: none (default), low, medium, and high.", "status": "active", "listing": { - "order": 61, + "order": 77, "featured": false, "snapshot": false }, @@ -3917,7 +4771,7 @@ } }, "pricing": { - "input": "", + "input": "1.25", "output": "10", "cache_read_input": "0.125", "cache_write_input": "0", @@ -3943,7 +4797,7 @@ "description": "GPT-5 Pro uses more compute to think harder and provide consistently better answers.", "status": "active", "listing": { - "order": 65, + "order": 81, "featured": false, "snapshot": false }, @@ -4006,7 +4860,7 @@ } }, "pricing": { - "input": "", + "input": "15", "output": "120", "cache_read_input": "0", "cache_write_input": "0", @@ -4032,7 +4886,7 @@ "description": "GPT-5 Nano is our fastest, cheapest version of GPT-5. It's great for summarization and classification tasks. For most new speed- and cost-sensitive workloads, we recommend starting with GPT-5.6 Luna . Learn more in our Model guidance page.", "status": "active", "listing": { - "order": 69, + "order": 85, "featured": false, "snapshot": false }, @@ -4107,7 +4961,7 @@ } }, "pricing": { - "input": "", + "input": "0.05", "output": "0.4", "cache_read_input": "0.005", "cache_write_input": "0", @@ -4121,19 +4975,19 @@ "object": "model", "created": 1754425928, "owned_by": "openai", - "display_name": "GPT-5 mini", + "display_name": "GPT-5 Mini", "context_length": 400000, "router": { "schema_version": 1, "request_name": "gpt-5-mini", "catalog_id": "openai:gpt-5-mini", "provider_model": "gpt-5-mini", - "display_name": "GPT-5 mini", + "display_name": "GPT-5 Mini", "provider_display_name": "OpenAI", - "description": "GPT-5 mini is a faster, more cost-efficient version of GPT-5. It's great for well-defined tasks and precise prompts. For most new low-latency, high-volume workloads, we recommend starting with GPT-5.6 Terra .", + "description": "GPT-5 Mini is a faster, more cost-efficient version of GPT-5. It's great for well-defined tasks and precise prompts. For most new low-latency, high-volume workloads, we recommend starting with GPT-5.6 Terra .", "status": "active", "listing": { - "order": 71, + "order": 87, "featured": false, "snapshot": false }, @@ -4208,7 +5062,7 @@ } }, "pricing": { - "input": "", + "input": "0.25", "output": "2", "cache_read_input": "0.025", "cache_write_input": "0", @@ -4231,10 +5085,10 @@ "provider_model": "gpt-5", "display_name": "GPT-5", "provider_display_name": "OpenAI", - "description": "GPT-5 is our previous model for coding, reasoning, and agentic tasks across domains. We recommend using the latest GPT-5.6 . Learn more on the Model guidance page. Reasoning.effort supports: minimal, low, medium, and high.", + "description": "GPT-5 is our previous model for coding, reasoning, and agentic tasks across domains. We recommend using the latest GPT-6 Astra . Learn more on the Model guidance page. Reasoning.effort supports: minimal, low, medium, and high.", "status": "active", "listing": { - "order": 73, + "order": 89, "featured": false, "snapshot": false }, @@ -4309,7 +5163,7 @@ } }, "pricing": { - "input": "", + "input": "1.25", "output": "10", "cache_read_input": "0.125", "cache_write_input": "0", @@ -4319,7 +5173,7 @@ } }, { - "id": "accounts/fireworks/models/gpt-oss-120b", + "id": "gpt-oss-120b", "object": "model", "created": 1754265600, "owned_by": "fireworks", @@ -4327,15 +5181,19 @@ "context_length": 131072, "router": { "schema_version": 1, - "request_name": "accounts/fireworks/models/gpt-oss-120b", + "request_name": "gpt-oss-120b", "catalog_id": "fireworks:accounts/fireworks/models/gpt-oss-120b", "provider_model": "accounts/fireworks/models/gpt-oss-120b", "display_name": "OpenAI gpt-oss-120b", "provider_display_name": "Fireworks", + "aliases": [ + "openai/gpt-oss-120b", + "accounts/fireworks/models/gpt-oss-120b" + ], "description": "Welcome to the gpt-oss series, OpenAI's open-weight models designed for powerful reasoning, agentic tasks, and versatile developer use cases. gpt-oss-120b is used for production, general purpose, high reasoning use-cases that fits into a single H100 GPU.", "status": "active", "listing": { - "order": 74, + "order": 91, "featured": false, "snapshot": false }, @@ -4401,7 +5259,7 @@ } }, "pricing": { - "input": "", + "input": "0.15", "output": "0.6", "cache_read_input": "0.015", "cache_write_input": "0", @@ -4427,7 +5285,7 @@ "description": "The o-series of models are trained with reinforcement learning to think before they answer and perform complex reasoning. The o3-pro model uses more compute to think harder and provide consistently better answers.", "status": "active", "listing": { - "order": 81, + "order": 98, "featured": false, "snapshot": false }, @@ -4493,7 +5351,7 @@ } }, "pricing": { - "input": "", + "input": "20", "output": "80", "cache_read_input": "0", "cache_write_input": "0", @@ -4519,7 +5377,7 @@ "description": "GPT-4.1 nano excels at instruction following and tool calling. It features a 1M token context window, and low latency without a reasoning step.", "status": "deprecated", "listing": { - "order": 82, + "order": 99, "featured": false, "snapshot": false }, @@ -4564,7 +5422,7 @@ } }, "pricing": { - "input": "", + "input": "0.1", "output": "0.4", "cache_read_input": "0.025", "cache_write_input": "0", @@ -4578,19 +5436,19 @@ "object": "model", "created": 1744318173, "owned_by": "openai", - "display_name": "GPT-4.1 mini", + "display_name": "GPT-4.1 Mini", "context_length": 1047576, "router": { "schema_version": 1, "request_name": "gpt-4.1-mini", "catalog_id": "openai:gpt-4.1-mini", "provider_model": "gpt-4.1-mini", - "display_name": "GPT-4.1 mini", + "display_name": "GPT-4.1 Mini", "provider_display_name": "OpenAI", - "description": "GPT-4.1 mini excels at instruction following and tool calling. It features a 1M token context window, and low latency without a reasoning step.", + "description": "GPT-4.1 Mini excels at instruction following and tool calling. It features a 1M token context window, and low latency without a reasoning step.", "status": "active", "listing": { - "order": 84, + "order": 101, "featured": false, "snapshot": false }, @@ -4635,7 +5493,7 @@ } }, "pricing": { - "input": "", + "input": "0.4", "output": "1.6", "cache_read_input": "0.1", "cache_write_input": "0", @@ -4661,7 +5519,7 @@ "description": "GPT-4.1 excels at instruction following and tool calling, with broad knowledge across domains. It features a 1M token context window, and low latency without a reasoning step.", "status": "active", "listing": { - "order": 86, + "order": 103, "featured": false, "snapshot": false }, @@ -4706,7 +5564,7 @@ } }, "pricing": { - "input": "", + "input": "2", "output": "8", "cache_read_input": "0.5", "cache_write_input": "0", @@ -4729,10 +5587,10 @@ "provider_model": "o4-mini", "display_name": "o4-mini", "provider_display_name": "OpenAI", - "description": "o4-mini is our latest small o-series model. It's optimized for fast, effective reasoning with exceptionally efficient performance in coding and visual tasks. It's succeeded by GPT-5 mini .", + "description": "o4-mini is our latest small o-series model. It's optimized for fast, effective reasoning with exceptionally efficient performance in coding and visual tasks. It's succeeded by GPT-5 Mini .", "status": "deprecated", "listing": { - "order": 88, + "order": 105, "featured": false, "snapshot": false }, @@ -4798,7 +5656,7 @@ } }, "pricing": { - "input": "", + "input": "1.1", "output": "4.4", "cache_read_input": "0.275", "cache_write_input": "0", @@ -4824,7 +5682,7 @@ "description": "o3 is a well-rounded and powerful model across domains. It sets a new standard for math, science, coding, and visual reasoning tasks. It also excels at technical writing and instruction-following. Use it to think through multi-step problems that involve analysis across text, code, and images.", "status": "active", "listing": { - "order": 89, + "order": 106, "featured": false, "snapshot": false }, @@ -4890,7 +5748,7 @@ } }, "pricing": { - "input": "", + "input": "2", "output": "8", "cache_read_input": "0.5", "cache_write_input": "0", @@ -4916,7 +5774,7 @@ "description": "o3-mini is our newest small reasoning model, providing high intelligence at the same cost and latency targets of o1-mini. o3-mini supports key developer features, like Structured Outputs, function calling, and Batch API.", "status": "deprecated", "listing": { - "order": 94, + "order": 111, "featured": false, "snapshot": false }, @@ -4981,7 +5839,7 @@ } }, "pricing": { - "input": "", + "input": "1.1", "output": "4.4", "cache_read_input": "0.55", "cache_write_input": "0", @@ -5007,7 +5865,7 @@ "description": "The o1 series of models are trained with reinforcement learning to perform complex reasoning. o1 models think before they answer, producing a long internal chain of thought before responding to the user.", "status": "deprecated", "listing": { - "order": 95, + "order": 112, "featured": false, "snapshot": false }, @@ -5073,7 +5931,7 @@ } }, "pricing": { - "input": "", + "input": "15", "output": "60", "cache_read_input": "7.5", "cache_write_input": "0", @@ -5087,19 +5945,19 @@ "object": "model", "created": 1721172741, "owned_by": "openai", - "display_name": "GPT-4o mini", + "display_name": "GPT-4o Mini", "context_length": 128000, "router": { "schema_version": 1, "request_name": "gpt-4o-mini", "catalog_id": "openai:gpt-4o-mini", "provider_model": "gpt-4o-mini", - "display_name": "GPT-4o mini", + "display_name": "GPT-4o Mini", "provider_display_name": "OpenAI", - "description": "GPT-4o mini (“o” for “omni”) is a fast, affordable small model for focused tasks. It accepts both text and image inputs, and produces text outputs (including Structured Outputs). It is ideal for fine-tuning, and model outputs from a larger model like GPT-4o can be distilled to GPT-4o-mini to produce similar results at lower cost and latency.", + "description": "GPT-4o Mini (\u201co\u201d for \u201comni\u201d) is a fast, affordable small model for focused tasks. It accepts both text and image inputs, and produces text outputs (including Structured Outputs). It is ideal for fine-tuning, and model outputs from a larger model like GPT-4o can be distilled to GPT-4o-Mini to produce similar results at lower cost and latency.", "status": "active", "listing": { - "order": 98, + "order": 115, "featured": false, "snapshot": false }, @@ -5144,7 +6002,7 @@ } }, "pricing": { - "input": "", + "input": "0.15", "output": "0.6", "cache_read_input": "0.075", "cache_write_input": "0", @@ -5167,10 +6025,10 @@ "provider_model": "gpt-4o", "display_name": "GPT-4o", "provider_display_name": "OpenAI", - "description": "GPT-4o (“o” for “omni”) is our versatile, high-intelligence flagship model. It accepts both text and image inputs, and produces text outputs (including Structured Outputs). It is the best model for most tasks, and is our most capable model outside of our o-series models.", + "description": "GPT-4o (\u201co\u201d for \u201comni\u201d) is our versatile, high-intelligence flagship model. It accepts both text and image inputs, and produces text outputs (including Structured Outputs). It is the best model for most tasks, and is our most capable model outside of our o-series models.", "status": "active", "listing": { - "order": 100, + "order": 117, "featured": false, "snapshot": false }, @@ -5215,7 +6073,7 @@ } }, "pricing": { - "input": "", + "input": "2.5", "output": "10", "cache_read_input": "1.25", "cache_write_input": "0", @@ -5241,7 +6099,7 @@ "description": "GPT-4 Turbo is the next generation of GPT-4, an older high-intelligence GPT model. It was designed to be a cheaper, better version of GPT-4. Today, we recommend using a newer model like GPT-4o.", "status": "deprecated", "listing": { - "order": 101, + "order": 118, "featured": false, "snapshot": false }, @@ -5286,7 +6144,7 @@ } }, "pricing": { - "input": "", + "input": "10", "output": "30", "cache_read_input": "0", "cache_write_input": "0", @@ -5312,7 +6170,7 @@ "description": "GPT-4 is an older version of a high-intelligence GPT model, usable in Chat Completions.", "status": "deprecated", "listing": { - "order": 102, + "order": 119, "featured": false, "snapshot": false }, @@ -5356,7 +6214,7 @@ } }, "pricing": { - "input": "", + "input": "30", "output": "60", "cache_read_input": "0", "cache_write_input": "0", @@ -5382,7 +6240,7 @@ "description": "GPT-3.5 Turbo models can understand and generate natural language or code and have been optimized for chat using the Chat Completions API but work well for non-chat tasks as well. As of July 2024, use gpt-4o-mini in place of GPT-3.5 Turbo, as it is cheaper, more capable, multimodal, and just as fast. GPT-3.5 Turbo is still available for use in the API.", "status": "deprecated", "listing": { - "order": 103, + "order": 120, "featured": false, "snapshot": false }, @@ -5426,7 +6284,7 @@ } }, "pricing": { - "input": "", + "input": "0.5", "output": "1.5", "cache_read_input": "0", "cache_write_input": "0", @@ -5452,7 +6310,7 @@ "description": "The fastest model with near-frontier intelligence", "status": "active", "listing": { - "order": 104, + "order": 121, "featured": false, "snapshot": false }, @@ -5523,7 +6381,7 @@ } }, "pricing": { - "input": "", + "input": "1", "output": "5", "cache_read_input": "0.1", "cache_write_input": "0", diff --git a/tests/unit/adapters/fixtures/ramp_router/11_real_messages_plain.json b/tests/unit/adapters/fixtures/ramp_router/11_real_messages_plain.json new file mode 100644 index 0000000..5364f5d --- /dev/null +++ b/tests/unit/adapters/fixtures/ramp_router/11_real_messages_plain.json @@ -0,0 +1,52 @@ +{ + "_probe": "P11", + "_question": "does the Anthropic-shaped surface report usage in Anthropic's native shape, service_tier inside usage?", + "_captured": "real capture against a live Ramp Router account", + "_scrubbed": "credentials, emails, IPs, dashboard URLs; prompt and completion text blanked to \"\"", + "_status": 200, + "_headers": { + "date": "Mon, 07 Sep 2026 10:46:49 GMT", + "content-type": "application/json", + "transfer-encoding": "chunked", + "connection": "close", + "request-id": "f5398bf7-e0dd-499b-b4be-904429e3badb", + "vary": "Origin", + "x-request-id": "f5398bf7-e0dd-499b-b4be-904429e3badb", + "x-trace-id": "f5398bf7-e0dd-499b-b4be-904429e3badb", + "cf-cache-status": "DYNAMIC", + "strict-transport-security": "max-age=31536000; includeSubDomains; preload", + "x-content-type-options": "nosniff", + "server": "cloudflare", + "cf-ray": "a37521bb5974fe58-DFW", + "alt-svc": "h3=\":443\"; ma=86400" + }, + "_body": { + "content": [ + { + "text": "", + "type": "text" + } + ], + "id": "msg_011Ceownws8VxtE9rrkSEu3i", + "model": "claude-haiku-4-5-20251001", + "role": "assistant", + "stop_reason": "end_turn", + "stop_sequence": null, + "type": "message", + "usage": { + "cache_creation": { + "ephemeral_1h_input_tokens": 0, + "ephemeral_5m_input_tokens": 0 + }, + "cache_creation_input_tokens": 0, + "cache_read_input_tokens": 0, + "input_tokens": 16, + "output_tokens": 5, + "output_tokens_details": { + "thinking_tokens": 0 + }, + "service_tier": "standard" + } + }, + "_body_was_json": true +} diff --git a/tests/unit/adapters/fixtures/ramp_router/12_real_messages_cache_control_cold.json b/tests/unit/adapters/fixtures/ramp_router/12_real_messages_cache_control_cold.json new file mode 100644 index 0000000..8ea607b --- /dev/null +++ b/tests/unit/adapters/fixtures/ramp_router/12_real_messages_cache_control_cold.json @@ -0,0 +1,52 @@ +{ + "_probe": "P12", + "_question": "does a cold cache_control write report cache_creation_input_tokens, with the 5m/1h split?", + "_captured": "real capture against a live Ramp Router account", + "_scrubbed": "credentials, emails, IPs, dashboard URLs; prompt and completion text blanked to \"\"", + "_status": 200, + "_headers": { + "date": "Mon, 07 Sep 2026 10:46:51 GMT", + "content-type": "application/json", + "transfer-encoding": "chunked", + "connection": "close", + "request-id": "ba8ede4c-7579-433f-9ce2-d5612d4a979a", + "vary": "Origin", + "x-request-id": "ba8ede4c-7579-433f-9ce2-d5612d4a979a", + "x-trace-id": "ba8ede4c-7579-433f-9ce2-d5612d4a979a", + "cf-cache-status": "DYNAMIC", + "strict-transport-security": "max-age=31536000; includeSubDomains; preload", + "x-content-type-options": "nosniff", + "server": "cloudflare", + "cf-ray": "a37521c45d8d2ccc-DFW", + "alt-svc": "h3=\":443\"; ma=86400" + }, + "_body": { + "content": [ + { + "text": "", + "type": "text" + } + ], + "id": "msg_011Ceowo42DH8aK9Jnm6QQPp", + "model": "claude-haiku-4-5-20251001", + "role": "assistant", + "stop_reason": "end_turn", + "stop_sequence": null, + "type": "message", + "usage": { + "cache_creation": { + "ephemeral_1h_input_tokens": 0, + "ephemeral_5m_input_tokens": 7481 + }, + "cache_creation_input_tokens": 7481, + "cache_read_input_tokens": 0, + "input_tokens": 15, + "output_tokens": 5, + "output_tokens_details": { + "thinking_tokens": 0 + }, + "service_tier": "standard" + } + }, + "_body_was_json": true +} diff --git a/tests/unit/adapters/fixtures/ramp_router/13_real_messages_cache_control_warm.json b/tests/unit/adapters/fixtures/ramp_router/13_real_messages_cache_control_warm.json new file mode 100644 index 0000000..ea94c42 --- /dev/null +++ b/tests/unit/adapters/fixtures/ramp_router/13_real_messages_cache_control_warm.json @@ -0,0 +1,52 @@ +{ + "_probe": "P13", + "_question": "warm repeat of P12 \u2014 cache_read_input_tokens beside input_tokens (additive)?", + "_captured": "real capture against a live Ramp Router account", + "_scrubbed": "credentials, emails, IPs, dashboard URLs; prompt and completion text blanked to \"\"", + "_status": 200, + "_headers": { + "date": "Mon, 07 Sep 2026 10:46:52 GMT", + "content-type": "application/json", + "transfer-encoding": "chunked", + "connection": "close", + "request-id": "1462dd72-addb-437d-965b-189cdec82e02", + "vary": "Origin", + "x-request-id": "1462dd72-addb-437d-965b-189cdec82e02", + "x-trace-id": "1462dd72-addb-437d-965b-189cdec82e02", + "cf-cache-status": "DYNAMIC", + "strict-transport-security": "max-age=31536000; includeSubDomains; preload", + "x-content-type-options": "nosniff", + "server": "cloudflare", + "cf-ray": "a37521ccda51d230-DFW", + "alt-svc": "h3=\":443\"; ma=86400" + }, + "_body": { + "content": [ + { + "text": "", + "type": "text" + } + ], + "id": "msg_011Ceowo9aLABKUwtMW3sryo", + "model": "claude-haiku-4-5-20251001", + "role": "assistant", + "stop_reason": "end_turn", + "stop_sequence": null, + "type": "message", + "usage": { + "cache_creation": { + "ephemeral_1h_input_tokens": 0, + "ephemeral_5m_input_tokens": 0 + }, + "cache_creation_input_tokens": 0, + "cache_read_input_tokens": 7481, + "input_tokens": 15, + "output_tokens": 6, + "output_tokens_details": { + "thinking_tokens": 0 + }, + "service_tier": "standard" + } + }, + "_body_was_json": true +} diff --git a/tests/unit/adapters/fixtures/ramp_router/14_real_messages_streamed.json b/tests/unit/adapters/fixtures/ramp_router/14_real_messages_streamed.json new file mode 100644 index 0000000..3d25099 --- /dev/null +++ b/tests/unit/adapters/fixtures/ramp_router/14_real_messages_streamed.json @@ -0,0 +1,95 @@ +{ + "_probe": "P14", + "_question": "where do usage and service_tier land across message_start / message_delta?", + "_captured": "real capture against a live Ramp Router account", + "_scrubbed": "credentials, emails, IPs, dashboard URLs; prompt and completion text blanked to \"\"", + "_status": 200, + "_headers": { + "date": "Mon, 07 Sep 2026 10:46:53 GMT", + "content-type": "text/event-stream", + "transfer-encoding": "chunked", + "connection": "close", + "cache-control": "no-store", + "request-id": "006ebb53-a396-4642-8d08-dd1e51bcf344", + "vary": "Origin", + "x-request-id": "006ebb53-a396-4642-8d08-dd1e51bcf344", + "x-trace-id": "006ebb53-a396-4642-8d08-dd1e51bcf344", + "cf-cache-status": "DYNAMIC", + "strict-transport-security": "max-age=31536000; includeSubDomains; preload", + "x-content-type-options": "nosniff", + "server": "cloudflare", + "cf-ray": "a37521d4fd702c4e-DFW", + "alt-svc": "h3=\":443\"; ma=86400" + }, + "_body": null, + "_body_was_json": false, + "_events": [ + { + "message": { + "content": [], + "id": "msg_011CeowoEXjqGsKxgZ4vhTeL", + "model": "claude-haiku-4-5-20251001", + "role": "assistant", + "stop_reason": null, + "stop_sequence": null, + "type": "message", + "usage": { + "cache_creation": { + "ephemeral_1h_input_tokens": 0, + "ephemeral_5m_input_tokens": 0 + }, + "cache_creation_input_tokens": 0, + "cache_read_input_tokens": 0, + "input_tokens": 16, + "output_tokens": 4, + "service_tier": "standard" + } + }, + "type": "message_start" + }, + { + "content_block": { + "text": "", + "type": "text" + }, + "index": 0, + "type": "content_block_start" + }, + { + "delta": { + "text": "", + "type": "text_delta" + }, + "index": 0, + "type": "content_block_delta" + }, + { + "index": 0, + "type": "content_block_stop" + }, + { + "delta": { + "stop_reason": "end_turn", + "stop_sequence": null + }, + "type": "message_delta", + "usage": { + "cache_creation": { + "ephemeral_1h_input_tokens": 0, + "ephemeral_5m_input_tokens": 0 + }, + "cache_creation_input_tokens": 0, + "cache_read_input_tokens": 0, + "input_tokens": 16, + "output_tokens": 5, + "output_tokens_details": { + "thinking_tokens": 0 + }, + "service_tier": "standard" + } + }, + { + "type": "message_stop" + } + ] +} diff --git a/tests/unit/adapters/fixtures/ramp_router/15_real_messages_openai_served.json b/tests/unit/adapters/fixtures/ramp_router/15_real_messages_openai_served.json new file mode 100644 index 0000000..e8092c9 --- /dev/null +++ b/tests/unit/adapters/fixtures/ramp_router/15_real_messages_openai_served.json @@ -0,0 +1,53 @@ +{ + "_probe": "P15", + "_question": "does a non-Anthropic vendor's usage arrive in Anthropic's additive shape on this surface?", + "_captured": "real capture against a live Ramp Router account", + "_scrubbed": "credentials, emails, IPs, dashboard URLs; prompt and completion text blanked to \"\"", + "_status": 200, + "_headers": { + "date": "Mon, 07 Sep 2026 10:46:54 GMT", + "content-type": "application/json", + "transfer-encoding": "chunked", + "connection": "close", + "request-id": "600e95e4-f964-4a32-9a49-370d2f7dd574", + "vary": "Origin", + "x-request-id": "600e95e4-f964-4a32-9a49-370d2f7dd574", + "x-trace-id": "600e95e4-f964-4a32-9a49-370d2f7dd574", + "cf-cache-status": "DYNAMIC", + "strict-transport-security": "max-age=31536000; includeSubDomains; preload", + "x-content-type-options": "nosniff", + "server": "cloudflare", + "cf-ray": "a37521dc3d76a8b3-DFW", + "alt-svc": "h3=\":443\"; ma=86400" + }, + "_body": { + "content": [ + { + "text": "", + "type": "text" + } + ], + "id": "msg_0c9b66cc00585c74006a9e961e3ad487d1872cedbe0c57efe5", + "model": "gpt-5.4-nano-2026-03-17", + "role": "assistant", + "stop_reason": "end_turn", + "stop_sequence": null, + "type": "message", + "usage": { + "cache_creation": { + "ephemeral_1h_input_tokens": 0, + "ephemeral_5m_input_tokens": 0 + }, + "cache_creation_input_tokens": 0, + "cache_read_input_tokens": 0, + "inference_geo": "us", + "input_tokens": 14, + "output_tokens": 5, + "output_tokens_details": { + "thinking_tokens": 0 + }, + "service_tier": "default" + } + }, + "_body_was_json": true +} diff --git a/tests/unit/fixtures/pricing/money_golden.json b/tests/unit/fixtures/pricing/money_golden.json index 3bf652a..fc70ec4 100644 --- a/tests/unit/fixtures/pricing/money_golden.json +++ b/tests/unit/fixtures/pricing/money_golden.json @@ -241,6 +241,88 @@ "base": "0.00423", "total": "0.00423", "total_cents": "0.423" + }, + { + "name": "ramp_router: OpenAI-served cache write at the catalog write rate (gpt-5.6-luna, 2026-09-07; Router billed 1.1x this \u2014 documented mismatch, not corrected)", + "provider": "ramp_router", + "api": "ramp_router", + "prices": { + "input": "0.0000002", + "output": "0.0000012", + "cache_read": "0.00000002", + "cache_write": "0.00000025" + }, + "counts": { + "input": 4493, + "cache_write": 4490, + "output": 5 + }, + "markup": "1", + "base": "0.0011291", + "total": "0.0011291", + "total_cents": "0.11291" + }, + { + "name": "ramp_router: cache read inside input (grok-build-0.1, dashboard-reconciled 2026-09-04)", + "provider": "ramp_router", + "api": "ramp_router", + "prices": { + "input": "0.000001", + "output": "0.000002", + "cache_read": "0.0000002" + }, + "counts": { + "input": 194, + "cache_read": 192, + "output": 134 + }, + "markup": "1", + "base": "0.0003084", + "total": "0.0003084", + "total_cents": "0.03084" + }, + { + "name": "ramp_router /v1/messages: Anthropic cache write at the 5m TTL rate (claude-haiku-4-5, dashboard-reconciled 2026-09-04)", + "provider": "ramp_router", + "api": "ramp_router_messages", + "prices": { + "input": "0.000001", + "output": "0.000005", + "cache_read": "0.0000001", + "cache_write_5m": "0.00000125", + "cache_write_1h": "0.000002" + }, + "counts": { + "input": 16, + "cache_write": 20113, + "cache_write_5m": 20113, + "output": 5 + }, + "markup": "1", + "base": "0.02518225", + "total": "0.02518225", + "total_cents": "2.518225" + }, + { + "name": "ramp_router /v1/messages: Anthropic cache read beside input (claude-haiku-4-5, dashboard-reconciled 2026-09-04)", + "provider": "ramp_router", + "api": "ramp_router_messages", + "prices": { + "input": "0.000001", + "output": "0.000005", + "cache_read": "0.0000001", + "cache_write_5m": "0.00000125", + "cache_write_1h": "0.000002" + }, + "counts": { + "input": 16, + "cache_read": 20113, + "output": 5 + }, + "markup": "1", + "base": "0.0020523", + "total": "0.0020523", + "total_cents": "0.20523" } ], "precomputed_cases": [ diff --git a/tests/unit/gateway/test_ramp_router.py b/tests/unit/gateway/test_ramp_router.py index 59cde64..2a2f3b4 100644 --- a/tests/unit/gateway/test_ramp_router.py +++ b/tests/unit/gateway/test_ramp_router.py @@ -5,19 +5,26 @@ import json import pathlib import threading +from decimal import Decimal from typing import Any import pytest from lago_agent_sdk import LagoSDK +from lago_agent_sdk.adapters.anthropic_native import RAMP_ROUTER_MESSAGES_API, extract_anthropic_native from lago_agent_sdk.adapters.openai_native import RAMP_ROUTER_PROVIDER, extract_openai_native +from lago_agent_sdk.exceptions import PricingUnavailableError from lago_agent_sdk.pricing import ( TOKEN_BILLED_PROVIDERS, PricingProvider, + lookup_ramp_router, parse_openrouter, + parse_ramp_router, ) -from lago_agent_sdk.token_semantics import KNOWN_PROVIDERS, token_semantics +from lago_agent_sdk.token_semantics import KNOWN_PROVIDERS, OPENAI_SHAPED_APIS, token_semantics +from lago_agent_sdk.wrappers.anthropic import _merge_stream_usage from lago_agent_sdk.wrappers.openai import _provider_hint_for +from lago_agent_sdk.wrappers.ramp_router import client_points_at_ramp_router, is_ramp_router_base_url ROUTER_BASE_URL = "https://api.router.com/v1" @@ -428,29 +435,38 @@ def test_degrades_to_zero_rather_than_throwing_on_malformed_payloads(payload: An # ---------------------------------------------------------------------- -# Price mode. Every Router call currently takes a clean pricing MISS and falls -# back to token events, because no vendor can be assigned to it safely yet. +# Price mode. A Router call prices against Router's OWN catalog — the rate the +# gateway bills, reconciled exact against a live account's dashboard export — +# never against OpenRouter's listing for the "same" model. # -# A real price table is loaded for these tests, and the same model is billed -# both directly and through Router. Without that contrast the tests would pass -# on an empty table, proving nothing: everything misses when nothing is priced. +# The Router table is built through the real parser from the REAL captured +# catalog, and an OpenRouter table listing the same model at a DIFFERENT rate +# is loaded beside it. Without that contrast a test could pass by pricing from +# the wrong table, and a table that silently failed to load would make every +# assertion below vacuous — so the control test prices the same model directly. # ---------------------------------------------------------------------- -PRICED_MODEL = "gpt-5.4-mini" -# Built through the real parser from a real-shaped OpenRouter payload, not from a -# hand-written key: `norm()` rewrites "." to "-", and a test whose table silently fails -# to load proves nothing about a miss. -# -# $0.75/M input and $4.50/M output are Router's own published base rates for this model. +_CATALOG_FIXTURE = ( + pathlib.Path(__file__).parents[1] + / "adapters" + / "fixtures" + / "ramp_router" + / "01_real_models_catalog.json" +) +_ROUTER_TABLE = parse_ramp_router(json.loads(_CATALOG_FIXTURE.read_text())["_body"]) +PRICED_MODEL = "gpt-5.4-nano" # Router's catalog: $0.20/M input, $1.25/M output, $0.02/M cached +SERVED_MODEL = f"{PRICED_MODEL}-2026-03-17" # what Router actually answers with (fixture 02) +# OpenRouter deliberately lists it at a rate that is NOT Router's, so a cost event priced +# from the wrong table shows up in the numbers, not only in `price_source`. _OPENROUTER_TABLE = parse_openrouter( - { - "data": [ - {"id": f"openai/{PRICED_MODEL}", "pricing": {"prompt": "0.00000075", "completion": "0.0000045"}} - ] - } + {"data": [{"id": f"openai/{PRICED_MODEL}", "pricing": {"prompt": "0.000001", "completion": "0.000001"}}]} ) class _StubFetcher: + def __init__(self, router_table: dict[str, Any] | None = None) -> None: + self._router = _ROUTER_TABLE if router_table is None else router_table + self.ramp_router_keys: list[str | None] = [] + def fetch_openrouter(self) -> dict[str, Any]: return _OPENROUTER_TABLE @@ -463,19 +479,40 @@ def fetch_cloudflare_workers_ai(self) -> dict[str, Any]: def fetch_mistral_aliases(self, api_key: str | None = None) -> dict[str, str]: return {} + def fetch_ramp_router(self, api_key: str | None = None) -> dict[str, Any]: + self.ramp_router_keys.append(api_key) + return self._router -def _priced_sdk() -> tuple[LagoSDK, list[dict], PricingProvider]: - provider = PricingProvider(fetcher=_StubFetcher(), ttl_seconds=3600.0) - sdk, received = _new_sdk(pricing_mode="price", pricing_provider=provider) - # The table has to be warm before the call, or the miss under test is just a cold + +def _priced_sdk( + router_table: dict[str, Any] | None = None, on_error: Any = None +) -> tuple[LagoSDK, list[dict], PricingProvider]: + provider = PricingProvider(fetcher=_StubFetcher(router_table), ttl_seconds=3600.0) + config: dict[str, Any] = {"pricing_mode": "price", "pricing_provider": provider} + if on_error is not None: + config["on_error"] = on_error + sdk, received = _new_sdk(**config) + # Both tables have to be warm before the call, or a miss under test is just a cold # cache. `maybe_refresh` is the queue worker's own warm-up, called synchronously. - provider.prime(["openrouter"]) + provider.prime(["ramp_router"]) provider.maybe_refresh() return sdk, received, provider -def test_the_same_model_does_price_when_called_directly_the_table_is_real() -> None: - """The control. If this fails, every "misses" assertion below is vacuous.""" +def _tiered(model: str, tier: str | None = "default", usage: dict[str, Any] | None = None) -> dict[str, Any]: + """A Router response carrying its top-level `service_tier`, as every captured one does.""" + body = router_response(model, usage) + if tier is not None: + body["service_tier"] = tier + return body + + +def _cost_by_type(received: list[dict]) -> dict[str, dict]: + return {e["properties"]["token_type"]: e for e in received if e["code"] == "llm_cost"} + + +def test_the_same_model_priced_directly_comes_from_openrouter_the_control() -> None: + """If this fails, every Router assertion below proves nothing about which table won.""" sdk, received, provider = _priced_sdk() assert provider.lookup("openai", PRICED_MODEL, "responses") is not None client = sdk.wrap(FakeRouterClient("https://api.openai.com/v1", lambda kw: router_response(PRICED_MODEL))) @@ -483,49 +520,188 @@ def test_the_same_model_does_price_when_called_directly_the_table_is_real() -> N assert sdk.flush(timeout=2.0) sdk.shutdown(timeout=1.0) - codes = [e["code"] for e in received] - assert "llm_cost" in codes - assert "llm_input_tokens" not in codes + costs = _cost_by_type(received) + assert costs and all(e["properties"]["price_source"] == "openrouter" for e in costs.values()) + assert costs["input"]["properties"]["unit_price"] == "0.000001" + assert "llm_input_tokens" not in [e["code"] for e in received] -def test_the_identical_model_through_router_misses_and_falls_back_to_token_events() -> None: - """Same table, same model, same usage — only the base URL differs. The miss is - caused by the Router provider vocabulary, which is the decision under test: Router - bills $0 for a BYOK-served request and a non-default tier at a rate its catalog says - "may differ", so a list-price lookup can be flatly wrong.""" +def test_the_identical_model_through_router_prices_from_routers_own_catalog() -> None: + """Same usage, same model family — only the base URL differs — and the money comes + from Router's table: $0.20/M input, not OpenRouter's $1/M.""" sdk, received, _ = _priced_sdk() - client = sdk.wrap(FakeRouterClient(ROUTER_BASE_URL, lambda kw: router_response(f"openai:{PRICED_MODEL}"))) + client = sdk.wrap(FakeRouterClient(ROUTER_BASE_URL, lambda kw: _tiered(SERVED_MODEL))) client.responses.create(model=PRICED_MODEL, input="ping") assert sdk.flush(timeout=2.0) sdk.shutdown(timeout=1.0) + costs = _cost_by_type(received) + assert set(costs) == {"input", "output"} + for e in costs.values(): + assert e["properties"]["price_source"] == "ramp_router" + assert e["properties"]["provider"] == RAMP_ROUTER_PROVIDER + # Billed under the served snapshot, the same row a direct call to it reports. + assert e["properties"]["model"] == SERVED_MODEL + assert costs["input"]["properties"]["unit_price"] == "0.0000002" + assert costs["input"]["properties"]["value"] == "0.0000022" # 11 tokens + assert costs["output"]["properties"]["value"] == "0.00000375" # 3 tokens x $1.25/M + assert "llm_input_tokens" not in [e["code"] for e in received] + + +@pytest.mark.parametrize("tier", ["flex", "priority", "turbo"]) +def test_a_non_default_tier_is_a_named_miss_never_a_multiplied_rate(tier: str) -> None: + """flex measured 0.5x, priority 2.0x, and a tier Router adds later is unknown. None + of them bill at the catalog rate, and the SDK applies no factor of its own: token + events, plus an on_error that says WHICH tier, since the same model priced fine a + moment ago. Decided 2026-09-07.""" + errors: list[tuple[Exception, str]] = [] + sdk, received, _ = _priced_sdk(on_error=lambda exc, where: errors.append((exc, where))) + client = sdk.wrap(FakeRouterClient(ROUTER_BASE_URL, lambda kw: _tiered(SERVED_MODEL, tier))) + client.responses.create(model="x", input="ping") + assert sdk.flush(timeout=2.0) + sdk.shutdown(timeout=1.0) + by_code = _by_code(received) assert "llm_cost" not in by_code # Not a silent drop. The usage is billed, exactly, as tokens. assert by_code["llm_input_tokens"] == 11 assert by_code["llm_output_tokens"] == 3 + misses = [(exc, where) for exc, where in errors if isinstance(exc, PricingUnavailableError)] + assert len(misses) == 1 + exc, where = misses[0] + assert where == "pricing" + assert exc.detail is not None and tier in exc.detail + assert tier in str(exc) + + +def test_a_router_response_reporting_no_tier_bills_at_the_base_rate() -> None: + """Sweep 2026-09-07: Router omitted `service_tier` on six `incomplete` zero-output + responses (both surfaces) and billed every one at standard, while flex and priority + were always reported explicitly. Absence means standard; only a reported non-base tier + is a miss.""" + errors: list[Exception] = [] + sdk, received, _ = _priced_sdk(on_error=lambda exc, where: errors.append(exc)) + client = sdk.wrap(FakeRouterClient(ROUTER_BASE_URL, lambda kw: _tiered(SERVED_MODEL, tier=None))) + client.responses.create(model="x", input="ping") + assert sdk.flush(timeout=2.0) + sdk.shutdown(timeout=1.0) + assert "llm_cost" in [e["code"] for e in received] + assert not any(isinstance(e, PricingUnavailableError) for e in errors) + + +_LUNA_COLD_WRITE = { + "input_tokens": 4493, + "output_tokens": 5, + "total_tokens": 4498, + "input_tokens_details": {"cache_write_tokens": 4490, "cached_tokens": 0}, +} +_LUNA_WARM_READ = { + "input_tokens": 4493, + "output_tokens": 5, + "total_tokens": 4498, + "input_tokens_details": {"cache_write_tokens": 0, "cached_tokens": 4490}, +} + + +def test_an_openai_served_cache_write_bills_at_the_catalog_write_rate() -> None: + """Reconciled against the dashboard on 2026-09-07 (gpt-5.6-luna, default tier): at the + published rates 3 x $0.20/M + 4490 x $0.25/M + 5 x $1.20/M = $0.0011291; Router charged + exactly 1.1x that, a documented per-model mismatch the SDK does not correct. What this + pins is the write arithmetic: the count sits INSIDE input_tokens, so it is moved out + before pricing — never billed at the input rate AND the write rate.""" + sdk, received, _ = _priced_sdk() + client = sdk.wrap( + FakeRouterClient(ROUTER_BASE_URL, lambda kw: _tiered("gpt-5.6-luna", usage=_LUNA_COLD_WRITE)) + ) + client.responses.create(model="x", input="ping") + assert sdk.flush(timeout=2.0) + sdk.shutdown(timeout=1.0) + costs = _cost_by_type(received) + assert set(costs) == {"input", "cache_write", "output"} + assert costs["input"]["properties"]["unit"] == "3" + assert costs["cache_write"]["properties"]["unit"] == "4490" + assert costs["cache_write"]["properties"]["unit_price"] == "0.00000025" + assert sum(Decimal(e["properties"]["value"]) for e in costs.values()) == Decimal("0.0011291") -def test_a_flex_tier_call_is_never_billed_at_the_base_rate() -> None: - """supported-models: "Service tiers, long contexts, caching, and other features may - use different rates." Billing flex at the standard rate over-bills.""" + +def test_the_warm_repeat_bills_the_cached_block_at_the_cache_read_rate() -> None: + """Same prompt a second later: 4490 cached at $0.02/M: $0.0000964 at the published rates + (Router charged 1.1x that — the documented luna mismatch).""" sdk, received, _ = _priced_sdk() client = sdk.wrap( - FakeRouterClient(ROUTER_BASE_URL, lambda kw: router_response(f"openai:{PRICED_MODEL}:flex")) + FakeRouterClient(ROUTER_BASE_URL, lambda kw: _tiered("gpt-5.6-luna", usage=_LUNA_WARM_READ)) ) client.responses.create(model="x", input="ping") assert sdk.flush(timeout=2.0) sdk.shutdown(timeout=1.0) - assert "llm_cost" not in [e["code"] for e in received] + + costs = _cost_by_type(received) + assert set(costs) == {"input", "cache_read", "output"} + assert costs["cache_read"]["properties"]["unit"] == "4490" + assert sum(Decimal(e["properties"]["value"]) for e in costs.values()) == Decimal("0.0000964") + + +def test_the_cache_write_count_is_mapped_for_router_and_stays_in_extras_for_openai() -> None: + """Same wire shape, two measured billing conventions: Router bills the write at its + catalog rate (mapped, not drift); OpenAI-native was metered at the plain input rate + (unmapped, surfaced in extras — see _MAPPED_DETAIL_FIELDS).""" + body = _tiered("gpt-5.6-luna", usage=_LUNA_COLD_WRITE) + via_router = extract_openai_native(body, provider_hint=RAMP_ROUTER_PROVIDER) + assert via_router.cache_write == 4490 + assert "input_tokens_details.cache_write_tokens" not in via_router.extras + direct = extract_openai_native(body) + assert direct.cache_write == 0 + assert direct.extras["input_tokens_details.cache_write_tokens"] == 4490 + + +def test_a_streamed_router_call_carries_the_served_tier_and_prices() -> None: + """The terminal `response.completed` event carries `service_tier` (fixture 04). The + stream wrapper used to forward usage and model only, so every streamed Router call + reached price mode tier-less — and a missing tier is a miss.""" + events = [ + _FakeStreamChunk( + {"type": "response.created", "response": {"model": SERVED_MODEL, "service_tier": "default"}} + ), + _FakeStreamChunk({"type": "response.output_text.delta", "delta": "po"}), + _FakeStreamChunk({"type": "response.completed", "response": _tiered(SERVED_MODEL)}), + ] + sdk, received, _ = _priced_sdk() + client = sdk.wrap(FakeRouterClient(ROUTER_BASE_URL, lambda kw: iter(events))) + for _ in client.responses.create(model="x", input="ping", stream=True): + pass + assert sdk.flush(timeout=2.0) + sdk.shutdown(timeout=1.0) + + costs = _cost_by_type(received) + assert set(costs) == {"input", "output"} + assert costs["input"]["properties"]["price_source"] == "ramp_router" def test_a_pricing_miss_never_reaches_the_caller_as_an_exception() -> None: sdk, _, _ = _priced_sdk() - client = sdk.wrap(FakeRouterClient(ROUTER_BASE_URL, lambda kw: router_response(f"openai:{PRICED_MODEL}"))) + client = sdk.wrap(FakeRouterClient(ROUTER_BASE_URL, lambda kw: _tiered(SERVED_MODEL, "flex"))) assert client.responses.create(model="x", input="ping") is not None sdk.shutdown(timeout=1.0) +def test_a_cold_or_empty_router_table_is_a_reported_miss_not_a_silent_token_fallback() -> None: + """Router used to sit in TOKEN_BILLED_PROVIDERS, which swallowed the miss on purpose + because nothing could fix it. Now a miss is actionable — no Router key learned, table + still cold, catalog missing the model — so it must reach on_error like any other.""" + assert RAMP_ROUTER_PROVIDER not in TOKEN_BILLED_PROVIDERS + + errors: list[Exception] = [] + sdk, received, _ = _priced_sdk(router_table={}, on_error=lambda exc, where: errors.append(exc)) + client = sdk.wrap(FakeRouterClient(ROUTER_BASE_URL, lambda kw: _tiered(SERVED_MODEL))) + client.responses.create(model="x", input="ping") + assert sdk.flush(timeout=2.0) + sdk.shutdown(timeout=1.0) + assert sorted(e["code"] for e in received) == ["llm_input_tokens", "llm_output_tokens"] + misses = [e for e in errors if isinstance(e, PricingUnavailableError)] + assert len(misses) == 1 and misses[0].detail is None + + # ---------------------------------------------------------------------- # The hot path. Billing is enqueue-only, so concurrency must not lose or # duplicate an event, and detection must not add per-call work. @@ -551,31 +727,10 @@ def call() -> None: # ---------------------------------------------------------------------- -# The two recorded decisions behind "ramp_router", pinned so neither can be -# reverted silently. The generic roster tests cannot see them: the hint comes -# from the wrapper's HOST arm, not from _PROVIDER_BY_BASE_URL_PATH, so nothing -# else in the suite fails if either set entry disappears. +# The recorded token-convention decision behind "ramp_router", pinned so it +# cannot be reverted silently. The generic roster tests cannot see it: the hint +# comes from the wrapper's HOST arm, not from _PROVIDER_BY_BASE_URL_PATH. # ---------------------------------------------------------------------- -def test_ramp_router_is_token_billed_a_price_mode_call_emits_token_events_with_no_error_report() -> None: - """Router is structurally unpriceable today (BYOK requests bill $0, tiers have - unpublished rates, every observed catalog input rate is empty), so a price miss is - permanent — and a permanent miss must not cry wolf on the error hook per call. Same - decision as Databricks and Snowflake.""" - assert RAMP_ROUTER_PROVIDER in TOKEN_BILLED_PROVIDERS - - errors: list[Any] = [] - provider = PricingProvider(fetcher=_StubFetcher(), ttl_seconds=3600.0) - sdk, received = _new_sdk( - pricing_mode="price", pricing_provider=provider, on_error=lambda exc, where: errors.append(exc) - ) - provider.prime(["openrouter"]) - provider.maybe_refresh() - client = sdk.wrap(FakeRouterClient(ROUTER_BASE_URL, lambda kw: router_response("openai:gpt-5.4-mini"))) - client.responses.create(model="x", input="ping") - assert sdk.flush(timeout=2.0) - sdk.shutdown(timeout=1.0) - assert sorted(e["code"] for e in received) == ["llm_input_tokens", "llm_output_tokens"] - assert errors == [] def test_ramp_routers_token_convention_is_a_recorded_measurement_openai_shaped_on_every_axis() -> None: @@ -645,10 +800,17 @@ def test_a_router_remainder_smaller_than_its_subsets_still_folds_rather_than_van _CAPTURES = pathlib.Path(__file__).parents[1] / "adapters" / "fixtures" / "ramp_router" -def _captured_bodies() -> list[tuple[str, dict[str, Any]]]: - """Every captured 200 that carries usage, buffered or streamed.""" +def _captured_bodies(surface: str = "responses") -> list[tuple[str, dict[str, Any]]]: + """Every captured 200 that carries usage, buffered or streamed, on ONE surface. + + Router has two: `/v1/responses` (OpenAI-shaped, fixtures 01-10) and `/v1/messages` + (Anthropic-shaped, fixtures 11-15, named `_messages_`). They report `service_tier` in + different places and go through different adapters, so a test must say which it means. + """ out: list[tuple[str, dict[str, Any]]] = [] for path in sorted(_CAPTURES.glob("*.json")): + if ("_messages_" in path.name) != (surface == "messages"): + continue blob = json.loads(path.read_text()) body = blob.get("_body") if not (isinstance(body, dict) and body.get("usage")): @@ -686,3 +848,280 @@ def test_every_captured_response_bills_the_bare_served_snapshot(name: str, body: assert u.model == body["model"] assert ":" not in u.model assert u.provider == RAMP_ROUTER_PROVIDER + + +@pytest.mark.skipif(not _captured_bodies(), reason="Router fixtures not captured") +@pytest.mark.parametrize("name,body", _captured_bodies(), ids=lambda v: v if isinstance(v, str) else "") +def test_every_captured_served_model_resolves_in_the_real_catalog(name: str, body: dict[str, Any]) -> None: + """The served name is what price mode looks up, and it is never the catalog id: a + dated snapshot for OpenAI and Anthropic, the vendor's own path for Fireworks. Every + response Router has actually sent must land on exactly one catalog entry.""" + u = extract_openai_native(body, provider_hint=RAMP_ROUTER_PROVIDER) + assert lookup_ramp_router(_ROUTER_TABLE, u.model) is not None + + +# ---------------------------------------------------------------------- +# Router's SECOND surface: `POST /v1/messages`, reached with an Anthropic client. +# Same host, same catalog, same key — but Anthropic's schema and Anthropic's +# ADDITIVE convention for every vendor, and the one place an Anthropic-served +# cache WRITE is reported. Detection is the shared host helper; the wrapper +# threads a provider hint into the Anthropic adapter, which stamps a distinct +# `api` so the token semantics cannot be confused with the Responses surface. +# ---------------------------------------------------------------------- +def messages_response(model: str, usage: dict[str, Any] | None = None) -> dict[str, Any]: + """A Router `/v1/messages` response, in the shape fixture 11 actually carries: + Anthropic's schema, `service_tier` INSIDE usage.""" + return { + "id": "msg_test", + "type": "message", + "role": "assistant", + "model": model, + "content": [{"type": "text", "text": "pong"}], + "usage": { + "input_tokens": 16, + "output_tokens": 5, + "cache_read_input_tokens": 0, + "cache_creation_input_tokens": 0, + "cache_creation": {"ephemeral_5m_input_tokens": 0, "ephemeral_1h_input_tokens": 0}, + "service_tier": "standard", + **(usage or {}), + }, + } + + +class FakeRouterAnthropicMessages: + def __init__(self, reply: Any) -> None: + self._reply = reply + + def create(self, **kwargs: Any) -> Any: + assert "extra_lago" not in kwargs + return self._reply(kwargs) + + +class FakeRouterAnthropicClient: + def __init__(self, base_url: str, reply: Any, api_key: str = "sk-router-from-client") -> None: + self.base_url = base_url + self.api_key = api_key + self.messages = FakeRouterAnthropicMessages(reply) + + +# The detector keys on the module; Router's Messages surface is reached with an Anthropic client. +FakeRouterAnthropicClient.__module__ = "anthropic.fake" + +ANTHROPIC_ROUTER_BASE_URL = "https://api.router.com" +HAIKU_SERVED = "claude-haiku-4-5-20251001" # what Router answers with (fixture 11) + + +@pytest.mark.parametrize( + "base_url,expected", + [ + ("https://api.router.com", True), + ("https://api.router.com/v1", True), + ("https://api-eu.router.com", True), + ("https://api.anthropic.com", False), + ("https://evil.example.com/api.router.com", False), + ("https://evilrouter.com", False), + ("/v1", False), + (None, False), + (42, False), + ], +) +def test_the_shared_host_helper_is_the_one_answer_both_wrappers_read(base_url: Any, expected: bool) -> None: + assert is_ramp_router_base_url(base_url) is expected + assert client_points_at_ramp_router(_Base(base_url)) is expected + + +def test_the_openai_wrapper_and_the_shared_helper_cannot_disagree() -> None: + for url in ("https://api.router.com/v1", "https://api-eu.router.com/v1", "https://api.openai.com/v1"): + assert (_provider_hint_for(_Base(url)) == RAMP_ROUTER_PROVIDER) is is_ramp_router_base_url(url) + + +def test_an_anthropic_client_pointed_at_router_bills_as_router_on_the_messages_surface() -> None: + sdk, received = _new_sdk() + client = sdk.wrap( + FakeRouterAnthropicClient(ANTHROPIC_ROUTER_BASE_URL, lambda kw: messages_response(HAIKU_SERVED)) + ) + client.messages.create( + model="claude-haiku-4-5", max_tokens=16, messages=[{"role": "user", "content": "ping"}] + ) + assert sdk.flush(timeout=2.0) + sdk.shutdown(timeout=1.0) + + assert _by_code(received) == {"llm_input_tokens": 16, "llm_output_tokens": 5} + for e in received: + assert e["properties"]["provider"] == RAMP_ROUTER_PROVIDER + assert e["properties"]["api"] == RAMP_ROUTER_MESSAGES_API + assert e["properties"]["model"] == HAIKU_SERVED + + +def test_an_anthropic_client_pointed_at_anthropic_is_untouched() -> None: + sdk, received = _new_sdk() + client = sdk.wrap( + FakeRouterAnthropicClient("https://api.anthropic.com", lambda kw: messages_response(HAIKU_SERVED)) + ) + client.messages.create(model="claude-haiku-4-5", max_tokens=16, messages=[]) + assert sdk.flush(timeout=2.0) + sdk.shutdown(timeout=1.0) + assert {(e["properties"]["provider"], e["properties"]["api"]) for e in received} == { + ("anthropic", "native") + } + + +def test_the_messages_surface_keeps_anthropics_additive_convention_for_every_vendor() -> None: + """Measured: haiku `input_tokens: 16` beside `cache_read_input_tokens: 20113` (2026-09-04, + reconciled exactly); an xAI model `input_tokens: 65` beside `cache_read_input_tokens: + 128` with thinking inside output (2026-09-07). The Responses stamp is in + OPENAI_SHAPED_APIS; this one must never be.""" + assert RAMP_ROUTER_MESSAGES_API not in OPENAI_SHAPED_APIS + assert token_semantics(RAMP_ROUTER_PROVIDER, RAMP_ROUTER_MESSAGES_API) == (False, False, False) + assert token_semantics(RAMP_ROUTER_PROVIDER, RAMP_ROUTER_PROVIDER) == (True, True, True) + + +def test_the_adapter_stamps_router_only_when_the_wrapper_says_so() -> None: + body = messages_response(HAIKU_SERVED) + assert (extract_anthropic_native(body).provider, extract_anthropic_native(body).api) == ( + "anthropic", + "native", + ) + hinted = extract_anthropic_native(body, provider_hint=RAMP_ROUTER_PROVIDER) + assert (hinted.provider, hinted.api) == (RAMP_ROUTER_PROVIDER, RAMP_ROUTER_MESSAGES_API) + # The tier rides inside usage on this surface and lands in extras with no special code. + assert hinted.extras["service_tier"] == "standard" + + +# Fixture 12 / 13, verbatim: a 7,481-token cache_control prefix on claude-haiku-4-5. +_MESSAGES_COLD_WRITE = { + "input_tokens": 15, + "output_tokens": 5, + "cache_creation_input_tokens": 7481, + "cache_creation": {"ephemeral_5m_input_tokens": 7481, "ephemeral_1h_input_tokens": 0}, +} +_MESSAGES_WARM_READ = {"input_tokens": 15, "output_tokens": 6, "cache_read_input_tokens": 7481} + + +def test_an_anthropic_cache_write_on_the_messages_surface_bills_at_the_ttl_write_rate() -> None: + """The gap the Responses surface cannot close. Router publishes cache_write_input_5m + ($1.25/M on haiku) and the Messages surface reports the written count with its TTL, so + the write bills at its own rate: 15 x $1/M + 7481 x $1.25/M + 5 x $5/M = $0.00939125. + The lump `cache_write` line is consumed entirely by the split — never billed twice.""" + sdk, received, _ = _priced_sdk() + client = sdk.wrap( + FakeRouterAnthropicClient( + ANTHROPIC_ROUTER_BASE_URL, lambda kw: messages_response(HAIKU_SERVED, _MESSAGES_COLD_WRITE) + ) + ) + client.messages.create(model="x", max_tokens=16, messages=[]) + assert sdk.flush(timeout=2.0) + sdk.shutdown(timeout=1.0) + + costs = _cost_by_type(received) + assert set(costs) == {"input", "cache_write_5m", "output"} + assert costs["input"]["properties"]["unit"] == "15" # additive: NOT reduced by the write + assert costs["cache_write_5m"]["properties"]["unit"] == "7481" + assert costs["cache_write_5m"]["properties"]["unit_price"] == "0.00000125" + assert all(e["properties"]["api"] == RAMP_ROUTER_MESSAGES_API for e in costs.values()) + assert sum(Decimal(e["properties"]["value"]) for e in costs.values()) == Decimal("0.00939125") + + +def test_the_warm_repeat_on_the_messages_surface_bills_the_read_beside_input() -> None: + """Additive: 15 input tokens stay 15; the 7,481 cached bill at $0.10/M. $0.0007931.""" + sdk, received, _ = _priced_sdk() + client = sdk.wrap( + FakeRouterAnthropicClient( + ANTHROPIC_ROUTER_BASE_URL, lambda kw: messages_response(HAIKU_SERVED, _MESSAGES_WARM_READ) + ) + ) + client.messages.create(model="x", max_tokens=16, messages=[]) + assert sdk.flush(timeout=2.0) + sdk.shutdown(timeout=1.0) + + costs = _cost_by_type(received) + assert set(costs) == {"input", "cache_read", "output"} + assert costs["input"]["properties"]["unit"] == "15" + assert costs["cache_read"]["properties"]["unit"] == "7481" + assert sum(Decimal(e["properties"]["value"]) for e in costs.values()) == Decimal("0.0007931") + + +@pytest.mark.parametrize("tier,priced", [("standard", True), ("default", True), ("priority", False)]) +def test_the_tier_gate_reads_the_messages_surfaces_in_usage_tier(tier: str, priced: bool) -> None: + errors: list[Exception] = [] + sdk, received, _ = _priced_sdk(on_error=lambda exc, where: errors.append(exc)) + client = sdk.wrap( + FakeRouterAnthropicClient( + ANTHROPIC_ROUTER_BASE_URL, lambda kw: messages_response(HAIKU_SERVED, {"service_tier": tier}) + ) + ) + client.messages.create(model="x", max_tokens=16, messages=[]) + assert sdk.flush(timeout=2.0) + sdk.shutdown(timeout=1.0) + assert ("llm_cost" in [e["code"] for e in received]) is priced + assert any(isinstance(e, PricingUnavailableError) and tier in str(e) for e in errors) is not priced + + +def test_a_streamed_messages_call_carries_the_tier_through_the_merge_and_prices() -> None: + """Fixture 14: `service_tier` sits inside `message_start.message.usage` AND + `message_delta.usage`; the wrapper's merge keeps it, so the adapter's drift sweep + lands it in extras and the tier gate passes.""" + start = { + "type": "message_start", + "message": { + "model": HAIKU_SERVED, + "usage": { + "input_tokens": 16, + "output_tokens": 4, + "cache_read_input_tokens": 0, + "service_tier": "standard", + }, + }, + } + delta = {"type": "message_delta", "usage": {"output_tokens": 5, "service_tier": "standard"}} + events = [ + _FakeStreamChunk(start), + _FakeStreamChunk({"type": "content_block_delta"}), + _FakeStreamChunk(delta), + ] + sdk, received, _ = _priced_sdk() + client = sdk.wrap(FakeRouterAnthropicClient(ANTHROPIC_ROUTER_BASE_URL, lambda kw: iter(events))) + for _ in client.messages.create(model="x", max_tokens=16, messages=[], stream=True): + pass + assert sdk.flush(timeout=2.0) + sdk.shutdown(timeout=1.0) + + costs = _cost_by_type(received) + assert set(costs) == {"input", "output"} + assert costs["input"]["properties"]["unit"] == "16" + assert costs["output"]["properties"]["unit"] == "5" + assert costs["input"]["properties"]["api"] == RAMP_ROUTER_MESSAGES_API + + +@pytest.mark.skipif(not _captured_bodies("messages"), reason="Router /v1/messages fixtures not captured") +@pytest.mark.parametrize( + "name,body", _captured_bodies("messages"), ids=lambda v: v if isinstance(v, str) else "" +) +def test_every_captured_messages_response_stamps_router_and_resolves_in_the_catalog( + name: str, body: dict[str, Any] +) -> None: + u = extract_anthropic_native(body, provider_hint=RAMP_ROUTER_PROVIDER) + assert (u.provider, u.api) == (RAMP_ROUTER_PROVIDER, RAMP_ROUTER_MESSAGES_API) + assert u.model == body["model"] + # The tier is INSIDE usage on this surface — the OpenAI-served model included. + assert u.extras["service_tier"] == body["usage"]["service_tier"] + assert lookup_ramp_router(_ROUTER_TABLE, u.model) is not None + # Additive: the lump write equals the TTL split (Anthropic's contract, every capture). + assert u.cache_write == u.cache_write_5m + u.cache_write_1h + + +@pytest.mark.skipif( + not (_CAPTURES / "14_real_messages_streamed.json").exists(), reason="fixture not captured" +) +def test_the_captured_messages_stream_merges_to_a_priced_tier() -> None: + blob = json.loads((_CAPTURES / "14_real_messages_streamed.json").read_text()) + accumulated: dict[str, Any] = {} + model: str | None = None + for event in blob["_events"]: + model = _merge_stream_usage(accumulated, event) or model + u = extract_anthropic_native({"usage": accumulated, "model": model}, provider_hint=RAMP_ROUTER_PROVIDER) + assert u.model == HAIKU_SERVED + assert u.input == 16 and u.output == 5 + assert u.extras["service_tier"] == "standard" + assert lookup_ramp_router(_ROUTER_TABLE, u.model) is not None diff --git a/tests/unit/test_auto_prime_pricing.py b/tests/unit/test_auto_prime_pricing.py index 5d47e6e..2e2f7d8 100644 --- a/tests/unit/test_auto_prime_pricing.py +++ b/tests/unit/test_auto_prime_pricing.py @@ -56,8 +56,12 @@ class FakeOpenAIClient: __module__ = "openai.client" - def __init__(self, base_url: str): + def __init__(self, base_url: str, api_key: str | None = None): self.base_url = base_url + # `openai.OpenAI(api_key=...)` exposes the key as `.api_key` (verified on 2.38); + # None here means "a client variant without it", and the attribute is absent. + if api_key is not None: + self.api_key = api_key _MISTRAL_ALIASES = parse_mistral_aliases( @@ -187,3 +191,100 @@ def fetch_mistral_aliases(self, api_key=None): provider.maybe_refresh() assert fetcher.mistral_calls == 0 + + +class _RouterCallCountingFetcher(HttpPricingFetcher): + def __init__(self): + super().__init__() + self.router_keys: list[str | None] = [] + + def fetch_ramp_router(self, api_key=None): + self.router_keys.append(api_key) + return {} + + +def test_wrap_openai_pointed_at_router_learns_the_key_and_primes_the_catalog(): + """Router's catalog is account-scoped, so the key the client already carries is the + one that unlocks it — no LagoConfig.ramp_router_api_key required.""" + fetcher = _RouterCallCountingFetcher() + provider = PricingProvider(fetcher=fetcher, ttl_seconds=3600) + sdk = _sdk_with_provider(provider) + + sdk.wrap(FakeOpenAIClient(base_url="https://api.router.com/v1", api_key="sk-router-abc")) + + assert _wait_until(lambda: fetcher.router_keys == ["sk-router-abc"]) + + +def test_wrap_openai_pointed_at_router_without_a_readable_key_still_primes(): + """A client variant with no `.api_key` degrades to "no key learned": the fetch runs + with None (then LagoConfig.ramp_router_api_key, then an empty table and a reported + miss) rather than raising out of wrap().""" + fetcher = _RouterCallCountingFetcher() + provider = PricingProvider(fetcher=fetcher, ttl_seconds=3600) + sdk = _sdk_with_provider(provider) + + sdk.wrap(FakeOpenAIClient(base_url="https://api.router.com/v1")) + + assert _wait_until(lambda: fetcher.router_keys == [None]) + + +def test_wrap_openai_pointed_at_real_openai_does_not_prime_router(): + fetcher = _RouterCallCountingFetcher() + provider = PricingProvider(fetcher=fetcher, ttl_seconds=3600) + sdk = _sdk_with_provider(provider) + + sdk.wrap(FakeOpenAIClient(base_url="https://api.openai.com/v1", api_key="sk-openai")) + provider.maybe_refresh() + + assert fetcher.router_keys == [] + + +def test_an_explicit_config_router_key_reaches_the_default_fetcher(): + cfg = LagoConfig( + api_key="dummy", default_subscription_id="sub_test", ramp_router_api_key="cfg-router-key" + ) + sdk = LagoSDK(api_key="dummy", config=cfg) + try: + assert sdk._pricing._fetcher._ramp_router_api_key == "cfg-router-key" # type: ignore[attr-defined] + finally: + sdk.shutdown(timeout=1.0) + + +class _Messages: + def create(self, **kwargs): + return {"usage": {"input_tokens": 1, "output_tokens": 1}} + + +class FakeAnthropicClient: + """Mimics anthropic.Anthropic: `.base_url`, `.api_key`, `.messages` (verified 0.103.1).""" + + __module__ = "anthropic.client" + + def __init__(self, base_url: str, api_key: str | None = None): + self.base_url = base_url + self.messages = _Messages() + if api_key is not None: + self.api_key = api_key + + +def test_wrap_anthropic_pointed_at_router_learns_the_key_and_primes_the_catalog(): + """Router's second surface. The Anthropic client carries the same Router key, read the + same way — one helper, so the two wrappers cannot learn it differently.""" + fetcher = _RouterCallCountingFetcher() + provider = PricingProvider(fetcher=fetcher, ttl_seconds=3600) + sdk = _sdk_with_provider(provider) + + sdk.wrap(FakeAnthropicClient(base_url="https://api.router.com", api_key="sk-router-via-anthropic")) + + assert _wait_until(lambda: fetcher.router_keys == ["sk-router-via-anthropic"]) + + +def test_wrap_anthropic_pointed_at_anthropic_does_not_prime_router(): + fetcher = _RouterCallCountingFetcher() + provider = PricingProvider(fetcher=fetcher, ttl_seconds=3600) + sdk = _sdk_with_provider(provider) + + sdk.wrap(FakeAnthropicClient(base_url="https://api.anthropic.com", api_key="sk-ant")) + provider.maybe_refresh() + + assert fetcher.router_keys == [] diff --git a/tests/unit/test_pricing.py b/tests/unit/test_pricing.py index f71974c..c61c511 100644 --- a/tests/unit/test_pricing.py +++ b/tests/unit/test_pricing.py @@ -16,8 +16,11 @@ from lago_agent_sdk.adapters.openai_native import extract_openai_native from lago_agent_sdk.canonical import WORKERS_AI_COMPAT_PREFIX from lago_agent_sdk.pricing import ( + RAMP_ROUTER_MODELS_URL, + TOKEN_BILLED_PROVIDERS, HttpPricingFetcher, PricingProvider, + _is_foreign_backend_alias, _parse_price, _pick_mistral_canonical, _strip_version, @@ -30,11 +33,14 @@ lookup_bedrock, lookup_cloudflare_workers_ai, lookup_openrouter, + lookup_ramp_router, parse_bedrock_offer, parse_bedrock_region, parse_cloudflare_workers_ai, parse_mistral_aliases, parse_openrouter, + parse_ramp_router, + ramp_router_unpriced_tier, ) FIXTURES = pathlib.Path(__file__).parent / "fixtures" / "pricing" @@ -50,16 +56,20 @@ def __init__( bedrock: dict | None = None, cloudflare_workers_ai: dict[str, ModelPrice] | None = None, mistral_aliases: dict[str, str] | None = None, + ramp_router: dict[str, ModelPrice] | None = None, ) -> None: self._openrouter = openrouter or {"exact": {}, "norm": {}} self._bedrock = bedrock or {} self._cloudflare_workers_ai = cloudflare_workers_ai or {} self._mistral_aliases = mistral_aliases or {} + self._ramp_router = ramp_router or {} self.openrouter_calls = 0 self.bedrock_calls: list[str] = [] self.cloudflare_workers_ai_calls = 0 self.mistral_aliases_calls = 0 self.last_mistral_api_key: str | None = None + self.ramp_router_calls = 0 + self.last_ramp_router_api_key: str | None = None def fetch_openrouter(self) -> dict[str, Any]: self.openrouter_calls += 1 @@ -78,6 +88,11 @@ def fetch_mistral_aliases(self, api_key: str | None = None) -> dict[str, str]: self.last_mistral_api_key = api_key return self._mistral_aliases + def fetch_ramp_router(self, api_key: str | None = None) -> dict[str, ModelPrice]: + self.ramp_router_calls += 1 + self.last_ramp_router_api_key = api_key + return self._ramp_router + _OPENROUTER_RAW = { "data": [ @@ -2213,3 +2228,472 @@ def test_pricing_backoff_growth_matches_the_queues_and_is_capped() -> None: p._note_failure("openrouter") seq.append(p._failure_backoff_seconds["openrouter"]) assert seq == [1.0, 2.0, 4.0, 8.0, 16.0, 32.0, 60.0, 60.0] + + +# ---------------------------------------------------------------------- +# Ramp Router — its own catalog is the price source +# +# Built through the real parser from the REAL captured catalog. The numbers +# asserted below are the ones that reconciled against Router's dashboard. +# ---------------------------------------------------------------------- +_ROUTER_CATALOG = json.loads( + ( + pathlib.Path(__file__).parent + / "adapters" + / "fixtures" + / "ramp_router" + / "01_real_models_catalog.json" + ).read_text() +)["_body"] +_ROUTER_TABLE = parse_ramp_router(_ROUTER_CATALOG) + + +def test_ramp_router_is_no_longer_token_billed() -> None: + """A Router miss is actionable now — no key, cold table, non-default tier — so it + must report like any other provider's instead of being swallowed as structural.""" + assert "ramp_router" not in TOKEN_BILLED_PROVIDERS + + +def test_ramp_router_parses_the_real_catalog_per_token() -> None: + """68 entries, six rate keys each, strings in USD per 1M tokens (measured 2026-09-07). + claude-haiku-4-5 publishes $1/M in, $5/M out, $0.10/M cached.""" + ids = {m["id"] for m in _ROUTER_CATALOG["data"]} + assert len(ids) == 68 + assert all(i in _ROUTER_TABLE for i in ids) + mp = _ROUTER_TABLE["claude-haiku-4-5"] + assert mp.source == "ramp_router" + assert mp.input == Decimal("0.000001") + assert mp.output == Decimal("0.000005") + assert mp.cache_read == Decimal("0.0000001") + assert mp.reasoning is None + + +def test_ramp_router_zero_cache_rate_means_no_separate_rate_not_free() -> None: + """Anthropic entries publish cache_write_input "0" because their write price lives + in the _5m/_1h keys; the pro and legacy OpenAI entries publish cache_read_input "0" + because they do not cache. Neither is a $0 rate: stored as None so compute_cost + leaves those tokens inside `input` at the input rate.""" + assert _ROUTER_TABLE["claude-haiku-4-5"].cache_write is None + assert _ROUTER_TABLE["gpt-5-pro"].cache_read is None + # ...while a genuinely published write rate is kept: gpt-5.6-luna, $0.25/M. + assert _ROUTER_TABLE["gpt-5.6-luna"].cache_write == Decimal("0.00000025") + + +def test_ramp_router_no_catalog_rate_is_lossy_at_twelve_places() -> None: + """Per-million -> per-token is a division by 1e6 floored to 12 dp. If Router ever + publishes a rate with more than six significant decimals this fails, which is the + moment to widen _SCALE in BOTH repos rather than silently floor a price to 0.""" + for m in _ROUTER_CATALOG["data"]: + for key, value in m["router"]["pricing"].items(): + d = Decimal(value) + per_token = (d / Decimal(1_000_000)).quantize(Decimal("1e-12")) + assert per_token * Decimal(1_000_000) == d, (m["id"], key, value) + + +def test_ramp_router_bills_the_published_rate_even_where_router_measurably_does_not() -> None: + """Dashboard-measured 2026-09-07: Router bills gpt-5.6-luna at 1.1x its own catalog and + gpt-5.6-sol at 0.55x. The SDK deliberately stores the PUBLISHED rate anyway — a factor + in the SDK would be the thing out of sync the day Router corrects its catalog — and + the docs hand the customer the measured factor as a `markup` they can drop that day.""" + assert _ROUTER_TABLE["gpt-5.6-luna"].input == Decimal("0.0000002") + assert _ROUTER_TABLE["gpt-5.6-sol"].input == Decimal("0.000004") + gpt54 = next(m for m in _ROUTER_CATALOG["data"] if m["id"] == "gpt-5.4") + assert _ROUTER_TABLE["gpt-5.4"].input == Decimal(gpt54["router"]["pricing"]["input"]) / Decimal(1_000_000) + + +def test_ramp_router_foreign_backend_aliases_are_not_indexed() -> None: + """A Fireworks-owned entry served through Baseten bills Baseten's rate, which the + catalog does not publish. The Baseten spelling is the entry's alias under another + path prefix, and it must MISS rather than price at the Fireworks rate (measured 1.11x + to 2.4x off, 2026-09-07). The entry's own id and Fireworks path still price.""" + assert "deepseek-ai/DeepSeek-V4-Flash-0731" not in _ROUTER_TABLE + assert "zai-org/GLM-5.2" not in _ROUTER_TABLE + assert "moonshotai/Kimi-K2.7-Code" not in _ROUTER_TABLE + assert "deepseek-v4-flash-0731" in _ROUTER_TABLE + assert "accounts/fireworks/models/deepseek-v4-flash-0731" in _ROUTER_TABLE + # A Baseten-OWNED entry's own path is its provider_model, not a foreign alias. + assert "thinkingmachines/inkling-small" in _ROUTER_TABLE + assert lookup_ramp_router(_ROUTER_TABLE, "deepseek-ai/DeepSeek-V4-Flash-0731") is None + + +@pytest.mark.parametrize( + "alias,provider_model,foreign", + [ + ("deepseek-ai/DeepSeek-V4-Flash-0731", "accounts/fireworks/models/deepseek-v4-flash-0731", True), + ("accounts/fireworks/models/glm-5p3", "accounts/fireworks/models/glm-5p3", False), + ("gpt-5-chat-latest", "gpt-5-chat", False), # no path on the alias: a plain synonym + ("vendor/x", "bare-provider-model", False), # no path on the entry: nothing to compare + ], +) +def test_is_foreign_backend_alias(alias: str, provider_model: str, foreign: bool) -> None: + assert _is_foreign_backend_alias(alias, provider_model) is foreign + + +def test_ramp_router_a_synthetic_entry_keeps_a_same_backend_alias_and_drops_a_foreign_one() -> None: + t = parse_ramp_router( + { + "data": [ + _router_entry( + "m", + {}, + provider_model="accounts/fireworks/models/m", + aliases=["accounts/fireworks/models/m-alias", "other-host/M", "plain-synonym"], + ) + ] + } + ) + assert sorted(t) == [ + "accounts/fireworks/models/m", + "accounts/fireworks/models/m-alias", + "m", + "plain-synonym", + ] + + +def test_ramp_router_keys_every_name_a_response_can_report() -> None: + """Fireworks- and Baseten-served responses report the vendor's own path, which is the + entry's provider_model or an alias, never its id (fixtures 03, a2_baseten).""" + lightning = _ROUTER_TABLE["nemotron-lightning-3p5-30b-a3b"] + assert _ROUTER_TABLE["accounts/fireworks/models/nemotron-lightning-3p5-30b-a3b"] is lightning + assert _ROUTER_TABLE["thinkingmachines/inkling-small"] is _ROUTER_TABLE["inkling-small"] + + +@pytest.mark.parametrize( + "served,catalog_id", + [ + ("gpt-5.4-nano-2026-03-17", "gpt-5.4-nano"), + ("claude-haiku-4-5-20251001", "claude-haiku-4-5"), + ("o3-2025-04-16", "o3"), + ("grok-build-0.1", "grok-build-0.1"), + ("accounts/fireworks/models/nemotron-lightning-3p5-30b-a3b", "nemotron-lightning-3p5-30b-a3b"), + ], +) +def test_ramp_router_lookup_resolves_every_served_name_shape(served: str, catalog_id: str) -> None: + """The five shapes Router has actually answered with: OpenAI and Anthropic dated + snapshots (version-strip), an xAI bare id, and a Fireworks vendor path.""" + assert lookup_ramp_router(_ROUTER_TABLE, served) is _ROUTER_TABLE[catalog_id] + + +def test_ramp_router_lookup_miss_returns_none() -> None: + assert lookup_ramp_router(_ROUTER_TABLE, "definitely-not-a-model") is None + assert lookup_ramp_router({}, "gpt-5.4-nano") is None + + +def _router_entry(mid: str, rates: dict[str, str], **router: Any) -> dict[str, Any]: + pricing = { + "input": "1", + "output": "2", + "cache_read_input": "0", + "cache_write_input": "0", + "cache_write_input_5m": "0", + "cache_write_input_1h": "0", + **rates, + } + return { + "id": mid, + "router": {**{"request_name": mid, "provider_model": mid}, "pricing": pricing, **router}, + } + + +def test_ramp_router_shared_name_with_identical_rates_prices() -> None: + """The live catalog's one shared name (`…/nemotron-3-ultra-nvfp4`, provider_model of + two entries) carries identical rates on both, so it stays priced.""" + t = parse_ramp_router( + {"data": [_router_entry("a", {}, aliases=["shared"]), _router_entry("b", {}, aliases=["shared"])]} + ) + assert t["shared"].input == Decimal("0.000001") + + +def test_ramp_router_shared_name_with_different_rates_is_unpriced_whatever_the_order() -> None: + """Guessing between two rates is a mispricing, not a miss. The name is removed AND + pinned, so a third entry cannot re-add it; the entries' own ids still price.""" + entries = [ + _router_entry("a", {}, aliases=["shared"]), + _router_entry("b", {"input": "3"}, aliases=["shared"]), + _router_entry("c", {}, aliases=["shared"]), + ] + t = parse_ramp_router({"data": entries}) + assert "shared" not in t + assert t["a"].input == Decimal("0.000001") + assert t["b"].input == Decimal("0.000003") + assert t["c"].input == Decimal("0.000001") + assert "shared" not in parse_ramp_router({"data": list(reversed(entries))}) + + +def test_ramp_router_empty_string_rate_is_a_missing_field_not_a_zero() -> None: + """The shape the committed fixture carried until the capture scrub was fixed: every + input rate read "". It must neither crash nor price input at $0.""" + t = parse_ramp_router({"data": [_router_entry("a", {"input": ""})]}) + assert t["a"].input is None + assert t["a"].output == Decimal("0.000002") + + +def test_ramp_router_malformed_entries_are_skipped_not_fatal() -> None: + t = parse_ramp_router( + { + "data": [ + {"id": "no-router-block"}, + {"id": "empty-router", "router": {}}, + {"id": "unparseable", "router": {"pricing": {"input": "abc", "output": None}}}, + "junk", + None, + _router_entry("ok", {}), + ] + } + ) + assert list(t) == ["ok"] + assert parse_ramp_router(None) == {} + assert parse_ramp_router({"data": "nope"}) == {} + + +@pytest.mark.parametrize("tier", ["default", "standard", "Default"]) +def test_ramp_router_base_rate_tiers_price(tier: str) -> None: + u = CanonicalUsage(model="m", provider="ramp_router", api="ramp_router", extras={"service_tier": tier}) + assert ramp_router_unpriced_tier(u) is None + + +@pytest.mark.parametrize( + "tier,expected", + [ + ("flex", "flex"), # measured 0.5x — a discount, so the base rate would OVER-bill + ("priority", "priority"), # measured 2.0x on two vendors + ("turbo", "turbo"), # a tier Router adds later must not silently bill at 1.0x + (7, "7"), # a non-string is drift, reported as-is + ], +) +def test_ramp_router_every_other_tier_is_a_named_miss(tier: Any, expected: str) -> None: + extras: dict[str, Any] = {} if tier is None else {"service_tier": tier} + u = CanonicalUsage(model="m", provider="ramp_router", api="ramp_router", extras=extras) + assert ramp_router_unpriced_tier(u) == expected + + +@pytest.mark.parametrize("extras", [{}, {"service_tier": None}, {"service_tier": ""}]) +def test_ramp_router_a_missing_tier_bills_at_the_base_rate(extras: dict[str, Any]) -> None: + """Sweep 2026-09-07: Router omitted the tier on six `incomplete` zero-output responses + and billed all six at standard; flex/priority were always explicit. Absence is standard.""" + u = CanonicalUsage(model="m", provider="ramp_router", api="ramp_router", extras=extras) + assert ramp_router_unpriced_tier(u) is None + + +def test_ramp_router_tier_gate_ignores_every_other_provider() -> None: + """OpenAI reports its own `service_tier`; only Router's tiers are unpriced.""" + u = CanonicalUsage(model="m", provider="openai", api="responses", extras={"service_tier": "flex"}) + assert ramp_router_unpriced_tier(u) is None + + +def test_ramp_router_cache_write_bills_at_the_catalog_write_rate_luna_reconciled() -> None: + """The 2026-09-07 dashboard row, default tier: gpt-5.6-luna, 4493 in / 4490 written / + 5 out. At the PUBLISHED rates: 3 x $0.20/M + 4490 x $0.25/M + 5 x $1.20/M = $0.0011291. + Router charged $0.00124201 — exactly 1.1x — a documented mismatch the SDK does not + correct (see the docs' markup recommendation); the write-rate arithmetic is what this + test pins.""" + usage = CanonicalUsage( + model="gpt-5.6-luna", + provider="ramp_router", + api="ramp_router", + input=4493, + cache_write=4490, + output=5, + ) + b = compute_cost(usage, _ROUTER_TABLE["gpt-5.6-luna"], Decimal(1)) + assert b.base == "0.0011291" + assert b.fields["input"]["tokens"] == "3" + assert b.fields["cache_write"]["tokens"] == "4490" + assert deoverlapped_token_total(usage) == 4498 + + +def test_ramp_router_cache_read_bills_at_the_catalog_read_rate_grok_reconciled() -> None: + """The 2026-09-04 dashboard row: grok-build-0.1, 194 in / 192 cached / 134 out. + Router charged 2 x $1/M + 192 x $0.20/M + 134 x $2/M = $0.0003084, exactly.""" + usage = CanonicalUsage( + model="grok-build-0.1", + provider="ramp_router", + api="ramp_router", + input=194, + cache_read=192, + output=134, + ) + b = compute_cost(usage, _ROUTER_TABLE["grok-build-0.1"], Decimal(1)) + assert b.base == "0.0003084" + + +def test_provider_ramp_router_cold_then_warm() -> None: + fetcher = StubFetcher(ramp_router=_ROUTER_TABLE) + p = PricingProvider(fetcher=fetcher, ttl_seconds=3600) + # cold: no table yet -> None, and flags it for refresh + assert p.lookup("ramp_router", "gpt-5.4-nano-2026-03-17", "ramp_router") is None + assert fetcher.ramp_router_calls == 0 + p.maybe_refresh() + assert fetcher.ramp_router_calls == 1 + mp = p.lookup("ramp_router", "gpt-5.4-nano-2026-03-17", "ramp_router") + assert mp is not None and mp.input == Decimal("0.0000002") + # warm: within the TTL nothing is refetched + p.maybe_refresh() + assert fetcher.ramp_router_calls == 1 + + +def test_provider_ramp_router_only_fetched_for_the_ramp_router_provider() -> None: + fetcher = StubFetcher(openrouter=parse_openrouter(_OPENROUTER_RAW), ramp_router=_ROUTER_TABLE) + p = PricingProvider(fetcher=fetcher, ttl_seconds=3600) + p.lookup("anthropic", "claude-opus-4-8", "native") + p.maybe_refresh() + assert fetcher.ramp_router_calls == 0 + + +def test_provider_ramp_router_lookup_never_consults_openrouter() -> None: + """Router serves models literally named `o4-mini` and `claude-haiku-4-5`; a fall-through + to OpenRouter would price them at another company's rate.""" + fetcher = StubFetcher(openrouter=parse_openrouter(_OPENROUTER_RAW), ramp_router={}) + p = PricingProvider(fetcher=fetcher, ttl_seconds=3600) + p.lookup("ramp_router", "claude-opus-4-8", "ramp_router") + p.maybe_refresh() + assert fetcher.openrouter_calls == 0 + assert p.lookup("ramp_router", "claude-opus-4-8", "ramp_router") is None + + +def test_provider_prime_ramp_router_warms_the_source() -> None: + fetcher = StubFetcher(ramp_router=_ROUTER_TABLE) + p = PricingProvider(fetcher=fetcher, ttl_seconds=3600) + p.prime(["ramp_router"]) + p.maybe_refresh() + assert fetcher.ramp_router_calls == 1 + + +def test_learn_ramp_router_api_key_is_used_on_next_fetch() -> None: + fetcher = StubFetcher(ramp_router=_ROUTER_TABLE) + p = PricingProvider(fetcher=fetcher, ttl_seconds=3600) + p.learn_ramp_router_api_key("sk-router-learned") + p.lookup("ramp_router", "o3", "ramp_router") + p.maybe_refresh() + assert fetcher.last_ramp_router_api_key == "sk-router-learned" + + +def test_learn_ramp_router_api_key_keeps_the_first_and_ignores_empty() -> None: + p = PricingProvider(fetcher=StubFetcher(), ttl_seconds=3600) + p.learn_ramp_router_api_key("") + assert p._ramp_router_api_key_override is None + p.learn_ramp_router_api_key("first") + p.learn_ramp_router_api_key("second") + assert p._ramp_router_api_key_override == "first" + + +def test_ramp_router_fetcher_returns_empty_without_credentials() -> None: + assert HttpPricingFetcher().fetch_ramp_router() == {} + + +def _capture_requests_get(fn): # type: ignore[no-untyped-def] + calls: list[tuple[str, dict[str, str] | None]] = [] + + class _FakeResp: + def raise_for_status(self) -> None: + pass + + def json(self) -> dict[str, Any]: + return _ROUTER_CATALOG + + def _fake_get(url: str, headers: dict[str, str] | None = None, timeout: float | None = None) -> _FakeResp: + calls.append((url, headers)) + return _FakeResp() + + import requests as _requests + + orig = _requests.get + _requests.get = _fake_get # type: ignore[assignment] + try: + result = fn() + finally: + _requests.get = orig # type: ignore[assignment] + return calls, result + + +def test_ramp_router_fetcher_accepts_a_learned_key_and_parses_the_catalog() -> None: + fetcher = HttpPricingFetcher() + calls, table = _capture_requests_get(lambda: fetcher.fetch_ramp_router(api_key="sk-router-learned")) + assert calls == [(RAMP_ROUTER_MODELS_URL, {"Authorization": "Bearer sk-router-learned"})] + assert table["gpt-5.4-nano"].input == Decimal("0.0000002") + + +def test_ramp_router_fetcher_explicit_config_key_wins_over_learned_key() -> None: + fetcher = HttpPricingFetcher(ramp_router_api_key="configured-key") + calls, _ = _capture_requests_get(lambda: fetcher.fetch_ramp_router(api_key="sk-router-learned")) + assert calls == [(RAMP_ROUTER_MODELS_URL, {"Authorization": "Bearer configured-key"})] + + +def test_ramp_router_catalog_carries_the_ttl_split_write_rates() -> None: + """haiku publishes cache_write_input_5m $1.25/M and _1h $2/M — 1.25x and 2x its input + rate — while its lump cache_write_input is "0" (None). OpenAI's luna is the mirror + image: a lump rate and no split.""" + haiku = _ROUTER_TABLE["claude-haiku-4-5"] + assert haiku.cache_write is None + assert haiku.cache_write_5m == Decimal("0.00000125") + assert haiku.cache_write_1h == Decimal("0.000002") + luna = _ROUTER_TABLE["gpt-5.6-luna"] + assert luna.cache_write == Decimal("0.00000025") + assert luna.cache_write_5m is None and luna.cache_write_1h is None + + +def _messages_usage(**counts: int) -> CanonicalUsage: + return CanonicalUsage( + model="claude-haiku-4-5", provider="ramp_router", api="ramp_router_messages", **counts + ) + + +def test_ttl_split_write_bills_each_part_at_its_own_rate_dashboard_reconciled() -> None: + """Router's dashboard, 2026-09-04, `/v1/messages`, haiku: 16 in + 20,113 written (5m) + + 5 out charged $0.02518225 — exactly 16 x $1/M + 20113 x $1.25/M + 5 x $5/M.""" + usage = _messages_usage(input=16, output=5, cache_write=20113, cache_write_5m=20113) + b = compute_cost(usage, _ROUTER_TABLE["claude-haiku-4-5"], Decimal(1)) + assert b.base == "0.02518225" + assert set(b.fields) == {"input", "cache_write_5m", "output"} + assert b.fields["cache_write_5m"]["tokens"] == "20113" + # The lump was consumed by the split — no second line for the same tokens. + assert "cache_write" not in b.fields + # And the token total counts the write ONCE (the split is a breakdown, not an addition). + assert deoverlapped_token_total(usage) == 16 + 5 + 20113 + + +def test_ttl_split_with_both_ttls_and_a_lump_remainder() -> None: + """1h at 2x, 5m at 1.25x, and 100 unattributed write tokens fall back to the lump rate + when one exists.""" + price = ModelPrice( + source="ramp_router", + input=Decimal("0.000001"), + cache_write=Decimal("0.0000011"), + cache_write_5m=Decimal("0.00000125"), + cache_write_1h=Decimal("0.000002"), + ) + usage = _messages_usage(input=10, cache_write=1100, cache_write_5m=600, cache_write_1h=400) + b = compute_cost(usage, price, Decimal(1)) + assert b.fields["cache_write_5m"]["tokens"] == "600" + assert b.fields["cache_write_1h"]["tokens"] == "400" + assert b.fields["cache_write"]["tokens"] == "100" + # 10 x 1e-6 + 600 x 1.25e-6 + 400 x 2e-6 + 100 x 1.1e-6 + assert b.base == "0.00167" + + +def test_ttl_split_never_bills_more_than_the_lump_reports() -> None: + """A surface whose split exceeds its lump is misreporting; the lump is authoritative.""" + usage = _messages_usage(input=10, cache_write=100, cache_write_5m=150, cache_write_1h=80) + b = compute_cost(usage, _ROUTER_TABLE["claude-haiku-4-5"], Decimal(1)) + assert b.fields["cache_write_5m"]["tokens"] == "100" + assert "cache_write_1h" not in b.fields + assert "cache_write" not in b.fields + + +def test_ttl_split_is_inert_without_split_rates_openrouter_anthropic_unchanged() -> None: + """OpenRouter publishes one `input_cache_write` for Anthropic. The split counts are + reported by native Anthropic too, and must change nothing there: the lump bills at the + lump rate exactly as before this code existed.""" + price = ModelPrice(source="openrouter", input=Decimal("0.000001"), cache_write=Decimal("0.00000125")) + usage = CanonicalUsage( + model="claude-haiku-4-5", + provider="anthropic", + api="native", + input=16, + cache_write=20113, + cache_write_5m=20113, + ) + b = compute_cost(usage, price, Decimal(1)) + assert set(b.fields) == {"input", "cache_write"} + assert b.fields["cache_write"]["tokens"] == "20113" + assert b.base == "0.02515725"