diff --git a/CHANGELOG.md b/CHANGELOG.md index 071959f..1478537 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,21 @@ All notable changes to this project will be documented here. Format follows [Kee ## [Unreleased] +### Changed + +- **README restructured.** The gateway deep-dives (Cloudflare, Databricks, Snowflake, Ramp Router) and the provider coverage/token-semantics tables moved to `docs/`, verbatim; the README keeps a short quickstart per provider and per gateway with a link to each full guide. + +### Added + +- **Ramp Router support: a customer pointing a client at Router billed nothing at all.** Router is an OpenAI-Responses-compatible gateway in front of OpenAI, Anthropic, Google Vertex, Fireworks and xAI, so a wrapped OpenAI client aimed at it made real, paid calls that produced **zero events** — the SDK had no way to tell a Router-routed call from a direct OpenAI one. The trick that identifies Cloudflare Workers AI cannot work here: Router's model ids are account-specific and opaque (its docs: "Never invent one or reuse a provider's public model name"), and an Anthropic-served response arrives in OpenAI's schema, byte-indistinguishable from a real OpenAI one. `base_url` is the only signal, read once at wrap time so the hot path gains no per-call work — and matched on the parsed HOST, because `"api.router.com" in base_url` also matches `https://evil.example.com/api.router.com/v1` and would stamp an unrelated endpoint's traffic as Router-served. + - **The model that answered is billed, not the one requested.** Two Router features make those diverge: a `models` fallback list sends no `model` field at all, and Switchyard routing can serve a different model than the one asked for. Router answers with a resolved bare vendor snapshot (`openai:gpt-5.4-nano` in, `gpt-5.4-nano-2026-03-17` out), which is what bills; the served tier is read from the response's own top-level `service_tier` field. As a fallback for an id that reaches the adapter unresolved, an explicit `provider:provider-model[:service-tier]` candidate is split on the **first** colon only — Fireworks candidates carry a path as their model segment (`fireworks:accounts/fireworks/models/kimi-k2p7-code`), so a naive split silently loses everything after the second separator. The service tier is matched against Router's documented set rather than read as "text after the last colon": a model segment can contain a colon of its own, and mistaking one for a tier renames the model and splits it into a second row in Lago against the live path's own name. + - **Price mode deliberately misses, and falls back to exact token events.** Router is stamped as its own provider, matching no vendor in the price tables, for three separate money reasons rather than as a shortcut. A **BYOK-served request costs $0** through Router ("your provider bills you directly and Ramp Router does not charge for that usage") and nothing in the response says which key served it, so a list-price lookup bills the full amount for usage Router never charged for. A **non-default service tier** does not bill at the published base rate — Router's own catalog says tiers "may use different rates". And the **overlap semantics belong to Router, not the served vendor** — measured live on an Anthropic-served model: Router normalizes the _numbers_ to OpenAI's convention, not just the schema (a warm `cache_control` call reports the cached block inside `input_tokens`; reasoning reports inside `output_tokens`), so `ramp_router` carries its own `OPENAI_SHAPED_APIS` entry, and stamping the served vendor would de-overlap with the wrong convention whenever that vendor's native one differs. Token mode, the default, is exact either way. The served tier is recorded in `extras` (diagnostic only — `extras` is not serialized into Lago events), and the catalog's own `router.pricing` rates make a Router price mode buildable later — but the response still cannot say whether a BYOK key served the call for $0, so token counts stay the honest default. + - **No backfill adapter, stated rather than invented.** Router is the first gateway here with no programmatic usage surface: its only routes are `GET /v1/models`, `POST /v1/responses`, `POST /v1/messages` and `POST /v1/messages/count_tokens`, and usage lives in the dashboard. An "analytics API" appears exactly once across all 20 documentation pages, in a limits table, with no path, auth or record shape. An adapter over guessed field names would have tests proving only that it matches the guess, so `src/lago_agent_sdk/gateway/__init__.py` carries the seam and the reason instead. + - **Nothing is skipped as a gateway cache hit**, because Router has no response cache — "Self-service Router response caching ... is not currently configurable". There is no `cf-aig-cache-status` analogue to read, and `_is_cache_hit` is untouched in both wrappers. Provider _prompt_ caching does pass through and those tokens bill normally. + - **Rebased onto the provider-hint table, the token-semantics roster and the REST dedup rather than around them.** Router's detection is the one hint that must match a HOST instead of a path, so `_provider_hint_for` keeps the path table's first say and gains a parsed-host arm after it. `ramp_router` joins `TOKEN_BILLED_PROVIDERS` — a structurally unpriceable provider must route to token events without a per-call price-miss report, the same decision Databricks and Snowflake carry — and joins `KNOWN_PROVIDERS` with its convention recorded in `OPENAI_SHAPED_APIS` — first entered conservatively as additive, then settled by measurement against a live account: Router normalizes the numbers to OpenAI's convention on cache and reasoning alike, so the guard reads Router counts as OpenAI-shaped. Both decisions are pinned by tests that fail if either set entry is reverted. + - **The `total_tokens` guard reads Router's stamped `api`, not the surface the call arrived on.** The adapter reassigns `api` to `ramp_router` after parsing the model, and the guard originally ran before that — so it asked `token_semantics("ramp_router", "responses")`, which matches no subset set and answers all-additive, while `compute_cost` and `deoverlapped_token_total` read the stamped `api="ramp_router"` and de-overlapped the same row as subset: the exact divergence `token_semantics.py` exists to make impossible. Unreachable on every captured payload — all ten report `total == input + output`, streamed included — but the guard exists for the day Router misreports, and on that day it under-folds a genuine remainder by `cache_read + reasoning`; where that over-count exceeds the declared total it suppresses the fold entirely, dropping generated tokens with no `extras` key and no `on_error`. The stamp now lands above the guard. +- 57 new tests. The price-mode ones load a real price table and bill the same model both directly and through Router, because the first version of them passed against a table that had **silently failed to load** — everything misses when nothing is priced, so asserting a miss on its own proved nothing. Captured fixtures from a live Router account back the seven wire-behaviour questions (catalog with `router.pricing`, buffered + fallback + streamed calls, the cache pair that settled the overlap question, a reasoning call, and three error families), written by a capture script that scrubs in the same step it writes and swept for credentials, emails and routable addresses before committing. The unit-test shapes stay hand-built to the documented schema where only the SDK's own decisions are under test. + ### Fixed - **A `views=("rest",)` backfill of a window the live path already billed is now rejected by Lago instead of billing every call twice.** The wrapped OpenAI client stamps each Cortex call's events with the id the reader derives from that call's row — `sfc_rest__` — read from the response's `x-snowflake-request-id` header, which is byte-identical to the REST view's `REQUEST_ID` (measured live). Both paths build the key through one shared `snowflake_event_id` helper, so the two spellings cannot drift apart silently. The dedup holds only when the backfill runs with the default `event_id_prefix` and resolves the same subscription the live path billed; both constraints are documented on `backfill_snowflake`. Streamed calls carry the key too — the header is readable off the stream's own response before the body is consumed (verified live). A missing header, or a client without `.with_raw_response`, keeps today's per-event UUID; a real-OpenAI call's ids are unchanged even if a proxy injects the header, because the keying is gated on the Snowflake provider hint rather than on the header's presence. One residual is documented rather than closed: a cache-creation call's cached block still double-counts on a backfilled live window — the wire reports creations as reads, so the two sides bill that component under different metrics and different ids; the call's input and output are deduplicated. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 0c67e57..549a787 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -71,6 +71,7 @@ uv lock --upgrade-package X # bump a single package - `src/lago_agent_sdk/gateway/` — second front door: gateway usage logs → `CanonicalUsage`, for backfill - `tests/unit/` — unit tests, organized to mirror `src/` - `tests/unit/adapters/fixtures/` — captured real provider responses, used by adapter tests +- `tests/unit/adapters/fixtures/capture_*.{py,ts}` — capture scripts; each reads its credential from the environment and scrubs secrets in the same step that writes a fixture into the tree ## Adding a provider diff --git a/README.md b/README.md index 0b12db5..4ed0bb7 100644 --- a/README.md +++ b/README.md @@ -1,16 +1,25 @@ -# lago-agent-sdk - -Instrument LLM clients and emit usage events to [Lago](https://www.getlago.com) for billing. +

+ + + + Lago + + +

+ +

+ lago-agent-sdk — Instrument LLM clients and emit usage events to Lago for billing. +

```text - ┌──────────────┐ -your code ──────► │ wrapped client│ ──► provider (Bedrock / Mistral / …) - └──────┬───────┘ - │ (extract usage) - ▼ - ┌──────────────┐ - │ Lago events │ ──► api.getlago.com - └──────────────┘ + ┌────────────────┐ + your code ────────► │ wrapped client │ ───► provider (Bedrock / Mistral / …) + └───────┬────────┘ + │ (extract usage) + ▼ + ┌────────────────┐ + │ Lago events │ ───► api.getlago.com + └────────────────┘ ``` ## What it does @@ -72,8 +81,6 @@ resp = client.messages.create( sdk.flush() ``` -Works with `Anthropic` and `AsyncAnthropic`. Both `messages.create(..., stream=True)` and the `messages.stream(...)` context manager are instrumented — usage is captured from the final `message_delta` event in either case. - ## Quickstart — Mistral ```python @@ -107,10 +114,6 @@ resp = client.chat.completions.create( sdk.flush() ``` -Works with `OpenAI` and `AsyncOpenAI`. Covers both **Chat Completions** (`client.chat.completions.create`) and the newer **Responses API** (`client.responses.create`), sync + streaming. For streaming, the wrapper auto-injects `stream_options={"include_usage": True}` so the final chunk carries usage data — without it OpenAI emits no usage on streamed responses. - -**Reasoning tokens** (`llm_reasoning_tokens`) populate automatically when you call an o-series model (`o4-mini`, `o1`, etc.) — OpenAI is the first provider to expose this metric separately. - ## Quickstart — Gemini ```python @@ -127,231 +130,68 @@ resp = client.models.generate_content( sdk.flush() ``` -Wraps the modern `google-genai` SDK (`from google import genai`). Covers `client.models.generate_content` + `generate_content_stream`, sync + async (via `client.aio.models`). +Every provider is covered sync + async + streaming. The full coverage matrix, the token fields each provider populates, and the per-provider quirks that affect billing (cache/reasoning overlap semantics, OpenAI's `stream_options` auto-inject, Gemini's additive `thoughts_token_count`) live in [docs/providers.md](docs/providers.md). -**Reasoning tokens** populate automatically on Gemini 2.5 — the model reasons internally by default and surfaces `thoughts_token_count` (see the note on reasoning semantics below). +## Gateways -## Cloudflare AI Gateway +`wrap()` also detects a client pointed at a gateway and bills what the gateway actually did — each guide covers backfill, attribution and the measured billing caveats for that gateway. -Point any of the clients above at your gateway instead of the provider directly — `wrap()` detects it and bills correctly, with two behaviors on top of the plain provider case: +### Cloudflare AI Gateway -```python -from anthropic import Anthropic -from lago_agent_sdk import LagoSDK +Point any supported client at your gateway URL; cache hits (`cf-aig-cache-status: HIT`) are not billed, and Workers AI models get priced automatically from Cloudflare's published rates. -sdk = LagoSDK(api_key="...", default_subscription_id="sub_acme") +```python client = sdk.wrap(Anthropic( api_key="...", base_url=f"https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/anthropic", default_headers={"cf-aig-authorization": f"Bearer {gateway_auth}"}, )) -client.messages.create(model="claude-sonnet-4-6", max_tokens=200, messages=[{"role": "user", "content": "Hello"}]) -sdk.flush() ``` -- **Gateway cache hits aren't billed.** If the gateway serves a response from its own cache (`cf-aig-cache-status: HIT`), the provider was never called, so the SDK skips emitting for that response. -- **Workers AI gets priced automatically.** Wrap an OpenAI-shaped client against the gateway's `/compat` endpoint (`model="workers-ai/@cf/..."`) with `pricing_mode="price"`, and the SDK fetches Cloudflare's own published Workers AI rates in the background — no separate price table to maintain. +Full guide, including backfill from the gateway's Logs API: [docs/cloudflare.md](docs/cloudflare.md). -For usage that already happened, backfill straight from the gateway's own Logs API instead of replaying calls — `lago_agent_sdk.gateway.adapters` extracts a log entry into `CanonicalUsage` and bills Cloudflare's own metered `cost` for it, so there's no separate price lookup and re-running over the same window never double-bills: +### Databricks AI Gateway -```python -from lago_agent_sdk.gateway.adapters import extract_cloudflare_log, resolve_subscription - -for entry in fetch_gateway_logs(): # GET .../ai-gateway/gateways/{id}/logs - usage = extract_cloudflare_log(entry) - sub = resolve_subscription(entry) or "sub_default" # from the call's cf-aig-metadata, if set - sdk.emit(usage, subscription=sub, mode="price", usd_cost=entry.get("cost") or 0, event_id=f"cf_{entry['id']}") -sdk.flush() -``` - -The sections above are the complete picture; runnable notebooks are kept out of the repo (see `.gitignore`) because their saved cells and outputs carry account identifiers and live subscription ids. - -**Gateway-routed calls are billed at the gateway's metered cost.** Cloudflare reports its own `cost` per log entry and the backfill passes that straight through, so Lago reconciles against the dashboard you actually look at. One measured consequence to be aware of: that field excludes additive *reasoning* tokens, so a thinking-heavy Gemini call bills about 4% of what Google charges (verified live at 22.8x on one call, 39.6x on another — the ratio tracks each prompt's thinking-to-output ratio). Cloudflare is exact on input, output, cache-read and cache-write. - -**If you hand-roll a poller, don't use `urllib`.** `gateway.ai.cloudflare.com` returns `403` with body `error code: 1010` to `Python-urllib` — its bot-signature check. Any other User-Agent passes, and `requests` (which this SDK uses) is fine. The failure looks like an auth error because the body is otherwise empty. - -## Databricks AI Gateway - -Unlike Cloudflare, Databricks has **no unified endpoint** — each provider is reachable only through its own native surface, and two of them use the same `openai.OpenAI` class. Which `base_url` you point at decides how the call is priced. - -**Databricks-hosted foundation models** (`system.ai.*`) — billed by Databricks in DBUs: +BYOK calls bill dollar cost at the vendor's rates; Databricks-hosted models (`system.ai.*`) bill token counts, and `sdk.backfill_databricks(source, "7 days")` fills in what `wrap()` didn't see. ```python -from openai import OpenAI -from lago_agent_sdk import LagoSDK - -sdk = LagoSDK(api_key="...", default_subscription_id="sub_acme") client = sdk.wrap(OpenAI( api_key=DATABRICKS_TOKEN, base_url=f"{DATABRICKS_HOST}/ai-gateway/mlflow/v1", default_headers={"Databricks-Ai-Gateway-Request-Tags": json.dumps({"lago_subscription": "sub_acme"})}, )) -client.chat.completions.create(model="system.ai.llama-4-maverick", messages=[{"role": "user", "content": "Hi"}]) -``` - -**Your own vendor key (BYOK)** — Anthropic via its native passthrough, note `api_key="unused"` because the real credential goes in `Authorization`, and the Unity Catalog connection holding your Anthropic key is named in `Databricks-Model-Provider-Service`: - -```python -from anthropic import Anthropic -client = sdk.wrap(Anthropic( - api_key="unused", - base_url=f"{DATABRICKS_HOST}/ai-gateway/anthropic", - default_headers={ - "Authorization": f"Bearer {DATABRICKS_TOKEN}", - "Databricks-Model-Provider-Service": "workspace.default.anthropickey", - }, -)) ``` -OpenAI BYOK is the same `OpenAI` class as the hosted example, against `/ai-gateway/openai/v1` with its own `Databricks-Model-Provider-Service`. - -### What gets billed +Full guide, including BYOK setup, backfill and gotchas: [docs/databricks.md](docs/databricks.md). -| Path | Live `wrap()` | Backfill | -|---|---|---| -| BYOK (OpenAI / Anthropic) | **dollar cost**, priced from the vendor's published rates | dollar cost from Databricks' own `external_model_spend` | -| Hosted (`system.ai.*`) | **token counts** | **token counts** | +### Snowflake Cortex -BYOK prices live because you pay the vendor directly, so the vendor's rate *is* your cost — verified against Databricks' own metered spend on 38 of 38 real buckets, exactly. Hosted models bill in DBUs against a rate card published only as HTML and present in no system table, so there is no rate to look up: those calls emit token counts instead of a dollar cost. That is the complete answer for them, not a degraded one, so it is **not** reported as an error — `TOKEN_BILLED_PROVIDERS` lists the providers this applies to, and the SDK notes it once per model at info level rather than warning on every call. A genuine price miss — a cold table, an unmatched model name — still reports through `on_error` as before. - -**Hosted dollars exist, and are deliberately not billed from.** `system.billing.usage` × `list_prices` (or `account_prices` for your contract rate) does yield exact USD per hour and endpoint. It is not used because it comes from a *different Databricks screen* than the gateway view: it carries no `request_tags`, so per-subscription splits would be ours rather than Databricks', and it lags the gateway by roughly a day — measured at ~19h on a live workspace. Every number this connector sends is one you can find on a Databricks **gateway** page, which is the property that makes it checkable. - -**Grouping matches the Databricks page.** Each backfilled event carries the grouping key of the surface it came from — `endpoint_name` for hosted, `bucket` (the hour) for BYOK. Group Lago by `endpoint_name` and you get the AI Gateway → Usage table row for row. Pass `dimensions={...}` to add your own keys; yours win on a name collision. - -**Don't run the live path and the backfill over the same hosted traffic.** Both emit token events, with different `transaction_id`s, so Lago accepts both and the counts double. Pick one per traffic stream: `wrap()` for real time, the backfill for completeness. - -`Databricks-Ai-Gateway-Request-Tags` is what makes attribution work. It lands in `request_tags` on `system.ai_gateway.usage` **and** is a first-class aggregation dimension on `external_model_spend`, so tagging `lago_subscription` means BYOK cost arrives already split per subscription — no apportioning needed. - -### Backfill — give it a window, it does the rest +The Cortex REST surface wraps like any OpenAI-compatible client; the AI SQL functions (`AI_COMPLETE`, …) have no client to wrap and are backfilled from Snowflake's usage views. Everything bills as token counts. ```python -from lago_agent_sdk.gateway.databricks import DatabricksSource +client = sdk.wrap(OpenAI( + base_url=f"https://{os.environ['SNOWFLAKE_ACCOUNT']}.snowflakecomputing.com/api/v2/cortex/v1", + api_key=os.environ["SNOWFLAKE_PAT"], +)) -source = DatabricksSource.from_env() # DATABRICKS_HOST / _TOKEN / _WAREHOUSE_ID -print(sdk.backfill_databricks(source, "7 days", default_subscription="sub_default")) -sdk.flush() -# {'cost': 60, 'tokens': 47, 'skipped': 0} +sdk.backfill_snowflake(SnowflakeSource.from_env(), "7 days", default_subscription="sub_default") ``` -Pass a `datetime` instead of `"7 days"` for an exact lower bound, and `unified=True` to bill the whole window to `default_subscription` regardless of per-call tags. - -The window reads **whole closed hours only**: it is floored to the hour at both ends and the current, still-aggregating hour is excluded, because `external_model_spend` is an hourly aggregate whose row for an hour does not exist until that hour closes. So the newest hour of traffic arrives on the next run — pass a window comfortably wider than your run interval, since this reader keeps no cursor. +Full guide, including cache semantics, dedup, attribution via `QUERY_TAG` and account setup: [docs/snowflake.md](docs/snowflake.md). -Unlike Cloudflare's single paginated GET, this one is worth having in the SDK — hand-rolling it is ~100 lines with three money-losing traps in them. The Statement Execution API returns only **chunk 0** inline, so a wide window silently truncates and bills a fraction of it with no error. A BYOK call appears in **both** `ai_gateway.usage` and `external_model_spend`, so billing both charges twice. And `transaction_id` is unique account-wide, so an unscoped row id blocks that row from ever reaching a second subscription. +### Ramp Router -To inspect a window before billing it, or to route rows yourself, read them directly — each row is already shaped for `emit()`: +An OpenAI-Responses-compatible gateway in front of OpenAI, Anthropic, Google Vertex, Fireworks and xAI. The model that answered is the one billed — Router can serve a different model than the one requested. ```python -for row in source.read_usage("7 days"): - print(row.usage.model, row.subscription, row.usd_cost) # usd_cost is None for hosted -``` - -Reading the system tables needs a PAT with the **`sql`** scope plus a SQL warehouse — the live calls above need neither. The pure `extract_databricks_log(row)` / `resolve_databricks_subscription(row)` functions stay available from `lago_agent_sdk.gateway.adapters` if you already have rows from `databricks-sql-connector` or your own warehouse job. - -**One cost note:** a SQL warehouse is a real cost centre. Measured on a test workspace, the warehouse queries cost roughly 1,500× the model-serving usage they were reporting on. Run the backfill as one query over a wide window, never as a tight polling loop. - -### Gotchas worth knowing - -- **`gpt-oss` models inflate input by ~100 tokens** from a server-injected preamble — a 2-character prompt bills 102. Not an SDK error. -- **`claude-opus-4-5` does not cache through this gateway**: reproducibly `cache_read`/`cache_write` of 0 with the full prompt billed as input, on a request shape where `claude-sonnet-4-5` caches fine. An opus customer silently gets no cache discount. -- **Hosted models report three different name strings.** `system.ai.llama-4-maverick` and `databricks-llama-4-maverick` both work as requests, and the response echoes a third (`meta-llama-4-maverick-040225`). Pricing keys off the resolved name, so reconciling by requested id will not line up. -- **Embeddings** work on `/ai-gateway/mlflow/v1/embeddings` and report input only — no `completion_tokens` at all. - -## Snowflake Cortex - -Snowflake serves Cortex two ways, and they need two different halves of this SDK. That split is the thing to understand before anything else here. - -| Surface | How you call it | How Lago sees it | -|---|---|---| -| **Cortex REST** — `/api/v2/cortex/v1` | an OpenAI-compatible client you hand to `wrap()` | live, per call | -| **AI SQL functions** — `AI_COMPLETE`, `AI_EMBED`, … | SQL, inside the warehouse | **backfill only** — there is no client to wrap | - -**Everything on this path bills as token counts.** Snowflake meters Cortex in credits against a rate card that lives in no view, so there is no per-request dollar figure to pass through and no price mode here. `provider` is `"snowflake"`, which is listed in `TOKEN_BILLED_PROVIDERS`, so a customer running `pricing_mode="price"` globally still gets token events for Snowflake rows — with no price-miss error, because a structural absence of a rate card is not a lookup failure. - -### Live — the REST surface - -```python -import os -from openai import OpenAI - client = sdk.wrap( - OpenAI( - base_url=f"https://{os.environ['SNOWFLAKE_ACCOUNT']}.snowflakecomputing.com/api/v2/cortex/v1", - api_key=os.environ["SNOWFLAKE_PAT"], - ) + OpenAI(api_key=os.environ["RAMP_ROUTER_API_KEY"], base_url="https://api.router.com/v1"), + subscription="sub_acme", ) +client.responses.create(model=os.environ["RAMP_ROUTER_MODEL"], input="Summarize this invoice.") ``` -The `base_url` is what identifies these calls as Snowflake rather than OpenAI — an OpenAI-shaped endpoint says nothing about whose tokens they are. Use `max_completion_tokens`; Cortex rejects `max_tokens` outright. - -**Cortex's `cached_tokens` is additive, the opposite of OpenAI's convention.** A cached call reports `prompt_tokens: 7`, `cached_tokens: 8745`, `completion_tokens: 6`, `total_tokens: 8758` — the cached block is *not* inside `prompt_tokens`. Caching also only happens when you send an explicit `cache_control: {"type": "ephemeral"}` content part; the same prompt twice without one reports zero cached both times. - -**The wire cannot tell a cache creation from a read; the view can.** A creation call reports the same `cached_tokens` with `cache_write_tokens: 0`, so the live `wrap()` path bills a creation as a cache read (`llm_cached_input_tokens`). The REST view records the same call as `cache_write_input`, so a backfilled row bills `llm_cache_creation_tokens` instead. Verified live on a matched pair (INT-230): identical wire usage, one `cache_write_input` row and three `cache_read_input` rows. If you price creation and read differently, know that live-path traffic reports everything at the read metric. It also bounds the REST-view dedup: a backfilled creation row emits its cached block under a *different* transaction id than the live path did (`_tok_cache_write` vs `_tok_cache_read`), so that one component bills on both metrics if you backfill a live-billed window — the call's input and output stay deduplicated. - -### Backfill — the SQL functions surface - -```python -from lago_agent_sdk.gateway.snowflake import SnowflakeSource - -source = SnowflakeSource.from_env() # SNOWFLAKE_ACCOUNT / _PAT, plus a warehouse -print(sdk.backfill_snowflake(source, "7 days", default_subscription="sub_default")) -sdk.flush() -# {'tokens': 47, 'skipped': 0} -``` - -Two counts, and there cannot be more: `tokens` is what got billed, `skipped` is what did not. Both causes of a skip are also reported through `on_error` with `where="backfill"`, so an automated caller notices a gap without inspecting the return value. - -**It reads the functions view only.** The REST view reports the calls `wrap()` already billed above. Both sides derive one idempotency key from the call's `REQUEST_ID` — the wrapper reads it off the `x-snowflake-request-id` response header — so Lago rejects a backfill's copies as duplicate `transaction_id`s instead of billing them twice. That protection holds only when the backfill uses the default `event_id_prefix` and resolves the same subscription the live path billed, and it does not cover a cache-creation call's cached block (see the cache note above) or calls billed without the header. So the rule stands: pass `views=("rest",)` only for REST traffic `wrap()` never saw: - -```python -sdk.backfill_snowflake( - source, - "7 days", - default_subscription="sub_default", - views=("rest",), # ONLY if no wrapped client is billing this traffic -) -``` - -The window reads **whole closed hours only** — floored at both ends, with the current hour excluded, because a bucket is not complete until its hour closes and billing it early burns that row's idempotency key so the correction is rejected as a duplicate. The newest hour therefore arrives on the next run: pass a window comfortably wider than your run interval, since this reader keeps no cursor. One more boundary Lago itself draws: events stamped before the subscription started are **accepted and silently never billed** — a window reaching back past the subscription's start reports its rows as billed while nothing lands in usage, so start backfills at the subscription's start date. - -**Attribution comes from `QUERY_TAG`**, the only customer-injectable key on either view, and the same `lago_subscription` key Cloudflare and Databricks read from their own metadata: - -```sql -ALTER SESSION SET QUERY_TAG = '{"lago_subscription": "sub_123"}'; -SELECT AI_COMPLETE('claude-sonnet-4-5', 'summarize this'); -``` - -By default the tag is the **only** attribution source: an untagged row falls to `default_subscription`, and to a skip (counted, reported) when there is none. `role_names` and `user_id` are opt-in — `subscription_order=("query_tag", "role_names")` — for accounts that really map one Snowflake role or user to one customer. They are not in the default because every live row carries both, so they would swallow untagged rows and bill them to a Snowflake identity instead of your default: that is a wrong subscription, and unlike a skip it is not recoverable. - -**Grouping matches the view.** Each event carries the grouping key of the surface it came from — `function_name` + `model_name` for functions rows, `inference_region` for REST — so a `GROUP BY` on the view and the same grouping in Lago line up. `dimensions={...}` adds your own; yours win on a collision. - -### A long-running query is deferred, not guessed at - -`IS_COMPLETED` means "did the query finish *in this aggregation window*", and these views are hour-bucketed — Snowflake documents a query running 5:30→8:30 writing **four rows, one per hour, all sharing one `QUERY_ID`**. Two things follow. The key `{prefix}_{kind}_{sub}_{QUERY_ID}` collides across those rows, and whether each row's `METRICS` is incremental or cumulative is unmeasured. On a 3-hour query using 3,800 input tokens, summing four incremental rows bills 3,800 and summing four cumulative ones bills 9,500; billing only the last row gives 3,800 if cumulative and 900 if incremental. - -So a `QUERY_ID` that yields more than one row in a window is **not billed**. It is counted in `skipped`, reported through `on_error`, listed on `source.deferred_rows`, and billable once the shape is settled. Guessing over-bills by 2.5× or under-bills by 76%, neither recoverable once invoiced. Every query ever observed on a real account finished inside one bucket, so this fires on a shape nobody has seen. - -To inspect a window before billing it, read the rows directly — `read_usage()` is a generator and each row is already shaped for `emit()`: - -```python -for row in source.read_usage("7 days"): - print(row.kind, row.usage.model, row.subscription, row.occurred_at) -``` - -### Setting up the account - -Reading the views needs a PAT plus a **running warehouse**; the live calls above need neither. Four things block a first-time setup and none of them says so clearly: - -- **Model access moved to RBAC.** `CORTEX_MODELS_ALLOWLIST` is deprecated and accepts only `'NONE'`; you need `GRANT APPLICATION ROLE SNOWFLAKE."CORTEX-MODEL-ROLE-ALL" TO ROLE …`, without which the role can call zero models. -- **A PAT's `ROLE_RESTRICTION` is a quoted string literal**, so it is case-sensitive — `'LAGO_CORTEX_ROLE'`, not the lowercase spelling that works everywhere else. -- **A warehouse with `AUTO_RESUME = FALSE`** fails every statement with "warehouse is suspended", which reads like a privilege error. -- **A PAT cannot authenticate without an active network policy.** Prefer reusing a broad one: recovering from an IP lockout needs Snowflake Support, with no self-service path back. - -Error code `003001` has four distinct causes — account entitlement, unknown model, model not granted to the role, and a bare fine-tuned model name — so it is not diagnostic on its own. - -**One cost note:** a SQL warehouse is a real cost centre. Measured on the equivalent Databricks setup, warehouse queries cost roughly 1,500× the model-serving usage they reported on. Run the backfill as one query over a wide window, never as a tight polling loop. - -The pure `extract_snowflake_functions_log(row)` / `extract_snowflake_rest_log(row)` / `resolve_snowflake_subscription(row)` functions stay available from `lago_agent_sdk.gateway.adapters` if you already have rows from your own warehouse job. +Full guide, including why price mode falls back to token events for Router traffic: [docs/ramp-router.md](docs/ramp-router.md). ## Multi-tenant — pick a subscription per call @@ -371,38 +211,6 @@ sdk = LagoSDK(api_key="...", default_subscription_id="sub_default") Backed by `contextvars` for safe propagation across `asyncio` tasks. -## Supported providers - -| Provider | Access | Status | -|---|---|---| -| AWS Bedrock | `Converse` (sync + stream) | ✓ | -| AWS Bedrock | `InvokeModel` (sync + stream), 7 model families | ✓ | -| Anthropic | native SDK (`messages.create` + `messages.stream`, sync + async) | ✓ | -| Mistral | native SDK (`chat.complete` + `chat.stream`) | ✓ | -| OpenAI | native SDK (`chat.completions.create` + `responses.create`, sync + async + stream) | ✓ | -| Google Gemini | native SDK (`google-genai`: `models.generate_content` + `generate_content_stream`, sync + async) | ✓ | - -## Token dimensions captured - -`CanonicalUsage` carries 11 numeric fields. Which ones populate depends on the provider: - -| Field | Lago metric code | Bedrock | Anthropic | Mistral | OpenAI | Gemini | -|---|---|---|---|---|---|---| -| input | `llm_input_tokens` | ✓ | ✓ | ✓ | ✓ | ✓ | -| output | `llm_output_tokens` | ✓ | ✓ | ✓ | ✓ | ✓ | -| cache_read | `llm_cached_input_tokens` | ✓ (Anthropic) | ✓ | ✓ (when cache hits) | ✓ (auto-cache) | ✓ (CachedContent API) | -| cache_write | `llm_cache_creation_tokens` | ✓ (Anthropic) | ✓ | ✗ | ✗ | ✗ | -| cache_write_5m / 1h | `llm_cache_write_5m/1h_tokens` | ✓ (Anthropic InvokeModel) | ✓ | ✗ | ✗ | ✗ | -| reasoning | `llm_reasoning_tokens` | ✗ (folded into output) | ✗ (folded into output, even with extended thinking) | ✗ (folded into output) | **✓ (o-series, subset)** | **✓ (Gemini 2.5, additive)** | -| tool_calls | `llm_tool_calls` | ✓ | ✓ | ✓ | ✓ | ✓ | -| audio_input | `llm_audio_input_tokens` | ✗ | ✗ | ✗ | ✓ (GPT-4o-audio) | ✓ (multimodal AUDIO) | -| audio_output | `llm_audio_output_tokens` | ✗ | ✗ | ✗ | ✓ (GPT-4o-audio) | ✓ (multimodal AUDIO) | -| image_input | `llm_image_input_tokens` | ✗ | ✗ | ✗ | ✗ | ✓ (multimodal IMAGE) | - -**Reasoning:** OpenAI's `reasoning_tokens` is a *subset* of `output` (already counted in `completion_tokens`). Gemini's `thoughts_token_count` is *additive* to `output` (`candidates + thoughts = total billable output`). - -**Cache/audio/image on OpenAI and Gemini are subsets of `input`, not additive.** Both providers count cached/audio/image tokens *within* their input total, so summing `llm_input_tokens + llm_cached_input_tokens` (or `+ audio/image`) double-counts. Bill on `llm_input_tokens` alone; use the breakdown fields only for cost attribution (e.g. a discounted cache rate). - ## Pricing mode — send dollar cost instead of tokens By default the SDK emits **token counts** (`pricing_mode="tokens"`). Set `pricing_mode="price"` to instead emit the **dollar cost** of each call: `Σ(unit_price_per_token × tokens) × markup`. @@ -416,18 +224,9 @@ sdk = LagoSDK(api_key="...", config=LagoConfig( pricing_mode="price", # "tokens" (default) | "price" markup=1.2, # optional cost multiplier (1.2 = +20%) )) -client = sdk.wrap(anthropic_client) -# ... use the client normally ... ``` -Price mode emits one `llm_cost` event per priced field (input, output, cache, ...), each carrying `precise_total_amount_cents` for Lago's **dynamic charge model** plus a `token_type` property so a single billable metric can be grouped by both `model` and `token_type`. Prices come from public sources (OpenRouter for native providers, the AWS Bedrock price list for Bedrock), fetched and cached in the background — your LLM call is never blocked on pricing. If a price isn't available yet, the SDK falls back to token-count events and reports via `on_error` rather than under-billing. - -Per-call override via `extra_lago`: - -```python -client.messages.create(model="claude-...", messages=[...], - extra_lago={"mode": "price", "markup": 1.5}) -``` +Price mode emits one `llm_cost` event per priced field (input, output, cache, ...), each carrying `precise_total_amount_cents` for Lago's **dynamic charge model** plus a `token_type` property so a single billable metric can be grouped by both `model` and `token_type`. Prices come from public sources (OpenRouter for native providers, the AWS Bedrock price list for Bedrock), fetched and cached in the background — your LLM call is never blocked on pricing. If a price isn't available yet, the SDK falls back to token-count events and reports via `on_error` rather than under-billing. Per-call override: `extra_lago={"mode": "price", "markup": 1.5}`. ## Error policy diff --git a/docs/cloudflare.md b/docs/cloudflare.md new file mode 100644 index 0000000..21095b5 --- /dev/null +++ b/docs/cloudflare.md @@ -0,0 +1,40 @@ +# Cloudflare AI Gateway + +Point any of the supported clients at your gateway instead of the provider directly — `wrap()` detects it and bills correctly, with two behaviors on top of the plain provider case: + +```python +from anthropic import Anthropic +from lago_agent_sdk import LagoSDK + +sdk = LagoSDK(api_key="...", default_subscription_id="sub_acme") +client = sdk.wrap(Anthropic( + api_key="...", + base_url=f"https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/anthropic", + default_headers={"cf-aig-authorization": f"Bearer {gateway_auth}"}, +)) +client.messages.create(model="claude-sonnet-4-6", max_tokens=200, messages=[{"role": "user", "content": "Hello"}]) +sdk.flush() +``` + +- **Gateway cache hits aren't billed.** If the gateway serves a response from its own cache (`cf-aig-cache-status: HIT`), the provider was never called, so the SDK skips emitting for that response. +- **Workers AI gets priced automatically.** Wrap an OpenAI-shaped client against the gateway's `/compat` endpoint (`model="workers-ai/@cf/..."`) with `pricing_mode="price"`, and the SDK fetches Cloudflare's own published Workers AI rates in the background — no separate price table to maintain. + +## Backfill from the Logs API + +For usage that already happened, backfill straight from the gateway's own Logs API instead of replaying calls — `lago_agent_sdk.gateway.adapters` extracts a log entry into `CanonicalUsage` and bills Cloudflare's own metered `cost` for it, so there's no separate price lookup and re-running over the same window never double-bills: + +```python +from lago_agent_sdk.gateway.adapters import extract_cloudflare_log, resolve_subscription + +for entry in fetch_gateway_logs(): # GET .../ai-gateway/gateways/{id}/logs + usage = extract_cloudflare_log(entry) + sub = resolve_subscription(entry) or "sub_default" # from the call's cf-aig-metadata, if set + sdk.emit(usage, subscription=sub, mode="price", usd_cost=entry.get("cost") or 0, event_id=f"cf_{entry['id']}") +sdk.flush() +``` + +This page is the complete picture; runnable notebooks are kept out of the repo (see `.gitignore`) because their saved cells and outputs carry account identifiers and live subscription ids. + +**Gateway-routed calls are billed at the gateway's metered cost.** Cloudflare reports its own `cost` per log entry and the backfill passes that straight through, so Lago reconciles against the dashboard you actually look at. One measured consequence to be aware of: that field excludes additive *reasoning* tokens, so a thinking-heavy Gemini call bills about 4% of what Google charges (verified live at 22.8x on one call, 39.6x on another — the ratio tracks each prompt's thinking-to-output ratio). Cloudflare is exact on input, output, cache-read and cache-write. + +**If you hand-roll a poller, don't use `urllib`.** `gateway.ai.cloudflare.com` returns `403` with body `error code: 1010` to `Python-urllib` — its bot-signature check. Any other User-Agent passes, and `requests` (which this SDK uses) is fine. The failure looks like an auth error because the body is otherwise empty. diff --git a/docs/databricks.md b/docs/databricks.md new file mode 100644 index 0000000..b21ad81 --- /dev/null +++ b/docs/databricks.md @@ -0,0 +1,86 @@ +# Databricks AI Gateway + +Unlike [Cloudflare](cloudflare.md), Databricks has **no unified endpoint** — each provider is reachable only through its own native surface, and two of them use the same `openai.OpenAI` class. Which `base_url` you point at decides how the call is priced. + +**Databricks-hosted foundation models** (`system.ai.*`) — billed by Databricks in DBUs: + +```python +from openai import OpenAI +from lago_agent_sdk import LagoSDK + +sdk = LagoSDK(api_key="...", default_subscription_id="sub_acme") +client = sdk.wrap(OpenAI( + api_key=DATABRICKS_TOKEN, + base_url=f"{DATABRICKS_HOST}/ai-gateway/mlflow/v1", + default_headers={"Databricks-Ai-Gateway-Request-Tags": json.dumps({"lago_subscription": "sub_acme"})}, +)) +client.chat.completions.create(model="system.ai.llama-4-maverick", messages=[{"role": "user", "content": "Hi"}]) +``` + +**Your own vendor key (BYOK)** — Anthropic via its native passthrough, note `api_key="unused"` because the real credential goes in `Authorization`, and the Unity Catalog connection holding your Anthropic key is named in `Databricks-Model-Provider-Service`: + +```python +from anthropic import Anthropic +client = sdk.wrap(Anthropic( + api_key="unused", + base_url=f"{DATABRICKS_HOST}/ai-gateway/anthropic", + default_headers={ + "Authorization": f"Bearer {DATABRICKS_TOKEN}", + "Databricks-Model-Provider-Service": "workspace.default.anthropickey", + }, +)) +``` + +OpenAI BYOK is the same `OpenAI` class as the hosted example, against `/ai-gateway/openai/v1` with its own `Databricks-Model-Provider-Service`. + +## What gets billed + +| Path | Live `wrap()` | Backfill | +|---|---|---| +| BYOK (OpenAI / Anthropic) | **dollar cost**, priced from the vendor's published rates | dollar cost from Databricks' own `external_model_spend` | +| Hosted (`system.ai.*`) | **token counts** | **token counts** | + +BYOK prices live because you pay the vendor directly, so the vendor's rate *is* your cost — verified against Databricks' own metered spend on 38 of 38 real buckets, exactly. Hosted models bill in DBUs against a rate card published only as HTML and present in no system table, so there is no rate to look up: those calls emit token counts instead of a dollar cost. That is the complete answer for them, not a degraded one, so it is **not** reported as an error — `TOKEN_BILLED_PROVIDERS` lists the providers this applies to, and the SDK notes it once per model at info level rather than warning on every call. A genuine price miss — a cold table, an unmatched model name — still reports through `on_error` as before. + +**Hosted dollars exist, and are deliberately not billed from.** `system.billing.usage` × `list_prices` (or `account_prices` for your contract rate) does yield exact USD per hour and endpoint. It is not used because it comes from a *different Databricks screen* than the gateway view: it carries no `request_tags`, so per-subscription splits would be ours rather than Databricks', and it lags the gateway by roughly a day — measured at ~19h on a live workspace. Every number this connector sends is one you can find on a Databricks **gateway** page, which is the property that makes it checkable. + +**Grouping matches the Databricks page.** Each backfilled event carries the grouping key of the surface it came from — `endpoint_name` for hosted, `bucket` (the hour) for BYOK. Group Lago by `endpoint_name` and you get the AI Gateway → Usage table row for row. Pass `dimensions={...}` to add your own keys; yours win on a name collision. + +**Don't run the live path and the backfill over the same hosted traffic.** Both emit token events, with different `transaction_id`s, so Lago accepts both and the counts double. Pick one per traffic stream: `wrap()` for real time, the backfill for completeness. + +`Databricks-Ai-Gateway-Request-Tags` is what makes attribution work. It lands in `request_tags` on `system.ai_gateway.usage` **and** is a first-class aggregation dimension on `external_model_spend`, so tagging `lago_subscription` means BYOK cost arrives already split per subscription — no apportioning needed. + +## Backfill — give it a window, it does the rest + +```python +from lago_agent_sdk.gateway.databricks import DatabricksSource + +source = DatabricksSource.from_env() # DATABRICKS_HOST / _TOKEN / _WAREHOUSE_ID +print(sdk.backfill_databricks(source, "7 days", default_subscription="sub_default")) +sdk.flush() +# {'cost': 60, 'tokens': 47, 'skipped': 0} +``` + +Pass a `datetime` instead of `"7 days"` for an exact lower bound, and `unified=True` to bill the whole window to `default_subscription` regardless of per-call tags. + +The window reads **whole closed hours only**: it is floored to the hour at both ends and the current, still-aggregating hour is excluded, because `external_model_spend` is an hourly aggregate whose row for an hour does not exist until that hour closes. So the newest hour of traffic arrives on the next run — pass a window comfortably wider than your run interval, since this reader keeps no cursor. + +Unlike Cloudflare's single paginated GET, this one is worth having in the SDK — hand-rolling it is ~100 lines with three money-losing traps in them. The Statement Execution API returns only **chunk 0** inline, so a wide window silently truncates and bills a fraction of it with no error. A BYOK call appears in **both** `ai_gateway.usage` and `external_model_spend`, so billing both charges twice. And `transaction_id` is unique account-wide, so an unscoped row id blocks that row from ever reaching a second subscription. + +To inspect a window before billing it, or to route rows yourself, read them directly — each row is already shaped for `emit()`: + +```python +for row in source.read_usage("7 days"): + print(row.usage.model, row.subscription, row.usd_cost) # usd_cost is None for hosted +``` + +Reading the system tables needs a PAT with the **`sql`** scope plus a SQL warehouse — the live calls above need neither. The pure `extract_databricks_log(row)` / `resolve_databricks_subscription(row)` functions stay available from `lago_agent_sdk.gateway.adapters` if you already have rows from `databricks-sql-connector` or your own warehouse job. + +**One cost note:** a SQL warehouse is a real cost centre. Measured on a test workspace, the warehouse queries cost roughly 1,500× the model-serving usage they were reporting on. Run the backfill as one query over a wide window, never as a tight polling loop. + +## Gotchas worth knowing + +- **`gpt-oss` models inflate input by ~100 tokens** from a server-injected preamble — a 2-character prompt bills 102. Not an SDK error. +- **`claude-opus-4-5` does not cache through this gateway**: reproducibly `cache_read`/`cache_write` of 0 with the full prompt billed as input, on a request shape where `claude-sonnet-4-5` caches fine. An opus customer silently gets no cache discount. +- **Hosted models report three different name strings.** `system.ai.llama-4-maverick` and `databricks-llama-4-maverick` both work as requests, and the response echoes a third (`meta-llama-4-maverick-040225`). Pricing keys off the resolved name, so reconciling by requested id will not line up. +- **Embeddings** work on `/ai-gateway/mlflow/v1/embeddings` and report input only — no `completion_tokens` at all. diff --git a/docs/images/lago-logo-white.svg b/docs/images/lago-logo-white.svg new file mode 100644 index 0000000..f65faad --- /dev/null +++ b/docs/images/lago-logo-white.svg @@ -0,0 +1,19 @@ + + + + + + + + diff --git a/docs/images/lago-logo.svg b/docs/images/lago-logo.svg new file mode 100644 index 0000000..51a35a6 --- /dev/null +++ b/docs/images/lago-logo.svg @@ -0,0 +1,19 @@ + + + + + + + + diff --git a/docs/providers.md b/docs/providers.md new file mode 100644 index 0000000..984b4ab --- /dev/null +++ b/docs/providers.md @@ -0,0 +1,43 @@ +# Providers — coverage and token semantics + +The [README](../README.md) shows the minimal quickstart per provider. This page holds the full coverage matrix, the token fields each provider populates, and the per-provider quirks that affect billing. + +## Supported providers + +| Provider | Access | Status | +|---|---|---| +| AWS Bedrock | `Converse` (sync + stream) | ✓ | +| AWS Bedrock | `InvokeModel` (sync + stream), 7 model families | ✓ | +| Anthropic | native SDK (`messages.create` + `messages.stream`, sync + async) | ✓ | +| Mistral | native SDK (`chat.complete` + `chat.stream`) | ✓ | +| OpenAI | native SDK (`chat.completions.create` + `responses.create`, sync + async + stream) | ✓ | +| Google Gemini | native SDK (`google-genai`: `models.generate_content` + `generate_content_stream`, sync + async) | ✓ | + +## Per-provider notes + +**Anthropic** — works with `Anthropic` and `AsyncAnthropic`. Both `messages.create(..., stream=True)` and the `messages.stream(...)` context manager are instrumented — usage is captured from the final `message_delta` event in either case. + +**OpenAI** — works with `OpenAI` and `AsyncOpenAI`. Covers both **Chat Completions** (`client.chat.completions.create`) and the newer **Responses API** (`client.responses.create`), sync + streaming. For streaming, the wrapper auto-injects `stream_options={"include_usage": True}` so the final chunk carries usage data — without it OpenAI emits no usage on streamed responses. **Reasoning tokens** (`llm_reasoning_tokens`) populate automatically when you call an o-series model (`o4-mini`, `o1`, etc.) — OpenAI is the first provider to expose this metric separately. + +**Gemini** — wraps the modern `google-genai` SDK (`from google import genai`). Covers `client.models.generate_content` + `generate_content_stream`, sync + async (via `client.aio.models`). **Reasoning tokens** populate automatically on Gemini 2.5 — the model reasons internally by default and surfaces `thoughts_token_count` (see the note on reasoning semantics below). + +## Token dimensions captured + +`CanonicalUsage` carries 11 numeric fields. Which ones populate depends on the provider: + +| Field | Lago metric code | Bedrock | Anthropic | Mistral | OpenAI | Gemini | +|---|---|---|---|---|---|---| +| input | `llm_input_tokens` | ✓ | ✓ | ✓ | ✓ | ✓ | +| output | `llm_output_tokens` | ✓ | ✓ | ✓ | ✓ | ✓ | +| cache_read | `llm_cached_input_tokens` | ✓ (Anthropic) | ✓ | ✓ (when cache hits) | ✓ (auto-cache) | ✓ (CachedContent API) | +| cache_write | `llm_cache_creation_tokens` | ✓ (Anthropic) | ✓ | ✗ | ✗ | ✗ | +| cache_write_5m / 1h | `llm_cache_write_5m/1h_tokens` | ✓ (Anthropic InvokeModel) | ✓ | ✗ | ✗ | ✗ | +| reasoning | `llm_reasoning_tokens` | ✗ (folded into output) | ✗ (folded into output, even with extended thinking) | ✗ (folded into output) | **✓ (o-series, subset)** | **✓ (Gemini 2.5, additive)** | +| tool_calls | `llm_tool_calls` | ✓ | ✓ | ✓ | ✓ | ✓ | +| audio_input | `llm_audio_input_tokens` | ✗ | ✗ | ✗ | ✓ (GPT-4o-audio) | ✓ (multimodal AUDIO) | +| audio_output | `llm_audio_output_tokens` | ✗ | ✗ | ✗ | ✓ (GPT-4o-audio) | ✓ (multimodal AUDIO) | +| image_input | `llm_image_input_tokens` | ✗ | ✗ | ✗ | ✗ | ✓ (multimodal IMAGE) | + +**Reasoning:** OpenAI's `reasoning_tokens` is a *subset* of `output` (already counted in `completion_tokens`). Gemini's `thoughts_token_count` is *additive* to `output` (`candidates + thoughts = total billable output`). + +**Cache/audio/image on OpenAI and Gemini are subsets of `input`, not additive.** Both providers count cached/audio/image tokens *within* their input total, so summing `llm_input_tokens + llm_cached_input_tokens` (or `+ audio/image`) double-counts. Bill on `llm_input_tokens` alone; use the breakdown fields only for cost attribution (e.g. a discounted cache rate). diff --git a/docs/ramp-router.md b/docs/ramp-router.md new file mode 100644 index 0000000..04ff1e8 --- /dev/null +++ b/docs/ramp-router.md @@ -0,0 +1,58 @@ +# Ramp Router + +[Ramp Router](https://router.com) is an OpenAI-Responses-compatible gateway in front of OpenAI, Anthropic, Google Vertex, Fireworks and xAI. Point an OpenAI client at it and `wrap()` detects it from the `base_url` — no other code change: + +```python +from openai import OpenAI +from lago_agent_sdk import LagoSDK + +sdk = LagoSDK(api_key=os.environ["LAGO_API_KEY"]) +client = sdk.wrap( + OpenAI( + api_key=os.environ["RAMP_ROUTER_API_KEY"], + base_url="https://api.router.com/v1", + ), + subscription="sub_acme", +) + +# A model id from GET /v1/models — Router's ids are account-specific. +client.responses.create(model=os.environ["RAMP_ROUTER_MODEL"], input="Summarize this invoice.") +sdk.flush() +``` + +- **The model that answered is the one billed.** Router diverges from what you asked for in two ways: a `models` fallback list sends no `model` field at all, and Switchyard routing can serve a different model than the one requested. The SDK bills the model the response reports. +- **Router answers with a resolved vendor snapshot, so nothing needs stripping.** Ask for `openai:gpt-5.4-nano` and the response says `gpt-5.4-nano-2026-03-17`; that bare name is what bills, so a Router-served model rolls up in Lago against the same row a direct call to it reports. (A `provider:model[:tier]` candidate is still split if one ever reaches the adapter unresolved, but no captured response carries that shape.) +- **The served service tier is recorded** in `usage.extras["service_tier"]`, read from the response's own `service_tier` field — `flex`, `default`, and so on. Note `extras` is diagnostic: it is **not** sent to Lago, so the tier is visible to an `on_error`/debug hook but does not reach your events or split a charge. Pricing a tier differently needs the Lago plan to do it. +- **Streaming bills once**, from the terminal usage event. `models` fallback, buffered and streamed calls all work unmodified — note the typed Python client rejects the non-standard `models` kwarg, so a fallback list goes through `extra_body={"models": [...]}` (verified live; the served model bills either way). + +Attribution works the same way as anywhere else in this SDK — `subscription` at wrap time or `extra_lago={"subscription": ...}` per call. Separately, it is worth putting the same id in Router's own `metadata` field, which Router stores with its usage record and shows in the request detail: + +```python +client.responses.create( + model=os.environ["RAMP_ROUTER_MODEL"], + input="Summarize this invoice.", + # Router stores this with its usage record. The SDK does not send it for you. + metadata={"lago_subscription": "sub_acme"}, +) +``` + +That costs nothing today and is what a backfill would key off later. + +## Price mode falls back to token events + +In `pricing_mode="price"`, a Router call currently emits **token events rather than `llm_cost`**. That is deliberate, not a gap, and it is worth understanding before turning price mode on for Router traffic: + +- **A BYOK-served request costs $0 through Router.** Router's own words: "When a request is served with your provider key, your provider bills you directly and Ramp Router does not charge for that usage." Nothing in the response says which key served it, so pricing at list rate would bill a customer the full amount for usage Router never charged for. +- **A non-default service tier does not bill at the published base rate.** Router's catalog says outright that "service tiers, long contexts, caching, and other features may use different rates", and that a Fast tier's "pricing may differ from the base rates shown here". Billing flex at the standard rate over-bills. +- **The token overlap semantics are Router's own, and they are measured.** OpenAI counts cached tokens inside `input_tokens` and reasoning inside `output_tokens`; Anthropic counts both additively. Router normalizes the _numbers_ to OpenAI's convention, not just the schema — verified live on an Anthropic-served model, where a warm `cache_control` call reported the cached block inside an unchanged `input_tokens`. But that convention is Router's, not the served vendor's, which is exactly why the served vendor must not be stamped as the provider: it would de-overlap with the wrong rules whenever its native convention differs. + +So Router is treated as a provider of its own, matching no vendor in the price tables. Token mode — the default — is exact and unaffected: it emits the counts Router reported, the same per-field `llm_*` events a direct provider call produces. Price mode takes a clean miss and falls back to those same token events, with no error on your call path. Bill Router traffic in token mode and price it with a Lago plan. + +## Measured behaviours and limitations + +- **There is no backfill path**, because Router exposes no programmatic usage surface. Its only routes are `GET /v1/models`, `POST /v1/responses`, `POST /v1/messages` and `POST /v1/messages/count_tokens`; usage lives in the dashboard's Logs view. An "analytics API" is mentioned once in Router's limits table with no path, auth or record shape. Unlike the [Cloudflare connector](cloudflare.md), there is no Logs API loop to show you. +- **Nothing is skipped as a gateway cache hit**, because Router has no response cache: "Self-service Router response caching, which would reuse an entire previous response without calling a model provider, is a separate optimization and is not currently configurable." Provider _prompt_ caching does pass through, and those cache-read and cache-write tokens are billed like any other. +- **Do not point a wrapped `Anthropic` client at Router.** `POST /v1/messages` exists and routes to the same providers, and the call will _work_ — but the Anthropic wrapper has no `base_url` detection, so the SDK bills it as though it came from Anthropic directly. Events are stamped `provider=anthropic api=native`, priced at Anthropic's list rate even though Router may have served the call BYOK at $0, and de-overlapped with Anthropic's additive cache convention rather than Router's inside-`input` one. That last one double-counts every cached token if Router normalizes `/v1/messages` the way it demonstrably does `/v1/responses` (fixture `06b`: the cached block sits inside an unchanged `input_tokens`) — a warm 18,825-token call bills 37,650. Not yet measured on `/v1/messages`, which is the reason to stay off it. Use the Responses surface until the Anthropic wrapper learns the host. +- **A proxy in front of Router is not detected.** Detection matches the `api.router.com` host (and `*.router.com`). Reaching Router through your own hostname bills as plain OpenAI, with the wrong provider and an unparsed model id. +- **`api.router.com` sits behind bot management.** A rejected client can get an HTML challenge page rather than Router's documented JSON error envelope. The SDK degrades to zero usage rather than throwing, either way. +- **Failures never bill.** Every documented status — including 402 `insufficient_credits`, 429, and 502 `all_candidates_failed` — emits nothing, as does a response reporting zero usage. diff --git a/docs/snowflake.md b/docs/snowflake.md new file mode 100644 index 0000000..39fc268 --- /dev/null +++ b/docs/snowflake.md @@ -0,0 +1,95 @@ +# Snowflake Cortex + +Snowflake serves Cortex two ways, and they need two different halves of this SDK. That split is the thing to understand before anything else here. + +| Surface | How you call it | How Lago sees it | +|---|---|---| +| **Cortex REST** — `/api/v2/cortex/v1` | an OpenAI-compatible client you hand to `wrap()` | live, per call | +| **AI SQL functions** — `AI_COMPLETE`, `AI_EMBED`, … | SQL, inside the warehouse | **backfill only** — there is no client to wrap | + +**Everything on this path bills as token counts.** Snowflake meters Cortex in credits against a rate card that lives in no view, so there is no per-request dollar figure to pass through and no price mode here. `provider` is `"snowflake"`, which is listed in `TOKEN_BILLED_PROVIDERS`, so a customer running `pricing_mode="price"` globally still gets token events for Snowflake rows — with no price-miss error, because a structural absence of a rate card is not a lookup failure. + +## Live — the REST surface + +```python +import os +from openai import OpenAI + +client = sdk.wrap( + OpenAI( + base_url=f"https://{os.environ['SNOWFLAKE_ACCOUNT']}.snowflakecomputing.com/api/v2/cortex/v1", + api_key=os.environ["SNOWFLAKE_PAT"], + ) +) +``` + +The `base_url` is what identifies these calls as Snowflake rather than OpenAI — an OpenAI-shaped endpoint says nothing about whose tokens they are. Use `max_completion_tokens`; Cortex rejects `max_tokens` outright. + +**Cortex's `cached_tokens` is additive, the opposite of OpenAI's convention.** A cached call reports `prompt_tokens: 7`, `cached_tokens: 8745`, `completion_tokens: 6`, `total_tokens: 8758` — the cached block is *not* inside `prompt_tokens`. Caching also only happens when you send an explicit `cache_control: {"type": "ephemeral"}` content part; the same prompt twice without one reports zero cached both times. + +**The wire cannot tell a cache creation from a read; the view can.** A creation call reports the same `cached_tokens` with `cache_write_tokens: 0`, so the live `wrap()` path bills a creation as a cache read (`llm_cached_input_tokens`). The REST view records the same call as `cache_write_input`, so a backfilled row bills `llm_cache_creation_tokens` instead. Verified live on a matched pair (INT-230): identical wire usage, one `cache_write_input` row and three `cache_read_input` rows. If you price creation and read differently, know that live-path traffic reports everything at the read metric. It also bounds the REST-view dedup: a backfilled creation row emits its cached block under a *different* transaction id than the live path did (`_tok_cache_write` vs `_tok_cache_read`), so that one component bills on both metrics if you backfill a live-billed window — the call's input and output stay deduplicated. + +## Backfill — the SQL functions surface + +```python +from lago_agent_sdk.gateway.snowflake import SnowflakeSource + +source = SnowflakeSource.from_env() # SNOWFLAKE_ACCOUNT / _PAT, plus a warehouse +print(sdk.backfill_snowflake(source, "7 days", default_subscription="sub_default")) +sdk.flush() +# {'tokens': 47, 'skipped': 0} +``` + +Two counts, and there cannot be more: `tokens` is what got billed, `skipped` is what did not. Both causes of a skip are also reported through `on_error` with `where="backfill"`, so an automated caller notices a gap without inspecting the return value. + +**It reads the functions view only.** The REST view reports the calls `wrap()` already billed above. Both sides derive one idempotency key from the call's `REQUEST_ID` — the wrapper reads it off the `x-snowflake-request-id` response header — so Lago rejects a backfill's copies as duplicate `transaction_id`s instead of billing them twice. That protection holds only when the backfill uses the default `event_id_prefix` and resolves the same subscription the live path billed, and it does not cover a cache-creation call's cached block (see the cache note above) or calls billed without the header. So the rule stands: pass `views=("rest",)` only for REST traffic `wrap()` never saw: + +```python +sdk.backfill_snowflake( + source, + "7 days", + default_subscription="sub_default", + views=("rest",), # ONLY if no wrapped client is billing this traffic +) +``` + +The window reads **whole closed hours only** — floored at both ends, with the current hour excluded, because a bucket is not complete until its hour closes and billing it early burns that row's idempotency key so the correction is rejected as a duplicate. The newest hour therefore arrives on the next run: pass a window comfortably wider than your run interval, since this reader keeps no cursor. One more boundary Lago itself draws: events stamped before the subscription started are **accepted and silently never billed** — a window reaching back past the subscription's start reports its rows as billed while nothing lands in usage, so start backfills at the subscription's start date. + +**Attribution comes from `QUERY_TAG`**, the only customer-injectable key on either view, and the same `lago_subscription` key Cloudflare and Databricks read from their own metadata: + +```sql +ALTER SESSION SET QUERY_TAG = '{"lago_subscription": "sub_123"}'; +SELECT AI_COMPLETE('claude-sonnet-4-5', 'summarize this'); +``` + +By default the tag is the **only** attribution source: an untagged row falls to `default_subscription`, and to a skip (counted, reported) when there is none. `role_names` and `user_id` are opt-in — `subscription_order=("query_tag", "role_names")` — for accounts that really map one Snowflake role or user to one customer. They are not in the default because every live row carries both, so they would swallow untagged rows and bill them to a Snowflake identity instead of your default: that is a wrong subscription, and unlike a skip it is not recoverable. + +**Grouping matches the view.** Each event carries the grouping key of the surface it came from — `function_name` + `model_name` for functions rows, `inference_region` for REST — so a `GROUP BY` on the view and the same grouping in Lago line up. `dimensions={...}` adds your own; yours win on a collision. + +## A long-running query is deferred, not guessed at + +`IS_COMPLETED` means "did the query finish *in this aggregation window*", and these views are hour-bucketed — Snowflake documents a query running 5:30→8:30 writing **four rows, one per hour, all sharing one `QUERY_ID`**. Two things follow. The key `{prefix}_{kind}_{sub}_{QUERY_ID}` collides across those rows, and whether each row's `METRICS` is incremental or cumulative is unmeasured. On a 3-hour query using 3,800 input tokens, summing four incremental rows bills 3,800 and summing four cumulative ones bills 9,500; billing only the last row gives 3,800 if cumulative and 900 if incremental. + +So a `QUERY_ID` that yields more than one row in a window is **not billed**. It is counted in `skipped`, reported through `on_error`, listed on `source.deferred_rows`, and billable once the shape is settled. Guessing over-bills by 2.5× or under-bills by 76%, neither recoverable once invoiced. Every query ever observed on a real account finished inside one bucket, so this fires on a shape nobody has seen. + +To inspect a window before billing it, read the rows directly — `read_usage()` is a generator and each row is already shaped for `emit()`: + +```python +for row in source.read_usage("7 days"): + print(row.kind, row.usage.model, row.subscription, row.occurred_at) +``` + +## Setting up the account + +Reading the views needs a PAT plus a **running warehouse**; the live calls above need neither. Four things block a first-time setup and none of them says so clearly: + +- **Model access moved to RBAC.** `CORTEX_MODELS_ALLOWLIST` is deprecated and accepts only `'NONE'`; you need `GRANT APPLICATION ROLE SNOWFLAKE."CORTEX-MODEL-ROLE-ALL" TO ROLE …`, without which the role can call zero models. +- **A PAT's `ROLE_RESTRICTION` is a quoted string literal**, so it is case-sensitive — `'LAGO_CORTEX_ROLE'`, not the lowercase spelling that works everywhere else. +- **A warehouse with `AUTO_RESUME = FALSE`** fails every statement with "warehouse is suspended", which reads like a privilege error. +- **A PAT cannot authenticate without an active network policy.** Prefer reusing a broad one: recovering from an IP lockout needs Snowflake Support, with no self-service path back. + +Error code `003001` has four distinct causes — account entitlement, unknown model, model not granted to the role, and a bare fine-tuned model name — so it is not diagnostic on its own. + +**One cost note:** a SQL warehouse is a real cost centre. Measured on the equivalent Databricks setup, warehouse queries cost roughly 1,500× the model-serving usage they reported on. Run the backfill as one query over a wide window, never as a tight polling loop. + +The pure `extract_snowflake_functions_log(row)` / `extract_snowflake_rest_log(row)` / `resolve_snowflake_subscription(row)` functions stay available from `lago_agent_sdk.gateway.adapters` if you already have rows from your own warehouse job. diff --git a/src/lago_agent_sdk/adapters/openai_native.py b/src/lago_agent_sdk/adapters/openai_native.py index fe255dc..b03d20b 100644 --- a/src/lago_agent_sdk/adapters/openai_native.py +++ b/src/lago_agent_sdk/adapters/openai_native.py @@ -37,6 +37,7 @@ from __future__ import annotations +import re from typing import Any, cast from ..canonical import WORKERS_AI_COMPAT_PREFIX, CanonicalUsage @@ -52,6 +53,104 @@ _WORKERS_AI_MODEL_PREFIX = "@cf/" # Top-level usage fields we recognize across BOTH chat completions and responses APIs. +#: Provider stamped on any call that reached a model through Ramp Router. +#: +#: Router is an OpenAI-Responses-compatible gateway in front of OpenAI, Anthropic, Google +#: Vertex, Fireworks and xAI. It is treated as a provider in its own right here, rather +#: than resolved to the vendor that actually served the call, because Router's model ids +#: are ACCOUNT-SPECIFIC and opaque — its docs are explicit: "Valid model IDs are +#: account-specific. They come from `GET /v1/models`. Never invent one or reuse a +#: provider's public model name." So unless the caller named an explicit +#: `provider:provider-model` candidate, nothing in the response says who served it, and +#: the model-string rule `_infer_provider` uses cannot see it. +#: +#: "ramp_router" is in `TOKEN_BILLED_PROVIDERS` and deliberately absent from +#: `_VENDOR_MAP`. Two distinct things would otherwise go wrong at once: +#: +#: * A price lookup under a guessed vendor can be flatly wrong. Router bills at list +#: price on its shared key but $0 for a BYOK-served request, and a non-default +#: service tier bills at a rate its own catalog says "may differ" from the base one. +#: * The overlap semantics belong to ROUTER, not to the served vendor. Measured +#: 2026-08-28 on an Anthropic-served model — the case that would diverge if anything +#: did: Router normalizes the NUMBERS to OpenAI's convention, not just the schema +#: (cached block INSIDE input, reasoning inside output; fixtures +#: 06b_real_cache_control_warm.json / 07_real_reasoning.json). So stamping the served +#: vendor would de-overlap with the WRONG convention whenever that vendor's native +#: one differs — "ramp_router" carries its own OPENAI_SHAPED_APIS entry instead. +#: +#: Token mode is unaffected and exact either way: it emits the counts Router reported. +#: Price mode routes to those same token events via `TOKEN_BILLED_PROVIDERS`, with no +#: per-call price-miss report — a structural, permanent miss must not cry wolf on the +#: error hook (the same decision Databricks and Snowflake got). The catalog DOES publish +#: per-model rates (`router.pricing`, 01_real_models_catalog.json), so a Router price +#: mode is buildable — but the response still cannot say whether a BYOK key served the +#: call ($0) or which tier rate applied, and every observed catalog entry carries an +#: EMPTY input rate, so token counts stay the honest default. +RAMP_ROUTER_PROVIDER = "ramp_router" + +# Router's documented service tiers, appearing as the third segment of a REQUESTED +# candidate id (`openai:gpt-5.4-mini:flex`). OpenAI sells `auto`/`default`/`flex`/ +# `priority`, Fireworks `default`/`priority`. +# +# Only used to disambiguate that candidate suffix. The tier a response actually reports +# arrives in its own top-level `service_tier` field and is recorded verbatim. +# +# Matched against this set rather than read as "whatever follows the last colon": a model +# segment may contain a colon of its own, and mistaking one for a tier would silently +# rename the model and split it into a second row in Lago. An unrecognized trailing +# segment therefore stays part of the model, which is recoverable; a renamed model is +# not. +_ROUTER_SERVICE_TIERS = frozenset({"auto", "default", "flex", "priority"}) + +# A provider segment is a short lowercase token. Anything else is part of the model. +_ROUTER_PROVIDER_SEGMENT = re.compile(r"^[a-z0-9][a-z0-9_-]{1,31}$") + + +def _parse_router_model(model_id: str) -> tuple[str, str, str]: + """Split a Router model id into its (provider, model, service-tier) parts. + + Router names a model two ways, and only one of them is parseable. A plain `model` is + an account-specific id that reveals nothing; a `models` candidate is + `provider:provider-model[:service-tier]`. + + This is a FALLBACK, not the live path. Router resolves whatever was requested to a + bare vendor snapshot before answering — every captured response reports + `gpt-5.4-nano-2026-03-17` or `claude-haiku-4-5-20251001`, never a compound candidate + — and `resolve_model` prefers the response's model over the requested one. So this + fires only when the response carries no model at all and the caller's requested id + falls through. It is kept because that fallthrough would otherwise publish + `openai:gpt-5.4-mini:flex` as a Lago model name; it is not where the served tier + comes from. See the `service_tier` read in `extract_openai_native`. + + Split on the FIRST colon, never on all of them: Fireworks candidates carry a path as + their model segment (`fireworks:accounts/fireworks/models/kimi-k2p7-code`), so a + naive split loses everything after the second separator. + + The model comes back BARE, provider prefix and tier stripped, so a model served + through Router rolls up in Lago against the same name a direct call to it reports. + Leaving the prefix on splits one model across two rows for no billing benefit. + """ + first_colon = model_id.find(":") + if first_colon <= 0: + return "", model_id, "" + + head = model_id[:first_colon].lower() + rest = model_id[first_colon + 1 :] + # A head that is not a plausible provider token — a path, or something long — means + # this is an opaque id that merely happens to contain a colon, not a candidate. + if not rest or not _ROUTER_PROVIDER_SEGMENT.match(head): + return "", model_id, "" + + tier = "" + last_colon = rest.rfind(":") + if last_colon > 0: + trailing = rest[last_colon + 1 :].lower() + if trailing in _ROUTER_SERVICE_TIERS: + tier = trailing + rest = rest[:last_colon] + return head, rest, tier + + _KNOWN_USAGE_FIELDS = { # chat completions "prompt_tokens", @@ -237,6 +336,57 @@ def extract_openai_native(response: Any, model_id: str = "", provider_hint: str resolved_model = resolve_model(resp.get("model"), model_id) provider = provider_hint or _infer_provider(resolved_model) + # MUST stay ABOVE the total_tokens guard below. The guard asks + # `token_semantics(provider, api)` the same convention question `compute_cost` and + # `deoverlapped_token_total` ask, and Router is the one surface here that REASSIGNS + # `api` mid-function. Read before the reassignment, the guard sees + # ("ramp_router", "responses") — all-additive — while the money paths see the + # stamped api="ramp_router" and de-overlap as subset. That divergence is exactly + # what token_semantics.py exists to make impossible, and it under-folds a genuine + # remainder by cache_read + reasoning, or suppresses the fold entirely when the + # over-count exceeds the declared total — silently, with no on_error. + # + # `resolve_model` prefers the response's own model over the requested one, which is + # what makes a Router fallback bill correctly with no extra work: a `models` request + # sends no `model` at all, and Switchyard routing can serve a different model than + # the one asked for, so the response is the only place the SERVED model appears. + model = resolved_model + if provider_hint == RAMP_ROUTER_PROVIDER: + router_provider, parsed_model, tier = _parse_router_model(resolved_model) + if router_provider: + model = parsed_model + # Recorded, not promoted to `provider` — see RAMP_ROUTER_PROVIDER for why + # the served vendor cannot drive the de-overlap convention. + extras["router_provider"] = router_provider + # The tier is billing-relevant on its own: Router's catalog says tiers "may use + # different rates" than the base ones it publishes, so a pinned non-default tier + # is the difference between a correct price and an over-bill at the standard + # rate. + # + # It is read from the response's OWN top-level `service_tier`, which is where + # every captured Router response actually reports it (02/04 `flex`, 03/05b/06b/07 + # `default`). The candidate suffix above is only a fallback: Router resolves + # `model` to a bare vendor snapshot on the way back — `openai:gpt-5.4-nano` in, + # `gpt-5.4-nano-2026-03-17` out — so on live traffic the suffix parse never fires + # and reading only it dropped the tier on every real call. Sourcing it from the + # candidate alone also answers the wrong question: the candidate says what was + # ASKED for, `service_tier` says what SERVED, and a `models` fallback list can + # make those differ. + # + # NOT filtered through _ROUTER_SERVICE_TIERS. That set disambiguates a colon + # segment that might instead be part of a model name; a dedicated field has no + # such ambiguity, so a tier Router adds later is recorded rather than dropped. + served_tier = resp.get("service_tier") + if isinstance(served_tier, str) and served_tier: + extras["service_tier"] = served_tier + elif tier: + extras["service_tier"] = tier + # Which of Router's two OpenAI-shaped surfaces answered. Router documents only + # `/v1/responses` (`/v1/chat/completions` 404s), so a `chat_completions` value + # here is drift worth seeing rather than a case to handle. + extras["router_surface"] = api + api = RAMP_ROUTER_PROVIDER + # Consistency guard: for genuine OpenAI, total_tokens always equals # prompt + completion (reasoning is a SUBSET of completion, never additive). # Verified across every fixture under openai_native/ — zero deltas. So a @@ -320,7 +470,7 @@ def extract_openai_native(response: Any, model_id: str = "", provider_hint: str audio_input=audio_input, audio_output=audio_output, tool_calls=tool_calls, - model=resolved_model, + model=model, provider=provider, api=api, extras=extras, diff --git a/src/lago_agent_sdk/gateway/__init__.py b/src/lago_agent_sdk/gateway/__init__.py index 3cca14d..f4aa055 100644 --- a/src/lago_agent_sdk/gateway/__init__.py +++ b/src/lago_agent_sdk/gateway/__init__.py @@ -11,3 +11,26 @@ """ from __future__ import annotations + +# NO RAMP ROUTER ADAPTER, deliberately, and this is the note saying so rather than an +# omission to rediscover later. +# +# Router is the first gateway this SDK supports that exposes no programmatic usage +# surface at all. Checked against every page of its documentation: the only routes are +# `GET /v1/models`, `POST /v1/responses`, `POST /v1/messages` and +# `POST /v1/messages/count_tokens`. Usage lives in the dashboard's Logs view, and +# `guides/monitor` describes what that view DISPLAYS — model, provider, status, tokens, +# cost, latency, ids, API key, token breakdown, metadata, service tier, fallback +# candidates — without offering any way to fetch it. +# +# An "analytics API" is referenced exactly once in the whole corpus, in the limits table +# on `api/errors-and-limits` ("the analytics API accepts at most 93 days"), with no path, +# no auth and no record shape. That is not enough to build against: an adapter written +# over guessed field names would have tests proving only that it matches the guess, and +# the fixtures behind it would not be captures of anything. +# +# So Router's live `wrap()` path is the whole integration for now. When the analytics +# API is published, the adapter goes here as `extract_ramp_router_log` / +# `resolve_ramp_router_subscription` alongside the Cloudflare pair, keyed for idempotent +# replay off whatever per-record id it exposes. Tracked as LAGO-1853, which also carries +# the questions to ask Router. diff --git a/src/lago_agent_sdk/pricing.py b/src/lago_agent_sdk/pricing.py index e5f5284..c4bc022 100644 --- a/src/lago_agent_sdk/pricing.py +++ b/src/lago_agent_sdk/pricing.py @@ -107,7 +107,7 @@ # a real vendor prefix would let a near-miss model string match Anthropic's or # OpenAI's own OpenRouter rate — a silent mispricing of a call Snowflake charged in # credits. The absence is the guard; do not "fix" it. -TOKEN_BILLED_PROVIDERS = frozenset({"databricks", "snowflake"}) +TOKEN_BILLED_PROVIDERS = frozenset({"databricks", "snowflake", "ramp_router"}) # Canonical field -> OpenRouter pricing key. _OPENROUTER_FIELD_MAP = { diff --git a/src/lago_agent_sdk/token_semantics.py b/src/lago_agent_sdk/token_semantics.py index 3dbd2c6..c425e1e 100644 --- a/src/lago_agent_sdk/token_semantics.py +++ b/src/lago_agent_sdk/token_semantics.py @@ -144,7 +144,22 @@ # at all. Adding either would drop a cached row from 4,698 tokens to 14; INT-221's # reconciliation test asserts the sum against Snowflake's own TOKENS column and fails # if someone does. -OPENAI_SHAPED_APIS = frozenset({"databricks_gateway"}) +OPENAI_SHAPED_APIS = frozenset( + { + "databricks_gateway", + # Ramp Router normalizes the NUMBERS to OpenAI's convention, not just the schema — + # measured 2026-08-28 against a live account, on an ANTHROPIC-served model, which + # is the case that would diverge if anything did: a warm cache_control call + # reported input_tokens unchanged (18825) with cached_tokens 18810 INSIDE it, + # total = input + output exactly (06b_real_cache_control_warm.json); reasoning + # came back inside output (128 of 169, total = input + output, + # 07_real_reasoning.json). cache_write is inferred from the same normalization + # rather than measured — every observed write reported cache_write_tokens: 0 + # while the warm read proved the cache existed, and the arithmetic keeps the + # write inside input — re-verify the day a nonzero write appears in a capture. + "ramp_router", + } +) # Every provider name the SDK's own code can stamp on a CanonicalUsage, so that # absence from the sets above is always a recorded DECISION and never a default @@ -175,6 +190,11 @@ "mistral", "databricks", "snowflake", + # Ramp Router, from the wrapper's host hint. Its convention lives in + # OPENAI_SHAPED_APIS (the adapter stamps api="ramp_router", and the surface + # wins): measured OpenAI-normalized on cache_read and reasoning, see the entry + # there. + "ramp_router", # adapters/bedrock_*, from _provider_from_model "amazon", "meta", diff --git a/src/lago_agent_sdk/wrappers/openai.py b/src/lago_agent_sdk/wrappers/openai.py index cdef8dc..50cec40 100644 --- a/src/lago_agent_sdk/wrappers/openai.py +++ b/src/lago_agent_sdk/wrappers/openai.py @@ -33,10 +33,12 @@ from __future__ import annotations import logging +import urllib.parse from collections.abc import AsyncIterator, Iterator from typing import Any from ..adapters import extract_openai_native +from ..adapters.openai_native import RAMP_ROUTER_PROVIDER # The one import a wrapper takes from gateway code, and it is load-bearing: the REST-view # dedup only works if this wrapper and `gateway/snowflake.py` compute the IDENTICAL @@ -166,6 +168,18 @@ def _snowflake_request_id(header_owner: Any) -> str: ) +# Ramp Router cannot be a row in the path table above: it serves every provider it +# fronts through one dedicated host with no distinguishing path, so the HOST is the +# signal — and it must be the PARSED host, never a substring test. A substring row +# ("api.router.com") also matches `https://evil.example.com/api.router.com/v1`, which +# would stamp an unrelated endpoint's traffic as Router-served. The `.router.com` +# suffix arm covers a regional or staging host without widening to arbitrary domains — +# `evilrouter.com` does not end in `.router.com`. The path table keeps first say: its +# rows are more specific, and no Snowflake or Databricks URL lives under router.com. +_RAMP_ROUTER_HOST = "api.router.com" +_RAMP_ROUTER_DOMAIN = ".router.com" + + def _provider_hint_for(client: Any) -> str: """Return a provider override implied by the client's base_url, or "". @@ -191,6 +205,13 @@ def _provider_hint_for(client: Any) -> str: for path, provider in _PROVIDER_BY_BASE_URL_PATH: if path in base_url: return provider + try: + host = urllib.parse.urlsplit(base_url).hostname or "" + except ValueError: + # A relative or malformed base_url is not a gateway. Never throw out of wrap(). + return "" + if host == _RAMP_ROUTER_HOST or host.endswith(_RAMP_ROUTER_DOMAIN): + return RAMP_ROUTER_PROVIDER return "" diff --git a/tests/unit/adapters/fixtures/capture_ramp_router.py b/tests/unit/adapters/fixtures/capture_ramp_router.py new file mode 100644 index 0000000..0e53053 --- /dev/null +++ b/tests/unit/adapters/fixtures/capture_ramp_router.py @@ -0,0 +1,461 @@ +"""Capture real Ramp Router responses, scrubbing every capture as it is written. + +Router is an OpenAI-Responses-compatible gateway in front of OpenAI, Anthropic, Google +Vertex, Fireworks and xAI. Its docs describe the request surface but not the +billing-relevant response behaviour, so this script exists to MEASURE the seven +questions the adapter's mapping depends on rather than reason about them: + + P1 GET /v1/models — does the catalog publish per-model PRICING? (Answered 2026-08-28: + yes, `router.pricing` — but every observed input rate is empty.) + P2 buffered call — does the Response's `model` report the requested alias or the + served `provider:provider-model[:service-tier]`? + P3 models fallback — under a candidate list, does `model` name the SERVED candidate? + Candidates MUST be the catalog's `provider:model` ids — bare + display names 400 (measured; the error fixture shows it). + P4 stream: true — do the SSE events carry usage and the resolved model where + `_extract_stream_usage` already looks (`.response.usage`)? + P5 prompt cache — the money question, in two probes: `prompt_cache_key` alone + does NOT warm a cache (05/06, cached_tokens 0 both calls); + an explicit `cache_control` content part DOES, and the warm + call answered it (05b/06b): cached INSIDE input — Router + normalizes the NUMBERS to OpenAI semantics. + P6 reasoning model — is `reasoning_tokens` inside `output_tokens`? (Yes — measured + with o4-mini at effort medium; a too-small call reports 0.) + P7 error families — the envelope shape, and that a failure carries no usage. + +Raw `urllib`, not the OpenAI SDK, deliberately: this captures the wire JSON AND the +response headers, and no doc says whether Router signals service tier, cache state or +BYOK in a header. If it does, that is a billing signal we would otherwise never see. + +Reads RAMP_ROUTER_API_KEY from the environment and nothing else. Model ids are +account-specific ("Never invent one or reuse a provider's public model name"), so they +are DISCOVERED from P1 rather than hardcoded; override with RAMP_ROUTER_MODEL, +RAMP_ROUTER_ANTHROPIC_MODEL, RAMP_ROUTER_REASONING_MODEL if the heuristics pick badly. + +Run with: RAMP_ROUTER_API_KEY="..." python tests/unit/adapters/fixtures/capture_ramp_router.py +""" + +from __future__ import annotations + +import json +import os +import re +import sys +import time +import urllib.error +import urllib.parse +import urllib.request +from pathlib import Path +from typing import Any + +OUT = Path(__file__).parent / "ramp_router" +OUT.mkdir(exist_ok=True) + +API_KEY = os.environ.get("RAMP_ROUTER_API_KEY") +if not API_KEY: + print("RAMP_ROUTER_API_KEY is not set. Put it in a gitignored .env and export it.") + sys.exit(1) +BASE_URL = (os.environ.get("RAMP_ROUTER_BASE_URL") or "https://api.router.com/v1").rstrip("/") + +# Keep every probe to a few tokens. The whole run should cost cents, and the fixtures are +# read for their `usage` object, never for their prose. +MAX_OUTPUT_TOKENS = 16 +PROMPT = "Reply with exactly one word: pong." + +# --------------------------------------------------------------------------- +# Scrubbing. Runs on EVERY value before it reaches the tree, in the same step +# that writes it — there is no unscrubbed intermediate file to forget about. +# +# The bar is the one the Cloudflare fixtures set: remove credentials and anything +# account-identifying, keep opaque platform ids, timings, costs and token counts, +# because those are what the tests assert on. +# --------------------------------------------------------------------------- + +#: Header names never written to a fixture, whatever their value. +_DROP_HEADERS = frozenset({"authorization", "x-api-key", "set-cookie", "cookie", "proxy-authorization"}) + +_REDACTIONS: list[tuple[re.Pattern[str], str]] = [ + # Router's own key shape, plus the generic provider-key shapes, in case a key is + # ever echoed back inside an error message or a request-context field. + (re.compile(r"\bsk-[A-Za-z0-9_-]{10,}"), "rr-test-key-REDACTED"), + (re.compile(r"\bBearer\s+[A-Za-z0-9._~+/-]{10,}=*", re.I), "Bearer rr-test-key-REDACTED"), + # Emails, except ones already in a reserved documentation domain (RFC 2606). + ( + re.compile(r"\b[A-Za-z0-9._%+-]+@(?!example\.(?:com|org|net)\b)[A-Za-z0-9.-]+\.[A-Za-z]{2,}\b"), + "user@example.com", + ), + # Dotted quads. RFC 5737 documentation address. + (re.compile(r"\b(?:\d{1,3}\.){3}\d{1,3}\b"), "203.0.113.10"), + # Dashboard/console URLs carry account and key ids in the path. + ( + re.compile(r"https://(?:app|dashboard|router)\.(?:router|ramp)\.com/[^\s\"']*"), + "https://app.router.com/REDACTED", + ), +] + + +def _scrub_string(s: str) -> str: + """Redact one string. + + The live key is substituted by VALUE first, not only by pattern: a key whose shape + the patterns above do not anticipate would otherwise sail through. Pattern matching + is the backstop, not the primary defence. + """ + out = s.replace(API_KEY, "rr-test-key-REDACTED") + for pattern, replacement in _REDACTIONS: + out = pattern.sub(replacement, out) + return out + + +#: Keys whose value is model-generated or caller-supplied CONTENT, blanked, not shipped. +_CONTENT_KEYS = frozenset( + {"text", "input", "instructions", "output_text", "content", "refusal", "summary_text"} +) + + +def _scrub(value: Any, key: str = "") -> Any: + """Deep-scrub a captured payload. + + Content keys are blanked to "" rather than deleted, so the shape a test reads is the + shape Router really sent — deleting entries would change what the fixture proves. + """ + if isinstance(value, str): + if key in _CONTENT_KEYS: + return "" + return _scrub_string(value) + if isinstance(value, list): + return [_scrub(v, key) for v in value] + if isinstance(value, dict): + return {k: _scrub(v, k) for k, v in value.items() if k.lower() not in _DROP_HEADERS} + return value + + +# Provenance is derived, never asserted. A fixture captured against a mock or a staging +# host must not claim to be a real one — the whole point of the `NN_real_*` naming +# convention in this repo is that it means something. +try: + _HOST: str | None = urllib.parse.urlsplit(BASE_URL).hostname +except ValueError: + _HOST = None +_IS_PRODUCTION = _HOST == "api.router.com" +_PROVENANCE = ( + "real capture against a live Ramp Router account" + if _IS_PRODUCTION + else f"NOT a production capture — taken against {_HOST}. Do not commit as NN_real_*." +) + + +def save(name: str, probe: str, question: str, payload: dict[str, Any]) -> None: + """Write one fixture. The only path that touches the tree, so the only scrub site.""" + body = { + "_probe": probe, + "_question": question, + "_captured": _PROVENANCE, + "_scrubbed": 'credentials, emails, IPs, dashboard URLs; prompt and completion text blanked to ""', + **_scrub(payload), + } + (OUT / name).write_text(json.dumps(body, indent=2) + "\n") + print(f" wrote {name}") + + +# --------------------------------------------------------------------------- +# Transport +# --------------------------------------------------------------------------- + + +def call(method: str, path: str, body: Any | None = None) -> dict[str, Any]: + """One request, captured whole. + + The body is parsed as JSON when it is JSON and kept as text when it is not: + api.router.com sits behind Cloudflare bot management, and an unrecognized client + gets an HTML challenge page instead of Router's documented error envelope. A capture + script that assumed JSON would crash on exactly the case worth recording. + """ + req = urllib.request.Request( + f"{BASE_URL}{path}", + method=method, + headers={ + "Authorization": f"Bearer {API_KEY}", + "Content-Type": "application/json", + "User-Agent": "lago-agent-sdk-capture/0.2.0", + }, + data=None if body is None else json.dumps(body).encode(), + ) + try: + with urllib.request.urlopen(req, timeout=120) as resp: + status, headers, text = resp.status, dict(resp.headers), resp.read().decode() + except urllib.error.HTTPError as err: + status, headers, text = err.code, dict(err.headers), err.read().decode() + header_map = {k.lower(): v for k, v in headers.items() if k.lower() not in _DROP_HEADERS} + try: + parsed: Any = json.loads(text) + was_json = True + except ValueError: + parsed, was_json = text, False + return {"_status": status, "_headers": header_map, "_body": parsed, "_body_was_json": was_json} + + +def call_stream(body: Any) -> dict[str, Any]: + """A streamed request, captured as the ordered list of SSE events.""" + captured = call("POST", "/responses", body) + raw = captured["_body"] if isinstance(captured["_body"], str) else "" + events: list[Any] = [] + for line in raw.split("\n"): + if not line.startswith("data:"): + continue + payload = line[5:].strip() + if not payload or payload == "[DONE]": + continue + try: + events.append(json.loads(payload)) + except ValueError: + events.append({"_unparseable": payload}) + return {**captured, "_body": None, "_body_was_json": False, "_events": events} + + +# --------------------------------------------------------------------------- +# Model discovery. Ids are account-specific, so nothing here is hardcoded. +# --------------------------------------------------------------------------- + + +def pick_model(models: list[dict[str, Any]], vendor_hints: list[str], name_hints: list[str]) -> str | None: + by_vendor = [m for m in models if any(h in str(m.get("owned_by", "")).lower() for h in vendor_hints)] + pool = by_vendor or models + for hint in name_hints: + hit = next((m for m in pool if hint in str(m.get("id", "")).lower()), None) + if hit: + return str(hit["id"]) + return str(pool[0]["id"]) if pool else None + + +def candidate_id(models: list[dict[str, Any]], model_id: str | None) -> str | None: + """The catalog's `provider:model` candidate form for a model id. + + `models` entries MUST be this form — a bare display name 400s with + "`models` entry 0 must be a provider:model string" (measured).""" + for m in models: + if m.get("id") == model_id: + router = m.get("router") + if isinstance(router, dict) and router.get("catalog_id"): + return str(router["catalog_id"]) + return None + + +def main() -> None: + # ---- P1: the catalog. Does it publish prices? ---------------------------------- + print("[P1] GET /v1/models") + p1 = call("GET", "/models") + save("01_real_models_catalog.json", "P1", "does GET /v1/models publish per-model pricing?", p1) + + body = p1["_body"] if isinstance(p1["_body"], dict) else {} + catalog: list[dict[str, Any]] = body.get("data") if isinstance(body.get("data"), list) else [] + print(f" {len(catalog)} models visible to this key") + if not catalog: + print(" no models — cannot run P2-P7. Check the key's catalog in the dashboard.") + return + + # Report, loudly, whether P1 answered the pricing question. + price_keys: set[str] = set() + price_re = re.compile(r"(pric|cost|rate|per_m|per_million)", re.I) + for m in catalog: + for k, v in m.items(): + if price_re.search(k): + price_keys.add(k) + if isinstance(v, dict): + price_keys.update(f"{k}.{nk}" for nk in v if price_re.search(nk)) + print( + f" P1 ANSWER: catalog carries price-shaped fields: {', '.join(sorted(price_keys))}" + if price_keys + else " P1 ANSWER: no price-shaped field in the catalog — price mode must go through OpenRouter" + ) + + cheap = os.environ.get("RAMP_ROUTER_MODEL") or pick_model( + catalog, ["openai"], ["nano", "mini", "4o-mini"] + ) + anthropic = os.environ.get("RAMP_ROUTER_ANTHROPIC_MODEL") or pick_model( + catalog, ["anthropic"], ["haiku", "sonnet"] + ) + reasoning = os.environ.get("RAMP_ROUTER_REASONING_MODEL") or pick_model( + catalog, ["openai"], ["o4-mini", "o3-mini", "o3", "gpt-5"] + ) + print(f" using: cheap={cheap} anthropic={anthropic} reasoning={reasoning}") + + # ---- P2: requested alias, or served candidate? ---------------------------------- + if cheap: + print("[P2] buffered call, plain `model`") + p2 = call( + "POST", + "/responses", + { + "model": cheap, + "input": PROMPT, + "max_output_tokens": MAX_OUTPUT_TOKENS, + "metadata": {"lago_subscription": "rr_gateway_test_sub"}, + }, + ) + save( + "02_real_buffered_plain_model.json", + "P2", + "does response.model echo the requested id or the served candidate?", + {"_requested_model": cheap, **p2}, + ) + served = p2["_body"].get("model") if isinstance(p2["_body"], dict) else None + print(f' P2 ANSWER: requested "{cheap}" -> response.model "{served}"') + + # ---- P3: which candidate answered? ----------------------------------------------- + cheap_cand, anthropic_cand = candidate_id(catalog, cheap), candidate_id(catalog, anthropic) + if cheap_cand and anthropic_cand and cheap_cand != anthropic_cand: + print("[P3] models fallback list") + # Both candidates are real (an unroutable one fails the whole request rather + # than falling back), so the question is only which one Router names. + p3 = call( + "POST", + "/responses", + {"models": [cheap_cand, anthropic_cand], "input": PROMPT, "max_output_tokens": MAX_OUTPUT_TOKENS}, + ) + save( + "03_real_models_fallback.json", + "P3", + "under a candidate list, does response.model name the SERVED candidate?", + {"_requested_models": [cheap_cand, anthropic_cand], **p3}, + ) + served = p3["_body"].get("model") if isinstance(p3["_body"], dict) else None + print(f' P3 ANSWER: candidates [{cheap_cand}, {anthropic_cand}] -> response.model "{served}"') + + # ---- P4: streaming --------------------------------------------------------------- + if cheap: + print("[P4] stream: true") + p4 = call_stream( + {"model": cheap, "input": PROMPT, "max_output_tokens": MAX_OUTPUT_TOKENS, "stream": True} + ) + save( + "04_real_streamed.json", + "P4", + "do SSE events carry usage and the resolved model under .response?", + p4, + ) + with_usage = [ + e + for e in p4["_events"] + if ( + isinstance(e, dict) + and (e.get("usage") or (isinstance(e.get("response"), dict) and e["response"].get("usage"))) + ) + ] + print(f" P4 ANSWER: {len(p4['_events'])} events, {len(with_usage)} carrying usage") + + # ---- P5: the money question ------------------------------------------------------- + if anthropic: + print("[P5] prompt cache, two probes") + # Probe A: `prompt_cache_key` alone. Measured NOT to warm anything — kept + # because a negative that cost a capture is worth not re-buying. + filler = "You are a careful billing assistant. Answer in one word. " * 120 + cache_body: dict[str, Any] = { + "model": anthropic, + "instructions": filler, + "input": PROMPT, + "max_output_tokens": MAX_OUTPUT_TOKENS, + "prompt_cache_key": "lago-capture-p5", + } + write = call("POST", "/responses", cache_body) + save( + "05_real_cache_write.json", + "P5", + "first call — does it report cache_write/creation tokens?", + write, + ) + time.sleep(3) + read = call("POST", "/responses", cache_body) + save( + "06_real_cache_read.json", + "P5", + "second call — is cache_read INSIDE input_tokens or additive?", + read, + ) + # Probe B: an explicit cache_control content part, which Router forwards and the + # provider honours. THIS pair is the one that answered the question. + big = "You are a precise assistant. " + " ".join( + f"Fact {i}: item {i} maps to {(i * 7) % 1000}." for i in range(1300) + ) + cc_body = { + "model": anthropic, + "max_output_tokens": MAX_OUTPUT_TOKENS, + "input": [ + { + "role": "user", + "content": [ + {"type": "input_text", "text": big, "cache_control": {"type": "ephemeral"}}, + {"type": "input_text", "text": "What does item 3 map to? Number only."}, + ], + } + ], + } + cold = call("POST", "/responses", cc_body) + save( + "05b_real_cache_control_cold.json", + "P5b", + "does Router forward an explicit cache_control part?", + cold, + ) + time.sleep(2) + warm = call("POST", "/responses", cc_body) + save( + "06b_real_cache_control_warm.json", "P5b", "warm repeat of 05b — additive or inside-input?", warm + ) + u = warm["_body"].get("usage") if isinstance(warm["_body"], dict) else None + print(f" P5 ANSWER: usage on the warm call = {json.dumps(u)}") + print(" input unchanged with cached_tokens > 0 => cache_read is INSIDE input (OpenAI semantics).") + + # ---- P6: reasoning --------------------------------------------------------------- + if reasoning: + print("[P6] reasoning model") + p6 = call( + "POST", + "/responses", + { + "model": reasoning, + "input": "How many primes are there between 10 and 30? Answer with the count only.", + "max_output_tokens": 400, + "reasoning": {"effort": "medium"}, + }, + ) + save("07_real_reasoning.json", "P6", "is reasoning_tokens inside output_tokens?", p6) + u = p6["_body"].get("usage") if isinstance(p6["_body"], dict) else None + print(f" P6 ANSWER: usage = {json.dumps(u)}") + + # ---- P7: error families ----------------------------------------------------------- + print("[P7] error families") + errors: list[tuple[str, str, dict[str, Any]]] = [ + ( + "08_real_error_404_model_not_found.json", + "an id not in this key's catalog", + {"model": "definitely-not-a-model-id", "input": PROMPT, "max_output_tokens": MAX_OUTPUT_TOKENS}, + ), + ( + "09_real_error_400_both_selectors.json", + "both route selectors at once", + {"model": cheap, "models": [cheap_cand], "input": PROMPT, "max_output_tokens": MAX_OUTPUT_TOKENS}, + ), + ( + "10_real_error_400_no_selector.json", + "neither route selector", + {"input": PROMPT, "max_output_tokens": MAX_OUTPUT_TOKENS}, + ), + ] + for name, what, err_body in errors: + captured = call("POST", "/responses", err_body) + save(name, "P7", f"error envelope for: {what}", captured) + print(f" {name}: HTTP {captured['_status']} json={captured['_body_was_json']}") + + print("\nDone. Inspect tests/unit/adapters/fixtures/ramp_router/*.json") + print("Then grep the directory for the key value, Bearer, non-example emails and routable IPs.") + + +if __name__ == "__main__": + try: + main() + except Exception as err: # noqa: BLE001 + # Deliberately not dumping the error object whole: a failure can carry the + # request — headers included — and this script holds a live credential. + print(f"capture failed: {err.__class__.__name__}: {err}") + sys.exit(1) diff --git a/tests/unit/adapters/fixtures/ramp_router/01_real_models_catalog.json b/tests/unit/adapters/fixtures/ramp_router/01_real_models_catalog.json new file mode 100644 index 0000000..d5e3e1c --- /dev/null +++ b/tests/unit/adapters/fixtures/ramp_router/01_real_models_catalog.json @@ -0,0 +1,5538 @@ +{ + "_probe": "P1", + "_question": "does GET /v1/models publish per-model pricing?", + "_captured": "real capture against a live Ramp Router account", + "_scrubbed": "credentials, emails, IPs, dashboard URLs; prompt and completion text blanked to \"\"", + "_status": 200, + "_headers": { + "alt-svc": "h3=\":443\"; ma=86400", + "cf-cache-status": "DYNAMIC", + "cf-ray": "a3238328fd280956-EWR", + "connection": "keep-alive", + "content-encoding": "br", + "content-type": "application/json", + "date": "Fri, 28 Aug 2026 13:02:47 GMT", + "server": "cloudflare", + "strict-transport-security": "max-age=31536000; includeSubDomains; preload", + "transfer-encoding": "chunked", + "vary": "Origin", + "x-content-type-options": "nosniff", + "x-request-id": "52ee6546-2898-4565-8832-8913ef8c4eb1", + "x-trace-id": "52ee6546-2898-4565-8832-8913ef8c4eb1" + }, + "_body": { + "object": "list", + "data": [ + { + "id": "accounts/fireworks/models/deepseek-v4-pro-0813", + "object": "model", + "created": 1786579200, + "owned_by": "fireworks", + "display_name": "DeepSeek-V4-Pro-0813", + "context_length": 1048576, + "router": { + "schema_version": 1, + "request_name": "accounts/fireworks/models/deepseek-v4-pro-0813", + "catalog_id": "fireworks:accounts/fireworks/models/deepseek-v4-pro-0813", + "provider_model": "accounts/fireworks/models/deepseek-v4-pro-0813", + "display_name": "DeepSeek-V4-Pro-0813", + "provider_display_name": "Fireworks", + "description": "DeepSeek-V4-Pro-0813 is the official release of DeepSeek-V4-Pro, superseding the preview version, with greatly enhanced agentic capabilities and performance improvements that are especially pronounced in production environments. It is built on the DeepSeek-V4-Pro (Preview) model structure, with a DSpark speculative decoding module attached.", + "status": "active", + "listing": { + "order": 0, + "featured": false, + "snapshot": false + }, + "limits": { + "context_window": 1048576, + "max_input_tokens": 1000000, + "max_output_tokens": 384000 + }, + "capabilities": { + "modalities": { + "input": [ + "" + ], + "output": [ + "text" + ] + }, + "tools": { + "supported": true, + "parallel": null + }, + "structured_outputs": true, + "temperature": true, + "prompt_caching": true, + "reasoning": { + "supported": true, + "efforts": [ + { + "value": "none", + "description": "Answers without reasoning first" + }, + { + "value": "minimal", + "description": "Reasons briefly before answering" + }, + { + "value": "low", + "description": "Fast responses with lighter reasoning" + }, + { + "value": "medium", + "description": "Balances speed and reasoning depth" + }, + { + "value": "high", + "description": "Greater reasoning depth" + }, + { + "value": "xhigh", + "description": "Reasons longer than high" + }, + { + "value": "max", + "description": "Reasons as long as the model allows" + } + ], + "default_effort": "high", + "summary": { + "output_supported": true, + "request_parameter_supported": false, + "values": [] + }, + "continuation": { + "request_include": [], + "required": false + } + }, + "verbosity": { + "supported": false, + "values": [] + } + }, + "pricing": { + "input": "", + "output": "3.96", + "cache_read_input": "0.044", + "cache_write_input": "0", + "cache_write_input_5m": "0", + "cache_write_input_1h": "0" + } + } + }, + { + "id": "grok-4.6", + "object": "model", + "created": 1786492800, + "owned_by": "xai", + "display_name": "Grok 4.6", + "context_length": 500000, + "router": { + "schema_version": 1, + "request_name": "grok-4.6", + "catalog_id": "xai:grok-4.6", + "provider_model": "grok-4.6", + "display_name": "Grok 4.6", + "provider_display_name": "xAI", + "status": "active", + "listing": { + "order": 1, + "featured": false, + "snapshot": false + }, + "limits": { + "context_window": 500000, + "max_input_tokens": null, + "max_output_tokens": 500000 + }, + "capabilities": { + "modalities": { + "input": [ + "", + "" + ], + "output": [ + "text" + ] + }, + "tools": { + "supported": true, + "parallel": null + }, + "structured_outputs": true, + "temperature": true, + "prompt_caching": true, + "reasoning": { + "supported": true, + "efforts": [ + { + "value": "minimal", + "description": "Reasons briefly before answering" + }, + { + "value": "low", + "description": "Fast responses with lighter reasoning" + }, + { + "value": "medium", + "description": "Balances speed and reasoning depth" + }, + { + "value": "high", + "description": "Greater reasoning depth" + }, + { + "value": "xhigh", + "description": "Reasons longer than high" + } + ], + "default_effort": "high", + "summary": { + "output_supported": true, + "request_parameter_supported": false, + "values": [] + }, + "continuation": { + "request_include": [], + "required": false + } + }, + "verbosity": { + "supported": false, + "values": [] + } + }, + "pricing": { + "input": "", + "output": "6", + "cache_read_input": "0.5", + "cache_write_input": "0", + "cache_write_input_5m": "0", + "cache_write_input_1h": "0" + } + } + }, + { + "id": "accounts/fireworks/models/nemotron-lightning-3p5-30b-a3b", + "object": "model", + "created": 1786060800, + "owned_by": "fireworks", + "display_name": "Nemotron Lightning 3.5 30B A3B", + "context_length": 262144, + "router": { + "schema_version": 1, + "request_name": "accounts/fireworks/models/nemotron-lightning-3p5-30b-a3b", + "catalog_id": "fireworks:accounts/fireworks/models/nemotron-lightning-3p5-30b-a3b", + "provider_model": "accounts/fireworks/models/nemotron-lightning-3p5-30b-a3b", + "display_name": "Nemotron Lightning 3.5 30B A3B", + "provider_display_name": "Fireworks", + "description": "Nemotron-Lightning-3.5-30B-A3B is a 30B-parameter Mixture-of-Experts language model (3B active) from NVIDIA's Nemotron-H family, built on a hybrid Mamba-Transformer architecture for efficient long-context inference. Like other models in the family, it responds to queries by first generating a reasoning trace and then concluding with a final response, with reasoning behavior configurable through a flag in the chat template. It includes a multi-token prediction (MTP) speculative decoding head for low-latency serving.", + "status": "active", + "listing": { + "order": 2, + "featured": false, + "snapshot": false + }, + "limits": { + "context_window": 262144, + "max_input_tokens": null, + "max_output_tokens": 262144 + }, + "capabilities": { + "modalities": { + "input": [ + "" + ], + "output": [ + "text" + ] + }, + "tools": { + "supported": true, + "parallel": null + }, + "structured_outputs": true, + "temperature": true, + "prompt_caching": true, + "reasoning": { + "supported": null, + "efforts": [], + "summary": { + "output_supported": null, + "request_parameter_supported": null, + "values": [] + }, + "continuation": { + "request_include": [], + "required": null + } + }, + "verbosity": { + "supported": false, + "values": [] + } + }, + "pricing": { + "input": "", + "output": "0.2", + "cache_read_input": "0.01", + "cache_write_input": "0", + "cache_write_input_5m": "0", + "cache_write_input_1h": "0" + } + } + }, + { + "id": "accounts/fireworks/models/qwen3p8-max", + "object": "model", + "created": 1785888000, + "owned_by": "fireworks", + "display_name": "Qwen 3.8 Max", + "context_length": 1000000, + "router": { + "schema_version": 1, + "request_name": "accounts/fireworks/models/qwen3p8-max", + "catalog_id": "fireworks:accounts/fireworks/models/qwen3p8-max", + "provider_model": "accounts/fireworks/models/qwen3p8-max", + "display_name": "Qwen 3.8 Max", + "provider_display_name": "Fireworks", + "description": "Built on the architectural foundation of Qwen3.5, Qwen3.8 delivers substantial gains across coding, professional work, research, and long-horizon agentic tasks. Beyond answering harder questions, Qwen3.8 is designed to carry complex, multi-step tasks through to completion with greater reliability.", + "status": "active", + "listing": { + "order": 3, + "featured": false, + "snapshot": false + }, + "limits": { + "context_window": 1000000, + "max_input_tokens": null, + "max_output_tokens": 131072 + }, + "capabilities": { + "modalities": { + "input": [ + "", + "" + ], + "output": [ + "text" + ] + }, + "tools": { + "supported": true, + "parallel": null + }, + "structured_outputs": true, + "temperature": true, + "prompt_caching": true, + "reasoning": { + "supported": true, + "efforts": [ + { + "value": "none", + "description": "Answers without reasoning first" + }, + { + "value": "minimal", + "description": "Reasons briefly before answering" + }, + { + "value": "low", + "description": "Fast responses with lighter reasoning" + }, + { + "value": "medium", + "description": "Balances speed and reasoning depth" + }, + { + "value": "high", + "description": "Greater reasoning depth" + }, + { + "value": "xhigh", + "description": "Reasons longer than high" + } + ], + "default_effort": "none", + "summary": { + "output_supported": true, + "request_parameter_supported": false, + "values": [] + }, + "continuation": { + "request_include": [], + "required": false + } + }, + "verbosity": { + "supported": false, + "values": [] + } + }, + "pricing": { + "input": "", + "output": "6", + "cache_read_input": "0.25", + "cache_write_input": "0", + "cache_write_input_5m": "0", + "cache_write_input_1h": "0" + } + } + }, + { + "id": "accounts/fireworks/models/deepseek-v4-flash-0731", + "object": "model", + "created": 1785456000, + "owned_by": "fireworks", + "display_name": "DeepSeek-V4-Flash-0731", + "context_length": 1048576, + "router": { + "schema_version": 1, + "request_name": "accounts/fireworks/models/deepseek-v4-flash-0731", + "catalog_id": "fireworks:accounts/fireworks/models/deepseek-v4-flash-0731", + "provider_model": "accounts/fireworks/models/deepseek-v4-flash-0731", + "display_name": "DeepSeek-V4-Flash-0731", + "provider_display_name": "Fireworks", + "description": "DeepSeek-V4-Flash-0731 is the official release of DeepSeek-V4-Flash, superseding the preview version, with substantially enhanced agentic capabilities. It has the same model structure as DeepSeek-V4-Flash-DSpark, i.e. it comes with a speculative decoding module attached.", + "status": "active", + "listing": { + "order": 4, + "featured": false, + "snapshot": false + }, + "limits": { + "context_window": 1048576, + "max_input_tokens": null, + "max_output_tokens": 1048576 + }, + "capabilities": { + "modalities": { + "input": [ + "" + ], + "output": [ + "text" + ] + }, + "tools": { + "supported": true, + "parallel": null + }, + "structured_outputs": true, + "temperature": true, + "prompt_caching": true, + "reasoning": { + "supported": true, + "efforts": [ + { + "value": "none", + "description": "Answers without reasoning first" + }, + { + "value": "minimal", + "description": "Reasons briefly before answering" + }, + { + "value": "low", + "description": "Fast responses with lighter reasoning" + }, + { + "value": "medium", + "description": "Balances speed and reasoning depth" + }, + { + "value": "high", + "description": "Greater reasoning depth" + }, + { + "value": "xhigh", + "description": "Reasons longer than high" + }, + { + "value": "max", + "description": "Reasons as long as the model allows" + } + ], + "default_effort": "high", + "summary": { + "output_supported": true, + "request_parameter_supported": false, + "values": [] + }, + "continuation": { + "request_include": [], + "required": false + } + }, + "verbosity": { + "supported": false, + "values": [] + } + }, + "pricing": { + "input": "", + "output": "0.66", + "cache_read_input": "0.007", + "cache_write_input": "0", + "cache_write_input_5m": "0", + "cache_write_input_1h": "0" + } + } + }, + { + "id": "claude-opus-5", + "object": "model", + "created": 1784851200, + "owned_by": "anthropic", + "display_name": "Claude Opus 5", + "context_length": 1000000, + "router": { + "schema_version": 1, + "request_name": "claude-opus-5", + "catalog_id": "anthropic:claude-opus-5", + "provider_model": "claude-opus-5", + "display_name": "Claude Opus 5", + "provider_display_name": "Anthropic", + "description": "For complex agentic coding and enterprise work", + "status": "active", + "listing": { + "order": 5, + "featured": true, + "snapshot": false + }, + "limits": { + "context_window": 1000000, + "max_input_tokens": 1000000, + "max_output_tokens": 128000 + }, + "capabilities": { + "modalities": { + "input": [ + "", + "" + ], + "output": [ + "text" + ] + }, + "tools": { + "supported": true, + "parallel": null + }, + "structured_outputs": true, + "temperature": true, + "prompt_caching": true, + "reasoning": { + "supported": true, + "efforts": [ + { + "value": "none", + "description": "Answers without reasoning first" + }, + { + "value": "minimal", + "description": "Reasons briefly before answering" + }, + { + "value": "low", + "description": "Fast responses with lighter reasoning" + }, + { + "value": "medium", + "description": "Balances speed and reasoning depth" + }, + { + "value": "high", + "description": "Greater reasoning depth" + }, + { + "value": "xhigh", + "description": "Reasons longer than high" + }, + { + "value": "max", + "description": "Reasons as long as the model allows" + } + ], + "default_effort": "none", + "summary": { + "output_supported": true, + "request_parameter_supported": false, + "values": [] + }, + "continuation": { + "request_include": [], + "required": false + } + }, + "verbosity": { + "supported": false, + "values": [] + } + }, + "pricing": { + "input": "", + "output": "25", + "cache_read_input": "0.5", + "cache_write_input": "0", + "cache_write_input_5m": "6.25", + "cache_write_input_1h": "10" + } + } + }, + { + "id": "accounts/fireworks/models/kimi-k3", + "object": "model", + "created": 1784419200, + "owned_by": "fireworks", + "display_name": "Kimi K3", + "context_length": 1048576, + "router": { + "schema_version": 1, + "request_name": "accounts/fireworks/models/kimi-k3", + "catalog_id": "fireworks:accounts/fireworks/models/kimi-k3", + "provider_model": "accounts/fireworks/models/kimi-k3", + "display_name": "Kimi K3", + "provider_display_name": "Fireworks", + "description": "Kimi K3 is Kimi’s most capable flagship model to date, with 2.8 trillion parameters. It is built on Kimi Delta Attention (KDA), a hybrid linear attention mechanism, and Attention Residuals, with native visual understanding and a 1M-token context window. It is the world’s first open-source model in the 3-trillion-parameter class, designed for frontier intelligence scenarios including long-horizon coding, knowledge work, and reasoning.", + "status": "active", + "listing": { + "order": 8, + "featured": true, + "snapshot": false + }, + "limits": { + "context_window": 1048576, + "max_input_tokens": null, + "max_output_tokens": 1048576 + }, + "capabilities": { + "modalities": { + "input": [ + "", + "" + ], + "output": [ + "text" + ] + }, + "tools": { + "supported": true, + "parallel": null + }, + "structured_outputs": true, + "temperature": true, + "prompt_caching": true, + "reasoning": { + "supported": true, + "efforts": [ + { + "value": "minimal", + "description": "Reasons briefly before answering" + }, + { + "value": "low", + "description": "Fast responses with lighter reasoning" + }, + { + "value": "medium", + "description": "Balances speed and reasoning depth" + }, + { + "value": "high", + "description": "Greater reasoning depth" + }, + { + "value": "xhigh", + "description": "Reasons longer than high" + }, + { + "value": "max", + "description": "Reasons as long as the model allows" + } + ], + "default_effort": "medium", + "summary": { + "output_supported": true, + "request_parameter_supported": false, + "values": [] + }, + "continuation": { + "request_include": [], + "required": false + } + }, + "verbosity": { + "supported": false, + "values": [] + } + }, + "pricing": { + "input": "", + "output": "15", + "cache_read_input": "0.3", + "cache_write_input": "0", + "cache_write_input_5m": "0", + "cache_write_input_1h": "0" + } + } + }, + { + "id": "accounts/fireworks/routers/kimi-k3-fast", + "object": "model", + "created": 1784419200, + "owned_by": "fireworks", + "display_name": "Kimi K3 Fast", + "context_length": 1048576, + "router": { + "schema_version": 1, + "request_name": "accounts/fireworks/routers/kimi-k3-fast", + "catalog_id": "fireworks:accounts/fireworks/routers/kimi-k3-fast", + "provider_model": "accounts/fireworks/routers/kimi-k3-fast", + "display_name": "Kimi K3 Fast", + "provider_display_name": "Fireworks", + "description": "Kimi K3 is Kimi’s most capable flagship model to date, with 2.8 trillion parameters. It is built on Kimi Delta Attention (KDA), a hybrid linear attention mechanism, and Attention Residuals, with native visual understanding and a 1M-token context window. It is the world’s first open-source model in the 3-trillion-parameter class, designed for frontier intelligence scenarios including long-horizon coding, knowledge work, and reasoning.", + "status": "active", + "listing": { + "order": 9, + "featured": false, + "snapshot": false + }, + "limits": { + "context_window": 1048576, + "max_input_tokens": null, + "max_output_tokens": 1048576 + }, + "capabilities": { + "modalities": { + "input": [ + "", + "" + ], + "output": [ + "text" + ] + }, + "tools": { + "supported": true, + "parallel": null + }, + "structured_outputs": true, + "temperature": true, + "prompt_caching": true, + "reasoning": { + "supported": true, + "efforts": [ + { + "value": "minimal", + "description": "Reasons briefly before answering" + }, + { + "value": "low", + "description": "Fast responses with lighter reasoning" + }, + { + "value": "medium", + "description": "Balances speed and reasoning depth" + }, + { + "value": "high", + "description": "Greater reasoning depth" + }, + { + "value": "xhigh", + "description": "Reasons longer than high" + }, + { + "value": "max", + "description": "Reasons as long as the model allows" + } + ], + "default_effort": "medium", + "summary": { + "output_supported": true, + "request_parameter_supported": false, + "values": [] + }, + "continuation": { + "request_include": [], + "required": false + } + }, + "verbosity": { + "supported": false, + "values": [] + } + }, + "pricing": { + "input": "", + "output": "22.5", + "cache_read_input": "0.45", + "cache_write_input": "0", + "cache_write_input_5m": "0", + "cache_write_input_1h": "0" + } + } + }, + { + "id": "claude-sonnet-5", + "object": "model", + "created": 1782691200, + "owned_by": "anthropic", + "display_name": "Claude Sonnet 5", + "context_length": 1000000, + "router": { + "schema_version": 1, + "request_name": "claude-sonnet-5", + "catalog_id": "anthropic:claude-sonnet-5", + "provider_model": "claude-sonnet-5", + "display_name": "Claude Sonnet 5", + "provider_display_name": "Anthropic", + "description": "The best combination of speed and intelligence", + "status": "active", + "listing": { + "order": 10, + "featured": false, + "snapshot": false + }, + "limits": { + "context_window": 1000000, + "max_input_tokens": 1000000, + "max_output_tokens": 128000 + }, + "capabilities": { + "modalities": { + "input": [ + "", + "" + ], + "output": [ + "text" + ] + }, + "tools": { + "supported": true, + "parallel": null + }, + "structured_outputs": true, + "temperature": true, + "prompt_caching": true, + "reasoning": { + "supported": true, + "efforts": [ + { + "value": "none", + "description": "Answers without reasoning first" + }, + { + "value": "minimal", + "description": "Reasons briefly before answering" + }, + { + "value": "low", + "description": "Fast responses with lighter reasoning" + }, + { + "value": "medium", + "description": "Balances speed and reasoning depth" + }, + { + "value": "high", + "description": "Greater reasoning depth" + }, + { + "value": "xhigh", + "description": "Reasons longer than high" + }, + { + "value": "max", + "description": "Reasons as long as the model allows" + } + ], + "default_effort": "high", + "summary": { + "output_supported": true, + "request_parameter_supported": false, + "values": [] + }, + "continuation": { + "request_include": [], + "required": false + } + }, + "verbosity": { + "supported": false, + "values": [] + } + }, + "pricing": { + "input": "", + "output": "10", + "cache_read_input": "0.2", + "cache_write_input": "0", + "cache_write_input_5m": "2.5", + "cache_write_input_1h": "4" + } + } + }, + { + "id": "grok-4.5", + "object": "model", + "created": 1782691200, + "owned_by": "xai", + "display_name": "Grok 4.5", + "context_length": 500000, + "router": { + "schema_version": 1, + "request_name": "grok-4.5", + "catalog_id": "xai:grok-4.5", + "provider_model": "grok-4.5", + "display_name": "Grok 4.5", + "provider_display_name": "xAI", + "description": "SpaceXAI's intelligent coding model for agentic software, engineering, and workflow tasks. Trained in SpaceXAI's data centers in Memphis with new datasets spanning science, engineering, and math.", + "status": "active", + "listing": { + "order": 11, + "featured": false, + "snapshot": false + }, + "limits": { + "context_window": 500000, + "max_input_tokens": null, + "max_output_tokens": 500000 + }, + "capabilities": { + "modalities": { + "input": [ + "", + "" + ], + "output": [ + "text" + ] + }, + "tools": { + "supported": true, + "parallel": null + }, + "structured_outputs": true, + "temperature": true, + "prompt_caching": true, + "reasoning": { + "supported": true, + "efforts": [ + { + "value": "minimal", + "description": "Reasons briefly before answering" + }, + { + "value": "low", + "description": "Fast responses with lighter reasoning" + }, + { + "value": "medium", + "description": "Balances speed and reasoning depth" + }, + { + "value": "high", + "description": "Greater reasoning depth" + }, + { + "value": "xhigh", + "description": "Reasons longer than high" + } + ], + "default_effort": "high", + "summary": { + "output_supported": true, + "request_parameter_supported": false, + "values": [] + }, + "continuation": { + "request_include": [], + "required": false + } + }, + "verbosity": { + "supported": false, + "values": [] + } + }, + "pricing": { + "input": "", + "output": "6", + "cache_read_input": "0.3", + "cache_write_input": "0", + "cache_write_input_5m": "0", + "cache_write_input_1h": "0" + } + } + }, + { + "id": "gpt-5.6-luna", + "object": "model", + "created": 1782228658, + "owned_by": "openai", + "display_name": "GPT-5.6 Luna", + "context_length": 1050000, + "router": { + "schema_version": 1, + "request_name": "gpt-5.6-luna", + "catalog_id": "openai:gpt-5.6-luna", + "provider_model": "gpt-5.6-luna", + "display_name": "GPT-5.6 Luna", + "provider_display_name": "OpenAI", + "description": "GPT-5.6 Luna is designed for cost-sensitive, high-volume workloads. It roughly corresponds to the nano model tier used in earlier GPT-5 families.", + "status": "active", + "listing": { + "order": 12, + "featured": true, + "snapshot": false + }, + "limits": { + "context_window": 1050000, + "max_input_tokens": null, + "max_output_tokens": 128000 + }, + "capabilities": { + "modalities": { + "input": [ + "", + "" + ], + "output": [ + "text" + ] + }, + "tools": { + "supported": true, + "parallel": null + }, + "structured_outputs": true, + "temperature": false, + "prompt_caching": true, + "reasoning": { + "supported": true, + "efforts": [ + { + "value": "none", + "description": "Answers without reasoning first" + }, + { + "value": "low", + "description": "Fast responses with lighter reasoning" + }, + { + "value": "medium", + "description": "Balances speed and reasoning depth" + }, + { + "value": "high", + "description": "Greater reasoning depth" + }, + { + "value": "xhigh", + "description": "Reasons longer than high" + }, + { + "value": "max", + "description": "Reasons as long as the model allows" + } + ], + "default_effort": "medium", + "summary": { + "output_supported": true, + "request_parameter_supported": true, + "values": [ + "auto", + "concise", + "detailed" + ], + "default": "auto" + }, + "continuation": { + "request_include": [ + "reasoning.encrypted_content" + ], + "required": true + } + }, + "verbosity": { + "supported": true, + "values": [ + "low", + "medium", + "high" + ], + "default": "medium" + } + }, + "pricing": { + "input": "", + "output": "1.2", + "cache_read_input": "0.02", + "cache_write_input": "0.25", + "cache_write_input_5m": "0", + "cache_write_input_1h": "0" + } + } + }, + { + "id": "gpt-5.6-terra", + "object": "model", + "created": 1782228459, + "owned_by": "openai", + "display_name": "GPT-5.6 Terra", + "context_length": 1050000, + "router": { + "schema_version": 1, + "request_name": "gpt-5.6-terra", + "catalog_id": "openai:gpt-5.6-terra", + "provider_model": "gpt-5.6-terra", + "display_name": "GPT-5.6 Terra", + "provider_display_name": "OpenAI", + "description": "GPT-5.6 Terra is designed for workloads that balance intelligence and cost. It roughly corresponds to the mini model tier used in earlier GPT-5 families.", + "status": "active", + "listing": { + "order": 13, + "featured": false, + "snapshot": false + }, + "limits": { + "context_window": 1050000, + "max_input_tokens": null, + "max_output_tokens": 128000 + }, + "capabilities": { + "modalities": { + "input": [ + "", + "" + ], + "output": [ + "text" + ] + }, + "tools": { + "supported": true, + "parallel": null + }, + "structured_outputs": true, + "temperature": false, + "prompt_caching": true, + "reasoning": { + "supported": true, + "efforts": [ + { + "value": "none", + "description": "Answers without reasoning first" + }, + { + "value": "low", + "description": "Fast responses with lighter reasoning" + }, + { + "value": "medium", + "description": "Balances speed and reasoning depth" + }, + { + "value": "high", + "description": "Greater reasoning depth" + }, + { + "value": "xhigh", + "description": "Reasons longer than high" + }, + { + "value": "max", + "description": "Reasons as long as the model allows" + } + ], + "default_effort": "medium", + "summary": { + "output_supported": true, + "request_parameter_supported": true, + "values": [ + "auto", + "concise", + "detailed" + ], + "default": "auto" + }, + "continuation": { + "request_include": [ + "reasoning.encrypted_content" + ], + "required": true + } + }, + "verbosity": { + "supported": true, + "values": [ + "low", + "medium", + "high" + ], + "default": "medium" + } + }, + "pricing": { + "input": "", + "output": "12", + "cache_read_input": "0.2", + "cache_write_input": "2.5", + "cache_write_input_5m": "0", + "cache_write_input_1h": "0" + } + } + }, + { + "id": "gpt-5.6-sol", + "object": "model", + "created": 1782228018, + "owned_by": "openai", + "display_name": "GPT-5.6 Sol", + "context_length": 1050000, + "router": { + "schema_version": 1, + "request_name": "gpt-5.6-sol", + "catalog_id": "openai:gpt-5.6-sol", + "provider_model": "gpt-5.6-sol", + "display_name": "GPT-5.6 Sol", + "provider_display_name": "OpenAI", + "description": "GPT-5.6 Sol is the frontier model in the GPT-5.6 family. It roughly corresponds to the unsuffixed model tier used in earlier GPT-5 families. The gpt-5.6 alias routes requests to GPT-5.6 Sol.", + "status": "active", + "listing": { + "order": 14, + "featured": true, + "snapshot": false + }, + "limits": { + "context_window": 1050000, + "max_input_tokens": null, + "max_output_tokens": 128000 + }, + "capabilities": { + "modalities": { + "input": [ + "", + "" + ], + "output": [ + "text" + ] + }, + "tools": { + "supported": true, + "parallel": null + }, + "structured_outputs": true, + "temperature": false, + "prompt_caching": true, + "reasoning": { + "supported": true, + "efforts": [ + { + "value": "none", + "description": "Answers without reasoning first" + }, + { + "value": "low", + "description": "Fast responses with lighter reasoning" + }, + { + "value": "medium", + "description": "Balances speed and reasoning depth" + }, + { + "value": "high", + "description": "Greater reasoning depth" + }, + { + "value": "xhigh", + "description": "Reasons longer than high" + }, + { + "value": "max", + "description": "Reasons as long as the model allows" + } + ], + "default_effort": "medium", + "summary": { + "output_supported": true, + "request_parameter_supported": true, + "values": [ + "auto", + "concise", + "detailed" + ], + "default": "auto" + }, + "continuation": { + "request_include": [ + "reasoning.encrypted_content" + ], + "required": true + } + }, + "verbosity": { + "supported": true, + "values": [ + "low", + "medium", + "high" + ], + "default": "medium" + } + }, + "pricing": { + "input": "", + "output": "20", + "cache_read_input": "0.4", + "cache_write_input": "5", + "cache_write_input_5m": "0", + "cache_write_input_1h": "0" + } + } + }, + { + "id": "accounts/fireworks/models/glm-5p2", + "object": "model", + "created": 1781568000, + "owned_by": "fireworks", + "display_name": "GLM 5.2", + "context_length": 1048576, + "router": { + "schema_version": 1, + "request_name": "accounts/fireworks/models/glm-5p2", + "catalog_id": "fireworks:accounts/fireworks/models/glm-5p2", + "provider_model": "accounts/fireworks/models/glm-5p2", + "display_name": "GLM 5.2", + "provider_display_name": "Fireworks", + "aliases": [ + "glm-5p2" + ], + "description": "GLM-5.2 introduces a robust 1M-token context and advanced, multi-effort coding capabilities to significantly enhance performance on long-horizon tasks. Its new IndexShare architecture and improved MTP layer simultaneously boost efficiency by reducing per-token FLOPs and increasing speculative decoding lengths.", + "status": "active", + "listing": { + "order": 15, + "featured": true, + "snapshot": false + }, + "limits": { + "context_window": 1048576, + "max_input_tokens": null, + "max_output_tokens": 1048576 + }, + "capabilities": { + "modalities": { + "input": [ + "" + ], + "output": [ + "text" + ] + }, + "tools": { + "supported": true, + "parallel": null + }, + "structured_outputs": true, + "temperature": true, + "prompt_caching": true, + "reasoning": { + "supported": true, + "efforts": [ + { + "value": "none", + "description": "Answers without reasoning first" + }, + { + "value": "minimal", + "description": "Reasons briefly before answering" + }, + { + "value": "low", + "description": "Fast responses with lighter reasoning" + }, + { + "value": "medium", + "description": "Balances speed and reasoning depth" + }, + { + "value": "high", + "description": "Greater reasoning depth" + }, + { + "value": "xhigh", + "description": "Reasons longer than high" + }, + { + "value": "max", + "description": "Reasons as long as the model allows" + } + ], + "default_effort": "xhigh", + "summary": { + "output_supported": true, + "request_parameter_supported": false, + "values": [] + }, + "continuation": { + "request_include": [], + "required": false + } + }, + "verbosity": { + "supported": false, + "values": [] + } + }, + "pricing": { + "input": "", + "output": "4.4", + "cache_read_input": "0.14", + "cache_write_input": "0", + "cache_write_input_5m": "0", + "cache_write_input_1h": "0" + } + } + }, + { + "id": "accounts/fireworks/routers/glm-5p2-fast", + "object": "model", + "created": 1781568000, + "owned_by": "fireworks", + "display_name": "GLM 5.2 Fast", + "context_length": 1048576, + "router": { + "schema_version": 1, + "request_name": "accounts/fireworks/routers/glm-5p2-fast", + "catalog_id": "fireworks:accounts/fireworks/routers/glm-5p2-fast", + "provider_model": "accounts/fireworks/routers/glm-5p2-fast", + "display_name": "GLM 5.2 Fast", + "provider_display_name": "Fireworks", + "description": "GLM-5.2 introduces a robust 1M-token context and advanced, multi-effort coding capabilities to significantly enhance performance on long-horizon tasks. Its new IndexShare architecture and improved MTP layer simultaneously boost efficiency by reducing per-token FLOPs and increasing speculative decoding lengths.", + "status": "active", + "listing": { + "order": 17, + "featured": false, + "snapshot": false + }, + "limits": { + "context_window": 1048576, + "max_input_tokens": null, + "max_output_tokens": 1048576 + }, + "capabilities": { + "modalities": { + "input": [ + "" + ], + "output": [ + "text" + ] + }, + "tools": { + "supported": true, + "parallel": null + }, + "structured_outputs": true, + "temperature": true, + "prompt_caching": true, + "reasoning": { + "supported": true, + "efforts": [ + { + "value": "none", + "description": "Answers without reasoning first" + }, + { + "value": "minimal", + "description": "Reasons briefly before answering" + }, + { + "value": "low", + "description": "Fast responses with lighter reasoning" + }, + { + "value": "medium", + "description": "Balances speed and reasoning depth" + }, + { + "value": "high", + "description": "Greater reasoning depth" + }, + { + "value": "xhigh", + "description": "Reasons longer than high" + }, + { + "value": "max", + "description": "Reasons as long as the model allows" + } + ], + "default_effort": "xhigh", + "summary": { + "output_supported": true, + "request_parameter_supported": false, + "values": [] + }, + "continuation": { + "request_include": [], + "required": false + } + }, + "verbosity": { + "supported": false, + "values": [] + } + }, + "pricing": { + "input": "", + "output": "6.6", + "cache_read_input": "0.21", + "cache_write_input": "0", + "cache_write_input_5m": "0", + "cache_write_input_1h": "0" + } + } + }, + { + "id": "accounts/fireworks/models/kimi-k2p7-code", + "object": "model", + "created": 1781222400, + "owned_by": "fireworks", + "display_name": "Kimi K2.7 Code", + "context_length": 262144, + "router": { + "schema_version": 1, + "request_name": "accounts/fireworks/models/kimi-k2p7-code", + "catalog_id": "fireworks:accounts/fireworks/models/kimi-k2p7-code", + "provider_model": "accounts/fireworks/models/kimi-k2p7-code", + "display_name": "Kimi K2.7 Code", + "provider_display_name": "Fireworks", + "description": "Kimi K2.7 Code is a coding-focused agentic model built upon Kimi K2.6. With substantial improvements on real-world long-horizon coding tasks, it strengthens end-to-end task completion across complex software engineering workflows while improving token efficiency, reducing thinking-token usage by approximately 30% compared with Kimi K2.6.", + "status": "active", + "listing": { + "order": 18, + "featured": false, + "snapshot": false + }, + "limits": { + "context_window": 262144, + "max_input_tokens": null, + "max_output_tokens": 262144 + }, + "capabilities": { + "modalities": { + "input": [ + "", + "" + ], + "output": [ + "text" + ] + }, + "tools": { + "supported": true, + "parallel": null + }, + "structured_outputs": true, + "temperature": true, + "prompt_caching": true, + "reasoning": { + "supported": true, + "efforts": [ + { + "value": "none", + "description": "Answers without reasoning first" + }, + { + "value": "minimal", + "description": "Reasons briefly before answering" + }, + { + "value": "low", + "description": "Fast responses with lighter reasoning" + }, + { + "value": "medium", + "description": "Balances speed and reasoning depth" + }, + { + "value": "high", + "description": "Greater reasoning depth" + }, + { + "value": "xhigh", + "description": "Reasons longer than high" + } + ], + "default_effort": "high", + "summary": { + "output_supported": true, + "request_parameter_supported": false, + "values": [] + }, + "continuation": { + "request_include": [], + "required": false + } + }, + "verbosity": { + "supported": false, + "values": [] + } + }, + "pricing": { + "input": "", + "output": "4", + "cache_read_input": "0.19", + "cache_write_input": "0", + "cache_write_input_5m": "0", + "cache_write_input_1h": "0" + } + } + }, + { + "id": "accounts/fireworks/models/minimax-m3", + "object": "model", + "created": 1781136000, + "owned_by": "fireworks", + "display_name": "Minimax M3", + "context_length": 512000, + "router": { + "schema_version": 1, + "request_name": "accounts/fireworks/models/minimax-m3", + "catalog_id": "fireworks:accounts/fireworks/models/minimax-m3", + "provider_model": "accounts/fireworks/models/minimax-m3", + "display_name": "Minimax M3", + "provider_display_name": "Fireworks", + "description": "MiniMax-M3 is a native multimodal model with 1M context running ~428B parameters and ~23B activated parameters. It brings native multimodality. enabling deeper semantic fusion across text, image, and video. M3 also introduces MiniMax Sparse Attention (MSA) to improve long context efficiency, achieving frontier-level performance across long-horizon agentic benchmarks, excelling in both coding and cowork.", + "status": "active", + "listing": { + "order": 20, + "featured": false, + "snapshot": false + }, + "limits": { + "context_window": 512000, + "max_input_tokens": null, + "max_output_tokens": 512000 + }, + "capabilities": { + "modalities": { + "input": [ + "" + ], + "output": [ + "text" + ] + }, + "tools": { + "supported": true, + "parallel": null + }, + "structured_outputs": true, + "temperature": true, + "prompt_caching": true, + "reasoning": { + "supported": true, + "efforts": [ + { + "value": "none", + "description": "Answers without reasoning first" + }, + { + "value": "minimal", + "description": "Reasons briefly before answering" + }, + { + "value": "low", + "description": "Fast responses with lighter reasoning" + }, + { + "value": "medium", + "description": "Balances speed and reasoning depth" + }, + { + "value": "high", + "description": "Greater reasoning depth" + }, + { + "value": "xhigh", + "description": "Reasons longer than high" + } + ], + "default_effort": "none", + "summary": { + "output_supported": true, + "request_parameter_supported": false, + "values": [] + }, + "continuation": { + "request_include": [], + "required": false + } + }, + "verbosity": { + "supported": false, + "values": [] + } + }, + "pricing": { + "input": "", + "output": "1.2", + "cache_read_input": "0.06", + "cache_write_input": "0", + "cache_write_input_5m": "0", + "cache_write_input_1h": "0" + } + } + }, + { + "id": "claude-fable-5", + "object": "model", + "created": 1780790400, + "owned_by": "anthropic", + "display_name": "Claude Fable 5", + "context_length": 1000000, + "router": { + "schema_version": 1, + "request_name": "claude-fable-5", + "catalog_id": "anthropic:claude-fable-5", + "provider_model": "claude-fable-5", + "display_name": "Claude Fable 5", + "provider_display_name": "Anthropic", + "description": "Next-generation intelligence for long-running agents", + "status": "active", + "listing": { + "order": 22, + "featured": false, + "snapshot": false + }, + "limits": { + "context_window": 1000000, + "max_input_tokens": 1000000, + "max_output_tokens": 128000 + }, + "capabilities": { + "modalities": { + "input": [ + "", + "" + ], + "output": [ + "text" + ] + }, + "tools": { + "supported": true, + "parallel": null + }, + "structured_outputs": true, + "temperature": true, + "prompt_caching": true, + "reasoning": { + "supported": true, + "efforts": [ + { + "value": "minimal", + "description": "Reasons briefly before answering" + }, + { + "value": "low", + "description": "Fast responses with lighter reasoning" + }, + { + "value": "medium", + "description": "Balances speed and reasoning depth" + }, + { + "value": "high", + "description": "Greater reasoning depth" + }, + { + "value": "xhigh", + "description": "Reasons longer than high" + }, + { + "value": "max", + "description": "Reasons as long as the model allows" + } + ], + "default_effort": "high", + "summary": { + "output_supported": true, + "request_parameter_supported": false, + "values": [] + }, + "continuation": { + "request_include": [], + "required": false + } + }, + "verbosity": { + "supported": false, + "values": [] + } + }, + "pricing": { + "input": "", + "output": "50", + "cache_read_input": "1", + "cache_write_input": "0", + "cache_write_input_5m": "12.5", + "cache_write_input_1h": "20" + } + } + }, + { + "id": "accounts/fireworks/models/nemotron-3-ultra-nvfp4", + "object": "model", + "created": 1780358400, + "owned_by": "fireworks", + "display_name": "NVIDIA Nemotron 3 Ultra NVFP4", + "context_length": 262144, + "router": { + "schema_version": 1, + "request_name": "accounts/fireworks/models/nemotron-3-ultra-nvfp4", + "catalog_id": "fireworks:accounts/fireworks/models/nemotron-3-ultra-nvfp4", + "provider_model": "accounts/fireworks/models/nemotron-3-ultra-nvfp4", + "display_name": "NVIDIA Nemotron 3 Ultra NVFP4", + "provider_display_name": "Fireworks", + "description": "Nemotron-3-Ultra-550B-A55B-NVFP4 is a frontier-scale large language model (LLM) trained by NVIDIA, designed to deliver strong agentic, reasoning, and conversational capabilities. It is optimized for the most demanding workloads, including complex multi-step agents, long-context analysis, and high-accuracy reasoning over code, math, and science. The model employs a hybrid Latent Mixture-of-Experts (LatentMoE) architecture, utilizing interleaved Mamba-2 and MoE layers, along with select Attention layers. Like the Super model, the Ultra model incorporates Multi-Token Prediction (MTP) layers for faster text generation and improved quality, and it is trained using an NVFP4 pre-training recipe to maximize compute efficiency. The model has 55B active parameters and 550B parameters in total.", + "status": "active", + "listing": { + "order": 23, + "featured": false, + "snapshot": false + }, + "limits": { + "context_window": 262144, + "max_input_tokens": null, + "max_output_tokens": 262144 + }, + "capabilities": { + "modalities": { + "input": [ + "" + ], + "output": [ + "text" + ] + }, + "tools": { + "supported": true, + "parallel": null + }, + "structured_outputs": true, + "temperature": true, + "prompt_caching": true, + "reasoning": { + "supported": true, + "efforts": [ + { + "value": "none", + "description": "Answers without reasoning first" + }, + { + "value": "minimal", + "description": "Reasons briefly before answering" + }, + { + "value": "low", + "description": "Fast responses with lighter reasoning" + }, + { + "value": "medium", + "description": "Balances speed and reasoning depth" + }, + { + "value": "high", + "description": "Greater reasoning depth" + }, + { + "value": "xhigh", + "description": "Reasons longer than high" + } + ], + "default_effort": "none", + "summary": { + "output_supported": true, + "request_parameter_supported": false, + "values": [] + }, + "continuation": { + "request_include": [], + "required": false + } + }, + "verbosity": { + "supported": false, + "values": [] + } + }, + "pricing": { + "input": "", + "output": "2.4", + "cache_read_input": "0.12", + "cache_write_input": "0", + "cache_write_input_5m": "0", + "cache_write_input_1h": "0" + } + } + }, + { + "id": "claude-opus-4-8", + "object": "model", + "created": 1779926400, + "owned_by": "anthropic", + "display_name": "Claude Opus 4.8", + "context_length": 1000000, + "router": { + "schema_version": 1, + "request_name": "claude-opus-4-8", + "catalog_id": "anthropic:claude-opus-4-8", + "provider_model": "claude-opus-4-8", + "display_name": "Claude Opus 4.8", + "provider_display_name": "Anthropic", + "aliases": [ + "opus-4.8" + ], + "status": "active", + "listing": { + "order": 24, + "featured": false, + "snapshot": false + }, + "limits": { + "context_window": 1000000, + "max_input_tokens": 1000000, + "max_output_tokens": 128000 + }, + "capabilities": { + "modalities": { + "input": [ + "", + "" + ], + "output": [ + "text" + ] + }, + "tools": { + "supported": true, + "parallel": null + }, + "structured_outputs": true, + "temperature": true, + "prompt_caching": true, + "reasoning": { + "supported": true, + "efforts": [ + { + "value": "none", + "description": "Answers without reasoning first" + }, + { + "value": "minimal", + "description": "Reasons briefly before answering" + }, + { + "value": "low", + "description": "Fast responses with lighter reasoning" + }, + { + "value": "medium", + "description": "Balances speed and reasoning depth" + }, + { + "value": "high", + "description": "Greater reasoning depth" + }, + { + "value": "xhigh", + "description": "Reasons longer than high" + }, + { + "value": "max", + "description": "Reasons as long as the model allows" + } + ], + "default_effort": "none", + "summary": { + "output_supported": true, + "request_parameter_supported": false, + "values": [] + }, + "continuation": { + "request_include": [], + "required": false + } + }, + "verbosity": { + "supported": false, + "values": [] + } + }, + "pricing": { + "input": "", + "output": "25", + "cache_read_input": "0.5", + "cache_write_input": "0", + "cache_write_input_5m": "6.25", + "cache_write_input_1h": "10" + } + } + }, + { + "id": "accounts/fireworks/models/deepseek-v4-flash", + "object": "model", + "created": 1776988800, + "owned_by": "fireworks", + "display_name": "DeepSeek-V4-Flash", + "context_length": 1048576, + "router": { + "schema_version": 1, + "request_name": "accounts/fireworks/models/deepseek-v4-flash", + "catalog_id": "fireworks:accounts/fireworks/models/deepseek-v4-flash", + "provider_model": "accounts/fireworks/models/deepseek-v4-flash", + "display_name": "DeepSeek-V4-Flash", + "provider_display_name": "Fireworks", + "description": "DeepSeek-V4-Flash is a streamlined open-source Mixture-of-Experts model optimized for fast, cost-efficient inference while preserving strong reasoning and coding performance at 1M token context scale. It leverages the same hybrid attention innovations as Pro but is tuned for lower latency and higher throughput in real-time applications. It delivers near-Pro reasoning quality under sufficient compute budget, making it ideal for interactive agents and high-volume production workloads.", + "status": "active", + "listing": { + "order": 28, + "featured": true, + "snapshot": false + }, + "limits": { + "context_window": 1048576, + "max_input_tokens": null, + "max_output_tokens": 1048576 + }, + "capabilities": { + "modalities": { + "input": [ + "" + ], + "output": [ + "text" + ] + }, + "tools": { + "supported": true, + "parallel": null + }, + "structured_outputs": true, + "temperature": true, + "prompt_caching": true, + "reasoning": { + "supported": true, + "efforts": [ + { + "value": "none", + "description": "Answers without reasoning first" + }, + { + "value": "minimal", + "description": "Reasons briefly before answering" + }, + { + "value": "low", + "description": "Fast responses with lighter reasoning" + }, + { + "value": "medium", + "description": "Balances speed and reasoning depth" + }, + { + "value": "high", + "description": "Greater reasoning depth" + }, + { + "value": "xhigh", + "description": "Reasons longer than high" + }, + { + "value": "max", + "description": "Reasons as long as the model allows" + } + ], + "default_effort": "high", + "summary": { + "output_supported": true, + "request_parameter_supported": false, + "values": [] + }, + "continuation": { + "request_include": [], + "required": false + } + }, + "verbosity": { + "supported": false, + "values": [] + } + }, + "pricing": { + "input": "", + "output": "0.28", + "cache_read_input": "0.028", + "cache_write_input": "0", + "cache_write_input_5m": "0", + "cache_write_input_1h": "0" + } + } + }, + { + "id": "gpt-5.5-pro", + "object": "model", + "created": 1776894349, + "owned_by": "openai", + "display_name": "GPT-5.5 Pro", + "context_length": 1050000, + "router": { + "schema_version": 1, + "request_name": "gpt-5.5-pro", + "catalog_id": "openai:gpt-5.5-pro", + "provider_model": "gpt-5.5-pro", + "display_name": "GPT-5.5 Pro", + "provider_display_name": "OpenAI", + "description": "GPT-5.5 Pro uses more compute to think harder and provide consistently better answers.", + "status": "active", + "listing": { + "order": 31, + "featured": false, + "snapshot": false + }, + "limits": { + "context_window": 1050000, + "max_input_tokens": null, + "max_output_tokens": 128000 + }, + "capabilities": { + "modalities": { + "input": [ + "", + "" + ], + "output": [ + "text" + ] + }, + "tools": { + "supported": true, + "parallel": null + }, + "structured_outputs": true, + "temperature": false, + "prompt_caching": false, + "reasoning": { + "supported": true, + "efforts": [ + { + "value": "medium", + "description": "Balances speed and reasoning depth" + }, + { + "value": "high", + "description": "Greater reasoning depth" + }, + { + "value": "xhigh", + "description": "Reasons longer than high" + } + ], + "default_effort": "medium", + "summary": { + "output_supported": true, + "request_parameter_supported": true, + "values": [ + "auto", + "concise", + "detailed" + ], + "default": "auto" + }, + "continuation": { + "request_include": [ + "reasoning.encrypted_content" + ], + "required": true + } + }, + "verbosity": { + "supported": true, + "values": [ + "low", + "medium", + "high" + ], + "default": "medium" + } + }, + "pricing": { + "input": "", + "output": "180", + "cache_read_input": "0", + "cache_write_input": "0", + "cache_write_input_5m": "0", + "cache_write_input_1h": "0" + } + } + }, + { + "id": "gpt-5.5", + "object": "model", + "created": 1776824847, + "owned_by": "openai", + "display_name": "GPT-5.5", + "context_length": 1050000, + "router": { + "schema_version": 1, + "request_name": "gpt-5.5", + "catalog_id": "openai:gpt-5.5", + "provider_model": "gpt-5.5", + "display_name": "GPT-5.5", + "provider_display_name": "OpenAI", + "description": "GPT-5.5 is our newest frontier model for the most complex professional work. Learn more in our GPT-5.5 model guidance . Reasoning.effort supports: none, low, medium (default), high and xhigh.", + "status": "active", + "listing": { + "order": 33, + "featured": false, + "snapshot": false + }, + "limits": { + "context_window": 1050000, + "max_input_tokens": null, + "max_output_tokens": 128000 + }, + "capabilities": { + "modalities": { + "input": [ + "", + "" + ], + "output": [ + "text" + ] + }, + "tools": { + "supported": true, + "parallel": null + }, + "structured_outputs": true, + "temperature": false, + "prompt_caching": true, + "reasoning": { + "supported": true, + "efforts": [ + { + "value": "none", + "description": "Answers without reasoning first" + }, + { + "value": "low", + "description": "Fast responses with lighter reasoning" + }, + { + "value": "medium", + "description": "Balances speed and reasoning depth" + }, + { + "value": "high", + "description": "Greater reasoning depth" + }, + { + "value": "xhigh", + "description": "Reasons longer than high" + } + ], + "default_effort": "medium", + "summary": { + "output_supported": true, + "request_parameter_supported": true, + "values": [ + "auto", + "concise", + "detailed" + ], + "default": "auto" + }, + "continuation": { + "request_include": [ + "reasoning.encrypted_content" + ], + "required": true + } + }, + "verbosity": { + "supported": true, + "values": [ + "low", + "medium", + "high" + ], + "default": "medium" + } + }, + "pricing": { + "input": "", + "output": "30", + "cache_read_input": "0.5", + "cache_write_input": "0", + "cache_write_input_5m": "0", + "cache_write_input_1h": "0" + } + } + }, + { + "id": "accounts/fireworks/models/kimi-k2p6", + "object": "model", + "created": 1776384000, + "owned_by": "fireworks", + "display_name": "Kimi K2.6", + "context_length": 262144, + "router": { + "schema_version": 1, + "request_name": "accounts/fireworks/models/kimi-k2p6", + "catalog_id": "fireworks:accounts/fireworks/models/kimi-k2p6", + "provider_model": "accounts/fireworks/models/kimi-k2p6", + "display_name": "Kimi K2.6", + "provider_display_name": "Fireworks", + "description": "Kimi K2.6 is an open-source, native multimodal agentic model that advances practical capabilities in long-horizon coding, coding-driven design, proactive autonomous execution, and swarm-based task orchestration.", + "status": "active", + "listing": { + "order": 34, + "featured": false, + "snapshot": false + }, + "limits": { + "context_window": 262144, + "max_input_tokens": null, + "max_output_tokens": 262144 + }, + "capabilities": { + "modalities": { + "input": [ + "", + "" + ], + "output": [ + "text" + ] + }, + "tools": { + "supported": true, + "parallel": null + }, + "structured_outputs": true, + "temperature": true, + "prompt_caching": true, + "reasoning": { + "supported": true, + "efforts": [ + { + "value": "none", + "description": "Answers without reasoning first" + }, + { + "value": "minimal", + "description": "Reasons briefly before answering" + }, + { + "value": "low", + "description": "Fast responses with lighter reasoning" + }, + { + "value": "medium", + "description": "Balances speed and reasoning depth" + }, + { + "value": "high", + "description": "Greater reasoning depth" + }, + { + "value": "xhigh", + "description": "Reasons longer than high" + } + ], + "default_effort": "none", + "summary": { + "output_supported": true, + "request_parameter_supported": false, + "values": [] + }, + "continuation": { + "request_include": [], + "required": false + } + }, + "verbosity": { + "supported": false, + "values": [] + } + }, + "pricing": { + "input": "", + "output": "4", + "cache_read_input": "0.16", + "cache_write_input": "0", + "cache_write_input_5m": "0", + "cache_write_input_1h": "0" + } + } + }, + { + "id": "grok-4.3", + "object": "model", + "created": 1776384000, + "owned_by": "xai", + "display_name": "Grok 4.3", + "context_length": 1000000, + "router": { + "schema_version": 1, + "request_name": "grok-4.3", + "catalog_id": "xai:grok-4.3", + "provider_model": "grok-4.3", + "display_name": "Grok 4.3", + "provider_display_name": "xAI", + "description": "Fast, reliable model with strong tool calling and instruction following capabilities.", + "status": "active", + "listing": { + "order": 36, + "featured": false, + "snapshot": false + }, + "limits": { + "context_window": 1000000, + "max_input_tokens": null, + "max_output_tokens": 1000000 + }, + "capabilities": { + "modalities": { + "input": [ + "", + "" + ], + "output": [ + "text" + ] + }, + "tools": { + "supported": true, + "parallel": null + }, + "structured_outputs": true, + "temperature": true, + "prompt_caching": true, + "reasoning": { + "supported": true, + "efforts": [ + { + "value": "none", + "description": "Answers without reasoning first" + }, + { + "value": "minimal", + "description": "Reasons briefly before answering" + }, + { + "value": "low", + "description": "Fast responses with lighter reasoning" + }, + { + "value": "medium", + "description": "Balances speed and reasoning depth" + }, + { + "value": "high", + "description": "Greater reasoning depth" + }, + { + "value": "xhigh", + "description": "Reasons longer than high" + } + ], + "default_effort": "none", + "summary": { + "output_supported": true, + "request_parameter_supported": false, + "values": [] + }, + "continuation": { + "request_include": [], + "required": false + } + }, + "verbosity": { + "supported": false, + "values": [] + } + }, + "pricing": { + "input": "", + "output": "2.5", + "cache_read_input": "0.2", + "cache_write_input": "0", + "cache_write_input_5m": "0", + "cache_write_input_1h": "0" + } + } + }, + { + "id": "grok-build-0.1", + "object": "model", + "created": 1776297600, + "owned_by": "xai", + "display_name": "Grok Build 0.1", + "context_length": 256000, + "router": { + "schema_version": 1, + "request_name": "grok-build-0.1", + "catalog_id": "xai:grok-build-0.1", + "provider_model": "grok-build-0.1", + "display_name": "Grok Build 0.1", + "provider_display_name": "xAI", + "description": "SpaceXAI's intelligent coding model for agentic software, engineering, and workflow tasks.", + "status": "active", + "listing": { + "order": 37, + "featured": false, + "snapshot": false + }, + "limits": { + "context_window": 256000, + "max_input_tokens": null, + "max_output_tokens": 256000 + }, + "capabilities": { + "modalities": { + "input": [ + "", + "" + ], + "output": [ + "text" + ] + }, + "tools": { + "supported": true, + "parallel": null + }, + "structured_outputs": true, + "temperature": true, + "prompt_caching": true, + "reasoning": { + "supported": true, + "efforts": [], + "summary": { + "output_supported": true, + "request_parameter_supported": false, + "values": [] + }, + "continuation": { + "request_include": [], + "required": false + } + }, + "verbosity": { + "supported": false, + "values": [] + } + }, + "pricing": { + "input": "", + "output": "2", + "cache_read_input": "0.2", + "cache_write_input": "0", + "cache_write_input_5m": "0", + "cache_write_input_1h": "0" + } + } + }, + { + "id": "claude-opus-4-7", + "object": "model", + "created": 1776124800, + "owned_by": "anthropic", + "display_name": "Claude Opus 4.7", + "context_length": 1000000, + "router": { + "schema_version": 1, + "request_name": "claude-opus-4-7", + "catalog_id": "anthropic:claude-opus-4-7", + "provider_model": "claude-opus-4-7", + "display_name": "Claude Opus 4.7", + "provider_display_name": "Anthropic", + "status": "active", + "listing": { + "order": 38, + "featured": false, + "snapshot": false + }, + "limits": { + "context_window": 1000000, + "max_input_tokens": 1000000, + "max_output_tokens": 128000 + }, + "capabilities": { + "modalities": { + "input": [ + "", + "" + ], + "output": [ + "text" + ] + }, + "tools": { + "supported": true, + "parallel": null + }, + "structured_outputs": true, + "temperature": true, + "prompt_caching": true, + "reasoning": { + "supported": true, + "efforts": [ + { + "value": "none", + "description": "Answers without reasoning first" + }, + { + "value": "minimal", + "description": "Reasons briefly before answering" + }, + { + "value": "low", + "description": "Fast responses with lighter reasoning" + }, + { + "value": "medium", + "description": "Balances speed and reasoning depth" + }, + { + "value": "high", + "description": "Greater reasoning depth" + }, + { + "value": "xhigh", + "description": "Reasons longer than high" + }, + { + "value": "max", + "description": "Reasons as long as the model allows" + } + ], + "default_effort": "none", + "summary": { + "output_supported": true, + "request_parameter_supported": false, + "values": [] + }, + "continuation": { + "request_include": [], + "required": false + } + }, + "verbosity": { + "supported": false, + "values": [] + } + }, + "pricing": { + "input": "", + "output": "25", + "cache_read_input": "0.5", + "cache_write_input": "0", + "cache_write_input_5m": "6.25", + "cache_write_input_1h": "10" + } + } + }, + { + "id": "gpt-5.4-mini", + "object": "model", + "created": 1773451123, + "owned_by": "openai", + "display_name": "GPT-5.4 mini", + "context_length": 400000, + "router": { + "schema_version": 1, + "request_name": "gpt-5.4-mini", + "catalog_id": "openai:gpt-5.4-mini", + "provider_model": "gpt-5.4-mini", + "display_name": "GPT-5.4 mini", + "provider_display_name": "OpenAI", + "aliases": [ + "fast-default" + ], + "description": "GPT-5.4 mini brings the strengths of GPT-5.4 to a faster, more efficient model designed for high-volume workloads. Learn more in our Model guidance page. Reasoning.effort supports: none (default), low, medium, high and xhigh.", + "status": "active", + "listing": { + "order": 40, + "featured": false, + "snapshot": false + }, + "limits": { + "context_window": 400000, + "max_input_tokens": null, + "max_output_tokens": 128000 + }, + "capabilities": { + "modalities": { + "input": [ + "", + "" + ], + "output": [ + "text" + ] + }, + "tools": { + "supported": true, + "parallel": null + }, + "structured_outputs": true, + "temperature": true, + "prompt_caching": true, + "reasoning": { + "supported": true, + "efforts": [ + { + "value": "none", + "description": "Answers without reasoning first" + }, + { + "value": "low", + "description": "Fast responses with lighter reasoning" + }, + { + "value": "medium", + "description": "Balances speed and reasoning depth" + }, + { + "value": "high", + "description": "Greater reasoning depth" + }, + { + "value": "xhigh", + "description": "Reasons longer than high" + } + ], + "default_effort": "medium", + "summary": { + "output_supported": true, + "request_parameter_supported": true, + "values": [ + "auto", + "concise", + "detailed" + ], + "default": "auto" + }, + "continuation": { + "request_include": [ + "reasoning.encrypted_content" + ], + "required": true + } + }, + "verbosity": { + "supported": true, + "values": [ + "low", + "medium", + "high" + ], + "default": "medium" + } + }, + "pricing": { + "input": "", + "output": "4.5", + "cache_read_input": "0.075", + "cache_write_input": "0", + "cache_write_input_5m": "0", + "cache_write_input_1h": "0" + } + } + }, + { + "id": "gpt-5.4-nano", + "object": "model", + "created": 1773450870, + "owned_by": "openai", + "display_name": "GPT-5.4 nano", + "context_length": 400000, + "router": { + "schema_version": 1, + "request_name": "gpt-5.4-nano", + "catalog_id": "openai:gpt-5.4-nano", + "provider_model": "gpt-5.4-nano", + "display_name": "GPT-5.4 nano", + "provider_display_name": "OpenAI", + "description": "GPT-5.4 nano is designed for tasks where speed and cost matter most like classification, data extraction, ranking, and sub-agents. Learn more in our Model guidance page. Reasoning.effort supports: none (default), low, medium, high and xhigh.", + "status": "active", + "listing": { + "order": 43, + "featured": false, + "snapshot": false + }, + "limits": { + "context_window": 400000, + "max_input_tokens": null, + "max_output_tokens": 128000 + }, + "capabilities": { + "modalities": { + "input": [ + "", + "" + ], + "output": [ + "text" + ] + }, + "tools": { + "supported": true, + "parallel": null + }, + "structured_outputs": true, + "temperature": true, + "prompt_caching": true, + "reasoning": { + "supported": true, + "efforts": [ + { + "value": "none", + "description": "Answers without reasoning first" + }, + { + "value": "low", + "description": "Fast responses with lighter reasoning" + }, + { + "value": "medium", + "description": "Balances speed and reasoning depth" + }, + { + "value": "high", + "description": "Greater reasoning depth" + }, + { + "value": "xhigh", + "description": "Reasons longer than high" + } + ], + "default_effort": "medium", + "summary": { + "output_supported": true, + "request_parameter_supported": true, + "values": [ + "auto", + "concise", + "detailed" + ], + "default": "auto" + }, + "continuation": { + "request_include": [ + "reasoning.encrypted_content" + ], + "required": true + } + }, + "verbosity": { + "supported": true, + "values": [ + "low", + "medium", + "high" + ], + "default": "medium" + } + }, + "pricing": { + "input": "", + "output": "1.25", + "cache_read_input": "0.02", + "cache_write_input": "0", + "cache_write_input_5m": "0", + "cache_write_input_1h": "0" + } + } + }, + { + "id": "grok-4.20-0309-reasoning", + "object": "model", + "created": 1773014400, + "owned_by": "xai", + "display_name": "Grok 4.20", + "context_length": 1000000, + "router": { + "schema_version": 1, + "request_name": "grok-4.20-0309-reasoning", + "catalog_id": "xai:grok-4.20-0309-reasoning", + "provider_model": "grok-4.20-0309-reasoning", + "display_name": "Grok 4.20", + "provider_display_name": "xAI", + "description": "Grok 4.20 is a high-performance model with industry-leading speed and agentic tool calling capabilities. It combines the lowest hallucination rate on the market with strict prompt adherence, delivering consistently precise and truthful responses.", + "status": "active", + "listing": { + "order": 45, + "featured": false, + "snapshot": false + }, + "limits": { + "context_window": 1000000, + "max_input_tokens": null, + "max_output_tokens": 1000000 + }, + "capabilities": { + "modalities": { + "input": [ + "", + "" + ], + "output": [ + "text" + ] + }, + "tools": { + "supported": true, + "parallel": null + }, + "structured_outputs": true, + "temperature": true, + "prompt_caching": true, + "reasoning": { + "supported": true, + "efforts": [], + "summary": { + "output_supported": true, + "request_parameter_supported": false, + "values": [] + }, + "continuation": { + "request_include": [], + "required": false + } + }, + "verbosity": { + "supported": false, + "values": [] + } + }, + "pricing": { + "input": "", + "output": "2.5", + "cache_read_input": "0.2", + "cache_write_input": "0", + "cache_write_input_5m": "0", + "cache_write_input_1h": "0" + } + } + }, + { + "id": "grok-4.20-0309-non-reasoning", + "object": "model", + "created": 1773014400, + "owned_by": "xai", + "display_name": "Grok 4.20 (Non-Reasoning)", + "context_length": 1000000, + "router": { + "schema_version": 1, + "request_name": "grok-4.20-0309-non-reasoning", + "catalog_id": "xai:grok-4.20-0309-non-reasoning", + "provider_model": "grok-4.20-0309-non-reasoning", + "display_name": "Grok 4.20 (Non-Reasoning)", + "provider_display_name": "xAI", + "description": "Grok 4.20 is a high-performance model with industry-leading speed and agentic tool calling capabilities. It combines the lowest hallucination rate on the market with strict prompt adherence, delivering consistently precise and truthful responses.", + "status": "active", + "listing": { + "order": 46, + "featured": false, + "snapshot": false + }, + "limits": { + "context_window": 1000000, + "max_input_tokens": null, + "max_output_tokens": 1000000 + }, + "capabilities": { + "modalities": { + "input": [ + "", + "" + ], + "output": [ + "text" + ] + }, + "tools": { + "supported": true, + "parallel": null + }, + "structured_outputs": true, + "temperature": true, + "prompt_caching": true, + "reasoning": { + "supported": false, + "efforts": [], + "summary": { + "output_supported": null, + "request_parameter_supported": null, + "values": [] + }, + "continuation": { + "request_include": [], + "required": null + } + }, + "verbosity": { + "supported": false, + "values": [] + } + }, + "pricing": { + "input": "", + "output": "2.5", + "cache_read_input": "0.2", + "cache_write_input": "0", + "cache_write_input_5m": "0", + "cache_write_input_1h": "0" + } + } + }, + { + "id": "grok-4.20-multi-agent-0309", + "object": "model", + "created": 1773014400, + "owned_by": "xai", + "display_name": "Grok 4.20 Multi-Agent Beta", + "context_length": 1000000, + "router": { + "schema_version": 1, + "request_name": "grok-4.20-multi-agent-0309", + "catalog_id": "xai:grok-4.20-multi-agent-0309", + "provider_model": "grok-4.20-multi-agent-0309", + "display_name": "Grok 4.20 Multi-Agent Beta", + "provider_display_name": "xAI", + "description": "Multiple agents collaborate in parallel to perform deep research tasks.", + "status": "active", + "listing": { + "order": 47, + "featured": false, + "snapshot": false + }, + "limits": { + "context_window": 1000000, + "max_input_tokens": null, + "max_output_tokens": 1000000 + }, + "capabilities": { + "modalities": { + "input": [ + "", + "" + ], + "output": [ + "text" + ] + }, + "tools": { + "supported": true, + "parallel": null + }, + "structured_outputs": true, + "temperature": true, + "prompt_caching": true, + "reasoning": { + "supported": true, + "efforts": [ + { + "value": "none", + "description": "Answers without reasoning first" + }, + { + "value": "minimal", + "description": "Reasons briefly before answering" + }, + { + "value": "low", + "description": "Fast responses with lighter reasoning" + }, + { + "value": "medium", + "description": "Balances speed and reasoning depth" + }, + { + "value": "high", + "description": "Greater reasoning depth" + }, + { + "value": "xhigh", + "description": "Reasons longer than high" + }, + { + "value": "max", + "description": "Reasons as long as the model allows" + } + ], + "default_effort": "none", + "summary": { + "output_supported": true, + "request_parameter_supported": false, + "values": [] + }, + "continuation": { + "request_include": [], + "required": false + } + }, + "verbosity": { + "supported": false, + "values": [] + } + }, + "pricing": { + "input": "", + "output": "2.5", + "cache_read_input": "0.2", + "cache_write_input": "0", + "cache_write_input_5m": "0", + "cache_write_input_1h": "0" + } + } + }, + { + "id": "gpt-5.4", + "object": "model", + "created": 1772691852, + "owned_by": "openai", + "display_name": "GPT-5.4", + "context_length": 1050000, + "router": { + "schema_version": 1, + "request_name": "gpt-5.4", + "catalog_id": "openai:gpt-5.4", + "provider_model": "gpt-5.4", + "display_name": "GPT-5.4", + "provider_display_name": "OpenAI", + "description": "GPT-5.4 is our frontier model for complex professional work. Learn more in our GPT-5.4 model guidance . Reasoning.effort supports: none (default), low, medium, high and xhigh.", + "status": "active", + "listing": { + "order": 48, + "featured": false, + "snapshot": false + }, + "limits": { + "context_window": 1050000, + "max_input_tokens": null, + "max_output_tokens": 128000 + }, + "capabilities": { + "modalities": { + "input": [ + "", + "" + ], + "output": [ + "text" + ] + }, + "tools": { + "supported": true, + "parallel": null + }, + "structured_outputs": true, + "temperature": true, + "prompt_caching": true, + "reasoning": { + "supported": true, + "efforts": [ + { + "value": "none", + "description": "Answers without reasoning first" + }, + { + "value": "low", + "description": "Fast responses with lighter reasoning" + }, + { + "value": "medium", + "description": "Balances speed and reasoning depth" + }, + { + "value": "high", + "description": "Greater reasoning depth" + }, + { + "value": "xhigh", + "description": "Reasons longer than high" + } + ], + "default_effort": "medium", + "summary": { + "output_supported": true, + "request_parameter_supported": true, + "values": [ + "auto", + "concise", + "detailed" + ], + "default": "auto" + }, + "continuation": { + "request_include": [ + "reasoning.encrypted_content" + ], + "required": true + } + }, + "verbosity": { + "supported": true, + "values": [ + "low", + "medium", + "high" + ], + "default": "medium" + } + }, + "pricing": { + "input": "", + "output": "15", + "cache_read_input": "0.25", + "cache_write_input": "0", + "cache_write_input_5m": "0", + "cache_write_input_1h": "0" + } + } + }, + { + "id": "gpt-5.4-pro", + "object": "model", + "created": 1772659601, + "owned_by": "openai", + "display_name": "GPT-5.4 Pro", + "context_length": 1050000, + "router": { + "schema_version": 1, + "request_name": "gpt-5.4-pro", + "catalog_id": "openai:gpt-5.4-pro", + "provider_model": "gpt-5.4-pro", + "display_name": "GPT-5.4 Pro", + "provider_display_name": "OpenAI", + "description": "GPT-5.4 Pro uses more compute to think harder and provide consistently better answers.", + "status": "active", + "listing": { + "order": 50, + "featured": false, + "snapshot": false + }, + "limits": { + "context_window": 1050000, + "max_input_tokens": null, + "max_output_tokens": 128000 + }, + "capabilities": { + "modalities": { + "input": [ + "", + "" + ], + "output": [ + "text" + ] + }, + "tools": { + "supported": true, + "parallel": null + }, + "structured_outputs": false, + "temperature": false, + "prompt_caching": false, + "reasoning": { + "supported": true, + "efforts": [ + { + "value": "medium", + "description": "Balances speed and reasoning depth" + }, + { + "value": "high", + "description": "Greater reasoning depth" + }, + { + "value": "xhigh", + "description": "Reasons longer than high" + } + ], + "default_effort": "medium", + "summary": { + "output_supported": true, + "request_parameter_supported": true, + "values": [ + "auto", + "concise", + "detailed" + ], + "default": "auto" + }, + "continuation": { + "request_include": [ + "reasoning.encrypted_content" + ], + "required": true + } + }, + "verbosity": { + "supported": true, + "values": [ + "low", + "medium", + "high" + ], + "default": "medium" + } + }, + "pricing": { + "input": "", + "output": "180", + "cache_read_input": "0", + "cache_write_input": "0", + "cache_write_input_5m": "0", + "cache_write_input_1h": "0" + } + } + }, + { + "id": "claude-sonnet-4-6", + "object": "model", + "created": 1771286400, + "owned_by": "anthropic", + "display_name": "Claude Sonnet 4.6", + "context_length": 1000000, + "router": { + "schema_version": 1, + "request_name": "claude-sonnet-4-6", + "catalog_id": "anthropic:claude-sonnet-4-6", + "provider_model": "claude-sonnet-4-6", + "display_name": "Claude Sonnet 4.6", + "provider_display_name": "Anthropic", + "aliases": [ + "sonnet-4.6" + ], + "status": "active", + "listing": { + "order": 52, + "featured": false, + "snapshot": false + }, + "limits": { + "context_window": 1000000, + "max_input_tokens": 1000000, + "max_output_tokens": 128000 + }, + "capabilities": { + "modalities": { + "input": [ + "", + "" + ], + "output": [ + "text" + ] + }, + "tools": { + "supported": true, + "parallel": null + }, + "structured_outputs": true, + "temperature": true, + "prompt_caching": true, + "reasoning": { + "supported": true, + "efforts": [ + { + "value": "none", + "description": "Answers without reasoning first" + }, + { + "value": "minimal", + "description": "Reasons briefly before answering" + }, + { + "value": "low", + "description": "Fast responses with lighter reasoning" + }, + { + "value": "medium", + "description": "Balances speed and reasoning depth" + }, + { + "value": "high", + "description": "Greater reasoning depth" + }, + { + "value": "xhigh", + "description": "Reasons longer than high" + }, + { + "value": "max", + "description": "Reasons as long as the model allows" + } + ], + "default_effort": "none", + "summary": { + "output_supported": true, + "request_parameter_supported": false, + "values": [] + }, + "continuation": { + "request_include": [], + "required": false + } + }, + "verbosity": { + "supported": false, + "values": [] + } + }, + "pricing": { + "input": "", + "output": "15", + "cache_read_input": "0.3", + "cache_write_input": "0", + "cache_write_input_5m": "3.75", + "cache_write_input_1h": "6" + } + } + }, + { + "id": "gpt-5.3-codex", + "object": "model", + "created": 1770537915, + "owned_by": "openai", + "display_name": "GPT-5.3-Codex", + "context_length": 400000, + "router": { + "schema_version": 1, + "request_name": "gpt-5.3-codex", + "catalog_id": "openai:gpt-5.3-codex", + "provider_model": "gpt-5.3-codex", + "display_name": "GPT-5.3-Codex", + "provider_display_name": "OpenAI", + "description": "GPT-5.3-Codex is optimized for agentic coding tasks in Codex or similar environments. GPT-5.3-Codex supports low , medium , high , and xhigh reasoning effort settings. If you want to learn more about prompting GPT-5.3-Codex, refer to our dedicated guide .", + "status": "active", + "listing": { + "order": 54, + "featured": false, + "snapshot": false + }, + "limits": { + "context_window": 400000, + "max_input_tokens": null, + "max_output_tokens": 128000 + }, + "capabilities": { + "modalities": { + "input": [ + "", + "" + ], + "output": [ + "text" + ] + }, + "tools": { + "supported": true, + "parallel": null + }, + "structured_outputs": true, + "temperature": true, + "prompt_caching": true, + "reasoning": { + "supported": true, + "efforts": [ + { + "value": "none", + "description": "Answers without reasoning first" + }, + { + "value": "low", + "description": "Fast responses with lighter reasoning" + }, + { + "value": "medium", + "description": "Balances speed and reasoning depth" + }, + { + "value": "high", + "description": "Greater reasoning depth" + }, + { + "value": "xhigh", + "description": "Reasons longer than high" + } + ], + "default_effort": "medium", + "summary": { + "output_supported": true, + "request_parameter_supported": true, + "values": [ + "auto", + "concise", + "detailed" + ], + "default": "auto" + }, + "continuation": { + "request_include": [ + "reasoning.encrypted_content" + ], + "required": true + } + }, + "verbosity": { + "supported": true, + "values": [ + "low", + "medium", + "high" + ], + "default": "medium" + } + }, + "pricing": { + "input": "", + "output": "14", + "cache_read_input": "0.175", + "cache_write_input": "0", + "cache_write_input_5m": "0", + "cache_write_input_1h": "0" + } + } + }, + { + "id": "claude-opus-4-6", + "object": "model", + "created": 1770163200, + "owned_by": "anthropic", + "display_name": "Claude Opus 4.6", + "context_length": 1000000, + "router": { + "schema_version": 1, + "request_name": "claude-opus-4-6", + "catalog_id": "anthropic:claude-opus-4-6", + "provider_model": "claude-opus-4-6", + "display_name": "Claude Opus 4.6", + "provider_display_name": "Anthropic", + "status": "active", + "listing": { + "order": 55, + "featured": false, + "snapshot": false + }, + "limits": { + "context_window": 1000000, + "max_input_tokens": 1000000, + "max_output_tokens": 128000 + }, + "capabilities": { + "modalities": { + "input": [ + "", + "" + ], + "output": [ + "text" + ] + }, + "tools": { + "supported": true, + "parallel": null + }, + "structured_outputs": true, + "temperature": true, + "prompt_caching": true, + "reasoning": { + "supported": true, + "efforts": [ + { + "value": "none", + "description": "Answers without reasoning first" + }, + { + "value": "minimal", + "description": "Reasons briefly before answering" + }, + { + "value": "low", + "description": "Fast responses with lighter reasoning" + }, + { + "value": "medium", + "description": "Balances speed and reasoning depth" + }, + { + "value": "high", + "description": "Greater reasoning depth" + }, + { + "value": "xhigh", + "description": "Reasons longer than high" + }, + { + "value": "max", + "description": "Reasons as long as the model allows" + } + ], + "default_effort": "none", + "summary": { + "output_supported": true, + "request_parameter_supported": false, + "values": [] + }, + "continuation": { + "request_include": [], + "required": false + } + }, + "verbosity": { + "supported": false, + "values": [] + } + }, + "pricing": { + "input": "", + "output": "25", + "cache_read_input": "0.5", + "cache_write_input": "0", + "cache_write_input_5m": "6.25", + "cache_write_input_1h": "10" + } + } + }, + { + "id": "gpt-5.2-pro", + "object": "model", + "created": 1765343983, + "owned_by": "openai", + "display_name": "GPT-5.2 Pro", + "context_length": 400000, + "router": { + "schema_version": 1, + "request_name": "gpt-5.2-pro", + "catalog_id": "openai:gpt-5.2-pro", + "provider_model": "gpt-5.2-pro", + "display_name": "GPT-5.2 Pro", + "provider_display_name": "OpenAI", + "description": "GPT-5.2 Pro is our previous pro model for complex professional work. We recommend using GPT-5.5 Pro for the latest pro model. GPT-5.2 Pro is available in the Responses API only to enable support for multi-turn model interactions before responding to API requests, and other advanced API features in the future. Since GPT-5.2 Pro is designed to tackle tough problems, some requests may take several minutes to finish. To avoid timeouts, try using background mode. GPT-5.2 Pro supports reasoning.effort: medium, high, xhigh.", + "status": "active", + "listing": { + "order": 56, + "featured": false, + "snapshot": false + }, + "limits": { + "context_window": 400000, + "max_input_tokens": null, + "max_output_tokens": 128000 + }, + "capabilities": { + "modalities": { + "input": [ + "", + "" + ], + "output": [ + "text" + ] + }, + "tools": { + "supported": true, + "parallel": null + }, + "structured_outputs": false, + "temperature": false, + "prompt_caching": false, + "reasoning": { + "supported": true, + "efforts": [ + { + "value": "medium", + "description": "Balances speed and reasoning depth" + }, + { + "value": "high", + "description": "Greater reasoning depth" + }, + { + "value": "xhigh", + "description": "Reasons longer than high" + } + ], + "default_effort": "medium", + "summary": { + "output_supported": true, + "request_parameter_supported": true, + "values": [ + "auto", + "concise", + "detailed" + ], + "default": "auto" + }, + "continuation": { + "request_include": [ + "reasoning.encrypted_content" + ], + "required": true + } + }, + "verbosity": { + "supported": true, + "values": [ + "low", + "medium", + "high" + ], + "default": "medium" + } + }, + "pricing": { + "input": "", + "output": "168", + "cache_read_input": "0", + "cache_write_input": "0", + "cache_write_input_5m": "0", + "cache_write_input_1h": "0" + } + } + }, + { + "id": "gpt-5.2", + "object": "model", + "created": 1765313051, + "owned_by": "openai", + "display_name": "GPT-5.2", + "context_length": 400000, + "router": { + "schema_version": 1, + "request_name": "gpt-5.2", + "catalog_id": "openai:gpt-5.2", + "provider_model": "gpt-5.2", + "display_name": "GPT-5.2", + "provider_display_name": "OpenAI", + "description": "GPT-5.2 is our previous frontier model for complex professional work. We recommend using the latest GPT-5.6 . Learn more on the Model guidance page. Reasoning.effort supports: none (default), low, medium, high and xhigh.", + "status": "active", + "listing": { + "order": 58, + "featured": false, + "snapshot": false + }, + "limits": { + "context_window": 400000, + "max_input_tokens": null, + "max_output_tokens": 128000 + }, + "capabilities": { + "modalities": { + "input": [ + "", + "" + ], + "output": [ + "text" + ] + }, + "tools": { + "supported": true, + "parallel": null + }, + "structured_outputs": true, + "temperature": true, + "prompt_caching": true, + "reasoning": { + "supported": true, + "efforts": [ + { + "value": "none", + "description": "Answers without reasoning first" + }, + { + "value": "low", + "description": "Fast responses with lighter reasoning" + }, + { + "value": "medium", + "description": "Balances speed and reasoning depth" + }, + { + "value": "high", + "description": "Greater reasoning depth" + }, + { + "value": "xhigh", + "description": "Reasons longer than high" + } + ], + "default_effort": "medium", + "summary": { + "output_supported": true, + "request_parameter_supported": true, + "values": [ + "auto", + "concise", + "detailed" + ], + "default": "auto" + }, + "continuation": { + "request_include": [ + "reasoning.encrypted_content" + ], + "required": true + } + }, + "verbosity": { + "supported": true, + "values": [ + "low", + "medium", + "high" + ], + "default": "medium" + } + }, + "pricing": { + "input": "", + "output": "14", + "cache_read_input": "0.175", + "cache_write_input": "0", + "cache_write_input_5m": "0", + "cache_write_input_1h": "0" + } + } + }, + { + "id": "gpt-5.1", + "object": "model", + "created": 1762800673, + "owned_by": "openai", + "display_name": "GPT-5.1", + "context_length": 400000, + "router": { + "schema_version": 1, + "request_name": "gpt-5.1", + "catalog_id": "openai:gpt-5.1", + "provider_model": "gpt-5.1", + "display_name": "GPT-5.1", + "provider_display_name": "OpenAI", + "description": "GPT-5.1 is our flagship model for coding and agentic tasks with configurable reasoning and non-reasoning effort. Learn more in our GPT-5.1 model guidance . Reasoning.effort supports: none (default), low, medium, and high.", + "status": "active", + "listing": { + "order": 61, + "featured": false, + "snapshot": false + }, + "limits": { + "context_window": 400000, + "max_input_tokens": null, + "max_output_tokens": 128000 + }, + "capabilities": { + "modalities": { + "input": [ + "", + "" + ], + "output": [ + "text" + ] + }, + "tools": { + "supported": true, + "parallel": null + }, + "structured_outputs": true, + "temperature": true, + "prompt_caching": true, + "reasoning": { + "supported": true, + "efforts": [ + { + "value": "none", + "description": "Answers without reasoning first" + }, + { + "value": "low", + "description": "Fast responses with lighter reasoning" + }, + { + "value": "medium", + "description": "Balances speed and reasoning depth" + }, + { + "value": "high", + "description": "Greater reasoning depth" + } + ], + "default_effort": "none", + "summary": { + "output_supported": true, + "request_parameter_supported": true, + "values": [ + "auto", + "concise", + "detailed" + ], + "default": "auto" + }, + "continuation": { + "request_include": [ + "reasoning.encrypted_content" + ], + "required": true + } + }, + "verbosity": { + "supported": true, + "values": [ + "low", + "medium", + "high" + ], + "default": "medium" + } + }, + "pricing": { + "input": "", + "output": "10", + "cache_read_input": "0.125", + "cache_write_input": "0", + "cache_write_input_5m": "0", + "cache_write_input_1h": "0" + } + } + }, + { + "id": "gpt-5-pro", + "object": "model", + "created": 1759469822, + "owned_by": "openai", + "display_name": "GPT-5 Pro", + "context_length": 400000, + "router": { + "schema_version": 1, + "request_name": "gpt-5-pro", + "catalog_id": "openai:gpt-5-pro", + "provider_model": "gpt-5-pro", + "display_name": "GPT-5 Pro", + "provider_display_name": "OpenAI", + "description": "GPT-5 Pro uses more compute to think harder and provide consistently better answers.", + "status": "active", + "listing": { + "order": 65, + "featured": false, + "snapshot": false + }, + "limits": { + "context_window": 400000, + "max_input_tokens": null, + "max_output_tokens": 272000 + }, + "capabilities": { + "modalities": { + "input": [ + "", + "" + ], + "output": [ + "text" + ] + }, + "tools": { + "supported": true, + "parallel": null + }, + "structured_outputs": true, + "temperature": false, + "prompt_caching": false, + "reasoning": { + "supported": true, + "efforts": [ + { + "value": "high", + "description": "Greater reasoning depth" + } + ], + "default_effort": "high", + "summary": { + "output_supported": true, + "request_parameter_supported": true, + "values": [ + "auto", + "concise", + "detailed" + ], + "default": "auto" + }, + "continuation": { + "request_include": [ + "reasoning.encrypted_content" + ], + "required": true + } + }, + "verbosity": { + "supported": true, + "values": [ + "low", + "medium", + "high" + ], + "default": "medium" + } + }, + "pricing": { + "input": "", + "output": "120", + "cache_read_input": "0", + "cache_write_input": "0", + "cache_write_input_5m": "0", + "cache_write_input_1h": "0" + } + } + }, + { + "id": "gpt-5-nano", + "object": "model", + "created": 1754426384, + "owned_by": "openai", + "display_name": "GPT-5 nano", + "context_length": 400000, + "router": { + "schema_version": 1, + "request_name": "gpt-5-nano", + "catalog_id": "openai:gpt-5-nano", + "provider_model": "gpt-5-nano", + "display_name": "GPT-5 nano", + "provider_display_name": "OpenAI", + "description": "GPT-5 Nano is our fastest, cheapest version of GPT-5. It's great for summarization and classification tasks. For most new speed- and cost-sensitive workloads, we recommend starting with GPT-5.6 Luna . Learn more in our Model guidance page.", + "status": "active", + "listing": { + "order": 69, + "featured": false, + "snapshot": false + }, + "limits": { + "context_window": 400000, + "max_input_tokens": null, + "max_output_tokens": 128000 + }, + "capabilities": { + "modalities": { + "input": [ + "", + "" + ], + "output": [ + "text" + ] + }, + "tools": { + "supported": true, + "parallel": null + }, + "structured_outputs": true, + "temperature": false, + "prompt_caching": true, + "reasoning": { + "supported": true, + "efforts": [ + { + "value": "minimal", + "description": "Reasons briefly before answering" + }, + { + "value": "low", + "description": "Fast responses with lighter reasoning" + }, + { + "value": "medium", + "description": "Balances speed and reasoning depth" + }, + { + "value": "high", + "description": "Greater reasoning depth" + } + ], + "default_effort": "medium", + "summary": { + "output_supported": true, + "request_parameter_supported": true, + "values": [ + "auto", + "concise", + "detailed" + ], + "default": "auto" + }, + "continuation": { + "request_include": [ + "reasoning.encrypted_content" + ], + "required": true + } + }, + "verbosity": { + "supported": true, + "values": [ + "low", + "medium", + "high" + ], + "default": "medium" + } + }, + "pricing": { + "input": "", + "output": "0.4", + "cache_read_input": "0.005", + "cache_write_input": "0", + "cache_write_input_5m": "0", + "cache_write_input_1h": "0" + } + } + }, + { + "id": "gpt-5-mini", + "object": "model", + "created": 1754425928, + "owned_by": "openai", + "display_name": "GPT-5 mini", + "context_length": 400000, + "router": { + "schema_version": 1, + "request_name": "gpt-5-mini", + "catalog_id": "openai:gpt-5-mini", + "provider_model": "gpt-5-mini", + "display_name": "GPT-5 mini", + "provider_display_name": "OpenAI", + "description": "GPT-5 mini is a faster, more cost-efficient version of GPT-5. It's great for well-defined tasks and precise prompts. For most new low-latency, high-volume workloads, we recommend starting with GPT-5.6 Terra .", + "status": "active", + "listing": { + "order": 71, + "featured": false, + "snapshot": false + }, + "limits": { + "context_window": 400000, + "max_input_tokens": null, + "max_output_tokens": 128000 + }, + "capabilities": { + "modalities": { + "input": [ + "", + "" + ], + "output": [ + "text" + ] + }, + "tools": { + "supported": true, + "parallel": null + }, + "structured_outputs": true, + "temperature": false, + "prompt_caching": true, + "reasoning": { + "supported": true, + "efforts": [ + { + "value": "minimal", + "description": "Reasons briefly before answering" + }, + { + "value": "low", + "description": "Fast responses with lighter reasoning" + }, + { + "value": "medium", + "description": "Balances speed and reasoning depth" + }, + { + "value": "high", + "description": "Greater reasoning depth" + } + ], + "default_effort": "medium", + "summary": { + "output_supported": true, + "request_parameter_supported": true, + "values": [ + "auto", + "concise", + "detailed" + ], + "default": "auto" + }, + "continuation": { + "request_include": [ + "reasoning.encrypted_content" + ], + "required": true + } + }, + "verbosity": { + "supported": true, + "values": [ + "low", + "medium", + "high" + ], + "default": "medium" + } + }, + "pricing": { + "input": "", + "output": "2", + "cache_read_input": "0.025", + "cache_write_input": "0", + "cache_write_input_5m": "0", + "cache_write_input_1h": "0" + } + } + }, + { + "id": "gpt-5", + "object": "model", + "created": 1754425777, + "owned_by": "openai", + "display_name": "GPT-5", + "context_length": 400000, + "router": { + "schema_version": 1, + "request_name": "gpt-5", + "catalog_id": "openai:gpt-5", + "provider_model": "gpt-5", + "display_name": "GPT-5", + "provider_display_name": "OpenAI", + "description": "GPT-5 is our previous model for coding, reasoning, and agentic tasks across domains. We recommend using the latest GPT-5.6 . Learn more on the Model guidance page. Reasoning.effort supports: minimal, low, medium, and high.", + "status": "active", + "listing": { + "order": 73, + "featured": false, + "snapshot": false + }, + "limits": { + "context_window": 400000, + "max_input_tokens": null, + "max_output_tokens": 128000 + }, + "capabilities": { + "modalities": { + "input": [ + "", + "" + ], + "output": [ + "text" + ] + }, + "tools": { + "supported": true, + "parallel": null + }, + "structured_outputs": true, + "temperature": false, + "prompt_caching": true, + "reasoning": { + "supported": true, + "efforts": [ + { + "value": "minimal", + "description": "Reasons briefly before answering" + }, + { + "value": "low", + "description": "Fast responses with lighter reasoning" + }, + { + "value": "medium", + "description": "Balances speed and reasoning depth" + }, + { + "value": "high", + "description": "Greater reasoning depth" + } + ], + "default_effort": "medium", + "summary": { + "output_supported": true, + "request_parameter_supported": true, + "values": [ + "auto", + "concise", + "detailed" + ], + "default": "auto" + }, + "continuation": { + "request_include": [ + "reasoning.encrypted_content" + ], + "required": true + } + }, + "verbosity": { + "supported": true, + "values": [ + "low", + "medium", + "high" + ], + "default": "medium" + } + }, + "pricing": { + "input": "", + "output": "10", + "cache_read_input": "0.125", + "cache_write_input": "0", + "cache_write_input_5m": "0", + "cache_write_input_1h": "0" + } + } + }, + { + "id": "accounts/fireworks/models/gpt-oss-120b", + "object": "model", + "created": 1754265600, + "owned_by": "fireworks", + "display_name": "OpenAI gpt-oss-120b", + "context_length": 131072, + "router": { + "schema_version": 1, + "request_name": "accounts/fireworks/models/gpt-oss-120b", + "catalog_id": "fireworks:accounts/fireworks/models/gpt-oss-120b", + "provider_model": "accounts/fireworks/models/gpt-oss-120b", + "display_name": "OpenAI gpt-oss-120b", + "provider_display_name": "Fireworks", + "description": "Welcome to the gpt-oss series, OpenAI's open-weight models designed for powerful reasoning, agentic tasks, and versatile developer use cases. gpt-oss-120b is used for production, general purpose, high reasoning use-cases that fits into a single H100 GPU.", + "status": "active", + "listing": { + "order": 74, + "featured": false, + "snapshot": false + }, + "limits": { + "context_window": 131072, + "max_input_tokens": null, + "max_output_tokens": 131072 + }, + "capabilities": { + "modalities": { + "input": [ + "" + ], + "output": [ + "text" + ] + }, + "tools": { + "supported": true, + "parallel": null + }, + "structured_outputs": true, + "temperature": true, + "prompt_caching": true, + "reasoning": { + "supported": true, + "efforts": [ + { + "value": "minimal", + "description": "Reasons briefly before answering" + }, + { + "value": "low", + "description": "Fast responses with lighter reasoning" + }, + { + "value": "medium", + "description": "Balances speed and reasoning depth" + }, + { + "value": "high", + "description": "Greater reasoning depth" + }, + { + "value": "xhigh", + "description": "Reasons longer than high" + } + ], + "default_effort": "medium", + "summary": { + "output_supported": true, + "request_parameter_supported": false, + "values": [] + }, + "continuation": { + "request_include": [], + "required": false + } + }, + "verbosity": { + "supported": false, + "values": [] + } + }, + "pricing": { + "input": "", + "output": "0.6", + "cache_read_input": "0.015", + "cache_write_input": "0", + "cache_write_input_5m": "0", + "cache_write_input_1h": "0" + } + } + }, + { + "id": "o3-pro", + "object": "model", + "created": 1748475349, + "owned_by": "openai", + "display_name": "o3-pro", + "context_length": 200000, + "router": { + "schema_version": 1, + "request_name": "o3-pro", + "catalog_id": "openai:o3-pro", + "provider_model": "o3-pro", + "display_name": "o3-pro", + "provider_display_name": "OpenAI", + "description": "The o-series of models are trained with reinforcement learning to think before they answer and perform complex reasoning. The o3-pro model uses more compute to think harder and provide consistently better answers.", + "status": "active", + "listing": { + "order": 81, + "featured": false, + "snapshot": false + }, + "limits": { + "context_window": 200000, + "max_input_tokens": null, + "max_output_tokens": 100000 + }, + "capabilities": { + "modalities": { + "input": [ + "", + "" + ], + "output": [ + "text" + ] + }, + "tools": { + "supported": true, + "parallel": null + }, + "structured_outputs": true, + "temperature": false, + "prompt_caching": false, + "reasoning": { + "supported": true, + "efforts": [ + { + "value": "low", + "description": "Fast responses with lighter reasoning" + }, + { + "value": "medium", + "description": "Balances speed and reasoning depth" + }, + { + "value": "high", + "description": "Greater reasoning depth" + } + ], + "default_effort": "medium", + "summary": { + "output_supported": true, + "request_parameter_supported": true, + "values": [ + "auto", + "concise", + "detailed" + ], + "default": "auto" + }, + "continuation": { + "request_include": [ + "reasoning.encrypted_content" + ], + "required": true + } + }, + "verbosity": { + "supported": false, + "values": [] + } + }, + "pricing": { + "input": "", + "output": "80", + "cache_read_input": "0", + "cache_write_input": "0", + "cache_write_input_5m": "0", + "cache_write_input_1h": "0" + } + } + }, + { + "id": "gpt-4.1-nano", + "object": "model", + "created": 1744321707, + "owned_by": "openai", + "display_name": "GPT-4.1 nano", + "context_length": 1047576, + "router": { + "schema_version": 1, + "request_name": "gpt-4.1-nano", + "catalog_id": "openai:gpt-4.1-nano", + "provider_model": "gpt-4.1-nano", + "display_name": "GPT-4.1 nano", + "provider_display_name": "OpenAI", + "description": "GPT-4.1 nano excels at instruction following and tool calling. It features a 1M token context window, and low latency without a reasoning step.", + "status": "deprecated", + "listing": { + "order": 82, + "featured": false, + "snapshot": false + }, + "limits": { + "context_window": 1047576, + "max_input_tokens": null, + "max_output_tokens": 32768 + }, + "capabilities": { + "modalities": { + "input": [ + "", + "" + ], + "output": [ + "text" + ] + }, + "tools": { + "supported": true, + "parallel": null + }, + "structured_outputs": true, + "temperature": true, + "prompt_caching": true, + "reasoning": { + "supported": false, + "efforts": [], + "summary": { + "output_supported": null, + "request_parameter_supported": null, + "values": [] + }, + "continuation": { + "request_include": [], + "required": null + } + }, + "verbosity": { + "supported": false, + "values": [] + } + }, + "pricing": { + "input": "", + "output": "0.4", + "cache_read_input": "0.025", + "cache_write_input": "0", + "cache_write_input_5m": "0", + "cache_write_input_1h": "0" + } + } + }, + { + "id": "gpt-4.1-mini", + "object": "model", + "created": 1744318173, + "owned_by": "openai", + "display_name": "GPT-4.1 mini", + "context_length": 1047576, + "router": { + "schema_version": 1, + "request_name": "gpt-4.1-mini", + "catalog_id": "openai:gpt-4.1-mini", + "provider_model": "gpt-4.1-mini", + "display_name": "GPT-4.1 mini", + "provider_display_name": "OpenAI", + "description": "GPT-4.1 mini excels at instruction following and tool calling. It features a 1M token context window, and low latency without a reasoning step.", + "status": "active", + "listing": { + "order": 84, + "featured": false, + "snapshot": false + }, + "limits": { + "context_window": 1047576, + "max_input_tokens": null, + "max_output_tokens": 32768 + }, + "capabilities": { + "modalities": { + "input": [ + "", + "" + ], + "output": [ + "text" + ] + }, + "tools": { + "supported": true, + "parallel": null + }, + "structured_outputs": true, + "temperature": true, + "prompt_caching": true, + "reasoning": { + "supported": false, + "efforts": [], + "summary": { + "output_supported": null, + "request_parameter_supported": null, + "values": [] + }, + "continuation": { + "request_include": [], + "required": null + } + }, + "verbosity": { + "supported": false, + "values": [] + } + }, + "pricing": { + "input": "", + "output": "1.6", + "cache_read_input": "0.1", + "cache_write_input": "0", + "cache_write_input_5m": "0", + "cache_write_input_1h": "0" + } + } + }, + { + "id": "gpt-4.1", + "object": "model", + "created": 1744316542, + "owned_by": "openai", + "display_name": "GPT-4.1", + "context_length": 1047576, + "router": { + "schema_version": 1, + "request_name": "gpt-4.1", + "catalog_id": "openai:gpt-4.1", + "provider_model": "gpt-4.1", + "display_name": "GPT-4.1", + "provider_display_name": "OpenAI", + "description": "GPT-4.1 excels at instruction following and tool calling, with broad knowledge across domains. It features a 1M token context window, and low latency without a reasoning step.", + "status": "active", + "listing": { + "order": 86, + "featured": false, + "snapshot": false + }, + "limits": { + "context_window": 1047576, + "max_input_tokens": null, + "max_output_tokens": 32768 + }, + "capabilities": { + "modalities": { + "input": [ + "", + "" + ], + "output": [ + "text" + ] + }, + "tools": { + "supported": true, + "parallel": null + }, + "structured_outputs": true, + "temperature": true, + "prompt_caching": true, + "reasoning": { + "supported": false, + "efforts": [], + "summary": { + "output_supported": null, + "request_parameter_supported": null, + "values": [] + }, + "continuation": { + "request_include": [], + "required": null + } + }, + "verbosity": { + "supported": false, + "values": [] + } + }, + "pricing": { + "input": "", + "output": "8", + "cache_read_input": "0.5", + "cache_write_input": "0", + "cache_write_input_5m": "0", + "cache_write_input_1h": "0" + } + } + }, + { + "id": "o4-mini", + "object": "model", + "created": 1744225351, + "owned_by": "openai", + "display_name": "o4-mini", + "context_length": 200000, + "router": { + "schema_version": 1, + "request_name": "o4-mini", + "catalog_id": "openai:o4-mini", + "provider_model": "o4-mini", + "display_name": "o4-mini", + "provider_display_name": "OpenAI", + "description": "o4-mini is our latest small o-series model. It's optimized for fast, effective reasoning with exceptionally efficient performance in coding and visual tasks. It's succeeded by GPT-5 mini .", + "status": "deprecated", + "listing": { + "order": 88, + "featured": false, + "snapshot": false + }, + "limits": { + "context_window": 200000, + "max_input_tokens": null, + "max_output_tokens": 100000 + }, + "capabilities": { + "modalities": { + "input": [ + "", + "" + ], + "output": [ + "text" + ] + }, + "tools": { + "supported": true, + "parallel": null + }, + "structured_outputs": true, + "temperature": false, + "prompt_caching": true, + "reasoning": { + "supported": true, + "efforts": [ + { + "value": "low", + "description": "Fast responses with lighter reasoning" + }, + { + "value": "medium", + "description": "Balances speed and reasoning depth" + }, + { + "value": "high", + "description": "Greater reasoning depth" + } + ], + "default_effort": "medium", + "summary": { + "output_supported": true, + "request_parameter_supported": true, + "values": [ + "auto", + "concise", + "detailed" + ], + "default": "auto" + }, + "continuation": { + "request_include": [ + "reasoning.encrypted_content" + ], + "required": true + } + }, + "verbosity": { + "supported": false, + "values": [] + } + }, + "pricing": { + "input": "", + "output": "4.4", + "cache_read_input": "0.275", + "cache_write_input": "0", + "cache_write_input_5m": "0", + "cache_write_input_1h": "0" + } + } + }, + { + "id": "o3", + "object": "model", + "created": 1744225308, + "owned_by": "openai", + "display_name": "o3", + "context_length": 200000, + "router": { + "schema_version": 1, + "request_name": "o3", + "catalog_id": "openai:o3", + "provider_model": "o3", + "display_name": "o3", + "provider_display_name": "OpenAI", + "description": "o3 is a well-rounded and powerful model across domains. It sets a new standard for math, science, coding, and visual reasoning tasks. It also excels at technical writing and instruction-following. Use it to think through multi-step problems that involve analysis across text, code, and images.", + "status": "active", + "listing": { + "order": 89, + "featured": false, + "snapshot": false + }, + "limits": { + "context_window": 200000, + "max_input_tokens": null, + "max_output_tokens": 100000 + }, + "capabilities": { + "modalities": { + "input": [ + "", + "" + ], + "output": [ + "text" + ] + }, + "tools": { + "supported": true, + "parallel": null + }, + "structured_outputs": true, + "temperature": false, + "prompt_caching": true, + "reasoning": { + "supported": true, + "efforts": [ + { + "value": "low", + "description": "Fast responses with lighter reasoning" + }, + { + "value": "medium", + "description": "Balances speed and reasoning depth" + }, + { + "value": "high", + "description": "Greater reasoning depth" + } + ], + "default_effort": "medium", + "summary": { + "output_supported": true, + "request_parameter_supported": true, + "values": [ + "auto", + "concise", + "detailed" + ], + "default": "auto" + }, + "continuation": { + "request_include": [ + "reasoning.encrypted_content" + ], + "required": true + } + }, + "verbosity": { + "supported": false, + "values": [] + } + }, + "pricing": { + "input": "", + "output": "8", + "cache_read_input": "0.5", + "cache_write_input": "0", + "cache_write_input_5m": "0", + "cache_write_input_1h": "0" + } + } + }, + { + "id": "o3-mini", + "object": "model", + "created": 1737146383, + "owned_by": "openai", + "display_name": "o3-mini", + "context_length": 200000, + "router": { + "schema_version": 1, + "request_name": "o3-mini", + "catalog_id": "openai:o3-mini", + "provider_model": "o3-mini", + "display_name": "o3-mini", + "provider_display_name": "OpenAI", + "description": "o3-mini is our newest small reasoning model, providing high intelligence at the same cost and latency targets of o1-mini. o3-mini supports key developer features, like Structured Outputs, function calling, and Batch API.", + "status": "deprecated", + "listing": { + "order": 94, + "featured": false, + "snapshot": false + }, + "limits": { + "context_window": 200000, + "max_input_tokens": null, + "max_output_tokens": 100000 + }, + "capabilities": { + "modalities": { + "input": [ + "" + ], + "output": [ + "text" + ] + }, + "tools": { + "supported": true, + "parallel": null + }, + "structured_outputs": true, + "temperature": false, + "prompt_caching": true, + "reasoning": { + "supported": true, + "efforts": [ + { + "value": "low", + "description": "Fast responses with lighter reasoning" + }, + { + "value": "medium", + "description": "Balances speed and reasoning depth" + }, + { + "value": "high", + "description": "Greater reasoning depth" + } + ], + "default_effort": "medium", + "summary": { + "output_supported": true, + "request_parameter_supported": true, + "values": [ + "auto", + "concise", + "detailed" + ], + "default": "auto" + }, + "continuation": { + "request_include": [ + "reasoning.encrypted_content" + ], + "required": true + } + }, + "verbosity": { + "supported": false, + "values": [] + } + }, + "pricing": { + "input": "", + "output": "4.4", + "cache_read_input": "0.55", + "cache_write_input": "0", + "cache_write_input_5m": "0", + "cache_write_input_1h": "0" + } + } + }, + { + "id": "o1", + "object": "model", + "created": 1734375816, + "owned_by": "openai", + "display_name": "o1", + "context_length": 200000, + "router": { + "schema_version": 1, + "request_name": "o1", + "catalog_id": "openai:o1", + "provider_model": "o1", + "display_name": "o1", + "provider_display_name": "OpenAI", + "description": "The o1 series of models are trained with reinforcement learning to perform complex reasoning. o1 models think before they answer, producing a long internal chain of thought before responding to the user.", + "status": "deprecated", + "listing": { + "order": 95, + "featured": false, + "snapshot": false + }, + "limits": { + "context_window": 200000, + "max_input_tokens": null, + "max_output_tokens": 100000 + }, + "capabilities": { + "modalities": { + "input": [ + "", + "" + ], + "output": [ + "text" + ] + }, + "tools": { + "supported": true, + "parallel": null + }, + "structured_outputs": true, + "temperature": false, + "prompt_caching": true, + "reasoning": { + "supported": true, + "efforts": [ + { + "value": "low", + "description": "Fast responses with lighter reasoning" + }, + { + "value": "medium", + "description": "Balances speed and reasoning depth" + }, + { + "value": "high", + "description": "Greater reasoning depth" + } + ], + "default_effort": "medium", + "summary": { + "output_supported": true, + "request_parameter_supported": true, + "values": [ + "auto", + "concise", + "detailed" + ], + "default": "auto" + }, + "continuation": { + "request_include": [ + "reasoning.encrypted_content" + ], + "required": true + } + }, + "verbosity": { + "supported": false, + "values": [] + } + }, + "pricing": { + "input": "", + "output": "60", + "cache_read_input": "7.5", + "cache_write_input": "0", + "cache_write_input_5m": "0", + "cache_write_input_1h": "0" + } + } + }, + { + "id": "gpt-4o-mini", + "object": "model", + "created": 1721172741, + "owned_by": "openai", + "display_name": "GPT-4o mini", + "context_length": 128000, + "router": { + "schema_version": 1, + "request_name": "gpt-4o-mini", + "catalog_id": "openai:gpt-4o-mini", + "provider_model": "gpt-4o-mini", + "display_name": "GPT-4o mini", + "provider_display_name": "OpenAI", + "description": "GPT-4o mini (“o” for “omni”) is a fast, affordable small model for focused tasks. It accepts both text and image inputs, and produces text outputs (including Structured Outputs). It is ideal for fine-tuning, and model outputs from a larger model like GPT-4o can be distilled to GPT-4o-mini to produce similar results at lower cost and latency.", + "status": "active", + "listing": { + "order": 98, + "featured": false, + "snapshot": false + }, + "limits": { + "context_window": 128000, + "max_input_tokens": null, + "max_output_tokens": 16384 + }, + "capabilities": { + "modalities": { + "input": [ + "", + "" + ], + "output": [ + "text" + ] + }, + "tools": { + "supported": true, + "parallel": null + }, + "structured_outputs": true, + "temperature": true, + "prompt_caching": true, + "reasoning": { + "supported": false, + "efforts": [], + "summary": { + "output_supported": null, + "request_parameter_supported": null, + "values": [] + }, + "continuation": { + "request_include": [], + "required": null + } + }, + "verbosity": { + "supported": false, + "values": [] + } + }, + "pricing": { + "input": "", + "output": "0.6", + "cache_read_input": "0.075", + "cache_write_input": "0", + "cache_write_input_5m": "0", + "cache_write_input_1h": "0" + } + } + }, + { + "id": "gpt-4o", + "object": "model", + "created": 1715367049, + "owned_by": "openai", + "display_name": "GPT-4o", + "context_length": 128000, + "router": { + "schema_version": 1, + "request_name": "gpt-4o", + "catalog_id": "openai:gpt-4o", + "provider_model": "gpt-4o", + "display_name": "GPT-4o", + "provider_display_name": "OpenAI", + "description": "GPT-4o (“o” for “omni”) is our versatile, high-intelligence flagship model. It accepts both text and image inputs, and produces text outputs (including Structured Outputs). It is the best model for most tasks, and is our most capable model outside of our o-series models.", + "status": "active", + "listing": { + "order": 100, + "featured": false, + "snapshot": false + }, + "limits": { + "context_window": 128000, + "max_input_tokens": null, + "max_output_tokens": 16384 + }, + "capabilities": { + "modalities": { + "input": [ + "", + "" + ], + "output": [ + "text" + ] + }, + "tools": { + "supported": true, + "parallel": null + }, + "structured_outputs": true, + "temperature": true, + "prompt_caching": true, + "reasoning": { + "supported": false, + "efforts": [], + "summary": { + "output_supported": null, + "request_parameter_supported": null, + "values": [] + }, + "continuation": { + "request_include": [], + "required": null + } + }, + "verbosity": { + "supported": false, + "values": [] + } + }, + "pricing": { + "input": "", + "output": "10", + "cache_read_input": "1.25", + "cache_write_input": "0", + "cache_write_input_5m": "0", + "cache_write_input_1h": "0" + } + } + }, + { + "id": "gpt-4-turbo", + "object": "model", + "created": 1712361441, + "owned_by": "openai", + "display_name": "GPT-4 Turbo", + "context_length": 128000, + "router": { + "schema_version": 1, + "request_name": "gpt-4-turbo", + "catalog_id": "openai:gpt-4-turbo", + "provider_model": "gpt-4-turbo", + "display_name": "GPT-4 Turbo", + "provider_display_name": "OpenAI", + "description": "GPT-4 Turbo is the next generation of GPT-4, an older high-intelligence GPT model. It was designed to be a cheaper, better version of GPT-4. Today, we recommend using a newer model like GPT-4o.", + "status": "deprecated", + "listing": { + "order": 101, + "featured": false, + "snapshot": false + }, + "limits": { + "context_window": 128000, + "max_input_tokens": null, + "max_output_tokens": 4096 + }, + "capabilities": { + "modalities": { + "input": [ + "", + "" + ], + "output": [ + "text" + ] + }, + "tools": { + "supported": true, + "parallel": null + }, + "structured_outputs": false, + "temperature": true, + "prompt_caching": false, + "reasoning": { + "supported": false, + "efforts": [], + "summary": { + "output_supported": null, + "request_parameter_supported": null, + "values": [] + }, + "continuation": { + "request_include": [], + "required": null + } + }, + "verbosity": { + "supported": false, + "values": [] + } + }, + "pricing": { + "input": "", + "output": "30", + "cache_read_input": "0", + "cache_write_input": "0", + "cache_write_input_5m": "0", + "cache_write_input_1h": "0" + } + } + }, + { + "id": "gpt-4", + "object": "model", + "created": 1687882411, + "owned_by": "openai", + "display_name": "GPT-4", + "context_length": 8192, + "router": { + "schema_version": 1, + "request_name": "gpt-4", + "catalog_id": "openai:gpt-4", + "provider_model": "gpt-4", + "display_name": "GPT-4", + "provider_display_name": "OpenAI", + "description": "GPT-4 is an older version of a high-intelligence GPT model, usable in Chat Completions.", + "status": "deprecated", + "listing": { + "order": 102, + "featured": false, + "snapshot": false + }, + "limits": { + "context_window": 8192, + "max_input_tokens": null, + "max_output_tokens": 8192 + }, + "capabilities": { + "modalities": { + "input": [ + "" + ], + "output": [ + "text" + ] + }, + "tools": { + "supported": false, + "parallel": false + }, + "structured_outputs": false, + "temperature": true, + "prompt_caching": false, + "reasoning": { + "supported": false, + "efforts": [], + "summary": { + "output_supported": null, + "request_parameter_supported": null, + "values": [] + }, + "continuation": { + "request_include": [], + "required": null + } + }, + "verbosity": { + "supported": false, + "values": [] + } + }, + "pricing": { + "input": "", + "output": "60", + "cache_read_input": "0", + "cache_write_input": "0", + "cache_write_input_5m": "0", + "cache_write_input_1h": "0" + } + } + }, + { + "id": "gpt-3.5-turbo", + "object": "model", + "created": 1677610602, + "owned_by": "openai", + "display_name": "GPT-3.5 Turbo", + "context_length": 16385, + "router": { + "schema_version": 1, + "request_name": "gpt-3.5-turbo", + "catalog_id": "openai:gpt-3.5-turbo", + "provider_model": "gpt-3.5-turbo", + "display_name": "GPT-3.5 Turbo", + "provider_display_name": "OpenAI", + "description": "GPT-3.5 Turbo models can understand and generate natural language or code and have been optimized for chat using the Chat Completions API but work well for non-chat tasks as well. As of July 2024, use gpt-4o-mini in place of GPT-3.5 Turbo, as it is cheaper, more capable, multimodal, and just as fast. GPT-3.5 Turbo is still available for use in the API.", + "status": "deprecated", + "listing": { + "order": 103, + "featured": false, + "snapshot": false + }, + "limits": { + "context_window": 16385, + "max_input_tokens": null, + "max_output_tokens": 4096 + }, + "capabilities": { + "modalities": { + "input": [ + "" + ], + "output": [ + "text" + ] + }, + "tools": { + "supported": false, + "parallel": false + }, + "structured_outputs": false, + "temperature": true, + "prompt_caching": false, + "reasoning": { + "supported": false, + "efforts": [], + "summary": { + "output_supported": null, + "request_parameter_supported": null, + "values": [] + }, + "continuation": { + "request_include": [], + "required": null + } + }, + "verbosity": { + "supported": false, + "values": [] + } + }, + "pricing": { + "input": "", + "output": "1.5", + "cache_read_input": "0", + "cache_write_input": "0", + "cache_write_input_5m": "0", + "cache_write_input_1h": "0" + } + } + }, + { + "id": "claude-haiku-4-5", + "object": "model", + "created": 0, + "owned_by": "anthropic", + "display_name": "Claude Haiku 4.5", + "context_length": 200000, + "router": { + "schema_version": 1, + "request_name": "claude-haiku-4-5", + "catalog_id": "anthropic:claude-haiku-4-5", + "provider_model": "claude-haiku-4-5", + "display_name": "Claude Haiku 4.5", + "provider_display_name": "Anthropic", + "description": "The fastest model with near-frontier intelligence", + "status": "active", + "listing": { + "order": 104, + "featured": false, + "snapshot": false + }, + "limits": { + "context_window": 200000, + "max_input_tokens": 200000, + "max_output_tokens": 64000 + }, + "capabilities": { + "modalities": { + "input": [ + "", + "" + ], + "output": [ + "text" + ] + }, + "tools": { + "supported": true, + "parallel": null + }, + "structured_outputs": true, + "temperature": true, + "prompt_caching": true, + "reasoning": { + "supported": true, + "efforts": [ + { + "value": "none", + "description": "Answers without reasoning first" + }, + { + "value": "minimal", + "description": "Reasons briefly before answering" + }, + { + "value": "low", + "description": "Fast responses with lighter reasoning" + }, + { + "value": "medium", + "description": "Balances speed and reasoning depth" + }, + { + "value": "high", + "description": "Greater reasoning depth" + }, + { + "value": "xhigh", + "description": "Reasons longer than high" + } + ], + "default_effort": "none", + "summary": { + "output_supported": true, + "request_parameter_supported": false, + "values": [] + }, + "continuation": { + "request_include": [], + "required": false + } + }, + "verbosity": { + "supported": false, + "values": [] + } + }, + "pricing": { + "input": "", + "output": "5", + "cache_read_input": "0.1", + "cache_write_input": "0", + "cache_write_input_5m": "1.25", + "cache_write_input_1h": "2" + } + } + } + ] + }, + "_body_was_json": true +} diff --git a/tests/unit/adapters/fixtures/ramp_router/02_real_buffered_plain_model.json b/tests/unit/adapters/fixtures/ramp_router/02_real_buffered_plain_model.json new file mode 100644 index 0000000..638d163 --- /dev/null +++ b/tests/unit/adapters/fixtures/ramp_router/02_real_buffered_plain_model.json @@ -0,0 +1,124 @@ +{ + "_probe": "P2", + "_question": "does response.model echo the requested id or the served candidate?", + "_captured": "real capture against a live Ramp Router account", + "_scrubbed": "credentials, emails, IPs, dashboard URLs; prompt and completion text blanked to \"\"", + "_requested_model": "gpt-5.4-nano", + "_status": 200, + "_headers": { + "alt-svc": "h3=\":443\"; ma=86400", + "cf-cache-status": "DYNAMIC", + "cf-ray": "a323832a6ad50956-EWR", + "connection": "keep-alive", + "content-encoding": "br", + "content-type": "application/json", + "date": "Fri, 28 Aug 2026 13:02:48 GMT", + "server": "cloudflare", + "strict-transport-security": "max-age=31536000; includeSubDomains; preload", + "transfer-encoding": "chunked", + "vary": "Origin", + "x-content-type-options": "nosniff", + "x-request-id": "c8767890-1450-4010-bbf1-1f545a3aca2a", + "x-trace-id": "c8767890-1450-4010-bbf1-1f545a3aca2a" + }, + "_body": { + "background": false, + "billing": { + "payer": "developer" + }, + "completed_at": 1787922168, + "conversation": null, + "created_at": 1787922167, + "error": null, + "frequency_penalty": 0, + "id": "resp_0e30162a5607e9f1006a9186f7c65c87d2a054e3b761db3b11", + "incomplete_details": null, + "instructions": null, + "max_output_tokens": 16, + "max_tool_calls": null, + "metadata": { + "lago_subscription": "rr_gateway_test_sub" + }, + "model": "gpt-5.4-nano-2026-03-17", + "moderation": null, + "object": "response", + "output": [ + { + "id": "msg_0e30162a5607e9f1006a9186f8850c87d2ad3e33844fd0a54a", + "type": "message", + "status": "completed", + "content": [ + { + "type": "output_text", + "annotations": [], + "logprobs": [], + "text": "" + } + ], + "phase": "final_answer", + "role": "assistant" + } + ], + "parallel_tool_calls": true, + "presence_penalty": 0, + "previous_response_id": null, + "prompt": null, + "prompt_cache_key": null, + "prompt_cache_retention": "in_memory", + "reasoning": { + "context": "current_turn", + "effort": "none", + "generate_summary": null, + "mode": "standard", + "summary": null + }, + "safety_identifier": null, + "service_tier": "flex", + "status": "completed", + "store": true, + "temperature": 1, + "text": { + "format": { + "type": "text" + }, + "verbosity": "medium" + }, + "tool_choice": "auto", + "tool_usage": { + "image_gen": { + "input_tokens": 0, + "input_tokens_details": { + "image_tokens": 0, + "text_tokens": 0 + }, + "output_tokens": 0, + "output_tokens_details": { + "image_tokens": 0, + "text_tokens": 0 + }, + "total_tokens": 0 + }, + "web_search": { + "num_requests": 0 + } + }, + "tools": [], + "top_logprobs": 0, + "top_p": 0.98, + "truncation": "disabled", + "usage": { + "input_tokens": 14, + "input_tokens_details": { + "cache_write_tokens": 0, + "cached_tokens": 0 + }, + "output_tokens": 5, + "output_tokens_details": { + "reasoning_tokens": 0 + }, + "total_tokens": 19 + }, + "user": null + }, + "_body_was_json": true +} diff --git a/tests/unit/adapters/fixtures/ramp_router/03_real_models_fallback.json b/tests/unit/adapters/fixtures/ramp_router/03_real_models_fallback.json new file mode 100644 index 0000000..8a340c1 --- /dev/null +++ b/tests/unit/adapters/fixtures/ramp_router/03_real_models_fallback.json @@ -0,0 +1,125 @@ +{ + "_captured": "real capture against a live Ramp Router account", + "_probe": "P3", + "_question": "under a candidate list, does response.model name the SERVED candidate?", + "_scrubbed": "credentials, emails, IPs, dashboard URLs; prompt and completion text blanked to \"\"", + "_requested_models": [ + "openai:gpt-5.4-nano", + "anthropic:claude-haiku-4-5" + ], + "_status": 200, + "_headers": { + "alt-svc": "h3=\":443\"; ma=86400", + "cf-cache-status": "DYNAMIC", + "cf-ray": "a32386327f65db6d-EWR", + "connection": "keep-alive", + "content-encoding": "br", + "content-type": "application/json", + "date": "Fri, 28 Aug 2026 13:04:52 GMT", + "server": "cloudflare", + "strict-transport-security": "max-age=31536000; includeSubDomains; preload", + "transfer-encoding": "chunked", + "vary": "Origin", + "x-content-type-options": "nosniff", + "x-request-id": "17354c41-9ae5-4542-8f84-dbd12b6a78e0", + "x-trace-id": "17354c41-9ae5-4542-8f84-dbd12b6a78e0" + }, + "_body_was_json": true, + "_body": { + "background": false, + "billing": { + "payer": "developer" + }, + "completed_at": 1787922292, + "conversation": null, + "created_at": 1787922291, + "error": null, + "frequency_penalty": 0, + "id": "resp_0bb9681439173034006a918773aec087d2b9c1a9cf507b30c7", + "incomplete_details": null, + "instructions": null, + "max_output_tokens": 16, + "max_tool_calls": null, + "metadata": {}, + "model": "gpt-5.4-nano-2026-03-17", + "moderation": null, + "object": "response", + "output": [ + { + "id": "msg_0bb9681439173034006a9187745df487d2a7a0f36fe8d28b18", + "type": "message", + "status": "completed", + "content": [ + { + "type": "output_text", + "annotations": [], + "logprobs": [], + "text": "" + } + ], + "phase": "final_answer", + "role": "assistant" + } + ], + "parallel_tool_calls": true, + "presence_penalty": 0, + "previous_response_id": null, + "prompt": null, + "prompt_cache_key": null, + "prompt_cache_retention": "in_memory", + "reasoning": { + "context": "current_turn", + "effort": "none", + "generate_summary": null, + "mode": "standard", + "summary": null + }, + "safety_identifier": null, + "service_tier": "default", + "status": "completed", + "store": true, + "temperature": 1, + "text": { + "format": { + "type": "text" + }, + "verbosity": "medium" + }, + "tool_choice": "auto", + "tool_usage": { + "image_gen": { + "input_tokens": 0, + "input_tokens_details": { + "image_tokens": 0, + "text_tokens": 0 + }, + "output_tokens": 0, + "output_tokens_details": { + "image_tokens": 0, + "text_tokens": 0 + }, + "total_tokens": 0 + }, + "web_search": { + "num_requests": 0 + } + }, + "tools": [], + "top_logprobs": 0, + "top_p": 0.98, + "truncation": "disabled", + "usage": { + "input_tokens": 11, + "input_tokens_details": { + "cache_write_tokens": 0, + "cached_tokens": 0 + }, + "output_tokens": 5, + "output_tokens_details": { + "reasoning_tokens": 0 + }, + "total_tokens": 16 + }, + "user": null + } +} diff --git a/tests/unit/adapters/fixtures/ramp_router/04_real_streamed.json b/tests/unit/adapters/fixtures/ramp_router/04_real_streamed.json new file mode 100644 index 0000000..28a74ff --- /dev/null +++ b/tests/unit/adapters/fixtures/ramp_router/04_real_streamed.json @@ -0,0 +1,336 @@ +{ + "_probe": "P4", + "_question": "do SSE events carry usage and the resolved model under .response?", + "_captured": "real capture against a live Ramp Router account", + "_scrubbed": "credentials, emails, IPs, dashboard URLs; prompt and completion text blanked to \"\"", + "_status": 200, + "_headers": { + "alt-svc": "h3=\":443\"; ma=86400", + "cache-control": "no-store", + "cf-cache-status": "DYNAMIC", + "cf-ray": "a3238335f95b7a99-EWR", + "connection": "keep-alive", + "content-type": "text/event-stream", + "date": "Fri, 28 Aug 2026 13:02:49 GMT", + "server": "cloudflare", + "strict-transport-security": "max-age=31536000; includeSubDomains; preload", + "transfer-encoding": "chunked", + "vary": "Origin", + "x-content-type-options": "nosniff", + "x-request-id": "afba1210-daab-47a7-8ecd-d2ae6e92104e", + "x-trace-id": "afba1210-daab-47a7-8ecd-d2ae6e92104e" + }, + "_body": null, + "_body_was_json": false, + "_events": [ + { + "type": "response.created", + "response": { + "id": "resp_0a220bea710bdc08006a9186f964c887d29a7213cbc4006c10", + "object": "response", + "created_at": 1787922169, + "status": "in_progress", + "background": false, + "completed_at": null, + "error": null, + "frequency_penalty": 0, + "incomplete_details": null, + "instructions": null, + "max_output_tokens": 16, + "max_tool_calls": null, + "model": "gpt-5.4-nano-2026-03-17", + "moderation": null, + "output": [], + "parallel_tool_calls": true, + "presence_penalty": 0, + "previous_response_id": null, + "prompt_cache_key": null, + "prompt_cache_retention": "in_memory", + "reasoning": { + "context": "current_turn", + "effort": "none", + "mode": "standard", + "summary": null + }, + "safety_identifier": null, + "service_tier": "flex", + "store": true, + "temperature": 1, + "text": { + "format": { + "type": "text" + }, + "verbosity": "medium" + }, + "tool_choice": "auto", + "tool_usage": { + "image_gen": { + "input_tokens": 0, + "input_tokens_details": { + "image_tokens": 0, + "text_tokens": 0 + }, + "output_tokens": 0, + "output_tokens_details": { + "image_tokens": 0, + "text_tokens": 0 + }, + "total_tokens": 0 + }, + "web_search": { + "num_requests": 0 + } + }, + "tools": [], + "top_logprobs": 0, + "top_p": 0.98, + "truncation": "disabled", + "usage": null, + "user": null, + "metadata": {} + }, + "sequence_number": 0 + }, + { + "type": "response.in_progress", + "response": { + "id": "resp_0a220bea710bdc08006a9186f964c887d29a7213cbc4006c10", + "object": "response", + "created_at": 1787922169, + "status": "in_progress", + "background": false, + "completed_at": null, + "error": null, + "frequency_penalty": 0, + "incomplete_details": null, + "instructions": null, + "max_output_tokens": 16, + "max_tool_calls": null, + "model": "gpt-5.4-nano-2026-03-17", + "moderation": null, + "output": [], + "parallel_tool_calls": true, + "presence_penalty": 0, + "previous_response_id": null, + "prompt_cache_key": null, + "prompt_cache_retention": "in_memory", + "reasoning": { + "context": "current_turn", + "effort": "none", + "mode": "standard", + "summary": null + }, + "safety_identifier": null, + "service_tier": "flex", + "store": true, + "temperature": 1, + "text": { + "format": { + "type": "text" + }, + "verbosity": "medium" + }, + "tool_choice": "auto", + "tool_usage": { + "image_gen": { + "input_tokens": 0, + "input_tokens_details": { + "image_tokens": 0, + "text_tokens": 0 + }, + "output_tokens": 0, + "output_tokens_details": { + "image_tokens": 0, + "text_tokens": 0 + }, + "total_tokens": 0 + }, + "web_search": { + "num_requests": 0 + } + }, + "tools": [], + "top_logprobs": 0, + "top_p": 0.98, + "truncation": "disabled", + "usage": null, + "user": null, + "metadata": {} + }, + "sequence_number": 1 + }, + { + "type": "response.output_item.added", + "item": { + "id": "msg_0a220bea710bdc08006a9186f9a1fc87d28258dcc08730be5d", + "type": "message", + "status": "in_progress", + "content": [], + "phase": "final_answer", + "role": "assistant" + }, + "output_index": 0, + "sequence_number": 2 + }, + { + "type": "response.content_part.added", + "content_index": 0, + "item_id": "msg_0a220bea710bdc08006a9186f9a1fc87d28258dcc08730be5d", + "output_index": 0, + "part": { + "type": "output_text", + "annotations": [], + "logprobs": [], + "text": "" + }, + "sequence_number": 3 + }, + { + "type": "response.output_text.delta", + "content_index": 0, + "delta": "pong", + "item_id": "msg_0a220bea710bdc08006a9186f9a1fc87d28258dcc08730be5d", + "logprobs": [], + "obfuscation": "fcrcqxrR1YVO", + "output_index": 0, + "sequence_number": 4 + }, + { + "type": "response.output_text.done", + "content_index": 0, + "item_id": "msg_0a220bea710bdc08006a9186f9a1fc87d28258dcc08730be5d", + "logprobs": [], + "output_index": 0, + "sequence_number": 5, + "text": "" + }, + { + "type": "response.content_part.done", + "content_index": 0, + "item_id": "msg_0a220bea710bdc08006a9186f9a1fc87d28258dcc08730be5d", + "output_index": 0, + "part": { + "type": "output_text", + "annotations": [], + "logprobs": [], + "text": "" + }, + "sequence_number": 6 + }, + { + "type": "response.output_item.done", + "item": { + "id": "msg_0a220bea710bdc08006a9186f9a1fc87d28258dcc08730be5d", + "type": "message", + "status": "completed", + "content": [ + { + "type": "output_text", + "annotations": [], + "logprobs": [], + "text": "" + } + ], + "phase": "final_answer", + "role": "assistant" + }, + "output_index": 0, + "sequence_number": 7 + }, + { + "type": "response.completed", + "response": { + "id": "resp_0a220bea710bdc08006a9186f964c887d29a7213cbc4006c10", + "object": "response", + "created_at": 1787922169, + "status": "completed", + "background": false, + "completed_at": 1787922169, + "error": null, + "frequency_penalty": 0, + "incomplete_details": null, + "instructions": null, + "max_output_tokens": 16, + "max_tool_calls": null, + "model": "gpt-5.4-nano-2026-03-17", + "moderation": null, + "output": [ + { + "id": "msg_0a220bea710bdc08006a9186f9a1fc87d28258dcc08730be5d", + "type": "message", + "status": "completed", + "content": [ + { + "type": "output_text", + "annotations": [], + "logprobs": [], + "text": "" + } + ], + "phase": "final_answer", + "role": "assistant" + } + ], + "parallel_tool_calls": true, + "presence_penalty": 0, + "previous_response_id": null, + "prompt_cache_key": null, + "prompt_cache_retention": "in_memory", + "reasoning": { + "context": "current_turn", + "effort": "none", + "mode": "standard", + "summary": null + }, + "safety_identifier": null, + "service_tier": "flex", + "store": true, + "temperature": 1, + "text": { + "format": { + "type": "text" + }, + "verbosity": "medium" + }, + "tool_choice": "auto", + "tool_usage": { + "image_gen": { + "input_tokens": 0, + "input_tokens_details": { + "image_tokens": 0, + "text_tokens": 0 + }, + "output_tokens": 0, + "output_tokens_details": { + "image_tokens": 0, + "text_tokens": 0 + }, + "total_tokens": 0 + }, + "web_search": { + "num_requests": 0 + } + }, + "tools": [], + "top_logprobs": 0, + "top_p": 0.98, + "truncation": "disabled", + "usage": { + "input_tokens": 14, + "input_tokens_details": { + "cache_write_tokens": 0, + "cached_tokens": 0 + }, + "output_tokens": 5, + "output_tokens_details": { + "reasoning_tokens": 0 + }, + "total_tokens": 19 + }, + "user": null, + "metadata": {} + }, + "sequence_number": 8 + } + ] +} diff --git a/tests/unit/adapters/fixtures/ramp_router/05_real_cache_write.json b/tests/unit/adapters/fixtures/ramp_router/05_real_cache_write.json new file mode 100644 index 0000000..0377944 --- /dev/null +++ b/tests/unit/adapters/fixtures/ramp_router/05_real_cache_write.json @@ -0,0 +1,84 @@ +{ + "_probe": "P5", + "_question": "first call — does it report cache_write/creation tokens?", + "_captured": "real capture against a live Ramp Router account", + "_scrubbed": "credentials, emails, IPs, dashboard URLs; prompt and completion text blanked to \"\"", + "_status": 200, + "_headers": { + "alt-svc": "h3=\":443\"; ma=86400", + "cf-cache-status": "DYNAMIC", + "cf-ray": "a323833a594d0956-EWR", + "connection": "keep-alive", + "content-encoding": "br", + "content-type": "application/json", + "date": "Fri, 28 Aug 2026 13:02:50 GMT", + "server": "cloudflare", + "strict-transport-security": "max-age=31536000; includeSubDomains; preload", + "transfer-encoding": "chunked", + "vary": "Origin", + "x-content-type-options": "nosniff", + "x-request-id": "79334ffd-c833-4a08-97de-46e934325b4a", + "x-trace-id": "79334ffd-c833-4a08-97de-46e934325b4a" + }, + "_body": { + "id": "resp_msg_011CeVC5Nk6htG3Wwq2qnwNt", + "created_at": 1787922170.8472354, + "error": null, + "incomplete_details": null, + "instructions": null, + "metadata": null, + "model": "claude-haiku-4-5-20251001", + "object": "response", + "output": [ + { + "id": "msg_011CeVC5Nk6htG3Wwq2qnwNt", + "content": [ + { + "annotations": [], + "text": "", + "type": "output_text", + "logprobs": null + } + ], + "role": "assistant", + "status": "completed", + "type": "message", + "phase": null + } + ], + "parallel_tool_calls": false, + "temperature": null, + "tool_choice": "auto", + "tools": [], + "top_p": null, + "background": null, + "completed_at": null, + "conversation": null, + "max_output_tokens": null, + "max_tool_calls": null, + "previous_response_id": null, + "prompt": null, + "prompt_cache_key": null, + "prompt_cache_retention": null, + "reasoning": null, + "safety_identifier": null, + "service_tier": "default", + "status": "completed", + "text": null, + "top_logprobs": null, + "truncation": null, + "usage": { + "input_tokens": 1457, + "input_tokens_details": { + "cached_tokens": 0 + }, + "output_tokens": 5, + "output_tokens_details": { + "reasoning_tokens": 0 + }, + "total_tokens": 1462 + }, + "user": null + }, + "_body_was_json": true +} diff --git a/tests/unit/adapters/fixtures/ramp_router/05b_real_cache_control_cold.json b/tests/unit/adapters/fixtures/ramp_router/05b_real_cache_control_cold.json new file mode 100644 index 0000000..7122f29 --- /dev/null +++ b/tests/unit/adapters/fixtures/ramp_router/05b_real_cache_control_cold.json @@ -0,0 +1,87 @@ +{ + "_captured": "real capture against a live Ramp Router account", + "_probe": "P5b", + "_question": "does Router forward an explicit cache_control part, and how is the write reported?", + "_scrubbed": "credentials, emails, IPs, dashboard URLs; prompt and completion text blanked to \"\"", + "_requested_models": [ + "claude-haiku-4-5" + ], + "_status": 200, + "_headers": { + "alt-svc": "h3=\":443\"; ma=86400", + "cf-cache-status": "DYNAMIC", + "cf-ray": "a323863a7d4bdb6d-EWR", + "connection": "keep-alive", + "content-encoding": "br", + "content-type": "application/json", + "date": "Fri, 28 Aug 2026 13:04:53 GMT", + "server": "cloudflare", + "strict-transport-security": "max-age=31536000; includeSubDomains; preload", + "transfer-encoding": "chunked", + "vary": "Origin", + "x-content-type-options": "nosniff", + "x-request-id": "4c8b735c-74c9-4df9-92aa-e4b5828dd7cd", + "x-trace-id": "4c8b735c-74c9-4df9-92aa-e4b5828dd7cd" + }, + "_body_was_json": true, + "_body": { + "id": "resp_msg_011CeVCERxpn9sEf4T2XQZHR", + "created_at": 1787922293.7136817, + "error": null, + "incomplete_details": null, + "instructions": null, + "metadata": null, + "model": "claude-haiku-4-5-20251001", + "object": "response", + "output": [ + { + "id": "msg_011CeVCERxpn9sEf4T2XQZHR", + "content": [ + { + "annotations": [], + "text": "", + "type": "output_text", + "logprobs": null + } + ], + "role": "assistant", + "status": "completed", + "type": "message", + "phase": null + } + ], + "parallel_tool_calls": false, + "temperature": null, + "tool_choice": "auto", + "tools": [], + "top_p": null, + "background": null, + "completed_at": null, + "conversation": null, + "max_output_tokens": null, + "max_tool_calls": null, + "previous_response_id": null, + "prompt": null, + "prompt_cache_key": null, + "prompt_cache_retention": null, + "reasoning": null, + "safety_identifier": null, + "service_tier": "default", + "status": "completed", + "text": null, + "top_logprobs": null, + "truncation": null, + "usage": { + "input_tokens": 18825, + "input_tokens_details": { + "cached_tokens": 0 + }, + "output_tokens": 5, + "output_tokens_details": { + "reasoning_tokens": 0 + }, + "total_tokens": 18830 + }, + "user": null + } +} diff --git a/tests/unit/adapters/fixtures/ramp_router/06_real_cache_read.json b/tests/unit/adapters/fixtures/ramp_router/06_real_cache_read.json new file mode 100644 index 0000000..95ebc87 --- /dev/null +++ b/tests/unit/adapters/fixtures/ramp_router/06_real_cache_read.json @@ -0,0 +1,84 @@ +{ + "_probe": "P5", + "_question": "second call — is cache_read INSIDE input_tokens or additive to it?", + "_captured": "real capture against a live Ramp Router account", + "_scrubbed": "credentials, emails, IPs, dashboard URLs; prompt and completion text blanked to \"\"", + "_status": 200, + "_headers": { + "alt-svc": "h3=\":443\"; ma=86400", + "cf-cache-status": "DYNAMIC", + "cf-ray": "a32383531af40956-EWR", + "connection": "keep-alive", + "content-encoding": "br", + "content-type": "application/json", + "date": "Fri, 28 Aug 2026 13:02:54 GMT", + "server": "cloudflare", + "strict-transport-security": "max-age=31536000; includeSubDomains; preload", + "transfer-encoding": "chunked", + "vary": "Origin", + "x-content-type-options": "nosniff", + "x-request-id": "79756dfa-2935-4a1f-80ee-64cb21de32f9", + "x-trace-id": "79756dfa-2935-4a1f-80ee-64cb21de32f9" + }, + "_body": { + "id": "resp_msg_011CeVC5f29GdbVBct6pLwu8", + "created_at": 1787922174.5423331, + "error": null, + "incomplete_details": null, + "instructions": null, + "metadata": null, + "model": "claude-haiku-4-5-20251001", + "object": "response", + "output": [ + { + "id": "msg_011CeVC5f29GdbVBct6pLwu8", + "content": [ + { + "annotations": [], + "text": "", + "type": "output_text", + "logprobs": null + } + ], + "role": "assistant", + "status": "completed", + "type": "message", + "phase": null + } + ], + "parallel_tool_calls": false, + "temperature": null, + "tool_choice": "auto", + "tools": [], + "top_p": null, + "background": null, + "completed_at": null, + "conversation": null, + "max_output_tokens": null, + "max_tool_calls": null, + "previous_response_id": null, + "prompt": null, + "prompt_cache_key": null, + "prompt_cache_retention": null, + "reasoning": null, + "safety_identifier": null, + "service_tier": "default", + "status": "completed", + "text": null, + "top_logprobs": null, + "truncation": null, + "usage": { + "input_tokens": 1457, + "input_tokens_details": { + "cached_tokens": 0 + }, + "output_tokens": 7, + "output_tokens_details": { + "reasoning_tokens": 0 + }, + "total_tokens": 1464 + }, + "user": null + }, + "_body_was_json": true +} diff --git a/tests/unit/adapters/fixtures/ramp_router/06b_real_cache_control_warm.json b/tests/unit/adapters/fixtures/ramp_router/06b_real_cache_control_warm.json new file mode 100644 index 0000000..e9f0c6e --- /dev/null +++ b/tests/unit/adapters/fixtures/ramp_router/06b_real_cache_control_warm.json @@ -0,0 +1,87 @@ +{ + "_captured": "real capture against a live Ramp Router account", + "_probe": "P5b", + "_question": "warm repeat of 05b \u2014 additive or inside-input?", + "_scrubbed": "credentials, emails, IPs, dashboard URLs; prompt and completion text blanked to \"\"", + "_requested_models": [ + "claude-haiku-4-5" + ], + "_status": 200, + "_headers": { + "alt-svc": "h3=\":443\"; ma=86400", + "cf-cache-status": "DYNAMIC", + "cf-ray": "a323864ccd77db6d-EWR", + "connection": "keep-alive", + "content-encoding": "br", + "content-type": "application/json", + "date": "Fri, 28 Aug 2026 13:04:56 GMT", + "server": "cloudflare", + "strict-transport-security": "max-age=31536000; includeSubDomains; preload", + "transfer-encoding": "chunked", + "vary": "Origin", + "x-content-type-options": "nosniff", + "x-request-id": "86b1f6b5-559e-4503-a4b9-4ab96ccafe20", + "x-trace-id": "86b1f6b5-559e-4503-a4b9-4ab96ccafe20" + }, + "_body_was_json": true, + "_body": { + "id": "resp_msg_011CeVCEePhA1G6PCUiP9Qnm", + "created_at": 1787922296.5541952, + "error": null, + "incomplete_details": null, + "instructions": null, + "metadata": null, + "model": "claude-haiku-4-5-20251001", + "object": "response", + "output": [ + { + "id": "msg_011CeVCEePhA1G6PCUiP9Qnm", + "content": [ + { + "annotations": [], + "text": "", + "type": "output_text", + "logprobs": null + } + ], + "role": "assistant", + "status": "completed", + "type": "message", + "phase": null + } + ], + "parallel_tool_calls": false, + "temperature": null, + "tool_choice": "auto", + "tools": [], + "top_p": null, + "background": null, + "completed_at": null, + "conversation": null, + "max_output_tokens": null, + "max_tool_calls": null, + "previous_response_id": null, + "prompt": null, + "prompt_cache_key": null, + "prompt_cache_retention": null, + "reasoning": null, + "safety_identifier": null, + "service_tier": "default", + "status": "completed", + "text": null, + "top_logprobs": null, + "truncation": null, + "usage": { + "input_tokens": 18825, + "input_tokens_details": { + "cached_tokens": 18810 + }, + "output_tokens": 5, + "output_tokens_details": { + "reasoning_tokens": 0 + }, + "total_tokens": 18830 + }, + "user": null + } +} diff --git a/tests/unit/adapters/fixtures/ramp_router/07_real_reasoning.json b/tests/unit/adapters/fixtures/ramp_router/07_real_reasoning.json new file mode 100644 index 0000000..56a6ce6 --- /dev/null +++ b/tests/unit/adapters/fixtures/ramp_router/07_real_reasoning.json @@ -0,0 +1,131 @@ +{ + "_captured": "real capture against a live Ramp Router account", + "_probe": "P6", + "_question": "is reasoning_tokens inside output_tokens?", + "_scrubbed": "credentials, emails, IPs, dashboard URLs; prompt and completion text blanked to \"\"", + "_requested_models": [ + "o4-mini" + ], + "_status": 200, + "_headers": { + "alt-svc": "h3=\":443\"; ma=86400", + "cf-cache-status": "DYNAMIC", + "cf-ray": "a32386527cc5db6d-EWR", + "connection": "keep-alive", + "content-encoding": "br", + "content-type": "application/json", + "date": "Fri, 28 Aug 2026 13:05:00 GMT", + "server": "cloudflare", + "strict-transport-security": "max-age=31536000; includeSubDomains; preload", + "transfer-encoding": "chunked", + "vary": "Origin", + "x-content-type-options": "nosniff", + "x-request-id": "da80e6b8-0cf0-49f5-aec9-6603e544b278", + "x-trace-id": "da80e6b8-0cf0-49f5-aec9-6603e544b278" + }, + "_body_was_json": true, + "_body": { + "background": false, + "billing": { + "payer": "developer" + }, + "completed_at": 1787922300, + "conversation": null, + "created_at": 1787922297, + "error": null, + "frequency_penalty": 0, + "id": "resp_0e76a7cad17577e4006a9187790c0087d28cbf7b8e27d688ce", + "incomplete_details": null, + "instructions": null, + "max_output_tokens": 400, + "max_tool_calls": null, + "metadata": {}, + "model": "o4-mini-2025-04-16", + "moderation": null, + "object": "response", + "output": [ + { + "id": "rs_0e76a7cad17577e4006a91877a446887d2891c06c99d153959", + "type": "reasoning", + "content": [], + "encrypted_content": "gAAAAABqkYd8wABExbg1YgRxIt6ACsyD0C88HZSfimBjLNchkw4tVccXSk4whKjl4ooJ4nkEzZTJqlhHZLTjYHAERLPZRlhtjCxoKzBtkmJKnCfd26VRXHFLPxzemhVVNRc1I-4HPv_NYUYMYZffciPEia_NF1fk0v5ml5jrccUm269F8pYzIPQ7ViHlBgNlZsb7QnTG4KA_jg9MDfM6Pjh7orn1Wu4iRuVEVYHtnwSom-YXR5alo35o3RkssU8qTzXOt8aOJIRj1EJdaTOct34_yQYHEQCh9CuZ9vvp-cYaB76M3QDCYlFAPrcodm8jlx3Bqx_vjevfrdxiPuLVE4cS9e-WoIz1FKx6AMQTB96AC5yHtMN-nSQBYpOjhzI0EdWuF5rqcDaQvk2BB9Shy40AcUzmnSWctaDCRowj3JSdV4QYji9wnjmvK-1BNLG3-Lag04dUUXE08GYYrF9LdgEe9ca0rw5hGCOkrvtAOA21j62CE6ejb1E2OUEDqCqdwUWniAbhqVV2XAFXsiCIvWDJBUhrIBXlPALw_uyo5jbAGLfEhV5p6b9a4heLPNM2wDPcN9Zbo3nqiqOwbGah8vx9I_8mQVpkW6lytXb0aMtbHmHG1Avtw8xRn0zN_wIbmur0G-Yzvc46Llb-mvtdbGygFUoZDAY8XwPeBEm0YSVI6tMrzhYF3l_Odc9g6KSdCg4UVsnX3VHkqbfTMJYuWLvsT1Vt6-Ns_QmWgXUQM4_KHLoGrdA61bM7fX35lKI6qFWjs93A5MKDfss3-zxszvIvguilNSD58GX4bWlmXUtl4W8wrmZN2fWEod8PsF_X2cGoo2m4ml3sMi5y1wYJ7C1n7uSf813n2rwnhH_mXnsxyT4XHhnRtYylt1rCL8l7TGR1VebVC6boFtLWmOnLVx_8PhZ8ebA3XgmdS0chMncf29VB51yyq-2J_k3Vg2hfgw-plFwaGyR8uJ8pyMIdAjUR8w5aEIOQRCSBJyaABfigFAn5xu1LA5R-ryWsMgv_U9EF5GZdE8MVcXtXSVYiJpmZQTzlUYBDmW9RrY2DU-OPGPpt91rrqlBD2Bl4pwVv0NcsjaAA5pRuaVGr8XuyyMqMEuBkv_BvVGqhoslwUIIyeAwd5EyoHkgadq3hEw064CQfCPGezyjMEHUVzcEJxj5rjQfLNNpt9zDEQOjAHcW8Nxmxl3hSbOG_n3lWsfDAVOmDlISKNlXrt7G_RZLnlCA1zJ6E0a6QBEjaJeOW6ppcptrAKsPz3GKzqKN45ecgmmgP9s4bLNcGMP1_tsCJQ8EvhFECd8UeKzc0DwHstZqN_Np9VU2nbAmNXjLEl9nI6jVH8T_enoFCDxSLhDK7zHJHD7TrlYwAJv-5jgQoVCrr_PcdnPF3t_PVF0S-DlA708fHz0zu_nYKWRvffAKHp886ZSIBXLx0iYAL5fWLSvd-4REMirF27aw_sln1WZTvjjEdj7ogF3OD8U80PMY4gietEVPVvYz0l1b2ViDVdoQQVOdvETEKaQZN8rYNFjqOZrWhYRnkNYtuuy-o_WwkWIuNWDojspFR6f6mRv9kjRsDcw3nC8hokzwyN_SnWlTCBAfCbjmsxnx9qgVqNneXgGnKV2qgPLBGkq1RKxX03X5PhsKnC6uV0seVG-DSS2Bxi0nATDbipuxyyGr3MkHXQSm7QpACQXpIVGf5EZej6NGEqSUhhHKQY2iJGOYEfQkVSh1mxaKEdyzDhiY-1g1xWpQH71ZQ78eoIkULg8xqalllI8PCrSDwcPsBxzJwBRbfiTNpg-wfc5hrS6OmsNN4odiqSXreT7LV7sZbxeypiJm--uh0hIL923mXquNii3ai9Q75c0EusMfTHO4759dQsHDatVn9vh76gQ==", + "summary": [] + }, + { + "content": [ + { + "type": "output_text", + "annotations": [], + "logprobs": [], + "text": "" + } + ], + "id": "msg_0e76a7cad17577e4006a91877c41a887d2a0da07ebb7cf8044", + "phase": null, + "role": "assistant", + "status": "completed", + "type": "message" + } + ], + "parallel_tool_calls": true, + "presence_penalty": 0, + "previous_response_id": null, + "prompt": null, + "prompt_cache_key": null, + "prompt_cache_retention": "in_memory", + "reasoning": { + "context": "current_turn", + "effort": "medium", + "generate_summary": null, + "mode": "standard", + "summary": null + }, + "safety_identifier": null, + "service_tier": "default", + "status": "completed", + "store": true, + "temperature": 1, + "text": { + "format": { + "type": "text" + }, + "verbosity": "medium" + }, + "tool_choice": "auto", + "tool_usage": { + "image_gen": { + "input_tokens": 0, + "input_tokens_details": { + "image_tokens": 0, + "text_tokens": 0 + }, + "output_tokens": 0, + "output_tokens_details": { + "image_tokens": 0, + "text_tokens": 0 + }, + "total_tokens": 0 + }, + "web_search": { + "num_requests": 0 + } + }, + "tools": [], + "top_logprobs": 0, + "top_p": 1, + "truncation": "disabled", + "usage": { + "input_tokens": 24, + "input_tokens_details": { + "cache_write_tokens": 0, + "cached_tokens": 0 + }, + "output_tokens": 169, + "output_tokens_details": { + "reasoning_tokens": 128 + }, + "total_tokens": 193 + }, + "user": null + } +} diff --git a/tests/unit/adapters/fixtures/ramp_router/08_real_error_404_model_not_found.json b/tests/unit/adapters/fixtures/ramp_router/08_real_error_404_model_not_found.json new file mode 100644 index 0000000..aa462ba --- /dev/null +++ b/tests/unit/adapters/fixtures/ramp_router/08_real_error_404_model_not_found.json @@ -0,0 +1,32 @@ +{ + "_probe": "P7", + "_question": "error envelope for: an id not in this key's catalog", + "_captured": "real capture against a live Ramp Router account", + "_scrubbed": "credentials, emails, IPs, dashboard URLs; prompt and completion text blanked to \"\"", + "_status": 404, + "_headers": { + "alt-svc": "h3=\":443\"; ma=86400", + "cf-cache-status": "DYNAMIC", + "cf-ray": "a323837edfea0956-EWR", + "connection": "keep-alive", + "content-encoding": "br", + "content-type": "application/json", + "date": "Fri, 28 Aug 2026 13:03:00 GMT", + "server": "cloudflare", + "strict-transport-security": "max-age=31536000; includeSubDomains; preload", + "transfer-encoding": "chunked", + "vary": "Origin", + "x-content-type-options": "nosniff", + "x-request-id": "275b1569-5760-4ab4-9ac5-816b1714c31c", + "x-trace-id": "275b1569-5760-4ab4-9ac5-816b1714c31c" + }, + "_body": { + "error": { + "message": "The model `definitely-not-a-model-id` does not exist or you do not have access to it.", + "type": "invalid_request_error", + "param": "model", + "code": "model_not_found" + } + }, + "_body_was_json": true +} diff --git a/tests/unit/adapters/fixtures/ramp_router/09_real_error_400_both_selectors.json b/tests/unit/adapters/fixtures/ramp_router/09_real_error_400_both_selectors.json new file mode 100644 index 0000000..26e10dc --- /dev/null +++ b/tests/unit/adapters/fixtures/ramp_router/09_real_error_400_both_selectors.json @@ -0,0 +1,31 @@ +{ + "_probe": "P7", + "_question": "error envelope for: both route selectors at once", + "_captured": "real capture against a live Ramp Router account", + "_scrubbed": "credentials, emails, IPs, dashboard URLs; prompt and completion text blanked to \"\"", + "_status": 400, + "_headers": { + "alt-svc": "h3=\":443\"; ma=86400", + "cf-cache-status": "DYNAMIC", + "cf-ray": "a323837ffcfe0956-EWR", + "connection": "keep-alive", + "content-length": "174", + "content-type": "application/json", + "date": "Fri, 28 Aug 2026 13:03:01 GMT", + "server": "cloudflare", + "strict-transport-security": "max-age=31536000; includeSubDomains; preload", + "vary": "Origin", + "x-content-type-options": "nosniff", + "x-request-id": "21dcb9c3-d76d-4de6-b3df-a1d352e1dcf5", + "x-trace-id": "21dcb9c3-d76d-4de6-b3df-a1d352e1dcf5" + }, + "_body": { + "error": { + "message": "`models` cannot be provided with `model`; provide either `model` or `models`.", + "type": "invalid_request_error", + "param": "models", + "code": "invalid_request" + } + }, + "_body_was_json": true +} diff --git a/tests/unit/adapters/fixtures/ramp_router/10_real_error_400_no_selector.json b/tests/unit/adapters/fixtures/ramp_router/10_real_error_400_no_selector.json new file mode 100644 index 0000000..3ad7312 --- /dev/null +++ b/tests/unit/adapters/fixtures/ramp_router/10_real_error_400_no_selector.json @@ -0,0 +1,31 @@ +{ + "_probe": "P7", + "_question": "error envelope for: neither route selector", + "_captured": "real capture against a live Ramp Router account", + "_scrubbed": "credentials, emails, IPs, dashboard URLs; prompt and completion text blanked to \"\"", + "_status": 400, + "_headers": { + "alt-svc": "h3=\":443\"; ma=86400", + "cf-cache-status": "DYNAMIC", + "cf-ray": "a3238381ac3db785-EWR", + "connection": "keep-alive", + "content-length": "130", + "content-type": "application/json", + "date": "Fri, 28 Aug 2026 13:03:01 GMT", + "server": "cloudflare", + "strict-transport-security": "max-age=31536000; includeSubDomains; preload", + "vary": "Origin", + "x-content-type-options": "nosniff", + "x-request-id": "3654a521-8409-4349-8038-d974d36d0036", + "x-trace-id": "3654a521-8409-4349-8038-d974d36d0036" + }, + "_body": { + "error": { + "message": "you must provide a model parameter", + "type": "invalid_request_error", + "param": "model", + "code": "invalid_request" + } + }, + "_body_was_json": true +} diff --git a/tests/unit/gateway/test_ramp_router.py b/tests/unit/gateway/test_ramp_router.py new file mode 100644 index 0000000..59cde64 --- /dev/null +++ b/tests/unit/gateway/test_ramp_router.py @@ -0,0 +1,688 @@ +"""Ramp Router live path — fake client, no live API.""" + +from __future__ import annotations + +import json +import pathlib +import threading +from typing import Any + +import pytest + +from lago_agent_sdk import LagoSDK +from lago_agent_sdk.adapters.openai_native import RAMP_ROUTER_PROVIDER, extract_openai_native +from lago_agent_sdk.pricing import ( + TOKEN_BILLED_PROVIDERS, + PricingProvider, + parse_openrouter, +) +from lago_agent_sdk.token_semantics import KNOWN_PROVIDERS, token_semantics +from lago_agent_sdk.wrappers.openai import _provider_hint_for + +ROUTER_BASE_URL = "https://api.router.com/v1" + + +def router_response(model: str, usage: dict[str, Any] | None = None) -> dict[str, Any]: + """A Router response, in the shape its docs specify: "Every request and response + uses the OpenAI Responses schema, whichever provider serves it." + + Hand-built rather than captured, and deliberately so for now: these tests pin the + SDK's own decisions — detection, candidate parsing, which field becomes the model — + none of which depend on Router's exact numbers. The assertions that need real + numbers live with the captured fixtures. + """ + return { + "id": "resp_test", + "object": "response", + "model": model, + "output": [{"type": "message", "content": [{"type": "output_text", "text": "pong"}]}], + "usage": { + "input_tokens": 11, + "output_tokens": 3, + "total_tokens": 14, + "input_tokens_details": {"cached_tokens": 0}, + "output_tokens_details": {"reasoning_tokens": 0}, + **(usage or {}), + }, + } + + +class _FakeStreamChunk: + """Mimics a Responses-API stream event.""" + + def __init__(self, payload: dict[str, Any]) -> None: + self._payload = payload + + def model_dump(self) -> dict[str, Any]: + return self._payload + + +class FakeRouterResponses: + def __init__(self, reply: Any) -> None: + self._reply = reply + self.create_calls = 0 + self.last_kwargs: dict[str, Any] | None = None + + def create(self, **kwargs: Any) -> Any: + self.create_calls += 1 + assert "extra_lago" not in kwargs # wrapper must strip lago opts + self.last_kwargs = dict(kwargs) + return self._reply(kwargs) + + +class FakeRouterClient: + def __init__(self, base_url: str, reply: Any) -> None: + self.base_url = base_url + self.responses = FakeRouterResponses(reply) + + +# The detector keys on the module; Router is reached with an OpenAI client. +FakeRouterClient.__module__ = "openai.fake" + + +def _new_sdk(default_sub: str = "sub_test", **config: Any) -> tuple[LagoSDK, list[dict]]: + received: list[dict] = [] + + def sender(batch: list[dict]) -> None: + received.extend(batch) + + from lago_agent_sdk import LagoConfig + + cfg = LagoConfig(**config) if config else None + sdk = LagoSDK(api_key="dummy", default_subscription_id=default_sub, config=cfg) + sdk._queue._sender = sender # type: ignore[attr-defined] + return sdk, received + + +def _by_code(received: list[dict]) -> dict[str, float]: + """code -> numeric value, the reduction every wrapper test in this repo uses.""" + return {e["code"]: float(e["properties"]["value"]) for e in received} + + +# ---------------------------------------------------------------------- +# Detection. `base_url` is the ONLY signal: Router's model ids are +# account-specific and opaque, and an Anthropic-served response arrives in +# OpenAI's schema, so nothing in the response body distinguishes the two. +# ---------------------------------------------------------------------- +class _Base: + def __init__(self, base_url: Any) -> None: + self.base_url = base_url + + +@pytest.mark.parametrize( + "base_url,expected", + [ + ("https://api.router.com/v1", RAMP_ROUTER_PROVIDER), + ("https://api.router.com/v1/", RAMP_ROUTER_PROVIDER), + ("https://API.Router.COM/v1", RAMP_ROUTER_PROVIDER), + # A regional or staging host under the same domain still bills as Router. + ("https://api-eu.router.com/v1", RAMP_ROUTER_PROVIDER), + # Direct providers and other gateways must be untouched. + ("https://api.openai.com/v1", ""), + ("https://gateway.ai.cloudflare.com/v1/acct/gw/compat", ""), + ], +) +def test_detection_base_url_is_the_only_signal(base_url: str, expected: str) -> None: + assert _provider_hint_for(_Base(base_url)) == expected + + +def test_a_lookalike_host_that_merely_contains_the_router_path_is_not_router() -> None: + """The reason detection parses the host instead of a substring test: a substring + stamps this unrelated endpoint's traffic as Router-served.""" + assert _provider_hint_for(_Base("https://evil.example.com/api.router.com/v1")) == "" + assert _provider_hint_for(_Base("https://evilrouter.com/v1")) == "" + + +def test_a_missing_malformed_or_exotic_base_url_never_throws_out_of_wrap() -> None: + class _NoUrl: + pass + + class _Explodes: + @property + def base_url(self) -> str: + raise RuntimeError("client blew up") + + assert _provider_hint_for(_NoUrl()) == "" + assert _provider_hint_for(None) == "" + assert _provider_hint_for(_Base("/v1")) == "" + assert _provider_hint_for(_Base(42)) == "" + assert _provider_hint_for(_Explodes()) == "" + + +# ---------------------------------------------------------------------- +# Candidate parsing. Router names a model two ways and both arrive in the +# same response field: an opaque account-specific id, or an explicit +# `provider:provider-model[:service-tier]` candidate. +# ---------------------------------------------------------------------- +def _extract(model: str) -> Any: + return extract_openai_native(router_response(model), model_id="", provider_hint=RAMP_ROUTER_PROVIDER) + + +def test_stamps_api_and_provider_as_ramp_router_keeping_the_surface_in_extras() -> None: + u = _extract("gpt-5.4-nano") + assert u.api == RAMP_ROUTER_PROVIDER + # The provider is NOT the vendor that served the call: Router's overlap semantics + # are its OWN (measured OpenAI-shaped) — see RAMP_ROUTER_PROVIDER. + assert u.provider == RAMP_ROUTER_PROVIDER + assert u.extras["router_surface"] == "responses" + + +def test_leaves_an_opaque_account_specific_id_exactly_as_reported() -> None: + """ "Valid model IDs are account-specific... Never invent one or reuse a provider's + public model name." So there is nothing to parse and nothing to strip.""" + u = _extract("my-org-fast-tier-7") + assert u.model == "my-org-fast-tier-7" + assert "router_provider" not in u.extras + assert "service_tier" not in u.extras + + +def test_splits_an_explicit_candidate_into_a_bare_model_plus_the_provider() -> None: + u = _extract("openai:gpt-5.4-mini") + # Bare, so a Router-served model rolls up in Lago against the same name a direct + # call to it reports rather than splitting into a second row. + assert u.model == "gpt-5.4-mini" + assert u.extras["router_provider"] == "openai" + + +def test_keeps_a_fireworks_models_whole_path_which_contains_slashes() -> None: + """The reason the split is on the FIRST colon only. A naive split on every colon + would keep "accounts" and lose the rest of the id.""" + u = _extract("fireworks:accounts/fireworks/models/kimi-k2p7-code") + assert u.model == "accounts/fireworks/models/kimi-k2p7-code" + assert u.extras["router_provider"] == "fireworks" + + +def test_pulls_a_pinned_service_tier_out_into_extras() -> None: + """Billing-relevant on its own: Router's catalog says tiers "may use different + rates" than the base ones it publishes, so pricing must be able to see this.""" + u = _extract("openai:gpt-5.4-mini:flex") + assert u.model == "gpt-5.4-mini" + assert u.extras["router_provider"] == "openai" + assert u.extras["service_tier"] == "flex" + + +@pytest.mark.parametrize("tier", ["auto", "default", "flex", "priority"]) +def test_recognizes_the_documented_tiers(tier: str) -> None: + u = _extract(f"openai:gpt-5.4-mini:{tier}") + assert u.extras["service_tier"] == tier + assert u.model == "gpt-5.4-mini" + + +def test_treats_an_unrecognized_trailing_segment_as_part_of_the_model_not_a_tier() -> None: + """A wrongly-stripped segment silently renames the model and splits it into a second + row in Lago. Keeping it is recoverable; renaming is not.""" + u = _extract("openai:gpt-5.4-mini:turbo") + assert u.model == "gpt-5.4-mini:turbo" + assert "service_tier" not in u.extras + + +def test_does_not_read_a_path_shaped_prefix_as_a_provider() -> None: + u = _extract("accounts/fireworks/models/foo:bar") + assert u.model == "accounts/fireworks/models/foo:bar" + assert "router_provider" not in u.extras + + +def test_bills_the_served_model_not_the_requested_one() -> None: + """Two ways requested and served diverge on Router: a `models` fallback list sends + no `model` field at all, and Switchyard routing can serve a different model than the + one asked for. The response is the only place the served model appears.""" + u = extract_openai_native( + router_response("anthropic:claude-haiku-4-5"), + model_id="openai:gpt-5.4-mini", + provider_hint=RAMP_ROUTER_PROVIDER, + ) + assert u.model == "claude-haiku-4-5" + assert u.extras["router_provider"] == "anthropic" + + +def test_leaves_a_non_router_clients_provider_inference_alone() -> None: + u = extract_openai_native(router_response("gpt-4o-mini-2024-07-18"), model_id="") + assert u.provider == "openai" + assert u.api == "responses" + assert "router_surface" not in u.extras + + +# ---------------------------------------------------------------------- +# Token mode is the default and must be exact: the counts Router reported, +# no field invented, none derived. +# ---------------------------------------------------------------------- +def test_a_router_pointed_client_bills_with_no_code_change_but_base_url() -> None: + sdk, received = _new_sdk() + client = sdk.wrap(FakeRouterClient(ROUTER_BASE_URL, lambda kw: router_response("openai:gpt-5.4-mini"))) + client.responses.create(model="gpt-5.4-mini", input="ping") + assert sdk.flush(timeout=2.0) + sdk.shutdown(timeout=1.0) + + by_code = _by_code(received) + assert by_code["llm_input_tokens"] == 11 + assert by_code["llm_output_tokens"] == 3 + assert len(received) == 2 # input + output only — total_tokens is derived + assert all(e["properties"]["model"] == "gpt-5.4-mini" for e in received) + + +def test_emits_the_same_fields_a_direct_provider_call_would() -> None: + sdk, received = _new_sdk() + client = sdk.wrap( + FakeRouterClient( + ROUTER_BASE_URL, + lambda kw: router_response( + "anthropic:claude-haiku-4-5", + { + "input_tokens": 1200, + "output_tokens": 40, + "total_tokens": 1240, + "input_tokens_details": {"cached_tokens": 900}, + "output_tokens_details": {"reasoning_tokens": 25}, + }, + ), + ) + ) + client.responses.create(model="x", input="ping") + assert sdk.flush(timeout=2.0) + sdk.shutdown(timeout=1.0) + + by_code = _by_code(received) + # Faithful extraction. Whether cache_read sits inside input is a PRICING question, + # not an extraction one — token mode reports what Router reported either way. + assert by_code["llm_input_tokens"] == 1200 + assert by_code["llm_output_tokens"] == 40 + assert by_code["llm_cached_input_tokens"] == 900 + assert by_code["llm_reasoning_tokens"] == 25 + # Exactly four events. `total_tokens` is derived from the others, so mapping it + # would double-count — a fifth event here would mean it had been. + assert len(received) == 4 + + +def test_a_streamed_call_bills_exactly_once_from_the_terminal_event() -> None: + def reply(kwargs: dict[str, Any]) -> Any: + if kwargs.get("stream") is not True: + return router_response("openai:gpt-5.4-mini") + # Router returns "OpenAI Responses server-sent events", which nest both usage + # and the resolved model under `.response`. + events = [ + _FakeStreamChunk( + {"type": "response.created", "response": {"id": "resp_1", "model": "openai:gpt-5.4-mini"}} + ), + _FakeStreamChunk({"type": "response.output_text.delta", "delta": "po"}), + _FakeStreamChunk( + { + "type": "response.completed", + "response": { + "id": "resp_1", + "model": "openai:gpt-5.4-mini", + "usage": {"input_tokens": 11, "output_tokens": 3, "total_tokens": 14}, + }, + } + ), + ] + return iter(events) + + sdk, received = _new_sdk() + client = sdk.wrap(FakeRouterClient(ROUTER_BASE_URL, reply)) + list(client.responses.create(model="gpt-5.4-mini", input="ping", stream=True)) + assert sdk.flush(timeout=2.0) + sdk.shutdown(timeout=1.0) + + assert len(received) == 2 # one input + one output, not two of each + assert _by_code(received)["llm_input_tokens"] == 11 + # The stream carries the served candidate too, parsed the same way. + assert all(e["properties"]["model"] == "gpt-5.4-mini" for e in received) + + +def test_a_models_fallback_request_bills_the_candidate_that_answered() -> None: + sdk, received = _new_sdk() + client = sdk.wrap( + FakeRouterClient( + ROUTER_BASE_URL, + # Second candidate served it. Billing the requested list would bill the + # wrong model, and the request carried no `model` field to fall back on. + lambda kw: router_response("fireworks:accounts/fireworks/models/kimi-k2p7-code"), + ) + ) + client.responses.create( + models=["openai:gpt-5.4-mini", "fireworks:accounts/fireworks/models/kimi-k2p7-code"], + input="ping", + ) + assert sdk.flush(timeout=2.0) + sdk.shutdown(timeout=1.0) + + assert len(received) == 2 + assert received[0]["properties"]["model"] == "accounts/fireworks/models/kimi-k2p7-code" + + +# ---------------------------------------------------------------------- +# A failure must never bill, and a malformed payload must never throw on the +# customer's call path. +# ---------------------------------------------------------------------- +_ROUTER_ERRORS = [ + (400, "invalid_request"), + (401, "invalid_api_key"), + (401, "api_key_deactivated"), + (402, "insufficient_credits"), + (403, "provider_unavailable"), + (404, "model_not_found"), + (413, "request_too_large"), + (429, "rate_limit_exceeded"), + (500, "internal_error"), + (501, "not_implemented_error"), + (502, "provider_request_failed"), + (502, "all_candidates_failed"), + (503, "service_unavailable"), + (504, "provider_request_failed"), +] + + +@pytest.mark.parametrize("status,code", _ROUTER_ERRORS) +def test_router_errors_emit_nothing(status: int, code: str) -> None: + """Every status Router's errors-and-limits page documents, with its code.""" + + def reply(kwargs: dict[str, Any]) -> Any: + err = RuntimeError(f"router {status}") + err.status = status # type: ignore[attr-defined] + raise err + + sdk, received = _new_sdk() + client = sdk.wrap(FakeRouterClient(ROUTER_BASE_URL, reply)) + with pytest.raises(RuntimeError): + client.responses.create(model="x", input="ping", _code=code) + assert sdk.flush(timeout=2.0) + sdk.shutdown(timeout=1.0) + assert received == [] + + +def test_a_zero_usage_response_emits_nothing_rather_than_a_zero_valued_event() -> None: + sdk, received = _new_sdk() + client = sdk.wrap( + FakeRouterClient( + ROUTER_BASE_URL, + lambda kw: router_response( + "openai:gpt-5.4-mini", {"input_tokens": 0, "output_tokens": 0, "total_tokens": 0} + ), + ) + ) + client.responses.create(model="x", input="ping") + assert sdk.flush(timeout=2.0) + sdk.shutdown(timeout=1.0) + assert received == [] + + +@pytest.mark.parametrize( + "payload", + [ + # api.router.com sits behind Cloudflare bot management, so a non-2xx can + # genuinely be an HTML challenge page rather than Router's documented JSON. + "Attention Required! | Cloudflare", + None, + 7, + {"id": "resp_1", "model": "openai:gpt-5.4-mini"}, + {"id": "resp_1", "model": "openai:gpt-5.4-mini", "usage": None}, + {"model": "openai:gpt-5.4-mini", "usage": {"input_tokens": "nope"}}, + {"model": "openai:gpt-5.4-mini", "usage": {"input_tokens": -5}}, + {"model": None, "usage": {"input_tokens": 4, "output_tokens": 1}}, + ], +) +def test_degrades_to_zero_rather_than_throwing_on_malformed_payloads(payload: Any) -> None: + u = extract_openai_native(payload, model_id="", provider_hint=RAMP_ROUTER_PROVIDER) + assert u.api == RAMP_ROUTER_PROVIDER + assert u.input >= 0 + + +# ---------------------------------------------------------------------- +# Price mode. Every Router call currently takes a clean pricing MISS and falls +# back to token events, because no vendor can be assigned to it safely yet. +# +# A real price table is loaded for these tests, and the same model is billed +# both directly and through Router. Without that contrast the tests would pass +# on an empty table, proving nothing: everything misses when nothing is priced. +# ---------------------------------------------------------------------- +PRICED_MODEL = "gpt-5.4-mini" +# Built through the real parser from a real-shaped OpenRouter payload, not from a +# hand-written key: `norm()` rewrites "." to "-", and a test whose table silently fails +# to load proves nothing about a miss. +# +# $0.75/M input and $4.50/M output are Router's own published base rates for this model. +_OPENROUTER_TABLE = parse_openrouter( + { + "data": [ + {"id": f"openai/{PRICED_MODEL}", "pricing": {"prompt": "0.00000075", "completion": "0.0000045"}} + ] + } +) + + +class _StubFetcher: + def fetch_openrouter(self) -> dict[str, Any]: + return _OPENROUTER_TABLE + + def fetch_bedrock(self, region: str) -> dict[str, Any]: + return {} + + def fetch_cloudflare_workers_ai(self) -> dict[str, Any]: + return {} + + def fetch_mistral_aliases(self, api_key: str | None = None) -> dict[str, str]: + return {} + + +def _priced_sdk() -> tuple[LagoSDK, list[dict], PricingProvider]: + provider = PricingProvider(fetcher=_StubFetcher(), ttl_seconds=3600.0) + sdk, received = _new_sdk(pricing_mode="price", pricing_provider=provider) + # The table has to be warm before the call, or the miss under test is just a cold + # cache. `maybe_refresh` is the queue worker's own warm-up, called synchronously. + provider.prime(["openrouter"]) + provider.maybe_refresh() + return sdk, received, provider + + +def test_the_same_model_does_price_when_called_directly_the_table_is_real() -> None: + """The control. If this fails, every "misses" assertion below is vacuous.""" + sdk, received, provider = _priced_sdk() + assert provider.lookup("openai", PRICED_MODEL, "responses") is not None + client = sdk.wrap(FakeRouterClient("https://api.openai.com/v1", lambda kw: router_response(PRICED_MODEL))) + client.responses.create(model=PRICED_MODEL, input="ping") + assert sdk.flush(timeout=2.0) + sdk.shutdown(timeout=1.0) + + codes = [e["code"] for e in received] + assert "llm_cost" in codes + assert "llm_input_tokens" not in codes + + +def test_the_identical_model_through_router_misses_and_falls_back_to_token_events() -> None: + """Same table, same model, same usage — only the base URL differs. The miss is + caused by the Router provider vocabulary, which is the decision under test: Router + bills $0 for a BYOK-served request and a non-default tier at a rate its catalog says + "may differ", so a list-price lookup can be flatly wrong.""" + sdk, received, _ = _priced_sdk() + client = sdk.wrap(FakeRouterClient(ROUTER_BASE_URL, lambda kw: router_response(f"openai:{PRICED_MODEL}"))) + client.responses.create(model=PRICED_MODEL, input="ping") + assert sdk.flush(timeout=2.0) + sdk.shutdown(timeout=1.0) + + by_code = _by_code(received) + assert "llm_cost" not in by_code + # Not a silent drop. The usage is billed, exactly, as tokens. + assert by_code["llm_input_tokens"] == 11 + assert by_code["llm_output_tokens"] == 3 + + +def test_a_flex_tier_call_is_never_billed_at_the_base_rate() -> None: + """supported-models: "Service tiers, long contexts, caching, and other features may + use different rates." Billing flex at the standard rate over-bills.""" + sdk, received, _ = _priced_sdk() + client = sdk.wrap( + FakeRouterClient(ROUTER_BASE_URL, lambda kw: router_response(f"openai:{PRICED_MODEL}:flex")) + ) + client.responses.create(model="x", input="ping") + assert sdk.flush(timeout=2.0) + sdk.shutdown(timeout=1.0) + assert "llm_cost" not in [e["code"] for e in received] + + +def test_a_pricing_miss_never_reaches_the_caller_as_an_exception() -> None: + sdk, _, _ = _priced_sdk() + client = sdk.wrap(FakeRouterClient(ROUTER_BASE_URL, lambda kw: router_response(f"openai:{PRICED_MODEL}"))) + assert client.responses.create(model="x", input="ping") is not None + sdk.shutdown(timeout=1.0) + + +# ---------------------------------------------------------------------- +# The hot path. Billing is enqueue-only, so concurrency must not lose or +# duplicate an event, and detection must not add per-call work. +# ---------------------------------------------------------------------- +def test_200_concurrent_calls_bill_exactly_200_input_events() -> None: + sdk, received = _new_sdk() + client = sdk.wrap(FakeRouterClient(ROUTER_BASE_URL, lambda kw: router_response("openai:gpt-5.4-mini"))) + + def call() -> None: + client.responses.create(model="x", input="ping") + + threads = [threading.Thread(target=call) for _ in range(200)] + for t in threads: + t.start() + for t in threads: + t.join() + assert sdk.flush(timeout=5.0) + sdk.shutdown(timeout=2.0) + + inputs = [e for e in received if e["code"] == "llm_input_tokens"] + assert len(inputs) == 200 + assert len({e["transaction_id"] for e in received}) == len(received) + + +# ---------------------------------------------------------------------- +# The two recorded decisions behind "ramp_router", pinned so neither can be +# reverted silently. The generic roster tests cannot see them: the hint comes +# from the wrapper's HOST arm, not from _PROVIDER_BY_BASE_URL_PATH, so nothing +# else in the suite fails if either set entry disappears. +# ---------------------------------------------------------------------- +def test_ramp_router_is_token_billed_a_price_mode_call_emits_token_events_with_no_error_report() -> None: + """Router is structurally unpriceable today (BYOK requests bill $0, tiers have + unpublished rates, every observed catalog input rate is empty), so a price miss is + permanent — and a permanent miss must not cry wolf on the error hook per call. Same + decision as Databricks and Snowflake.""" + assert RAMP_ROUTER_PROVIDER in TOKEN_BILLED_PROVIDERS + + errors: list[Any] = [] + provider = PricingProvider(fetcher=_StubFetcher(), ttl_seconds=3600.0) + sdk, received = _new_sdk( + pricing_mode="price", pricing_provider=provider, on_error=lambda exc, where: errors.append(exc) + ) + provider.prime(["openrouter"]) + provider.maybe_refresh() + client = sdk.wrap(FakeRouterClient(ROUTER_BASE_URL, lambda kw: router_response("openai:gpt-5.4-mini"))) + client.responses.create(model="x", input="ping") + assert sdk.flush(timeout=2.0) + sdk.shutdown(timeout=1.0) + assert sorted(e["code"] for e in received) == ["llm_input_tokens", "llm_output_tokens"] + assert errors == [] + + +def test_ramp_routers_token_convention_is_a_recorded_measurement_openai_shaped_on_every_axis() -> None: + """Measured live 2026-08-28, on an Anthropic-served model — the case that would + diverge if anything did: a warm cache_control call reported the cached block INSIDE + input_tokens (06b_real_cache_control_warm.json), and reasoning came back inside + output (07_real_reasoning.json). Router normalizes the NUMBERS to OpenAI's + convention, not just the schema. The entry lives in OPENAI_SHAPED_APIS because the + adapter stamps api="ramp_router" and the surface wins over the vendor.""" + assert RAMP_ROUTER_PROVIDER in KNOWN_PROVIDERS + assert token_semantics(RAMP_ROUTER_PROVIDER, RAMP_ROUTER_PROVIDER) == (True, True, True) + + +# ---------------------------------------------------------------------- +# Ordering of the api stamp against the total_tokens guard. Router is the +# only surface in this tree that REASSIGNS `api` mid-extract, so the stamp +# has to land before the guard reads it. +# ---------------------------------------------------------------------- +def _misreporting_router_response(total: int) -> dict[str, Any]: + """A Router payload whose declared total does NOT equal input + output, with both + subsets non-zero. No captured fixture has this shape — all ten report + total == input + output, streamed included — so this is the only cover the guard's + Router branch has.""" + return router_response( + "gpt-5.4-nano", + { + "input_tokens": 100, + "output_tokens": 50, + "total_tokens": total, + "input_tokens_details": {"cached_tokens": 80}, + "output_tokens_details": {"reasoning_tokens": 30}, + }, + ) + + +def test_the_totals_guard_reads_the_stamped_router_api_not_the_pre_stamp_surface() -> None: + """The guard, compute_cost and deoverlapped_token_total must answer the overlap + question identically — the whole reason token_semantics.py exists. Read before the + stamp, the guard sees ("ramp_router", "responses"), which is in no subset set, and so + adds cache_read + reasoning to an accounted sum that already contains them. + """ + u = extract_openai_native(_misreporting_router_response(1000), provider_hint=RAMP_ROUTER_PROVIDER) + # 1000 - (100 + 50). The cached block sits INSIDE input and reasoning INSIDE output, + # so neither is accounted twice; folding 740 would lose exactly cache_read + reasoning. + assert u.extras["unaccounted_output_tokens"] == 850 + assert u.output == 50 + 850 + # Read from before the stamp — moving the block above the guard must not cost this. + assert u.extras["router_surface"] == "responses" + + +def test_a_router_remainder_smaller_than_its_subsets_still_folds_rather_than_vanishing() -> None: + """The suppression case, and the one that loses money silently rather than merely + under-counting: with the wrong semantics the accounted sum (260) EXCEEDS the declared + total, `unaccounted` goes negative, the guard never fires, and 50 generated tokens are + dropped with no extras key and no on_error report.""" + u = extract_openai_native(_misreporting_router_response(200), provider_hint=RAMP_ROUTER_PROVIDER) + assert u.extras["unaccounted_output_tokens"] == 50 + assert u.output == 100 + + +# ---------------------------------------------------------------------- +# The captured responses, run through the adapter. The tests above pin the +# SDK's decisions against a hand-built shape; these pin them against what +# Router actually sent. Skips cleanly when the captures are absent, so a +# missing capture reads as "not covered" rather than as a pass. +# ---------------------------------------------------------------------- +_CAPTURES = pathlib.Path(__file__).parents[1] / "adapters" / "fixtures" / "ramp_router" + + +def _captured_bodies() -> list[tuple[str, dict[str, Any]]]: + """Every captured 200 that carries usage, buffered or streamed.""" + out: list[tuple[str, dict[str, Any]]] = [] + for path in sorted(_CAPTURES.glob("*.json")): + blob = json.loads(path.read_text()) + body = blob.get("_body") + if not (isinstance(body, dict) and body.get("usage")): + # The streamed capture keeps its payload under `.response` per event; the + # terminal one is the only one carrying usage, and is what the wrapper bills. + body = None + for event in blob.get("_events") or []: + candidate = (event or {}).get("response") + if isinstance(candidate, dict) and candidate.get("usage"): + body = candidate + if isinstance(body, dict) and body.get("usage"): + out.append((path.name, body)) + return out + + +@pytest.mark.skipif(not _captured_bodies(), reason="Router fixtures not captured") +@pytest.mark.parametrize("name,body", _captured_bodies(), ids=lambda v: v if isinstance(v, str) else "") +def test_every_captured_response_reports_its_served_tier(name: str, body: dict[str, Any]) -> None: + """The regression this file previously had no way to catch. The tier was read only + from a `provider:model:tier` candidate suffix, a shape Router resolves away before + answering, so `service_tier` was dropped on 100% of live traffic while the hand-built + tests stayed green.""" + u = extract_openai_native(body, provider_hint=RAMP_ROUTER_PROVIDER) + assert u.extras["service_tier"] == body["service_tier"] + + +@pytest.mark.skipif(not _captured_bodies(), reason="Router fixtures not captured") +@pytest.mark.parametrize("name,body", _captured_bodies(), ids=lambda v: v if isinstance(v, str) else "") +def test_every_captured_response_bills_the_bare_served_snapshot(name: str, body: dict[str, Any]) -> None: + """Router answers with a resolved vendor snapshot, never a compound candidate — the + reason the suffix parse is a fallback rather than the live path. Billing the model + verbatim is what rolls a Router-served call up against the same Lago row a direct + call to that model reports.""" + u = extract_openai_native(body, provider_hint=RAMP_ROUTER_PROVIDER) + assert u.model == body["model"] + assert ":" not in u.model + assert u.provider == RAMP_ROUTER_PROVIDER