Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,26 @@

All notable changes to this project will be documented here. Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); versions follow [SemVer](https://semver.org).

## [Unreleased]

### Added

- **Ramp Router price mode: a Router call is now priced from Router's own catalog.** `GET /v1/models` publishes a per-model rate block, and it is the rate Router bills — measured against a live account's dashboard export on 2026-09-04 and 2026-09-07: every default-tier call whose counts the response fully reports reconciled at exactly the catalog rate across all five served vendors, including the cache split (xAI, 194 in / 192 cached, to the last digit). `ramp_router` leaves `TOKEN_BILLED_PROVIDERS`: its miss is no longer structural, so a Router call that cannot be priced reports through `on_error` like any other provider's.
- **The Router key is learned from the wrapped client.** The catalog is account-scoped, and `OpenAI(api_key=...)` already carries the key that unlocks it, so `wrap()` reads it (same mechanism as the Mistral alias source) and primes the catalog before the first call. `LagoConfig.ramp_router_api_key` exists for pricing Router usage without `wrap()` and always wins over a learned key.
- **Every name a response can report is indexed.** Router answers with the vendor's own dated snapshot (`gpt-5.4-nano-2026-03-17`, version-stripped to the catalog id) or, for Fireworks and Baseten, the vendor's own path (`accounts/fireworks/models/…`), which is the entry's `provider_model` or an alias — never its id. All nine distinct served names across every capture resolve. A name two entries claim at different rates is left unpriced rather than guessed; a zero cache rate is stored as "no separate rate" so those tokens stay inside `input` at the input rate instead of billing a cached block at $0.
- **Only the default tier is priced.** `flex` measured 0.5x the catalog rate and `priority` 2.0x on two vendors, and neither factor is published anywhere machine-readable, so an explicitly reported non-default tier is a price miss: token events plus a `PricingUnavailableError` whose new `detail` names the tier. A response with **no** tier bills at the base rate — a 237-call sweep showed Router omits `service_tier` only on responses that stopped with zero output, and bills those at standard.
- **Eight OpenAI models bill off Router's own catalog; the SDK bills the catalog and documents the mismatch.** Reconciled row by row against the dashboard export of that sweep: `gpt-5.4-mini`, `gpt-5.4-nano`, `gpt-5.5`, `gpt-5.5-pro`, `gpt-5.6-luna`, `gpt-5.6-terra` and `gpt-6-astra` bill exactly 1.1x their published rate on every field; `gpt-5.6-sol` bills 0.55x — a 50%-off promotion on Router's model page, times the same 1.1. Every other model on every vendor billed the catalog rate exactly. No factor is applied in code — the day Router corrects its catalog a baked-in factor would be the thing out of sync — and the docs give the measured factors, dated, as a `markup` a customer can apply to those models and drop when the catalog catches up.
- **A catalog entry served through a different backend is a miss, not a price.** Ten Fireworks-owned entries carry a Baseten alias; when Baseten serves, Router bills Baseten's unpublished rate (1.11x to 2.4x off the catalog's) and names the model by that alias. Those aliases are not indexed, so the call takes the honest token-event path.
- **OpenAI-served cache writes are billed at the catalog write rate.** `/v1/responses` reports `cache_write_tokens` inside `input_tokens` for OpenAI models, and Router bills it at `cache_write_input` (gpt-5.6-luna, 4493 in / 4490 written: `3 × input + 4490 × cache_write + output`, exact). The adapter maps the count for the Router surface only; OpenAI-native keeps it unmapped, where Databricks metered the same field at the plain input rate. Anthropic-served writes stay unpriceable on this surface — the count is never reported — and bill at the input rate, documented as a shortfall of up to 20% (5m TTL) or 50% (1h TTL) on a cold call, in proportion to how much of the prompt was the cached prefix.
- Two dashboard-reconciled rows join `money_golden.json` in both repos.
- **An Anthropic client pointed at Router is detected too, on Router's `/v1/messages` surface.** Both wrappers now read one shared host helper, so they cannot disagree about what counts as Router. The Anthropic adapter takes the wrapper's hint and stamps `provider=ramp_router api=ramp_router_messages` — a stamp distinct from the Responses surface's, because the two report the same vendor under different conventions: `/v1/messages` keeps Anthropic's additive shape for every vendor (measured on an Anthropic and an xAI model), where `/v1/responses` folds cached tokens inside `input_tokens`. The Router key is learned from the Anthropic client the same way. Five new captured fixtures (11-15) back it, including a cold and warm `cache_control` pair and a streamed call.
- **Anthropic cache writes through Router are billed at their TTL rate, exactly.** `/v1/messages` reports `cache_creation_input_tokens` with the 5-minute / 1-hour split, and Router's catalog publishes both rates, so `ModelPrice` carries `cache_write_5m` / `cache_write_1h` and `compute_cost` bills each part at its own rate, consuming the lump `cache_write` count so nothing is billed twice. Reconciled against the dashboard: 16 in + 20,113 written (5m) + 5 out on `claude-haiku-4-5` = $0.02518225. Inert everywhere else: OpenRouter publishes a single Anthropic write rate, and that path is byte-for-byte unchanged. Two more dashboard-reconciled rows join `money_golden.json`.

### Fixed

- **Streamed Router calls now carry the served `service_tier`.** The stream wrapper forwarded usage and model only, so every streamed Router call reached price mode tier-less — which is a miss.
- **The catalog fixture's input rates were blanked by our own capture scrub.** The scrub treated any key named `input` as prompt content, including `router.pricing.input`, and the resulting empty column was cited as Router's data. Content keys are no longer blanked under `pricing`; the fixture is recaptured with all 68 rates.

## [0.3.0] - 2026-09-02

### Changed
Expand Down
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -181,7 +181,7 @@ Full guide, including cache semantics, dedup, attribution via `QUERY_TAG` and ac

### Ramp Router

An OpenAI-Responses-compatible gateway in front of OpenAI, Anthropic, Google Vertex, Fireworks and xAI. The model that answered is the one billed — Router can serve a different model than the one requested.
A gateway in front of OpenAI, Anthropic, Google Vertex, Fireworks and xAI, with an OpenAI-shaped surface and an Anthropic-shaped one. Point either client at it. The model that answered is the one billed — Router can serve a different model than the one requested.

```python
client = sdk.wrap(
Expand All @@ -191,7 +191,7 @@ client = sdk.wrap(
client.responses.create(model=os.environ["RAMP_ROUTER_MODEL"], input="Summarize this invoice.")
```

Full guide, including why price mode falls back to token events for Router traffic: [docs/ramp-router.md](docs/ramp-router.md).
Full guide, including how price mode bills Router's own catalog rate: [docs/ramp-router.md](docs/ramp-router.md).

## Multi-tenant — pick a subscription per call

Expand Down Expand Up @@ -226,7 +226,7 @@ sdk = LagoSDK(api_key="...", config=LagoConfig(
))
```

Price mode emits one `llm_cost` event per priced field (input, output, cache, ...), each carrying `precise_total_amount_cents` for Lago's **dynamic charge model** plus a `token_type` property so a single billable metric can be grouped by both `model` and `token_type`. Prices come from public sources (OpenRouter for native providers, the AWS Bedrock price list for Bedrock), fetched and cached in the background — your LLM call is never blocked on pricing. If a price isn't available yet, the SDK falls back to token-count events and reports via `on_error` rather than under-billing. Per-call override: `extra_lago={"mode": "price", "markup": 1.5}`.
Price mode emits one `llm_cost` event per priced field (input, output, cache, ...), each carrying `precise_total_amount_cents` for Lago's **dynamic charge model** plus a `token_type` property so a single billable metric can be grouped by both `model` and `token_type`. Prices come from public sources (OpenRouter for native providers, the AWS Bedrock price list for Bedrock) and from a gateway's own catalog where it publishes one (Cloudflare Workers AI, Ramp Router), fetched and cached in the background — your LLM call is never blocked on pricing. If a price isn't available yet, the SDK falls back to token-count events and reports via `on_error` rather than under-billing. Per-call override: `extra_lago={"mode": "price", "markup": 1.5}`.

## Error policy

Expand Down
Loading
Loading