Skip to content

Commit b635e95

Browse files
Sync public snapshot from freebuff-private
Source: CodebuffAI/freebuff-private@3c7ed1866cb560021d75636337903bc788db6b67
1 parent 2e39c0d commit b635e95

2 files changed

Lines changed: 44 additions & 1 deletion

File tree

bun.lock

Lines changed: 3 additions & 1 deletion
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

common/src/constants/provider-routes.ts

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@ export const PROVIDER_ROUTE_IDS = [
88
'infron/makora',
99
'deepseek/openrouter',
1010
'deepseek/crof',
11+
'deepseek/luminal',
1112
'deepseek/runinfra',
1213
'deepseek/official',
1314
] as const
@@ -176,6 +177,46 @@ export const DEEPSEEK_CROF_PROVIDER_ROUTE =
176177
* watchdog is DeepSeek-direct-only (see `handleDeepSeekStream`); no equivalent
177178
* guards the CrofAI lane now serving in front of it.
178179
*/
180+
/**
181+
* DeepSeek V4 Flash's Luminal lane — a small FREE grant, and the only pin in
182+
* this file that is RATIONED rather than reactive.
183+
*
184+
* Every other route id here records where a session ENDED UP after something
185+
* failed. This one records that a session WON a slot: Luminal donated a slice
186+
* of Flash capacity far below our volume, so the pin is minted by an admission
187+
* controller (web/src/server/free-session/luminal-admission.ts) that hands out
188+
* a bounded number of them and stops when Luminal starts refusing.
189+
*
190+
* It is FIRST in the cascade for the sessions that carry it, which no other
191+
* cheap-lane experiment has earned, because it is free and the lanes behind it
192+
* are not: CrofAI's cache reads are $0.0030/M, DeepSeek direct's are
193+
* $0.0070-0.0140/M, and this is $0. The OpenRouter retirement on {@link
194+
* DEEPSEEK_CROF_PROVIDER_ROUTE} is the cautionary tale for adding a lane for
195+
* depth; this is the opposite — a lane added for price, capped so it cannot
196+
* become depth.
197+
*
198+
* Measured against the endpoint on 2026-08-20, which is what made this
199+
* routable at all:
200+
*
201+
* - It SHEDS rather than queues: HTTP 429 in 87-98ms with `retry-after: 1`
202+
* and a structured `rate_limit_error` body. A refused session costs one
203+
* fast round trip and diverts.
204+
* - REQUEST-bound, not token-bound. 32k prompts shed at ~175-224k tok/s
205+
* while 128k prompts sustained 498k tok/s untouched, so the served budget
206+
* (~5-7 req/s) does not shrink as prompts grow.
207+
* - Prefix caching holds at 99.4-99.9% across concurrency, against
208+
* production Flash's 98.8% — which is why admission is per SESSION. A
209+
* request-level share would make every request a cold prefill and consume
210+
* the grant on prefill alone.
211+
*
212+
* A 429 here is TERMINAL for the session, unlike a divert off any other lane:
213+
* the cascade re-pins it onward and it never comes back. That is deliberate.
214+
* The lanes behind this one are sized to take our whole volume, so there is
215+
* nothing to gain by retrying a rationed lane and one wasted round trip per
216+
* turn to lose.
217+
*/
218+
export const DEEPSEEK_LUMINAL_PROVIDER_ROUTE =
219+
'deepseek/luminal' satisfies ProviderRouteId
179220
export const DEEPSEEK_OFFICIAL_PROVIDER_ROUTE =
180221
'deepseek/official' satisfies ProviderRouteId
181222
/**

0 commit comments

Comments
 (0)