Skip to content

Latest commit

 

History

History
73 lines (52 loc) · 2.88 KB

File metadata and controls

73 lines (52 loc) · 2.88 KB

CLIProxyAPI

A proxy that pools many provider subscriptions behind one endpoint, so coding clients can spend all of their quota without manual account switching.

Language

Pool

Provider: An upstream AI service that the proxy sends requests to, such as Claude, Codex, xAI, or Kimi. Avoid: backend, vendor

Credential: One signed-in subscription or API key for a provider, which the proxy can send requests through. Avoid: account, auth file, model

Quota window: A period over which a provider limits how much one credential can use, such as the Claude 5-hour and 7-day limits or the Codex weekly limit. Avoid: session, limit period

Quota reading: What the proxy last learned about one credential's quota window: the share left, the reset time, and when it was learned. Avoid: signal, watermark, snapshot

Reset time: The moment a quota window ends and the credential's usage in that window returns to zero. Avoid: session end, renewal

Wasted quota: The share of a quota window still left when its reset time arrives, which expiring-first routing exists to keep near zero. Avoid: leftover quota, unused quota

Clients

Client: A program that sends requests to the proxy on Dan's behalf, such as T3 Code, the Claude Code CLI, or the Codex CLI. Avoid: app, frontend, consumer

Client key: The secret a client presents to the proxy to be let in. It identifies the client, not a provider, and is never sent upstream. Avoid: API key, token, credential

Routing

Thread: One ongoing conversation in a client, whose prompts share context and build on each other. Avoid: session, chat

Affinity: The rule that sends every request in a thread to the same credential while that credential stays usable. Avoid: sticky routing, account affinity, session affinity

Binding: The link between one thread, on one model, and the credential that affinity sends it to. It ends when the thread has been idle for a set time or the credential can no longer be used, and the next request then picks a credential again. Avoid: pin, sticky session

Expiring-first routing: The rule that, among usable credentials, prefers the one with the highest urgency, so the quota that would be lost at a reset time is spent first. Avoid: smart routing, intelligent routing

Urgency: The share of a credential's quota window still left, divided by the time until its reset time: how fast the credential must be used to lose nothing at reset. Avoid: score, burn rate, pace

Cache write: A request that makes the provider store a new prompt prefix for reuse, instead of reading one it already stored. Avoid: cache miss, re-read

Requests

Cloaking: The proxy rewriting a request from a client that isn't Claude Code so that it reaches the provider looking like a Claude Code request. A request from Claude Code itself passes through unchanged. Avoid: disguise, spoofing