Add CheaperInference provider - #7010
Open
aiapienthusiast wants to merge 14 commits into
Open
aiapienthusiast wants to merge 14 commits into
aiapienthusiast wants to merge 14 commits into
Conversation
CheaperInference is an OpenAI-compatible gateway that serves each request from one of several providers for the requested model, ranked by discount, speed or a balance of both, at or below the model maker's list price. Adds the provider entry plus 16 relayed models, all using base_model against existing lab metadata with provider-side cost, cache rates, long-context tiers and reasoning_options only. Rates come from GET /v1/models (pricing_version sha256:d7fdd65d…, checked 2026-09-13T10:00:38Z). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Contributor
Action items
|
The gateway documents that the reasoning field is forwarded to the serving provider but names no on/off control of its own, so only the effort levels inherited from the lab entries are authored. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Contributor
Action items
|
Tested on the live chat completions endpoint: reasoning.enabled = false removes reasoning tokens and the reasoning field on kimi-k3, qwen-3-8-max, deepseek-v4.1-flash and deepseek-v4-flash-0731, and true restores them, so those four carry toggle with the wire path as a leading comment. On claude-sonnet-5 neither reasoning.enabled nor thinking had any observable effect on this path, so it stays effort-only. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Registers a sync module for the CheaperInference provider so the hourly workflow keeps its rates current instead of leaving them to drift. The gateway's catalog endpoint is authoritative for pricing: exact per-million input, output, cache-read and cache-write rates, an above_threshold band with its own token threshold for models priced in two bands, and the limits this host actually serves. It exposes no reasoning controls, so the module sets skipCreates and refuses to sync a reasoning model whose reasoning_options were not hand-authored; image and video routes are skipped because their per-unit media pricing cannot be expressed by the cost schema. Reading the catalog needs a key, so the workflow gains a CHEAPERINFERENCE_API_KEY secret. The endpoint accepts a key restricted to read-only scope, which cannot spend from the account behind it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Author
|
Addressed all three.
|
Contributor
|
No actionable findings. |
github-actions
Bot
added
the
reviewer: ready
github-actions
Bot
removed
the
reviewer: ready
Contributor
Action items
|
Contributor
Action items
|
Contributor
Action items
|
Re-probed every model the review flagged on the live chat completions endpoint, with a unique prompt per request so nothing came from a cache. Reasoning comes back where the gateway catalog says it does not. claude-opus-4-8-fast returns reasoning_content on every plain request (7 of 7) and drops it for reasoning.enabled = false, and glm-4.6 returns reasoning_content with 128 to 592 reasoning tokens for every graded effort value. Both lose the reasoning = false line: opus-4-8-fast carries the verified toggle plus the lab effort set, glm-4.6 the toggle its first-party file also uses. gemini-3-5-flash reasons as well, and the gateway documents its own set in a 400 - minimal, low, medium, high, thinking that cannot be disabled - so it stays effort-only with those four values. reasoning.enabled = false and reasoning.effort = none both switch reasoning off on deepseek-v4-flash, deepseek-v4-pro and deepseek-v4-flash-0731, so each keeps a single off control, the toggle, with graded values and no none: high|xhigh, high|xhigh and low|high|max. minimax-m2.7 has a real toggle (no reasoning in 4 of 4 requests with reasoning.enabled = false, reasoning in every other form), so it replaces the empty options list. The four Claude routes the review asked about - opus 4.5, opus 4.6, sonnet 4.5 and sonnet 4.6 - return no reasoning_content and no reasoning token count at all, and effort, reasoning.enabled and thinking.budget_tokens are each accepted with 200 and change nothing. budget_tokens goes, and reasoning_options is empty rather than an effort list this host does not honour. qwen-3-8-max, re-verified as the review asked: reasoning.enabled = false no longer suppresses anything in 6 of 6 requests, so its toggle and effort list go the same way. The luna, sol and terra context tiers move from 271_999 to 272_000, the band start the sync module emits and gpt-6-astra already carries. claude-haiku-4.5, claude-opus-4.7, gemini-2.5-flash and gemini-3.1-flash-lite keep reasoning = false: the catalog declares them non-reasoning and five wire forms return nothing on each. bun run validate passes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
qwen3-5-35b-a3b, qwen3-6-35b-a3b and qwen3.6-27b carried budget_tokens, but this gateway has no reasoning-budget field on the OpenAI-compatible path: reasoning.max_tokens = 64 and thinking.budget_tokens = 64 are both accepted with 200 and the reply still carries around a thousand characters of reasoning, the same as a plain request. Their toggles are re-verified and stay: reasoning.enabled = false returns no reasoning content on all three, and true restores it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Contributor
Action items
|
Answers the third review round, all of it re-probed on the live endpoint with a unique prompt per request. reasoning = false replaces reasoning_options = [] on claude-opus-4.5, claude-opus-4.6, claude-sonnet-4.5 and claude-sonnet-4.6: no request form - plain, reasoning.enabled, reasoning.effort, thinking.budget_tokens - returns reasoning_content or a reasoning token count on those routes, so an empty options list would claim an always-on reasoner that is not there. The four that already carried reasoning = false keep it and now cite the probe in a leading comment. qwen-3-8-max keeps reasoning = true with an empty options list, because it does return reasoning_content on every request and neither the toggle nor effort = "none" suppresses it. interleaved records the side channel where the reply carries one: 33 models return reasoning_content, gemini-3-flash-preview returns reasoning_details, and glm-4.7 returns its reasoning in a plain reasoning field, which the schema covers as interleaved = true. The OpenAI-shaped routes count reasoning tokens without exposing any content, so they carry no interleaved. attachment is dropped on gemini-3-flash-preview and gemini-3.1-pro-preview, which accept an image_url part; the other sixteen reject it (400, or every upstream route failing) and now also override modalities.input to text only so the two agree. qwen-3-8-27b gets the toggle it actually forwards plus low|medium|xhigh and no none. gpt-5.5-pro is responses-shaped: /v1/chat/completions returns 400 and /v1/responses answers, where effort medium and high work. bun run validate passes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Contributor
Action items
|
Author
|
Both review rounds are answered here, and every claim below is a live probe on Reasoning: what this host actually returnsNine models return nothing on any wire form I could find - plain,
Three models the catalog reports as non-reasoning do reason, so
One off control per model
budget_tokens
interleavedAdded wherever the reply carries a side channel, left off where it does not. 33 models return attachment and modalitiesProbed with an Tiers
One thing the review did not ask about
Worth knowing for the sync module: the catalog's own |
…routes claude-fable-5, claude-fable-5.1 and claude-opus-5-fast advertise capabilities.vision true, but an image_url part is rejected on every attempt (502 or 400, "all available provider routes failed"), exactly like the other Anthropic routes on this host, so they get attachment = false and a text-only input modality. All three do return the reasoning side channel in reasoning_content, so interleaved stays. gemini-3.6-flash reports usage.completion_tokens_details.reasoning_tokens but the message carries only content on 8 of 8 probes, so no side channel is declared for it. muse-spark-1.2 returns its reasoning in reasoning_details on 3 of 3 untruncated probes (runs cut short by max_tokens carry none), so interleaved.field is set accordingly.
…he sync A model that reasons on this host but exposes no reasoning controls in the catalog still needs hand-authored reasoning_options. Throwing a plain Error aborted the whole run, so a single new model could silence the hourly sync for every other model. MissingReasoningOptionsError is the shape the runner already understands: it keeps the local file, records the ID and carries on, so the gap is reported without stopping the run.
…e reasoner The old assertion only matched the message text, so a plain Error would have passed it. It now asserts the error type and the model ID it carries. A second test drives syncProvider over a temporary models directory holding one researched and one unresearched reasoner, and checks what the run actually does with them: the unresearched file is left byte-for-byte alone, nothing is deleted, the other model still syncs, and the ID is handed to the missing-model issue opener.
Contributor
Action items
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Cheaper Inference is an OpenAI-compatible gateway that serves each request from one of several providers for the requested model, ranked by discount, speed or a balance of both (a
rankingfield;balanceby default), at or below the model maker's list price.Adds
providers/cheaperinference/withprovider.toml,logo.svgand 58 models, eachbase_model-ed against the existing lab entry and carrying onlycost,reasoning_optionsandlimitdeltas. No new files under the top-levelmodels/.The 58 are every chat model the gateway serves:
openai14,anthropic13,zhipuai9,google8,deepseek5,alibaba5, and one each fromxai,moonshotai,minimaxandmeta. The image-generation and video routes it also serves are left out.Rates are from
GET /v1/models(read 2026-09-16T08:00:58Z,pricing_versionsha256:1099de72…), including cache rates. Four OpenAI models price a second band above 272k input tokens, recorded as[[cost.tiers]]:gpt-6-astra,gpt-5.6-sol,gpt-5.6-terraandgpt-5.6-luna. The gateway reprices often, so these are a timestamped snapshot; each file's first two comment lines carry the source, thepricing_versionand the discount off the maker's list price.reasoning_optionsare copied from the lab entries (51 of the 58 have them).toggleis on the 13 models where a live request confirms the gateway forwards it, each carrying a wire comment naming the exact field:{"reasoning": {"enabled": false}}onPOST /v1/chat/completionsfor 12 of them, and{"thinking": {"type": "disabled"}}forglm-4.5-air, which ignores the first form. In each case the disabled form returns no reasoning content and the enabled form returns it.claude-sonnet-5andqwen-3-8-maxkeep their other reasoning options but notoggle, becausereasoning.enabledchanged nothing in the reply;gemini-2.5-flashandglm-4.6carry no reasoning options at all, because the gateway reports no reasoning for them even though their lab entries have a toggle.limit.outputis set for every model from the gateway's declared maximum;limit.contextonly where the gateway's declared window differs from the lab entry (23 models).It also wires the provider into the daily sync so the snapshot does not go stale:
packages/core/src/sync/providers/cheaperinference.tsreads the same/v1/modelsendpoint and reusesfactorBaseModelfrom the OpenRouter module to keep every entrybase_model-ed,packages/core/src/sync/index.tsregisters it in theaggregatorsgroup, and.github/workflows/sync-models.ymlpasses aCHEAPERINFERENCE_API_KEYsecret. The endpoint needs a key to answer, so the sync run is a no-op until that secret exists — nothing else in the workflow changes.packages/core/test/cheaperinference.test.tscovers the module with 6 tests, which pass.bun validatepasses on the full tree.