fix(providers): repair the AI/ML API integration — 6 stale models to 353 live, and 400s on every message - #1
Open
Lookoff-AIMLAPI wants to merge 5 commits into
Open
Conversation
… catalog schema
AI/ML API has been discovering zero models since its catalog changed shape
upstream, so every user of the provider saw the 6-entry static seed instead of
the live list — and four of those six ids no longer exist, so most of what was
offered 404'd on first use.
Three independent defects, each verified against the live catalog on 2026-09-03
(936 rows, 353 of them chat):
- The parser tested `Array.isArray(data)` against a response that is now the
OpenAI-style envelope `{ "object": "list", "data": [...] }`. Every row was
discarded before any filter ran, so discovery returned [] and the route took
its local_catalog branch. Sibling entries (thebai, openrouter) already unwrap
the envelope themselves; this one never did.
- The chat filter looked for `type === "chat-completion"`, a spelling the
catalog no longer publishes. The current value is `openai/chat-completions`
and the old one matches 0 of 936 rows. Both are accepted now, so a further
rename degrades to "some models missing" rather than "no models at all".
- The `chat.length ? chat : all` fallback would, once the type filter stopped
matching, have published 583 video/image/TTS/batch ids into a chat model
picker. Dropping it is what makes defect 2 observable instead of silent.
The static seed replaces the four dead ids (claude-3-5-sonnet-20241022,
gemini-1.5-pro, meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo,
mistral-large-latest — absent from the catalog as both id AND alias) with six
current ones. Each replacement was checked twice, because neither check alone is
sufficient: it must appear on the catalog's chat surface as an id or an alias,
AND answer 200 to a real chat completion. `llama-3.3-70b-versatile` is why —
it is listed as a chat model, advertises tools and structured output, and still
404s "model does not exist" on inference.
The seed keeps the previous list's UNPREFIXED spelling. A `vendor/model` id in
this registry is matched by parseModel() as an exact model id, which makes it
report provider = null: seeding "anthropic/claude-sonnet-4-6" here stops that
string resolving to the anthropic provider anywhere in the app, collapsing its
context window from 1M to the 128k default and dropping the provider prefix that
diegosouzapw#8716 exists to preserve. That regression is not hypothetical — an earlier draft
of this change caused it, and combo-target-token-limit-8716 caught it.
Anthropic ids use the dotted spelling. The catalog carries `claude-sonnet-4-6`
and `claude-sonnet-4.6` as separate entries and the dashed one advertises only
`streaming` in its capabilities, while the dotted one advertises tools, vision,
reasoning and structured output. The same split exists for claude-opus-4.7/4.8.
Both existing tests passed throughout the outage because their mocks fed the
parser a bare array of `chat-completion` rows — a shape the endpoint has not
returned since the change. They now assert the real envelope, which is what
turns this from a green suite over a broken provider into a regression guard.
…se its own brand name
OmniRoute's AI/ML API traffic currently reaches the gateway untagged, so none of
it is attributable to this project. Four headers fix that, using the mechanism
the registry already has rather than any new machinery: openrouter, orcarouter,
cline and gitlawb all declare a `headers` block on their registry entry, and
BaseExecutor.buildHeadersPreamble() spreads it into a freshly built map per
request. That gives the two properties this needs for free — the headers are
scoped to this provider's own dispatch, so they can never ride a request to
another upstream, and the shared registry constant is never mutated. The
regenerated translate-path golden shows exactly that: the four headers appear
under `aimlapi` and under no other provider.
HTTP-Referer and X-Title follow the OpenRouter convention and name the CALLING
application, so they point at OmniRoute's own repository, not at the gateway
being called.
The partner id is asserted against `^part_[A-Za-z0-9]{1,64}$` in a test because
a malformed one has no runtime symptom: the gateway accepts the request either
way and simply records the usage as untagged, so a typo would cost attribution
silently and forever.
The dashboard label becomes `aimlapi.com`, the name the provider ships under.
The machine identifier (`aimlapi`) and alias (`aiml`) are untouched — those are
what existing user configs and stored connections key on, and renaming them
would break them.
…to AI/ML API
A second live defect, independent of the discovery bug and one that would have
survived fixing it: AI/ML API validates optional fields with a strict schema and
answers 400 "Expected number, received null" when a field arrives as a literal
`null`, instead of reading null as "unset". OmniRoute relays the caller's body
verbatim, and the OpenAI SDKs serialise an unset optional as `null` — so an SDK
client that never sets temperature still puts `temperature: null` on the wire and
gets a 400 on every single message. Repairing discovery alone would have handed
those users a 353-model picker attached to a provider that still could not answer.
Field-by-field sweep against POST /v1/chat/completions on 2026-09-03, read off
`details[].path` in the 400 bodies (the top-level `message` is generic and names
no field, which is most of why this is hard to diagnose from a log):
400 on null, every model seed, tools, tool_choice, response_format, stream,
stream_options, parallel_tool_calls, max_tokens,
max_completion_tokens, reasoning_effort
400, model-dependent temperature, top_p — 400 on claude-sonnet-4.6 and
deepseek-chat, 200 on gpt-5, so an integration
smoke-tested only against gpt-5 looks healthy
200 on null (untouched) stop, presence_penalty, frequency_penalty, n, user,
logprobs, logit_bias, top_logprobs, metadata
Expressed as a `dropIfNull` rule in the existing STRIP_RULES table, whose stated
purpose is "params a given provider/model rejects upstream" — no new mechanism,
and it stays provider-scoped. Dropping is the correct reading: every field listed
means "unset" when the caller sends null.
`dropIfNull` fires only on a literal null, which is the whole point: a plain
`drop` would also discard a deliberate `temperature: 0`, `stream: false`,
`parallel_tool_calls: false` or `tools: []`. Those are covered by a test.
Pins aimlapi.com in the dashboard provider grids for this fork only. It is isolated in a single commit, with its guard test, so it can be dropped whole before anything is offered upstream: a placement request without a partnership behind it is the wrong thing to put in front of a maintainer, and it has no business travelling with the functional repair. The rank map is the only lever that reaches the rendered order. filterConfiguredProviderEntries() sorts every grid by display name and docs/reference/PROVIDER_REFERENCE.md is generated alphabetically, so the provider catalog's key order never surfaces to a user and is left untouched. Ranked 3, below Kimi (1) and Cheaper Inference (2). Those two encode an explicit operator decision dated 2026-07-31 and are asserted in featured-providers-rank.test.ts; reordering them here would mean editing someone else's stated commitment to move ourselves above it. Rank 3 still pins us above every other aggregator in the grid. No supporter chip is added. ProviderCard renders those from per-sponsor predicates tied to the "Open Source Friend" programme, and rendering one would assert a sponsorship that does not exist.
The placeholder part_omniroute was a readable stand-in chosen before the partner was registered. Registration mints the id server-side, so the real value is part_T2iNtMuQ3JBmEPwyOKCLOxaP. A wrong or unknown partner id is accepted with a 200 and silently not attributed, so this would not have surfaced at runtime.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The
aimlapiprovider is fully wired in this repo and has been showing users 6 models, 4 ofwhich no longer exist upstream. It should be showing 353. And once discovery is fixed, most
of those models still 400 on every message, for a second and unrelated reason. This PR fixes both,
and tags the resulting traffic with partner attribution.
Defect 1 — discovery returns zero models
Verified against the live catalog on 2026-09-03 (
GET https://api.aimlapi.com/models— public, nokey; 936 rows, 353 of them chat):
discovery/providerModelsConfig.tsArray.isArray(data)against a response that is now the OpenAI-style envelope{"object":"list","data":[…]}. Always false → every row discarded → discovery returned[].type === "chat-completion"; the catalog publishesopenai/chat-completions. The old spelling matches 0 of 936 rows.chat.length ? chat : allfall-through would, once 1b started missing, have published 583 video/image/TTS/batch ids into a chat model picker — which is what kept 1b invisible.1a alone is fatal:
parseResponsereturns[],buildApiDiscoveryResponsetakes itsdiscoveredModels.length > 0false branch, and the route re-derives from the static seed. The usersees exactly the 6 seed entries.
passthroughModels: truemeans a hand-typed id still routes; it isthe browsable list that was dead.
parseResponseagainst the real live catalog: 0 models before, 353 after.Both the response shape and the
typevocabulary changed upstream without a deprecation path, sothe parser now accepts both spellings and unwraps either shape — a further rename degrades to
"some models missing" rather than "the provider has no models".
Defect 2 — 400 on every message, even with a full model list
Independent of the above, and it would have survived the discovery fix: AI/ML API validates
optional fields with a strict schema and answers 400 "Expected number, received null" when a
field arrives as a literal
null, rather than reading null as "unset". OmniRoute relays thecaller's body verbatim, and the OpenAI SDKs serialise an unset optional as
null— so an SDKclient that never sets
temperaturestill sendstemperature: nulland gets a 400 on every turn.Fixing discovery alone would have handed those users a 353-model picker attached to a provider that
still could not answer.
Field-by-field sweep against
POST /v1/chat/completions, 2026-09-03:nullseed,tools,tool_choice,response_format,stream,stream_options,parallel_tool_calls,max_tokens,max_completion_tokens,reasoning_efforttemperature,top_p— 400 onclaude-sonnet-4.6anddeepseek-chat, 200 ongpt-5stop,presence_penalty,frequency_penalty,n,user,logprobs,logit_bias,top_logprobs,metadataTwo things make this expensive to diagnose from the outside:
temperature/top_paremodel-dependent, so an integration smoke-tested against
gpt-5looks healthy; and the 400 body'stop-level
messageis generic — onlydetails[].path/.reasonname the offending field. Thelist above was read off
details[].path.Fixed as a
dropIfNullrule in the existingSTRIP_RULEStable (translator/paramSupport.ts),whose stated purpose is "params a given provider/model rejects upstream". No new mechanism, and it
stays provider-scoped. Dropping is the correct reading — every field listed means "unset" when the
caller sends null.
dropIfNullfires only on a literalnull, which is the point: a plaindropwould also discard a deliberate
temperature: 0,stream: false,parallel_tool_calls: falseortools: [].Defect 3 — the seed model ids
Every id was checked twice, because neither check alone is sufficient: present on the catalog's
chat surface as an id OR alias, and answering 200 to a real
POST /v1/chat/completions.gpt-4ogpt-5claude-3-5-sonnet-20241022claude-sonnet-4.6gemini-1.5-progemini-2.5-prometa-llama/Meta-Llama-3.1-70B-Instruct-Turboglm-5deepseek-chatdeepseek-chatmistral-large-latestmistral-largeThree things this turned up that are worth knowing before anyone edits this list again:
llama-3.3-70b-versatileis listed as a chatmodel, advertises tools and structured output, and still returns
404 "The model does not exist or you do not have access to it"on inference. It was in anearlier draft of this seed purely on catalog evidence.
vendor/modelid in this registry is matched byparseModel()as an exact model id, which then reportsprovider = null. An earlier draft seededanthropic/claude-sonnet-4-6and that string stopped resolving to the anthropic providerapp-wide — context window collapsed from 1M to the 128k default, and combo targets lost the
provider prefix.
combo-target-token-limit-8716caught it. The previous seed's bare ids wereright about this.
claude-sonnet-4-6andclaude-sonnet-4.6as separate entries; the dashed one advertises onlystreamingincapabilitieswhile the dotted one advertises tools, vision, reasoning and structured output.Same split for
claude-opus-4.7/4.8— 3 such pairs out of 7.Why the tests did not catch any of this
Both existing discovery tests fed the parser a bare array of
chat-completionrows — a shapethe endpoint has not returned since the schema change. They passed for the entire outage. They now
assert the real envelope and the real type value, plus new cases for the legacy spellings and for
the "never fall through to non-chat rows" rule.
Attribution
Four headers on the
aimlapiregistry entry, using the mechanism already present foropenrouter/orcarouter/cline/gitlawb:BaseExecutor.buildHeadersPreamble()spreadsconfig.headersinto a freshly built map per request,so they are scoped to this provider's dispatch (they cannot ride a request to another upstream) and
the shared registry constant is never mutated — both asserted, and the regenerated
translate-pathgolden shows the four headers under
aimlapiand under no other provider.HTTP-Referer/X-Titlefollow the OpenRouter convention and name the calling app, not thegateway. The partner id is asserted against
^part_[A-Za-z0-9]{1,64}$, because a malformed id hasno runtime symptom — the gateway accepts the request and records the usage as untagged.
Display label is now
aimlapi.com. The machine id (aimlapi) and alias (aiml) are unchanged;those are what existing configs and stored connections key on.
The last commit is fork-only
chore(aimlapi): fork-only placement — do not send upstreampins aimlapi.com in the dashboardgrids. Drop that commit before offering anything upstream. It is isolated and carries its own
guard test so it comes out in one piece.
It ranks aimlapi 3rd, below the two sponsors the operator ranked explicitly on 2026-07-31, and
adds no supporter chip — rendering one would assert an "Open Source Friend" sponsorship that
does not exist. The provider catalog's key order is untouched:
filterConfiguredProviderEntries()sorts every grid by display name and
PROVIDER_REFERENCE.mdis generated alphabetically, so therank map is the only lever that reaches the rendered order.
Related Issues
response shape have since gone stale)
Validation
npm run lint— clean on every changed filenpm run check:docs-all— exit 0release/v3.8.51Unit suite — same command, same host, before and after:
release/v3.8.51)+22 tests, +22 passing — the new and updated files. The failing set is the baseline set plus
exactly one, and every member of it is pre-existing and unrelated to providers: event-loop timing
(
9147-catalog-eventloop-yield), Redis-dependent suites (quota-redis-store,rate-limit-manager), an unreleased DB handle (combo-context-overflow-compression-probe), andshell/binary-manager tests. The base branch is independently confirmed red:
diegosouzapw#12518.
The one extra is
resolve-npm-entry→ "live environment: the real node install can resolvenpm-cli.js". It is a host-layout probe of
scripts/build/resolveNpmEntry.ts, a file this PR doesnot touch: Homebrew keeps npm at
/opt/homebrew/lib/node_modules/npm/bin/npm-cli.jswhile theresolver derives
/opt/homebrew/Cellar/node/26.7.0/lib/node_modules/…fromprocess.execPath. Itfails deterministically on this machine (2/2 standalone runs) and will not reproduce on CI's runner
layout.
Vitest UI:
ProviderIcon-icon-url,providerCardKimiPartnerAccent,providerPageHeaderKimiPartnerLink— 3 files / 101 tests pass.Production build:
npm run buildfails on this branch and on a pristine base for a reasonneither introduced nor touched here:
docs/reference/REMOVED_PROVIDERS.mdhas no YAML frontmatter,so the MDX pipeline throws
title: Invalid input: expected string, received undefined. It is theonly file under
docs/reference/missing frontmatter, it arrived in diegosouzapw#12478 one commit before thebase tip, and it is the "Turbopack build failed" hard failure in diegosouzapw#12518. With that single
pre-existing defect temporarily patched out locally,
npm run buildexits 0 with this branch'schanges. Not fixed here, per the base-red rule that such fixes belong in their own PR.
Live verification — real inference through the integration
Not a mock and not a raw curl: through
getDefaultExecutor("aimlapi")→DefaultExecutor.execute(),the registry entry and header path this PR changes.
Tests Added Or Updated
tests/unit/aimlapi-catalog-repair.test.ts(new) — seed ids, known-dead-id blocklist (nowincluding the catalog-listed-but-404 id and the dashed Anthropic spelling), no-vendor-prefix
guard, endpoint shape
tests/unit/aimlapi-attribution-headers.test.ts(new) — the four headers, partner-id regex,provider scoping, no-mutation of the shared constant, display label
tests/unit/aimlapi-null-param-strip.test.ts(new) — all 12 rejected fields dropped, falsy-but-realvalues (
temperature: 0,stream: false,tools: []) preserved, the 9 fields the upstream acceptsas null left alone, the two lists asserted disjoint, no leakage to other providers
tests/unit/aimlapi-fork-placement.test.ts(new, fork-only commit) — dashboard pintests/unit/provider-models-discovery-split.test.ts(updated) — envelope, both type spellings,bare-array back-compat, no fall-through to non-chat rows
tests/unit/provider-models-route.test.ts(updated) — mock now returns the real envelopetests/unit/executors-strip-unsupported-params.test.ts(updated) — the STRIP_RULES shape guardnow accepts a
dropIfNull-only ruleCoverage Notes
Four production files change (
providerModelsConfig.ts,registry/aimlapi/index.ts,apikey/gateways.ts,translator/paramSupport.ts; plusfeaturedProviders.tsin the fork-onlycommit). Each is covered above. The two edited discovery tests are RED against the pre-change parser
(it returns
[]for the real envelope) and the null-param tests are RED against the pre-changerule table, so these are genuine regression guards rather than restatements.
Reviewer Notes
models missing" rather than the failure mode being fixed.
dropIfNullrule is deliberately provider-scoped. A global null-strip is arguably correct(the OpenAI schema treats null as unset) but that is a behaviour change for 355 providers and
belongs in its own PR with its own evidence. Other OpenAI-compatible upstreams in this registry
may well have the same intolerance; nothing here surveys them.
discovery-parser and executor level plus live upstream calls. Streaming was not exercised live,
only the non-streaming path.
GET /v1/modelsreturns 200 for any key, including a bogus one,so the common "validate the key by GETting /models" pattern silently accepts garbage here. This
repo already dodged that trap for OpenRouter via
testKeyModelsUrl(
registry/openrouter/index.ts:12-16); AI/ML API exposes no equivalent authenticated endpoint topoint at, so none is added.