Skip to content

feat: add The Grid as an ai-proxy provider - #13920

Open
CastilloLuis wants to merge 1 commit into
apache:masterfrom
the-gridai:thegridai/thegrid-provider
Open

CastilloLuis wants to merge 1 commit into
apache:masterfrom
the-gridai:thegridai/thegrid-provider

Conversation

@CastilloLuis

Copy link
Copy Markdown

Description

Adds The Grid as an ai-proxy provider, and adds the opt-in redirect following in the HTTP transport that it needs in order to work through the gateway.

The Grid is an OpenAI-compatible inference API whose model ids are market instruments rather than fixed models: a task type and a quality tier (text-standard, code-prime, agent-max), or a lab-scoped market (claude-opus-latest, kimi-latest). It serves both /v1/chat/completions and /v1/responses, so it maps onto the existing openai-chat and openai-responses capabilities with no new protocol.

apisix/plugins/ai-providers/thegrid.lua mirrors openai.lua, including the max_tokens override mapping — verified against the live API rather than assumed:

endpoint field result
/v1/chat/completions max_completion_tokens: 64 200, usage.completion_tokens = 64
/v1/responses max_output_tokens: 64 200, usage.output_tokens = 64, status: incomplete

The routing redirect

The Grid's Consumption API validates a request and then answers 307 with a Location pointing at its routing layer on the same host (api.thegrid.ai/v1/...api.thegrid.ai/r/v1/...), where inference is fulfilled. It is documented and intentional, not an error condition.

lua-resty-http does not follow redirects, so without this change ai-proxy would return the 307 downstream and:

  • ai-rate-limiting would meter zero tokens on every request, since the gateway never sees a body carrying usage;
  • ai-proxy-multi fallback would never trigger, because every request looks successful;
  • the Location URL is self-authenticating — I confirmed it returns 200 when replayed with no Authorization header — so the gateway would be turning its configured provider credentials into a bearer URL handed to the caller.

So the hop is followed inside the transport instead:

  • A provider opts in with follow_redirects = true. Every existing provider is untouched and keeps today's behaviour.
  • One hop, 307/308 only. 301/302/303 are deliberately excluded — they permit rewriting the method to GET, which would silently drop the prompt.
  • Method, body and headers are replayed unchanged. The redirect's own query string replaces the original when it has one, so auth.query providers stay authenticated when it does not.
  • Same-origin only. A cross-origin hop would replay the provider credentials in Authorization against a host named by the response rather than by the configuration, so it is refused and surfaces as a request error.

I am happy to split the transport change into its own PR if you would rather review it separately — the provider is not functional without it, which is why they are together here.

Tests

t/plugin/ai-proxy-thegrid.t covers the provider through both ai-proxy and ai-proxy-multi, rejection of an unknown provider name, the same-origin redirect being followed through to the model output, and the cross-origin redirect being refused. The mock endpoints added to t/lib/server.lua assert that the replay preserved the method, the body, the target query string and the request headers, so a regression that drops any of them fails rather than passing quietly.

I could not run the Test::Nginx suite locally on macOS; it will run here in CI. Locally I verified luacheck is clean on every changed Lua file and that all of them compile.

Which issue(s) this PR fixes:

Fixes #13919

Checklist

  • I have explained the need for this PR and the problem it solves
  • I have explained the changes or the new features added to this PR
  • I have added tests corresponding to this change
  • I have updated the documentation to reflect this change
  • I have verified that this change is backward compatible

The Grid is an OpenAI-compatible inference API. Add it as a provider
module for ai-proxy, ai-proxy-multi and ai-request-rewrite, covering the
openai-chat and openai-responses capabilities, and map the plugin-level
max_tokens override onto max_completion_tokens and max_output_tokens
respectively, matching the openai provider.

The Grid's Consumption API validates a request and then answers with a
307 to its routing layer on the same host, where inference is fulfilled.
Returning that redirect downstream would leave the Plugin blind to the
real response, so ai-rate-limiting would meter zero tokens and
ai-proxy-multi would never fall back. Add opt-in redirect following to
the HTTP transport for this: a provider sets follow_redirects, and one
307/308 hop is replayed with the method, body and headers preserved.
Only same-origin redirects are followed, since a cross-origin hop would
replay the provider credentials against a host named by the response
rather than by the configuration.

Signed-off-by: CastilloLuis <luisedcastillog@gmail.com>
@CastilloLuis

Copy link
Copy Markdown
Author

A note on verification, since I said above that I could not run Test::Nginx locally.

To avoid leaving the transport change resting only on luacheck, I drove the real _M.request in ai-transport/http.lua under LuaJIT with the HTTP client, core and socket.url stubbed, and asserted the redirect behaviour directly. All nine cases pass:

case asserted
same-origin 307 followed; method, body, Authorization and target query all preserved on the replay
cross-origin host refused, request errors
port differs refused
scheme differs refused
relative Location treated as same-origin, followed
redirect carries no query original query is kept, so auth.query providers stay authenticated
no Location header errors rather than passing through silently
provider without follow_redirects unchanged: the 307 is returned as-is, one request made
non-redirect status untouched, one request made

That harness is a local scratch tool, not something I have added to the PR — the committed coverage is t/plugin/ai-proxy-thegrid.t, which still needs a run of the real suite. It looks like the workflows are waiting on maintainer approval, so please do kick them off; if anything in that file is wrong I would rather find out now than after review time is spent on it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: As a user, I want to route to The Grid through ai-proxy, so that I can consume market-priced inference from APISIX

1 participant