feat: add unified LLM API proxy endpoint /v1/chat/completions - #190
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR is part of issue #179 "make a UI-first LLM API security gateway". The unified proxy endpoint is the routing layer (milestone 1.2): it reads the
modelfield from the request body, matches an enabled channel from the 1.1 Channel table (ordered by Priority), forwards the request to the upstream with the channel's API key injected, and relays the response back, including SSE streaming.What changed
controllers/proxy.go: OpenAI-compatiblePOST /v1/chat/completionsendpoint. It parses the body (onlymodelandstreamare read; everything else is passed through), looks up the channel viaGetChannelByModel, injectsAuthorization: Bearer ***and forwards the raw body. Non-stream responses are relayed header/status/body as-is; streaming responses are flushed chunk-by-chunk viac.Ctx.ResponseWriter.Write() + Flush(). Errors follow the OpenAI error format: 400 for invalid JSON / missing model / no available channel, 502 for upstream connection failure or misconfigured channel, 504 on upstream timeout. A shared package-levelhttp.Clientis reused across requests for connection pooling. This endpoint does NOT require Casdoor authentication (auth is deferred to milestone 1.3 tokens).object/channel.go: AddedGetChannelByModel()which queries onlystatus=enabledchannels ordered bypriority(ascending) and returns the first channel supporting the requested model. Introduced theErrNoChannelAvailablesentinel error so callers can distinguish "no match" (HTTP 400) from database failures (HTTP 502).routers/filter.go:TransparentStaticnow excludes the/v1/prefix in addition to/api/, so the new endpoint is not swallowed by static file handling and bypasses the CasdoorApiFilter.routers/router.go: RegisteredPOST /v1/chat/completions.Verification
go build ./...passed.API tests against a local mock upstream all passed:
model/ invalid JSON / unknown model → 400 with clear OpenAI-format errorstream=true) relays all chunks withtext/event-stream+ flush/api/*returns "please sign in first")go test ./...has 3 pre-existing failures unrelated to this PR (Alibaba Cloud RDS client and local MySQL credentials inrun/andsync/tests).Note
Per the plan, this endpoint is intentionally unauthenticated for now (auth lands in milestone 1.3). Do not deploy this PR alone to production — it should be merged and released back-to-back with 1.3.