Skip to content

feat: add unified LLM API proxy endpoint /v1/chat/completions - #190

Merged
hsluoyz merged 1 commit into
apache:masterfrom
BeiBaiNian:feat-proxy-1.2
Aug 13, 2026
Merged

feat: add unified LLM API proxy endpoint /v1/chat/completions#190
hsluoyz merged 1 commit into
apache:masterfrom
BeiBaiNian:feat-proxy-1.2

Conversation

@BeiBaiNian

Copy link
Copy Markdown
Contributor

This PR is part of issue #179 "make a UI-first LLM API security gateway". The unified proxy endpoint is the routing layer (milestone 1.2): it reads the model field from the request body, matches an enabled channel from the 1.1 Channel table (ordered by Priority), forwards the request to the upstream with the channel's API key injected, and relays the response back, including SSE streaming.

What changed

  • Added controllers/proxy.go: OpenAI-compatible POST /v1/chat/completions endpoint. It parses the body (only model and stream are read; everything else is passed through), looks up the channel via GetChannelByModel, injects Authorization: Bearer *** and forwards the raw body. Non-stream responses are relayed header/status/body as-is; streaming responses are flushed chunk-by-chunk via c.Ctx.ResponseWriter.Write() + Flush(). Errors follow the OpenAI error format: 400 for invalid JSON / missing model / no available channel, 502 for upstream connection failure or misconfigured channel, 504 on upstream timeout. A shared package-level http.Client is reused across requests for connection pooling. This endpoint does NOT require Casdoor authentication (auth is deferred to milestone 1.3 tokens).
  • Modified object/channel.go: Added GetChannelByModel() which queries only status=enabled channels ordered by priority (ascending) and returns the first channel supporting the requested model. Introduced the ErrNoChannelAvailable sentinel error so callers can distinguish "no match" (HTTP 400) from database failures (HTTP 502).
  • Modified routers/filter.go: TransparentStatic now excludes the /v1/ prefix in addition to /api/, so the new endpoint is not swallowed by static file handling and bypasses the Casdoor ApiFilter.
  • Modified routers/router.go: Registered POST /v1/chat/completions.

Verification

  • go build ./... passed.

  • API tests against a local mock upstream all passed:

    • Missing model / invalid JSON / unknown model → 400 with clear OpenAI-format error
    • Disabled channels are skipped; channels with empty base URL → 502
    • Model routing honors Priority (lowest priority wins)
    • Non-stream forwarding returns upstream status/body as-is (200, 429, 500 verified)
    • SSE streaming (stream=true) relays all chunks with text/event-stream + flush
    • Upstream connection failure → 502; slow upstream is waited for and relayed
    • Endpoint is reachable without any Casdoor cookie (contrast: /api/* returns "please sign in first")
  • go test ./... has 3 pre-existing failures unrelated to this PR (Alibaba Cloud RDS client and local MySQL credentials in run/ and sync/ tests).

Note

Per the plan, this endpoint is intentionally unauthenticated for now (auth lands in milestone 1.3). Do not deploy this PR alone to production — it should be merged and released back-to-back with 1.3.

@hsluoyz
hsluoyz merged commit 712cd5d into apache:master Aug 13, 2026
3 checks passed
@BeiBaiNian
BeiBaiNian deleted the feat-proxy-1.2 branch August 13, 2026 13:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants