Repository navigation
Harden the gateway: live key reload, honest errors, real 429 rotation, keyed access - #1
Merged
Merged
Conversation
added 4 commits
October 6, 2026 21:45
The running server kept the in-memory key list from startup, so a key issued by the CLI (a separate process) only started working after a service restart. mtime/size of keys.json is now checked on every store operation and the file re-read when it changes.
- A stream that never started returned without writing anything, so net/http sent an empty 200 that hid the failure. It now answers 502 with a JSON error. - Rotate cleared the retry-after limit immediately and only relabeled the lane country, so the same limited exit was picked again. The limit is kept until the exit is actually replaced; the OnRotate hook (tor respawn) brings up a fresh exit in the background and RotateDone lifts the limit. Rotating lanes leave the pool meanwhile. - tor: track each lane process (Lane.SetProcess) and add Respawn.
…y cap - requireKey / LANVELLO_REQUIRE_KEY refuses keyless /v1 requests even when no key exists yet, so a public deployment never falls back to open mode. - Native /v1/responses with stream:false now folds the terminal event into one response object. The stream flag was read after fingerprintResponses forced stream=true on the same map, so every non-stream request got raw sse. - Request bodies over the 128MB cap are refused with 413 instead of silently truncated.
A model whose upstream endpoint returned 503 on every lane made the retry loop spin until the whole 90s wait budget and then answer 'no lane available', hiding the real error (seen live with opencode's exo-free: Endpoint is unavailable). Retryable far-end statuses are now capped at two full passes over the lanes and the last drained answer is relayed, so the real status and message reach the caller in seconds. The streaming path keeps the last retryable response for the same case instead of ending as a silent empty 200.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A round of gateway correctness fixes found while running it in production.
Fixes
keys: live reload (
internal/keys)keys.jsonchanges made by the CLI (lanvello key add/key revoke, a separate process) are picked up on the next request instead of requiring a restart. mtime/size is checked per operation and the file is re-read when it changes; a torn read keeps the in-memory list.upstream: honest stream errors (
internal/upstream)A stream that never started (no lane up, upstream unreachable) now answers 502 with the real error. Returning without writing anything let net/http send an empty 200 that hid the failure.
upstream: bounded retries for a plainly down endpoint
Persistent 502/503/504 from the far end no longer spins until the whole wait budget (90s) and then answers an unhelpful "no lane available". After two full passes over the lanes the real status and body are relayed in seconds; the streaming path keeps the last retryable answer too, instead of ending as an empty 200. (Found live: a model whose endpoint was down returned 503 "Endpoint is unavailable" and the gateway hid it behind a 90s hang.)
lanes: real 429 rotation (
internal/lanes,internal/tor)Rotateno longer clears the retry-after limit immediately (which handed the same limited exit back to the next request) and no longer only relabels the country: theOnRotatehook respawns the lane's tor in the background for a fresh exit, the lane stays out of the pick pool while rotating, andRotateDone/RotateFailedbring it back.server: stream:false native responses (
internal/server)/v1/responseswithstream:falsereturns one aggregated response object. The stream flag was read after fingerprinting forced it true on the same map, so every non-stream request previously got raw SSE back.server/config: requireKey (
internal/server,internal/config)Opt-in (
requireKeyin the config file orLANVELLO_REQUIRE_KEY=1) refuses keyless/v1requests even when no key exists yet, so a public deployment never falls back to open mode.server: 413 for oversized bodies
Request bodies over the 128MB cap are refused with
413instead of being silently truncated.Testing
go test ./...is green; every fix has a dedicated test (external key changes, stream 502, bounded 503 + relayed body, rotation lifecycle, non-stream responses aggregation, requireKey, 413).