Repository navigation
Docs: lead with the agentcore_gateway backend; diagram the private Lattice path - #46
Merged
Merged
Conversation
README no longer describes llm-edge as the only way to keep the model key out of containers. It now presents both gateway modes, with agentcore_gateway (AgentCore Gateway in front of LiteLLM, key in the token vault, per-session SigV4) first, and llm-edge plus the fixed NAT EIP as the alternative. The quick start's step 2 branches by backend instead of always storing a key for llm-edge. New diagram docs/images/agentcore-gateway-litellm.svg and an architecture.md section that walks every hop: loopback shim and per-session STS, the gateway interface endpoint, what the inference provider target does (match path and model, swap the key, relay verbatim, no translation), and the managed VPC Lattice private endpoint, including what provider.endpoint and routingDomain each do and the 350-second Lattice idle timeout that hangs long non-streaming calls. The hero diagram caption and deployment.md Option A2 point at it. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Follow-up to #40 and #45.
llm-edgeas the way model traffic is governed. They now present both gateway modes.agentcore_gatewaycomes first: AgentCore Gateway sits in front of LiteLLM, the key is held in the token vault, kernels sign with per-session SigV4, the runtime reaches the gateway over PrivateLink and the gateway reaches LiteLLM over VPC Lattice.llm-edgewith the fixed NAT EIP is listed as the alternative. Step 2 of the quick start now branches by backend instead of always storing a key forllm-edge.docs/images/agentcore-gateway-litellm.svg(animated, light theme), with a matchingarchitecture.mdsection that goes through each hop. Most of this behaviour is not written down in one place in the AgentCore docs yet:model, swaps in the key, and relays the body and SSE stream verbatim, with no protocol translation; only three paths are acceptedprovider.endpointis the name on the publicly trusted certificate and is sent as SNI/Host;routingDomainis the internal ALB DNS that Lattice connects todeployment.mdOption A2 now link to the new diagram and section.Docs only, no code changes.
Test plan
xmllint; rendered headless and checked by eye for clipped or overlapping labels#how-the-agentcore_gateway-backend-reaches-a-private-litellmmatches GitHub's slug for the new heading🤖 Generated with Claude Code