Skip to content

docs: add AWS Bedrock gateway guide for Helm and Replicated - #874

Open
rajshah4 wants to merge 6 commits into
mainfrom
codex/bedrock-llm-gateway-docs
Open

rajshah4 wants to merge 6 commits into
mainfrom
codex/bedrock-llm-gateway-docs

Conversation

@rajshah4

@rajshah4 rajshah4 commented Oct 2, 2026

Copy link
Copy Markdown
Member

Adds an AWS Bedrock guide for both Replicated and licensed Helm installations. The Helm path explains how to give the bundled LiteLLM pod a scoped EKS IAM role, configure a Bedrock inference-profile route, select it in OpenHands, and verify a sandbox conversation. It links the guide from the enterprise integration overview, Helm install guide, and existing VM Admin Console configuration without removing the Replicated instructions.

Validation:

  • mint broken-links and git diff --check passed.
  • On OpenHands Enterprise Helm chart 0.71.1 in EKS, the LiteLLM Bedrock route answered a gateway request and a Canvas conversation selected the Bedrock profile, ran pwd in a sandbox, and returned /workspace/project.

This is a companion to the Azure gateway guide in #872. The Bedrock example is based on the documented Helm values and an IRSA-backed EKS test; other Kubernetes credential mechanisms are mentioned but were not tested here.

@mintlify

mintlify Bot commented Oct 2, 2026 •

Copy link
Copy Markdown

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated
all-hands-ai 🟢 Ready View Preview Oct 8, 2026, 12:38 PM

💡 Tip: Enable Automations to automatically generate PRs for you.

@enyst enyst left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm an AI agent based on Opus 5.5, helping Engel Nyst (@enyst) with project work.

  • Rebase: main removed llms.txt and llms-full.txt in #815 (Mintlify generates them now), so dropping those two files from this PR clears the conflict. #879 adds a card at the same spot in integrations/overview.mdx, and #872/#879 change the same docs.json line.
  • Checked: the Replicated fields and per-line bedrock/<id> routes (openhands.yaml#L1091), and the litellm-helm.serviceAccount keys.
  • Two points inline. The credential claim on line 9 doesn't hold for Replicated, and the profile settings differ from #872 and #879.


OpenHands Enterprise can use Amazon Bedrock through its bundled LiteLLM
gateway. Configure the gateway according to how you installed OpenHands.
The IAM identity that calls Bedrock belongs to the **LiteLLM pod**, not to an

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This holds for the Helm IRSA setup, but not for Replicated:

  • With Access Key + Secret, the installer also copies the keys into global.agentServerEnv, which is the sandbox environment (openhands.yaml#L1074-L1077).
  • With EC2 Instance Profile and a hop limit of 2, any pod on the node can reach IMDS, sandboxes included. The chart ships no NetworkPolicy.

Could you limit this sentence to the Helm path, or describe the Replicated behavior? Admins may scope IAM permissions based on it.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for catching this distinction. Updated the introduction to limit the IAM statement to the Helm IRSA example, and added the Replicated credential behavior separately. The note now explains that static AWS keys also reach sandbox environments and that an EC2 response hop limit of 2 does not provide gateway-only credential isolation.

| --- | --- |
| Profile Name | `Bedrock-Haiku-4-5` |
| Custom Model | `litellm_proxy/bedrock-haiku-4-5` |
| Base URL | `http://openhands-litellm.openhands.svc.cluster.local:4000/v1` |

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#872 and #879 use openhands/<alias> with a base URL that exactly matches LITE_LLM_API_URL (no /v1). The app treats that form as the managed proxy: it compares the base URL exactly (saas_settings_store.py#L110-L113) and stores bundled-proxy routes as openhands/<route> (constants.py#L243-L256). The /v1 form evidently works, since you tested it, but the three guides would be less confusing if they matched:

Suggested change
| Base URL | `http://openhands-litellm.openhands.svc.cluster.local:4000/v1` |
| Base URL | `http://openhands-litellm.openhands.svc.cluster.local:4000` |

Then use openhands/bedrock-haiku-4-5 on line 158 and openhands/bedrock-sonnet-4-5 on line 165, and drop "/v1" on line 188.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We validated litellm_proxy/... through working conversations on both Helm and Replicated. Additional testing confirmed that openhands/... works after an explicit profile switch, but the EKS profile-save workflow removed the gateway URL and the initial conversation failed authentication. We’re retaining the validated setup instructions until the replacement workflow passes end to end.

This branch was successfully deployed

1 active deployment
staging — 1633317b Deployed Oct 8, 2026 by mintlify[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants