Skip to content

docs: add Google Vertex AI and Gemini gateway guide - #879

Open
rajshah4 wants to merge 5 commits into
mainfrom
codex/google-llm-gateway-guide
Open

rajshah4 wants to merge 5 commits into
mainfrom
codex/google-llm-gateway-guide

Conversation

@rajshah4

@rajshah4 rajshah4 commented Oct 3, 2026

Copy link
Copy Markdown
Member

Adds a Google LLM gateway guide covering Google Cloud Vertex AI and Google AI Studio (Gemini API), with Replicated first and Helm second. Matches the AWS Bedrock and Azure gateway guides in #874 and #872: provider selection, credential storage, gateway routes, model profiles, conversation verification and troubleshooting. Adds navigation, integration and installation links, and LLM index content.

Validation: Vertex Gemini 2.5 Flash passed a gateway request and a completed sandbox conversation on GKE with chart 0.74.0 / OpenHands 1.67.0. Replicated release 0.74.0 also passed managed Vertex inference, terminal file operations and a read-only repository conversation. Gemini API configuration is documented and source-checked, but has not yet passed an end-to-end test in this evaluation.

Checks: Mintlify broken links, navigation JSON, shell syntax, YAML parsing and git diff --check passed.

Source verification: Replicated Google configuration, Vertex gateway generation, and Gemini gateway generation.

@mintlify

mintlify Bot commented Oct 3, 2026 •

Copy link
Copy Markdown

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated
all-hands-ai 🟢 Ready View Preview Oct 4, 2026, 10:34 PM

💡 Tip: Enable Automations to automatically generate PRs for you.

@enyst enyst left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm an AI agent based on Opus 5.5, helping Engel Nyst (@enyst) with project work.

  • Rebase: main removed llms.txt and llms-full.txt in #815 (Mintlify generates them now), so dropping those two files from this PR clears the conflict. #874 adds a card at the same spot in integrations/overview.mdx.
  • Checked: the Replicated field titles and per-line routes (config.yaml#L389-L436, openhands.yaml#L1046-L1054). Also the volumes/volumeMounts/envVars pattern, which matches what Replicated renders, and the openhands/<alias> profile with the :4000 base URL, which matches LITE_LLM_API_URL (_env.yaml#L354-L359).
  • One accuracy point inline: where the Replicated credential ends up. It also affects the opening sentence (lines 9–10).
  • #875 refers to this guide, so merging this one first lets #875 link to it.

to your account and, for Vertex, in your selected location.
4. Save the configuration and deploy the updated version.

The uploaded service-account file is used by the bundled LiteLLM pod. Keep it

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On Replicated, this file doesn't stay in the LiteLLM pod. The installer also copies the service-account JSON into global.agentServerEnv as VERTEXAI_CREDENTIALS, so every sandbox gets it (openhands.yaml#L1017-L1020). Could you say that here, and soften "The Google credential belongs to the bundled LiteLLM gateway" on lines 9–10? The Helm path is described correctly. Admins will want to scope that service account accordingly.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for pointing this out. Updated both the introduction and Replicated section to explain that Vertex credentials also reach sandbox environments. We confirmed this with a boolean-only terminal check on our evaluation instance, without displaying credential values.

This branch was successfully deployed

1 active deployment
staging — ad3f6557 Deployed Oct 4, 2026 by mintlify[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants