Skip to content

docs, examples: OpenShell and Substrate on one mesh, on your Kubernetes or on GKE - #747

Merged
aojea merged 1 commit into
google:docs-openshell-sandbox-guidefrom
aojea:openshell-substrate-guide
Oct 18, 2026
Merged

aojea merged 1 commit into
google:docs-openshell-sandbox-guidefrom
aojea:openshell-substrate-guide

Conversation

@aojea

@aojea aojea commented Oct 18, 2026 •

Copy link
Copy Markdown
Collaborator

Stacked on #655.

A tutorial, site/content/docs/guides/openshell-and-substrate.md: how to put NVIDIA OpenShell sandboxes and Agent Substrate actors on one Agent Mesh, on Kubernetes you manage or on GKE, and how to move a workload between the two runtimes with a policy change. One section per deployment with the commands to run and the output to expect; the hub in the cluster or on Cloud Run; both runtimes on one mesh; moving a workload; a Google API (Gemini on Vertex AI, an agent on Agent Engine) from either runtime through an egress node with a federated credential; clean-up. The federation setup is in #749.

Examples:

  • development/examples/openshell-kubernetes/ (new): OpenShell Helm chart 0.1.4, Agent Sandbox v1.0.7, SPIRE 0.29.0 with a ClusterSPIFFEID on the supervisor pods and provider_spiffe_workload_api_socket_path. KUBE_CONTEXT for kind or GKE, HUB=local|<url>, OPENSHELL_GATEWAY per cluster. On GKE: Ubuntu node pool (Landlock), allowed_ips from the edge Service's ClusterIP, SPIRE OIDC discovery on a reserved address. The provider profile allows /mesh/** on the edge node.
  • development/examples/substrate-actor/: remote-hub mode (provider signs with an operator key; discovery and JWKS as static files), KUBE_CONTEXT and CALC_MCP_IMAGE for a cluster from Substrate's tools/setup-gcp bootstrap.
  • Both: make gemini NAME=<workload> GCP_PROJECT=<project> [AGENT_ENGINE=<id>] runs, from inside the sandbox or the actor, generateContent on a Gemini model (allowed), a GET on the model and a POST to another Vertex AI path (both refused at the serving node), and a query to an agent on Agent Engine.
  • All three examples pin v0.1.1 and select their own gateway in the openshell CLI. The Kubernetes ones: --metrics-addr and a /readyz readiness probe on the nodes; port-forward helpers refuse a local port that is in use; down keeps the Service the Substrate egress gateway was installed with. The Docker example gets allowed_publish on the node role.

Every section was run on the released images: Agent Mesh v0.1.1, OpenShell 0.1.3/0.1.4, Substrate v0.5.0, GKE 1.36.4 Dataplane V2, agentmesh-one on Cloud Run with operator-owned keys. All four deployments; the cross-cluster call in both directions with the audit line at the serving node; cutover and rollback by policy with the latencies as measured; Gemini and Agent Engine from an OpenShell sandbox and from a Substrate actor, with the two refusals and Google's GenerateAccessToken audit entry.

Related: agent-substrate/substrate#2827 (DNS relay REFUSED vs search-list resolvers; the example uses absolute names).

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces Kubernetes examples and guides for integrating NVIDIA OpenShell sandboxes and Agent Substrate actors onto a single Agent Mesh, including support for GKE and Cloud Run deployments. The review feedback highlights a bug in the bind.py script where usage instructions are not printed due to missing docstrings, and security issues in both Makefiles where sensitive tokens are generated with insecure, world-readable permissions (umask 022) instead of owner-only permissions (umask 077).

Comment on lines +1 to +5
#!/usr/bin/env python3
# Adds a sandbox's SPIFFE identity to the sandbox role in policy.json.
#
# bind.py <member> e.g. user:spiffe://openshell.local/openshell/sandbox/<id>
# bind.py --remove <member>

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The script uses sys.exit(__doc__) to print usage instructions on invalid arguments. However, the usage instructions at the top of the file are written as comments (#) rather than a module-level docstring ("""). Consequently, __doc__ resolves to None, causing the script to exit silently with a success status (0) instead of displaying the usage guide and returning an error status.

Suggested change
#!/usr/bin/env python3
# Adds a sandbox's SPIFFE identity to the sandbox role in policy.json.
#
# bind.py <member> e.g. user:spiffe://openshell.local/openshell/sandbox/<id>
# bind.py --remove <member>
#!/usr/bin/env python3
"""Adds a sandbox's SPIFFE identity to the sandbox role in policy.json.
bind.py <member> e.g. user:spiffe://openshell.local/openshell/sandbox/<id>
bind.py --remove <member>
"""

Comment on lines +239 to +245
mkdir -p run && umask 022 && printf 'mesh_tok_%s' "$$(head -c 16 /dev/urandom | od -An -tx1 | tr -d ' \n')" > $@

run/admin-token:
mkdir -p run && umask 022 && printf 'mesh_adm_%s' "$$(head -c 16 /dev/urandom | od -An -tx1 | tr -d ' \n')" > $@

run/api-token:
mkdir -p run && umask 022 && head -c 16 /dev/urandom | od -An -tx1 | tr -d ' \n' > $@

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

security-medium medium

The generated tokens (join-token, admin-token, and api-token) contain highly sensitive credentials. Using umask 022 creates these files with world-readable permissions (0644). It is highly recommended to use umask 077 instead, so that the generated files are only readable by the owner (0600).

	mkdir -p run && umask 077 && printf 'mesh_tok_%s' "$$(head -c 16 /dev/urandom | od -An -tx1 | tr -d ' \n')" > $@

run/admin-token:
	mkdir -p run && umask 077 && printf 'mesh_adm_%s' "$$(head -c 16 /dev/urandom | od -An -tx1 | tr -d ' \n')" > $@

run/api-token:
	mkdir -p run && umask 077 && head -c 16 /dev/urandom | od -An -tx1 | tr -d ' \n' > $@

Comment on lines +119 to +125
mkdir -p run && umask 022 && printf 'mesh_tok_%s' "$$(head -c 16 /dev/urandom | od -An -tx1 | tr -d ' \n')" > $@

run/admin-token:
mkdir -p run && umask 022 && printf 'mesh_adm_%s' "$$(head -c 16 /dev/urandom | od -An -tx1 | tr -d ' \n')" > $@

run/api-token:
mkdir -p run && umask 022 && head -c 16 /dev/urandom | od -An -tx1 | tr -d ' \n' > $@

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

security-medium medium

The generated tokens (join-token, admin-token, and api-token) contain highly sensitive credentials. Using umask 022 creates these files with world-readable permissions (0644). It is highly recommended to use umask 077 instead, so that the generated files are only readable by the owner (0600).

	mkdir -p run && umask 077 && printf 'mesh_tok_%s' "$$(head -c 16 /dev/urandom | od -An -tx1 | tr -d ' \n')" > $@

run/admin-token:
	mkdir -p run && umask 077 && printf 'mesh_adm_%s' "$$(head -c 16 /dev/urandom | od -An -tx1 | tr -d ' \n')" > $@

run/api-token:
	mkdir -p run && umask 077 && head -c 16 /dev/urandom | od -An -tx1 | tr -d ' \n' > $@

…es or on GKE

A tutorial, site/content/docs/guides/openshell-and-substrate.md: how to
put NVIDIA OpenShell sandboxes and Agent Substrate actors on one Agent
Mesh, each runtime on Kubernetes you manage or on GKE, with the hub in
the cluster or on Cloud Run; how to move a workload between the two
runtimes with a policy change; and how a workload on either runtime
calls Gemini on Vertex AI and an agent on Agent Engine through an egress
node with a federated credential. One section per deployment, with the
commands to run and the output to expect. Every section was run on
Agent Mesh v0.1.1, OpenShell 0.1.3 and 0.1.4, Substrate v0.5.0 and
GKE 1.36.4.

development/examples/openshell-kubernetes is new: OpenShell from its
Helm chart (0.1.4) with Agent Sandbox v1.0.7 and SPIRE 0.29.0, one
ClusterSPIFFEID on the supervisor pods and one chart value; KUBE_CONTEXT
for kind or GKE, HUB=local or a URL, OPENSHELL_GATEWAY per cluster. On
GKE the sandbox node pool is Ubuntu (Landlock), allowed_ips is rendered
from the edge Service's ClusterIP, and SPIRE's OIDC discovery is exposed
on a reserved address as the issuer. The provider profile allows the
/mesh/** path on the edge node, which a request to an egress destination
served by another node uses.

development/examples/substrate-actor joins a hub outside the cluster:
the provider signs with a key the operator keeps and its discovery
document and JWKS are static files at a public URL; KUBE_CONTEXT and
CALC_MCP_IMAGE put the mesh on a cluster from Substrate's own tooling.

Both Kubernetes examples get make gemini: from inside the sandbox or the
actor, generateContent on a Gemini model (allowed), a GET on the model
and a POST to another Vertex AI path (both refused at the serving node),
and with AGENT_ENGINE=<id> a query to an agent on Agent Engine.

All three examples pin v0.1.1 and select their own gateway in the
openshell CLI. The Kubernetes ones: the nodes serve /readyz through
--metrics-addr and the manifests probe it, so rollout status waits for
a node on the mesh; the port-forward helpers refuse a local port that
is in use; the Substrate down keeps the Service the egress gateway was
installed with. The Docker example gets allowed_publish on the node
role.
@aojea
aojea force-pushed the openshell-substrate-guide branch from 4e64e96 to b180d9c Compare October 18, 2026 17:41
@aojea
aojea merged commit 01aad62 into google:docs-openshell-sandbox-guide Oct 18, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant