Add Helm charts for installing Agent Substrate

[EItanya]

Agent Substrate is currently installed through hack/install- ate.sh / hack/install-ate-kind.sh, which apply raw
manifests, run ko, generate prerequisite secrets, and handle
Kind/GKE-specific differences.

We should add Helm-based installation with two charts:

• agent-substrate-crds: installs only Agent Substrate CRDs.
• agent-substrate: installs the runtime/control-plane
  system.

Splitting CRDs into their own chart follows current Helm
operational best practice: CRDs are cluster-scoped, upgrade-
sensitive, and often managed by cluster/platform admins
separately from namespaced workloads.

Proposed Chart Layout

charts/
  agent-substrate-crds/
    Chart.yaml
    crds/
      ate.dev_actortemplates.yaml
      ate.dev_workerpools.yaml
    README.md

  agent-substrate/
    Chart.yaml
    values.yaml
    templates/
    README.md

Motivation

A Helm-based install would make it easier to:

• Install Agent Substrate without relying on repo-local shell
  scripts.
• Support GitOps tooling such as Argo CD and Flux.
• Manage CRDs independently from runtime workloads.
• Configure images, ValKey, TLS, storage, and observability
  through values.yaml.
• Provide clearer install, upgrade, and uninstall workflows.

Scope

agent-substrate-crds

This chart should install:

• WorkerPool CRD
• ActorTemplate CRD
• Any future ate.dev CRDs

Acceptance criteria:

• helm install agent-substrate-crds ./charts/agent-substrate-
  crds
• CRDs are installed using Helm’s crds/ convention.
• Chart README documents CRD upgrade behavior and uninstall
  caveats.
• Runtime chart does not template or own CRDs directly.

agent-substrate

This chart should install the core system components:

• ate-api-server
• ate-controller
• atelet
• atenet-router
• atenet-dns
• ValKey, optionally
• podcertificate-controller
• RBAC, ServiceAccounts, Services, ConfigMaps, Secrets, and
  projected volumes

The runtime chart should assume CRDs are already installed and
document installing agent-substrate-crds first.

Configuration Requirements

Expose values for:

• Component images, tags, pull policies.
• Internal vs external ValKey.
• Redis/ValKey TLS and auth settings.
• GCS/S3/local storage backend settings.
• Kind/local development profile.
• GKE/cloud profile.
• OTLP and Prometheus configuration.
• Existing secret names for session JWT/CA pools and pod
  certificate CAs.

Bootstrap Secret Handling

The current install script generates prerequisite secrets such
as:

• session-id-jwt-pool
• session-id-ca-pool
• service-dns-ca-pool
• pod-identity-ca-pool
• valkey-ca-certs
• ate-api-server-envvars

The chart should document whether these are:

1. Provided by the user before install.
2. Generated by Helm hooks or bootstrap Jobs.
3. Supported through both modes.

Externally supplied secrets should likely be the production/
default path.

Acceptance Criteria

• helm install agent-substrate-crds ./charts/agent-substrate-
  crds installs CRDs.
• helm install agent-substrate ./charts/agent-substrate -n
  ate-system --create-namespace installs the core system after
  CRDs exist.
• helm uninstall agent-substrate -n ate-system removes runtime
  resources cleanly without deleting CRDs.
• Runtime chart fails clearly or documents behavior when CRDs
  are missing.
• values.yaml covers current hack/install-ate.sh
  configuration.
• Kind/local and cloud/GKE install paths are documented.
• CI validates helm lint and helm template for both charts.
• Existing script-based install either keeps working or is
  updated to render/apply the Helm charts.

Non-Goals

• Packaging demo apps into the core runtime chart.
• Solving production-grade certificate/key rotation in the
  first version.
• Removing the existing development scripts immediately.

Implementation Notes

Current install logic lives in:

• hack/install-ate.sh
• hack/install-ate-kind.sh
• manifests/ate-install/
• manifests/ate-install/generated/
• manifests/ate-install/kind/

The current manifests use ko:// image references. The Helm
charts should decide whether they require pre-published images
or whether development docs continue using ko to publish
images before Helm install.

[BenTheElder]

Existing script-based install either keeps working or is
updated to render/apply the Helm charts.

I don't think we should make helm a requirement. Not everyone uses it, and it's a pretty heavy dependency.

I strongly encourage structure-aware yaml manipulation (e.g. kustomize) over text templating.

I also think the ko based workflows have been very helpful for rapid iteration and I don't think we want to lose that.

The Helm charts should decide whether they require pre-published images
or whether development docs continue using ko to publish
images before Helm install.

I think it may be slightly premature to start publishing builds. We don't have defined versions and releases yet.
We will need to do so eventually, but see:
https://github.com/agent-substrate/substrate/blob/9bd41df2a36d4779206babbad8c4b6bf1f6e0884/README.md#status-and-compatibility

[a4-a4s1]

Stepping back from the helm-vs-kustomize debate: both produce manifests that a GitOps tool can sync. The harder constraint may sit elsewhere.

Two needs #79 is folding together:

• Install-time UX: helm, kustomize, raw YAML — all work. The productive Q is whether install can be done against already-published images (no ko, no shell scripts at sync time).
• GitOps reconciliation: ArgoCD/Flux want a deterministic source of truth. Helm and kustomize both deliver that; ko-rendered manifests with build-local SHAs do not — image refs aren't stable across builds.

Both needs trace back to the same thing: when stable image references become available. The comment above flagging "premature to publish builds" + the README status-and-compatibility note both point there.

One Q for maintainers: is there a target where stable builds become the goal? That timing probably anchors the packaging shape more than helm-vs-kustomize directly.

[🤖a4s1]

[EItanya]

Existing script-based install either keeps working or is
updated to render/apply the Helm charts.

I don't think we should make helm a requirement. Not everyone uses it, and it's a pretty heavy dependency.

I strongly encourage structure-aware yaml manipulation (e.g. kustomize) over text templating.

I also think the ko based workflows have been very helpful for rapid iteration and I don't think we want to lose that.

The Helm charts should decide whether they require pre-published images
or whether development docs continue using ko to publish
images before Helm install.

I think it may be slightly premature to start publishing builds. We don't have defined versions and releases yet. We will need to do so eventually, but see: https://github.com/agent-substrate/substrate/blob/9bd41df2a36d4779206babbad8c4b6bf1f6e0884/README.md#status-and-compatibility

Just to be clear, I’m mostly focused on reproducible installs via popular tooling. From my experience helm is BY FAR the most popular and widely used kubernetes installation mechanism, usually via Argo and other gitops tools at the org level. My main goal in opening this issue is starting the discussion on how users will install/manage installations of substrate.

I saw #64, but my experience is that GitOps is the preferred method most of the time, so I wanted to get that discussion going here. Happy to modify the issue to be more about reproducible gitops friendly installs if that would help.

[a4-a4s1]

+1 on broadening the issue toward "reproducible GitOps-friendly installs" — that reframe sidesteps the tooling-comparison and surfaces the actual question.

A few observations on how that broader scope intersects with what's already in-tree:

• ko-rapid-iteration and pre-published images aren't mutually exclusive. Dev loop keeps ko publish for the tight feedback path; release path publishes versioned images that the GitOps install consumes. The two paths don't share a build but can share a manifest shape (kustomize base + overlay), so the "ko vs published" Q collapses into "two consumers of the same manifests."

• The structural Q the broader scope surfaces: where does manifests/ate-install/ sit relative to the GitOps install path? Today's manifests/ is ko resolve output; a published-image install would either (a) re-render the same base with image refs swapped in, or (b) ship a separate rendered tree. (a) keeps one source of truth; (b) decouples release cadence from manifest changes.

• ate-operator: Make it easy to install and manage (multiple) substrates #64 (mtaufen ate-operator install) and this issue overlap on the install-time concerns but differ on the operator-vs-charts axis. Worth a forward-link between the two once the scope here settles, so the operator-install path has a place to land.

If the issue gets retitled, would it be useful to enumerate the install-time decisions explicitly (release artifact format, version pinning, RuntimeClass parameterization, multi-tenant overlay shape) so the conversation can converge on each in turn?

[🤖a4s1]