feat(chart): add trustBundle, and exercise real auth in the e2e sandbox job

[dcmcand]

Follow-up to #76. Makes the e2e job test the OIDC path it has always skipped.

Why the job ran in dev mode

/readyz only turns 200 once the pod completes OIDC discovery against https://keycloak.nebari.local, and the sandbox gateway's certificate is signed by a self-signed CA the pod has no reason to trust. #76 fixed the other half of that blocker - the sandbox now resolves *.nebari.local from inside the cluster - leaving CA trust as the only thing standing in the way.

Why this goes in the chart

Trusting a private CA is a real deployment need, not a CI-only one: any Nebari behind a private CA or a TLS-terminating proxy has it. So rather than work around it in the workflow, the chart gains a values surface for it:

trustBundle:
  configMapName: ""
  key: ca-certificates.crt

When configMapName is set, the chart mounts that key read-only at /etc/ssl/nebari and points SSL_CERT_FILE at it. No Go change is needed - oidc.NewProvider uses Go's default pool, which honors that variable. Only the configured key is projected, so an unrelated entry in the ConfigMap cannot show up beside the bundle.

The ConfigMap must hold a complete bundle - public roots plus the private CA - because SSL_CERT_FILE replaces Go's default pool rather than extending it. NIC already produces exactly that when trust_bundle is configured: a nebari-trust-bundle ConfigMap, public roots included, projected into every namespace via trust-manager. Both values.yaml and the chart README say so, because a CA-only file is the easy mistake and it breaks TLS to anything publicly signed.

One targeted refactor came with it: three features now contribute volumes, and the gate was spelled out twice (once for volumeMounts, once for volumes). A third term repeated in both places is where that stops being readable, so the condition moved into a nebari-frames.hasVolumes helper.

Guards against a test that tests nothing

Two, both learned from #76:

• The chart job now asserts on rendered content rather than only that a render succeeded. A silently-empty render would previously still pass kubeconform. It also asserts the feature stays opt-in - no SSL_CERT_FILE without trustBundle.configMapName.
• The e2e job asserts the pod is really in real-auth mode: FRAMES_DEV_MODE absent, OIDC_ISSUER_URL present, SSL_CERT_FILE pointing at the mount. Dev mode also reports ready, so a quiet fallback to it would otherwise look completely green. That is precisely how v3's gateway-ip test only regex-checks the address, never connects to it action-nebari-sandbox#103 sat unnoticed for months.

Otherwise no new wait logic: Synced/Healthy already gates on the readiness probe, so it is itself the assertion that in-cluster DNS, CA trust, and the operator's client provisioning all work.

Scope

Deliberately readiness only. An authenticated API call is a natural next step but a separate piece of work: the operator sets DirectAccessGrantsEnabled: false on both clients it provisions, and frames validates an ID token whose aud must be the SPA client, so there is no simple grant to reach for. Getting a token means either scripting the authorization-code + PKCE flow or mutating Keycloak first - a real tradeoff that deserves its own design pass rather than being smuggled in here.

Known gap: the sandbox ConfigMap holds the gateway CA alone, not public-roots-plus-CA, so SSL_CERT_FILE narrows the pod's trust to that one CA. Harmless here - frames only talks to in-cluster Keycloak - but it means this job does not exercise the complete-bundle contract the chart documents.

Verification

Locally: helm lint, all four volume-gate combinations (including the no-volumes case the new helper has to preserve), the exact e2e value set rendering with OIDC_ISSUER_URL and SSL_CERT_FILE present and FRAMES_DEV_MODE absent, schema rejecting both an empty key and unknown properties, kubeconform on all three renders, plus make test and make lint (unchanged - no Go code touched).

The real verification is the e2e job going green with auth on, which is what CI here decides.

Also corrects AGENTS.md, which still described the sandbox as k3d after #76.

[dcmcand]

The first e2e run failed, and it turned out not to be the trust bundle - the pod never reached TLS at all:

auth: OIDC discovery not ready; retrying
issuer: https://keycloak.nebari.local/realms/nebari
error: ... dial tcp: lookup keycloak.nebari.local on 10.96.0.10:53: server misbehaving

DNS, not certificates. The sandbox action's in-cluster DNS step is a no-op on kind: it writes a coredns-custom ConfigMap, which is a k3s convention (k3s CoreDNS mounts it and does import /etc/coredns/custom/*.server). kind's CoreDNS comes from kubeadm with a plain .:53 Corefile and no such import, so the ConfigMap is created, CoreDNS is restarted, and the file is never read. The step still logs In-cluster DNS ready, ~10 minutes before the pod started getting SERVFAIL in the same run.

Filed upstream as nebari-dev/action-nebari-sandbox#104. Same blind spot as #103: both are features whose entire purpose is reachability from inside the cluster, and neither has a test that resolves or connects to anything from a pod.

Worked around here by adding the zone to the Corefile kind actually reads - a merge patch on just that key, skipped when the zone is already present - and then proving it with a throwaway pod that resolves the name and must see the gateway IP. Without that assertion the failure shows up fifteen minutes later as an unexplained unhealthy Application, which is exactly how the upstream no-op stayed invisible.

That makes three workarounds in this job for v3 regressions, all from the same k3d-to-kind migration: the MetalLB pool (nebari-dev/nebari-infrastructure-core#612), now DNS, plus the gateway-ip test gap that let both ship green. Each carries its issue reference in a comment and comes out when upstream lands.

Verified locally by extracting the rendered step out of the workflow YAML and executing it against a stubbed kubectl: the Corefile patch produces the right block with {{ .Name }} intact, is idempotent on a second application, and both failure paths (probe cannot resolve, gateway-ip empty) exit non-zero with a clear message rather than passing quietly.

Note the trust bundle is still unproven either way - the pod has never yet got far enough to attempt TLS. This run is the first real test of it.