[TaylorMutch]

Summary

Add an opt-in supervisor TLS listener for declared HTTP and WebSocket services. Application traffic connects directly to the owning supervisor and sandbox loopback connector, bypassing the gateway.

Related Issue

Closes #4666. Part of RFC 0017.

Changes

• Add listener enablement, domain, app-port, and Kubernetes TLS configuration; default to disabled.
• Use the owning supervisor Pod IP and application port in both dedicated and shared topologies. The driver creates no application Services; ingress integrations own any routing resources they require.
• Bound service declarations before persistence and fence route registration against supervisor shutdown.
• Route exact declared hosts through the sandbox connector, isolate per-sandbox teardown, and document DNS, TLS, and ingress authentication responsibilities.

Release Notes

The application listener remains opt-in and disabled by default.

Application listener request flow

This PR adds an opt-in TLS listener on the supervisor. The gateway supplies
configuration, while application traffic reaches declared sandbox services
through the selected sandbox instance's connector.

flowchart TB
    subgraph Configuration["Configuration and deployment"]
        SETTINGS["Operator enables app listener<br/>Domain, port, TLS Secret"]
        KUBE["Kubernetes driver / Helm<br/>Configure supervisor port and mount TLS material"]
        DECLARE["Sandbox service declaration<br/>Example: web → port 8080"]
        GW["OpenShell gateway<br/>Include service allowlist in sandbox configuration"]

        SETTINGS --> KUBE
        DECLARE --> GW
    end

    subgraph HOST["Supervisor process — dedicated or shared"]
        TLS["One process-level TLS listener<br/>Example: Pod IP:8443"]
        ROUTER["SNI dispatcher<br/>Select sandbox instance"]

        subgraph INSTANCE["Selected sandbox instance"]
            CONFIG["Apply accepted configuration<br/>Install routes and renew freshness"]
            CHECK["Validate installed route and freshness<br/>Require HTTP Host to match TLS SNI"]
            PROXY["HTTP/1.1 or WebSocket proxy<br/>Apply limits and sanitize headers"]
            CONNECTOR["This instance's<br/>BoundaryLoopbackConnector"]

            CONFIG -.->|"Route table and validity"| CHECK
            CHECK -->|"Accepted"| PROXY
            PROXY --> CONNECTOR
        end

        TLS --> ROUTER
        ROUTER --> CHECK

        OTHER["Other sandbox instances<br/>Separate routes, connectors and lifetimes"]
        ROUTER -->|"Other SNI hostname"| OTHER

        REVOKE["Route removed or changed,<br/>configuration rejected, freshness expires,<br/>or instance stops"]
        CONFIG -.-> REVOKE
        REVOKE -.->|"Reject affected traffic<br/>Close affected active streams"| PROXY
    end

    subgraph SANDBOX["Selected sandbox"]
        APP["Application<br/>127.0.0.1:8080"]
    end

    CLIENT["Client<br/>team--demo--web.apps.example.com"]
    EDGE["Externally configured ingress / proxy<br/>or direct connection"]

    KUBE -.->|"Startup configuration"| TLS
    GW -->|"Existing configuration push / poll"| CONFIG
    CLIENT --> EDGE
    EDGE -->|"TLS with application SNI"| TLS
    CONNECTOR -->|"Connect to declared loopback port"| APP

Loading

• Shared supervisor support: one listener dispatches to separate sandbox
  instances. Each retains its own connector, limits, and lifetime.
• Configuration freshness: routes have a 30-second freshness window, renewed
  through successful configuration delivery or an accepted live push session.
  Withdrawal or expiry also closes affected streams.
• Kubernetes reachability: this PR configures ports and TLS material. Both
  dedicated and shared supervisors expose Pod-IP endpoints without application
  Services; callers must target the owning supervisor.
• Authentication: this direct path relies on the application or trusted
  ingress for authentication. Gateway-mediated authentication is separate.

Advertisements, discovery, authoritative public URLs, the sandbox proxy, and
HTTPRoute reconciliation are outside this PR. The ingress box represents
infrastructure configured externally at this stage. Addresses and ports are
illustrative.

Automated Kubernetes listener checks

mise run e2e:kubernetes:app-listener deploys an isolated Helm release with
matching branch images, ephemeral TLS material and in-cluster clients, then
removes its test resources. It needs neither discovery nor the reference proxy.

The test:e2e CI lane now runs four cases: dedicated push, dedicated poll,
shared graceful takeover and shared abrupt takeover. These jobs use the existing
Kubernetes reusable workflow, enable pinned Calico in kind to verify NetworkPolicy
denial, upload failure evidence, and contribute to Core E2E result.

Coverage includes HTTP/WebSockets, verified application TLS, SNI/Host validation,
header handling, named/unnamed routes, Pod-IP ingress without app Services,
quiet-session freshness, selective withdrawal, gateway-outage expiry/recovery,
workload identity/memory preservation through shared-owner changes, and disabled
listener compatibility with gateway-mediated application access.

Local k3d validation covered dedicated push/poll and abrupt shared takeover,
including disabled-listener gateway compatibility. The combined stack also
exercised Go SDK discovery and two proxy replicas routing directly to supervisor
Pod IPs, including owner handover and route withdrawal. This live validation
preceded integration of the latest route-limit and shutdown-fencing hardening;
focused regression tests cover that integration. Current-head remote CI results
are tracked in this PR's checks.

Testing

• 40 focused combined-stack Rust regression tests cover listener routing, route-limit concurrency, shutdown fencing, advertisements, discovery, and proxy behavior.
• Local k3d validated the Service removal with dedicated push/poll, abrupt shared-owner takeover, HTTP/WebSockets, network-policy denial, freshness/withdrawal, outage recovery, and disabled-listener gateway access. Workload identity and memory survived owner changes.
• Real Go SDK snapshots/watches and workspace-restricted discovery credentials drove two proxy replicas; both followed Pod-IP endpoints, graceful owner handover, and route withdrawal without per-sandbox app Services.
• Helm lint/unit/parser/coherence/scaling checks, Python lint/format/type checks, and topology assertions passed.

Live Kubernetes validation preceded integration of the latest listener hardening;
focused Rust regressions validate that final integration. New-head CI is pending.

Qualification limits

Live HA validation exercised supervisor process/Pod loss on a local k3d cluster. Whole-node failure, real network partitions, metric-driven HPA, sustained production load, and published release artifacts remain unqualified. Grant-file projection contributes to revocation propagation time. Main now includes #4842 for gateway-mediated hashed hostnames. Direct application-listener/discovery hostnames remain readable; extending hashed naming to that path and installation-status presentation (#4670) remain separate work.

The example Kubernetes HTTPRoute controller is a deferred compatibility POC outside this stack.

Checklist

• Follows Conventional Commits
• Commits are signed off (DCO)
• Relevant implementation, configuration, and API documentation updated

PR dependency order: #4936 → #4940 → #4941 → #4956 → #4963 → #4964 → #4957 → #4958.

[copy-pr-bot]

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

[github-actions]

🌿 Preview your docs: https://nvidia-preview-pr-4936.docs.buildwithfern.com/openshell

[github-actions]

Compatibility review (advisory)

New or updated breaking changes; see the full report for all findings.

1. rust-sdk — No explicit risk acceptance found

The SDK-reachable raw::proto::SandboxConfigSnapshot and GetSandboxConfigResponse structs gain a required service_routes field, invalidating previously exhaustive literals and patterns.

• Impact: Consumers constructing or exhaustively matching these messages, including custom servers and mocks, stop compiling; ordinary RPC callers and server-trait signatures are unaffected.
• Action: Keep existing raw message shapes unchanged on this no-break train; any later permitted break needs migration guidance for service_routes initialization and non-exhaustive matching.
• PR: feat(supervisor): add independent application listener and endpoints #4936

2. typescript-sdk — No explicit risk acceptance found

Regenerated SandboxConfigSnapshot and GetSandboxConfigResponse types gain a required serviceRoutes array in the public raw entry point.

• Impact: Complete object literals typed as either raw message fail type-checking for missing serviceRoutes, while create() and MessageInitShape-based initialization remain compatible.
• Action: Preserve existing raw message shapes on this no-break train; any later permitted break needs guidance to add serviceRoutes: [] or use schema-based initialization.
• PR: feat(supervisor): add independent application listener and endpoints #4936

Review details

• Execution: complete
• Candidate: PR #4936 (623b1575bf49333608f856d0930c9aa6e862269b)
• Baseline: 9f1cdb4ef6ed6be9fd86507580f9656b61b9c754 (9f1cdb4ef6ed6be9fd86507580f9656b61b9c754)
• Reviewed: rust-sdk, python-sdk, go-sdk, typescript-sdk, api-behavior, cli, configuration, policy, helm, state

Advisory review; publication gates are unchanged.

Branch check run

[TaylorMutch]

/ok to test d46a19f

[github-actions]

Label test:e2e applied for d46a19f. Open the existing run and click Re-run all jobs to execute with the label set. The run will execute the standard E2E suite after building the required gateway, sandbox, and supervisor images once. The matching required CI gate status on this PR will flip green automatically once the run finishes.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[FrostGod]

Can we add a shared-mode ingress example showing how to reach the owning supervisor before discovery lands?

[FrostGod]

What happens if a sandbox already has over 1,024 services when we enable the listener?

[FrostGod]

Can we automate checking that rejected configuration closes existing streams?

[FrostGod]

Do raw SDK users need a migration note for the new service_routes field?

[TaylorMutch]

@FrostGod, addressed the test/documentation follow-ups in 5759d05. Decisions on all four comments:

• Shared-mode ingress example: kept this to a brief ownership note alongside the existing direct-access curl --connect-to example in e2e/kubernetes/app-listener/README.md. Before discovery lands, select the owning supervisor and repeat that selection after takeover; a Service balancing across arbitrary shared supervisors cannot select the owner. The downstream discovery/routing PRs automate this. A full manual ingress configuration would introduce a temporary operator workflow we don't intend to require for the completed stack.

• More than 1,024 services: the bound is per sandbox, not an aggregate across a shared supervisor. Currently an already oversized sandbox makes its complete supervisor configuration build return ResourceExhausted; it does not silently install a partial allowlist. This can also block that sandbox's policy/settings delivery until declarations are reduced. We consider over 1,024 services in one sandbox unrealistic for the intended use cases, so we're retaining the internal defensive bound without adding configurable service quotas or special oversized-state migration handling in this PR.

• Rejected configuration closes streams: added rejected_configuration_closes_streams_without_affecting_another_sandbox. It opens real TLS WebSockets for two sandbox instances sharing one listener, installs a rejected configuration for one, verifies new requests fail closed and its existing stream closes, and checks the other sandbox's stream remains usable. Restoring accepted configuration permits a new connection without reviving the old stream. This automates the listener configuration boundary; injecting a rejection through a deployed gateway remains a separate manual Kubernetes check.

• Raw SDK migration: added notes to the Rust and TypeScript SDK READMEs. The protobuf change is additive on the wire, but complete Rust literals need service_routes: Vec::new() or ..Default::default(), and exhaustive patterns need the field or ... Complete TypeScript message literals need serviceRoutes: [], or schema-based create() initialization. The notes also distinguish this supervisor allowlist from discovery; curated SDK APIs are unchanged.

Validation: all 14 supervisor application-listener tests passed locally, including the new regression; cargo fmt --all -- --check passed. The regression runs with the existing Rust unit-test suite.

[github-actions]

Compatibility review (advisory)

New or updated breaking changes; see the full report for all findings.

1. api-behavior — Risk explicitly accepted

GetSandboxConfig and configuration push previously worked independently of service count but now fail with ResourceExhausted for sandboxes exceeding 1,024 declarations when supervisor.app_listener.enabled is true.

• Impact: Enabling the listener on existing oversized sandboxes blocks their policy/settings delivery, although leaving the listener disabled preserves previous behavior.

• Action: Preserve policy/settings delivery for existing oversized sandboxes on v0.1.7 and document how to check and reduce declarations or leave supervisor.app_listener.enabled=false before enablement.

• PR: feat(supervisor): add independent application listener and endpoints #4936

• Acceptance: comment

Review details

• Execution: complete
• Candidate: PR #4936 (22ac5d226cf1f0927f7729176cf8b539167913f5)
• Baseline: d7b616895d8778320f994ddf14f4e89ab777f29c (d7b616895d8778320f994ddf14f4e89ab777f29c)
• Reviewed: rust-sdk, python-sdk, go-sdk, typescript-sdk, api-behavior, cli, configuration, policy, helm, state

Advisory review; publication gates are unchanged.

Branch check run