Skip to content

feat(rpc,bench): emit per-pod URLs in up envelopes from SND status#119

Merged
bdchatham merged 3 commits intomainfrom
feat/rpc-emit-per-pod-urls
May 1, 2026
Merged

feat(rpc,bench): emit per-pod URLs in up envelopes from SND status#119
bdchatham merged 3 commits intomainfrom
feat/rpc-emit-per-pod-urls

Conversation

@bdchatham
Copy link
Copy Markdown
Contributor

@bdchatham bdchatham commented May 1, 2026

Summary

Closes #118. After rpc up --apply and bench up --apply, poll the SND's status.perPodServices (published by sei-k8s-controller#158) and emit endpoints.perPod[] with dial-ready URLs:

http://<pod>.<namespace>.svc:<evmHttp>
ws://<pod>.<namespace>.svc:<evmWs>

This closes the loop on platform's nightly JSONPath workaround (sei-protocol/platform#334) — consumers that need per-pod connectivity (seiload's WebSocket block collector, future qa harness) can read .data.endpoints.perPod[].evmWs from the standard envelope instead of re-implementing label-and-filter Service discovery.

Envelope shape

Before:

{
  "data": {
    "endpoints": {
      "tendermintRpc": ["http://...-rpc-primary-internal..."],
      "evmJsonRpc":    ["http://...-rpc-primary-internal..."]
    }
  }
}

After (with --apply and a controller publishing the new field):

{
  "data": {
    "endpoints": {
      "tendermintRpc": ["http://...-rpc-primary-internal..."],
      "evmJsonRpc":    ["http://...-rpc-primary-internal..."],
      "perPod": [
        {
          "name": "pacific-1-rpc-primary-0",
          "evmJsonRpc": "http://pacific-1-rpc-primary-0.nightly.svc:8545",
          "evmWs":      "ws://pacific-1-rpc-primary-0.nightly.svc:8546"
        },
        { "name": "pacific-1-rpc-primary-1", ... }
      ]
    }
  }
}

perPod is omitempty. Dry-run never populates it; controllers without the new status field also produce an empty result.

Implementation notes

  • New kube.GetSND returns *unstructured.Unstructured. Dynamic client preserves the architectural constraint that seictl never imports the typed sei-k8s-controller API surface.
  • perPodPoller is dependency-injected for tests. Production polls every 1s up to 30s, then emits a stderr warning + perPod omitted; the apply itself is unaffected.
  • PerPod lives inside the existing Endpoints struct rather than as a sibling on the result types — keeps URL handles under one umbrella and applies uniformly to chain/rpc/bench (chain currently doesn't poll, so it stays empty there).
  • Order in the envelope mirrors the controller's order (sorted by ordinal in populatePerPodServices); the parser does not re-sort.

Test plan

  • go test ./cluster/... — green
  • Parser tests cover: nil unstructured, missing status, missing perPodServices, controller-ordered output, malformed entries dropped, partial result returned for poller to retry.
  • Wiring tests cover: rpc up --apply invokes the poller with the right SND name + replicas; dry-run skips it; bench up --apply queries ${chainID}-rpc with sizeProfile.RPC replicas.
  • Live verify post-merge: seictl rpc up --apply -o json | jq '.data.endpoints.perPod[].evmWs' against harbor.
  • Follow-up: open platform PR replacing the JSONPath block in k8s_nightly.yml with a jq over the new envelope (after a new seictl release ships).

🤖 Generated with Claude Code

bdchatham and others added 3 commits May 1, 2026 16:36
Closes #118. After `rpc up --apply` and `bench up --apply`, poll the
SND's `status.perPodServices` (published by sei-k8s-controller#158) and
emit `endpoints.perPod[]` with dial-ready URLs:

  http://<pod>.<namespace>.svc:<evmHttp>
  ws://<pod>.<namespace>.svc:<evmWs>

This closes the loop on platform's nightly JSONPath workaround
(sei-protocol/platform#334) — consumers that need per-pod connectivity
(seiload's WebSocket block collector, future qa harness) can read
`.endpoints.perPod[].evmWs` from the standard envelope instead of
re-implementing label-and-filter Service discovery.

Implementation notes:
- New `kube.GetSND` returns `*unstructured.Unstructured`. Dynamic client
  preserves the architectural constraint that seictl never imports the
  typed sei-k8s-controller API surface.
- `perPodPoller` is dependency-injected for tests; production polls every
  1s up to 30s, then emits a stderr warning + `perPod` omitted.
- `PerPod` lives inside the existing `Endpoints` struct rather than as a
  sibling on the result types — keeps URL handles under one umbrella and
  applies uniformly to chain/rpc/bench.
- Dry-run skips the poll; envelope omits `perPod` via `omitempty`.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Coral cross-review (kubernetes-specialist + product-engineer) on #119:

- Tighten poll completion: replace `len >= replicas` with strict equality
  AND `status.observedGeneration == metadata.generation`. Closes a
  scale-down race where the controller has not yet pruned per-pod
  entries from the prior generation.
- Short-circuit terminal apiserver errors (Forbidden, Unauthorized,
  NoMatchError on the CRD). Without this, an RBAC misconfig waits the
  full 30s before warning. NotFound on the SND itself stays transient
  (apply just landed; client cache may trail).
- Extract `sndGetter` DI seam so the poll loop is unit-testable without
  a real cluster. Production wraps `kc.GetSND`; tests inject a fake.
- Switch per-pod URL form from `<pod>.<ns>.svc:<port>` to
  `<pod>.<ns>.svc.cluster.local:<port>` to match the aggregate
  Endpoints style and resolve cleanly via in-cluster-DNS-via-VPN flows.
- Tighten doc comments: `Endpoints` env-var lock now scopes correctly
  to TendermintRpc/EvmJsonRpc only; `PerPodEndpoint` documents the
  deferred TendermintRpc field and the order-by-ordinal contract.
- New unit tests: terminal-error short-circuit, partial-then-ready
  retries, stale-generation keeps polling, NotFound is transient,
  ctx cancellation exits promptly. 16 new test cases pass.

RBAC verified on harbor: `kubectl auth can-i get seinodedeployments.sei.io`
returns yes — engineers' Access-Entry-bound role covers `get` alongside
`list`/`patch`. No platform-side change needed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Drop narration that restates code/names; keep comments that document a
constraint, a contract, or a surprise (env-var lock, scale-down race
guard, NotFound transience).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@bdchatham bdchatham merged commit f554bd4 into main May 1, 2026
3 checks passed
@bdchatham bdchatham deleted the feat/rpc-emit-per-pod-urls branch May 1, 2026 23:58
bdchatham added a commit that referenced this pull request May 2, 2026
Picks up #119: per-pod URLs in rpc up / bench up envelopes via
endpoints.perPod[]. Additive only, no breaking changes.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(rpc): emit per-pod URLs in rpc up envelope from Status.PerPodServices

1 participant