Add the ProxyAdminService proto and its generation pipeline - #258
Open
liam-lowe wants to merge 2 commits into
Open
Add the ProxyAdminService proto and its generation pipeline#258liam-lowe wants to merge 2 commits into
liam-lowe wants to merge 2 commits into
Conversation
liam-lowe
force-pushed
the
liam-lowe/proxyadmin-service-proto
branch
from
August 21, 2026 00:55
279d226 to
3e4f0c0
Compare
liam-lowe
force-pushed
the
liam-lowe/proxyadmin-service-proto
branch
9 times, most recently
from
August 24, 2026 22:10
4a6b862 to
b4d1ef4
Compare
liam-lowe
force-pushed
the
liam-lowe/proxyadmin-service-proto
branch
2 times, most recently
from
August 24, 2026 22:49
8ccc7d4 to
b6525a5
Compare
liam-lowe
force-pushed
the
liam-lowe/proxyadmin-service-proto
branch
3 times, most recently
from
August 25, 2026 01:32
149245c to
37e2601
Compare
The schema, the generated stubs, the make targets and the drift workflow are one unit: CI regenerates and diffs api/, so a change to any of them without the others fails the gate. Requests carry no routing fields. Scope and target travel as gRPC metadata, because an interceptor cannot read a request field without reflection, so with fields every listener's limits would have to be re-checked inside every handler and any RPC added later would be exposed until someone remembered. The response is one list. Cluster connections carry their members, and each member row nests the member that reported it, so reading a member's version next to its session counts needs no join. A member appears once per connection it reports on and those facts repeat, which is the cost of not having a second list. Nothing describes the group as a whole. The responder is the member marked self. Session counts carry a target as well as a total. total is what a member holds now, target is what it was configured to hold. Without target a connection holding three sessions of a configured ten reports connected == total, which is the condition this endpoint exists to reveal. ConnectionState is UNSPECIFIED, CONNECTED and ERROR, and will grow. Partial connectivity is ERROR rather than CONNECTED: three sessions of a configured ten is the condition this endpoint exists to reveal, not a healthy link. UNSPECIFIED means no state was observed, which covers a field nobody set and a connection this API does not describe, such as one that is not multiplexed. A member that answered nothing has no row to appear in. It shows up only where every member that did answer was connected, because a connection that would otherwise read CONNECTED is held at ERROR instead. On a connection already reading ERROR it leaves no trace, and a three-member group with one member down is then indistinguishable from a two-member group. buf is pinned in develop/buf.mod and run through the Go toolchain rather than installed separately. The codegen plugins were already remote and version pinned in buf.gen.yaml, so the buf version only governed the CLI, and CI and a developer's machine could still disagree about it. buf gets its own module rather than joining tools.mod. Its dependency graph pulls quic-go and docker, whose versions do not resolve against tools.mod's existing pins: adding it there produces a module where buf does not compile. proto-lint and proto-breaking are make targets so the workflow runs what a developer runs. The drift step diffs the whole tree rather than api/ alone, so generation that writes anywhere is caught. Nothing describes the Temporal cluster a connection fronts. That is ServerDescription under another name, it is not topology, and it would make the proxy originate its own DescribeCluster call to the local frontend. Field 5 on ClusterConnection is left unused rather than renumbered.
liam-lowe
force-pushed
the
liam-lowe/proxyadmin-service-proto
branch
from
August 25, 2026 01:32
37e2601 to
1b9f883
Compare
clean: true removes everything under out: before generating, and out: is api/. The setting is safe today: api/ does not exist on main and is generated in full. Anything that ever lands there by another route is deleted on the next generation run without warning.
liam-lowe
force-pushed
the
liam-lowe/proxyadmin-service-proto
branch
from
August 25, 2026 04:36
2b490f6 to
7298853
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds the
ProxyAdminServiceschema, its generated Go stubs, the generation targets, and a CI workflow that keeps them in step. No server or client implementation: this PR is the wire contract only, and nothing calls it yet.These pieces ship together because the new workflow regenerates
api/and diffs it against the tree. A change to the proto, thebufconfig, or a make target without the others fails that gate.The service exposes one RPC,
DescribeClusterConnections, which reports a proxy and the cluster connections it has to other proxies.Routing lives in metadata, not in the request
Requests carry no routing fields. Two pieces of gRPC metadata control how far a call travels. Metadata rather than request fields because an interceptor cannot read a request field without reflection. With fields, every listener's limits would have to be re-checked inside every handler, and any RPC added later would be exposed until someone remembered to add the check.
1.
s2s-proxy-scope: how many members of this group answerflowchart LR subgraph M["s2s-proxy-scope: member"] direction LR O1(["operator"]) -->|call| A1["member-1"] A1 -->|"answers for itself"| R1(["one member"]) endflowchart LR subgraph G["s2s-proxy-scope: group (the default when absent)"] direction LR O2(["operator"]) -->|call| B1["member-1"] B1 -.->|fan out| B2["member-2"] B1 -.->|fan out| B3["member-3"] B2 -.->|reply| B1 B3 -.->|reply| B1 B1 -->|"one merged answer"| R2(["whole group"]) endUnder
group, the member that received the call answers for itself and fans out to its siblings, then merges. Each connection'smemberslist then holds one row per member that reported it, and the member that received the call is the row markedself.2.
s2s-proxy-target: hand the question to the counterpartys2s-proxy-targetnames a cluster connection. The call is forwarded once to that connection's counterparty, which answers at group scope for its own group. Each listener enforces a ceiling on both headers, so a call travels at most two hops: one forward, then one fan-out. Metadata is not propagated to outgoing calls, so a forwarded or fanned-out request always starts clean.DescribeClusterConnectionsThe request is empty. Everything that varies is in metadata.
classDiagram class DescribeClusterConnectionsResponse { ClusterConnection[] cluster_connections } class ClusterConnection { string name ConnectionState state int32 mux_sessions_connected int32 mux_sessions_total int32 mux_sessions_target ClusterConnectionMember[] members } class ClusterConnectionMember { Member identity ConnectionState state int32 mux_sessions_connected int32 mux_sessions_total int32 mux_sessions_target } class Member { string id bool self string address string version Timestamp start_time } DescribeClusterConnectionsResponse --> ClusterConnection : cluster_connections ClusterConnection --> ClusterConnectionMember : members ClusterConnectionMember --> Member : identityThe response is a single list. Each cluster connection carries its members, and each member row nests the member that reported it, so a member's version sits next to its session counts with no join.
A member appears once per connection it reports on, so those per-member facts repeat down the response. There is no top-level description of the member that answered: it is the member marked
self.What the shape gives up
Folding the roster into the connection list removes the only place a member with nothing to report could appear:
CONNECTEDatERROR. On a connection already readingERRORit leaves no trace. What is lost is which member, why, and the discovered-against-responding counts.MemberStateand the dial error fields are gone with it, having nothing left to describe.MemberDiscoveryis gone too.id,self,address,versionandstart_timeare properties of a member, not of a member-connection pair, and they appear once per connection the member reports on.In exchange the response has one list instead of two, and no id join.
There is no description of the Temporal cluster a connection fronts. That is
ServerDescriptionunder another name, it is not topology, and it would make the proxy originate its ownDescribeClustercall to the local frontend. Nothing here populates it. The node that needs the remote cluster's identity already makes that hop and will own the field.A forwarded response does not mark which of the far side's connections the request arrived over. The two sides name their connections independently, and the chart advises picking a name "not shared by other s2s-proxy deployments", so there is no correspondence to rely on either. A caller making a targeted request is assumed to know what it asked about.
mux_sessions_targetsits besidetotalbecause they answer different questions.totalis what a member holds now.targetis what it was configured to hold. A connection holding three of a configured ten reportsconnected == total, so withouttargetthe API cannot express the condition it exists to reveal. The metriccluster_connection_mux_sessions_targetalready exports that denominator.ConnectionStateis deliberately coarse:UNSPECIFIED,CONNECTED,ERROR. Partial connectivity isERRORrather thanCONNECTED, because three sessions of a configured ten is the condition this endpoint exists to reveal, not a healthy link.UNSPECIFIEDmeans no state was observed, which covers both a field nobody set and a connection this API does not describe, such as one that is not multiplexed. More values will be added;buf.yamlpinsbreaking: use: FILE, and adding enum values is not a breaking change.A member that holds zero sessions and a member whose sessions failed both read
ERROR.mux_sessions_connectedandmux_sessions_totalseparate them: zero-of-ten is a customer proxy that has not dialled in, three-of-ten is a link that is degraded. Distinguishing "their cluster is down" from "the ACL rejected us" needs a secondDescribeClusterhop either way.CONNECTION_STATE_CLOSEDcan be added later; adding enum values is not aFILE-level break.No message declares a
reservedrange. Nothing consumes this wire format yet.ClusterConnectionleaves field 5 unused where the cluster description was removed, rather than renumbering every field after it.buf is vendored
bufis pinned indevelop/buf.modand run through the Go toolchain, the same waygolangci-lintandmockgenare pinned indevelop/tools.mod. Contributors need no separate install, and CI uses the same pinned version, so CI and a developer's machine cannot disagree about it.Two things worth knowing:
tools.mod. buf's dependency graph pullsquic-goanddocker, whose versions do not resolve againsttools.mod's existing pins. Adding it there produces a module where buf itself fails to compile, and it forces unrelated upgrades ongolangci-lintandmockgen. A separate modfile keeps both graphs independent.tools.modandtools.sumare untouched by this PR.buf.gen.yamluses remote plugins pinned toprotocolbuffers/go:v1.36.11andgrpc/go:v1.5.1, so buf only drives the CLI.proto-lintandproto-breakingare make targets so the workflow runs what a developer runs.Testing
make generate-proxy-protoreproduces the checked-inapi/exactly, run twice from a cold build cache. That is what the workflow's drift job asserts.make proto-lintpasses. The workflow also runsbuf breakingagainstmain, skipped on this PR becauseproto/buf.yamldoes not exist onmainyet.make test(0 failures),make lint(0 issues) andgo build ./...all pass, andgolangci-lintstill resolves from the unmodifiedtools.mod.api/proxyadmin/v1has no hand-written logic to test.Worked example
Requested at the default
groupscope against a three-member group with three cluster connections configured.Request. Empty. Everything that varies travels in metadata.
grpcurl -plaintext \ -H 's2s-proxy-scope: group' \ localhost:6061 \ temporal.s2sproxy.proxyadmin.v1.ProxyAdminService/DescribeClusterConnections{}Response. Every connection in config is listed, whether or not it is up.
{ "clusterConnections": [ { "name": "prod-migration", "state": "CONNECTION_STATE_CONNECTED", "muxSessionsConnected": 30, "muxSessionsTotal": 30, "members": [ { "identity": { "id": "s2s-proxy-0", "self": true, "version": "1.42.0", "startTime": "2026-08-24T09:14:02Z" }, "state": "CONNECTION_STATE_CONNECTED", "muxSessionsConnected": 10, "muxSessionsTotal": 10 }, { "identity": { "id": "s2s-proxy-1", "address": "10.4.1.19:9234", "version": "1.42.0", "startTime": "2026-08-24T09:14:07Z" }, "state": "CONNECTION_STATE_CONNECTED", "muxSessionsConnected": 10, "muxSessionsTotal": 10 }, { "identity": { "id": "s2s-proxy-2", "address": "10.4.1.23:9234", "version": "1.41.2", "startTime": "2026-08-21T18:02:55Z" }, "state": "CONNECTION_STATE_CONNECTED", "muxSessionsConnected": 10, "muxSessionsTotal": 10 } ] }, { "name": "dr-standby", "state": "CONNECTION_STATE_ERROR", "muxSessionsTotal": 30, "members": [ { "identity": { "id": "s2s-proxy-0", "self": true, "version": "1.42.0", "startTime": "2026-08-24T09:14:02Z" }, "state": "CONNECTION_STATE_ERROR", "muxSessionsTotal": 10 }, { "identity": { "id": "s2s-proxy-1", "address": "10.4.1.19:9234", "version": "1.42.0", "startTime": "2026-08-24T09:14:07Z" }, "state": "CONNECTION_STATE_ERROR", "muxSessionsTotal": 10 }, { "identity": { "id": "s2s-proxy-2", "address": "10.4.1.23:9234", "version": "1.41.2", "startTime": "2026-08-21T18:02:55Z" }, "state": "CONNECTION_STATE_ERROR", "muxSessionsTotal": 10 } ] }, { "name": "eu-migration", "state": "CONNECTION_STATE_ERROR", "muxSessionsConnected": 20, "muxSessionsTotal": 20, "members": [ { "identity": { "id": "s2s-proxy-0", "self": true, "version": "1.42.0", "startTime": "2026-08-24T09:14:02Z" }, "state": "CONNECTION_STATE_CONNECTED", "muxSessionsConnected": 10, "muxSessionsTotal": 10 }, { "identity": { "id": "s2s-proxy-1", "address": "10.4.1.19:9234", "version": "1.42.0", "startTime": "2026-08-24T09:14:07Z" }, "state": "CONNECTION_STATE_CONNECTED", "muxSessionsConnected": 10, "muxSessionsTotal": 10 }, { "identity": { "id": "s2s-proxy-2", "address": "10.4.1.23:9234", "version": "1.41.2", "startTime": "2026-08-21T18:02:55Z" }, "state": "CONNECTION_STATE_ERROR" } ] } ] }Things to read out of that response:
dr-standbyisERRORwith nothing connected, and it is still listed by name with its members. A connection missing from this list is a connection missing from config.eu-migrationisERRORbecause the members disagree.s2s-proxy-2has no connection by that name, which is config drift across the group rather than a network problem. The coarse enum cannot yet say which of the two it is.muxSessionsTotalis summed over members that reported the connection. It is 30 forprod-migrationacross three members, and 20 foreu-migrationbecause only two members have it configured.s2s-proxy-0is markedselfand has noaddress. It served the call and was never dialled. Every other member was reached over the peer listener at the address shown.s2s-proxy-2is a version behind. The group is mid-rollout, visible as two distinct versions within one connection's member list.identityobject repeats. Each member's id and version appear once per connection it reports on, three times each here. That is the cost of the response having one list rather than two.muxSessionsConnectedis absent rather than0on the errored rows.initialFailoverVersionandfailoverVersionIncrementare quoted. protojson rendersint64as a string. Theint32counts are plain numbers.A member that answers nothing appears nowhere in this response. There is no roster to place it in, so a three-member group with one member down is indistinguishable from a two-member group.
At
memberscope eachclusterConnections[].membersholds exactly one row, the one markedself.