Skip to content

Rewrite force replication's verify target for pre-1.22.2 servers - #259

Open
JayChung0258 wants to merge 1 commit into
mainfrom
jayhung/s2c-fr-endpoint-override
Open

Rewrite force replication's verify target for pre-1.22.2 servers#259
JayChung0258 wants to merge 1 commit into
mainfrom
jayhung/s2c-fr-endpoint-override

Conversation

@JayChung0258

@JayChung0258 JayChung0258 commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

What

Adds an opt-in translator that rewrites ForceReplicationParams.TargetClusterEndpoint in the StartWorkflowExecution request that starts the force replication workflow, pointing it at this proxy's replicationEndpoint.

Why

Temporal servers older than v1.22.2 ignore TargetClusterName in the VerifyReplicationTasks activity and dial TargetClusterEndpoint verbatim. The caller fills that with the remote cluster's own address, which is generally not routable from the local cluster. Replication succeeds, verification never completes, and the migration sits in progress while the activity retries for a week.

The routable address is the one this proxy already writes into the local cluster's registry when it rewrites FrontendAddress during AddOrUpdateRemoteCluster. Same correction, different carrier: that one is a typed proto field, this one rides inside a workflow-args payload.

TargetClusterName is left untouched, so servers from v1.22.2 on prefer the name and ignore this field whether or not the flag stays set.

Config

clusterConnections:
  - name: my-migration-cluster
    replicationEndpoint: "my-s2s-proxy.svc.cluster.local:9233"
    forceReplicationEndpointOverride: true

Off by default. The translator is only constructed when the flag is true and replicationEndpoint is non-empty. Belongs on the self-hosted side, not on a cloud-side proxy.

Scope

Inbound only, StartWorkflowExecution only, and only when the workflow type is force-replication. It type-asserts before anything else, which keeps it inert on the stream path where MatchMethod is not consulted. Only the one JSON key is replaced; every other param and the payload metadata are preserved.

Testing

Unit tests cover the rewrite, other workflow types, other RPCs, near-miss method names, nil input, empty payloads, a nil payload element, wrong encoding, malformed JSON, a missing key, idempotency, and that other params survive.

go build ./..., go test -race -tags test_dep ./interceptor/... ./config/... and make lint are all clean.

Known gap: no e2e case. StartWorkflowExecution is stubbed to PermissionDenied in the echo fixture, so one would require implementing it there first. Follow-up.

Temporal servers older than v1.22.2 ignore TargetClusterName in the
VerifyReplicationTasks activity and dial TargetClusterEndpoint verbatim.
The caller fills that with the remote cluster's own address, which is
generally not routable from the local cluster, so verification never
completes and the migration stalls while replication itself succeeds.

Add an opt-in translator that rewrites that one field to this proxy's
replicationEndpoint, which is the same correction already applied to
FrontendAddress in AddOrUpdateRemoteCluster. That one is a typed proto
field; this one rides inside a workflow-args payload.

TargetClusterName is left alone, so servers from v1.22.2 on are
unaffected whether or not the flag stays set.

Scoped narrowly: inbound only, StartWorkflowExecution only, and only
when the workflow type is force-replication. Enabled by a new
forceReplicationEndpointOverride key, which also requires
replicationEndpoint to be set.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@JayChung0258
JayChung0258 requested a review from a team as a code owner August 24, 2026 04:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant