Skip to content

fix: Retry a Salesforce request once after refreshing the token - #70

Open
keelerm84 wants to merge 1 commit into
mainfrom
mk/SDK-2822/oauth-refresh-retry
Open

fix: Retry a Salesforce request once after refreshing the token#70
keelerm84 wants to merge 1 commit into
mainfrom
mk/SDK-2822/oauth-refresh-retry

Conversation

@keelerm84

@keelerm84 keelerm84 commented Aug 31, 2026

Copy link
Copy Markdown
Member

Summary

The bridge only learns its Salesforce token has expired by having a request refused,
because Salesforce ends the session on its own schedule. requestWithOauth handled that
notification and then discarded the benefit: it refreshed the token and returned the
original 401 to the caller, leaving the fresh token unused until the next cycle. So
every routine token expiry cost a full poll interval, on loops with a 30-second floor.

In featureLoop that cycle is wasted in a particular way -- the push failure jumps past
etag = pollResponse.Header.Get("ETag"), so the cycle spends a full flag poll against
LaunchDarkly and a push against Salesforce and stores nothing, and the org keeps serving
stale flag data for another interval. In eventLoop the refused request is the drain of
/event, so events accumulate rather than being collected.

requestWithOauth is now straight-line: send, return if the status is not 401 or 403,
otherwise finish with the rejection, refresh, send once more, return the second response.
An expiry now costs one round trip instead of a cycle.

The body replay is the subtle part

client.Do consumes and closes the request body while writing it, so sending one
*http.Request twice would write the payload once and nothing after.

The tempting story is that bytes.NewBuffer causes this and bytes.NewReader cures it.
That is not so: http.NewRequest builds a GetBody closure for *bytes.Buffer and
*bytes.Reader alike. The actual fix is sendWithToken calling GetBody() and
reinstalling a fresh body before each send.

What makes it non-optional is how it fails when omitted. The transport sees a request
that declared a ContentLength and wrote nothing, classifies it as
nothingWrittenError, and rewinds through GetBody itself -- but only when the attempt
went out on a reused connection. On a fresh dial the send fails with
http: ContentLength=N with Body length 0. Omitting the replay therefore passes on a warm
pool and breaks intermittently in production, exactly when the connection is fresh or was
reaped while idle, which is the situation around a session expiry. The body test forces
Connection: close on the rejection so it can distinguish a real replay from the
transport covering for a missing one.

bytes.NewReader is kept at the flag push for a different, stated reason: a Reader is a
read-only view of pollBytes, where a Buffer is a writable staging area that
http.NewRequest snapshots once. Both poll requests pass a nil body, so GetBody is nil
and the replay skips them with no special case.

Response ownership, and a leak

When the refresh itself failed, the old code returned nil, err, permanent and never
drained or closed the 401 response. Go can only pool a connection once its body reaches
EOF and is closed, and eventLoop treats a non-permanent error as recoverable and keeps
looping -- so a token endpoint failing transiently leaked one connection per cycle
indefinitely. The rejection is now drained before authorizing, which also returns that
connection to the pool in time for the token request to reuse it.

The rule is now checkable by eye: the caller owns the returned response, and every
response the function does not return is released inside it. No path leaves one open and
none is closed twice.

Bounded by structure

The second send is a single statement -- no loop, no recursion -- so a second rejection is
returned as it stands. A token minted seconds ago being refused is not an expiry; it means
the connected app permissions or its run-as identity changed, which refreshing again
cannot fix. Looping would spend the org API allocation on a rejection that repeats.

Not addressed here

The refresh itself gets no retry, there is no proactive refresh ahead of expiry, and no
backoff, jitter or general status classification -- the ticket tracks those separately.

Each of the three changes was mutation-tested independently: reverting requestWithOauth
fails the retry and loop tests, deleting only the GetBody replay fails the two body
tests, and deleting only the drainAndClose fails the connection-count test. Verified on
Go 1.26 and on the CI-pinned Go 1.15.


Note

Overview
requestWithOauth now retries the same Salesforce call once after a 401/403, instead of refreshing the token and still returning the original rejection. Callers get the second response, so routine session expiry costs one extra round trip rather than a full poll interval of stale flags or backed-up events.

The send path is split into sendWithToken, which re-applies the current bearer token (and scope header) and replays the request body via GetBody() before each client.Do, so a second attempt is safe when the first send consumed the body. Flag pushes use bytes.NewReader instead of bytes.NewBuffer so both attempts stay a read-only view of the polled payload.

On auth failure the 401/403 response is drained and closed before refresh, fixing a connection leak when refresh failed transiently and clarifying response ownership. Retries are capped at one after refresh; other status codes are unchanged.

Adds oauth_retry_test.go covering retry success, single-retry limits, non-auth statuses, full body replay on flag push, ETag advancement after a retried push, and connection reuse when refresh fails.

Reviewed by Cursor Bugbot for commit 56f787a. Bugbot is set up for automated code reviews on this repo. Configure here.

requestWithOauth detected a 401 or 403, refreshed the OAuth token, and then
returned the original rejection without using the new token. The caller saw a
non-200, logged it and waited out a whole poll interval, so every routine
Salesforce session expiry cost a full cycle of stale flag data or undelivered
events. It now sends the request a second time with the refreshed token and
returns that response.

The retry is not repeated. A token minted seconds ago being refused is not an
expiry, so another refresh cannot change the answer -- it means the connected
app's permissions or its run-as identity changed. The second response is
returned as it stands, which keeps a standing misconfiguration from turning
into unbounded requests against the org's API allocation. This change adds no
backoff and no general retryable-status classification; the only retry is the
one that follows a successful refresh.

The flag push now builds its body with bytes.NewReader instead of
bytes.NewBuffer, and requestWithOauth replays the body from Request.GetBody
before each send. Both readers give http.NewRequest enough to build a GetBody,
so either replays correctly, but a Reader is a read-only view of the polled
bytes where a Buffer is a writable staging area that http.NewRequest snapshots
once -- so the Reader states at the call site the guarantee the retry depends
on. Without the explicit replay the second attempt fails only intermittently:
the transport covers for a missing body by rewinding through GetBody itself,
but only when the attempt went out on a pooled connection, so the failure
appears just when the retry has to dial.

The rejection is now drained and closed on both paths. Previously, when the
refresh that followed a 401 failed transiently, requestWithOauth returned a nil
response and the rejection was released by nobody; eventLoop and featureLoop
treat a non-permanent error as recoverable, so a bridge leaked a connection
every cycle for as long as the refresh kept failing. Now that no rejection is
handed back to the caller, requestWithOauth is the only place either path can
release one.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant