Back off between live FedEx sandbox retries - #10
Closed
freshlogic wants to merge 2 commits into
Closed
Conversation
The live sandbox tests wrap their calls in async.retry, but with default options: five attempts at a zero interval. On 2026-07-25 the sandbox answered 503 SERVICE.UNAVAILABLE.ERROR for a SMART_POST rate request and all five attempts landed inside the same ~15s outage window (test/index.js:645), so the retry bought nothing and main went red. The test itself is sound — it passed in 1.8s and 2.3s on the two prior green runs. Give async.retry an exponential interval (2s, 4s, 8s over four attempts) so a short vendor blip is actually ridden out. Verified against a local server that 503s for a fixed 5s window: the default config fails after burning five attempts in 24ms, the new config passes on attempt three. Healthy runs are unaffected — the backoff only applies after a failed attempt. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…bin.org The non-200 access-token test pointed at https://httpbin.org/status/500, which is unreliable: it answered in 20886ms on run 30162824011 and returned '503 Service Temporarily Unavailable' rather than a 500 on run 30281894500, failing the /^500/ assertion. That is the same third-party dependency that turned dhl-ecommerce-solutions red. Stand up a throwaway node:http server answering 500 for the duration of the test. The assertions are unchanged and the test now runs in 33ms without network access. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Coverage Report for CI Build 30282039568Warning No base build found for commit Coverage: 100.0%Details
Uncovered ChangesNo uncovered changes found. Coverage RegressionsRequires a base build to compare against. How to fix this → Coverage Stats
💛 - Coveralls |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
async.retrywith default options — 5 attempts at a zero interval.503 SERVICE.UNAVAILABLE.ERRORfor the SMART_POST rate request. All 5 attempts fired inside the same ~15s outage window (15563ms ÷ 5 ≈ 3.1s each), so the retry was decorative andmainwent red.Root cause
Not a code defect and not a permanently-broken sandbox:
should return rate quotes for a SmartPost shipmentpassed in 1792ms (run 28098919820) and 2307ms (run 26487320976) on the two prior green runs, so SMART_POST is provisioned and working. The 503 was a transient vendor outage that the zero-interval retry could not survive.The
startup_failurecurrently showing on main is a separate artifact — attempts 1 and 2 of run 30162824011 were genuinefailures, and the run was subsequently re-run up toattempt: 50by the CI watcher routine, which is what produced thestartup_failurestate.Test plan
default (times:5, interval:0)→ FAIL, 5 attempts in 24msnew (times:4, backoff)→ PASS, 3 attempts over 6008msnpx eslint .cleanFEDEX_*credential-driven and pass in CI)Healthy runs incur no delay — backoff only applies after a failed attempt.
🤖 Generated with Claude Code