fix: coordinate REST API rate limits and bound retries

[KAJdev]

Honor Retry-After and use jittered exponential backoff when no valid hint is available. Retry and wait budgets surface RateLimitError when exhausted.

[Copilot]

🟡 Changes recommended

Quota reconciliation can incorrectly block available capacity, and retry-delay calculation and reporting need correction.

4 open findings

• In-flight reservations are double-counted during header reconciliation · New
• Quota rollovers before the local deadline are not recognized · New
• Naive HTTP dates are interpreted in the host's local timezone · New
• Returned cooldown omits the existing shared cooldown · New

What changed in this PR

Adds coordinated rate-limit handling to the management REST transport for synchronous and asynchronous callers.

Changes:

• Shares quota reservations across routes by origin and API key.
• Adds bounded retries, Retry-After handling, and jittered backoff.
• Introduces RateLimitError, documentation, and regression tests.

File	Description
tests/​test_api/​test_rate_limit.py	Tests quota coordination, retries, budgets, and cancellation.
runpod/​error.py	Adds RateLimitError with retry-delay details.
runpod/​api/​rest.py	Integrates shared quotas and bounded retries.
runpod/​api/​_rate_limit.py	Implements parsing, reservations, cooldowns, and registry management.
docs/​api/​handling_errors.md	Documents rate-limit behavior and configuration.

🧠 Review effort: Balanced

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[deanq]

10 inline comments below.

[deanq]

After a window passes reset_at and remaining is None, every acquire() with something in flight takes the now >= reset_at branch again and returns 0.05. The check doesn't look at whether remaining was already cleared.

Result: after each minute reset, concurrent callers drop to one-at-a-time with 50ms polling until a response refreshes the window (and if the server stops sending that window, until the next update deletes it). Combined with the budget accounting above, this can raise RateLimitError with quota available.

[deanq]

Still present in 3d1c1a2 (_rate_limit.py:92-96). The now >= reset_at branch rewrites the window to (None, reset_at) and sets probing = True whenever in_flight, without checking whether remaining is already None. So an already-reset window keeps forcing the 50ms probe path on every acquire() until a response refreshes it. Guarding with remaining is not None (or dropping the window once reset) should fix it. A test with two concurrent requests after a reset, asserting the second is not serialized, would cover it.

[deanq]

Waiting on in-flight requests is a fixed 50ms busy-poll. N waiters each wake 20x/s and take the bucket lock, and the async path takes a threading.Lock on the event loop. A threading.Condition notified from release()/update() (or for async, a computed sleep until the next release) removes the wakeups and up to 50ms of added latency.

[deanq]

Still present in 3d1c1a2 (_rate_limit.py:104, return 0.05). The budget accounting part is fixed, since the probe delay no longer goes through budget.wait(), but waiters still busy-poll at 20 Hz under the bucket lock, and the async path still takes a threading.Lock on the event loop. A threading.Condition notified from release()/update() would remove the wakeups and the added latency.