choiyounggi · choiyounggi · Aug 5, 2026
@@ -9,15 +9,15 @@ follow the cross-pointers in their index or take the next matching seeded domain
 
 | Domain | Status | Route here when |
 |--------|--------|-----------------|
-| [databases](wiki/databases/index.md) | **seeded** | Designing schemas/tables/keys, choosing or evaluating indexes, writing or optimizing queries, choosing transaction/isolation behavior |
-| [backend](wiki/backend/index.md) | **seeded** | Server-side application code — language-agnostic (`common/`: API contracts, idempotency, JWT, timeouts/retries, caching, jobs, transactions in app code, shared state/pools, errors, LLM completion validation & context budgeting, consuming external-API responses, externally-owned defaults, object-storage references) plus stack subtrees: `java/` (JPA, Spring proxies, JVM threads/memory), `node/` (event loop, promises, runtime validation, shutdown), `python/` (GIL/asyncio, pydantic, WSGI/ASGI workers, language traps) |
+| [databases](wiki/databases/index.md) | **seeded** | Designing schemas/tables/keys, choosing or evaluating indexes, writing or optimizing queries, choosing transaction/isolation behavior, verifying that a hand-rolled additive migration reaches an already-deployed database |
+| [backend](wiki/backend/index.md) | **seeded** | Server-side application code — language-agnostic (`common/`: API contracts, idempotency, JWT, timeouts/retries, caching, jobs, transactions in app code, shared state/pools, errors, LLM completion validation & context budgeting, consuming external-API responses, externally-owned defaults, object-storage references, pacing calls under a provider rate limit) plus stack subtrees: `java/` (JPA, Spring proxies, JVM threads/memory), `node/` (event loop, promises, runtime validation, shutdown), `python/` (GIL/asyncio, pydantic, WSGI/ASGI workers, language traps) |
 | [frontend](wiki/frontend/index.md) | **seeded** | Web UI code: state placement, rendering performance, in-UI data fetching (races, infinite scroll), auth token handling, forms, XSS-safe output, accessibility |
 | [infrastructure](wiki/infrastructure/index.md) | **seeded** | CI/CD pipelines, secrets in build/deploy, container image builds, rollout/rollback strategy, observability (logs/metrics/alerting) |
-| [testing](wiki/testing/index.md) | **seeded** | Writing or structuring automated tests: level choice, cases/assertions, test data, mock decisions, flaky tests (release-process quality → qa) |
+| [testing](wiki/testing/index.md) | **seeded** | Writing or structuring automated tests: level choice, cases/assertions, test data, mock decisions, flaky tests, testing code that bulk-deletes resources from a machine-wide daemon (release-process quality → qa) |
 | [qa](wiki/qa/index.md) | **seeded** | Release-quality process: release gates, regression scoping, bug reports, severity/priority triage, exploratory testing, automated verification of document deliverables (spec/RFC gates) (writing automated test code → testing) |
 | [debugging](wiki/debugging/index.md) | **seeded** | Diagnosing a failure — finding what is wrong and why: reproducing, bisection, hypothesis testing, traces/logs, intermittent failures (fixing the diagnosed fault → its owning domain) |
 | [security](wiki/security/index.md) | **seeded** | Trust-boundary decisions: input validation, session-vs-token auth choice, per-resource authorization (IDOR), secrets hygiene, dependency trust, PII handling (XSS rendering → frontend; CI secrets → infrastructure; JWT implementation → backend/frontend auth) |
-| [platforms](wiki/platforms/index.md) | **seeded** | OS-level differences breaking code across macOS/Linux/Windows: shell portability, BSD-vs-GNU CLI, filesystem case/line endings, Unicode normalization in text/file-name matching, commands inspected before execution, background services/cron, invoking prompt-capable CLIs non-interactively, toolchain version pinning |
+| [platforms](wiki/platforms/index.md) | **seeded** | OS-level differences breaking code across macOS/Linux/Windows: shell portability, BSD-vs-GNU CLI, filesystem case/line endings, Unicode normalization in text/file-name matching, commands inspected before execution, background services/cron, invoking prompt-capable CLIs non-interactively, driving a TUI in a tmux pane, agent-harness hooks that replace a tool's result, toolchain version pinning |
 | [mobile](wiki/mobile/index.md) | **seeded** | App-side iOS/Android/cross-platform: process death/state survival, offline-first sync, mobile-network calls, store rollout/hotfix strategy, startup time |
 
 All ten domains are seeded. New categories grow via `skills/wiki-ingest/SKILL.md`.
@@ -37,3 +37,5 @@ Append-only. Format: `## [YYYY-MM-DD] <ingest|revise|lint|gap|contradiction|drif
 ## [2026-08-03] ingest | Consolidated review of knowledge PRs #6–#13 (8 fork PRs) into 12 pages. New: backend/common/llm (completion-response-validation, context-window-budget), backend/common/integrations (externally-owned-defaults), backend/common/storage (object-key-persistence), infrastructure/containers/host-cgroup-visibility, infrastructure/observability/missing-container-metrics, platforms/environment/unicode-text-matching, platforms/shells/command-text-inspected-before-execution, platforms/processes/non-interactive-cli-invocation, qa/document-verification (spec-document-gates, editing-a-gated-document), testing/quality (checks-that-cannot-pass, spec-artifact-checks, harness-reverse-controls). All cited URLs are per-PR live-verified; three pages were reconciled from two overlapping PR versions each (see revise/dedup entries below).
 ## [2026-08-03] revise | Reconciled 3 pages from overlapping PR pairs, taking the more complete/better-sourced body and folding in the other's unique cases: backend/common/llm/completion-response-validation (#12 body — tool_calls/function_call carve-out, streaming, Responses API status==incomplete, "reasoning is scratch, not deliverable" — kept in llm/ per #6/#13 category, folded in #6's DeepSeek-first-party edge + the 8,173-char reasoning_content field incident); backend/common/integrations/externally-owned-defaults (#12 generalized body — any repo-external resource — folded in #6's LiteLLM-alias-removed field incident + gateway-config-vs-live-upstream nuance); platforms/processes/non-interactive-cli-invocation (#12 body — GNU-nohup extension precision, ssh -n stdin-detach vs BatchMode, pre-log DNS/TLS/proxy + curl -v — folded in #11's DEBIAN_FRONTEND, pager/color TTY case, wrapper-CLI case, and the no-request-in-gateway-log field incident).
 ## [2026-08-03] dedup | Dropped 3 candidate pages as duplicates/superseded during the #6–#13 consolidation: testing/docs-as-spec/document-conformance-checks (#9 — same case as testing/quality/spec-artifact-checks from #8: coverage-vs-validity split, per-check negative controls, GFM pipe parsing; #8 kept as canonical, docs-as-spec category not created); backend/common/llm/gateway-model-alias-defaults (#6 — subsumed by the generalized integrations/externally-owned-defaults; the model-alias case is one instance); backend/common/integrations/llm-response-completeness (#12 — folded into llm/completion-response-validation, kept in llm/ for category coherence with context-window-budget).
+## [2026-08-05] ingest | 5 new pages from a 9-candidate knowledge flush. databases/schema-design/verifying-additive-migrations (create_all() skips existing tables whole, so only the hand-written ALTER adds a column; the only test state that exercises the migration path is the *previous* shape — DROP the new columns, re-run init, assert information_schema type/nullability/default plus what a pre-existing row reads back; restore in finally). platforms/processes/driving-a-tui-in-a-tmux-pane (a capture-pane diff is not delivery evidence — the tty echoes keystrokes while the process is busy; check the TUI's busy indicator first, then an effect only it can produce; pass send-keys payloads after `--`). platforms/processes/harness-tool-result-interception (PostToolUse `updatedToolOutput` replaces a tool's result, so a one-line read may be substitution — size-check with wc, try the hook's remediation once, then switch to grep -n + awk and put the fallback in every spawn brief). testing/mocking/destructive-operations-on-shared-daemons (never point a pattern-sweep test at the live daemon; PATH-inject a recording fake, assert targets present AND bystanders absent, plus a deletes-nothing control). backend/common/reliability/client-side-rate-limit-pacing (auth/token requests issued inside header-building bypass a method-level throttle — throttle at the lowest HTTP-issuing layer, stamp the clock immediately before the send, reproduce with a cold credential cache). Sources: SQLAlchemy metadata/defaults, PostgreSQL ddl-alter, POSIX chap02/11/12, tmux.1, Claude Code hooks, Okta/Auth0/GitHub rate-limit docs.
+## [2026-08-05] revise | platforms/shells +2 merges (no new pages — both candidates were the same cases as existing pages). portable-shell-scripts: step 5 gains the POSIX-sh `set --` reordering idiom with the rule that it must stay inline in the dispatcher (POSIX 2.9.5 restores a function's caller positional parameters, so a `parse_flags` helper detects the flag via globals while leaving the caller's operand list unfiltered — a silent wrong-argument run), plus new step 6 on single- vs double-quoting a text payload (POSIX 2.2.3: backquote keeps its command-substitution meaning inside double quotes, so a quoted command example executes and the message ships gutted at exit 0); 3 edge cases + 3 Instead-of rows; reproduced under /bin/sh, dash and zsh. command-text-inspected-before-execution: new steps 6-7 — a blocked command's empty stdout is byte-identical to a silent success, so stat the artifact rather than trusting silence, and hand a blocked status signal back as un-emitted instead of routing around the gate; 2 edge cases + 2 Instead-of rows; field context from a worktree_escape guardrail blocking the orchestrator's own status-update.sh (Write to the same tree succeeded, proving the block is command-text-scoped). Both last_verified bumped to 2026-08-05.
@@ -0,0 +1,80 @@
+---
+id: backend-common-reliability-client-side-rate-limit-pacing
+domain: backend
+category: reliability
+applies_to: [general]
+confidence: field-tested
+sources:
+  - https://developer.okta.com/docs/reference/rate-limits/
+  - https://auth0.com/docs/troubleshoot/customer-support/operational-policies/rate-limit-policy
+  - https://docs.github.com/en/rest/using-the-rest-api/rate-limits-for-the-rest-api
+last_verified: 2026-08-05
+related: [backend-common-reliability-timeouts-and-retries, backend-common-auth-jwt-server-side, debugging-methodology-hypothesis-testing]
+---
+
+# Pacing Your Own Calls Under a Provider's Per-Second Quota
+
+## When this applies
+
+You wrapped an external API client with a throttle (minimum interval, token
+bucket) to stay under a documented per-second quota, and rate-limit errors still
+appear — characteristically on the first call after process start, or on some
+days and not others.
+
+## Do this
+
+1. **Count every HTTP request the client issues, including the ones it makes to
+   get credentials.** Token issuance and refresh normally happen inside
+   header-building or a request interceptor — below the layer the throttle
+   decorates — so a token POST and the API GET it enables leave in the same second
+   and deterministically exceed a small quota. Providers rate-limit their token
+   endpoints as their own buckets (Okta publishes per-endpoint buckets for
+   `/oauth2/v1/token`; Auth0 publishes a `/oauth/token` limit), and OAuth-app
+   traffic counts against the caller's quota (GitHub). Route the credential request
+   through the same throttle as everything else.
+
+2. **Place the throttle at the lowest layer that issues HTTP** — the send/execute
+   method, not the public wrapper methods. That is the only position auth refresh
+   cannot bypass.
+
+3. **Stamp the throttle clock immediately before the request goes out**, not on
+   entry to the wrapper. Anything between the stamp and the send — token refresh,
+   payload serialization, signing — shortens the real gap below the interval you
+   think you are enforcing.
+
+4. **Check what the first call of a process does.** A "last request at" state
+   initialized to zero correctly exempts the first call from waiting; the defect is
+   that an unthrottled token request then fires inside it, making two requests
+   where the throttle counted one. Assert the first call's request count, not just
+   its spacing.
+
+5. **Reproduce with a cold credential cache.** Clear the cached token, then run.
+   A live token removes the extra request, which is why the failure looks
+   intermittent and gets filed as a flaky provider.
+
+## Edge cases
+
+| Case | Then |
+|------|------|
+| Several processes or workers share one API key | An in-process throttle cannot see the others — move pacing to a shared store (Redis token bucket) or divide the quota explicitly per worker; the per-process interval is not the quota |
+| The token is cached on disk and shared between runs | Only the first process after expiry pays the extra request — schedule a warm-up call at startup so the cost lands outside a user-facing request |
+| The provider's limiter is a sliding window | Pace below the nominal limit rather than exactly at it; requests bunched at a window boundary breach a limit that per-second averaging satisfies |
+| The provider publishes a separate quota for token issuance | Confirm it in their docs before exempting it from the shared throttle; when the limits are unpublished, count it |
+| A 429 still arrives despite correct pacing | Honour `Retry-After` and back off — pacing prevents self-inflicted breaches, it does not cover provider-side or cross-tenant limits ([backend-common-reliability-timeouts-and-retries]) |
+| Mock/sandbox credentials have a lower quota than production | Pace to the environment's own limit, read at startup from config — a throttle tuned to production silently breaches in sandbox |
+
+## Instead of
+
+| If you are about to | Do this instead | Why |
+|---------------------|-----------------|-----|
+| Decorate the public API methods with the throttle | Put it at the lowest HTTP-issuing layer | Requests issued from header-building or an interceptor never pass through a method-level decorator |
+| Record the throttle timestamp when the wrapper method is entered | Record it immediately before the send | Token refresh between the two makes the enforced interval shorter than the configured one |
+| File a first-call rate-limit error as an intermittent provider fault | Re-run with the credential cache cleared and count the outbound requests | The bug reproduces only on token-issuance runs, so most runs are green and hide it |
+| Raise the throttle interval until the errors stop | Count the requests per second the client actually emits | Padding the interval hides an uncounted request instead of counting it, and it returns under any change in timing |
+
+## Sources
+
+- https://developer.okta.com/docs/reference/rate-limits/ — rate-limit buckets are per-endpoint collections sharing a quota; OAuth endpoints (`/oauth2/v1/authorize`, token) carry their own buckets, with nested per-client quotas
+- https://auth0.com/docs/troubleshoot/customer-support/operational-policies/rate-limit-policy — the `/oauth/token` endpoint has a published production rate limit, i.e. token issuance is metered like any other call
+- https://docs.github.com/en/rest/using-the-rest-api/rate-limits-for-the-rest-api — requests made by an OAuth/GitHub App on a user's behalf "count towards" that user's rate limit
+- Field context: a trading-API client throttled its public methods, while `_headers()` called `_throttle()` and then `_get_token()`. On the two days the cached token had expired, the logs show token POST at `…:00.354`, issuance at `…:00.495`, and the following balance call rejected at `…:00.543` — two requests inside one second against a 2/second quota. On days with a valid cached token the identical code passed
@@ -31,6 +31,7 @@ Match your situation to a "load when" line; load only matching pages.
 | Page | Load when |
 |------|-----------|
 | [timeouts-and-retries](common/reliability/timeouts-and-retries.md) | Your service calls another service/external API/DB over the network — setting timeouts and deadlines, deciding what to retry per failure type, backoff/jitter, capping concurrency against a slow dependency; debugging pool exhaustion or retry storms |
+| [client-side-rate-limit-pacing](common/reliability/client-side-rate-limit-pacing.md) | You throttle your own calls to stay under a provider's per-second quota and rate-limit errors still occur; deciding which layer the throttle belongs at and when its clock is stamped; auth/token requests issued inside header-building or an interceptor; a rate-limit failure that only reproduces on the process's first call or on days the credential cache is cold; sharing one API key across workers |
 
 ### caching
 

@@ -38,6 +38,7 @@ Match your situation to a "load when" line; load only matching pages.
 | [nullability-and-defaults](schema-design/nullability-and-defaults.md) | Declaring column nullability/defaults; queries dropping rows around NULLs |
 | [soft-delete](schema-design/soft-delete.md) | Deleted records themselves must be restorable or kept (deleted_at schemas); deciding what a parent's deletion does to children that must survive (for who-changed-what history → requirements-to-tables) |
 | [online-schema-changes](schema-design/online-schema-changes.md) | Running ALTER TABLE / CREATE INDEX on a large table under live traffic; a migration blocks reads/writes (ACCESS EXCLUSIVE); adding a column/constraint/NOT NULL/index/type change safely; expand-and-contract to decouple DB migration from app deploy |
+| [verifying-additive-migrations](schema-design/verifying-additive-migrations.md) | The project has no migration tool and schema comes from an ORM `create_all()` plus hand-written `ALTER TABLE ADD COLUMN IF NOT EXISTS`; writing the test that proves a new column reaches an already-deployed database; deciding what a migration test's starting DB state must be; asserting a column's type/nullability/default from the catalog; a column added on the model never appeared on the deployed table |
 
 ## operations