Skip to content

Commit 0398dad

Browse files
jirhikerclaude
andcommitted
fix(deploy): prevent App Engine request starvation under burst load
Prod cycled between 0-1 instances and returned site-wide 500/503 with "Request was aborted after waiting too long to attempt to service your request." Requests died in the pending queue (blank instanceId, ~2ms latency), not in app code — App Engine never scaled out past one instance despite max_instances=10. Root cause: the scheduler had no visibility into real per-instance concurrency (gunicorn -w 4), so it kept routing bursts to a single saturated instance instead of spinning up more. min_instances=0 added cold-start pile-ups on top. - app.template.yaml: add max_concurrent_requests: 6 so the scheduler scales out before an instance saturates (2-worker headroom for bursts) - CD_production.yml: MIN_INSTANCES 0 -> 1 to kill cold-start pile-ups - CD_production.yml: gunicorn -w 4 -> -w 8 for more concurrency per F4 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1 parent 0ac283d commit 0398dad

2 files changed

Lines changed: 3 additions & 2 deletions

File tree

.github/app.template.yaml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@ inbound_services:
88
automatic_scaling:
99
min_instances: ${MIN_INSTANCES}
1010
max_instances: ${MAX_INSTANCES}
11+
max_concurrent_requests: 6
1112
handlers:
1213
- url: /.*
1314
secure: always

.github/workflows/CD_production.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -148,8 +148,8 @@ jobs:
148148
run: |
149149
export MAX_INSTANCES="10"
150150
export SERVICE_NAME="ocotillo-api"
151-
export ENTRYPOINT="gunicorn -w 4 -k uvicorn.workers.UvicornWorker main:app"
152-
export MIN_INSTANCES="0"
151+
export ENTRYPOINT="gunicorn -w 8 -k uvicorn.workers.UvicornWorker main:app"
152+
export MIN_INSTANCES="1"
153153
envsubst < .github/app.template.yaml > app.yaml
154154
155155
- name: Deploy to Google Cloud

0 commit comments

Comments
 (0)