fix(deploy): prevent App Engine request starvation under burst load

[jirhiker]

What

Three App Engine scaling fixes to stop site-wide 500/503 outages under burst load:

• app.template.yaml — add max_concurrent_requests: 6
• CD_production.yml — MIN_INSTANCES 0 → 1
• CD_production.yml — gunicorn -w 4 → -w 8

Why

Prod cycled between 0 and 1 instances and returned site-wide 500/503 — including / and /health. Every failing request logged:

ERROR - Request was aborted after waiting too long to attempt to service your request.

Status 500/503, latency ~2ms, blank instanceId = App Engine killed the request in the pending queue; it never reached app code. Not OOM (zero "Exceeded soft memory limit" lines), not a code crash (same-instance requests returned 200).

Root cause: the scheduler had no visibility into real per-instance concurrency (gunicorn -w 4), so under a burst (map UI firing many ?f=json&limit=10000 collection requests at once) it kept routing to a single saturated instance instead of scaling out toward max_instances=10. min_instances=0 added cold-start pile-ups.

How the fixes address it

1. max_concurrent_requests: 6 gives the scheduler an explicit per-instance ceiling → scales out before an instance saturates. Set below the 8 workers on purpose: 2-worker headroom absorbs bursts (safer than packing to 8).
2. MIN_INSTANCES=1 keeps one warm instance → no cold-start pile-ups.
3. -w 8 doubles concurrency per F4 (1GB) instance.

Reviewer notes

• Config-only; no app code touched. Effective on next production deploy.
• Cap intentionally 6 < 8 workers for scale-out headroom — not a typo.
• Separate pre-existing pygeoapi SensorThings KeyError: '@iot.id' 500s seen in logs are not addressed here (unrelated per-item bug).

🤖 Generated with Claude Code