fix: harden IBKR runtime against historical failures

[Pigbibi]

Summary

• add a deterministic production WSGI startup validator to CI and the Docker image build
• re-resolve zoned GCE gateway addresses on every IBKR connection retry
• classify exhausted gateway connection failures as ibkr_gateway_unavailable and return retryable HTTP 503
• add regression coverage for startup-gate wiring, host refresh, and connection-failure reporting

Historical evidence (2026-06-21 through 2026-07-21)

Across the four gateway services, Cloud Logging showed:

• u159: 17 x 429, 20 x 500, 276 x 503, 1 x 504
• u166: 5 x 500, 3 x 503
• u18308207: 17 x 503
• u18336562: 2 x 500, 3 x 503

The dominant 503 clusters were production startup/import failures, including:

• TypeError: non-default argument 'connect_ib_fn' follows default argument
• missing/incompatible QuantPlatformKit APIs (load_strategy_entrypoint, common.health, try_record_platform_execution, notifications.renderer_base)
• duplicate Flask health endpoint registration

The operational failures were:

• /monitor-dispatch 429s while the single instance was occupied (the preceding fix: harden IBKR precheck runtime dispatch #353 already removed the self-call/schedule collision)
• repeated TCP connection refusals for u166
• IBKR handshake timeouts for u18336562 and u159

The startup validator imports the same locked production dependency set used by the image and verifies the required Flask route contract without contacting IBKR. It therefore fails CI and docker build before a broken image can be pushed or deployed.

Verification

• uv run --no-sync ruff check --exclude external .
• 21 focused tests passed
• python scripts/validate_cloud_run_startup.py passed with 7 registered routes
• current deployed revisions: zero ERROR/4xx/5xx entries since 2026-07-20T21:54Z at time of analysis

[github-actions]

🤖 Codex PR Review

✅ Merge allowed: 1 finding(s) reported but none are blocking

ℹ️ Other Findings

1. 🟡 [MEDIUM] Logic in application/runtime_broker_adapters.py

connect_ib() now raises IBKRGatewayUnavailableError after exhausting retries for every exception from its try block, not just transient socket/handshake failures. That try block also covers post-connect checks such as validate_configured_accounts(), so a successful IBKR login with the wrong account entitlement will now be surfaced by main._handle_request() as failure_category="ibkr_gateway_unavailable" with HTTP 503. This is a permanent configuration/auth mismatch, not a retryable gateway outage, so callers will be told to retry a request that cannot succeed until the configuration is fixed. (line 137)

Suggestion: Only retry and wrap transient connection failures (for example TimeoutError, ConnectionError, or the specific connect-library exceptions). Let deterministic post-connect validation/configuration errors bubble unchanged, or map them to a separate non-retryable failure category/status instead of ibkr_gateway_unavailable.

---

Review by Codex PR Review bot • PR

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b1870016db

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".