feat: reach external providers through a LiteLLM proxy with declared fallback

[Yash-Chindam]

Eighth PR closing gaps between the design spec and the implementation.

Gap this closes

§6/§7.1 name LiteLLM as the layer that normalizes external endpoints and centralizes fallback; §10 has "apply declared fallback if required". There was no external provider path: an external decision was sent to the local engine under the name approved-external-fallback.

What changed

• DispatchingBackend sends local decisions to the engine and external ones to a LiteLLM proxy (ROUTER_EXTERNAL_BASE_URL, ROUTER_EXTERNAL_API_KEY). Provider credentials live in the proxy.
• The privacy rule is enforced again at the dispatch boundary: a non-public request for an external model is refused (policy_violation) without contacting the provider.
• Separate circuit breakers for the engine and the provider.
• Fallback: when the local engine fails, a request already entitled to external routing (public, opted in, operator and tenant allow it) is re-routed. The route reason says so and router_fallbacks_total counts it.
• Start-up refuses external_fallback_enabled with the vLLM backend and no proxy address.
• config/litellm.yaml, and deploy/kubernetes/external-provider.yaml: proxy Deployment, Service, and a NetworkPolicy making it the only workload with internet egress, reachable only from the gateway.
• The manifest secret check now allows a secret-shaped key only when its value is an os.environ/ reference.

Not done

• Fallback applies to buffered requests only; a streamed request that fails is not re-routed.
• Nothing has been run against a real LiteLLM proxy or provider. The provider model in config/litellm.yaml is a placeholder for whichever provider is approved.
• The README does not document this yet.
• The Playwright suite was not re-run for this PR locally; CI runs it.

Test plan

• 24 new tests: dispatch, boundary refusal for private and restricted data, fallback on engine failure and out-of-memory, no fallback for ineligible requests, per-target circuits, manifest and proxy-config contracts
• ruff format, ruff check ., mypy clean
• pytest tests/unit tests/integration: 338 passed, coverage 98%

🤖 Generated with Claude Code