Clarify AI Gateway response-cache contract and metadata

[stackbilt-admin]

Summary

AI Gateway response caching is different from provider prompt/prefix caching. Cloudflare's current AI Gateway caching docs make the distinction concrete:

• Gateway cache only applies to identical requests unless callers provide cf-aig-cache-key.
• Per-request controls are cf-aig-cache-ttl, cf-aig-skip-cache, and cf-aig-cache-key.
• Responses expose cache status with cf-aig-cache-status as HIT or MISS.

llm-providers already has pieces of this surface (GatewayMetadata.cacheKey, GatewayMetadata.cacheTtl, ResponseCacheAdapter, and CacheHints.strategy: 'response' | 'both'), but downstream consumers still need to know too much about which fields affect Cloudflare AI Gateway versus local response-cache adapters.

Docs:

• https://developers.cloudflare.com/ai-gateway/features/caching/

Current code surface

• BaseProvider.getAIGatewayHeaders() forwards cf-aig-cache-key and cf-aig-cache-ttl when the provider base URL is AI Gateway.
• GatewayMetadata does not expose skipCache.
• CacheHints.strategy: 'response' is documented, but does not itself create Gateway response-cache headers or an explicit local adapter policy.
• Provider response metadata does not normalize cf-aig-cache-status when Gateway returns it.
• The factory-level ResponseCacheAdapter key is internal and separate from AI Gateway's cache key behavior.

Proposed work

1. Add explicit GatewayMetadata.skipCache?: boolean mapped to cf-aig-skip-cache: true for HTTP-provider Gateway calls.
2. Add response metadata normalization for cf-aig-cache-status where provider adapters can access response headers.
3. Document how CacheHints.strategy relates to GatewayMetadata and ResponseCacheAdapter:
   • provider-prefix: provider/native prompt cache only.
   • response: response-cache policy only, but caller must provide Gateway metadata or a response cache adapter.
   • both: both layers, still separately observable.
4. Consider a small helper for deterministic cache key generation that consumers can call before dispatch, so gateway/custom cache keys are stable without copying factory internals.
5. Add tests that prove old Cloudflare headers still work and the new skip-cache/status surfaces are backward-compatible.

Acceptance criteria

• GatewayMetadata.skipCache is exported, documented, and mapped to the current Cloudflare header name.
• Provider responses routed through AI Gateway can expose normalized cache status, at least as response.metadata.aiGatewayCacheStatus.
• README shows a minimal AI Gateway response-cache example with cacheKey, cacheTtl, and skipCache.
• Tests cover new headers and avoid sending Cloudflare-specific headers to non-Gateway base URLs.

Notes

Do not make response caching default for arbitrary chat/agent turns. Gateway response caching is exact-request caching and should remain an explicit policy choice.

[stackbilt-admin]

AEGIS has started using the newer Cloudflare Gateway/binding contract from @stackbilt/llm-providers@^1.14.1.

Additional acceptance criteria from the AEGIS integration:

• Document Workers AI binding gateway options separately from HTTP AI Gateway headers.
• Show an example combining provider-prefix caching with gateway metadata without implying exact-response caching is safe for arbitrary agent turns.
• Keep x-session-affinity / prefix caching separate from AI Gateway response caching in examples and metadata naming.
• Normalize cache-hit fields from Workers AI responses so AEGIS can report cached input tokens per route class.

AEGIS direct binding calls now attach route metadata when AI_GATEWAY_ID is set; provider-backed calls pass gatewayMetadata.customMetadata.app = 'aegis' and route class where available.

[stackbilt-admin]

Update 2026-06-09: llm-gateway now forwards Workers AI calls through AI Gateway when AI_GATEWAY_ID is set and passes cache/observability metadata through the binding shim. Gateway commit: Stackbilt-dev/llm-gateway@b2062c1.\n\nThe gateway now sends cf-aig-gateway-id, cf-aig-cache-key/cache-ttl, cf-aig-metadata, and x-session-affinity for provider-prefix cache affinity. Added a focused gateway unit test for the AI Gateway chat-completions path and header forwarding.\n\nRelated provider catalog work is pushed as 273b314 for Kimi K2.6, GLM-4.7-Flash, and deepseek/deepseek-v4-pro; npm publish is blocked by registry auth (npm whoami 401).