| Flow | Participants | Request/model types and identifiers | Entrypoints | Observable outputs | Failure/fallback boundaries | Tests/demos | Not yet implemented |
|---|---|---|---|---|---|---|---|
| Client -> Scheduler -> Proxy -> Instance -> vLLM | Client, Scheduler, Proxy, Instance, vLLM | OpenAI-compatible chat/completion bodies; model name; Scheduler internal request ID; optional RAG and Injection_type |
test/demo_client.py, test/demo_scheduler.py, test/demo_proxy.py, test/demo_instance.py |
Scheduler /debug/status, /debug/strategy; Proxy UI/control plane; client response |
Scheduler resource selection may fail; Proxy may fall back by local policy; Instance/vLLM errors return through chain | demo scripts, source checkout tests | Canonical service entrypoints under cacheroute.entrypoints |
| Scheduler <-> KDN discovery and knowledge selection | Scheduler, KDN | Knowledge IDs, KDN resource records, queue/load state | test/demo_kdn.py, KDN registration CLI |
Scheduler debug status and strategy snapshots | Missing KDN or knowledge keeps routing in compatible fallback paths | KDN and scheduler demos | Canonical cacheroute.knowledge service migration |
| Knowledge registration and lookup | KDN, store/text database, optional KV builder | kid, text chunks, kv_ready, artifact compatibility fields |
kdn_server/kdn_register_cli.py |
KDN README commands, Scheduler pool visibility | Registration does not guarantee derived KVCache readiness | contract tests and demos | Canonical knowledge repository ports |
| Text versus KVCache injection | Scheduler, Proxy, KDN, Instance | Injection_type: text, kvcache, hybrid; knowledge IDs |
Proxy strategies and request preparation | Proxy task-level timing/logging and UI state | If KV is unavailable, current compatibility may recompute text depending on path | Proxy demos | Unified policy package migration |
| Cache artifact and cache-operation flow | cacheroute.cache, KDN contract models, legacy KDN paths |
artifact_, cacheop_, endpoint compatibility IDs |
KDN contract requests | Contract JSON models | Unsupported or unknown compatibility must remain explicit | contract foundation tests | Gateway/LMCache operation execution adapters |
| Instance capability registration | Instance, Proxy | capability identity, compatibility profile, endpoint-like data | instance/control_plane.py, demo instance |
Proxy Instance list and resource snapshots | Capability mismatch prevents unsafe reuse | instance/proxy demos | Canonical topology registry migration |
| Scheduler-to-Proxy-to-Instance observability propagation | Scheduler, Proxy, Instance | reserved trace headers, trace_, request ID, runtime profile |
Scheduler forwarding path, Proxy request path, Proxy-to-Instance forwarding, Instance handlers | Separate process-local Proxy and Instance trace stages | Proxy and Instance validate headers or create local fallback context; exact reserved set stops at Instance | test/observability |
Client-returned canonical traces, cross-process aggregation |
| Proxy-local and Instance-local request-stage collection | Proxy, Instance | prepare queue, ready queue, first response, decode, completion | Proxy queue execution; Instance chat/completion handlers | Request-local collectors and immutable process-local traces | These are transport-observed intervals, not authoritative vLLM prefill/decode timings; no trace header reaches vLLM | test/observability |
authoritative vLLM/LMCache timing instrumentation |
| Legacy compatibility paths | Root packages and shims | legacy runtime profile, Redis/LMCache key projections | existing CLIs/demos | Existing debug/README outputs | Shims preserve imports and wire compatibility | governance and wheel tests | Removal after approved migration milestones |
Proxy- and Instance-observed transport intervals are useful for experiments but are not authoritative vLLM prefill/decode execution timings. Treat vLLM prefill/decode as not currently instrumented by canonical trace collection.