feat: support multi-token KV cache pages - #1522

Open

hiworldwzj wants to merge 15 commits into

mainfrom

Commits on Aug 28, 2026

feat: support multi-token KV cache pages
hiworldwzj
committed
83567c3
refactor: preallocate KV cache during scheduling
hiworldwzj
committed
1be6cb0
refactor: derive KV indexes in infer state
hiworldwzj
committed
edda3b6
fix: assert prompt cache page alignment
hiworldwzj
committed
dc1364d
refactor: align KV allocation needs to pages
hiworldwzj
committed
dc74842
refactor: reduce MTP KV reservation window
hiworldwzj
committed
585f857
refactor: move token scheduling control to infer backend
hiworldwzj
committed
2392dad
refactor: remove duplicate KV page reservation
hiworldwzj
committed
3e1b0d7
fix: restrict fixed KV cache to single-token pages
hiworldwzj
committed
329f5b6
refactor: split PD queues by inference stage
hiworldwzj
committed
7d9166b

Commits on Aug 31, 2026