feat: dynamically split PD decode requests - #1535

Merged

hiworldwzj merged 17 commits into

mainfrom

bsh/dynamic_pd_chunk

Sep 3, 2026

Commits on Sep 2, 2026

feat: dynamically split PD decode requests

shihaobai
authored and
hiworldwzj
committed
7830122
refactor: end PD chunks through max token limit

shihaobai
authored and
hiworldwzj
committed
e94865d
refactor: clarify resource shortage handling

shihaobai
authored and
hiworldwzj
committed
1d0e7fc
refactor: keep existing pause counter names

shihaobai
authored and
hiworldwzj
committed
474edb0
refactor: return decode shortage count

shihaobai
authored and
hiworldwzj
committed
06d40a6
refactor: assume overlap in PD decode yielding

shihaobai
authored and
hiworldwzj
committed
d389174
fix(pd-master): count segment output tokens locally
hiworldwzj
committed
4c12d0f
feat(pd): enable advanced scheduling on decode nodes
hiworldwzj
committed
71d3b64
refactor(pd): use EMA scheduling on decode nodes
hiworldwzj
committed
e0f8fcb
refactor(pd): inline decode allocation fallback
hiworldwzj
committed
99c6379
refactor: restore PDQueue name
hiworldwzj
committed
8b82046
fix(pd): force early finish on decode capacity shortage
hiworldwzj
committed
9cc8f44
fix(pd): use EMA output length for decode admission
hiworldwzj
committed
a78e9ed
fix(pd): distinguish decode capacity segments
hiworldwzj
committed
05c34d8
refactor(pd): simplify segment finish handling
hiworldwzj
committed
922df29
fix(pd): cap decode admission estimate by request limit
hiworldwzj
committed
b09c681

Commits on Sep 3, 2026

fix
hiworldwzj
committed
e844deb