feat(qwen4_exp): tensor parallelism for Qwen3.8-Flash-Next (offload backend) - #385

Draft

gdevenyi wants to merge 5 commits into

FlashML-org:mainfrom

gdevenyi:feat/qwen4-exp-tp

Commits on Sep 10, 2026

feat(qwen4_exp): tensor parallelism for Qwen3.8-Flash-Next (offload backend)

gdevenyi
and
claude
committed
c3d0f97
fix(qwen4_exp): load the HF config for sharding only when TP > 1

gdevenyi
and
claude
committed
96e190c
test(qwen4_exp): TP shard test against the new expert-piece stream
gdevenyi
committed
a99433c
fix(qwen4_exp): forward prefix through Qwen4ExpMoE, and test the CPU build

gdevenyi
and
claude
committed
62a9a8c
feat(quant): let the triton NVFP4 expert kernel serve a tensor-parallel shard

gdevenyi
and
claude
committed
d2ad39b