[Klaud Cold] Update qwen3.5-fp4-mi355x-sglang-agentic-mtp to v0.5.20-rocm720-mi35x-20260924 and port #3364 TP2 conc / hicache-size tuning - #3455
Conversation
… image to v0.5.20-rocm720-mi35x-20260924 Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
89504e6 to
0c10a66
Compare
|
Thanks for the contribution!
中文感谢你的贡献！
|
TP2 GPU-resident KV now covers [1, 4, 8, 12] and TP2 HiCache [16, 20, 24, 28, 32, 36, 40]; every HiCache variant pins hicache-size 253. Drops the now-unused tp2_c16/tp2_c20 resident variants and adds tp2_c16_hicache so each matrix point resolves to exactly one recipe variant. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…55x-sglang-agentic-mtp-v0.5.20-20260924 # Conflicts: # perf-changelog.yaml
functionstackx
changed the title
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
| - "This image bump does not change the DSpark draft model data type. Only the image: line changes and kimik3_fp4_mi355x_mtp.sh is unchanged; the draft loads unmodified from the published Inferact/Kimi-K3-DSpark checkpoint via --speculative-config (model=Inferact/Kimi-K3-DSpark, method=dspark). The only dtype in that speculative-config is kv_cache_dtype=fp8, which sets the draft KV-cache storage precision, not the draft weights. No flag overrides or re-quantizes the draft-model weights, so the draft dtype is preserved from its checkpoint across this re-sweep." | ||
| pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/3419 | ||
| - "Update SGLang ROCm image from v0.5.19-rocm720-mi35x-20260915 to v0.5.20-rocm720-mi35x-20260924 (latest nightly)" | ||
| pr-link: PRLINK_PLACEHOLDER |
There was a problem hiding this comment.
🟡 (optional) perf-changelog.yaml's new entry has pr-link: PRLINK_PLACEHOLDER instead of a real PR URL, violating the requirement that every changelog entry carry a real pr-link. Since the file is append-only and byte-sensitive, merging this leaves a permanent placeholder in history that never gets corrected retroactively. Fix: replace PRLINK_PLACEHOLDER with the actual PR URL before merge (compare other entries, e.g. the prior entry's pr-link: #3334, for the expected format).
Why this was flagged
perf-changelog.yaml:8958 sets pr-link: PRLINK_PLACEHOLDER for the new qwen3.5-fp4-mi355x-sglang-agentic-mtp entry. The file is documented as append-only, so once merged this placeholder cannot be fixed by editing the entry later without violating that invariant. Anyone auditing the changelog for this image bump (e.g. to find the PR that introduced v0.5.20-rocm720-mi35x-20260924) gets a dead literal string instead of a link. Base branch entries all carry real GitHub PR URLs (e.g. pull/3334 just above), so this is a regression in changelog completeness introduced by this diff.
Verification: nit. The candidate is factually present. perf-changelog.yaml:8958 (the new qwen3.5-fp4-mi355x-sglang-agentic-mtp entry appended at the tail) sets pr-link: PRLINK_PLACEHOLDER, a literal placeholder rather than a real URL. Every other entry carries a real GitHub PR URL, e.g. the immediately preceding entry at line 8952 pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/3334, and…
|
View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=36220889205 View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=36220889205 |
This branch has not been deployed
1 participant
Summary
Update the SGLang ROCm image from
v0.5.19-rocm720-mi35x-20260915tov0.5.20-rocm720-mi35x-20260924(latest nightly) for the Qwen3.5 FP4 MI355X AgentX recipe, and port the concurrency / HiCache tuning from #3364 onto the srt-slurm recipe that #3428 switched this config to. The same image passed the Qwen3.5 FP8/FP4 MI355X 8k1k sweeps (#3423, #3439).Recipes touched:
qwen3.5-fp4-mi355x-sglang-agentic-mtpChanges:
configs/amd-master.yaml: image bump; TP2 GPU-resident conc-list[1, 4, 8, 12, 16, 20]→[1, 4, 8, 12], TP2 HiCache conc-list[20, …, 40]→[16, 20, 24, 28, 32, 36, 40]. TP4 stays[1, 4, 8, 12, 16].srt-slurm-recipes/qwen3.5/sglang/mi355x-fp4-mtp/agentic.yaml:containerbump; every HiCache variant pinshicache-size: 253(overrideshicache-ratio); drops the now-unusedtp2_c16/tp2_c20GPU-resident variants and addstp2_c16_hicache. Verified locally withinfx.srt_slurm.single_node.select_recipethat all 16 matrix points resolve to exactly one variant.perf-changelog.yaml: new entry.Supersedes #3364, which made the same changes against the retired
benchmarks/single_node/agentic/qwen3.5_fp4_mi355x_sglang_mtp.shbash script.Test plan
🤖 Generated with Claude Code