[Klaud Cold] Port #3393: dsv41flash-fp4-b200-vllm-agentic-dspark to vLLM nightly-0961bbae with FlashInfer sparse indexer + fp8 KV - #3459
Conversation
…LLM nightly-0961bbae with FlashInfer sparse indexer + fp8 KV Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
0c7637a to
b15f933
Compare
|
Thanks for the contribution!
中文感谢你的贡献！
|
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
Beyond the inline finding, I checked whether the new kv-cache-dtype: fp8 and indexer_kv_dtype: mxfp4 settings lower the DSpark draft's effective precision (CONTRIBUTING's draft-model-precision rule) — these are KV-cache/sparse-indexer storage dtypes, not draft-weight overrides, and the unchanged speculative-config block still loads the draft unmodified, consistent with this repo's own precedent for the same kv_cache_dtype=fp8 setting a few lines above in the Kimi-K3 changelog entry.
Extended reasoning...
Config-only PR bumping a vLLM nightly tag, health-check timeouts, and adding a FlashInfer sparse-indexer/fp8-KV attention config plus one perf-changelog entry; no source, launcher, or auth/crypto/permissions code is touched. A CONFIRMED inline finding already flags the unresolved PRLINK_PLACEHOLDER, which is enough on its own to warrant human review, so I deferred rather than approved. I additionally traced the draft-precision question the applicable guidance calls out and found no weight-precision regression, which is worth recording since it's not covered by the inline finding.
| - dsv41flash-fp4-b200-vllm-agentic-dspark | ||
| description: | ||
| - "Port #3393 onto the srt-slurm recipe: pin B200 DeepSeek-V4.1-Flash vLLM to the 2026-09-22 nightly 0961bbae (from nightly-cd10ed6f); enable FlashInfer sparse attention (FLASHINFER_MLA_SPARSE_DSV41) with MXFP4 indexer KV and sparse indexer logits at both TP2 and TP4, plus fp8 KV cache; raise the engine readiness timeout to 7200 s (health check 720 x 10 s). Topology, conc lists and memory limits unchanged." | ||
| pr-link: PRLINK_PLACEHOLDER |
There was a problem hiding this comment.
🔴 The new changelog entry's pr-link uses the literal string PRLINK_PLACEHOLDER, which the repo's merge tooling does not recognize as a placeholder, so merge-time canonicalization for this entry will fail. infx/workflows/validate_perf_changelog.py's PR_LINK_PLACEHOLDERS set only accepts "XXX" or "https://github.com/SemiAnalysisAI/InferenceX/pull/XXX" (validate_added_pr_link, line 142-145); "PRLINK_PLACEHOLDER" matches neither and isn't the expected pull/<pr_number> URL either. Fix: replace pr-link with the documented XXX placeholder (or the real PR URL) so utils/merge_with_reuse.sh's prepare_perf_changelog_merge step can canonicalize it at merge time instead of raising ChangelogValidationError.
Why this was flagged
perf-changelog.yaml:8970 sets pr-link: PRLINK_PLACEHOLDER for the new dsv41flash-fp4-b200-vllm-agentic-dspark entry. When this PR is merged via utils/merge_with_reuse.sh, prepare_perf_changelog_merge.py calls compare_entries -> validate_added_pr_link (infx/workflows/validate_perf_changelog.py:134-145), which only tolerates "XXX"/".../pull/XXX" or the exact expected pull/<pr_number> URL for a new entry. "PRLINK_PLACEHOLDER" is neither, so validate_added_pr_link raises ChangelogValidationError("new PR entry must use ... found 'PRLINK_PLACEHOLDER'"), blocking the reuse-merge path/gate that main relies on for append-only changelog integrity, unlike the base branch where no such stray placeholder exists in this file.
Verification: normal. The new tail entry sets pr-link: PRLINK_PLACEHOLDER (git diff perf-changelog.yaml, added lines under config-keys dsv41flash-fp4-b200-vllm-agentic-dspark). The supported merge command utils/merge_with_reuse.sh <pr> runs python -m infx.workflows.prepare_perf_changelog_merge canonicalize --pr-number "$PR" (merge_with_reuse.sh:169-174). That calls canonicalize_bytes →… | normal.…
This branch has not been deployed
1 participant
Summary
Port of #3393 (by @xinli-sw) onto the srt-slurm recipe that #3428 moved this config to. #3393 edits the shared legacy
benchmarks/single_node/agentic/dsv41flash_fp4_vllm_mtp.shscript plus a B200 launcher opt-in, neither of which the B200 DeepSeek-V4.1-Flash vLLM AgentX config uses any more.Recipes touched:
dsv41flash-fp4-b200-vllm-agentic-dsparkChanges:
configs/nvidia-master.yaml: imagenightly-cd10ed6f…→vllm/vllm-openai:nightly-0961bbae2894d574be790d219651824eb199318e(2026-09-22 nightly).srt-slurm-recipes/dsv41flash/vllm/b200-fp4-mtp/agentic.yaml:containerbump to the same nightly.attention-config: {"backend":"FLASHINFER_MLA_SPARSE_DSV41","indexer_kv_dtype":"mxfp4","indexer_sparse_logits":true}at both TP2 and TP4, pluskv-cache-dtype: fp8— exactly what Qualify B200 DeepSeek-V4.1-Flash on vLLM nightly 0961bbae / 在 B200 上验证 DeepSeek-V4.1-Flash 的 vLLM nightly 0961bbae #3393's script passes whenDSV41_BLACKWELL_ATTENTION=true(its description mentions MegaAttention at TP4, but its code, and its green sweep, use FlashInfer at both TP sizes).VLLM_ENGINE_READY_TIMEOUT_S3600 → 7200 andhealth_check.max_attempts360 → 720 (10 s interval), replacing Qualify B200 DeepSeek-V4.1-Flash on vLLM nightly 0961bbae / 在 B200 上验证 DeepSeek-V4.1-Flash 的 vLLM nightly 0961bbae #3393's launcher export.perf-changelog.yaml: new entry.Topology, conc lists, CUDA-graph ladders and memory limits are unchanged. #3393's settings passed a full sweep on the legacy path (run 35902738208: all 16 AgentX points, GSM8K strict EM 0.978); this PR re-validates them on the native srt-slurm path. Companion B300 port: #3458.
Test plan
🤖 Generated with Claude Code