From 24084cf7f76cf07dd7a83cbbd5daff1dc7167cbd Mon Sep 17 00:00:00 2001 From: xxi Date: Wed, 5 Aug 2026 08:30:13 +0000 Subject: [PATCH] [https://nvbugs/5945081][fix] un-waive DeepSeek-V3-Lite NVFP4 pp4 CUTLASS test The bug's signature is "Test terminated unexpectedly" -- process death rather than an accuracy assertion. Attempting to reproduce it against TOT on an exclusive 8-GPU B200 node gave three consecutive passes (GSM8K 64.59 / 63.12 / 64.75, ~7 min each), with every iteration verified to have actually exercised the bug path: four distinct ranks reporting ep=1 pp=4 tp=1 world=4, the NVFP4 path taken, GSM8K evaluated to completion, and no stack dumps. Runs that hit an environment problem instead of the bug were excluded rather than counted. Three passes on a flaky process death establish "not currently reproducible", not "fixed", so this un-waive hands the sampling to CI: the case is in the post-merge lists for DGX_B300, GB200, GB300 and RTXPro6000D, and every post-merge run now adds evidence. If it flakes again the waive can be restored with a fresh occurrence to analyze, which is more useful than keeping a possibly-stale waive indefinitely. Note that B200 no longer covers this case -- it was dropped from l0_dgx_b200.yml by #15624, so the stage where the bug was originally reported (DGX_B200-4_GPUs-PyTorch-Post-Merge-1) does not run it anymore. Signed-off-by: xxi --- tests/integration/test_lists/waives.txt | 1 - 1 file changed, 1 deletion(-) diff --git a/tests/integration/test_lists/waives.txt b/tests/integration/test_lists/waives.txt index 11da19460ee4..e366ebae61f4 100644 --- a/tests/integration/test_lists/waives.txt +++ b/tests/integration/test_lists/waives.txt @@ -52,7 +52,6 @@ accuracy/test_llm_api_pytorch.py::TestDeepSeekV3Lite::test_fp8_block_scales_4gpu accuracy/test_llm_api_pytorch.py::TestDeepSeekV3Lite::test_nvfp4[moe_backend=CUTLASS-mtp_nextn=2-fp8kv=False-attention_dp=False-cuda_graph=True-overlap_scheduler=False-torch_compile=False] SKIP (https://nvbugs/6388363) accuracy/test_llm_api_pytorch.py::TestDeepSeekV3Lite::test_nvfp4_4gpus[moe_backend=CUTEDSL-mtp_nextn=0-tp2pp2-fp8kv=False-attention_dp=False-cuda_graph=False-overlap_scheduler=False-low_precision_combine=False-torch_compile=True] SKIP (https://nvbugs/6428087) accuracy/test_llm_api_pytorch.py::TestDeepSeekV3Lite::test_nvfp4_4gpus[moe_backend=CUTEDSL-mtp_nextn=0-tp2pp2-fp8kv=True-attention_dp=True-cuda_graph=True-overlap_scheduler=True-low_precision_combine=False-torch_compile=False] SKIP (https://nvbugs/6427411) -accuracy/test_llm_api_pytorch.py::TestDeepSeekV3Lite::test_nvfp4_4gpus[moe_backend=CUTLASS-mtp_nextn=0-pp4-fp8kv=False-attention_dp=False-cuda_graph=False-overlap_scheduler=False-low_precision_combine=False-torch_compile=False] SKIP (https://nvbugs/5945081) accuracy/test_llm_api_pytorch.py::TestDeepSeekV3Lite::test_nvfp4_4gpus[moe_backend=CUTLASS-mtp_nextn=0-pp4-fp8kv=False-attention_dp=False-cuda_graph=False-overlap_scheduler=False-low_precision_combine=False-torch_compile=True] SKIP (https://nvbugs/6384625) accuracy/test_llm_api_pytorch.py::TestDeepSeekV3Lite::test_nvfp4_4gpus[moe_backend=CUTLASS-mtp_nextn=0-pp4-fp8kv=True-attention_dp=True-cuda_graph=True-overlap_scheduler=True-low_precision_combine=False-torch_compile=False] SKIP (https://nvbugs/6427411) accuracy/test_llm_api_pytorch.py::TestDeepSeekV3Lite::test_nvfp4_4gpus[moe_backend=CUTLASS-mtp_nextn=0-pp4-fp8kv=True-attention_dp=True-cuda_graph=True-overlap_scheduler=True-low_precision_combine=False-torch_compile=True] SKIP (https://nvbugs/6428063)