From b135aa2afb70792d7fd54048da1c840889ab2aaa Mon Sep 17 00:00:00 2001 From: trtllm-agent <296075020+trtllm-agent@users.noreply.github.com> Date: Mon, 3 Aug 2026 10:31:22 -0700 Subject: [PATCH] [nvbugs/6539942][fix] Unwaive B300 NemotronNanoV3 bf16 attn_dp_off trtllm accuracy test The hang was not reproducible at HEAD on healthy B300 hardware. The test passed 5/5 consecutive runs (MMLU 74.05-74.22, GSM8K 68.46-69.75 against a 66.17 threshold). All four ranks' hang-detector stacks ended at the same plain, non-collective torch.cuda.synchronize() in PiecewiseCapturedGraph.forward, with no rank inside an NCCL collective. Instrumenting DualModeCapturedGraph.forward confirmed every rank agreed on the graph path and token bucket for all 2747 prefill iterations, ruling out cross-rank divergence in piecewise CUDA graph dispatch. That synchronize is the first host-blocking point after graph-launched work, so a device-side stall surfaces there regardless of cause. Only the B300 bf16 entry is removed; the GB200 fp8 and GB300 bf16 entries filed under the same bug ID are left in place since that hardware was not available to verify. Signed-off-by: trtllm-agent <296075020+trtllm-agent@users.noreply.github.com> --- tests/integration/test_lists/waives.txt | 1 - 1 file changed, 1 deletion(-) diff --git a/tests/integration/test_lists/waives.txt b/tests/integration/test_lists/waives.txt index fc8bae3f0f10..58f24e804341 100644 --- a/tests/integration/test_lists/waives.txt +++ b/tests/integration/test_lists/waives.txt @@ -165,7 +165,6 @@ full:B200/llmapi/test_llm_api_pytorch_moe_lora.py::test_qwen_moe_routed_expert_m full:B200/test_e2e.py::test_multi_nodes_eval[Qwen3/Qwen3-235B-A22B-tp16-mmlu] SKIP (https://nvbugs/6424188) full:B200/test_e2e.py::test_multi_nodes_eval[Qwen3/saved_models_Qwen3-235B-A22B_nvfp4_hf-tp16-mmlu] SKIP (https://nvbugs/6424188) full:B300/accuracy/test_disaggregated_serving.py::TestDeepSeekV3Lite::test_auto_dtype_with_helix[fifo_v2-cudagraph:with_padding-pp1tp2cp2] SKIP (https://nvbugs/6410881) -full:B300/accuracy/test_llm_api_autodeploy.py::TestNemotronNanoV3::test_accuracy[bf16-4-attn_dp_off-trtllm] SKIP (https://nvbugs/6539942) full:B300/accuracy/test_llm_api_pytorch.py::TestDeepSeekV32::test_nvfp4_multi_gpus[latency] SKIP (https://nvbugs/6423866) full:B300/accuracy/test_llm_api_pytorch.py::TestDeepSeekV32::test_nvfp4_multi_gpus_chunked_prefill[latency] SKIP (https://nvbugs/6483369) full:B300/accuracy/test_llm_api_pytorch.py::TestDeepSeekV32::test_nvfp4_multi_gpus_chunked_prefill[latency_qsplit] SKIP (https://nvbugs/6423866)