[TRTLLM-14822][feat] deprecate WIDEEP MoE backend

docs/source/deployment-guide/deployment-guide-for-deepseek-r1-on-trtllm.md

@@ -29,8 +29,8 @@ There are multiple MOE backends inside TensorRT LLM, not all of them supporting
 | H100/H200 | FP8 | CUTLASS |
 | B200/GB200 EP<=8 | NVFP4 | CUTLASS, TRTLLM |
 | B200/GB200 EP<=8 | FP8 | DEEPGEMM |
-| GB200 NVL72 EP>8 | NVFP4 |  WIDEEP |
-| GB200 NVL72 EP>8 | FP8 | WIDEEP without EPLB |
+| GB200 NVL72 EP>8 | NVFP4 |  CUTEDSL |
+| GB200 NVL72 EP>8 | FP8 | DEEPGEMM |
 
 The default moe backend is `CUTLASS`, so for the combination which is not supported by `CUTLASS`, one must set the `moe_config.backend` explicitly to run the model.
 
@@ -207,7 +207,7 @@ See the [`TorchLlmArgs` class](https://nvidia.github.io/TensorRT-LLM/llm-api/ref
 Add the following fields to the YAML configuration file `/tmp/config.yml` to enable wide EP:
 ```yaml
 moe_config:
-    backend: WIDEEP
+    backend: CUTEDSL
     max_num_tokens: 9216
     load_balancer:  # configure online EP balancer
       num_slots: 288

examples/layer_wise_benchmarks/README.md

@@ -86,8 +86,8 @@ NP=4 ./mpi_launch.sh ./run.sh config_gen.yaml --batch-size 32 --seq-len-q 4
 NP=4 ./mpi_launch.sh ./run.sh config_ctx.yaml --layer-indices 5,6,7,8
 NP=4 ./mpi_launch.sh ./run.sh config_gen.yaml --layer-indices 5,6,7,8
 
-# Scale DEP=16 to 4 GPUs: reduces the number of experts; uses MNNVL A2A if applicable
-NP=4 ./mpi_launch.sh ./run.sh config_gen.yaml --scaled-from 16 --moe-backend WIDEEP
+# Scale DEP=16 to 4 GPUs: reduces the number of experts
+NP=4 ./mpi_launch.sh ./run.sh config_gen.yaml --scaled-from 16 --moe-backend CUTEDSL
 
 # Scale TEP=16 to 4 GPUs: reduces the number of attention heads and experts
 NP=4 ./mpi_launch.sh ./run.sh config_gen.yaml --scaled-from 16 --no-enable-attention-dp
@@ -101,8 +101,8 @@ NP=2 ./mpi_launch.sh ./run.sh config_ctx.yaml --model Qwen/Qwen3-Next-80B-A3B-In
 NP=2 ./mpi_launch.sh ./run.sh config_gen.yaml --model Qwen/Qwen3-Next-80B-A3B-Instruct --layer-indices 6,7 --no-enable-attention-dp --mamba-ssm-cache-dtype float16 --batch-size 512
 
 # Run with DeepEP A2A
-NP=4 ./mpi_launch.sh -x TRTLLM_FORCE_ALLTOALL_METHOD=DeepEP ./run.sh config_ctx.yaml --moe-backend WIDEEP
-NP=4 ./mpi_launch.sh -x TRTLLM_FORCE_ALLTOALL_METHOD=DeepEP ./run.sh config_gen.yaml --moe-backend WIDEEP
+NP=4 ./mpi_launch.sh -x TRTLLM_FORCE_COMM_METHOD=DEEPEP ./run.sh config_ctx.yaml --moe-backend CUTEDSL
+NP=4 ./mpi_launch.sh -x TRTLLM_FORCE_COMM_METHOD=DEEPEP ./run.sh config_gen.yaml --moe-backend CUTEDSL
 
 # Run with imbalanced ranks: in addition to activating all experts, the specified ratio of tokens is sent to rank 0
 # Note: if balance ratio is 0, the "activate all experts" behavior is not applied
@@ -157,14 +157,14 @@ python3 scripts/build_wheel.py --cuda_architectures native --no-venv --skip_buil
 **Step 3:** Run benchmarks to generate profiles. Run the following command on the controller node, where `NODES` ≤ the number of allocated nodes:
 
 ```bash
-# Run DeepSeek-R1 NVFP4 with wide EP; uses MNNVL A2A if applicable
-NODES=4 NP=16 ./slurm_launch.sh ./run.sh config_gen.yaml --moe-backend WIDEEP
+# Run DeepSeek-R1 NVFP4 with wide EP
+NODES=4 NP=16 ./slurm_launch.sh ./run.sh config_gen.yaml --moe-backend CUTEDSL
 
 # Run with TRTLLMGen
 NODES=4 NP=16 ./slurm_launch.sh ./run.sh config_gen.yaml --moe-backend TRTLLM
 
 # Run with DeepEPLowLatency
-NODES=4 NP=16 TRTLLM_FORCE_ALLTOALL_METHOD=DeepEPLowLatency ./slurm_launch.sh ./run.sh config_gen.yaml --moe-backend WIDEEP
+NODES=4 NP=16 TRTLLM_FORCE_COMM_METHOD=DEEPEPLOWLATENCY ./slurm_launch.sh ./run.sh config_gen.yaml --moe-backend CUTEDSL
 
 # You can run 4-GPU and 8-GPU tasks without reallocating the Slurm job
 NODES=1 NP=4 ./slurm_launch.sh ./run.sh config_ctx.yaml
@@ -187,7 +187,7 @@ Run with OpenMPI:
 
 ```bash
 NP=4 ./mpi_launch.sh ./run.sh config_ctx.yaml --batch-size 1,2,4 --seq-len-q 1024,8192
-NP=4 ./mpi_launch.sh ./run.sh config_gen.yaml --scaled-from 16 --moe-backend WIDEEP --batch-size 32,64,128,256,512 --seq-len-q 1,2,3,4
+NP=4 ./mpi_launch.sh ./run.sh config_gen.yaml --scaled-from 16 --moe-backend CUTEDSL --batch-size 32,64,128,256,512 --seq-len-q 1,2,3,4
 ```
 
 ## Parse profiles
@@ -354,7 +354,7 @@ Two E2E traces are required because the two pieces of information cannot be capt
 Limitations:
 
 1. Pipeline parallelism is not supported.
-2. Only the CUTLASS and WIDEEP MoE backends are supported.
+2. Only the CUTEDSL, CUTLASS, DEEPGEMM and TRTLLM MoE backends are supported.
 3. Only tested with the GEN phase and attention DP.
 
 ## Developer utilities
@@ -372,7 +372,7 @@ Limitations:
 
 1. Error `fp8 blockscale gemm only support Hopper` on Blackwell.
 
-   The default MoE backend "CUTLASS" does not support FP8 weights. Please choose the same MoE backend as your end-to-end config. A typical solution is to add the `--moe-backend DEEPGEMM` (or `TRTLLM`, `WIDEEP`) and `--moe-backend-for-prefill DEEPGEMM` (or `WIDEEP`) options.
+   The default MoE backend "CUTLASS" does not support FP8 weights. Please choose the same MoE backend as your end-to-end config. A typical solution is to add the `--moe-backend DEEPGEMM` (or `TRTLLM`, `CUTEDSL`) and `--moe-backend-for-prefill DEEPGEMM` options.
 
 2. Error `huggingface_hub.errors.HfHubHTTPError: 429 Client Error: Too Many Requests for url: https://huggingface.co/nvidia/DeepSeek-R1-0528-FP4-v2/resolve/main/config.json`.
 

examples/layer_wise_benchmarks/run.py

@@ -53,9 +53,7 @@ def comma_separated_floats(s):
 parser.add_argument("--load-format", type=str, choices=["AUTO", "DUMMY"])
 parser.add_argument("--max-num-tokens", type=int)
 parser.add_argument("--moe-backend", type=str)
-parser.add_argument(
-    "--moe-backend-for-prefill", type=str, choices=["CUTLASS", "DEEPGEMM", "WIDEEP"]
-)
+parser.add_argument("--moe-backend-for-prefill", type=str, choices=["CUTLASS", "DEEPGEMM"])
 parser.add_argument("--moe-max-num-tokens", type=int)
 group = parser.add_mutually_exclusive_group()
 group.add_argument(

examples/llm-api/llm_sparse_attention.py

@@ -176,8 +176,8 @@ def parse_arguments():
                         type=str,
                         default='CUTLASS',
                         choices=[
-                            'CUTLASS', 'TRTLLM', 'VANILLA', 'WIDEEP',
-                            'DEEPGEMM', 'CUTEDSL', 'TRITON'
+                            'CUTLASS', 'TRTLLM', 'VANILLA', 'DEEPGEMM',
+                            'CUTEDSL', 'TRITON'
                         ])
     parser.add_argument('--tp_size', type=int, default=1)
     parser.add_argument('--moe_ep_size', type=int, default=-1)

examples/llm-api/quickstart_advanced.py

@@ -87,8 +87,8 @@ def add_llm_args(parser):
         type=str,
         default='AUTO',
         choices=[
-            'AUTO', 'CUTLASS', 'TRTLLM', 'VANILLA', 'WIDEEP', 'DEEPGEMM',
-            'CUTEDSL', 'TRITON'
+            'AUTO', 'CUTLASS', 'TRTLLM', 'VANILLA', 'DEEPGEMM', 'CUTEDSL',
+            'TRITON'
         ],
         help=
         'MoE backend to use. AUTO selects default backend based on model. It currently doesn\'t always give the best choice for all scenarios. The capabilities of auto selection will be improved in future releases.'

examples/longbench/eval_longbench_v1.py

@@ -138,8 +138,8 @@ def parse_arguments() -> argparse.Namespace:
                         type=str,
                         default='CUTLASS',
                         choices=[
-                            'CUTLASS', 'TRTLLM', 'VANILLA', 'WIDEEP',
-                            'DEEPGEMM', 'CUTEDSL', 'TRITON'
+                            'CUTLASS', 'TRTLLM', 'VANILLA', 'DEEPGEMM',
+                            'CUTEDSL', 'TRITON'
                         ])
     parser.add_argument('--tp_size', type=int, default=1)
     parser.add_argument('--moe_ep_size', type=int, default=-1)

examples/models/core/exaone/README.md

@@ -173,7 +173,7 @@ K-EXAONE supports the following MoE backends:
 |---------|-------------|
 | `CUTLASS` | Default backend, optimized for general use cases |
 | `TRTLLM` | TensorRT-LLM backend using TRT-LLM Gen kernels, optimized for low-latency inference |
-| `WIDEEP` | Wide expert parallelism backend for cases where EP size exceeds the number of experts |
+| `CUTEDSL` | CuTe DSL backend, used for wide expert parallelism where EP size exceeds the number of experts |
 
 You can specify the MoE backend using the `--moe_backend` argument:
 

examples/wide_ep/README.md

@@ -49,7 +49,7 @@ When GDRCopy is installed and the kernel module is loaded, you should be able to
 An example yaml file to enable wide EP:
 ```yaml
 moe_config:
-    backend: WIDEEP
+    backend: CUTEDSL
     max_num_tokens: 9216
     load_balancer:
       num_slots: 288
@@ -59,7 +59,7 @@ moe_config:
 #### `backend`
 
  - MoE backend type, defaults to `CUTLASS`.
- - Currently, TensorRT LLM has multiple MoE backends that support wide EP, including `WIDEEP`, `CUTLASS`, `TRTLLM` and `CUTEDSL`. There are on-going efforts to refactor the backends so that we don't necessarily need a specific `WIDEEP` backend, and each other backend will support wide EP functionality.
+ - TensorRT LLM has multiple MoE backends that support wide EP, including `CUTEDSL`, `CUTLASS` and `TRTLLM`. The dedicated `WIDEEP` backend is deprecated and can no longer be selected; use `CUTEDSL` for large-EP NVFP4 deployments, or `DEEPGEMM` for FP8 block-scale checkpoints on Blackwell.
 
 #### `max_num_tokens`
 

examples/wide_ep/ep_load_balancer/README.md

@@ -27,9 +27,8 @@ Run 32-way expert parallelism inference on the prepared dataset. Please refer to
 cat > ./config.yaml <<EOF
 enable_attention_dp: true
 cuda_graph_config: {}
-moe_backend: WideEP
 moe_config:
-    backend: WideEP
+    backend: CUTEDSL
     max_num_tokens: 8192
 EOF
 
@@ -119,7 +118,7 @@ cat > ./config_eplb.yaml <<EOF
 enable_attention_dp: true
 cuda_graph_config: {}
 moe_config:
-    backend: WideEP
+    backend: CUTEDSL
     max_num_tokens: 9216
     load_balancer: ./moe_load_balancer.yaml
 EOF
@@ -185,7 +184,7 @@ cat > ./config_eplb.yaml <<EOF
 enable_attention_dp: true
 cuda_graph_config: {}
 moe_config:
-    backend: WideEP
+    backend: CUTEDSL
     max_num_tokens: 9216
     load_balancer: ./moe_load_balancer.yaml
 EOF

examples/wide_ep/slurm_scripts/config.yaml

@@ -81,7 +81,7 @@ worker_config:
       free_gpu_memory_fraction: 0.6
       dtype: fp8
     moe_config:
-      backend: WIDEEP
+      backend: CUTEDSL
       use_low_precision_moe_combine: true
       load_balancer:
         num_slots: 288

examples/wide_ep/slurm_scripts/kimi-k2-thinking.yaml

@@ -66,7 +66,7 @@ worker_config:
       free_gpu_memory_fraction: 0.6
       dtype: fp8
     moe_config:
-      backend: WIDEEP
+      backend: CUTEDSL
       use_low_precision_moe_combine: true
       load_balancer:
         num_slots: 416

tensorrt_llm/_torch/models/modeling_deepseekv4.py

@@ -1515,16 +1515,14 @@ def __init__(
                 CutlassFusedMoE,
                 TritonFusedMoE,
                 TRTLLMGenFusedMoE,
-                WideEPMoE,
                 DeepGemmFusedMoE,
             )
             # NVFP4 routed-expert path: the TRTLLM-Gen fp4-block-scale fused-MoE
             # cubin produces near-zero accuracy without bias even when
             # swiglu_limit is supplied; drop the limit there until the cubin
             # gains a no-bias clamp variant. MXFP4 variants are unaffected.
             kernel_requires_bias_for_swiglu_limit = (
-                moe_cls in (TRTLLMGenFusedMoE, WideEPMoE)
-                and experts_quant_config.quant_mode.has_nvfp4()
+                moe_cls is TRTLLMGenFusedMoE and experts_quant_config.quant_mode.has_nvfp4()
             )
             # DeepSeek-V4 supplies a uniform scalar limit. The TRTLLM-Gen FP8
             # path consumes it directly and rejects the redundant tensor.

tensorrt_llm/_torch/modules/fused_moe/MOE_DEVELOPER_GUIDE.md

@@ -119,7 +119,8 @@ ConfigurableMoE currently supports these backends (`create_moe.py`):
 - `CutlassFusedMoE`, `TRTLLMGenFusedMoE`, `DeepGemmFusedMoE`, `CuteDslFusedMoE`, `DenseGEMMFusedMoE`, `MegaMoEDeepGemm`, `MegaMoECuteDsl`
 
 Still on old path (standalone, with embedded communication):
-- `TritonFusedMoE`, `WideEPMoE`, `VanillaMoE`
+- `TritonFusedMoE`, `VanillaMoE`
+- `WideEPMoE` — deprecated, the `WIDEEP` backend can no longer be selected
 
 **Rule: All new features should target ConfigurableMoE + Backend + Scheduler architecture.**
 
@@ -152,7 +153,7 @@ Still on old path (standalone, with embedded communication):
 | `mega_moe/mega_moe_cute_dsl.py` | `MegaMoECuteDsl` | SM100/SM103 | NVFP4 via ported CuteDSL `Sm100MegaMoEKernel` fused dispatch+FC1+act+FC2+combine kernel; requires CUDA 13 Cutlass DSL runtime (PR #14354) and NVSHMEM provider (hard gate); threads per-expert `fc31_alpha`/`fc2_alpha`/`fc1_norm_const` through the kernel ABI and supports SwiGLU clamp via `swiglu_limit`; default deepgemm graph (topk score folded before fc1-out quant, host `combine_output.sum(dim=1)`) | `FUSED_COMM` |
 | `fused_moe_marlin.py` | `MarlinFusedMoE` | SM89-SM99 | W4A16 NVFP4 on Ada/Hopper (BF16 activations + FP4 weights, fused single-launch `marlin_nvfp4_moe_gemm` kernel); supports attention-DP + EP via external comm (scheduler precomputes routing; dispatch payload is plain BF16, no activation scales); non-NVFP4 layers (e.g. unquantized MTP draft layers) fall back to Cutlass in `get_moe_cls`; no dynamic EPLB | `EXTERNAL_COMM` |
 | `fused_moe_triton.py` | `TritonFusedMoE` | SM90 only | GPT-OSS on Hopper (requires `swiglu_gptoss_style=True`) | (legacy path) |
-| `fused_moe_wide_ep.py` | `WideEPMoE` | All GPUs | Deprecating — use ConfigurableMoE instead | (legacy path) |
+| `fused_moe_wide_ep.py` | `WideEPMoE` | All GPUs | Deprecated — `create_moe.py` rejects the `WIDEEP` backend. Wide EP and EPLB are available on the other backends: use `DEEPGEMM` for FP8 block-scale checkpoints, or `TRTLLM` / `CUTEDSL` / `CUTLASS` otherwise. Class kept for reference only | (legacy path) |
 | `fused_moe_vanilla.py` | `VanillaMoE` | All devices | Reference / debugging only | (legacy path) |
 
 ### Communication (`fused_moe/communication/`)
@@ -198,7 +199,7 @@ is available.
 
 Each backend's `can_implement(quant_algo, dtype_activation, swiglu_gptoss_style, ...)` method declares supported quantizations. Source of truth: the `can_implement` classmethod in each backend file.
 
-| Quantization | Cutlass | TRTLLMGen | DeepGemm | DenseGEMM | CuteDSL | MegaMoE-DG | MegaMoE-CuteDSL | Triton | Marlin | WideEP | Vanilla |
+| Quantization | Cutlass | TRTLLMGen | DeepGemm | DenseGEMM | CuteDSL | MegaMoE-DG | MegaMoE-CuteDSL | Triton | Marlin | WideEP (deprecated) | Vanilla |
 |---|---|---|---|---|---|---|---|---|---|---|---|
 | Unquantized (BF16/FP16) | Y (SM80+) | N | N | N | N | N | N | Y (SM90, BF16) | N | Y | Y |
 | FP8 QDQ | Y (SM89+) | N | N | N | N | N | N | Y (SM90) | N | Y | Y |

tensorrt_llm/_torch/modules/fused_moe/create_moe.py

@@ -26,6 +26,12 @@
 from .moe_load_balancer import get_moe_load_balancer
 from .routing import BaseMoeRoutingMethod
 
+WIDEEP_DEPRECATION_MESSAGE = (
+    "The WIDEEP MoE backend is deprecated and can no longer be selected. Wide "
+    "expert parallelism and EPLB are supported by the other backends: use "
+    "DEEPGEMM for FP8 block-scale checkpoints, or TRTLLM / CUTEDSL / CUTLASS "
+    "otherwise.")
+
 
 def _get_pretrained_megamoe_capability_args(
         model_config: ModelConfig) -> Dict[str, Optional[object]]:
@@ -154,7 +160,7 @@ def get_moe_cls(
             )
             return CutlassFusedMoE
     elif moe_backend.upper() == "WIDEEP":
-        return WideEPMoE
+        raise ValueError(WIDEEP_DEPRECATION_MESSAGE)
     elif moe_backend.upper() == "TRITON":
         return TritonFusedMoE
     elif moe_backend.upper() == "MEGAMOE_DEEPGEMM":
@@ -288,6 +294,9 @@ def create_moe_backend(
     Returns:
         MoE: MoE backend instance
     """
+    if moe_cls is WideEPMoE:
+        raise ValueError(WIDEEP_DEPRECATION_MESSAGE)
+
     # Get parameters from pretrained_config if not explicitly provided
     pretrained_config = model_config.pretrained_config
     if num_experts is None:
@@ -310,7 +319,6 @@ def create_moe_backend(
     moe_load_balancer = get_moe_load_balancer()
     if moe_load_balancer is not None:
         supported_load_balancer_backends = (
-            WideEPMoE,
             CutlassFusedMoE,
             TRTLLMGenFusedMoE,
             CuteDslFusedMoE,
@@ -336,15 +344,15 @@ def create_moe_backend(
 
     if swiglu_limit is not None:
         assert moe_cls in [
-            CutlassFusedMoE, TritonFusedMoE, TRTLLMGenFusedMoE, WideEPMoE,
+            CutlassFusedMoE, TritonFusedMoE, TRTLLMGenFusedMoE,
             DeepGemmFusedMoE, MegaMoECuteDsl
         ], f"swiglu_limit is not supported in {moe_cls.__name__}."
 
     if swiglu_limit_scalar is not None:
         # MegaMoECuteDsl uses the scalar only as a fallback when no per-expert
         # tensor limit is given (see the MegaMoE branch below).
         assert moe_cls in [
-            CutlassFusedMoE, TRTLLMGenFusedMoE, WideEPMoE, DeepGemmFusedMoE,
+            CutlassFusedMoE, TRTLLMGenFusedMoE, DeepGemmFusedMoE,
             MegaMoEDeepGemm, CuteDslFusedMoE, MegaMoECuteDsl
         ], f"swiglu_limit_scalar is not supported in {moe_cls.__name__}."
 
@@ -394,22 +402,6 @@ def create_moe_backend(
             init_load_balancer=init_load_balancer,
             activation_type=activation_type,
         )
-    elif moe_cls == WideEPMoE:
-        return moe_cls(
-            routing_method=routing_method,
-            num_experts=num_experts,
-            hidden_size=hidden_size,
-            intermediate_size=intermediate_size,
-            dtype=dtype,
-            reduce_results=reduce_results,
-            model_config=model_config,
-            aux_stream_dict=aux_stream_dict,
-            weight_loading_mode=weight_loading_mode,
-            apply_router_weight_on_input=apply_router_weight_on_input,
-            layer_idx=layer_idx,
-            swiglu_limit=swiglu_limit,
-            swiglu_limit_scalar=swiglu_limit_scalar,
-            activation_type=activation_type)
     elif moe_cls == VanillaMoE:
         assert not apply_router_weight_on_input, "apply_router_weight_on_input is not supported in VanillaMoE."
 

tensorrt_llm/_torch/peft/lora/validation.py

@@ -93,7 +93,7 @@ def check_moe_lora_supported(
 
     Args:
         moe_backend_name: The resolved `moe_backend` string (e.g. "CUTLASS",
-            "WIDEEP", "TRTLLM"). Comparison is case-insensitive.
+            "CUTEDSL", "TRTLLM"). Comparison is case-insensitive.
         lora_config: The model's `LoraConfig`, or None.
         quant_config: The model's `QuantConfig`, or None. We only reject when
             the layer is actually quantized (`quant_mode.has_any_quant`).

tensorrt_llm/tools/layer_wise_benchmarks/runner.py

@@ -702,7 +702,6 @@ def replace_routing_method_ctx(self, balance_method: BalanceMethod, balance_rati
             "CUTLASS",
             "DEEPGEMM",
             "TRTLLM",
-            "WIDEEP",
         ]:
             raise NotImplementedError(
                 f'Not support replace routing method for moe_backend "{self.model_config.moe_backend}",'

tests/integration/defs/accuracy/references/gsm8k.yaml

@@ -145,6 +145,18 @@ deepseek-ai/DeepSeek-V4-Flash:
   #   95.11 reference still holds for the hypothesis test.
   - quant_algo: FP8_BLOCK_SCALES
     accuracy: 95.11
+deepseek-ai/DeepSeek-V4-Flash-Base:
+  # Base (pretrained, non-instruct) checkpoint, so GSM8K lands well below the
+  # instruct DeepSeek-V4-Flash above. GSM8K measurements from
+  # test_fp8_4gpus_static_eplb at TP=4/EP=4 over the full 1319 samples:
+  # * 90.90 / 91.02 / 91.43 on 4x B200 183GB.
+  # * 91.13 on 4x B300 (SM103).
+  # SM100 and SM103 route FP8 block scales through the same DeepGEMM path, so
+  # the spread above is run-to-run noise rather than a per-GPU difference. The
+  # reference records the low end; the hypothesis test's ~3.2-point margin
+  # absorbs the drift.
+  - quant_algo: FP8_BLOCK_SCALES
+    accuracy: 90.90
 deepseek-ai/DeepSeek-V4-Pro:
   # Full GSM8K aggregate gate for the Pro deployment path: TP=8, EP=8,
   # attention DP, TRTLLM MoE, FP8 KV cache, MTP max_draft_len=1, padded CUDA