Add Intel WebGPU subgroup-matrix MatMul implementation#29592
Open
jchen10 wants to merge 3 commits into
Open
Conversation
jchen10
commented
Jul 7, 2026
Contributor
- Add an Intel-specific F16 MatMul path built on the 8x16x16 subgroup-matrix config (Xe2/Xe3). A and B are loaded directly from global memory with no prepacking, and results are written through workgroup scratch with bounds-checked stores, so M and N may be any size while only K must be a multiple of 16. The impl is created in MatMul::PrePackInternal and dispatched from Compute when the device reports the required config.
- The tile shape and split-K factor are chosen per call rather than fixed: TileM in {8,16,32,64}, TileN in {16,32,64}, and split-K to top up occupancy, sized to the device's resident subgroups. The WGSL kernel is parameterized by sg_mat_count_m/n and split_k, emitting the needed subgroup-matrix accumulators via generation-time guards.
- An optional offline-tuned config table (generated from an autotuner sweep) can override the heuristic; selection precedence is tuned table > heuristic.
- Adds MatMul subgroup-matrix unit tests covering tile selections and partial/large non-aligned dims.
- Add an Intel-specific F16 MatMul path built on the 8x16x16
subgroup-matrix config (Xe2/Xe3). A and B are loaded directly from
global memory with no prepacking, and results are written through
workgroup scratch with bounds-checked stores, so M and N may be any
size while only K must be a multiple of 16. The impl is created in
MatMul::PrePackInternal and dispatched from Compute when the device
reports the required config.
- The tile shape and split-K factor are chosen per call rather than
fixed: TileM in {8,16,32,64}, TileN in {16,32,64}, and split-K to top
up occupancy, sized to the device's resident subgroups. The WGSL kernel
is parameterized by sg_mat_count_m/n and split_k, emitting the needed
subgroup-matrix accumulators via generation-time guards.
- An optional offline-tuned config table (generated from an autotuner
sweep) can override the heuristic; selection precedence is tuned table
> heuristic.
- Adds MatMul subgroup-matrix unit tests covering tile selections and partial/large non-aligned dims.
Contributor
Author
|
@qjia7 PTAL |
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds an Intel-specific WebGPU MatMul fast-path that uses the Chromium experimental subgroup-matrix feature (8x16x16 F16 config) for Xe2/Xe3 GPUs, with adaptive tile/split-K selection and optional baked-in tuned configs.
Changes:
- Introduces an Intel subgroup-matrix MatMul implementation (WGSL template + C++ dispatch/config selection) and wires it into WebGPU MatMul via a vendor opt-impl hook.
- Adds Intel device-info helpers for architecture-based occupancy heuristics and subgroup-matrix capability detection.
- Adds new MatMul tests intended to exercise the subgroup-matrix path across representative tile/split-K/edge cases.
Reviewed changes
Copilot reviewed 4 out of 10 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| onnxruntime/test/providers/cpu/math/matmul_test.cc | Adds a new fp16 MatMul test suite aimed at covering subgroup-matrix tile/split-K behavior. |
| onnxruntime/core/providers/webgpu/vendor/intel/math/subgroup_matrix_matmul.wgsl.template | Adds the parameterized WGSL subgroup-matrix MatMul kernel template with optional split-K reduction. |
| onnxruntime/core/providers/webgpu/vendor/intel/math/subgroup_matrix_matmul.h | Declares the Intel subgroup-matrix MatMul program/impl factory and related constants. |
| onnxruntime/core/providers/webgpu/vendor/intel/math/subgroup_matrix_matmul.cc | Implements config selection (tuned table + heuristic) and the WebGPU dispatch path for subgroup-matrix MatMul. |
| onnxruntime/core/providers/webgpu/vendor/intel/intel_device_info.h | Declares Intel adapter helpers for subgroup occupancy estimation and subgroup-matrix config support checks. |
| onnxruntime/core/providers/webgpu/vendor/intel/intel_device_info.cc | Implements Intel architecture-to-occupancy mapping and config enumeration checks. |
| onnxruntime/core/providers/webgpu/math/matmul.h | Adds a MatMul optimized-implementation abstraction and cached implementation pointer. |
| onnxruntime/core/providers/webgpu/math/matmul.cc | Dispatches to the vendor optimized impl when present and instantiates it during PrePackInternal. |
| onnxruntime/contrib_ops/webgpu/quantization/subgroup_matrix_matmul_nbits.cc | Refactors Intel occupancy logic to use the new HwSubgroups helper. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.