Skip to content

Add Intel WebGPU subgroup-matrix MatMul implementation#29592

Open
jchen10 wants to merge 3 commits into
microsoft:mainfrom
jchen10:sgmm_up
Open

Add Intel WebGPU subgroup-matrix MatMul implementation#29592
jchen10 wants to merge 3 commits into
microsoft:mainfrom
jchen10:sgmm_up

Conversation

@jchen10

@jchen10 jchen10 commented Jul 7, 2026

Copy link
Copy Markdown
Contributor
  • Add an Intel-specific F16 MatMul path built on the 8x16x16 subgroup-matrix config (Xe2/Xe3). A and B are loaded directly from global memory with no prepacking, and results are written through workgroup scratch with bounds-checked stores, so M and N may be any size while only K must be a multiple of 16. The impl is created in MatMul::PrePackInternal and dispatched from Compute when the device reports the required config.
  • The tile shape and split-K factor are chosen per call rather than fixed: TileM in {8,16,32,64}, TileN in {16,32,64}, and split-K to top up occupancy, sized to the device's resident subgroups. The WGSL kernel is parameterized by sg_mat_count_m/n and split_k, emitting the needed subgroup-matrix accumulators via generation-time guards.
  • An optional offline-tuned config table (generated from an autotuner sweep) can override the heuristic; selection precedence is tuned table > heuristic.
  • Adds MatMul subgroup-matrix unit tests covering tile selections and partial/large non-aligned dims.

- Add an Intel-specific F16 MatMul path built on the 8x16x16
subgroup-matrix config (Xe2/Xe3). A and B are loaded directly from
global memory with no prepacking, and results are written through
workgroup scratch with bounds-checked stores, so M and N may be any
size while only K must be a multiple of 16. The impl is created in
MatMul::PrePackInternal and dispatched from Compute when the device
reports the required config.
- The tile shape and split-K factor are chosen per call rather than
fixed: TileM in {8,16,32,64}, TileN in {16,32,64}, and split-K to top
up occupancy, sized to the device's resident subgroups. The WGSL kernel
is parameterized by sg_mat_count_m/n and split_k, emitting the needed
subgroup-matrix accumulators via generation-time guards.
- An optional offline-tuned config table (generated from an autotuner
sweep) can override the heuristic; selection precedence is tuned table
> heuristic.
- Adds MatMul subgroup-matrix unit tests covering tile selections and partial/large non-aligned dims.
@jchen10

jchen10 commented Jul 7, 2026

Copy link
Copy Markdown
Contributor Author

@qjia7 PTAL

@@ -0,0 +1,281 @@
// Copyright (c) Microsoft Corporation. All rights reserved.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done!

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds an Intel-specific WebGPU MatMul fast-path that uses the Chromium experimental subgroup-matrix feature (8x16x16 F16 config) for Xe2/Xe3 GPUs, with adaptive tile/split-K selection and optional baked-in tuned configs.

Changes:

  • Introduces an Intel subgroup-matrix MatMul implementation (WGSL template + C++ dispatch/config selection) and wires it into WebGPU MatMul via a vendor opt-impl hook.
  • Adds Intel device-info helpers for architecture-based occupancy heuristics and subgroup-matrix capability detection.
  • Adds new MatMul tests intended to exercise the subgroup-matrix path across representative tile/split-K/edge cases.

Reviewed changes

Copilot reviewed 4 out of 10 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
onnxruntime/test/providers/cpu/math/matmul_test.cc Adds a new fp16 MatMul test suite aimed at covering subgroup-matrix tile/split-K behavior.
onnxruntime/core/providers/webgpu/vendor/intel/math/subgroup_matrix_matmul.wgsl.template Adds the parameterized WGSL subgroup-matrix MatMul kernel template with optional split-K reduction.
onnxruntime/core/providers/webgpu/vendor/intel/math/subgroup_matrix_matmul.h Declares the Intel subgroup-matrix MatMul program/impl factory and related constants.
onnxruntime/core/providers/webgpu/vendor/intel/math/subgroup_matrix_matmul.cc Implements config selection (tuned table + heuristic) and the WebGPU dispatch path for subgroup-matrix MatMul.
onnxruntime/core/providers/webgpu/vendor/intel/intel_device_info.h Declares Intel adapter helpers for subgroup occupancy estimation and subgroup-matrix config support checks.
onnxruntime/core/providers/webgpu/vendor/intel/intel_device_info.cc Implements Intel architecture-to-occupancy mapping and config enumeration checks.
onnxruntime/core/providers/webgpu/math/matmul.h Adds a MatMul optimized-implementation abstraction and cached implementation pointer.
onnxruntime/core/providers/webgpu/math/matmul.cc Dispatches to the vendor optimized impl when present and instantiates it during PrePackInternal.
onnxruntime/contrib_ops/webgpu/quantization/subgroup_matrix_matmul_nbits.cc Refactors Intel occupancy logic to use the new HwSubgroups helper.

Comment thread onnxruntime/test/providers/cpu/math/matmul_test.cc
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants