WebGPU backend + async as a kernel property (asyncMode, mode 'async')

[fuzzie360]

Summary

Two things, in dependency order:

1. A WebGPU backend: new GPU({ mode: 'webgpu' }). Kernels compile to WGSL compute shaders running over f32 storage buffers — no float packing, no texture data layout, no fragment-shader detour, because WebGPU has the primitives WebGL made us emulate. The API in this mode is async: kernel(args) returns a Promise (WebGPU readback is inherently async), resolving to the same result shapes the other backends return synchronously. pipeline: true resolves to a GPU-resident WebGPUBufferResult handle that downstream kernels accept without readback. Opt-in only, never auto-selected.

2. Async as a property any kernel can have — the generalization the WebGPU contract made obvious:

   • asyncMode: true (or .setAsyncMode(true)) makes every call return a Promise on every backend. On webgl2 it buys a genuinely non-blocking readback: readPixels into a PIXEL_PACK_BUFFER, a fence, zero-timeout clientWaitSync polled via MessageChannel tasks (dodging the nested-setTimeout clamp), then getBufferSubData once signaled. webgpu is natively async; cpu/webgl/headlessgl resolve their synchronous result — the calling contract is uniform everywhere.
   • mode: 'async' is auto-selection under that contract: pick the best provable backend now (webgl2 → webgl → cpu), force asyncMode on, and let the first kernel call upgrade to webgpu once an adapter has actually answered — the async contract is exactly what buys the room to probe. A declined or failed upgrade falls back to the proven kernel. Write await kernel(...) once and the same code runs on webgpu, webgl2, or cpu with no fallback rewrite.

Performance

Apple M1 Max, Chromium headed; WebGL2 = ANGLE Metal, WebGPU adapter = apple/metal-3. Results cross-checked between modes (worst relative error ≤ 2.2e-6) before any timing; medians after warmup; every timed run synced by reading back its own result; inputs ping-ponged so no dispatch is elidable. Reproduce: node scripts/benchmark-webgpu.mjs.

Throughput

Workload	cpu	webgl2 (single)	webgpu
matmul 512×512 (incl. readback)	284 ms	3.40 ms (83.5× vs cpu)	1.80 ms (157.8× vs cpu)
matmul 1024×1024 (incl. readback)	2340 ms	18.5 ms (126.5× vs cpu)	6.30 ms (371.5× vs cpu)
4M-element map (incl. readback)	6.25 ms	12.9 ms (0.5× vs cpu)	8.90 ms (0.7× vs cpu)
3-kernel pipeline chain, final readback only	18.5 ms	36.6 ms (0.5× vs cpu)	9.00 ms (2.1× vs cpu)
3-kernel chain, excl. final readback	—	6.35 ms	5.80 ms

Honest reading: compute-bound kernels run ~2–3× faster than WebGL2 and readback is dramatically cheaper (see the chain row: 4× end-to-end); trivially memory-bound single kernels remain transfer-dominated in every GPU mode, exactly as on WebGL.

Main-thread availability — what asyncMode buys

30 × matmul 512×512 with readback, 4 ms heartbeat running beside it (node scripts/benchmark-async.mjs):

configuration	backend	wall time	main thread blocked	worst single stall
webgl2 (sync)	WebGL2Kernel	105 ms	101 ms (96.3%)	105 ms
webgl2 + asyncMode	WebGL2Kernel	234 ms	3.3 ms (1.4%)	4.8 ms
webgpu	WebGPUKernel	77 ms	14.9 ms (19.4%)	9.6 ms
mode 'async' (→ webgpu)	WebGPUKernel	65 ms	4.5 ms (7%)	4.9 ms

The sync loop freezes the page for its entire duration in one 105 ms stall. asyncMode trades ~4 ms per-readback latency (Metal fence-completion granularity) for a main thread that never stalls past 5 ms — for UI-adjacent GPU compute, that's the whole game. Pipeline intermediates and await only final results to amortize the latency.

Scope (v1)

WebGPU supports: scalar and Array(2/3/4) returns; 1D/2D/3D outputs (large 1D dispatches fold across Y past the 65,535-workgroup limit); array / TypedArray / Input / nested-array arguments, numbers, booleans; constants; user functions; dynamicOutput; pipeline handles with refcounted buffers. Deferred with clear WebGPU backend does not yet support … errors: kernel maps, graphical, toString(), precision: 'unsigned', Math.random — in mode: 'async' these decline the upgrade and stay on the GL backend rather than erroring. The upgraded kernel is built with dynamicArguments so it is never stricter about argument sizes than the GL kernel-switching it replaces.

Verification

• New suites: 9 webgpu files (60 async QUnit tests) + test/features/async-mode.js (16 tests) — the async suite covers the Promise contract per backend, rejection-not-throw, the upgrade firing exactly when an adapter answers, chained setters landing before the upgrade decision, pipeline handles across the upgrade, and all three webgl2 transfer shapes against their synchronous reads.
• The main-thread-availability test is verified discriminating by mutation: wrap the blocking read in Promise.resolve() and exactly that test fails.
• Full suites: headed M1 browser 3806/0 (0 crashes), Node 2526/0, SwiftShader headless failure set identical to the 131-test known baseline (byte-for-byte diff of clean HEAD vs branch).
• Bring-up bugs found by the suites and fixed on the branch: pipeline-handle reuse by its producing kernel, immutable output-buffer lifetime, folded-dispatch thread.x corruption past 2²¹ elements, and the upgraded webgpu kernel refusing grown arguments that GL kernel-switching absorbs.
• README documents mode: 'webgpu', mode: 'async', and asyncMode; index.d.ts now carries the webgpu statics, both new modes, asyncMode, and setAsyncMode.

Adversarially-verified review (round 2)

A 19-agent multi-dimension review (5 specialist finders → per-finding adversarial verifiers, each instructed to refute) produced 34 raw findings; 14 were verified, 12 confirmed with concrete mechanisms, 2 refuted with evidence. All 12 are fixed on the branch (c147681, ec91b1c, 3be52bb), each with a regression test where the failure is testable:

• Critical, silent wrong results (2): a break behind an if inside a switch case compiled to WGSL that breaks the enclosing loop (now throws, matching the GL backends); outputs/arguments past the device's storage-binding limit resolved fully-shaped all-zeros results (the device now requests the adapter's real limits and throws a named error at the actual ceiling).
• WGSL codegen (3): mixed int/float ternaries rounded the float branch away (now promote to f32); prefix ++i emitted invalid WGSL (now postfix); helpers named after WGSL builtins lost their definitions to premature name-mangling (registry now keyed by original names).
• Async-contract seams (5): arguments are sampled at call time again (the review's nastiest find — const p = k(buf); buf[0] = 9 used to poison the queued run); asyncMode now survives kernel switches and CPU fallback; the mode-'async' upgrade only swaps to a webgpu kernel whose full async build succeeded, harvests per-kernel functions correctly, and no longer masks real errors behind a silent re-run; webgpu pipeline handles reaching a kernel that stayed on GL are read back transparently; combineKernels refuses the async contract with a clear error.
• Robustness (2): device loss now rejects with a rebuild instruction instead of resolving handles over dead buffers; a rejected mapAsync no longer strands pooled staging buffers.

New tests: test/internal/wgsl-codegen.js (7 tests, pure Node, 6 fail with the fixes reverted) plus 7 additions to the async-mode and webgpu-errors suites. Post-fix verification: Node 2539/0, headed M1 3821/0, SwiftShader failure set byte-identical to the develop baseline.

🤖 Generated with Claude Code

https://claude.ai/code/session_01RUTVDFaHav3uAdN3XfZLyx

[robertleeplummerjr]

[robertleeplummerjr]

It warms my heart to see this today. Wow! Nice work.