OffscreenCanvas in Workers

[triniwiz]

An OffscreenCanvas can now move to a Worker, which then owns its context. This covers 2d, bitmaprenderer, webgl, webgl2 and webgpu, for canvases sent from the main thread and canvases made in the Worker. On NativeScript Windows it moves with worker.postMessage({ canvas }, [canvas]), with NativeScript/windows#23. The Android and iOS runtimes can't transfer it through postMessage yet, so there it goes as a handle:

// main
worker.postMessage({ canvas: OffscreenCanvas._toHandle(canvas.transferControlToOffscreen()) });

// worker
import '@nativescript/canvas/worker';
const canvas = OffscreenCanvas._fromHandle(data.canvas);

_toHandle / _fromHandle / _releaseHandle (take, adopt once, drop) are what a runtime transfer hook calls (OffscreenCanvas registers itself with __nsRegisterTransferable where the runtime has it), so postMessage(…, [canvas]) can work there later without app changes.

How it works

• crates/canvas-c/src/offscreen: CanvasOffscreenSurface, made on the UI thread from a transferred canvas's view, or viewless.
  • It holds the view's layers or window and the one context made on it.
  • The UI thread only reaches that context through the 2D render thread (a new weak TargetHandle split out of RenderTarget), the WebGL thread, or wgpu's locks.
  • It hears back through a sink: the bitmap resized, or a context was made (to unhide or set opacity on the view).
  • A view-backed context is always threaded.
• Bindings:
  • CanvasModule.OffscreenSurface (OffscreenSurfaceImpl, and crates/canvas-napi/src/offscreen.rs).
  • NSCCanvas.transferToOffscreenSurface() in Swift and Kotlin, with the Android JNI in org_nativescript_canvas_NSCCanvas_offscreen.rs.
• TS:
  • OffscreenCanvas draws through surfaces and no longer imports Canvas; Canvas/common.ts imports it instead.
  • @nativescript/canvas/worker is the API without the view.
  • Context classes take an existing native object, and recognize a Canvas view by a brand instead of instanceof.
• Native support:
  • Viewless threaded WebGL on every platform.
  • WebGPU contexts with no surface.
  • Metal layers whose drawable the render thread sizes.
  • A per-thread canvases-behind count, so a Worker's canvases only hold back its own requestAnimationFrame.

Fixes found on the way

• Android: toDataURL of a threaded 2D canvas read whichever GL context was current on the render thread.
• Android: a 2D context on a pbuffer (no window) didn't grow on resize.
• Isolate install: a new isolate could get a dead Worker's cached templates (Caches::Remove on install).
• iOS: file helper callbacks from a Worker ran on main; NSOperationQueueWrapper now retains its run loop.
• Image sources: GPUCanvasContext.__toDataURL when unconfigured, and iOS drawAtlas with an OffscreenCanvas source.

Windows

A transferred canvas moves to a Worker as on Android and iOS; the hidden host (OffscreenCanvas._legacyHost) is gone.

• offscreen/windows.rs: an OffscreenSurface over the view's SwapChainPanel.
  • The context lives where the OffscreenCanvas is used: 2D on the render thread, WebGL on the WebGL thread, WebGPU through wgpu.
  • Swapchains are bound on the UI thread through a message-only window (dxgi::UiThread). wgpu's SetSwapChain from a Worker goes through it too.
  • Transparent canvases present into the view's SurfaceImageSource.
  • The surface keeps the fit, composition scale and view size, and tracks what it shows itself instead of in the UI thread's 2D SHOWN table.
• Bindings: NSCCanvas.transferToOffscreenSurface(listener) (plus offscreenToDataURL and detachOffscreenSurface) in host/windows.rs, and _createOffscreenSurface() in napi-host.ts.
• Fixes on Windows:
  • Viewless WebGL now uses texture contexts, so resizing it works.
  • A transparent canvas whose fit overflows its view was cut in the wrong place and drew over its neighbours. It is now placed with a margin and clipped to its panel.
  • A Worker's env no longer takes the native log console from the main thread.
• Demo: canvas-offscreen-worker shows a Worker drawing a transparent 2D and an opaque WebGL view.
• Runtime: Workers need Transfer lists in worker postMessage, native addons in workers windows#23 (Node-API callbacks on the Worker's thread, requestAnimationFrame and ~/ paths in Workers, transfer lists). 0.1.0-beta.5 can't run bundled Workers.

On macOS cargo check -p canvas-napi hits two errors, both in Windows-only #[napi] methods outside this change (get_adapter_luid, draw_d3d_shared_frame); none of them is in offscreen.rs.

Tests

• offscreen.worker.handle, offscreen.worker.transfer, offscreen.worker:
  • Each context type drawn in a Worker and read back there.
  • A Worker's frames shown in the transferred view.
  • A resize in the Worker reaching the view.
  • Drawing on after the view is disposed.
  • Terminating a drawing Worker.
• offscreen.worker.postMessage: a canvas transferred to a Worker and back, a view's canvas transferred to a Worker, and what can't be transferred.
• Rust host test: crates/canvas-c/tests/offscreen_surface.rs.

iOS simulator:

• offscreen passes 64/64 and 2d passes 180/180.
• webgpu, canvassource, scaling and lifecycle each pass on their own.
• One full run stalled at webgpu.video after 313 passes and no failures. That suite passes alone; I haven't checked whether master stalls the same way.

Android emulator:

• offscreen.worker passes 27/27, 2d 180/180 and webgl 83/83, and the remaining groups pass.
• The only failures are WebGPU tests: the emulator has no adapter.

Windows (with NativeScript/windows#23):

• offscreen passes 69/69.
• A full run passes 425/427. The two failures are in webgpu.video and fail the same way on 0.1.0-beta.5.
• Checked on screen: a Worker's 2D and WebGL frames show in their views.

The checked-in Windows canvasnative.node is rebuilt for x64 and arm64. The checked-in xcframework and AAR are not rebuilt in this PR; the native libraries still need a full rebuild.

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

• Configuration used: Organization UI
• Review profile: CHILL
• Plan: Advanced
• Run ID: b5cfe620-22bd-49d3-83f5-7439670655f4

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

• Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}