feat(controlnet): add Qwen Image Fun-ControlNet-Union with canvas control layers

docs/src/content/docs/Users Guide/03.Models/03.model-format-support.mdx

@@ -1,7 +1,7 @@
 ---
 title: Model Format Support
 description: Which file formats and quantizations InvokeAI loads for every model family, text encoder, VAE and adapter.
-lastUpdated: 2026-10-16
+lastUpdated: 2026-10-18
 sidebar:
   order: 3
 ---
@@ -102,7 +102,7 @@ not check, so a quantized file there fails with an unclear error or decodes inco
 | SDXL | ✓ (LyCORIS, Diffusers, OMI) | ✓ | IP-Adapter, T2I-Adapter, textual inversion |
 | FLUX.1 | ✓ (LyCORIS, Diffusers, OMI) | ✓ | Control LoRA, IP-Adapter, Redux |
 | FLUX.2, Z-Image | ✓ (LyCORIS, Diffusers) | Z-Image only | |
-| Qwen-Image, Krea-2, Wan 2.2, Anima | ✓ (LyCORIS/kohya, including LoKR where noted on the family page) | Anima (ControlNet-LLLite) | |
+| Qwen-Image, Krea-2, Wan 2.2, Anima | ✓ (LyCORIS/kohya, including LoKR where noted on the family page) | Qwen-Image (single-file checkpoint), Anima (ControlNet-LLLite) | |
 | LTX-2, MiniMax H3 | ✓ (plain low-rank; LoKR, LoHA and DoRA are refused) | ✗ | |
 | SD 3, CogView4, Ideogram 4, ERNIE-Image, Qwen-Image-2.1 | ✗ | ✗ | |
 

docs/src/content/docs/Users Guide/03.Models/Local Models/qwen-image.mdx

@@ -1,7 +1,7 @@
 ---
 title: Qwen Image
-description: Generate and edit images with the Qwen Image 2512 text-to-image model and the Qwen Image Edit 2511 reference-image editing model, including GGUF installs and the Lightning LoRAs.
-lastUpdated: 2026-10-16
+description: Generate and edit images with the Qwen Image 2512 text-to-image model and the Qwen Image Edit 2511 reference-image editing model, including GGUF installs, the Lightning LoRAs and the Fun-ControlNet-Union.
+lastUpdated: 2026-10-18
 sidebar:
   order: 8
 ---
@@ -17,11 +17,13 @@ variants:
   person in image 2").
 
 The variant is detected automatically on install. Both variants can also run text-to-image,
-image-to-image, inpainting and outpainting on the Canvas; only the Edit variant accepts reference images.
+image-to-image, inpainting and outpainting on the Canvas; only the Edit variant accepts reference images,
+and only the text-to-image variant takes a [ControlNet](#controlnet).
 
 ## License
 
-Qwen Image 2512, Qwen Image Edit 2511, the Lightning LoRAs and the Qwen2.5-VL text encoder are released under **Apache 2.0**, which allows commercial use. None of the downloads is gated.
+Qwen Image 2512, Qwen Image Edit 2511, the Lightning LoRAs, the Qwen2.5-VL text encoder and the Qwen Image
+ControlNet Union are released under **Apache 2.0**, which allows commercial use. None of the downloads is gated.
 
 This is a summary, not legal advice. Read the full license on the model's HuggingFace page before using a model commercially.
 
@@ -135,6 +137,29 @@ Image**; connect an **Image to Latents - Qwen Image** node to the **Reference La
 Qwen Image LoRAs (diffusers PEFT, Kohya and LoKR formats) are supported and apply to the transformer.
 In the workflow editor use **Apply LoRA - Qwen Image** or **Apply LoRA Collection - Qwen Image**.
 
+## ControlNet
+
+The **Qwen Image ControlNet Union** starter model is alibaba-pai's *Fun-ControlNet-Union* for Qwen Image 2512
+(~3.5 GB, Apache 2.0). One model handles every condition: **Canny**, **HED**, **Depth**, **Pose**, **MLSD**,
+**Scribble** and **Gray** control images, plus **inpainting**. It is trained for the text-to-image transformer:
+with a Qwen Image Edit model selected, a control layer's settings and the **Invoke** button say that the
+ControlNet is not compatible, and nothing is generated until the layer is disabled or the model changed.
+
+On the Canvas, add a **Control Layer** while a Qwen Image model is selected. Its adapter type is **Qwen Image
+ControlNet** and its pixels are the control image; the layer's **Filter** turns a raster into one of the
+conditions (Canny, HED, Depth Anything, DW Openpose, MLSD, Scribble or Grayscale). The layer's **Weight** is
+the model's control scale: it ranges from `0` to `2`, starts at `0.85`, and `0.70` to `0.95` is the
+recommended range. **Only one** Qwen Image control layer can be used per generation.
+
+When the Canvas inpaints or outpaints with a control layer present, the ControlNet also receives the image being
+regenerated and the mask of the region as its inpainting condition. Without a control layer, inpainting and
+outpainting work as before.
+
+In the workflow editor, the same thing is the **Qwen Image ControlNet** node feeding the **Control** input of
+**Denoise - Qwen Image**. Give it a control **Image**, an **Inpaint Image** together with its **Inpaint Mask**
+(white is regenerated), or both. **Denoise - Qwen Image** then also needs its **VAE** input connected, which it
+uses to encode these images.
+
 ## PiD super-resolution decode
 
 Qwen Image works with [PiD Decode](/users-guide/image-generation/advanced-topics/pid-decode/), which replaces the VAE
@@ -147,7 +172,8 @@ Decoder Qwen-Image (2K to 4K)* decoders (including an int8 build of the latter)
 | ------------------------------------------- | ------------------------------------------------------------------- |
 | **Main Model - Qwen Image**                 | Loads the transformer, VAE and Qwen VL encoder (or a component source) |
 | **Prompt - Qwen Image**                     | Encodes the prompt and any reference images                         |
-| **Denoise - Qwen Image**                    | Runs sampling; accepts reference latents, img2img latents and masks |
+| **Denoise - Qwen Image**                    | Runs sampling; accepts reference latents, img2img latents, masks and a ControlNet |
+| **Qwen Image ControlNet**                   | Configures the ControlNet Union: control image and/or inpaint image with mask, scale, step range |
 | **Image to Latents - Qwen Image**           | VAE encode                                                          |
 | **Latents to Image - Qwen Image**           | VAE decode                                                          |
 | **Latents to Image - Qwen-Image + PiD (4x SR)** | PiD super-resolution decode                                     |

docs/src/content/docs/Users Guide/07.Canvas/02.canvas-guide.mdx

@@ -1,7 +1,7 @@
 ---
 title: Canvas guide
 description: A detailed guide to the Canvas — its tools and their options, navigation, the header buttons, the generation frame, selections, transform, filters, the staging area, menus, settings and keyboard shortcuts.
-lastUpdated: 2026-10-05
+lastUpdated: 2026-10-18
 ---
 
 import { Image } from 'astro:assets';
@@ -117,7 +117,7 @@ To crop a layer to the generation frame, choose **Crop layer to bbox** from its
 * **Edge detection** — Canny, HED, Lineart, Lineart Anime, MLSD, PiDiNet and Scribble.
 * **Depth & normal** — Depth Anything and Normal map.
 * **Pose & face** — DW Openpose and Mediapipe Face.
-* **Color & tone** — Color map and Adjust image.
+* **Color & tone** — Color map, Adjust image and Grayscale.
 * **Blur & noise** — Content shuffle, Blur and Noise.
 * **Upscale** — an upscaling model.
 

docs/src/content/docs/configuration/Optimization/low-vram-mode.mdx

@@ -210,8 +210,8 @@ To always decode untiled unless you ask for tiling, turn this off. A decode too
 auto_tiled_decode: false
 ```
 
-- **Always tile:** set `force_tiled_decode: true` in `invokeai.yaml`. It covers the VAE decode and encode for SD1.5/SDXL, FLUX.1, FLUX.2, Z-Image, Qwen-Image and Qwen-Image-2.1, the Anima encode, the control image of a Z-Image ControlNet, and the Krea-2 style reference. Other model families (for example SD3, Wan and LTX-2) ignore it.
-- **Per node:** the SD, FLUX.1, FLUX.2, Z-Image, Qwen-Image and Qwen-Image-2.1 **Latents to Image** and **Image to Latents** nodes, and **Image to Latents - Anima**, have **Tiled** and **Tile Size** fields for workflows. So does the **Z-Image ControlNet** node, for the control image the denoise encodes. The [Upscale panel](/users-guide/image-generation/upscale/) turns tiling on by itself.
+- **Always tile:** set `force_tiled_decode: true` in `invokeai.yaml`. It covers the VAE decode and encode for SD1.5/SDXL, FLUX.1, FLUX.2, Z-Image, Qwen-Image and Qwen-Image-2.1, the Anima encode, the control image of a Z-Image ControlNet, the control and inpaint images of a Qwen Image ControlNet, and the Krea-2 style reference. Other model families (for example SD3, Wan and LTX-2) ignore it.
+- **Per node:** the SD, FLUX.1, FLUX.2, Z-Image, Qwen-Image and Qwen-Image-2.1 **Latents to Image** and **Image to Latents** nodes, and **Image to Latents - Anima**, have **Tiled** and **Tile Size** fields for workflows. So do the **Z-Image ControlNet** and **Qwen Image ControlNet** nodes, for the images the denoise encodes. The [Upscale panel](/users-guide/image-generation/upscale/) turns tiling on by itself.
 - **Automatic retry:** if a FLUX.1, FLUX.2, Z-Image or Anima decode, or a FLUX.1 or FLUX.2 encode, runs out of memory, InvokeAI retries it once with tiling and logs that it did.
 
 A tiled decode is not pixel-identical to a single-pass one. The VAE looks at the whole image at once, so the difference is spread over the image rather than limited to the tile seams; it is small (for a 1024x1024 FLUX.2 Klein image, 5.6 of 255 per pixel value on average, with no visible seams), but the same seed will not reproduce an untiled result exactly.

invokeai/app/invocations/qwen_image/qwen_image_control.py

@@ -0,0 +1,118 @@
+"""Qwen Image ControlNet: the alibaba-pai Fun-ControlNet-Union for spatial control and inpainting."""
+
+from typing import Optional
+
+from pydantic import BaseModel, Field, model_validator
+
+from invokeai.app.invocations.baseinvocation import (
+    BaseInvocation,
+    BaseInvocationOutput,
+    Classification,
+    invocation,
+    invocation_output,
+)
+from invokeai.app.invocations.fields import FieldDescriptions, ImageField, InputField, OutputField
+from invokeai.app.invocations.model import ModelIdentifierField
+from invokeai.app.invocations.util import validate_begin_end_step
+from invokeai.app.services.shared.invocation_context import InvocationContext
+from invokeai.backend.model_manager.taxonomy import BaseModelType, ModelType
+from invokeai.backend.util.qwen_image_vae import QWEN_IMAGE_VAE_MIN_TILE_SIZE
+
+_CONTROL_SCALE_DESCRIPTION = "Strength of the control signal. Recommended range: 0.70-0.95."
+_BEGIN_DESCRIPTION = "When the control is first applied (% of total steps)"
+_END_DESCRIPTION = "When the control is last applied (% of total steps)"
+_TILE_SIZE_DESCRIPTION = (
+    f"{FieldDescriptions.vae_tile_size} Values between 1 and {QWEN_IMAGE_VAE_MIN_TILE_SIZE} are raised to "
+    f"{QWEN_IMAGE_VAE_MIN_TILE_SIZE}."
+)
+
+
+class QwenImageControlField(BaseModel):
+    """Qwen Image Fun-ControlNet conditioning: a control image, an inpaint image with its mask, or both."""
+
+    image_name: str | None = Field(default=None, description="The preprocessed control image (Canny, depth, pose, ...)")
+    inpaint_image_name: str | None = Field(default=None, description="The image to inpaint")
+    inpaint_mask_name: str | None = Field(default=None, description="The inpaint mask; white is regenerated")
+    control_model: ModelIdentifierField = Field(description="The Qwen Image ControlNet model")
+    control_context_scale: float = Field(default=0.85, ge=0.0, le=2.0, description=_CONTROL_SCALE_DESCRIPTION)
+    begin_step_percent: float = Field(default=0.0, ge=0.0, le=1.0, description=_BEGIN_DESCRIPTION)
+    end_step_percent: float = Field(default=1.0, ge=0.0, le=1.0, description=_END_DESCRIPTION)
+    tiled: bool = Field(
+        default=False,
+        description="Encode the control images in tiles. The denoise encodes them at generation size, which for an "
+        "upscale is the enlarged frame -- too large for an untiled encode on consumer hardware.",
+    )
+    tile_size: int = Field(default=0, ge=0, multiple_of=8, description=_TILE_SIZE_DESCRIPTION)
+
+
+@invocation_output("qwen_image_control_output")
+class QwenImageControlOutput(BaseInvocationOutput):
+    """Qwen Image Control output containing control configuration."""
+
+    control: QwenImageControlField = OutputField(description="Qwen Image control conditioning")
+
+
+@invocation(
+    "qwen_image_control",
+    title="Qwen Image ControlNet",
+    tags=["image", "qwen_image", "control", "controlnet", "inpaint"],
+    category="conditioning",
+    version="1.0.0",
+    classification=Classification.Prototype,
+)
+class QwenImageControlInvocation(BaseInvocation):
+    """Configure the Qwen Image Fun-ControlNet-Union for spatial control, inpainting, or both.
+
+    Takes a preprocessed control image (Canny, HED, depth, pose, MLSD, ...) and/or an image to inpaint with its mask
+    (white is regenerated), plus the Qwen Image ControlNet model. Recommended control scale: 0.70-0.95.
+    """
+
+    image: Optional[ImageField] = InputField(
+        default=None, description="The preprocessed control image (Canny, HED, depth, pose, MLSD, ...)"
+    )
+    inpaint_image: Optional[ImageField] = InputField(
+        default=None, description="The image to inpaint. Goes together with the inpaint mask."
+    )
+    inpaint_mask: Optional[ImageField] = InputField(
+        default=None,
+        description="The inpaint mask: white is regenerated, black is kept. Goes together with the inpaint image.",
+    )
+    control_model: ModelIdentifierField = InputField(
+        description=FieldDescriptions.controlnet_model,
+        title="Control Model",
+        ui_model_base=BaseModelType.QwenImage,
+        ui_model_type=ModelType.ControlNet,
+    )
+    control_context_scale: float = InputField(
+        default=0.85, ge=0.0, le=2.0, description=_CONTROL_SCALE_DESCRIPTION, title="Control Scale"
+    )
+    begin_step_percent: float = InputField(default=0.0, ge=0.0, le=1.0, description=_BEGIN_DESCRIPTION)
+    end_step_percent: float = InputField(default=1.0, ge=0.0, le=1.0, description=_END_DESCRIPTION)
+    tiled: bool = InputField(default=False, description=FieldDescriptions.tiled)
+    # NOTE: tile_size = 0 is a special value, as on the Qwen Image VAE nodes: the workflow UI cannot set None.
+    tile_size: int = InputField(default=0, ge=0, multiple_of=8, description=_TILE_SIZE_DESCRIPTION)
+
+    @model_validator(mode="after")
+    def validate_begin_end_step_percent(self):
+        validate_begin_end_step(self.begin_step_percent, self.end_step_percent)
+        return self
+
+    def invoke(self, context: InvocationContext) -> QwenImageControlOutput:
+        # Checked here rather than by a validator: connected inputs are filled in at execution time.
+        if (self.inpaint_image is None) != (self.inpaint_mask is None):
+            raise ValueError("The inpaint image and the inpaint mask go together: connect both or neither.")
+        if self.image is None and self.inpaint_image is None:
+            raise ValueError("Connect a control image, an inpaint image with its mask, or both.")
+        return QwenImageControlOutput(
+            control=QwenImageControlField(
+                image_name=self.image.image_name if self.image is not None else None,
+                inpaint_image_name=self.inpaint_image.image_name if self.inpaint_image is not None else None,
+                inpaint_mask_name=self.inpaint_mask.image_name if self.inpaint_mask is not None else None,
+                control_model=self.control_model,
+                control_context_scale=self.control_context_scale,
+                begin_step_percent=self.begin_step_percent,
+                end_step_percent=self.end_step_percent,
+                tiled=self.tiled,
+                tile_size=self.tile_size,
+            )
+        )