[Pfannkuchensack]

Summary

Adds Qwen-Image-2.1 (Base and the 8-step Turbo) as a new qwen-image-2-1 base. This covers text-to-image plus canvas img2img, inpaint and outpaint. Before this, the released files were misidentified: the GGUF as FLUX schnell, the VAE as SD1.

Model support

• Formats: Diffusers folders (QwenImage21Pipeline); single-file transformers in bf16, ComfyUI scaled fp8 or int8_convrot; GGUF transformers; standalone VAEs in Diffusers or ComfyUI layout.
• Nodes: model loader, Qwen3-VL 8B text encoder, denoise, i2l and l2i.
• Parity: the denoise and the decode reproduce QwenImage21Pipeline bit for bit. That covers the prompt template and pre-norm hidden states, the Base shift and Turbo sigma table, CPU noise, the scheduler's bf16 step, the prefix KV cache, and the VAE's denormalization and pixel mapping, both done in the VAE's dtype as the pipeline does.
• Strength: on Base, img2img and inpaint pick the start step by strength (step index), as Qwen-Image, Z-Image and diffusers' img2img pipelines do, so a strength changes the image about as much as on those models (0.75 starts at sigma ~0.85 at 40 steps). Turbo picks by noise level (clip_timestep_schedule_fractional): its table is front-loaded, so by index any strength from 0.5 to 0.85 would start above sigma 0.89.
• RGBA VAE: a decode is saved as RGB when at most 0.1% of pixels have alpha < 224, otherwise as RGBA. Encodes treat the input as opaque. The VAE, bundled or standalone, runs at the configured precision, except float16, which is unstable for it.
• Memory: encodes and decodes tile up front when the untiled frame would not fit (1024/768/512 px tiles, via the shared Qwen-Image tiling helper at 16x). The working-memory estimates cover the KV cache, CFG, SDPA scores and the GGUF dequant transient, and scale with the VAE and compute dtypes.
• Refused at install (InvalidMatchError): nvfp4 files, torchao-quantized files (such as unsloth's FP8 build; loading them is [enhancement]: Load torchao-quantized single-file transformers (unsloth FP8) #9798), int8 files without the convrot marker, and Q8_CR GGUFs. The check is shared with Ideogram 4.
• Identification: Qwen-Image LoRA configs no longer claim 2.1 LoRAs; attention-only ones are told apart by their 4096 width. The FLUX GGUF config now requires double_blocks, so the 2.1 GGUF is no longer taken for FLUX schnell.

Starter models

• Diffusers Base and Turbo, the Q4_K_M GGUF and the ComfyUI VAE.
• The Qwen3-VL 8B encoder starter moves to common.py, shared with Ideogram 4. previous_names keeps existing installs matched.

Frontend

• webv2: base identity, generation policy (VAE and Qwen3-VL 8B encoder are required for non-Diffusers models), graph builder, canvas mapping and metadata modes.
• webv1 enums and labels for shared-schema compatibility.

Docs: a new model page, plus the requirements, model-families, generate-panel, parameters, low-VRAM and FP8 pages.

Reference images (Edit) and LoRA support follow in separate PRs.

Related Issues / Discussions

Part of #9594. Builds on #9774 (diffusers 0.41.0).

QA Instructions

• Python: the affected suites pass locally, ruff clean; the full suite runs in CI.
• webv2: pnpm lint passes. vitest: 10073 passed; 3 failures are locale-dependent tests that fail on a German-locale machine, unrelated to this change.
• webv1: tsc shows the same 16 errors as on the base commit 9d30e6a148; none are new.
• Docs: the build passes and all internal links are valid.
• Parity (RTX 4090): final latents are bit-identical to QwenImage21Pipeline at 1024² and 2048² (40 steps). The decoded 1024² image is RGB bit-identical to the pipeline's (it was off by up to 12/255 on a quarter of the pixels before the dtype fix); a tiled decode differs from the pipeline's untiled one by the tiling.
• Browser (webv2, GGUF Q4_K_M + fp8 encoder + ComfyUI VAE):
  • txt2img, about 47 s for 40 steps at 1024².
  • CFG 4 with a negative prompt.
  • Recall All, which restores model, VAE, encoder, CFG, seed and prompts.
  • Canvas img2img, inpaint and outpaint.
  • A 2048² img2img, where the encode tiles.
• Real files (RTX 4090, 1024², 40 steps, same seed, against the Diffusers transformer): ComfyUI bf16 is bit-identical; int8_convrot 37 dB; GGUF Q4_K_M 21 dB (visibly the same image). The ComfyUI Turbo bf16 file is identified as Turbo and gives clean 8-step images. unsloth's FP8 (torchao) file is refused at install. No ComfyUI scaled-fp8 2.1 build is published; that path is covered by fixtures.

Review

All material findings were resolved, with regression tests:

• the Turbo strength mapping;
• NaN sigmas at one Base step;
• missing encode auto-tiling;
• the dtype-blind VAE estimate;
• install-time refusal of nvfp4 and unmarked int8;
• attention-only LoRA misidentification;
• the webv1 enum.

From the maintainer review: the decode now matches the pipeline's dtype order (bit-identical image), the bundled VAE honours the configured precision and never runs in float16, the denoise estimate scales with the compute dtype, the Turbo fixture carries the real Turbo scheduler config, and the webv2 builder is tested at CFG > 1 and for the GGUF component requirements.

From the second maintainer review: Base image-to-image starts by step index (above), Qwen-Image-2.1 is listed on the Model Format Support page, the builder's unreachable component checks are gone, and tests pin the dtype width at the call site, the strength rounding, the guidance slice and the encoder edge's source field.

Remaining limitation: LoRAs that target only to_out or the MLP halves, and attention-only LoKr, are told apart from Qwen-Image LoRAs only with #9793.

Compatibility / Rollout

• Models already installed: Qwen-Image-2.1 files installed before this PR keep their wrong records (FLUX/SD1) until they are removed and reinstalled. There is no migration.
• Starter rename: the 8B encoder starter is renamed "Qwen3-VL 8B Encoder" and keeps its old name in previous_names.
• Generated API files: openapi.json and schema.ts are regenerated; the changes are additive.

Checklist

• The PR has a short but descriptive title, suitable for a changelog
• Meaningful regression coverage added / updated where needed; obsolete tests/code removed
• Persisted-state and API changes include required migrations / compatibility validation
• Relevant performance/efficiency opportunities considered; material claims have evidence
• Material review findings resolved and relevant checks rerun
• Documentation added / updated (if applicable)
• Updated What's New copy (if doing a release after this PR)

[lstein]

Approving at 37efce3. I found no blockers in an adversarial review of the inference path, model identification and the frontend/API contract.

Please fix before merge: the image-parity claim in the description

The decoded image is not bit-identical to QwenImage21Pipeline when the VAE runs in bf16, which is the default on CUDA and MPS.

• qwen_image_2_1_latents_to_image.py:58 denormalizes in fp32 (latents.float() * std + mean) and casts to vae.dtype once.
• The pipeline (pipeline_qwenimage21.py, end of __call__) first casts the latents to vae.dtype, then casts latents_mean/latents_std to bf16 and does the multiply-add in bf16.

I loaded the real ComfyUI VAE through convert_comfy_vae_to_diffusers and decoded the same latents both ways on CPU in bf16. About 40% of the decoder input elements differ, and about 25% of output pixels differ by up to 12/255. Running the post-processing on its own gives 0 differing pixels, so the denormalization order accounts for the whole difference.

The fp32 order is arguably the more accurate one, so either fix is fine:

• match the pipeline order if exact parity matters, or
• narrow the Parity bullet and the QA line ("the image is identical at 2048²") to say the latents are bit-identical.

Non-blocking notes

1. Bundled VAE precision. QwenImage21DiffusersModel._load_model loads every submodel at choose_bfloat16_safe_dtype(...), the VAE included. As a result:

   • precision: float32 is ignored for the bundled VAE.
   • On a GPU without bf16 the VAE gets fp16, which _wan_family_dtype documents as unstable for this autoencoder.

   The standalone VAE path does use _wan_family_dtype.

2. Denoise working-memory estimate. The 0.25 MiB/token activation term and the 0.5 MiB/text-token KV term in _estimate_working_memory assume 2-byte weights. Where the model falls back to fp32 (some MPS/XPU devices), the estimate is about half the real requirement. The VAE estimate already scales with dtype.itemsize; this one could too.

3. Turbo fixture scheduler config. The Turbo stripped-model fixture contains the Base scheduler_config.json (use_dynamic_shifting: true, shift_terminal: 0.02). The runtime is unaffected, because TURBO_SCHEDULER_CONFIG is hard-coded and matches the real HF file. The fixture would still mislead anything that read it later.

4. No CFG > 1 coverage for the graph builder. graph.test.ts has no qwen-image-2-1 case, and the node-type snapshot only covers cfg 1. Dropping the neg_cond → negative_conditioning edge would go unnoticed, and CFG > 1 would silently run without a negative prompt. One builder test at CFG > 1 would also be a good place to assert the GGUF/checkpoint requireComponent errors.

5. LoRA identification gap (already listed as a limitation). The _has_qwen_image21_lora_keys width check only looks at lora_A/lora_down on to_q/to_k/to_v. Two kinds of 2.1 LoRA are therefore still identified as qwen-image:

   • a 2.1 LoRA that touches only to_out.0 plus img_mlp.proj/img_mlp.out;
   • an attention-only LoKr.

   Fine to leave for the LoRA PR.

6. Smaller items:

   • The legacy UI lists 2.1 models but fails at Invoke with "No graph builders", like other bases it cannot build.
   • A standalone Qwen3-VL 8B encoder left selected from an Ideogram 4 checkpoint overrides the encoder bundled in a 2.1 Diffusers model, and Recall doesn't clear it. Krea-2 and Ideogram 4 already behave this way.
   • Turbo is detected from the filename alone for single files, so a renamed Turbo GGUF gets Base defaults.

Verified clean

• Identification regressions. I ran from_model_on_disk over all 197 stripped fixtures on both the base and head trees. Only the 8 new Qwen-Image-2.1 fixtures changed. I also built 26 synthetic files of realistic shape and none changed identification. Together these cover:

  • FLUX/Kontext/Krea/Schnell GGUFs, including prefixed ones;
  • Qwen-Image 1 LoRAs in PEFT, Kohya, attention-only and LoKr formats;
  • Qwen-Image/Wan/Anima/SD/FLUX VAEs.

  pytest tests/model_identification tests/backend/model_manager: 2276 passed.

• Install-time refusals. An InvalidMatchError does not stop other configs from matching, and the 2.1 refusal only runs after the 2.1 key check passes. The Ideogram 4 refusal and GGUF dequant helper were moved without changes.

• Sigmas. Base sigmas match retrieve_timesteps exactly (torch.equal) across step counts and sequence lengths. Turbo resampling is correct, and 1-step Base has no NaN.

• Strength slicing. clip_sigmas handles every strength edge case tried.

• Other inference checks:

  • CPU noise is seed-reproducible.
  • The scheduler step matches the scheduler's dtype handling.
  • The prefix KV caches are local to each invocation and separate for positive and negative.
  • The system-prefix length is 14.
  • Inpaint mask polarity and re-noising are correct.

• Shared tiling helper. The default spatial_scale=8 keeps existing Qwen-Image and Wan callers byte-identical.

• API schema. openapi.json regenerates identically from the Python, and the schema diff only adds things.

• Frontend. webv2 lint:tsc passes and vitest across generation/core, models/core, workflow/core and workbench/image-actions passes (1515 tests). The graph builder's field names match the invocations.

• Starters. The encoder starter rename keeps both source-based and previous_names matching working.

[lstein]

I smoke-tested this PR. With both Base and Turbo, inpaint and img2img need a denoising strength of about 0.90 before the image changes materially. On most other models, 0.75–0.85 already has a strong effect.

The cause is the strength mapping, not the inpaint code. clip_sigmas picks the start by noise level (strength 0.75 → sigma 0.75), the same way FLUX, FLUX.2, SD3 and CogView4 do. The Qwen-Image, Krea-2, Z-Image, Anima and Wan nodes pick by step index instead, and so do diffusers' own img2img pipelines (get_timesteps in pipeline_qwenimage_img2img.py). Their shifted schedules stay at high sigma for most of the steps, so picking by step index injects much more noise at the same strength.

Starting sigma at 1024², computed with this PR's own schedules (build_sigmas, TURBO_SIGMAS):

Strength	Base 40 steps, by step index	This PR (by noise level)	Turbo, by step index
0.50	0.657	0.50	0.895
0.75	0.853	0.75	0.954
0.85	0.916	0.85	0.978
0.90	0.946	0.90	0.978

So strength 0.90 on 2.1 is roughly 0.80 on Qwen-Image 1 or Z-Image, which matches what I saw.

I understand why noise-level selection was chosen for Turbo: picking by step index wastes most of the slider, since strength 0.50–0.85 all starts above sigma 0.89. My suggestion:

• Base: pick by step index, which matches Qwen-Image 1 and the diffusers img2img convention.
• Turbo: keep noise-level selection.

If one rule for both variants matters more, the alternative is to keep the current mapping and note in the model docs that 2.1 needs a higher denoising strength than Qwen-Image 1 for comparable change.

Not a blocker; the approval stands.

[lstein]

PR is still a work in progress. Withdrawing approval to avoid confusion.

[Pfannkuchensack]

Thanks for the thorough review. Addressed in faf49ed:

• Parity: l2i now denormalizes and maps to pixels in the VAE dtype exactly like the pipeline / VaeImageProcessor (the post-processing order was a second, smaller source). The 1024² image is now RGB bit-identical to QwenImage21Pipeline on a 4090; tests compare against VaeImageProcessor directly.
• Notes 1–4: the bundled VAE uses _torch_dtype_avoiding_float16() (honours precision, never fp16); the denoise estimate scales with the compute dtype; the Turbo fixture has the real Turbo scheduler config; there's a graph test at CFG > 1 plus the GGUF component errors.
• Note 5 is in feat(models): add LoRA support for Qwen-Image-2.1 #9793, along with norm-diff support (Comfy's Turbo LoRA) and a narrower FLUX Kohya detector.
• Also new: torchao-quantized files (unsloth FP8) are refused at install.

Real-file and GPU results are in the descriptions; the stack is ready from my side.

[lstein]

Re-reviewed at 43e614e (fix faf49ed plus the two main merges). Thanks, this addresses the parity request and the non-blocking notes well.

Verified:

• Decode parity. l2i plus to_image now produce exactly what QwenImage21Pipeline + VaeImageProcessor.postprocess produce, in bf16, fp16 and fp32, including NaN/±inf edge values (checked on CPU with synthetic tensors). The new decode test checks against diffusers' own processor, and reverting either half of the fix makes it fail.
• Bundled VAE dtype. It honours precision and is never fp16. Note that on CPU-only setups it now runs in fp32 rather than bf16, which is correct but doubles VAE memory.
• Denoise working memory now scales with the compute dtype, and quantized transformers are still sized by their compute dtype.
• Turbo fixture. Its scheduler config now equals TURBO_SCHEDULER_CONFIG, and identification is unchanged.
• CFG > 1 graph test. It catches a dropped or miswired negative-conditioning edge and an inverted CFG gate.
• torchao refusal. It only runs after the 2.1/Ideogram 4 fingerprint matches, so it can't refuse other bases, and it raises InvalidMatchError, so other configs still get to match.
• ERNIE-Image merge. Both sides are kept in factory.py, configs/main.py, the starters and the loaders, with no duplicate registrations. openapi.json/schema.ts regenerate byte-identically.
• Local runs.
  • tests/model_identification + tests/backend/model_manager: 2290 passed, 150 hardware/env skips.
  • Qwen-Image-2.1 node/VAE/config tests: 51 passed.
  • graph.test.ts: 83 passed.
  • lint:tsc: clean.

Follow-ups (non-blocking)

1. Img2img/inpaint strength is unchanged and undocumented. clip_sigmas still picks the start by noise level for both Base and Turbo, so Base needs about 0.90 strength for the change that ~0.80 gives on Qwen-Image 1, Z-Image and similar models (see my earlier comment). If noise-level selection stays for both variants, please at least note in qwen-image-2-1.mdx that 2.1 needs a higher denoising strength than Qwen-Image 1 for comparable change.
2. Model Format Support page has no Qwen-Image-2.1 entries. Main added 03.model-format-support.mdx, and the merged 02.model-families.mdx:45 now points to it for "which format loads for which model". The family matrix, the encoder table (Qwen3-VL 8B) and the VAE table there all omit 2.1, and its torchao refusal is mentioned only on the family page.
3. The requireComponent calls at graph.ts:1300-1301 are dead code. compileGenerateGraph throws the first policy validation reason (baseGenerationPolicies.ts missingMessage) before the builder runs, so these never execute. Deleting both still passes the new GGUF/checkpoint test, which is really testing the policy. Their label also says "Qwen3-VL Encoder" rather than "Qwen3-VL 8B Encoder". Either drop them or make them the source of the message.
4. Small test gaps:
   • The float32 working-memory test calls the static estimator directly, so hard-coding 2 instead of dtype.itemsize at the call site (qwen_image_2_1_denoise.py:239) passes every test.
   • The CFG > 1 graph test checks only the source node_id of the neg_cond.qwen3_vl_encoder edge, so wiring it from the wrong loader output field (e.g. vae) passes.