feat(models): add Qwen-Image-2.1 edits from reference images

[Pfannkuchensack]

Summary

Adds edits from reference images to Qwen-Image-2.1 (up to 4). The prompt says what to do with the references, for example put the jacket from the first image on the person in the second.

Stacked on #9784. This PR's own change is the last commit; the first one is #9784.

The model reads each reference twice, at the same size (about 1 MP at its aspect ratio, on the 32 px grid):

• Through the encoder. Qwen3-VL's vision tower reads the reference alongside the prompt, composited over white when it is transparent.
• As latents. The transformer places the reference's clean latents in the prompt's <|image_pad|> slots, 2×2 latents per slot. Alpha is kept, and the latents are normalized in the VAE dtype, as the pipeline does.

Backend

• qwen_image_2_1_text_encoder takes reference_images. It stores the slot mask and each reference's slot grid in the conditioning.
• qwen_image_2_1_i2l gains a reference mode. Its preprocessing is bit-identical to the pipeline's VaeImageProcessor.
• qwen_image_2_1_denoise takes reference_latents. It checks each reference against the prompt's slot grid, so a wrong order, count or size fails before the transformer loads, and it requires the negative prompt to be encoded with the same references.
• Vision tower: only the Qwen-Image-2.1 Diffusers model's encoder keeps it (+1.1 GB). The shared standalone Qwen3-VL loaders, which Ideogram 4 and Krea-2 also use, keep dropping it. A standalone encoder with references is refused with a clear message.
• New capability: FeaturesFacet.reference_images_require_format, analogous to reference_images_require_variant. Qwen-Image-2.1 declares "diffusers".
• Prefix KV cache: about 2 GiB per reference per guidance branch. When the caches would take more than half the GPU, the denoise runs without them and recomputes the prefix every step.

webv2

• New config type qwen_image_2_1_reference_image.
• Each reference is wired into both prompts, and through one reference-mode i2l node into the denoise node, in order.
• Metadata and recall support.
• References are offered only on Diffusers models. Invoke is blocked, with a reason, while a standalone encoder replaces the one that reads them.

Docs: the "Reference images" section on the Qwen-Image-2.1 page, the generate-panel and model-families rows, the recall API, and the integration guide.

Related Issues / Discussions

Part of #9594. Depends on #9784.

QA Instructions

• Python: the affected suites pass locally, ruff clean; the full suite runs in CI.
• webv2: pnpm lint passes. vitest: 10077 passed; 3 failures are locale-dependent tests that fail on a German-locale machine, unrelated to this change.
• Docs: the build passes, internal links are valid.
• Covered on CPU:
  • the encoder template and slot grids, using a tiny Qwen3-VL with its vision tower;
  • the vision tower reading the pixels;
  • white compositing;
  • bit-equality of the reference preprocessing with VaeImageProcessor;
  • the denoise sequence layout;
  • the refusals: grid mismatch, swapped references, channels, a negative prompt without the references;
  • the cache fallback;
  • the loader keeping the tower;
  • the webv2 graph order and the validation.
• GPU (RTX 4090, reserve_vram_gb: 1):
  • Edit parity: the reference pixels, the VAE input and the prompt embeddings (with the image-pad mask) are bit-identical to QwenImage21Pipeline; the edited image matches it at 45.6 dB, the rest being cuDNN's choice of convolution kernels for the reference encode. The model card's yacht-sketch edit gives a clean photorealistic result.
  • Encoder memory: reading references costs ~286 B·T² + 224 KiB·T above the encoder's weights (T ≈ 1030 tokens per reference): 0.5 / 1.6 / 3.2 / 5.4 GiB for 1-4 references. Without reserving it, four references made the driver spill the encode to system memory (> 8 min); the node now reserves it, and the encode takes 10.9 s.
  • No-cache path: 3 references at CFG 4 recompute the prefix every step (the caches would hold 12 GiB): 140 s for 20 steps against 75 s for 2 references with the cache. 4 references at CFG 4 run stably in 199 s.

Review

All material findings were resolved, with regression tests:

• a standalone encoder silently breaking Invoke on the Generate route, which is now a validation reason;
• KV caches that could not fit 4 references with CFG, which now use the no-cache fallback;
• order mismatches between same-count references, which are now matched by grid;
• the vision-tower check not recognizing the stubbed tower;
• checks running only after the transformer load.

Found in GPU testing and fixed, with a regression test: the encoder node reserved no room for the references' attention (above).

Remaining limitation: two references of identical size in swapped order cannot be told apart.

Compatibility / Rollout

• Generated API files: openapi.json and schema.ts are regenerated. The changes are additive: new node fields and the new capability field.
• Capabilities: reference_images_require_format is new and null for every other base.
• Memory: the Qwen-Image-2.1 Diffusers encoder now loads with its vision tower, about 1.1 GB more for every generation with it.

Checklist

• The PR has a short but descriptive title, suitable for a changelog
• Meaningful regression coverage added / updated where needed; obsolete tests/code removed
• Persisted-state and API changes include required migrations / compatibility validation
• Relevant performance/efficiency opportunities considered; material claims have evidence
• Material review findings resolved and relevant checks rerun
• Documentation added / updated (if applicable)
• Updated What's New copy (if doing a release after this PR)