Skip to content

feat(upscale): support FLUX.2 and Qwen-Image in the Upscale widget - #9934

Merged
joshistoast merged 1 commit into
invoke-ai:mainfrom
Pfannkuchensack:feat/upscale-flux2-qwen-image
Oct 19, 2026
Merged

joshistoast merged 1 commit into
invoke-ai:mainfrom
Pfannkuchensack:feat/upscale-flux2-qwen-image

Conversation

@Pfannkuchensack

@Pfannkuchensack Pfannkuchensack commented Oct 19, 2026 •

Copy link
Copy Markdown
Member

Summary

The Upscale widget can now use FLUX.2 main models (Klein 4B/9B, Klein Base, [dev]) and Qwen Image main models. It runs the same pass as the FLUX.1, Z-Image, Anima and Krea-2 entries: Spandrel enlarges the image, then a single img2img pass covers the whole frame, with tiled VAE encode and decode. No backend changes; the prerequisites are merged (#9890 FLUX.2 VAE tiling, #9892 Qwen Image shift hold).

FLUX.2

  • Creativity: flux2_denoise clips denoising_start by sigma value, so Creativity maps exactly as it does for FLUX.1.
  • Scheduler: above pure noise the node always steps with its own Euler, so the scheduler field is hidden and the metadata records Euler.
  • Components:
    • Klein gets the Qwen3 encoder of its own size: the 4B for the 4B models, the 8B for the 9B models.
    • [dev] gets the FLUX.2 Mistral encoder.
    • A Diffusers install (or a complete SDNQ Klein) brings its own encoder and VAE.
  • Guidance goes to guidance and the negative prompt is never wired, as in Generate.
  • Model change: picking a FLUX.2 model sets Steps and Guidance to that model's served defaults: Klein 4 steps and guidance 1, Klein Base 28 steps, [dev] 28 steps and guidance 3.5.
    • Steps follow the model because flux2_denoise spreads the requested steps over the clipped window. Every step runs at full upscale resolution, so Klein at the widget's 30 would cost about 7× what it was distilled for.
    • Switching to another family puts Steps back to 30 and CFG back to 2.
  • Guidance cap: the widget refuses guidance above the served guidance_max (20 for FLUX.2), which the node would otherwise refuse after the upscale has already run.

Qwen Image

  • Creativity: qwen_image_denoise clips by step index with Python's round, so the widget translates Creativity to the step nearest FLUX's entry sigma.
  • Schedule: the node's schedule is the exponential shift, held at 8192 tokens, followed by diffusers' stretch to a terminal sigma of 0.02. A shift value alone cannot describe it.
    • UpscaleIndexClipping.shift(model, size) therefore becomes sigmas(model, size, steps).
    • Z-Image, Anima and Krea-2 share a helper for the plain shifted schedule, and their recorded starts are unchanged byte for byte.
  • CFG: 4 on model pick. The negative branch is wired only above CFG 1, and no shift is sent, so the node's held dynamic shift applies.
  • Components: a GGUF or single-file model needs a Qwen VL encoder and a Qwen Image VAE; a Diffusers install brings both.
  • Edit models are refused. Without a reference image the node builds a blank reference at the full output size. That doubles the image tokens on every call, the negative pass included: about 2.5–3× the time and unreserved activation memory, for no benefit.

Qwen Image starts, Balanced at 30 steps: step 19 at 1024², step 21 at 2048² and 4096².

Related Issues / Discussions

Follow-up to #9825 and #9889. Builds on #9890 and #9892. Qwen Image 2.1 is not included: its Base model clips by step index, but Turbo clips by noise level, so the table would need clipping per variant.

QA Instructions

Automated (webv2, run one by one)

  • lint (oxfmt, oxlint, tsc --noEmit, architecture) and build: pass.
  • test: 10,847 passed. Three tests that format numbers with the system locale fail on this German-locale machine without this change too.
  • Browser tests for the Upscale and Generate panels: 229 passed.
  • test:fixtures (40), test:performance:architecture (build and browser, no baseline change), test:project-files:browser and test:accessibility:browser: pass.

Automated (Python)

  • test_upscale_index_clipped_starts.py and test_frontend_graph_node_types.py: 1,076 passed, 3 skipped.
  • The recorded starts now include Qwen Image at 512/1024/2048/4096 with 8 and 30 steps. Each case drives the real qwen_image_denoise with the real diffusers scheduler and checks that the pass enters at the step nearest FLUX's sigma.
    • Translating without the terminal stretch fails 16 of the 48 Qwen cases.
    • Using the floor rule fails 27 of them.
  • New: the real flux2_denoise is driven to the img2img preblend for the starts the widget sends. The test checks that it enters at exactly 1 - denoising_start and steps with its own Euler.
    • Clipping before the shift fails it.
    • Dropping the exact-entry insertion fails it.
    • Handing img2img to a scheduler fails it.
  • The upscale graph contract now covers 8 architectures: 58 node types and 239 literal values.

End to end (RTX 4090, 1024² source, Balanced, 2× to 2048², through the Invoke button):

Model Start Time
FLUX.2 Klein 4B fp8 0.499, 4 steps, guidance 1, Qwen3 4B 40 s
FLUX.2 Klein 9B fp8 0.499, 4 steps, Qwen3 8B picked automatically 72 s
Qwen Image 2512 Q4_K_M GGUF 0.7 (step 21/30), CFG 4, negative branch 2:24 min
  • All three keep the composition and the lettering.
  • Clouds, rock and paper texture come out sharper.
  • No tile seams are visible at 100%.
  • Encode and decode run tiled (1024) in every case.
  • The metadata records the encoders, the VAE, guidance or CFG, the steps and the scheduler.
  • FLUX.2 [dev] was not run on a GPU; the graph tests cover its loader, Mistral encoder and guidance.

UI, checked in the browser

  • Klein 4B shows the Qwen3 4B encoder (not the 8B), Steps 4, CFG 1, no scheduler and no negative prompt.
  • Qwen Image shows the Qwen VL encoder, the Qwen Image VAE, CFG 4 and Steps 30.
  • Switching from Klein to SDXL brings back Steps 30, CFG 2 and the scheduler field.

Review

Resolved:

  • Qwen Image Edit is refused instead of running at double cost.
  • FLUX.2 takes its served step count.
  • A contract test now ties FLUX.2's value clipping to the real node.
  • Guidance above FLUX.2's cap is refused before queueing.

Remaining, shared with existing entries:

  • If an Edit or Fill model is the first installed main model, the widget picks it and then shows the refusal message.
  • The guidance slider is labelled "CFG scale" for FLUX models, as it already is for FLUX.1.

Compatibility / Rollout

  • Persisted widget values: two new nullable fields, mistralEncoderModel and qwenVlEncoderModel. Older values normalize them to null, and the model sync fills them in.

Checklist

  • The PR has a short but descriptive title, suitable for a changelog
  • Meaningful regression coverage added / updated where needed; obsolete tests/code removed
  • Persisted-state and API changes include required migrations / compatibility validation
  • Relevant performance/efficiency opportunities considered; material claims have evidence
  • Material review findings resolved and relevant checks rerun
  • Documentation added / updated (if applicable)
  • Updated What's New copy (if doing a release after this PR)

View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

Index-clipped Creativity now aims at each node's own sigma schedule, which Qwen-Image's terminal stretch needs.
FLUX.2 takes its served steps and guidance on model change and refuses guidance above its cap; Qwen-Image Edit is refused.
@github-actions github-actions Bot added frontend PRs that change frontend files python-tests PRs that change python tests docs PRs that change docs labels Oct 19, 2026
@joshistoast
joshistoast merged commit 1d6369c into invoke-ai:main Oct 19, 2026
22 of 23 checks passed
@Pfannkuchensack
Pfannkuchensack deleted the feat/upscale-flux2-qwen-image branch October 19, 2026 10:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs PRs that change docs frontend PRs that change frontend files python-tests PRs that change python tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants