Repository navigation
fix(flux2,z-image): run Diffusers schedulers on the node's sigma schedule - #9851
Open
Pfannkuchensack wants to merge 1 commit into
Open
Pfannkuchensack wants to merge 1 commit into
Pfannkuchensack wants to merge 1 commit into
Conversation
…dule FLUX.2 shifts before clipping on every path; Euler/LCM keep the terminal sigma and Heun runs the shifted schedule. Z-Image Heun runs the clipped sigmas from the img2img start; Heun progress reaches n/n on both. Upscale offers Heun for FLUX.1 and Z-Image, which now start at the blend. Closes invoke-ai#9792
Member
Author
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Summary
#9724 fixed FLUX.1's Diffusers scheduler paths. FLUX.2 and Z-Image still had the same problems:
denoising_end < 1finished the image instead of stopping. txt2img also clipped the unshifted schedule, while img2img clipped the shifted one, so a pass ending atdenoising_end = xand a pass continuing fromdenoising_start = xdid not meet at the same sigma.muanddenoising_end.order=2is halved bycalc_percentage).Now:
time_shift_flux2(mu)once, before clipping, on every path. Its Diffusers schedulers are built without a shift of their own. Euler and LCM run those sigmas plus the requested terminal sigma via a newset_timesteps_from_sigmas()helper (shared with FLUX.1's Euler); Heun runs them viaset_heun_timesteps_from_sigmas()from Fix FLUX scheduler integration #9724.order=1on both; FLUX.2 previews are taken from the sample and sigma of the model evaluation, as on FLUX.1.Same seed on an RTX 4090, mean absolute pixel difference (0–255) against the Euler result of the same graph, before → after:
add_noise=false) vs single Euler runRelated Issues / Discussions
Closes #9792
Follows #9724, which added the Heun helper for FLUX.1.
QA Instructions
Automated:
uv run --no-sync pytest tests/app/invocations/test_denoise_noise_inputs.py tests/app/invocations/test_flux2_denoise_wiring.py tests/app/invocations/test_flux2_denoise_img2img_normalization.py tests/app/invocations/test_flux2_working_memory.py tests/app/invocations/test_z_image_schedule_shift.py tests/app/invocations/test_z_image_denoise_working_memory.py tests/app/invocations/test_z_image_denoise_schedulers.py tests/app/invocations/test_upscale_index_clipped_starts.py tests/app/invocations/test_module_compatibility.py tests/backend/flux tests/backend/flux2 tests/backend/architectures/test_scheduler_set.py→ 525 passed, 4 xfailed (the known LCM preblend cases).main: terminal sigma, Heun schedule and progress, previews from the evaluation sample, shift-before-clip on txt2img, Z-Image Heun schedule/img2img start/progress, and a node built with a scheduler that shifts again (shift=3.0).uv tool run ruff@0.11.2 check/format --checkon the changed files: clean.pnpm lintpassed;vitest run src/features/upscale97 passed;vitest run --config vitest.browser.config.mts src/app/UpscaleCapabilityGate.browser.test.tsxpassed (fails when Z-Image's list is reverted to Euler only).Manual (RTX 4090, FLUX.2 Klein 4B fp8 and Z-Image Turbo int8, 1024², 8 steps, seed 42): the graphs in the table above, run against
mainand this branch on the same data root. In the Upscale widget (Vite dev server against this branch), Z-Image's Scheduler field lists "Euler" and "Heun (2nd order)"; a 2× Z-Image upscale with Heun ran end to end and matched the Euler upscale visually.Not run: the full Python suite and
pnpm check:release; no GPU check on AMD/ROCm or macOS.Review
Resolved findings:
Remaining limitations:
timesteps[0]).timestep=int(t_curr), i.e. 0, in its progress state (pre-existing).shift=0(allowed by the field'sge=0) gives Heun evaluation sigmas of 0 and NaN latents; Euler stays finite. Tightening the field would be a schema change that could reject saved workflows, so it is left as is.Compatibility / Rollout
denoising_end < 1, which now returns latents at sigma1 − denoising_endon the shifted schedule instead of a finished image. Generate, Canvas and Upscale always senddenoising_end = 1; only workflow-editor graphs use it.invokeai.backend.flux2.denoise.denoise()no longer takesmu, andFLUX2_TXT2IMG_SCHEDULER_KWARGSis gone frominvokeai.backend.flux2.sampling_utils. Neither is part ofinvokeai.invocation_apiand nothing in the repo uses them; out-of-tree code that forked_run_diffusionfails with aTypeError/ImportErrorinstead of shifting twice.scheduler.step(...)line ininvokeai/backend/flux2/denoise.pyand the removedtest_flux2_lcm_scheduler_setup_passes_mu; whichever merges second keepsgenerator=generatoron the step call and drops that test. There is no semantic conflict.Checklist
What's Newcopy (if doing a release after this PR)Need help on this PR? Tag
@codesmith-botwith what you need.