diff --git a/docs/src/content/docs/Users Guide/02.Projects/introduction.mdx b/docs/src/content/docs/Users Guide/02.Projects/introduction.mdx index 36ac9747983..ed5b05f6d33 100644 --- a/docs/src/content/docs/Users Guide/02.Projects/introduction.mdx +++ b/docs/src/content/docs/Users Guide/02.Projects/introduction.mdx @@ -29,7 +29,7 @@ Invoke saves projects automatically on the server, so there is no Save button to | Panel layout and floating widgets | Saved layout presets | | Prompt history, and the Invoke routing (source and destination) | Preferences, theme and hotkeys | | Widget settings, such as gallery filters | Boards in the Library | -| Project settings: **Use CPU noise**, **Antialias progress images** and the Preview's progress image toggle | Generate recipes | +| Project settings: **Use CPU noise**, the **NSFW checker** and **Invisible watermark** output filters, **Antialias progress images** and the Preview's progress image toggle | Generate recipes | | The project's boards and the images and videos on them | | In **Settings**, a badge next to each option says where it is stored: **Saved with this project**, **Saved with this widget in this project**, or **Saved in your preferences** for options that follow you across projects. diff --git a/docs/src/content/docs/Users Guide/03.Models/Local Models/flux-1.mdx b/docs/src/content/docs/Users Guide/03.Models/Local Models/flux-1.mdx index 14eea2ba68a..4a6e54db6f1 100644 --- a/docs/src/content/docs/Users Guide/03.Models/Local Models/flux-1.mdx +++ b/docs/src/content/docs/Users Guide/03.Models/Local Models/flux-1.mdx @@ -93,6 +93,10 @@ Selecting a FLUX.1 model applies these defaults: - **Scheduler** — Euler, Heun (2nd order; better quality at about twice the time per step) or LCM (for few steps). - **Size** — width and height must be multiples of 16. +- **DyPE** — Dynamic Position Extrapolation for sizes above the 1024px training resolution: *Auto* turns it on + above 1536px, *Area* by image area, *4K optimized* is a fixed tuning, and *Manual* exposes the magnitude + (**λs**, 0–8) and decay speed (**λt**; about 2 for 4K, 1 around 2K–3K, 0.5 just above native). Off by + default; recorded in metadata and restored by Recall All. - **Negative prompt** — FLUX.1 has no negative prompt in the Generate tab or on the Canvas. Text-to-image, image-to-image, inpainting and outpainting all work with dev and schnell models on the @@ -159,5 +163,4 @@ Decode](/users-guide/image-generation/advanced-topics/pid-decode/). | **FLUX Redux**, **FLUX IP-Adapter**, **FLUX ControlNet**, **Control LoRA - FLUX**, **FLUX Fill Conditioning** | Conditioning for the features above | In workflows, **FLUX Denoise** also exposes **CFG Scale** with a negative conditioning input (CFG is off at -the default of 1.0), and a **DyPE** preset for resolutions above the model's native size: *Auto -(>1536px)*, *Area (auto)*, *4K Optimized* or *Manual*. +the default of 1.0), and the same **DyPE** preset, scale and exponent as the Generate panel. diff --git a/docs/src/content/docs/Users Guide/03.Models/Local Models/krea-2.mdx b/docs/src/content/docs/Users Guide/03.Models/Local Models/krea-2.mdx index d87a444e5cd..aae0692f430 100644 --- a/docs/src/content/docs/Users Guide/03.Models/Local Models/krea-2.mdx +++ b/docs/src/content/docs/Users Guide/03.Models/Local Models/krea-2.mdx @@ -132,20 +132,20 @@ Krea-2 can transfer the *look* of a reference image — palette, texture, render driving the content. There is **no adapter model and no LoRA**: the reference's attention keys and values are spliced into the target's, so it works with any Krea-2 checkpoint out of the box. -On the canvas, add a **Reference Image** while a Krea-2 model is selected, pick an image, and set **Style -Strength**. In the workflow editor the same thing is the **Style Reference - Krea-2** node, feeding the -**Style Reference** input of **Denoise - Krea-2**. +In the Generate panel, add a **Reference Image** while a Krea-2 model is selected, pick an image (cropping +it if you like), and set **Style strength**. The reference applies to Generate and to every Canvas mode. In +the workflow editor the same thing is the **Style Reference - Krea-2** node, feeding the **Style Reference** +input of **Denoise - Krea-2**. -- **Style Strength** is the one knob you need. `1.0` is the recommended setting; the slider goes to `2.0` +- **Style strength** is the one knob you need. `1.0` is the recommended setting; the slider goes to `2.0` for a heavier effect. `0` disables the reference entirely and costs nothing. - The remaining node inputs (block range, key/value scaling, AdaIN strengths) are for tuning and should be - left at their defaults. Style Strength already modulates several of them. -- Style Strength is recorded in image metadata, so you can see what a given image was generated with. It has - no recall button of its own — the reference image itself is not part of the metadata. + left at their defaults. Style strength already modulates several of them. +- The reference image and its Style strength are recorded in image metadata, so **Recall All** and **Remix + Image** restore them along with the rest of the recipe. :::caution[One reference, matching size, roughly 2x runtime] -- **Exactly one** reference image is used. If several are enabled on the canvas, only the first is applied - and the others are flagged with a warning. +- **Exactly one** reference image is used; the Generate panel accepts a single Krea-2 reference. - The reference must be encoded at **the same width and height as the denoise node** — its image tokens are appended to the target's. On the canvas this is wired up for you. In the workflow editor, set the same width and height on both nodes, or the denoise node will refuse the reference. diff --git a/docs/src/content/docs/Users Guide/03.Models/Local Models/qwen-image.mdx b/docs/src/content/docs/Users Guide/03.Models/Local Models/qwen-image.mdx index 36f80a60b3a..a682e1011b5 100644 --- a/docs/src/content/docs/Users Guide/03.Models/Local Models/qwen-image.mdx +++ b/docs/src/content/docs/Users Guide/03.Models/Local Models/qwen-image.mdx @@ -101,9 +101,9 @@ generation to **4** or **8** steps. Use the one that matches your model variant, - **Steps:** 4 or 8, matching the LoRA - **CFG Scale:** 1 -The LoRAs are trained for a fixed schedule shift of `3`. In the workflow editor, set **Shift** on the -**Denoise - Qwen Image** node to `3.0` when using them; leave it empty for the base model's default -schedule. The Generate tab does not expose this setting. +The LoRAs are trained for a fixed schedule shift of `3`. Turn on the **Shift** override in the Generate +panel's Render section when using them (it starts at `3`), and leave it off for the base model's default +schedule. In the workflow editor the same thing is the **Shift** input of **Denoise - Qwen Image**. ## Image editing with reference images @@ -143,4 +143,7 @@ Decoder Qwen-Image (2K to 4K)* decoders (including an int8 build of the latter) | **Latents to Image - Qwen-Image + PiD (4x SR)** | PiD super-resolution decode | The **Prompt - Qwen Image** node also has a **Quantization** option (`none`, `int8`, `nf4`) that -quantizes the Qwen2.5-VL encoder on load to reduce its VRAM use. +quantizes the Qwen2.5-VL encoder on load to reduce its VRAM use. The Generate panel offers the same choice +as **Encoder quantization** under **Advanced**, applied to both prompt encoders. The saving has a cost: a +quantized encoder cannot stay in the model cache, so it is loaded from disk for every prompt it encodes, twice +per generation when CFG is above 1. Leave it at `none` if the bf16 encoder fits. diff --git a/docs/src/content/docs/Users Guide/03.Models/Local Models/z-image.mdx b/docs/src/content/docs/Users Guide/03.Models/Local Models/z-image.mdx index 956cf447a56..4e6b7632193 100644 --- a/docs/src/content/docs/Users Guide/03.Models/Local Models/z-image.mdx +++ b/docs/src/content/docs/Users Guide/03.Models/Local Models/z-image.mdx @@ -95,8 +95,9 @@ Selecting a Z-Image model applies these defaults: - **Prompt weighting**: Compel-style [prompt syntax](/users-guide/image-generation/advanced-prompt-syntax/#attention-weighting) does not apply to Z-Image. Write prompts in plain language. -In the workflow editor, **Denoise - Z-Image** also has a **Shift** input that overrides the timestep -shift. Leave it blank to have it calculated from the image size, which is the recommended setting. +- **Shift**: the Render section's **Shift** override fixes the timestep shift of the sampling schedule. + Leave the override off to have it calculated from the image size, which is the recommended setting; the + override starts at `3`. In the workflow editor the same thing is the **Shift** input of **Denoise - Z-Image**. ## ControlNet @@ -130,14 +131,16 @@ Collection - Z-Image**. ## Seed Variance Enhancer -Z-Image Turbo tends to produce similar images across different seeds. The **Seed Variance Enhancer - -Z-Image** node, available in the workflow editor only, adds reproducible, seed-based noise to the prompt -conditioning to increase variety: +Z-Image Turbo tends to produce similar images across different seeds. **Seed variance**, next to the seed +in the Generate panel's Render section, adds reproducible, seed-based noise to the prompt conditioning to +increase variety: -- **Strength**: `0` is off, `0.1` (the default) is subtle, `0.5` is strong. -- **Randomize Percent**: the share of embedding values that receive noise (default `50`). +- **Variance strength**: `0` is off, `0.1` (the default) is subtle, `0.5` is strong. +- **Randomized channels**: the share of embedding values that receive noise (default `50%`). -Connect it between **Prompt - Z-Image** and the denoise node's positive conditioning. +Both settings are recorded in image metadata and restored by Recall All. In the workflow editor the same +thing is the **Seed Variance Enhancer - Z-Image** node, connected between **Prompt - Z-Image** and the +denoise node's positive conditioning. ## PiD decoding diff --git a/docs/src/content/docs/Users Guide/04.Image Generation/generate-panel.mdx b/docs/src/content/docs/Users Guide/04.Image Generation/generate-panel.mdx index eeb01458102..c4e337fd7fd 100644 --- a/docs/src/content/docs/Users Guide/04.Image Generation/generate-panel.mdx +++ b/docs/src/content/docs/Users Guide/04.Image Generation/generate-panel.mdx @@ -45,6 +45,16 @@ The **Seed** sets the starting noise for the image. The same model, prompt, seed * **Seed mode** sets how seeds are chosen: **Random** picks a fresh starting seed each time you press Invoke, **Fixed** reuses the seed in the field, and **Increment** and **Decrement** use successive seeds, counting up or down by one. A line under the field shows the seeds the next batch will use. * **Recent seeds** shows thumbnails of recent images. Click one to reuse its seed; the mode switches to Fixed. +### Model-specific options + +Some families add their own controls to the Render section: + +* **DyPE** (FLUX.1) — position extrapolation for sizes above the 1024px training resolution. **Auto** turns it on above 1536px, **Area** by image area and **4K optimized** is a fixed tuning; **Manual** exposes the **scale (λs)** and **exponent (λt)**. +* **Shift** (Z-Image, Qwen Image) — an override for the sampling schedule's timestep shift, from 0.1 to 10. Left on auto, the model derives it from the image size; Qwen Image's Lightning LoRAs are trained for `3`. +* **Seed variance** (Krea-2, Z-Image) — perturbs the prompt conditioning so nearby seeds give more varied images; **Variance strength** and **Randomized channels** set how much. +* **Conditioning rebalance** (Krea-2) — see [Krea-2](/users-guide/models/local-models/krea-2/). +* **Low-noise guidance** (Wan) — guidance for the low-noise expert; left unset, it reuses the main guidance. + ### Negative prompt A negative prompt lists what you do *not* want in the image, such as *blurry, watermark, extra fingers*. Models that support one show a **+ Negative prompt…** button under the prompt; click it to open the box, and click **Turn off negative prompt** to hide it again without losing the text. @@ -60,8 +70,11 @@ The **Advanced** section appears for models with extra options, and its header c * **HiDiffusion** — improves structure at large sizes for SD 1.5 and SDXL; see [HiDiffusion](/users-guide/image-generation/advanced-topics/hidiffusion/). * **Color compensation** — SDXL only. * **PiD Decode** — decodes the image at up to four times its size; see [PiD Super-Resolution Decode](/users-guide/image-generation/advanced-topics/pid-decode/). +* **Encoder quantization** — Qwen Image only; quantizes the Qwen VL text encoder to 8-bit or 4-bit on load to save VRAM. A quantized encoder cannot stay in the model cache, so it is loaded from disk on every generation, twice when CFG is above 1. * **Seamless tiling** — makes the image tile along the X axis, the Y axis, or both. +Two output filters live in **Settings** under **Project** rather than in the panel, because they apply to every image the project's Generate panel and Canvas produce: the **NSFW checker** blurs images it flags as explicit, and the **Invisible watermark** stamps every saved image. Both are off by default. Upscale results, workflow runs and images from external models are not filtered. + ### Resetting to defaults Each model has recommended settings, which you can change in the Model Manager. **Reset all to model defaults**, the ↺ button in the Generate panel's title bar, applies them: size, steps, CFG or guidance, scheduler and VAE settings. Your prompts are left alone. The button is disabled when your settings already match the defaults. @@ -102,6 +115,7 @@ What a reference does, and what options it has, depends on the model family: | **FLUX.1** | An IP-Adapter, or a **FLUX Redux** model, which transfers the reference's content. | IP-Adapter options as above, or **Image influence** (Lowest to Highest) for Redux. | | **FLUX.1 Kontext** | Kontext edits or recombines the reference images according to your prompt. | None beyond the model. | | **FLUX.2, Qwen Image Edit** | The model reads the references directly and uses them as your prompt instructs, for example *put the jacket from the first image on the person in the second*. | None. | +| **Krea-2** | One reference's look — palette, texture, rendering — is transferred into the image while the prompt keeps driving the content; see [Krea-2](/users-guide/models/local-models/krea-2/#style-reference). | **Style strength** (0–2; 0 turns the reference off). | Reference images require an extra model for SD 1.5, SDXL and FLUX.1 (an IP-Adapter or FLUX Redux model). These are available as starter models in the Model Manager, and an installed IP-Adapter is selected automatically. diff --git a/invokeai/backend/architectures/defs/krea_2.py b/invokeai/backend/architectures/defs/krea_2.py index 631a17c3387..cb6074ef520 100644 --- a/invokeai/backend/architectures/defs/krea_2.py +++ b/invokeai/backend/architectures/defs/krea_2.py @@ -34,6 +34,9 @@ dimension_grid=16, guidance_label="CFG", scheduler_set="flow", + # Training-free style reference: shared-KV attention splices one reference's keys and + # values into the target, so there is no adapter model and exactly one image. + max_reference_images=1, supports_regional_guidance=True, ), VaeFacet( diff --git a/invokeai/backend/image_util/invisible_watermark.py b/invokeai/backend/image_util/invisible_watermark.py index 95c483848cc..a7af5174391 100644 --- a/invokeai/backend/image_util/invisible_watermark.py +++ b/invokeai/backend/image_util/invisible_watermark.py @@ -24,7 +24,11 @@ def add_watermark(cls, image: Image.Image, watermark_text: str) -> Image.Image: encoder = WatermarkEncoder() encoder.set_watermark("bytes", watermark_text.encode("utf-8")) bgr_encoded = encoder.encode(bgr, "dwtDct") - return Image.fromarray(cv2.cvtColor(bgr_encoded, cv2.COLOR_BGR2RGB)).convert("RGBA") + watermarked = Image.fromarray(cv2.cvtColor(bgr_encoded, cv2.COLOR_BGR2RGB)) + # The watermark lives in the color channels. Carry the source alpha over, so a masked Canvas result keeps + # its transparent surround instead of becoming an opaque rectangle; an image without alpha stays opaque. + watermarked.putalpha(image.convert("RGBA").getchannel("A")) + return watermarked @classmethod def decode_watermark(cls, image: Image.Image, length: int = 8) -> str: diff --git a/invokeai/frontend/webv2/performance/browser-baseline.json b/invokeai/frontend/webv2/performance/browser-baseline.json index 9e45cb28593..e04bb3bd8c8 100644 --- a/invokeai/frontend/webv2/performance/browser-baseline.json +++ b/invokeai/frontend/webv2/performance/browser-baseline.json @@ -1,6 +1,6 @@ { "browserExecutable": "/home/toast/.cache/ms-playwright/chromium-1243/chrome-linux64/chrome", - "capturedAt": "2026-10-09", + "capturedAt": "2026-10-15", "routes": [ { "activatedResourceBaseline": { @@ -29,12 +29,12 @@ "cssRawBytes": 2159, "fontRawBytes": 23664, "imageRawBytes": 0, - "largestAssetRawBytes": 749029, - "otherRawBytes": 200480, + "largestAssetRawBytes": 752421, + "otherRawBytes": 205544, "requestCount": 30, - "scriptRawBytes": 1842561, + "scriptRawBytes": 1847804, "scriptRequestCount": 26, - "totalRawBytes": 2068864 + "totalRawBytes": 2079171 }, "routeReadyMedianMs": 275.80000000447035, "scriptSourceOwnerSet": "launchpad-static", @@ -67,12 +67,12 @@ "cssRawBytes": 2159, "fontRawBytes": 23664, "imageRawBytes": 0, - "largestAssetRawBytes": 749029, - "otherRawBytes": 200480, + "largestAssetRawBytes": 752421, + "otherRawBytes": 205544, "requestCount": 30, - "scriptRawBytes": 1842561, + "scriptRawBytes": 1847804, "scriptRequestCount": 26, - "totalRawBytes": 2068864 + "totalRawBytes": 2079171 }, "routeReadyMedianMs": 277, "scriptSourceOwnerSet": "launchpad-static", @@ -105,12 +105,12 @@ "cssRawBytes": 2159, "fontRawBytes": 23664, "imageRawBytes": 44398, - "largestAssetRawBytes": 749029, - "otherRawBytes": 200480, + "largestAssetRawBytes": 752421, + "otherRawBytes": 205544, "requestCount": 88, - "scriptRawBytes": 4063609, + "scriptRawBytes": 4106504, "scriptRequestCount": 83, - "totalRawBytes": 4334310 + "totalRawBytes": 4382269 }, "routeReadyMedianMs": 430.19999999552965, "scriptSourceOwnerSet": "editor-minimal-static", @@ -143,12 +143,12 @@ "cssRawBytes": 2159, "fontRawBytes": 23664, "imageRawBytes": 44398, - "largestAssetRawBytes": 749029, - "otherRawBytes": 200480, + "largestAssetRawBytes": 752421, + "otherRawBytes": 205544, "requestCount": 88, - "scriptRawBytes": 4063609, + "scriptRawBytes": 4106504, "scriptRequestCount": 83, - "totalRawBytes": 4334310 + "totalRawBytes": 4382269 }, "routeReadyMedianMs": 466.3999999985099, "scriptSourceOwnerSet": "editor-minimal-static", @@ -159,12 +159,12 @@ "cssRawBytes": 0, "fontRawBytes": 0, "imageRawBytes": 0, - "largestAssetRawBytes": 305162, + "largestAssetRawBytes": 305459, "otherRawBytes": 0, "requestCount": 9, - "scriptRawBytes": 514518, + "scriptRawBytes": 514291, "scriptRequestCount": 9, - "totalRawBytes": 514518 + "totalRawBytes": 514291 }, "domContentLoadedMedianMs": 125.5, "id": "editor-canvas", @@ -181,12 +181,12 @@ "cssRawBytes": 2159, "fontRawBytes": 23664, "imageRawBytes": 44398, - "largestAssetRawBytes": 749029, - "otherRawBytes": 200480, + "largestAssetRawBytes": 752421, + "otherRawBytes": 205544, "requestCount": 97, - "scriptRawBytes": 4578127, + "scriptRawBytes": 4620795, "scriptRequestCount": 92, - "totalRawBytes": 4848828 + "totalRawBytes": 4896560 }, "routeReadyMedianMs": 1153.7999999970198, "scriptSourceOwnerSet": "editor-canvas-static", @@ -197,12 +197,12 @@ "cssRawBytes": 0, "fontRawBytes": 0, "imageRawBytes": 0, - "largestAssetRawBytes": 305162, + "largestAssetRawBytes": 305459, "otherRawBytes": 0, "requestCount": 9, - "scriptRawBytes": 514518, + "scriptRawBytes": 514291, "scriptRequestCount": 9, - "totalRawBytes": 514518 + "totalRawBytes": 514291 }, "domContentLoadedMedianMs": 126.59999999403954, "id": "editor-canvas", @@ -219,12 +219,12 @@ "cssRawBytes": 2159, "fontRawBytes": 23664, "imageRawBytes": 44398, - "largestAssetRawBytes": 749029, - "otherRawBytes": 200480, + "largestAssetRawBytes": 752421, + "otherRawBytes": 205544, "requestCount": 97, - "scriptRawBytes": 4578127, + "scriptRawBytes": 4620795, "scriptRequestCount": 92, - "totalRawBytes": 4848828 + "totalRawBytes": 4896560 }, "routeReadyMedianMs": 1147.1000000014901, "scriptSourceOwnerSet": "editor-canvas-static", @@ -235,12 +235,12 @@ "cssRawBytes": 15413, "fontRawBytes": 0, "imageRawBytes": 0, - "largestAssetRawBytes": 187094, + "largestAssetRawBytes": 187100, "otherRawBytes": 0, "requestCount": 7, - "scriptRawBytes": 420919, + "scriptRawBytes": 420925, "scriptRequestCount": 6, - "totalRawBytes": 436332 + "totalRawBytes": 436338 }, "domContentLoadedMedianMs": 125, "id": "editor-workflow", @@ -257,12 +257,12 @@ "cssRawBytes": 17572, "fontRawBytes": 23664, "imageRawBytes": 44398, - "largestAssetRawBytes": 749029, - "otherRawBytes": 200480, + "largestAssetRawBytes": 752421, + "otherRawBytes": 205544, "requestCount": 95, - "scriptRawBytes": 4484528, + "scriptRawBytes": 4527429, "scriptRequestCount": 89, - "totalRawBytes": 4770642 + "totalRawBytes": 4818607 }, "routeReadyMedianMs": 1171.6000000014901, "scriptSourceOwnerSet": "editor-workflow-static", @@ -273,12 +273,12 @@ "cssRawBytes": 15413, "fontRawBytes": 0, "imageRawBytes": 0, - "largestAssetRawBytes": 187094, + "largestAssetRawBytes": 187100, "otherRawBytes": 0, "requestCount": 7, - "scriptRawBytes": 420919, + "scriptRawBytes": 420925, "scriptRequestCount": 6, - "totalRawBytes": 436332 + "totalRawBytes": 436338 }, "domContentLoadedMedianMs": 123.70000000298023, "id": "editor-workflow", @@ -295,12 +295,12 @@ "cssRawBytes": 17572, "fontRawBytes": 23664, "imageRawBytes": 44398, - "largestAssetRawBytes": 749029, - "otherRawBytes": 200480, + "largestAssetRawBytes": 752421, + "otherRawBytes": 205544, "requestCount": 95, - "scriptRawBytes": 4484528, + "scriptRawBytes": 4527429, "scriptRequestCount": 89, - "totalRawBytes": 4770642 + "totalRawBytes": 4818607 }, "routeReadyMedianMs": 1192.8999999985099, "scriptSourceOwnerSet": "editor-workflow-static", @@ -333,12 +333,12 @@ "cssRawBytes": 2159, "fontRawBytes": 23664, "imageRawBytes": 44398, - "largestAssetRawBytes": 749029, - "otherRawBytes": 200480, + "largestAssetRawBytes": 752421, + "otherRawBytes": 205544, "requestCount": 88, - "scriptRawBytes": 4063609, + "scriptRawBytes": 4106504, "scriptRequestCount": 83, - "totalRawBytes": 4334310 + "totalRawBytes": 4382269 }, "routeReadyMedianMs": 1431.3999999985099, "scriptSourceOwnerSet": "editor-minimal-static", @@ -371,12 +371,12 @@ "cssRawBytes": 2159, "fontRawBytes": 23664, "imageRawBytes": 44398, - "largestAssetRawBytes": 749029, - "otherRawBytes": 200480, + "largestAssetRawBytes": 752421, + "otherRawBytes": 205544, "requestCount": 88, - "scriptRawBytes": 4063609, + "scriptRawBytes": 4106504, "scriptRequestCount": 83, - "totalRawBytes": 4334310 + "totalRawBytes": 4382269 }, "routeReadyMedianMs": 1396.3999999985099, "scriptSourceOwnerSet": "editor-minimal-static", diff --git a/invokeai/frontend/webv2/public/locales/en.json b/invokeai/frontend/webv2/public/locales/en.json index 2274602a242..b3acd165de8 100644 --- a/invokeai/frontend/webv2/public/locales/en.json +++ b/invokeai/frontend/webv2/public/locales/en.json @@ -859,6 +859,33 @@ "heading": "Height", "paragraphs": ["Height of the generated image. Must be a multiple of 8."] }, + "dypePreset": { + "heading": "DyPE (High-Resolution)", + "paragraphs": [ + "Dynamic Position Extrapolation improves FLUX.1 generation quality at sizes above the 1024px training resolution.", + "Off: standard generation. Auto: enables above 1536px. Area: enables by image area. 4K: tuned for 4K output. Manual: set the magnitude (λs) and decay speed (λt) yourself; 2.0 suits 4K and above, 1.0 around 2K–3K, 0.5 just above 1024px." + ] + }, + "encoderQuantization": { + "heading": "Encoder Quantization", + "paragraphs": [ + "Quantizes the Qwen2.5-VL text encoder on load to reduce VRAM use. 4-bit (nf4) saves the most memory; 8-bit (int8) is a middle ground. None keeps the bf16 weights.", + "A quantized encoder cannot stay in the model cache, so it is loaded from disk on every generation, twice when CFG is above 1. None keeps the cached encoder and loads it once." + ] + }, + "styleStrength": { + "heading": "Style Strength", + "paragraphs": [ + "How strongly the reference's look is transferred. 1.0 is the recommended setting; 2.0 is a heavier effect and 0 turns the reference off." + ] + }, + "timestepShift": { + "heading": "Timestep Shift", + "paragraphs": [ + "Overrides the sigma-schedule shift. Left on auto the model derives it from the image size, which is recommended.", + "Qwen Image Lightning LoRAs are trained for a fixed shift of 3. Lower values spend more steps at low noise; higher values at high noise." + ] + }, "imageInfluence": { "heading": "Denoising Strength", "paragraphs": [ @@ -1938,6 +1965,14 @@ "label": "Use CPU noise", "description": "Use CPU noise generation for deterministic legacy-compatible outputs." }, + "useNsfwChecker": { + "label": "NSFW checker", + "description": "Blur Generate and Canvas results the checker flags as explicit before they are saved." + }, + "useWatermark": { + "label": "Invisible watermark", + "description": "Stamp an invisible watermark into every Generate and Canvas result." + }, "confirmImageDeletion": { "label": "Confirm image deletion", "description": "Ask for confirmation before permanently deleting images." @@ -2463,10 +2498,27 @@ "krea2RebalanceWeights": "Per-tap weights", "krea2RebalanceWeightsHelp": "{{count}} comma-separated numbers, one per tap.", "krea2RebalanceWeightsInvalid": "Enter exactly {{count}} comma-separated numbers.", - "krea2SeedVariance": "Seed variance", - "krea2SeedVarianceHelp": "Perturbs conditioning so nearby seeds diverge more.", - "krea2SeedVarianceRandomize": "Randomized channels", - "krea2SeedVarianceStrength": "Variance strength", + "dypePreset": "DyPE", + "dypePresetOff": "Off", + "dypePresetManual": "Manual", + "dypePresetAuto": "Auto (above 1536px)", + "dypePresetArea": "Area (auto)", + "dypePreset4k": "4K optimized", + "dypeScale": "DyPE scale (λs)", + "dypeExponent": "DyPE exponent (λt)", + "encoderQuantization": "Encoder quantization", + "encoderQuantizationHelp": "Quantizes the Qwen VL encoder on load to save VRAM, but reloads it from disk for every generation.", + "encoderQuantizationNone": "None (bf16)", + "encoderQuantizationInt8": "8-bit (int8)", + "encoderQuantizationNf4": "4-bit (nf4)", + "styleStrength": "Style strength", + "timestepShift": "Shift", + "timestepShiftHelp": "Auto derives the schedule shift from the image size.", + "qwenImageShiftHelp": "Auto derives the schedule shift from the image size; Lightning LoRAs expect 3.", + "seedVariance": "Seed variance", + "seedVarianceHelp": "Perturbs conditioning so nearby seeds diverge more.", + "seedVarianceRandomize": "Randomized channels", + "seedVarianceStrength": "Variance strength", "llmTaskGenerating": "Generating", "llmTaskLoadingModel": "Loading model", "llmTaskProgress": "LLM task progress", diff --git a/invokeai/frontend/webv2/src/features/generation/core/__fixtures__/architectureCapabilities.json b/invokeai/frontend/webv2/src/features/generation/core/__fixtures__/architectureCapabilities.json index 01f6efbee4f..005e93e4968 100644 --- a/invokeai/frontend/webv2/src/features/generation/core/__fixtures__/architectureCapabilities.json +++ b/invokeai/frontend/webv2/src/features/generation/core/__fixtures__/architectureCapabilities.json @@ -563,7 +563,7 @@ "scheduler_set": "flow", "scheduler_applies_to_graph": false, "control_kinds": [], - "max_reference_images": 0, + "max_reference_images": 1, "reference_images_require_variant": null, "supports_regional_guidance": true, "regional_negative": false, @@ -615,7 +615,7 @@ "scheduler_set": "flow", "scheduler_applies_to_graph": false, "control_kinds": [], - "max_reference_images": 0, + "max_reference_images": 1, "reference_images_require_variant": null, "supports_regional_guidance": true, "regional_negative": false, diff --git a/invokeai/frontend/webv2/src/features/generation/core/__snapshots__/animaLLLiteCanvasGraphs.json b/invokeai/frontend/webv2/src/features/generation/core/__snapshots__/animaLLLiteCanvasGraphs.json index 8315b8ad6bd..d5a5deb9d7f 100644 --- a/invokeai/frontend/webv2/src/features/generation/core/__snapshots__/animaLLLiteCanvasGraphs.json +++ b/invokeai/frontend/webv2/src/features/generation/core/__snapshots__/animaLLLiteCanvasGraphs.json @@ -406,12 +406,6 @@ "image_name": "canvas-composite.png" } }, - "canvas_l2i": { - "is_intermediate": true, - "use_cache": false, - "id": "canvas_l2i", - "type": "anima_l2i" - }, "create_gradient_mask": { "is_intermediate": true, "use_cache": true, @@ -470,14 +464,20 @@ "id": "expand_mask", "type": "expand_mask_with_fade" }, + "canvas_l2i": { + "is_intermediate": true, + "use_cache": false, + "id": "canvas_l2i", + "type": "anima_l2i" + }, "canvas_output": { "is_intermediate": true, "use_cache": false, - "id": "canvas_output", "layer_base": { "image_name": "canvas-composite.png" }, - "type": "invokeai_img_blend" + "type": "invokeai_img_blend", + "id": "canvas_output" } } }, @@ -888,12 +888,6 @@ "image_name": "canvas-composite.png" } }, - "canvas_l2i": { - "is_intermediate": true, - "use_cache": false, - "id": "canvas_l2i", - "type": "anima_l2i" - }, "create_gradient_mask": { "is_intermediate": true, "use_cache": true, @@ -917,14 +911,20 @@ "id": "expand_mask", "type": "expand_mask_with_fade" }, + "canvas_l2i": { + "is_intermediate": true, + "use_cache": false, + "id": "canvas_l2i", + "type": "anima_l2i" + }, "canvas_output": { "is_intermediate": true, "use_cache": false, - "id": "canvas_output", "layer_base": { "image_name": "canvas-composite.png" }, - "type": "invokeai_img_blend" + "type": "invokeai_img_blend", + "id": "canvas_output" }, "anima_lllite_sketch-layer": { "is_intermediate": true, @@ -1481,12 +1481,6 @@ "id": "canvas_i2l", "type": "anima_i2l" }, - "canvas_l2i": { - "is_intermediate": true, - "use_cache": false, - "id": "canvas_l2i", - "type": "anima_l2i" - }, "create_gradient_mask": { "is_intermediate": true, "use_cache": true, @@ -1536,14 +1530,20 @@ "id": "expand_mask", "type": "expand_mask_with_fade" }, + "canvas_l2i": { + "is_intermediate": true, + "use_cache": false, + "id": "canvas_l2i", + "type": "anima_l2i" + }, "canvas_output": { "is_intermediate": true, "use_cache": false, - "id": "canvas_output", "layer_base": { "image_name": "canvas-composite.png" }, - "type": "invokeai_img_blend" + "type": "invokeai_img_blend", + "id": "canvas_output" }, "anima_lllite_sketch-layer": { "is_intermediate": true, diff --git a/invokeai/frontend/webv2/src/features/generation/core/__snapshots__/generateGraphNodeTypes.json b/invokeai/frontend/webv2/src/features/generation/core/__snapshots__/generateGraphNodeTypes.json index c82de6d94d3..61bcecb9982 100644 --- a/invokeai/frontend/webv2/src/features/generation/core/__snapshots__/generateGraphNodeTypes.json +++ b/invokeai/frontend/webv2/src/features/generation/core/__snapshots__/generateGraphNodeTypes.json @@ -12,6 +12,8 @@ "compel", "core_metadata", "denoise_latents", + "img_nsfw", + "img_watermark", "integer", "l2i", "main_model_loader", @@ -30,6 +32,8 @@ "compel", "core_metadata", "denoise_latents", + "img_nsfw", + "img_watermark", "integer", "l2i", "main_model_loader", @@ -46,6 +50,8 @@ "collect", "core_metadata", "denoise_latents", + "img_nsfw", + "img_watermark", "integer", "l2i", "noise", @@ -61,6 +67,8 @@ }, "nodeTypes": [ "core_metadata", + "img_nsfw", + "img_watermark", "integer", "sd3_denoise", "sd3_l2i", @@ -89,6 +97,8 @@ "flux_model_loader", "flux_text_encoder", "flux_vae_decode", + "img_nsfw", + "img_watermark", "integer", "string" ] @@ -114,6 +124,8 @@ "flux2_klein_model_loader", "flux2_klein_text_encoder", "flux2_vae_decode", + "img_nsfw", + "img_watermark", "integer", "string" ] @@ -129,6 +141,8 @@ "cogview4_model_loader", "cogview4_text_encoder", "core_metadata", + "img_nsfw", + "img_watermark", "integer", "string" ] @@ -147,6 +161,8 @@ "ernie_image_model_loader", "ernie_image_text_encoder", "ernie_image_vae_decode", + "img_nsfw", + "img_watermark", "integer", "string" ] @@ -161,6 +177,8 @@ }, "nodeTypes": [ "core_metadata", + "img_nsfw", + "img_watermark", "integer", "qwen_image_denoise", "qwen_image_l2i", @@ -180,11 +198,14 @@ "nodeTypes": [ "collect", "core_metadata", + "img_nsfw", + "img_watermark", "integer", "string", "z_image_denoise", "z_image_l2i", "z_image_model_loader", + "z_image_seed_variance_enhancer", "z_image_text_encoder" ] }, @@ -204,6 +225,8 @@ "ideogram4_l2i", "ideogram4_model_loader", "ideogram4_text_encoder", + "img_nsfw", + "img_watermark", "integer", "string" ] @@ -219,9 +242,12 @@ "nodeTypes": [ "collect", "core_metadata", + "img_nsfw", + "img_watermark", "integer", "krea2_denoise", "krea2_model_loader", + "krea2_style_reference", "krea2_text_encoder", "qwen_image_l2i", "string" @@ -237,6 +263,8 @@ }, "nodeTypes": [ "core_metadata", + "img_nsfw", + "img_watermark", "integer", "qwen_image_2_1_denoise", "qwen_image_2_1_l2i", @@ -255,6 +283,8 @@ }, "nodeTypes": [ "core_metadata", + "img_nsfw", + "img_watermark", "integer", "string", "wan_denoise", @@ -282,6 +312,8 @@ "anima_text_encoder", "collect", "core_metadata", + "img_nsfw", + "img_watermark", "integer", "string" ] @@ -338,13 +370,16 @@ ], "literalInputs": { "is_intermediate": [ - false + false, + true ], "use_cache": [ false ] }, - "outputs": [] + "outputs": [ + "image" + ] }, "anima_model_loader": { "inputs": [], @@ -443,13 +478,16 @@ ], "literalInputs": { "is_intermediate": [ - false + false, + true ], "use_cache": [ false ] }, - "outputs": [] + "outputs": [ + "image" + ] }, "cogview4_model_loader": { "inputs": [], @@ -540,6 +578,16 @@ "clip_skip": [ 0 ], + "dype_exponent": [ + 1 + ], + "dype_preset": [ + "manual", + "off" + ], + "dype_scale": [ + 3 + ], "generation_mode": [ "anima_txt2img", "cogview4_txt2img", @@ -605,9 +653,20 @@ "krea2_seed_variance_strength": [ 0.1 ], + "krea2_style_strength": [ + 1 + ], "mistral_encoder": [], "model": [], + "qwen_image_quantization": [ + "nf4", + "none" + ], "qwen_image_qwen_vl_encoder": [], + "qwen_image_shift": [ + "auto", + 3 + ], "qwen_image_vae": [], "qwen3_5_encoder": [], "qwen3_encoder": [], @@ -616,6 +675,7 @@ "rand_device": [ "cpu" ], + "ref_images": [], "scheduler": [ "euler_a", "euler" @@ -648,6 +708,21 @@ 1024, 512, 768 + ], + "z_image_seed_variance_enabled": [ + false, + true + ], + "z_image_seed_variance_randomize_percent": [ + 50 + ], + "z_image_seed_variance_strength": [ + 0.1, + 0.2 + ], + "z_image_shift": [ + "auto", + 3 ] }, "outputs": [ @@ -787,13 +862,16 @@ ], "literalInputs": { "is_intermediate": [ - false + false, + true ], "use_cache": [ false ] }, - "outputs": [] + "outputs": [ + "image" + ] }, "flux_denoise": { "inputs": [ @@ -809,6 +887,15 @@ "denoising_start": [ 0 ], + "dype_exponent": [ + 1 + ], + "dype_preset": [ + "manual" + ], + "dype_scale": [ + 3 + ], "guidance": [ 3.5 ], @@ -884,13 +971,16 @@ ], "literalInputs": { "is_intermediate": [ - false + false, + true ], "use_cache": [ false ] }, - "outputs": [] + "outputs": [ + "image" + ] }, "flux2_denoise": { "inputs": [ @@ -1021,13 +1111,16 @@ ], "literalInputs": { "is_intermediate": [ - false + false, + true ], "use_cache": [ false ] }, - "outputs": [] + "outputs": [ + "image" + ] }, "ideogram4_caption_builder": { "inputs": [ @@ -1082,13 +1175,16 @@ ], "literalInputs": { "is_intermediate": [ - false + false, + true ], "use_cache": [ false ] }, - "outputs": [] + "outputs": [ + "image" + ] }, "ideogram4_model_loader": { "inputs": [], @@ -1128,6 +1224,37 @@ "conditioning" ] }, + "img_nsfw": { + "inputs": [ + "image" + ], + "literalInputs": { + "is_intermediate": [ + true + ], + "use_cache": [ + false + ] + }, + "outputs": [ + "image" + ] + }, + "img_watermark": { + "inputs": [ + "image", + "metadata" + ], + "literalInputs": { + "is_intermediate": [ + false + ], + "use_cache": [ + false + ] + }, + "outputs": [] + }, "integer": { "inputs": [], "literalInputs": { @@ -1146,6 +1273,7 @@ "inputs": [ "positive_conditioning", "seed", + "style_reference", "transformer" ], "literalInputs": { @@ -1191,6 +1319,32 @@ "vae" ] }, + "krea2_style_reference": { + "inputs": [ + "vae" + ], + "literalInputs": { + "height": [ + 1024 + ], + "image": [], + "is_intermediate": [ + true + ], + "style_strength": [ + 1 + ], + "use_cache": [ + true + ], + "width": [ + 1024 + ] + }, + "outputs": [ + "style_reference" + ] + }, "krea2_text_encoder": { "inputs": [ "prompt", @@ -1222,13 +1376,16 @@ true ], "is_intermediate": [ - false + false, + true ], "use_cache": [ false ] }, - "outputs": [] + "outputs": [ + "image" + ] }, "main_model_loader": { "inputs": [], @@ -1314,13 +1471,16 @@ ], "literalInputs": { "is_intermediate": [ - false + false, + true ], "use_cache": [ false ] }, - "outputs": [] + "outputs": [ + "image" + ] }, "qwen_image_2_1_model_loader": { "inputs": [], @@ -1375,6 +1535,9 @@ "is_intermediate": [ true ], + "shift": [ + 3 + ], "steps": [ 40 ], @@ -1397,13 +1560,16 @@ ], "literalInputs": { "is_intermediate": [ - false + false, + true ], "use_cache": [ false ] }, - "outputs": [] + "outputs": [ + "image" + ] }, "qwen_image_model_loader": { "inputs": [], @@ -1433,6 +1599,9 @@ "is_intermediate": [ true ], + "quantization": [ + "nf4" + ], "use_cache": [ true ] @@ -1486,13 +1655,16 @@ ], "literalInputs": { "is_intermediate": [ - false + false, + true ], "use_cache": [ false ] }, - "outputs": [] + "outputs": [ + "image" + ] }, "sd3_model_loader": { "inputs": [], @@ -1622,13 +1794,16 @@ ], "literalInputs": { "is_intermediate": [ - false + false, + true ], "use_cache": [ false ] }, - "outputs": [] + "outputs": [ + "image" + ] }, "wan_model_loader": { "inputs": [], @@ -1693,6 +1868,9 @@ "scheduler": [ "euler" ], + "shift": [ + 3 + ], "steps": [ 9 ], @@ -1715,13 +1893,16 @@ ], "literalInputs": { "is_intermediate": [ - false + false, + true ], "use_cache": [ false ] }, - "outputs": [] + "outputs": [ + "image" + ] }, "z_image_model_loader": { "inputs": [], @@ -1743,6 +1924,29 @@ "vae" ] }, + "z_image_seed_variance_enhancer": { + "inputs": [ + "conditioning", + "seed" + ], + "literalInputs": { + "is_intermediate": [ + true + ], + "randomize_percent": [ + 50 + ], + "strength": [ + 0.2 + ], + "use_cache": [ + true + ] + }, + "outputs": [ + "conditioning" + ] + }, "z_image_text_encoder": { "inputs": [ "prompt", diff --git a/invokeai/frontend/webv2/src/features/generation/core/baseGenerationPolicies.test.ts b/invokeai/frontend/webv2/src/features/generation/core/baseGenerationPolicies.test.ts index 01b7a59d70d..6d7e77b5766 100644 --- a/invokeai/frontend/webv2/src/features/generation/core/baseGenerationPolicies.test.ts +++ b/invokeai/frontend/webv2/src/features/generation/core/baseGenerationPolicies.test.ts @@ -22,6 +22,7 @@ import { getComponentSectionPolicy, getAutoFlux2ComponentSourceModel, getDefaultGenerateSettings, + getDefaultReferenceImageConfig, getGenerateModelSelectionResult, getGenerationDimensions, getGenerationModelAvailabilityReasons, @@ -980,6 +981,116 @@ describe('component policies', () => { expect(result.clearedLabels).not.toContain('Reference Images'); }); + it('routes Krea-2 to its single style reference and re-targets other kinds to it', () => { + // A Diffusers bundle needs no standalone components, which keeps the reasons to the reference check. + const krea2 = createModel('krea-2', { format: 'diffusers' }); + + expect(isReferenceImageSupported(krea2)).toBe(true); + expect(getMaxReferenceImages(krea2)).toBe(1); + expect(getDefaultReferenceImageConfig(krea2, [sdxlIpAdapter], referenceImage)).toEqual({ + image: referenceImage, + styleStrength: 1, + type: 'krea2_reference_image', + }); + + const fromSdxl = getGenerateModelSelectionResult({ + currentValues: createSettings(createModel('sdxl'), { + referenceImages: [ + { + id: 'ref-1', + isEnabled: true, + config: { + beginEndStepPct: [0, 1], + clipVisionModel: 'ViT-H', + image: referenceImage, + method: 'style', + model: sdxlIpAdapter, + type: 'ip_adapter', + weight: 1, + }, + }, + ], + }), + model: krea2, + models: [sdxlIpAdapter], + }); + + expect(fromSdxl.settings.referenceImages).toEqual([ + { + id: 'ref-1', + isEnabled: true, + config: { image: referenceImage, styleStrength: 1, type: 'krea2_reference_image' }, + }, + ]); + + const styleSettings = createSettings(krea2, { + referenceImages: [ + { + id: 'style', + isEnabled: true, + config: { image: referenceImage, styleStrength: 1, type: 'krea2_reference_image' }, + }, + ], + }); + + expect(getGenerationValidationReasons(krea2, styleSettings)).toEqual([]); + expect(getGenerationValidationReasons(createModel('sd-3'), styleSettings)).toEqual([ + 'Reference Image #1 is not supported by sd-3 model.', + ]); + }); + + it('switches off the enabled references a newly selected model has no room for, keeping every reference', () => { + const krea2 = createModel('krea-2', { format: 'diffusers' }); + const ipAdapter = (id: string, isEnabled: boolean) => ({ + id, + isEnabled, + config: { + beginEndStepPct: [0, 1] as [number, number], + clipVisionModel: 'ViT-H' as const, + image: { + ...referenceImage, + original: { image: { ...referenceImage.original.image, image_name: `${id}.png` } }, + }, + method: 'style' as const, + model: sdxlIpAdapter, + type: 'ip_adapter' as const, + weight: 1, + }, + }); + + const result = getGenerateModelSelectionResult({ + currentValues: createSettings(createModel('sdxl'), { + referenceImages: [ipAdapter('bypassed', false), ipAdapter('used', true), ipAdapter('extra', true)], + }), + model: krea2, + models: [sdxlIpAdapter], + }); + + // The first enabled reference is the one that keeps running, not the first in the list. + expect( + result.settings.referenceImages.map(({ config, id, isEnabled }) => ({ id, isEnabled, type: config.type })) + ).toEqual([ + { id: 'bypassed', isEnabled: false, type: 'krea2_reference_image' }, + { id: 'used', isEnabled: true, type: 'krea2_reference_image' }, + { id: 'extra', isEnabled: false, type: 'krea2_reference_image' }, + ]); + expect(result.clearedLabels).toContain('Reference Images'); + expect(getGenerationValidationReasons(krea2, result.settings)).toEqual([]); + }); + + it('names a limit of one reference in the singular', () => { + const krea2 = createModel('krea-2', { format: 'diffusers' }); + const style = (id: string) => ({ + id, + isEnabled: true, + config: { image: referenceImage, styleStrength: 1, type: 'krea2_reference_image' as const }, + }); + + expect( + getGenerationValidationReasons(krea2, createSettings(krea2, { referenceImages: [style('a'), style('b')] })) + ).toContain(`Generate supports at most 1 reference image for ${krea2.name}.`); + }); + it('reports reference image support per model', () => { expect(isReferenceImageSupported(createModel('sdxl'))).toBe(true); expect(isReferenceImageSupported(createModel('cogview4'))).toBe(false); diff --git a/invokeai/frontend/webv2/src/features/generation/core/baseGenerationPolicies.ts b/invokeai/frontend/webv2/src/features/generation/core/baseGenerationPolicies.ts index b3bf6a01141..f57ebb0435b 100644 --- a/invokeai/frontend/webv2/src/features/generation/core/baseGenerationPolicies.ts +++ b/invokeai/frontend/webv2/src/features/generation/core/baseGenerationPolicies.ts @@ -61,12 +61,17 @@ import { DEFAULT_ANIMA_INPAINT_ADAPTER_WEIGHT, DEFAULT_HIDIFFUSION_T1_RATIO, DEFAULT_HIDIFFUSION_T2_RATIO, + DEFAULT_FLUX_DYPE_EXPONENT, + DEFAULT_FLUX_DYPE_SCALE, DEFAULT_IDEOGRAM4_SAMPLER_PRESET, DEFAULT_KREA2_REBALANCE_MULTIPLIER, DEFAULT_KREA2_REBALANCE_WEIGHTS, DEFAULT_KREA2_SEED_VARIANCE_RANDOMIZE_PERCENT, DEFAULT_KREA2_SEED_VARIANCE_STRENGTH, + DEFAULT_KREA2_STYLE_STRENGTH, DEFAULT_REFERENCE_IMAGE_LIMIT, + DEFAULT_ZIMAGE_SEED_VARIANCE_RANDOMIZE_PERCENT, + DEFAULT_ZIMAGE_SEED_VARIANCE_STRENGTH, deriveAspectRatioId, IDEOGRAM4_GUIDANCE_MAX, IDEOGRAM4_GUIDANCE_MIN, @@ -358,6 +363,8 @@ export const getGenerationUiPolicy = (model: GenerateModelConfig | undefined) => seedVisible, // Show PiD controls on supported bases even while PiD is disabled. pidVisible: getIsPidSupportedBase(model?.base), + // Both Qwen-Image text encoders take the quantization choice; no other family's encoder does. + encoderQuantizationVisible: model?.type === 'main' && model.base === 'qwen-image', }; }; @@ -477,6 +484,15 @@ export const getDefaultGenerateSettings = (model?: GenerateModelConfig): Generat krea2SeedVarianceEnabled: false, krea2SeedVarianceStrength: DEFAULT_KREA2_SEED_VARIANCE_STRENGTH, krea2SeedVarianceRandomizePercent: DEFAULT_KREA2_SEED_VARIANCE_RANDOMIZE_PERCENT, + fluxDypePreset: 'off', + fluxDypeScale: DEFAULT_FLUX_DYPE_SCALE, + fluxDypeExponent: DEFAULT_FLUX_DYPE_EXPONENT, + zImageShift: null, + zImageSeedVarianceEnabled: false, + zImageSeedVarianceStrength: DEFAULT_ZIMAGE_SEED_VARIANCE_STRENGTH, + zImageSeedVarianceRandomizePercent: DEFAULT_ZIMAGE_SEED_VARIANCE_RANDOMIZE_PERCENT, + qwenImageShift: null, + qwenImageEncoderQuantization: 'none', pidMode: 'off', pidDecoderModel: null, gemma2EncoderModel: null, @@ -1313,6 +1329,10 @@ export const getDefaultReferenceImageConfig = ( return { image, type: 'qwen_image_reference_image' }; } + if (modelBase === 'krea-2') { + return { image, styleStrength: DEFAULT_KREA2_STYLE_STRENGTH, type: 'krea2_reference_image' }; + } + if (isFluxKontextModel(model)) { return { image, model, type: 'flux_kontext_reference_image' }; } @@ -1342,6 +1362,8 @@ const getReferenceImageConfigSupported = ( return model.type !== 'external_image_generator' && model.base === 'flux2'; case 'qwen_image_reference_image': return model.type !== 'external_image_generator' && model.base === 'qwen-image' && model.variant === 'edit'; + case 'krea2_reference_image': + return model.type !== 'external_image_generator' && model.base === 'krea-2'; case 'flux_kontext_reference_image': return isFluxKontextModel(model); case 'flux_redux': @@ -1427,7 +1449,22 @@ const getSettingsWithCompatibleModelSelections = ( addClearedLabel(clearedLabels, 'LoRAs'); } - const compatibleReferenceImages = getCompatibleReferenceImages(settings.referenceImages, model, models); + // Enabled references beyond the new family's limit are switched off rather than deleted: Invoke is not left + // blocked, the first enabled ones keep running, and switching back can turn the rest on again. + const retargetedReferenceImages = getCompatibleReferenceImages(settings.referenceImages, model, models); + const maxReferenceImages = getMaxReferenceImages(model); + let enabledKept = 0; + const compatibleReferenceImages = + retargetedReferenceImages.filter((referenceImage) => referenceImage.isEnabled).length > maxReferenceImages + ? retargetedReferenceImages.map((referenceImage) => { + if (!referenceImage.isEnabled) { + return referenceImage; + } + + enabledKept += 1; + return enabledKept > maxReferenceImages ? { ...referenceImage, isEnabled: false } : referenceImage; + }) + : retargetedReferenceImages; if (compatibleReferenceImages !== settings.referenceImages) { nextSettings.referenceImages = compatibleReferenceImages; @@ -1618,7 +1655,9 @@ const getReferenceImageValidationReasons = (model: GenerateModelConfig, settings const maxReferenceImages = getMaxReferenceImages(model); if (isReferenceImageSupported(model) && enabled.length > maxReferenceImages) { - reasons.push(`Generate supports at most ${maxReferenceImages} reference images for ${model.name}.`); + reasons.push( + `Generate supports at most ${maxReferenceImages} reference image${maxReferenceImages === 1 ? '' : 's'} for ${model.name}.` + ); } enabled.forEach((referenceImage, index) => { diff --git a/invokeai/frontend/webv2/src/features/generation/core/canvas/compileCanvasGraph.test.ts b/invokeai/frontend/webv2/src/features/generation/core/canvas/compileCanvasGraph.test.ts index c97d954c1e7..433f8e0fe94 100644 --- a/invokeai/frontend/webv2/src/features/generation/core/canvas/compileCanvasGraph.test.ts +++ b/invokeai/frontend/webv2/src/features/generation/core/canvas/compileCanvasGraph.test.ts @@ -95,7 +95,7 @@ const qwen3Encoder: ComponentModelConfig = { variant: 'qwen3_06b', }; -const PROJECT_SETTINGS: GenerationProjectSettings = { useCpuNoise: true }; +const PROJECT_SETTINGS: GenerationProjectSettings = { useCpuNoise: true, useNsfwChecker: false, useWatermark: false }; const settingsFor = (model: GenerateModelConfig, overrides: Partial = {}): GenerateSettings => ({ ...getDefaultGenerateSettings(model), @@ -140,6 +140,7 @@ const compile = ( outputOnlyMaskedRegions?: boolean; controlLayers?: readonly ControlLayerGraphInput[]; regionalGuidance?: readonly RegionalGuidanceInput[]; + projectSettings?: Partial; } = {} ) => compileCanvasGraph({ @@ -160,7 +161,7 @@ const compile = ( mode, model, noiseMaskImageName: overrides.noiseMaskImageName ?? null, - projectSettings: PROJECT_SETTINGS, + projectSettings: { ...PROJECT_SETTINGS, ...overrides.projectSettings }, settings: settingsFor(model, { ...COMPONENT_OVERRIDES[model.base], ...overrides.settings, @@ -1511,3 +1512,97 @@ describe('compileCanvasGraph — regional guidance', () => { expect(collectedSources).toEqual(['krea2_seed_variance', 'rg_krea2_r1_seed_variance', 'rg_krea2_r2_seed_variance']); }); }); + +describe('compileCanvasGraph — output filters', () => { + const metadataTarget = (graph: BackendGraphContract) => + graph.edges.find((edge) => edge.destination.field === 'metadata')?.destination.node_id; + + it('filters the resized txt2img output, keeping the resize intermediate and the metadata on the result', () => { + const { backendGraph } = compile(sd1Model, 'txt2img', { + // Off the model grid, so the base decode is resized back to the bbox before the filters run. + bbox: { height: 1000, width: 700, x: 0, y: 0 }, + destination: 'gallery', + projectSettings: { useNsfwChecker: true, useWatermark: true }, + }); + + expect(backendGraph.nodes.canvas_unfiltered_output).toMatchObject({ is_intermediate: true, type: 'img_resize' }); + expect(backendGraph.nodes.canvas_img_nsfw?.type).toBe('img_nsfw'); + expect(backendGraph.nodes.canvas_output).toMatchObject({ is_intermediate: false, type: 'img_watermark' }); + expect(metadataTarget(backendGraph)).toBe('canvas_output'); + }); + + it('filters the composited inpaint result rather than the decode behind it', () => { + const { backendGraph } = compile(sd1Model, 'inpaint', { projectSettings: { useNsfwChecker: true } }); + + expect(backendGraph.nodes.canvas_unfiltered_output?.type).toBe('invokeai_img_blend'); + expect(backendGraph.nodes.canvas_output).toMatchObject({ is_intermediate: true, type: 'img_nsfw' }); + expect( + backendGraph.edges.find( + (edge) => edge.destination.node_id === 'canvas_output' && edge.destination.field === 'image' + )?.source.node_id + ).toBe('canvas_unfiltered_output'); + expect(metadataTarget(backendGraph)).toBe('canvas_output'); + }); + + it('runs a lone watermark as the staged outpaint result', () => { + const { backendGraph } = compile(sd1Model, 'outpaint', { + destination: 'canvas', + projectSettings: { useWatermark: true }, + }); + + expect(backendGraph.nodes.canvas_unfiltered_output?.type).toBe('invokeai_img_blend'); + expect(backendGraph.nodes.canvas_img_nsfw).toBeUndefined(); + expect(backendGraph.nodes.canvas_output).toMatchObject({ is_intermediate: true, type: 'img_watermark' }); + expect( + backendGraph.edges.find( + (edge) => edge.destination.node_id === 'canvas_output' && edge.destination.field === 'image' + )?.source.node_id + ).toBe('canvas_unfiltered_output'); + expect(metadataTarget(backendGraph)).toBe('canvas_output'); + }); + + it('leaves every mode untouched while both filters are off', () => { + for (const mode of ['txt2img', 'img2img', 'inpaint', 'outpaint'] as const) { + const { backendGraph } = compile(sd1Model, mode, { + projectSettings: { useNsfwChecker: false, useWatermark: false }, + }); + + expect(backendGraph.nodes.canvas_unfiltered_output).toBeUndefined(); + expect(getNodeByType(backendGraph, 'img_nsfw')).toBeUndefined(); + expect(getNodeByType(backendGraph, 'img_watermark')).toBeUndefined(); + } + }); +}); + +describe('compileCanvasGraph — Krea-2 style reference', () => { + it('encodes the reference at the processing size, not the bbox', () => { + const { backendGraph } = compile(krea2Model, 'img2img', { + // 700x1000 is off the 16px grid, so processing rounds it and resizes back. + bbox: { height: 1000, width: 700, x: 0, y: 0 }, + settings: { + referenceImages: [ + { + config: { + image: { original: { image: { height: 512, image_name: 'style.png', width: 512 } } }, + styleStrength: 1.2, + type: 'krea2_reference_image', + }, + id: 'style', + isEnabled: true, + }, + ], + }, + strength: 0.5, + }); + const styleReference = getNodeByType(backendGraph, 'krea2_style_reference'); + const denoise = backendGraph.nodes.denoise_latents; + + expect(styleReference).toMatchObject({ image: { image_name: 'style.png' }, style_strength: 1.2 }); + expect(styleReference?.width).toBe(denoise?.width); + expect(styleReference?.height).toBe(denoise?.height); + expect(denoise?.width).not.toBe(700); + expect(backendGraph.edges.find((edge) => edge.destination.field === 'style_reference')?.source.node_id).toBe( + styleReference?.id + ); + }); +}); diff --git a/invokeai/frontend/webv2/src/features/generation/core/canvas/compileCanvasGraph.ts b/invokeai/frontend/webv2/src/features/generation/core/canvas/compileCanvasGraph.ts index 39082c7264a..4260197de7f 100644 --- a/invokeai/frontend/webv2/src/features/generation/core/canvas/compileCanvasGraph.ts +++ b/invokeai/frontend/webv2/src/features/generation/core/canvas/compileCanvasGraph.ts @@ -9,7 +9,13 @@ import type { ComponentModelConfig, GenerateModelConfig, GenerateSettings } from import { getGenerationValidationReasons } from '@features/generation/core/baseGenerationPolicies'; import { findVaeSource, GRAPH_BUILDERS } from '@features/generation/core/graph'; -import { addEdge, addNode, toGraphContract } from '@features/generation/core/graphBuilder'; +import { + addEdge, + addNode, + addOutputPostprocessors, + replaceCanvasOutput, + toGraphContract, +} from '@features/generation/core/graphBuilder'; import { addKrea2ConditioningEnhancers } from '@features/generation/core/krea2Conditioning'; import { getIsPidActive } from '@features/generation/core/pid'; import { shouldUsePidDecode } from '@features/generation/core/pidGraph'; @@ -187,26 +193,6 @@ const getDenoiseSize = (graph: BackendGraphContract, denoise: BackendInvocationC const findMetadataNode = (graph: BackendGraphContract): BackendInvocationContract | undefined => Object.values(graph.nodes).find((node) => node.type === 'core_metadata'); -/** Rename every matching node ID and edge reference. */ -const renameNode = (graph: BackendGraphContract, oldId: string, newId: string): BackendInvocationContract => { - const node = graph.nodes[oldId]; - if (!node) { - throw new Error(`Canvas generation could not find the "${oldId}" node to rename.`); - } - delete graph.nodes[oldId]; - node.id = newId; - graph.nodes[newId] = node; - for (const edge of graph.edges) { - if (edge.source.node_id === oldId) { - edge.source.node_id = newId; - } - if (edge.destination.node_id === oldId) { - edge.destination.node_id = newId; - } - } - return node; -}; - /** Resizes a model-grid processing output back to the bbox's exact final footprint. */ const resizeCanvasOutputToBbox = ( graph: BackendGraphContract, @@ -229,25 +215,14 @@ const resizeCanvasOutputToBbox = ( return; } - const processingOutput = renameNode(graph, 'canvas_output', 'canvas_processing_output'); - processingOutput.is_intermediate = true; - - const output = addNode(graph, { + const { output, retired } = replaceCanvasOutput(graph, 'canvas_processing_output', { height: bbox.height, - id: 'canvas_output', is_intermediate: destination === 'canvas', type: 'img_resize', use_cache: false, width: bbox.width, }); - addEdge(graph, processingOutput, 'image', output, 'image'); - - const metadataEdge = graph.edges.find( - (edge) => edge.destination.node_id === processingOutput.id && edge.destination.field === 'metadata' - ); - if (metadataEdge) { - metadataEdge.destination.node_id = output.id; - } + addEdge(graph, retired, 'image', output, 'image'); }; /** Keeps saved-image metadata aligned to the bbox footprint, not the internal processing size. */ @@ -458,9 +433,11 @@ const graftMaskTail = ( const { compositing, denoise, destination, i2lType, initialImageName, model, settings, vaeSource } = args; const needsResize = !sizesMatch(args.bbox, args.processingSize); - // The final output owns canvas_output; the base decoder becomes intermediate. - const l2i = renameNode(graph, 'canvas_output', 'canvas_l2i'); - l2i.is_intermediate = true; + // The base decoder, until the composited output below takes over canvas_output from it. + const l2i = graph.nodes.canvas_output; + if (!l2i) { + throw new Error('Canvas generation could not find the base graph output.'); + } const gradientMask = addNode(graph, { coherence_mode: compositing.coherenceMode, @@ -526,31 +503,25 @@ const graftMaskTail = ( addEdge(graph, expandMask, 'image', outputMaskSource, 'image'); } - const output = compositing.outputOnlyMaskedRegions - ? addNode(graph, { - id: 'canvas_output', - invert_mask: true, - is_intermediate: destination === 'canvas', - type: 'apply_mask_to_image', - use_cache: false, - }) - : addNode(graph, { - id: 'canvas_output', - is_intermediate: destination === 'canvas', - layer_base: { image_name: initialImageName }, - type: 'invokeai_img_blend', - use_cache: false, - }); + const { output } = replaceCanvasOutput( + graph, + 'canvas_l2i', + compositing.outputOnlyMaskedRegions + ? { + invert_mask: true, + is_intermediate: destination === 'canvas', + type: 'apply_mask_to_image', + use_cache: false, + } + : { + is_intermediate: destination === 'canvas', + layer_base: { image_name: initialImageName }, + type: 'invokeai_img_blend', + use_cache: false, + } + ); addEdge(graph, generatedImageSource, 'image', output, compositing.outputOnlyMaskedRegions ? 'image' : 'layer_upper'); addEdge(graph, outputMaskSource, 'image', output, 'mask'); - - // Move metadata to the final saved output after renaming. - const metadataEdge = graph.edges.find( - (edge) => edge.destination.node_id === 'canvas_l2i' && edge.destination.field === 'metadata' - ); - if (metadataEdge) { - metadataEdge.destination.node_id = 'canvas_output'; - } }; /** Optionally inserts an `img_noise` node before encode; returns the node feeding `i2l.image`. */ @@ -828,6 +799,8 @@ export const compileCanvasGraph = (input: CompileCanvasGraphInput): CompiledCanv shouldUsePidDecode(settings, model.base) && settings.pidMode === 'fit' ); } + // After every mode has settled its final image node, so the filters see the composited/resized result. + addOutputPostprocessors(backendGraph, runtimeProjectSettings, outputIsIntermediate); setCanvasMetadataDimensions(backendGraph, bbox); return { diff --git a/invokeai/frontend/webv2/src/features/generation/core/contracts.ts b/invokeai/frontend/webv2/src/features/generation/core/contracts.ts index 60489f4a5a4..6c957a81714 100644 --- a/invokeai/frontend/webv2/src/features/generation/core/contracts.ts +++ b/invokeai/frontend/webv2/src/features/generation/core/contracts.ts @@ -4,6 +4,12 @@ export type ResultDestination = 'canvas' | 'gallery'; export interface GenerationProjectSettings { useCpuNoise: boolean; + // The Generate and Canvas output filters are required so that a caller passing its settings structurally breaks + // the build when it renames one, instead of silently turning a safety filter off. + /** Blur images the NSFW checker flags before they are saved. */ + useNsfwChecker: boolean; + /** Stamp an invisible watermark into every saved image. */ + useWatermark: boolean; } export interface PromptHistoryItem { diff --git a/invokeai/frontend/webv2/src/features/generation/core/graph.test.ts b/invokeai/frontend/webv2/src/features/generation/core/graph.test.ts index f61e441084f..4a97ab2594f 100644 --- a/invokeai/frontend/webv2/src/features/generation/core/graph.test.ts +++ b/invokeai/frontend/webv2/src/features/generation/core/graph.test.ts @@ -12,6 +12,7 @@ import type { import { getDefaultGenerateSettings, isSupportedGenerateModel } from './baseGenerationPolicies'; import { compileGenerateGraph, resolveGenerateSeed } from './graph'; +import { generateGraphCases, satisfiedSettingsFor } from './graphCoverage.testing'; const sd1Model: MainModelConfig = { base: 'sd-1', key: 'sd1-model', name: 'SD 1.5', type: 'main' }; const sd2Model: MainModelConfig = { base: 'sd-2', key: 'sd2-model', name: 'SD 2', type: 'main' }; @@ -160,7 +161,11 @@ const refImage = { }; const compile = (model: GenerateModelConfig, overrides: Partial = {}) => - compileGenerateGraph(createSettings(model, overrides), model, 'gallery', { useCpuNoise: true }).backendGraph; + compileGenerateGraph(createSettings(model, overrides), model, 'gallery', { + useCpuNoise: true, + useNsfwChecker: false, + useWatermark: false, + }).backendGraph; const getEdge = (graph: ReturnType, targetNodeId: string, targetField: string) => graph.edges.find((edge) => edge.destination.node_id === targetNodeId && edge.destination.field === targetField); @@ -208,7 +213,7 @@ describe('compileGenerateGraph', () => { createSettings(sd1Model), sd1Model, 'gallery', - { useCpuNoise: false }, + { useCpuNoise: false, useNsfwChecker: false, useWatermark: false }, 'xpu' ).backendGraph; @@ -1352,3 +1357,235 @@ describe('PiD decode', () => { ).toThrow(/multiple of 64/); }); }); + +describe('model-family tuning', () => { + const fluxComponents: Partial = { + clipEmbedModel: clipEmbed, + t5EncoderModel: t5Encoder, + vae: fluxVae, + }; + const krea2Model: MainModelConfig = { + base: 'krea-2', + format: 'diffusers', + key: 'krea2-tuning', + name: 'Krea-2 Turbo', + type: 'main', + variant: 'krea2_turbo', + }; + const styleReference = (styleStrength: number, isEnabled = true, id = 'style-ref') => ({ + config: { image: refImage, styleStrength, type: 'krea2_reference_image' as const }, + id, + isEnabled, + }); + + it('sends no DyPE fields while the preset is off, but records off so recall can restore it', () => { + const graph = compile(fluxModel, fluxComponents); + const metadata = getNodeByType(graph, 'core_metadata'); + + expect(graph.nodes.denoise_latents).not.toHaveProperty('dype_preset'); + expect(graph.nodes.denoise_latents).not.toHaveProperty('dype_scale'); + expect(metadata).toMatchObject({ dype_preset: 'off' }); + expect(metadata).not.toHaveProperty('dype_scale'); + }); + + it('sends only the DyPE preset for the derived presets, and scale and exponent under manual', () => { + const auto = compile(fluxModel, { ...fluxComponents, fluxDypePreset: 'auto', fluxDypeScale: 3 }); + const manual = compile(fluxModel, { + ...fluxComponents, + fluxDypeExponent: 0.5, + fluxDypePreset: 'manual', + fluxDypeScale: 3, + }); + + expect(auto.nodes.denoise_latents).toMatchObject({ dype_preset: 'auto' }); + expect(auto.nodes.denoise_latents).not.toHaveProperty('dype_scale'); + expect(manual.nodes.denoise_latents).toMatchObject({ dype_exponent: 0.5, dype_preset: 'manual', dype_scale: 3 }); + expect(getNodeByType(manual, 'core_metadata')).toMatchObject({ + dype_exponent: 0.5, + dype_preset: 'manual', + dype_scale: 3, + }); + }); + + it('leaves the Z-Image shift to the node unless overridden, and records which', () => { + const auto = compile(zImageModel); + const fixed = compile(zImageModel, { zImageShift: 3 }); + + expect(auto.nodes.denoise_latents).not.toHaveProperty('shift'); + expect(getNodeByType(auto, 'core_metadata')?.z_image_shift).toBe('auto'); + expect(fixed.nodes.denoise_latents?.shift).toBe(3); + expect(getNodeByType(fixed, 'core_metadata')?.z_image_shift).toBe(3); + }); + + it('inserts the Z-Image seed variance enhancer between the encoder and the collector, fed by the seed', () => { + const graph = compile(zImageModel, { + zImageSeedVarianceEnabled: true, + zImageSeedVarianceRandomizePercent: 25, + zImageSeedVarianceStrength: 0.3, + }); + const enhancer = getNodeByType(graph, 'z_image_seed_variance_enhancer'); + + expect(enhancer).toMatchObject({ randomize_percent: 25, strength: 0.3 }); + expect(getEdge(graph, enhancer!.id, 'conditioning')?.source.node_id).toBe('pos_cond'); + expect(getEdge(graph, enhancer!.id, 'seed')?.source.node_id).toBe('seed'); + expect(getEdge(graph, 'pos_cond_collect', 'item')?.source.node_id).toBe(enhancer!.id); + expect(getNodeByType(graph, 'core_metadata')).toMatchObject({ + z_image_seed_variance_enabled: true, + z_image_seed_variance_randomize_percent: 25, + z_image_seed_variance_strength: 0.3, + }); + }); + + it('skips the Z-Image enhancer while it is off or at zero strength, keeping the direct conditioning edge', () => { + for (const graph of [ + compile(zImageModel), + compile(zImageModel, { zImageSeedVarianceEnabled: true, zImageSeedVarianceStrength: 0 }), + ]) { + expect(getNodeByType(graph, 'z_image_seed_variance_enhancer')).toBeUndefined(); + expect(getEdge(graph, 'pos_cond_collect', 'item')?.source.node_id).toBe('pos_cond'); + } + }); + + it('sends the Qwen-Image shift only when overridden', () => { + expect(compile(qwenImageModel).nodes.denoise_latents).not.toHaveProperty('shift'); + expect(getNodeByType(compile(qwenImageModel), 'core_metadata')?.qwen_image_shift).toBe('auto'); + + const fixed = compile(qwenImageModel, { qwenImageShift: 3 }); + + expect(fixed.nodes.denoise_latents?.shift).toBe(3); + expect(getNodeByType(fixed, 'core_metadata')?.qwen_image_shift).toBe(3); + }); + + it('passes the Qwen-Image encoder quantization to both text encoders, and nothing for none', () => { + const plain = compile(qwenImageModel, { cfgScale: 4 }); + const quantized = compile(qwenImageModel, { cfgScale: 4, qwenImageEncoderQuantization: 'nf4' }); + + expect(plain.nodes.pos_cond).not.toHaveProperty('quantization'); + expect(plain.nodes.neg_cond).not.toHaveProperty('quantization'); + expect(quantized.nodes.pos_cond?.quantization).toBe('nf4'); + expect(quantized.nodes.neg_cond?.quantization).toBe('nf4'); + expect(getNodeByType(quantized, 'core_metadata')?.qwen_image_quantization).toBe('nf4'); + }); + + it('encodes the enabled Krea-2 style reference at the denoise size and feeds it to the denoise node', () => { + // The served limit is one enabled reference; a disabled card stays out of the graph. + const graph = compile(krea2Model, { + height: 768, + referenceImages: [styleReference(1.5, false, 'disabled'), styleReference(0.8, true, 'used')], + width: 1280, + }); + const styleNodes = Object.values(graph.nodes).filter((node) => node.type === 'krea2_style_reference'); + + expect(styleNodes).toHaveLength(1); + expect(styleNodes[0]).toMatchObject({ + height: 768, + image: { image_name: 'reference.png' }, + style_strength: 0.8, + width: 1280, + }); + expect(getEdge(graph, styleNodes[0]!.id, 'vae')?.source.node_id).toBe('model_loader'); + expect(getEdge(graph, 'denoise_latents', 'style_reference')?.source.node_id).toBe(styleNodes[0]!.id); + expect(getNodeByType(graph, 'core_metadata')?.krea2_style_strength).toBe(0.8); + expect((getNodeByType(graph, 'core_metadata')!.ref_images as { id: string }[]).map((entry) => entry.id)).toEqual([ + 'used', + ]); + }); + + it('encodes a cropped Krea-2 reference from its crop', () => { + const cropped = { + crop: { + box: { height: 256, width: 256, x: 0, y: 0 }, + image: { height: 256, image_name: 'crop.png', width: 256 }, + ratio: 1, + }, + original: refImage.original, + }; + const graph = compile(krea2Model, { + referenceImages: [ + { config: { image: cropped, styleStrength: 1, type: 'krea2_reference_image' }, id: 'r', isEnabled: true }, + ], + }); + + expect(getNodeByType(graph, 'krea2_style_reference')?.image).toEqual({ image_name: 'crop.png' }); + }); + + it('leaves the Krea-2 graph without a style node when every reference is bypassed', () => { + const graph = compile(krea2Model, { referenceImages: [styleReference(0)] }); + + expect(getNodeByType(graph, 'krea2_style_reference')).toBeUndefined(); + expect(getEdge(graph, 'denoise_latents', 'style_reference')).toBeUndefined(); + expect(getNodeByType(graph, 'core_metadata')).not.toHaveProperty('krea2_style_strength'); + }); +}); + +describe('output filters', () => { + it.each(generateGraphCases)('$label chains the checker and watermark after its decode', ({ base, shape }) => { + const { model, settings } = satisfiedSettingsFor(base, shape); + const plain = compileGenerateGraph(settings, model, 'gallery', { + useCpuNoise: true, + useNsfwChecker: false, + useWatermark: false, + }).backendGraph; + const graph = compileGenerateGraph(settings, model, 'canvas', { + useCpuNoise: true, + useNsfwChecker: true, + useWatermark: true, + }).backendGraph; + + expect(Object.values(plain.nodes).some((node) => node.type === 'img_nsfw' || node.type === 'img_watermark')).toBe( + false + ); + // The image that used to be the output now feeds the checker, and only the watermark is the result. + expect(graph.nodes.canvas_unfiltered_output?.type).toBe(plain.nodes.canvas_output?.type); + expect(graph.nodes.canvas_unfiltered_output?.is_intermediate).toBe(true); + expect(graph.nodes.canvas_img_nsfw).toMatchObject({ is_intermediate: true, type: 'img_nsfw', use_cache: false }); + expect(graph.nodes.canvas_output).toMatchObject({ is_intermediate: true, type: 'img_watermark', use_cache: false }); + expect(getEdge(graph, 'canvas_img_nsfw', 'image')?.source).toEqual({ + field: 'image', + node_id: 'canvas_unfiltered_output', + }); + expect(getEdge(graph, 'canvas_output', 'image')?.source).toEqual({ field: 'image', node_id: 'canvas_img_nsfw' }); + expect(getEdge(graph, 'canvas_output', 'metadata')?.source.node_id).toBe(getNodeByType(graph, 'core_metadata')?.id); + expect(getEdge(graph, 'canvas_unfiltered_output', 'metadata')).toBeUndefined(); + }); + + it('runs a single filter as the durable output itself', () => { + const graph = compileGenerateGraph(createSettings(sd1Model), sd1Model, 'gallery', { + useCpuNoise: true, + useNsfwChecker: false, + useWatermark: true, + }).backendGraph; + + expect(graph.nodes.canvas_img_nsfw).toBeUndefined(); + expect(graph.nodes.canvas_output).toMatchObject({ is_intermediate: false, type: 'img_watermark' }); + expect(getEdge(graph, 'canvas_output', 'image')?.source.node_id).toBe('canvas_unfiltered_output'); + }); + + it('keeps the filters after a PiD decode, which already ends on the output node', () => { + const gemma2: ComponentModelConfig = { base: 'any', key: 'gemma2', name: 'Gemma 2 2B', type: 'gemma2_encoder' }; + const graph = compileGenerateGraph( + createSettings(sdxlModel, { + gemma2EncoderModel: gemma2, + pidDecoderModel: { base: 'sdxl', key: 'pid-sdxl', name: 'PiD SDXL', type: 'pid_decoder' }, + pidMode: 'native', + }), + sdxlModel, + 'gallery', + { useCpuNoise: true, useNsfwChecker: true, useWatermark: false } + ).backendGraph; + + expect(graph.nodes.canvas_unfiltered_output?.type).toBe('sdxl_pid_decode'); + expect(graph.nodes.canvas_output?.type).toBe('img_nsfw'); + }); + + it('does not filter an external provider, which returns finished images', () => { + const graph = compileGenerateGraph(createSettings(externalModel), externalModel, 'gallery', { + useCpuNoise: true, + useNsfwChecker: true, + useWatermark: true, + }).backendGraph; + + expect(getNodeByType(graph, 'img_nsfw')).toBeUndefined(); + expect(getNodeByType(graph, 'img_watermark')).toBeUndefined(); + }); +}); diff --git a/invokeai/frontend/webv2/src/features/generation/core/graph.ts b/invokeai/frontend/webv2/src/features/generation/core/graph.ts index 4df2bcc73cb..13009af1045 100644 --- a/invokeai/frontend/webv2/src/features/generation/core/graph.ts +++ b/invokeai/frontend/webv2/src/features/generation/core/graph.ts @@ -46,6 +46,7 @@ import { addEdge, addMetadata, addNode, + addOutputPostprocessors, createId, getActiveCompatibleLoras, toModelIdentifier, @@ -623,6 +624,25 @@ const buildSD3Graph = ( return graph; }; +/** + * The node's own default is `off`, so that preset sends nothing and the graph stays as it was. Scale and exponent + * only mean anything under `manual`; the other presets derive theirs from the image size. + */ +const getFluxDypeFields = ( + settings: GenerateSettings +): { dype_preset?: string; dype_scale?: number; dype_exponent?: number } => { + if (settings.fluxDypePreset === 'off') { + return {}; + } + + return { + dype_preset: settings.fluxDypePreset, + ...(settings.fluxDypePreset === 'manual' + ? { dype_exponent: settings.fluxDypeExponent, dype_scale: settings.fluxDypeScale } + : {}), + }; +}; + const buildFluxGraph = ( settings: GenerateSettings, model: MainModelConfig, @@ -659,6 +679,7 @@ const buildFluxGraph = ( const posCond = addNode(graph, { id: 'pos_cond', type: 'flux_text_encoder' }); const posCondCollect = addNode(graph, { id: 'pos_cond_collect', type: 'collect' }); const denoiseSize = getDenoiseSize(settings, model); + const dype = getFluxDypeFields(settings); const denoise = addNode(graph, { denoising_end: 1, denoising_start: 0, @@ -669,6 +690,7 @@ const buildFluxGraph = ( scheduler, type: 'flux_denoise', width: denoiseSize.width, + ...dype, }); const output = addDecodeOutput({ denoise, @@ -699,6 +721,9 @@ const buildFluxGraph = ( addMetadata(graph, output, settings, model, 'flux_txt2img', projectSettings, { clip_embed_model: clipEmbedModel ?? undefined, guidance: settings.cfgScale, + ...dype, + // Recorded even when off, so recall can restore it over a preset the panel has since turned on. + dype_preset: settings.fluxDypePreset, scheduler, t5_encoder: t5EncoderModel ?? undefined, vae: vaeModel ?? undefined, @@ -960,13 +985,20 @@ const buildQwenImageGraph = ( 'transformer', ]) : modelLoader; - const posCond = addNode(graph, { id: 'pos_cond', type: 'qwen_image_text_encoder' }); - const negCond = useCfg ? addNode(graph, { id: 'neg_cond', type: 'qwen_image_text_encoder' }) : null; + // The encoders default to `none`, so only an actual quantization choice reaches the graph. + const encoderQuantization = + settings.qwenImageEncoderQuantization === 'none' ? {} : { quantization: settings.qwenImageEncoderQuantization }; + const posCond = addNode(graph, { id: 'pos_cond', type: 'qwen_image_text_encoder', ...encoderQuantization }); + const negCond = useCfg + ? addNode(graph, { id: 'neg_cond', type: 'qwen_image_text_encoder', ...encoderQuantization }) + : null; const qwenDenoiseSize = getDenoiseSize(settings, model); const denoise = addNode(graph, { cfg_scale: settings.cfgScale, height: qwenDenoiseSize.height, id: 'denoise_latents', + // Null keeps the node's dynamic shift; a value fixes the schedule (3 for the Lightning LoRAs). + ...(settings.qwenImageShift === null ? {} : { shift: settings.qwenImageShift }), steps: settings.steps, type: 'qwen_image_denoise', width: qwenDenoiseSize.width, @@ -1040,7 +1072,9 @@ const buildQwenImageGraph = ( } addMetadata(graph, output, settings, model, 'qwen_image_txt2img', projectSettings, { qwen_image_component_source: sourceModel, + qwen_image_quantization: settings.qwenImageEncoderQuantization, qwen_image_qwen_vl_encoder: settings.qwenVLEncoderModel ?? undefined, + qwen_image_shift: settings.qwenImageShift ?? 'auto', qwen_image_vae: vaeModel ?? undefined, scheduler: undefined, ...(shouldUsePidDecode(settings, model.base) ? getPidMetadata(settings) : {}), @@ -1098,6 +1132,8 @@ const buildZImageGraph = ( height: zImageDenoiseSize.height, id: 'denoise_latents', scheduler, + // Null keeps the node's size-derived shift. + ...(settings.zImageShift === null ? {} : { shift: settings.zImageShift }), steps: settings.steps, type: 'z_image_denoise', width: zImageDenoiseSize.width, @@ -1118,7 +1154,21 @@ const buildZImageGraph = ( addEdge(graph, loraSource, 'qwen3_encoder', posCond, 'qwen3_encoder'); addEdge(graph, modelLoader, 'vae', denoise, 'vae'); addEdge(graph, positivePrompt, 'value', posCond, 'prompt'); - addEdge(graph, posCond, 'conditioning', posCondCollect, 'item'); + // Zero strength would be a no-op node, so it is left out like Krea-2's enhancer. + if (settings.zImageSeedVarianceEnabled && settings.zImageSeedVarianceStrength > 0) { + const seedVariance = addNode(graph, { + id: 'seed_variance_enhancer', + randomize_percent: settings.zImageSeedVarianceRandomizePercent, + strength: settings.zImageSeedVarianceStrength, + type: 'z_image_seed_variance_enhancer', + }); + + addEdge(graph, seed, 'value', seedVariance, 'seed'); + addEdge(graph, posCond, 'conditioning', seedVariance, 'conditioning'); + addEdge(graph, seedVariance, 'conditioning', posCondCollect, 'item'); + } else { + addEdge(graph, posCond, 'conditioning', posCondCollect, 'item'); + } addEdge(graph, posCondCollect, 'collection', denoise, 'positive_conditioning'); if (negCond && negCondCollect) { @@ -1134,6 +1184,10 @@ const buildZImageGraph = ( qwen3_source: sourceModel, scheduler, vae: vaeModel ?? undefined, + z_image_seed_variance_enabled: settings.zImageSeedVarianceEnabled, + z_image_seed_variance_randomize_percent: settings.zImageSeedVarianceRandomizePercent, + z_image_seed_variance_strength: settings.zImageSeedVarianceStrength, + z_image_shift: settings.zImageShift ?? 'auto', ...(shouldUsePidDecode(settings, model.base) ? getPidMetadata(settings) : {}), }); @@ -1296,6 +1350,24 @@ const buildKrea2Graph = ( addEdge(graph, seed, 'value', denoise, 'seed'); addEdge(graph, denoise, 'latents', output, 'latents'); + // Shared-KV reference attention takes exactly one reference; zero strength is a full bypass, so it is not run. + const styleReference = getEnabledReferenceImages(settings, 'krea2_reference_image').find( + (referenceImage) => referenceImage.config.styleStrength > 0 + ); + if (styleReference?.config.image) { + const styleReferenceNode = addNode(graph, { + // The reference's image tokens are appended to the target's, so both must be the same size. + height: settings.height, + id: 'krea2_style_reference', + image: toImageField(styleReference.config.image), + style_strength: styleReference.config.styleStrength, + type: 'krea2_style_reference', + width: settings.width, + }); + + addEdge(graph, modelLoader, 'vae', styleReferenceNode, 'vae'); + addEdge(graph, styleReferenceNode, 'style_reference', denoise, 'style_reference'); + } addMetadata(graph, output, settings, model, 'krea2_txt2img', projectSettings, { krea2_rebalance_enabled: settings.krea2RebalanceEnabled, krea2_rebalance_multiplier: settings.krea2RebalanceMultiplier, @@ -1303,9 +1375,11 @@ const buildKrea2Graph = ( krea2_seed_variance_enabled: settings.krea2SeedVarianceEnabled, krea2_seed_variance_randomize_percent: settings.krea2SeedVarianceRandomizePercent, krea2_seed_variance_strength: settings.krea2SeedVarianceStrength, + ...(styleReference ? { krea2_style_strength: styleReference.config.styleStrength } : {}), qwen3_vl_encoder: qwen3VlEncoderModel ?? undefined, vae: vaeModel ?? undefined, }); + addReferenceImageMetadata(graph, output, settings); return graph; }; @@ -1622,7 +1696,11 @@ export const compileGenerateGraph = ( throw new Error(`${model.name} dimensions must be multiples of ${dimensionGrid}.`); } - return builder(settings, model, outputIsIntermediate, runtimeProjectSettings); + const graph = builder(settings, model, outputIsIntermediate, runtimeProjectSettings); + // External providers return finished images; the filters apply to every locally decoded family. + addOutputPostprocessors(graph, runtimeProjectSettings, outputIsIntermediate); + + return graph; })(); return { diff --git a/invokeai/frontend/webv2/src/features/generation/core/graphBuilder.ts b/invokeai/frontend/webv2/src/features/generation/core/graphBuilder.ts index 9dfc8d04492..163d568b082 100644 --- a/invokeai/frontend/webv2/src/features/generation/core/graphBuilder.ts +++ b/invokeai/frontend/webv2/src/features/generation/core/graphBuilder.ts @@ -41,6 +41,77 @@ export const addEdge = ( return edge; }; +/** Rename every matching node ID and edge reference. */ +const renameNode = (graph: BackendGraphContract, oldId: string, newId: string): BackendInvocationContract => { + const node = graph.nodes[oldId]; + if (!node) { + throw new Error(`Could not find the "${oldId}" node to rename.`); + } + delete graph.nodes[oldId]; + node.id = newId; + graph.nodes[newId] = node; + for (const edge of graph.edges) { + if (edge.source.node_id === oldId) { + edge.source.node_id = newId; + } + if (edge.destination.node_id === oldId) { + edge.destination.node_id = newId; + } + } + return node; +}; + +/** + * Makes `output` the graph's `canvas_output`, the node result hydration follows. The node that held the id is renamed + * to `retiredId` and made intermediate, and its metadata edge moves to `output` so the saved record stays on the node + * that emits the final image. Edges into `output` are the caller's; the retired node keeps its edges under its new id. + */ +export const replaceCanvasOutput = ( + graph: BackendGraphContract, + retiredId: string, + output: { type: string; [key: string]: unknown } +): { output: BackendInvocationContract; retired: BackendInvocationContract } => { + const retired = renameNode(graph, 'canvas_output', retiredId); + retired.is_intermediate = true; + + const next = addNode(graph, { ...output, id: 'canvas_output' }); + const metadataEdge = graph.edges.find( + (edge) => edge.destination.node_id === retiredId && edge.destination.field === 'metadata' + ); + if (metadataEdge) { + metadataEdge.destination.node_id = next.id; + } + + return { output: next, retired }; +}; + +/** + * Chains the project's optional output filters after the image the graph ends on. Each filter in turn takes over + * `canvas_output`, so the last one is the durable result and the unfiltered image becomes intermediate. + */ +export const addOutputPostprocessors = ( + graph: BackendGraphContract, + projectSettings: Pick, + outputIsIntermediate: boolean +): void => { + const filters = [ + ...(projectSettings.useNsfwChecker ? ['img_nsfw'] : []), + ...(projectSettings.useWatermark ? ['img_watermark'] : []), + ]; + let retiredId = 'canvas_unfiltered_output'; + + for (const type of filters) { + const { output, retired } = replaceCanvasOutput(graph, retiredId, { + is_intermediate: outputIsIntermediate, + type, + use_cache: false, + }); + + addEdge(graph, retired, 'image', output, 'image'); + retiredId = `canvas_${type}`; + } +}; + export const getActiveCompatibleLoras = ( settings: Pick, model: GenerateModelConfig diff --git a/invokeai/frontend/webv2/src/features/generation/core/graphCoverage.test.ts b/invokeai/frontend/webv2/src/features/generation/core/graphCoverage.test.ts index 594703a4de0..468e6b7478e 100644 --- a/invokeai/frontend/webv2/src/features/generation/core/graphCoverage.test.ts +++ b/invokeai/frontend/webv2/src/features/generation/core/graphCoverage.test.ts @@ -22,18 +22,25 @@ import { type ModelShape, satisfiedSettingsFor, SHAPE_OVERRIDES, + tunedSettingsFor, } from './graphCoverage.testing'; const compileForShape = ( base: SupportedGenerateBase, - shape: ModelShape + shape: ModelShape, + tuned = false ): { filled: GenerateComponentValueKey[]; graph: BackendGraphContract } => { - const { filled, model, settings } = satisfiedSettingsFor(base, shape); + const { filled, model, projectSettings, settings } = tuned + ? tunedSettingsFor(base, shape) + : { + ...satisfiedSettingsFor(base, shape), + projectSettings: { useCpuNoise: true, useNsfwChecker: false, useWatermark: false }, + }; // Assert every requirement before compilation for complete diagnostics. expect(getGenerationValidationReasons(model, settings), `${base}/${shape.label} is not satisfiable`).toEqual([]); - return { filled, graph: compileGenerateGraph(settings, model, 'gallery', { useCpuNoise: true }).backendGraph }; + return { filled, graph: compileGenerateGraph(settings, model, 'gallery', projectSettings).backendGraph }; }; seedArchitectureCapabilities(); @@ -95,6 +102,8 @@ describe('generate graph coverage', () => { for (const vae of offered.values()) { const { backendGraph } = compileGenerateGraph({ ...settings, vae: vae as VaeModelConfig }, model, 'gallery', { useCpuNoise: true, + useNsfwChecker: false, + useWatermark: false, }); // Metadata records the selection whether or not the builder used it, so it does not count. const sent = Object.values(backendGraph.nodes).some( @@ -134,7 +143,7 @@ describe('generate graph coverage', () => { }, model, 'gallery', - { useCpuNoise: true } + { useCpuNoise: true, useNsfwChecker: false, useWatermark: false } ); const nodes = Object.values(backendGraph.nodes); const loaded = nodes @@ -162,8 +171,12 @@ describe('generate graph coverage', () => { const fieldsFor = (nodeType: string) => (fields[nodeType] ??= { inputs: new Set(), literals: new Map(), outputs: new Set() }); - for (const { base, shape } of generateGraphCases) { - const { filled, graph } = compileForShape(base, shape); + // Each shape compiles twice: at its defaults, and with every optional tuning and output filter switched on. + const compilations = generateGraphCases.flatMap(({ base, shape }) => + [false, true].map((tuned) => ({ base, shape, ...compileForShape(base, shape, tuned) })) + ); + + for (const { base, filled, graph, shape } of compilations) { const entry = (byBase[base] ??= { componentsFilled: {}, nodeTypes: [] }); entry.componentsFilled[shape.label] = filled; diff --git a/invokeai/frontend/webv2/src/features/generation/core/graphCoverage.testing.ts b/invokeai/frontend/webv2/src/features/generation/core/graphCoverage.testing.ts index 6340317d653..81e4ec21c27 100644 --- a/invokeai/frontend/webv2/src/features/generation/core/graphCoverage.testing.ts +++ b/invokeai/frontend/webv2/src/features/generation/core/graphCoverage.testing.ts @@ -209,3 +209,39 @@ export const fullyFilledSettingsFor = (base: SupportedGenerateBase, shape: Model return { filled: [...allFilled].sort(), model, settings }; }; + +/** + * The optional per-family tuning and output filters, all switched on, so the backend contract also covers the nodes + * and literal values a default recipe never emits. Settings a family ignores are harmless to carry. + */ +export const tunedSettingsFor = (base: SupportedGenerateBase, shape: ModelShape) => { + const { filled, model, settings } = satisfiedSettingsFor(base, shape); + const image = { original: { image: { height: 1024, image_name: 'style-reference.png', width: 1024 } } }; + + return { + filled, + model, + projectSettings: { useCpuNoise: true, useNsfwChecker: true, useWatermark: true }, + settings: { + ...settings, + fluxDypeExponent: 1, + fluxDypePreset: 'manual' as const, + fluxDypeScale: 3, + qwenImageEncoderQuantization: 'nf4' as const, + qwenImageShift: 3, + referenceImages: + base === 'krea-2' + ? [ + { + config: { image, styleStrength: 1, type: 'krea2_reference_image' as const }, + id: 'style', + isEnabled: true, + }, + ] + : settings.referenceImages, + zImageSeedVarianceEnabled: true, + zImageSeedVarianceStrength: 0.2, + zImageShift: 3, + }, + }; +}; diff --git a/invokeai/frontend/webv2/src/features/generation/core/previewGraph.test.ts b/invokeai/frontend/webv2/src/features/generation/core/previewGraph.test.ts index a9f8d2c127f..8858477edb8 100644 --- a/invokeai/frontend/webv2/src/features/generation/core/previewGraph.test.ts +++ b/invokeai/frontend/webv2/src/features/generation/core/previewGraph.test.ts @@ -49,7 +49,12 @@ afterEach(resetArchitectureCapabilities); describe('compileGeneratePreviewGraph', () => { it('compiles a ready graph with deterministic node ids across repeated compiles', () => { - const input = { destination: 'gallery' as const, models, storedValues, useCpuNoise: false }; + const input = { + destination: 'gallery' as const, + models, + storedValues, + projectSettings: { useCpuNoise: false, useNsfwChecker: false, useWatermark: false }, + }; const first = compileGeneratePreviewGraph(input); const second = compileGeneratePreviewGraph(input); @@ -71,7 +76,12 @@ describe('compileGeneratePreviewGraph', () => { }); it('keeps the literal seed value instead of resolving a random one', () => { - const result = compileGeneratePreviewGraph({ destination: 'gallery', models, storedValues, useCpuNoise: false }); + const result = compileGeneratePreviewGraph({ + destination: 'gallery', + models, + storedValues, + projectSettings: { useCpuNoise: false, useNsfwChecker: false, useWatermark: false }, + }); if (result.status !== 'ready') { throw new Error('expected ready'); @@ -83,7 +93,12 @@ describe('compileGeneratePreviewGraph', () => { }); it('rewrites edges consistently with rewritten node ids', () => { - const result = compileGeneratePreviewGraph({ destination: 'gallery', models, storedValues, useCpuNoise: false }); + const result = compileGeneratePreviewGraph({ + destination: 'gallery', + models, + storedValues, + projectSettings: { useCpuNoise: false, useNsfwChecker: false, useWatermark: false }, + }); if (result.status !== 'ready') { throw new Error('expected ready'); @@ -104,7 +119,7 @@ describe('compileGeneratePreviewGraph', () => { destination: 'gallery', models, storedValues: badValues, - useCpuNoise: false, + projectSettings: { useCpuNoise: false, useNsfwChecker: false, useWatermark: false }, }); expect(result.status).toBe('invalid'); @@ -119,14 +134,19 @@ describe('compileGeneratePreviewGraph', () => { destination: 'gallery', models: [], storedValues, - useCpuNoise: false, + projectSettings: { useCpuNoise: false, useNsfwChecker: false, useWatermark: false }, }); expect(result.status).toBe('invalid'); }); it('pins a deterministic seed across repeated compiles when generate settings are uninitialized', () => { - const input = { destination: 'gallery' as const, models, storedValues: undefined, useCpuNoise: false }; + const input = { + destination: 'gallery' as const, + models, + storedValues: undefined, + projectSettings: { useCpuNoise: false, useNsfwChecker: false, useWatermark: false }, + }; const first = compileGeneratePreviewGraph(input); const second = compileGeneratePreviewGraph(input); diff --git a/invokeai/frontend/webv2/src/features/generation/core/previewGraph.ts b/invokeai/frontend/webv2/src/features/generation/core/previewGraph.ts index 105177ac794..12c3659af8e 100644 --- a/invokeai/frontend/webv2/src/features/generation/core/previewGraph.ts +++ b/invokeai/frontend/webv2/src/features/generation/core/previewGraph.ts @@ -1,4 +1,10 @@ -import type { BackendGraphContract, GenerationModelCatalogItem, GraphContract, ResultDestination } from './contracts'; +import type { + BackendGraphContract, + GenerationModelCatalogItem, + GenerationProjectSettings, + GraphContract, + ResultDestination, +} from './contracts'; import type { GenerateWidgetValues } from './types'; import { getGenerationModelAvailabilityReasons, getGenerationValidationReasons } from './baseGenerationPolicies'; @@ -16,7 +22,7 @@ export interface GeneratePreviewInput { destination: ResultDestination; models: readonly GenerationModelCatalogItem[]; storedValues: unknown; - useCpuNoise: boolean; + projectSettings: GenerationProjectSettings; } export type GeneratePreviewResult = @@ -102,9 +108,7 @@ export const compileGeneratePreviewGraph = (input: GeneratePreviewInput): Genera } try { - const compiled = compileGenerateGraph(settings, settings.model, input.destination, { - useCpuNoise: input.useCpuNoise, - }); + const compiled = compileGenerateGraph(settings, settings.model, input.destination, input.projectSettings); // Preview graphs bake a literal seed because they have no queue-batch injection step. const seedNode = compiled.backendGraph.nodes[compiled.seedNodeId]; const backendGraphWithSeed: BackendGraphContract = seedNode diff --git a/invokeai/frontend/webv2/src/features/generation/core/settings.test.ts b/invokeai/frontend/webv2/src/features/generation/core/settings.test.ts index d117035a449..acd73b2f713 100644 --- a/invokeai/frontend/webv2/src/features/generation/core/settings.test.ts +++ b/invokeai/frontend/webv2/src/features/generation/core/settings.test.ts @@ -279,6 +279,62 @@ describe('normalizeGenerateSettings', () => { expect(normalized?.vaePrecision).toBe('fp32'); }); + it('bounds the model-family tuning fields and defaults the rest', () => { + const normalized = normalizeGenerateSettings({ + ...legacyStoredValues, + fluxDypeExponent: 5000, + fluxDypePreset: 'turbo', + fluxDypeScale: -1, + qwenImageEncoderQuantization: 'fp8', + qwenImageShift: 3, + zImageSeedVarianceEnabled: 'yes', + zImageSeedVarianceRandomizePercent: 0, + zImageSeedVarianceStrength: 9, + zImageShift: 42, + }); + + expect(normalized).toMatchObject({ + fluxDypeExponent: 1000, + fluxDypePreset: 'off', + fluxDypeScale: 0, + qwenImageEncoderQuantization: 'none', + qwenImageShift: 3, + zImageSeedVarianceEnabled: false, + zImageSeedVarianceRandomizePercent: 1, + zImageSeedVarianceStrength: 2, + // Out of the node's range means auto rather than a clamped override. + zImageShift: null, + }); + expect(normalizeGenerateSettings(legacyStoredValues)).toMatchObject({ + fluxDypeExponent: 2, + fluxDypePreset: 'off', + fluxDypeScale: 2, + qwenImageEncoderQuantization: 'none', + qwenImageShift: null, + zImageSeedVarianceEnabled: false, + zImageSeedVarianceRandomizePercent: 50, + zImageSeedVarianceStrength: 0.1, + zImageShift: null, + }); + }); + + it('normalizes a Krea-2 style reference, defaulting and bounding its strength', () => { + const image = { original: { image: { height: 512, image_name: 'style.png', width: 512 } } }; + const normalized = normalizeGenerateSettings({ + ...legacyStoredValues, + referenceImages: [ + { config: { image, type: 'krea2_reference_image' }, id: 'defaulted', isEnabled: true }, + { config: { image, styleStrength: 7, type: 'krea2_reference_image' }, id: 'bounded', isEnabled: true }, + ], + }); + + expect(normalized?.referenceImages.map((referenceImage) => referenceImage.config)).toEqual([ + { image, styleStrength: 1, type: 'krea2_reference_image' }, + { image, styleStrength: 2, type: 'krea2_reference_image' }, + ]); + expect(normalized && isGenerateSettings(normalized)).toBe(true); + }); + it('drops malformed persisted model identifiers missing a base', () => { const normalized = normalizeGenerateSettings({ ...legacyStoredValues, diff --git a/invokeai/frontend/webv2/src/features/generation/core/settings.ts b/invokeai/frontend/webv2/src/features/generation/core/settings.ts index 4f1eb5be30d..b7609dc38dc 100644 --- a/invokeai/frontend/webv2/src/features/generation/core/settings.ts +++ b/invokeai/frontend/webv2/src/features/generation/core/settings.ts @@ -11,10 +11,12 @@ import type { GenerateReferenceImageAsset, GenerateReferenceImageConfig, GenerateSettings, + FluxDypePreset, GenerateWidgetValues, Ideogram4SamplerPreset, LoraModelConfig, MainModelConfig, + QwenImageEncoderQuantization, VaeModelConfig, VaePrecision, } from './types'; @@ -215,10 +217,48 @@ export const IDEOGRAM4_MU_MAX = 4; export const DEFAULT_KREA2_SEED_VARIANCE_STRENGTH = 0.1; export const MAX_KREA2_SEED_VARIANCE_STRENGTH = 2; export const DEFAULT_KREA2_SEED_VARIANCE_RANDOMIZE_PERCENT = 50; +/** `krea2_style_reference.style_strength` is `ge=0, le=2`; 1 is the recommended setting. */ +export const DEFAULT_KREA2_STYLE_STRENGTH = 1; +export const MAX_KREA2_STYLE_STRENGTH = 2; +/** `flux_denoise.dype_scale` is `le=8` and `dype_exponent` `le=1000`; both default to 2 under the manual preset. */ +export const FLUX_DYPE_PRESETS: FluxDypePreset[] = ['off', 'manual', 'auto', 'area', '4k']; +export const DEFAULT_FLUX_DYPE_SCALE = 2; +export const MAX_FLUX_DYPE_SCALE = 8; +export const DEFAULT_FLUX_DYPE_EXPONENT = 2; +export const MAX_FLUX_DYPE_EXPONENT = 1000; +/** + * 3 is the Lightning value and the auto-derived one at one megapixel. The nodes accept 0, but Qwen-Image takes its log + * and fails the queue item, and Z-Image's schedule collapses, so the override starts above it. + */ +export const DEFAULT_TIMESTEP_SHIFT = 3; +export const MIN_TIMESTEP_SHIFT = 0.1; +export const MAX_TIMESTEP_SHIFT = 10; +/** `z_image_seed_variance_enhancer`: strength `le=2`, randomize percent `ge=1, le=100`. */ +export const DEFAULT_ZIMAGE_SEED_VARIANCE_STRENGTH = 0.1; +export const MAX_ZIMAGE_SEED_VARIANCE_STRENGTH = 2; +export const MIN_ZIMAGE_SEED_VARIANCE_RANDOMIZE_PERCENT = 1; +export const DEFAULT_ZIMAGE_SEED_VARIANCE_RANDOMIZE_PERCENT = 50; +export const QWEN_IMAGE_ENCODER_QUANTIZATIONS: QwenImageEncoderQuantization[] = ['none', 'int8', 'nf4']; const isIdeogram4SamplerPreset = (value: unknown): value is Ideogram4SamplerPreset => IDEOGRAM4_SAMPLER_PRESETS.includes(value as Ideogram4SamplerPreset); +export const isFluxDypePreset = (value: unknown): value is FluxDypePreset => + FLUX_DYPE_PRESETS.includes(value as FluxDypePreset); + +export const isQwenImageEncoderQuantization = (value: unknown): value is QwenImageEncoderQuantization => + QWEN_IMAGE_ENCODER_QUANTIZATIONS.includes(value as QwenImageEncoderQuantization); + +/** + * A shift override, kept only inside the node's bounds. Anything else means auto rather than a clamped value: null is + * the legacy sentinel for "let the node derive it", and silently clamping a stray value would turn auto recipes into + * fixed ones. + */ +const getTimestepShift = (value: unknown): number | null => + typeof value === 'number' && Number.isFinite(value) && value >= MIN_TIMESTEP_SHIFT && value <= MAX_TIMESTEP_SHIFT + ? value + : null; + const getOptionalNumber = (value: unknown): number | null => typeof value === 'number' && Number.isFinite(value) ? value : null; @@ -246,6 +286,7 @@ const REFERENCE_IMAGE_CONFIG_TYPES = new Set([ 'flux_kontext_reference_image', 'flux_redux', 'ip_adapter', + 'krea2_reference_image', 'qwen_image_reference_image', ]); @@ -285,6 +326,18 @@ const normalizeReferenceImageConfig = (value: unknown): GenerateReferenceImageCo return { image, type: value.type }; case 'flux_kontext_reference_image': return { image, model: getMainModelOrNull(value.model), type: value.type }; + case 'krea2_reference_image': + return { + image, + styleStrength: getClampedNumber( + value, + 'styleStrength', + 0, + MAX_KREA2_STYLE_STRENGTH, + DEFAULT_KREA2_STYLE_STRENGTH + ), + type: value.type, + }; case 'flux_redux': return { image, @@ -340,6 +393,7 @@ const cloneReferenceImageConfig = (config: GenerateReferenceImageConfig): Genera switch (config.type) { case 'external_reference_image': case 'flux2_reference_image': + case 'krea2_reference_image': case 'qwen_image_reference_image': return { ...config, image: config.image ? cloneCroppableImage(config.image) : null }; case 'flux_kontext_reference_image': @@ -508,6 +562,7 @@ const syncReferenceImageConfigWithModels = ( } case 'external_reference_image': case 'flux2_reference_image': + case 'krea2_reference_image': case 'qwen_image_reference_image': return config; } @@ -807,6 +862,35 @@ export const normalizeGenerateSettings = (values: unknown): GenerateSettings | n 100, DEFAULT_KREA2_SEED_VARIANCE_RANDOMIZE_PERCENT ), + fluxDypePreset: isFluxDypePreset(values.fluxDypePreset) ? values.fluxDypePreset : 'off', + fluxDypeScale: getClampedNumber(values, 'fluxDypeScale', 0, MAX_FLUX_DYPE_SCALE, DEFAULT_FLUX_DYPE_SCALE), + fluxDypeExponent: getClampedNumber( + values, + 'fluxDypeExponent', + 0, + MAX_FLUX_DYPE_EXPONENT, + DEFAULT_FLUX_DYPE_EXPONENT + ), + zImageShift: getTimestepShift(values.zImageShift), + zImageSeedVarianceEnabled: values.zImageSeedVarianceEnabled === true, + zImageSeedVarianceStrength: getClampedNumber( + values, + 'zImageSeedVarianceStrength', + 0, + MAX_ZIMAGE_SEED_VARIANCE_STRENGTH, + DEFAULT_ZIMAGE_SEED_VARIANCE_STRENGTH + ), + zImageSeedVarianceRandomizePercent: getClampedNumber( + values, + 'zImageSeedVarianceRandomizePercent', + MIN_ZIMAGE_SEED_VARIANCE_RANDOMIZE_PERCENT, + 100, + DEFAULT_ZIMAGE_SEED_VARIANCE_RANDOMIZE_PERCENT + ), + qwenImageShift: getTimestepShift(values.qwenImageShift), + qwenImageEncoderQuantization: isQwenImageEncoderQuantization(values.qwenImageEncoderQuantization) + ? values.qwenImageEncoderQuantization + : 'none', pidMode: isPidMode(values.pidMode) ? values.pidMode : 'off', pidDecoderModel: getModelIdentifierOrNull(values.pidDecoderModel), gemma2EncoderModel: getModelIdentifierOrNull(values.gemma2EncoderModel), diff --git a/invokeai/frontend/webv2/src/features/generation/core/types.ts b/invokeai/frontend/webv2/src/features/generation/core/types.ts index 285150c2352..fb0b372ecfc 100644 --- a/invokeai/frontend/webv2/src/features/generation/core/types.ts +++ b/invokeai/frontend/webv2/src/features/generation/core/types.ts @@ -140,6 +140,8 @@ export type GenerateReferenceImageConfig = } | { type: 'flux2_reference_image'; image: GenerateReferenceImageAsset | null } | { type: 'qwen_image_reference_image'; image: GenerateReferenceImageAsset | null } + /** Krea-2 training-free style transfer: no adapter model, a single reference, `styleStrength` 0–2 (0 bypasses). */ + | { type: 'krea2_reference_image'; image: GenerateReferenceImageAsset | null; styleStrength: number } | { type: 'external_reference_image'; image: GenerateReferenceImageAsset | null }; export interface GenerateReferenceImage { @@ -268,6 +270,21 @@ export interface GenerateSettings { krea2SeedVarianceEnabled: boolean; krea2SeedVarianceStrength: number; krea2SeedVarianceRandomizePercent: number; + /** FLUX.1 DyPE position extrapolation for sizes above the training resolution. `off` sends nothing. */ + fluxDypePreset: FluxDypePreset; + /** DyPE magnitude (λs) and decay speed (λt); the denoise node reads them only under the `manual` preset. */ + fluxDypeScale: number; + fluxDypeExponent: number; + /** Z-Image sigma-schedule shift; null lets the node derive it from the image size. */ + zImageShift: number | null; + /** Perturbs Z-Image conditioning for variety between seeds. Default off. */ + zImageSeedVarianceEnabled: boolean; + zImageSeedVarianceStrength: number; + zImageSeedVarianceRandomizePercent: number; + /** Qwen-Image sigma-schedule shift (Lightning LoRAs train at 3); null keeps the model's dynamic shift. */ + qwenImageShift: number | null; + /** BitsAndBytes quantization applied to the Qwen2.5-VL encoder on load, trading quality for VRAM. */ + qwenImageEncoderQuantization: QwenImageEncoderQuantization; /** PiD replaces VAE decode with caption-conditioned 4× output; requires a PiD decoder and Gemma-2 encoder. */ pidMode: PidMode; /** PiD decoder checkpoint. Trained per backbone, so it must match the main model's base. */ @@ -288,6 +305,12 @@ export type PidMode = 'off' | 'fit' | 'native'; /** Use preset defaults unless steps, guidance, or mu is explicitly overridden. */ export type Ideogram4SamplerPreset = 'V4_QUALITY_48' | 'V4_DEFAULT_20' | 'V4_TURBO_12'; +/** `flux_denoise.dype_preset`: `auto` turns on above 1536px, `area` by image area, `4k` is a fixed tuning. */ +export type FluxDypePreset = 'off' | 'manual' | 'auto' | 'area' | '4k'; + +/** `qwen_image_text_encoder.quantization`. */ +export type QwenImageEncoderQuantization = 'none' | 'int8' | 'nf4'; + export interface GenerateWidgetValues extends GenerateSettings { model: GenerateModelConfig; } diff --git a/invokeai/frontend/webv2/src/features/generation/settings.ts b/invokeai/frontend/webv2/src/features/generation/settings.ts index 70160f0d43b..7b53b1a09d8 100644 --- a/invokeai/frontend/webv2/src/features/generation/settings.ts +++ b/invokeai/frontend/webv2/src/features/generation/settings.ts @@ -13,18 +13,26 @@ export { getDefaultLoraWeight, getModelDefaultVae, hasModelDefaultVae, + isFluxDypePreset, isLoraCompatibleWithModel, isLoraModelConfig, isMainModelConfig, isModelIdentifierConfig, + isQwenImageEncoderQuantization, isVaeModelConfig, isWanLoraTargetingMain, + MAX_FLUX_DYPE_EXPONENT, + MAX_FLUX_DYPE_SCALE, MAX_HIDIFFUSION_RATIO, MAX_NEGATIVE_PROMPT_HEIGHT_PX, MAX_POSITIVE_PROMPT_HEIGHT_PX, + MAX_TIMESTEP_SHIFT, + MAX_ZIMAGE_SEED_VARIANCE_STRENGTH, MIN_NEGATIVE_PROMPT_HEIGHT_PX, MIN_HIDIFFUSION_T1_RATIO, MIN_POSITIVE_PROMPT_HEIGHT_PX, + MIN_TIMESTEP_SHIFT, + MIN_ZIMAGE_SEED_VARIANCE_RANDOMIZE_PERCENT, normalizeGenerateSettings, normalizeGenerateWidgetValues, normalizeReferenceImages, diff --git a/invokeai/frontend/webv2/src/features/generation/ui/GenerateAdvancedFields.browser.test.tsx b/invokeai/frontend/webv2/src/features/generation/ui/GenerateAdvancedFields.browser.test.tsx index 459ee6c1ce6..e62179851b5 100644 --- a/invokeai/frontend/webv2/src/features/generation/ui/GenerateAdvancedFields.browser.test.tsx +++ b/invokeai/frontend/webv2/src/features/generation/ui/GenerateAdvancedFields.browser.test.tsx @@ -33,6 +33,11 @@ void i18n.use(initReactI18next).init({ advanced: 'Advanced', cfgRescale: 'CFG rescale', clipSkip: 'CLIP skip', + encoderQuantization: 'Encoder quantization', + encoderQuantizationHelp: 'Quantizes the Qwen VL encoder on load to save VRAM.', + encoderQuantizationInt8: '8-bit (int8)', + encoderQuantizationNf4: '4-bit (nf4)', + encoderQuantizationNone: 'None (bf16)', hiDiffusion: 'HiDiffusion', hiDiffusionRauNet: 'HiDiffusion: RAU-Net', hiDiffusionT1Ratio: 'HiDiffusion: T1 Ratio', @@ -61,6 +66,13 @@ void i18n.use(initReactI18next).init({ const sd1Model: MainModelConfig = { base: 'sd-1', key: 'sd1', name: 'SD 1.5', type: 'main' }; const sd2Model: MainModelConfig = { base: 'sd-2', key: 'sd2', name: 'SD 2', type: 'main' }; const fluxModel: MainModelConfig = { base: 'flux', key: 'flux', name: 'FLUX.1 dev', type: 'main' }; +const qwenImageModel: MainModelConfig = { + base: 'qwen-image', + format: 'diffusers', + key: 'qwen-image', + name: 'Qwen Image', + type: 'main', +}; let host: HTMLDivElement | null = null; let root: Root | null = null; (globalThis as typeof globalThis & { IS_REACT_ACT_ENVIRONMENT: boolean }).IS_REACT_ACT_ENVIRONMENT = true; @@ -159,3 +171,17 @@ describe('GenerateAdvancedFields PiD control', () => { expect(selectByLabel('PiD Decode')).toBeNull(); }); }); + +describe('GenerateAdvancedFields encoder quantization', () => { + it('offers the Qwen VL encoder quantization for Qwen Image only', async () => { + await render(qwenImageModel, false); + + expect(selectByLabel('Encoder quantization')).not.toBeNull(); + expect(selectByLabel('Encoder quantization')?.textContent).toContain('None (bf16)'); + + await settle(() => root?.unmount()); + await render(fluxModel, false); + + expect(selectByLabel('Encoder quantization')).toBeNull(); + }); +}); diff --git a/invokeai/frontend/webv2/src/features/generation/ui/GenerateAdvancedFields.tsx b/invokeai/frontend/webv2/src/features/generation/ui/GenerateAdvancedFields.tsx index 023de62930e..3973fbaeb7f 100644 --- a/invokeai/frontend/webv2/src/features/generation/ui/GenerateAdvancedFields.tsx +++ b/invokeai/frontend/webv2/src/features/generation/ui/GenerateAdvancedFields.tsx @@ -1,5 +1,9 @@ /* oxlint-disable react-perf/jsx-no-new-object-as-prop, react-perf/jsx-no-new-function-as-prop, react-perf/jsx-no-new-array-as-prop, react-perf/jsx-no-jsx-as-prop */ -import type { GenerateModelConfig, GenerateSettings } from '@features/generation/core/types'; +import type { + GenerateModelConfig, + GenerateSettings, + QwenImageEncoderQuantization, +} from '@features/generation/core/types'; import { Badge, createListCollection, HStack, Stack } from '@chakra-ui/react'; import { getDefaultGenerateSettings, getGenerationUiPolicy } from '@features/generation/core/baseGenerationPolicies'; @@ -7,9 +11,11 @@ import { isPidMode, MAX_PID_STEPS, MIN_PID_STEPS } from '@features/generation/co import { DEFAULT_HIDIFFUSION_T1_RATIO, DEFAULT_HIDIFFUSION_T2_RATIO, + isQwenImageEncoderQuantization, isVaeModelConfig, MAX_HIDIFFUSION_RATIO, MIN_HIDIFFUSION_T1_RATIO, + QWEN_IMAGE_ENCODER_QUANTIZATIONS, } from '@features/generation/core/settings'; import { useExternalStoreSelector } from '@platform/state/selectors'; import { Field, Select } from '@platform/ui'; @@ -38,6 +44,12 @@ const VAE_PRECISION_COLLECTION = createListCollection({ ] as const, }); +const ENCODER_QUANTIZATION_LABEL_KEYS: Record = { + int8: 'widgets.generate.encoderQuantizationInt8', + nf4: 'widgets.generate.encoderQuantizationNf4', + none: 'widgets.generate.encoderQuantizationNone', +}; + const selectAdvancedSettings = pickGenerateSettings([ 'cfgRescaleMultiplier', 'clipSkip', @@ -49,6 +61,7 @@ const selectAdvancedSettings = pickGenerateSettings([ 'hiDiffusionWindowAttentionEnabled', 'pidMode', 'pidSteps', + 'qwenImageEncoderQuantization', 'seamlessXAxis', 'seamlessYAxis', 'vae', @@ -83,6 +96,16 @@ export const GenerateAdvancedFields = ({ }), [t] ); + const encoderQuantizationCollection = useMemo( + () => + createListCollection({ + items: QWEN_IMAGE_ENCODER_QUANTIZATIONS.map((value) => ({ + label: t(ENCODER_QUANTIZATION_LABEL_KEYS[value]), + value, + })), + }), + [t] + ); const clipSkipMax = policy.clipSkipMax ?? 0; const updateNumber = @@ -109,7 +132,8 @@ export const GenerateAdvancedFields = ({ !policy.hiDiffusionVisible && !policy.clipSkipMax && !policy.cfgRescaleVisible && - !policy.pidVisible + !policy.pidVisible && + !policy.encoderQuantizationVisible ) { return null; } @@ -124,6 +148,7 @@ export const GenerateAdvancedFields = ({ policy.colorCompensationVisible && settings.colorCompensation, policy.hiDiffusionVisible && settings.hiDiffusionEnabled, policy.pidVisible && settings.pidMode !== 'off', + policy.encoderQuantizationVisible && settings.qwenImageEncoderQuantization !== 'none', ].filter(Boolean).length; const badges = @@ -339,6 +364,28 @@ export const GenerateAdvancedFields = ({ ) : null} + {policy.encoderQuantizationVisible ? ( + + { + const preset = value[0]; + + if (isFluxDypePreset(preset)) { + onCommit({ fluxDypePreset: preset }); + } + }} + /> + + {settings.fluxDypePreset === 'manual' ? ( + <> + onCommit({ fluxDypeScale: value })} + /> + onCommit({ fluxDypeExponent: value })} /> ) : null} @@ -272,6 +419,49 @@ const Krea2SeedVarianceFields = ({ onCommit, settings }: RenderFieldsProps) => { ); }; +/** Null lets the denoise node derive the shift from the image size; the override starts at the Lightning value. */ +const TimestepShiftField = ({ + helpText, + onChange, + shift, +}: { + helpText: string; + shift: number | null; + onChange: (shift: number | null) => void; +}) => { + const { t } = useTranslation(); + + return ( + + onChange(checked ? DEFAULT_TIMESTEP_SHIFT : null)} + /> + {shift !== null ? ( + { + if (next >= MIN_TIMESTEP_SHIFT && next <= MAX_TIMESTEP_SHIFT) { + onChange(next); + } + }} + /> + ) : null} + + ); +}; + const selectSeedHistory = (insights: GenerationQueueInsights) => insights.seedHistory; /** Clicking an executed seed switches to fixed mode. */ @@ -414,6 +604,21 @@ export const GenerateRenderSection = ({ ) : null} {familyBase === 'wan' ? : null} + {familyBase === 'z-image' ? ( + onCommit({ zImageShift })} + /> + ) : null} + {familyBase === 'qwen-image' ? ( + onCommit({ qwenImageShift })} + /> + ) : null} + {familyBase === 'flux' ? : null} {uiPolicy.schedulerVisible ? ( settings.scheduler} @@ -448,6 +653,12 @@ export const GenerateRenderSection = ({ ) : null} + {familyBase === 'z-image' ? ( + <> + + + + ) : null} ); diff --git a/invokeai/frontend/webv2/src/features/generation/ui/reference-images/ReferenceImageCard.tsx b/invokeai/frontend/webv2/src/features/generation/ui/reference-images/ReferenceImageCard.tsx index d6b3cec080b..fb89a3e1648 100644 --- a/invokeai/frontend/webv2/src/features/generation/ui/reference-images/ReferenceImageCard.tsx +++ b/invokeai/frontend/webv2/src/features/generation/ui/reference-images/ReferenceImageCard.tsx @@ -19,7 +19,13 @@ import { memo, useCallback, useRef, useState } from 'react'; import { useTranslation } from 'react-i18next'; import { formatWeight, getModeLabelKey } from './referenceImageConfig'; -import { ClipVisionSelect, FieldHeader, FluxReduxControls, IPAdapterControls } from './ReferenceImageControls'; +import { + ClipVisionSelect, + FieldHeader, + FluxReduxControls, + IPAdapterControls, + Krea2StyleReferenceControls, +} from './ReferenceImageControls'; import { ReferenceImageCropDialog } from './ReferenceImageCropDialog'; /** Reference image config variants that carry a selectable model. */ @@ -148,6 +154,14 @@ const ReferenceImageCardBase = ({ {formatWeight(config.weight)} + ) : config.type === 'krea2_reference_image' ? ( + <> + {' · '} + {t('widgets.generate.styleStrength')}{' '} + + {formatWeight(config.styleStrength)} + + ) : null} @@ -258,6 +272,8 @@ const ReferenceImageCardBase = ({ selectedBase={selectedBase} onConfigChange={changeConfig} /> + ) : config.type === 'krea2_reference_image' ? ( + ) : null} diff --git a/invokeai/frontend/webv2/src/features/generation/ui/reference-images/ReferenceImageControls.tsx b/invokeai/frontend/webv2/src/features/generation/ui/reference-images/ReferenceImageControls.tsx index cd8b419e136..c963445f3fb 100644 --- a/invokeai/frontend/webv2/src/features/generation/ui/reference-images/ReferenceImageControls.tsx +++ b/invokeai/frontend/webv2/src/features/generation/ui/reference-images/ReferenceImageControls.tsx @@ -9,6 +9,7 @@ import type { FeatureHintId } from '@platform/ui/hints'; import type { ReactNode } from 'react'; import { Collapsible, createListCollection, HStack, SegmentGroup, Stack, Text } from '@chakra-ui/react'; +import { DEFAULT_KREA2_STYLE_STRENGTH, MAX_KREA2_STYLE_STRENGTH } from '@features/generation/core/settings'; import { Field, Select, Slider } from '@platform/ui'; import { FeatureHint } from '@platform/ui/hints'; import { ScrubberField } from '@platform/ui/ScrubberField'; @@ -378,3 +379,38 @@ export const FluxReduxControls = ); }; + +const STYLE_STRENGTH_MARKS = [DEFAULT_KREA2_STYLE_STRENGTH]; + +/** Krea-2 style transfer has one knob; 0 bypasses the reference entirely. */ +export const Krea2StyleReferenceControls = ({ + config, + disabled, + onChange, +}: { + config: T; + disabled: boolean; + onChange: (config: T) => void; +}) => { + const { t } = useTranslation(); + const handleStrengthChange = useCallback( + (styleStrength: number) => onChange({ ...config, styleStrength }), + [config, onChange] + ); + + return ( + + ); +}; diff --git a/invokeai/frontend/webv2/src/features/upscale/core/graph.ts b/invokeai/frontend/webv2/src/features/upscale/core/graph.ts index 8d606cbaac8..f0cebe4aeac 100644 --- a/invokeai/frontend/webv2/src/features/upscale/core/graph.ts +++ b/invokeai/frontend/webv2/src/features/upscale/core/graph.ts @@ -91,11 +91,14 @@ export const getZImageTileControlValues = (structure: number) => ({ endStepPercent: 0.75 + structure * 0.025, }); +/** Upscale runs no output filters, so only the noise device reaches its graph. */ +type UpscaleProjectSettings = Pick; + const addUpscaleMetadata = ( graph: BackendGraphContract, { frame, output }: UpscalePathResult, settings: UpscaleWidgetValues, - projectSettings: GenerationProjectSettings, + projectSettings: UpscaleProjectSettings, randDevice: string ): void => { if (!settings.model || !settings.upscaleModel || !settings.inputImage) { @@ -159,7 +162,7 @@ interface UpscalePathContext { /** The encoders this main model reads. A field it has no slot for is null here, whatever the widget holds. */ components: ReturnType; destination: ResultDestination; - projectSettings: GenerationProjectSettings; + projectSettings: UpscaleProjectSettings; positivePrompt: BackendInvocationContract; negativePrompt: BackendInvocationContract; seed: BackendInvocationContract; @@ -677,7 +680,7 @@ const UPSCALE_PATH_BUILDERS = { export const compileUpscaleGraph = ( settings: UpscaleWidgetValues, destination: ResultDestination, - projectSettings: GenerationProjectSettings, + projectSettings: UpscaleProjectSettings, randDevice = projectSettings.useCpuNoise ? 'cpu' : 'cuda' ): CompiledUpscaleGraph => { const reasons = getUpscaleValidationReasons(settings); diff --git a/invokeai/frontend/webv2/src/platform/ui/hints/hintRegistry.ts b/invokeai/frontend/webv2/src/platform/ui/hints/hintRegistry.ts index 77f751b0f1c..59b2a51eab0 100644 --- a/invokeai/frontend/webv2/src/platform/ui/hints/hintRegistry.ts +++ b/invokeai/frontend/webv2/src/platform/ui/hints/hintRegistry.ts @@ -26,6 +26,8 @@ export const FEATURE_HINTS = { concepts: { href: SUPPORT_CONCEPTS }, conditioningRebalance: {}, creativity: {}, + dypePreset: { href: 'https://github.com/guyyariv/DyPE' }, + encoderQuantization: {}, guidance: {}, height: {}, hidiffusion: { href: 'https://github.com/megvii-research/HiDiffusion' }, @@ -61,10 +63,12 @@ export const FEATURE_HINTS = { }, steps: {}, structure: {}, + styleStrength: {}, t5Encoder: {}, tileControlNet: { href: 'https://support.invoke.ai/support/solutions/articles/151000105880' }, tileOverlap: {}, tileSize: {}, + timestepShift: {}, upscaleModel: {}, upscaleScale: {}, vae: { href: SUPPORT_ADVANCED_SETTINGS }, diff --git a/invokeai/frontend/webv2/src/workbench/canvas-operations/importGalleryImages.test.ts b/invokeai/frontend/webv2/src/workbench/canvas-operations/importGalleryImages.test.ts index c6e7f44ec2d..e85a2530fdd 100644 --- a/invokeai/frontend/webv2/src/workbench/canvas-operations/importGalleryImages.test.ts +++ b/invokeai/frontend/webv2/src/workbench/canvas-operations/importGalleryImages.test.ts @@ -128,6 +128,15 @@ const setModel = (project: Project, base: GenerateWidgetValues['model']['base']) krea2SeedVarianceEnabled: false, krea2SeedVarianceStrength: 0.1, krea2SeedVarianceRandomizePercent: 50, + fluxDypePreset: 'off', + fluxDypeScale: 2, + fluxDypeExponent: 2, + zImageShift: null, + zImageSeedVarianceEnabled: false, + zImageSeedVarianceStrength: 0.1, + zImageSeedVarianceRandomizePercent: 50, + qwenImageShift: null, + qwenImageEncoderQuantization: 'none', pidMode: 'off', pidDecoderModel: null, gemma2EncoderModel: null, diff --git a/invokeai/frontend/webv2/src/workbench/image-actions/imageRecall.test.ts b/invokeai/frontend/webv2/src/workbench/image-actions/imageRecall.test.ts index ff88676be2e..18a5426dc58 100644 --- a/invokeai/frontend/webv2/src/workbench/image-actions/imageRecall.test.ts +++ b/invokeai/frontend/webv2/src/workbench/image-actions/imageRecall.test.ts @@ -151,6 +151,15 @@ const createValues = (overrides: Partial = {}): GenerateWi krea2SeedVarianceEnabled: false, krea2SeedVarianceStrength: 0.1, krea2SeedVarianceRandomizePercent: 50, + fluxDypePreset: 'off', + fluxDypeScale: 2, + fluxDypeExponent: 2, + zImageShift: null, + zImageSeedVarianceEnabled: false, + zImageSeedVarianceStrength: 0.1, + zImageSeedVarianceRandomizePercent: 50, + qwenImageShift: null, + qwenImageEncoderQuantization: 'none', referenceImages: [], scheduler: 'euler_a', seamlessXAxis: false, @@ -1074,7 +1083,11 @@ describe('image recall', () => { Object.values(backendGraph.nodes) .filter((node) => node.type === 'lora_selector') .map((node) => [(node.lora as { key: string }).key, node.weight]); - const generated = compileGenerateGraph(generatedWith, model, 'gallery', { useCpuNoise: true }).backendGraph; + const generated = compileGenerateGraph(generatedWith, model, 'gallery', { + useCpuNoise: true, + useNsfwChecker: false, + useWatermark: false, + }).backendGraph; const result = buildImageRecallSettings({ currentValues: createValues({ loras: [{ isEnabled: true, model: staleLora, weight: 0.9 }], model: sd1Model }), image, @@ -1084,7 +1097,11 @@ describe('image recall', () => { supportedModels: [model], vaeModels: [], }); - const recalled = compileGenerateGraph(result!.values, result!.values.model, 'gallery', { useCpuNoise: true }); + const recalled = compileGenerateGraph(result!.values, result!.values.model, 'gallery', { + useCpuNoise: true, + useNsfwChecker: false, + useWatermark: false, + }); expect(loraWeights(recalled.backendGraph)).toEqual([ [grainLora.key, -0.4], @@ -1182,10 +1199,271 @@ describe('image recall', () => { }); }); +describe('model-family tuning recall', () => { + const fluxModel: MainModelConfig = { base: 'flux', key: 'flux-model', name: 'FLUX', type: 'main' }; + const zImageModel: MainModelConfig = { base: 'z-image', key: 'z-model', name: 'Z-Image', type: 'main' }; + const qwenModel: MainModelConfig = { base: 'qwen-image', key: 'qwen-model', name: 'Qwen', type: 'main' }; + const recall = ( + metadata: Record, + model: MainModelConfig, + kind: ImageRecallKind = 'all', + current: Partial = {} + ) => + buildImageRecallSettings({ + currentValues: createValues(current), + image, + kind, + metadata: { model: { key: model.key }, ...metadata }, + models: [], + supportedModels: [model], + vaeModels: [], + }); + + it('restores the DyPE recipe a FLUX image was generated with', () => { + const result = recall({ dype_exponent: 0.5, dype_preset: 'manual', dype_scale: 3 }, fluxModel, 'remix'); + + expect(result?.values).toMatchObject({ fluxDypeExponent: 0.5, fluxDypePreset: 'manual', fluxDypeScale: 3 }); + expect(result?.fields).toContain('modelTuning'); + }); + + it('restores the Z-Image shift and seed variance, reading auto back as no override', () => { + const fixed = recall( + { + z_image_seed_variance_enabled: true, + z_image_seed_variance_randomize_percent: 25, + z_image_seed_variance_strength: 0.3, + z_image_shift: 2.5, + }, + zImageModel + ); + const auto = recall({ z_image_shift: 'auto' }, zImageModel, 'all', { zImageShift: 4 }); + + expect(fixed?.values).toMatchObject({ + zImageSeedVarianceEnabled: true, + zImageSeedVarianceRandomizePercent: 25, + zImageSeedVarianceStrength: 0.3, + zImageShift: 2.5, + }); + expect(auto?.values.zImageShift).toBeNull(); + expect(auto?.fields).toContain('modelTuning'); + }); + + it('does not count tuning the recall leaves as it was', () => { + const unchanged = recall({ dype_preset: 'off', generation_mode: 'flux_txt2img', seed: 7 }, fluxModel); + + expect(unchanged?.values.fluxDypePreset).toBe('off'); + expect(unchanged?.fields).not.toContain('modelTuning'); + }); + + it('restores the Qwen-Image shift and encoder quantization', () => { + const result = recall({ qwen_image_quantization: 'nf4', qwen_image_shift: 3 }, qwenModel); + + expect(result?.values).toMatchObject({ qwenImageEncoderQuantization: 'nf4', qwenImageShift: 3 }); + }); + + it('ignores tuning recorded for another family and values outside the node bounds', () => { + const wrongFamily = recall({ dype_preset: 'manual', z_image_shift: 2 }, qwenModel); + const outOfRange = recall({ dype_preset: 'turbo', dype_scale: 9 }, fluxModel); + + expect(wrongFamily?.values.fluxDypePreset).toBe('off'); + expect(wrongFamily?.values.zImageShift).toBeNull(); + expect(wrongFamily?.fields ?? []).not.toContain('modelTuning'); + expect(outOfRange?.values).toMatchObject({ fluxDypePreset: 'off', fluxDypeScale: 2 }); + }); + + describe('over tuning the panel has since changed', () => { + const changedTuning = { + fluxDypePreset: '4k', + qwenImageEncoderQuantization: 'nf4', + qwenImageShift: 5, + zImageSeedVarianceEnabled: true, + zImageShift: 5, + } as const; + const recallOverChanged = (metadata: Record, model: MainModelConfig) => + buildImageRecallSettings({ + currentValues: createValues(changedTuning), + image, + kind: 'all', + metadata, + models: [], + supportedModels: [model], + vaeModels: [], + })?.values; + const record = (model: MainModelConfig, extra: Record = {}) => ({ + generation_mode: `${model.base}_txt2img`, + model: { base: model.base, key: model.key, name: model.name, type: 'main' }, + ...extra, + }); + + it('restores the defaults a record of the same family leaves out', () => { + expect(recallOverChanged(record(fluxModel), fluxModel)?.fluxDypePreset).toBe('off'); + expect(recallOverChanged(record(zImageModel), zImageModel)).toMatchObject({ + zImageSeedVarianceEnabled: false, + zImageShift: null, + }); + expect(recallOverChanged(record(qwenModel), qwenModel)).toMatchObject({ + qwenImageEncoderQuantization: 'none', + qwenImageShift: null, + }); + }); + + it('reads the legacy null shift as auto', () => { + expect(recallOverChanged(record(qwenModel, { qwen_image_shift: null }), qwenModel)?.qwenImageShift).toBeNull(); + }); + + it('keeps the current tuning when the record is not a generation of that family', () => { + const { generation_mode: _mode, ...withoutMode } = record(fluxModel); + const otherFamily = record(fluxModel, { + model: { base: 'sdxl', key: fluxModel.key, name: 'SDXL', type: 'main' }, + }); + + expect(recallOverChanged(withoutMode, fluxModel)?.fluxDypePreset).toBe('4k'); + expect(recallOverChanged(otherFamily, fluxModel)?.fluxDypePreset).toBe('4k'); + }); + + // What a graph records must bring the panel back to that recipe from either direction. A switch the graph stops + // recording still passes here, since recall reads a missing key as the default; graph.test.ts pins that FLUX + // records DyPE off. + const tunedBases = generateGraphCases.filter(({ base }) => ['flux', 'qwen-image', 'z-image'].includes(base)); + const defaultTuning = { + fluxDypePreset: 'off', + qwenImageEncoderQuantization: 'none', + qwenImageShift: null, + zImageSeedVarianceEnabled: false, + zImageShift: null, + } as const; + const roundTrip = ({ base, shape }: (typeof generateGraphCases)[number], recorded: object, current: object) => { + const { model, settings } = fullyFilledSettingsFor(base, shape); + const graph = compileGenerateGraph({ ...settings, ...recorded }, model, 'gallery', { + useCpuNoise: true, + useNsfwChecker: false, + useWatermark: false, + }).backendGraph; + const values = buildImageRecallSettings({ + currentValues: createValues({ ...current, model }), + image, + kind: 'all', + metadata: Object.values(graph.nodes).find((node) => node.type === 'core_metadata'), + models: [], + supportedModels: [model], + vaeModels: [], + })?.values; + const keys = { + flux: ['fluxDypePreset'], + 'qwen-image': ['qwenImageEncoderQuantization', 'qwenImageShift'], + 'z-image': ['zImageSeedVarianceEnabled', 'zImageShift'], + }[base as 'flux' | 'qwen-image' | 'z-image'] as (keyof typeof defaultTuning)[]; + + return { + expected: Object.fromEntries(keys.map((key) => [key, (recorded as typeof defaultTuning)[key]])), + restored: Object.fromEntries(keys.map((key) => [key, values?.[key]])), + }; + }; + + it.each(tunedBases)('$label restores its default tuning over changed settings', (graphCase) => { + const { expected, restored } = roundTrip(graphCase, defaultTuning, changedTuning); + + expect(restored).toEqual(expected); + }); + + it.each(tunedBases)('$label restores its changed tuning over the defaults', (graphCase) => { + const { expected, restored } = roundTrip(graphCase, changedTuning, defaultTuning); + + expect(restored).toEqual(expected); + }); + }); + + it('offers recall for an image whose only recallable delta is its tuning', () => { + const capabilities = getImageRecallCapabilities({ + currentValues: createValues({ model: fluxModel }), + image, + metadata: { dype_preset: 'auto' }, + models: [], + supportedModels: [], + vaeModels: [], + }); + + expect(capabilities.all).toBe(true); + expect(capabilities.remix).toBe(true); + }); + + it('restores a Krea-2 style reference with its strength through the recorded reference images', () => { + const result = buildImageRecallSettings({ + currentValues: createValues(), + image, + kind: 'all', + metadata: { + krea2_style_strength: 1.4, + model: { key: krea2Model.key }, + ref_images: [ + { + config: { + image: { original: { image: { height: 512, image_name: 'style.png', width: 512 } } }, + styleStrength: 1.4, + type: 'krea2_reference_image', + }, + id: 'style', + isEnabled: true, + }, + ], + }, + models: [], + supportedModels: [krea2Model], + vaeModels: [], + }); + + expect(result?.values.referenceImages).toEqual([ + { + config: { + image: { original: { image: { height: 512, image_name: 'style.png', width: 512 } } }, + styleStrength: 1.4, + type: 'krea2_reference_image', + }, + id: 'style', + isEnabled: true, + }, + ]); + expect(result?.fields).toContain('referenceImages'); + }); +}); + +describe('Krea-2 style reference without an image', () => { + it('recalls the rest of the record and leaves reference images alone', () => { + const result = buildImageRecallSettings({ + currentValues: createValues(), + image, + kind: 'all', + metadata: { + krea2_style_strength: 1.4, + model: { key: krea2Model.key }, + ref_images: [ + { + config: { image: null, styleStrength: 1.4, type: 'krea2_reference_image' }, + id: 'style', + isEnabled: true, + }, + ], + steps: 8, + }, + models: [], + supportedModels: [krea2Model], + vaeModels: [], + }); + + expect(result?.fields).toEqual(expect.arrayContaining(['model', 'steps'])); + expect(result?.fields).not.toContain('referenceImages'); + expect(result?.values.referenceImages).toEqual([]); + }); +}); + /** Remix from the metadata a fully filled graph records, starting from SDXL defaults. */ const remixRoundTrip = ({ base, shape }: (typeof generateGraphCases)[number]) => { const { filled, model, settings } = fullyFilledSettingsFor(base, shape); - const graph = compileGenerateGraph(settings, model, 'gallery', { useCpuNoise: true }).backendGraph; + const graph = compileGenerateGraph(settings, model, 'gallery', { + useCpuNoise: true, + useNsfwChecker: false, + useWatermark: false, + }).backendGraph; const recordedMetadata = Object.values(graph.nodes).find((node) => node.type === 'core_metadata'); const serializedMetadata = JSON.stringify(recordedMetadata); // A slot the graph leaves unused (e.g. a bundled model's) is not recorded, so there is nothing to restore. diff --git a/invokeai/frontend/webv2/src/workbench/image-actions/imageRecall.ts b/invokeai/frontend/webv2/src/workbench/image-actions/imageRecall.ts index 9d72e8ec608..cdb0cc39075 100644 --- a/invokeai/frontend/webv2/src/workbench/image-actions/imageRecall.ts +++ b/invokeai/frontend/webv2/src/workbench/image-actions/imageRecall.ts @@ -29,9 +29,17 @@ import { isMainModelConfig, isModelIdentifierConfig, isVaeCompatibleWithGenerateModel, + isFluxDypePreset, + isQwenImageEncoderQuantization, isValidKrea2RebalanceWeights, + MAX_FLUX_DYPE_EXPONENT, + MAX_FLUX_DYPE_SCALE, MAX_HIDIFFUSION_RATIO, + MAX_TIMESTEP_SHIFT, + MAX_ZIMAGE_SEED_VARIANCE_STRENGTH, MIN_HIDIFFUSION_T1_RATIO, + MIN_TIMESTEP_SHIFT, + MIN_ZIMAGE_SEED_VARIANCE_RANDOMIZE_PERCENT, normalizeReferenceImages, } from '@features/generation/settings'; import { SEED_MAX } from '@platform/core/seed'; @@ -75,6 +83,7 @@ type RecalledField = | 'loras' | 'referenceImages' | 'krea2Rebalance' + | 'modelTuning' | 'animaInpaintAdapterWeight'; export type ImageRecallSkipReason = @@ -421,6 +430,113 @@ const getMetadataKrea2Rebalance = ( const hasKrea2Rebalance = (metadata: unknown, model: GenerateModelConfig | undefined): boolean => Object.keys(getMetadataKrea2Rebalance(metadata, model)).length > 0; +/** + * Generation graphs stamp `generation_mode` and record what ran, so on such a record a missing list or switch means + * none or the node default. Other records that name a model, such as canvas saves, never record concepts or tuning. + */ +const isGenerationRecord = (metadata: unknown): boolean => getString(metadata, 'generation_mode') !== null; + +const getBoundedNumber = (metadata: unknown, key: string, min: number, max: number): number | null => { + const value = getNumber(metadata, key); + + return value !== null && value >= min && value <= max ? value : null; +}; + +/** + * A recorded shift: `'auto'` (webv2) or `null` (legacy) for one the node derived, a number for an override. + * Undefined when the key is missing or holds something else. + */ +const getRecordedTimestepShift = (metadata: unknown, key: string): number | null | undefined => { + if (!isRecord(metadata) || !(key in metadata)) { + return undefined; + } + + if (metadata[key] === null || metadata[key] === 'auto') { + return null; + } + + return getBoundedNumber(metadata, key, MIN_TIMESTEP_SHIFT, MAX_TIMESTEP_SHIFT) ?? undefined; +}; + +/** + * Per-family tuning the graphs record under their own keys. Gated on the model recall settles on, since each + * setting only reaches that family's denoise or encoder nodes. + * + * A generation record of that same family that leaves a switch out ran the node's default: webv2 recorded no tuning + * before the panel offered it. Restoring the default there means a preset turned on since does not survive recall. + * Values that only matter while a switch is on (DyPE scale, variance strength) stay as they are when missing. + */ +const getMetadataModelTuning = ( + metadata: unknown, + model: GenerateModelConfig | undefined +): Partial => { + if (!model || model.type !== 'main' || !isRecord(metadata)) { + return {}; + } + + const recordsFamily = isGenerationRecord(metadata) && getRecord(metadata, 'model')?.base === model.base; + const recordedOrDefault = (key: string, recorded: T | undefined, fallback: T): T | undefined => + recorded !== undefined ? recorded : recordsFamily && !(key in metadata) ? fallback : undefined; + + if (model.base === 'flux') { + const preset = recordedOrDefault( + 'dype_preset', + isFluxDypePreset(metadata.dype_preset) ? metadata.dype_preset : undefined, + 'off' + ); + const scale = getBoundedNumber(metadata, 'dype_scale', 0, MAX_FLUX_DYPE_SCALE); + const exponent = getBoundedNumber(metadata, 'dype_exponent', 0, MAX_FLUX_DYPE_EXPONENT); + + return { + ...(preset === undefined ? {} : { fluxDypePreset: preset }), + ...(scale === null ? {} : { fluxDypeScale: scale }), + ...(exponent === null ? {} : { fluxDypeExponent: exponent }), + }; + } + + if (model.base === 'z-image') { + const shift = recordedOrDefault('z_image_shift', getRecordedTimestepShift(metadata, 'z_image_shift'), null); + const enabled = recordedOrDefault( + 'z_image_seed_variance_enabled', + getBoolean(metadata, 'z_image_seed_variance_enabled') ?? undefined, + false + ); + const strength = getBoundedNumber(metadata, 'z_image_seed_variance_strength', 0, MAX_ZIMAGE_SEED_VARIANCE_STRENGTH); + const randomizePercent = getBoundedNumber( + metadata, + 'z_image_seed_variance_randomize_percent', + MIN_ZIMAGE_SEED_VARIANCE_RANDOMIZE_PERCENT, + 100 + ); + + return { + ...(shift === undefined ? {} : { zImageShift: shift }), + ...(enabled === undefined ? {} : { zImageSeedVarianceEnabled: enabled }), + ...(strength === null ? {} : { zImageSeedVarianceStrength: strength }), + ...(randomizePercent === null ? {} : { zImageSeedVarianceRandomizePercent: randomizePercent }), + }; + } + + if (model.base === 'qwen-image') { + const shift = recordedOrDefault('qwen_image_shift', getRecordedTimestepShift(metadata, 'qwen_image_shift'), null); + const quantization = recordedOrDefault( + 'qwen_image_quantization', + isQwenImageEncoderQuantization(metadata.qwen_image_quantization) ? metadata.qwen_image_quantization : undefined, + 'none' + ); + + return { + ...(shift === undefined ? {} : { qwenImageShift: shift }), + ...(quantization === undefined ? {} : { qwenImageEncoderQuantization: quantization }), + }; + } + + return {}; +}; + +const hasModelTuning = (metadata: unknown, model: GenerateModelConfig | undefined): boolean => + Object.keys(getMetadataModelTuning(metadata, model)).length > 0; + /** * The canvas records the adapter's weight beside the adapter; only an Anima image can carry it. An out-of-range * weight is clamped rather than applied, which would block every inpaint until the user edits it. @@ -583,12 +699,6 @@ const resolveRecordedLora = ( return matches.length > 1 ? 'ambiguous' : (matches[0] ?? null); }; -/** - * Generation graphs stamp `generation_mode` and omit `loras` when no concept was active, so on such a record a - * missing list means "none". Other records that name a model, such as canvas saves, never record concepts. - */ -const isGenerationRecord = (metadata: unknown): boolean => getString(metadata, 'generation_mode') !== null; - interface ConceptRecall { /** The concept list to apply, or null when the metadata says nothing reliable about concepts. */ loras: GenerateLora[] | null; @@ -748,6 +858,7 @@ export const getImageRecallCapabilities = ({ const hasReferenceImages = getMetadataReferenceImages(metadata).length > 0; const hasModel = supportedMetadataModel !== null; const hasRebalance = hasKrea2Rebalance(metadata, clipSkipModel); + const hasTuning = hasModelTuning(metadata, clipSkipModel); const hasAdapterWeight = hasAnimaInpaintAdapterWeight(metadata, clipSkipModel); const hasConcepts = (getMetadataConcepts(metadata, models, clipSkipModel, supportedMetadataModel).loras?.length ?? 0) > 0; @@ -764,6 +875,7 @@ export const getImageRecallCapabilities = ({ hasReferenceImages || hasConcepts || hasRebalance || + hasTuning || hasAdapterWeight; const hasNonSeedMetadata = hasModel || @@ -777,6 +889,7 @@ export const getImageRecallCapabilities = ({ hasReferenceImages || hasConcepts || hasRebalance || + hasTuning || hasAdapterWeight; return { @@ -847,6 +960,14 @@ export const buildImageRecallSettings = ({ fields.push('krea2Rebalance'); } + const tuningPatch = getMetadataModelTuning(metadata, values.model); + + // A same-family record always restores its switches, so only count the field when one actually changes. + if (Object.entries(tuningPatch).some(([key, value]) => values[key as keyof GenerateWidgetValues] !== value)) { + values = { ...values, ...tuningPatch }; + fields.push('modelTuning'); + } + const adapterWeightPatch = getMetadataAnimaInpaintAdapterWeight(metadata, values.model); if (Object.keys(adapterWeightPatch).length > 0) { diff --git a/invokeai/frontend/webv2/src/workbench/settings/applicationContributions.ts b/invokeai/frontend/webv2/src/workbench/settings/applicationContributions.ts index 76c78a0c0ad..a8dc59cd1be 100644 --- a/invokeai/frontend/webv2/src/workbench/settings/applicationContributions.ts +++ b/invokeai/frontend/webv2/src/workbench/settings/applicationContributions.ts @@ -78,6 +78,14 @@ export const projectSettings = section('project', 'Project', [ ...preference('useCpuNoise', 'Use CPU noise'), scope: 'project', }, + { + ...preference('useNsfwChecker', 'NSFW checker', 'safety blur'), + scope: 'project', + }, + { + ...preference('useWatermark', 'Invisible watermark', 'output'), + scope: 'project', + }, ]); export const galleryPreferenceSettings = section('gallery', 'Gallery', [ preference('confirmImageDeletion', 'Confirm image deletion', 'delete safety'), diff --git a/invokeai/frontend/webv2/src/workbench/settings/contracts.ts b/invokeai/frontend/webv2/src/workbench/settings/contracts.ts index ba250a87af9..5d130227b74 100644 --- a/invokeai/frontend/webv2/src/workbench/settings/contracts.ts +++ b/invokeai/frontend/webv2/src/workbench/settings/contracts.ts @@ -15,6 +15,10 @@ export type { WorkbenchLanguage } from '@platform/i18n/languages'; */ export interface ProjectSettings { useCpuNoise: boolean; + /** Blur images the NSFW checker flags before they are saved. */ + useNsfwChecker: boolean; + /** Stamp an invisible watermark into every saved image. */ + useWatermark: boolean; antialiasProgressImages: boolean; /** Preview owns this live toggle; persist it per project rather than exposing it in Settings. */ showProgressImagesInViewer: boolean; diff --git a/invokeai/frontend/webv2/src/workbench/settings/store.test.ts b/invokeai/frontend/webv2/src/workbench/settings/store.test.ts index 8d6e54249ad..b04734654e1 100644 --- a/invokeai/frontend/webv2/src/workbench/settings/store.test.ts +++ b/invokeai/frontend/webv2/src/workbench/settings/store.test.ts @@ -347,6 +347,14 @@ describe('normalizeProjectSettings', () => { ); }); + it('defaults the output filters off for projects saved before them', () => { + expect(store.normalizeProjectSettings({})).toMatchObject({ useNsfwChecker: false, useWatermark: false }); + expect(store.normalizeProjectSettings({ useNsfwChecker: true, useWatermark: true })).toMatchObject({ + useNsfwChecker: true, + useWatermark: true, + }); + }); + /** * Discard legacy project highlighting/attention settings so imports cannot overwrite personal editor * preferences. diff --git a/invokeai/frontend/webv2/src/workbench/settings/store.ts b/invokeai/frontend/webv2/src/workbench/settings/store.ts index 7c890fbfce0..3e13c1dd05b 100644 --- a/invokeai/frontend/webv2/src/workbench/settings/store.ts +++ b/invokeai/frontend/webv2/src/workbench/settings/store.ts @@ -39,6 +39,8 @@ export const DEFAULT_PROJECT_SETTINGS: ProjectSettings = { antialiasProgressImages: false, showProgressImagesInViewer: true, useCpuNoise: true, + useNsfwChecker: false, + useWatermark: false, }; export const DEFAULT_PREFERENCES: WorkbenchPreferences = { @@ -263,6 +265,10 @@ export const normalizeProjectSettings = (settings?: Partial): P ? settings.showProgressImagesInViewer : DEFAULT_PROJECT_SETTINGS.showProgressImagesInViewer, useCpuNoise: typeof settings?.useCpuNoise === 'boolean' ? settings.useCpuNoise : DEFAULT_PROJECT_SETTINGS.useCpuNoise, + useNsfwChecker: + typeof settings?.useNsfwChecker === 'boolean' ? settings.useNsfwChecker : DEFAULT_PROJECT_SETTINGS.useNsfwChecker, + useWatermark: + typeof settings?.useWatermark === 'boolean' ? settings.useWatermark : DEFAULT_PROJECT_SETTINGS.useWatermark, }); type WorkbenchPreferencesInput = Omit< diff --git a/invokeai/frontend/webv2/src/workbench/widget-frame/graphPreviewSource.ts b/invokeai/frontend/webv2/src/workbench/widget-frame/graphPreviewSource.ts index f1c3826537e..f8afb9efe69 100644 --- a/invokeai/frontend/webv2/src/workbench/widget-frame/graphPreviewSource.ts +++ b/invokeai/frontend/webv2/src/workbench/widget-frame/graphPreviewSource.ts @@ -101,8 +101,8 @@ const buildGenerateSource = ( result = compileGeneratePreviewGraph({ destination: project.invocation.destination, models: models ?? [], + projectSettings: project.settings, storedValues: getProjectWidgetValues(project, 'generate'), - useCpuNoise: project.settings.useCpuNoise, }); } catch (error) { return { diff --git a/invokeai/frontend/webv2/src/workbench/widgets/canvas/invoke/prepareCanvasInvocation.test.ts b/invokeai/frontend/webv2/src/workbench/widgets/canvas/invoke/prepareCanvasInvocation.test.ts index d157fd3f2f4..2b2aaf5dd48 100644 --- a/invokeai/frontend/webv2/src/workbench/widgets/canvas/invoke/prepareCanvasInvocation.test.ts +++ b/invokeai/frontend/webv2/src/workbench/widgets/canvas/invoke/prepareCanvasInvocation.test.ts @@ -281,7 +281,7 @@ const makeHarness = (options: HarnessOptions = {}): Harness => { inFlight: options.inFlight ?? new Set(), models: options.models, projectId: options.projectId ?? 'project-1', - projectSettings: { useCpuNoise: true }, + projectSettings: { useCpuNoise: true, useNsfwChecker: false, useWatermark: false }, signal: new AbortController().signal, strength: options.strength ?? 0.75, }; @@ -364,7 +364,7 @@ describe('prepareCanvasInvocation generation-device boundary', () => { models: harness.deps.models, projectId: harness.deps.projectId, canvasValues: { scaleMethod: 'manual', scaledHeight: 768, scaledWidth: 1024 }, - projectSettings: { useCpuNoise: false }, + projectSettings: { useCpuNoise: false, useNsfwChecker: false, useWatermark: false }, strength: harness.deps.strength, }); @@ -392,7 +392,7 @@ describe('prepareCanvasInvocation generation-device boundary', () => { generateValues: harness.deps.generateValues, models: harness.deps.models, projectId: harness.deps.projectId, - projectSettings: { useCpuNoise: false }, + projectSettings: { useCpuNoise: false, useNsfwChecker: false, useWatermark: false }, strength: harness.deps.strength, }); diff --git a/invokeai/frontend/webv2/src/workbench/widgets/canvas/invoke/prepareCanvasInvocation.ts b/invokeai/frontend/webv2/src/workbench/widgets/canvas/invoke/prepareCanvasInvocation.ts index 4215cfc6b1f..230fa3ab377 100644 --- a/invokeai/frontend/webv2/src/workbench/widgets/canvas/invoke/prepareCanvasInvocation.ts +++ b/invokeai/frontend/webv2/src/workbench/widgets/canvas/invoke/prepareCanvasInvocation.ts @@ -4,7 +4,11 @@ * notices. A per-project in-flight guard prevents overlapping preparation. */ -import type { CanvasScalingSettings, GenerateModelConfig } from '@features/generation/contracts'; +import type { + CanvasScalingSettings, + GenerateModelConfig, + GenerationProjectSettings, +} from '@features/generation/contracts'; import type { ModelConfig } from '@features/models'; import type { CanvasControlLayerContract, @@ -14,7 +18,6 @@ import type { import type { ComposeForGenerationOptions, ComposeForGenerationResult } from '@workbench/canvas-operations/api'; import type { ExpandedPositivePrompts, ResultDestination } from '@workbench/invocationContracts'; import type { WorkbenchNotificationKind } from '@workbench/projectContracts'; -import type { ProjectSettings } from '@workbench/settings/contracts'; import type { WorkbenchCommands, WorkbenchNotificationCommands } from '@workbench/workbenchStore'; import { compileCanvasGraph, type CanvasCompileMode } from '@features/generation/canvasGraph'; @@ -95,8 +98,8 @@ export interface RunCanvasInvocationDeps { generateValues: Record; /** Loaded models, for the same value/model sync the generate path performs. */ models?: readonly ModelConfig[]; - /** Project settings (only `useCpuNoise` is consulted by the compiler). */ - projectSettings: Pick; + /** The project settings the compiler consults. */ + projectSettings: GenerationProjectSettings; /** Runtime-derived accelerator family recorded in generation metadata. */ randDevice?: string; /** Persisted denoising strength (already defaulted + clamped). Used for every image mode. */ @@ -409,7 +412,7 @@ export interface PrepareCanvasInvocationArgs { owner?: AccountScope; /** Expanded by the caller before submitting. */ expansion?: ExpandedPositivePrompts; - projectSettings: Pick; + projectSettings: GenerationProjectSettings; /** * The canvas widget's persisted values. The scaling policy is read here, in * the lazily loaded orchestrator, so its reader stays out of the eager submit path. diff --git a/invokeai/frontend/webv2/src/workbench/workbenchState.test.ts b/invokeai/frontend/webv2/src/workbench/workbenchState.test.ts index 762418334ba..26f90f889e1 100644 --- a/invokeai/frontend/webv2/src/workbench/workbenchState.test.ts +++ b/invokeai/frontend/webv2/src/workbench/workbenchState.test.ts @@ -256,6 +256,15 @@ const createGenerateValues = (overrides: Partial = {}): Ge krea2SeedVarianceEnabled: false, krea2SeedVarianceStrength: 0.1, krea2SeedVarianceRandomizePercent: 50, + fluxDypePreset: 'off', + fluxDypeScale: 2, + fluxDypeExponent: 2, + zImageShift: null, + zImageSeedVarianceEnabled: false, + zImageSeedVarianceStrength: 0.1, + zImageSeedVarianceRandomizePercent: 50, + qwenImageShift: null, + qwenImageEncoderQuantization: 'none', referenceImages: [], scheduler: 'euler_a', seamlessXAxis: false, @@ -443,6 +452,13 @@ const primeGenerate = ( const submitGenerate = (state: WorkbenchState) => workbenchReducer(state, { backendSupportsCancellation: true, type: 'submitInvocationSnapshot' }); +const getQueuedOutputNode = (state: WorkbenchState) => { + const submission = getActiveProject(state).queue.items[0]?.snapshot.backendSubmission; + const nodes = submission?.kind === 'invalid' ? {} : (submission?.graph.nodes ?? {}); + + return nodes.canvas_output; +}; + const getQueuedRandDevice = (state: WorkbenchState): unknown => { const submission = getActiveProject(state).queue.items[0]?.snapshot.backendSubmission; const nodes = submission?.kind === 'invalid' ? {} : (submission?.graph.nodes ?? {}); @@ -464,6 +480,35 @@ const createUpscaleModel = (key: string, type: string, base: string, name = key) type, }); +describe('project output filters', () => { + it('queues the NSFW checker as the generate output when the project turns it on', () => { + const initial = workbenchReducer(createInitialWorkbenchState(), { + settings: { useNsfwChecker: true }, + type: 'setActiveProjectSettings', + }); + const state = workbenchReducer(primeGenerate(initial), { + backendSupportsCancellation: true, + route: { destination: 'gallery', destinationLocked: false, sourceId: 'generate', sourceLocked: false }, + projectId: initial.activeProjectId, + type: 'submitResolvedInvocationSnapshot', + }); + + expect(getQueuedOutputNode(state)).toMatchObject({ is_intermediate: false, type: 'img_nsfw' }); + }); + + it('queues the plain decode while the filters are off', () => { + const initial = createInitialWorkbenchState(); + const state = workbenchReducer(primeGenerate(initial), { + backendSupportsCancellation: true, + route: { destination: 'gallery', destinationLocked: false, sourceId: 'generate', sourceLocked: false }, + projectId: initial.activeProjectId, + type: 'submitResolvedInvocationSnapshot', + }); + + expect(getQueuedOutputNode(state)?.type).toBe('l2i'); + }); +}); + describe('generation-device orchestration metadata', () => { it('passes the XPU runtime snapshot through the generate submission boundary', () => { generationDeviceMock.options = [{ device: 'xpu:0', name: 'Intel Arc' }]; @@ -6098,6 +6143,8 @@ describe('workbench account and project settings', () => { antialiasProgressImages: true, showProgressImagesInViewer: true, useCpuNoise: false, + useNsfwChecker: false, + useWatermark: false, }); }); }); diff --git a/tests/backend/architectures/test_features.py b/tests/backend/architectures/test_features.py index 4d92ee605a7..5b8aa98956e 100644 --- a/tests/backend/architectures/test_features.py +++ b/tests/backend/architectures/test_features.py @@ -86,7 +86,7 @@ def test_reference_images_and_the_one_variant_condition() -> None: supported = { b.value for b in generative_bases() if (f := get(b, FeaturesFacet)) is not None and f.supports_reference_images } - assert supported == {"flux", "flux2", "sd-1", "sdxl", "qwen-image"} + assert supported == {"flux", "flux2", "sd-1", "sdxl", "qwen-image", "krea-2"} conditional = { b.value diff --git a/tests/backend/image_util/test_invisible_watermark.py b/tests/backend/image_util/test_invisible_watermark.py new file mode 100644 index 00000000000..8010f98b9d5 --- /dev/null +++ b/tests/backend/image_util/test_invisible_watermark.py @@ -0,0 +1,34 @@ +import numpy as np +from PIL import Image + +from invokeai.backend.image_util.invisible_watermark import InvisibleWatermark + + +def _textured_rgb(size: int = 512) -> np.ndarray: + """A smooth gradient with some texture: enough detail for the DWT-DCT watermark to embed and decode.""" + y, x = np.mgrid[0:size, 0:size] + rgb = np.stack([(x * 255) // size, (y * 255) // size, ((x + y) * 127) // size], axis=-1) + noise = np.random.default_rng(0).integers(0, 32, size=rgb.shape) + return np.clip(rgb + noise, 0, 255).astype(np.uint8) + + +def test_watermarking_keeps_the_transparent_surround_of_a_masked_result(): + rgb = _textured_rgb() + alpha = np.zeros(rgb.shape[:2], dtype=np.uint8) + alpha[128:384, 128:384] = 255 + image = Image.fromarray(np.dstack([rgb, alpha]), mode="RGBA") + + watermarked = InvisibleWatermark.add_watermark(image, "InvokeAI") + + assert watermarked.mode == "RGBA" + assert np.array_equal(np.array(watermarked.getchannel("A")), alpha) + assert InvisibleWatermark.decode_watermark(watermarked, length=8) == "InvokeAI" + + +def test_watermarking_an_image_without_alpha_returns_it_opaque(): + image = Image.fromarray(_textured_rgb(), mode="RGB") + + watermarked = InvisibleWatermark.add_watermark(image, "InvokeAI") + + assert watermarked.mode == "RGBA" + assert np.all(np.array(watermarked.getchannel("A")) == 255)