feat(generation): restore the SDXL Refiner in Generate and Canvas

docs/src/content/docs/Users Guide/03.Models/Local Models/sdxl.mdx

@@ -181,8 +181,14 @@ image on its own.
 - **Installing** — install **SDXL Refiner** from the starter models, or any refiner checkpoint or
   Diffusers folder; InvokeAI recognizes refiners automatically and lists them under their own
   **SDXL Refiner** family.
-- **Using it** — the refiner is not selectable in the Generate panel or on the Canvas. Use it in the
-  workflow editor.
+- **Using it** — with an SDXL main model selected, the Generate panel shows a **Refiner** section.
+  Pick the refiner there and set **Refiner start** (the point where the base model hands over, 0.8
+  by default), its steps, CFG, scheduler and the positive and negative **aesthetic scores**. The
+  same settings apply to SDXL generations from the Canvas, including inpainting and outpainting; there
+  the base pass cannot start after the hand-over, so the effective denoising strength is capped at
+  1 − Refiner start. The refiner is never offered as a main model, and it cannot be combined with PiD
+  decoding. An image made with a refiner records it in its metadata, so **Recall All** and
+  **Remix Image** restore it. The workflow editor offers the same nodes for custom graphs.
 
 A refiner workflow runs two denoise passes over the same latents:
 

docs/src/content/docs/Users Guide/04.Image Generation/generate-panel.mdx

@@ -61,6 +61,10 @@ A negative prompt lists what you do *not* want in the image, such as *blurry, wa
 
 Not every model uses a negative prompt. SD 1.5, SDXL, SD 3.5 and CogView4 always use it. Several newer models, including Z-Image, Qwen Image, Qwen-Image-2.1 and Krea-2, use it only when CFG is above 1, and the box says so. FLUX.1, FLUX.2 and Ideogram 4 do not use a negative prompt, and the box is not shown for them.
 
+### Refiner
+
+The **Refiner** section appears for SDXL models. Pick an **SDXL Refiner** model to have it finish the last part of the denoising schedule; the base model stops at **Refiner start** (0.8 by default, so the refiner handles the final 20%) and the refiner continues from there with its own **Refiner steps**, **Refiner CFG** and **Refiner scheduler**. The **Positive** and **Negative aesthetic score** condition the refiner on the aesthetic rating it was trained with. A **Refiner start** of 1 turns the refiner **Off** and disables its tuning controls. Lower **Refiner start** to reactivate the selected refiner; the start control and model picker stay available while it is off. The controls are disabled until a refiner is selected, and refiner models are only offered here, never as the main model. The refiner also runs for SDXL generations from the Canvas. It cannot be combined with **PiD Decode**. See [SDXL](/users-guide/models/local-models/sdxl/#sdxl-refiner).
+
 ### Advanced settings
 
 The **Advanced** section appears for models with extra options, and its header counts the settings you have changed. Depending on the model it can include:
@@ -79,7 +83,7 @@ Two output filters live in **Settings** under **Project** rather than in the pan
 
 Each model has recommended settings, which you can change in the Model Manager. **Reset all to model defaults**, the ↺ button in the Generate panel's title bar, applies them: size, steps, CFG or guidance, scheduler and VAE settings. Your prompts are left alone. The button is disabled when your settings already match the defaults.
 
-To return to the settings of an earlier image instead, right-click it in the Gallery or Preview and open **Recall Metadata**. **Recall All** restores its model, concepts, prompts, seed, size and render settings; **Remix Image** does the same but keeps a new seed; and **Use Prompt**, **Use Seed** and **Use Size** recall one thing at a time. Recalled concepts replace the ones in the panel at the weights the image used; any that are not installed or do not fit the model are listed in a notice. If the image's model is not installed, or the image was saved from the canvas, your current concepts are kept.
+To return to the settings of an earlier image instead, right-click it in the Gallery or Preview and open **Recall Metadata**. **Recall All** restores its model, concepts, prompts, seed, size, render and refiner settings; **Remix Image** does the same but keeps a new seed; and **Use Prompt**, **Use Seed** and **Use Size** recall one thing at a time. Recalled concepts replace the ones in the panel at the weights the image used; any that are not installed or do not fit the model are listed in a notice. If the image's model is not installed, or the image was saved from the canvas, your current concepts are kept.
 
 To keep a combination of settings for later, use **Presets** in the panel's title bar to save the current settings under a name and apply them again.
 

docs/src/content/docs/development/Guides/recall-api.mdx

@@ -513,11 +513,26 @@ administrators' sessions ignore other users' recalls.
 | `loras`                                                                                                | Replaces the LoRA list. Entries that are not installed or not compatible with the effective model are dropped and reported.              |
 | `ip_adapters`, `reference_images`                                                                      | Feed the reference image list, retargeted to the effective model's reference-image type. Files no longer in the gallery are dropped and reported; if none of the recalled images exist the project keeps its current list. `append=true` adds behind the existing entries without re-checking them; the model's reference-image limit is enforced (newest entries are dropped first). |
 | `control_layers`                                                                                       | **Not applied.** Control layers live on the Canvas in webv2, not in the Generate panel.                                                  |
-| `denoise_strength`, `refiner_*`                                                                        | **Not applied.** The Generate panel has no image-to-image strength or SDXL refiner settings.                                              |
+| `refiner_model`, `refiner_steps`, `refiner_cfg_scale`, `refiner_denoise_start`, `refiner_positive_aesthetic_score`, `refiner_negative_aesthetic_score` | Applied for an effective SDXL main model. The refiner must be an installed `sdxl-refiner` main model, matched by key or name. Steps must be a whole number ≥1, CFG ≥1, denoising start 0–1, and aesthetic scores 1–10. |
+| `denoise_strength`                                                                                    | **Not applied.** The Generate panel has no image-to-image strength setting.                                                              |
+
+A refiner identifier that reaches the frontend but is missing, invalid or
+incompatible is reported; after any main-model compatibility transition, the
+current refiner selection and tuning are kept together, and the request's
+refiner tuning is skipped. Invalid tuning values
+are reported and leave their current values unchanged. If a requested main-model
+key is unavailable in the frontend catalog, its refiner selection and tuning are
+also skipped. This differs from an unresolved main-model name dropped by the
+backend: as described in [Model Name Resolution](#model-name-resolution), the
+remaining parameters then apply to the current model.
 
 In `strict` mode, omitted scalar parameters reset to the effective model's
 defaults (empty prompts, randomized seed, default steps/CFG/scheduler/size),
-and omitted lists clear. An omitted `model` keeps the current model.
+and omitted lists clear. An omitted `model` keeps the current model. An omitted
+`refiner_model` clears the refiner slot, and the five omitted refiner tuning
+fields listed above reset to their defaults, unless a failed model lookup
+prevents the refiner update. The refiner scheduler is unchanged; this API has no
+`refiner_scheduler` parameter.
 
 Events are applied in arrival order; an `append` request that overlaps a
 still-loading replace sees the replaced list. The handler is loaded on the

invokeai/frontend/webv2/performance/architecture-baseline.json

@@ -563,6 +563,8 @@
           "source:src/features/generation/core/promptTemplates.ts",
           "source:src/features/generation/core/referenceImage.ts",
           "source:src/features/generation/core/resolveGenerateWidgetValues.ts",
+          "source:src/features/generation/core/sdxlRefiner.ts",
+          "source:src/features/generation/core/sdxlRefinerGraph.ts",
           "source:src/features/generation/core/settings.ts",
           "source:src/features/generation/core/supportedBases.ts",
           "source:src/features/generation/data/architectureCapabilitiesApi.ts",

invokeai/frontend/webv2/performance/browser-baseline.json

@@ -990,6 +990,8 @@
       "source:src/features/generation/core/promptTemplates.ts",
       "source:src/features/generation/core/referenceImage.ts",
       "source:src/features/generation/core/resolveGenerateWidgetValues.ts",
+      "source:src/features/generation/core/sdxlRefiner.ts",
+      "source:src/features/generation/core/sdxlRefinerGraph.ts",
       "source:src/features/generation/core/settings.ts",
       "source:src/features/generation/core/supportedBases.ts",
       "source:src/features/generation/core/systemPrompts.ts",
@@ -1020,6 +1022,7 @@
       "source:src/features/generation/ui/GenerateHeaderActions.tsx",
       "source:src/features/generation/ui/GenerateModelCard.tsx",
       "source:src/features/generation/ui/GeneratePresetsPopover.tsx",
+      "source:src/features/generation/ui/GenerateRefinerSection.tsx",
       "source:src/features/generation/ui/GenerateRenderSection.tsx",
       "source:src/features/generation/ui/GenerateSettingsForm.tsx",
       "source:src/features/generation/ui/GenerateWidgetView.tsx",
@@ -2080,6 +2083,8 @@
       "source:src/features/generation/core/promptTemplates.ts",
       "source:src/features/generation/core/referenceImage.ts",
       "source:src/features/generation/core/resolveGenerateWidgetValues.ts",
+      "source:src/features/generation/core/sdxlRefiner.ts",
+      "source:src/features/generation/core/sdxlRefinerGraph.ts",
       "source:src/features/generation/core/settings.ts",
       "source:src/features/generation/core/supportedBases.ts",
       "source:src/features/generation/core/systemPrompts.ts",
@@ -2110,6 +2115,7 @@
       "source:src/features/generation/ui/GenerateHeaderActions.tsx",
       "source:src/features/generation/ui/GenerateModelCard.tsx",
       "source:src/features/generation/ui/GeneratePresetsPopover.tsx",
+      "source:src/features/generation/ui/GenerateRefinerSection.tsx",
       "source:src/features/generation/ui/GenerateRenderSection.tsx",
       "source:src/features/generation/ui/GenerateSettingsForm.tsx",
       "source:src/features/generation/ui/GenerateWidgetView.tsx",
@@ -3329,6 +3335,8 @@
       "source:src/features/generation/core/promptTemplates.ts",
       "source:src/features/generation/core/referenceImage.ts",
       "source:src/features/generation/core/resolveGenerateWidgetValues.ts",
+      "source:src/features/generation/core/sdxlRefiner.ts",
+      "source:src/features/generation/core/sdxlRefinerGraph.ts",
       "source:src/features/generation/core/settings.ts",
       "source:src/features/generation/core/supportedBases.ts",
       "source:src/features/generation/core/systemPrompts.ts",
@@ -3359,6 +3367,7 @@
       "source:src/features/generation/ui/GenerateHeaderActions.tsx",
       "source:src/features/generation/ui/GenerateModelCard.tsx",
       "source:src/features/generation/ui/GeneratePresetsPopover.tsx",
+      "source:src/features/generation/ui/GenerateRefinerSection.tsx",
       "source:src/features/generation/ui/GenerateRenderSection.tsx",
       "source:src/features/generation/ui/GenerateSettingsForm.tsx",
       "source:src/features/generation/ui/GenerateWidgetView.tsx",

invokeai/frontend/webv2/public/locales/en.json

@@ -1108,6 +1108,39 @@
     "width": {
       "heading": "Width",
       "paragraphs": ["Width of the generated image. Must be a multiple of 8."]
+    },
+    "refinerModel": {
+      "heading": "Refiner model",
+      "paragraphs": [
+        "An SDXL Refiner model that takes over the last part of the denoising schedule from the SDXL base model, adding fine detail.",
+        "Only SDXL Refiner models are listed here; they cannot be used as the main model."
+      ]
+    },
+    "refinerSteps": {
+      "heading": "Refiner steps",
+      "paragraphs": ["Number of steps the refiner schedule is divided into. Only the part after Refiner start runs."]
+    },
+    "refinerCfgScale": {
+      "heading": "Refiner CFG",
+      "paragraphs": ["How strictly the refiner follows the prompt during its pass, like the main CFG."]
+    },
+    "refinerScheduler": {
+      "heading": "Refiner scheduler",
+      "paragraphs": ["Scheduler used for the refiner pass, independent of the base model's scheduler."]
+    },
+    "refinerStart": {
+      "heading": "Refiner start",
+      "paragraphs": [
+        "Where in the schedule the base model hands over to the refiner.",
+        "0.8 means the base model denoises the first 80% and the refiner finishes the last 20%; 0 hands the whole schedule to the refiner, and 1 leaves it nothing to do, so the refiner is not run.",
+        "On the Canvas the base pass cannot start after the hand-over, so an image-to-image strength above 1 − Refiner start is capped there."
+      ]
+    },
+    "refinerAestheticScore": {
+      "heading": "Aesthetic score",
+      "paragraphs": [
+        "The refiner was trained with an aesthetic rating from 1 to 10. The positive score steers the result toward images rated that highly; the negative score steers it away from images rated that low."
+      ]
     }
   },
   "models": {
@@ -2750,6 +2783,18 @@
       "pidHelp": "Replaces the VAE decode with a 4x super-resolution decode.",
       "pidFitHelp": "Generates at the requested size, then decodes 4x and scales back down.",
       "pidNativeHelp": "The requested size is the 4x target; generation runs at a quarter of it.",
+      "presetSettingsCleared": "Settings cleared by preset",
+      "presetSettingsClearedDescription": "Cleared: {{labels}}.",
+      "refiner": "Refiner",
+      "refinerModel": "Refiner model",
+      "refinerNone": "None",
+      "refinerModelsScope": "SDXL Refiner models",
+      "refinerSteps": "Refiner steps",
+      "refinerCfgScale": "Refiner CFG",
+      "refinerScheduler": "Refiner scheduler",
+      "refinerStart": "Refiner start",
+      "refinerPositiveAestheticScore": "Positive aesthetic score",
+      "refinerNegativeAestheticScore": "Negative aesthetic score",
       "animaInpaintAdapterWeight": "Inpaint adapter weight"
     },
     "upscale": {
@@ -4347,13 +4392,20 @@
           "nothingRecalled": "No recallable image data",
           "partiallyRecalled": "Some image data was not recalled",
           "unnamedConcept": "unnamed concept",
+          "unnamedRefiner": "unnamed refiner",
           "skippedConcept": {
             "ambiguous": "{{name}} (matches more than one installed model)",
             "duplicate": "{{name}} (listed more than once)",
             "incompatible": "{{name}} (incompatible with the selected model)",
             "invalid": "{{name}} (unreadable metadata)",
             "modelUnavailable": "{{name}} (the image's model is not available)",
             "unresolved": "{{name}} (not installed)"
+          },
+          "skippedRefiner": {
+            "ambiguous": "Refiner not restored: {{name}} matches more than one installed model; the current refiner settings were kept.",
+            "invalid": "Refiner not restored: unreadable metadata; the current refiner settings were kept.",
+            "modelUnavailable": "Refiner not restored: the image's model is not available; the current refiner settings were kept.",
+            "unresolved": "Refiner not restored: {{name}} is not installed; the current refiner settings were kept."
           }
         },
         "star": {

invokeai/frontend/webv2/scripts/mock-backend-fixtures.mjs

@@ -395,14 +395,20 @@ const createQueueItems = (count) =>
 
 const MODEL_BASES = ['sd-1', 'sdxl', 'flux', 'flux2', 'qwen-image'];
 const MODEL_TYPES = ['main', 'lora', 'controlnet', 'vae', 'embedding'];
+// Main models cycle through their own bases so Generate sees SDXL mains and refiners, not only SD 1.5.
+const MAIN_MODEL_BASES = ['sd-1', 'sdxl', 'sdxl-refiner'];
 
 const createModels = (count) =>
   range(count, (index) => {
     const id = ordinal(index);
     const type = MODEL_TYPES[index % MODEL_TYPES.length];
+    const base =
+      type === 'main'
+        ? MAIN_MODEL_BASES[Math.floor(index / MODEL_TYPES.length) % MAIN_MODEL_BASES.length]
+        : MODEL_BASES[index % MODEL_BASES.length];
 
     return {
-      base: MODEL_BASES[index % MODEL_BASES.length],
+      base,
       cover_image: null,
       default_settings: type === 'main' ? { cfg_scale: 7, height: 1024, steps: 30, width: 1024 } : null,
       description: `Deterministic representative ${type} model ${id}.`,

invokeai/frontend/webv2/src/features/generation/core/__snapshots__/generateGraphNodeTypes.json

@@ -44,7 +44,8 @@
     "sdxl": {
       "componentsFilled": {
         "diffusers": [],
-        "standalone-components": []
+        "standalone-components": [],
+        "diffusers-with-refiner": []
       },
       "nodeTypes": [
         "collect",
@@ -57,6 +58,8 @@
         "noise",
         "sdxl_compel_prompt",
         "sdxl_model_loader",
+        "sdxl_refiner_compel_prompt",
+        "sdxl_refiner_model_loader",
         "string"
       ]
     },
@@ -676,6 +679,25 @@
           "cpu"
         ],
         "ref_images": [],
+        "refiner_cfg_scale": [
+          7.5
+        ],
+        "refiner_model": [],
+        "refiner_negative_aesthetic_score": [
+          2.5
+        ],
+        "refiner_positive_aesthetic_score": [
+          6
+        ],
+        "refiner_scheduler": [
+          "euler"
+        ],
+        "refiner_start": [
+          0.8
+        ],
+        "refiner_steps": [
+          20
+        ],
         "scheduler": [
           "euler_a",
           "euler"
@@ -731,6 +753,7 @@
     },
     "denoise_latents": {
       "inputs": [
+        "latents",
         "negative_conditioning",
         "noise",
         "positive_conditioning",
@@ -741,13 +764,16 @@
           0
         ],
         "cfg_scale": [
-          7
+          7,
+          7.5
         ],
         "denoising_end": [
+          0.8,
           1
         ],
         "denoising_start": [
-          0
+          0,
+          0.8
         ],
         "hidiffusion": [
           false
@@ -762,9 +788,11 @@
           true
         ],
         "scheduler": [
-          "euler_a"
+          "euler_a",
+          "euler"
         ],
         "steps": [
+          20,
           30
         ],
         "use_cache": [
@@ -1741,6 +1769,43 @@
         "vae"
       ]
     },
+    "sdxl_refiner_compel_prompt": {
+      "inputs": [
+        "clip2",
+        "style"
+      ],
+      "literalInputs": {
+        "aesthetic_score": [
+          2.5,
+          6
+        ],
+        "is_intermediate": [
+          true
+        ],
+        "use_cache": [
+          true
+        ]
+      },
+      "outputs": [
+        "conditioning"
+      ]
+    },
+    "sdxl_refiner_model_loader": {
+      "inputs": [],
+      "literalInputs": {
+        "is_intermediate": [
+          true
+        ],
+        "model": [],
+        "use_cache": [
+          true
+        ]
+      },
+      "outputs": [
+        "clip2",
+        "unet"
+      ]
+    },
     "string": {
       "inputs": [],
       "literalInputs": {