feat(generation): restore hosted-provider options, Canvas modes and model record editing

[joshistoast]

Summary

Hosted image providers (OpenAI, Gemini, Seedream and the other external generators) lost several legacy capabilities in webv2.

Provider options and named sizes. Generate offers the options each provider's node accepts (OpenAI quality, background and input fidelity; Gemini temperature and thinking level; Seedream watermark and optimize-prompt) only for models whose provider and capabilities support them, plus the provider's named image sizes and preset resolutions. They compile into the external invocation with the node's own field names, are recorded in metadata and restored by Recall All and Remix. The named size follows the dimensions: a hand-typed size equal to a preset sends that preset's name, any other size sends none, and switching to a provider model snaps the frame to its nearest preset. Recall restores a named size only when the metadata pins one preset, and a same-provider recall resets options the image left at the provider default.

Canvas. External models generate from Canvas in text-to-image and image-to-image, with the composited frame as the init image. Providers bucket the request to their own size table, so the compiled graph scales the returned image back to the exact bounding box before staging or saving; metadata keeps the provider's request identity and seed. Inpaint is refused with a named reason because the Canvas keep-mask matches no provider's edit-mask contract; outpaint, control layers and regional guidance are refused with named reasons both in readiness and at invoke. Readiness also checks the frame against the model's maximum size and the Seedream image budget (references, init image and output). The render-size slot and strength control are hidden for providers.

Model record editor. External models gain an editor for the provider (a select over the registered providers), provider model id, supported modes, seed and reference support, image and reference limits, and default width/height/image count, saved through the model-records update contract with untouched capability tables preserved. A record whose source matches a starter-catalog entry is shown read-only, routing ids included, because the backend overlays that entry's capabilities on every response, so a changed route would keep the original entry's capabilities. Edits wait for the catalog lookup, with a Retry on failure; a retained draft is shown with a note that it cannot be saved.

Related Issues / Discussions

Follows up the legacy parity audit behind #9803.

QA Instructions

From invokeai/frontend/webv2 (Node 22.14, pnpm 10.12): pnpm check:release passes; three browser files in untouched code (GalleryMaintenance, WorkbenchHotkeyRuntime.modal, ToastContrast) timed out once under a load average above 12 and pass when rerun alone. uv run pytest tests/app/invocations/test_frontend_external_canvas_graphs.py validates the compiled Canvas graphs against the real invocation fields and edge types.

Mock backend (representative profile; three fixture models are external):

1. Models → "Fixture GPT Image": the External Provider section shows provider (select), model id, modes, limits and defaults. Toggle "Accepts a seed", Save: toast, unsaved marker clears, and a reload shows the saved value. A model installed from the starter catalog shows the whole section read-only with a "Managed by the starter catalog" notice; while the catalog is loading or unavailable the section stays locked and offers Retry.
2. Editor → model picker → External → "Fixture Gemini Image": the Size section shows Provider size presets (pick "16:9 (2K)" → 2048×1152; type 1024×1024 → the select reads "1:1 (1K)"); Provider options show Temperature override and Thinking level. "Fixture GPT Image" shows Quality, Background and Input fidelity; "Fixture Seedream" shows the two toggles.
3. Canvas with an external model: the Size section shows the frame size only and the Render section has no strength. A control or regional guidance layer, an inpaint mask, or a frame over the model's maximum size disables Invoke with the reason. Invoke and the queue cannot be exercised on the mock (no enqueue route); graph shape is covered by the compile tests and the Python graph test. A real provider key is needed to see output and recall end to end.

Review

Resolved from the first review: hosted-provider Canvas output was placed at the provider's bucketed size instead of the bounding box (now resized back, with metadata preserved); Canvas inpaint reached providers that ignore or misread the mask (now refused); the control-layer and regional-guidance rejections were unreachable (now named in readiness and at invoke); readiness validated only text-to-image, maximum size and the Seedream budget were checked only by the backend; recall could restore a named size without its dimensions and did not reset omitted provider defaults; a typed preset size read as custom and model switches never selected a preset; starter-managed records could change routing ids while keeping the old catalog overlay, and a failed catalog lookup exposed edits the backend reverted. The browser timing baselines are back to the main values rather than re-recorded: the branch adds no eager work on those routes, and the browser performance gate passes against them.

Limitations: Canvas readiness derives the mode from pixel coverage at invoke time, so a Seedream budget or inpaint refusal that depends on the painted content appears as an invoke notice rather than a disabled button; no real provider execution was performed; OpenAI input fidelity is offered only when the model takes image inputs (the backend uses it only on the edits endpoint).

Compatibility / Rollout

New generate settings are optional with defaults and normalize on load; no project-document migration. No backend or API changes; the Python addition is a test over the frontend's graph fixture.

Checklist

• The PR has a short but descriptive title, suitable for a changelog
• Meaningful regression coverage added / updated where needed; obsolete tests/code removed
• Persisted-state and API changes include required migrations / compatibility validation
• Relevant performance/efficiency opportunities considered; material claims have evidence
• Material review findings resolved and relevant checks rerun
• Documentation added / updated (if applicable)
• Updated What's New copy (if doing a release after this PR)

---

^{Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.}

[lstein]

Reviewed at 172193b. Format, oxlint, tsc, the architecture check, the unit suite (10,150 tests) and the browser tests in the touched files all pass. The performance gate was not run.

Blocker

Canvas output from hosted providers is not sized to the bounding box.

canvasProcessingSize.ts sends the raw bbox. compileCanvasGraph.ts adds no resize back to it. The backend then snaps the request to the provider's size table (_bucket_request / _bucket_to_size in external_generation_default.py), resizing the init image along with it. Only inpaint is resized back (_get_resize_target_for_inpaint). Running the backend's bucketing against the starter capabilities gives:

Model	bbox	sent / returned
gpt-image-1	640×512	1024×1024 (init squashed to 1024²)
gpt-image-1	512×512	1024×1024
seedream-4-0	1024×1024	2048×2048
seedream-4-0	1000×700	2496×1664

Gemini 3 with no matching preset gets no image_size, so a 2048 bbox comes back as a 1K image.

Staging places each result at its own size at the bbox origin (workbenchState.ts: placement = { width: image.width, height: image.height, x: bbox.x, y: bbox.y }). The staged layer therefore overflows or underfills the frame and changes aspect ratio. For img2img the content is also stretched. Core metadata records the bbox size (640×512) while the image is 1024×1024.

To reproduce: pick GPT Image 1, draw a 640×512 bbox over a full raster layer, and invoke. This is the same bbox the compile test uses, whose comment says "the provider returns one image at the bbox size".

The local path restores to the bbox, and the external path needs the same: either a resize after the provider node, or a backend resize back to the requested size for every canvas mode.

Non-blockers

1. Canvas inpaint does not work correctly with any shipped provider. graph.ts sends the canvas grayscale mask (white = keep, dark = edit), which the backend loads as L.
   • OpenAI (providers/openai.py) uploads that L PNG as the edits mask, but the endpoint expects alpha transparency to mark the edit region.
   • Gemini, Seedream, Alibaba and Atlas never read mask_image and repaint the whole frame. No blend tail is added, so unmasked pixels change.
   • Only custom records reach this today, because no starter advertises inpaint. Consider refusing inpaint until a provider handles masks, or compositing the result over the init image through the mask.
2. The regional guidance rejection is unreachable. In prepareCanvasInvocation.ts, createRegionalGuidanceCollector returns false for base external, because getRegionalGuidanceSupport is null. The regional layer is silently skipped and the run is queued without it, so the "does not support regional guidance" reason in compileCanvasGraph.ts never fires. Control layers throw ControlLayerValidationError('unsupported_adapter') in shouldComposite before compile runs, so users get the generic adapter message, not the named one. The compile tests pass regionalGuidance / controlLayers in directly, which hides this.
3. Canvas readiness validates only txt2img. workbench/invocation.ts calls getGenerationValidationReasons(values.model, values) with the default mode on the canvas branch too. A model advertising ['img2img'] is blocked from Canvas even when a full raster layer would compile it as img2img.
4. max_image_size is not checked before queueing. For example, Atlas FLUX.2 Pro / Z-Image Turbo (max 2048, no aspect buckets) with a 2500×1500 bbox passes the frontend and then fails in _validate_request. Separately, Seedream with 14 references plus the canvas init image plus one output is 16, over the 15 limit, so it raises at runtime (seedream.py). This happens only on Canvas.
5. Recall can restore a named size that is never sent. imageRecall.ts applies image_size when the model lists that name but does not apply the preset's width and height. resolveExternalImageSize then drops it.
   • Trigger: recall an image from a backend node run in the workflow editor without core metadata. It has image_size but no width/height.
   • Result: the size select shows "Custom" while the toast counts the field as applied.
   • Either apply the preset's dimensions with the name, or skip it. The test "restores … the named size … on a remix too" uses metadata without width/height, so it asserts this behaviour.
6. Recall does not reset nullable options. gemini_temperature, gemini_thinking_level and openai_input_fidelity are omitted from metadata when null, and recall only applies keys that are present. Remixing a default-temperature Gemini image while the current temperature is 2 keeps 2. A same-provider recall should reset these to null, as LoRAs are cleared when the image had none.
7. A typed size equal to a preset reads as "Custom" and sends no image_size. For example, Gemini 3 Pro with 2048×2048 typed in gets its default 1K. Model switches never set externalImageSize, whereas webv1 selects the model's first preset on switch (modelSelected.ts).
8. Performance timing baselines look loosened rather than re-measured (57273a4).
   • The launchpad's DOMContentLoaded, load and route-ready medians rose 21–34%, and editor-minimal rose 18–44% (projectSwitch included).
   • Canvas, workflow and gallery are flat.
   • The launchpad bundle grew about 9 KB, with the same request count.
   • With a 10% tolerance, this raises those routes' limits by several times the tolerance. Please re-record on a quiet machine, or explain the regression.
9. Starter-managed records: changing provider ids keeps the old catalog overlay. apply_external_starter_model_overrides matches on config.source, and a PATCH of provider_id / provider_model_id does not change source.
   • Trigger: change gemini-2.5-flash-image to gemini-3-pro-image-preview on an installed starter record.
   • Result: the record keeps 2.5-flash's modes, limits and defaults, and the read-only panel presents them as describing the new target.
   • Either lock the routing ids on starter-managed records, or explain in the notice that capabilities stay tied to the catalog entry.
10. Smaller editor issues:
    • If /starter_models fails for an admin, the full form shows. A capability edit then saves, the response re-applies the overlay, and every draft field is dropped as "submitted" behind a success toast, with no sign of the revert.
    • A capability draft made in that fallback form stays hidden, and "Unsaved changes" still shows, once a retry turns the panel read-only. Save then discards it.
    • After picking a different provider, a record's original unconfigured provider_id disappears from the select. Getting it back needs Reset, which discards the other edits.
    • The provider list comes from all registered providers, not only configured ones as the description says.

Test gaps

• In GenerateDimensionFields.tsx, replacing externalImageSize: settings.externalImageSize with null passes every GenerateDimensionFields and ExternalSizeSelect browser test. The wiring between the dimension fields and the select is untested; the select is only tested on its own.
• Removing the >= GEMINI_TEMPERATURE_MIN bound (in normalization or recall) fails no test.
• Removing the local-model early return in getMetadataExternalProvider fails no test.