Skip to content

feat(canvas): restore PiD decoding and the Anima inpaint adapter on Canvas - #9805

Merged
lstein merged 4 commits into
mainfrom
feat/parity-canvas-pid-anima
Oct 17, 2026
Merged

lstein merged 4 commits into
mainfrom
feat/parity-canvas-pid-anima

Conversation

@joshistoast

@joshistoast joshistoast commented Oct 11, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Two Canvas generation paths the legacy frontend supported were missing from webv2.

PiD decoding on Canvas. The canvas compiler rejected every PiD mode ("not supported on the canvas yet"). It now compiles Canvas text-to-image and image-to-image, in both Fit and Native modes, on every base where Generate already supports PiD. Native image-to-image resizes the source to the quarter-size denoise resolution and decodes straight to the bounding box, as the legacy builders did; Fit keeps the full-size encode and resizes the decoded output. The encode VAE is resolved from the graph (the decode's VAE edge, or the model loader for the FLUX and SD3 decoders that take none). Inpaint/outpaint and Scale Before Processing that changes the processing size remain rejected with the legacy messages.

Anima inpaint adapter. Anima models get an optional four-channel inpaint/outpaint adapter slot (Anima LLLite models that are not three-channel) with a weight control. During Canvas inpaint and outpaint the graph adds the inverted gradient mask and an anima_lllite node fed by the resized/initial image (inpaint) or the infilled image (outpaint), joined to the shared control collector exactly as the legacy addInpaint/addOutpaint builders wired it. Ordinary control layers still reject four-channel models. The adapter and weight are recorded in metadata and restored by Recall All and Remix.

Graphs without PiD or without an adapter are unchanged.

Related Issues / Discussions

Follows up the legacy parity audit behind #9803.

QA Instructions

From invokeai/frontend/webv2 (Node 22.14, pnpm 10.12): pnpm check:release passes (lint, unit, browser, fixtures, performance, project-file journeys, accessibility). One browser suite unrelated to this change (gallery maintenance settings) timed out once under heavy machine load and passed on rerun. Python: pytest tests/app/invocations/test_frontend_anima_lllite_graphs.py tests/app/invocations/test_frontend_graph_node_types.py passes (474 passed, 2 skipped).

Follow-up at 9b892f1: pnpm check:release passes again (lint, unit, browser, performance, project-file journeys, accessibility); pytest tests/app/invocations/test_frontend_anima_lllite_graphs.py tests/app/invocations/test_frontend_graph_node_types.py: 480 passed, 2 skipped.

The mock backend has no Anima or PiD models, so UI behavior is covered by browser tests (GenerateSettingsForm.browser.test.tsx: the Inpaint Adapter slot lists only four-channel models, the weight control appears once one is picked, and the slot is absent for non-Anima models). Real-model verification was not performed. With a real install:

  1. Select an SDXL or FLUX model with a PiD decoder and Gemma-2 encoder, set PiD to Fit or Native, draw a bounding box on Canvas and invoke text-to-image and image-to-image: both run. Paint an inpaint mask: the error reads "PiD decode currently supports Text to Image and Image to Image only…". Set Scale Before Processing to Auto on a small box: "Turn off Scale Before Processing…".
  2. Select an Anima model, open Generate → Components → Inpaint Adapter: only the inpainting LLLite is listed; pick it and the weight control appears. Add an inpaint mask on Canvas and invoke: the queued graph contains anima_inpaint_adapter and invert_anima_inpaint_adapter_mask. Recall All on the result restores both.

Review

No material findings. Reviewers confirmed the compiled graphs match the legacy wiring field for field on every PiD base and both adapter paths, and that non-PiD and non-adapter graphs are unchanged. Follow-ups from review: compile tests for scaled outpaint and noise-mask inpaint with the adapter, a recall test for an uninstalled adapter, and the browser test above.

Follow-ups from the second review: PiD Fit on an off-grid bbox resized the decoded image twice (to the grid size, then to the bbox); it now resizes once, straight to the bbox. Recall All and Remix did not clear a selected inpaint adapter when the image was made without one; core_metadata drops null on write, so recall now treats an absent anima_inpaint_adapter on an anima_inpaint/anima_outpaint record as none, and a Python test pins that the invocation persists the key only when an adapter ran. A recorded adapter weight is clamped to the setting's range on recall. Tests added for SDXL Native encode size, the off-grid Fit resize, and an inpaint-with-adapter graph in the backend fixture. Adapters with cond_in_channels: null are still offered, as in legacy.

Limitations: no real-model execution; a legacy-made Anima inpaint image never recorded its adapter, so recalling one clears the adapter slot (before, it kept whatever was selected; neither restores it); the Scale Before Processing rule compares grid-clamped sizes (webv2's auto scaling only grows), so an already-optimal box is accepted where legacy's auto mode would have rejected it.

Compatibility / Rollout

The new settings are optional with defaults and normalize on project load; no project-document migration. No API changes.

Checklist

  • The PR has a short but descriptive title, suitable for a changelog
  • Meaningful regression coverage added / updated where needed; obsolete tests/code removed
  • Persisted-state and API changes include required migrations / compatibility validation
  • Relevant performance/efficiency opportunities considered; material claims have evidence
  • Material review findings resolved and relevant checks rerun
  • Documentation added / updated (if applicable)
  • Updated What's New copy (if doing a release after this PR)

View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.

Canvas txt2img and img2img decode through PiD in fit and native modes, resolving the encode VAE from the PiD graph and encoding at the denoise size; inpaint/outpaint and Scale Before Processing keep the legacy rejections.
Anima inpaint/outpaint can add the optional 4-channel ControlNet-LLLite adapter with a weight, fed the raster image and inverted mask through the shared collector, recorded in metadata and restored by Recall All and Remix.
…picker

Compile tests pin the scaled outpaint mask and pre-noise image the Anima inpaint adapter reads, recall keeps an uninstalled adapter pick while restoring the weight, and a browser case covers the Anima-only slot, its 4-channel filter and the weight scrubber.
@github-actions github-actions Bot added frontend PRs that change frontend files python-tests PRs that change python tests docs PRs that change docs labels Oct 11, 2026
@lstein lstein self-assigned this Oct 16, 2026

@lstein lstein left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at d7f8137. I found no blockers. I tried to break both halves: image-to-image encode size across every base, VAE source lookup (seamless, control, regional guidance, custom VAE on FLUX/SD3), Native/Fit geometry, the Scale Before Processing and inpaint/outpaint rejections, adapter settings lifecycle, mask polarity and sources, collector sharing with control layers, and leakage into non-canvas or non-Anima graphs. None of these turned up a defect. Both adapter snapshot graphs pass the backend's graph validation, and reverting each part of the change makes an existing test fail.

Non-blocking notes:

  1. PiD Fit mode resizes twice on an off-grid bbox (pidGraph.ts:134, then resizeCanvasOutputToBbox in compileCanvasGraph.ts:209). Example: SDXL Fit with a 1000×700 bbox snaps to 1000×704. The 4000×2816 decode is resized to 1000×704, then resized again to 1000×700. Legacy resizes once, straight to the final size. The output is valid, just slightly softer.
  2. Recall All / Remix never clears a stale inpaint adapter (imageRecall.ts:475). anima_inpaint_adapter is written to metadata only when an adapter was used, and recall treats a missing key as "no change". So recalling an Anima inpaint image made without the adapter keeps the currently selected adapter, and the next inpaint doesn't reproduce the image. Writing anima_inpaint_adapter: null on Anima inpaint/outpaint graphs that don't use one would fix it.
  3. The recalled anima_inpaint_adapter_weight isn't clamped (imageRecall.ts:424-435). Out-of-range metadata (for example 25) is applied as-is, and canvas validation then rejects every inpaint until the user edits the weight. Only hand-made or foreign metadata can carry such a value.
  4. Adapters with cond_in_channels: null are offered in the inpaint slot (controlValidation.ts:89). A 3-channel LLLite model installed before the field existed passes the check, and the backend logs "mask will be ignored" and produces a wrong image with no error. This matches legacy and the code documents it, so I'm noting it only.

Test gaps:

  • Native image-to-image encode size is asserted only for FLUX. SDXL, which reads the size from its noise node, has no size assertion.
  • Fit mode with an off-grid bbox (note 1) isn't tested.
  • The Python fixture covers the adapter only for outpaint; inpaint is covered by the TypeScript tests alone.

Not run: the frontend and Python test suites, and real-model execution.

…on recall

PiD Fit graphs on an off-grid bbox now resize the decoded image straight to the bbox instead of twice.
Recall All and Remix clear the Anima inpaint adapter when an Anima inpaint or outpaint record carries none,
and clamp a recorded adapter weight to the setting's range.
Covers SDXL Native encode size, the off-grid Fit resize, and the inpaint adapter in the backend fixture test.
@joshistoast

Copy link
Copy Markdown
Collaborator Author

Thanks for the review. Addressed in 9b892f1:

  1. Double resize in Fit mode: resizeCanvasOutputToBbox now retargets the PiD Fit resize to the bbox instead of adding a second img_resize. SDXL Fit with a 1000×700 bbox decodes at 4000×2816 and resizes once to 1000×700; metadata keeps the bbox size. Native and non-PiD graphs take the unchanged path. Test added.
  2. Stale adapter on Recall All / Remix: writing anima_inpaint_adapter: null turned out not to work, since CoreMetadataInvocation serialises with exclude_none=True and drops the key. Recall now follows the loras convention: on an anima_inpaint/anima_outpaint record an absent key means no adapter and clears the slot; elsewhere an absent key still means no change. A Python test runs the invocation on the fixture graphs and pins that the key is persisted only when an adapter ran.
  3. Unclamped recalled weight: clamped to the setting's range with the same helper the settings normaliser uses (25 → 10, −25 → −10).
  4. cond_in_channels: null adapters: left as is, per your note.

Test gaps: SDXL Native image-to-image now asserts the quarter-size encode; the off-grid Fit case is tested; the Python fixture carries an inpaint-with-adapter graph that goes through the same validation as the outpaint one.

One caveat noted in the description: legacy-made Anima inpaint images never recorded the adapter, so recalling one now clears the slot rather than keeping the current pick. Neither behaviour restores it.

@lstein
lstein enabled auto-merge October 17, 2026 01:00
@lstein
lstein merged commit ed5346d into main Oct 17, 2026
17 checks passed
@lstein
lstein deleted the feat/parity-canvas-pid-anima branch October 17, 2026 01:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs PRs that change docs frontend PRs that change frontend files python-tests PRs that change python tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants