Repository navigation
feat(canvas): restore PiD decoding and the Anima inpaint adapter on Canvas - #9805
Conversation
Canvas txt2img and img2img decode through PiD in fit and native modes, resolving the encode VAE from the PiD graph and encoding at the denoise size; inpaint/outpaint and Scale Before Processing keep the legacy rejections. Anima inpaint/outpaint can add the optional 4-channel ControlNet-LLLite adapter with a weight, fed the raster image and inverted mask through the shared collector, recorded in metadata and restored by Recall All and Remix.
…picker Compile tests pin the scaled outpaint mask and pre-noise image the Anima inpaint adapter reads, recall keeps an uninstalled adapter pick while restoring the weight, and a browser case covers the Anima-only slot, its 4-channel filter and the weight scrubber.
lstein
left a comment
There was a problem hiding this comment.
Reviewed at d7f8137. I found no blockers. I tried to break both halves: image-to-image encode size across every base, VAE source lookup (seamless, control, regional guidance, custom VAE on FLUX/SD3), Native/Fit geometry, the Scale Before Processing and inpaint/outpaint rejections, adapter settings lifecycle, mask polarity and sources, collector sharing with control layers, and leakage into non-canvas or non-Anima graphs. None of these turned up a defect. Both adapter snapshot graphs pass the backend's graph validation, and reverting each part of the change makes an existing test fail.
Non-blocking notes:
- PiD Fit mode resizes twice on an off-grid bbox (
pidGraph.ts:134, thenresizeCanvasOutputToBboxincompileCanvasGraph.ts:209). Example: SDXL Fit with a 1000×700 bbox snaps to 1000×704. The 4000×2816 decode is resized to 1000×704, then resized again to 1000×700. Legacy resizes once, straight to the final size. The output is valid, just slightly softer. - Recall All / Remix never clears a stale inpaint adapter (
imageRecall.ts:475).anima_inpaint_adapteris written to metadata only when an adapter was used, and recall treats a missing key as "no change". So recalling an Anima inpaint image made without the adapter keeps the currently selected adapter, and the next inpaint doesn't reproduce the image. Writinganima_inpaint_adapter: nullon Anima inpaint/outpaint graphs that don't use one would fix it. - The recalled
anima_inpaint_adapter_weightisn't clamped (imageRecall.ts:424-435). Out-of-range metadata (for example 25) is applied as-is, and canvas validation then rejects every inpaint until the user edits the weight. Only hand-made or foreign metadata can carry such a value. - Adapters with
cond_in_channels: nullare offered in the inpaint slot (controlValidation.ts:89). A 3-channel LLLite model installed before the field existed passes the check, and the backend logs "mask will be ignored" and produces a wrong image with no error. This matches legacy and the code documents it, so I'm noting it only.
Test gaps:
- Native image-to-image encode size is asserted only for FLUX. SDXL, which reads the size from its
noisenode, has no size assertion. - Fit mode with an off-grid bbox (note 1) isn't tested.
- The Python fixture covers the adapter only for outpaint; inpaint is covered by the TypeScript tests alone.
Not run: the frontend and Python test suites, and real-model execution.
…on recall PiD Fit graphs on an off-grid bbox now resize the decoded image straight to the bbox instead of twice. Recall All and Remix clear the Anima inpaint adapter when an Anima inpaint or outpaint record carries none, and clamp a recorded adapter weight to the setting's range. Covers SDXL Native encode size, the off-grid Fit resize, and the inpaint adapter in the backend fixture test.
|
Thanks for the review. Addressed in 9b892f1:
Test gaps: SDXL Native image-to-image now asserts the quarter-size encode; the off-grid Fit case is tested; the Python fixture carries an inpaint-with-adapter graph that goes through the same validation as the outpaint one. One caveat noted in the description: legacy-made Anima inpaint images never recorded the adapter, so recalling one now clears the slot rather than keeping the current pick. Neither behaviour restores it. |
Summary
Two Canvas generation paths the legacy frontend supported were missing from webv2.
PiD decoding on Canvas. The canvas compiler rejected every PiD mode ("not supported on the canvas yet"). It now compiles Canvas text-to-image and image-to-image, in both Fit and Native modes, on every base where Generate already supports PiD. Native image-to-image resizes the source to the quarter-size denoise resolution and decodes straight to the bounding box, as the legacy builders did; Fit keeps the full-size encode and resizes the decoded output. The encode VAE is resolved from the graph (the decode's VAE edge, or the model loader for the FLUX and SD3 decoders that take none). Inpaint/outpaint and Scale Before Processing that changes the processing size remain rejected with the legacy messages.
Anima inpaint adapter. Anima models get an optional four-channel inpaint/outpaint adapter slot (Anima LLLite models that are not three-channel) with a weight control. During Canvas inpaint and outpaint the graph adds the inverted gradient mask and an
anima_lllitenode fed by the resized/initial image (inpaint) or the infilled image (outpaint), joined to the shared control collector exactly as the legacyaddInpaint/addOutpaintbuilders wired it. Ordinary control layers still reject four-channel models. The adapter and weight are recorded in metadata and restored by Recall All and Remix.Graphs without PiD or without an adapter are unchanged.
Related Issues / Discussions
Follows up the legacy parity audit behind #9803.
QA Instructions
From
invokeai/frontend/webv2(Node 22.14, pnpm 10.12):pnpm check:releasepasses (lint, unit, browser, fixtures, performance, project-file journeys, accessibility). One browser suite unrelated to this change (gallery maintenance settings) timed out once under heavy machine load and passed on rerun. Python:pytest tests/app/invocations/test_frontend_anima_lllite_graphs.py tests/app/invocations/test_frontend_graph_node_types.pypasses (474 passed, 2 skipped).Follow-up at 9b892f1:
pnpm check:releasepasses again (lint, unit, browser, performance, project-file journeys, accessibility);pytest tests/app/invocations/test_frontend_anima_lllite_graphs.py tests/app/invocations/test_frontend_graph_node_types.py: 480 passed, 2 skipped.The mock backend has no Anima or PiD models, so UI behavior is covered by browser tests (
GenerateSettingsForm.browser.test.tsx: the Inpaint Adapter slot lists only four-channel models, the weight control appears once one is picked, and the slot is absent for non-Anima models). Real-model verification was not performed. With a real install:anima_inpaint_adapterandinvert_anima_inpaint_adapter_mask. Recall All on the result restores both.Review
No material findings. Reviewers confirmed the compiled graphs match the legacy wiring field for field on every PiD base and both adapter paths, and that non-PiD and non-adapter graphs are unchanged. Follow-ups from review: compile tests for scaled outpaint and noise-mask inpaint with the adapter, a recall test for an uninstalled adapter, and the browser test above.
Follow-ups from the second review: PiD Fit on an off-grid bbox resized the decoded image twice (to the grid size, then to the bbox); it now resizes once, straight to the bbox. Recall All and Remix did not clear a selected inpaint adapter when the image was made without one;
core_metadatadropsnullon write, so recall now treats an absentanima_inpaint_adapteron ananima_inpaint/anima_outpaintrecord as none, and a Python test pins that the invocation persists the key only when an adapter ran. A recorded adapter weight is clamped to the setting's range on recall. Tests added for SDXL Native encode size, the off-grid Fit resize, and an inpaint-with-adapter graph in the backend fixture. Adapters withcond_in_channels: nullare still offered, as in legacy.Limitations: no real-model execution; a legacy-made Anima inpaint image never recorded its adapter, so recalling one clears the adapter slot (before, it kept whatever was selected; neither restores it); the Scale Before Processing rule compares grid-clamped sizes (webv2's auto scaling only grows), so an already-optimal box is accepted where legacy's auto mode would have rejected it.
Compatibility / Rollout
The new settings are optional with defaults and normalize on project load; no project-document migration. No API changes.
Checklist
What's Newcopy (if doing a release after this PR)Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is disabled.