Repository navigation
feat(canvas): noise adjustment for film grain - #9933
Conversation
Adds a Noise raster/group adjustment with Gaussian or Uniform distribution, monochromatic or color grain, amount, and a re-rollable seed. Noise is a deterministic function of position, so display, generation, export, merges and PSD share the same grain. Layer bakes and PSD now apply a layer's own adjustments before resampling, matching the display.
JPPhoto
left a comment
There was a problem hiding this comment.
Merge blockers
-
[P1 · Must fix] Merge Down drops Noise.
mergeLayerController.ts:139composites raw cache pixels; the reducer removes the layer adjustments. Impact / recovery: Grain disappears from either adjusted layer; undo is the recovery. Test: Add a Noise-specific pre/post Merge Down pixel comparison. Fix: Bake adjustments before compositing. If skipped: Merge Down violates the PR’s promise that merges preserve grain. -
[P1 · Must fix] Older builds can erase the whole adjustment stack.
canvasMigration.ts:125accepts Noise, while the writer still emits schema 3 or 4. An older client can drop the unknown stack on load, then autosave that loss. Impact / recovery: Permanent after the save. Test: Open and save a Noise document with an older schema reader. Fix: Mark Noise documents with a schema version older builds refuse. The PR documents this limitation, but it remains a data-loss path.
Other findings/issues
-
[P2 · Should fix] Rasterize changes the Noise pattern.
rasterizeLayerController.ts:86transforms raw parametric pixels, resets the transform, then keeps the adjustment. Noise runs on a new pixel grid. Impact / recovery: Grain changes after rasterizing a transformed text, shape or gradient layer; undo recovers it. Test: Compare pixels before and after rasterizing a transformed noisy layer. Fix: Apply Noise before baking the transform. -
[P2 · Should fix] Text noise can exceed its memory reservation.
rasterComposite.ts:247reserves from estimated text bounds, while rasterization uses browser-measured bounds. Wide text under heavy downscaling can allocate a larger scratch region than reserved. Impact / likelihood: Possible budget overrun or browser allocation failure with wide text and tight memory. Test: Use text whose measured bounds exceed the estimate. Fix: Reserve from actual layer bounds or a conservative text bound.
Noise gains a 1–8 px grain size (coordinate-pure Gaussian-correlated samples), a midtones response that fades grain to none at black and white, and a chroma grain on OKLab a/b that keeps lightness. Re-records the editor source-owner set and canvas route bytes for the larger kernel.
Any noise adjustment, enabled or not, raises the canvas and the project floor to 5 and never lowers it, so older builds refuse the project instead of dropping the adjustment stack and saving the loss.
Merge Down bakes both layers' stacks layer-local before compositing; Rasterize bakes a noise stack before the transform and drops it. Composites reserve measured text scratch past its estimate or refuse as over budget.
Covers grain size, tonal response and chroma, how Merge Down and Rasterize bake adjustments, the noise kernel's cost and region-independence invariants, and that older versions refuse projects with noise.
…ustment # Conflicts: # invokeai/frontend/webv2/performance/browser-baseline.json # invokeai/frontend/webv2/src/workbench/projects/projectFile.test.ts
JPPhoto
left a comment
There was a problem hiding this comment.
Merge Down, Rasterize, and text-reservation fixes hold. But...
Merge blockers:
- P2:
invokeai/frontend/webv2/src/workbench/projects/durableSyncedPersistence.ts:1465-1473: A legacy whole project with repeated workflow IDs can save Noise as canvas schema 5 without raisingminimum_canvas_schema_version. The backend preserves the old floor, so older builds see the project as compatible until they try to open it.
Effect: The saved compatibility floor breaks the PR's promise to raise it to 5.
Likelihood: Every affected legacy whole project; uncommon overall.
Recovery: Save again after the floor is raised.
Test: Save Noise in the repeated-ID whole-project fixture; assert the request and stored floor are 5, and an older client is refused.
Other findings/issues:
- P2:
invokeai/frontend/webv2/src/workbench/canvas-engine/render/rasterComposite.ts:351-365: When a layer is downscaled, nonlinear color adjustments can run after resampling. Generation supplies no adjusted surface (invokeai/frontend/webv2/src/workbench/canvas-operations/generationComposite.ts:183-193), so its pixels can differ from the display, despite the PR's promise that adjustments run before resampling on every path.
Effect: The generated base image can differ from the canvas preview.
Likelihood: Any generated composite with a downscaled layer using a nonlinear color adjustment.
Recovery: Apply these adjustments to layer-local pixels before resampling, or narrow the stated promise.
Test: Compare display and generation pixels for a downscaled raster with a gamma or levels adjustment.
A legacy project kept whole because its workflow repeats an item id now sends the raised canvas floor like split saves do, so saving Noise into it stores floor 5 and older clients refuse it.
JPPhoto
left a comment
There was a problem hiding this comment.
Merge blockers:
- P2:
invokeai/frontend/webv2/src/workbench/canvas-engine/engine.ts:2552-2553: On an export cache miss,getAdjustedCacheSurface()builds a full layer-sized adjusted surface.exportRasterCompositereserves only the output and sampled scratch (invokeai/frontend/webv2/src/workbench/canvas-engine/exportRasterComposite.ts:57-59). Exporting a small region of a large downscaled noisy layer can therefore allocate beyond its reservation; the budget controller records the cache but cannot refuse or extend that lease.
Effect: Export may exceed the raster budget or fail allocation instead of returningover-budget.
Likelihood: Medium for large layers, small export regions, and tight budgets.
Recovery: Retry after cache eviction; document state is unchanged.
Test: Export a small region of a large downscaled noisy layer with no adjusted cache; verify it uses only the reserved sampled scratch path.
Export reuses the display's adjusted copy only when it is current and never builds a layer-sized one, so a cache miss adjusts just the reserved sampled region.
Main's growth and the Noise adjustment together exceed the 1% byte allowance on the editor build and on canvas activation.
Summary
Adds a Noise adjustment for raster layers and groups, aimed at film grain.
Noise is a deterministic hash of seed, pixel position and channel. A layer's grain lives in layer-local pixels and moves, scales and rotates with the layer; a group's grain is anchored to document pixels. Adjusting any sub-rectangle at its own origin gives exactly the pixels a whole-surface pass gives there, at every size, so partial redraws, exports, bakes and PSD all match the canvas.
A layer's own stack runs on layer-local pixels before resampling on the display, in merge/crop/copy bakes and in PSD. Generation and composite export do the same whenever the stack contains noise, and otherwise unless the layer is downscaled: a color-only stack on a downscaled layer, with no current display copy to reuse, is adjusted after resampling so the work stays bounded by the output size. Noise never takes that shortcut, because resampling must average its grain exactly as on screen. Merge Down bakes both layers' stacks into the merged pixels, which also fixes color adjustments being dropped on merge. Rasterize bakes a stack that contains noise into the source render before the transform and removes it, so the grain stays where it was.
Related Issues / Discussions
Tonal response, grain size and chroma grain follow ideas from the FilmGrain nodes in JPPhoto's cinematic-effects-pack.
QA Instructions
pnpm check:releaseininvokeai/frontend/webv2: lint, unit (10,904), browser, fixtures, architecture and browser performance, project-file journeys and accessibility all pass. Across three full runs, the browser suite failed one different, unrelated test each time (hotkey modal, workflow style-preset picker, gallery starred strip) under machine load; each passes 3/3 alone and none is touched by this branch.pnpm dev:mock):architecture-baseline.jsonaddsrender/thumbnail.tsto the editor's source-owner set. The bundler moved that small, side-effect-free module from a lazy chunk into an existing eager one after the kernel grew; no import changed and request counts are unchanged.browser-baseline.jsonraises only the byte figures that failed on the tree merged with main, to their measured values, since main's growth and the larger kernel together exceed the allowance: editor-canvas activated largest asset 305,523 → 311,154, editor-canvas script 4,718,489 → 4,767,087 and total 5,014,895 → 5,065,152, and editor-workflow script 4,644,510 → 4,691,655 (the kernel sits in a chunk every editor route loads). Timings are unchanged.Review
Resolved:
Remaining limits:
Compatibility / Rollout
noisearm to the persisted adjustment schema. Existing documents load unchanged..invkexport, as made by a newer version, instead of loading it without the stack. The backend needed no change; its existing floor check refuses clients below the floor.Checklist
What's Newcopy (if doing a release after this PR)