[lstein]

Summary

Anima models had no way to carry a character from a reference image into a new scene. This adds support for the SigLIP2-based LuciferTC/Anima-IP-Adapter (ip_adapter-Character_Reference-10.safetensors). It follows the reference ComfyUI implementation (ComfyUI-Anima_IP-Adapter).

• Backend
  • Adds a model config and loader, IPAdapter_Checkpoint_Anima_Config.
  • Fixes the SigLIP probe so it accepts google/siglip2-base-patch16-512, whose config has no architectures.
  • New anima_ip_adapter node. It encodes a reference with SigLIP2 and auto-installs the encoder if it is missing.
  • anima_denoise 1.9.0 gains an ip_adapter input:
    • The per-block IP term is added after each DiT block.
    • Only the conditional pass gets the IP term.
    • The adapter's bundled cross-attention LoRA is applied once per model.
  • Several images that share an adapter are attended together: their tokens are concatenated. Each image keeps its own Strength (0–2) and step window.
• Generate / Canvas (webv2)
  • The IP-Adapter reference card now works for Anima, with up to 4 references.
  • Anima cards show a single Strength control plus Active steps. Mode, Style variant and CLIP Vision are hidden.
  • A reference with Strength 0 is not sent to the graph.
  • References are only offered on the Qwen3 (2B / 2.9B) releases. Anima-3.8B gets none.
  • Recall works through ref_images metadata.
• Starter model: "Anima IP-Adapter (Character Reference)", with the SigLIP2 encoder as a dependency-only starter.
• Docs: the Anima page and the Generate panel reference table.

What transfers, from GPU runs on Anima Base 1.0 and PerfectRSBmix Mu:

• Character: strong transfer. Hair, outfit and accessories carry over from a prompt that doesn't describe them. Fine details such as heterochromia do not.
• Several images of one character: these combine well. Different characters blend their attributes, even with a 2girls prompt.
• Stylized references: these carry their composition and content, not style alone.
• Cost: no measurable time added per generation (~26–35 s with or without).

Related Issues / Discussions

None.

QA Instructions

1. Install the Anima IP-Adapter (Character Reference) starter. SigLIP2 installs with it.
2. Select an Anima 2B model. Add a reference image of an anime character and use a prompt that doesn't describe it, e.g. 1girl, solo, sitting in a cafe, holding a coffee cup. Set Strength to 1.0–1.5 and invoke. The character's hair, outfit and accessories should carry over.
3. Setting Strength to 0 should give output identical to having no reference.
4. Add up to 4 references, then try a 5th. The 5th is refused.
5. Switch to an Anima-3.8B model. Reference images are cleared and the section is hidden.

Checks, run on the rebased commit:

• uv run --no-sync pytest -n 4: 14,542 passed, 205 skipped, 7 xfailed
• uv tool run ruff@0.11.2 check and format --check: clean
• webv2: pnpm lint, pnpm test, pnpm check:release: pass (10,326 unit tests; release gate with 2,923 accessibility/browser tests)
• GPU on a W7900 (ROCm), driving the graph from the built UI in headless Chromium:
  • Strength 0 vs no adapter: max pixel difference 0 on both models.
  • Character, style and multi-image comparisons gave the transfer results listed above.
• Not tested:
  • Anima-2.9B. The adapter is mapped onto its 40 blocks the same way LoRAs are, but this was never run.
  • The Canvas path in a real browser. It uses the same graph builder and is covered by unit tests.
  • CUDA.

Review

Findings that were resolved:

• Model-switch re-targeting: an SD reference with no adapter and a negative weight now re-targets to a valid Anima reference.
• Strength 0: the encoder is skipped entirely, so it never triggers the encoder download.
• Depth check: an adapter deeper than the transformer is refused before any patching.
• Step window validation: begin_step_percent > end_step_percent is rejected.
• No-adapter memory: a working-memory increase on the no-adapter path was removed.
• Docs: they now state the multi-image and step-window behaviour.

Remaining limitations:

• Encoder precision: the SigLIP2 encoder runs fp32 activations on session-precision weights. The shared SigLIP loader is unchanged because FLUX Redux uses it too. On random weights this measured about 0.4% relative L2 error in bf16.
• Strength 0 and the bundled LoRA: Strength 0 drops the IP term and the bundled LoRA, so 0 matches no adapter exactly. The reference keeps the LoRA on at 0.
• Active steps: they gate the IP term only; the LoRA stays on for the whole run.
• Low Strength with several images: a low but non-zero Strength on an extra image still takes attention from the others (the docs say to remove it instead).
• Image count: the backend doesn't cap how many images a hand-built workflow sends; the UI caps it at 4.
• Encoder auto-install: same behaviour as FLUX Redux and the SD IP-Adapter. It can run for a non-admin user, and it can hold a borrowed idle GPU while it downloads (wait_for_job with a 600 s timeout).
• Switching from 5 SD/FLUX references: switching to Anima keeps all 5. Generation is then blocked with "at most 4" until one is removed.
• Existing bug, recorded only: webv2 sends method to flux_ip_adapter, which has no such field, so the FLUX card's Mode control does nothing. It's listed in SILENTLY_IGNORED and isn't fixed here.

Compatibility / Rollout

• anima_denoise goes 1.8.0 → 1.9.0 with a new optional input, and anima_denoise_meta goes to 1.1.0. Existing graphs and workflows validate unchanged.
• The model config union gains a member and the SigLIP probe accepts more configs. Existing models identify the same.
• openapi.json, schema.ts and the webv2 capabilities fixture are regenerated.
• License: the adapter weights are under the CircleStone Labs Non-Commercial License v1.0 according to the upstream GitHub README; the HF card says "unknown". The code is Apache-2.0 and SigLIP2 is Apache-2.0. The starter description and the docs state this.

Checklist

• The PR has a short but descriptive title, suitable for a changelog
• Meaningful regression coverage added / updated where needed; obsolete tests/code removed
• Persisted-state and API changes include required migrations / compatibility validation
• Relevant performance/efficiency opportunities considered; material claims have evidence
• Material review findings resolved and relevant checks rerun
• Documentation added / updated (if applicable)
• Updated What's New copy (if doing a release after this PR)

🤖 Generated with Claude Code

---

^{Need help on this PR? Tag @codesmith-bot with what you need.}

[Pfannkuchensack]

Approve with nits. The math matches the reference, other bases are unchanged, and none of the points below block a merge.

What I verified

• Math against the reference nodes.py:
  • The IP query is the block's flattened, adaLN-modulated cross-attention input, run through q_proj and then q_norm.
  • K/V come from ip_k_proj/ip_v_proj, with no key norm and no output projection.
  • The gate is adaln_ip on the normed t_emb, and the term is added after the MLP.
  • Only the positive pass gets the term. In the reference, the zero null tokens switch it off through scale_mask.
  • Strength scales the fp32 tokens before the biased projections.
  • Only the cross-attention part of the bundled LoRA is applied, at alpha/rank = 1.
  • Preprocessing letterboxes to 512 px. The _reference_forward transcription in the tests is faithful.
• Real checkpoint: I read the safetensors header of ip_adapter-Character_Reference-10.safetensors. It has 728 BF16 tensors, blocks 0–27, both flags False, lora_alpha = lora_rank = 32 and no null_tokens, so it matches anima_ip_adapter_format.py.
• Classification: only IPAdapter_Checkpoint_Anima_Config claims the real layout. The widened SigLIP probe matches google/siglip2-base-patch16-512's config.json verbatim, and it still checks the class name when a config has one.
• Other bases: getReferenceImageConfigSupported gives the same answers for sd-1/sdxl/flux as before, and the SD/FLUX graph fields are unchanged. Regional prompting only patches cross_attn.forward, so it doesn't affect the IP term.
• Checks:
  • The focused pytest suites pass on the head and with current main merged in. They cover the Anima backend and denoise, the IP-Adapter, configs, features, starter models, fp8 capability and graph node types.
  • On the merged tree, main's union-collector validation (#9853) accepts anima_ip_adapter → collect → anima_denoise.ip_adapter.
  • openapi.json regenerated on the head is identical to the committed one.
  • ruff is clean.

Merging main: the only conflict is one hunk in generateGraphNodeTypes.json, in fieldsByNodeType.denoise_latents.inputs. Main adds "latents" (SDXL Refiner) and this PR adds "ip_adapter", so take the sorted union. Main also changed openapi.json and schema.ts since your base. They auto-merge, but please regenerate both rather than trusting the textual merge.

I didn't run the webv2 vitest/browser suites or a GPU generation; for those I'm relying on your runs.

[lstein]

@Pfannkuchensack thanks for the review! All seven nits are addressed in 110f44b; the open part of the cache-size nit is explained in its thread.

Merging main: I built on your merge and then merged current main (2c1c5ae). I regenerated openapi.json, schema.ts and the graph node-type snapshot rather than trusting the text merge. The snapshot has the union you described: main's latents plus this PR's ip_adapter.

Checks:

• Full pytest passes. The only failures were 3 test_app_info tests, caused by a local webv2 dist/ build; they pass without it.
• ruff is clean.
• webv2: lint, 11,629 unit tests and 3,496 browser tests all pass.
• The bundle-size budget in check:release fails, but main fails it too (otherRawBytes 242,974 vs a 240,751 limit). This branch adds 324 bytes on top, so I didn't re-baseline it here.