Model cache undercounts the Anima IP-Adapter's bundled LoRA with partial loading #10037

Summary

The model cache undercounts the Anima IP-Adapter's RAM use by about 26 MB, the size of its bundled cross-attention LoRA, whenever partial loading is enabled. Partial loading is the default on GPUs with dedicated VRAM. This was left open in a review nit on #9903.

Details

AnimaIPAdapter stores its bundled LoRA as a plain attribute, self._lora (a ModelPatchRaw, rank 32 bf16), in invokeai/backend/anima/ip_adapter.py. Because it isn't a submodule, it is missing from module.state_dict().

#9903 added an AnimaIPAdapter branch to calc_model_size_by_data (invokeai/backend/model_manager/load/model_util.py) that adds lora_patch().calc_size(). That figure reaches only one place: _make_room_internal(size) in model_cache.py, which runs when the model is first put into the cache.

When partial loading is on, the cache wraps the module in CachedModelWithPartialLoad. That wrapper's total_bytes() is the sum of the state dict's tensors, so it leaves out the LoRA. Every later use of the entry's size relies on total_bytes() and undercounts the adapter:

• the RAM budget registration (self._ram_budget.add_non_shared(wrapped_model.total_bytes(), ...));
• the eviction size sorting;
• the bytes_freed figures.

With partial loading off (CachedModelOnlyFullLoad), the passed size is used everywhere. That gives a small overcount instead: a full load to the device counts LoRA bytes that module.to() never moves, because the LoRA patch is applied to the transformer rather than moved with the adapter.

Why it wasn't fixed in #9903

CachedModelWithPartialLoad.total_bytes() also serves as the full-load VRAM target, via partial_load_to_vram(self.total_bytes()). Making it larger would break the "fully loaded" detection. That code is shared by every model, and the gap is small: about 26 MB on a ~500 MB adapter.

Suggested fix

Have the Anima IP-Adapter loader expose the bundled LoRA as its own cache entry, a ModelPatchRaw that the cache already knows how to size, instead of keeping it as an attribute of the adapter module. Then drop the AnimaIPAdapter special case from calc_model_size_by_data.

The cache would then count both parts correctly in both wrappers, and the LoRA's lifecycle would follow the cache like any other user LoRA.