fix(gguf): avoid device over-reservation for host decoding

[wunianze666-netizen]

Summary

Correct GGUF Linear device working-memory reservations for types decoded through the NumPy host fallback. IQ4_NL/IQ4_XS currently receive the worst-case device-kernel intermediate allowance (8 bytes per element), although that route decodes on the host and transfers only its cast result.

Use the decoder's existing dispatch registries to distinguish host fallback from torch-compatible/device-kernel types. Preserve the packed input, compute-dtype result and maximum-single-layer accounting. Existing kernel table entries are unchanged; newly registered device kernels without a measured entry still receive the conservative 8-byte allowance.

This removes an incorrect reservation, not measured allocated VRAM or a demonstrated speed gain. No decoder, numerical output, kernel, dtype, API or stored format changes.

Related Issues / Discussions

Closes #9718

QA Instructions

On main 358257bca6acb9268c89d61c052c729c5c6a1a0a, the new focused cases produce 6 failed, 1 passed. Both IQ formats decode real nonzero packed blocks; only the byte-reservation assertions fail. For example, 512 IQ4_NL elements with fp16 compute reserve 5,408 bytes instead of 1,312. After the patch the same seven cases pass. The control registers an unmeasured device kernel and retains its conservative bound.

Commands below use the existing test environment and repository root. The recorded Windows runs set application roots, caches and temporary paths to isolated task directories, restrict BLAS threads to one and disable network model downloads.

python -m pytest tests/backend/quantization/gguf/test_ggml_tensor.py -o addopts= -p no:cov -n 0 -q -k 'host_dequant_reservation or new_device_dequant_kernel'
7 passed, 21 deselected, 4 existing warnings

python -m pytest tests/backend/quantization/gguf tests/backend/quantization/test_int8_convrot.py::TestThePerForwardDequantTransient tests/app/invocations/test_flux1_denoise_working_memory.py -o addopts= -p no:cov -n 0 -q
43 passed, 8 skipped, 4 existing warnings

ruff check invokeai/backend/quantization/gguf/ggml_tensor.py tests/backend/quantization/gguf/test_ggml_tensor.py
All checks passed!
ruff format --check invokeai/backend/quantization/gguf/ggml_tensor.py tests/backend/quantization/gguf/test_ggml_tensor.py
2 files already formatted
git diff --check

CPU-only execution covers IQ4_NL/IQ4_XS at fp16, bf16 and fp32, exact decoded outputs/shapes/dtypes, both owning and shared estimators, existing Q8/int8/mixed-layer behavior and the FLUX.1 reservation wire. No bf16 matmul is introduced. Eight CUDA-dependent cases are skipped; no local GPU allocation benchmark, full-model generation, full repository suite or build is claimed. The W7900 measurements in #9718 belong to its maintainer author, not this submission.

Review

No material review findings remain on the frozen candidate. The submission changes only reservation accounting; it does not change the decoder or generated values.

The host float32 decode still consumes host RAM; this change corrects the device reservation only. The existing non-IQ kernel estimates and their measurement precision are deliberately outside scope. A separate per-IQ lookup was rejected because it would diverge when a type gains a device kernel.

AI-assisted tooling was used to inspect the maintainer-reported path and develop the patch and tests. Listed results were executed; unavailable hardware checks are explicit.

Checklist

• The PR has a short but descriptive title, suitable for a changelog
• Meaningful regression coverage added / updated where needed; obsolete tests/code removed
• Persisted-state and API changes include required migrations / compatibility validation (not applicable; no stored/API changes)
• Relevant performance/efficiency opportunities considered; material claims have evidence (reservation accounting only, no measured VRAM/speed claim)
• Material review findings resolved and relevant checks rerun
• Documentation added / updated (if applicable) (dispatch comment updated)
• Updated What's New copy (if doing a release after this PR) (not a release PR)

---

^{Need help on this PR? Tag @codesmith-bot with what you need.}