feat(ltx2): run LTX-2 video self-attention on SageAttention when enabled

[Pfannkuchensack]

Note

Stacked PR 7 of 8: merge after #9663. It targets main, so until the PRs before it merge, its diff also shows their commits and the merges that carry them forward. This PR's own changes are two commits, 9507eccfb6 (the scope) and b3c3e4300a (the audio result in the docs).

Summary

With attention_backend: sage, LTX-2 denoising enters the SageAttention scope from #9657. Its video self-attention runs on SageAttention; the audio attention is below the size threshold and the text cross-attention passes a mask, so both stay on PyTorch SDPA. The default auto is unchanged. The docs page lists LTX-2 under Supported models.

Related Issues / Discussions

Stacked on #9657 (SageAttention backend) and #9663, the PR before it.

QA Instructions

LTX-2.5 distilled int8, model loaded, three arms per seed as described in #9657:

	1248×704, 121 frames
Step, default SDPA	5263 ms
Step, cuDNN-first SDPA	4726 ms
Step, SageAttention	4178 ms (−21 %)
Whole generation	72.0 → 63.4 s (−12 %)
Peak VRAM in the denoise scope	unchanged
Per-frame LPIPS against default SDPA, median (cuDNN-first)	0.142 (0.090)
Flicker ratio against default SDPA	1.001

• Per step, the 48 video self-attention calls run on SageAttention. Audio self-attention and the two audio–video cross-attentions (3 × 48) are below the size threshold, and the text cross-attention (2 × 48) passes a mask; all of these stay on SDPA. Dropping the text mask where it masks nothing is a possible follow-up.

• Accuracy on real activations, every call of step 0 (130 calls): worst cosine 0.9993, relative L2 0.027.

• Frames were compared per frame by LPIPS and by frame-to-frame flicker.

• Audio. It reads the video through cross-attention, so it changes even though its own attention stays on SDPA. Measured with the code of this PR on 8 clips (4 prompts × 2 seeds, 1248×704, 121 frames; two prompts with clear sync events: four hand claps, a spoken sentence), against default SDPA, with cuDNN-first as the exact-kernel noise floor:

  	SageAttention	cuDNN-first (floor)
  Log-mel distance, median (max)	0.92 dB (1.58)	0.74 dB (1.36)
  Onset-envelope cosine, median	0.980	0.986
  Shift against the SDPA audio	0 ms in all 8	0 ms
  Loudness	within ±0.2 dB	within ±0.26 dB
  Clipping / silence	none	none

  Sync to the picture: on the clap and speech clips, motion and sound onsets correlate clearly (0.4–0.65), and the lag between them is identical in all three arms on every clip, so SageAttention does not move the sound against the picture. On the other two prompts there are no sync events to measure. Nobody listened to the clips; the numbers are the evidence.

• At 1792×1024 all three arms ran out of memory on 24 GB without partial loading, so that size was not measured.

• The timings and LPIPS come from the measurement prototype, which still used sageattn()'s default K smoothing; the selection feat(attention): add opt-in SageAttention backend for diffusion models #9657 ships is faster per call.

To try it: install SageAttention as the docs page from #9657 describes, set attention_backend: sage, restart and generate with LTX-2. The first generation logs SageAttention ... serves diffusion-model attention on cuda:0 (...) with the GPU and kernel name, and with log_level: debug every generation logs how many calls ran on SageAttention and why the others did not. With attention_backend: auto the output is bit-identical to main.

Checks: ruff clean; pytest tests/app/invocations passes with the whole stack applied (2090 passed). No call-site test was added: this change only enters the scope, whose behavior #9657 tests.

Review

Resolved: the audio was not compared at first. It now is (see QA), and the docs state the result.

Compatibility / Rollout

Merge after #9663. No API or persisted-state change; only attention_backend: sage behaves differently.

Checklist

• The PR has a short but descriptive title, suitable for a changelog
• Meaningful regression coverage added / updated where needed; obsolete tests/code removed
• Persisted-state and API changes include required migrations / compatibility validation
• Relevant performance/efficiency opportunities considered; material claims have evidence
• Material review findings resolved and relevant checks rerun
• Documentation added / updated (if applicable)
• Updated What's New copy (if doing a release after this PR)

[JPPhoto]

This is now fixed, thanks:

• invokeai/app/invocations/ltx2/ltx2_denoise.py:620: the scope wraps joint denoising; invokeai/backend/ltx2/denoise.py:693 feeds both modalities through the transformer and uses both predictions. Changing video self-attention can therefore alter audio through cross-modal attention, despite audio attention itself staying on SDPA. The PR says its QA did not compare audio. Effect: Sage-enabled video/audio generations may produce degraded or desynchronized sound. Likelihood: normal when Sage is enabled for LTX-2. Recovery: set attention_backend: auto. Test: compare decoded audio quality and A/V sync against SDPA on fixed-seed clips; document the result in docs/src/content/docs/configuration/Optimization/sage-attention.mdx.

[JPPhoto]

Approved.