the quality of video downgrade very much when calling pipe for 50 different prompts in wan 2.2 with lighting lora

[chaowenguo]

I (https://huggingface.co/datasets/chaowenguoback/wan?image-viewer=video-0-C91E732593BE818C54EA8F13CF4E0818D52B6588) use #12074 (comment) to generate a long video, using the current prompt and the last frame of video generated by the previous prompt. the beginning 12 prompt generate good videos, but the rest are completely unacceptable. pretty bad. I use pipe.enable_sequential_cpu_offload() and torch.cuda.empty_cache() after every pipe(prompt) call. Please help. the quality downgrade in 1:30

[Advait251206]

Hi @chaowenguo,

From your description, I would first investigate cumulative visual drift from feeding each generated video's last frame into the next generation, rather than assuming that the pipeline is gradually losing quality.

Your process is effectively:

1. Generate a segment from prompt 1.
2. Extract its final frame.
3. Use that frame as the input image for prompt 2.
4. Repeat this for all 50 prompts.

Even if each individual generation is reasonable, small errors in composition, subject appearance, lighting, or image detail can accumulate when every segment is conditioned on the previous segment's output. After many iterations, the model may be working from a frame that has drifted significantly from the original scene.

1. Check whether the degradation is caused by chaining frames

I would run these two tests before changing the model or LoRA configuration:

• Independent-generation test: Generate several prompts using the same original input image rather than the previous segment's last frame.
• Chained-generation test: Generate the same prompts using the previous segment's last frame, as in your current workflow.

Keep the resolution, number of frames, inference steps, guidance settings, LoRA weights, and seeds consistent where possible.

If independent generations remain visually consistent while chained generations deteriorate, that would strongly suggest that the repeated image-conditioning process is contributing to the problem.

2. torch.cuda.empty_cache() does not reset generation state

Calling torch.cuda.empty_cache() releases unused cached memory held by PyTorch's CUDA allocator. It does not reset the model weights, restore visual details in the input frame, or undo changes already introduced by the previous generation.

Likewise, sequential CPU offloading is a memory-management technique; it is not intended to improve or maintain visual quality between generations.

So I would not expect either of those operations to prevent visual drift caused by repeatedly reusing generated frames.

3. Try periodic anchor frames

Instead of using the last frame from every segment indefinitely, consider periodically returning to a carefully selected reference frame or generating new anchor frames.

For example:

• Use the previous segment's final frame for short transitions.
• After a few segments, use a selected high-quality frame as the new reference.
• For longer sequences, generate keyframes for major scene changes and build transitions between them.

This can help prevent small visual changes from accumulating across the entire sequence.

If your goal is to connect a known first and last frame, investigate a first-and-last-frame video generation workflow rather than relying exclusively on repeated image-to-video calls. Diffusers documents Wan first/last-frame generation here: [Wan pipeline documentation](https://huggingface.co/docs/diffusers/api/pipelines/wan).

4. Verify the Wan 2.2 LoRA configuration

If by “lighting LoRA” you mean the Wan 2.2 Lightning LoRA, check that the LoRA weights are loaded into the appropriate Wan 2.2 denoisers. Wan 2.2 uses a dual-transformer setup, and the Diffusers documentation notes that LoRA loading into the second transformer may require explicitly setting load_into_transformer_2=True.

Also verify that the LoRA strength, scheduler, inference steps, and prompt trigger words remain consistent across all calls. Avoid changing multiple settings at once while debugging.

Reference: [Wan 2.2 LoRA notes in the Diffusers documentation](https://huggingface.co/docs/diffusers/api/pipelines/wan).

5. A useful debugging check

Save the exact input image passed to every call, not just the resulting videos. Then inspect the sequence of input frames side by side.

If the inputs themselves progressively lose detail, change appearance, or drift away from the intended scene, that points toward accumulated conditioning drift. If the input frames remain consistent but output quality still drops, I would next compare the generation parameters and inspect whether the LoRA and scheduler configuration are identical for every call.

It would also help to share the loop that extracts and passes the final frame, along with the exact Wan 2.2 pipeline class, LoRA loading code, and inference parameters. Those details would make it easier to distinguish frame-conditioning drift from a pipeline or configuration issue.

Hope this helps narrow it down!

---

If this answer helped or pointed you in the right direction, I'd appreciate it if you could mark it as the accepted answer so it's easier for others with the same issue to find.

Also, if you found my contribution useful, I'd appreciate it if you could check out my GitHub profile, follow me, and star any repositories you find interesting.

GitHub: https://github.com/Advait251206