Repository navigation
Replies: 3 comments
|
Hi @chaowenguo, From your description, I would first investigate cumulative visual drift from feeding each generated video's last frame into the next generation, rather than assuming that the pipeline is gradually losing quality. Your process is effectively:
Even if each individual generation is reasonable, small errors in composition, subject appearance, lighting, or image detail can accumulate when every segment is conditioned on the previous segment's output. After many iterations, the model may be working from a frame that has drifted significantly from the original scene. 1. Check whether the degradation is caused by chaining framesI would run these two tests before changing the model or LoRA configuration:
Keep the resolution, number of frames, inference steps, guidance settings, LoRA weights, and seeds consistent where possible. If independent generations remain visually consistent while chained generations deteriorate, that would strongly suggest that the repeated image-conditioning process is contributing to the problem. 2.
|
|
That's error accumulation, not something wrong with the 50th call. Every iteration decodes the last frame to pixels and re-encodes it through the Wan VAE, and Lightning's few-step outputs already carry extra artifacts. Each clip inherits the previous one's defects, so after a dozen or so the drift becomes visible. Two things actually help. Pass latents forward instead of a decoded frame if your pipeline allows it, skipping the VAE round trip per clip. And re-anchor periodically, every N clips run one segment at higher steps without the Lightning LoRA to reset the drift. Quick check to confirm the diagnosis: save the frame you feed into each call. If the input frame for clip 13 is already degraded, it's VAE + lora compounding, not a prompt problem. — |
|
This is error accumulation from the pixel round-trip, and it compounds exactly the way you describe: the first few prompts look fine, then every later clip inherits the artifacts of everything before it. Two mechanisms stack up in your loop:
What actually helps, in order of effectiveness:
If you want a sanity check that isolates the mechanism: run the same 50-prompt chain but skip the pixel round-trip (latent-chained), and separately run one 50×-longer single generation. If the latent-chained version holds up where the pixel-chained one degrades, that confirms the VAE round-trip is the culprit and not something specific to prompt 13. |
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
I (https://huggingface.co/datasets/chaowenguoback/wan?image-viewer=video-0-C91E732593BE818C54EA8F13CF4E0818D52B6588) use #12074 (comment) to generate a long video, using the current prompt and the last frame of video generated by the previous prompt. the beginning 12 prompt generate good videos, but the rest are completely unacceptable. pretty bad. I use pipe.enable_sequential_cpu_offload() and torch.cuda.empty_cache() after every pipe(prompt) call. Please help. the quality downgrade in 1:30
All reactions