2.4k
497
Having encountered extremely annoying memory issues trying to create animations with just 8-Gig of VRAM on my RTX3070, I started tinkering around with different ways to create animations. This workflow is the result of one such experiment.
The workflow is divided into two parts, you can do one or the other.
1. Create frames: Creating frames uses ControlNet to extract pose and lineart from a video file. My testing was all done with TikTok videos. The WF uses a mask to get rid of the background which prevents the lineart from affecting your scene, but uses it to help improve the rendering of humans. This part of the workflow will create a folder of individual frames for your final animation.
2. Assemble Video: This section will assemble the many frames you created into a finished video.
Please read the annotations for further details.
I have added a turbo mode for the rendering, you'll need to download and install this LoRA:
https://civitai.com/models/216190/lcmandturbomix-lora-only-12mb-8-step-sampling-effect-is-superior-to-using-lcm-or-turbo-alone
If you aren't getting satisfactory results with the LoRA, bypass it, but remember to change the Steps and CFG in both the KSampler and the Face Detailer.
I can't offer any support on running this, it was simply an experiment for me, but it's a good starting point for exploring ComfyUI animation when you are limited to only 8gb of VRAM. Oh and go take a nap once you've started it. It can take a bit.
I hope you find this useful.