8-Gig VRAM Animation Creator
created a year ago
controlnet
lora
video
face
ipadapter
62 nodes

2.4k

497

Outputs
Description

Having encountered extremely annoying memory issues trying to create animations with just 8-Gig of VRAM on my RTX3070, I started tinkering around with different ways to create animations. This workflow is the result of one such experiment.

The workflow is divided into two parts, you can do one or the other.

1. Create frames: Creating frames uses ControlNet to extract pose and lineart from a video file. My testing was all done with TikTok videos. The WF uses a mask to get rid of the background which prevents the lineart from affecting your scene, but uses it to help improve the rendering of humans. This part of the workflow will create a folder of individual frames for your final animation.

2. Assemble Video: This section will assemble the many frames you created into a finished video.

Please read the annotations for further details.

I have added a turbo mode for the rendering, you'll need to download and install this LoRA:

https://civitai.com/models/216190/lcmandturbomix-lora-only-12mb-8-step-sampling-effect-is-superior-to-using-lcm-or-turbo-alone

If you aren't getting satisfactory results with the LoRA, bypass it, but remember to change the Steps and CFG in both the KSampler and the Face Detailer.

I can't offer any support on running this, it was simply an experiment for me, but it's a good starting point for exploring ComfyUI animation when you are limited to only 8gb of VRAM. Oh and go take a nap once you've started it. It can take a bit.

I hope you find this useful.

Built-in nodes
Custom nodes
CR SDXL Aspect Ratio
MaskToImage
OpenposePreprocessor
ControlNetApply
PreviewImage
LoadImage
Image Remove Background Rembg (mtb)
ImageCompositeMasked
ControlNetLoader
ControlNetLoaderAdvanced
CheckpointLoaderSimple
LoraLoader
LineArtPreprocessor
VHS_VideoCombine
FILM VFI
VHS_BatchManager
LoadFramesFromFolder [n-suite]
IPAdapterUnifiedLoaderFaceID
CLIPTextEncode
IPAdapterAdvanced
VAEDecode
LoadVideo [n-suite]
VAEEncode
Seed Everywhere
UltralyticsDetectorProvider
BboxDetectorSEGS
CR Image Output
KSampler
DetailerForEach
Models
3
0
0
0