What this workflow does

You can easily transform any boring video or clip and create a fantastic animation, completely change objects or change scenes. It is ideal to create reels in Instagram or Tik Tok videos.

It is a relatively simple workflow that uses the new RAVE method in combination with AnimateDiff.

🎥 Video demo link

👉 Tutorial (incl. resources): https://youtu.be/4826j---2LU

IHow to use this workflow (download)

•The tutorial contains more details in how to get the different resources

•Download the workflow and the resources (videos) if you want to follow the tutorial

•Install custom nodes via custom manager

•Install required models (checkpoints, ControlNet, AnimateDiff)

•Update ComfyUI and restart

•Check that the models are properly loaded (check that the names are the same as your model names)

•Do a test run with a low number of frames (16) to check everything works

•Adjust your settings, change models, add CN's, add IP Adapters, etc., until you get satisfactory results

•Run the complete animation: beware of VRAM use of RAVE KSampler.

Tips about this workflow

•This workflow is setup to work with AnimateDiff version 3. For other versions, it is not necessary to use the Domain Adapter (Lora). You only need to deactivate or bypass the Lora Loader node.

•Unsampling originally works with Depth, but with LooseDepth you can achieve bigger transformation of objects and scenes.

•The workflow works good with other controlnets besides ControlGIF.

•IP Adapter also provides nice consistent results when applied with the first image generated from the RAVE KSampler.

•I did not manage yet to get it working nicely with SDXL, any suggestion/trick is appreciated.

•The RAVE Ksampler also uses quite VRAM. I manage to process 96 frames with a 4090 24 GB with SD1.5 type of videos.

Additional information

•RAVE method: https://rave-video.github.io/

•Rave ComfyUI node: https://github.com/spacepxl/ComfyUI-RAVE?tab=readme-ov-file

•AnimateDiff Evolved: https://github.com/Kosinkadink/ComfyUI-AnimateDiff-Evolved

•AnimateDiff version 3: https://github.com/guoyww/AnimateDiff an