Motion Brush Replica with Grounding Dino Support
created a year ago
upscale
animatediff
lora
video
77 nodes

5.4k

1.0k

Outputs
Description

Hi, and welcome to my modified version of House of Dim's Motion Brush workflow (https://www.youtube.com/watch?v=l7g58GrJIdE)

I have modified this workflow to use either hand drawn masks or Grounding Dino Segment Everything.

I have also modified this workflow to use RGThree's Bookmarks, Fast Groups Bypasser, and Use Everywhere's Anything Everywhere nodes.

If you would like to see the physical links between the Anything Everywhere nodes right click on a blank spot on the canvas and select "Show UE links" or if you would like a fresh plate of spaghetti then right click on the blank canvas and select "Convert all UE's to real links"

🚨🚨🚨 Convert all UE's to real links can be undone by pressing ctrl+z 🚨🚨🚨

Grounding Dino is the selected mask method by default (In the Mask Generation group) This allows you to set a prompt in it for the subject and then continuously generate new images with the same subject type without having to manually draw a new mask each time.

image showing how Dino works

(https://github.com/IDEA-Research/GroundingDINO/blob/main/.asset/hero_figure.png)

Remember to set your animation prompt in the Animation Generation group (bookmark "4")

If you generated a river in your image prompt then in the animation prompt you might put something like "rushing river" to animate the river.


uses bookmarks

Press the number on the left side of this legend to goto that section of the workflow.

1: Loaders and welcome notes

2: Image Generation

3: Mask Generation

4: Animation Generation


IMPORTANT

Some of the Anything Everywhere Nodes use regex so unless you are familiar with how RegEx works I'd suggest not modifying those nodes as you could break the workflow.

If you'd like to learn about RegEx expressions you can learn more about them here. (https://www.howtogeek.com/devops/how-do-you-actually-use-regex/)

Built-in nodes
Custom nodes
EmptyLatentImage
CLIPTextEncode
PreviewBridge
ImageToMask
VAEDecode
SAMLoader
GroundingDinoModelLoader (segment anything)
PreviewImage
GroundingDinoSAMSegment (segment anything)
GrowMask
NNLatentUpscale
ADE_AnimateDiffLoRALoader
SetLatentNoiseMask
ADE_AnimateDiffUniformContextOptions
MaskToImage
ImageBlur
CR Latent Batch Size
VHS_VideoCombine
FILM VFI
ADE_AnimateDiffLoaderWithContext
VAELoader
KSampler
CheckpointLoaderSimple
Models
2
0
5
0