Universal Video Generator (modelscope+ADLCM w/SD15) now with SUPIR+SDXL Lightning)
created 2 months ago
animatediff
video
ipadapter
lora
upscale
controlnet
272 nodes

2.0k

450

Tip this creator
Credits
https://www.youtube.com/@stephantual
Outputs
example image output 0 for workflow.
example image output 1 for workflow.
example image output 2 for workflow.
example image output 3 for workflow.
example image output optimized_comfy_workflows_user_uploads/cd4e44f0-99b9-487e-ad8a-1a8fb45dacff/assets/vlcsnap-2024-03-14-11h40m44s975.webp for workflow.
Description

This is an offshoot of my work on the "not SORA" video generator. My ultimate goal is to create a universal tool that will let you use ADLCM+SVD+Zeroscope with pupeteering, create logical scenes then upscale to 4k native.

Currently it integrates:

  1. Zeroscope v2 (T2V, V2V, upscale via model+SUPIR)

  2. Modelscope v2 nodes (with SD15 diffuser)

  3. ADLCM upscale

  4. SDXL lightning upscale

If you make an amazing video with this, please post it on my discord at tinyurl.com/URSIUM - we'd love to see your work!

If you or your studio find this useful, consider a donation at https://ko-fi.com/stephantual to help me continue my work, as this is a huge project and will be released in phases. All future updates will be on this page.

The LORA for MS resolution normalization was compiled by exponentialML (the creator of the new SD15 + modelscope nodes). It's included as a zip file on this site.

Important:

  • This is not a beginner project - but it's a great way to learn!

  • The horror film look is just something I like. You can just switch from lcm>squarert_linear to lcm for 'normal' (real life) results.

  • I don't always use comfyUI manager - if comfy complains about missing nodes, please download them on github from the main branches (links in the video description)

  • This is bleeding edge (really) - things will change, break and have bugs. This was compiled on March 14th 2024. If you find a bug in any of the nodes, please report them to the indivual creators on their respective github to help the community.

  • This was built and run on windows 11 with 24gb ram. I tested it to successfully run at 16gb if you drop the resolution to 720p and drop the FILMVFI clear cache to 5, SUPIR steps: 8. The quality will be lower, evidently.

Do not ask for support here. Please use the discord at tinyurl.com/URSIUM in the correct forum category. Please understand this is released 'AS IS' and comes with no support or warranty of any kind. Everything is released under CC-BY - meaning you can do whatever you want with it, but please give attribution to https://www.youtube.com/@stephantual

Built-in nodes
FreeU_V2
SelfAttentionGuidance
CLIPTextEncode
CLIPSetLastLayer
CheckpointLoaderSimple
PatchModelAddDownscale
ImageUpscaleWithModel
VAEEncode
PreviewImage
VAEDecode
LoraLoaderModelOnly
EmptyLatentImage
CLIPVisionLoader
LoraLoader
KSampler
ImageScale
UpscaleModelLoader
Custom nodes
Automatic CFG
ADE_ApplyAnimateDiffModelSimple
ADE_StandardUniformContextOptions
ADE_LoadAnimateDiffModel
PlaySound|pysssss
VHS_VideoCombine
Modelscopet2v
0246.Beautify
CR Text Concatenate
IPAdapterEncoder
ADE_UseEvolvedSampling
ImpactFloat
Modelscopev2v
CR Apply LoRA Stack
CR LoRA Stack
IPAdapterModelLoader
If ANY execute A else B
GetImageSize+
Compare
PrepImageForClipVision
ADE_AnimateDiffSamplingSettings
ControlNetLoaderAdvanced
ADE_StandardStaticViewOptions
ADE_ViewsOnlyContextOptions
AIO_Preprocessor
ACN_AdvancedControlNetApply
ADE_IterationOptsFreeInit
ModelScopeT2VLoader
ADE_ApplyAnimateDiffModel
ImageFromBatch+
FileNamePrefix
FILM VFI
Upscale Model Loader
UltimateSDUpscale
VHS_LoadVideo
Text Multiline
ADE_MultivalDynamic
SUPIR_Upscale
Custom files
new

These are separate files that the creator has uploaded for this workflow.

resolution_normalization.zip

Models
1
0
0
0