AnimateDiff for ComfyUI

AnimateDiff integration for ComfyUI, adapts from sd-webui-animatediff. Please read the original repo README for more information.

How to Use

Clone this repo into custom_nodes folder.
Download motion modules and put them under comfyui-animatediff/models/.

Original modules: Google Drive | HuggingFace | CivitAI | Baidu NetDisk
Community modules: manshoety/AD_Stabilized_Motion | CiaraRowles/TemporalDiff
AnimateDiff v2 mm_sd_v15_v2.ckpt

Nodes

AnimateDiffLoader

AnimateDiffSampler

Mostly the same with KSampler
Use AnimateDiffLoader to load the motion module
inject_method: should left default
frame_number: animation length
latent_image: You can pass an EmptyLatentImage

AnimateDiffCombine

Combine GIF frames and produce the GIF image
frame_rate: number of frame per second
loop_count: use 0 for infinite loop
save_image: should GIF be saved to disk
format: supports image/gif, image/webp (better compression) or video/webm (need ffmpeg installed and available in PATH)

Example Workflow

Workflow file: https://github.com/ArtVentureX/comfyui-animatediff/blob/main/workflow.json

Samples

Known Issues

GIF split into multiple scenes

Work around:

Shorter your prompt and negative prompt
Reduce resolution. AnimateDiff is trained on 512x512 images so it works best with 512x512 output.
Disable xformers with --disable-xformers

GIF has Wartermark (especially when using mm_sd_v15)

See: https://github.com/continue-revolution/sd-webui-animatediff/issues/31

Training data used by the authors of the AnimateDiff paper contained Shutterstock watermarks. Since mm_sd_v15 was finetuned on finer, less drastic movement, the motion module attempts to replicate the transparency of that watermark and does not get blurred away like mm_sd_v14. Try other community finetuned modules.

3.3 KiB Raw Blame History