AnimateDiff for ComfyUI

AnimateDiff integration for ComfyUI, adapts from sd-webui-animatediff. Please read the original repo README for more information.

How to Use

1. Clone this repo into custom_nodes folder.
2. Download motion modules and put them under comfyui-animatediff/models/.

• Original modules: Google Drive | HuggingFace | CivitAI | Baidu NetDisk
• Community modules: manshoety/AD_Stabilized_Motion | CiaraRowles/TemporalDiff
• AnimateDiff v2 mm_sd_v15_v2.ckpt

Nodes

AnimateDiffLoader

AnimateDiffSampler

• Mostly the same with KSampler
• Use AnimateDiffLoader to load the motion module
• inject_method: should left default
• frame_number: animation length
• latent_image: You can pass an EmptyLatentImage

AnimateDiffCombine

• Combine GIF frames and produce the GIF image
• frame_rate: number of frame per second
• loop_count: use 0 for infinite loop
• save_image: should GIF be saved to disk
• format: supports image/gif, image/webp (better compression) or video/webm (need ffmpeg installed and available in PATH)

Example Workflow

Workflow file: https://github.com/ArtVentureX/comfyui-animatediff/blob/main/workflow.json

Samples

Known Issues

GIF split into multiple scenes

Work around:

• Shorter your prompt and negative prompt
• Reduce resolution. AnimateDiff is trained on 512x512 images so it works best with 512x512 output.
• Disable xformers with --disable-xformers

GIF has Wartermark (especially when using mm_sd_v15)

See: https://github.com/continue-revolution/sd-webui-animatediff/issues/31

Training data used by the authors of the AnimateDiff paper contained Shutterstock watermarks. Since mm_sd_v15 was finetuned on finer, less drastic movement, the motion module attempts to replicate the transparency of that watermark and does not get blurred away like mm_sd_v14. Try other community finetuned modules.