AnimateDiff for ComfyUI

AnimateDiff integration for ComfyUI, adapts from sd-webui-animatediff. Please read the original repo README for more information.

How to Use

Clone this repo into custom_nodes folder.
Download motion modules from Google Drive | HuggingFace | CivitAI | Baidu NetDisk. You only need to download one of mm_sd_v14.ckpt | mm_sd_v15.ckpt. Put the model weights under comfyui-animatediff/models/. DO NOT change model filename.

Update 2023/09/15

You can now use community models from manshoety/AD_Stabilized_Motion or CiaraRowles/TemporalDiff
Supports AnimateDiff v2 mm_sd_v15_v2.ckpt model
Fix image is grayed out.
New node: AnimateDiffSampler and AnimateDiffLoader
- Mostly the same with KSampler
- Use AnimateDiffLoader to load the motion module
- inject_method: should left default. See this issue for more details.
- frame_number: animation length

Example Workflow

Workflow file: https://github.com/ArtVentureX/comfyui-animatediff/blob/main/workflow.json

Samples

Known Issues

GIF split into multiple scenes

See: https://github.com/continue-revolution/sd-webui-animatediff/issues/38

Main reasons:

Promt are too long (more than 75 tokens)
Resolution are too high
Number of frame too high

Work around:

Shorter your prompt and negative prompt
Reduce resolution. AnimateDiff is trained on 512x512 images so it works best with 512x512 output.
Shouldn't generate longer than 16 frames. AnimateDiff is trained to output the best results with 16 frames.

GIF has Wartermark after update to the latest version

See: https://github.com/continue-revolution/sd-webui-animatediff/issues/31

As mentioned in the issue thread, it seems to be due to the training dataset. The new version is the correct implementation and produces smoother GIFs compared to the older version.

Old revision	New revision

I played around with both version and found that the watermark only present in some models, not always. To use the old (legacy) method, change injection_method to legacy in the AnimateDiffSampler node.

4.0 KiB Raw Blame History