AnimateDiff for ComfyUI
AnimateDiff integration for ComfyUI, adapts from sd-webui-animatediff. Please read the original repo README for more information.
How to Use
- Clone this repo into
custom_nodesfolder. - Download motion modules from Google Drive | HuggingFace | CivitAI | Baidu NetDisk. You only need to download one of
mm_sd_v14.ckpt|mm_sd_v15.ckpt. Put the model weights undercomfyui-animatediff/models/. DO NOT change model filename.
Update 2023/09/15
- You can now use community models from manshoety/AD_Stabilized_Motion or CiaraRowles/TemporalDiff
- Supports AnimateDiff v2 mm_sd_v15_v2.ckpt model
- Fix image is grayed out.
- New node: AnimateDiffSampler and AnimateDiffLoader
- Mostly the same with
KSampler - Use
AnimateDiffLoaderto load the motion module inject_method: should left default. See this issue for more details.frame_number: animation length
- Mostly the same with
Example Workflow
Samples
Known Issues
GIF split into multiple scenes
Known Issues
GIF split into multiple scenes
This is usually due to memory (VRAM) is not enough to process the whole image batch at the same time. Try reduce the image size and frame number.
GIF has Wartermark after update to the latest version
See https://github.com/continue-revolution/sd-webui-animatediff/issues/31
See: https://github.com/continue-revolution/sd-webui-animatediff/issues/38
Main reasons:
- Promt are too long (more than 75 tokens)
- Resolution are too high
- Number of frame too high
Work around:
- Shorter your prompt and negative prompt
- Reduce resolution. AnimateDiff is trained on 512x512 images so it works best with 512x512 output.
- Shouldn't generate longer than 16 frames. AnimateDiff is trained to output the best results with 16 frames.
GIF has Wartermark after update to the latest version
See: https://github.com/continue-revolution/sd-webui-animatediff/issues/31
Stashed changes
As mentioned in the issue thread, it seems to be due to the training dataset. The new version is the correct implementation and produces smoother GIFs compared to the older version.
| Old revision | New revision |
<<<<<<< Updated upstream
I played around with both version and found that the watermark only present in some models, not always. So I've brought back the old method and also created a new node with the new method. You can try both to find the best fit for each model.
I played around with both version and found that the watermark only present in some models, not always. To use the old (legacy) method, change injection_method to legacy in the AnimateDiffSampler node.
Stashed changes