2023-09-15 21:56:35 +07:00
2023-09-15 16:47:42 +07:00
2023-07-22 22:11:47 +07:00
2023-07-22 22:11:47 +07:00
2023-07-22 22:11:47 +07:00
2023-09-15 21:56:35 +07:00

AnimateDiff for ComfyUI

AnimateDiff integration for ComfyUI, adapts from sd-webui-animatediff. Please read the original repo README for more information.

How to Use

  1. Clone this repo into custom_nodes folder.
  2. Download motion modules from Google Drive | HuggingFace | CivitAI | Baidu NetDisk. You only need to download one of mm_sd_v14.ckpt | mm_sd_v15.ckpt. Put the model weights under comfyui-animatediff/models/. DO NOT change model filename.

Update 2023/09/15

image

Example Workflow

image

Samples

23b44c29-29e8-4f48-ab3c-4df87c90c13f

25f6c60c-f8ac-4abe-984f-1559c355d7f6

Known Issues

GIF split into multiple scenes

<<<<<<< Updated upstream AnimateDiff_00002

Known Issues

GIF split into multiple scenes

AnimateDiff_00007_

This is usually due to memory (VRAM) is not enough to process the whole image batch at the same time. Try reduce the image size and frame number.

GIF has Wartermark after update to the latest version

See https://github.com/continue-revolution/sd-webui-animatediff/issues/31

AnimateDiff_00007_

See: https://github.com/continue-revolution/sd-webui-animatediff/issues/38

Main reasons:

  • Promt are too long (more than 75 tokens)
  • Resolution are too high
  • Number of frame too high

Work around:

  • Shorter your prompt and negative prompt
  • Reduce resolution. AnimateDiff is trained on 512x512 images so it works best with 512x512 output.
  • Shouldn't generate longer than 16 frames. AnimateDiff is trained to output the best results with 16 frames.

GIF has Wartermark after update to the latest version

See: https://github.com/continue-revolution/sd-webui-animatediff/issues/31

Stashed changes

As mentioned in the issue thread, it seems to be due to the training dataset. The new version is the correct implementation and produces smoother GIFs compared to the older version.

Old revision New revision

<<<<<<< Updated upstream

I played around with both version and found that the watermark only present in some models, not always. So I've brought back the old method and also created a new node with the new method. You can try both to find the best fit for each model.

Screenshot 2023-07-28 at 18 14 14

I played around with both version and found that the watermark only present in some models, not always. To use the old (legacy) method, change injection_method to legacy in the AnimateDiffSampler node.

Stashed changes

S
Description
No description provided
Readme Apache-2.0
217 KiB
Languages
Python 87.6%
JavaScript 12.4%