2023-07-22 22:11:47 +07:00
2023-07-22 22:11:47 +07:00
2023-07-22 22:11:47 +07:00
2023-09-15 21:58:38 +07:00
2023-09-15 21:58:38 +07:00

AnimateDiff for ComfyUI

AnimateDiff integration for ComfyUI, adapts from sd-webui-animatediff. Please read the original repo README for more information.

How to Use

  1. Clone this repo into custom_nodes folder.
  2. Download motion modules from Google Drive | HuggingFace | CivitAI | Baidu NetDisk. You only need to download one of mm_sd_v14.ckpt | mm_sd_v15.ckpt. Put the model weights under comfyui-animatediff/models/. DO NOT change model filename.

Update 2023/09/15

image

Example Workflow

image

Workflow file: https://github.com/ArtVentureX/comfyui-animatediff/blob/main/workflow.json

Samples

23b44c29-29e8-4f48-ab3c-4df87c90c13f

25f6c60c-f8ac-4abe-984f-1559c355d7f6

Known Issues

GIF split into multiple scenes

AnimateDiff_00007_

See: https://github.com/continue-revolution/sd-webui-animatediff/issues/38

Main reasons:

  • Promt are too long (more than 75 tokens)
  • Resolution are too high
  • Number of frame too high

Work around:

  • Shorter your prompt and negative prompt
  • Reduce resolution. AnimateDiff is trained on 512x512 images so it works best with 512x512 output.
  • Shouldn't generate longer than 16 frames. AnimateDiff is trained to output the best results with 16 frames.

GIF has Wartermark after update to the latest version

See: https://github.com/continue-revolution/sd-webui-animatediff/issues/31

As mentioned in the issue thread, it seems to be due to the training dataset. The new version is the correct implementation and produces smoother GIFs compared to the older version.

Old revision New revision

I played around with both version and found that the watermark only present in some models, not always. To use the old (legacy) method, change injection_method to legacy in the AnimateDiffSampler node.

S
Description
No description provided
Readme Apache-2.0
217 KiB
Languages
Python 87.6%
JavaScript 12.4%