Austin Mroz
414c5d3bb8
This ports the video format code written for the upstream changes to the ffmpeg implementation. It improves the quality of webm outputs and adds support for additional codecs (h264, h265, av1) It also improves the logging by passing errors and more selectively blocking the logging of encoders. While h265 has been included, most browsers will be unable to display the resulting video.
AnimateDiff for ComfyUI
AnimateDiff integration for ComfyUI, adapts from sd-webui-animatediff. Please read the original repo README for more information.
How to Use
- Clone this repo into
custom_nodesfolder. - Download motion modules and put them under
comfyui-animatediff/models/.
- Original modules: Google Drive | HuggingFace | CivitAI | Baidu NetDisk
- Community modules: manshoety/AD_Stabilized_Motion | CiaraRowles/TemporalDiff
- AnimateDiff v2 mm_sd_v15_v2.ckpt
Nodes
AnimateDiffLoader
AnimateDiffSampler
- Mostly the same with
KSampler - Use
AnimateDiffLoaderto load the motion module inject_method: should left defaultframe_number: animation lengthlatent_image: You can pass anEmptyLatentImage
AnimateDiffCombine
- Combine GIF frames and produce the GIF image
frame_rate: number of frame per secondloop_count: use 0 for infinite loopsave_image: should GIF be saved to diskformat: supportsimage/gif,image/webp(better compression) orvideo/webm(needffmpeginstalled and available in PATH)
Example Workflow
Workflow file: https://github.com/ArtVentureX/comfyui-animatediff/blob/main/workflow.json
Samples
Known Issues
GIF split into multiple scenes
Work around:
- Shorter your prompt and negative prompt
- Reduce resolution. AnimateDiff is trained on 512x512 images so it works best with 512x512 output.
- Disable xformers with
--disable-xformers
GIF has Wartermark (especially when using mm_sd_v15)
See: https://github.com/continue-revolution/sd-webui-animatediff/issues/31
Training data used by the authors of the AnimateDiff paper contained Shutterstock watermarks. Since mm_sd_v15 was finetuned on finer, less drastic movement, the motion module attempts to replicate the transparency of that watermark and does not get blurred away like mm_sd_v14. Try other community finetuned modules.