ComfyUI-MuseTalk_FSH

the comfyui custom node of MuseTalk to make audio driven videos!

How to use

make sure ffmpeg is worked in your commandline for Linux

apt update
apt install ffmpeg

for Windows,you can install ffmpeg by WingetUI automatically

then!

git clone https://github.com/AIFSH/ComfyUI-MuseTalk_FSH.git
cd ComfyUI-MuseTalk_FSH
pip install -r requirements.txt

mmlab packages

pip install --no-cache-dir -U openmim 
mim install mmengine 
mim install "mmcv>=2.0.1" 
mim install "mmdet>=3.1.0" 
mim install "mmpose>=1.1.0" 

Download weights

You can download weights manually as follows:

1. Download our trained weights.

2. Download the weights of other components:

   • sd-vae-ft-mse
   • whisper
   • dwpose
   • face-parse-bisent
   • resnet18

Finally, these weights should be organized in models as follows:

ComfyUI-MuseTalk_FSH/models/
├── musetalk
│   └── musetalk.json
│   └── pytorch_model.bin
├── dwpose
│   └── dw-ll_ucoco_384.pth
├── face-parse-bisent
│   ├── 79999_iter.pth
│   └── resnet18-5c106cde.pth
├── sd-vae-ft-mse
│   ├── config.json
│   └── diffusion_pytorch_model.bin
└── whisper
    └── tiny.pt

Tutorial

todo

WeChat Group && Donate

Thanks

• MuseTalk