ComfyUI Sonic

Custom node that takes an audio mp3 and an image, and creates a lip synced video from them.

Install ComfyUI Sonic Custom Node

Install the custom node ComfyUI Sonic using the manager,

Or, you can use your command/terminal prompt.

1. Navigate to your ./ComfyUI_windows_portable/ComfyUI/custom_nodes folder.
2. Run,

   git clone https://github.com/Sean-Bradley/ComfyUI-Sonic
   

3. Navigate to your ./ComfyUI_windows_portable folder.
4. Run

   python_embeded\python -m pip install -r ComfyUI/custom_nodes/comfyui-sonic/requirements.txt
   

5. Restart ComfyUI

Install Models

We need to create this folder structure and download each model into its place.

📂 ComfyUI/
├── 📂 models/
│   └── 📂 sonic/
│       ├── audio2bucket.pth
│       ├── audio2token.pth
│       ├── unet.pth
│       ├── face_yolov8m.pt
│       ├── 📂 whisper-tiny/
│       │   ├── config.json
│       │   ├── model.safetensors
│       │   └── preprocessor_config.json
│       └── 📂 RIFE/
│           └── flownet.pkl

Download each file and place into your ./ComfyUI/models/sonic/ folder

• audio2bucket.pth
• audio2token.pth
• unet.pth
• face_yolov8m.pt

Download each file and place into your ./ComfyUI/models/sonic/whisper-tiny/ folder

• config.json
• model.safetensors
• preprocessor_config.json

Download this file and place into your ./ComfyUI/models/sonic/RIFE/ folder

• flownet.pkl

Sample Workflows

Tip

For easier drag/drop of workflows into ComfyUI, visit https://sbcode.net/genai/lipsync-sonic/

Download this Example Audio and save into your ComfyUI/input folder.

Initial Image	Workflow