avatar-graph-comfyui
A custom_nodes module for creating real-time interactive avatars powered by blender bpy mesh api + Avatech Shape Flow runtime.
Demo
Contents
Custom Nodes
Expand to see all the nodes available✨
| Name | Description | Preview |
|---|---|---|
| Segmentation (SAM) | Integrative SAM node allowing you to directly select and create multiple image segment output. | |
| Create Mesh Layer | Create a mesh object from the input images (usually a segmented part of the entire image) | |
| Join Meshes | Combine multiple meshes into a single mesh object | |
| Mesh Modify Shape Key | Given shape key name & target vertex_group, modify the vertex / all vertex’s transform | |
| Create Shape Flow | Create runtime shape flow graph, allowing interactive inputs affecting shape keys value in runtime | |
| Match Texture Aspect Ratio | Since the mesh is created in 1:1 aspect ratio, a re-scale is needed at the end of the operation | |
| Plane Texture Unwrap | Will perform mesh face fill and UV Cube project on the target plane mesh, scaled to bounds. | |
| Avatar Main Output | The primary output of the .ava file. The embeded Avatar View will auto update with this node's output |
Workflow Template
Simple Rig Workflow
To enable the character to blink eyes and talking.
🎯Notice
For optimal results, please input a character image with an open mouth and a minimum resolution of 768x768. This higher resolution will enable the tool to accurately recognize and work with facial features.
💡Make your character mouth open Guide
Download: 📂Simple Rig Template
Image Preprocess Guide
💡If you want to generate a new character image
you can download this Template and refer to the Guide!
Character Gen Prompting Guide
Character Gen Prompting Guide
🎯Notice
We need a character image with an open mouth and enable the tool to easily recognize facial features, so please add to the prompt:
looking at viewer, detailed face, open mouth, [smile], solo,eye-level angle
Download: 📂Character Gen Template
Feel free to change any checkpoint model that suits your needs.
💡If you have a character image but it's not mouth open
you can download this Template and refer to the Guide!
Mouth Open Guide (Inpaint)
Mouth Open Guide (Inpaint)
To maintain consistency with the base image, it is recommended to utilize a checkpoint model that aligns with its style.
Download: 📂MouthOpen Template
Inpaint Demonstration
Recommend Checkpoint Model List
Anime Style SD1.5
- https://civitai.com/models/35960/flat-2d-animerge
- https://civitai.com/models/24149/mistoonanime
- https://civitai.com/models/22364/kizuki-anime-hentai-checkpoint
Realistic Style SD1.5
Shape Flow
Installation
-
cd custom_nodes -
git clone https://github.com/avatechgg/avatar-graph-comfyui.git -
Install deps
cd avatar-graph-comfyui && python -m pip install -r requirements.txt -
Navigate to in your comfyui installation folder, open
comfyui/main.pyand add the following lines:
def prompt_worker(q, server):
e = execution.PromptExecutor(server)
# This is a workaround since using blender as a python module, the bpy module has to be imported after in the custom thread, otherwise it will cause a segfault if imported in the custom nodes.
# Add next line
import global_bpy
while True:
item, item_id = q.get()
# Add next line
global_bpy.reset_bpy()
-
Restart comfyui
-
Run comfyui with enable-cors-header
python main.py --enable-cors-headeror (mac)python main.py --force-fp16 --enable-cors-header
Development
If you are interested in contributing expand to see development details✨
For comfyui frontend extension, frontend js located at avatar-graph-comfyui/js
Web stack used: vanjs tailwindcss
Install deps
pnpm i
Run the dev command to start the tailwindcss watcher
pnpm dev
For each changes, simply refresh the comfyui page to see the changes.
p.s. For tailwind autocomplete, add the following to your vscode settings.json.
{
"tailwindCSS.experimental.classRegex": [
["class\\s?:\\s?([\\s\\S]*)", "(?:\"|')([^\"']*)(?:\"|')"]
]
}