avatar-graph-comfyui
A custom_nodes module for creating real-time interactive avatars powered by blender bpy mesh api + Avatech Shape Flow runtime.
Demo
Custom Nodes
Expand to see all the nodes available✨
| Preview | Node Name | Description |
|---|---|---|
| Segmentation (SAM) | Integrative SAM node allowing you to directly select and create multiple image segment output. | |
| Create Mesh Layer | Create a mesh object from the input images (usually a segmented part of the entire image) | |
| Join Meshes | Combine multiple meshes into a single mesh object | |
| Mesh Modify Shape Key | Given shape key name & target vertex_group, modify the vertex / all vertex’s transform | |
| Create Shape Flow | Create runtime shape flow graph, allowing interactive inputs affecting shape keys value in runtime | |
| Match Texture Aspect Ratio | Since the mesh is created in 1:1 aspect ratio, a re-scale is needed at the end of the operation | |
| Plane Texture Unwrap | Will perform mesh face fill and UV Cube project on the target plane mesh, scaled to bounds. | |
| Avatar Main Output | The primary output of the .ava file. The embeded Avatar View will auto update with this node's output |
Workflow Tempalte
🎯Notice
To achieve the best results, please ensure character image with an open mouth and have a minimum resolution of 768x768, it enable the tool to easily recognize facial features.
Simple Eyeblink and Talking Workflow
Image Preprocess Guide
💡If you want to generate a new character image, you can refer to this Guide!
Character Gen Prompting Guide
🎯Notice
We need a character image with an open mouth and enable the tool to easily recognize facial features, so please add to the prompt:
looking at viewer, detailed face, open mouth, [smile], solo,eye-level angle
Workflow Download:
Feel free to change any checkpoint model that suits your needs.
💡If you have a character image but it's not mouth open, you can refer to this Guide!
Mouth Open(Inpaint) Guide
🎯Notice
To maintain consistency with the base image, it is recommended to utilize a checkpoint model that aligns with its style.
Workflow Download:
Inpaint Demonstration
Recommend Checkpoint Model List
Anime Style SD1.5
- https://civitai.com/models/35960/flat-2d-animerge
- https://civitai.com/models/24149/mistoonanime
- https://civitai.com/models/22364/kizuki-anime-hentai-checkpoint
Realistic Style SD1.5
Shape Flow
Installation
-
cd custom_nodes -
git clone https://github.com/avatechgg/avatar-graph-comfyui.git -
Install deps
cd avatar-graph-comfyui && python -m pip install -r requirements.txt -
Navigate to in your comfyui installation folder, open
comfyui/main.pyand add the following lines:
def prompt_worker(q, server):
e = execution.PromptExecutor(server)
# This is a workaround since using blender as a python module, the bpy module has to be imported after in the custom thread, otherwise it will cause a segfault if imported in the custom nodes.
# Add next line
import global_bpy
while True:
item, item_id = q.get()
# Add next line
global_bpy.reset_bpy()
-
Restart comfyui
-
Run comfyui with enable-cors-header
python main.py --enable-cors-headeror (mac)python main.py --force-fp16 --enable-cors-header
Development
If you are interested in contributing expand to see development details✨
For comfyui frontend extension, frontend js located at avatar-graph-comfyui/js
Web stack used: vanjs tailwindcss
Install deps
pnpm i
Run the dev command to start the tailwindcss watcher
pnpm dev
For each changes, simply refresh the comfyui page to see the changes.
p.s. For tailwind autocomplete, add the following to your vscode settings.json.
{
"tailwindCSS.experimental.classRegex": [
["class\\s?:\\s?([\\s\\S]*)", "(?:\"|')([^\"']*)(?:\"|')"]
]
}