Update Install Methods & Instruction, LGM Workflow
This commit is contained in:
MrForExample
@@ -12,7 +12,7 @@
|
||||
### Note: this project is still a WIP and not been released into ComFyUI package database yet
|
||||
|
||||
## Currently support:
|
||||
- For use case please check [Example Workflows](./_Example_Workflows/). [**Last update: 09/02/2024**]
|
||||
- For use case please check [Example Workflows](./_Example_Workflows/). [**Last update: 11/02/2024**]
|
||||
- **Note:** you need to put [Example Inputs Files & Folders](_Example_Workflows/_Example_Inputs_Files/) under ComfyUI Root Directory\ComfyUI\input folder before you can run the example workflow
|
||||
|
||||
- **Large Multiview Gaussian Model**: [3DTopia/LGM](https://github.com/3DTopia/LGM)
|
||||
@@ -92,17 +92,20 @@
|
||||
## Install:
|
||||
|
||||
**[IMPORTANT!!!]** <br> Currently this package is only been tested in following setups:
|
||||
- Windows 10/11
|
||||
- Miniconda/Conda Python 3.11.7
|
||||
- Windows 10/11 (Tested on my laptop) & Ubuntu 23.10 [(Tested by @watsieboi)](https://github.com/MrForExample/ComfyUI-3D-Pack/issues/16)
|
||||
- Miniconda/Conda Python 3.11.x
|
||||
- I tried install this package with ComfyUI embed python env first, but I can't find a way to build CUDA related libraries, e.g. diff-gaussian-rasterization, nvdiffrast, simple-knn.
|
||||
- Torch version: 2.1.2+cu121/V.2.1.2+cu118
|
||||
- Torch version >= 2.1.2+cu121
|
||||
|
||||
<br>
|
||||
|
||||
Assume you have already downloaded [ComfyUI](https://github.com/comfyanonymous/ComfyUI)
|
||||
|
||||
### Setup Method 0: Using Miniconda (Recommend! Should works in both Windows & Linux)
|
||||
|
||||
First download [Miniconda](https://docs.conda.io/projects/miniconda/en/latest/) (*One of the best way to manage a clean and separated python envirments*)
|
||||
- Alternatively you can check this tutorial: [Installing ComfyUI with Miniconda On Windows and Mac](https://www.comflowy.com/preparation-for-study/install#step-two-download-comfyui)
|
||||
|
||||
Then running following commands to setup the Miniconda environment for ComfyUI:
|
||||
|
||||
```bash
|
||||
# Go to your Your ComfyUI root directory, for my example:
|
||||
@@ -113,75 +116,63 @@ conda create -p ./python_miniconda_env/ComfyUI python=3.11
|
||||
# conda will tell what command to use to activate the env
|
||||
conda activate C:\Users\reall\Softwares\ComfyUI_windows_portable\python_miniconda_env\ComfyUI
|
||||
|
||||
# This package also works with cu118
|
||||
# update pip
|
||||
python -m pip install --upgrade pip
|
||||
|
||||
# You can using following command to installing CUDA only in the miniconda environment you just created if you don't want to donwload and install it manually & globally:
|
||||
# conda install -c "nvidia/label/cuda-12.1.0" cuda-toolkit
|
||||
|
||||
# Install the main packahes
|
||||
pip install torch torchvision --index-url https://download.pytorch.org/whl/cu121
|
||||
|
||||
pip install -r ./ComfyUI/requirements.txt
|
||||
|
||||
# Then go to ComfyUI-3D-Pack directory under the ComfyUI Root Directory\ComfyUI\custom_nodes for my example is:
|
||||
cd C:\Users\reall\Softwares\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-3D-Pack
|
||||
# Finally you can double click following .bat script or run it in CLI:
|
||||
install.bat
|
||||
```
|
||||
|
||||
- ***Install Plan B:***
|
||||
- Alternatively you can check this tutorial: [Installing ComfyUI with Miniconda On Windows and Mac](https://www.comflowy.com/preparation-for-study/install#step-two-download-comfyui)
|
||||
|
||||
Just in case `install.bat` may not working in your PC, you could also run the following commands under this package's root directory:
|
||||
|
||||
```bash
|
||||
# First make sure the Conda env: python_miniconda_env\ComfyUI is activated, then go to ComfyUI Root Directory\ComfyUI\custom_nodes\ComfyUI-3D-Pack and:
|
||||
pip install -r requirements.txt
|
||||
|
||||
git clone --recursive https://github.com/ashawkey/diff-gaussian-rasterization
|
||||
pip install ./diff-gaussian-rasterization
|
||||
|
||||
pip install ./simple-knn
|
||||
|
||||
pip install -U xformers --index-url https://download.pytorch.org/whl/cu121
|
||||
|
||||
git clone --recursive https://github.com/NVlabs/nvdiffrast/`
|
||||
pip install ./nvdiffrast
|
||||
|
||||
# Install pointnet2_ops
|
||||
cd tgs/models/snowflake/pointnet2_ops_lib && python setup.py install && cd ../../../../
|
||||
|
||||
# Install pytorch_scatter
|
||||
pip install git+https://github.com/rusty1s/pytorch_scatter.git
|
||||
|
||||
# Install pytorch3d
|
||||
pip install git+https://github.com/facebookresearch/pytorch3d.git@stable
|
||||
|
||||
# Install kiuikit (Computer Vision Library)
|
||||
pip install git+https://github.com/ashawkey/kiuikit.git
|
||||
```
|
||||
|
||||
- ***Install with ComfyUI's embed python env on Windows:***
|
||||
### Setup Method 1: Using Chocolatey (Windows Only!)
|
||||
|
||||
If you have already installed ComfyUI on Windows with embed python environment and you don't want to switch to Miniconda/Conda and reinstall all your packages <br>
|
||||
*(Which is ideal, but unfortunately so far it seems I'm not experienced enough in this matter, but I'll continue to looking for the better solutions and please give me some suggestions if ou know better, thanks :)*
|
||||
*(Which is ideal, but unfortunately so far it seems I'm not experienced enough in this matter, but I'll continue to looking for the better solutions and please give me some suggestions if you know better, thanks :)*
|
||||
|
||||
According to [@doctorpangloss](https://github.com/MrForExample/ComfyUI-3D-Pack/issues/5), you can setup the c++/cuda build environments in windows by using [chocolatey](https://chocolatey.org/) with following command:
|
||||
```bash
|
||||
# using git bash for the sake of simplicity
|
||||
# enable developer mode
|
||||
# google this: allow os.symlink on windows by adding your username to the local security policy entry for it.
|
||||
# you will have to restart your computer
|
||||
# install chocolatey using powershell, then install the prereqs for compilation on Windows
|
||||
choco install -y visualstudio2022buildtools
|
||||
choco install -y visualstudio2022-workload-vctools --package-parameters "--add Microsoft.VisualStudio.Component.VC.Llvm.ClangToolset --add Microsoft.VisualStudio.Component.VC.Llvm.Clang"
|
||||
# I couldn't find cuda version 12.1, you may need to install cudav12.1 manually
|
||||
choco install -y cuda
|
||||
```
|
||||
Then just go to ComfyUI-3D-Pack directory under the ComfyUI Root Directory\ComfyUI\custom_nodes and running:
|
||||
```bash
|
||||
pip install -r ./ComfyUI/requirements.txt
|
||||
According to [@doctorpangloss](https://github.com/MrForExample/ComfyUI-3D-Pack/issues/5), you can setup the c++/cuda build environments in windows by using [chocolatey](https://chocolatey.org/) with following command:
|
||||
|
||||
install.bat
|
||||
```
|
||||
```bash
|
||||
# using git bash for the sake of simplicity
|
||||
# enable developer mode
|
||||
# google this: allow os.symlink on windows by adding your username to the local security policy entry for it.
|
||||
# you will have to restart your computer
|
||||
# install chocolatey using powershell, then install the prereqs for compilation on Windows
|
||||
choco install -y visualstudio2022buildtools
|
||||
choco install -y visualstudio2022-workload-vctools --package-parameters "--add Microsoft.VisualStudio.Component.VC.Llvm.ClangToolset --add Microsoft.VisualStudio.Component.VC.Llvm.Clang"
|
||||
# I couldn't find cuda version 12.1 in chocolatey package database, you may need to install cudav12.1 manually
|
||||
choco install -y cuda
|
||||
```
|
||||
|
||||
### Install Comfy3D:
|
||||
Go to the Comfy3D root directory: *ComfyUI Root Directory\ComfyUI\custom_nodes\ComfyUI-3D-Pack* and run:
|
||||
|
||||
```bash
|
||||
install_miniconda.bat
|
||||
```
|
||||
Just in case `install_miniconda.bat` may not working in your OS, you could also run the following commands under the same directory: (Works with Linux & macOS)
|
||||
|
||||
```bash
|
||||
pip install -r requirements.txt
|
||||
|
||||
pip install -r requirements_post.txt
|
||||
```
|
||||
<br>
|
||||
|
||||
**Plus:**<br>
|
||||
For those who want to run it inside Google Colab, you can check the [install instruction from @lovisdotio](https://github.com/MrForExample/ComfyUI-3D-Pack/issues/13)
|
||||
|
||||
## Run:
|
||||
Copy the files inside folder [__New_ComfyUI_Bats](./_New_ComfyUI_Bats/) to your ComfyUI root directory, and double click run_nvidia_gpu_miniconda.bat to start ComfyUI!
|
||||
- Alternatively you can just activate the Conda env: python_miniconda_env\ComfyUI, and go to your ComfyUI root directory then run command `python ./ComfyUI/main.py`
|
||||
- Alternatively you can just activate the Conda env: `python_miniconda_env\ComfyUI`, and go to your ComfyUI root directory then run command `python ./ComfyUI/main.py`
|
||||
|
||||
## Tips
|
||||
* The world & camera coordinate system is the same as OpenGL:
|
||||
@@ -202,3 +193,4 @@ azimuth: in (-180, 180), from +z to +x is (0, 90)
|
||||
```
|
||||
|
||||
* If you encounter OpenGL errors (e.g., `[F glutil.cpp:338] eglInitialize() failed`), then set `force_cuda_rasterize` to true on corresponding node
|
||||
* If after the installation, your ComfyUI get stucked at starting or running, you could following the instruction in following link to solve the problem: [Code Hangs Indefinitely When Evaluating Neuron Models on GPU](https://github.com/lava-nc/lava-dl/discussions/211)
|
||||
|
||||
@@ -1,7 +1,121 @@
|
||||
{
|
||||
"last_node_id": 17,
|
||||
"last_link_id": 22,
|
||||
"last_node_id": 19,
|
||||
"last_link_id": 27,
|
||||
"nodes": [
|
||||
{
|
||||
"id": 9,
|
||||
"type": "[Comfy3D] Save 3DGS",
|
||||
"pos": [
|
||||
1798,
|
||||
459
|
||||
],
|
||||
"size": {
|
||||
"0": 315,
|
||||
"1": 58
|
||||
},
|
||||
"flags": {},
|
||||
"order": 9,
|
||||
"mode": 0,
|
||||
"inputs": [
|
||||
{
|
||||
"name": "gs_ply",
|
||||
"type": "GS_PLY",
|
||||
"link": 13
|
||||
}
|
||||
],
|
||||
"outputs": [
|
||||
{
|
||||
"name": "save_path",
|
||||
"type": "STRING",
|
||||
"links": [
|
||||
11
|
||||
],
|
||||
"shape": 3,
|
||||
"slot_index": 0
|
||||
}
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "[Comfy3D] Save 3DGS"
|
||||
},
|
||||
"widgets_values": [
|
||||
"LGMTest/FatCat.ply"
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": 17,
|
||||
"type": "[Comfy3D] Large Multiview Gaussian Model",
|
||||
"pos": [
|
||||
1270,
|
||||
461
|
||||
],
|
||||
"size": {
|
||||
"0": 252,
|
||||
"1": 46
|
||||
},
|
||||
"flags": {},
|
||||
"order": 5,
|
||||
"mode": 0,
|
||||
"inputs": [
|
||||
{
|
||||
"name": "multiview_images",
|
||||
"type": "IMAGE",
|
||||
"link": 26
|
||||
},
|
||||
{
|
||||
"name": "lgm_model",
|
||||
"type": "LGM_MODEL",
|
||||
"link": 22
|
||||
}
|
||||
],
|
||||
"outputs": [
|
||||
{
|
||||
"name": "gs_ply",
|
||||
"type": "GS_PLY",
|
||||
"links": [
|
||||
19,
|
||||
20
|
||||
],
|
||||
"shape": 3,
|
||||
"slot_index": 0
|
||||
}
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "[Comfy3D] Large Multiview Gaussian Model"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": 2,
|
||||
"type": "[Comfy3D] Load Diffusers Pipeline",
|
||||
"pos": [
|
||||
380,
|
||||
345
|
||||
],
|
||||
"size": {
|
||||
"0": 375.3948669433594,
|
||||
"1": 83.33004760742188
|
||||
},
|
||||
"flags": {},
|
||||
"order": 0,
|
||||
"mode": 0,
|
||||
"outputs": [
|
||||
{
|
||||
"name": "pipe",
|
||||
"type": "DIFFUSERS_PIPE",
|
||||
"links": [
|
||||
23
|
||||
],
|
||||
"shape": 3,
|
||||
"slot_index": 0
|
||||
}
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "[Comfy3D] Load Diffusers Pipeline"
|
||||
},
|
||||
"widgets_values": [
|
||||
"MVDreamPipeline",
|
||||
"ashawkey/imagedream-ipmv-diffusers"
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": 4,
|
||||
"type": "LoadImage",
|
||||
@@ -14,14 +128,14 @@
|
||||
"1": 314
|
||||
},
|
||||
"flags": {},
|
||||
"order": 0,
|
||||
"order": 1,
|
||||
"mode": 0,
|
||||
"outputs": [
|
||||
{
|
||||
"name": "IMAGE",
|
||||
"type": "IMAGE",
|
||||
"links": [
|
||||
2
|
||||
24
|
||||
],
|
||||
"shape": 3,
|
||||
"slot_index": 0
|
||||
@@ -44,11 +158,151 @@
|
||||
"image"
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": 5,
|
||||
"type": "InvertMask",
|
||||
"pos": [
|
||||
542,
|
||||
545
|
||||
],
|
||||
"size": {
|
||||
"0": 210,
|
||||
"1": 26
|
||||
},
|
||||
"flags": {},
|
||||
"order": 3,
|
||||
"mode": 0,
|
||||
"inputs": [
|
||||
{
|
||||
"name": "mask",
|
||||
"type": "MASK",
|
||||
"link": 3
|
||||
}
|
||||
],
|
||||
"outputs": [
|
||||
{
|
||||
"name": "MASK",
|
||||
"type": "MASK",
|
||||
"links": [
|
||||
25
|
||||
],
|
||||
"shape": 3,
|
||||
"slot_index": 0
|
||||
}
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "InvertMask"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": 1,
|
||||
"type": "[Comfy3D] Load Large Multiview Gaussian Model",
|
||||
"pos": [
|
||||
898,
|
||||
796
|
||||
],
|
||||
"size": {
|
||||
"0": 315,
|
||||
"1": 82
|
||||
},
|
||||
"flags": {},
|
||||
"order": 2,
|
||||
"mode": 0,
|
||||
"outputs": [
|
||||
{
|
||||
"name": "lgm_model",
|
||||
"type": "LGM_MODEL",
|
||||
"links": [
|
||||
22
|
||||
],
|
||||
"shape": 3,
|
||||
"slot_index": 0
|
||||
}
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "[Comfy3D] Load Large Multiview Gaussian Model"
|
||||
},
|
||||
"widgets_values": [
|
||||
"model_fp16.safetensors",
|
||||
"big"
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": 15,
|
||||
"type": "[Comfy3D] Preview 3DMesh",
|
||||
"pos": [
|
||||
2326.1025658000017,
|
||||
214.90587429004222
|
||||
],
|
||||
"size": [
|
||||
600,
|
||||
500
|
||||
],
|
||||
"flags": {},
|
||||
"order": 12,
|
||||
"mode": 0,
|
||||
"inputs": [
|
||||
{
|
||||
"name": "mesh_file_path",
|
||||
"type": "STRING",
|
||||
"link": 16,
|
||||
"widget": {
|
||||
"name": "mesh_file_path"
|
||||
}
|
||||
}
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "[Comfy3D] Preview 3DMesh"
|
||||
},
|
||||
"widgets_values": [
|
||||
"",
|
||||
null
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": 14,
|
||||
"type": "[Comfy3D] Save 3D Mesh",
|
||||
"pos": [
|
||||
1996.1025658,
|
||||
214.90587429004222
|
||||
],
|
||||
"size": {
|
||||
"0": 307.8253173828125,
|
||||
"1": 60.248291015625
|
||||
},
|
||||
"flags": {},
|
||||
"order": 10,
|
||||
"mode": 0,
|
||||
"inputs": [
|
||||
{
|
||||
"name": "mesh",
|
||||
"type": "MESH",
|
||||
"link": 17
|
||||
}
|
||||
],
|
||||
"outputs": [
|
||||
{
|
||||
"name": "save_path",
|
||||
"type": "STRING",
|
||||
"links": [
|
||||
16
|
||||
],
|
||||
"shape": 3,
|
||||
"slot_index": 0
|
||||
}
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "[Comfy3D] Save 3D Mesh"
|
||||
},
|
||||
"widgets_values": [
|
||||
"LGMTest/FatCat.obj"
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": 12,
|
||||
"type": "[Comfy3D] Switch 3DGS Axis",
|
||||
"pos": [
|
||||
1554,
|
||||
1558,
|
||||
460
|
||||
],
|
||||
"size": {
|
||||
@@ -56,7 +310,7 @@
|
||||
"1": 106
|
||||
},
|
||||
"flags": {},
|
||||
"order": 6,
|
||||
"order": 7,
|
||||
"mode": 0,
|
||||
"inputs": [
|
||||
{
|
||||
@@ -85,58 +339,19 @@
|
||||
"-z"
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": 9,
|
||||
"type": "[Comfy3D] Save 3DGS",
|
||||
"pos": [
|
||||
1798,
|
||||
459
|
||||
],
|
||||
"size": {
|
||||
"0": 315,
|
||||
"1": 58
|
||||
},
|
||||
"flags": {},
|
||||
"order": 8,
|
||||
"mode": 0,
|
||||
"inputs": [
|
||||
{
|
||||
"name": "gs_ply",
|
||||
"type": "GS_PLY",
|
||||
"link": 13
|
||||
}
|
||||
],
|
||||
"outputs": [
|
||||
{
|
||||
"name": "save_path",
|
||||
"type": "STRING",
|
||||
"links": [
|
||||
11
|
||||
],
|
||||
"shape": 3,
|
||||
"slot_index": 0
|
||||
}
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "[Comfy3D] Save 3DGS"
|
||||
},
|
||||
"widgets_values": [
|
||||
"LGMTest/FatCat.ply"
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": 10,
|
||||
"type": "[Comfy3D] Preview 3DGS",
|
||||
"pos": [
|
||||
2309.8418660992434,
|
||||
768.9263430496216
|
||||
2331,
|
||||
776
|
||||
],
|
||||
"size": [
|
||||
600,
|
||||
500
|
||||
],
|
||||
"flags": {},
|
||||
"order": 10,
|
||||
"order": 11,
|
||||
"mode": 0,
|
||||
"inputs": [
|
||||
{
|
||||
@@ -156,51 +371,19 @@
|
||||
null
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": 15,
|
||||
"type": "[Comfy3D] Preview 3DMesh",
|
||||
"pos": [
|
||||
2312,
|
||||
226
|
||||
],
|
||||
"size": [
|
||||
600,
|
||||
500
|
||||
],
|
||||
"flags": {},
|
||||
"order": 11,
|
||||
"mode": 0,
|
||||
"inputs": [
|
||||
{
|
||||
"name": "mesh_file_path",
|
||||
"type": "STRING",
|
||||
"link": 16,
|
||||
"widget": {
|
||||
"name": "mesh_file_path"
|
||||
}
|
||||
}
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "[Comfy3D] Preview 3DMesh"
|
||||
},
|
||||
"widgets_values": [
|
||||
"",
|
||||
null
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": 16,
|
||||
"type": "[Comfy3D] Convert 3DGS to Mesh with NeRF and Marching Cubes",
|
||||
"pos": [
|
||||
1550,
|
||||
224
|
||||
1566,
|
||||
215
|
||||
],
|
||||
"size": {
|
||||
"0": 411.6000061035156,
|
||||
"1": 65.46688079833984
|
||||
},
|
||||
"flags": {},
|
||||
"order": 7,
|
||||
"order": 8,
|
||||
"mode": 0,
|
||||
"inputs": [
|
||||
{
|
||||
@@ -228,96 +411,15 @@
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": 14,
|
||||
"type": "[Comfy3D] Save 3D Mesh",
|
||||
"pos": [
|
||||
1980,
|
||||
225
|
||||
],
|
||||
"size": {
|
||||
"0": 307.8253173828125,
|
||||
"1": 60.248291015625
|
||||
},
|
||||
"flags": {},
|
||||
"order": 9,
|
||||
"mode": 0,
|
||||
"inputs": [
|
||||
{
|
||||
"name": "mesh",
|
||||
"type": "MESH",
|
||||
"link": 17
|
||||
}
|
||||
],
|
||||
"outputs": [
|
||||
{
|
||||
"name": "save_path",
|
||||
"type": "STRING",
|
||||
"links": [
|
||||
16
|
||||
],
|
||||
"shape": 3,
|
||||
"slot_index": 0
|
||||
}
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "[Comfy3D] Save 3D Mesh"
|
||||
},
|
||||
"widgets_values": [
|
||||
"LGMTest/FatCat.obj"
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": 17,
|
||||
"type": "[Comfy3D] Large Multiview Gaussian Model",
|
||||
"pos": [
|
||||
1270,
|
||||
461
|
||||
],
|
||||
"size": [
|
||||
252,
|
||||
46
|
||||
],
|
||||
"flags": {},
|
||||
"order": 5,
|
||||
"mode": 0,
|
||||
"inputs": [
|
||||
{
|
||||
"name": "multiview_images",
|
||||
"type": "IMAGE",
|
||||
"link": 21
|
||||
},
|
||||
{
|
||||
"name": "lgm_model",
|
||||
"type": "LGM_MODEL",
|
||||
"link": 22
|
||||
}
|
||||
],
|
||||
"outputs": [
|
||||
{
|
||||
"name": "gs_ply",
|
||||
"type": "GS_PLY",
|
||||
"links": [
|
||||
19,
|
||||
20
|
||||
],
|
||||
"shape": 3,
|
||||
"slot_index": 0
|
||||
}
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "[Comfy3D] Large Multiview Gaussian Model"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": 3,
|
||||
"id": 18,
|
||||
"type": "[Comfy3D] MVDream Model",
|
||||
"pos": [
|
||||
792,
|
||||
809,
|
||||
460
|
||||
],
|
||||
"size": {
|
||||
"0": 405.5999755859375,
|
||||
"1": 238
|
||||
"1": 286
|
||||
},
|
||||
"flags": {},
|
||||
"order": 4,
|
||||
@@ -326,17 +428,17 @@
|
||||
{
|
||||
"name": "mvdream_pipe",
|
||||
"type": "DIFFUSERS_PIPE",
|
||||
"link": 1
|
||||
"link": 23
|
||||
},
|
||||
{
|
||||
"name": "reference_image",
|
||||
"type": "IMAGE",
|
||||
"link": 2
|
||||
"link": 24
|
||||
},
|
||||
{
|
||||
"name": "reference_mask",
|
||||
"type": "MASK",
|
||||
"link": 5
|
||||
"link": 25
|
||||
}
|
||||
],
|
||||
"outputs": [
|
||||
@@ -344,7 +446,8 @@
|
||||
"name": "multiview_images",
|
||||
"type": "IMAGE",
|
||||
"links": [
|
||||
21
|
||||
26,
|
||||
27
|
||||
],
|
||||
"shape": 3,
|
||||
"slot_index": 0
|
||||
@@ -356,131 +459,40 @@
|
||||
"widgets_values": [
|
||||
"",
|
||||
"ugly, blurry, pixelated obscure, unnatural colors, poor lighting, dull, unclear, cropped, lowres, low quality, artifacts, duplicate",
|
||||
0,
|
||||
"fixed",
|
||||
5,
|
||||
30,
|
||||
0
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": 1,
|
||||
"type": "[Comfy3D] Load Large Multiview Gaussian Model",
|
||||
"id": 19,
|
||||
"type": "PreviewImage",
|
||||
"pos": [
|
||||
879,
|
||||
752
|
||||
1279,
|
||||
120
|
||||
],
|
||||
"size": [
|
||||
210,
|
||||
246
|
||||
],
|
||||
"size": {
|
||||
"0": 315,
|
||||
"1": 82
|
||||
},
|
||||
"flags": {},
|
||||
"order": 1,
|
||||
"mode": 0,
|
||||
"outputs": [
|
||||
{
|
||||
"name": "lgm_model",
|
||||
"type": "LGM_MODEL",
|
||||
"links": [
|
||||
22
|
||||
],
|
||||
"shape": 3,
|
||||
"slot_index": 0
|
||||
}
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "[Comfy3D] Load Large Multiview Gaussian Model"
|
||||
},
|
||||
"widgets_values": [
|
||||
"model_fp16.safetensors",
|
||||
"big"
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": 5,
|
||||
"type": "InvertMask",
|
||||
"pos": [
|
||||
542,
|
||||
545
|
||||
],
|
||||
"size": {
|
||||
"0": 210,
|
||||
"1": 26
|
||||
},
|
||||
"flags": {},
|
||||
"order": 3,
|
||||
"order": 6,
|
||||
"mode": 0,
|
||||
"inputs": [
|
||||
{
|
||||
"name": "mask",
|
||||
"type": "MASK",
|
||||
"link": 3
|
||||
}
|
||||
],
|
||||
"outputs": [
|
||||
{
|
||||
"name": "MASK",
|
||||
"type": "MASK",
|
||||
"links": [
|
||||
5
|
||||
],
|
||||
"shape": 3,
|
||||
"slot_index": 0
|
||||
"name": "images",
|
||||
"type": "IMAGE",
|
||||
"link": 27
|
||||
}
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "InvertMask"
|
||||
"Node name for S&R": "PreviewImage"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": 2,
|
||||
"type": "[Comfy3D] Load Diffusers Pipeline",
|
||||
"pos": [
|
||||
380,
|
||||
345
|
||||
],
|
||||
"size": {
|
||||
"0": 375.3948669433594,
|
||||
"1": 83.33004760742188
|
||||
},
|
||||
"flags": {},
|
||||
"order": 2,
|
||||
"mode": 0,
|
||||
"outputs": [
|
||||
{
|
||||
"name": "pipe",
|
||||
"type": "DIFFUSERS_PIPE",
|
||||
"links": [
|
||||
1
|
||||
],
|
||||
"shape": 3,
|
||||
"slot_index": 0
|
||||
}
|
||||
],
|
||||
"properties": {
|
||||
"Node name for S&R": "[Comfy3D] Load Diffusers Pipeline"
|
||||
},
|
||||
"widgets_values": [
|
||||
"MVDreamPipeline",
|
||||
"ashawkey/imagedream-ipmv-diffusers"
|
||||
]
|
||||
}
|
||||
],
|
||||
"links": [
|
||||
[
|
||||
1,
|
||||
2,
|
||||
0,
|
||||
3,
|
||||
0,
|
||||
"DIFFUSERS_PIPE"
|
||||
],
|
||||
[
|
||||
2,
|
||||
4,
|
||||
0,
|
||||
3,
|
||||
1,
|
||||
"IMAGE"
|
||||
],
|
||||
[
|
||||
3,
|
||||
4,
|
||||
@@ -489,14 +501,6 @@
|
||||
0,
|
||||
"MASK"
|
||||
],
|
||||
[
|
||||
5,
|
||||
5,
|
||||
0,
|
||||
3,
|
||||
2,
|
||||
"MASK"
|
||||
],
|
||||
[
|
||||
11,
|
||||
9,
|
||||
@@ -545,14 +549,6 @@
|
||||
0,
|
||||
"GS_PLY"
|
||||
],
|
||||
[
|
||||
21,
|
||||
3,
|
||||
0,
|
||||
17,
|
||||
0,
|
||||
"IMAGE"
|
||||
],
|
||||
[
|
||||
22,
|
||||
1,
|
||||
@@ -560,6 +556,46 @@
|
||||
17,
|
||||
1,
|
||||
"LGM_MODEL"
|
||||
],
|
||||
[
|
||||
23,
|
||||
2,
|
||||
0,
|
||||
18,
|
||||
0,
|
||||
"DIFFUSERS_PIPE"
|
||||
],
|
||||
[
|
||||
24,
|
||||
4,
|
||||
0,
|
||||
18,
|
||||
1,
|
||||
"IMAGE"
|
||||
],
|
||||
[
|
||||
25,
|
||||
5,
|
||||
0,
|
||||
18,
|
||||
2,
|
||||
"MASK"
|
||||
],
|
||||
[
|
||||
26,
|
||||
18,
|
||||
0,
|
||||
17,
|
||||
0,
|
||||
"IMAGE"
|
||||
],
|
||||
[
|
||||
27,
|
||||
18,
|
||||
0,
|
||||
19,
|
||||
0,
|
||||
"IMAGE"
|
||||
]
|
||||
],
|
||||
"groups": [],
|
||||
|
||||
-32
@@ -1,32 +0,0 @@
|
||||
@echo off
|
||||
|
||||
set "requirements_txt=%~dp0\requirements.txt"
|
||||
set "python_miniconda_exec=..\..\..\python_miniconda_env\ComfyUI\python.exe"
|
||||
|
||||
echo Installing ComfyUI-3D-Pack ...
|
||||
|
||||
|
||||
if exist "%python_miniconda_exec%" (
|
||||
echo Installing with Miniconda Environment
|
||||
"%python_miniconda_exec%" -s -m pip install -r "%requirements_txt%"
|
||||
|
||||
"%python_miniconda_exec%" -s -m pip install -U xformers --index-url https://download.pytorch.org/whl/cu121
|
||||
|
||||
"%python_miniconda_exec%" -s -m pip install -U https://github.com/ashawkey/diff-gaussian-rasterization
|
||||
"%python_miniconda_exec%" -s -m pip install -U https://github.com/NVlabs/nvdiffrast/
|
||||
"%python_miniconda_exec%" -s -m pip install -U https://github.com/facebookresearch/pytorch3d.git@stable
|
||||
"%python_miniconda_exec%" -s -m pip install -U https://github.com/rusty1s/pytorch_scatter.git
|
||||
"%python_miniconda_exec%" -s -m pip install -U https://github.com/YixunLiang/simple-knn.git
|
||||
"%python_miniconda_exec%" -s -m pip install -U https://github.com/ashawkey/kiuikit.git
|
||||
|
||||
set "python_miniconda_exec=..\..\..\..\..\..\..\python_miniconda_env\ComfyUI\python.exe"
|
||||
|
||||
cd tgs/models/snowflake/pointnet2_ops_lib && "%python_miniconda_exec%" -s setup.py install && cd ../../../../
|
||||
) else (
|
||||
echo ERROR: Cannot find Miniconda Environment "%python_miniconda_exec%"
|
||||
)
|
||||
|
||||
|
||||
echo Install Finished!
|
||||
|
||||
pause
|
||||
@@ -0,0 +1,21 @@
|
||||
@echo off
|
||||
|
||||
set "requirements_txt=%~dp0\requirements.txt"
|
||||
set "requirements_post_txt=%~dp0\requirements_post.txt"
|
||||
set "python_miniconda_exec=..\..\..\python_miniconda_env\ComfyUI\python.exe"
|
||||
|
||||
echo Starting to install ComfyUI-3D-Pack...
|
||||
|
||||
if exist "%python_miniconda_exec%" (
|
||||
echo Installing with Miniconda Environment
|
||||
|
||||
"%python_miniconda_exec%" -s -m pip install -r "%requirements_txt%"
|
||||
"%python_miniconda_exec%" -s -m pip install -r "%requirements_post_txt%"
|
||||
|
||||
) else (
|
||||
echo ERROR: Cannot find Miniconda Environment "%python_miniconda_exec%"
|
||||
)
|
||||
|
||||
echo Install Finished. Press any key to continue...
|
||||
|
||||
pause
|
||||
@@ -63,6 +63,12 @@ SUPPORTED_3DGS_EXTENSIONS = (
|
||||
'.ply',
|
||||
)
|
||||
|
||||
SUPPORTED_CHECKPOINTS_EXTENSIONS = (
|
||||
'.ckpt',
|
||||
'.bin',
|
||||
'.safetensors',
|
||||
)
|
||||
|
||||
ELEVATION_MIN = -90
|
||||
ELEVATION_MAX = 90.0
|
||||
AZIMUTH_MIN = -180.0
|
||||
@@ -1008,9 +1014,14 @@ class Load_Triplane_Gaussian_Transformers:
|
||||
|
||||
@classmethod
|
||||
def INPUT_TYPES(cls):
|
||||
cls.checkpoints_dir_abs = os.path.join(ROOT_PATH, cls.checkpoints_dir)
|
||||
all_models_names = get_list_filenames(cls.checkpoints_dir_abs, SUPPORTED_CHECKPOINTS_EXTENSIONS)
|
||||
if cls.default_ckpt_name not in all_models_names:
|
||||
all_models_names += [cls.default_ckpt_name]
|
||||
|
||||
return {
|
||||
"required": {
|
||||
"model_name": (get_list_filenames(os.path.join(ROOT_PATH, cls.checkpoints_dir)), ),
|
||||
"model_name": (all_models_names, ),
|
||||
},
|
||||
}
|
||||
|
||||
@@ -1031,12 +1042,13 @@ class Load_Triplane_Gaussian_Transformers:
|
||||
cfg: ExperimentConfig = load_config(config_path)
|
||||
|
||||
# Download pre-trained model if it not exist locally
|
||||
ckpt_path = os.path.join(ROOT_PATH, self.checkpoints_dir, model_name)
|
||||
ckpt_path = os.path.join(self.checkpoints_dir_abs, model_name)
|
||||
if not os.path.isfile(ckpt_path):
|
||||
from huggingface_hub import hf_hub_download
|
||||
hf_hub_download(repo_id="VAST-AI/TriplaneGaussian", local_dir=self.checkpoints_dir, filename=self.default_ckpt_name, repo_type="model")
|
||||
cstr(f"[{self.__class__.__name__}] can't find checkpoint {ckpt_path}, will download it from repo VAST-AI/TriplaneGaussian instead").warning.print()
|
||||
ckpt_path = os.path.join(ROOT_PATH, self.checkpoints_dir, self.default_ckpt_name)
|
||||
|
||||
from huggingface_hub import hf_hub_download
|
||||
hf_hub_download(repo_id="VAST-AI/TriplaneGaussian", local_dir=self.checkpoints_dir_abs, filename=self.default_ckpt_name, repo_type="model")
|
||||
ckpt_path = os.path.join(self.checkpoints_dir_abs, self.default_ckpt_name)
|
||||
|
||||
cfg.system.weights=ckpt_path
|
||||
tgs_model = TGS(cfg=cfg.system).to(device)
|
||||
@@ -1158,6 +1170,7 @@ class MVDream_Model:
|
||||
"default": "ugly, blurry, pixelated obscure, unnatural colors, poor lighting, dull, unclear, cropped, lowres, low quality, artifacts, duplicate",
|
||||
"multiline": True
|
||||
}),
|
||||
"seed": ("INT", {"default": 0, "min": 0, "max": 0xffffffffffffffff}),
|
||||
"mv_guidance_scale": ("FLOAT", {"default": 5.0, "min": 0.0, "step": 0.01}),
|
||||
"num_inference_steps": ("INT", {"default": 30, "min": 1}),
|
||||
"elevation": ("FLOAT", {"default": 0.0, "min": ELEVATION_MIN, "max": ELEVATION_MAX, 'step': 0.0001}),
|
||||
@@ -1180,6 +1193,7 @@ class MVDream_Model:
|
||||
reference_mask, # [1, H, W]
|
||||
prompt,
|
||||
prompt_neg,
|
||||
seed,
|
||||
mv_guidance_scale,
|
||||
num_inference_steps,
|
||||
elevation,
|
||||
@@ -1189,12 +1203,14 @@ class MVDream_Model:
|
||||
if len(reference_mask.shape) == 3:
|
||||
reference_mask = reference_mask.squeeze(0)
|
||||
|
||||
generator = torch.manual_seed(seed)
|
||||
|
||||
reference_mask = reference_mask.unsqueeze(2)
|
||||
# give the white background to reference_image
|
||||
reference_image = (reference_image * reference_mask + (1 - reference_mask)).detach().cpu().numpy()
|
||||
|
||||
# generate multi-view images
|
||||
mv_images = mvdream_pipe(prompt, reference_image, negative_prompt=prompt_neg, guidance_scale=mv_guidance_scale, num_inference_steps=num_inference_steps, elevation=elevation)
|
||||
mv_images = mvdream_pipe(prompt, reference_image, generator=generator, negative_prompt=prompt_neg, guidance_scale=mv_guidance_scale, num_inference_steps=num_inference_steps, elevation=elevation)
|
||||
mv_images = torch.from_numpy(np.stack([mv_images[1], mv_images[2], mv_images[3], mv_images[0]], axis=0)).float() # [4, H, W, 3], float32
|
||||
|
||||
return (mv_images,)
|
||||
@@ -1206,9 +1222,14 @@ class Load_Large_Multiview_Gaussian_Model:
|
||||
|
||||
@classmethod
|
||||
def INPUT_TYPES(cls):
|
||||
cls.checkpoints_dir_abs = os.path.join(ROOT_PATH, cls.checkpoints_dir)
|
||||
all_models_names = get_list_filenames(cls.checkpoints_dir_abs, SUPPORTED_CHECKPOINTS_EXTENSIONS)
|
||||
if cls.default_ckpt_name not in all_models_names:
|
||||
all_models_names += [cls.default_ckpt_name]
|
||||
|
||||
return {
|
||||
"required": {
|
||||
"model_name": (get_list_filenames(os.path.join(ROOT_PATH, cls.checkpoints_dir)), ),
|
||||
"model_name": (all_models_names, ),
|
||||
"lgb_config": (['big', 'default', 'small', 'tiny'], )
|
||||
},
|
||||
}
|
||||
@@ -1227,18 +1248,19 @@ class Load_Large_Multiview_Gaussian_Model:
|
||||
lgm_model = LGM(config_defaults[lgb_config])
|
||||
|
||||
# resume pretrained checkpoint
|
||||
ckpt_path = os.path.join(ROOT_PATH, self.checkpoints_dir, model_name)
|
||||
ckpt_path = os.path.join(self.checkpoints_dir_abs, model_name)
|
||||
if not os.path.isfile(ckpt_path):
|
||||
from huggingface_hub import hf_hub_download
|
||||
hf_hub_download(repo_id="ashawkey/LGM", local_dir=self.checkpoints_dir, filename=self.default_ckpt_name, repo_type="model")
|
||||
cstr(f"[{self.__class__.__name__}] can't find checkpoint {ckpt_path}, will download it from repo ashawkey/LGM instead").warning.print()
|
||||
ckpt_path = os.path.join(ROOT_PATH, self.checkpoints_dir, self.default_ckpt_name)
|
||||
|
||||
elif ckpt_path.endswith('safetensors'):
|
||||
from huggingface_hub import hf_hub_download
|
||||
hf_hub_download(repo_id="ashawkey/LGM", local_dir=self.checkpoints_dir_abs, filename=self.default_ckpt_name, repo_type="model")
|
||||
ckpt_path = os.path.join(self.checkpoints_dir_abs, self.default_ckpt_name)
|
||||
|
||||
if ckpt_path.endswith('safetensors'):
|
||||
ckpt = load_file(ckpt_path, device='cpu')
|
||||
else:
|
||||
ckpt = torch.load(ckpt_path, map_location='cpu')
|
||||
|
||||
|
||||
lgm_model.load_state_dict(ckpt, strict=False)
|
||||
cstr(f"[{self.__class__.__name__}] loaded model ckpt from {ckpt_path}").msg.print()
|
||||
|
||||
|
||||
+3
-2
@@ -13,8 +13,8 @@ nerfacc>=0.5.3
|
||||
PyMCubes
|
||||
|
||||
# for use ML models
|
||||
diffusers
|
||||
transformers
|
||||
diffusers>=0.26.1
|
||||
transformers>=4.36.2
|
||||
safetensors
|
||||
|
||||
# for training differentiable tensors
|
||||
@@ -23,6 +23,7 @@ pytorch_msssim
|
||||
# for process images & videos
|
||||
imageio
|
||||
imageio-ffmpeg
|
||||
matplotlib
|
||||
|
||||
# for dmtet and mesh import & export
|
||||
trimesh
|
||||
|
||||
@@ -0,0 +1,13 @@
|
||||
# local
|
||||
./tgs/models/snowflake/pointnet2_ops_lib
|
||||
./simple-knn
|
||||
|
||||
# remote
|
||||
git+https://github.com/ashawkey/diff-gaussian-rasterization
|
||||
git+https://github.com/NVlabs/nvdiffrast/
|
||||
git+https://github.com/facebookresearch/pytorch3d.git@stable
|
||||
git+https://github.com/rusty1s/pytorch_scatter.git
|
||||
git+https://github.com/ashawkey/kiuikit.git
|
||||
|
||||
xformers --index-url https://download.pytorch.org/whl/cu121
|
||||
|
||||
@@ -1,3 +0,0 @@
|
||||
set "python_miniconda_exec=..\..\..\..\..\..\..\python_miniconda_env\ComfyUI\python.exe"
|
||||
|
||||
cd tgs/models/snowflake/pointnet2_ops_lib && "%python_miniconda_exec%" -s setup.py install && cd ../../../../
|
||||
+1
-1
@@ -4,7 +4,7 @@ import os
|
||||
import time
|
||||
import subprocess
|
||||
|
||||
from shared_utils.common_utils import cstr
|
||||
from ..shared_utils.common_utils import cstr
|
||||
|
||||
web = server.web
|
||||
|
||||
|
||||
Reference in New Issue
Block a user