Update preview nodes

- Support custom background color in preview node
- Fix Preview node does not update automatically with same mesh name
- Add system config file that allow custom clients IP address
This commit is contained in:
MrForExample
2024-03-10 20:25:05 +01:00
parent f442e095f9
commit b4e136aa55
14 changed files with 195 additions and 45 deletions
+13 -7
View File
@@ -38,6 +38,7 @@
- **Preview 3DGS and 3D Mesh**: 3D Visualization inside ComfyUI:
- Using [gsplat.js](https://github.com/huggingface/gsplat.js/tree/main) and [three.js](https://github.com/mrdoob/three.js/tree/dev) for 3DGS & 3D Mesh visualization respectively
- Custumizable background base on JS library: [mdbassit/Coloris](https://github.com/mdbassit/Coloris)
<video controls autoplay loop src="https://github.com/MrForExample/ComfyUI-3D-Pack/assets/62230687/9f3c56b1-afb3-4bf1-8845-ab1025a87463"></video>
@@ -64,7 +65,8 @@
- **Gaussian Splatting Orbit Renderer**
- Render 3DGS to images sequences or video, given a 3DGS file and camera poses generated by **Stack Orbit Camera Poses** node
- Bake Multi-View images into UVTexture of given 3D mesh using [Nvdiffrast](https://github.com/NVlabs/nvdiffrast), supports:
- **Multi-View images to UVTexture**
- Bake Multi-View images into UVTexture of given 3D mesh using [Nvdiffrast](https://github.com/NVlabs/nvdiffrast), supports:
- Export to .obj, .ply, .glb
- **NeuS**
@@ -81,6 +83,9 @@
- **Switch Axis for 3DGS & 3D Mesh**
- Since different algorithms likely use different coordinate system, so the ability to re-mapping the axis of coordinate is crucial for passing generated result between differnt nodes.
- **[Customizable system config file](configs/system.conf)**
- Custom clients IP address
## Roadmap:
- [x] Add DMTet algorithm to allow conversion from points cloud(Gaussian/.ply) to mesh (.obj, .ply, .glb)
@@ -92,16 +97,17 @@
- [x] Integrate [LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creation](https://me.kiui.moe/lgm/)
- [ ] Add a general SDS/ISM Optimization algorithm to allow training 3D representations with diffusion model, *The real fun starts here* ;)
- Need to do some in-depth research on Interval Score Matching (ISM), since math behind it makes perfect sense and also there are so many ways we could improve upon the result obtained from [LucidDreamer](https://github.com/EnVision-Research/LucidDreamer)
- [ ] Add camera pose estimation from raw multi-views images
- [ ] Improve 3DGS to Mesh conversion algorithms:
- [ ] Add & Improve a few best MVS algorithms (e.g instant-ngp, NeuS2, GaussianPro, etc.)
- [ ] Improve 3DGS/Nerf to Mesh conversion algorithms:
- Support to training DMTet with images(RGB, Alpha, Normal Map)
- Find better methods to converts 3DGS or Points Cloud to Mesh (Normal maps reconstruction maybe?)
- [ ] Add Structure from motion (SfM) initialization for 3DGS (Better first guess -> Faster convergence & Better result)
- [ ] Add a few best Nerf algorithms (No idea yet, [instant-ngp](https://github.com/NVlabs/instant-ngp) maybe?)
- Add a general SDS/ISM Optimization algorithm to allow training 3D representations with diffusion model
- Need to do some in-depth research on Interval Score Matching (ISM), since math behind it makes perfect sense and also there are so many ways we could improve upon the result obtained from [LucidDreamer](https://github.com/EnVision-Research/LucidDreamer)
- On Hold since runtime cost to generate an is too big (3+hours for an average RTX GPU like 3080)
## Install: