Update preview nodes

- Support custom background color in preview node
- Fix Preview node does not update automatically with same mesh name
- Add system config file that allow custom clients IP address

README.md

@@ -38,6 +38,7 @@
 
 - **Preview 3DGS and 3D Mesh**: 3D Visualization inside ComfyUI:
   - Using [gsplat.js](https://github.com/huggingface/gsplat.js/tree/main) and [three.js](https://github.com/mrdoob/three.js/tree/dev) for 3DGS & 3D Mesh visualization respectively
+  - Custumizable background base on JS library: [mdbassit/Coloris](https://github.com/mdbassit/Coloris)
 
     <video controls autoplay loop src="https://github.com/MrForExample/ComfyUI-3D-Pack/assets/62230687/9f3c56b1-afb3-4bf1-8845-ab1025a87463"></video>
 
@@ -64,7 +65,8 @@
 - **Gaussian Splatting Orbit Renderer**
   - Render 3DGS to images sequences or video, given a 3DGS file and camera poses generated by **Stack Orbit Camera Poses** node
   
-- Bake Multi-View images into UVTexture of given 3D mesh using [Nvdiffrast](https://github.com/NVlabs/nvdiffrast), supports:
+- **Multi-View images to UVTexture**
+  - Bake Multi-View images into UVTexture of given 3D mesh using [Nvdiffrast](https://github.com/NVlabs/nvdiffrast), supports:
   - Export to .obj, .ply, .glb
 
 - **NeuS**
@@ -81,6 +83,9 @@
 - **Switch Axis for 3DGS & 3D Mesh**
   - Since different algorithms likely use different coordinate system, so the ability to re-mapping the axis of coordinate is crucial for passing generated result between differnt nodes.
 
+- **[Customizable system config file](configs/system.conf)**
+  - Custom clients IP address 
+
 ## Roadmap:
 - [x] Add DMTet algorithm to allow conversion from points cloud(Gaussian/.ply) to mesh (.obj, .ply, .glb)
 
@@ -92,16 +97,17 @@
 
 - [x] Integrate [LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creation](https://me.kiui.moe/lgm/)
 
-- [ ] Add a general SDS/ISM Optimization algorithm to allow training 3D representations with diffusion model, *The real fun starts here* ;)
-  - Need to do some in-depth research on Interval Score Matching (ISM), since math behind it makes perfect sense and also there are so many ways we could improve upon the result obtained from [LucidDreamer](https://github.com/EnVision-Research/LucidDreamer)
+- [ ] Add camera pose estimation from raw multi-views images
 
-- [ ] Improve 3DGS to Mesh conversion algorithms:
+- [ ] Add & Improve a few best MVS algorithms (e.g instant-ngp, NeuS2, GaussianPro, etc.)
+
+- [ ] Improve 3DGS/Nerf to Mesh conversion algorithms:
   -  Support to training DMTet with images(RGB, Alpha, Normal Map)
   -  Find better methods to converts 3DGS or Points Cloud to Mesh (Normal maps reconstruction maybe?)
 
-- [ ] Add Structure from motion (SfM) initialization for 3DGS (Better first guess -> Faster convergence & Better result)
-
-- [ ] Add a few best Nerf algorithms (No idea yet, [instant-ngp](https://github.com/NVlabs/instant-ngp) maybe?)
+- Add a general SDS/ISM Optimization algorithm to allow training 3D representations with diffusion model
+  - Need to do some in-depth research on Interval Score Matching (ISM), since math behind it makes perfect sense and also there are so many ways we could improve upon the result obtained from [LucidDreamer](https://github.com/EnVision-Research/LucidDreamer)
+  - On Hold since runtime cost to generate an is too big (3+hours for an average RTX GPU like 3080)
 
 ## Install:
 

__init__.py

@@ -1,6 +1,7 @@
 import os
 import sys
 import folder_paths as comfy_paths
+from pyhocon import ConfigFactory
 
 ROOT_PATH = os.path.join(comfy_paths.get_folder_paths("custom_nodes")[0], "ComfyUI-3D-Pack")
 sys.path.append(ROOT_PATH)
@@ -9,7 +10,16 @@ import shutil
 import __main__
 import importlib
 import inspect
-from .webserver.server import server
+from .webserver.server import server, set_web_conf
+
+conf_path = os.path.join(ROOT_PATH, "configs/system.conf")
+# Configuration
+f = open(conf_path)
+conf_text = f.read()
+f.close()
+sys_conf = ConfigFactory.parse_string(conf_text)
+
+set_web_conf(sys_conf['web'])
 
 NODE_CLASS_MAPPINGS = {}
 NODE_DISPLAY_NAME_MAPPINGS = {}

configs/system.conf

@@ -0,0 +1,6 @@
+web{
+  clients_ip = [
+    127.0.0.1,
+    0.0.0.0
+  ]
+}

web/html/gsVisualizer.html

@@ -4,19 +4,20 @@
   <meta charset="UTF-8" />
   <meta name="viewport" content="width=device-width, initial-scale=1.0" />
   <link rel="stylesheet" href="/extensions/ComfyUI-3D-Pack/style/gsStyle.css">
-  <link type="text/css" rel="stylesheet" href="/extensions/ComfyUI-3D-Pack/style/progressStyle.css">
+  <link type="text/css" rel="stylesheet" href="/extensions/ComfyUI-3D-Pack/style/libs/coloris.min.css">
+  <link type="text/css" rel="stylesheet" href="/extensions/ComfyUI-3D-Pack/style/sharedStyle.css">
 </head>
 <body>
-  <div id="root">
-    <div id="progress-container">
-        <dialog open id="progress-dialog">
-            <p>
-                <label for="progress-indicator">Loading scene...</label>
-            </p>
-            <progress max="100" id="progress-indicator"></progress>
-        </dialog>
-    </div>
+  <div id="progress-container">
+    <dialog open id="progress-dialog">
+        <p>
+            <label for="progress-indicator">Loading scene...</label>
+        </p>
+        <progress max="100" id="progress-indicator"></progress>
+    </dialog>
+  </div>
 
+  <div id="container">
     <canvas id="canvas"></canvas>
 		<script type="importmap">
 			{
@@ -25,7 +26,35 @@
 			  }
 			}
 		</script>
-    <script id="visualizer" type="module" filepath="" crossorigin src="/extensions/ComfyUI-3D-Pack/js/gsVisualizer.js"></script>
+    <script id="visualizer" type="module" filepath="" timestamp="" crossorigin src="/extensions/ComfyUI-3D-Pack/js/gsVisualizer.js"></script>
   </div>
+
+  <script src="/extensions/ComfyUI-3D-Pack/js/libs/coloris.min.js"></script>
+  <script>
+    Coloris({
+      el: '.coloris',
+      theme: 'polaroid',
+      themeMode: 'dark',
+      format: 'rgb',
+      alpha: false,
+      swatches: [
+        'rgb(0, 0, 0)',
+        'rgb(128, 128, 128)',
+        'rgb(255, 255, 255)',
+        'rgb(132, 188, 218)',
+        'rgb(128, 227, 119)',
+        'rgb(213, 96, 98)'
+      ]
+    });
+
+    const canvas = document.getElementById("canvas");
+    const handleClick = () => {
+      Coloris.close();
+      document.activeElement.blur();
+    };
+    canvas.addEventListener('click', handleClick);
+    
+  </script>
+  <input type="text" id="color-picker" class="coloris" value="rgb(0, 0, 0)"></input>
 </body>
 </html>

web/html/threeVisualizer.html

@@ -4,10 +4,11 @@
 		<meta charset="utf-8">
 		<meta name="viewport" content="width=device-width, user-scalable=no, minimum-scale=1.0, maximum-scale=1.0">
 		<link type="text/css" rel="stylesheet" href="/extensions/ComfyUI-3D-Pack/style/threeStyle.css">
-		<link type="text/css" rel="stylesheet" href="/extensions/ComfyUI-3D-Pack/style/progressStyle.css">
+		<link type="text/css" rel="stylesheet" href="/extensions/ComfyUI-3D-Pack/style/libs/coloris.min.css">
+		<link type="text/css" rel="stylesheet" href="/extensions/ComfyUI-3D-Pack/style/sharedStyle.css">
 	</head>
 
-	<body>
+	<body>		
 		<div id="progress-container">
 			<dialog open id="progress-dialog">
 				<p>
@@ -17,16 +18,37 @@
 			</dialog>
 		</div>
 
-		<div id="container"></div>
-		<script type="importmap">
-			{
-			  "imports": {
-				"three": "https://unpkg.com/three@latest/build/three.module.js",
-				"three/addons/": "https://unpkg.com/three@latest/examples/jsm/"
-			  }
-			}
+		<div id="container">
+			<script type="importmap">
+				{
+				  "imports": {
+					"three": "https://unpkg.com/three@latest/build/three.module.js",
+					"three/addons/": "https://unpkg.com/three@latest/examples/jsm/"
+				  }
+				}
+			</script>
+			<script id="visualizer" type="module" filepath="" timestamp="" crossorigin src="/extensions/ComfyUI-3D-Pack/js/threeVisualizer.js"></script>
+		</div>
+
+		<script src="/extensions/ComfyUI-3D-Pack/js/libs/coloris.min.js"></script>
+		<script>
+			Coloris({
+				el: '.coloris',
+				theme: 'polaroid',
+				themeMode: 'dark',
+				format: 'rgb',
+				alpha: false,
+				swatches: [
+					'rgb(0, 0, 0)',
+					'rgb(128, 128, 128)',
+					'rgb(255, 255, 255)',
+					'rgb(132, 188, 218)',
+					'rgb(128, 227, 119)',
+					'rgb(213, 96, 98)'
+				]
+			});
 		</script>
-		<script id="visualizer" type="module" filepath="" crossorigin src="/extensions/ComfyUI-3D-Pack/js/threeVisualizer.js"></script>
+		<input type="text" id="color-picker" class="coloris" value="rgb(0, 0, 0)"></input>
 	</body>
 
 </html>

web/js/gsVisualizer.js

@@ -1,10 +1,12 @@
 import * as SPLAT from 'gsplat';
 import { api } from '/scripts/api.js'
+import {getRGBValue} from '/extensions/ComfyUI-3D-Pack/js/sharedFunctions.js';
 
 const visualizer = document.getElementById("visualizer");
 const canvas = document.getElementById("canvas");
 const progressDialog = document.getElementById("progress-dialog");
 const progressIndicator = document.getElementById("progress-indicator");
+const colorPicker = document.getElementById("color-picker");
 
 const renderer = new SPLAT.WebGLRenderer(canvas);
 const scene = new SPLAT.Scene();
@@ -19,14 +21,14 @@ const handleResize = () => {
 handleResize();
 window.addEventListener("resize", handleResize);
 
-
-var lastFilepath = "";
+var lastTimestamp = "";
 var needUpdate = false;
 
 function frameUpdate() {
 
     var filepath = visualizer.getAttribute("filepath");
-    if (filepath == lastFilepath){
+    var timestamp = visualizer.getAttribute("timestamp");
+    if (timestamp == lastTimestamp){
         if (needUpdate){
             controls.update();
             renderer.render( scene, camera );
@@ -36,8 +38,13 @@ function frameUpdate() {
         needUpdate = false;
         scene.reset();
         progressDialog.open = true;
-        lastFilepath = filepath;
-        main(lastFilepath);
+        lastTimestamp = timestamp;
+        main(filepath);
+    }
+
+    var color = getRGBValue(colorPicker.value);
+    if (color[0] != renderer.backgroundColor.r || color[1] != renderer.backgroundColor.g || color[2] != renderer.backgroundColor.b){
+        renderer.backgroundColor = new SPLAT.Color32(color[0], color[1], color[2], 255);  // It will automatically update background color in preview scene
     }
 };
 

web/js/sharedFunctions.js

@@ -0,0 +1,14 @@
+export function getRGBValue(colorString="rgb(0, 0, 0)", scale01=false) {
+    var color = colorString.split(/[\D]+/).filter(Boolean)
+    for (let index = 0; index < color.length; index++) {
+        color[index] = Number(color[index]);
+    }
+
+    if (scale01) {
+        for (let index = 0; index < color.length; index++) {
+            color[index] = color[index] / 255;
+        }
+    }
+
+    return color;
+}

web/js/threeVisualizer.js

@@ -1,5 +1,6 @@
 import * as THREE from 'three';
-import { api } from '/scripts/api.js'
+import { api } from '/scripts/api.js';
+import {getRGBValue} from '/extensions/ComfyUI-3D-Pack/js/sharedFunctions.js';
 
 import { OrbitControls } from 'three/addons/controls/OrbitControls.js';
 import { RoomEnvironment } from 'three/addons/environments/RoomEnvironment.js';
@@ -13,6 +14,7 @@ const visualizer = document.getElementById("visualizer");
 const container = document.getElementById( 'container' );
 const progressDialog = document.getElementById("progress-dialog");
 const progressIndicator = document.getElementById("progress-indicator");
+const colorPicker = document.getElementById("color-picker");
 
 const renderer = new THREE.WebGLRenderer( { antialias: true } );
 renderer.setPixelRatio( window.devicePixelRatio );
@@ -50,13 +52,14 @@ window.onresize = function () {
 };
 
 
-var lastFilepath = "";
+var lastTimestamp = "";
 var needUpdate = false;
 
 function frameUpdate() {
     
     var filepath = visualizer.getAttribute("filepath");
-    if (filepath == lastFilepath){
+    var timestamp = visualizer.getAttribute("timestamp");
+    if (timestamp == lastTimestamp){
         if (needUpdate){
             controls.update();
             renderer.render( scene, camera );
@@ -66,8 +69,14 @@ function frameUpdate() {
         needUpdate = false;
         scene.clear();
         progressDialog.open = true;
-        lastFilepath = filepath;
-        main(lastFilepath);
+        lastTimestamp = timestamp;
+        main(filepath);
+    }
+
+    var color = getRGBValue(colorPicker.value, true);
+    if (color[0] != scene.background.r || color[1] != scene.background.g || color[2] != scene.background.b){
+        scene.background.setStyle(colorPicker.value);
+        renderer.render( scene, camera ); // Force update background color in preview scene
     }
 }
 

web/style/sharedStyle.css

@@ -33,4 +33,34 @@ progress::-webkit-progress-value {
 
 progress::-moz-progress-bar {
     background-color: #eee;
+}
+
+
+#container {
+    position: absolute;
+    top: 0%;
+    left: 0%;
+    width: 100%;
+    height: 100%;
+}
+
+.clr-field {
+    margin-left: 5px;
+    margin-top: 5px;
+}
+
+.clr-field button {
+    width: 20px;
+    height: 20px;
+    left: 5px;
+    right: auto;
+    border-radius: 5px;
+}
+
+.clr-field input {
+    padding-left: 36px;
+    width: 120px;
+    height: 24px;
+    border: 2px solid #aaa;
+    border-radius: 5px;
 }

web/visualization.js

@@ -17,6 +17,9 @@ class Visualizer {
         const iframeDocument = this.iframe.contentWindow.document
         const previewScript = iframeDocument.getElementById('visualizer')
         previewScript.setAttribute("filepath", filepath)
+
+        const timestamp = Date.now().toString()
+        previewScript.setAttribute("timestamp", timestamp)
     }
 
     remove() {
@@ -132,7 +135,7 @@ function registerVisualizer(nodeType, nodeData, nodeClassName, typeName){
             let Preview3DNode = app.graph._nodes.filter(
                 (wi) => wi.type == nodeClassName
             )
-            let nodeName = `Preview3DNode_${Preview3DNode.length}`
+            let nodeName = `Preview3DNode_${nodeClassName}`
 
             console.log(`[Comfy3D] Create: ${nodeName}`)
 

webserver/server.py

@@ -1,11 +1,11 @@
 import server
 import folder_paths as comfy_paths
 import os
-import time
-import subprocess
 
 from ..shared_utils.common_utils import cstr
 
+ROOT_PATH = os.path.join(comfy_paths.get_folder_paths("custom_nodes")[0], "ComfyUI-3D-Pack")
+
 web = server.web
 
 SUPPORTED_VIEW_EXTENSIONS = (
@@ -19,11 +19,18 @@ SUPPORTED_VIEW_EXTENSIONS = (
     '.splat'
 )
 
+
+web_conf = None
+
+def set_web_conf(new_web_conf):
+    global web_conf
+    web_conf = new_web_conf
+
 @server.PromptServer.instance.routes.get("/extensions/ComfyUI-3D-Pack/html/viewfile")
 async def view_file(request):
     query = request.rel_url.query
     # Security check to see if query client is local
-    if request.remote == "127.0.0.1" and "filepath" in query:
+    if request.remote in web_conf['clients_ip'] and "filepath" in query:
         filepath = query["filepath"]
         
         cstr(f"[Server Query view_file] Get file {filepath}").msg.print()
@@ -31,4 +38,4 @@ async def view_file(request):
         if filepath.lower().endswith(SUPPORTED_VIEW_EXTENSIONS) and os.path.exists(filepath):
             return web.FileResponse(filepath)
     
-    return web.Response(status=404)
+    return web.Response(status=404)