ComfyUI 3DGS Preview Node

A ComfyUI custom node that takes two images as input and outputs a novel-view preview rendered with 3D Gaussian Splatting.

---

Overview

This node reconstructs a sparse 3D scene from a pair of photographs and renders it from a user-defined viewpoint using Gaussian Splatting. No pre-trained model is required — everything runs from scratch using classical computer vision (feature matching + triangulation) followed by a differentiable Gaussian renderer.

[Image 1] ──┐
             ├──► Feature Matching ──► Camera Pose ──► Triangulation ──► Gaussian Render ──► [Preview]
[Image 2] ──┘

Note: Full 3DGS training typically requires dozens to hundreds of images. With only two images the scene is reconstructed as a sparse point cloud; the output is an informative structural preview, not a dense photorealistic render.

---

Installation

1. Place the node in ComfyUI

# From your ComfyUI root directory
cd custom_nodes
git clone <this-repo-url> ComfyUI_3DGS
# — or — copy the folder manually into custom_nodes/

2. Install dependencies

pip install -r custom_nodes/ComfyUI_3DGS/requirements.txt

3. Restart ComfyUI

After restart, search for "3DGS Preview" in the node browser (under 3D / Gaussian Splatting).

---

Requirements

Package	Minimum Version
torch	2.0.0
opencv-python	4.7.0
numpy	1.24.0
Pillow	9.0.0

GPU (CUDA) is used automatically when available; CPU fallback is supported.

---

Node Reference

Category: 3D / Gaussian Splatting Node name: GaussianSplattingPreview

Inputs

Name	Type	Required	Description
image1	IMAGE	Yes	First input image
image2	IMAGE	Yes	Second input image (different viewpoint of the same scene)
render_width	INT	No	Output image width in pixels (default: 512)
render_height	INT	No	Output image height in pixels (default: 512)
max_gaussians	INT	No	Maximum number of Gaussians (= triangulated points). Range: 100–5000, default: 1000
gaussian_scale	FLOAT	No	Size of each Gaussian relative to focal length. Larger = softer/bigger blobs. Range: 0.001–0.3, default: 0.025
novel_view_yaw	FLOAT	No	Horizontal rotation of the novel viewpoint in degrees. 0 = front, +/− = left/right. Range: −90–90, default: 15
novel_view_pitch	FLOAT	No	Vertical tilt of the novel viewpoint in degrees. Range: −45–45, default: −5
background_color	ENUM	No	Background fill: dark / white / black / transparent (default: dark)

Outputs

Name	Type	Description
rendered_preview	IMAGE	Novel-view image rendered with Gaussian Splatting

---

How It Works

Step 1 — Feature Detection & Matching

SIFT keypoints are detected in both images (falls back to ORB if SIFT is unavailable). Matches are filtered with Lowe's ratio test (threshold = 0.75).

Step 2 — Camera Pose Estimation

The Essential Matrix is computed via RANSAC, then decomposed into rotation R and translation t between the two cameras.

Step 3 — Point Triangulation

Matched feature pairs are triangulated using the two projection matrices, yielding a sparse 3D point cloud. Points are filtered for validity (finite values, positive depth, depth-outlier removal) and normalized to unit scale.

Step 4 — Gaussian Initialization

Each triangulated 3D point becomes one isotropic Gaussian:

• Position — triangulated 3D coordinate
• Color — sampled from image1 at the feature pixel location
• Scale (sigma) — derived from gaussian_scale and depth at render time

Step 5 — Novel-View Rendering

A virtual camera is placed at z = 2.5 (normalized scene units) and rotated by the specified yaw/pitch angles. Gaussians are:

1. Projected onto the screen using a pinhole camera model
2. Sorted back-to-front (painter's algorithm)
3. Rendered using batched weighted-additive splatting on PyTorch tensors
4. Composited over the chosen background using accumulated opacity

alpha(h,w) = 1 − exp(−weight(h,w) × 2.5)
rgb(h,w)   = (Σ gauss_i · color_i) / (Σ gauss_i)
out(h,w)   = rgb · alpha + background · (1 − alpha)

Fallback Behavior

If reconstruction fails (too few keypoints, poor matches, degenerate pose), the node falls back to a side-by-side display of the two input images instead of returning an error.

---

Tips for Best Results

Situation	Recommendation
Blank or nearly blank output	Increase novel_view_yaw closer to 0, or check that images have enough texture
Too sparse / many holes	Increase max_gaussians (up to 5000) or use higher-resolution inputs
Blobs too large	Decrease gaussian_scale (try 0.01–0.015)
Blobs too small / invisible	Increase gaussian_scale (try 0.04–0.08)
Reconstruction always fails	Ensure the two images show the same scene from slightly different viewpoints (10–40° apart works best)
Flat / textureless images	SIFT/ORB need texture to find keypoints; add some surface detail

---

File Structure

ComfyUI_3DGS/
├── __init__.py          # Node registration for ComfyUI
├── gs_preview_node.py   # Full pipeline implementation
├── requirements.txt     # Python dependencies
└── README.md            # This file

---

Limitations

• Two images only — the scene is reconstructed as a sparse point cloud; dense 3DGS training is out of scope.
• No optimization loop — Gaussians are not refined; colors and scales are fixed at initialization.
• Unknown intrinsics — focal length is estimated as max(H, W) × 1.2; distorted or fisheye lenses will degrade accuracy.
• Baseline sensitivity — very small or very large viewpoint differences between the two images reduce reconstruction quality.

---

License

MIT