Kelly Yang
3fb16edc48
The Load3D model_file input serializes as COMBO in object_info, which comfy-test's validator skips as a connection slot. That misaligned widgets_values, mapping '3d/cube.obj' onto the width INT and failing VALIDATION. Drive the cube test through Preview3D's string path widget instead, which validates and executes cleanly on CPU runners.
ComfyUI-Recgen
ComfyUI custom nodes for RecGen: turn a single RGB image + depth map + object mask into a 3D mesh and optional Gaussian splat.
Use this extension when you want image-to-3D or RGB-D to 3D inside ComfyUI without writing Python glue code. It targets creators who already have (or can generate) aligned depth and segmentation, and who want .obj / .ply outputs in the ComfyUI output folder.
| License | GPL-3.0 |
| CI test report | comfy-test results (updates after Actions publish to gh-pages) |
| AI / LLM summary | llms.txt · full context |
Features
- RecGen 2D to 3D — one ComfyUI node for full RecGen inference
- RecGen Load Example Inputs — demo RGB, depth, mask, and intrinsics from bundled examples
- macOS Apple Silicon (MPS) — PyTorch
spconvfallback and SDPA attention (noflash-attnrequired) - ComfyUI-native I/O —
IMAGE/MASKtensors, string output path, overlay preview image
Requirements
- Python ≥ 3.10, PyTorch 2.x
- ComfyUI
- Sibling RecGen inference repo (
pip install -e recgen) - ~10 GB disk for first-time HuggingFace weights (
TRI-ML/RecGen) - CUDA recommended; MPS on Apple Silicon supported; CPU works for small/smoke resolutions
Install
ComfyUI (recommended)
cd ComfyUI/custom_nodes
git clone git@github.com:kaili-yang/ComfyUI-Recgen.git
Install RecGen inference and this extension (adjust paths):
pip install -e /path/to/recgen
pip install -e /path/to/ComfyUI/custom_nodes/ComfyUI-Recgen
Restart ComfyUI. Nodes appear under category RecGen.
Development layout
your-workspace/
├── recgen/ # git clone TRI-ML/RecGen
└── ComfyUI/
└── custom_nodes/
└── ComfyUI-Recgen/
Smoke test (no ComfyUI UI)
chmod +x ComfyUI-Recgen/scripts/run_demo.sh
./ComfyUI-Recgen/scripts/run_demo.sh
# or: python3 ComfyUI-Recgen/tests/test_run_node.py
Uses 256×256 example data and matching intrinsics. Outputs: outputs/recgen_outputs/.
Quick workflow
- RecGen Load Example Inputs → RecGen 2D to 3D (demo), or
- Wire your own RGB, depth, mask, and camera fx / fy / cx / cy into RecGen 2D to 3D
| Node | Description |
|---|---|
| RecGen Load Example Inputs | Demo assets from recgen/examples/ |
| RecGen 2D to 3D | Full reconstruction |
Outputs: posed_mesh.obj, overlay.png, optional posed_gaussian.ply under output/recgen_outputs/.
Depth: metric (metres) or uint16 mm PNG. ComfyUI Load Image depth normalized to 0–1 is auto-rescaled when needed.
Documentation
| Doc | Audience |
|---|---|
| FAQ | Common questions |
| Node reference | Inputs / outputs |
| llms.txt | Short summary for AI crawlers |
| llms-full.md | Detailed context for AI assistants |
| AGENTS.md | Coding agents |
FAQ (short)
Do I need RecGen installed separately?
Yes — pip install -e the TRI-ML/RecGen repo; this pack is only the ComfyUI adapter.
Does it work on Mac?
Yes (MPS + CPU fallbacks). First run downloads large weights; use the smoke script at 256² for a quick check.
What inputs are required?
RGB + depth + object mask + pinhole intrinsics (or use the example loader).
More: docs/FAQ.md
Develop
cd ComfyUI-Recgen
pip install -e ".[dev]"
pre-commit install
pytest tests/test_comfyui_recgen.py
Related projects
- TRI-ML/RecGen — upstream model and
recgen_inference - ComfyUI — node-based UI
- ComfyUI Manager — optional installer
Citation / attribution
If you use RecGen weights or method, cite and follow the license of TRI-ML/RecGen. This ComfyUI extension is maintained by Kaili Yang under GPL-3.0.