snomiao-ComfyUI-InfiniDepth/docs/development-notes.md

ComfyUI-InfiniDepth Development Notes

Notes from the initial development and testing of this custom node.

Setup Experience

ComfyUI Version

• Tested with ComfyUI v0.18.1 (latest as of 2026-03)
• Both V1 (NODE_CLASS_MAPPINGS) and V3 (comfy_entrypoint) registration styles are supported
• We use V1 style for broader compatibility

Node Registration

• ComfyUI discovers custom nodes by importing __init__.py from directories under custom_nodes/
• The module must export NODE_CLASS_MAPPINGS dict (node_id -> class) and optionally NODE_DISPLAY_NAME_MAPPINGS
• Import errors during node loading are silently swallowed — check "Import times for custom nodes" in startup logs to verify the node loaded

sys.path for InfiniDepth

• The InfiniDepth Python package lives at the repo root, not on pip
• __init__.py auto-detects the repo root via os.path.realpath(__file__) — this resolves symlinks correctly
• Three install methods supported:
  1. Symlink: ln -s <repo>/ComfyUI-InfiniDepth <ComfyUI>/custom_nodes/ComfyUI-InfiniDepth (recommended)
  2. Environment variable: export INFINIDEPTH_ROOT=/path/to/repo
  3. Direct copy into custom_nodes/ (less ideal, requires INFINIDEPTH_ROOT env var)

Issues Encountered & Solutions

1. CUDA requirement in model init

Problem: InfiniDepth/model/model.py line ~99 does:

if not torch.cuda.is_available():
    raise RuntimeError("CUDA is required to initialize InfiniDepth models.")
self.cuda()
self.eval()

This hard-blocks loading on macOS (MPS) or CPU-only systems.

Solution: Monkey-patch _BaseInfiniDepthModel.__init__ in the loader node to:

• Temporarily mock torch.cuda.is_available() to return True
• Replace self.cuda() with a no-op
• Let ComfyUI handle device placement via comfy.model_management.get_torch_device()

2. Missing grpc module

Problem: InfiniDepth/model/block/implicit_decoder.py imports from grpc import insecure_channel. This is not in requirements.txt.

Solution: Add grpcio to the custom node's requirements.txt.

3. Missing MoGe dependency

Problem: MoGe (Microsoft Monocular Geometry Estimation) is a git-only dependency needed for metric scale recovery when no depth sensor input is provided.

Solution: install.py handles this via pip install git+https://github.com/microsoft/MoGe.git. Users without MoGe will see a clear error at execution time.

4. HuggingFace checkpoint download

Problem: Checkpoints are hosted at ritianyu/InfiniDepth (not YvanYin/InfiniDepth). The repo downloads files flat, but InfiniDepth expects them in subdirectories.

Expected structure:

checkpoints/
  depth/infinidepth.ckpt
  depth/infinidepth_depthsensor.ckpt
  gs/infinidepth_gs.ckpt
  gs/infinidepth_depthsensor_gs.ckpt
  moge-2-vitl-normal/model.pt
  sky/skyseg.onnx

5. RGBA images (4 channels instead of 3)

Problem: Some ComfyUI image sources (e.g. LoadImage with PNG alpha, or nodes that output RGBA) produce [B, H, W, 4] tensors. InfiniDepth and MoGe-2 expect exactly 3 channels [1, 3, H, W], causing:

ValueError: Expected image shape [1,3,H,W], got (1, 4, 768, 1024)

Solution: Slice to RGB before processing: image[i][:, :, :3]. Alpha channel is irrelevant for depth estimation.

6. Port conflict during testing

Problem: Default ComfyUI port 8188 was already in use by another instance.

Solution: Use --port 8199 for testing. Always check before assuming the API returned data from your instance.

IMAGE Format

• ComfyUI: [B, H, W, C] float32, channels-last, 0-1 range
• InfiniDepth: [B, C, H, W] float32, channels-first, 0-1 range
• Conversion: .permute(0, 3, 1, 2) before model, .permute(0, 2, 3, 1) after

Performance

• Full inference on CPU (Apple M4 Max, 36GB): ~340 seconds for a single 1200x900 image at original resolution
• Model loading takes ~10 seconds (1.1GB checkpoint + DINOv3 backbone)
• MoGe-2 metric depth estimation adds ~60 seconds on CPU
• On CUDA GPU this should be dramatically faster (seconds, not minutes)

Testing Workflow (API)

Minimal workflow JSON for testing via POST /prompt:

{
  "prompt": {
    "1": {"class_type": "LoadImage", "inputs": {"image": "courtyard.jpg"}},
    "2": {"class_type": "InfiniDepthModelLoader", "inputs": {
      "model_type": "InfiniDepth",
      "depth_ckpt": "checkpoints/depth/infinidepth.ckpt",
      "moge2_ckpt": "checkpoints/moge-2-vitl-normal/model.pt"
    }},
    "3": {"class_type": "InfiniDepthEstimate", "inputs": {
      "model": ["2", 0], "image": ["1", 0],
      "output_resolution_mode": "original",
      "upsample_ratio": 1, "output_height": 768, "output_width": 1024
    }},
    "4": {"class_type": "PreviewImage", "inputs": {"images": ["3", 0]}},
    "5": {"class_type": "PreviewImage", "inputs": {"images": ["3", 1]}}
  }
}