# ComfyUI-InfiniDepth Development Notes

Notes from the initial development and testing of this custom node.

## Setup Experience

### ComfyUI Version
- Tested with ComfyUI v0.18.1 (latest as of 2026-03)
- Both V1 (NODE_CLASS_MAPPINGS) and V3 (comfy_entrypoint) registration styles are supported
- We use V1 style for broader compatibility

### Node Registration
- ComfyUI discovers custom nodes by importing `__init__.py` from directories under `custom_nodes/`
- The module must export `NODE_CLASS_MAPPINGS` dict (node_id -> class) and optionally `NODE_DISPLAY_NAME_MAPPINGS`
- Import errors during node loading are silently swallowed — check "Import times for custom nodes" in startup logs to verify the node loaded

### sys.path for InfiniDepth
- The `InfiniDepth` Python package lives at the repo root, not on pip
- `__init__.py` auto-detects the repo root via `os.path.realpath(__file__)` — this resolves symlinks correctly
- Three install methods supported:
  1. Symlink: `ln -s <repo>/ComfyUI-InfiniDepth <ComfyUI>/custom_nodes/ComfyUI-InfiniDepth` (recommended)
  2. Environment variable: `export INFINIDEPTH_ROOT=/path/to/repo`
  3. Direct copy into `custom_nodes/` (less ideal, requires INFINIDEPTH_ROOT env var)

## Issues Encountered & Solutions

### 1. CUDA requirement in model init
**Problem:** `InfiniDepth/model/model.py` line ~99 does:
```python
if not torch.cuda.is_available():
    raise RuntimeError("CUDA is required to initialize InfiniDepth models.")
self.cuda()
self.eval()
```
This hard-blocks loading on macOS (MPS) or CPU-only systems.

**Solution:** Monkey-patch `_BaseInfiniDepthModel.__init__` in the loader node to:
- Temporarily mock `torch.cuda.is_available()` to return `True`
- Replace `self.cuda()` with a no-op
- Let ComfyUI handle device placement via `comfy.model_management.get_torch_device()`

### 2. Missing grpc module
**Problem:** `InfiniDepth/model/block/implicit_decoder.py` imports `from grpc import insecure_channel`. This is not in `requirements.txt`.

**Solution:** Add `grpcio` to the custom node's `requirements.txt`.

### 3. Missing MoGe dependency
**Problem:** MoGe (Microsoft Monocular Geometry Estimation) is a git-only dependency needed for metric scale recovery when no depth sensor input is provided.

**Solution:** `install.py` handles this via `pip install git+https://github.com/microsoft/MoGe.git`. Users without MoGe will see a clear error at execution time.

### 4. HuggingFace checkpoint download
**Problem:** Checkpoints are hosted at `ritianyu/InfiniDepth` (not `YvanYin/InfiniDepth`). The repo downloads files flat, but InfiniDepth expects them in subdirectories.

**Expected structure:**
```
checkpoints/
  depth/infinidepth.ckpt
  depth/infinidepth_depthsensor.ckpt
  gs/infinidepth_gs.ckpt
  gs/infinidepth_depthsensor_gs.ckpt
  moge-2-vitl-normal/model.pt
  sky/skyseg.onnx
```

### 5. RGBA images (4 channels instead of 3)
**Problem:** Some ComfyUI image sources (e.g. LoadImage with PNG alpha, or nodes that output RGBA) produce `[B, H, W, 4]` tensors. InfiniDepth and MoGe-2 expect exactly 3 channels `[1, 3, H, W]`, causing:
```
ValueError: Expected image shape [1,3,H,W], got (1, 4, 768, 1024)
```

**Solution:** Slice to RGB before processing: `image[i][:, :, :3]`. Alpha channel is irrelevant for depth estimation.

### 6. Port conflict during testing
**Problem:** Default ComfyUI port 8188 was already in use by another instance.

**Solution:** Use `--port 8199` for testing. Always check before assuming the API returned data from your instance.

## IMAGE Format
- ComfyUI: `[B, H, W, C]` float32, channels-last, 0-1 range
- InfiniDepth: `[B, C, H, W]` float32, channels-first, 0-1 range
- Conversion: `.permute(0, 3, 1, 2)` before model, `.permute(0, 2, 3, 1)` after

## Performance
- Full inference on CPU (Apple M4 Max, 36GB): ~340 seconds for a single 1200x900 image at original resolution
- Model loading takes ~10 seconds (1.1GB checkpoint + DINOv3 backbone)
- MoGe-2 metric depth estimation adds ~60 seconds on CPU
- On CUDA GPU this should be dramatically faster (seconds, not minutes)

## Testing Workflow (API)
Minimal workflow JSON for testing via `POST /prompt`:
```json
{
  "prompt": {
    "1": {"class_type": "LoadImage", "inputs": {"image": "courtyard.jpg"}},
    "2": {"class_type": "InfiniDepthModelLoader", "inputs": {
      "model_type": "InfiniDepth",
      "depth_ckpt": "checkpoints/depth/infinidepth.ckpt",
      "moge2_ckpt": "checkpoints/moge-2-vitl-normal/model.pt"
    }},
    "3": {"class_type": "InfiniDepthEstimate", "inputs": {
      "model": ["2", 0], "image": ["1", 0],
      "output_resolution_mode": "original",
      "upsample_ratio": 1, "output_height": 768, "output_width": 1024
    }},
    "4": {"class_type": "PreviewImage", "inputs": {"images": ["3", 0]}},
    "5": {"class_type": "PreviewImage", "inputs": {"images": ["3", 1]}}
  }
}
```