[00:00] Crash dump logging enabled: /logs/DepthAnythingV3-0902/dev/linux-gpu/crash_dump.log [00:00] [00:00] [1/5] SYNTAX [00:00] ---------------------------------------- [00:00] [DEBUG] server=None, api=None [00:00] Found pyproject.toml (modern format) [00:00] Found requirements.txt (legacy format) [00:00] Unicode check: OK (all characters cp1252-safe) [00:00] Forbidden patterns check: OK [00:00] [SYNTAX] PASSED [00:00] [00:00] [2/5] INSTALL [00:00] ---------------------------------------- [00:00] [DEBUG] server=None, api=None [00:00] Setting up ComfyUI... [00:00] Creating virtual environment... [00:00] Running: uv venv /workspaces/DepthAnythingV3-0902/.venv --python 3.12 [00:00] torch_version='2.11.0' -> pinning torch==2.11.0 torchvision==0.26.0 torchaudio==2.11.0 [00:00] Cloning ComfyUI (latest)... [00:00] Running: git clone --depth 1 https://github.com/comfyanonymous/ComfyUI.git /workspaces/DepthAnythingV3-0902/ComfyUI [00:01] Pinning torch family from https://download.pytorch.org/whl/cu128: torch==2.11.0 torchvision==0.26.0 torchaudio==2.11.0 [00:01] Running: uv pip install --python /workspaces/DepthAnythingV3-0902/.venv/bin/python --index-url https://download.pytorch.org/whl/cu128 --extra-index-url https://pypi.org/simple --index-strategy first-index torch==2.11.0 torchvision==0.26.0 torchaudio==2.11.0 [00:52] Installing ComfyUI requirements... [00:52] Running: uv pip install --python /workspaces/DepthAnythingV3-0902/.venv/bin/python --index-url https://download.pytorch.org/whl/cu128 --extra-index-url https://pypi.org/simple --index-strategy first-index -r /workspaces/DepthAnythingV3-0902/ComfyUI/requirements.txt [01:03] Installing custom node... [01:03] Copying ComfyUI-DepthAnythingV3 to custom_nodes/... [01:03] Installing node requirements... [01:03] Running: uv pip install --python /workspaces/DepthAnythingV3-0902/.venv/bin/python --index-url https://download.pytorch.org/whl/cu128 --extra-index-url https://pypi.org/simple --index-strategy first-index -r /workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes/ComfyUI-DepthAnythingV3/requirements.txt [01:03] Running install.py... [01:03] Running: /workspaces/DepthAnythingV3-0902/.venv/bin/python /workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes/ComfyUI-DepthAnythingV3/install.py [01:04] [01:04] Installing 3 node dependencies... [01:04] Cloning ComfyUI-VideoHelperSuite... [01:05] Installing requirements for ComfyUI-VideoHelperSuite... [01:07] Cloning ComfyUI-KJNodes... [01:09] Installing requirements for ComfyUI-KJNodes... [01:11] Cloning ComfyUI-GeometryPack... [01:14] Installing requirements for ComfyUI-GeometryPack... [01:15] Running install.py for ComfyUI-GeometryPack... [01:18] [comfy-env] _discover: scanning /workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes [01:18] [comfy-env] _discover: ComfyUI-DepthAnythingV3 -> depthanythingv3-nodes (custom_nodes/ComfyUI-DepthAnythingV3/nodes/comfy-env.toml) [01:18] [comfy-env] _discover: ComfyUI-GeometryPack -> geometrypack-nodes (custom_nodes/ComfyUI-GeometryPack/nodes/comfy-env.toml) [01:18] [comfy-env] _discover: ComfyUI-KJNodes: no comfy-env.toml found [01:18] [comfy-env] _discover: ComfyUI-VideoHelperSuite: no comfy-env.toml found [01:18] [comfy-env] Workspace: /root/.ce [01:18] [comfy-env] ComfyUI: /workspaces/DepthAnythingV3-0902/ComfyUI [01:18] [comfy-env] Found 2 node config(s): [01:18] - depthanythingv3-nodes <- custom_nodes/ComfyUI-DepthAnythingV3/nodes/comfy-env.toml (python=host) [01:18] - geometrypack-nodes <- custom_nodes/ComfyUI-GeometryPack/nodes/comfy-env.toml (python=host) [01:18] [comfy-env] Bootstrap interpreter has python 3.12 + torch 2.11.0 (cpu). Looking for cuda-only wheels matching cpu/torch2.11/cp312. [01:18] [comfy-env] GPU: Detection: smi [01:18] Driver: 580.159.04 [01:18] CUDA: 13.0 [01:18] Recommended: CUDA 13.0 [01:18] [01:18] GPU 0: NVIDIA GeForce RTX 5060 (sm_120) [Blackwell] 8151MB [01:18] [comfy-env] Comfyui feature -> torch 2.11.0 from https://download.pytorch.org/whl/cu130 (CUDA 13.0 via GPU driver) [01:18] [comfy-env] cuda-wheels: 3 package(s) need a matched (cuda, torch, python) combo: ['flash_attn', 'sageattention', 'cumesh'] [01:18] [comfy-env] cuda-wheels tier 1 (bootstrap): probing cu13.0/torch2.11/cp312 [01:18] [cuda-wheels] Looking up flash_attn: cu130 torch2.11 cp312 manylinux linux [01:18] [cuda-wheels] Found: flash_attn-2.8.3+cu130torch2.11-cp312-cp312-manylinux_2_34_x86_64.manylinux_2_35_x86_64.whl [01:18] [cuda-wheels] Looking up sageattention: cu130 torch2.11 cp312 manylinux linux [01:18] [cuda-wheels] Found: sageattention-2.2.0+cu130torch2.11-cp312-cp312-manylinux_2_34_x86_64.manylinux_2_35_x86_64.whl [01:18] [cuda-wheels] Looking up cumesh: cu130 torch2.11 cp312 manylinux linux [01:18] [cuda-wheels] Found: cumesh-0.0.1+cu130torch2.11-cp312-cp312-manylinux_2_35_x86_64.whl [01:18] [comfy-env] cuda-wheels combo: cu13.0/torch2.11 (bootstrap matches; per-node envs will pin to this) [01:18] [comfy-env] Host glibc 2.35 -> system-requirements [01:18] [cuda-wheels] Looking up flash_attn: cu130 torch2.11 cp312 manylinux linux [01:18] [cuda-wheels] Found: flash_attn-2.8.3+cu130torch2.11-cp312-cp312-manylinux_2_34_x86_64.manylinux_2_35_x86_64.whl [01:18] [cuda-wheels] Looking up sageattention: cu130 torch2.11 cp312 manylinux linux [01:19] [cuda-wheels] Found: sageattention-2.2.0+cu130torch2.11-cp312-cp312-manylinux_2_34_x86_64.manylinux_2_35_x86_64.whl [01:19] [comfy-env] depthanythingv3-nodes: cuda-wheels deferred for post-pixi install (flash_attn, sageattention) [01:19] [cuda-wheels] Looking up cumesh: cu130 torch2.11 cp312 manylinux linux [01:19] [cuda-wheels] Found: cumesh-0.0.1+cu130torch2.11-cp312-cp312-manylinux_2_35_x86_64.whl [01:19] [comfy-env] geometrypack-nodes: cuda-wheels deferred for post-pixi install (cumesh) [01:19] Generated /root/.ce/pixi.toml [01:19] [comfy-env] Installing 2 environment(s): [01:19] - depthanythingv3-nodes (python=host, 11 pypi deps, 2 cuda wheels) [01:19] - geometrypack-nodes (python=host, 14 pypi deps, 1 cuda wheels) [01:19] [comfy-env] Running `pixi install -e depthanythingv3-nodes -e geometrypack-nodes`... [01:25] WARN Skipped running the post-link scripts because `run-post-link-scripts` = `false` [01:25] - bin/.librsvg-pre-unlink.sh [01:25] To enable them, run: [01:25] pixi config set --local run-post-link-scripts insecure [01:25] More info: [01:25] https://pixi.sh/latest/reference/pixi_configuration/#run-post-link-scripts [01:42] WARN Skipped running the post-link scripts because `run-post-link-scripts` = `false` [01:42] - bin/.librsvg-pre-unlink.sh [01:42] To enable them, run: [01:42] pixi config set --local run-post-link-scripts insecure [01:42] More info: [01:42] https://pixi.sh/latest/reference/pixi_configuration/#run-post-link-scripts [02:24] The following environments have been installed: depthanythingv3-nodes, geometrypack-nodes. [02:24] [comfy-env] depthanythingv3-nodes: installing cuda-wheels with --no-deps (flash_attn, sageattention) [02:24] Using Python 3.12.13 environment at: .pixi/envs/depthanythingv3-nodes [02:26] Resolved 2 packages in 1.90s [02:26] Downloading sageattention (46.9MiB) [02:26] Downloading flash-attn (231.5MiB) [02:28] Downloaded sageattention [02:33] Downloaded flash-attn [02:33] Prepared 2 packages in 6.69s [02:33] Installed 2 packages in 6ms [02:33] + flash-attn==2.8.3+cu130torch2.11 (from https://github.com/PozzettiAndrea/cuda-wheels/releases/download/flash_attn-latest/flash_attn-2.8.3%2Bcu130torch2.11-cp312-cp312-manylinux_2_34_x86_64.manylinux_2_35_x86_64.whl) [02:33] + sageattention==2.2.0+cu130torch2.11 (from https://github.com/PozzettiAndrea/cuda-wheels/releases/download/sageattention-latest/sageattention-2.2.0%2Bcu130torch2.11-cp312-cp312-manylinux_2_34_x86_64.manylinux_2_35_x86_64.whl) [02:33] [comfy-env] geometrypack-nodes: installing cuda-wheels with --no-deps (cumesh) [02:33] Using Python 3.12.13 environment at: .pixi/envs/geometrypack-nodes [02:34] Resolved 1 package in 831ms [02:34] Downloading cumesh (16.5MiB) [02:34] Downloaded cumesh [02:34] Prepared 1 package in 291ms [02:34] Installed 1 package in 2ms [02:34] + cumesh==0.0.1+cu130torch2.11 (from https://github.com/PozzettiAndrea/cuda-wheels/releases/download/cumesh-latest/cumesh-0.0.1%2Bcu130torch2.11-cp312-cp312-manylinux_2_35_x86_64.whl) [02:34] [comfy-env] Recorded install hash 43376444b298 -> /root/.ce/install.hash [02:34] [comfy-env] Install log: /root/.ce/install.log [02:34] [02:34] Installation complete! [02:34] [02:34] [requirements] Re-installing main package requirements... [02:34] Installing requirements for ComfyUI-DepthAnythingV3... [02:34] [comfy-env] _discover: scanning /workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes [02:34] [comfy-env] _discover: ComfyUI-DepthAnythingV3 -> depthanythingv3-nodes (custom_nodes/ComfyUI-DepthAnythingV3/nodes/comfy-env.toml) [02:34] [comfy-env] _discover: ComfyUI-GeometryPack -> geometrypack-nodes (custom_nodes/ComfyUI-GeometryPack/nodes/comfy-env.toml) [02:34] [comfy-env] _discover: ComfyUI-KJNodes: no comfy-env.toml found [02:34] [comfy-env] _discover: ComfyUI-VideoHelperSuite: no comfy-env.toml found [02:34] [comfy-env] Workspace inputs unchanged (hash 43376444b298) -- skipping install. Delete /root/.ce/install.hash to force. [02:34] [02:34] Installation complete! [02:34] Installing 3 node dependency(ies)... [02:34] ComfyUI-VideoHelperSuite from Kosinkadink/ComfyUI-VideoHelperSuite [02:34] ComfyUI-VideoHelperSuite already exists, skipping... [02:34] ComfyUI-KJNodes from kijai/ComfyUI-KJNodes [02:34] ComfyUI-KJNodes already exists, skipping... [02:34] ComfyUI-GeometryPack from PozzettiAndrea/ComfyUI-GeometryPack [02:34] ComfyUI-GeometryPack already exists, skipping... [02:34] Installing validation endpoint... [02:34] Cloning PozzettiAndrea/ComfyUI-validate-endpoint... [02:34] Running: git clone --depth 1 https://github.com/PozzettiAndrea/ComfyUI-validate-endpoint.git /workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes/ComfyUI-validate-endpoint [02:35] CUDA packages absent (will mock): flash_attn, sageattention [02:35] [INSTALL] PASSED [02:35] [02:35] [3/5] REGISTRATION [02:35] ---------------------------------------- [02:35] [DEBUG] server=None, api=None [02:35] Starting ComfyUI server... [02:35] Starting ComfyUI server on port 41896... [02:35] CUDA mock packages: flash_attn, sageattention [02:35] Starting ComfyUI server on port 41896... [02:35] Waiting for server to be ready (timeout: 600s)... [02:36] [ComfyUI] setup plugin alembic.autogenerate.schemas [02:36] [ComfyUI] setup plugin alembic.autogenerate.tables [02:36] [ComfyUI] setup plugin alembic.autogenerate.types [02:36] [ComfyUI] setup plugin alembic.autogenerate.constraints [02:36] [ComfyUI] setup plugin alembic.autogenerate.defaults [02:36] [ComfyUI] setup plugin alembic.autogenerate.comments [02:36] [ComfyUI] [comfy-env] ComfyUI-GeometryPack: 1 isolation env(s): [02:36] [ComfyUI] [comfy-env] nodes -> /workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes/ComfyUI-GeometryPack/nodes [02:36] [ComfyUI] [comfy-env] Workspace: /root/.ce [02:36] [ComfyUI] [comfy-env] env: /root/.ce/.pixi/envs/geometrypack-nodes [OK] [02:36] [ComfyUI] [comfy-env] prestartup complete [02:36] [ComfyUI] [comfy-env] ComfyUI-DepthAnythingV3: 1 isolation env(s): [02:36] [ComfyUI] [comfy-env] nodes -> /workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes/ComfyUI-DepthAnythingV3/nodes [02:36] [ComfyUI] [comfy-env] env: /root/.ce/.pixi/envs/depthanythingv3-nodes [OK] [02:36] [ComfyUI] [comfy-env] prestartup complete [02:36] [ComfyUI] [02:36] [ComfyUI] Prestartup times for custom nodes: [02:36] [ComfyUI] 0.0 seconds: /workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes/ComfyUI-DepthAnythingV3 [02:36] [ComfyUI] 0.2 seconds: /workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes/ComfyUI-GeometryPack [02:36] [ComfyUI] [02:41] [ComfyUI] Found comfy_kitchen backend cuda: {'available': True, 'disabled': False, 'unavailable_reason': None, 'capabilities': ['apply_rope', 'apply_rope1', 'dequantize_nvfp4', 'dequantize_per_tensor_fp8', 'quantize_mxfp8', 'quantize_nvfp4', 'quantize_per_tensor_fp8', 'scaled_mm_nvfp4']} [02:41] [ComfyUI] Found comfy_kitchen backend triton: {'available': True, 'disabled': True, 'unavailable_reason': None, 'capabilities': ['apply_rope', 'apply_rope1', 'dequantize_nvfp4', 'dequantize_per_tensor_fp8', 'quantize_mxfp8', 'quantize_nvfp4', 'quantize_per_tensor_fp8']} [02:41] [ComfyUI] Found comfy_kitchen backend eager: {'available': True, 'disabled': False, 'unavailable_reason': None, 'capabilities': ['apply_rope', 'apply_rope1', 'dequantize_mxfp8', 'dequantize_nvfp4', 'dequantize_per_tensor_fp8', 'quantize_mxfp8', 'quantize_nvfp4', 'quantize_per_tensor_fp8', 'scaled_mm_mxfp8', 'scaled_mm_nvfp4']} [02:41] [ComfyUI] Checkpoint files will always be loaded safely. [02:41] [ComfyUI] Total VRAM 7672 MB, total RAM 23913 MB [02:41] [ComfyUI] pytorch version: 2.11.0+cu130 [02:41] [ComfyUI] Set vram state to: NORMAL_VRAM [02:41] [ComfyUI] Device: cuda:0 NVIDIA GeForce RTX 5060 : cudaMallocAsync [02:41] [ComfyUI] Using async weight offloading with 2 streams [02:41] [ComfyUI] Enabled pinned memory 21522.0 [02:42] [ComfyUI] Using pytorch attention [02:46] [ComfyUI] aimdo: /project/src-posix/cuda-funchooks.c:52:DEBUG:aimdo_setup_hooks: hooks successfully installed [02:46] [ComfyUI] aimdo: /project/src/control.c:151:INFO:comfy-aimdo inited for GPU: NVIDIA GeForce RTX 5060 (VRAM: 7672 MB) [02:46] [ComfyUI] DynamicVRAM support detected and enabled [02:46] [ComfyUI] Python version: 3.12.13 (main, May 8 2026, 18:32:55) [Clang 22.1.3 ] [02:46] [ComfyUI] ComfyUI version: 0.21.1 [02:46] [ComfyUI] comfy-aimdo version: 0.3.0 [02:46] [ComfyUI] comfy-kitchen version: 0.2.8 [02:46] [ComfyUI] ****** User settings have been changed to be stored on the server instead of browser storage. ****** [02:46] [ComfyUI] ****** For multi-user setups add the --multi-user CLI argument to enable multiple user profiles. ****** [02:46] [ComfyUI] comfyui-frontend-package version: 1.43.18 [02:46] [ComfyUI] comfyui-workflow-templates version: 0.9.77 [02:46] [ComfyUI] comfyui-embedded-docs version: 0.5.0 [02:46] [ComfyUI] comfy-kitchen version: 0.2.8 [02:46] [ComfyUI] comfy-aimdo version: 0.3.0 [02:46] [ComfyUI] [Prompt Server] web root: /workspaces/DepthAnythingV3-0902/.venv/lib/python3.12/site-packages/comfyui_frontend_package/static [02:46] [ComfyUI] Asset seeder disabled [02:48] [ComfyUI] loading... [02:48] [ComfyUI] calling register_nodes [02:48] [ComfyUI] [comfy-env] Version: 0.3.89 [02:48] [ComfyUI] [comfy-env] register_nodes: ComfyUI-GeometryPack [02:48] [ComfyUI] [comfy-env] ComfyUI source dir: /workspaces/DepthAnythingV3-0902/ComfyUI [02:48] [ComfyUI] [comfy-env] Found 1 isolation env(s) [02:48] [ComfyUI] [comfy-env] Scanning nodes metadata (isolation root)... [02:48] [ComfyUI] [comfy-env] Metadata scan: /root/.pixi/bin/pixi run --as-is --manifest-path /root/.ce/pixi.toml -e geometrypack-nodes python /tmp/comfy_meta_re4ohy0w.py /workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes/ComfyUI-GeometryPack nodes /tmp/comfy_meta_out_fcmr_6sj.pkl [02:48] [ComfyUI] [comfy-env] Scan env PATH for nodes: [02:48] [ComfyUI] [comfy-env] [0] /root/.local/bin [02:48] [ComfyUI] [comfy-env] [1] /usr/local/nvidia/bin [02:48] [ComfyUI] [comfy-env] [2] /usr/local/cuda/bin [02:48] [ComfyUI] [comfy-env] [3] /usr/local/sbin [02:48] [ComfyUI] [comfy-env] [4] /usr/local/bin [02:48] [ComfyUI] [comfy-env] [5] /usr/sbin [02:48] [ComfyUI] [comfy-env] [6] /usr/bin [02:48] [ComfyUI] [comfy-env] [7] /sbin [02:48] [ComfyUI] [comfy-env] [8] /bin [02:53] [ComfyUI] [comfy-env] Metadata scan stderr for nodes: [02:53] [ComfyUI] [comfy-env] [meta-scan] importing nodes from /workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes/ComfyUI-GeometryPack [02:53] [ComfyUI] [comfy-env] ERROR:root:Failed to import comfy_kitchen, Error: No module named 'comfy_kitchen', fp8 and fp4 support will not be available. [02:53] [ComfyUI] [comfy-env] [meta-scan] import OK [02:53] [ComfyUI] [comfy-env] Scanned nodes: 125 nodes (5.2s) [02:53] [ComfyUI] [comfy-env] Scanned nodes root: 125 nodes (5.2s) [02:53] [ComfyUI] [comfy-env] Imported nodes root: 125 nodes (isolation) [02:53] [ComfyUI] [comfy-env] Registered 125 total nodes [02:53] [ComfyUI] [comfy-env] Version: 0.3.89 [02:53] [ComfyUI] [comfy-env] register_nodes: ComfyUI-DepthAnythingV3 [02:53] [ComfyUI] [comfy-env] ComfyUI source dir: /workspaces/DepthAnythingV3-0902/ComfyUI [02:53] [ComfyUI] [comfy-env] Found 1 isolation env(s) [02:53] [ComfyUI] [comfy-env] Scanning nodes metadata (isolation root)... [02:53] [ComfyUI] [comfy-env] Metadata scan: /root/.pixi/bin/pixi run --as-is --manifest-path /root/.ce/pixi.toml -e depthanythingv3-nodes python /tmp/comfy_meta_qw2d5cec.py /workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes/ComfyUI-DepthAnythingV3 nodes /tmp/comfy_meta_out_j1ngx00g.pkl [02:53] [ComfyUI] [comfy-env] Scan env PATH for nodes: [02:53] [ComfyUI] [comfy-env] [0] /root/.local/bin [02:53] [ComfyUI] [comfy-env] [1] /usr/local/nvidia/bin [02:53] [ComfyUI] [comfy-env] [2] /usr/local/cuda/bin [02:53] [ComfyUI] [comfy-env] [3] /usr/local/sbin [02:53] [ComfyUI] [comfy-env] [4] /usr/local/bin [02:53] [ComfyUI] [comfy-env] [5] /usr/sbin [02:53] [ComfyUI] [comfy-env] [6] /usr/bin [02:53] [ComfyUI] [comfy-env] [7] /sbin [02:53] [ComfyUI] [comfy-env] [8] /bin [02:56] [ComfyUI] [comfy-env] Metadata scan stderr for nodes: [02:56] [ComfyUI] [comfy-env] [meta-scan] importing nodes from /workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes/ComfyUI-DepthAnythingV3 [02:56] [ComfyUI] [comfy-env] ERROR:root:Failed to import comfy_kitchen, Error: No module named 'comfy_kitchen', fp8 and fp4 support will not be available. [02:56] [ComfyUI] [comfy-env] [meta-scan] import OK [02:56] [ComfyUI] [comfy-env] Scanned nodes: 15 nodes (2.8s) [02:56] [ComfyUI] [comfy-env] Scanned nodes root: 15 nodes (2.8s) [02:56] [ComfyUI] [comfy-env] Imported nodes root: 15 nodes (isolation) [02:56] [ComfyUI] [comfy-env] Registered 15 total nodes [02:56] [ComfyUI] [02:56] [ComfyUI] Import times for custom nodes: [02:56] [ComfyUI] 0.0 seconds: /workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes/websocket_image_save.py [02:56] [ComfyUI] 0.0 seconds: /workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes/ComfyUI-validate-endpoint [02:56] [ComfyUI] 0.0 seconds: /workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes/ComfyUI-VideoHelperSuite [02:56] [ComfyUI] 0.1 seconds: /workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes/ComfyUI-KJNodes [02:56] [ComfyUI] 2.9 seconds: /workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes/ComfyUI-DepthAnythingV3 [02:56] [ComfyUI] 5.4 seconds: /workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes/ComfyUI-GeometryPack [02:56] [ComfyUI] [02:56] [ComfyUI] Context impl SQLiteImpl. [02:56] [ComfyUI] Will assume non-transactional DDL. [02:56] [ComfyUI] Context impl SQLiteImpl. [02:56] [ComfyUI] Will assume non-transactional DDL. [02:56] [ComfyUI] Running upgrade -> 0001_assets, Initial assets schema [02:56] [ComfyUI] Revision ID: 0001_assets [02:56] [ComfyUI] Revises: None [02:56] [ComfyUI] Create Date: 2025-12-10 00:00:00 [02:56] [ComfyUI] Running upgrade 0001_assets -> 0002_merge_to_asset_references, Merge AssetInfo and AssetCacheState into unified asset_references table. [02:56] [ComfyUI] Running upgrade 0002_merge_to_asset_references -> 0003_add_metadata_job_id, Add system_metadata and job_id columns to asset_references. [02:56] [ComfyUI] Change preview_id FK from assets.id to asset_references.id. [02:56] [ComfyUI] Database upgraded from None to 0003_add_metadata_job_id [02:56] [ComfyUI] Starting server [02:56] [ComfyUI] [02:56] [ComfyUI] To see the GUI go to: http://127.0.0.1:41896 [02:57] Server responding, waiting for nodes to load... [03:17] Server is ready! [03:17] Checking for import errors in server logs... [03:17] No import errors detected [03:17] Found 1135 registered nodes [03:17] [REGISTRATION] PASSED [03:17] [03:17] [4/5] INSTANTIATION [03:17] ---------------------------------------- [03:17] [DEBUG] server=, api= [03:17] Testing node constructors... [03:17] All 15 node(s) instantiated successfully! [03:17] [INSTANTIATION] PASSED [03:17] [03:17] [5/5] EXECUTION [03:17] ---------------------------------------- [03:17] [DEBUG] server=, api= [03:17] GPU runner - will execute 7 workflow(s) [03:17] Running 7 workflow(s) (all with videos)... [03:17] Installing screenshot dependencies (playwright, pillow)... [03:19] Packages installed, downloading chromium browser... [03:35] Screenshot dependencies installed successfully [03:35] Starting headless browser... [03:35] Installing Vulkan stack (apt)... [03:35] Vulkan install error: [Errno 2] No such file or directory: 'sudo' [03:35] GPU detected (no Vulkan) -- using ANGLE/OpenGL rendering [03:35] Capturing execution frames: advanced.json [03:37] [ComfyUI] [DEPRECATION WARNING] Detected import of deprecated legacy API: /extensions/core/widgetInputs.js. This is likely caused by a custom node extension using outdated APIs. Please update your extensions or contact the extension author for an updated version. [03:42] Frame 1 saved (frame_000.jpg, t=0.0s) [03:42] Validating workflow... [03:42] [ComfyUI] === /validate endpoint called === [03:42] [ComfyUI] Received JSON with keys: ['prompt'] [03:42] [ComfyUI] Prompt has 11 nodes [03:42] [ComfyUI] Node 2: LoadImage [03:42] [ComfyUI] Inputs: {"image": "bridge.jpg"} [03:42] [ComfyUI] Node 3: DownloadAndLoadDepthAnythingV3Model [03:42] Queuing workflow for execution... [03:42] [ComfyUI] Inputs: {"model": "da3_large.safetensors", "precision": "auto", "attention": "auto"} [03:42] [ComfyUI] Node 4: PreviewImage [03:42] [ComfyUI] Inputs: {"images": ["12", 0]} [03:42] [ComfyUI] Node 6: PreviewImage [03:42] [ComfyUI] Inputs: {"images": ["12", 1]} [03:42] [ComfyUI] Node 7: PreviewImage [03:42] [ComfyUI] Inputs: {"images": ["12", 2]} [03:42] [ComfyUI] Node 8: PreviewImage [03:42] [ComfyUI] Inputs: {"images": ["12", 3]} [03:42] [ComfyUI] Node 9: PreviewAny [03:42] [ComfyUI] Inputs: {"source": ["12", 6]} [03:42] [ComfyUI] Node 10: PreviewAny [03:42] [ComfyUI] Inputs: {"source": ["12", 5]} [03:42] [ComfyUI] Node 11: MaskPreview [03:42] [ComfyUI] Inputs: {"mask": ["12", 7]} [03:42] [ComfyUI] Node 12: DepthAnything_V3 [03:42] [ComfyUI] Inputs: {"normalization_mode": "V2-Style", "resize_method": "resize", "invert_depth": false, "keep_model_size": false, "da3_model": ["3", 0], "images": ["2", 0]} [03:42] [ComfyUI] Node 13: PreviewImage [03:42] [ComfyUI] Inputs: {"images": ["12", 4]} [03:42] [ComfyUI] Generated prompt_id: 0681e1b7-6bcf-474d-b7d6-6ecb277fd4a8 [03:42] [ComfyUI] Calling execution.validate_prompt()... [03:42] [ComfyUI] Validation result: valid=True [03:42] [ComfyUI] === Validation PASSED === [03:42] [ComfyUI] got prompt [03:42] [ComfyUI] [comfy-env] Running DownloadAndLoadDepthAnythingV3Model: model: 'da3_large.safetensors', precision: 'auto', attention: 'auto' [03:42] [ComfyUI] [comfy-env] SubprocessWorker: /root/.ce/.pixi/envs/depthanythingv3-nodes/bin/python [03:42] [ComfyUI] [VRAM] Before DownloadAndLoadDepthAnythingV3Model: 620 / 7672 MB [03:42] [ComfyUI] [comfy-env] env_vars: {'COMFYUI_BASE': '/workspaces/DepthAnythingV3-0902/ComfyUI'} [03:42] [ComfyUI] [SubprocessWorker] call_method: nodes.load_model.DownloadAndLoadDepthAnythingV3Model.EXECUTE_NORMALIZED [03:42] [ComfyUI] [ComfyUI-DepthAnythingV3] python: /root/.ce/.pixi/envs/depthanythingv3-nodes/bin/python [03:42] [ComfyUI] [SubprocessWorker] acquired lock, ensuring started... [03:42] [ComfyUI] [ComfyUI-DepthAnythingV3] sys_path sent to worker: ['/workspaces/DepthAnythingV3-0902/ComfyUI', '/workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes/ComfyUI-DepthAnythingV3', '/root/.ce/.pixi/envs/depthanythingv3-nodes/lib/python3.12/site-packages', '/workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes/ComfyUI-DepthAnythingV3'] [03:42] [ComfyUI] [SubprocessWorker] is_pixi=True, python=/root/.ce/.pixi/envs/depthanythingv3-nodes/bin/python [03:42] [ComfyUI] [SubprocessWorker] launching cmd=['/root/.pixi/bin/pixi', 'run', '--as-is', '--manifest-path', '/root/.ce/pixi.toml', '-e', 'depthanythingv3-nodes', 'python', '/tmp/comfyui_pvenv_lusourms/persistent_worker.py', 'abstract://comfy_worker_d7bb9d2f6ef8']... [03:42] [ComfyUI] [SubprocessWorker] socket_addr=abstract://comfy_worker_d7bb9d2f6ef8 (abstract, no filesystem path) [03:43] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] ERROR: Failed to import comfy_kitchen, Error: No module named 'comfy_kitchen', fp8 and fp4 support will not be available. [03:43] [ComfyUI] [SubprocessWorker] worker started/confirmed [03:43] [ComfyUI] [SubprocessWorker] serializing kwargs to shm... [03:43] [ComfyUI] [SubprocessWorker] created 0 shm blocks [03:43] [ComfyUI] [SubprocessWorker] sending request via socket... [03:43] [ComfyUI] [SubprocessWorker] call_id=1 sending call_method [03:44] Frame 2 saved (frame_001.jpg, t=2.4s) [03:44] [capture-periodic] freeze=2ms shot=128ms unfreeze=8ms saved=True [03:44] [ComfyUI] [03:46] Frame 3 saved (frame_002.jpg, t=4.7s) [03:46] [capture-periodic] freeze=2ms shot=137ms unfreeze=3ms saved=True [03:47] [capture-loop] iter=50 t=5.7s state={'complete': False, 'error': None, 'executedCount': 0, 'wsState': 1} eval_ms=2 ram=4.2/23.4GB (18%) swap=1.8/32.0GB (6%) cpu=8% browser_rss=0.86GB [03:51] [ComfyUI] model.safetensors: 0%| | 0.00/1.64G [00:00 cpu: 3405 / 7672 MB [04:13] [ComfyUI] [ComfyUI-DepthAnythingV3] Health check: ping (timeout=5.0s)... [04:13] Capturing execution frames: advanced_3d.json [04:13] [ComfyUI] [ComfyUI-DepthAnythingV3] Health check: ok [04:13] [ComfyUI] [SubprocessWorker] call_id=3 sending model_to_device [04:14] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [VRAM] -274MB (now 519MB) reserved=2560MB peak=1505MB [04:14] [ComfyUI] [comfy-env] [VRAM] After offload 'DepthAnything3Net_1' (783 MB): 909 / 7672 MB [04:19] Frame 1 saved (frame_000.jpg, t=0.0s) [04:19] Validating workflow... [04:19] [ComfyUI] === /validate endpoint called === [04:19] [ComfyUI] Received JSON with keys: ['prompt'] [04:19] Queuing workflow for execution... [04:19] [ComfyUI] Prompt has 6 nodes [04:19] [ComfyUI] Node 2: LoadImage [04:19] [ComfyUI] Inputs: {"image": "kaffi.jpg"} [04:19] [ComfyUI] Node 3: DownloadAndLoadDepthAnythingV3Model [04:19] [ComfyUI] Inputs: {"model": "da3_large.safetensors", "precision": "auto", "attention": "auto"} [04:19] [ComfyUI] Node 21: DepthAnything_V3 [04:19] [ComfyUI] Inputs: {"normalization_mode": "Raw", "resize_method": "resize", "invert_depth": false, "keep_model_size": false, "da3_model": ["3", 0], "images": ["2", 0]} [04:19] [ComfyUI] Node 22: DA3_ToPointCloud [04:19] [ComfyUI] Inputs: {"confidence_threshold": 0.2, "downsample": 1, "allow_around_1": 1, "filter_outliers": false, "outlier_percentage": 5, "depth_raw": ["21", 0], "confidence": ["21", 1], "intrinsics": ["21", 6], "sky_ma [04:19] [ComfyUI] Node 23: DA3_SavePointCloud [04:19] [ComfyUI] Inputs: {"filename_prefix": "pointcloud", "pointcloud": ["22", 0]} [04:19] [ComfyUI] Node 25: DA3_PreviewPointCloud [04:19] [ComfyUI] Inputs: {"file_path": ["23", 0], "color_mode": "RGB", "preview_vtk": ""} [04:19] [ComfyUI] Generated prompt_id: 46a4ae7e-59d8-4fe2-a60e-9e4653f5866e [04:19] [ComfyUI] Calling execution.validate_prompt()... [04:19] [ComfyUI] Validation result: valid=True [04:19] [ComfyUI] === Validation PASSED === [04:19] [ComfyUI] got prompt [04:19] [ComfyUI] [comfy-env] Running DownloadAndLoadDepthAnythingV3Model: model: 'da3_large.safetensors', precision: 'auto', attention: 'auto' [04:19] [ComfyUI] [VRAM] Before DownloadAndLoadDepthAnythingV3Model: 909 / 7672 MB [04:19] [ComfyUI] [SubprocessWorker] call_method: nodes.load_model.DownloadAndLoadDepthAnythingV3Model.EXECUTE_NORMALIZED [04:19] [ComfyUI] [SubprocessWorker] acquired lock, ensuring started... [04:19] [ComfyUI] [ComfyUI-DepthAnythingV3] python: /root/.ce/.pixi/envs/depthanythingv3-nodes/bin/python [04:19] [ComfyUI] [ComfyUI-DepthAnythingV3] Health check: ping (timeout=5.0s)... [04:19] [ComfyUI] [ComfyUI-DepthAnythingV3] sys_path sent to worker: ['/workspaces/DepthAnythingV3-0902/ComfyUI', '/workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes/ComfyUI-DepthAnythingV3', '/root/.ce/.pixi/envs/depthanythingv3-nodes/lib/python3.12/site-packages', '/workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes/ComfyUI-DepthAnythingV3'] [04:19] [ComfyUI] [ComfyUI-DepthAnythingV3] Health check: failed [04:19] [ComfyUI] [SubprocessWorker] is_pixi=True, python=/root/.ce/.pixi/envs/depthanythingv3-nodes/bin/python [04:19] [ComfyUI] [ComfyUI-DepthAnythingV3] Socket unhealthy, restarting worker... [04:19] [ComfyUI] [SubprocessWorker] launching cmd=['/root/.pixi/bin/pixi', 'run', '--as-is', '--manifest-path', '/root/.ce/pixi.toml', '-e', 'depthanythingv3-nodes', 'python', '/tmp/comfyui_pvenv_lusourms/persistent_worker.py', 'abstract://comfy_worker_b392e3fa4b3e']... [04:19] [ComfyUI] [comfy-env] Invalidated 1 stale model patchers (will be cleaned up during next unload) [04:19] [ComfyUI] [SubprocessWorker] socket_addr=abstract://comfy_worker_b392e3fa4b3e (abstract, no filesystem path) [04:21] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] ERROR: Failed to import comfy_kitchen, Error: No module named 'comfy_kitchen', fp8 and fp4 support will not be available. [04:21] [ComfyUI] [SubprocessWorker] worker started/confirmed [04:21] [ComfyUI] [SubprocessWorker] serializing kwargs to shm... [04:21] [ComfyUI] [SubprocessWorker] created 0 shm blocks [04:21] [ComfyUI] [SubprocessWorker] sending request via socket... [04:21] [ComfyUI] [SubprocessWorker] call_id=4 sending call_method [04:21] [ComfyUI] [SubprocessWorker] got response: ok [04:21] [ComfyUI] [comfy-env] [VRAM] Before offload 'DepthAnything3Net_1' (783 MB) -> cpu: 737 / 7672 MB [04:21] [ComfyUI] [ComfyUI-DepthAnythingV3] Health check: ping (timeout=5.0s)... [04:21] [ComfyUI] [ComfyUI-DepthAnythingV3] Health check: ok [04:21] [ComfyUI] [SubprocessWorker] call_id=5 sending model_to_device [04:21] [ComfyUI] Model 'DepthAnything3Net_1' gone from worker (restarted?), treating as offloaded [04:21] [ComfyUI] [comfy-env] [VRAM] After offload 'DepthAnything3Net_1' (783 MB): 737 / 7672 MB [04:21] [ComfyUI] [comfy-env] DownloadAndLoadDepthAnythingV3Model done (1.94s): result: NodeOutput [04:21] [ComfyUI] [VRAM] After DownloadAndLoadDepthAnythingV3Model: 737 / 7672 MB [04:21] [ComfyUI] [comfy-env] Running DepthAnything_V3: normalization_mode: 'Raw', resize_method: 'resize', invert_depth: False, keep_model_size: False, da3_model: dict, images: torch.float32 [1,533,800,3] cpu [04:21] [ComfyUI] [VRAM] Before DepthAnything_V3: 737 / 7672 MB [04:21] [ComfyUI] [SubprocessWorker] call_method: nodes.nodes_inference.DepthAnything_V3.EXECUTE_NORMALIZED [04:21] [ComfyUI] [SubprocessWorker] acquired lock, ensuring started... [04:21] [ComfyUI] [ComfyUI-DepthAnythingV3] Health check: ping (timeout=5.0s)... [04:21] [ComfyUI] [ComfyUI-DepthAnythingV3] Health check: ok [04:21] [ComfyUI] [SubprocessWorker] worker started/confirmed [04:21] [ComfyUI] [comfy-env] PRE-SERIALIZE 'images' shape: torch.Size([1, 533, 800, 3]) [04:21] [ComfyUI] [SubprocessWorker] serializing kwargs to shm... [04:21] [ComfyUI] [SubprocessWorker] created 0 shm blocks [04:21] [ComfyUI] [SubprocessWorker] sending request via socket... [04:21] [ComfyUI] [SubprocessWorker] call_id=6 sending call_method [04:21] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] _from_shm got dict with keys: ['__type__', 'strategy', 'parent_pid', 'fd', 'storage_size'] [04:21] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] DESERIALIZE TensorRef: tensor_size=[1, 533, 800, 3] [04:21] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] DESERIALIZED tensor shape: torch.Size([1, 533, 800, 3]) [04:21] Frame 2 saved (frame_001.jpg, t=2.3s) [04:22] [capture-periodic] freeze=2ms shot=140ms unfreeze=5ms saved=True [04:22] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] WARNING: No comfy kitchen, using old apply_rope functions. [04:24] Frame 3 saved (frame_002.jpg, t=4.6s) [04:24] [capture-periodic] freeze=2ms shot=129ms unfreeze=5ms saved=True [04:24] [ComfyUI] [SubprocessWorker] callback 'request_vram_budget' from call_id=6 [04:24] [ComfyUI] [comfy-env] VRAM request: 0.82GB, free before eviction: 7.27GB [04:24] [ComfyUI] [comfy-env] VRAM after eviction: 7.27GB [04:24] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [VRAM] +548MB (now 548MB) reserved=576MB peak=548MB [04:25] [capture-loop] iter=50 t=6.0s state={'complete': False, 'error': None, 'executedCount': 0, 'wsState': 1} eval_ms=3 ram=5.3/23.4GB (23%) swap=1.8/32.0GB (6%) cpu=11% browser_rss=0.95GB [04:25] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=6 [04:25] [ComfyUI] [SubprocessWorker] got response: ok [04:25] [ComfyUI] [comfy-env] Created 1 model patchers: ['DepthAnything3Net_1'] [04:25] [ComfyUI] [comfy-env] DepthAnything_V3 done (4.15s): result: NodeOutput [04:25] [ComfyUI] [VRAM] After DepthAnything_V3: 2699 / 7672 MB [04:25] [ComfyUI] [VRAM] Loaded: SubprocessModel (783 MB) [04:25] [ComfyUI] [comfy-env] Running DA3_ToPointCloud: confidence_threshold: 0.2, downsample: 1, allow_around_1: True, filter_outliers: False, outlier_percentage: 5.0, depth_raw: torch.float32 [1,532,800,3] cpu, confidence: torch.float32 [1,532,800,3] cpu, intrinsics: '{\n "intrinsics": [\n {\n "image_0": [\n ..., sky_mask: torch.float32 [1,532,800] cpu, source_image: torch.float32 [1,532,800,3] cpu [04:25] [ComfyUI] [VRAM] Before DA3_ToPointCloud: 2699 / 7672 MB [04:25] [ComfyUI] [VRAM] Loaded: SubprocessModel (783 MB) [04:25] [ComfyUI] [SubprocessWorker] call_method: nodes.nodes_3d.DA3_ToPointCloud.EXECUTE_NORMALIZED [04:25] [ComfyUI] [SubprocessWorker] acquired lock, ensuring started... [04:25] [ComfyUI] [ComfyUI-DepthAnythingV3] Health check: ping (timeout=5.0s)... [04:25] [ComfyUI] [ComfyUI-DepthAnythingV3] Health check: ok [04:25] [ComfyUI] [SubprocessWorker] worker started/confirmed [04:25] [ComfyUI] [comfy-env] PRE-SERIALIZE 'depth_raw' shape: torch.Size([1, 532, 800, 3]) [04:25] [ComfyUI] [comfy-env] PRE-SERIALIZE 'confidence' shape: torch.Size([1, 532, 800, 3]) [04:25] [ComfyUI] [comfy-env] PRE-SERIALIZE 'sky_mask' shape: torch.Size([1, 532, 800]) [04:25] [ComfyUI] [comfy-env] PRE-SERIALIZE 'source_image' shape: torch.Size([1, 532, 800, 3]) [04:25] [ComfyUI] [SubprocessWorker] serializing kwargs to shm... [04:25] [ComfyUI] [SubprocessWorker] created 0 shm blocks [04:25] [ComfyUI] [SubprocessWorker] sending request via socket... [04:25] [ComfyUI] [SubprocessWorker] call_id=7 sending call_method [04:25] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] _from_shm got dict with keys: ['__type__', 'strategy', 'parent_pid', 'fd', 'storage_size'] [04:25] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] DESERIALIZE TensorRef: tensor_size=[1, 532, 800, 3] [04:25] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] DESERIALIZED tensor shape: torch.Size([1, 532, 800, 3]) [04:25] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] _from_shm got dict with keys: ['__type__', 'strategy', 'parent_pid', 'fd', 'storage_size'] [04:25] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] DESERIALIZE TensorRef: tensor_size=[1, 532, 800, 3] [04:25] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] DESERIALIZED tensor shape: torch.Size([1, 532, 800, 3]) [04:25] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] _from_shm got dict with keys: ['__type__', 'strategy', 'parent_pid', 'fd', 'storage_size'] [04:25] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] DESERIALIZE TensorRef: tensor_size=[1, 532, 800] [04:25] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] DESERIALIZED tensor shape: torch.Size([1, 532, 800]) [04:25] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] _from_shm got dict with keys: ['__type__', 'strategy', 'parent_pid', 'fd', 'storage_size'] [04:25] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] DESERIALIZE TensorRef: tensor_size=[1, 532, 800, 3] [04:25] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] DESERIALIZED tensor shape: torch.Size([1, 532, 800, 3]) [04:25] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [VRAM] +244MB (now 792MB) reserved=1856MB peak=1353MB [04:26] [ComfyUI] [SubprocessWorker] got response: ok [04:26] [ComfyUI] [comfy-env] DA3_ToPointCloud done (0.17s): result: NodeOutput [04:26] [ComfyUI] [VRAM] After DA3_ToPointCloud: 2699 / 7672 MB [04:26] [ComfyUI] [VRAM] Loaded: SubprocessModel (783 MB) [04:26] [ComfyUI] [comfy-env] Running DA3_SavePointCloud: filename_prefix: 'pointcloud', pointcloud: list[1] [04:26] [ComfyUI] [VRAM] Before DA3_SavePointCloud: 2699 / 7672 MB [04:26] [ComfyUI] [VRAM] Loaded: SubprocessModel (783 MB) [04:26] [ComfyUI] [SubprocessWorker] call_method: nodes.nodes_3d.DA3_SavePointCloud.EXECUTE_NORMALIZED [04:26] [ComfyUI] [SubprocessWorker] acquired lock, ensuring started... [04:26] [ComfyUI] [ComfyUI-DepthAnythingV3] Health check: ping (timeout=5.0s)... [04:26] [ComfyUI] [ComfyUI-DepthAnythingV3] Health check: ok [04:26] [ComfyUI] [SubprocessWorker] worker started/confirmed [04:26] [ComfyUI] [SubprocessWorker] serializing kwargs to shm... [04:26] [ComfyUI] [SubprocessWorker] created 0 shm blocks [04:26] [ComfyUI] [SubprocessWorker] sending request via socket... [04:26] [ComfyUI] [SubprocessWorker] call_id=8 sending call_method [04:26] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] _from_shm got dict with keys: ['__type__', 'strategy', 'parent_pid', 'fd', 'storage_size'] [04:26] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] DESERIALIZE TensorRef: tensor_size=[310588, 3] [04:26] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] DESERIALIZED tensor shape: torch.Size([310588, 3]) [04:26] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] _from_shm got dict with keys: ['__type__', 'strategy', 'parent_pid', 'fd', 'storage_size'] [04:26] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] DESERIALIZE TensorRef: tensor_size=[310588] [04:26] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] DESERIALIZED tensor shape: torch.Size([310588]) [04:26] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] _from_shm got dict with keys: ['__type__', 'strategy', 'parent_pid', 'fd', 'storage_size'] [04:26] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] DESERIALIZE TensorRef: tensor_size=[310588, 3] [04:26] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] DESERIALIZED tensor shape: torch.Size([310588, 3]) [04:26] Frame 4 saved (frame_003.jpg, t=7.1s) [04:26] [capture-periodic] freeze=2ms shot=144ms unfreeze=4ms saved=True [04:28] [ComfyUI] [SubprocessWorker] got response: ok [04:28] [ComfyUI] [comfy-env] DA3_SavePointCloud done (2.35s): result: NodeOutput [04:28] [ComfyUI] [VRAM] After DA3_SavePointCloud: 2699 / 7672 MB [04:28] [ComfyUI] [VRAM] Loaded: SubprocessModel (783 MB) [04:28] [ComfyUI] [comfy-env] Running DA3_PreviewPointCloud: file_path: '/workspaces/DepthAnythingV3-0902/ComfyUI/output/pointclo..., color_mode: 'RGB' [04:28] [ComfyUI] [VRAM] Before DA3_PreviewPointCloud: 2699 / 7672 MB [04:28] [ComfyUI] [VRAM] Loaded: SubprocessModel (783 MB) [04:28] [ComfyUI] [SubprocessWorker] call_method: nodes.preview_nodes.DA3_PreviewPointCloud.EXECUTE_NORMALIZED [04:28] [ComfyUI] [SubprocessWorker] acquired lock, ensuring started... [04:28] [ComfyUI] [ComfyUI-DepthAnythingV3] Health check: ping (timeout=5.0s)... [04:28] [ComfyUI] [ComfyUI-DepthAnythingV3] Health check: ok [04:28] [ComfyUI] [SubprocessWorker] worker started/confirmed [04:28] [ComfyUI] [SubprocessWorker] serializing kwargs to shm... [04:28] [ComfyUI] [SubprocessWorker] created 0 shm blocks [04:28] [ComfyUI] [SubprocessWorker] sending request via socket... [04:28] [ComfyUI] [SubprocessWorker] call_id=9 sending call_method [04:28] Node executed (2 total), capturing... [04:28] [ComfyUI] [SubprocessWorker] got response: ok [04:28] [ComfyUI] [comfy-env] DA3_PreviewPointCloud done (0.05s): result: NodeOutput [04:28] [ComfyUI] [VRAM] After DA3_PreviewPointCloud: 2699 / 7672 MB [04:28] [ComfyUI] [VRAM] Loaded: SubprocessModel (783 MB) [04:28] [ComfyUI] Prompt executed in 8.71 seconds [04:28] [ComfyUI] [comfy-env] [VRAM] Before offload 'DepthAnything3Net_1' (783 MB) -> cpu: 2699 / 7672 MB [04:28] [ComfyUI] [ComfyUI-DepthAnythingV3] Health check: ping (timeout=5.0s)... [04:28] [ComfyUI] [ComfyUI-DepthAnythingV3] Health check: ok [04:28] [ComfyUI] [SubprocessWorker] call_id=10 sending model_to_device [04:28] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [VRAM] -224MB (now 569MB) reserved=1856MB peak=1353MB [04:28] [ComfyUI] [comfy-env] [VRAM] After offload 'DepthAnything3Net_1' (783 MB): 907 / 7672 MB [04:29] Frame 5 saved (frame_004.jpg, t=9.1s) [04:29] [capture-node] freeze=1225ms shot=93ms unfreeze=3ms saved=True [04:30] Execution complete (t=10.6s) [04:30] Captured 5 unique frames over 10.55s [04:30] [debug] iframes: 1 [04:30] [debug] iframe[0]: src=http://127.0.0.1:41896/extensions/ComfyUI-DepthAnythingV3/viewer_vtk.html?v=1778, canvas=486x394, rafFrozen=False [04:30] [timing] trigger_3d_previews: 0.0s [04:32] Saved high-quality screenshot: advanced_3d_executed.png [04:32] [timing] final_screenshot: 0.3s [04:32] Capturing execution frames: advanced_3d_multiview.json [04:38] Frame 1 saved (frame_000.jpg, t=0.0s) [04:38] Validating workflow... [04:38] [ComfyUI] === /validate endpoint called === [04:38] [ComfyUI] Received JSON with keys: ['prompt'] [04:38] Queuing workflow for execution... [04:38] [ComfyUI] Prompt has 8 nodes [04:38] [ComfyUI] Node 3: DownloadAndLoadDepthAnythingV3Model [04:38] [ComfyUI] Inputs: {"model": "da3_large.safetensors", "precision": "auto", "attention": "auto"} [04:38] [ComfyUI] Node 21: LoadImageDataSetFromFolder [04:38] [ComfyUI] Inputs: {"folder": "SOH"} [04:38] [ComfyUI] Node 27: PreviewImage [04:38] [ComfyUI] Inputs: {"images": ["21", 0]} [04:38] [ComfyUI] Node 38: DA3_MultiViewPointCloud [04:38] [ComfyUI] Inputs: {"extrinsics": ["39", 4], "intrinsics": ["39", 5], "confidence_threshold": 0.3, "downsample": 4, "use_icp": false, "allow_around_1": false, "filter_outliers": true, "outlier_percentage": 5, "depths": [04:38] [ComfyUI] Node 39: DepthAnythingV3_MultiView [04:38] [ComfyUI] Inputs: {"normalization_mode": "Raw", "resize_method": "resize", "invert_depth": false, "keep_model_size": false, "da3_model": ["3", 0], "images": ["21", 0]} [04:38] [ComfyUI] Node 44: DA3_SavePointCloud [04:38] [ComfyUI] Inputs: {"filename_prefix": "pointcloud", "pointcloud": ["38", 0]} [04:38] [ComfyUI] Node 45: PreviewAny [04:38] [ComfyUI] Inputs: {"source": ["44", 0]} [04:38] [ComfyUI] Node 48: DA3_PreviewPointCloud [04:38] [ComfyUI] Inputs: {"file_path": ["44", 0], "color_mode": "RGB", "preview_vtk": ""} [04:38] [ComfyUI] Generated prompt_id: 258038a0-f52f-4788-9312-a3ca5e272ec7 [04:38] [ComfyUI] Calling execution.validate_prompt()... [04:38] [ComfyUI] Validation result: valid=True [04:38] [ComfyUI] === Validation PASSED === [04:38] [ComfyUI] got prompt [04:39] [ComfyUI] [comfy-env] Running DownloadAndLoadDepthAnythingV3Model: model: 'da3_large.safetensors', precision: 'auto', attention: 'auto' [04:39] [ComfyUI] [VRAM] Before DownloadAndLoadDepthAnythingV3Model: 907 / 7672 MB [04:39] [ComfyUI] [ComfyUI-DepthAnythingV3] python: /root/.ce/.pixi/envs/depthanythingv3-nodes/bin/python [04:39] [ComfyUI] [SubprocessWorker] call_method: nodes.load_model.DownloadAndLoadDepthAnythingV3Model.EXECUTE_NORMALIZED [04:39] [ComfyUI] [ComfyUI-DepthAnythingV3] sys_path sent to worker: ['/workspaces/DepthAnythingV3-0902/ComfyUI', '/workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes/ComfyUI-DepthAnythingV3', '/root/.ce/.pixi/envs/depthanythingv3-nodes/lib/python3.12/site-packages', '/workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes/ComfyUI-DepthAnythingV3'] [04:39] [ComfyUI] [SubprocessWorker] acquired lock, ensuring started... [04:39] [ComfyUI] [SubprocessWorker] is_pixi=True, python=/root/.ce/.pixi/envs/depthanythingv3-nodes/bin/python [04:39] [ComfyUI] [ComfyUI-DepthAnythingV3] Health check: ping (timeout=5.0s)... [04:39] [ComfyUI] [SubprocessWorker] launching cmd=['/root/.pixi/bin/pixi', 'run', '--as-is', '--manifest-path', '/root/.ce/pixi.toml', '-e', 'depthanythingv3-nodes', 'python', '/tmp/comfyui_pvenv_lusourms/persistent_worker.py', 'abstract://comfy_worker_21a5fc171fb8']... [04:39] [ComfyUI] [ComfyUI-DepthAnythingV3] Health check: failed [04:39] [ComfyUI] [SubprocessWorker] socket_addr=abstract://comfy_worker_21a5fc171fb8 (abstract, no filesystem path) [04:39] [ComfyUI] [ComfyUI-DepthAnythingV3] Socket unhealthy, restarting worker... [04:39] [ComfyUI] [comfy-env] Invalidated 1 stale model patchers (will be cleaned up during next unload) [04:39] Node executed (1 total), capturing... [04:39] Frame 2 saved (frame_001.jpg, t=0.6s) [04:39] [capture-node] freeze=2ms shot=127ms unfreeze=2ms saved=True [04:40] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] ERROR: Failed to import comfy_kitchen, Error: No module named 'comfy_kitchen', fp8 and fp4 support will not be available. [04:40] [ComfyUI] [SubprocessWorker] worker started/confirmed [04:40] [ComfyUI] [SubprocessWorker] serializing kwargs to shm... [04:40] [ComfyUI] [SubprocessWorker] created 0 shm blocks [04:40] [ComfyUI] [SubprocessWorker] sending request via socket... [04:40] [ComfyUI] [SubprocessWorker] call_id=11 sending call_method [04:40] [ComfyUI] [SubprocessWorker] got response: ok [04:40] [ComfyUI] [comfy-env] [VRAM] Before offload 'DepthAnything3Net_1' (783 MB) -> cpu: 737 / 7672 MB [04:40] [ComfyUI] [ComfyUI-DepthAnythingV3] Health check: ping (timeout=5.0s)... [04:40] [ComfyUI] [ComfyUI-DepthAnythingV3] Health check: ok [04:40] [ComfyUI] [SubprocessWorker] call_id=12 sending model_to_device [04:40] [ComfyUI] Model 'DepthAnything3Net_1' gone from worker (restarted?), treating as offloaded [04:40] [ComfyUI] [comfy-env] [VRAM] After offload 'DepthAnything3Net_1' (783 MB): 737 / 7672 MB [04:40] [ComfyUI] [comfy-env] DownloadAndLoadDepthAnythingV3Model done (1.70s): result: NodeOutput [04:40] [ComfyUI] [VRAM] After DownloadAndLoadDepthAnythingV3Model: 737 / 7672 MB [04:40] [ComfyUI] [comfy-env] Running DepthAnythingV3_MultiView: normalization_mode: 'Raw', resize_method: 'resize', invert_depth: False, keep_model_size: False, da3_model: dict, images: torch.float32 [1,680,1208,3] cpu [04:40] [ComfyUI] [VRAM] Before DepthAnythingV3_MultiView: 737 / 7672 MB [04:40] [ComfyUI] [SubprocessWorker] call_method: nodes.nodes_multiview.DepthAnythingV3_MultiView.EXECUTE_NORMALIZED [04:40] [ComfyUI] [SubprocessWorker] acquired lock, ensuring started... [04:40] [ComfyUI] [ComfyUI-DepthAnythingV3] Health check: ping (timeout=5.0s)... [04:40] [ComfyUI] [ComfyUI-DepthAnythingV3] Health check: ok [04:40] [ComfyUI] [SubprocessWorker] worker started/confirmed [04:40] [ComfyUI] [comfy-env] PRE-SERIALIZE 'images' shape: torch.Size([1, 680, 1208, 3]) [04:40] [ComfyUI] [SubprocessWorker] serializing kwargs to shm... [04:40] [ComfyUI] [SubprocessWorker] created 0 shm blocks [04:40] [ComfyUI] [SubprocessWorker] sending request via socket... [04:40] [ComfyUI] [SubprocessWorker] call_id=13 sending call_method [04:40] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] _from_shm got dict with keys: ['__type__', 'strategy', 'parent_pid', 'fd', 'storage_size'] [04:40] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] DESERIALIZE TensorRef: tensor_size=[1, 680, 1208, 3] [04:40] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] DESERIALIZED tensor shape: torch.Size([1, 680, 1208, 3]) [04:41] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] WARNING: No comfy kitchen, using old apply_rope functions. [04:41] Frame 3 saved (frame_002.jpg, t=2.9s) [04:41] [capture-periodic] freeze=1ms shot=132ms unfreeze=4ms saved=True [04:43] [ComfyUI] [SubprocessWorker] callback 'request_vram_budget' from call_id=13 [04:43] [ComfyUI] [comfy-env] VRAM request: 0.82GB, free before eviction: 7.27GB [04:43] [ComfyUI] [comfy-env] VRAM after eviction: 7.27GB [04:43] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [VRAM] +506MB (now 506MB) reserved=512MB peak=506MB [04:43] Frame 4 saved (frame_003.jpg, t=5.1s) [04:43] [capture-periodic] freeze=2ms shot=122ms unfreeze=6ms saved=True [04:44] [capture-loop] iter=50 t=6.1s state={'complete': False, 'error': None, 'executedCount': 1, 'wsState': 1} eval_ms=2 ram=5.2/23.4GB (22%) swap=1.8/32.0GB (6%) cpu=14% browser_rss=0.89GB [04:44] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [VRAM] +326MB (now 832MB) reserved=2880MB peak=1240MB [04:44] [ComfyUI] [SubprocessWorker] got response: ok [04:44] [ComfyUI] [comfy-env] Created 1 model patchers: ['DepthAnything3Net_1'] [04:44] [ComfyUI] [comfy-env] DepthAnythingV3_MultiView done (4.23s): result: NodeOutput [04:44] [ComfyUI] [VRAM] After DepthAnythingV3_MultiView: 3723 / 7672 MB [04:44] [ComfyUI] [VRAM] Loaded: SubprocessModel (783 MB) [04:44] [ComfyUI] [comfy-env] Running DepthAnythingV3_MultiView: normalization_mode: 'Raw', resize_method: 'resize', invert_depth: False, keep_model_size: False, da3_model: dict, images: torch.float32 [1,680,1208,3] cpu [04:44] [ComfyUI] [VRAM] Before DepthAnythingV3_MultiView: 3723 / 7672 MB [04:45] [ComfyUI] [VRAM] Loaded: SubprocessModel (783 MB) [04:45] [ComfyUI] [SubprocessWorker] call_method: nodes.nodes_multiview.DepthAnythingV3_MultiView.EXECUTE_NORMALIZED [04:45] [ComfyUI] [SubprocessWorker] acquired lock, ensuring started... [04:45] [ComfyUI] [ComfyUI-DepthAnythingV3] Health check: ping (timeout=5.0s)... [04:45] [ComfyUI] [ComfyUI-DepthAnythingV3] Health check: ok [04:45] [ComfyUI] [SubprocessWorker] worker started/confirmed [04:45] [ComfyUI] [comfy-env] PRE-SERIALIZE 'images' shape: torch.Size([1, 680, 1208, 3]) [04:45] [ComfyUI] [SubprocessWorker] serializing kwargs to shm... [04:45] [ComfyUI] [SubprocessWorker] created 0 shm blocks [04:45] [ComfyUI] [SubprocessWorker] sending request via socket... [04:45] [ComfyUI] [SubprocessWorker] call_id=14 sending call_method [04:45] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] _from_shm got dict with keys: ['__type__', 'strategy', 'parent_pid', 'fd', 'storage_size'] [04:45] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] DESERIALIZE TensorRef: tensor_size=[1, 680, 1208, 3] [04:45] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] DESERIALIZED tensor shape: torch.Size([1, 680, 1208, 3]) [04:45] [ComfyUI] [SubprocessWorker] callback 'request_vram_budget' from call_id=14 [04:45] [ComfyUI] [comfy-env] VRAM request: 0.82GB, free before eviction: 4.14GB [04:45] [ComfyUI] [comfy-env] VRAM after eviction: 4.14GB [04:45] [ComfyUI] [SubprocessWorker] got response: ok [04:45] [ComfyUI] [comfy-env] DepthAnythingV3_MultiView done (0.62s): result: NodeOutput [04:45] [ComfyUI] [VRAM] After DepthAnythingV3_MultiView: 3691 / 7672 MB [04:45] [ComfyUI] [VRAM] Loaded: SubprocessModel (783 MB) [04:45] [ComfyUI] [comfy-env] Running DA3_MultiViewPointCloud: extrinsics: '{\n "extrinsics": [\n {\n "image_0": [\n ..., intrinsics: '{\n "intrinsics": [\n {\n "image_0": [\n ..., confidence_threshold: 0.3, downsample: 4, use_icp: False, allow_around_1: False, filter_outliers: True, outlier_percentage: 5.0, depths: torch.float32 [1,680,1208,3] cpu, images: torch.float32 [1,680,1208,3] cpu, confidence: torch.float32 [1,680,1208,3] cpu, sky_mask: torch.float32 [1,680,1208] cpu [04:45] [ComfyUI] [VRAM] Before DA3_MultiViewPointCloud: 3691 / 7672 MB [04:45] [ComfyUI] [VRAM] Loaded: SubprocessModel (783 MB) [04:45] [ComfyUI] [SubprocessWorker] call_method: nodes.nodes_multiview.DA3_MultiViewPointCloud.EXECUTE_NORMALIZED [04:45] [ComfyUI] [SubprocessWorker] acquired lock, ensuring started... [04:45] [ComfyUI] [ComfyUI-DepthAnythingV3] Health check: ping (timeout=5.0s)... [04:45] [ComfyUI] [ComfyUI-DepthAnythingV3] Health check: ok [04:45] [ComfyUI] [SubprocessWorker] worker started/confirmed [04:45] [ComfyUI] [comfy-env] PRE-SERIALIZE 'depths' shape: torch.Size([1, 680, 1208, 3]) [04:45] [ComfyUI] [comfy-env] PRE-SERIALIZE 'images' shape: torch.Size([1, 680, 1208, 3]) [04:45] [ComfyUI] [comfy-env] PRE-SERIALIZE 'confidence' shape: torch.Size([1, 680, 1208, 3]) [04:45] [ComfyUI] [comfy-env] PRE-SERIALIZE 'sky_mask' shape: torch.Size([1, 680, 1208]) [04:45] [ComfyUI] [SubprocessWorker] serializing kwargs to shm... [04:45] [ComfyUI] [SubprocessWorker] created 0 shm blocks [04:45] [ComfyUI] [SubprocessWorker] sending request via socket... [04:45] [ComfyUI] [SubprocessWorker] call_id=15 sending call_method [04:45] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] _from_shm got dict with keys: ['__type__', 'strategy', 'parent_pid', 'fd', 'storage_size'] [04:45] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] DESERIALIZE TensorRef: tensor_size=[1, 680, 1208, 3] [04:45] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] DESERIALIZED tensor shape: torch.Size([1, 680, 1208, 3]) [04:45] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] _from_shm got dict with keys: ['__type__', 'strategy', 'parent_pid', 'fd', 'storage_size'] [04:45] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] DESERIALIZE TensorRef: tensor_size=[1, 680, 1208, 3] [04:45] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] DESERIALIZED tensor shape: torch.Size([1, 680, 1208, 3]) [04:45] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] _from_shm got dict with keys: ['__type__', 'strategy', 'parent_pid', 'fd', 'storage_size'] [04:45] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] DESERIALIZE TensorRef: tensor_size=[1, 680, 1208, 3] [04:45] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] DESERIALIZED tensor shape: torch.Size([1, 680, 1208, 3]) [04:45] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] _from_shm got dict with keys: ['__type__', 'strategy', 'parent_pid', 'fd', 'storage_size'] [04:45] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] DESERIALIZE TensorRef: tensor_size=[1, 680, 1208] [04:45] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] DESERIALIZED tensor shape: torch.Size([1, 680, 1208]) [04:45] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=15 [04:45] [ComfyUI] [SubprocessWorker] got response: ok [04:45] [ComfyUI] [comfy-env] DA3_MultiViewPointCloud done (0.17s): result: NodeOutput [04:45] [ComfyUI] [VRAM] After DA3_MultiViewPointCloud: 3691 / 7672 MB [04:45] [ComfyUI] [VRAM] Loaded: SubprocessModel (783 MB) [04:45] [ComfyUI] [comfy-env] Running DA3_MultiViewPointCloud: extrinsics: '{\n "extrinsics": [\n {\n "image_0": [\n ..., intrinsics: '{\n "intrinsics": [\n {\n "image_0": [\n ..., confidence_threshold: 0.3, downsample: 4, use_icp: False, allow_around_1: False, filter_outliers: True, outlier_percentage: 5.0, depths: torch.float32 [1,680,1208,3] cpu, images: torch.float32 [1,680,1208,3] cpu, confidence: torch.float32 [1,680,1208,3] cpu, sky_mask: torch.float32 [1,680,1208] cpu [04:45] [ComfyUI] [VRAM] Before DA3_MultiViewPointCloud: 3691 / 7672 MB [04:45] [ComfyUI] [VRAM] Loaded: SubprocessModel (783 MB) [04:45] [ComfyUI] [SubprocessWorker] call_method: nodes.nodes_multiview.DA3_MultiViewPointCloud.EXECUTE_NORMALIZED [04:45] [ComfyUI] [SubprocessWorker] acquired lock, ensuring started... [04:45] [ComfyUI] [ComfyUI-DepthAnythingV3] Health check: ping (timeout=5.0s)... [04:45] [ComfyUI] [ComfyUI-DepthAnythingV3] Health check: ok [04:45] [ComfyUI] [SubprocessWorker] worker started/confirmed [04:45] [ComfyUI] [comfy-env] PRE-SERIALIZE 'depths' shape: torch.Size([1, 680, 1208, 3]) [04:45] [ComfyUI] [comfy-env] PRE-SERIALIZE 'images' shape: torch.Size([1, 680, 1208, 3]) [04:45] [ComfyUI] [comfy-env] PRE-SERIALIZE 'confidence' shape: torch.Size([1, 680, 1208, 3]) [04:45] [ComfyUI] [comfy-env] PRE-SERIALIZE 'sky_mask' shape: torch.Size([1, 680, 1208]) [04:45] [ComfyUI] [SubprocessWorker] serializing kwargs to shm... [04:45] [ComfyUI] [SubprocessWorker] created 0 shm blocks [04:45] [ComfyUI] [SubprocessWorker] sending request via socket... [04:45] [ComfyUI] [SubprocessWorker] call_id=16 sending call_method [04:45] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] _from_shm got dict with keys: ['__type__', 'strategy', 'parent_pid', 'fd', 'storage_size'] [04:45] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] DESERIALIZE TensorRef: tensor_size=[1, 680, 1208, 3] [04:45] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] DESERIALIZED tensor shape: torch.Size([1, 680, 1208, 3]) [04:45] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] _from_shm got dict with keys: ['__type__', 'strategy', 'parent_pid', 'fd', 'storage_size'] [04:45] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] DESERIALIZE TensorRef: tensor_size=[1, 680, 1208, 3] [04:45] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] DESERIALIZED tensor shape: torch.Size([1, 680, 1208, 3]) [04:45] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] _from_shm got dict with keys: ['__type__', 'strategy', 'parent_pid', 'fd', 'storage_size'] [04:45] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] DESERIALIZE TensorRef: tensor_size=[1, 680, 1208, 3] [04:45] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] DESERIALIZED tensor shape: torch.Size([1, 680, 1208, 3]) [04:45] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] _from_shm got dict with keys: ['__type__', 'strategy', 'parent_pid', 'fd', 'storage_size'] [04:45] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] DESERIALIZE TensorRef: tensor_size=[1, 680, 1208] [04:45] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] DESERIALIZED tensor shape: torch.Size([1, 680, 1208]) [04:45] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=16 [04:45] [ComfyUI] [SubprocessWorker] got response: ok [04:45] [ComfyUI] [comfy-env] DA3_MultiViewPointCloud done (0.17s): result: NodeOutput [04:45] [ComfyUI] [VRAM] After DA3_MultiViewPointCloud: 3691 / 7672 MB [04:45] [ComfyUI] [VRAM] Loaded: SubprocessModel (783 MB) [04:45] [ComfyUI] [comfy-env] Running DA3_SavePointCloud: filename_prefix: 'pointcloud', pointcloud: list[1] [04:45] [ComfyUI] [VRAM] Before DA3_SavePointCloud: 3691 / 7672 MB [04:45] [ComfyUI] [VRAM] Loaded: SubprocessModel (783 MB) [04:45] [ComfyUI] [SubprocessWorker] call_method: nodes.nodes_3d.DA3_SavePointCloud.EXECUTE_NORMALIZED [04:45] [ComfyUI] [SubprocessWorker] acquired lock, ensuring started... [04:45] [ComfyUI] [ComfyUI-DepthAnythingV3] Health check: ping (timeout=5.0s)... [04:45] [ComfyUI] [ComfyUI-DepthAnythingV3] Health check: ok [04:45] [ComfyUI] [SubprocessWorker] worker started/confirmed [04:45] [ComfyUI] [SubprocessWorker] serializing kwargs to shm... [04:45] [ComfyUI] [SubprocessWorker] created 0 shm blocks [04:45] [ComfyUI] [SubprocessWorker] sending request via socket... [04:45] [ComfyUI] [SubprocessWorker] call_id=17 sending call_method [04:45] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] _from_shm got dict with keys: ['__type__', 'strategy', 'parent_pid', 'fd', 'storage_size'] [04:46] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] DESERIALIZE TensorRef: tensor_size=[18834, 3] [04:46] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] DESERIALIZED tensor shape: torch.Size([18834, 3]) [04:46] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] _from_shm got dict with keys: ['__type__', 'strategy', 'parent_pid', 'fd', 'storage_size'] [04:46] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] DESERIALIZE TensorRef: tensor_size=[18834, 3] [04:46] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] DESERIALIZED tensor shape: torch.Size([18834, 3]) [04:46] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] _from_shm got dict with keys: ['__type__', 'strategy', 'parent_pid', 'fd', 'storage_size'] [04:46] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] DESERIALIZE TensorRef: tensor_size=[18834] [04:46] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] DESERIALIZED tensor shape: torch.Size([18834]) [04:46] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] _from_shm got dict with keys: ['__type__', 'strategy', 'parent_pid', 'fd', 'storage_size'] [04:46] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] DESERIALIZE TensorRef: tensor_size=[18834] [04:46] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] DESERIALIZED tensor shape: torch.Size([18834]) [04:46] Frame 5 saved (frame_004.jpg, t=7.3s) [04:46] [capture-periodic] freeze=2ms shot=134ms unfreeze=5ms saved=True [04:46] [ComfyUI] [SubprocessWorker] got response: ok [04:46] [ComfyUI] [comfy-env] DA3_SavePointCloud done (0.29s): result: NodeOutput [04:46] [ComfyUI] [VRAM] After DA3_SavePointCloud: 3691 / 7672 MB [04:46] [ComfyUI] [VRAM] Loaded: SubprocessModel (783 MB) [04:46] [ComfyUI] [comfy-env] Running DA3_SavePointCloud: filename_prefix: 'pointcloud', pointcloud: list[1] [04:46] [ComfyUI] [VRAM] Before DA3_SavePointCloud: 3691 / 7672 MB [04:46] [ComfyUI] [VRAM] Loaded: SubprocessModel (783 MB) [04:46] [ComfyUI] [SubprocessWorker] call_method: nodes.nodes_3d.DA3_SavePointCloud.EXECUTE_NORMALIZED [04:46] [ComfyUI] [SubprocessWorker] acquired lock, ensuring started... [04:46] [ComfyUI] [ComfyUI-DepthAnythingV3] Health check: ping (timeout=5.0s)... [04:46] [ComfyUI] [ComfyUI-DepthAnythingV3] Health check: ok [04:46] [ComfyUI] [SubprocessWorker] worker started/confirmed [04:46] [ComfyUI] [SubprocessWorker] serializing kwargs to shm... [04:46] [ComfyUI] [SubprocessWorker] created 0 shm blocks [04:46] [ComfyUI] [SubprocessWorker] sending request via socket... [04:46] [ComfyUI] [SubprocessWorker] call_id=18 sending call_method [04:46] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] _from_shm got dict with keys: ['__type__', 'strategy', 'parent_pid', 'fd', 'storage_size'] [04:46] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] DESERIALIZE TensorRef: tensor_size=[20466, 3] [04:46] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] DESERIALIZED tensor shape: torch.Size([20466, 3]) [04:46] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] _from_shm got dict with keys: ['__type__', 'strategy', 'parent_pid', 'fd', 'storage_size'] [04:46] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] DESERIALIZE TensorRef: tensor_size=[20466, 3] [04:46] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] DESERIALIZED tensor shape: torch.Size([20466, 3]) [04:46] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] _from_shm got dict with keys: ['__type__', 'strategy', 'parent_pid', 'fd', 'storage_size'] [04:46] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] DESERIALIZE TensorRef: tensor_size=[20466] [04:46] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] DESERIALIZED tensor shape: torch.Size([20466]) [04:46] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] _from_shm got dict with keys: ['__type__', 'strategy', 'parent_pid', 'fd', 'storage_size'] [04:46] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] DESERIALIZE TensorRef: tensor_size=[20466] [04:46] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] DESERIALIZED tensor shape: torch.Size([20466]) [04:46] [ComfyUI] [SubprocessWorker] got response: ok [04:46] [ComfyUI] [comfy-env] DA3_SavePointCloud done (0.28s): result: NodeOutput [04:46] [ComfyUI] [VRAM] After DA3_SavePointCloud: 3691 / 7672 MB [04:46] [ComfyUI] [VRAM] Loaded: SubprocessModel (783 MB) [04:46] [ComfyUI] [comfy-env] Running DA3_PreviewPointCloud: file_path: '/workspaces/DepthAnythingV3-0902/ComfyUI/output/pointclo..., color_mode: 'RGB' [04:46] [ComfyUI] [VRAM] Before DA3_PreviewPointCloud: 3691 / 7672 MB [04:46] [ComfyUI] [VRAM] Loaded: SubprocessModel (783 MB) [04:46] [ComfyUI] [SubprocessWorker] call_method: nodes.preview_nodes.DA3_PreviewPointCloud.EXECUTE_NORMALIZED [04:46] [ComfyUI] [SubprocessWorker] acquired lock, ensuring started... [04:46] [ComfyUI] [ComfyUI-DepthAnythingV3] Health check: ping (timeout=5.0s)... [04:46] [ComfyUI] [ComfyUI-DepthAnythingV3] Health check: ok [04:46] [ComfyUI] [SubprocessWorker] worker started/confirmed [04:46] [ComfyUI] [SubprocessWorker] serializing kwargs to shm... [04:46] [ComfyUI] [SubprocessWorker] created 0 shm blocks [04:46] [ComfyUI] [SubprocessWorker] sending request via socket... [04:46] [ComfyUI] [SubprocessWorker] call_id=19 sending call_method [04:46] [ComfyUI] [SubprocessWorker] got response: ok [04:46] [ComfyUI] [comfy-env] DA3_PreviewPointCloud done (0.05s): result: NodeOutput [04:46] Node executed (3 total), capturing... [04:46] [ComfyUI] [VRAM] After DA3_PreviewPointCloud: 3691 / 7672 MB [04:46] [ComfyUI] [VRAM] Loaded: SubprocessModel (783 MB) [04:46] [ComfyUI] [comfy-env] Running DA3_PreviewPointCloud: file_path: '/workspaces/DepthAnythingV3-0902/ComfyUI/output/pointclo..., color_mode: 'RGB' [04:46] [ComfyUI] [VRAM] Before DA3_PreviewPointCloud: 3691 / 7672 MB [04:46] [ComfyUI] [VRAM] Loaded: SubprocessModel (783 MB) [04:46] [ComfyUI] [SubprocessWorker] call_method: nodes.preview_nodes.DA3_PreviewPointCloud.EXECUTE_NORMALIZED [04:46] [ComfyUI] [SubprocessWorker] acquired lock, ensuring started... [04:46] [ComfyUI] [ComfyUI-DepthAnythingV3] Health check: ping (timeout=5.0s)... [04:46] [ComfyUI] [ComfyUI-DepthAnythingV3] Health check: ok [04:46] [ComfyUI] [SubprocessWorker] worker started/confirmed [04:46] [ComfyUI] [SubprocessWorker] serializing kwargs to shm... [04:46] [ComfyUI] [SubprocessWorker] created 0 shm blocks [04:46] [ComfyUI] [SubprocessWorker] sending request via socket... [04:46] [ComfyUI] [SubprocessWorker] call_id=20 sending call_method [04:46] [ComfyUI] [SubprocessWorker] got response: ok [04:46] [ComfyUI] [comfy-env] DA3_PreviewPointCloud done (0.06s): result: NodeOutput [04:46] [ComfyUI] [VRAM] After DA3_PreviewPointCloud: 3691 / 7672 MB [04:46] [ComfyUI] [VRAM] Loaded: SubprocessModel (783 MB) [04:46] [ComfyUI] Prompt executed in 7.77 seconds [04:46] Frame 6 saved (frame_005.jpg, t=8.1s) [04:46] [capture-node] freeze=6ms shot=139ms unfreeze=2ms saved=True [04:47] Node executed (4 total), capturing... [04:47] Frame 7 saved (frame_006.jpg, t=8.6s) [04:47] [capture-node] freeze=2ms shot=141ms unfreeze=0ms saved=True [04:47] Execution complete (t=8.6s) [04:47] Captured 7 unique frames over 8.76s [04:47] [debug] iframes: 1 [04:47] [debug] iframe[0]: src=http://127.0.0.1:41896/extensions/ComfyUI-GeometryPack/viewer_vtk.html?v=1778836, canvas=486x394, rafFrozen=True [04:47] [timing] trigger_3d_previews: 0.0s [04:49] Saved high-quality screenshot: advanced_3d_multiview_executed.png [04:49] [timing] final_screenshot: 0.3s [04:49] [ComfyUI] [comfy-env] [VRAM] Before offload 'DepthAnything3Net_1' (783 MB) -> cpu: 3691 / 7672 MB [04:49] Capturing execution frames: bas_relief.json [04:49] [ComfyUI] [ComfyUI-DepthAnythingV3] Health check: ping (timeout=5.0s)... [04:49] [ComfyUI] [ComfyUI-DepthAnythingV3] Health check: ok [04:49] [ComfyUI] [SubprocessWorker] call_id=21 sending model_to_device [04:49] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [VRAM] -234MB (now 599MB) reserved=2848MB peak=1240MB [04:50] [ComfyUI] [comfy-env] [VRAM] After offload 'DepthAnything3Net_1' (783 MB): 907 / 7672 MB [04:55] Frame 1 saved (frame_000.jpg, t=0.0s) [04:55] Validating workflow... [04:55] [ComfyUI] === /validate endpoint called === [04:55] [ComfyUI] Received JSON with keys: ['prompt'] [04:55] Queuing workflow for execution... [04:55] [ComfyUI] Prompt has 6 nodes [04:55] [ComfyUI] Node 2: LoadImage [04:55] [ComfyUI] Inputs: {"image": "bas_relief.png"} [04:55] [ComfyUI] Node 3: DownloadAndLoadDepthAnythingV3Model [04:55] [ComfyUI] Inputs: {"model": "da3_large.safetensors", "precision": "auto", "attention": "auto"} [04:55] [ComfyUI] Node 21: DepthAnything_V3 [04:55] [ComfyUI] Inputs: {"normalization_mode": "Raw", "resize_method": "resize", "invert_depth": false, "keep_model_size": true, "da3_model": ["3", 0], "images": ["2", 0]} [04:55] [ComfyUI] Node 27: PreviewImage [04:55] [ComfyUI] Inputs: {"images": ["21", 2]} [04:55] [ComfyUI] Node 30: Preview3D [04:55] [ComfyUI] Inputs: {"model_file": ["32", 0], "image": ""} [04:55] [ComfyUI] Node 32: DA3_ToMesh [04:55] [ComfyUI] Inputs: {"confidence_threshold": 0.1, "depth_edge_threshold": 0.1, "downsample": 1, "target_faces": 100000, "filename_prefix": "mesh", "allow_around_1": true, "use_draco_compression": false, "depth_raw": ["21 [04:55] [ComfyUI] Generated prompt_id: 2961d08d-4375-4f30-9e55-00cc8a2501f1 [04:55] [ComfyUI] Calling execution.validate_prompt()... [04:55] [ComfyUI] Validation result: valid=True [04:55] [ComfyUI] === Validation PASSED === [04:55] [ComfyUI] got prompt [04:55] [ComfyUI] [comfy-env] Running DownloadAndLoadDepthAnythingV3Model: model: 'da3_large.safetensors', precision: 'auto', attention: 'auto' [04:55] [ComfyUI] [VRAM] Before DownloadAndLoadDepthAnythingV3Model: 907 / 7672 MB [04:55] [ComfyUI] [ComfyUI-DepthAnythingV3] python: /root/.ce/.pixi/envs/depthanythingv3-nodes/bin/python [04:55] [ComfyUI] [ComfyUI-DepthAnythingV3] sys_path sent to worker: ['/workspaces/DepthAnythingV3-0902/ComfyUI', '/workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes/ComfyUI-DepthAnythingV3', '/root/.ce/.pixi/envs/depthanythingv3-nodes/lib/python3.12/site-packages', '/workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes/ComfyUI-DepthAnythingV3'] [04:55] [ComfyUI] [SubprocessWorker] call_method: nodes.load_model.DownloadAndLoadDepthAnythingV3Model.EXECUTE_NORMALIZED [04:55] [ComfyUI] [SubprocessWorker] acquired lock, ensuring started... [04:55] [ComfyUI] [SubprocessWorker] is_pixi=True, python=/root/.ce/.pixi/envs/depthanythingv3-nodes/bin/python [04:55] [ComfyUI] [ComfyUI-DepthAnythingV3] Health check: ping (timeout=5.0s)... [04:55] [ComfyUI] [SubprocessWorker] launching cmd=['/root/.pixi/bin/pixi', 'run', '--as-is', '--manifest-path', '/root/.ce/pixi.toml', '-e', 'depthanythingv3-nodes', 'python', '/tmp/comfyui_pvenv_lusourms/persistent_worker.py', 'abstract://comfy_worker_64c53aa6607b']... [04:55] [ComfyUI] [ComfyUI-DepthAnythingV3] Health check: failed [04:56] [ComfyUI] [SubprocessWorker] socket_addr=abstract://comfy_worker_64c53aa6607b (abstract, no filesystem path) [04:56] [ComfyUI] [ComfyUI-DepthAnythingV3] Socket unhealthy, restarting worker... [04:56] [ComfyUI] [comfy-env] Invalidated 1 stale model patchers (will be cleaned up during next unload) [04:57] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] ERROR: Failed to import comfy_kitchen, Error: No module named 'comfy_kitchen', fp8 and fp4 support will not be available. [04:57] [ComfyUI] [SubprocessWorker] worker started/confirmed [04:57] [ComfyUI] [SubprocessWorker] serializing kwargs to shm... [04:57] [ComfyUI] [SubprocessWorker] created 0 shm blocks [04:57] [ComfyUI] [SubprocessWorker] sending request via socket... [04:57] [ComfyUI] [SubprocessWorker] call_id=22 sending call_method [04:57] [ComfyUI] [SubprocessWorker] got response: ok [04:57] [ComfyUI] [comfy-env] [VRAM] Before offload 'DepthAnything3Net_1' (783 MB) -> cpu: 737 / 7672 MB [04:57] [ComfyUI] [ComfyUI-DepthAnythingV3] Health check: ping (timeout=5.0s)... [04:57] [ComfyUI] [ComfyUI-DepthAnythingV3] Health check: ok [04:57] [ComfyUI] [SubprocessWorker] call_id=23 sending model_to_device [04:57] [ComfyUI] Model 'DepthAnything3Net_1' gone from worker (restarted?), treating as offloaded [04:57] [ComfyUI] [comfy-env] [VRAM] After offload 'DepthAnything3Net_1' (783 MB): 737 / 7672 MB [04:57] [ComfyUI] [comfy-env] DownloadAndLoadDepthAnythingV3Model done (1.64s): result: NodeOutput [04:57] [ComfyUI] [VRAM] After DownloadAndLoadDepthAnythingV3Model: 737 / 7672 MB [04:57] [ComfyUI] [comfy-env] Running DepthAnything_V3: normalization_mode: 'Raw', resize_method: 'resize', invert_depth: False, keep_model_size: True, da3_model: dict, images: torch.float32 [1,2000,1336,3] cpu [04:57] [ComfyUI] [VRAM] Before DepthAnything_V3: 737 / 7672 MB [04:57] [ComfyUI] [SubprocessWorker] call_method: nodes.nodes_inference.DepthAnything_V3.EXECUTE_NORMALIZED [04:57] [ComfyUI] [SubprocessWorker] acquired lock, ensuring started... [04:57] [ComfyUI] [ComfyUI-DepthAnythingV3] Health check: ping (timeout=5.0s)... [04:57] [ComfyUI] [ComfyUI-DepthAnythingV3] Health check: ok [04:57] [ComfyUI] [SubprocessWorker] worker started/confirmed [04:57] [ComfyUI] [comfy-env] PRE-SERIALIZE 'images' shape: torch.Size([1, 2000, 1336, 3]) [04:57] [ComfyUI] [SubprocessWorker] serializing kwargs to shm... [04:57] [ComfyUI] [SubprocessWorker] created 0 shm blocks [04:57] [ComfyUI] [SubprocessWorker] sending request via socket... [04:57] [ComfyUI] [SubprocessWorker] call_id=24 sending call_method [04:57] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] _from_shm got dict with keys: ['__type__', 'strategy', 'parent_pid', 'fd', 'storage_size'] [04:57] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] DESERIALIZE TensorRef: tensor_size=[1, 2000, 1336, 3] [04:57] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] DESERIALIZED tensor shape: torch.Size([1, 2000, 1336, 3]) [04:57] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] WARNING: No comfy kitchen, using old apply_rope functions. [04:58] Frame 2 saved (frame_001.jpg, t=2.3s) [04:58] [capture-periodic] freeze=1ms shot=124ms unfreeze=1ms saved=True [05:00] Frame 3 saved (frame_002.jpg, t=4.5s) [05:00] [capture-periodic] freeze=1ms shot=169ms unfreeze=3ms saved=True [05:00] [ComfyUI] [SubprocessWorker] callback 'request_vram_budget' from call_id=24 [05:00] [ComfyUI] [comfy-env] VRAM request: 0.82GB, free before eviction: 7.27GB [05:00] [ComfyUI] [comfy-env] VRAM after eviction: 7.27GB [05:00] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [VRAM] +202MB (now 202MB) reserved=224MB peak=202MB [05:01] [capture-loop] iter=50 t=5.8s state={'complete': False, 'error': None, 'executedCount': 0, 'wsState': 1} eval_ms=2 ram=5.2/23.4GB (22%) swap=1.8/32.0GB (6%) cpu=12% browser_rss=1.06GB [05:01] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [VRAM] +978MB (now 1180MB) reserved=1376MB peak=1180MB [05:02] [ComfyUI] [SubprocessWorker] got response: error [05:02] [ComfyUI] !!! Exception during processing !!! Allocation on device 0 would exceed allowed memory. (out of memory) [05:02] [ComfyUI] Currently allocated : 4.33 GiB [05:02] [ComfyUI] Requested : 1.27 GiB [05:02] [ComfyUI] Device limit : 7.49 GiB [05:02] [ComfyUI] Free (according to CUDA): 109.19 MiB [05:02] [ComfyUI] PyTorch limit (set by user-supplied memory fraction) [05:02] [ComfyUI] : 17179869184.00 GiB [05:02] [ComfyUI] [05:02] [ComfyUI] Worker traceback: [05:02] [ComfyUI] Traceback (most recent call last): [05:02] [ComfyUI] File "/tmp/comfyui_pvenv_lusourms/persistent_worker.py", line 1762, in main [05:02] [ComfyUI] result = method(**inputs) [05:02] [ComfyUI] ^^^^^^^^^^^^^^^^ [05:02] [ComfyUI] File "/workspaces/DepthAnythingV3-0902/ComfyUI/comfy_api/latest/_io.py", line 1833, in EXECUTE_NORMALIZED [05:02] [ComfyUI] to_return = cls.execute(*args, **kwargs) [05:02] [ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [05:02] [ComfyUI] File "/workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes/ComfyUI-DepthAnythingV3/nodes/nodes_inference.py", line 156, in execute [05:02] [ComfyUI] output = model(img, extrinsics=ext_i, intrinsics=int_i, infer_gs=infer_gs) [05:02] [ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [05:02] [ComfyUI] File "/root/.ce/.pixi/envs/depthanythingv3-nodes/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1779, in _wrapped_call_impl [05:02] [ComfyUI] return self._call_impl(*args, **kwargs) [05:02] [ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [05:02] [ComfyUI] File "/root/.ce/.pixi/envs/depthanythingv3-nodes/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1790, in _call_impl [05:02] [ComfyUI] return forward_call(*args, **kwargs) [05:02] [ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [05:02] [ComfyUI] File "/workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes/ComfyUI-DepthAnythingV3/nodes/depth_anything_v3/model.py", line 1577, in forward [05:02] [ComfyUI] output = self._process_depth_head(feats, H, W) [05:02] [ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [05:02] [ComfyUI] File "/workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes/ComfyUI-DepthAnythingV3/nodes/depth_anything_v3/model.py", line 1591, in _process_depth_head [05:02] [ComfyUI] return self.head(feats, H, W, patch_start_idx=0) [05:02] [ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [05:02] [ComfyUI] File "/root/.ce/.pixi/envs/depthanythingv3-nodes/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1779, in _wrapped_call_impl [05:02] [ComfyUI] return self._call_impl(*args, **kwargs) [05:02] [ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [05:02] [ComfyUI] File "/root/.ce/.pixi/envs/depthanythingv3-nodes/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1790, in _call_impl [05:02] [ComfyUI] return forward_call(*args, **kwargs) [05:02] [ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [05:02] [ComfyUI] File "/workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes/ComfyUI-DepthAnythingV3/nodes/depth_anything_v3/model.py", line 1366, in forward [05:02] [ComfyUI] out_dict = self._forward_impl(feats, H, W, patch_start_idx) [05:02] [ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [05:02] [ComfyUI] File "/workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes/ComfyUI-DepthAnythingV3/nodes/depth_anything_v3/model.py", line 1419, in _forward_impl [05:02] [ComfyUI] fused_main = self._add_pos_embed(fused_main, W, H) [05:02] [ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [05:02] [ComfyUI] File "/workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes/ComfyUI-DepthAnythingV3/nodes/depth_anything_v3/model.py", line 1477, in _add_pos_embed [05:02] [ComfyUI] pe = position_grid_to_embed(pe, x.shape[1]) * ratio [05:02] [ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [05:02] [ComfyUI] File "/workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes/ComfyUI-DepthAnythingV3/nodes/depth_anything_v3/model.py", line 131, in position_grid_to_embed [05:02] [ComfyUI] emb = torch.cat([emb_x, emb_y], dim=-1) [05:02] [ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [05:02] [ComfyUI] torch.OutOfMemoryError: Allocation on device 0 would exceed allowed memory. (out of memory) [05:02] [ComfyUI] Currently allocated : 4.33 GiB [05:02] [ComfyUI] Requested : 1.27 GiB [05:02] [ComfyUI] Device limit : 7.49 GiB [05:02] [ComfyUI] Free (according to CUDA): 109.19 MiB [05:02] [ComfyUI] PyTorch limit (set by user-supplied memory fraction) [05:02] [ComfyUI] : 17179869184.00 GiB [05:02] [ComfyUI] [05:02] [ComfyUI] Traceback (most recent call last): [05:02] [ComfyUI] File "/workspaces/DepthAnythingV3-0902/ComfyUI/execution.py", line 535, in execute [05:02] [ComfyUI] output_data, output_ui, has_subgraph, has_pending_tasks = await get_output_data(prompt_id, unique_id, obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data) [05:02] [ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [05:02] [ComfyUI] File "/workspaces/DepthAnythingV3-0902/ComfyUI/execution.py", line 335, in get_output_data [05:02] [ComfyUI] return_values = await _async_map_node_over_list(prompt_id, unique_id, obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data) [05:02] [ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [05:02] [ComfyUI] File "/workspaces/DepthAnythingV3-0902/ComfyUI/execution.py", line 309, in _async_map_node_over_list [05:02] [ComfyUI] await process_inputs(input_dict, i) [05:02] [ComfyUI] File "/workspaces/DepthAnythingV3-0902/ComfyUI/execution.py", line 297, in process_inputs [05:02] [ComfyUI] result = f(**inputs) [05:02] [ComfyUI] ^^^^^^^^^^^ [05:02] [ComfyUI] File "/workspaces/DepthAnythingV3-0902/.venv/lib/python3.12/site-packages/comfy_env/isolation/metadata.py", line 556, in proxy [05:02] [ComfyUI] result = worker.call_method( [05:02] [ComfyUI] ^^^^^^^^^^^^^^^^^^^ [05:02] [ComfyUI] File "/workspaces/DepthAnythingV3-0902/.venv/lib/python3.12/site-packages/comfy_env/isolation/workers/subprocess.py", line 674, in call_method [05:02] [ComfyUI] raise WorkerError( [05:02] [ComfyUI] comfy_env.isolation.workers.base.WorkerError: Allocation on device 0 would exceed allowed memory. (out of memory) [05:02] [ComfyUI] Currently allocated : 4.33 GiB [05:02] [ComfyUI] Requested : 1.27 GiB [05:02] [ComfyUI] Device limit : 7.49 GiB [05:02] [ComfyUI] Free (according to CUDA): 109.19 MiB [05:02] [ComfyUI] PyTorch limit (set by user-supplied memory fraction) [05:02] [ComfyUI] : 17179869184.00 GiB [05:02] [ComfyUI] [05:02] [ComfyUI] Worker traceback: [05:02] Execution complete (t=7.1s) [05:02] [ComfyUI] Traceback (most recent call last): [05:02] Execution error: {'prompt_id': 'a38e9f56-8351-4933-8312-f068d5b8399a', 'node_id': '21', 'node_type': 'DepthAnything_V3', 'executed': ['3', '2'], 'exception_message': 'Allocation on device 0 would exceed allowed memory. (out of memory)\nCurrently allocated : 4.33 GiB\nRequested : 1.27 GiB\nDevice limit : 7.49 GiB\nFree (according to CUDA): 109.19 MiB\nPyTorch limit (set by user-supplied memory fraction)\n : 17179869184.00 GiB\n\nWorker traceback:\nTraceback (most recent call last):\n File "/tmp/comfyui_pvenv_lusourms/persistent_worker.py", line 1762, in main\n result = method(**inputs)\n ^^^^^^^^^^^^^^^^\n File "/workspaces/DepthAnythingV3-0902/ComfyUI/comfy_api/latest/_io.py", line 1833, in EXECUTE_NORMALIZED\n to_return = cls.execute(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes/ComfyUI-DepthAnythingV3/nodes/nodes_inference.py", line 156, in execute\n output = model(img, extrinsics=ext_i, intrinsics=int_i, infer_gs=infer_gs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/root/.ce/.pixi/envs/depthanythingv3-nodes/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1779, in _wrapped_call_impl\n return self._call_impl(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/root/.ce/.pixi/envs/depthanythingv3-nodes/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1790, in _call_impl\n return forward_call(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes/ComfyUI-DepthAnythingV3/nodes/depth_anything_v3/model.py", line 1577, in forward\n output = self._process_depth_head(feats, H, W)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes/ComfyUI-DepthAnythingV3/nodes/depth_anything_v3/model.py", line 1591, in _process_depth_head\n return self.head(feats, H, W, patch_start_idx=0)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/root/.ce/.pixi/envs/depthanythingv3-nodes/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1779, in _wrapped_call_impl\n return self._call_impl(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/root/.ce/.pixi/envs/depthanythingv3-nodes/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1790, in _call_impl\n return forward_call(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes/ComfyUI-DepthAnythingV3/nodes/depth_anything_v3/model.py", line 1366, in forward\n out_dict = self._forward_impl(feats, H, W, patch_start_idx)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes/ComfyUI-DepthAnythingV3/nodes/depth_anything_v3/model.py", line 1419, in _forward_impl\n fused_main = self._add_pos_embed(fused_main, W, H)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes/ComfyUI-DepthAnythingV3/nodes/depth_anything_v3/model.py", line 1477, in _add_pos_embed\n pe = position_grid_to_embed(pe, x.shape[1]) * ratio\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes/ComfyUI-DepthAnythingV3/nodes/depth_anything_v3/model.py", line 131, in position_grid_to_embed\n emb = torch.cat([emb_x, emb_y], dim=-1)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\ntorch.OutOfMemoryError: Allocation on device 0 would exceed allowed memory. (out of memory)\nCurrently allocated : 4.33 GiB\nRequested : 1.27 GiB\nDevice limit : 7.49 GiB\nFree (according to CUDA): 109.19 MiB\nPyTorch limit (set by user-supplied memory fraction)\n : 17179869184.00 GiB\n\n', 'exception_type': 'comfy_env.isolation.workers.base.WorkerError', 'traceback': [' File "/workspaces/DepthAnythingV3-0902/ComfyUI/execution.py", line 535, in execute\n output_data, output_ui, has_subgraph, has_pending_tasks = await get_output_data(prompt_id, unique_id, obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n', ' File "/workspaces/DepthAnythingV3-0902/ComfyUI/execution.py", line 335, in get_output_data\n return_values = await _async_map_node_over_list(prompt_id, unique_id, obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n', ' File "/workspaces/DepthAnythingV3-0902/ComfyUI/execution.py", line 309, in _async_map_node_over_list\n await process_inputs(input_dict, i)\n', ' File "/workspaces/DepthAnythingV3-0902/ComfyUI/execution.py", line 297, in process_inputs\n result = f(**inputs)\n ^^^^^^^^^^^\n', ' File "/workspaces/DepthAnythingV3-0902/.venv/lib/python3.12/site-packages/comfy_env/isolation/metadata.py", line 556, in proxy\n result = worker.call_method(\n ^^^^^^^^^^^^^^^^^^^\n', ' File "/workspaces/DepthAnythingV3-0902/.venv/lib/python3.12/site-packages/comfy_env/isolation/workers/subprocess.py", line 674, in call_method\n raise WorkerError(\n'], 'current_inputs': {'normalization_mode': ['Raw'], 'resize_method': ['resize'], 'invert_depth': [False], 'keep_model_size': [True], 'da3_model': ["{'model_path': '/workspaces/DepthAnythingV3-0902/ComfyUI/models/depthanything3/da3_large.safetensors', 'model_key': 'da3-large', 'dtype': 'bf16', 'attention': 'auto'}"], 'images': ['tensor([[[[0.5922, 0.4510, 0.2863],\n [0.5922, 0.4588, 0.2902],\n [0.5529, 0.4235, 0.2667],\n [0.5176, 0.3922, 0.2588],\n [0.5059, 0.3882, 0.2706],\n [0.4863, 0.3765, 0.2549],\n ...,\n [0.5608, 0.4471, 0.3059],\n [0.5608, 0.4471, 0.3098],\n [0.5490, 0.4471, 0.3255],\n [0.5373, 0.4510, 0.3373],\n [0.5176, 0.4392, 0.3216],\n [0.5059, 0.4314, 0.3216]],\n\n [[0.5686, 0.4235, 0.2745],\n [0.5451, 0.4157, 0.2627],\n [0.5373, 0.4078, 0.2627],\n [0.5216, 0.4000, 0.2667],\n [0.5020, 0.3843, 0.2627],\n [0.5020, 0.3843, 0.2706],\n ...,\n [0.5608, 0.4471, 0.3020],\n [0.5647, 0.4549, 0.3098],\n [0.5647, 0.4627, 0.3373],\n [0.5412, 0.4549, 0.3451],\n [0.5137, 0.4392, 0.3294],\n [0.5020, 0.4314, 0.3255]],\n\n [[0.4745, 0.3529, 0.2235],\n [0.4706, 0.3569, 0.2235],\n [0.4745, 0.3647, 0.2392],\n [0.4824, 0.3686, 0.2431],\n [0.4824, 0.3686, 0.2471],\n [0.4941, 0.3843, 0.2627],\n ...,\n [0.5647, 0.4471, 0.3059],\n [0.5686, 0.4549, 0.3176],\n [0.5529, 0.4510, 0.3255],\n [0.5255, 0.4314, 0.3294],\n [0.5255, 0.4471, 0.3412],\n [0.5059, 0.4353, 0.3216]],\n\n [[0.4000, 0.2902, 0.1765],\n [0.3882, 0.2863, 0.1725],\n [0.3647, 0.2627, 0.1529],\n [0.3961, 0.2902, 0.1804],\n [0.4627, 0.3569, 0.2392],\n [0.4941, 0.3882, 0.2627],\n ...,\n [0.5765, 0.4588, 0.3255],\n [0.5765, 0.4667, 0.3255],\n [0.5686, 0.4588, 0.3294],\n [0.5451, 0.4510, 0.3333],\n [0.5333, 0.4510, 0.3412],\n [0.5294, 0.4471, 0.3333]],\n\n [[0.5098, 0.3882, 0.2627],\n [0.5176, 0.4000, 0.2706],\n [0.5059, 0.3843, 0.2627],\n [0.4745, 0.3569, 0.2392],\n [0.4706, 0.3647, 0.2431],\n [0.4902, 0.3804, 0.2627],\n ...,\n [0.5804, 0.4706, 0.3373],\n [0.5647, 0.4549, 0.3176],\n [0.5804, 0.4627, 0.3255],\n [0.5608, 0.4510, 0.3216],\n [0.5294, 0.4392, 0.3216],\n [0.5412, 0.4471, 0.3451]],\n\n [[0.4980, 0.3686, 0.2392],\n [0.5490, 0.4275, 0.2863],\n [0.5294, 0.4196, 0.2863],\n [0.4745, 0.3569, 0.2314],\n [0.4745, 0.3647, 0.2392],\n [0.4980, 0.3922, 0.2667],\n ...,\n [0.5529, 0.4471, 0.3137],\n [0.5569, 0.4431, 0.3137],\n [0.5725, 0.4549, 0.3176],\n [0.5569, 0.4471, 0.3020],\n [0.5569, 0.4549, 0.3137],\n [0.5451, 0.4431, 0.3294]],\n\n ...,\n\n [[0.5765, 0.4824, 0.3725],\n [0.5725, 0.4824, 0.3686],\n [0.5686, 0.4745, 0.3608],\n [0.5490, 0.4471, 0.3216],\n [0.5373, 0.4235, 0.2902],\n [0.5294, 0.4118, 0.2784],\n ...,\n [0.5373, 0.4196, 0.2745],\n [0.5765, 0.4471, 0.3020],\n [0.5333, 0.4000, 0.2588],\n [0.5647, 0.4353, 0.2784],\n [0.5608, 0.4275, 0.2627],\n [0.6000, 0.4588, 0.3059]],\n\n [[0.5451, 0.4471, 0.3373],\n [0.5647, 0.4706, 0.3569],\n [0.5608, 0.4627, 0.3490],\n [0.5255, 0.4196, 0.2980],\n [0.5255, 0.4078, 0.2784],\n [0.5176, 0.3961, 0.2627],\n ...,\n [0.5255, 0.4039, 0.2549],\n [0.5490, 0.4275, 0.2706],\n [0.5333, 0.4039, 0.2510],\n [0.5490, 0.4157, 0.2706],\n [0.5647, 0.4314, 0.2784],\n [0.5961, 0.4588, 0.2980]],\n\n [[0.5529, 0.4510, 0.3294],\n [0.5490, 0.4431, 0.3176],\n [0.5686, 0.4588, 0.3373],\n [0.5412, 0.4275, 0.3059],\n [0.5412, 0.4275, 0.2980],\n [0.5333, 0.4157, 0.2824],\n ...,\n [0.5569, 0.4235, 0.2863],\n [0.5608, 0.4353, 0.2980],\n [0.5490, 0.4157, 0.2824],\n [0.5608, 0.4196, 0.2863],\n [0.5647, 0.4314, 0.2863],\n [0.5922, 0.4588, 0.3098]],\n\n [[0.5294, 0.4196, 0.3059],\n [0.5333, 0.4275, 0.2941],\n [0.5412, 0.4235, 0.2941],\n [0.5333, 0.4118, 0.2902],\n [0.5373, 0.4196, 0.2941],\n [0.5294, 0.4118, 0.2902],\n ...,\n [0.5765, 0.4431, 0.3137],\n [0.5765, 0.4471, 0.3176],\n [0.5490, 0.4118, 0.2824],\n [0.5647, 0.4275, 0.2863],\n [0.5765, 0.4431, 0.2941],\n [0.5765, 0.4353, 0.2863]],\n\n [[0.5176, 0.4078, 0.2902],\n [0.5412, 0.4392, 0.3098],\n [0.5098, 0.3922, 0.2745],\n [0.5098, 0.3922, 0.2706],\n [0.5255, 0.4078, 0.2902],\n [0.5137, 0.4000, 0.2824],\n ...,\n [0.5608, 0.4235, 0.2941],\n [0.5765, 0.4353, 0.3059],\n [0.5451, 0.4078, 0.2627],\n [0.5451, 0.4078, 0.2549],\n [0.5451, 0.4000, 0.2471],\n [0.5255, 0.3686, 0.2157]],\n\n [[0.5176, 0.4196, 0.3098],\n [0.5725, 0.4667, 0.3569],\n [0.5490, 0.4392, 0.3294],\n [0.5412, 0.4353, 0.3137],\n [0.5373, 0.4275, 0.3059],\n [0.5176, 0.4039, 0.2863],\n ...,\n [0.5647, 0.4314, 0.2980],\n [0.5529, 0.4118, 0.2745],\n [0.5098, 0.3686, 0.2196],\n [0.5373, 0.3882, 0.2353],\n [0.5176, 0.3647, 0.2078],\n [0.4745, 0.3176, 0.1765]]]])']}, 'current_outputs': ['2', '32', '21', '3', '30', '27'], 'timestamp': 1778836033559} [05:02] [ComfyUI] File "/tmp/comfyui_pvenv_lusourms/persistent_worker.py", line 1762, in main [05:02] [ComfyUI] result = method(**inputs) [05:02] [ComfyUI] ^^^^^^^^^^^^^^^^ [05:02] [ComfyUI] File "/workspaces/DepthAnythingV3-0902/ComfyUI/comfy_api/latest/_io.py", line 1833, in EXECUTE_NORMALIZED [05:02] [ComfyUI] to_return = cls.execute(*args, **kwargs) [05:02] [ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [05:02] [ComfyUI] File "/workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes/ComfyUI-DepthAnythingV3/nodes/nodes_inference.py", line 156, in execute [05:02] [ComfyUI] output = model(img, extrinsics=ext_i, intrinsics=int_i, infer_gs=infer_gs) [05:02] [ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [05:02] [ComfyUI] File "/root/.ce/.pixi/envs/depthanythingv3-nodes/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1779, in _wrapped_call_impl [05:02] [ComfyUI] return self._call_impl(*args, **kwargs) [05:02] [ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [05:02] [ComfyUI] File "/root/.ce/.pixi/envs/depthanythingv3-nodes/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1790, in _call_impl [05:02] [ComfyUI] return forward_call(*args, **kwargs) [05:02] [ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [05:02] [ComfyUI] File "/workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes/ComfyUI-DepthAnythingV3/nodes/depth_anything_v3/model.py", line 1577, in forward [05:02] [ComfyUI] output = self._process_depth_head(feats, H, W) [05:02] [ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [05:02] [ComfyUI] File "/workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes/ComfyUI-DepthAnythingV3/nodes/depth_anything_v3/model.py", line 1591, in _process_depth_head [05:02] [ComfyUI] return self.head(feats, H, W, patch_start_idx=0) [05:02] [ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [05:02] [ComfyUI] File "/root/.ce/.pixi/envs/depthanythingv3-nodes/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1779, in _wrapped_call_impl [05:02] [ComfyUI] return self._call_impl(*args, **kwargs) [05:02] [ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [05:02] [ComfyUI] File "/root/.ce/.pixi/envs/depthanythingv3-nodes/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1790, in _call_impl [05:02] [ComfyUI] return forward_call(*args, **kwargs) [05:02] [ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [05:02] [ComfyUI] File "/workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes/ComfyUI-DepthAnythingV3/nodes/depth_anything_v3/model.py", line 1366, in forward [05:02] [ComfyUI] out_dict = self._forward_impl(feats, H, W, patch_start_idx) [05:02] [ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [05:02] [ComfyUI] File "/workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes/ComfyUI-DepthAnythingV3/nodes/depth_anything_v3/model.py", line 1419, in _forward_impl [05:02] [ComfyUI] fused_main = self._add_pos_embed(fused_main, W, H) [05:02] [ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [05:02] [ComfyUI] File "/workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes/ComfyUI-DepthAnythingV3/nodes/depth_anything_v3/model.py", line 1477, in _add_pos_embed [05:02] [ComfyUI] pe = position_grid_to_embed(pe, x.shape[1]) * ratio [05:02] [ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [05:02] [ComfyUI] File "/workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes/ComfyUI-DepthAnythingV3/nodes/depth_anything_v3/model.py", line 131, in position_grid_to_embed [05:02] [ComfyUI] emb = torch.cat([emb_x, emb_y], dim=-1) [05:02] [ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [05:02] [ComfyUI] torch.OutOfMemoryError: Allocation on device 0 would exceed allowed memory. (out of memory) [05:02] [ComfyUI] Currently allocated : 4.33 GiB [05:02] [ComfyUI] Requested : 1.27 GiB [05:02] [ComfyUI] Device limit : 7.49 GiB [05:02] [ComfyUI] Free (according to CUDA): 109.19 MiB [05:02] [ComfyUI] PyTorch limit (set by user-supplied memory fraction) [05:02] [ComfyUI] : 17179869184.00 GiB [05:02] [ComfyUI] [05:02] [ComfyUI] [05:02] [ComfyUI] Prompt executed in 6.72 seconds [05:03] Capturing execution frames: da3_streaming.json [05:08] Frame 1 saved (frame_000.jpg, t=0.0s) [05:08] Validating workflow... [05:08] [ComfyUI] === /validate endpoint called === [05:08] [ComfyUI] Received JSON with keys: ['prompt'] [05:08] Queuing workflow for execution... [05:08] [ComfyUI] Prompt has 5 nodes [05:08] [ComfyUI] Node 2: DownloadAndLoadDepthAnythingV3Model [05:08] [ComfyUI] Inputs: {"model": "da3_large.safetensors", "precision": "auto", "attention": "auto"} [05:08] [ComfyUI] Node 3: LoadVideo [05:08] [ComfyUI] Inputs: {"file": "robot_unitree.mp4", "video-preview": ""} [05:08] [ComfyUI] [ComfyUI-DepthAnythingV3] python: /root/.ce/.pixi/envs/depthanythingv3-nodes/bin/python [05:08] [ComfyUI] Node 4: SaveVideo [05:08] [ComfyUI] [ComfyUI-DepthAnythingV3] sys_path sent to worker: ['/workspaces/DepthAnythingV3-0902/ComfyUI', '/workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes/ComfyUI-DepthAnythingV3', '/root/.ce/.pixi/envs/depthanythingv3-nodes/lib/python3.12/site-packages', '/workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes/ComfyUI-DepthAnythingV3'] [05:08] [ComfyUI] Inputs: {"filename_prefix": "video/preview_da3", "format": "auto", "codec": "auto", "video": ["6", 0]} [05:08] [ComfyUI] [SubprocessWorker] is_pixi=True, python=/root/.ce/.pixi/envs/depthanythingv3-nodes/bin/python [05:08] [ComfyUI] Node 5: LoadSALADModel [05:08] [ComfyUI] [SubprocessWorker] launching cmd=['/root/.pixi/bin/pixi', 'run', '--as-is', '--manifest-path', '/root/.ce/pixi.toml', '-e', 'depthanythingv3-nodes', 'python', '/tmp/comfyui_pvenv_lusourms/persistent_worker.py', 'abstract://comfy_worker_f2b86d71b296']... [05:08] [ComfyUI] Inputs: {} [05:08] [ComfyUI] [SubprocessWorker] socket_addr=abstract://comfy_worker_f2b86d71b296 (abstract, no filesystem path) [05:08] [ComfyUI] Node 6: DepthAnythingV3_Streaming [05:08] [ComfyUI] Inputs: {"normalization_mode": "Standard", "chunk_size": 30, "overlap": 8, "align_lib": "auto", "align_method": "sim3", "resize_method": "resize", "invert_depth": true, "save_pointcloud": false, "sample_ratio [05:08] [ComfyUI] Generated prompt_id: 8b388a06-1ed2-462d-9ad3-8f3f9e5c67dc [05:09] [ComfyUI] Calling execution.validate_prompt()... [05:09] [ComfyUI] Validation result: valid=True [05:09] [ComfyUI] === Validation PASSED === [05:09] [ComfyUI] got prompt [05:09] [ComfyUI] [comfy-env] Running LoadSALADModel: [05:09] [ComfyUI] [VRAM] Before LoadSALADModel: 7563 / 7672 MB [05:09] [ComfyUI] [SubprocessWorker] call_method: nodes.load_model.LoadSALADModel.EXECUTE_NORMALIZED [05:09] [ComfyUI] [SubprocessWorker] acquired lock, ensuring started... [05:09] [ComfyUI] [ComfyUI-DepthAnythingV3] Health check: ping (timeout=5.0s)... [05:09] [ComfyUI] [ComfyUI-DepthAnythingV3] Health check: failed [05:09] [ComfyUI] [ComfyUI-DepthAnythingV3] Socket unhealthy, restarting worker... [05:10] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] ERROR: Failed to import comfy_kitchen, Error: No module named 'comfy_kitchen', fp8 and fp4 support will not be available. [05:10] [ComfyUI] [SubprocessWorker] worker started/confirmed [05:10] [ComfyUI] [SubprocessWorker] sending request via socket... [05:10] [ComfyUI] [SubprocessWorker] call_id=25 sending call_method [05:10] [ComfyUI] Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads. [05:10] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] WARNING: Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads. [05:11] [ComfyUI] [05:11] Frame 2 saved (frame_001.jpg, t=2.2s) [05:11] [capture-periodic] freeze=1ms shot=126ms unfreeze=5ms saved=True [05:13] Frame 3 saved (frame_002.jpg, t=4.5s) [05:13] [capture-periodic] freeze=2ms shot=137ms unfreeze=6ms saved=True [05:14] [capture-loop] iter=50 t=5.7s state={'complete': False, 'error': None, 'executedCount': 0, 'wsState': 1} eval_ms=2 ram=4.9/23.4GB (21%) swap=1.8/32.0GB (6%) cpu=11% browser_rss=1.04GB [05:14] [ComfyUI] dino_salad.safetensors: 0%| | 0.00/352M [00:00'], 'salad_model': ["{'checkpoint_path': '/workspaces/DepthAnythingV3-0902/ComfyUI/models/salad/dino_salad.safetensors'}"]}, 'current_outputs': ['2', '5', '4', '6', '3'], 'timestamp': 1778836064087} [05:33] Capturing execution frames: simple.json [05:39] Frame 1 saved (frame_000.jpg, t=0.0s) [05:39] Validating workflow... [05:39] [ComfyUI] === /validate endpoint called === [05:39] [ComfyUI] Received JSON with keys: ['prompt'] [05:39] Queuing workflow for execution... [05:39] [ComfyUI] Prompt has 4 nodes [05:39] [ComfyUI] Node 2: LoadImage [05:39] [ComfyUI] Inputs: {"image": "bridge.jpg"} [05:39] [ComfyUI] Node 3: DownloadAndLoadDepthAnythingV3Model [05:39] [ComfyUI] Inputs: {"model": "da3_large.safetensors", "precision": "auto", "attention": "auto"} [05:39] [ComfyUI] Node 4: PreviewImage [05:39] [ComfyUI] Inputs: {"images": ["14", 0]} [05:39] [ComfyUI] Node 14: DepthAnything_V3 [05:39] [ComfyUI] Inputs: {"normalization_mode": "V2-Style", "resize_method": "resize", "invert_depth": true, "keep_model_size": false, "da3_model": ["3", 0], "images": ["2", 0]} [05:39] [ComfyUI] Generated prompt_id: 49d98b4b-8fab-4ee0-b8e4-f9aab24f76ee [05:39] [ComfyUI] Calling execution.validate_prompt()... [05:39] [ComfyUI] Validation result: valid=True [05:39] [ComfyUI] === Validation PASSED === [05:39] [ComfyUI] [ComfyUI-DepthAnythingV3] python: /root/.ce/.pixi/envs/depthanythingv3-nodes/bin/python [05:39] [ComfyUI] got prompt [05:39] [ComfyUI] [ComfyUI-DepthAnythingV3] sys_path sent to worker: ['/workspaces/DepthAnythingV3-0902/ComfyUI', '/workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes/ComfyUI-DepthAnythingV3', '/root/.ce/.pixi/envs/depthanythingv3-nodes/lib/python3.12/site-packages', '/workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes/ComfyUI-DepthAnythingV3'] [05:39] [ComfyUI] [comfy-env] Running DownloadAndLoadDepthAnythingV3Model: model: 'da3_large.safetensors', precision: 'auto', attention: 'auto' [05:39] [ComfyUI] [SubprocessWorker] is_pixi=True, python=/root/.ce/.pixi/envs/depthanythingv3-nodes/bin/python [05:39] [ComfyUI] [VRAM] Before DownloadAndLoadDepthAnythingV3Model: 7563 / 7672 MB [05:39] [ComfyUI] [SubprocessWorker] launching cmd=['/root/.pixi/bin/pixi', 'run', '--as-is', '--manifest-path', '/root/.ce/pixi.toml', '-e', 'depthanythingv3-nodes', 'python', '/tmp/comfyui_pvenv_lusourms/persistent_worker.py', 'abstract://comfy_worker_91c412a2e107']... [05:39] [ComfyUI] [SubprocessWorker] call_method: nodes.load_model.DownloadAndLoadDepthAnythingV3Model.EXECUTE_NORMALIZED [05:39] [ComfyUI] [SubprocessWorker] socket_addr=abstract://comfy_worker_91c412a2e107 (abstract, no filesystem path) [05:39] [ComfyUI] [SubprocessWorker] acquired lock, ensuring started... [05:39] [ComfyUI] [ComfyUI-DepthAnythingV3] Health check: ping (timeout=5.0s)... [05:39] [ComfyUI] [ComfyUI-DepthAnythingV3] Health check: failed [05:39] [ComfyUI] [ComfyUI-DepthAnythingV3] Socket unhealthy, restarting worker... [05:40] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] ERROR: Failed to import comfy_kitchen, Error: No module named 'comfy_kitchen', fp8 and fp4 support will not be available. [05:40] [ComfyUI] [SubprocessWorker] worker started/confirmed [05:40] [ComfyUI] [SubprocessWorker] serializing kwargs to shm... [05:40] [ComfyUI] [SubprocessWorker] created 0 shm blocks [05:40] [ComfyUI] [SubprocessWorker] sending request via socket... [05:40] [ComfyUI] [SubprocessWorker] call_id=28 sending call_method [05:40] [ComfyUI] [SubprocessWorker] got response: ok [05:40] [ComfyUI] [comfy-env] DownloadAndLoadDepthAnythingV3Model done (1.72s): result: NodeOutput [05:40] [ComfyUI] [VRAM] After DownloadAndLoadDepthAnythingV3Model: 737 / 7672 MB [05:40] [ComfyUI] [comfy-env] Running DepthAnything_V3: normalization_mode: 'V2-Style', resize_method: 'resize', invert_depth: True, keep_model_size: False, da3_model: dict, images: torch.float32 [1,683,1024,3] cpu [05:40] [ComfyUI] [VRAM] Before DepthAnything_V3: 737 / 7672 MB [05:40] [ComfyUI] [SubprocessWorker] call_method: nodes.nodes_inference.DepthAnything_V3.EXECUTE_NORMALIZED [05:40] [ComfyUI] [SubprocessWorker] acquired lock, ensuring started... [05:40] [ComfyUI] [ComfyUI-DepthAnythingV3] Health check: ping (timeout=5.0s)... [05:41] [ComfyUI] [ComfyUI-DepthAnythingV3] Health check: ok [05:41] [ComfyUI] [SubprocessWorker] worker started/confirmed [05:41] [ComfyUI] [comfy-env] PRE-SERIALIZE 'images' shape: torch.Size([1, 683, 1024, 3]) [05:41] [ComfyUI] [SubprocessWorker] serializing kwargs to shm... [05:41] [ComfyUI] [SubprocessWorker] created 0 shm blocks [05:41] [ComfyUI] [SubprocessWorker] sending request via socket... [05:41] [ComfyUI] [SubprocessWorker] call_id=29 sending call_method [05:41] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] _from_shm got dict with keys: ['__type__', 'strategy', 'parent_pid', 'fd', 'storage_size'] [05:41] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] DESERIALIZE TensorRef: tensor_size=[1, 683, 1024, 3] [05:41] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] DESERIALIZED tensor shape: torch.Size([1, 683, 1024, 3]) [05:41] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] WARNING: No comfy kitchen, using old apply_rope functions. [05:41] Frame 2 saved (frame_001.jpg, t=2.3s) [05:41] [capture-periodic] freeze=1ms shot=134ms unfreeze=3ms saved=True [05:43] Frame 3 saved (frame_002.jpg, t=4.6s) [05:43] [capture-periodic] freeze=2ms shot=140ms unfreeze=3ms saved=True [05:43] [ComfyUI] [SubprocessWorker] callback 'request_vram_budget' from call_id=29 [05:43] [ComfyUI] [comfy-env] VRAM request: 0.82GB, free before eviction: 7.27GB [05:43] [ComfyUI] [comfy-env] VRAM after eviction: 7.27GB [05:44] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [VRAM] +218MB (now 218MB) reserved=224MB peak=218MB [05:44] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] WARNING: WARNING: This model does not support sky segmentation. V2-Style normalization will work but without sky masking. Use Mono/Metric/Nested models for best V2-Style results. [05:44] [capture-loop] iter=50 t=5.7s state={'complete': False, 'error': None, 'executedCount': 0, 'wsState': 1} eval_ms=2 ram=5.2/23.4GB (22%) swap=1.8/32.0GB (6%) cpu=12% browser_rss=1.00GB [05:45] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [VRAM] +702MB (now 921MB) reserved=2560MB peak=1917MB [05:45] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=29 [05:45] [ComfyUI] [SubprocessWorker] got response: ok [05:45] [ComfyUI] [comfy-env] Created 1 model patchers: ['DepthAnything3Net_1'] [05:45] [ComfyUI] [comfy-env] DepthAnything_V3 done (4.31s): result: NodeOutput [05:45] [ComfyUI] [VRAM] After DepthAnything_V3: 3405 / 7672 MB [05:45] [ComfyUI] [VRAM] Loaded: SubprocessModel (783 MB) [05:45] [ComfyUI] Prompt executed in 6.08 seconds [05:45] Node executed (1 total), capturing... [05:45] Frame 4 saved (frame_003.jpg, t=6.7s) [05:45] [capture-node] freeze=2ms shot=147ms unfreeze=0ms saved=True [05:45] Execution complete (t=6.7s) [05:45] Captured 4 unique frames over 6.93s [05:45] [debug] iframes: 0 [05:45] [timing] trigger_3d_previews: 0.0s [05:48] Saved high-quality screenshot: simple_executed.png [05:48] [timing] final_screenshot: 0.3s [05:48] [ComfyUI] [comfy-env] [VRAM] Before offload 'DepthAnything3Net_1' (783 MB) -> cpu: 3405 / 7672 MB [05:48] Capturing execution frames: video_multiview_depth.json [05:48] [ComfyUI] [ComfyUI-DepthAnythingV3] Health check: ping (timeout=5.0s)... [05:48] [ComfyUI] [ComfyUI-DepthAnythingV3] Health check: ok [05:48] [ComfyUI] [SubprocessWorker] call_id=30 sending model_to_device [05:48] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [VRAM] -290MB (now 631MB) reserved=2560MB peak=1917MB [05:48] [ComfyUI] [comfy-env] [VRAM] After offload 'DepthAnything3Net_1' (783 MB): 909 / 7672 MB [05:53] Frame 1 saved (frame_000.jpg, t=0.0s) [05:53] Validating workflow... [05:53] [ComfyUI] === /validate endpoint called === [05:53] Queuing workflow for execution... [05:53] [ComfyUI] Received JSON with keys: ['prompt'] [05:53] [ComfyUI] Prompt has 9 nodes [05:53] [ComfyUI] Node 3: DownloadAndLoadDepthAnythingV3Model [05:53] [ComfyUI] Inputs: {"model": "da3_base.safetensors", "precision": "auto", "attention": "auto"} [05:53] [ComfyUI] Node 52: CreateVideo [05:53] [ComfyUI] Inputs: {"fps": 10, "images": ["62", 0]} [05:53] [ComfyUI] Node 53: CreateVideo [05:53] [ComfyUI] Inputs: {"fps": 10, "images": ["61", 0]} [05:53] [ComfyUI] Node 54: SaveVideo [05:53] [ComfyUI] Inputs: {"filename_prefix": "video/ComfyUI", "format": "auto", "codec": "auto", "video": ["52", 0]} [05:53] [ComfyUI] Node 55: SaveVideo [05:53] [ComfyUI] Inputs: {"filename_prefix": "video/ComfyUI", "format": "auto", "codec": "auto", "video": ["53", 0]} [05:53] [ComfyUI] Node 61: DepthAnything_V3 [05:53] [ComfyUI] Inputs: {"normalization_mode": "V2-Style", "resize_method": "resize", "invert_depth": false, "keep_model_size": false, "da3_model": ["3", 0], "images": ["71", 0]} [05:53] [ComfyUI] Node 62: DepthAnythingV3_MultiView [05:53] [ComfyUI] Inputs: {"normalization_mode": "V2-Style", "resize_method": "resize", "invert_depth": false, "keep_model_size": false, "da3_model": ["3", 0], "images": ["71", 0]} [05:53] [ComfyUI] Node 66: PreviewImage [05:53] [ComfyUI] Inputs: {"images": ["71", 0]} [05:53] [ComfyUI] Node 71: LoadImagesFromFolderKJ [05:53] [ComfyUI] Inputs: {"folder": "input/robotframes", "width": 1024, "height": 576, "keep_aspect_ratio": "crop", "image_load_cap": 0, "start_index": 0, "include_subfolders": false} [05:53] [ComfyUI] Generated prompt_id: a1c5f04a-4550-4b6b-99af-f9dfabe04419 [05:53] [ComfyUI] Calling execution.validate_prompt()... [05:53] [ComfyUI] Validation result: valid=True [05:53] [ComfyUI] === Validation PASSED === [05:53] [ComfyUI] got prompt [05:54] [ComfyUI] [comfy-env] Running DownloadAndLoadDepthAnythingV3Model: model: 'da3_base.safetensors', precision: 'auto', attention: 'auto' [05:54] [ComfyUI] [VRAM] Before DownloadAndLoadDepthAnythingV3Model: 909 / 7672 MB [05:54] [ComfyUI] [SubprocessWorker] call_method: nodes.load_model.DownloadAndLoadDepthAnythingV3Model.EXECUTE_NORMALIZED [05:54] [ComfyUI] [SubprocessWorker] acquired lock, ensuring started... [05:54] [ComfyUI] [ComfyUI-DepthAnythingV3] python: /root/.ce/.pixi/envs/depthanythingv3-nodes/bin/python [05:54] [ComfyUI] [ComfyUI-DepthAnythingV3] sys_path sent to worker: ['/workspaces/DepthAnythingV3-0902/ComfyUI', '/workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes/ComfyUI-DepthAnythingV3', '/root/.ce/.pixi/envs/depthanythingv3-nodes/lib/python3.12/site-packages', '/workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes/ComfyUI-DepthAnythingV3'] [05:54] [ComfyUI] [ComfyUI-DepthAnythingV3] Health check: ping (timeout=5.0s)... [05:54] [ComfyUI] [SubprocessWorker] is_pixi=True, python=/root/.ce/.pixi/envs/depthanythingv3-nodes/bin/python [05:54] [ComfyUI] [ComfyUI-DepthAnythingV3] Health check: failed [05:54] [ComfyUI] [SubprocessWorker] launching cmd=['/root/.pixi/bin/pixi', 'run', '--as-is', '--manifest-path', '/root/.ce/pixi.toml', '-e', 'depthanythingv3-nodes', 'python', '/tmp/comfyui_pvenv_lusourms/persistent_worker.py', 'abstract://comfy_worker_b29795b6ff15']... [05:54] [ComfyUI] [ComfyUI-DepthAnythingV3] Socket unhealthy, restarting worker... [05:54] [ComfyUI] [SubprocessWorker] socket_addr=abstract://comfy_worker_b29795b6ff15 (abstract, no filesystem path) [05:54] [ComfyUI] [comfy-env] Invalidated 1 stale model patchers (will be cleaned up during next unload) [05:54] Node executed (1 total), capturing... [05:55] Frame 2 saved (frame_001.jpg, t=1.3s) [05:55] [capture-node] freeze=2ms shot=131ms unfreeze=4ms saved=True [05:56] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] ERROR: Failed to import comfy_kitchen, Error: No module named 'comfy_kitchen', fp8 and fp4 support will not be available. [05:56] [ComfyUI] [SubprocessWorker] worker started/confirmed [05:56] [ComfyUI] [SubprocessWorker] serializing kwargs to shm... [05:56] [ComfyUI] [SubprocessWorker] created 0 shm blocks [05:56] [ComfyUI] [SubprocessWorker] sending request via socket... [05:56] [ComfyUI] [SubprocessWorker] call_id=31 sending call_method [05:56] [ComfyUI] [05:59] Frame 3 saved (frame_002.jpg, t=5.8s) [05:59] [capture-periodic] freeze=2ms shot=139ms unfreeze=4ms saved=True [05:59] [capture-loop] iter=50 t=6.1s state={'complete': False, 'error': None, 'executedCount': 1, 'wsState': 1} eval_ms=2 ram=5.0/23.4GB (22%) swap=1.8/32.0GB (6%) cpu=10% browser_rss=1.14GB [06:01] [ComfyUI] model.safetensors: 0%| | 0.00/542M [00:00 cpu: 737 / 7672 MB [06:03] [ComfyUI] [ComfyUI-DepthAnythingV3] Health check: ping (timeout=5.0s)... [06:03] [ComfyUI] [ComfyUI-DepthAnythingV3] Health check: ok [06:03] [ComfyUI] [SubprocessWorker] call_id=32 sending model_to_device [06:03] [ComfyUI] Model 'DepthAnything3Net_1' gone from worker (restarted?), treating as offloaded [06:03] [ComfyUI] [comfy-env] [VRAM] After offload 'DepthAnything3Net_1' (783 MB): 737 / 7672 MB [06:03] [ComfyUI] [comfy-env] DownloadAndLoadDepthAnythingV3Model done (8.29s): result: NodeOutput [06:03] [ComfyUI] [VRAM] After DownloadAndLoadDepthAnythingV3Model: 737 / 7672 MB [06:03] [ComfyUI] [comfy-env] Running DepthAnythingV3_MultiView: normalization_mode: 'V2-Style', resize_method: 'resize', invert_depth: False, keep_model_size: False, da3_model: dict, images: torch.float32 [13,576,1024,3] cpu [06:03] [ComfyUI] [VRAM] Before DepthAnythingV3_MultiView: 737 / 7672 MB [06:03] [ComfyUI] [SubprocessWorker] call_method: nodes.nodes_multiview.DepthAnythingV3_MultiView.EXECUTE_NORMALIZED [06:03] [ComfyUI] [SubprocessWorker] acquired lock, ensuring started... [06:03] [ComfyUI] [ComfyUI-DepthAnythingV3] Health check: ping (timeout=5.0s)... [06:03] [ComfyUI] [ComfyUI-DepthAnythingV3] Health check: ok [06:03] [ComfyUI] [SubprocessWorker] worker started/confirmed [06:03] [ComfyUI] [comfy-env] PRE-SERIALIZE 'images' shape: torch.Size([13, 576, 1024, 3]) [06:03] [ComfyUI] [SubprocessWorker] serializing kwargs to shm... [06:03] [ComfyUI] [SubprocessWorker] created 0 shm blocks [06:03] [ComfyUI] [SubprocessWorker] sending request via socket... [06:03] [ComfyUI] [SubprocessWorker] call_id=33 sending call_method [06:03] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] _from_shm got dict with keys: ['__type__', 'strategy', 'parent_pid', 'fd', 'storage_size'] [06:03] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] DESERIALIZE TensorRef: tensor_size=[13, 576, 1024, 3] [06:03] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] DESERIALIZED tensor shape: torch.Size([13, 576, 1024, 3]) [06:03] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] WARNING: No comfy kitchen, using old apply_rope functions. [06:04] Frame 5 saved (frame_004.jpg, t=10.3s) [06:04] [capture-periodic] freeze=2ms shot=133ms unfreeze=6ms saved=True [06:05] [capture-loop] iter=100 t=11.6s state={'complete': False, 'error': None, 'executedCount': 1, 'wsState': 1} eval_ms=2 ram=4.9/23.4GB (21%) swap=1.8/32.0GB (6%) cpu=14% browser_rss=0.98GB [06:05] [ComfyUI] [SubprocessWorker] callback 'request_vram_budget' from call_id=33 [06:05] [ComfyUI] [comfy-env] VRAM request: 0.27GB, free before eviction: 7.27GB [06:05] [ComfyUI] [comfy-env] VRAM after eviction: 7.27GB [06:05] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] WARNING: WARNING: This model does not support sky segmentation. V2-Style normalization will work but without sky masking. Use Mono/Metric/Nested models for best V2-Style results. [06:05] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [VRAM] +258MB (now 258MB) reserved=288MB peak=258MB [06:06] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [VRAM] +795MB (now 1053MB) reserved=1696MB peak=1053MB [06:07] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [VRAM] +294MB (now 1347MB) reserved=2688MB peak=1447MB [06:08] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [VRAM] -695MB (now 653MB) reserved=2688MB peak=1447MB [06:09] [ComfyUI] [SubprocessWorker] got response: ok [06:09] [ComfyUI] [comfy-env] Created 1 model patchers: ['DepthAnything3Net_1'] [06:09] [ComfyUI] [comfy-env] DepthAnythingV3_MultiView done (6.46s): result: NodeOutput [06:09] [ComfyUI] [VRAM] After DepthAnythingV3_MultiView: 3533 / 7672 MB [06:09] [ComfyUI] [VRAM] Loaded: SubprocessModel (258 MB) [06:09] [ComfyUI] [comfy-env] Running DepthAnything_V3: normalization_mode: 'V2-Style', resize_method: 'resize', invert_depth: False, keep_model_size: False, da3_model: dict, images: torch.float32 [13,576,1024,3] cpu [06:09] [ComfyUI] [VRAM] Before DepthAnything_V3: 3533 / 7672 MB [06:09] [ComfyUI] [VRAM] Loaded: SubprocessModel (258 MB) [06:09] [ComfyUI] [SubprocessWorker] call_method: nodes.nodes_inference.DepthAnything_V3.EXECUTE_NORMALIZED [06:09] [ComfyUI] [SubprocessWorker] acquired lock, ensuring started... [06:09] [ComfyUI] [ComfyUI-DepthAnythingV3] Health check: ping (timeout=5.0s)... [06:09] [ComfyUI] [ComfyUI-DepthAnythingV3] Health check: failed [06:09] [ComfyUI] [ComfyUI-DepthAnythingV3] Socket unhealthy, restarting worker... [06:09] [ComfyUI] [ComfyUI-DepthAnythingV3] python: /root/.ce/.pixi/envs/depthanythingv3-nodes/bin/python [06:09] [ComfyUI] [comfy-env] Invalidated 1 stale model patchers (will be cleaned up during next unload) [06:09] [ComfyUI] [ComfyUI-DepthAnythingV3] sys_path sent to worker: ['/workspaces/DepthAnythingV3-0902/ComfyUI', '/workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes/ComfyUI-DepthAnythingV3', '/root/.ce/.pixi/envs/depthanythingv3-nodes/lib/python3.12/site-packages', '/workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes/ComfyUI-DepthAnythingV3'] [06:09] [ComfyUI] [SubprocessWorker] is_pixi=True, python=/root/.ce/.pixi/envs/depthanythingv3-nodes/bin/python [06:09] [ComfyUI] [SubprocessWorker] launching cmd=['/root/.pixi/bin/pixi', 'run', '--as-is', '--manifest-path', '/root/.ce/pixi.toml', '-e', 'depthanythingv3-nodes', 'python', '/tmp/comfyui_pvenv_lusourms/persistent_worker.py', 'abstract://comfy_worker_90daabc7ff84']... [06:09] [ComfyUI] [SubprocessWorker] socket_addr=abstract://comfy_worker_90daabc7ff84 (abstract, no filesystem path) [06:09] Node executed (2 total), capturing... [06:10] Frame 6 saved (frame_005.jpg, t=16.3s) [06:10] [capture-node] freeze=2ms shot=150ms unfreeze=1ms saved=True [06:11] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] ERROR: Failed to import comfy_kitchen, Error: No module named 'comfy_kitchen', fp8 and fp4 support will not be available. [06:11] [capture-loop] iter=150 t=17.5s state={'complete': False, 'error': None, 'executedCount': 2, 'wsState': 1} eval_ms=2 ram=5.2/23.4GB (22%) swap=1.8/32.0GB (6%) cpu=15% browser_rss=1.02GB [06:11] [ComfyUI] [SubprocessWorker] worker started/confirmed [06:11] [ComfyUI] [comfy-env] PRE-SERIALIZE 'images' shape: torch.Size([13, 576, 1024, 3]) [06:11] [ComfyUI] [SubprocessWorker] serializing kwargs to shm... [06:11] [ComfyUI] [SubprocessWorker] created 0 shm blocks [06:11] [ComfyUI] [SubprocessWorker] sending request via socket... [06:11] [ComfyUI] [SubprocessWorker] call_id=34 sending call_method [06:11] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] _from_shm got dict with keys: ['__type__', 'strategy', 'parent_pid', 'fd', 'storage_size'] [06:11] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] DESERIALIZE TensorRef: tensor_size=[13, 576, 1024, 3] [06:11] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [comfy-env] DESERIALIZED tensor shape: torch.Size([13, 576, 1024, 3]) [06:11] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] WARNING: No comfy kitchen, using old apply_rope functions. [06:12] Frame 7 saved (frame_006.jpg, t=18.6s) [06:12] [capture-periodic] freeze=2ms shot=141ms unfreeze=5ms saved=True [06:13] [ComfyUI] [SubprocessWorker] callback 'request_vram_budget' from call_id=34 [06:13] [ComfyUI] [comfy-env] VRAM request: 0.27GB, free before eviction: 7.27GB [06:13] [ComfyUI] [comfy-env] VRAM after eviction: 7.27GB [06:13] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] WARNING: WARNING: This model does not support sky segmentation. V2-Style normalization will work but without sky masking. Use Mono/Metric/Nested models for best V2-Style results. [06:13] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [VRAM] +258MB (now 258MB) reserved=288MB peak=258MB [06:14] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=34 [06:14] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=34 [06:14] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=34 [06:15] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=34 [06:15] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [VRAM] +210MB (now 469MB) reserved=640MB peak=835MB [06:15] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=34 [06:15] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=34 [06:15] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=34 [06:15] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=34 [06:15] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=34 [06:15] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=34 [06:15] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=34 [06:15] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=34 [06:16] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=34 [06:16] [capture-loop] iter=200 t=23.0s state={'complete': False, 'error': None, 'executedCount': 2, 'wsState': 1} eval_ms=2 ram=6.7/23.4GB (29%) swap=1.8/32.0GB (6%) cpu=15% browser_rss=1.01GB [06:16] Frame 8 saved (frame_007.jpg, t=23.1s) [06:16] [capture-periodic] freeze=1ms shot=125ms unfreeze=7ms saved=True [06:17] [ComfyUI] [worker:ComfyUI-DepthAnythingV3] [VRAM] -202MB (now 267MB) reserved=1120MB peak=835MB [06:17] [ComfyUI] [SubprocessWorker] got response: ok [06:17] [ComfyUI] [comfy-env] [VRAM] Before offload 'DepthAnything3Net_1' (258 MB) -> cpu: 1963 / 7672 MB [06:17] [ComfyUI] [ComfyUI-DepthAnythingV3] Health check: ping (timeout=5.0s)... [06:17] [ComfyUI] [ComfyUI-DepthAnythingV3] Health check: ok [06:17] [ComfyUI] [SubprocessWorker] call_id=35 sending model_to_device [06:17] [ComfyUI] [comfy-env] [VRAM] After offload 'DepthAnything3Net_1' (258 MB): 907 / 7672 MB [06:17] [ComfyUI] [comfy-env] Created 1 model patchers: ['DepthAnything3Net_1'] [06:17] [ComfyUI] [comfy-env] DepthAnything_V3 done (7.95s): result: NodeOutput [06:17] [ComfyUI] [VRAM] After DepthAnything_V3: 907 / 7672 MB [06:17] [ComfyUI] [VRAM] Loaded: SubprocessModel (258 MB) [06:17] [ComfyUI] Prompt executed in 24.01 seconds [06:17] [ComfyUI] [comfy-env] [VRAM] Before offload 'DepthAnything3Net_1' (258 MB) -> cpu: 907 / 7672 MB [06:17] [ComfyUI] [ComfyUI-DepthAnythingV3] Health check: ping (timeout=5.0s)... [06:17] [ComfyUI] [ComfyUI-DepthAnythingV3] Health check: ok [06:17] [ComfyUI] [SubprocessWorker] call_id=36 sending model_to_device [06:17] [ComfyUI] [comfy-env] [VRAM] After offload 'DepthAnything3Net_1' (258 MB): 907 / 7672 MB [06:18] Node executed (3 total), capturing... [06:18] Frame 9 saved (frame_008.jpg, t=24.5s) [06:18] [capture-node] freeze=2ms shot=150ms unfreeze=0ms saved=True [06:18] Execution complete (t=24.5s) [06:18] Captured 9 unique frames over 24.68s [06:18] [debug] iframes: 0 [06:18] [timing] trigger_3d_previews: 0.0s [06:20] Saved high-quality screenshot: video_multiview_depth_executed.png [06:20] [timing] final_screenshot: 0.3s [06:20] Results saved to /logs/DepthAnythingV3-0902/dev/linux-gpu/results.json [06:20] Model report: 9 files, 2.4 GB -> /logs/DepthAnythingV3-0902/dev/linux-gpu/models.json [06:20] Saved: /logs/DepthAnythingV3-0902/dev/linux-gpu/index.html [06:20] linux: FAILED [06:20] Error: Workflow execution failed (2 error(s)) [06:20] Details: bas_relief.json: Workflow execution failed: {'prompt_id': 'a38e9f56-8351-4933-8312-f068d5b8399a', 'node_id': '21', 'node_type': 'DepthAnything_V3', 'executed': ['3', '2'], 'exception_message': 'Allocation on device 0 would exceed allowed memory. (out of memory)\nCurrently allocated : 4.33 GiB\nRequested : 1.27 GiB\nDevice limit : 7.49 GiB\nFree (according to CUDA): 109.19 MiB\nPyTorch limit (set by user-supplied memory fraction)\n : 17179869184.00 GiB\n\nWorker traceback:\nTraceback (most recent call last):\n File "/tmp/comfyui_pvenv_lusourms/persistent_worker.py", line 1762, in main\n result = method(**inputs)\n ^^^^^^^^^^^^^^^^\n File "/workspaces/DepthAnythingV3-0902/ComfyUI/comfy_api/latest/_io.py", line 1833, in EXECUTE_NORMALIZED\n to_return = cls.execute(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes/ComfyUI-DepthAnythingV3/nodes/nodes_inference.py", line 156, in execute\n output = model(img, extrinsics=ext_i, intrinsics=int_i, infer_gs=infer_gs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/root/.ce/.pixi/envs/depthanythingv3-nodes/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1779, in _wrapped_call_impl\n return self._call_impl(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/root/.ce/.pixi/envs/depthanythingv3-nodes/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1790, in _call_impl\n return forward_call(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes/ComfyUI-DepthAnythingV3/nodes/depth_anything_v3/model.py", line 1577, in forward\n output = self._process_depth_head(feats, H, W)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes/ComfyUI-DepthAnythingV3/nodes/depth_anything_v3/model.py", line 1591, in _process_depth_head\n return self.head(feats, H, W, patch_start_idx=0)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/root/.ce/.pixi/envs/depthanythingv3-nodes/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1779, in _wrapped_call_impl\n return self._call_impl(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/root/.ce/.pixi/envs/depthanythingv3-nodes/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1790, in _call_impl\n return forward_call(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes/ComfyUI-DepthAnythingV3/nodes/depth_anything_v3/model.py", line 1366, in forward\n out_dict = self._forward_impl(feats, H, W, patch_start_idx)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes/ComfyUI-DepthAnythingV3/nodes/depth_anything_v3/model.py", line 1419, in _forward_impl\n fused_main = self._add_pos_embed(fused_main, W, H)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes/ComfyUI-DepthAnythingV3/nodes/depth_anything_v3/model.py", line 1477, in _add_pos_embed\n pe = position_grid_to_embed(pe, x.shape[1]) * ratio\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes/ComfyUI-DepthAnythingV3/nodes/depth_anything_v3/model.py", line 131, in position_grid_to_embed\n emb = torch.cat([emb_x, emb_y], dim=-1)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\ntorch.OutOfMemoryError: Allocation on device 0 would exceed allowed memory. (out of memory)\nCurrently allocated : 4.33 GiB\nRequested : 1.27 GiB\nDevice limit : 7.49 GiB\nFree (according to CUDA): 109.19 MiB\nPyTorch limit (set by user-supplied memory fraction)\n : 17179869184.00 GiB\n\n', 'exception_type': 'comfy_env.isolation.workers.base.WorkerError', 'traceback': [' File "/workspaces/DepthAnythingV3-0902/ComfyUI/execution.py", line 535, in execute\n output_data, output_ui, has_subgraph, has_pending_tasks = await get_output_data(prompt_id, unique_id, obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n', ' File "/workspaces/DepthAnythingV3-0902/ComfyUI/execution.py", line 335, in get_output_data\n return_values = await _async_map_node_over_list(prompt_id, unique_id, obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n', ' File "/workspaces/DepthAnythingV3-0902/ComfyUI/execution.py", line 309, in _async_map_node_over_list\n await process_inputs(input_dict, i)\n', ' File "/workspaces/DepthAnythingV3-0902/ComfyUI/execution.py", line 297, in process_inputs\n result = f(**inputs)\n ^^^^^^^^^^^\n', ' File "/workspaces/DepthAnythingV3-0902/.venv/lib/python3.12/site-packages/comfy_env/isolation/metadata.py", line 556, in proxy\n result = worker.call_method(\n ^^^^^^^^^^^^^^^^^^^\n', ' File "/workspaces/DepthAnythingV3-0902/.venv/lib/python3.12/site-packages/comfy_env/isolation/workers/subprocess.py", line 674, in call_method\n raise WorkerError(\n'], 'current_inputs': {'normalization_mode': ['Raw'], 'resize_method': ['resize'], 'invert_depth': [False], 'keep_model_size': [True], 'da3_model': ["{'model_path': '/workspaces/DepthAnythingV3-0902/ComfyUI/models/depthanything3/da3_large.safetensors', 'model_key': 'da3-large', 'dtype': 'bf16', 'attention': 'auto'}"], 'images': ['tensor([[[[0.5922, 0.4510, 0.2863],\n [0.5922, 0.4588, 0.2902],\n [0.5529, 0.4235, 0.2667],\n [0.5176, 0.3922, 0.2588],\n [0.5059, 0.3882, 0.2706],\n [0.4863, 0.3765, 0.2549],\n ...,\n [0.5608, 0.4471, 0.3059],\n [0.5608, 0.4471, 0.3098],\n [0.5490, 0.4471, 0.3255],\n [0.5373, 0.4510, 0.3373],\n [0.5176, 0.4392, 0.3216],\n [0.5059, 0.4314, 0.3216]],\n\n [[0.5686, 0.4235, 0.2745],\n [0.5451, 0.4157, 0.2627],\n [0.5373, 0.4078, 0.2627],\n [0.5216, 0.4000, 0.2667],\n [0.5020, 0.3843, 0.2627],\n [0.5020, 0.3843, 0.2706],\n ...,\n [0.5608, 0.4471, 0.3020],\n [0.5647, 0.4549, 0.3098],\n [0.5647, 0.4627, 0.3373],\n [0.5412, 0.4549, 0.3451],\n [0.5137, 0.4392, 0.3294],\n [0.5020, 0.4314, 0.3255]],\n\n [[0.4745, 0.3529, 0.2235],\n [0.4706, 0.3569, 0.2235],\n [0.4745, 0.3647, 0.2392],\n [0.4824, 0.3686, 0.2431],\n [0.4824, 0.3686, 0.2471],\n [0.4941, 0.3843, 0.2627],\n ...,\n [0.5647, 0.4471, 0.3059],\n [0.5686, 0.4549, 0.3176],\n [0.5529, 0.4510, 0.3255],\n [0.5255, 0.4314, 0.3294],\n [0.5255, 0.4471, 0.3412],\n [0.5059, 0.4353, 0.3216]],\n\n [[0.4000, 0.2902, 0.1765],\n [0.3882, 0.2863, 0.1725],\n [0.3647, 0.2627, 0.1529],\n [0.3961, 0.2902, 0.1804],\n [0.4627, 0.3569, 0.2392],\n [0.4941, 0.3882, 0.2627],\n ...,\n [0.5765, 0.4588, 0.3255],\n [0.5765, 0.4667, 0.3255],\n [0.5686, 0.4588, 0.3294],\n [0.5451, 0.4510, 0.3333],\n [0.5333, 0.4510, 0.3412],\n [0.5294, 0.4471, 0.3333]],\n\n [[0.5098, 0.3882, 0.2627],\n [0.5176, 0.4000, 0.2706],\n [0.5059, 0.3843, 0.2627],\n [0.4745, 0.3569, 0.2392],\n [0.4706, 0.3647, 0.2431],\n [0.4902, 0.3804, 0.2627],\n ...,\n [0.5804, 0.4706, 0.3373],\n [0.5647, 0.4549, 0.3176],\n [0.5804, 0.4627, 0.3255],\n [0.5608, 0.4510, 0.3216],\n [0.5294, 0.4392, 0.3216],\n [0.5412, 0.4471, 0.3451]],\n\n [[0.4980, 0.3686, 0.2392],\n [0.5490, 0.4275, 0.2863],\n [0.5294, 0.4196, 0.2863],\n [0.4745, 0.3569, 0.2314],\n [0.4745, 0.3647, 0.2392],\n [0.4980, 0.3922, 0.2667],\n ...,\n [0.5529, 0.4471, 0.3137],\n [0.5569, 0.4431, 0.3137],\n [0.5725, 0.4549, 0.3176],\n [0.5569, 0.4471, 0.3020],\n [0.5569, 0.4549, 0.3137],\n [0.5451, 0.4431, 0.3294]],\n\n ...,\n\n [[0.5765, 0.4824, 0.3725],\n [0.5725, 0.4824, 0.3686],\n [0.5686, 0.4745, 0.3608],\n [0.5490, 0.4471, 0.3216],\n [0.5373, 0.4235, 0.2902],\n [0.5294, 0.4118, 0.2784],\n ...,\n [0.5373, 0.4196, 0.2745],\n [0.5765, 0.4471, 0.3020],\n [0.5333, 0.4000, 0.2588],\n [0.5647, 0.4353, 0.2784],\n [0.5608, 0.4275, 0.2627],\n [0.6000, 0.4588, 0.3059]],\n\n [[0.5451, 0.4471, 0.3373],\n [0.5647, 0.4706, 0.3569],\n [0.5608, 0.4627, 0.3490],\n [0.5255, 0.4196, 0.2980],\n [0.5255, 0.4078, 0.2784],\n [0.5176, 0.3961, 0.2627],\n ...,\n [0.5255, 0.4039, 0.2549],\n [0.5490, 0.4275, 0.2706],\n [0.5333, 0.4039, 0.2510],\n [0.5490, 0.4157, 0.2706],\n [0.5647, 0.4314, 0.2784],\n [0.5961, 0.4588, 0.2980]],\n\n [[0.5529, 0.4510, 0.3294],\n [0.5490, 0.4431, 0.3176],\n [0.5686, 0.4588, 0.3373],\n [0.5412, 0.4275, 0.3059],\n [0.5412, 0.4275, 0.2980],\n [0.5333, 0.4157, 0.2824],\n ...,\n [0.5569, 0.4235, 0.2863],\n [0.5608, 0.4353, 0.2980],\n [0.5490, 0.4157, 0.2824],\n [0.5608, 0.4196, 0.2863],\n [0.5647, 0.4314, 0.2863],\n [0.5922, 0.4588, 0.3098]],\n\n [[0.5294, 0.4196, 0.3059],\n [0.5333, 0.4275, 0.2941],\n [0.5412, 0.4235, 0.2941],\n [0.5333, 0.4118, 0.2902],\n [0.5373, 0.4196, 0.2941],\n [0.5294, 0.4118, 0.2902],\n ...,\n [0.5765, 0.4431, 0.3137],\n [0.5765, 0.4471, 0.3176],\n [0.5490, 0.4118, 0.2824],\n [0.5647, 0.4275, 0.2863],\n [0.5765, 0.4431, 0.2941],\n [0.5765, 0.4353, 0.2863]],\n\n [[0.5176, 0.4078, 0.2902],\n [0.5412, 0.4392, 0.3098],\n [0.5098, 0.3922, 0.2745],\n [0.5098, 0.3922, 0.2706],\n [0.5255, 0.4078, 0.2902],\n [0.5137, 0.4000, 0.2824],\n ...,\n [0.5608, 0.4235, 0.2941],\n [0.5765, 0.4353, 0.3059],\n [0.5451, 0.4078, 0.2627],\n [0.5451, 0.4078, 0.2549],\n [0.5451, 0.4000, 0.2471],\n [0.5255, 0.3686, 0.2157]],\n\n [[0.5176, 0.4196, 0.3098],\n [0.5725, 0.4667, 0.3569],\n [0.5490, 0.4392, 0.3294],\n [0.5412, 0.4353, 0.3137],\n [0.5373, 0.4275, 0.3059],\n [0.5176, 0.4039, 0.2863],\n ...,\n [0.5647, 0.4314, 0.2980],\n [0.5529, 0.4118, 0.2745],\n [0.5098, 0.3686, 0.2196],\n [0.5373, 0.3882, 0.2353],\n [0.5176, 0.3647, 0.2078],\n [0.4745, 0.3176, 0.1765]]]])']}, 'current_outputs': ['2', '32', '21', '3', '30', '27'], 'timestamp': 1778836033559} Details: Workflow: /node/ComfyUI-DepthAnythingV3/workflows/bas_relief.json Node error: DepthAnything_V3 da3_streaming.json: Workflow execution failed: {'prompt_id': 'a88b9bb9-da9c-4a06-8c80-471998281ad1', 'node_id': '6', 'node_type': 'DepthAnythingV3_Streaming', 'executed': ['3', '5', '2'], 'exception_message': 'Allocation on device 0 would exceed allowed memory. (out of memory)\nCurrently allocated : 4.85 GiB\nRequested : 286.44 MiB\nDevice limit : 7.49 GiB\nFree (according to CUDA): 109.19 MiB\nPyTorch limit (set by user-supplied memory fraction)\n : 17179869184.00 GiB\n\nWorker traceback:\nTraceback (most recent call last):\n File "/tmp/comfyui_pvenv_lusourms/persistent_worker.py", line 1762, in main\n result = method(**inputs)\n ^^^^^^^^^^^^^^^^\n File "/workspaces/DepthAnythingV3-0902/ComfyUI/comfy_api/latest/_io.py", line 1833, in EXECUTE_NORMALIZED\n to_return = cls.execute(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes/ComfyUI-DepthAnythingV3/nodes/streaming/node.py", line 249, in execute\n result = pipeline.run(normalized_images, pbar=pbar,\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes/ComfyUI-DepthAnythingV3/nodes/streaming/pipeline.py", line 277, in run\n result = self._process_chunk(normalized_images, start, end)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes/ComfyUI-DepthAnythingV3/nodes/streaming/pipeline.py", line 330, in _process_chunk\n output = self.model(chunk_input)\n ^^^^^^^^^^^^^^^^^^^^^^^\n File "/root/.ce/.pixi/envs/depthanythingv3-nodes/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1779, in _wrapped_call_impl\n return self._call_impl(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/root/.ce/.pixi/envs/depthanythingv3-nodes/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1790, in _call_impl\n return forward_call(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes/ComfyUI-DepthAnythingV3/nodes/depth_anything_v3/model.py", line 1577, in forward\n output = self._process_depth_head(feats, H, W)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes/ComfyUI-DepthAnythingV3/nodes/depth_anything_v3/model.py", line 1591, in _process_depth_head\n return self.head(feats, H, W, patch_start_idx=0)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/root/.ce/.pixi/envs/depthanythingv3-nodes/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1779, in _wrapped_call_impl\n return self._call_impl(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/root/.ce/.pixi/envs/depthanythingv3-nodes/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1790, in _call_impl\n return forward_call(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes/ComfyUI-DepthAnythingV3/nodes/depth_anything_v3/model.py", line 1379, in forward\n self._forward_impl([feat[s0:s1] for feat in feats], H, W, patch_start_idx)\n File "/workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes/ComfyUI-DepthAnythingV3/nodes/depth_anything_v3/model.py", line 1419, in _forward_impl\n fused_main = self._add_pos_embed(fused_main, W, H)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/workspaces/DepthAnythingV3-0902/ComfyUI/custom_nodes/ComfyUI-DepthAnythingV3/nodes/depth_anything_v3/model.py", line 1477, in _add_pos_embed\n pe = position_grid_to_embed(pe, x.shape[1]) * ratio\n ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~\ntorch.OutOfMemoryError: Allocation on device 0 would exceed allowed memory. (out of memory)\nCurrently allocated : 4.85 GiB\nRequested : 286.44 MiB\nDevice limit : 7.49 GiB\nFree (according to CUDA): 109.19 MiB\nPyTorch limit (set by user-supplied memory fraction)\n : 17179869184.00 GiB\n\n', 'exception_type': 'comfy_env.isolation.workers.base.WorkerError', 'traceback': [' File "/workspaces/DepthAnythingV3-0902/ComfyUI/execution.py", line 535, in execute\n output_data, output_ui, has_subgraph, has_pending_tasks = await get_output_data(prompt_id, unique_id, obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n', ' File "/workspaces/DepthAnythingV3-0902/ComfyUI/execution.py", line 335, in get_output_data\n return_values = await _async_map_node_over_list(prompt_id, unique_id, obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n', ' File "/workspaces/DepthAnythingV3-0902/ComfyUI/execution.py", line 309, in _async_map_node_over_list\n await process_inputs(input_dict, i)\n', ' File "/workspaces/DepthAnythingV3-0902/ComfyUI/execution.py", line 297, in process_inputs\n result = f(**inputs)\n ^^^^^^^^^^^\n', ' File "/workspaces/DepthAnythingV3-0902/.venv/lib/python3.12/site-packages/comfy_env/isolation/metadata.py", line 556, in proxy\n result = worker.call_method(\n ^^^^^^^^^^^^^^^^^^^\n', ' File "/workspaces/DepthAnythingV3-0902/.venv/lib/python3.12/site-packages/comfy_env/isolation/workers/subprocess.py", line 674, in call_method\n raise WorkerError(\n'], 'current_inputs': {'normalization_mode': ['Standard'], 'chunk_size': [30], 'overlap': [8], 'align_lib': ['auto'], 'align_method': ['sim3'], 'resize_method': ['resize'], 'invert_depth': [True], 'save_pointcloud': [False], 'sample_ratio': [0.015], 'conf_threshold_coef': [0.75], 'da3_model': ["{'model_path': '/workspaces/DepthAnythingV3-0902/ComfyUI/models/depthanything3/da3_large.safetensors', 'model_key': 'da3-large', 'dtype': 'bf16', 'attention': 'auto'}"], 'video': [''], 'salad_model': ["{'checkpoint_path': '/workspaces/DepthAnythingV3-0902/ComfyUI/models/salad/dino_salad.safetensors'}"]}, 'current_outputs': ['2', '5', '4', '6', '3'], 'timestamp': 1778836064087} Details: Workflow: /node/ComfyUI-DepthAnythingV3/workflows/da3_streaming.json Node error: DepthAnythingV3_Streaming [06:20] Stopping ComfyUI server...