[00:00] Crash dump logging enabled: /logs/Sharp-1914/dev/linux-gpu/crash_dump.log [00:00] [00:00] [1/5] SYNTAX [00:00] ---------------------------------------- [00:00] [DEBUG] server=None, api=None [00:00] Found pyproject.toml (modern format) [00:00] Found requirements.txt (legacy format) [00:00] Unicode check: OK (all characters cp1252-safe) [00:00] Forbidden patterns check: OK [00:00] [SYNTAX] PASSED [00:00] [00:00] [2/5] INSTALL [00:00] ---------------------------------------- [00:00] [DEBUG] server=None, api=None [00:00] Setting up ComfyUI... [00:00] Creating virtual environment... [00:00] Running: uv venv /workspaces/Sharp-1914/.venv --python 3.10 [00:00] torch_version='2.10.0' -> pinning torch==2.10.0 torchvision==0.25.0 torchaudio==2.10.0 [00:00] Cloning ComfyUI (latest)... [00:00] Running: git clone --depth 1 https://github.com/comfyanonymous/ComfyUI.git /workspaces/Sharp-1914/ComfyUI [00:02] Pinning torch family from https://download.pytorch.org/whl/cu128: torch==2.10.0 torchvision==0.25.0 torchaudio==2.10.0 [00:02] Running: uv pip install --python /workspaces/Sharp-1914/.venv/bin/python --index-url https://download.pytorch.org/whl/cu128 --extra-index-url https://pypi.org/simple --index-strategy unsafe-best-match torch==2.10.0 torchvision==0.25.0 torchaudio==2.10.0 [02:05] Installing ComfyUI requirements... [02:05] Running: uv pip install --python /workspaces/Sharp-1914/.venv/bin/python --index-url https://download.pytorch.org/whl/cu128 --extra-index-url https://pypi.org/simple --index-strategy unsafe-best-match -r /workspaces/Sharp-1914/ComfyUI/requirements.txt [02:23] Installing custom node... [02:23] Copying ComfyUI-Sharp to custom_nodes/... [02:23] Installing node requirements... [02:23] Running: uv pip install --python /workspaces/Sharp-1914/.venv/bin/python --index-url https://download.pytorch.org/whl/cu128 --extra-index-url https://pypi.org/simple --index-strategy unsafe-best-match -r /workspaces/Sharp-1914/ComfyUI/custom_nodes/ComfyUI-Sharp/requirements.txt [02:28] Running install.py... [02:28] Running: /workspaces/Sharp-1914/.venv/bin/python /workspaces/Sharp-1914/ComfyUI/custom_nodes/ComfyUI-Sharp/install.py [02:28] [02:28] Installing 2 node dependencies... [02:28] Cloning ComfyUI-GaussianPack... [02:35] Installing requirements for ComfyUI-GaussianPack... [02:37] Running install.py for ComfyUI-GaussianPack... [02:40] [comfy-env] _discover: scanning /workspaces/Sharp-1914/ComfyUI/custom_nodes [02:40] [comfy-env] _discover: ComfyUI-GaussianPack -> gaussianpack-nodes (custom_nodes/ComfyUI-GaussianPack/nodes/comfy-env.toml) [02:40] [comfy-env] _discover: ComfyUI-Sharp -> sharp-nodes (custom_nodes/ComfyUI-Sharp/nodes/comfy-env.toml) [02:40] [comfy-env] Workspace: /tmp/comfytest-home/.ce [02:40] [comfy-env] ComfyUI: /workspaces/Sharp-1914/ComfyUI [02:40] [comfy-env] Found 2 node config(s): [02:40] - gaussianpack-nodes <- custom_nodes/ComfyUI-GaussianPack/nodes/comfy-env.toml (python=host) [02:40] - sharp-nodes <- custom_nodes/ComfyUI-Sharp/nodes/comfy-env.toml (python=host) [02:40] [comfy-env] Bootstrap interpreter has python 3.10 + torch 2.10.0 (cu128). Looking for cuda-only wheels matching cu128/torch2.10/cp310. [02:41] [comfy-env] GPU: Detection: smi [02:41] Driver: 580.126.20 [02:41] CUDA: 13.0 [02:41] Recommended: CUDA 13.0 [02:41] [02:41] GPU 0: NVIDIA GeForce RTX 3090 (sm_86) [Ampere] 24576MB [02:41] [comfy-env] Comfyui feature -> torch 2.10.0 from https://download.pytorch.org/whl/cu128 (CUDA 12.8 via bootstrap torch) [02:41] [comfy-env] cuda-wheels: 1 package(s) need a matched (cuda, torch, python) combo: ['gsplat'] [02:41] [comfy-env] cuda-wheels tier 1 (bootstrap): probing cu12.8/torch2.10/cp310 [02:41] [cuda-wheels] Looking up gsplat: cu128 torch2.10 cp310 manylinux linux [02:41] [cuda-wheels] Found: gsplat-1.5.3+cu128torch2.10-cp310-cp310-manylinux_2_34_x86_64.manylinux_2_35_x86_64.whl [02:41] [comfy-env] cuda-wheels combo: cu12.8/torch2.10 (bootstrap matches; per-node envs will pin to this) [02:41] [comfy-env] Host glibc 2.35 -> system-requirements [02:41] [cuda-wheels] Looking up gsplat: cu128 torch2.10 cp310 manylinux linux [02:41] [cuda-wheels] Found: gsplat-1.5.3+cu128torch2.10-cp310-cp310-manylinux_2_34_x86_64.manylinux_2_35_x86_64.whl [02:41] [comfy-env] gaussianpack-nodes: cuda-wheels deferred for post-pixi install (gsplat) [02:41] Generated /tmp/comfytest-home/.ce/pixi.toml [02:41] [comfy-env] Installing 2 environment(s): [02:41] - gaussianpack-nodes (python=host, 8 pypi deps, 1 cuda wheels) [02:41] - sharp-nodes (python=host, 15 pypi deps) [02:41] [comfy-env] Running `pixi install -e gaussianpack-nodes -e sharp-nodes`... [02:45] WARN Skipped running the post-link scripts because `run-post-link-scripts` = `false` [02:45] - bin/.librsvg-pre-unlink.sh [02:45] To enable them, run: [02:45] pixi config set --local run-post-link-scripts insecure [02:45] More info: [02:45] https://pixi.sh/latest/reference/pixi_configuration/#run-post-link-scripts [02:51] WARN Skipped running the post-link scripts because `run-post-link-scripts` = `false` [02:51] - bin/.librsvg-pre-unlink.sh [02:51] To enable them, run: [02:51] pixi config set --local run-post-link-scripts insecure [02:51] More info: [02:51] https://pixi.sh/latest/reference/pixi_configuration/#run-post-link-scripts [04:14] The following environments have been installed: gaussianpack-nodes, sharp-nodes. [04:14] [comfy-env] gaussianpack-nodes: installing cuda-wheels with --no-deps (gsplat) [04:14] Using Python 3.10.20 environment at: .pixi/envs/gaussianpack-nodes [04:17] Resolved 1 package in 2.37s [04:17] Downloading gsplat (33.9MiB) [04:18] Downloaded gsplat [04:18] Prepared 1 package in 1.06s [04:18] Installed 1 package in 46ms [04:18] + gsplat==1.5.3+cu128torch2.10 (from https://github.com/PozzettiAndrea/cuda-wheels/releases/download/gsplat-latest/gsplat-1.5.3%2Bcu128torch2.10-cp310-cp310-manylinux_2_34_x86_64.manylinux_2_35_x86_64.whl) [04:18] [comfy-env] Recorded install hash c13fd71e660f -> /tmp/comfytest-home/.ce/install.hash [04:18] [comfy-env] Install log: /tmp/comfytest-home/.ce/install.log [04:18] [04:18] Installation complete! [04:18] Cloning ComfyUI-PanoPack... [04:21] Installing requirements for ComfyUI-PanoPack... [04:21] Running install.py for ComfyUI-PanoPack... [04:21] [04:21] Installing 2 node dependencies... [04:21] Cloning ComfyUI-MoGe2... [04:22] Installing requirements for ComfyUI-MoGe2... [04:22] Running install.py for ComfyUI-MoGe2... [04:22] [comfy-env] _discover: scanning /workspaces/Sharp-1914/ComfyUI/custom_nodes [04:22] [comfy-env] _discover: ComfyUI-GaussianPack -> gaussianpack-nodes (custom_nodes/ComfyUI-GaussianPack/nodes/comfy-env.toml) [04:22] [comfy-env] _discover: ComfyUI-MoGe2 -> moge2-nodes (custom_nodes/ComfyUI-MoGe2/nodes/comfy-env.toml) [04:22] [comfy-env] _discover: ComfyUI-PanoPack -> panopack-nodes (custom_nodes/ComfyUI-PanoPack/nodes/comfy-env.toml) [04:22] [comfy-env] _discover: ComfyUI-Sharp -> sharp-nodes (custom_nodes/ComfyUI-Sharp/nodes/comfy-env.toml) [04:22] [comfy-env] Workspace: /tmp/comfytest-home/.ce [04:22] [comfy-env] ComfyUI: /workspaces/Sharp-1914/ComfyUI [04:22] [comfy-env] Found 4 node config(s): [04:22] - gaussianpack-nodes <- custom_nodes/ComfyUI-GaussianPack/nodes/comfy-env.toml (python=host) [04:22] - moge2-nodes <- custom_nodes/ComfyUI-MoGe2/nodes/comfy-env.toml (python=host) [04:22] - panopack-nodes <- custom_nodes/ComfyUI-PanoPack/nodes/comfy-env.toml (python=host) [04:22] - sharp-nodes <- custom_nodes/ComfyUI-Sharp/nodes/comfy-env.toml (python=host) [04:22] [comfy-env] Bootstrap interpreter has python 3.10 + torch 2.10.0 (cu128). Looking for cuda-only wheels matching cu128/torch2.10/cp310. [04:22] [comfy-env] GPU: Detection: smi [04:22] Driver: 580.126.20 [04:22] CUDA: 13.0 [04:22] Recommended: CUDA 13.0 [04:22] [04:22] GPU 0: NVIDIA GeForce RTX 3090 (sm_86) [Ampere] 24576MB [04:22] [comfy-env] Comfyui feature -> torch 2.10.0 from https://download.pytorch.org/whl/cu128 (CUDA 12.8 via bootstrap torch) [04:22] [comfy-env] cuda-wheels: 3 package(s) need a matched (cuda, torch, python) combo: ['gsplat', 'sageattention', 'flash_attn'] [04:22] [comfy-env] cuda-wheels tier 1 (bootstrap): probing cu12.8/torch2.10/cp310 [04:22] [cuda-wheels] Looking up gsplat: cu128 torch2.10 cp310 manylinux linux [04:22] [cuda-wheels] Found: gsplat-1.5.3+cu128torch2.10-cp310-cp310-manylinux_2_34_x86_64.manylinux_2_35_x86_64.whl [04:22] [cuda-wheels] Looking up sageattention: cu128 torch2.10 cp310 manylinux linux [04:22] [cuda-wheels] Found: sageattention-2.2.0+cu128torch2.10-cp310-cp310-manylinux_2_34_x86_64.manylinux_2_35_x86_64.whl [04:22] [cuda-wheels] Looking up flash_attn: cu128 torch2.10 cp310 manylinux linux [04:23] [cuda-wheels] Found: flash_attn-2.8.3+cu128torch2.10-cp310-cp310-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl [04:23] [comfy-env] cuda-wheels combo: cu12.8/torch2.10 (bootstrap matches; per-node envs will pin to this) [04:23] [comfy-env] Host glibc 2.35 -> system-requirements [04:23] [cuda-wheels] Looking up gsplat: cu128 torch2.10 cp310 manylinux linux [04:23] [cuda-wheels] Found: gsplat-1.5.3+cu128torch2.10-cp310-cp310-manylinux_2_34_x86_64.manylinux_2_35_x86_64.whl [04:23] [comfy-env] gaussianpack-nodes: cuda-wheels deferred for post-pixi install (gsplat) [04:23] [cuda-wheels] Looking up sageattention: cu128 torch2.10 cp310 manylinux linux [04:23] [cuda-wheels] Found: sageattention-2.2.0+cu128torch2.10-cp310-cp310-manylinux_2_34_x86_64.manylinux_2_35_x86_64.whl [04:23] [cuda-wheels] Looking up flash_attn: cu128 torch2.10 cp310 manylinux linux [04:23] [cuda-wheels] Found: flash_attn-2.8.3+cu128torch2.10-cp310-cp310-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl [04:23] [comfy-env] moge2-nodes: cuda-wheels deferred for post-pixi install (sageattention, flash_attn) [04:23] Generated /tmp/comfytest-home/.ce/pixi.toml [04:23] [comfy-env] Installing 4 environment(s): [04:23] - gaussianpack-nodes (python=host, 8 pypi deps, 1 cuda wheels) [04:23] - moge2-nodes (python=host, 12 pypi deps, 2 cuda wheels) [04:23] - panopack-nodes (python=host, 21 pypi deps) [04:23] - sharp-nodes (python=host, 15 pypi deps) [04:23] [comfy-env] Running `pixi install -e gaussianpack-nodes -e moge2-nodes -e panopack-nodes -e sharp-nodes`... [04:28] WARN Skipped running the post-link scripts because `run-post-link-scripts` = `false` [04:28] - bin/.librsvg-pre-unlink.sh [04:28] To enable them, run: [04:28] pixi config set --local run-post-link-scripts insecure [04:28] More info: [04:28] https://pixi.sh/latest/reference/pixi_configuration/#run-post-link-scripts [04:28] WARN Skipped running the post-link scripts because `run-post-link-scripts` = `false` [04:28] - bin/.librsvg-pre-unlink.sh [04:28] To enable them, run: [04:28] pixi config set --local run-post-link-scripts insecure [04:28] More info: [04:28] https://pixi.sh/latest/reference/pixi_configuration/#run-post-link-scripts [04:31] The following environments have been installed: gaussianpack-nodes, moge2-nodes, panopack-nodes, sharp-nodes. [04:31] [comfy-env] gaussianpack-nodes: installing cuda-wheels with --no-deps (gsplat) [04:32] Using Python 3.10.20 environment at: .pixi/envs/gaussianpack-nodes [04:32] Resolved 1 package in 944ms [04:32] Checked 1 package in 2ms [04:33] [comfy-env] moge2-nodes: installing cuda-wheels with --no-deps (sageattention, flash_attn) [04:33] Using Python 3.10.20 environment at: .pixi/envs/moge2-nodes [04:35] Resolved 2 packages in 2.03s [04:35] Downloading sageattention (53.1MiB) [04:35] Downloading flash-attn (241.9MiB) [04:37] Downloaded sageattention [04:43] Downloaded flash-attn [04:43] Prepared 2 packages in 8.05s [04:43] Installed 2 packages in 7ms [04:43] + flash-attn==2.8.3+cu128torch2.10 (from https://github.com/PozzettiAndrea/cuda-wheels/releases/download/flash_attn-latest/flash_attn-2.8.3%2Bcu128torch2.10-cp310-cp310-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl) [04:43] + sageattention==2.2.0+cu128torch2.10 (from https://github.com/PozzettiAndrea/cuda-wheels/releases/download/sageattention-latest/sageattention-2.2.0%2Bcu128torch2.10-cp310-cp310-manylinux_2_34_x86_64.manylinux_2_35_x86_64.whl) [04:43] [comfy-env] Recorded install hash ead5c265d7a8 -> /tmp/comfytest-home/.ce/install.hash [04:43] [comfy-env] Install log: /tmp/comfytest-home/.ce/install.log [04:43] [04:43] Installation complete! [04:43] Cloning ComfyUI-GeometryPack... [04:47] Installing requirements for ComfyUI-GeometryPack... [04:47] Running install.py for ComfyUI-GeometryPack... [04:48] [comfy-env] _discover: scanning /workspaces/Sharp-1914/ComfyUI/custom_nodes [04:48] [comfy-env] _discover: ComfyUI-GaussianPack -> gaussianpack-nodes (custom_nodes/ComfyUI-GaussianPack/nodes/comfy-env.toml) [04:48] [comfy-env] _discover: ComfyUI-GeometryPack -> geometrypack-nodes (custom_nodes/ComfyUI-GeometryPack/nodes/comfy-env.toml) [04:48] [comfy-env] _discover: ComfyUI-MoGe2 -> moge2-nodes (custom_nodes/ComfyUI-MoGe2/nodes/comfy-env.toml) [04:48] [comfy-env] _discover: ComfyUI-PanoPack -> panopack-nodes (custom_nodes/ComfyUI-PanoPack/nodes/comfy-env.toml) [04:48] [comfy-env] _discover: ComfyUI-Sharp -> sharp-nodes (custom_nodes/ComfyUI-Sharp/nodes/comfy-env.toml) [04:48] [comfy-env] Workspace: /tmp/comfytest-home/.ce [04:48] [comfy-env] ComfyUI: /workspaces/Sharp-1914/ComfyUI [04:48] [comfy-env] Found 5 node config(s): [04:48] - gaussianpack-nodes <- custom_nodes/ComfyUI-GaussianPack/nodes/comfy-env.toml (python=host) [04:48] - geometrypack-nodes <- custom_nodes/ComfyUI-GeometryPack/nodes/comfy-env.toml (python=host) [04:48] - moge2-nodes <- custom_nodes/ComfyUI-MoGe2/nodes/comfy-env.toml (python=host) [04:48] - panopack-nodes <- custom_nodes/ComfyUI-PanoPack/nodes/comfy-env.toml (python=host) [04:48] - sharp-nodes <- custom_nodes/ComfyUI-Sharp/nodes/comfy-env.toml (python=host) [04:48] [comfy-env] Bootstrap interpreter has python 3.10 + torch 2.10.0 (cu128). Looking for cuda-only wheels matching cu128/torch2.10/cp310. [04:48] [comfy-env] GPU: Detection: smi [04:48] Driver: 580.126.20 [04:48] CUDA: 13.0 [04:48] Recommended: CUDA 13.0 [04:48] [04:48] GPU 0: NVIDIA GeForce RTX 3090 (sm_86) [Ampere] 24576MB [04:48] [comfy-env] Comfyui feature -> torch 2.10.0 from https://download.pytorch.org/whl/cu128 (CUDA 12.8 via bootstrap torch) [04:48] [comfy-env] cuda-wheels: 4 package(s) need a matched (cuda, torch, python) combo: ['gsplat', 'cumesh', 'sageattention', 'flash_attn'] [04:48] [comfy-env] cuda-wheels tier 1 (bootstrap): probing cu12.8/torch2.10/cp310 [04:48] [cuda-wheels] Looking up gsplat: cu128 torch2.10 cp310 manylinux linux [04:48] [cuda-wheels] Found: gsplat-1.5.3+cu128torch2.10-cp310-cp310-manylinux_2_34_x86_64.manylinux_2_35_x86_64.whl [04:48] [cuda-wheels] Looking up cumesh: cu128 torch2.10 cp310 manylinux linux [04:48] [cuda-wheels] Found: cumesh-0.0.1+cu128torch2.10-cp310-cp310-manylinux_2_35_x86_64.whl [04:48] [cuda-wheels] Looking up sageattention: cu128 torch2.10 cp310 manylinux linux [04:48] [cuda-wheels] Found: sageattention-2.2.0+cu128torch2.10-cp310-cp310-manylinux_2_34_x86_64.manylinux_2_35_x86_64.whl [04:48] [cuda-wheels] Looking up flash_attn: cu128 torch2.10 cp310 manylinux linux [04:48] [cuda-wheels] Found: flash_attn-2.8.3+cu128torch2.10-cp310-cp310-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl [04:48] [comfy-env] cuda-wheels combo: cu12.8/torch2.10 (bootstrap matches; per-node envs will pin to this) [04:48] [comfy-env] Host glibc 2.35 -> system-requirements [04:48] [cuda-wheels] Looking up gsplat: cu128 torch2.10 cp310 manylinux linux [04:48] [cuda-wheels] Found: gsplat-1.5.3+cu128torch2.10-cp310-cp310-manylinux_2_34_x86_64.manylinux_2_35_x86_64.whl [04:48] [comfy-env] gaussianpack-nodes: cuda-wheels deferred for post-pixi install (gsplat) [04:48] [cuda-wheels] Looking up cumesh: cu128 torch2.10 cp310 manylinux linux [04:48] [cuda-wheels] Found: cumesh-0.0.1+cu128torch2.10-cp310-cp310-manylinux_2_35_x86_64.whl [04:48] [comfy-env] geometrypack-nodes: cuda-wheels deferred for post-pixi install (cumesh) [04:48] [cuda-wheels] Looking up sageattention: cu128 torch2.10 cp310 manylinux linux [04:48] [cuda-wheels] Found: sageattention-2.2.0+cu128torch2.10-cp310-cp310-manylinux_2_34_x86_64.manylinux_2_35_x86_64.whl [04:48] [cuda-wheels] Looking up flash_attn: cu128 torch2.10 cp310 manylinux linux [04:49] [cuda-wheels] Found: flash_attn-2.8.3+cu128torch2.10-cp310-cp310-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl [04:49] [comfy-env] moge2-nodes: cuda-wheels deferred for post-pixi install (sageattention, flash_attn) [04:49] Generated /tmp/comfytest-home/.ce/pixi.toml [04:49] [comfy-env] Installing 5 environment(s): [04:49] - gaussianpack-nodes (python=host, 8 pypi deps, 1 cuda wheels) [04:49] - geometrypack-nodes (python=host, 14 pypi deps, 1 cuda wheels) [04:49] - moge2-nodes (python=host, 12 pypi deps, 2 cuda wheels) [04:49] - panopack-nodes (python=host, 21 pypi deps) [04:49] - sharp-nodes (python=host, 15 pypi deps) [04:49] [comfy-env] Running `pixi install -e gaussianpack-nodes -e geometrypack-nodes -e moge2-nodes -e panopack-nodes -e sharp-nodes`... [05:00] WARN Skipped running the post-link scripts because `run-post-link-scripts` = `false` [05:00] - bin/.librsvg-pre-unlink.sh [05:00] To enable them, run: [05:00] pixi config set --local run-post-link-scripts insecure [05:00] More info: [05:00] https://pixi.sh/latest/reference/pixi_configuration/#run-post-link-scripts [05:01] The following environments have been installed: gaussianpack-nodes, geometrypack-nodes, moge2-nodes, panopack-nodes, sharp-nodes. [05:02] [comfy-env] gaussianpack-nodes: installing cuda-wheels with --no-deps (gsplat) [05:02] Using Python 3.10.20 environment at: .pixi/envs/gaussianpack-nodes [05:03] Resolved 1 package in 985ms [05:03] Checked 1 package in 0.08ms [05:03] [comfy-env] geometrypack-nodes: installing cuda-wheels with --no-deps (cumesh) [05:03] Using Python 3.10.20 environment at: .pixi/envs/geometrypack-nodes [05:04] Resolved 1 package in 1.15s [05:04] Downloading cumesh (22.7MiB) [05:04] Downloaded cumesh [05:04] Prepared 1 package in 483ms [05:04] Installed 1 package in 2ms [05:04] + cumesh==0.0.1+cu128torch2.10 (from https://github.com/PozzettiAndrea/cuda-wheels/releases/download/cumesh-latest/cumesh-0.0.1%2Bcu128torch2.10-cp310-cp310-manylinux_2_35_x86_64.whl) [05:04] [comfy-env] moge2-nodes: installing cuda-wheels with --no-deps (sageattention, flash_attn) [05:04] Using Python 3.10.20 environment at: .pixi/envs/moge2-nodes [05:04] Checked 2 packages in 91ms [05:04] [comfy-env] Recorded install hash 6a7bc6f93489 -> /tmp/comfytest-home/.ce/install.hash [05:04] [comfy-env] Install log: /tmp/comfytest-home/.ce/install.log [05:04] [05:04] Installation complete! [05:04] [05:04] [requirements] Re-installing main package requirements... [05:04] Installing requirements for ComfyUI-PanoPack... [05:04] [comfy-env] _discover: scanning /workspaces/Sharp-1914/ComfyUI/custom_nodes [05:04] [comfy-env] _discover: ComfyUI-GaussianPack -> gaussianpack-nodes (custom_nodes/ComfyUI-GaussianPack/nodes/comfy-env.toml) [05:04] [comfy-env] _discover: ComfyUI-GeometryPack -> geometrypack-nodes (custom_nodes/ComfyUI-GeometryPack/nodes/comfy-env.toml) [05:04] [comfy-env] _discover: ComfyUI-MoGe2 -> moge2-nodes (custom_nodes/ComfyUI-MoGe2/nodes/comfy-env.toml) [05:04] [comfy-env] _discover: ComfyUI-PanoPack -> panopack-nodes (custom_nodes/ComfyUI-PanoPack/nodes/comfy-env.toml) [05:04] [comfy-env] _discover: ComfyUI-Sharp -> sharp-nodes (custom_nodes/ComfyUI-Sharp/nodes/comfy-env.toml) [05:04] [comfy-env] Workspace inputs unchanged (hash 6a7bc6f93489) -- skipping install. Delete /tmp/comfytest-home/.ce/install.hash to force. [05:04] [05:04] Installation complete! [05:04] ComfyUI-MoGe2: exists [05:04] ComfyUI-GeometryPack: exists [05:04] [05:04] [requirements] Re-installing main package requirements... [05:04] Installing requirements for ComfyUI-Sharp... [05:04] [comfy-env] _discover: scanning /workspaces/Sharp-1914/ComfyUI/custom_nodes [05:04] [comfy-env] _discover: ComfyUI-GaussianPack -> gaussianpack-nodes (custom_nodes/ComfyUI-GaussianPack/nodes/comfy-env.toml) [05:05] [comfy-env] _discover: ComfyUI-GeometryPack -> geometrypack-nodes (custom_nodes/ComfyUI-GeometryPack/nodes/comfy-env.toml) [05:05] [comfy-env] _discover: ComfyUI-MoGe2 -> moge2-nodes (custom_nodes/ComfyUI-MoGe2/nodes/comfy-env.toml) [05:05] [comfy-env] _discover: ComfyUI-PanoPack -> panopack-nodes (custom_nodes/ComfyUI-PanoPack/nodes/comfy-env.toml) [05:05] [comfy-env] _discover: ComfyUI-Sharp -> sharp-nodes (custom_nodes/ComfyUI-Sharp/nodes/comfy-env.toml) [05:05] [comfy-env] Workspace inputs unchanged (hash 6a7bc6f93489) -- skipping install. Delete /tmp/comfytest-home/.ce/install.hash to force. [05:05] [05:05] Installation complete! [05:05] Installing 2 node dependency(ies)... [05:05] ComfyUI-GaussianPack from PozzettiAndrea/ComfyUI-GaussianPack [05:05] ComfyUI-GaussianPack already exists, skipping... [05:05] ComfyUI-PanoPack from PozzettiAndrea/ComfyUI-PanoPack [05:05] ComfyUI-PanoPack already exists, skipping... [05:05] Installing validation endpoint... [05:05] Cloning PozzettiAndrea/ComfyUI-validate-endpoint... [05:05] Running: git clone --depth 1 https://x-access-token:***@github.com/PozzettiAndrea/ComfyUI-validate-endpoint.git /workspaces/Sharp-1914/ComfyUI/custom_nodes/ComfyUI-validate-endpoint [05:05] [INSTALL] PASSED [05:05] [05:05] [3/5] REGISTRATION [05:05] ---------------------------------------- [05:05] [DEBUG] server=None, api=None [05:05] Starting ComfyUI server... [05:05] Starting ComfyUI server on port 41899... [05:05] Starting ComfyUI server on port 41899... [05:05] Waiting for server to be ready (timeout: 600s)... [05:06] [ComfyUI] [INFO] setup plugin alembic.autogenerate.schemas [05:06] [ComfyUI] [INFO] setup plugin alembic.autogenerate.tables [05:06] [ComfyUI] [INFO] setup plugin alembic.autogenerate.types [05:06] [ComfyUI] [INFO] setup plugin alembic.autogenerate.constraints [05:06] [ComfyUI] [INFO] setup plugin alembic.autogenerate.defaults [05:06] [ComfyUI] [INFO] setup plugin alembic.autogenerate.comments [05:06] [ComfyUI] [comfy-env] ComfyUI-PanoPack: 1 isolation env(s): [05:06] [ComfyUI] [comfy-env] nodes -> /workspaces/Sharp-1914/ComfyUI/custom_nodes/ComfyUI-PanoPack/nodes [05:06] [ComfyUI] [comfy-env] Workspace: /tmp/comfytest-home/.ce [05:06] [ComfyUI] [comfy-env] env: /tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes [OK] [05:06] [ComfyUI] [comfy-env] prestartup complete [05:06] [ComfyUI] [comfy-env] ComfyUI-MoGe2: 1 isolation env(s): [05:06] [ComfyUI] [comfy-env] nodes -> /workspaces/Sharp-1914/ComfyUI/custom_nodes/ComfyUI-MoGe2/nodes [05:06] [ComfyUI] [comfy-env] env: /tmp/comfytest-home/.ce/.pixi/envs/moge2-nodes [OK] [05:06] [ComfyUI] [comfy-env] prestartup complete [05:06] [ComfyUI] [comfy-env] ComfyUI-Sharp: 1 isolation env(s): [05:06] [ComfyUI] [comfy-env] nodes -> /workspaces/Sharp-1914/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes [05:06] [ComfyUI] [comfy-env] env: /tmp/comfytest-home/.ce/.pixi/envs/sharp-nodes [OK] [05:06] [ComfyUI] [comfy-env] prestartup complete [05:06] [ComfyUI] [comfy-env] ComfyUI-GeometryPack: 1 isolation env(s): [05:06] [ComfyUI] [comfy-env] nodes -> /workspaces/Sharp-1914/ComfyUI/custom_nodes/ComfyUI-GeometryPack/nodes [05:06] [ComfyUI] [comfy-env] env: /tmp/comfytest-home/.ce/.pixi/envs/geometrypack-nodes [OK] [05:06] [ComfyUI] [comfy-env] prestartup complete [05:06] [ComfyUI] [INFO] [05:06] [ComfyUI] Prestartup times for custom nodes: [05:06] [ComfyUI] [INFO] 0.0 seconds: /workspaces/Sharp-1914/ComfyUI/custom_nodes/ComfyUI-MoGe2 [05:06] [ComfyUI] [INFO] 0.0 seconds: /workspaces/Sharp-1914/ComfyUI/custom_nodes/ComfyUI-Sharp [05:06] [ComfyUI] [INFO] 0.1 seconds: /workspaces/Sharp-1914/ComfyUI/custom_nodes/ComfyUI-GaussianPack [05:06] [ComfyUI] [INFO] 0.1 seconds: /workspaces/Sharp-1914/ComfyUI/custom_nodes/ComfyUI-GeometryPack [05:06] [ComfyUI] [INFO] 0.2 seconds: /workspaces/Sharp-1914/ComfyUI/custom_nodes/ComfyUI-PanoPack [05:06] [ComfyUI] [INFO] [05:11] [ComfyUI] [WARNING] WARNING: You need pytorch with cu130 or higher to use optimized CUDA operations. [05:11] [ComfyUI] [INFO] Found comfy_kitchen backend eager: {'available': True, 'disabled': False, 'unavailable_reason': None, 'capabilities': ['apply_rope', 'apply_rope1', 'apply_rope_split_half', 'apply_rope_split_half1', 'dequantize_mxfp8', 'dequantize_nvfp4', 'dequantize_per_tensor_fp8', 'gemv_awq_w4a16', 'quantize_mxfp8', 'quantize_nvfp4', 'quantize_per_tensor_fp8', 'quantize_svdquant_w4a4', 'scaled_mm_mxfp8', 'scaled_mm_nvfp4', 'scaled_mm_svdquant_w4a4', 'stochastic_rounding_fp8']} [05:11] [ComfyUI] [INFO] Found comfy_kitchen backend cuda: {'available': True, 'disabled': True, 'unavailable_reason': None, 'capabilities': ['apply_rope', 'apply_rope1', 'apply_rope_split_half', 'apply_rope_split_half1', 'dequantize_nvfp4', 'dequantize_per_tensor_fp8', 'gemv_awq_w4a16', 'quantize_mxfp8', 'quantize_nvfp4', 'quantize_per_tensor_fp8', 'quantize_svdquant_w4a4', 'scaled_mm_svdquant_w4a4', 'stochastic_rounding_fp8']} [05:11] [ComfyUI] [INFO] Found comfy_kitchen backend triton: {'available': True, 'disabled': True, 'unavailable_reason': None, 'capabilities': ['apply_rope', 'apply_rope1', 'apply_rope_split_half', 'apply_rope_split_half1', 'dequantize_nvfp4', 'dequantize_per_tensor_fp8', 'quantize_mxfp8', 'quantize_nvfp4', 'quantize_per_tensor_fp8']} [05:11] [ComfyUI] [INFO] Checkpoint files will always be loaded safely. [05:11] [ComfyUI] [INFO] Total VRAM 24122 MB, total RAM 32010 MB [05:11] [ComfyUI] [INFO] pytorch version: 2.10.0+cu128 [05:11] [ComfyUI] [INFO] Set vram state to: NORMAL_VRAM [05:11] [ComfyUI] [INFO] Device: cuda:0 NVIDIA GeForce RTX 3090 : cudaMallocAsync [05:11] [ComfyUI] [INFO] Using async weight offloading with 2 streams [05:11] [ComfyUI] [INFO] Enabled pinned memory 28809.0 [05:11] [ComfyUI] [INFO] working around nvidia conv3d memory bug. [05:11] [ComfyUI] [INFO] Using pytorch attention [05:14] [ComfyUI] aimdo: /project/src-posix/cuda-funchooks.c:52:DEBUG:aimdo_setup_hooks: hooks successfully installed [05:14] [ComfyUI] aimdo: /project/src/control.c:240:INFO:comfy-aimdo inited for GPU: NVIDIA GeForce RTX 3090 (VRAM: 24122 MB) [05:14] [ComfyUI] [INFO] DynamicVRAM support detected and enabled [05:14] [ComfyUI] [INFO] Python version: 3.10.20 (main, Jun 2 2026, 22:25:53) [Clang 22.1.3 ] [05:14] [ComfyUI] [INFO] ComfyUI version: 0.24.0 [05:14] [ComfyUI] [INFO] comfy-aimdo version: 0.4.9 [05:14] [ComfyUI] [INFO] comfy-kitchen version: 0.2.10 [05:14] [ComfyUI] [WARNING] ****** User settings have been changed to be stored on the server instead of browser storage. ****** [05:14] [ComfyUI] [WARNING] ****** For multi-user setups add the --multi-user CLI argument to enable multiple user profiles. ****** [05:14] [ComfyUI] [INFO] comfyui-frontend-package version: 1.45.15 [05:14] [ComfyUI] [INFO] comfyui-workflow-templates version: 0.9.98 [05:14] [ComfyUI] [INFO] comfyui-embedded-docs version: 0.5.2 [05:14] [ComfyUI] [INFO] comfy-kitchen version: 0.2.10 [05:14] [ComfyUI] [INFO] comfy-aimdo version: 0.4.9 [05:14] [ComfyUI] [INFO] [Prompt Server] web root: /workspaces/Sharp-1914/.venv/lib/python3.10/site-packages/comfyui_frontend_package/static [05:14] [ComfyUI] [INFO] Asset seeder disabled [05:16] [ComfyUI] [comfy-env] Version: 0.3.89 [05:16] [ComfyUI] [comfy-env] register_nodes: ComfyUI-PanoPack [05:16] [ComfyUI] [comfy-env] ComfyUI source dir: /workspaces/Sharp-1914/ComfyUI [05:16] [ComfyUI] [comfy-env] Found 1 isolation env(s) [05:16] [ComfyUI] [comfy-env] Scanning nodes metadata (isolation root)... [05:16] [ComfyUI] [comfy-env] Metadata scan: /tmp/comfytest-home/.pixi/bin/pixi run --as-is --manifest-path /tmp/comfytest-home/.ce/pixi.toml -e panopack-nodes python /tmp/comfy_meta_dd9m07mn.py /workspaces/Sharp-1914/ComfyUI/custom_nodes/ComfyUI-PanoPack nodes /tmp/comfy_meta_out__t6jioa4.pkl [05:16] [ComfyUI] [comfy-env] Scan env PATH for nodes: [05:16] [ComfyUI] [comfy-env] [0] /tmp/comfytest-tools/bin [05:16] [ComfyUI] [comfy-env] [1] /opt/uv/bin [05:16] [ComfyUI] [comfy-env] [2] /usr/local/sbin [05:16] [ComfyUI] [comfy-env] [3] /usr/local/bin [05:16] [ComfyUI] [comfy-env] [4] /usr/sbin [05:16] [ComfyUI] [comfy-env] [5] /usr/bin [05:16] [ComfyUI] [comfy-env] [6] /sbin [05:16] [ComfyUI] [comfy-env] [7] /bin [05:18] [ComfyUI] [comfy-env] Metadata scan stderr for nodes: [05:18] [ComfyUI] [comfy-env] [meta-scan] importing nodes from /workspaces/Sharp-1914/ComfyUI/custom_nodes/ComfyUI-PanoPack [05:18] [ComfyUI] [comfy-env] [meta-scan] import OK [05:18] [ComfyUI] [comfy-env] Scanned nodes: 15 nodes (2.4s) [05:18] [ComfyUI] [comfy-env] Scanned nodes root: 15 nodes (2.4s) [05:19] [ComfyUI] [comfy-env] Imported nodes root: 15 nodes (isolation) [05:19] [ComfyUI] [comfy-env] Registered 15 total nodes [05:19] [ComfyUI] [comfy-env] Version: 0.3.89 [05:19] [ComfyUI] [comfy-env] register_nodes: ComfyUI-MoGe2 [05:19] [ComfyUI] [comfy-env] ComfyUI source dir: /workspaces/Sharp-1914/ComfyUI [05:19] [ComfyUI] [comfy-env] Found 1 isolation env(s) [05:19] [ComfyUI] [comfy-env] Scanning nodes metadata (isolation root)... [05:19] [ComfyUI] [comfy-env] Metadata scan: /tmp/comfytest-home/.pixi/bin/pixi run --as-is --manifest-path /tmp/comfytest-home/.ce/pixi.toml -e moge2-nodes python /tmp/comfy_meta_scanqzpi.py /workspaces/Sharp-1914/ComfyUI/custom_nodes/ComfyUI-MoGe2 nodes /tmp/comfy_meta_out_wv94jhkw.pkl [05:19] [ComfyUI] [comfy-env] Scan env PATH for nodes: [05:19] [ComfyUI] [comfy-env] [0] /tmp/comfytest-tools/bin [05:19] [ComfyUI] [comfy-env] [1] /opt/uv/bin [05:19] [ComfyUI] [comfy-env] [2] /usr/local/sbin [05:19] [ComfyUI] [comfy-env] [3] /usr/local/bin [05:19] [ComfyUI] [comfy-env] [4] /usr/sbin [05:19] [ComfyUI] [comfy-env] [5] /usr/bin [05:19] [ComfyUI] [comfy-env] [6] /sbin [05:19] [ComfyUI] [comfy-env] [7] /bin [05:21] [ComfyUI] [comfy-env] Metadata scan stderr for nodes: [05:21] [ComfyUI] [comfy-env] [meta-scan] importing nodes from /workspaces/Sharp-1914/ComfyUI/custom_nodes/ComfyUI-MoGe2 [05:21] [ComfyUI] [comfy-env] ERROR:root:Failed to import comfy_kitchen, Error: No module named 'comfy_kitchen', fp8 and fp4 support will not be available. [05:21] [ComfyUI] [comfy-env] WARNING:root:comfy_kitchen does not support stochastic FP8 rounding, please update comfy_kitchen. [05:21] [ComfyUI] [comfy-env] [meta-scan] import OK [05:21] [ComfyUI] [comfy-env] Scanned nodes: 3 nodes (2.4s) [05:21] [ComfyUI] [comfy-env] Scanned nodes root: 3 nodes (2.4s) [05:21] [ComfyUI] [comfy-env] Imported nodes root: 3 nodes (isolation) [05:21] [ComfyUI] [comfy-env] Registered 3 total nodes [05:21] [ComfyUI] [comfy-env] Version: 0.3.89 [05:21] [ComfyUI] [comfy-env] register_nodes: ComfyUI-Sharp [05:21] [ComfyUI] [comfy-env] ComfyUI source dir: /workspaces/Sharp-1914/ComfyUI [05:21] [ComfyUI] [comfy-env] Found 1 isolation env(s) [05:21] [ComfyUI] [comfy-env] Scanning nodes metadata (isolation root)... [05:21] [ComfyUI] [comfy-env] Metadata scan: /tmp/comfytest-home/.pixi/bin/pixi run --as-is --manifest-path /tmp/comfytest-home/.ce/pixi.toml -e sharp-nodes python /tmp/comfy_meta_b2_6lc64.py /workspaces/Sharp-1914/ComfyUI/custom_nodes/ComfyUI-Sharp nodes /tmp/comfy_meta_out_a8p45fda.pkl [05:21] [ComfyUI] [comfy-env] Scan env PATH for nodes: [05:21] [ComfyUI] [comfy-env] [0] /tmp/comfytest-tools/bin [05:21] [ComfyUI] [comfy-env] [1] /opt/uv/bin [05:21] [ComfyUI] [comfy-env] [2] /usr/local/sbin [05:21] [ComfyUI] [comfy-env] [3] /usr/local/bin [05:21] [ComfyUI] [comfy-env] [4] /usr/sbin [05:21] [ComfyUI] [comfy-env] [5] /usr/bin [05:21] [ComfyUI] [comfy-env] [6] /sbin [05:21] [ComfyUI] [comfy-env] [7] /bin [05:23] [ComfyUI] [comfy-env] Metadata scan stderr for nodes: [05:23] [ComfyUI] [comfy-env] [meta-scan] importing nodes from /workspaces/Sharp-1914/ComfyUI/custom_nodes/ComfyUI-Sharp [05:23] [ComfyUI] [comfy-env] [meta-scan] import OK [05:23] [ComfyUI] [comfy-env] Scanned nodes: 17 nodes (2.6s) [05:23] [ComfyUI] [comfy-env] Scanned nodes root: 17 nodes (2.6s) [05:23] [ComfyUI] [comfy-env] Imported nodes root: 17 nodes (isolation) [05:23] [ComfyUI] [comfy-env] Registered 17 total nodes [05:23] [ComfyUI] [comfy-env] Version: 0.3.89 [05:24] [ComfyUI] [comfy-env] register_nodes: ComfyUI-GaussianPack [05:24] [ComfyUI] [comfy-env] ComfyUI source dir: /workspaces/Sharp-1914/ComfyUI [05:24] [ComfyUI] [comfy-env] Found 1 isolation env(s) [05:24] [ComfyUI] [comfy-env] Scanning nodes metadata (isolation root)... [05:24] [ComfyUI] [comfy-env] Metadata scan: /tmp/comfytest-home/.pixi/bin/pixi run --as-is --manifest-path /tmp/comfytest-home/.ce/pixi.toml -e gaussianpack-nodes python /tmp/comfy_meta_7z1vhg8x.py /workspaces/Sharp-1914/ComfyUI/custom_nodes/ComfyUI-GaussianPack nodes /tmp/comfy_meta_out_wiowns0a.pkl [05:24] [ComfyUI] [comfy-env] Scan env PATH for nodes: [05:24] [ComfyUI] [comfy-env] [0] /tmp/comfytest-tools/bin [05:24] [ComfyUI] [comfy-env] [1] /opt/uv/bin [05:24] [ComfyUI] [comfy-env] [2] /usr/local/sbin [05:24] [ComfyUI] [comfy-env] [3] /usr/local/bin [05:24] [ComfyUI] [comfy-env] [4] /usr/sbin [05:24] [ComfyUI] [comfy-env] [5] /usr/bin [05:24] [ComfyUI] [comfy-env] [6] /sbin [05:24] [ComfyUI] [comfy-env] [7] /bin [05:26] [ComfyUI] [comfy-env] Metadata scan stderr for nodes: [05:26] [ComfyUI] [comfy-env] [meta-scan] importing nodes from /workspaces/Sharp-1914/ComfyUI/custom_nodes/ComfyUI-GaussianPack [05:26] [ComfyUI] [comfy-env] SPZ route not registered (ModuleNotFoundError: No module named 'comfy_aimdo') [05:26] [ComfyUI] [comfy-env] [meta-scan] import OK [05:26] [ComfyUI] [comfy-env] Scanned nodes: 11 nodes (2.5s) [05:26] [ComfyUI] [comfy-env] Scanned nodes root: 11 nodes (2.5s) [05:26] [ComfyUI] [comfy-env] Imported nodes root: 11 nodes (isolation) [05:26] [ComfyUI] [comfy-env] Registered 11 total nodes [05:26] [ComfyUI] [INFO] loading... [05:26] [ComfyUI] [INFO] calling register_nodes [05:26] [ComfyUI] [comfy-env] Version: 0.3.89 [05:26] [ComfyUI] [comfy-env] register_nodes: ComfyUI-GeometryPack [05:26] [ComfyUI] [comfy-env] ComfyUI source dir: /workspaces/Sharp-1914/ComfyUI [05:26] [ComfyUI] [comfy-env] Found 1 isolation env(s) [05:26] [ComfyUI] [comfy-env] Scanning nodes metadata (isolation root)... [05:26] [ComfyUI] [comfy-env] Metadata scan: /tmp/comfytest-home/.pixi/bin/pixi run --as-is --manifest-path /tmp/comfytest-home/.ce/pixi.toml -e geometrypack-nodes python /tmp/comfy_meta_ke08f0ef.py /workspaces/Sharp-1914/ComfyUI/custom_nodes/ComfyUI-GeometryPack nodes /tmp/comfy_meta_out_hm7mr16r.pkl [05:26] [ComfyUI] [comfy-env] Scan env PATH for nodes: [05:26] [ComfyUI] [comfy-env] [0] /tmp/comfytest-tools/bin [05:26] [ComfyUI] [comfy-env] [1] /opt/uv/bin [05:26] [ComfyUI] [comfy-env] [2] /usr/local/sbin [05:26] [ComfyUI] [comfy-env] [3] /usr/local/bin [05:26] [ComfyUI] [comfy-env] [4] /usr/sbin [05:26] [ComfyUI] [comfy-env] [5] /usr/bin [05:26] [ComfyUI] [comfy-env] [6] /sbin [05:26] [ComfyUI] [comfy-env] [7] /bin [05:29] [ComfyUI] [comfy-env] Metadata scan stderr for nodes: [05:29] [ComfyUI] [comfy-env] [meta-scan] importing nodes from /workspaces/Sharp-1914/ComfyUI/custom_nodes/ComfyUI-GeometryPack [05:29] [ComfyUI] [comfy-env] ERROR:root:Failed to import comfy_kitchen, Error: No module named 'comfy_kitchen', fp8 and fp4 support will not be available. [05:29] [ComfyUI] [comfy-env] WARNING:root:comfy_kitchen does not support stochastic FP8 rounding, please update comfy_kitchen. [05:29] [ComfyUI] [comfy-env] [meta-scan] import OK [05:29] [ComfyUI] [comfy-env] Scanned nodes: 125 nodes (3.2s) [05:29] [ComfyUI] [comfy-env] Scanned nodes root: 125 nodes (3.3s) [05:29] [ComfyUI] [comfy-env] Imported nodes root: 125 nodes (isolation) [05:29] [ComfyUI] [comfy-env] Registered 125 total nodes [05:29] [ComfyUI] [INFO] [05:29] [ComfyUI] Import times for custom nodes: [05:29] [ComfyUI] [INFO] 0.0 seconds: /workspaces/Sharp-1914/ComfyUI/custom_nodes/ComfyUI-validate-endpoint [05:29] [ComfyUI] [INFO] 0.0 seconds: /workspaces/Sharp-1914/ComfyUI/custom_nodes/websocket_image_save.py [05:29] [ComfyUI] [INFO] 2.4 seconds: /workspaces/Sharp-1914/ComfyUI/custom_nodes/ComfyUI-MoGe2 [05:29] [ComfyUI] [INFO] 2.5 seconds: /workspaces/Sharp-1914/ComfyUI/custom_nodes/ComfyUI-PanoPack [05:29] [ComfyUI] [INFO] 2.5 seconds: /workspaces/Sharp-1914/ComfyUI/custom_nodes/ComfyUI-GaussianPack [05:29] [ComfyUI] [INFO] 2.6 seconds: /workspaces/Sharp-1914/ComfyUI/custom_nodes/ComfyUI-Sharp [05:29] [ComfyUI] [INFO] 3.3 seconds: /workspaces/Sharp-1914/ComfyUI/custom_nodes/ComfyUI-GeometryPack [05:29] [ComfyUI] [INFO] [05:29] [ComfyUI] [INFO] Context impl SQLiteImpl. [05:29] [ComfyUI] [INFO] Will assume non-transactional DDL. [05:29] [ComfyUI] [INFO] Context impl SQLiteImpl. [05:29] [ComfyUI] [INFO] Will assume non-transactional DDL. [05:29] [ComfyUI] [INFO] Running upgrade -> 0001_assets, Initial assets schema [05:29] [ComfyUI] Revision ID: 0001_assets [05:29] [ComfyUI] Revises: None [05:29] [ComfyUI] Create Date: 2025-12-10 00:00:00 [05:30] [ComfyUI] [INFO] Running upgrade 0001_assets -> 0002_merge_to_asset_references, Merge AssetInfo and AssetCacheState into unified asset_references table. [05:30] [ComfyUI] [INFO] Running upgrade 0002_merge_to_asset_references -> 0003_add_metadata_job_id, Add system_metadata and job_id columns to asset_references. [05:30] [ComfyUI] Change preview_id FK from assets.id to asset_references.id. [05:30] [ComfyUI] [INFO] Database upgraded from None to 0003_add_metadata_job_id [05:30] [ComfyUI] [INFO] Using RAM pressure cache. [05:30] [ComfyUI] [INFO] Starting server [05:30] [ComfyUI] [05:30] [ComfyUI] [INFO] To see the GUI go to: http://127.0.0.1:41899 [05:30] Server responding, waiting for nodes to load... [05:50] Server is ready! [05:50] Checking for import errors in server logs... [05:50] No import errors detected [05:50] Found 942 registered nodes [05:50] [REGISTRATION] PASSED [05:50] [05:50] [4/5] INSTANTIATION [05:50] ---------------------------------------- [05:50] [DEBUG] server=, api= [05:50] Testing node constructors... [05:50] All 17 node(s) instantiated successfully! [05:50] [INSTANTIATION] PASSED [05:50] [05:50] [5/5] EXECUTION [05:50] ---------------------------------------- [05:50] [DEBUG] server=, api= [05:51] GPU runner - will execute 4 workflow(s) [05:51] Running 4 workflow(s) (all with videos)... [05:51] Installing screenshot dependencies (playwright, pillow)... [05:52] Packages installed, downloading chromium browser... [06:08] Screenshot dependencies installed successfully [06:08] Starting headless browser... [06:08] Installing Vulkan stack (apt)... [06:08] Vulkan install error: [Errno 2] No such file or directory: 'sudo' [06:08] GPU detected (no Vulkan) -- using ANGLE/OpenGL rendering [06:08] [ComfyUI] [INFO] Using RAM pressure cache. [06:08] Capturing execution frames: pano.json [06:08] [ComfyUI] [INFO] Using RAM pressure cache. [06:14] Frame 1 saved (frame_000.jpg, t=0.0s) [06:14] Validating workflow... [06:14] [ComfyUI] [INFO] === /validate endpoint called === [06:14] [ComfyUI] [INFO] Received JSON with keys: ['prompt'] [06:14] Queuing workflow for execution... [06:14] [ComfyUI] [INFO] Prompt has 18 nodes [06:14] [ComfyUI] [INFO] Node 7: LoadImage [06:14] [ComfyUI] [INFO] Inputs: {"image": "pano.png"} [06:14] [ComfyUI] [INFO] Node 9: LoadSharpModel [06:14] [ComfyUI] [INFO] Inputs: {"precision": "auto"} [06:14] [ComfyUI] [INFO] Node 54: MergeGaussians [06:14] [ComfyUI] [INFO] Inputs: {"ply_folder": ["76", 0], "output_prefix": "merged", "max_depth": 0, "min_opacity": 0} [06:14] [ComfyUI] [INFO] Node 59: PanoramaWrap [06:14] [ComfyUI] [INFO] Inputs: {"image": ["7", 0]} [06:14] [ComfyUI] [INFO] Node 60: PanoramaDepthMerge [06:14] [ComfyUI] [INFO] Inputs: {"out_width": ["61", 0], "out_height": ["61", 1], "use_gpu": true, "scale_anchor": false, "chunk_size": 8, "center_weight_power": 2, "normal_edge_threshold_deg": 0, "normal_consistency_boost": false, [06:14] [ComfyUI] [INFO] Node 61: GetImageSize [06:14] [ComfyUI] [comfy-env] SubprocessWorker: /tmp/comfytest-home/.ce/.pixi/envs/sharp-nodes/bin/python [06:14] [ComfyUI] [INFO] Inputs: {"image": ["7", 0]} [06:14] [ComfyUI] [comfy-env] env_vars: {'COMFYUI_BASE': '/workspaces/Sharp-1914/ComfyUI'} [06:14] [ComfyUI] [INFO] Node 62: PreviewImage [06:14] [ComfyUI] [ComfyUI-Sharp] python: /tmp/comfytest-home/.ce/.pixi/envs/sharp-nodes/bin/python [06:14] [ComfyUI] [INFO] Inputs: {"images": ["60", 2]} [06:14] [ComfyUI] [ComfyUI-Sharp] sys_path sent to worker: ['/workspaces/Sharp-1914/ComfyUI', '/workspaces/Sharp-1914/ComfyUI/custom_nodes/ComfyUI-Sharp', '/tmp/comfytest-home/.ce/.pixi/envs/sharp-nodes/lib/python3.1/site-packages', '/workspaces/Sharp-1914/ComfyUI/custom_nodes/ComfyUI-Sharp'] [06:14] [ComfyUI] [INFO] Node 68: PreviewImage [06:14] [ComfyUI] [SubprocessWorker] is_pixi=True, python=/tmp/comfytest-home/.ce/.pixi/envs/sharp-nodes/bin/python [06:14] [ComfyUI] [INFO] Inputs: {"images": ["96", 0]} [06:14] [ComfyUI] [SubprocessWorker] launching cmd=['/tmp/comfytest-home/.pixi/bin/pixi', 'run', '--as-is', '--manifest-path', '/tmp/comfytest-home/.ce/pixi.toml', '-e', 'sharp-nodes', 'python', '/tmp/comfyui_pvenv_qlpb5zhh/persistent_worker.py', 'abstract://comfy_worker_375e4707e383']... [06:14] [ComfyUI] [INFO] Node 70: PreviewImage [06:14] [ComfyUI] [SubprocessWorker] socket_addr=abstract://comfy_worker_375e4707e383 (abstract, no filesystem path) [06:14] [ComfyUI] [INFO] Inputs: {"images": ["81", 0]} [06:14] [ComfyUI] [INFO] Node 73: SharpRayToPlanarDepth [06:14] [ComfyUI] [INFO] Inputs: {"image": ["87", 1], "intrinsics": ["87", 4], "extrinsics": ["87", 3]} [06:14] [ComfyUI] [INFO] Node 76: SharpPredictGaussiansFromMetricDepth [06:14] [ComfyUI] [INFO] Inputs: {"snap_to_external_depth": "none", "depth_convention": "planar_z", "focal_length_mm": 30, "output_prefix": "sharp_aligned", "save_background_layer": true, "edge_crop": 0, "model": ["9", 0], "image": [ [06:14] [ComfyUI] [INFO] Node 78: TransformGaussian [06:14] [ComfyUI] [INFO] Inputs: {"rotate_x": -90, "rotate_y": 0, "rotate_z": 0, "ply_path": ["54", 1]} [06:14] [ComfyUI] [INFO] Node 80: DownloadAndLoadMoGe2Model [06:14] [ComfyUI] [INFO] Inputs: {"model": "moge-2-vitl-normal.pt", "precision": "auto", "attention": "auto"} [06:14] [ComfyUI] [INFO] Node 81: MoGe2Inference [06:14] [ComfyUI] [INFO] Inputs: {"resolution_level": 9, "num_tokens": 0, "fov_x_deg": ["96", 3], "force_projection": true, "apply_mask": true, "moge2_model": ["80", 0], "images": ["96", 0]} [06:14] [ComfyUI] [INFO] Node 87: PanoramaSplitAdaptive [06:14] [ComfyUI] [INFO] Inputs: {"split_method": "icosahedron_42", "resolution": 1536, "fov_degrees": 90, "use_gpu": true, "create_masks": true, "mask_method": "closest_to", "panorama": ["59", 0], "depth_panorama": ["60", 0], "norma [06:14] [ComfyUI] [INFO] Node 89: PreviewImage [06:14] [ComfyUI] [INFO] Inputs: {"images": ["87", 8]} [06:14] [ComfyUI] [INFO] Node 95: PreviewGaussians [06:14] [ComfyUI] [INFO] Inputs: {"camera_mode": "orbit", "fov_degrees": 71, "renderer": "playcanvas", "transport_format": "ply", "ply_path": ["78", 0]} [06:14] [ComfyUI] [INFO] Node 96: PanoramaSplit [06:14] [ComfyUI] [INFO] Inputs: {"split_method": "icosahedron_42", "resolution": 952, "fov_degrees": 90, "use_gpu": true, "create_masks": false, "fname": "pano_bank", "panorama": ["59", 0]} [06:14] [ComfyUI] [INFO] Generated prompt_id: 31085ed5-c99f-4caa-adc5-5d3b2d8a0840 [06:14] [ComfyUI] [INFO] Calling execution.validate_prompt()... [06:14] [ComfyUI] [INFO] Validation result: valid=True [06:14] [ComfyUI] [INFO] === Validation PASSED === [06:14] [ComfyUI] [INFO] got prompt [06:14] [ComfyUI] [comfy-env] Running LoadSharpModel: precision: 'auto' [06:14] [ComfyUI] [VRAM] Before LoadSharpModel: 697 / 24122 MB [06:14] [ComfyUI] [SubprocessWorker] call_method: nodes.load_model.LoadSharpModel.EXECUTE_NORMALIZED [06:14] [ComfyUI] [SubprocessWorker] acquired lock, ensuring started... [06:15] [ComfyUI] [worker:ComfyUI-Sharp] ERROR: Failed to import comfy_kitchen, Error: No module named 'comfy_kitchen', fp8 and fp4 support will not be available. [06:16] [ComfyUI] [worker:ComfyUI-Sharp] WARNING: comfy_kitchen does not support stochastic FP8 rounding, please update comfy_kitchen. [06:16] [ComfyUI] [SubprocessWorker] worker started/confirmed [06:16] [ComfyUI] [SubprocessWorker] serializing kwargs to shm... [06:16] [ComfyUI] [SubprocessWorker] created 0 shm blocks [06:16] [ComfyUI] [SubprocessWorker] sending request via socket... [06:16] [ComfyUI] [SubprocessWorker] call_id=1 sending call_method [06:16] [ComfyUI] [worker:ComfyUI-Sharp] [LoadSharpModel] precision='auto' -> dtype=bf16 (device=cuda:0) [06:16] [ComfyUI] [worker:ComfyUI-Sharp] [LoadSharpModel] checkpoint not found at /workspaces/Sharp-1914/ComfyUI/models/sharp/sharp_2572gikvuh.pt — downloading from HuggingFace repo 'apple/Sharp' file 'sharp_2572gikvuh.pt' -> /workspaces/Sharp-1914/ComfyUI/models/sharp [06:16] [ComfyUI] [06:17] Frame 2 saved (frame_001.jpg, t=2.3s) [06:17] [capture-periodic] freeze=1ms shot=91ms unfreeze=7ms saved=True [06:19] Frame 3 saved (frame_002.jpg, t=4.5s) [06:19] [capture-periodic] freeze=1ms shot=121ms unfreeze=5ms saved=True [06:20] [capture-loop] iter=50 t=5.7s state={'complete': False, 'error': None, 'executedCount': 0, 'wsState': 1} eval_ms=2 ram=4.7/31.3GB (15%) swap=1.7/32.0GB (5%) cpu=9% browser_rss=1.01GB [06:22] [ComfyUI] sharp_2572gikvuh.pt: 0%| | 0.00/2.81G [00:00 dtype=bf16 [06:41] [ComfyUI] [SubprocessWorker] got response: ok [06:41] [ComfyUI] [comfy-env] SubprocessWorker: /tmp/comfytest-home/.ce/.pixi/envs/moge2-nodes/bin/python [06:41] [ComfyUI] [comfy-env] LoadSharpModel done (26.77s): result: NodeOutput [06:41] [ComfyUI] [comfy-env] env_vars: {'COMFYUI_BASE': '/workspaces/Sharp-1914/ComfyUI'} [06:41] [ComfyUI] [VRAM] After LoadSharpModel: 959 / 24122 MB [06:41] [ComfyUI] [ComfyUI-MoGe2] python: /tmp/comfytest-home/.ce/.pixi/envs/moge2-nodes/bin/python [06:41] [ComfyUI] [comfy-env] Running DownloadAndLoadMoGe2Model: model: 'moge-2-vitl-normal.pt', precision: 'auto', attention: 'auto' [06:41] [ComfyUI] [ComfyUI-MoGe2] sys_path sent to worker: ['/workspaces/Sharp-1914/ComfyUI', '/workspaces/Sharp-1914/ComfyUI/custom_nodes/ComfyUI-MoGe2', '/tmp/comfytest-home/.ce/.pixi/envs/moge2-nodes/lib/python3.1/site-packages', '/workspaces/Sharp-1914/ComfyUI/custom_nodes/ComfyUI-MoGe2'] [06:41] [ComfyUI] [VRAM] Before DownloadAndLoadMoGe2Model: 959 / 24122 MB [06:41] [ComfyUI] [SubprocessWorker] is_pixi=True, python=/tmp/comfytest-home/.ce/.pixi/envs/moge2-nodes/bin/python [06:41] [ComfyUI] [SubprocessWorker] call_method: nodes.load_model.DownloadAndLoadMoGe2Model.EXECUTE_NORMALIZED [06:41] [ComfyUI] [SubprocessWorker] launching cmd=['/tmp/comfytest-home/.pixi/bin/pixi', 'run', '--as-is', '--manifest-path', '/tmp/comfytest-home/.ce/pixi.toml', '-e', 'moge2-nodes', 'python', '/tmp/comfyui_pvenv_bmmahm2z/persistent_worker.py', 'abstract://comfy_worker_b1be181c7b7c']... [06:41] [ComfyUI] [SubprocessWorker] acquired lock, ensuring started... [06:41] [ComfyUI] [SubprocessWorker] socket_addr=abstract://comfy_worker_b1be181c7b7c (abstract, no filesystem path) [06:42] [capture-loop] iter=250 t=27.5s state={'complete': False, 'error': None, 'executedCount': 0, 'wsState': 1} eval_ms=2 ram=4.6/31.3GB (15%) swap=1.7/32.0GB (5%) cpu=12% browser_rss=0.94GB [06:42] [ComfyUI] [worker:ComfyUI-MoGe2] ERROR: Failed to import comfy_kitchen, Error: No module named 'comfy_kitchen', fp8 and fp4 support will not be available. [06:42] [ComfyUI] [worker:ComfyUI-MoGe2] WARNING: comfy_kitchen does not support stochastic FP8 rounding, please update comfy_kitchen. [06:43] [ComfyUI] [SubprocessWorker] worker started/confirmed [06:43] [ComfyUI] [SubprocessWorker] serializing kwargs to shm... [06:43] [ComfyUI] [SubprocessWorker] created 0 shm blocks [06:43] [ComfyUI] [SubprocessWorker] sending request via socket... [06:43] [ComfyUI] [SubprocessWorker] call_id=1 sending call_method [06:43] Frame 10 saved (frame_009.jpg, t=28.6s) [06:43] [capture-periodic] freeze=1ms shot=88ms unfreeze=6ms saved=True [06:43] [ComfyUI] [06:46] [ComfyUI] model.pt: 0%| | 0.00/1.32G [00:00 42 crops @ 952x952, fov=(90,90) via grid_sample (GPU); cameras 1.199s, split 5.180s, total 7.251s [07:06] [ComfyUI] [SubprocessWorker] got response: ok [07:06] [ComfyUI] [comfy-env] PanoramaSplit done (8.29s): result: NodeOutput [07:06] [ComfyUI] [VRAM] After PanoramaSplit: 2855 / 24122 MB [07:07] Frame 17 saved (frame_016.jpg, t=52.6s) [07:07] [capture-periodic] freeze=1ms shot=88ms unfreeze=6ms saved=True [07:08] [ComfyUI] [comfy-env] Running MoGe2Inference: resolution_level: 9, num_tokens: 0, fov_x_deg: 90.0, force_projection: True, apply_mask: True, moge2_model: dict, images: torch.float32 [42,952,952,3] cpu [07:08] [ComfyUI] [VRAM] Before MoGe2Inference: 2855 / 24122 MB [07:08] [ComfyUI] [SubprocessWorker] call_method: nodes.nodes_inference.MoGe2Inference.EXECUTE_NORMALIZED [07:08] [ComfyUI] [SubprocessWorker] acquired lock, ensuring started... [07:08] [ComfyUI] [ComfyUI-MoGe2] Health check: ping (timeout=5.0s)... [07:08] [ComfyUI] [ComfyUI-MoGe2] Health check: ok [07:08] [ComfyUI] [SubprocessWorker] worker started/confirmed [07:08] [ComfyUI] [comfy-env] PRE-SERIALIZE 'images' shape: torch.Size([42, 952, 952, 3]) [07:08] [ComfyUI] [SubprocessWorker] serializing kwargs to shm... [07:08] Node executed (1 total), capturing... [07:09] [ComfyUI] [SubprocessWorker] created 0 shm blocks [07:09] [ComfyUI] [SubprocessWorker] sending request via socket... [07:09] [ComfyUI] [SubprocessWorker] call_id=2 sending call_method [07:09] [ComfyUI] [worker:ComfyUI-MoGe2] [comfy-env] _from_shm got dict with keys: ['__type__', 'strategy', 'parent_pid', 'fd', 'storage_size'] [07:09] [ComfyUI] [worker:ComfyUI-MoGe2] [comfy-env] DESERIALIZE TensorRef: tensor_size=[42, 952, 952, 3] [07:09] [ComfyUI] [worker:ComfyUI-MoGe2] [comfy-env] DESERIALIZED tensor shape: torch.Size([42, 952, 952, 3]) [07:09] Frame 18 saved (frame_017.jpg, t=54.6s) [07:09] [capture-node] freeze=13ms shot=126ms unfreeze=1ms saved=True [07:09] [capture-loop] iter=500 t=55.1s state={'complete': False, 'error': None, 'executedCount': 1, 'wsState': 1} eval_ms=1 ram=7.0/31.3GB (22%) swap=1.7/32.0GB (5%) cpu=14% browser_rss=1.10GB [07:14] [capture-loop] iter=550 t=60.4s state={'complete': False, 'error': None, 'executedCount': 1, 'wsState': 1} eval_ms=1 ram=9.0/31.3GB (29%) swap=1.7/32.0GB (5%) cpu=14% browser_rss=1.08GB [07:16] [ComfyUI] [SubprocessWorker] callback 'request_vram_budget' from call_id=2 [07:16] [ComfyUI] [comfy-env] VRAM request: 0.66GB, free before eviction: 22.30GB [07:16] [ComfyUI] [comfy-env] VRAM after eviction: 22.30GB [07:16] [ComfyUI] [worker:ComfyUI-MoGe2] [VRAM] +216MB (now 216MB) reserved=224MB peak=216MB [07:17] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [07:17] [ComfyUI] [worker:ComfyUI-MoGe2] [VRAM] +455MB (now 671MB) reserved=1600MB peak=782MB [07:17] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [07:17] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [07:17] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [07:17] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [07:18] Frame 19 saved (frame_018.jpg, t=63.3s) [07:18] [capture-periodic] freeze=1ms shot=112ms unfreeze=1ms saved=True [07:18] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [07:18] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [07:18] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [07:18] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [07:18] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [07:19] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [07:19] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [07:19] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [07:19] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [07:20] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [07:20] Frame 20 saved (frame_019.jpg, t=65.5s) [07:20] [capture-periodic] freeze=1ms shot=96ms unfreeze=10ms saved=True [07:20] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [07:20] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [07:20] [capture-loop] iter=600 t=66.0s state={'complete': False, 'error': None, 'executedCount': 1, 'wsState': 1} eval_ms=1 ram=9.6/31.3GB (31%) swap=1.7/32.0GB (5%) cpu=14% browser_rss=1.09GB [07:20] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [07:20] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [07:21] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [07:21] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [07:21] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [07:21] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [07:22] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [07:22] Frame 21 saved (frame_020.jpg, t=67.8s) [07:22] [capture-periodic] freeze=1ms shot=86ms unfreeze=1ms saved=True [07:22] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [07:22] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [07:23] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [07:23] [ComfyUI] [worker:ComfyUI-MoGe2] [VRAM] +239MB (now 911MB) reserved=1632MB peak=911MB [07:23] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [07:23] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [07:23] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [07:24] [ComfyUI] [worker:ComfyUI-MoGe2] [VRAM] -240MB (now 671MB) reserved=1632MB peak=911MB [07:24] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [07:24] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [07:24] Frame 22 saved (frame_021.jpg, t=69.9s) [07:24] [capture-periodic] freeze=1ms shot=89ms unfreeze=6ms saved=True [07:24] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [07:24] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [07:25] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [07:25] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [07:25] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [07:25] [capture-loop] iter=650 t=71.4s state={'complete': False, 'error': None, 'executedCount': 1, 'wsState': 1} eval_ms=2 ram=10.6/31.3GB (34%) swap=1.7/32.0GB (5%) cpu=13% browser_rss=1.13GB [07:26] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [07:26] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [07:26] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [07:26] Frame 23 saved (frame_022.jpg, t=72.2s) [07:26] [capture-periodic] freeze=1ms shot=97ms unfreeze=7ms saved=True [07:26] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [07:27] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [07:29] Frame 24 saved (frame_023.jpg, t=74.3s) [07:29] [capture-periodic] freeze=1ms shot=93ms unfreeze=6ms saved=True [07:30] [ComfyUI] [SubprocessWorker] got response: ok [07:30] [ComfyUI] [comfy-env] Created 1 model patchers: ['MoGeWrapper_1'] [07:30] [ComfyUI] [comfy-env] MoGe2Inference done (21.24s): result: NodeOutput [07:30] [ComfyUI] [VRAM] After MoGe2Inference: 4605 / 24122 MB [07:30] [ComfyUI] [VRAM] Loaded: SubprocessModel (631 MB) [07:31] Frame 25 saved (frame_024.jpg, t=76.5s) [07:31] [capture-periodic] freeze=1ms shot=102ms unfreeze=5ms saved=True [07:31] [ComfyUI] [comfy-env] Running PanoramaDepthMerge: out_width: 4096, out_height: 2048, use_gpu: True, scale_anchor: False, chunk_size: 8, center_weight_power: 2.0, normal_edge_threshold_deg: 0.0, normal_consistency_boost: False, max_ray_distance_m: 0.0, face_points: torch.float32 [42,952,952,3] cpu, extrinsics: torch.float32 [42,4,4] cpu, intrinsics: torch.float32 [42,3,3] cpu, face_valid_masks: torch.float32 [42,952,952] cpu, face_normals: torch.float32 [42,952,952,3] cpu [07:31] [ComfyUI] [VRAM] Before PanoramaDepthMerge: 4605 / 24122 MB [07:31] [ComfyUI] [VRAM] Loaded: SubprocessModel (631 MB) [07:31] [ComfyUI] [SubprocessWorker] call_method: nodes.depth_merge.PanoramaDepthMerge.EXECUTE_NORMALIZED [07:31] [ComfyUI] [SubprocessWorker] acquired lock, ensuring started... [07:31] [ComfyUI] [ComfyUI-PanoPack] Health check: ping (timeout=5.0s)... [07:31] [ComfyUI] [ComfyUI-PanoPack] Health check: ok [07:31] [ComfyUI] [SubprocessWorker] worker started/confirmed [07:31] [ComfyUI] [comfy-env] PRE-SERIALIZE 'face_points' shape: torch.Size([42, 952, 952, 3]) [07:31] [ComfyUI] [comfy-env] PRE-SERIALIZE 'extrinsics' shape: torch.Size([42, 4, 4]) [07:31] [ComfyUI] [comfy-env] PRE-SERIALIZE 'intrinsics' shape: torch.Size([42, 3, 3]) [07:31] [ComfyUI] [comfy-env] PRE-SERIALIZE 'face_valid_masks' shape: torch.Size([42, 952, 952]) [07:31] [ComfyUI] [comfy-env] PRE-SERIALIZE 'face_normals' shape: torch.Size([42, 952, 952, 3]) [07:31] [ComfyUI] [SubprocessWorker] serializing kwargs to shm... [07:31] Node executed (2 total), capturing... [07:31] Frame 26 saved (frame_025.jpg, t=76.9s) [07:31] [capture-node] freeze=1ms shot=104ms unfreeze=1ms saved=True [07:31] [ComfyUI] [SubprocessWorker] created 0 shm blocks [07:31] [ComfyUI] [SubprocessWorker] sending request via socket... [07:31] [ComfyUI] [SubprocessWorker] call_id=3 sending call_method [07:31] [ComfyUI] [worker:ComfyUI-PanoPack] [comfy-env] _from_shm got dict with keys: ['__type__', 'strategy', 'parent_pid', 'fd', 'storage_size'] [07:31] [ComfyUI] [worker:ComfyUI-PanoPack] [comfy-env] DESERIALIZE TensorRef: tensor_size=[42, 952, 952, 3] [07:31] [ComfyUI] [worker:ComfyUI-PanoPack] [comfy-env] DESERIALIZED tensor shape: torch.Size([42, 952, 952, 3]) [07:31] [ComfyUI] [worker:ComfyUI-PanoPack] [comfy-env] _from_shm got dict with keys: ['__type__', 'strategy', 'parent_pid', 'fd', 'storage_size'] [07:31] [ComfyUI] [worker:ComfyUI-PanoPack] [comfy-env] DESERIALIZE TensorRef: tensor_size=[42, 4, 4] [07:31] [ComfyUI] [worker:ComfyUI-PanoPack] [comfy-env] DESERIALIZED tensor shape: torch.Size([42, 4, 4]) [07:31] [ComfyUI] [worker:ComfyUI-PanoPack] [comfy-env] _from_shm got dict with keys: ['__type__', 'strategy', 'parent_pid', 'fd', 'storage_size'] [07:31] [ComfyUI] [worker:ComfyUI-PanoPack] [comfy-env] DESERIALIZE TensorRef: tensor_size=[42, 3, 3] [07:31] [ComfyUI] [worker:ComfyUI-PanoPack] [comfy-env] DESERIALIZED tensor shape: torch.Size([42, 3, 3]) [07:31] [ComfyUI] [worker:ComfyUI-PanoPack] [comfy-env] _from_shm got dict with keys: ['__type__', 'strategy', 'parent_pid', 'fd', 'storage_size'] [07:31] [ComfyUI] [worker:ComfyUI-PanoPack] [comfy-env] DESERIALIZE TensorRef: tensor_size=[42, 952, 952] [07:31] [ComfyUI] [worker:ComfyUI-PanoPack] [comfy-env] DESERIALIZED tensor shape: torch.Size([42, 952, 952]) [07:31] [ComfyUI] [worker:ComfyUI-PanoPack] [comfy-env] _from_shm got dict with keys: ['__type__', 'strategy', 'parent_pid', 'fd', 'storage_size'] [07:31] [ComfyUI] [worker:ComfyUI-PanoPack] [comfy-env] DESERIALIZE TensorRef: tensor_size=[42, 952, 952, 3] [07:31] [capture-loop] iter=700 t=77.3s state={'complete': False, 'error': None, 'executedCount': 2, 'wsState': 1} eval_ms=2 ram=13.6/31.3GB (44%) swap=1.7/32.0GB (5%) cpu=15% browser_rss=1.19GB [07:31] [ComfyUI] [worker:ComfyUI-PanoPack] [comfy-env] DESERIALIZED tensor shape: torch.Size([42, 952, 952, 3]) [07:33] Frame 27 saved (frame_026.jpg, t=79.1s) [07:33] [capture-periodic] freeze=1ms shot=100ms unfreeze=5ms saved=True [07:37] [capture-loop] iter=750 t=82.7s state={'complete': False, 'error': None, 'executedCount': 2, 'wsState': 1} eval_ms=2 ram=15.4/31.3GB (49%) swap=1.7/32.0GB (5%) cpu=14% browser_rss=1.18GB [07:42] [capture-loop] iter=800 t=88.2s state={'complete': False, 'error': None, 'executedCount': 2, 'wsState': 1} eval_ms=1 ram=15.5/31.3GB (50%) swap=1.7/32.0GB (5%) cpu=20% browser_rss=1.10GB [07:48] [capture-loop] iter=850 t=93.6s state={'complete': False, 'error': None, 'executedCount': 2, 'wsState': 1} eval_ms=2 ram=15.3/31.3GB (49%) swap=1.7/32.0GB (5%) cpu=21% browser_rss=1.10GB [07:53] [capture-loop] iter=900 t=99.2s state={'complete': False, 'error': None, 'executedCount': 2, 'wsState': 1} eval_ms=3 ram=15.5/31.3GB (50%) swap=1.7/32.0GB (5%) cpu=20% browser_rss=1.10GB [07:54] [ComfyUI] [worker:ComfyUI-PanoPack] [PanoramaDepthMerge] normals merged: 2048x4096 [07:54] [ComfyUI] [worker:ComfyUI-PanoPack] [PanoramaDepthMerge] merging 42 faces @ 952x952 -> equirect 2048x4096 (use_gpu=True); input = points (||.|| -> ray distance) [07:54] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=3 [07:54] [ComfyUI] [worker:ComfyUI-PanoPack] [PanoramaDepthMerge] top-level peak VRAM estimate: 3.46 GB (4096x2048, N=42, chunk_size=8) [07:54] [ComfyUI] [SubprocessWorker] callback 'request_vram_budget' from call_id=3 [07:54] [ComfyUI] [comfy-env] VRAM request: 3.46GB, free before eviction: 20.47GB [07:54] [ComfyUI] [comfy-env] VRAM after eviction: 20.47GB [07:54] [ComfyUI] [worker:ComfyUI-PanoPack] [PanoramaDepthMerge] -> requested 3.46 GB eviction via comfy_worker.call_parent [07:55] [ComfyUI] [worker:ComfyUI-PanoPack] [VRAM] +436MB (now 436MB) reserved=608MB peak=822MB [07:55] [ComfyUI] /workspaces/Sharp-1914/ComfyUI/custom_nodes/ComfyUI-PanoPack/nodes/_vendor/worldgen/src/panorama_utils.py:349: UserWarning: Sparse CSR tensor support is in beta state. If you miss a functionality in the sparse tensor support, please submit a feature request to https://github.com/pytorch/pytorch/issues. (Triggered internally at /pytorch/aten/src/ATen/SparseCsrTensorImpl.cpp:49.) [07:55] [ComfyUI] A_gpu = torch.sparse_csr_tensor( [07:55] [ComfyUI] [worker:ComfyUI-PanoPack] [merge_panorama_depth_gpu 256x128 ] done 0.52s (warp 0.29, build 0.01, LSMR 0.22); N=42 K=8 chunks=6; depth min/med/max = 0.384/0.945/3.091; valid 32768/32768 [07:55] [ComfyUI] [worker:ComfyUI-PanoPack] [merge_panorama_depth_gpu 512x256 ] done 0.71s (warp 0.08, build 0.02, LSMR 0.07); N=42 K=8 chunks=6; depth min/med/max = 0.384/0.945/3.094; valid 131072/131072 [07:55] Frame 28 saved (frame_027.jpg, t=101.0s) [07:55] [capture-periodic] freeze=1ms shot=101ms unfreeze=6ms saved=True [07:56] [ComfyUI] [worker:ComfyUI-PanoPack] [merge_panorama_depth_gpu 1024x512 ] done 1.04s (warp 0.10, build 0.06, LSMR 0.14); N=42 K=8 chunks=6; depth min/med/max = 0.384/0.945/3.094; valid 524288/524288 [07:56] [ComfyUI] [worker:ComfyUI-PanoPack] [VRAM] -259MB (now 177MB) reserved=224MB peak=822MB [07:56] [ComfyUI] [worker:ComfyUI-PanoPack] [07:56] [ComfyUI] === WATCHDOG TICK 1 === [07:56] [ComfyUI] [worker:ComfyUI-PanoPack] Thread 0x0000757e127ff640 (most recent call first): [07:56] [ComfyUI] File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/socket.py", line 293 in accept [07:56] [ComfyUI] File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/multiprocessing/connection.py", line 609 in accept [07:56] [ComfyUI] File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/multiprocessing/connection.py", line 463 in accept [07:56] [ComfyUI] File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/multiprocessing/resource_sharer.py", line 138 in _serve [07:56] [ComfyUI] File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/threading.py", line 953 in run [07:56] [ComfyUI] File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/threading.py", line 1016 in _bootstrap_inner [07:56] [ComfyUI] File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/threading.py", line 973 in _bootstrap [07:56] [ComfyUI] [07:56] [ComfyUI] Thread 0x0000757ff35a0640 (most recent call first): [07:56] [ComfyUI] File "/tmp/comfyui_pvenv_0kgugsfd/persistent_worker.py", line 106 in _vram_poller [07:56] [ComfyUI] File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/threading.py", line 953 in run [07:56] [ComfyUI] File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/threading.py", line 1016 in _bootstrap_inner [07:56] [ComfyUI] File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/threading.py", line 973 in _bootstrap [07:56] [ComfyUI] [07:56] [ComfyUI] Current thread 0x0000757ff3da1640 (most recent call first): [07:56] [ComfyUI] File "/tmp/comfyui_pvenv_0kgugsfd/persistent_worker.py", line 52 in _watchdog [07:56] [ComfyUI] File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/threading.py", line 953 in run [07:56] [ComfyUI] File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/threading.py", line 1016 in _bootstrap_inner [07:56] [ComfyUI] File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/threading.py", line 973 in _bootstrap [07:56] [ComfyUI] [07:56] [ComfyUI] Thread 0x0000757ff4377740 (most recent call first): [07:56] [ComfyUI] File "/workspaces/Sharp-1914/ComfyUI/custom_nodes/ComfyUI-PanoPack/nodes/_vendor/worldgen/src/panorama_utils.py", line 834 in merge_panorama_depth_gpu [07:56] [ComfyUI] File "/workspaces/Sharp-1914/ComfyUI/custom_nodes/ComfyUI-PanoPack/nodes/_vendor/worldgen/src/panorama_utils.py", line 550 in merge_panorama_depth_gpu [07:56] [ComfyUI] File "/workspaces/Sharp-1914/ComfyUI/custom_nodes/ComfyUI-PanoPack/nodes/depth_merge.py", line 953 in execute [07:56] [ComfyUI] File "/workspaces/Sharp-1914/ComfyUI/comfy_api/latest/_io.py", line 1892 in EXECUTE_NORMALIZED [07:56] [ComfyUI] File "/tmp/comfyui_pvenv_0kgugsfd/persistent_worker.py", line 1762 in main [07:56] [ComfyUI] File "/tmp/comfyui_pvenv_0kgugsfd/persistent_worker.py", line 1807 in [07:56] [ComfyUI] [07:56] [ComfyUI] [worker:ComfyUI-PanoPack] === END === [07:56] [ComfyUI] [07:57] [ComfyUI] [worker:ComfyUI-PanoPack] [merge_panorama_depth_gpu 2048x1024] done 2.05s (warp 0.20, build 0.25, LSMR 0.51); N=42 K=8 chunks=6; depth min/med/max = 0.384/0.945/3.095; valid 2097152/2097152 [07:57] [ComfyUI] [worker:ComfyUI-PanoPack] [VRAM] +682MB (now 859MB) reserved=4960MB peak=859MB [07:58] [ComfyUI] [worker:ComfyUI-PanoPack] [VRAM] -562MB (now 298MB) reserved=416MB peak=859MB [07:59] [capture-loop] iter=950 t=104.6s state={'complete': False, 'error': None, 'executedCount': 2, 'wsState': 1} eval_ms=1 ram=16.7/31.3GB (53%) swap=1.7/32.0GB (5%) cpu=13% browser_rss=1.11GB [08:01] [ComfyUI] [worker:ComfyUI-PanoPack] [VRAM] +671MB (now 970MB) reserved=1088MB peak=970MB [08:02] [ComfyUI] [worker:ComfyUI-PanoPack] [VRAM] +319MB (now 1290MB) reserved=1376MB peak=1290MB [08:03] [ComfyUI] [worker:ComfyUI-PanoPack] [merge_panorama_depth_gpu 4096x2048] done 8.27s (warp 0.58, build 3.18, LSMR 2.26); N=42 K=8 chunks=6; depth min/med/max = 0.384/0.945/3.094; valid 8388608/8388608 [08:03] [ComfyUI] [worker:ComfyUI-PanoPack] [VRAM] -1283MB (now 8MB) reserved=2176MB peak=1898MB [08:03] [ComfyUI] [worker:ComfyUI-PanoPack] [PanoramaDepthMerge] merged depth: shape (2048, 4096), min=0.384, median=0.945, max=3.094; valid pixels: 8388608 / 8388608 [08:04] [capture-loop] iter=1000 t=110.1s state={'complete': False, 'error': None, 'executedCount': 2, 'wsState': 1} eval_ms=2 ram=16.1/31.3GB (51%) swap=1.7/32.0GB (5%) cpu=12% browser_rss=1.11GB [08:10] [capture-loop] iter=1050 t=115.5s state={'complete': False, 'error': None, 'executedCount': 2, 'wsState': 1} eval_ms=2 ram=16.2/31.3GB (52%) swap=1.7/32.0GB (5%) cpu=17% browser_rss=1.11GB [08:15] [capture-loop] iter=1100 t=121.0s state={'complete': False, 'error': None, 'executedCount': 2, 'wsState': 1} eval_ms=2 ram=16.1/31.3GB (52%) swap=1.7/32.0GB (5%) cpu=17% browser_rss=1.11GB [08:18] [ComfyUI] [worker:ComfyUI-PanoPack] [PanoramaDepthMerge] debug_image: H=2048+102, mult_disagreement (log-d, sphere-uniform): mean=11.22%, p99=24.69%, max=28.22%; overlap_pixels=8388608 [08:18] [ComfyUI] [worker:ComfyUI-PanoPack] [headless_gl] prefix=/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes [08:18] [ComfyUI] [worker:ComfyUI-PanoPack] [headless_gl] OSMesa: ['libOSMesa.so', 'libOSMesa.so.8', 'libOSMesa.so.8.0.0'] [08:18] [ComfyUI] [worker:ComfyUI-PanoPack] [headless_gl] env: VTK_DEFAULT_OPENGL_WINDOW=vtkOSOpenGLRenderWindow PYVISTA_OFF_SCREEN=true [08:18] [ComfyUI] [worker:ComfyUI-PanoPack] [PanoramaDepthMerge] xyz_views: cloud bbox center=(-0.79, -0.31, +0.03) m, size=(4.27 x 3.57 x 1.96) m, N=80000 (subsampled) [08:20] [ComfyUI] [worker:ComfyUI-PanoPack] [PanoramaDepthMerge] points: merged equirect point map (1, 2048, 4096, 3) [08:20] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=3 [08:20] [capture-loop] iter=1150 t=126.4s state={'complete': False, 'error': None, 'executedCount': 2, 'wsState': 1} eval_ms=2 ram=15.3/31.3GB (49%) swap=1.7/32.0GB (5%) cpu=14% browser_rss=1.11GB [08:21] [ComfyUI] [SubprocessWorker] got response: ok [08:21] [ComfyUI] [comfy-env] PanoramaDepthMerge done (49.81s): result: NodeOutput [08:21] [ComfyUI] [VRAM] After PanoramaDepthMerge: 3335 / 24122 MB [08:21] [ComfyUI] [VRAM] Loaded: SubprocessModel (631 MB) [08:21] [ComfyUI] [comfy-env] Running PanoramaSplitAdaptive: split_method: 'icosahedron_42', resolution: 1536, fov_degrees: 90.0, use_gpu: True, create_masks: True, mask_method: 'closest_to', panorama: dict, depth_panorama: torch.float32 [1,2048,4096,3] cpu, normals_panorama: torch.float32 [1,2048,4096,3] cpu [08:21] [ComfyUI] [VRAM] Before PanoramaSplitAdaptive: 3335 / 24122 MB [08:21] [ComfyUI] [VRAM] Loaded: SubprocessModel (631 MB) [08:21] [ComfyUI] [SubprocessWorker] call_method: nodes.panorama_split_adaptive.PanoramaSplitAdaptive.EXECUTE_NORMALIZED [08:21] [ComfyUI] [SubprocessWorker] acquired lock, ensuring started... [08:21] [ComfyUI] [ComfyUI-PanoPack] Health check: ping (timeout=5.0s)... [08:21] [ComfyUI] [ComfyUI-PanoPack] Health check: ok [08:21] [ComfyUI] [SubprocessWorker] worker started/confirmed [08:21] [ComfyUI] [comfy-env] PRE-SERIALIZE 'depth_panorama' shape: torch.Size([1, 2048, 4096, 3]) [08:21] [ComfyUI] [comfy-env] PRE-SERIALIZE 'normals_panorama' shape: torch.Size([1, 2048, 4096, 3]) [08:21] [ComfyUI] [SubprocessWorker] serializing kwargs to shm... [08:21] [ComfyUI] [SubprocessWorker] created 0 shm blocks [08:21] Node executed (3 total), capturing... [08:21] [ComfyUI] [SubprocessWorker] sending request via socket... [08:21] [ComfyUI] [SubprocessWorker] call_id=4 sending call_method [08:21] [ComfyUI] [worker:ComfyUI-PanoPack] [comfy-env] _from_shm got dict with keys: ['__type__', 'strategy', 'parent_pid', 'fd', 'storage_size'] [08:21] [ComfyUI] [worker:ComfyUI-PanoPack] [comfy-env] DESERIALIZE TensorRef: tensor_size=[1, 2048, 4096, 3] [08:21] [ComfyUI] [worker:ComfyUI-PanoPack] [comfy-env] DESERIALIZED tensor shape: torch.Size([1, 2048, 4096, 3]) [08:21] [ComfyUI] [worker:ComfyUI-PanoPack] [comfy-env] _from_shm got dict with keys: ['__type__', 'strategy', 'parent_pid', 'fd', 'storage_size'] [08:21] [ComfyUI] [worker:ComfyUI-PanoPack] [comfy-env] DESERIALIZE TensorRef: tensor_size=[1, 2048, 4096, 3] [08:21] [ComfyUI] [worker:ComfyUI-PanoPack] [comfy-env] DESERIALIZED tensor shape: torch.Size([1, 2048, 4096, 3]) [08:21] [ComfyUI] [worker:ComfyUI-PanoPack] [comfy-env] _from_shm got dict with keys: ['__type__', 'strategy', 'parent_pid', 'fd', 'storage_size'] [08:21] [ComfyUI] [worker:ComfyUI-PanoPack] [comfy-env] DESERIALIZE TensorRef: tensor_size=[1, 2048, 4096, 3] [08:21] [ComfyUI] [worker:ComfyUI-PanoPack] [comfy-env] DESERIALIZED tensor shape: torch.Size([1, 2048, 4096, 3]) [08:21] Frame 29 saved (frame_028.jpg, t=127.0s) [08:21] [capture-node] freeze=2ms shot=105ms unfreeze=5ms saved=True [08:21] [ComfyUI] [worker:ComfyUI-PanoPack] [PanoramaSplitAdaptive] computing area weights from depth (2048x4096) + normals... [08:21] [ComfyUI] [worker:ComfyUI-PanoPack] [PanoramaSplitAdaptive] weight range: [1.06, 131], mean=11.7 [08:21] [ComfyUI] [worker:ComfyUI-PanoPack] [PanoramaSplitAdaptive] adaptive relax: icosahedron_42 (42 cameras), weighted Lloyd... [08:26] [capture-loop] iter=1200 t=132.1s state={'complete': False, 'error': None, 'executedCount': 3, 'wsState': 1} eval_ms=2 ram=16.2/31.3GB (52%) swap=1.7/32.0GB (5%) cpu=12% browser_rss=1.12GB [08:31] [capture-loop] iter=1250 t=137.4s state={'complete': False, 'error': None, 'executedCount': 3, 'wsState': 1} eval_ms=2 ram=16.1/31.3GB (52%) swap=1.7/32.0GB (5%) cpu=11% browser_rss=1.12GB [08:37] [capture-loop] iter=1300 t=142.9s state={'complete': False, 'error': None, 'executedCount': 3, 'wsState': 1} eval_ms=2 ram=16.3/31.3GB (52%) swap=1.7/32.0GB (5%) cpu=12% browser_rss=1.12GB [08:42] [capture-loop] iter=1350 t=148.3s state={'complete': False, 'error': None, 'executedCount': 3, 'wsState': 1} eval_ms=1 ram=16.5/31.3GB (53%) swap=1.7/32.0GB (5%) cpu=11% browser_rss=1.12GB [08:47] [ComfyUI] [worker:ComfyUI-PanoPack] [PanoramaSplitAdaptive] Lloyd ran 24 iters (max move 1.42e-02) [08:47] [ComfyUI] [worker:ComfyUI-PanoPack] [PanoramaSplitAdaptive] Lloyd relaxation done: 42 camera directions in 25.326s [08:47] [ComfyUI] [worker:ComfyUI-PanoPack] [PanoramaSplitAdaptive] rasterizing 42 faces @ 1536x1536... [08:48] [capture-loop] iter=1400 t=153.8s state={'complete': False, 'error': None, 'executedCount': 3, 'wsState': 1} eval_ms=2 ram=16.0/31.3GB (51%) swap=1.7/32.0GB (5%) cpu=13% browser_rss=1.12GB [08:53] [capture-loop] iter=1450 t=159.2s state={'complete': False, 'error': None, 'executedCount': 3, 'wsState': 1} eval_ms=2 ram=16.3/31.3GB (52%) swap=1.7/32.0GB (5%) cpu=11% browser_rss=1.12GB [08:55] [ComfyUI] [worker:ComfyUI-PanoPack] [VRAM] +756MB (now 764MB) reserved=800MB peak=1898MB [08:56] [ComfyUI] [worker:ComfyUI-PanoPack] [VRAM] +1230MB (now 1994MB) reserved=3168MB peak=1994MB [08:56] [ComfyUI] [worker:ComfyUI-PanoPack] [08:56] [ComfyUI] === WATCHDOG TICK 2 === [08:56] [ComfyUI] [worker:ComfyUI-PanoPack] Thread 0x0000757e127ff640 (most recent call first): [08:56] [ComfyUI] File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/socket.py", line 293 in accept [08:56] [ComfyUI] File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/multiprocessing/connection.py", line 609 in accept [08:56] [ComfyUI] File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/multiprocessing/connection.py", line 463 in accept [08:56] [ComfyUI] File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/multiprocessing/resource_sharer.py", line 138 in _serve [08:56] [ComfyUI] File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/threading.py", line 953 in run [08:56] [ComfyUI] File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/threading.py", line 1016 in _bootstrap_inner [08:56] [ComfyUI] File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/threading.py", line 973 in _bootstrap [08:56] [ComfyUI] [08:56] [ComfyUI] Thread 0x0000757ff35a0640 (most recent call first): [08:56] [ComfyUI] File "/tmp/comfyui_pvenv_0kgugsfd/persistent_worker.py", line 106 in _vram_poller [08:56] [ComfyUI] File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/threading.py", line 953 in run [08:56] [ComfyUI] File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/threading.py", line 1016 in _bootstrap_inner [08:56] [ComfyUI] File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/threading.py", line 973 in _bootstrap [08:56] [ComfyUI] [08:56] [ComfyUI] Current thread 0x0000757ff3da1640 (most recent call first): [08:56] [ComfyUI] File "/tmp/comfyui_pvenv_0kgugsfd/persistent_worker.py", line 52 in _watchdog [08:56] [ComfyUI] File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/threading.py", line 953 in run [08:56] [ComfyUI] File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/threading.py", line 1016 in _bootstrap_inner [08:56] [ComfyUI] File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/threading.py", line 973 in _bootstrap [08:56] [ComfyUI] [08:56] [ComfyUI] Thread 0x0000757ff4377740 (most recent call first): [08:56] [ComfyUI] File "/workspaces/Sharp-1914/ComfyUI/custom_nodes/ComfyUI-PanoPack/nodes/panorama_split_adaptive.py", line 439 in execute [08:56] [ComfyUI] File "/workspaces/Sharp-1914/ComfyUI/comfy_api/latest/_io.py", line 1892 in EXECUTE_NORMALIZED [08:56] [ComfyUI] File "/tmp/comfyui_pvenv_0kgugsfd/persistent_worker.py", line 1762 in main [08:56] [ComfyUI] File "/tmp/comfyui_pvenv_0kgugsfd/persistent_worker.py", line 1807 in [08:56] [ComfyUI] [08:56] [ComfyUI] [worker:ComfyUI-PanoPack] === END === [08:56] [ComfyUI] [08:57] [ComfyUI] [worker:ComfyUI-PanoPack] [VRAM] -1986MB (now 8MB) reserved=3168MB peak=1994MB [08:57] [ComfyUI] [worker:ComfyUI-PanoPack] [PanoramaSplitAdaptive] rasterized RGB in 10.315s (GPU) [08:57] [ComfyUI] [worker:ComfyUI-PanoPack] [PanoramaSplitAdaptive] rasterizing depth crops... [08:59] [capture-loop] iter=1500 t=164.6s state={'complete': False, 'error': None, 'executedCount': 3, 'wsState': 1} eval_ms=1 ram=17.8/31.3GB (57%) swap=1.7/32.0GB (5%) cpu=12% browser_rss=1.12GB [09:04] [capture-loop] iter=1550 t=170.0s state={'complete': False, 'error': None, 'executedCount': 3, 'wsState': 1} eval_ms=1 ram=18.3/31.3GB (59%) swap=1.7/32.0GB (5%) cpu=11% browser_rss=1.12GB [09:07] [ComfyUI] [worker:ComfyUI-PanoPack] [VRAM] +756MB (now 764MB) reserved=3168MB peak=1994MB [09:08] [ComfyUI] [worker:ComfyUI-PanoPack] [VRAM] +1230MB (now 1994MB) reserved=3264MB peak=1994MB [09:09] [ComfyUI] [worker:ComfyUI-PanoPack] [VRAM] -1986MB (now 8MB) reserved=3264MB peak=1994MB [09:10] [capture-loop] iter=1600 t=175.5s state={'complete': False, 'error': None, 'executedCount': 3, 'wsState': 1} eval_ms=2 ram=20.8/31.3GB (67%) swap=1.7/32.0GB (5%) cpu=12% browser_rss=1.13GB [09:10] [ComfyUI] [worker:ComfyUI-PanoPack] [PanoramaSplitAdaptive] rasterizing normals crops... [09:15] [capture-loop] iter=1650 t=180.8s state={'complete': False, 'error': None, 'executedCount': 3, 'wsState': 1} eval_ms=2 ram=20.4/31.3GB (65%) swap=1.7/32.0GB (5%) cpu=11% browser_rss=1.13GB [09:19] [ComfyUI] [worker:ComfyUI-PanoPack] [VRAM] +756MB (now 764MB) reserved=3264MB peak=1994MB [09:20] [capture-loop] iter=1700 t=186.3s state={'complete': False, 'error': None, 'executedCount': 3, 'wsState': 1} eval_ms=1 ram=23.0/31.3GB (74%) swap=1.7/32.0GB (5%) cpu=12% browser_rss=1.13GB [09:20] [ComfyUI] [worker:ComfyUI-PanoPack] [VRAM] +1230MB (now 1994MB) reserved=3264MB peak=1994MB [09:21] [ComfyUI] [worker:ComfyUI-PanoPack] [VRAM] -1986MB (now 8MB) reserved=3264MB peak=1994MB [09:22] [ComfyUI] [worker:ComfyUI-PanoPack] [PanoramaSplitAdaptive] generating closest_to masks... [09:26] [capture-loop] iter=1750 t=191.6s state={'complete': False, 'error': None, 'executedCount': 3, 'wsState': 1} eval_ms=1 ram=23.0/31.3GB (74%) swap=1.7/32.0GB (5%) cpu=11% browser_rss=1.13GB [09:31] [capture-loop] iter=1800 t=197.1s state={'complete': False, 'error': None, 'executedCount': 3, 'wsState': 1} eval_ms=2 ram=23.1/31.3GB (74%) swap=1.7/32.0GB (5%) cpu=11% browser_rss=1.13GB [09:36] [capture-loop] iter=1850 t=202.4s state={'complete': False, 'error': None, 'executedCount': 3, 'wsState': 1} eval_ms=1 ram=22.9/31.3GB (73%) swap=1.7/32.0GB (5%) cpu=11% browser_rss=1.13GB [09:42] [capture-loop] iter=1900 t=207.9s state={'complete': False, 'error': None, 'executedCount': 3, 'wsState': 1} eval_ms=1 ram=23.2/31.3GB (74%) swap=1.7/32.0GB (5%) cpu=11% browser_rss=1.13GB [09:47] [ComfyUI] [worker:ComfyUI-PanoPack] [PanoramaSplitAdaptive] generating debug_masks equirect image... [09:47] [capture-loop] iter=1950 t=213.2s state={'complete': False, 'error': None, 'executedCount': 3, 'wsState': 1} eval_ms=1 ram=22.9/31.3GB (73%) swap=1.7/32.0GB (5%) cpu=12% browser_rss=1.12GB [09:53] [capture-loop] iter=2000 t=218.7s state={'complete': False, 'error': None, 'executedCount': 3, 'wsState': 1} eval_ms=2 ram=24.5/31.3GB (78%) swap=1.8/32.0GB (6%) cpu=13% browser_rss=1.12GB [09:55] [ComfyUI] [worker:ComfyUI-PanoPack] [PanoramaSplitAdaptive] pixels/area: lowest 0.243 -> 0.336 (+38% on the worst-sampled cell), highest 6.44 -> 2.85; spread max/min 26.5x -> 8.5x [09:55] [ComfyUI] [worker:ComfyUI-PanoPack] [PanoramaSplitAdaptive] done: 42 adaptive faces, 94.124s total [09:56] [ComfyUI] [worker:ComfyUI-PanoPack] [09:56] [ComfyUI] === WATCHDOG TICK 3 === [09:56] [ComfyUI] [worker:ComfyUI-PanoPack] Thread 0x0000757e127ff640 (most recent call first): [09:56] [ComfyUI] File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/socket.py", line 293 in accept [09:56] [ComfyUI] File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/multiprocessing/connection.py", line 609 in accept [09:56] [ComfyUI] File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/multiprocessing/connection.py", line 463 in accept [09:56] [ComfyUI] File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/multiprocessing/resource_sharer.py", line 138 in _serve [09:56] [ComfyUI] File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/threading.py", line 953 in run [09:56] [ComfyUI] File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/threading.py", line 1016 in _bootstrap_inner [09:56] [ComfyUI] File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/threading.py", line 973 in _bootstrap [09:56] [ComfyUI] [09:56] [ComfyUI] Thread 0x0000757ff35a0640 (most recent call first): [09:56] [ComfyUI] File "/tmp/comfyui_pvenv_0kgugsfd/persistent_worker.py", line 106 in _vram_poller [09:56] [ComfyUI] File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/threading.py", line 953 in run [09:56] [ComfyUI] File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/threading.py", line 1016 in _bootstrap_inner [09:56] [ComfyUI] File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/threading.py", line 973 in _bootstrap [09:56] [ComfyUI] [09:56] [ComfyUI] Current thread 0x0000757ff3da1640 (most recent call first): [09:56] [ComfyUI] File "/tmp/comfyui_pvenv_0kgugsfd/persistent_worker.py", line 52 in _watchdog [09:56] [ComfyUI] File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/threading.py", line 953 in run [09:56] [ComfyUI] File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/threading.py", line 1016 in _bootstrap_inner [09:56] [ComfyUI] File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/threading.py", line 973 in _bootstrap [09:56] [ComfyUI] [09:56] [ComfyUI] Thread 0x0000757ff4377740 (most recent call first): [09:56] [ComfyUI] File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/site-packages/torch/storage.py", line 524 in _share_fd_cpu_ [09:56] [ComfyUI] File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/site-packages/torch/storage.py", line 449 in wrapper [09:56] [ComfyUI] File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/site-packages/torch/storage.py", line 400 in share_memory_ [09:56] [ComfyUI] File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/site-packages/torch/storage.py", line 520 in share_memory_ [09:56] [ComfyUI] File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/site-packages/torch/storage.py", line 449 in wrapper [09:56] [ComfyUI] File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/site-packages/torch/storage.py", line 1193 in _share_memory_ [09:56] [ComfyUI] File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/site-packages/torch/_tensor.py", line 853 in share_memory_ [09:56] [ComfyUI] File "/tmp/comfyui_pvenv_0kgugsfd/persistent_worker.py", line 563 in _serialize_tensor_native [09:56] [ComfyUI] File "/tmp/comfyui_pvenv_0kgugsfd/persistent_worker.py", line 628 in _to_shm [09:56] [ComfyUI] File "/tmp/comfyui_pvenv_0kgugsfd/persistent_worker.py", line 705 in [09:56] [ComfyUI] File "/tmp/comfyui_pvenv_0kgugsfd/persistent_worker.py", line 705 in _to_shm [09:56] [ComfyUI] File "/tmp/comfyui_pvenv_0kgugsfd/persistent_worker.py", line 692 in _to_shm [09:56] [ComfyUI] File "/tmp/comfyui_pvenv_0kgugsfd/persistent_worker.py", line 1772 in main [09:56] [ComfyUI] File "/tmp/comfyui_pvenv_0kgugsfd/persistent_worker.py", line 1807 in [09:56] [ComfyUI] [09:56] [ComfyUI] [worker:ComfyUI-PanoPack] === END === [09:56] [ComfyUI] [09:57] [ComfyUI] [SubprocessWorker] got response: error [09:57] [ComfyUI] [ERROR] !!! Exception during processing !!! unable to allocate shared memory(shm) for file : Success (0) [09:57] [ComfyUI] [09:57] [ComfyUI] Worker traceback: [09:57] [ComfyUI] Traceback (most recent call last): [09:57] [ComfyUI] File "/tmp/comfyui_pvenv_0kgugsfd/persistent_worker.py", line 1772, in main [09:57] [ComfyUI] result_meta = _to_shm(result, shm_registry) [09:57] [ComfyUI] File "/tmp/comfyui_pvenv_0kgugsfd/persistent_worker.py", line 692, in _to_shm [09:57] [ComfyUI] "args": _to_shm(list(obj.args), registry, visited), [09:57] [ComfyUI] File "/tmp/comfyui_pvenv_0kgugsfd/persistent_worker.py", line 705, in _to_shm [09:57] [ComfyUI] result = [_to_shm(v, registry, visited) for v in obj] [09:57] [ComfyUI] File "/tmp/comfyui_pvenv_0kgugsfd/persistent_worker.py", line 705, in [09:57] [ComfyUI] result = [_to_shm(v, registry, visited) for v in obj] [09:57] [ComfyUI] File "/tmp/comfyui_pvenv_0kgugsfd/persistent_worker.py", line 628, in _to_shm [09:57] [ComfyUI] result = _serialize_tensor_native(tensor, registry) [09:57] [ComfyUI] File "/tmp/comfyui_pvenv_0kgugsfd/persistent_worker.py", line 563, in _serialize_tensor_native [09:57] [ComfyUI] t.share_memory_() [09:57] [ComfyUI] File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/site-packages/torch/_tensor.py", line 853, in share_memory_ [09:57] [ComfyUI] self._typed_storage()._share_memory_() [09:57] [ComfyUI] File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/site-packages/torch/storage.py", line 1193, in _share_memory_ [09:57] [ComfyUI] self._untyped_storage.share_memory_() [09:57] [ComfyUI] File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/site-packages/torch/storage.py", line 449, in wrapper [09:57] [ComfyUI] return fn(self, *args, **kwargs) [09:57] [ComfyUI] File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/site-packages/torch/storage.py", line 520, in share_memory_ [09:57] [ComfyUI] return super().share_memory_(*args, **kwargs) [09:57] [ComfyUI] File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/site-packages/torch/storage.py", line 400, in share_memory_ [09:57] [ComfyUI] self._share_fd_cpu_() [09:57] [ComfyUI] File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/site-packages/torch/storage.py", line 449, in wrapper [09:57] [ComfyUI] return fn(self, *args, **kwargs) [09:57] [ComfyUI] File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/site-packages/torch/storage.py", line 524, in _share_fd_cpu_ [09:57] [ComfyUI] return super()._share_fd_cpu_(*args, **kwargs) [09:57] [ComfyUI] RuntimeError: unable to allocate shared memory(shm) for file : Success (0) [09:57] [ComfyUI] [09:57] [ComfyUI] [ERROR] Traceback (most recent call last): [09:57] [ComfyUI] File "/workspaces/Sharp-1914/ComfyUI/execution.py", line 536, in execute [09:57] [ComfyUI] output_data, output_ui, has_subgraph, has_pending_tasks = await get_output_data(prompt_id, unique_id, obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data) [09:57] [ComfyUI] File "/workspaces/Sharp-1914/ComfyUI/execution.py", line 336, in get_output_data [09:57] [ComfyUI] return_values = await _async_map_node_over_list(prompt_id, unique_id, obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data) [09:57] [ComfyUI] File "/workspaces/Sharp-1914/ComfyUI/execution.py", line 310, in _async_map_node_over_list [09:57] [ComfyUI] await process_inputs(input_dict, i) [09:57] [ComfyUI] File "/workspaces/Sharp-1914/ComfyUI/execution.py", line 298, in process_inputs [09:57] [ComfyUI] result = f(**inputs) [09:57] [ComfyUI] File "/workspaces/Sharp-1914/.venv/lib/python3.10/site-packages/comfy_env/isolation/metadata.py", line 556, in proxy [09:57] [ComfyUI] result = worker.call_method( [09:57] [ComfyUI] File "/workspaces/Sharp-1914/.venv/lib/python3.10/site-packages/comfy_env/isolation/workers/subprocess.py", line 674, in call_method [09:57] [ComfyUI] raise WorkerError( [09:57] [ComfyUI] comfy_env.isolation.workers.base.WorkerError: unable to allocate shared memory(shm) for file : Success (0) [09:57] [ComfyUI] [09:57] [ComfyUI] Worker traceback: [09:57] [ComfyUI] Traceback (most recent call last): [09:57] [ComfyUI] File "/tmp/comfyui_pvenv_0kgugsfd/persistent_worker.py", line 1772, in main [09:57] [ComfyUI] result_meta = _to_shm(result, shm_registry) [09:57] [ComfyUI] File "/tmp/comfyui_pvenv_0kgugsfd/persistent_worker.py", line 692, in _to_shm [09:57] [ComfyUI] "args": _to_shm(list(obj.args), registry, visited), [09:57] [ComfyUI] File "/tmp/comfyui_pvenv_0kgugsfd/persistent_worker.py", line 705, in _to_shm [09:57] [ComfyUI] result = [_to_shm(v, registry, visited) for v in obj] [09:57] [ComfyUI] File "/tmp/comfyui_pvenv_0kgugsfd/persistent_worker.py", line 705, in [09:57] [ComfyUI] result = [_to_shm(v, registry, visited) for v in obj] [09:58] [ComfyUI] File "/tmp/comfyui_pvenv_0kgugsfd/persistent_worker.py", line 628, in _to_shm [09:58] [ComfyUI] result = _serialize_tensor_native(tensor, registry) [09:58] [ComfyUI] File "/tmp/comfyui_pvenv_0kgugsfd/persistent_worker.py", line 563, in _serialize_tensor_native [09:58] [ComfyUI] t.share_memory_() [09:58] [ComfyUI] File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/site-packages/torch/_tensor.py", line 853, in share_memory_ [09:58] [ComfyUI] self._typed_storage()._share_memory_() [09:58] [ComfyUI] File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/site-packages/torch/storage.py", line 1193, in _share_memory_ [09:58] [ComfyUI] self._untyped_storage.share_memory_() [09:58] [ComfyUI] File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/site-packages/torch/storage.py", line 449, in wrapper [09:58] [ComfyUI] return fn(self, *args, **kwargs) [09:58] [ComfyUI] File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/site-packages/torch/storage.py", line 520, in share_memory_ [09:58] [ComfyUI] return super().share_memory_(*args, **kwargs) [09:58] [ComfyUI] File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/site-packages/torch/storage.py", line 400, in share_memory_ [09:58] [ComfyUI] self._share_fd_cpu_() [09:58] [ComfyUI] File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/site-packages/torch/storage.py", line 449, in wrapper [09:58] [ComfyUI] return fn(self, *args, **kwargs) [09:58] [ComfyUI] File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/site-packages/torch/storage.py", line 524, in _share_fd_cpu_ [09:58] [ComfyUI] return super()._share_fd_cpu_(*args, **kwargs) [09:58] [ComfyUI] RuntimeError: unable to allocate shared memory(shm) for file : Success (0) [09:58] [ComfyUI] [09:58] [ComfyUI] [09:58] [ComfyUI] [INFO] Prompt executed in 223.04 seconds [09:58] Execution complete (t=223.6s) [09:58] Execution error: {'prompt_id': '16f9d308-d076-4c5d-9a60-1719bd41817a', 'node_id': '87', 'node_type': 'PanoramaSplitAdaptive', 'executed': ['7', '81', '61', '96', '9', '70', '62', '59', '80', '60', '68'], 'exception_message': 'unable to allocate shared memory(shm) for file : Success (0)\n\nWorker traceback:\nTraceback (most recent call last):\n File "/tmp/comfyui_pvenv_0kgugsfd/persistent_worker.py", line 1772, in main\n result_meta = _to_shm(result, shm_registry)\n File "/tmp/comfyui_pvenv_0kgugsfd/persistent_worker.py", line 692, in _to_shm\n "args": _to_shm(list(obj.args), registry, visited),\n File "/tmp/comfyui_pvenv_0kgugsfd/persistent_worker.py", line 705, in _to_shm\n result = [_to_shm(v, registry, visited) for v in obj]\n File "/tmp/comfyui_pvenv_0kgugsfd/persistent_worker.py", line 705, in \n result = [_to_shm(v, registry, visited) for v in obj]\n File "/tmp/comfyui_pvenv_0kgugsfd/persistent_worker.py", line 628, in _to_shm\n result = _serialize_tensor_native(tensor, registry)\n File "/tmp/comfyui_pvenv_0kgugsfd/persistent_worker.py", line 563, in _serialize_tensor_native\n t.share_memory_()\n File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/site-packages/torch/_tensor.py", line 853, in share_memory_\n self._typed_storage()._share_memory_()\n File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/site-packages/torch/storage.py", line 1193, in _share_memory_\n self._untyped_storage.share_memory_()\n File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/site-packages/torch/storage.py", line 449, in wrapper\n return fn(self, *args, **kwargs)\n File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/site-packages/torch/storage.py", line 520, in share_memory_\n return super().share_memory_(*args, **kwargs)\n File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/site-packages/torch/storage.py", line 400, in share_memory_\n self._share_fd_cpu_()\n File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/site-packages/torch/storage.py", line 449, in wrapper\n return fn(self, *args, **kwargs)\n File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/site-packages/torch/storage.py", line 524, in _share_fd_cpu_\n return super()._share_fd_cpu_(*args, **kwargs)\nRuntimeError: unable to allocate shared memory(shm) for file : Success (0)\n\n', 'exception_type': 'comfy_env.isolation.workers.base.WorkerError', 'traceback': [' File "/workspaces/Sharp-1914/ComfyUI/execution.py", line 536, in execute\n output_data, output_ui, has_subgraph, has_pending_tasks = await get_output_data(prompt_id, unique_id, obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)\n', ' File "/workspaces/Sharp-1914/ComfyUI/execution.py", line 336, in get_output_data\n return_values = await _async_map_node_over_list(prompt_id, unique_id, obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)\n', ' File "/workspaces/Sharp-1914/ComfyUI/execution.py", line 310, in _async_map_node_over_list\n await process_inputs(input_dict, i)\n', ' File "/workspaces/Sharp-1914/ComfyUI/execution.py", line 298, in process_inputs\n result = f(**inputs)\n', ' File "/workspaces/Sharp-1914/.venv/lib/python3.10/site-packages/comfy_env/isolation/metadata.py", line 556, in proxy\n result = worker.call_method(\n', ' File "/workspaces/Sharp-1914/.venv/lib/python3.10/site-packages/comfy_env/isolation/workers/subprocess.py", line 674, in call_method\n raise WorkerError(\n'], 'current_inputs': {'split_method': ['icosahedron_42'], 'resolution': [1536], 'fov_degrees': [90], 'use_gpu': [True], 'create_masks': [True], 'mask_method': ['closest_to'], 'panorama': ["{'image': tensor([[[[0.1647, 0.1020, 0.0745],\n [0.1765, 0.1098, 0.0863],\n [0.1765, 0.1137, 0.0941],\n ...,\n [0.1647, 0.1137, 0.0941],\n [0.1647, 0.1098, 0.0902],\n [0.1333, 0.0784, 0.0667]],\n\n [[0.1922, 0.1255, 0.0902],\n [0.1843, 0.1216, 0.0902],\n [0.1922, 0.1255, 0.0980],\n ...,\n [0.1765, 0.1255, 0.0980],\n [0.1725, 0.1255, 0.1020],\n [0.1647, 0.1059, 0.0824]],\n\n [[0.1882, 0.1216, 0.0980],\n [0.1843, 0.1216, 0.0980],\n [0.1882, 0.1216, 0.0980],\n ...,\n [0.1804, 0.1333, 0.1020],\n [0.1804, 0.1373, 0.1098],\n [0.1569, 0.1059, 0.0863]],\n\n ...,\n\n [[0.1882, 0.1255, 0.1176],\n [0.1922, 0.1255, 0.1176],\n [0.1922, 0.1255, 0.1176],\n ...,\n [0.1176, 0.1333, 0.1412],\n [0.1686, 0.1529, 0.1451],\n [0.1843, 0.1569, 0.1333]],\n\n [[0.1882, 0.1255, 0.1176],\n [0.1882, 0.1255, 0.1137],\n [0.1922, 0.1255, 0.1176],\n ...,\n [0.1647, 0.1412, 0.1098],\n [0.1922, 0.1529, 0.1176],\n [0.2039, 0.1490, 0.1294]],\n\n [[0.1765, 0.1137, 0.1137],\n [0.1804, 0.1255, 0.1216],\n [0.1765, 0.1216, 0.1137],\n ...,\n [0.1961, 0.1725, 0.1412],\n [0.1647, 0.1451, 0.1333],\n [0.1843, 0.1137, 0.1373]]]]), 'meta': {}}"], 'depth_panorama': ['tensor([[[[0.7430, 0.7430, 0.7430],\n [0.7430, 0.7430, 0.7430],\n [0.7430, 0.7430, 0.7430],\n ...,\n [0.7430, 0.7430, 0.7430],\n [0.7430, 0.7430, 0.7430],\n [0.7430, 0.7430, 0.7430]],\n\n [[0.7429, 0.7429, 0.7429],\n [0.7429, 0.7429, 0.7429],\n [0.7429, 0.7429, 0.7429],\n ...,\n [0.7429, 0.7429, 0.7429],\n [0.7429, 0.7429, 0.7429],\n [0.7429, 0.7429, 0.7429]],\n\n [[0.7430, 0.7430, 0.7430],\n [0.7430, 0.7430, 0.7430],\n [0.7430, 0.7430, 0.7430],\n ...,\n [0.7430, 0.7430, 0.7430],\n [0.7430, 0.7430, 0.7430],\n [0.7430, 0.7430, 0.7430]],\n\n ...,\n\n [[0.8810, 0.8810, 0.8810],\n [0.8810, 0.8810, 0.8810],\n [0.8810, 0.8810, 0.8810],\n ...,\n [0.8810, 0.8810, 0.8810],\n [0.8810, 0.8810, 0.8810],\n [0.8810, 0.8810, 0.8810]],\n\n [[0.8811, 0.8811, 0.8811],\n [0.8811, 0.8811, 0.8811],\n [0.8811, 0.8811, 0.8811],\n ...,\n [0.8811, 0.8811, 0.8811],\n [0.8811, 0.8811, 0.8811],\n [0.8811, 0.8811, 0.8811]],\n\n [[0.8813, 0.8813, 0.8813],\n [0.8813, 0.8813, 0.8813],\n [0.8813, 0.8813, 0.8813],\n ...,\n [0.8813, 0.8813, 0.8813],\n [0.8813, 0.8813, 0.8813],\n [0.8813, 0.8813, 0.8813]]]])'], 'normals_panorama': ['tensor([[[[4.8400e-01, 5.0399e-01, 2.7195e-04],\n [4.8401e-01, 5.0400e-01, 2.7180e-04],\n [4.8401e-01, 5.0400e-01, 2.7180e-04],\n ...,\n [4.8401e-01, 5.0399e-01, 2.7168e-04],\n [4.8401e-01, 5.0399e-01, 2.7168e-04],\n [4.8400e-01, 5.0399e-01, 2.7195e-04]],\n\n [[4.8384e-01, 5.0488e-01, 2.8512e-04],\n [4.8384e-01, 5.0488e-01, 2.8512e-04],\n [4.8384e-01, 5.0488e-01, 2.8512e-04],\n ...,\n [4.8384e-01, 5.0488e-01, 2.8512e-04],\n [4.8384e-01, 5.0488e-01, 2.8512e-04],\n [4.8384e-01, 5.0488e-01, 2.8512e-04]],\n\n [[4.8386e-01, 5.0571e-01, 2.9308e-04],\n [4.8386e-01, 5.0571e-01, 2.9308e-04],\n [4.8386e-01, 5.0571e-01, 2.9308e-04],\n ...,\n [4.8386e-01, 5.0571e-01, 2.9308e-04],\n [4.8386e-01, 5.0571e-01, 2.9308e-04],\n [4.8386e-01, 5.0571e-01, 2.9308e-04]],\n\n ...,\n\n [[4.9929e-01, 5.1112e-01, 9.9988e-01],\n [4.9929e-01, 5.1112e-01, 9.9988e-01],\n [4.9929e-01, 5.1112e-01, 9.9988e-01],\n ...,\n [4.9929e-01, 5.1112e-01, 9.9988e-01],\n [4.9929e-01, 5.1112e-01, 9.9988e-01],\n [4.9929e-01, 5.1112e-01, 9.9988e-01]],\n\n [[4.9880e-01, 5.1144e-01, 9.9987e-01],\n [4.9880e-01, 5.1144e-01, 9.9987e-01],\n [4.9880e-01, 5.1144e-01, 9.9987e-01],\n ...,\n [4.9880e-01, 5.1144e-01, 9.9987e-01],\n [4.9880e-01, 5.1144e-01, 9.9987e-01],\n [4.9880e-01, 5.1144e-01, 9.9987e-01]],\n\n [[4.9943e-01, 5.1242e-01, 9.9985e-01],\n [4.9943e-01, 5.1242e-01, 9.9985e-01],\n [4.9943e-01, 5.1242e-01, 9.9985e-01],\n ...,\n [4.9943e-01, 5.1242e-01, 9.9985e-01],\n [4.9943e-01, 5.1242e-01, 9.9985e-01],\n [4.9943e-01, 5.1242e-01, 9.9985e-01]]]])']}, 'current_outputs': ['7', '61', '81', '54', '96', '87', '9', '62', '70', '59', '76', '73', '80', '89', '95', '68', '78', '60'], 'timestamp': 1780860292737} [09:58] [ComfyUI] [comfy-env] [VRAM] Before offload 'MoGeWrapper_1' (631 MB) -> cpu: 6567 / 24122 MB [09:58] Capturing execution frames: sharp_basic.json [09:58] [ComfyUI] [ComfyUI-MoGe2] Health check: ping (timeout=5.0s)... [09:58] [ComfyUI] [ComfyUI-MoGe2] python: /tmp/comfytest-home/.ce/.pixi/envs/moge2-nodes/bin/python [09:58] [ComfyUI] [ComfyUI-MoGe2] sys_path sent to worker: ['/workspaces/Sharp-1914/ComfyUI', '/workspaces/Sharp-1914/ComfyUI/custom_nodes/ComfyUI-MoGe2', '/tmp/comfytest-home/.ce/.pixi/envs/moge2-nodes/lib/python3.1/site-packages', '/workspaces/Sharp-1914/ComfyUI/custom_nodes/ComfyUI-MoGe2'] [09:58] [ComfyUI] [ComfyUI-MoGe2] Health check: failed [09:58] [ComfyUI] [ComfyUI-MoGe2] Socket unhealthy, restarting worker... [09:58] [ComfyUI] [SubprocessWorker] is_pixi=True, python=/tmp/comfytest-home/.ce/.pixi/envs/moge2-nodes/bin/python [09:58] [ComfyUI] [comfy-env] Invalidated 1 stale model patchers (will be cleaned up during next unload) [09:58] [ComfyUI] [SubprocessWorker] launching cmd=['/tmp/comfytest-home/.pixi/bin/pixi', 'run', '--as-is', '--manifest-path', '/tmp/comfytest-home/.ce/pixi.toml', '-e', 'moge2-nodes', 'python', '/tmp/comfyui_pvenv_bmmahm2z/persistent_worker.py', 'abstract://comfy_worker_e79ad090de81']... [09:58] [ComfyUI] [SubprocessWorker] socket_addr=abstract://comfy_worker_e79ad090de81 (abstract, no filesystem path) [09:59] [ComfyUI] [worker:ComfyUI-MoGe2] ERROR: Failed to import comfy_kitchen, Error: No module named 'comfy_kitchen', fp8 and fp4 support will not be available. [09:59] [ComfyUI] [worker:ComfyUI-MoGe2] WARNING: comfy_kitchen does not support stochastic FP8 rounding, please update comfy_kitchen. [09:59] [ComfyUI] [SubprocessWorker] call_id=3 sending model_to_device [09:59] [ComfyUI] [WARNING] Model 'MoGeWrapper_1' gone from worker (restarted?), treating as offloaded [09:59] [ComfyUI] [comfy-env] [VRAM] After offload 'MoGeWrapper_1' (631 MB): 4817 / 24122 MB [09:59] [ComfyUI] [INFO] Using RAM pressure cache. [10:03] Frame 1 saved (frame_000.jpg, t=0.0s) [10:03] Validating workflow... [10:03] [ComfyUI] [INFO] === /validate endpoint called === [10:03] [ComfyUI] [INFO] Received JSON with keys: ['prompt'] [10:03] Queuing workflow for execution... [10:03] [ComfyUI] [INFO] Prompt has 4 nodes [10:03] [ComfyUI] [INFO] Node 1: LoadImage [10:03] [ComfyUI] [INFO] Inputs: {"image": "beach.jpg"} [10:03] [ComfyUI] [INFO] Node 2: LoadSharpModel [10:03] [ComfyUI] [INFO] Inputs: {"precision": "auto"} [10:03] [ComfyUI] [INFO] Node 5: SharpPredict [10:03] [ComfyUI] [INFO] Inputs: {"focal_length_mm": 30, "output_prefix": "sharp", "model": ["2", 0], "image": ["1", 0]} [10:03] [ComfyUI] [INFO] Node 6: PreviewGaussianSharp [10:03] [ComfyUI] [INFO] Inputs: {"preview_gaussian": "", "ply_path": ["5", 0], "extrinsics": ["5", 1], "intrinsics": ["5", 2]} [10:03] [ComfyUI] [INFO] Generated prompt_id: 1f141f5e-7c30-4e18-9d61-64bb3fb15caa [10:03] [ComfyUI] [INFO] Calling execution.validate_prompt()... [10:03] [ComfyUI] [ComfyUI-Sharp] python: /tmp/comfytest-home/.ce/.pixi/envs/sharp-nodes/bin/python [10:03] [ComfyUI] [INFO] Validation result: valid=True [10:03] [ComfyUI] [ComfyUI-Sharp] sys_path sent to worker: ['/workspaces/Sharp-1914/ComfyUI', '/workspaces/Sharp-1914/ComfyUI/custom_nodes/ComfyUI-Sharp', '/tmp/comfytest-home/.ce/.pixi/envs/sharp-nodes/lib/python3.1/site-packages', '/workspaces/Sharp-1914/ComfyUI/custom_nodes/ComfyUI-Sharp'] [10:03] [ComfyUI] [INFO] === Validation PASSED === [10:03] [ComfyUI] [SubprocessWorker] is_pixi=True, python=/tmp/comfytest-home/.ce/.pixi/envs/sharp-nodes/bin/python [10:03] [ComfyUI] [INFO] got prompt [10:03] [ComfyUI] [SubprocessWorker] launching cmd=['/tmp/comfytest-home/.pixi/bin/pixi', 'run', '--as-is', '--manifest-path', '/tmp/comfytest-home/.ce/pixi.toml', '-e', 'sharp-nodes', 'python', '/tmp/comfyui_pvenv_qlpb5zhh/persistent_worker.py', 'abstract://comfy_worker_8bd7d9f62285']... [10:03] [ComfyUI] [comfy-env] Running LoadSharpModel: precision: 'auto' [10:03] [ComfyUI] [SubprocessWorker] socket_addr=abstract://comfy_worker_8bd7d9f62285 (abstract, no filesystem path) [10:03] [ComfyUI] [VRAM] Before LoadSharpModel: 4817 / 24122 MB [10:03] [ComfyUI] [SubprocessWorker] call_method: nodes.load_model.LoadSharpModel.EXECUTE_NORMALIZED [10:03] [ComfyUI] [SubprocessWorker] acquired lock, ensuring started... [10:03] [ComfyUI] [ComfyUI-Sharp] Health check: ping (timeout=5.0s)... [10:03] [ComfyUI] [ComfyUI-Sharp] Health check: failed [10:03] [ComfyUI] [ComfyUI-Sharp] Socket unhealthy, restarting worker... [10:04] [ComfyUI] [worker:ComfyUI-Sharp] ERROR: Failed to import comfy_kitchen, Error: No module named 'comfy_kitchen', fp8 and fp4 support will not be available. [10:04] [ComfyUI] [worker:ComfyUI-Sharp] WARNING: comfy_kitchen does not support stochastic FP8 rounding, please update comfy_kitchen. [10:04] [ComfyUI] [SubprocessWorker] worker started/confirmed [10:04] [ComfyUI] [SubprocessWorker] serializing kwargs to shm... [10:04] [ComfyUI] [SubprocessWorker] created 0 shm blocks [10:04] [ComfyUI] [SubprocessWorker] sending request via socket... [10:04] [ComfyUI] [SubprocessWorker] call_id=2 sending call_method [10:05] [ComfyUI] [worker:ComfyUI-Sharp] [LoadSharpModel] precision='auto' -> dtype=bf16 (device=cuda:0) [10:05] [ComfyUI] [worker:ComfyUI-Sharp] [LoadSharpModel] checkpoint already present, reusing (no download): /workspaces/Sharp-1914/ComfyUI/models/sharp/sharp_2572gikvuh.pt (2679.6 MB) [10:05] [ComfyUI] [worker:ComfyUI-Sharp] [LoadSharpModel] checkpoint ready: /workspaces/Sharp-1914/ComfyUI/models/sharp/sharp_2572gikvuh.pt (2679.6 MB) [10:05] [ComfyUI] [worker:ComfyUI-Sharp] [LoadSharpModel] config: precision=auto -> dtype=bf16 [10:05] [ComfyUI] [SubprocessWorker] got response: ok [10:05] [ComfyUI] [comfy-env] [VRAM] Before offload 'MoGeWrapper_1' (631 MB) -> cpu: 4817 / 24122 MB [10:05] [ComfyUI] [ComfyUI-MoGe2] Health check: ping (timeout=5.0s)... [10:05] [ComfyUI] [ComfyUI-MoGe2] Health check: ok [10:05] [ComfyUI] [SubprocessWorker] call_id=4 sending model_to_device [10:05] [ComfyUI] [WARNING] Model 'MoGeWrapper_1' gone from worker (restarted?), treating as offloaded [10:05] [ComfyUI] [comfy-env] [VRAM] After offload 'MoGeWrapper_1' (631 MB): 4817 / 24122 MB [10:05] [ComfyUI] [comfy-env] LoadSharpModel done (1.64s): result: NodeOutput [10:05] [ComfyUI] [VRAM] After LoadSharpModel: 4817 / 24122 MB [10:05] [ComfyUI] [comfy-env] Running SharpPredict: focal_length_mm: 30.0, output_prefix: 'sharp', model: dict, image: torch.float32 [1,1024,1536,3] cpu [10:05] [ComfyUI] [VRAM] Before SharpPredict: 4817 / 24122 MB [10:05] [ComfyUI] [SubprocessWorker] call_method: nodes.predict.SharpPredict.EXECUTE_NORMALIZED [10:05] [ComfyUI] [SubprocessWorker] acquired lock, ensuring started... [10:05] [ComfyUI] [ComfyUI-Sharp] Health check: ping (timeout=5.0s)... [10:05] [ComfyUI] [ComfyUI-Sharp] Health check: ok [10:05] [ComfyUI] [SubprocessWorker] worker started/confirmed [10:05] [ComfyUI] [comfy-env] PRE-SERIALIZE 'image' shape: torch.Size([1, 1024, 1536, 3]) [10:05] [ComfyUI] [SubprocessWorker] serializing kwargs to shm... [10:05] [ComfyUI] [SubprocessWorker] created 0 shm blocks [10:05] [ComfyUI] [SubprocessWorker] sending request via socket... [10:05] [ComfyUI] [SubprocessWorker] call_id=3 sending call_method [10:05] [ComfyUI] [worker:ComfyUI-Sharp] [comfy-env] _from_shm got dict with keys: ['__type__', 'strategy', 'parent_pid', 'fd', 'storage_size'] [10:05] [ComfyUI] [worker:ComfyUI-Sharp] [comfy-env] DESERIALIZE TensorRef: tensor_size=[1, 1024, 1536, 3] [10:05] [ComfyUI] [worker:ComfyUI-Sharp] [comfy-env] DESERIALIZED tensor shape: torch.Size([1, 1024, 1536, 3]) [10:05] Frame 2 saved (frame_001.jpg, t=2.3s) [10:05] [capture-periodic] freeze=1ms shot=108ms unfreeze=3ms saved=True [10:08] Frame 3 saved (frame_002.jpg, t=4.5s) [10:08] [capture-periodic] freeze=1ms shot=108ms unfreeze=3ms saved=True [10:08] [capture-loop] iter=50 t=5.6s state={'complete': False, 'error': None, 'executedCount': 0, 'wsState': 1} eval_ms=1 ram=10.8/31.3GB (34%) swap=1.8/32.0GB (6%) cpu=15% browser_rss=1.19GB [10:09] [ComfyUI] [worker:ComfyUI-Sharp] [LoadSharpModel] loading checkpoint from /workspaces/Sharp-1914/ComfyUI/models/sharp/sharp_2572gikvuh.pt (dtype=bf16) [10:11] [ComfyUI] [worker:ComfyUI-Sharp] [LoadSharpModel] initializing model on meta device... [10:11] [ComfyUI] [worker:ComfyUI-Sharp] [LoadSharpModel] model ready (torch.bfloat16) [10:12] [ComfyUI] [SubprocessWorker] callback 'request_vram_budget' from call_id=3 [10:12] [ComfyUI] [comfy-env] VRAM request: 2.81GB, free before eviction: 20.24GB [10:12] [ComfyUI] [comfy-env] VRAM after eviction: 20.24GB [10:12] [ComfyUI] [worker:ComfyUI-Sharp] [VRAM] +715MB (now 715MB) reserved=736MB peak=715MB [10:13] [ComfyUI] [worker:ComfyUI-Sharp] [VRAM] +2563MB (now 3278MB) reserved=3456MB peak=3350MB [10:14] [ComfyUI] [worker:ComfyUI-Sharp] [VRAM] +415MB (now 3693MB) reserved=4032MB peak=3693MB [10:14] [capture-loop] iter=100 t=11.0s state={'complete': False, 'error': None, 'executedCount': 0, 'wsState': 1} eval_ms=2 ram=11.1/31.3GB (36%) swap=1.8/32.0GB (6%) cpu=13% browser_rss=1.09GB [10:15] [ComfyUI] [worker:ComfyUI-Sharp] [VRAM] +470MB (now 4164MB) reserved=7840MB peak=7053MB [10:16] [ComfyUI] [worker:ComfyUI-Sharp] [VRAM] +234MB (now 4398MB) reserved=7360MB peak=7053MB [10:17] [ComfyUI] [worker:ComfyUI-Sharp] [VRAM] -1544MB (now 2854MB) reserved=7360MB peak=7053MB [10:17] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=3 [10:17] [ComfyUI] [SubprocessWorker] got response: ok [10:17] [ComfyUI] [comfy-env] SubprocessWorker: /tmp/comfytest-home/.ce/.pixi/envs/gaussianpack-nodes/bin/python [10:17] [ComfyUI] [comfy-env] Created 1 model patchers: ['RGBGaussianPredictor_1'] [10:17] [ComfyUI] [comfy-env] env_vars: {'COMFYUI_BASE': '/workspaces/Sharp-1914/ComfyUI'} [10:17] [ComfyUI] [ComfyUI-GaussianPack] python: /tmp/comfytest-home/.ce/.pixi/envs/gaussianpack-nodes/bin/python [10:17] [ComfyUI] [comfy-env] SharpPredict done (12.18s): result: NodeOutput [10:17] [ComfyUI] [ComfyUI-GaussianPack] sys_path sent to worker: ['/workspaces/Sharp-1914/ComfyUI', '/workspaces/Sharp-1914/ComfyUI/custom_nodes/ComfyUI-GaussianPack', '/tmp/comfytest-home/.ce/.pixi/envs/gaussianpack-nodes/lib/python3.1/site-packages', '/workspaces/Sharp-1914/ComfyUI/custom_nodes/ComfyUI-GaussianPack'] [10:17] [ComfyUI] [VRAM] After SharpPredict: 12291 / 24122 MB [10:17] [ComfyUI] [VRAM] Loaded: SubprocessModel (2679 MB) [10:17] [ComfyUI] [SubprocessWorker] is_pixi=True, python=/tmp/comfytest-home/.ce/.pixi/envs/gaussianpack-nodes/bin/python [10:17] [ComfyUI] [comfy-env] Running PreviewGaussianSharp: ply_path: '/workspaces/Sharp-1914/ComfyUI/output/sharp_178086030688..., extrinsics: list[4], intrinsics: list[3] [10:17] [ComfyUI] [SubprocessWorker] launching cmd=['/tmp/comfytest-home/.pixi/bin/pixi', 'run', '--as-is', '--manifest-path', '/tmp/comfytest-home/.ce/pixi.toml', '-e', 'gaussianpack-nodes', 'python', '/tmp/comfyui_pvenv_ii0358nh/persistent_worker.py', 'abstract://comfy_worker_26954ef0c0d3']... [10:17] [ComfyUI] [VRAM] Before PreviewGaussianSharp: 12291 / 24122 MB [10:17] [ComfyUI] [SubprocessWorker] socket_addr=abstract://comfy_worker_26954ef0c0d3 (abstract, no filesystem path) [10:17] [ComfyUI] [VRAM] Loaded: SubprocessModel (2679 MB) [10:17] [ComfyUI] [SubprocessWorker] call_method: nodes.preview_gaussian_sharp.PreviewGaussianSharp.EXECUTE_NORMALIZED [10:17] [ComfyUI] [SubprocessWorker] acquired lock, ensuring started... [10:18] [ComfyUI] [SubprocessWorker] worker started/confirmed [10:18] [ComfyUI] [SubprocessWorker] serializing kwargs to shm... [10:18] [ComfyUI] [SubprocessWorker] created 0 shm blocks [10:18] [ComfyUI] [SubprocessWorker] sending request via socket... [10:18] [ComfyUI] [SubprocessWorker] call_id=1 sending call_method [10:18] Frame 4 saved (frame_003.jpg, t=15.4s) [10:18] [capture-periodic] freeze=1ms shot=96ms unfreeze=2ms saved=True [10:18] [ComfyUI] [worker:ComfyUI-GaussianPack] WARNING: SPZ route not registered (ModuleNotFoundError: No module named 'comfy_aimdo') [10:18] [ComfyUI] [SubprocessWorker] got response: ok [10:18] [ComfyUI] [comfy-env] PreviewGaussianSharp done (1.53s): result: NodeOutput [10:18] [ComfyUI] [VRAM] After PreviewGaussianSharp: 12291 / 24122 MB [10:18] [ComfyUI] [VRAM] Loaded: SubprocessModel (2679 MB) [10:18] [ComfyUI] [INFO] Prompt executed in 15.38 seconds [10:18] [ComfyUI] [comfy-env] [VRAM] Before offload 'RGBGaussianPredictor_1' (2679 MB) -> cpu: 12291 / 24122 MB [10:18] [ComfyUI] [ComfyUI-Sharp] Health check: ping (timeout=5.0s)... [10:18] [ComfyUI] [ComfyUI-Sharp] Health check: ok [10:18] [ComfyUI] [SubprocessWorker] call_id=4 sending model_to_device [10:19] [ComfyUI] [worker:ComfyUI-Sharp] [VRAM] -360MB (now 2495MB) reserved=7360MB peak=7053MB [10:19] Node executed (1 total), capturing... [10:20] [ComfyUI] [worker:ComfyUI-Sharp] [VRAM] -1769MB (now 726MB) reserved=7360MB peak=7053MB [10:20] Frame 5 saved (frame_004.jpg, t=15.9s) [10:20] [capture-node] freeze=631ms shot=409ms unfreeze=0ms saved=True [10:20] Execution complete (t=15.9s) [10:20] [3d-wait] Phase 1: Waiting for 'Loading 3D Model' text to disappear... [10:20] [3d-wait] Phase 1 done in 0.0s [10:20] [3d-wait] Phase 2: iframes=1 [10:20] [3d-wait] iframe viewer_ready=True (matched 1 console logs) [10:20] [3d-wait] Unfrozen in 0.001s, rendering for 5s... [10:20] [ComfyUI] [comfy-env] [VRAM] After offload 'RGBGaussianPredictor_1' (2679 MB): 5027 / 24122 MB [10:20] [ComfyUI] [INFO] Using RAM pressure cache. [10:28] [3d-wait] JS halted via CDP after 7.7s [10:51] [3d-wait] Re-frozen in 31.3s [10:51] [3d-wait] Total 3D wait: 31.3s [10:51] Captured 5 unique frames over 48.25s [10:51] [debug] iframes: 1 [10:51] [debug] iframe[0]: src=http://127.0.0.1:41899/extensions/ComfyUI-GaussianPack/viewer_gaussian_sharp.htm, canvas=1536x1024, rafFrozen=True [10:51] [timing] trigger_3d_previews: 0.0s [10:53] Saved high-quality screenshot: sharp_basic_executed.png [10:53] [timing] final_screenshot: 0.3s [10:54] [ComfyUI] [INFO] Using RAM pressure cache. [10:54] Capturing execution frames: sharp_exif.json [10:54] [ComfyUI] [INFO] Using RAM pressure cache. [10:59] Frame 1 saved (frame_000.jpg, t=0.0s) [10:59] Validating workflow... [10:59] [ComfyUI] [INFO] === /validate endpoint called === [10:59] Queuing workflow for execution... [10:59] [ComfyUI] [INFO] Received JSON with keys: ['prompt'] [10:59] [ComfyUI] [INFO] Prompt has 4 nodes [10:59] [ComfyUI] [INFO] Node 2: LoadSharpModel [10:59] [ComfyUI] [INFO] Inputs: {"precision": "auto"} [10:59] [ComfyUI] [ComfyUI-Sharp] python: /tmp/comfytest-home/.ce/.pixi/envs/sharp-nodes/bin/python [10:59] [ComfyUI] [INFO] Node 7: SharpPredict [10:59] [ComfyUI] [ComfyUI-Sharp] sys_path sent to worker: ['/workspaces/Sharp-1914/ComfyUI', '/workspaces/Sharp-1914/ComfyUI/custom_nodes/ComfyUI-Sharp', '/tmp/comfytest-home/.ce/.pixi/envs/sharp-nodes/lib/python3.1/site-packages', '/workspaces/Sharp-1914/ComfyUI/custom_nodes/ComfyUI-Sharp'] [10:59] [ComfyUI] [INFO] Inputs: {"focal_length_mm": ["9", 2], "output_prefix": "sharp", "model": ["2", 0], "image": ["9", 0]} [10:59] [ComfyUI] [SubprocessWorker] is_pixi=True, python=/tmp/comfytest-home/.ce/.pixi/envs/sharp-nodes/bin/python [10:59] [ComfyUI] [INFO] Node 9: LoadImageWithExif [10:59] [ComfyUI] [SubprocessWorker] launching cmd=['/tmp/comfytest-home/.pixi/bin/pixi', 'run', '--as-is', '--manifest-path', '/tmp/comfytest-home/.ce/pixi.toml', '-e', 'sharp-nodes', 'python', '/tmp/comfyui_pvenv_qlpb5zhh/persistent_worker.py', 'abstract://comfy_worker_3201e25c98bb']... [10:59] [ComfyUI] [INFO] Inputs: {"image": "eth3d.png", "default_focal_mm": 30} [10:59] [ComfyUI] [SubprocessWorker] socket_addr=abstract://comfy_worker_3201e25c98bb (abstract, no filesystem path) [10:59] [ComfyUI] [INFO] Node 11: PreviewGaussianSharp [10:59] [ComfyUI] [INFO] Inputs: {"preview_gaussian": "", "ply_path": ["7", 0], "extrinsics": ["7", 1], "intrinsics": ["7", 2]} [10:59] [ComfyUI] [INFO] Generated prompt_id: 35f229fe-8d91-4024-98dd-a9461e299a0b [10:59] [ComfyUI] [INFO] Calling execution.validate_prompt()... [10:59] [ComfyUI] [INFO] Validation result: valid=True [10:59] [ComfyUI] [INFO] === Validation PASSED === [10:59] [ComfyUI] [INFO] got prompt [10:59] [ComfyUI] [comfy-env] Running LoadImageWithExif: image: 'eth3d.png', default_focal_mm: 30.0 [10:59] [ComfyUI] [VRAM] Before LoadImageWithExif: 5027 / 24122 MB [10:59] [ComfyUI] [SubprocessWorker] call_method: nodes.load_image_exif.LoadImageWithExif.EXECUTE_NORMALIZED [10:59] [ComfyUI] [SubprocessWorker] acquired lock, ensuring started... [10:59] [ComfyUI] [ComfyUI-Sharp] Health check: ping (timeout=5.0s)... [10:59] [ComfyUI] [ComfyUI-Sharp] Health check: failed [10:59] [ComfyUI] [ComfyUI-Sharp] Socket unhealthy, restarting worker... [10:59] [ComfyUI] [comfy-env] Invalidated 1 stale model patchers (will be cleaned up during next unload) [11:00] [ComfyUI] [worker:ComfyUI-Sharp] ERROR: Failed to import comfy_kitchen, Error: No module named 'comfy_kitchen', fp8 and fp4 support will not be available. [11:00] [ComfyUI] [worker:ComfyUI-Sharp] WARNING: comfy_kitchen does not support stochastic FP8 rounding, please update comfy_kitchen. [11:01] [ComfyUI] [SubprocessWorker] worker started/confirmed [11:01] [ComfyUI] [SubprocessWorker] serializing kwargs to shm... [11:01] [ComfyUI] [SubprocessWorker] created 0 shm blocks [11:01] [ComfyUI] [SubprocessWorker] sending request via socket... [11:01] [ComfyUI] [SubprocessWorker] call_id=5 sending call_method [11:01] [ComfyUI] [SubprocessWorker] got response: ok [11:01] [ComfyUI] [comfy-env] [VRAM] Before offload 'RGBGaussianPredictor_1' (2679 MB) -> cpu: 4817 / 24122 MB [11:01] [ComfyUI] [ComfyUI-Sharp] Health check: ping (timeout=5.0s)... [11:01] [ComfyUI] [ComfyUI-Sharp] Health check: ok [11:01] [ComfyUI] [SubprocessWorker] call_id=6 sending model_to_device [11:01] [ComfyUI] [WARNING] Model 'RGBGaussianPredictor_1' gone from worker (restarted?), treating as offloaded [11:01] [ComfyUI] [comfy-env] [VRAM] After offload 'RGBGaussianPredictor_1' (2679 MB): 4817 / 24122 MB [11:01] [ComfyUI] [comfy-env] LoadImageWithExif done (1.96s): result: NodeOutput [11:01] [ComfyUI] [VRAM] After LoadImageWithExif: 4817 / 24122 MB [11:01] [ComfyUI] [comfy-env] Running LoadSharpModel: precision: 'auto' [11:01] [ComfyUI] [VRAM] Before LoadSharpModel: 4817 / 24122 MB [11:01] [ComfyUI] [SubprocessWorker] call_method: nodes.load_model.LoadSharpModel.EXECUTE_NORMALIZED [11:01] [ComfyUI] [SubprocessWorker] acquired lock, ensuring started... [11:01] [ComfyUI] [ComfyUI-Sharp] Health check: ping (timeout=5.0s)... [11:01] [ComfyUI] [ComfyUI-Sharp] Health check: ok [11:01] [ComfyUI] [SubprocessWorker] worker started/confirmed [11:01] [ComfyUI] [SubprocessWorker] serializing kwargs to shm... [11:01] [ComfyUI] [SubprocessWorker] created 0 shm blocks [11:01] [ComfyUI] [SubprocessWorker] sending request via socket... [11:01] [ComfyUI] [SubprocessWorker] call_id=7 sending call_method [11:01] [ComfyUI] [worker:ComfyUI-Sharp] [LoadSharpModel] precision='auto' -> dtype=bf16 (device=cuda:0) [11:01] [ComfyUI] [worker:ComfyUI-Sharp] [LoadSharpModel] checkpoint already present, reusing (no download): /workspaces/Sharp-1914/ComfyUI/models/sharp/sharp_2572gikvuh.pt (2679.6 MB) [11:01] [ComfyUI] [worker:ComfyUI-Sharp] [LoadSharpModel] checkpoint ready: /workspaces/Sharp-1914/ComfyUI/models/sharp/sharp_2572gikvuh.pt (2679.6 MB) [11:01] [ComfyUI] [worker:ComfyUI-Sharp] [LoadSharpModel] config: precision=auto -> dtype=bf16 [11:01] [ComfyUI] [SubprocessWorker] got response: ok [11:01] [ComfyUI] [comfy-env] LoadSharpModel done (0.08s): result: NodeOutput [11:01] [ComfyUI] [VRAM] After LoadSharpModel: 4817 / 24122 MB [11:01] [ComfyUI] [comfy-env] Running SharpPredict: focal_length_mm: 20.0, output_prefix: 'sharp', model: dict, image: torch.float32 [1,2022,3040,3] cpu [11:01] [ComfyUI] [VRAM] Before SharpPredict: 4817 / 24122 MB [11:01] [ComfyUI] [SubprocessWorker] call_method: nodes.predict.SharpPredict.EXECUTE_NORMALIZED [11:01] [ComfyUI] [SubprocessWorker] acquired lock, ensuring started... [11:01] [ComfyUI] [ComfyUI-Sharp] Health check: ping (timeout=5.0s)... [11:01] [ComfyUI] [ComfyUI-Sharp] Health check: ok [11:01] [ComfyUI] [SubprocessWorker] worker started/confirmed [11:01] [ComfyUI] [comfy-env] PRE-SERIALIZE 'image' shape: torch.Size([1, 2022, 3040, 3]) [11:01] [ComfyUI] [SubprocessWorker] serializing kwargs to shm... [11:01] [ComfyUI] [SubprocessWorker] created 0 shm blocks [11:01] [ComfyUI] [SubprocessWorker] sending request via socket... [11:01] [ComfyUI] [SubprocessWorker] call_id=8 sending call_method [11:01] [ComfyUI] [worker:ComfyUI-Sharp] [comfy-env] _from_shm got dict with keys: ['__type__', 'strategy', 'parent_pid', 'fd', 'storage_size'] [11:01] [ComfyUI] [worker:ComfyUI-Sharp] [comfy-env] DESERIALIZE TensorRef: tensor_size=[1, 2022, 3040, 3] [11:01] [ComfyUI] [worker:ComfyUI-Sharp] [comfy-env] DESERIALIZED tensor shape: torch.Size([1, 2022, 3040, 3]) [11:01] Frame 2 saved (frame_001.jpg, t=2.3s) [11:02] [capture-periodic] freeze=2ms shot=142ms unfreeze=2ms saved=True [11:03] [ComfyUI] [worker:ComfyUI-Sharp] [LoadSharpModel] loading checkpoint from /workspaces/Sharp-1914/ComfyUI/models/sharp/sharp_2572gikvuh.pt (dtype=bf16) [11:04] Frame 3 saved (frame_002.jpg, t=4.5s) [11:04] [capture-periodic] freeze=1ms shot=99ms unfreeze=2ms saved=True [11:05] [ComfyUI] [worker:ComfyUI-Sharp] [LoadSharpModel] initializing model on meta device... [11:05] [ComfyUI] [worker:ComfyUI-Sharp] [LoadSharpModel] model ready (torch.bfloat16) [11:05] [ComfyUI] [SubprocessWorker] callback 'request_vram_budget' from call_id=8 [11:05] [ComfyUI] [comfy-env] VRAM request: 2.81GB, free before eviction: 20.24GB [11:05] [ComfyUI] [comfy-env] VRAM after eviction: 20.24GB [11:05] [capture-loop] iter=50 t=5.6s state={'complete': False, 'error': None, 'executedCount': 0, 'wsState': 1} eval_ms=1 ram=13.9/31.3GB (44%) swap=1.8/32.0GB (6%) cpu=66% browser_rss=1.09GB [11:05] [ComfyUI] [worker:ComfyUI-Sharp] [VRAM] +1115MB (now 1115MB) reserved=1120MB peak=1115MB [11:06] [ComfyUI] [worker:ComfyUI-Sharp] [VRAM] +2287MB (now 3402MB) reserved=3520MB peak=3402MB [11:07] [ComfyUI] [worker:ComfyUI-Sharp] [VRAM] +3702MB (now 7105MB) reserved=7968MB peak=7105MB [11:08] [ComfyUI] [worker:ComfyUI-Sharp] [VRAM] -2890MB (now 4216MB) reserved=7968MB peak=7105MB [11:09] [ComfyUI] [worker:ComfyUI-Sharp] [VRAM] +300MB (now 4516MB) reserved=7392MB peak=7105MB [11:09] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=8 [11:09] [ComfyUI] [SubprocessWorker] got response: ok [11:09] [ComfyUI] [comfy-env] Created 1 model patchers: ['RGBGaussianPredictor_1'] [11:09] [ComfyUI] [comfy-env] SharpPredict done (7.83s): result: NodeOutput [11:09] [ComfyUI] [VRAM] After SharpPredict: 12323 / 24122 MB [11:09] [ComfyUI] [VRAM] Loaded: SubprocessModel (2679 MB) [11:09] [ComfyUI] [comfy-env] Running PreviewGaussianSharp: ply_path: '/workspaces/Sharp-1914/ComfyUI/output/sharp_178086035996..., extrinsics: list[4], intrinsics: list[3] [11:09] [ComfyUI] [VRAM] Before PreviewGaussianSharp: 12323 / 24122 MB [11:09] [ComfyUI] [VRAM] Loaded: SubprocessModel (2679 MB) [11:09] [ComfyUI] [SubprocessWorker] call_method: nodes.preview_gaussian_sharp.PreviewGaussianSharp.EXECUTE_NORMALIZED [11:09] [ComfyUI] [SubprocessWorker] acquired lock, ensuring started... [11:09] [ComfyUI] [ComfyUI-GaussianPack] Health check: ping (timeout=5.0s)... [11:09] [ComfyUI] [ComfyUI-GaussianPack] Health check: ok [11:09] [ComfyUI] [SubprocessWorker] worker started/confirmed [11:09] [ComfyUI] [SubprocessWorker] serializing kwargs to shm... [11:09] [ComfyUI] [SubprocessWorker] created 0 shm blocks [11:09] [ComfyUI] [SubprocessWorker] sending request via socket... [11:09] [ComfyUI] [SubprocessWorker] call_id=2 sending call_method [11:09] [ComfyUI] [SubprocessWorker] got response: ok [11:09] [ComfyUI] [comfy-env] PreviewGaussianSharp done (0.06s): result: NodeOutput [11:09] [ComfyUI] [VRAM] After PreviewGaussianSharp: 12323 / 24122 MB [11:09] [ComfyUI] [VRAM] Loaded: SubprocessModel (2679 MB) [11:09] [ComfyUI] [INFO] Prompt executed in 9.95 seconds [11:09] Node executed (1 total), capturing... [11:10] Frame 4 saved (frame_003.jpg, t=10.4s) [11:10] [capture-node] freeze=1ms shot=954ms unfreeze=0ms saved=True [11:10] Execution complete (t=10.4s) [11:11] [3d-wait] Phase 1: Waiting for 'Loading 3D Model' text to disappear... [11:11] [3d-wait] Phase 1 done in 0.0s [11:11] [3d-wait] Phase 2: iframes=1 [11:11] [3d-wait] iframe viewer_ready=True (matched 1 console logs) [11:11] [3d-wait] Unfrozen in 0.001s, rendering for 5s... [11:19] [3d-wait] JS halted via CDP after 8.3s [11:44] [3d-wait] Re-frozen in 33.1s [11:44] [3d-wait] Total 3D wait: 33.2s [11:44] Captured 4 unique frames over 44.66s [11:44] [debug] iframes: 1 [11:44] [debug] iframe[0]: src=http://127.0.0.1:41899/extensions/ComfyUI-GaussianPack/viewer_gaussian_sharp.htm, canvas=3040x2022, rafFrozen=True [11:44] [timing] trigger_3d_previews: 0.0s [11:46] Saved high-quality screenshot: sharp_exif_executed.png [11:46] [timing] final_screenshot: 0.3s [11:46] [ComfyUI] [comfy-env] [VRAM] Before offload 'RGBGaussianPredictor_1' (2679 MB) -> cpu: 12323 / 24122 MB [11:46] Capturing execution frames: vanilla_vs_image_depth_gaussians.json [11:46] [ComfyUI] [ComfyUI-Sharp] Health check: ping (timeout=5.0s)... [11:46] [ComfyUI] [ComfyUI-Sharp] python: /tmp/comfytest-home/.ce/.pixi/envs/sharp-nodes/bin/python [11:46] [ComfyUI] [ComfyUI-Sharp] Health check: failed [11:46] [ComfyUI] [ComfyUI-Sharp] sys_path sent to worker: ['/workspaces/Sharp-1914/ComfyUI', '/workspaces/Sharp-1914/ComfyUI/custom_nodes/ComfyUI-Sharp', '/tmp/comfytest-home/.ce/.pixi/envs/sharp-nodes/lib/python3.1/site-packages', '/workspaces/Sharp-1914/ComfyUI/custom_nodes/ComfyUI-Sharp'] [11:46] [ComfyUI] [ComfyUI-Sharp] Socket unhealthy, restarting worker... [11:46] [ComfyUI] [SubprocessWorker] is_pixi=True, python=/tmp/comfytest-home/.ce/.pixi/envs/sharp-nodes/bin/python [11:46] [ComfyUI] [comfy-env] Invalidated 1 stale model patchers (will be cleaned up during next unload) [11:46] [ComfyUI] [SubprocessWorker] launching cmd=['/tmp/comfytest-home/.pixi/bin/pixi', 'run', '--as-is', '--manifest-path', '/tmp/comfytest-home/.ce/pixi.toml', '-e', 'sharp-nodes', 'python', '/tmp/comfyui_pvenv_qlpb5zhh/persistent_worker.py', 'abstract://comfy_worker_32e55c77d639']... [11:46] [ComfyUI] [SubprocessWorker] socket_addr=abstract://comfy_worker_32e55c77d639 (abstract, no filesystem path) [11:47] [ComfyUI] [worker:ComfyUI-Sharp] ERROR: Failed to import comfy_kitchen, Error: No module named 'comfy_kitchen', fp8 and fp4 support will not be available. [11:47] [ComfyUI] [worker:ComfyUI-Sharp] WARNING: comfy_kitchen does not support stochastic FP8 rounding, please update comfy_kitchen. [11:48] [ComfyUI] [SubprocessWorker] call_id=9 sending model_to_device [11:48] [ComfyUI] [WARNING] Model 'RGBGaussianPredictor_1' gone from worker (restarted?), treating as offloaded [11:48] [ComfyUI] [comfy-env] [VRAM] After offload 'RGBGaussianPredictor_1' (2679 MB): 4817 / 24122 MB [11:48] [ComfyUI] [INFO] Using RAM pressure cache. [11:52] Frame 1 saved (frame_000.jpg, t=0.0s) [11:52] Validating workflow... [11:52] [ComfyUI] [INFO] === /validate endpoint called === [11:52] Queuing workflow for execution... [11:52] [ComfyUI] [INFO] Received JSON with keys: ['prompt'] [11:52] [ComfyUI] [INFO] Prompt has 8 nodes [11:52] [ComfyUI] [INFO] Node 7: LoadImage [11:52] [ComfyUI] [INFO] Inputs: {"image": "middlebury.jpg"} [11:52] [ComfyUI] [INFO] Node 9: LoadSharpModel [11:52] [ComfyUI] [INFO] Inputs: {"precision": "auto"} [11:52] [ComfyUI] [INFO] Node 47: SharpPredictMetricDepth [11:52] [ComfyUI] [INFO] Inputs: {"focal_length_mm": 30, "model": ["9", 0], "image": ["7", 0]} [11:52] [ComfyUI] [INFO] Node 57: SharpPredict [11:52] [ComfyUI] [INFO] Inputs: {"focal_length_mm": 30, "output_prefix": "sharp", "model": ["9", 0], "image": ["58", 0]} [11:52] [ComfyUI] [INFO] Node 58: ImageFromBatch [11:52] [ComfyUI] [INFO] Inputs: {"batch_index": 0, "length": 1, "image": ["7", 0]} [11:52] [ComfyUI] [INFO] Node 83: SharpPredictGaussiansFromMetricDepth [11:52] [ComfyUI] [INFO] Inputs: {"snap_to_external_depth": "none", "depth_convention": "planar_z", "focal_length_mm": 30, "output_prefix": "sharp_aligned", "save_background_layer": true, "edge_crop": 0, "model": ["9", 0], "image": [ [11:52] [ComfyUI] [INFO] Node 92: PreviewGaussianSharp [11:52] [ComfyUI] [INFO] Inputs: {"preview_gaussian": "", "ply_path": ["83", 0], "extrinsics": ["83", 2], "intrinsics": ["83", 3]} [11:52] [ComfyUI] [INFO] Node 93: PreviewGaussianSharp [11:52] [ComfyUI] [INFO] Inputs: {"preview_gaussian": "", "ply_path": ["57", 0], "extrinsics": ["57", 1], "intrinsics": ["57", 2]} [11:52] [ComfyUI] [INFO] Generated prompt_id: e934353c-f7d4-4b26-9f6a-de2185353c0e [11:52] [ComfyUI] [INFO] Calling execution.validate_prompt()... [11:52] [ComfyUI] [INFO] Validation result: valid=True [11:52] [ComfyUI] [INFO] === Validation PASSED === [11:52] [ComfyUI] [INFO] got prompt [11:52] [ComfyUI] [comfy-env] Running LoadSharpModel: precision: 'auto' [11:52] [ComfyUI] [VRAM] Before LoadSharpModel: 4817 / 24122 MB [11:52] [ComfyUI] [SubprocessWorker] call_method: nodes.load_model.LoadSharpModel.EXECUTE_NORMALIZED [11:52] [ComfyUI] [SubprocessWorker] acquired lock, ensuring started... [11:52] [ComfyUI] [ComfyUI-Sharp] Health check: ping (timeout=5.0s)... [11:52] [ComfyUI] [ComfyUI-Sharp] Health check: ok [11:52] [ComfyUI] [SubprocessWorker] worker started/confirmed [11:52] [ComfyUI] [SubprocessWorker] serializing kwargs to shm... [11:52] [ComfyUI] [SubprocessWorker] created 0 shm blocks [11:52] [ComfyUI] [SubprocessWorker] sending request via socket... [11:52] [ComfyUI] [SubprocessWorker] call_id=10 sending call_method [11:52] [ComfyUI] [worker:ComfyUI-Sharp] [LoadSharpModel] precision='auto' -> dtype=bf16 (device=cuda:0) [11:52] [ComfyUI] [worker:ComfyUI-Sharp] [LoadSharpModel] checkpoint already present, reusing (no download): /workspaces/Sharp-1914/ComfyUI/models/sharp/sharp_2572gikvuh.pt (2679.6 MB) [11:52] [ComfyUI] [worker:ComfyUI-Sharp] [LoadSharpModel] checkpoint ready: /workspaces/Sharp-1914/ComfyUI/models/sharp/sharp_2572gikvuh.pt (2679.6 MB) [11:52] [ComfyUI] [worker:ComfyUI-Sharp] [LoadSharpModel] config: precision=auto -> dtype=bf16 [11:52] [ComfyUI] [SubprocessWorker] got response: ok [11:52] [ComfyUI] [comfy-env] [VRAM] Before offload 'RGBGaussianPredictor_1' (2679 MB) -> cpu: 4817 / 24122 MB [11:52] [ComfyUI] [ComfyUI-Sharp] Health check: ping (timeout=5.0s)... [11:52] [ComfyUI] [ComfyUI-Sharp] Health check: ok [11:52] [ComfyUI] [SubprocessWorker] call_id=11 sending model_to_device [11:52] [ComfyUI] [WARNING] Model 'RGBGaussianPredictor_1' gone from worker (restarted?), treating as offloaded [11:52] [ComfyUI] [comfy-env] [VRAM] After offload 'RGBGaussianPredictor_1' (2679 MB): 4817 / 24122 MB [11:52] [ComfyUI] [comfy-env] LoadSharpModel done (0.25s): result: NodeOutput [11:52] [ComfyUI] [VRAM] After LoadSharpModel: 4817 / 24122 MB [11:52] [ComfyUI] [comfy-env] Running SharpPredict: focal_length_mm: 30.0, output_prefix: 'sharp', model: dict, image: torch.float32 [1,960,1416,3] cpu [11:52] [ComfyUI] [VRAM] Before SharpPredict: 4817 / 24122 MB [11:52] [ComfyUI] [SubprocessWorker] call_method: nodes.predict.SharpPredict.EXECUTE_NORMALIZED [11:52] [ComfyUI] [SubprocessWorker] acquired lock, ensuring started... [11:52] [ComfyUI] [ComfyUI-Sharp] Health check: ping (timeout=5.0s)... [11:52] [ComfyUI] [ComfyUI-Sharp] Health check: ok [11:52] [ComfyUI] [SubprocessWorker] worker started/confirmed [11:52] [ComfyUI] [comfy-env] PRE-SERIALIZE 'image' shape: torch.Size([1, 960, 1416, 3]) [11:52] [ComfyUI] [SubprocessWorker] serializing kwargs to shm... [11:52] [ComfyUI] [SubprocessWorker] created 0 shm blocks [11:52] [ComfyUI] [SubprocessWorker] sending request via socket... [11:52] [ComfyUI] [SubprocessWorker] call_id=12 sending call_method [11:52] [ComfyUI] [worker:ComfyUI-Sharp] [comfy-env] _from_shm got dict with keys: ['__type__', 'strategy', 'parent_pid', 'fd', 'storage_size'] [11:52] [ComfyUI] [worker:ComfyUI-Sharp] [comfy-env] DESERIALIZE TensorRef: tensor_size=[1, 960, 1416, 3] [11:52] [ComfyUI] [worker:ComfyUI-Sharp] [comfy-env] DESERIALIZED tensor shape: torch.Size([1, 960, 1416, 3]) [11:54] [ComfyUI] [worker:ComfyUI-Sharp] [LoadSharpModel] loading checkpoint from /workspaces/Sharp-1914/ComfyUI/models/sharp/sharp_2572gikvuh.pt (dtype=bf16) [11:54] Frame 2 saved (frame_001.jpg, t=2.3s) [11:54] [capture-periodic] freeze=1ms shot=114ms unfreeze=3ms saved=True [11:55] [ComfyUI] [worker:ComfyUI-Sharp] [LoadSharpModel] initializing model on meta device... [11:55] [ComfyUI] [worker:ComfyUI-Sharp] [LoadSharpModel] model ready (torch.bfloat16) [11:55] [ComfyUI] [SubprocessWorker] callback 'request_vram_budget' from call_id=12 [11:55] [ComfyUI] [comfy-env] VRAM request: 2.81GB, free before eviction: 20.24GB [11:55] [ComfyUI] [comfy-env] VRAM after eviction: 20.24GB [11:55] [ComfyUI] [worker:ComfyUI-Sharp] [VRAM] +314MB (now 314MB) reserved=320MB peak=314MB [11:56] Frame 3 saved (frame_002.jpg, t=4.5s) [11:56] [capture-periodic] freeze=1ms shot=108ms unfreeze=1ms saved=True [11:56] [ComfyUI] [worker:ComfyUI-Sharp] [VRAM] +2924MB (now 3239MB) reserved=3456MB peak=3311MB [11:57] [capture-loop] iter=50 t=5.6s state={'complete': False, 'error': None, 'executedCount': 0, 'wsState': 1} eval_ms=1 ram=11.3/31.3GB (36%) swap=1.8/32.0GB (6%) cpu=68% browser_rss=1.03GB [11:57] [ComfyUI] [worker:ComfyUI-Sharp] [VRAM] -267MB (now 2973MB) reserved=3360MB peak=3311MB [11:58] [ComfyUI] [worker:ComfyUI-Sharp] [VRAM] +1188MB (now 4161MB) reserved=7872MB peak=7051MB [11:59] [ComfyUI] [worker:ComfyUI-Sharp] [VRAM] +234MB (now 4395MB) reserved=7328MB peak=7051MB [12:00] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=12 [12:00] [ComfyUI] [SubprocessWorker] got response: ok [12:00] [ComfyUI] [comfy-env] Created 1 model patchers: ['RGBGaussianPredictor_1'] [12:00] [ComfyUI] [ComfyUI-GaussianPack] python: /tmp/comfytest-home/.ce/.pixi/envs/gaussianpack-nodes/bin/python [12:00] [ComfyUI] [comfy-env] SharpPredict done (8.25s): result: NodeOutput [12:00] [ComfyUI] [ComfyUI-GaussianPack] sys_path sent to worker: ['/workspaces/Sharp-1914/ComfyUI', '/workspaces/Sharp-1914/ComfyUI/custom_nodes/ComfyUI-GaussianPack', '/tmp/comfytest-home/.ce/.pixi/envs/gaussianpack-nodes/lib/python3.1/site-packages', '/workspaces/Sharp-1914/ComfyUI/custom_nodes/ComfyUI-GaussianPack'] [12:00] [ComfyUI] [VRAM] After SharpPredict: 12259 / 24122 MB [12:00] [ComfyUI] [SubprocessWorker] is_pixi=True, python=/tmp/comfytest-home/.ce/.pixi/envs/gaussianpack-nodes/bin/python [12:00] [ComfyUI] [VRAM] Loaded: SubprocessModel (2679 MB) [12:00] [ComfyUI] [SubprocessWorker] launching cmd=['/tmp/comfytest-home/.pixi/bin/pixi', 'run', '--as-is', '--manifest-path', '/tmp/comfytest-home/.ce/pixi.toml', '-e', 'gaussianpack-nodes', 'python', '/tmp/comfyui_pvenv_ii0358nh/persistent_worker.py', 'abstract://comfy_worker_029717b14254']... [12:00] [ComfyUI] [comfy-env] Running PreviewGaussianSharp: ply_path: '/workspaces/Sharp-1914/ComfyUI/output/sharp_178086041060..., extrinsics: list[4], intrinsics: list[3] [12:00] [ComfyUI] [SubprocessWorker] socket_addr=abstract://comfy_worker_029717b14254 (abstract, no filesystem path) [12:00] [ComfyUI] [VRAM] Before PreviewGaussianSharp: 12259 / 24122 MB [12:00] [ComfyUI] [VRAM] Loaded: SubprocessModel (2679 MB) [12:00] [ComfyUI] [SubprocessWorker] call_method: nodes.preview_gaussian_sharp.PreviewGaussianSharp.EXECUTE_NORMALIZED [12:00] [ComfyUI] [SubprocessWorker] acquired lock, ensuring started... [12:00] [ComfyUI] [ComfyUI-GaussianPack] Health check: ping (timeout=5.0s)... [12:00] [ComfyUI] [ComfyUI-GaussianPack] Health check: failed [12:00] [ComfyUI] [ComfyUI-GaussianPack] Socket unhealthy, restarting worker... [12:00] Frame 4 saved (frame_003.jpg, t=8.9s) [12:00] [capture-periodic] freeze=4ms shot=105ms unfreeze=2ms saved=True [12:01] [ComfyUI] [SubprocessWorker] worker started/confirmed [12:01] [ComfyUI] [SubprocessWorker] serializing kwargs to shm... [12:01] [ComfyUI] [SubprocessWorker] created 0 shm blocks [12:01] [ComfyUI] [SubprocessWorker] sending request via socket... [12:01] [ComfyUI] [SubprocessWorker] call_id=3 sending call_method [12:02] [ComfyUI] [worker:ComfyUI-GaussianPack] WARNING: SPZ route not registered (ModuleNotFoundError: No module named 'comfy_aimdo') [12:02] [ComfyUI] [SubprocessWorker] got response: ok [12:02] [ComfyUI] [comfy-env] PreviewGaussianSharp done (1.44s): result: NodeOutput [12:02] [ComfyUI] [ComfyUI-Sharp] python: /tmp/comfytest-home/.ce/.pixi/envs/sharp-nodes/bin/python [12:02] [ComfyUI] [VRAM] After PreviewGaussianSharp: 12259 / 24122 MB [12:02] [ComfyUI] [ComfyUI-Sharp] sys_path sent to worker: ['/workspaces/Sharp-1914/ComfyUI', '/workspaces/Sharp-1914/ComfyUI/custom_nodes/ComfyUI-Sharp', '/tmp/comfytest-home/.ce/.pixi/envs/sharp-nodes/lib/python3.1/site-packages', '/workspaces/Sharp-1914/ComfyUI/custom_nodes/ComfyUI-Sharp'] [12:02] [ComfyUI] [SubprocessWorker] is_pixi=True, python=/tmp/comfytest-home/.ce/.pixi/envs/sharp-nodes/bin/python [12:02] [ComfyUI] [VRAM] Loaded: SubprocessModel (2679 MB) [12:02] [ComfyUI] [SubprocessWorker] launching cmd=['/tmp/comfytest-home/.pixi/bin/pixi', 'run', '--as-is', '--manifest-path', '/tmp/comfytest-home/.ce/pixi.toml', '-e', 'sharp-nodes', 'python', '/tmp/comfyui_pvenv_qlpb5zhh/persistent_worker.py', 'abstract://comfy_worker_4a8b7d64501a']... [12:02] [ComfyUI] [comfy-env] Running SharpPredictMetricDepth: focal_length_mm: 30.0, model: dict, image: torch.float32 [1,960,1416,3] cpu [12:02] [ComfyUI] [SubprocessWorker] socket_addr=abstract://comfy_worker_4a8b7d64501a (abstract, no filesystem path) [12:02] [ComfyUI] [VRAM] Before SharpPredictMetricDepth: 12259 / 24122 MB [12:02] [ComfyUI] [VRAM] Loaded: SubprocessModel (2679 MB) [12:02] [ComfyUI] [SubprocessWorker] call_method: nodes.predict_metric_depth.SharpPredictMetricDepth.EXECUTE_NORMALIZED [12:02] [ComfyUI] [SubprocessWorker] acquired lock, ensuring started... [12:02] [ComfyUI] [ComfyUI-Sharp] Health check: ping (timeout=5.0s)... [12:02] [ComfyUI] [ComfyUI-Sharp] Health check: failed [12:02] [ComfyUI] [ComfyUI-Sharp] Socket unhealthy, restarting worker... [12:02] [ComfyUI] [comfy-env] Invalidated 1 stale model patchers (will be cleaned up during next unload) [12:02] Node executed (1 total), capturing... [12:03] [ComfyUI] [worker:ComfyUI-Sharp] ERROR: Failed to import comfy_kitchen, Error: No module named 'comfy_kitchen', fp8 and fp4 support will not be available. [12:03] [ComfyUI] [worker:ComfyUI-Sharp] WARNING: comfy_kitchen does not support stochastic FP8 rounding, please update comfy_kitchen. [12:03] [ComfyUI] [SubprocessWorker] worker started/confirmed [12:03] [ComfyUI] [comfy-env] PRE-SERIALIZE 'image' shape: torch.Size([1, 960, 1416, 3]) [12:03] Frame 5 saved (frame_004.jpg, t=11.2s) [12:03] [ComfyUI] [SubprocessWorker] serializing kwargs to shm... [12:03] [capture-node] freeze=142ms shot=129ms unfreeze=2ms saved=True [12:03] [ComfyUI] [SubprocessWorker] created 0 shm blocks [12:03] [ComfyUI] [SubprocessWorker] sending request via socket... [12:03] [ComfyUI] [SubprocessWorker] call_id=13 sending call_method [12:03] [ComfyUI] [worker:ComfyUI-Sharp] [comfy-env] _from_shm got dict with keys: ['__type__', 'strategy', 'parent_pid', 'fd', 'storage_size'] [12:03] [ComfyUI] [worker:ComfyUI-Sharp] [comfy-env] DESERIALIZE TensorRef: tensor_size=[1, 960, 1416, 3] [12:03] [ComfyUI] [worker:ComfyUI-Sharp] [comfy-env] DESERIALIZED tensor shape: torch.Size([1, 960, 1416, 3]) [12:06] [ComfyUI] [worker:ComfyUI-Sharp] [LoadSharpModel] loading checkpoint from /workspaces/Sharp-1914/ComfyUI/models/sharp/sharp_2572gikvuh.pt (dtype=bf16) [12:08] [ComfyUI] [worker:ComfyUI-Sharp] [LoadSharpModel] initializing model on meta device... [12:08] [ComfyUI] [worker:ComfyUI-Sharp] [LoadSharpModel] model ready (torch.bfloat16) [12:08] [ComfyUI] [worker:ComfyUI-Sharp] [SharpPredictMetricDepth] intrinsics not wired -> using identity-style K (focal=1186.2px @ image 1416×960, 35mm-equiv 30.0mm). Pass intrinsics from PanoramaSplit for per-face accurate K. [12:08] [ComfyUI] [SubprocessWorker] callback 'request_vram_budget' from call_id=13 [12:08] [ComfyUI] [comfy-env] VRAM request: 2.81GB, free before eviction: 20.24GB [12:08] [ComfyUI] [comfy-env] VRAM after eviction: 20.24GB [12:08] [ComfyUI] [worker:ComfyUI-Sharp] [VRAM] +443MB (now 443MB) reserved=448MB peak=443MB [12:09] [ComfyUI] [worker:ComfyUI-Sharp] [VRAM] +2859MB (now 3302MB) reserved=3456MB peak=3311MB [12:10] [ComfyUI] [worker:ComfyUI-Sharp] [VRAM] -330MB (now 2973MB) reserved=3360MB peak=3339MB [12:11] [ComfyUI] [worker:ComfyUI-Sharp] [VRAM] +1188MB (now 4161MB) reserved=7872MB peak=7051MB [12:12] [ComfyUI] [worker:ComfyUI-Sharp] [SharpPredictMetricDepth] 1 face(s) @ 1536²; layer0/layer1 depth median=2.12/2.12m, K_1536 fx->1286.7; points_raw ||v||=1.32–3.07m; 4.7s [12:12] [ComfyUI] [worker:ComfyUI-Sharp] [VRAM] -1474MB (now 2688MB) reserved=4384MB peak=7051MB [12:12] [ComfyUI] [SubprocessWorker] got response: ok [12:12] [ComfyUI] [comfy-env] [VRAM] Before offload 'RGBGaussianPredictor_1' (2679 MB) -> cpu: 9277 / 24122 MB [12:12] [ComfyUI] [ComfyUI-Sharp] Health check: ping (timeout=5.0s)... [12:12] [ComfyUI] [ComfyUI-Sharp] Health check: ok [12:12] [ComfyUI] [SubprocessWorker] call_id=14 sending model_to_device [12:13] [ComfyUI] [worker:ComfyUI-Sharp] [VRAM] -1534MB (now 1155MB) reserved=4384MB peak=7051MB [12:14] [ComfyUI] [comfy-env] [VRAM] After offload 'RGBGaussianPredictor_1' (2679 MB): 4989 / 24122 MB [12:14] [ComfyUI] [comfy-env] Created 1 model patchers: ['RGBGaussianPredictor_1'] [12:14] [ComfyUI] [comfy-env] SharpPredictMetricDepth done (12.77s): result: NodeOutput [12:14] [ComfyUI] [VRAM] After SharpPredictMetricDepth: 4989 / 24122 MB [12:14] [ComfyUI] [VRAM] Loaded: SubprocessModel (2679 MB) [12:14] [ComfyUI] [comfy-env] Running SharpPredictGaussiansFromMetricDepth: snap_to_external_depth: 'none', depth_convention: 'planar_z', focal_length_mm: 30.0, output_prefix: 'sharp_aligned', save_background_layer: True, edge_crop: 0.0, model: dict, image: torch.float32 [1,960,1416,3] cpu, metric_depth_layer0: torch.float32 [1,1536,1536,3] cpu [12:14] [ComfyUI] [VRAM] Before SharpPredictGaussiansFromMetricDepth: 4989 / 24122 MB [12:14] [ComfyUI] [VRAM] Loaded: SubprocessModel (2679 MB) [12:14] [ComfyUI] [SubprocessWorker] call_method: nodes.predict_gaussians_from_depth.SharpPredictGaussiansFromMetricDepth.EXECUTE_NORMALIZED [12:14] [ComfyUI] [SubprocessWorker] acquired lock, ensuring started... [12:14] [ComfyUI] [ComfyUI-Sharp] Health check: ping (timeout=5.0s)... [12:14] [ComfyUI] [ComfyUI-Sharp] Health check: ok [12:14] [ComfyUI] [SubprocessWorker] worker started/confirmed [12:14] [ComfyUI] [comfy-env] PRE-SERIALIZE 'image' shape: torch.Size([1, 960, 1416, 3]) [12:14] [ComfyUI] [comfy-env] PRE-SERIALIZE 'metric_depth_layer0' shape: torch.Size([1, 1536, 1536, 3]) [12:14] [ComfyUI] [SubprocessWorker] serializing kwargs to shm... [12:14] [ComfyUI] [SubprocessWorker] created 0 shm blocks [12:14] [ComfyUI] [SubprocessWorker] sending request via socket... [12:14] [ComfyUI] [SubprocessWorker] call_id=15 sending call_method [12:14] [ComfyUI] [worker:ComfyUI-Sharp] [comfy-env] _from_shm got dict with keys: ['__type__', 'strategy', 'parent_pid', 'fd', 'storage_size'] [12:14] [ComfyUI] [worker:ComfyUI-Sharp] [comfy-env] DESERIALIZE TensorRef: tensor_size=[1, 960, 1416, 3] [12:14] [ComfyUI] [worker:ComfyUI-Sharp] [comfy-env] DESERIALIZED tensor shape: torch.Size([1, 960, 1416, 3]) [12:14] [ComfyUI] [worker:ComfyUI-Sharp] [comfy-env] _from_shm got dict with keys: ['__type__', 'strategy', 'parent_pid', 'fd', 'storage_size'] [12:14] [ComfyUI] [worker:ComfyUI-Sharp] [comfy-env] DESERIALIZE TensorRef: tensor_size=[1, 1536, 1536, 3] [12:14] [ComfyUI] [worker:ComfyUI-Sharp] [comfy-env] DESERIALIZED tensor shape: torch.Size([1, 1536, 1536, 3]) [12:14] [ComfyUI] [worker:ComfyUI-Sharp] [VRAM] -1146MB (now 9MB) reserved=96MB peak=7051MB [12:14] [ComfyUI] [worker:ComfyUI-Sharp] [SharpPredictGaussiansFromMetricDepth] metric_depth_layer1 unwired -> layer 1 will duplicate layer 0 (legacy behavior) [12:14] [ComfyUI] [SubprocessWorker] callback 'request_vram_budget' from call_id=15 [12:15] [ComfyUI] [comfy-env] VRAM request: 2.81GB, free before eviction: 20.06GB [12:15] [ComfyUI] [comfy-env] VRAM after eviction: 20.06GB [12:15] [ComfyUI] [worker:ComfyUI-Sharp] [SharpPredictGaussiansFromMetricDepth] face 1/1: 1416×960, extrinsics=none, intrinsics=none, mask=no, snap=none, depth_conv=planar_z [12:15] [ComfyUI] [worker:ComfyUI-Sharp] [VRAM] +5538MB (now 5547MB) reserved=7456MB peak=7051MB [12:16] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=15 [12:16] [ComfyUI] [worker:ComfyUI-Sharp] [SharpPredictGaussiansFromMetricDepth] 1 face(s) -> 1.18M gaussians total (both layers, snap=none); saved to /workspaces/Sharp-1914/ComfyUI/output/sharp_aligned_1780860430286.ply; 1.1s [12:16] [ComfyUI] [SubprocessWorker] got response: ok [12:16] [ComfyUI] [comfy-env] SharpPredictGaussiansFromMetricDepth done (1.67s): result: NodeOutput [12:16] [ComfyUI] [VRAM] After SharpPredictGaussiansFromMetricDepth: 12387 / 24122 MB [12:16] [ComfyUI] [VRAM] Loaded: SubprocessModel (2679 MB) [12:16] [ComfyUI] [comfy-env] Running PreviewGaussianSharp: ply_path: '/workspaces/Sharp-1914/ComfyUI/output/sharp_aligned_1780..., extrinsics: list[4], intrinsics: list[3] [12:16] [ComfyUI] [VRAM] Before PreviewGaussianSharp: 12387 / 24122 MB [12:16] [ComfyUI] [VRAM] Loaded: SubprocessModel (2679 MB) [12:16] [ComfyUI] [SubprocessWorker] call_method: nodes.preview_gaussian_sharp.PreviewGaussianSharp.EXECUTE_NORMALIZED [12:16] [ComfyUI] [SubprocessWorker] acquired lock, ensuring started... [12:16] [ComfyUI] [ComfyUI-GaussianPack] Health check: ping (timeout=5.0s)... [12:16] [ComfyUI] [ComfyUI-GaussianPack] Health check: ok [12:16] [ComfyUI] [SubprocessWorker] worker started/confirmed [12:16] [ComfyUI] [SubprocessWorker] serializing kwargs to shm... [12:16] [ComfyUI] [SubprocessWorker] created 0 shm blocks [12:16] [ComfyUI] [SubprocessWorker] sending request via socket... [12:16] [ComfyUI] [SubprocessWorker] call_id=4 sending call_method [12:16] [ComfyUI] [SubprocessWorker] got response: ok [12:16] [ComfyUI] [comfy-env] PreviewGaussianSharp done (0.06s): result: NodeOutput [12:16] [ComfyUI] [VRAM] After PreviewGaussianSharp: 12387 / 24122 MB [12:16] [ComfyUI] [VRAM] Loaded: SubprocessModel (2679 MB) [12:16] [ComfyUI] [INFO] Prompt executed in 24.46 seconds [12:16] [ComfyUI] [comfy-env] [VRAM] Before offload 'RGBGaussianPredictor_1' (2679 MB) -> cpu: 12387 / 24122 MB [12:16] [ComfyUI] [ComfyUI-Sharp] Health check: ping (timeout=5.0s)... [12:16] [ComfyUI] [ComfyUI-Sharp] Health check: ok [12:16] [ComfyUI] [SubprocessWorker] call_id=16 sending model_to_device [12:17] [ComfyUI] [worker:ComfyUI-Sharp] [VRAM] -4874MB (now 674MB) reserved=7456MB peak=7051MB [12:17] [ComfyUI] [comfy-env] [VRAM] After offload 'RGBGaussianPredictor_1' (2679 MB): 5027 / 24122 MB [12:17] [ComfyUI] [INFO] Using RAM pressure cache. [12:29] Frame 6 saved (frame_005.jpg, t=20.0s) [12:29] [capture-periodic] freeze=8673ms shot=8653ms unfreeze=698ms saved=True [12:38] Node executed (2 total), capturing... [12:54] Frame 7 saved (frame_006.jpg, t=46.6s) [12:54] [capture-node] freeze=7706ms shot=7985ms unfreeze=0ms saved=True [12:54] Execution complete (t=46.6s) [12:54] [3d-wait] Phase 1: Waiting for 'Loading 3D Model' text to disappear... [12:54] [3d-wait] Phase 1 done in 0.0s [12:54] [3d-wait] Phase 2: iframes=2 [12:54] [3d-wait] iframe viewer_ready=True (matched 2 console logs) [12:54] [3d-wait] Unfrozen in 0.001s, rendering for 5s... [13:10] [3d-wait] JS halted via CDP after 15.9s [13:34] [3d-wait] Re-frozen in 40.2s [13:34] [3d-wait] Total 3D wait: 40.2s [13:34] Captured 7 unique frames over 102.54s [13:34] [debug] iframes: 2 [13:34] [debug] iframe[0]: src=http://127.0.0.1:41899/extensions/ComfyUI-GaussianPack/viewer_gaussian_sharp.htm, canvas=1416x960, rafFrozen=True [13:34] [debug] iframe[1]: src=http://127.0.0.1:41899/extensions/ComfyUI-GaussianPack/viewer_gaussian_sharp.htm, canvas=1416x960, rafFrozen=True [13:34] [timing] trigger_3d_previews: 0.0s [13:36] Saved high-quality screenshot: vanilla_vs_image_depth_gaussians_executed.png [13:36] [timing] final_screenshot: 0.3s [13:36] [ComfyUI] [INFO] Using RAM pressure cache. [13:37] Results saved to /logs/Sharp-1914/dev/linux-gpu/results.json [13:37] Model report: 8 files, 3.8 GB -> /logs/Sharp-1914/dev/linux-gpu/models.json [13:37] Saved: /logs/Sharp-1914/dev/linux-gpu/index.html [13:37] linux: FAILED [13:37] Error: Workflow execution failed (1 error(s)) [13:37] Details: pano.json: Workflow execution failed: {'prompt_id': '16f9d308-d076-4c5d-9a60-1719bd41817a', 'node_id': '87', 'node_type': 'PanoramaSplitAdaptive', 'executed': ['7', '81', '61', '96', '9', '70', '62', '59', '80', '60', '68'], 'exception_message': 'unable to allocate shared memory(shm) for file : Success (0)\n\nWorker traceback:\nTraceback (most recent call last):\n File "/tmp/comfyui_pvenv_0kgugsfd/persistent_worker.py", line 1772, in main\n result_meta = _to_shm(result, shm_registry)\n File "/tmp/comfyui_pvenv_0kgugsfd/persistent_worker.py", line 692, in _to_shm\n "args": _to_shm(list(obj.args), registry, visited),\n File "/tmp/comfyui_pvenv_0kgugsfd/persistent_worker.py", line 705, in _to_shm\n result = [_to_shm(v, registry, visited) for v in obj]\n File "/tmp/comfyui_pvenv_0kgugsfd/persistent_worker.py", line 705, in \n result = [_to_shm(v, registry, visited) for v in obj]\n File "/tmp/comfyui_pvenv_0kgugsfd/persistent_worker.py", line 628, in _to_shm\n result = _serialize_tensor_native(tensor, registry)\n File "/tmp/comfyui_pvenv_0kgugsfd/persistent_worker.py", line 563, in _serialize_tensor_native\n t.share_memory_()\n File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/site-packages/torch/_tensor.py", line 853, in share_memory_\n self._typed_storage()._share_memory_()\n File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/site-packages/torch/storage.py", line 1193, in _share_memory_\n self._untyped_storage.share_memory_()\n File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/site-packages/torch/storage.py", line 449, in wrapper\n return fn(self, *args, **kwargs)\n File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/site-packages/torch/storage.py", line 520, in share_memory_\n return super().share_memory_(*args, **kwargs)\n File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/site-packages/torch/storage.py", line 400, in share_memory_\n self._share_fd_cpu_()\n File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/site-packages/torch/storage.py", line 449, in wrapper\n return fn(self, *args, **kwargs)\n File "/tmp/comfytest-home/.ce/.pixi/envs/panopack-nodes/lib/python3.10/site-packages/torch/storage.py", line 524, in _share_fd_cpu_\n return super()._share_fd_cpu_(*args, **kwargs)\nRuntimeError: unable to allocate shared memory(shm) for file : Success (0)\n\n', 'exception_type': 'comfy_env.isolation.workers.base.WorkerError', 'traceback': [' File "/workspaces/Sharp-1914/ComfyUI/execution.py", line 536, in execute\n output_data, output_ui, has_subgraph, has_pending_tasks = await get_output_data(prompt_id, unique_id, obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)\n', ' File "/workspaces/Sharp-1914/ComfyUI/execution.py", line 336, in get_output_data\n return_values = await _async_map_node_over_list(prompt_id, unique_id, obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)\n', ' File "/workspaces/Sharp-1914/ComfyUI/execution.py", line 310, in _async_map_node_over_list\n await process_inputs(input_dict, i)\n', ' File "/workspaces/Sharp-1914/ComfyUI/execution.py", line 298, in process_inputs\n result = f(**inputs)\n', ' File "/workspaces/Sharp-1914/.venv/lib/python3.10/site-packages/comfy_env/isolation/metadata.py", line 556, in proxy\n result = worker.call_method(\n', ' File "/workspaces/Sharp-1914/.venv/lib/python3.10/site-packages/comfy_env/isolation/workers/subprocess.py", line 674, in call_method\n raise WorkerError(\n'], 'current_inputs': {'split_method': ['icosahedron_42'], 'resolution': [1536], 'fov_degrees': [90], 'use_gpu': [True], 'create_masks': [True], 'mask_method': ['closest_to'], 'panorama': ["{'image': tensor([[[[0.1647, 0.1020, 0.0745],\n [0.1765, 0.1098, 0.0863],\n [0.1765, 0.1137, 0.0941],\n ...,\n [0.1647, 0.1137, 0.0941],\n [0.1647, 0.1098, 0.0902],\n [0.1333, 0.0784, 0.0667]],\n\n [[0.1922, 0.1255, 0.0902],\n [0.1843, 0.1216, 0.0902],\n [0.1922, 0.1255, 0.0980],\n ...,\n [0.1765, 0.1255, 0.0980],\n [0.1725, 0.1255, 0.1020],\n [0.1647, 0.1059, 0.0824]],\n\n [[0.1882, 0.1216, 0.0980],\n [0.1843, 0.1216, 0.0980],\n [0.1882, 0.1216, 0.0980],\n ...,\n [0.1804, 0.1333, 0.1020],\n [0.1804, 0.1373, 0.1098],\n [0.1569, 0.1059, 0.0863]],\n\n ...,\n\n [[0.1882, 0.1255, 0.1176],\n [0.1922, 0.1255, 0.1176],\n [0.1922, 0.1255, 0.1176],\n ...,\n [0.1176, 0.1333, 0.1412],\n [0.1686, 0.1529, 0.1451],\n [0.1843, 0.1569, 0.1333]],\n\n [[0.1882, 0.1255, 0.1176],\n [0.1882, 0.1255, 0.1137],\n [0.1922, 0.1255, 0.1176],\n ...,\n [0.1647, 0.1412, 0.1098],\n [0.1922, 0.1529, 0.1176],\n [0.2039, 0.1490, 0.1294]],\n\n [[0.1765, 0.1137, 0.1137],\n [0.1804, 0.1255, 0.1216],\n [0.1765, 0.1216, 0.1137],\n ...,\n [0.1961, 0.1725, 0.1412],\n [0.1647, 0.1451, 0.1333],\n [0.1843, 0.1137, 0.1373]]]]), 'meta': {}}"], 'depth_panorama': ['tensor([[[[0.7430, 0.7430, 0.7430],\n [0.7430, 0.7430, 0.7430],\n [0.7430, 0.7430, 0.7430],\n ...,\n [0.7430, 0.7430, 0.7430],\n [0.7430, 0.7430, 0.7430],\n [0.7430, 0.7430, 0.7430]],\n\n [[0.7429, 0.7429, 0.7429],\n [0.7429, 0.7429, 0.7429],\n [0.7429, 0.7429, 0.7429],\n ...,\n [0.7429, 0.7429, 0.7429],\n [0.7429, 0.7429, 0.7429],\n [0.7429, 0.7429, 0.7429]],\n\n [[0.7430, 0.7430, 0.7430],\n [0.7430, 0.7430, 0.7430],\n [0.7430, 0.7430, 0.7430],\n ...,\n [0.7430, 0.7430, 0.7430],\n [0.7430, 0.7430, 0.7430],\n [0.7430, 0.7430, 0.7430]],\n\n ...,\n\n [[0.8810, 0.8810, 0.8810],\n [0.8810, 0.8810, 0.8810],\n [0.8810, 0.8810, 0.8810],\n ...,\n [0.8810, 0.8810, 0.8810],\n [0.8810, 0.8810, 0.8810],\n [0.8810, 0.8810, 0.8810]],\n\n [[0.8811, 0.8811, 0.8811],\n [0.8811, 0.8811, 0.8811],\n [0.8811, 0.8811, 0.8811],\n ...,\n [0.8811, 0.8811, 0.8811],\n [0.8811, 0.8811, 0.8811],\n [0.8811, 0.8811, 0.8811]],\n\n [[0.8813, 0.8813, 0.8813],\n [0.8813, 0.8813, 0.8813],\n [0.8813, 0.8813, 0.8813],\n ...,\n [0.8813, 0.8813, 0.8813],\n [0.8813, 0.8813, 0.8813],\n [0.8813, 0.8813, 0.8813]]]])'], 'normals_panorama': ['tensor([[[[4.8400e-01, 5.0399e-01, 2.7195e-04],\n [4.8401e-01, 5.0400e-01, 2.7180e-04],\n [4.8401e-01, 5.0400e-01, 2.7180e-04],\n ...,\n [4.8401e-01, 5.0399e-01, 2.7168e-04],\n [4.8401e-01, 5.0399e-01, 2.7168e-04],\n [4.8400e-01, 5.0399e-01, 2.7195e-04]],\n\n [[4.8384e-01, 5.0488e-01, 2.8512e-04],\n [4.8384e-01, 5.0488e-01, 2.8512e-04],\n [4.8384e-01, 5.0488e-01, 2.8512e-04],\n ...,\n [4.8384e-01, 5.0488e-01, 2.8512e-04],\n [4.8384e-01, 5.0488e-01, 2.8512e-04],\n [4.8384e-01, 5.0488e-01, 2.8512e-04]],\n\n [[4.8386e-01, 5.0571e-01, 2.9308e-04],\n [4.8386e-01, 5.0571e-01, 2.9308e-04],\n [4.8386e-01, 5.0571e-01, 2.9308e-04],\n ...,\n [4.8386e-01, 5.0571e-01, 2.9308e-04],\n [4.8386e-01, 5.0571e-01, 2.9308e-04],\n [4.8386e-01, 5.0571e-01, 2.9308e-04]],\n\n ...,\n\n [[4.9929e-01, 5.1112e-01, 9.9988e-01],\n [4.9929e-01, 5.1112e-01, 9.9988e-01],\n [4.9929e-01, 5.1112e-01, 9.9988e-01],\n ...,\n [4.9929e-01, 5.1112e-01, 9.9988e-01],\n [4.9929e-01, 5.1112e-01, 9.9988e-01],\n [4.9929e-01, 5.1112e-01, 9.9988e-01]],\n\n [[4.9880e-01, 5.1144e-01, 9.9987e-01],\n [4.9880e-01, 5.1144e-01, 9.9987e-01],\n [4.9880e-01, 5.1144e-01, 9.9987e-01],\n ...,\n [4.9880e-01, 5.1144e-01, 9.9987e-01],\n [4.9880e-01, 5.1144e-01, 9.9987e-01],\n [4.9880e-01, 5.1144e-01, 9.9987e-01]],\n\n [[4.9943e-01, 5.1242e-01, 9.9985e-01],\n [4.9943e-01, 5.1242e-01, 9.9985e-01],\n [4.9943e-01, 5.1242e-01, 9.9985e-01],\n ...,\n [4.9943e-01, 5.1242e-01, 9.9985e-01],\n [4.9943e-01, 5.1242e-01, 9.9985e-01],\n [4.9943e-01, 5.1242e-01, 9.9985e-01]]]])']}, 'current_outputs': ['7', '61', '81', '54', '96', '87', '9', '62', '70', '59', '76', '73', '80', '89', '95', '68', '78', '60'], 'timestamp': 1780860292737} Details: Workflow: /node/ComfyUI-Sharp/workflows/pano.json Node error: PanoramaSplitAdaptive [13:37] Stopping ComfyUI server...