78 KiB
78 KiB
[ComfyUI] [INFO] Using RAM pressure cache.
[ComfyUI] [INFO] Using RAM pressure cache.
Capturing execution frames: sharp_exif.json
[ComfyUI] [INFO] Using RAM pressure cache.
[ComfyUI] [INFO] Using RAM pressure cache.
Frame 1 saved (frame_000.jpg, t=0.0s)
Validating workflow...
[ComfyUI] [INFO] === /validate endpoint called ===
[ComfyUI] [INFO] === /validate endpoint called ===
[ComfyUI] [INFO] Received JSON with keys: ['prompt']
[ComfyUI] [INFO] Received JSON with keys: ['prompt']
[ComfyUI] [INFO] Prompt has 4 nodes
[ComfyUI] [INFO] Prompt has 4 nodes
[ComfyUI] [INFO] Node 2: LoadSharpModel
[ComfyUI] [INFO] Node 2: LoadSharpModel
[ComfyUI] [INFO] Inputs: {"precision": "auto"}
[ComfyUI] [INFO] Inputs: {"precision": "auto"}
[ComfyUI] [INFO] Node 7: SharpPredict
[ComfyUI] [INFO] Node 7: SharpPredict
[ComfyUI] [INFO] Inputs: {"focal_length_mm": ["9", 2], "output_prefix": "sharp", "model": ["2", 0], "image": ["9", 0]}
[ComfyUI] [INFO] Inputs: {"focal_length_mm": ["9", 2], "output_prefix": "sharp", "model": ["2", 0], "image": ["9", 0]}
[ComfyUI] [INFO] Node 9: LoadImageWithExif
[ComfyUI] [INFO] Node 9: LoadImageWithExif
[ComfyUI] [INFO] Inputs: {"image": "eth3d.png", "default_focal_mm": 30}
[ComfyUI] [INFO] Inputs: {"image": "eth3d.png", "default_focal_mm": 30}
[ComfyUI] [INFO] Node 11: PreviewGaussianSharp
[ComfyUI] [INFO] Node 11: PreviewGaussianSharp
[ComfyUI] [INFO] Inputs: {"preview_gaussian": "", "ply_path": ["7", 0], "extrinsics": ["7", 1], "intrinsics": ["7", 2]}
[ComfyUI] [INFO] Inputs: {"preview_gaussian": "", "ply_path": ["7", 0], "extrinsics": ["7", 1], "intrinsics": ["7", 2]}
[ComfyUI] [INFO] Generated prompt_id: 6b486fba-b946-4cbc-8766-4743667ede94
[ComfyUI] [INFO] Generated prompt_id: 6b486fba-b946-4cbc-8766-4743667ede94
[ComfyUI] [INFO] Calling execution.validate_prompt()...
[ComfyUI] [INFO] Calling execution.validate_prompt()...
[ComfyUI] [INFO] Validation result: valid=True
[ComfyUI] [INFO] Validation result: valid=True
[ComfyUI] [INFO] === Validation PASSED ===
[ComfyUI] [INFO] === Validation PASSED ===
Queuing workflow for execution...
[ComfyUI] [INFO] got prompt
[ComfyUI] [INFO] got prompt
[ComfyUI] [comfy-env] Running LoadImageWithExif: image: 'eth3d.png', default_focal_mm: 30.0
[ComfyUI] [comfy-env] Running LoadImageWithExif: image: 'eth3d.png', default_focal_mm: 30.0
[ComfyUI] [SubprocessWorker] call_method: nodes.load_image_exif.LoadImageWithExif.EXECUTE_NORMALIZED
[ComfyUI] [SubprocessWorker] call_method: nodes.load_image_exif.LoadImageWithExif.EXECUTE_NORMALIZED
[ComfyUI] [SubprocessWorker] acquired lock, ensuring started...
[ComfyUI] [SubprocessWorker] acquired lock, ensuring started...
[ComfyUI] [ComfyUI-Sharp] Health check: ping (timeout=5.0s)...
[ComfyUI] [ComfyUI-Sharp] Health check: ping (timeout=5.0s)...
[ComfyUI] [ComfyUI-Sharp] Health check: failed
[ComfyUI] [ComfyUI-Sharp] Health check: failed
[ComfyUI] [ComfyUI-Sharp] Socket unhealthy, restarting worker...
[ComfyUI] [ComfyUI-Sharp] Socket unhealthy, restarting worker...
[ComfyUI] [ComfyUI-Sharp] python: /Users/runner/.ce/.pixi/envs/sharp-nodes/bin/python
[ComfyUI] [ComfyUI-Sharp] python: /Users/runner/.ce/.pixi/envs/sharp-nodes/bin/python
[ComfyUI] [ComfyUI-Sharp] sys_path sent to worker: ['/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI', '/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp', '/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages', '/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp']
[ComfyUI] [ComfyUI-Sharp] sys_path sent to worker: ['/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI', '/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp', '/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages', '/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp']
[ComfyUI] [SubprocessWorker] is_pixi=True, python=/Users/runner/.ce/.pixi/envs/sharp-nodes/bin/python
[ComfyUI] [SubprocessWorker] is_pixi=True, python=/Users/runner/.ce/.pixi/envs/sharp-nodes/bin/python
[ComfyUI] [SubprocessWorker] launching cmd=['/Users/runner/.pixi/bin/pixi', 'run', '--as-is', '--manifest-path', '/Users/runner/.ce/pixi.toml', '-e', 'sharp-nodes', 'python', '/var/folders/8d/778wjbv96mq1760tv6gk374m0000gn/T/comfyui_pvenv_zcu5iw2_/persistent_worker.py', 'unix:///var/folders/8d/778wjbv96mq1760tv6gk374m0000gn/T/comfy_worker_be2eddec1a40.sock']...
[ComfyUI] [SubprocessWorker] launching cmd=['/Users/runner/.pixi/bin/pixi', 'run', '--as-is', '--manifest-path', '/Users/runner/.ce/pixi.toml', '-e', 'sharp-nodes', 'python', '/var/folders/8d/778wjbv96mq1760tv6gk374m0000gn/T/comfyui_pvenv_zcu5iw2_/persistent_worker.py', 'unix:///var/folders/8d/778wjbv96mq1760tv6gk374m0000gn/T/comfy_worker_be2eddec1a40.sock']...
[ComfyUI] [SubprocessWorker] socket_addr=unix:///var/folders/8d/778wjbv96mq1760tv6gk374m0000gn/T/comfy_worker_be2eddec1a40.sock exists=True
[ComfyUI] [SubprocessWorker] socket_addr=unix:///var/folders/8d/778wjbv96mq1760tv6gk374m0000gn/T/comfy_worker_be2eddec1a40.sock exists=True
Frame 2 saved (frame_001.jpg, t=2.5s)
[capture-periodic] freeze=6ms shot=244ms unfreeze=12ms saved=True
[ComfyUI] [worker:ComfyUI-Sharp] ERROR: Failed to import comfy_kitchen, Error: No module named 'comfy_kitchen', fp8 and fp4 support will not be available.
[ComfyUI] [worker:ComfyUI-Sharp] ERROR: Failed to import comfy_kitchen, Error: No module named 'comfy_kitchen', fp8 and fp4 support will not be available.
[ComfyUI] [worker:ComfyUI-Sharp] WARNING: comfy_kitchen does not support stochastic FP8 rounding, please update comfy_kitchen.
[ComfyUI] [worker:ComfyUI-Sharp] WARNING: comfy_kitchen does not support stochastic FP8 rounding, please update comfy_kitchen.
[ComfyUI] [SubprocessWorker] worker started/confirmed
[ComfyUI] [SubprocessWorker] worker started/confirmed
[ComfyUI] [SubprocessWorker] serializing kwargs to shm...
[ComfyUI] [SubprocessWorker] serializing kwargs to shm...
[ComfyUI] [SubprocessWorker] created 0 shm blocks
[ComfyUI] [SubprocessWorker] created 0 shm blocks
[ComfyUI] [SubprocessWorker] sending request via socket...
[ComfyUI] [SubprocessWorker] sending request via socket...
[ComfyUI] [SubprocessWorker] call_id=3 sending call_method
[ComfyUI] [SubprocessWorker] call_id=3 sending call_method
Frame 3 saved (frame_002.jpg, t=4.8s)
[capture-periodic] freeze=11ms shot=1239ms unfreeze=12ms saved=True
[capture-loop] iter=50 t=7.8s state={'complete': False, 'error': None, 'executedCount': 0, 'wsState': 1} eval_ms=4 ram=2.1/7.0GB (37%) swap=5.7/8.0GB (71%) cpu=65% browser_rss=0.53GB
[ComfyUI] [SubprocessWorker] got response: ok
[ComfyUI] [SubprocessWorker] got response: ok
[ComfyUI] [comfy-env] LoadImageWithExif done (7.97s): result: NodeOutput
[ComfyUI] [comfy-env] LoadImageWithExif done (7.97s): result: NodeOutput
[ComfyUI] [comfy-env] Running LoadSharpModel: precision: 'auto'
[ComfyUI] [comfy-env] Running LoadSharpModel: precision: 'auto'
[ComfyUI] [SubprocessWorker] call_method: nodes.load_model.LoadSharpModel.EXECUTE_NORMALIZED
[ComfyUI] [SubprocessWorker] call_method: nodes.load_model.LoadSharpModel.EXECUTE_NORMALIZED
[ComfyUI] [SubprocessWorker] acquired lock, ensuring started...
[ComfyUI] [SubprocessWorker] acquired lock, ensuring started...
[ComfyUI] [ComfyUI-Sharp] Health check: ping (timeout=5.0s)...
[ComfyUI] [ComfyUI-Sharp] Health check: ping (timeout=5.0s)...
[ComfyUI] [ComfyUI-Sharp] Health check: ok
[ComfyUI] [ComfyUI-Sharp] Health check: ok
[ComfyUI] [SubprocessWorker] worker started/confirmed
[ComfyUI] [SubprocessWorker] worker started/confirmed
[ComfyUI] [SubprocessWorker] serializing kwargs to shm...
[ComfyUI] [SubprocessWorker] serializing kwargs to shm...
[ComfyUI] [SubprocessWorker] created 0 shm blocks
[ComfyUI] [SubprocessWorker] created 0 shm blocks
[ComfyUI] [SubprocessWorker] sending request via socket...
[ComfyUI] [SubprocessWorker] sending request via socket...
[ComfyUI] [SubprocessWorker] call_id=4 sending call_method
[ComfyUI] [SubprocessWorker] call_id=4 sending call_method
[ComfyUI] [worker:ComfyUI-Sharp] [LoadSharpModel] precision='auto' -> dtype=bf16 (device=mps)
[ComfyUI] [worker:ComfyUI-Sharp] [LoadSharpModel] precision='auto' -> dtype=bf16 (device=mps)
[ComfyUI] [worker:ComfyUI-Sharp] [LoadSharpModel] checkpoint already present, reusing (no download): /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/models/sharp/sharp_2572gikvuh.pt (2679.6 MB)
[ComfyUI] [worker:ComfyUI-Sharp] [LoadSharpModel] checkpoint already present, reusing (no download): /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/models/sharp/sharp_2572gikvuh.pt (2679.6 MB)
[ComfyUI] [worker:ComfyUI-Sharp] [LoadSharpModel] checkpoint ready: /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/models/sharp/sharp_2572gikvuh.pt (2679.6 MB)
[ComfyUI] [worker:ComfyUI-Sharp] [LoadSharpModel] checkpoint ready: /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/models/sharp/sharp_2572gikvuh.pt (2679.6 MB)
[ComfyUI] [worker:ComfyUI-Sharp] [LoadSharpModel] config: precision=auto -> dtype=bf16
[ComfyUI] [worker:ComfyUI-Sharp] [LoadSharpModel] config: precision=auto -> dtype=bf16
[ComfyUI] [SubprocessWorker] got response: ok
[ComfyUI] [SubprocessWorker] got response: ok
[ComfyUI] [comfy-env] LoadSharpModel done (0.05s): result: NodeOutput
[ComfyUI] [comfy-env] LoadSharpModel done (0.05s): result: NodeOutput
[ComfyUI] [comfy-env] Running SharpPredict: focal_length_mm: 20.0, output_prefix: 'sharp', model: dict, image: torch.float32 [1,2022,3040,3] cpu
[ComfyUI] [comfy-env] Running SharpPredict: focal_length_mm: 20.0, output_prefix: 'sharp', model: dict, image: torch.float32 [1,2022,3040,3] cpu
[ComfyUI] [SubprocessWorker] call_method: nodes.predict.SharpPredict.EXECUTE_NORMALIZED
[ComfyUI] [SubprocessWorker] call_method: nodes.predict.SharpPredict.EXECUTE_NORMALIZED
[ComfyUI] [SubprocessWorker] acquired lock, ensuring started...
[ComfyUI] [SubprocessWorker] acquired lock, ensuring started...
[ComfyUI] [ComfyUI-Sharp] Health check: ping (timeout=5.0s)...
[ComfyUI] [ComfyUI-Sharp] Health check: ping (timeout=5.0s)...
[ComfyUI] [ComfyUI-Sharp] Health check: ok
[ComfyUI] [ComfyUI-Sharp] Health check: ok
[ComfyUI] [SubprocessWorker] worker started/confirmed
[ComfyUI] [SubprocessWorker] worker started/confirmed
[ComfyUI] [comfy-env] PRE-SERIALIZE 'image' shape: torch.Size([1, 2022, 3040, 3])
[ComfyUI] [comfy-env] PRE-SERIALIZE 'image' shape: torch.Size([1, 2022, 3040, 3])
[ComfyUI] [SubprocessWorker] serializing kwargs to shm...
[ComfyUI] [SubprocessWorker] serializing kwargs to shm...
[ComfyUI] [SubprocessWorker] created 0 shm blocks
[ComfyUI] [SubprocessWorker] created 0 shm blocks
[ComfyUI] [SubprocessWorker] sending request via socket...
[ComfyUI] [SubprocessWorker] sending request via socket...
[ComfyUI] [SubprocessWorker] call_id=5 sending call_method
[ComfyUI] [SubprocessWorker] call_id=5 sending call_method
[ComfyUI] [worker:ComfyUI-Sharp] [comfy-env] _from_shm got dict with keys: ['__type__', 'strategy', 'manager_path', 'storage_key', 'storage_size']
[ComfyUI] [worker:ComfyUI-Sharp] [comfy-env] _from_shm got dict with keys: ['__type__', 'strategy', 'manager_path', 'storage_key', 'storage_size']
[ComfyUI] [worker:ComfyUI-Sharp] [comfy-env] DESERIALIZE TensorRef: tensor_size=[1, 2022, 3040, 3]
[ComfyUI] [worker:ComfyUI-Sharp] [comfy-env] DESERIALIZE TensorRef: tensor_size=[1, 2022, 3040, 3]
[ComfyUI] [worker:ComfyUI-Sharp] [comfy-env] DESERIALIZED tensor shape: torch.Size([1, 2022, 3040, 3])
[ComfyUI] [worker:ComfyUI-Sharp] [comfy-env] DESERIALIZED tensor shape: torch.Size([1, 2022, 3040, 3])
Frame 4 saved (frame_003.jpg, t=8.2s)
[capture-periodic] freeze=5ms shot=246ms unfreeze=6ms saved=True
Frame 5 saved (frame_004.jpg, t=10.6s)
[capture-periodic] freeze=4ms shot=122ms unfreeze=4ms saved=True
[ComfyUI] objc[20942]: Class AVFFrameReceiver is implemented in both /Users/runner/.ce/.pixi/envs/sharp-nodes/lib/libavdevice.62.3.101.dylib (0x1283447a8) and /Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/cv2/.dylibs/libavdevice.61.3.100.dylib (0x15ef303a8). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed.
[ComfyUI] objc[20942]: Class AVFFrameReceiver is implemented in both /Users/runner/.ce/.pixi/envs/sharp-nodes/lib/libavdevice.62.3.101.dylib (0x1283447a8) and /Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/cv2/.dylibs/libavdevice.61.3.100.dylib (0x15ef303a8). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed.
[ComfyUI] objc[20942]: Class AVFAudioReceiver is implemented in both /Users/runner/.ce/.pixi/envs/sharp-nodes/lib/libavdevice.62.3.101.dylib (0x1283447f8) and /Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/cv2/.dylibs/libavdevice.61.3.100.dylib (0x15ef303f8). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed.
[ComfyUI] objc[20942]: Class AVFAudioReceiver is implemented in both /Users/runner/.ce/.pixi/envs/sharp-nodes/lib/libavdevice.62.3.101.dylib (0x1283447f8) and /Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/cv2/.dylibs/libavdevice.61.3.100.dylib (0x15ef303f8). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed.
[ComfyUI] [worker:ComfyUI-Sharp] [LoadSharpModel] loading checkpoint from /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/models/sharp/sharp_2572gikvuh.pt (dtype=bf16)
[ComfyUI] [worker:ComfyUI-Sharp] [LoadSharpModel] loading checkpoint from /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/models/sharp/sharp_2572gikvuh.pt (dtype=bf16)
[capture-loop] iter=100 t=14.1s state={'complete': False, 'error': None, 'executedCount': 0, 'wsState': 1} eval_ms=72 ram=2.6/7.0GB (44%) swap=3.5/8.0GB (44%) cpu=64% browser_rss=0.54GB
Frame 6 saved (frame_005.jpg, t=17.3s)
[capture-periodic] freeze=4ms shot=458ms unfreeze=13ms saved=True
[capture-loop] iter=150 t=21.4s state={'complete': False, 'error': None, 'executedCount': 0, 'wsState': 1} eval_ms=4 ram=2.7/7.0GB (76%) swap=0.6/2.0GB (30%) cpu=88% browser_rss=0.44GB
[ComfyUI] [worker:ComfyUI-Sharp] [LoadSharpModel] initializing model on meta device...
[ComfyUI] [worker:ComfyUI-Sharp] [LoadSharpModel] initializing model on meta device...
[ComfyUI] [worker:ComfyUI-Sharp] [LoadSharpModel] model ready (torch.bfloat16)
[ComfyUI] [worker:ComfyUI-Sharp] [LoadSharpModel] model ready (torch.bfloat16)
[ComfyUI] [SubprocessWorker] callback 'request_vram_budget' from call_id=5
[ComfyUI] [SubprocessWorker] callback 'request_vram_budget' from call_id=5
[ComfyUI] [comfy-env] VRAM request: 2.81GB, free before eviction: 1.80GB
[ComfyUI] [comfy-env] VRAM request: 2.81GB, free before eviction: 1.80GB
[ComfyUI] [comfy-env] VRAM after eviction: 1.80GB
[ComfyUI] [comfy-env] VRAM after eviction: 1.80GB
Frame 7 saved (frame_006.jpg, t=26.3s)
[capture-periodic] freeze=3ms shot=260ms unfreeze=12ms saved=True
[capture-loop] iter=200 t=29.8s state={'complete': False, 'error': None, 'executedCount': 0, 'wsState': 1} eval_ms=6 ram=4.5/7.0GB (79%) swap=0.6/2.0GB (30%) cpu=73% browser_rss=0.27GB
Frame 8 saved (frame_007.jpg, t=33.8s)
[capture-periodic] freeze=2ms shot=7203ms unfreeze=24ms saved=True
Frame 9 saved (frame_008.jpg, t=43.2s)
[capture-periodic] freeze=2ms shot=3247ms unfreeze=3ms saved=True
[capture-loop] iter=250 t=46.6s state={'complete': False, 'error': None, 'executedCount': 0, 'wsState': 1} eval_ms=5 ram=5.8/7.0GB (90%) swap=0.9/2.0GB (45%) cpu=79% browser_rss=0.24GB
Frame 10 saved (frame_009.jpg, t=53.8s)
[capture-periodic] freeze=97ms shot=5990ms unfreeze=5ms saved=True
[ComfyUI] [worker:ComfyUI-Sharp]
[ComfyUI] [worker:ComfyUI-Sharp]
[ComfyUI] === WATCHDOG TICK 1 ===
[ComfyUI] === WATCHDOG TICK 1 ===
[ComfyUI] [worker:ComfyUI-Sharp] Current thread 0x000000016c1f7000 (most recent call first):
[ComfyUI] [worker:ComfyUI-Sharp] Current thread 0x000000016c1f7000 (most recent call first):
[ComfyUI] File "/var/folders/8d/778wjbv96mq1760tv6gk374m0000gn/T/comfyui_pvenv_zcu5iw2_/persistent_worker.py", line 52 in _watchdog
[ComfyUI] File "/var/folders/8d/778wjbv96mq1760tv6gk374m0000gn/T/comfyui_pvenv_zcu5iw2_/persistent_worker.py", line 52 in _watchdog
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/threading.py", line 982 in run
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/threading.py", line 982 in run
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/threading.py", line 1045 in _bootstrap_inner
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/threading.py", line 1045 in _bootstrap_inner
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/threading.py", line 1002 in _bootstrap
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/threading.py", line 1002 in _bootstrap
[ComfyUI]
[ComfyUI]
[ComfyUI] Thread 0x00000001efc5a2c0 (most recent call first):
[ComfyUI] Thread 0x00000001efc5a2c0 (most recent call first):
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/conv.py", line 548 in _conv_forward
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/conv.py", line 548 in _conv_forward
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/conv.py", line 553 in forward
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/conv.py", line 553 in forward
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/comfy/ops.py", line 537 in forward
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/comfy/ops.py", line 537 in forward
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1787 in _call_impl
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1787 in _call_impl
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1776 in _wrapped_call_impl
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1776 in _wrapped_call_impl
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/sharp/model.py", line 969 in forward
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/sharp/model.py", line 969 in forward
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1787 in _call_impl
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1787 in _call_impl
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1776 in _wrapped_call_impl
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1776 in _wrapped_call_impl
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/sharp/model.py", line 1118 in forward
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/sharp/model.py", line 1118 in forward
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1787 in _call_impl
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1787 in _call_impl
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1776 in _wrapped_call_impl
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1776 in _wrapped_call_impl
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/sharp/model.py", line 2006 in encode
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/sharp/model.py", line 2006 in encode
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/predict.py", line 348 in _predict_image_cached
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/predict.py", line 348 in _predict_image_cached
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/predict.py", line 239 in execute
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/predict.py", line 239 in execute
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 124 in decorate_context
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 124 in decorate_context
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/comfy_api/latest/_io.py", line 1892 in EXECUTE_NORMALIZED
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/comfy_api/latest/_io.py", line 1892 in EXECUTE_NORMALIZED
[ComfyUI] File "/var/folders/8d/778wjbv96mq1760tv6gk374m0000gn/T/comfyui_pvenv_zcu5iw2_/persistent_worker.py", line 1762 in main
[ComfyUI] File "/var/folders/8d/778wjbv96mq1760tv6gk374m0000gn/T/comfyui_pvenv_zcu5iw2_/persistent_worker.py", line 1762 in main
[ComfyUI] File "/var/folders/8d/778wjbv96mq1760tv6gk374m0000gn/T/comfyui_pvenv_zcu5iw2_/persistent_worker.py", line 1807 in <module>
[ComfyUI] File "/var/folders/8d/778wjbv96mq1760tv6gk374m0000gn/T/comfyui_pvenv_zcu5iw2_/persistent_worker.py", line 1807 in <module>
[ComfyUI]
[ComfyUI]
[ComfyUI] [worker:ComfyUI-Sharp] === END ===
[ComfyUI] [worker:ComfyUI-Sharp] === END ===
[ComfyUI]
[ComfyUI]
[capture-loop] iter=300 t=61.4s state={'complete': False, 'error': None, 'executedCount': 0, 'wsState': 1} eval_ms=16 ram=5.2/7.0GB (84%) swap=1.2/2.0GB (62%) cpu=77% browser_rss=0.20GB
[ComfyUI] [SubprocessWorker] got response: error
[ComfyUI] [SubprocessWorker] got response: error
[ComfyUI] [ERROR] !!! Exception during processing !!! MPS backend out of memory (MPS allocated: 6.73 GiB, other allocations: 581.22 MiB, max allowed: 7.93 GiB). Tried to allocate 576.00 MiB on private pool. Use PYTORCH_MPS_HIGH_WATERMARK_RATIO=0.0 to disable upper limit for memory allocations (may cause system failure).
[ComfyUI] [ERROR] !!! Exception during processing !!! MPS backend out of memory (MPS allocated: 6.73 GiB, other allocations: 581.22 MiB, max allowed: 7.93 GiB). Tried to allocate 576.00 MiB on private pool. Use PYTORCH_MPS_HIGH_WATERMARK_RATIO=0.0 to disable upper limit for memory allocations (may cause system failure).
[ComfyUI]
[ComfyUI]
[ComfyUI] Worker traceback:
[ComfyUI] Worker traceback:
[ComfyUI] Traceback (most recent call last):
[ComfyUI] Traceback (most recent call last):
[ComfyUI] File "/var/folders/8d/778wjbv96mq1760tv6gk374m0000gn/T/comfyui_pvenv_zcu5iw2_/persistent_worker.py", line 1762, in main
[ComfyUI] File "/var/folders/8d/778wjbv96mq1760tv6gk374m0000gn/T/comfyui_pvenv_zcu5iw2_/persistent_worker.py", line 1762, in main
[ComfyUI] result = method(**inputs)
[ComfyUI] result = method(**inputs)
[ComfyUI] ^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/comfy_api/latest/_io.py", line 1892, in EXECUTE_NORMALIZED
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/comfy_api/latest/_io.py", line 1892, in EXECUTE_NORMALIZED
[ComfyUI] to_return = cls.execute(*args, **kwargs)
[ComfyUI] to_return = cls.execute(*args, **kwargs)
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 124, in decorate_context
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 124, in decorate_context
[ComfyUI] return func(*args, **kwargs)
[ComfyUI] return func(*args, **kwargs)
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/predict.py", line 239, in execute
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/predict.py", line 239, in execute
[ComfyUI] gaussians, refined_depth_l0, refined_depth_l1 = _predict_image_cached(
[ComfyUI] gaussians, refined_depth_l0, refined_depth_l1 = _predict_image_cached(
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/predict.py", line 348, in _predict_image_cached
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/predict.py", line 348, in _predict_image_cached
[ComfyUI] monodepth_output, _ = predictor.encode(image_resized_pt)
[ComfyUI] monodepth_output, _ = predictor.encode(image_resized_pt)
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/sharp/model.py", line 2006, in encode
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/sharp/model.py", line 2006, in encode
[ComfyUI] monodepth_output = self.monodepth_model(image)
[ComfyUI] monodepth_output = self.monodepth_model(image)
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
[ComfyUI] return self._call_impl(*args, **kwargs)
[ComfyUI] return self._call_impl(*args, **kwargs)
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl
[ComfyUI] return forward_call(*args, **kwargs)
[ComfyUI] return forward_call(*args, **kwargs)
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/sharp/model.py", line 1118, in forward
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/sharp/model.py", line 1118, in forward
[ComfyUI] decoder_features = self.monodepth_predictor.decoder(encoder_features)
[ComfyUI] decoder_features = self.monodepth_predictor.decoder(encoder_features)
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
[ComfyUI] return self._call_impl(*args, **kwargs)
[ComfyUI] return self._call_impl(*args, **kwargs)
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl
[ComfyUI] return forward_call(*args, **kwargs)
[ComfyUI] return forward_call(*args, **kwargs)
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/sharp/model.py", line 970, in forward
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/sharp/model.py", line 970, in forward
[ComfyUI] features = self.fusions[i](features, features_i)
[ComfyUI] features = self.fusions[i](features, features_i)
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
[ComfyUI] return self._call_impl(*args, **kwargs)
[ComfyUI] return self._call_impl(*args, **kwargs)
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl
[ComfyUI] return forward_call(*args, **kwargs)
[ComfyUI] return forward_call(*args, **kwargs)
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/sharp/model.py", line 862, in forward
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/sharp/model.py", line 862, in forward
[ComfyUI] x = self.out_conv(x)
[ComfyUI] x = self.out_conv(x)
[ComfyUI] ^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
[ComfyUI] return self._call_impl(*args, **kwargs)
[ComfyUI] return self._call_impl(*args, **kwargs)
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl
[ComfyUI] return forward_call(*args, **kwargs)
[ComfyUI] return forward_call(*args, **kwargs)
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/comfy/ops.py", line 537, in forward
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/comfy/ops.py", line 537, in forward
[ComfyUI] return super().forward(*args, **kwargs)
[ComfyUI] return super().forward(*args, **kwargs)
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/conv.py", line 553, in forward
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/conv.py", line 553, in forward
[ComfyUI] return self._conv_forward(input, self.weight, self.bias)
[ComfyUI] return self._conv_forward(input, self.weight, self.bias)
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/conv.py", line 548, in _conv_forward
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/conv.py", line 548, in _conv_forward
[ComfyUI] return F.conv2d(
[ComfyUI] return F.conv2d(
[ComfyUI] ^^^^^^^^^
[ComfyUI] ^^^^^^^^^
[ComfyUI] RuntimeError: MPS backend out of memory (MPS allocated: 6.73 GiB, other allocations: 581.22 MiB, max allowed: 7.93 GiB). Tried to allocate 576.00 MiB on private pool. Use PYTORCH_MPS_HIGH_WATERMARK_RATIO=0.0 to disable upper limit for memory allocations (may cause system failure).
[ComfyUI] RuntimeError: MPS backend out of memory (MPS allocated: 6.73 GiB, other allocations: 581.22 MiB, max allowed: 7.93 GiB). Tried to allocate 576.00 MiB on private pool. Use PYTORCH_MPS_HIGH_WATERMARK_RATIO=0.0 to disable upper limit for memory allocations (may cause system failure).
[ComfyUI]
[ComfyUI]
[ComfyUI] [ERROR] Traceback (most recent call last):
[ComfyUI] [ERROR] Traceback (most recent call last):
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/execution.py", line 536, in execute
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/execution.py", line 536, in execute
[ComfyUI] output_data, output_ui, has_subgraph, has_pending_tasks = await get_output_data(prompt_id, unique_id, obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)
[ComfyUI] output_data, output_ui, has_subgraph, has_pending_tasks = await get_output_data(prompt_id, unique_id, obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/execution.py", line 336, in get_output_data
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/execution.py", line 336, in get_output_data
[ComfyUI] return_values = await _async_map_node_over_list(prompt_id, unique_id, obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)
[ComfyUI] return_values = await _async_map_node_over_list(prompt_id, unique_id, obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/execution.py", line 310, in _async_map_node_over_list
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/execution.py", line 310, in _async_map_node_over_list
[ComfyUI] await process_inputs(input_dict, i)
[ComfyUI] await process_inputs(input_dict, i)
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/execution.py", line 298, in process_inputs
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/execution.py", line 298, in process_inputs
[ComfyUI] result = f(**inputs)
[ComfyUI] result = f(**inputs)
[ComfyUI] ^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/venv/lib/python3.11/site-packages/comfy_env/isolation/metadata.py", line 556, in proxy
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/venv/lib/python3.11/site-packages/comfy_env/isolation/metadata.py", line 556, in proxy
[ComfyUI] result = worker.call_method(
[ComfyUI] result = worker.call_method(
[ComfyUI] ^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/venv/lib/python3.11/site-packages/comfy_env/isolation/workers/subprocess.py", line 674, in call_method
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/venv/lib/python3.11/site-packages/comfy_env/isolation/workers/subprocess.py", line 674, in call_method
[ComfyUI] raise WorkerError(
[ComfyUI] raise WorkerError(
[ComfyUI] comfy_env.isolation.workers.base.WorkerError: MPS backend out of memory (MPS allocated: 6.73 GiB, other allocations: 581.22 MiB, max allowed: 7.93 GiB). Tried to allocate 576.00 MiB on private pool. Use PYTORCH_MPS_HIGH_WATERMARK_RATIO=0.0 to disable upper limit for memory allocations (may cause system failure).
[ComfyUI] comfy_env.isolation.workers.base.WorkerError: MPS backend out of memory (MPS allocated: 6.73 GiB, other allocations: 581.22 MiB, max allowed: 7.93 GiB). Tried to allocate 576.00 MiB on private pool. Use PYTORCH_MPS_HIGH_WATERMARK_RATIO=0.0 to disable upper limit for memory allocations (may cause system failure).
[ComfyUI]
[ComfyUI]
[ComfyUI] Worker traceback:
[ComfyUI] Worker traceback:
[ComfyUI] Traceback (most recent call last):
[ComfyUI] Traceback (most recent call last):
[ComfyUI] File "/var/folders/8d/778wjbv96mq1760tv6gk374m0000gn/T/comfyui_pvenv_zcu5iw2_/persistent_worker.py", line 1762, in main
[ComfyUI] File "/var/folders/8d/778wjbv96mq1760tv6gk374m0000gn/T/comfyui_pvenv_zcu5iw2_/persistent_worker.py", line 1762, in main
[ComfyUI] result = method(**inputs)
[ComfyUI] result = method(**inputs)
[ComfyUI] ^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/comfy_api/latest/_io.py", line 1892, in EXECUTE_NORMALIZED
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/comfy_api/latest/_io.py", line 1892, in EXECUTE_NORMALIZED
[ComfyUI] to_return = cls.execute(*args, **kwargs)
[ComfyUI] to_return = cls.execute(*args, **kwargs)
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 124, in decorate_context
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 124, in decorate_context
[ComfyUI] return func(*args, **kwargs)
[ComfyUI] return func(*args, **kwargs)
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/predict.py", line 239, in execute
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/predict.py", line 239, in execute
[ComfyUI] gaussians, refined_depth_l0, refined_depth_l1 = _predict_image_cached(
[ComfyUI] gaussians, refined_depth_l0, refined_depth_l1 = _predict_image_cached(
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/predict.py", line 348, in _predict_image_cached
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/predict.py", line 348, in _predict_image_cached
[ComfyUI] monodepth_output, _ = predictor.encode(image_resized_pt)
[ComfyUI] monodepth_output, _ = predictor.encode(image_resized_pt)
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/sharp/model.py", line 2006, in encode
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/sharp/model.py", line 2006, in encode
[ComfyUI] monodepth_output = self.monodepth_model(image)
[ComfyUI] monodepth_output = self.monodepth_model(image)
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
[ComfyUI] return self._call_impl(*args, **kwargs)
[ComfyUI] return self._call_impl(*args, **kwargs)
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl
[ComfyUI] return forward_call(*args, **kwargs)
[ComfyUI] return forward_call(*args, **kwargs)
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/sharp/model.py", line 1118, in forward
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/sharp/model.py", line 1118, in forward
[ComfyUI] decoder_features = self.monodepth_predictor.decoder(encoder_features)
[ComfyUI] decoder_features = self.monodepth_predictor.decoder(encoder_features)
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
[ComfyUI] return self._call_impl(*args, **kwargs)
[ComfyUI] return self._call_impl(*args, **kwargs)
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl
[ComfyUI] return forward_call(*args, **kwargs)
[ComfyUI] return forward_call(*args, **kwargs)
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/sharp/model.py", line 970, in forward
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/sharp/model.py", line 970, in forward
[ComfyUI] features = self.fusions[i](features, features_i)
[ComfyUI] features = self.fusions[i](features, features_i)
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
[ComfyUI] return self._call_impl(*args, **kwargs)
[ComfyUI] return self._call_impl(*args, **kwargs)
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl
[ComfyUI] return forward_call(*args, **kwargs)
[ComfyUI] return forward_call(*args, **kwargs)
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/sharp/model.py", line 862, in forward
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/sharp/model.py", line 862, in forward
[ComfyUI] x = self.out_conv(x)
[ComfyUI] x = self.out_conv(x)
[ComfyUI] ^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
[ComfyUI] return self._call_impl(*args, **kwargs)
[ComfyUI] return self._call_impl(*args, **kwargs)
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl
[ComfyUI] return forward_call(*args, **kwargs)
[ComfyUI] return forward_call(*args, **kwargs)
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/comfy/ops.py", line 537, in forward
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/comfy/ops.py", line 537, in forward
[ComfyUI] return super().forward(*args, **kwargs)
[ComfyUI] return super().forward(*args, **kwargs)
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/conv.py", line 553, in forward
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/conv.py", line 553, in forward
[ComfyUI] return self._conv_forward(input, self.weight, self.bias)
[ComfyUI] return self._conv_forward(input, self.weight, self.bias)
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/conv.py", line 548, in _conv_forward
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/conv.py", line 548, in _conv_forward
[ComfyUI] return F.conv2d(
[ComfyUI] return F.conv2d(
[ComfyUI] ^^^^^^^^^
[ComfyUI] ^^^^^^^^^
[ComfyUI] RuntimeError: MPS backend out of memory (MPS allocated: 6.73 GiB, other allocations: 581.22 MiB, max allowed: 7.93 GiB). Tried to allocate 576.00 MiB on private pool. Use PYTORCH_MPS_HIGH_WATERMARK_RATIO=0.0 to disable upper limit for memory allocations (may cause system failure).
[ComfyUI] RuntimeError: MPS backend out of memory (MPS allocated: 6.73 GiB, other allocations: 581.22 MiB, max allowed: 7.93 GiB). Tried to allocate 576.00 MiB on private pool. Use PYTORCH_MPS_HIGH_WATERMARK_RATIO=0.0 to disable upper limit for memory allocations (may cause system failure).
[ComfyUI]
[ComfyUI]
[ComfyUI]
[ComfyUI]
[ComfyUI] [INFO] Prompt executed in 86.82 seconds
[ComfyUI] [INFO] Prompt executed in 86.82 seconds
Execution complete (t=87.9s)
Execution error: {'prompt_id': '8a0433fa-7d52-4d8b-b620-62151ba4d2b5', 'node_id': '7', 'node_type': 'SharpPredict', 'executed': ['2', '9'], 'exception_message': 'MPS backend out of memory (MPS allocated: 6.73 GiB, other allocations: 581.22 MiB, max allowed: 7.93 GiB). Tried to allocate 576.00 MiB on private pool. Use PYTORCH_MPS_HIGH_WATERMARK_RATIO=0.0 to disable upper limit for memory allocations (may cause system failure).\n\nWorker traceback:\nTraceback (most recent call last):\n File "/var/folders/8d/778wjbv96mq1760tv6gk374m0000gn/T/comfyui_pvenv_zcu5iw2_/persistent_worker.py", line 1762, in main\n result = method(**inputs)\n ^^^^^^^^^^^^^^^^\n File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/comfy_api/latest/_io.py", line 1892, in EXECUTE_NORMALIZED\n to_return = cls.execute(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 124, in decorate_context\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/predict.py", line 239, in execute\n gaussians, refined_depth_l0, refined_depth_l1 = _predict_image_cached(\n ^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/predict.py", line 348, in _predict_image_cached\n monodepth_output, _ = predictor.encode(image_resized_pt)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/sharp/model.py", line 2006, in encode\n monodepth_output = self.monodepth_model(image)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl\n return self._call_impl(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl\n return forward_call(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/sharp/model.py", line 1118, in forward\n decoder_features = self.monodepth_predictor.decoder(encoder_features)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl\n return self._call_impl(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl\n return forward_call(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/sharp/model.py", line 970, in forward\n features = self.fusions[i](features, features_i)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl\n return self._call_impl(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl\n return forward_call(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/sharp/model.py", line 862, in forward\n x = self.out_conv(x)\n ^^^^^^^^^^^^^^^^\n File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl\n return self._call_impl(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl\n return forward_call(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/comfy/ops.py", line 537, in forward\n return super().forward(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/conv.py", line 553, in forward\n return self._conv_forward(input, self.weight, self.bias)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/conv.py", line 548, in _conv_forward\n return F.conv2d(\n ^^^^^^^^^\nRuntimeError: MPS backend out of memory (MPS allocated: 6.73 GiB, other allocations: 581.22 MiB, max allowed: 7.93 GiB). Tried to allocate 576.00 MiB on private pool. Use PYTORCH_MPS_HIGH_WATERMARK_RATIO=0.0 to disable upper limit for memory allocations (may cause system failure).\n\n', 'exception_type': 'comfy_env.isolation.workers.base.WorkerError', 'traceback': [' File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/execution.py", line 536, in execute\n output_data, output_ui, has_subgraph, has_pending_tasks = await get_output_data(prompt_id, unique_id, obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n', ' File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/execution.py", line 336, in get_output_data\n return_values = await _async_map_node_over_list(prompt_id, unique_id, obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n', ' File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/execution.py", line 310, in _async_map_node_over_list\n await process_inputs(input_dict, i)\n', ' File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/execution.py", line 298, in process_inputs\n result = f(**inputs)\n ^^^^^^^^^^^\n', ' File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/venv/lib/python3.11/site-packages/comfy_env/isolation/metadata.py", line 556, in proxy\n result = worker.call_method(\n ^^^^^^^^^^^^^^^^^^^\n', ' File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/venv/lib/python3.11/site-packages/comfy_env/isolation/workers/subprocess.py", line 674, in call_method\n raise WorkerError(\n'], 'current_inputs': {'focal_length_mm': [20], 'output_prefix': ['sharp'], 'model': ["{'model_path': '/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/models/sharp/sharp_2572gikvuh.pt', 'precision': 'auto', 'dtype': 'bf16'}"], 'image': ['tensor([[[[0.2157, 0.1647, 0.1569],\n [0.2275, 0.1608, 0.1608],\n [0.2275, 0.1647, 0.1529],\n ...,\n [0.4471, 0.4745, 0.4863],\n [0.4510, 0.4745, 0.4863],\n [0.4431, 0.4745, 0.4745]],\n\n [[0.2196, 0.1686, 0.1569],\n [0.2314, 0.1647, 0.1608],\n [0.2353, 0.1647, 0.1647],\n ...,\n [0.4549, 0.4784, 0.4706],\n [0.4549, 0.4706, 0.4784],\n [0.4431, 0.4706, 0.4667]],\n\n [[0.2314, 0.1725, 0.1569],\n [0.2510, 0.1725, 0.1608],\n [0.2392, 0.1725, 0.1686],\n ...,\n [0.4588, 0.4745, 0.4627],\n [0.4510, 0.4745, 0.4667],\n [0.4471, 0.4706, 0.4627]],\n\n ...,\n\n [[0.1569, 0.1647, 0.1843],\n [0.1647, 0.1725, 0.1882],\n [0.1490, 0.1686, 0.1922],\n ...,\n [0.1020, 0.1137, 0.1373],\n [0.1137, 0.1176, 0.1412],\n [0.0863, 0.1176, 0.1373]],\n\n [[0.1569, 0.1686, 0.1804],\n [0.1569, 0.1647, 0.1961],\n [0.1490, 0.1647, 0.1961],\n ...,\n [0.1059, 0.1216, 0.1294],\n [0.1176, 0.1216, 0.1333],\n [0.1059, 0.1137, 0.1373]],\n\n [[0.1608, 0.1686, 0.1843],\n [0.1608, 0.1686, 0.1882],\n [0.1569, 0.1686, 0.1882],\n ...,\n [0.1059, 0.1176, 0.1333],\n [0.1137, 0.1176, 0.1333],\n [0.1333, 0.1176, 0.1333]]]])'], 'prompt': ["{'2': {'inputs': {'precision': 'auto'}, 'class_type': 'LoadSharpModel', '_meta': {'title': '(Down)Load SHARP Model'}}, '7': {'inputs': {'focal_length_mm': ['9', 2], 'output_prefix': 'sharp', 'model': ['2', 0], 'image': ['9', 0]}, 'class_type': 'SharpPredict', '_meta': {'title': 'SHARP Predict (Image to PLY)'}}, '9': {'inputs': {'image': 'eth3d.png', 'default_focal_mm': 30.0}, 'class_type': 'LoadImageWithExif', '_meta': {'title': 'Load Image with EXIF (Focal Length)'}}, '11': {'inputs': {'preview_gaussian': '', 'ply_path': ['7', 0], 'extrinsics': ['7', 1], 'intrinsics': ['7', 2]}, 'class_type': 'PreviewGaussianSharp', '_meta': {'title': 'Preview Gaussian Sharp'}}}"], 'extra_pnginfo': ["{'workflow': {'id': 'sharp-basic-workflow', 'revision': 0, 'last_node_id': 11, 'last_link_id': 24, 'nodes': [{'id': 2, 'type': 'LoadSharpModel', 'pos': [-150, -400], 'size': [270, 106], 'flags': {}, 'order': 0, 'mode': 0, 'inputs': [], 'outputs': [{'name': 'model_config', 'type': 'SHARP_MODEL_CONFIG', 'links': [11]}], 'properties': {'Node name for S&R': 'LoadSharpModel', 'aux_id': 'PozzettiAndrea/ComfyUI-Sharp', 'ver': '9f1b9c726e2cfc0118866c3df783e7f6bc895226', 'cnr_id': 'comfyui-sharp'}, 'widgets_values': ['auto']}, {'id': 9, 'type': 'LoadImageWithExif', 'pos': [-208.74978010739324, -99.02778023242435], 'size': [342.28125, 420], 'flags': {}, 'order': 1, 'mode': 0, 'inputs': [], 'outputs': [{'name': 'image', 'type': 'IMAGE', 'links': [19]}, {'name': 'mask', 'type': 'MASK', 'links': None}, {'name': 'focal_length_mm', 'type': 'FLOAT', 'links': [20]}], 'properties': {'Node name for S&R': 'LoadImageWithExif', 'aux_id': 'PozzettiAndrea/ComfyUI-Sharp', 'ver': 'c69751a59b587c3b83d0e37cfe2bab3bc838574a', 'cnr_id': 'comfyui-sharp'}, 'widgets_values': ['eth3d.png', 30, 'image']}, {'id': 7, 'type': 'SharpPredict', 'pos': [226.15696022727278, -245.70490056818184], 'size': [298.890625, 236], 'flags': {}, 'order': 2, 'mode': 0, 'inputs': [{'name': 'model', 'type': 'SHARP_MODEL_CONFIG', 'link': 11}, {'name': 'image', 'type': 'IMAGE', 'link': 19}, {'name': 'extrinsics', 'shape': 7, 'type': 'EXTRINSICS', 'link': None}, {'name': 'intrinsics', 'shape': 7, 'type': 'INTRINSICS', 'link': None}, {'name': 'focal_length_mm', 'shape': 7, 'type': 'FLOAT', 'widget': {'name': 'focal_length_mm'}, 'link': 20}], 'outputs': [{'name': 'ply_path', 'type': 'STRING', 'links': [22]}, {'name': 'extrinsics', 'type': 'EXTRINSICS', 'links': [23]}, {'name': 'intrinsics', 'type': 'INTRINSICS', 'links': [24]}, {'name': 'layer_0_refined_depth', 'type': 'IMAGE', 'links': None}, {'name': 'layer_1_refined_depth', 'type': 'IMAGE', 'links': None}], 'properties': {'Node name for S&R': 'SharpPredict', 'aux_id': 'PozzettiAndrea/ComfyUI-Sharp', 'ver': 'c69751a59b587c3b83d0e37cfe2bab3bc838574a', 'cnr_id': 'comfyui-sharp'}, 'widgets_values': [30, 'sharp']}, {'id': 11, 'type': 'PreviewGaussianSharp', 'pos': [600.5860117362482, -287.72961200652577], 'size': [512, 658], 'flags': {}, 'order': 3, 'mode': 0, 'inputs': [{'name': 'ply_path', 'type': 'STRING', 'link': 22}, {'name': 'extrinsics', 'shape': 7, 'type': 'EXTRINSICS', 'link': 23}, {'name': 'intrinsics', 'shape': 7, 'type': 'INTRINSICS', 'link': 24}], 'outputs': [{'name': 'extrinsics', 'type': 'EXTRINSICS', 'links': None}, {'name': 'intrinsics', 'type': 'INTRINSICS', 'links': None}], 'properties': {'Node name for S&R': 'PreviewGaussianSharp', 'aux_id': 'PozzettiAndrea/ComfyUI-GaussianPack', 'ver': '059313a53ffa7ecc8697eddc02d2cee4727e5085'}, 'widgets_values': ['']}], 'links': [[11, 2, 0, 7, 0, 'SHARP_MODEL'], [19, 9, 0, 7, 1, 'IMAGE'], [20, 9, 2, 7, 4, 'FLOAT'], [22, 7, 0, 11, 0, 'STRING'], [23, 7, 1, 11, 1, 'EXTRINSICS'], [24, 7, 2, 11, 2, 'INTRINSICS']], 'groups': [], 'config': {}, 'extra': {'ds': {'scale': 0.9373016969629105, 'offset': [449.06099628194596, 596.4219590460706]}, 'workflowRendererVersion': 'LG', 'frontendVersion': '1.45.15'}, 'version': 0.4}}"]}, 'current_outputs': ['2', '9', '11', '7'], 'timestamp': 1780860197909}
Status: FAILED
Error: Workflow execution failed: {'prompt_id': '8a0433fa-7d52-4d8b-b620-62151ba4d2b5', 'node_id': '7', 'node_type': 'SharpPredict', 'executed': ['2', '9'], 'exception_message': 'MPS backend out of memory (MPS allocated: 6.73 GiB, other allocations: 581.22 MiB, max allowed: 7.93 GiB). Tried to allocate 576.00 MiB on private pool. Use PYTORCH_MPS_HIGH_WATERMARK_RATIO=0.0 to disable upper limit for memory allocations (may cause system failure).\n\nWorker traceback:\nTraceback (most recent call last):\n File "/var/folders/8d/778wjbv96mq1760tv6gk374m0000gn/T/comfyui_pvenv_zcu5iw2_/persistent_worker.py", line 1762, in main\n result = method(**inputs)\n ^^^^^^^^^^^^^^^^\n File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/comfy_api/latest/_io.py", line 1892, in EXECUTE_NORMALIZED\n to_return = cls.execute(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 124, in decorate_context\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/predict.py", line 239, in execute\n gaussians, refined_depth_l0, refined_depth_l1 = _predict_image_cached(\n ^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/predict.py", line 348, in _predict_image_cached\n monodepth_output, _ = predictor.encode(image_resized_pt)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/sharp/model.py", line 2006, in encode\n monodepth_output = self.monodepth_model(image)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl\n return self._call_impl(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl\n return forward_call(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/sharp/model.py", line 1118, in forward\n decoder_features = self.monodepth_predictor.decoder(encoder_features)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl\n return self._call_impl(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl\n return forward_call(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/sharp/model.py", line 970, in forward\n features = self.fusions[i](features, features_i)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl\n return self._call_impl(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl\n return forward_call(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/sharp/model.py", line 862, in forward\n x = self.out_conv(x)\n ^^^^^^^^^^^^^^^^\n File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl\n return self._call_impl(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl\n return forward_call(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/comfy/ops.py", line 537, in forward\n return super().forward(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/conv.py", line 553, in forward\n return self._conv_forward(input, self.weight, self.bias)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/conv.py", line 548, in _conv_forward\n return F.conv2d(\n ^^^^^^^^^\nRuntimeError: MPS backend out of memory (MPS allocated: 6.73 GiB, other allocations: 581.22 MiB, max allowed: 7.93 GiB). Tried to allocate 576.00 MiB on private pool. Use PYTORCH_MPS_HIGH_WATERMARK_RATIO=0.0 to disable upper limit for memory allocations (may cause system failure).\n\n', 'exception_type': 'comfy_env.isolation.workers.base.WorkerError', 'traceback': [' File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/execution.py", line 536, in execute\n output_data, output_ui, has_subgraph, has_pending_tasks = await get_output_data(prompt_id, unique_id, obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n', ' File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/execution.py", line 336, in get_output_data\n return_values = await _async_map_node_over_list(prompt_id, unique_id, obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n', ' File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/execution.py", line 310, in _async_map_node_over_list\n await process_inputs(input_dict, i)\n', ' File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/execution.py", line 298, in process_inputs\n result = f(**inputs)\n ^^^^^^^^^^^\n', ' File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/venv/lib/python3.11/site-packages/comfy_env/isolation/metadata.py", line 556, in proxy\n result = worker.call_method(\n ^^^^^^^^^^^^^^^^^^^\n', ' File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/venv/lib/python3.11/site-packages/comfy_env/isolation/workers/subprocess.py", line 674, in call_method\n raise WorkerError(\n'], 'current_inputs': {'focal_length_mm': [20], 'output_prefix': ['sharp'], 'model': ["{'model_path': '/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/models/sharp/sharp_2572gikvuh.pt', 'precision': 'auto', 'dtype': 'bf16'}"], 'image': ['tensor([[[[0.2157, 0.1647, 0.1569],\n [0.2275, 0.1608, 0.1608],\n [0.2275, 0.1647, 0.1529],\n ...,\n [0.4471, 0.4745, 0.4863],\n [0.4510, 0.4745, 0.4863],\n [0.4431, 0.4745, 0.4745]],\n\n [[0.2196, 0.1686, 0.1569],\n [0.2314, 0.1647, 0.1608],\n [0.2353, 0.1647, 0.1647],\n ...,\n [0.4549, 0.4784, 0.4706],\n [0.4549, 0.4706, 0.4784],\n [0.4431, 0.4706, 0.4667]],\n\n [[0.2314, 0.1725, 0.1569],\n [0.2510, 0.1725, 0.1608],\n [0.2392, 0.1725, 0.1686],\n ...,\n [0.4588, 0.4745, 0.4627],\n [0.4510, 0.4745, 0.4667],\n [0.4471, 0.4706, 0.4627]],\n\n ...,\n\n [[0.1569, 0.1647, 0.1843],\n [0.1647, 0.1725, 0.1882],\n [0.1490, 0.1686, 0.1922],\n ...,\n [0.1020, 0.1137, 0.1373],\n [0.1137, 0.1176, 0.1412],\n [0.0863, 0.1176, 0.1373]],\n\n [[0.1569, 0.1686, 0.1804],\n [0.1569, 0.1647, 0.1961],\n [0.1490, 0.1647, 0.1961],\n ...,\n [0.1059, 0.1216, 0.1294],\n [0.1176, 0.1216, 0.1333],\n [0.1059, 0.1137, 0.1373]],\n\n [[0.1608, 0.1686, 0.1843],\n [0.1608, 0.1686, 0.1882],\n [0.1569, 0.1686, 0.1882],\n ...,\n [0.1059, 0.1176, 0.1333],\n [0.1137, 0.1176, 0.1333],\n [0.1333, 0.1176, 0.1333]]]])'], 'prompt': ["{'2': {'inputs': {'precision': 'auto'}, 'class_type': 'LoadSharpModel', '_meta': {'title': '(Down)Load SHARP Model'}}, '7': {'inputs': {'focal_length_mm': ['9', 2], 'output_prefix': 'sharp', 'model': ['2', 0], 'image': ['9', 0]}, 'class_type': 'SharpPredict', '_meta': {'title': 'SHARP Predict (Image to PLY)'}}, '9': {'inputs': {'image': 'eth3d.png', 'default_focal_mm': 30.0}, 'class_type': 'LoadImageWithExif', '_meta': {'title': 'Load Image with EXIF (Focal Length)'}}, '11': {'inputs': {'preview_gaussian': '', 'ply_path': ['7', 0], 'extrinsics': ['7', 1], 'intrinsics': ['7', 2]}, 'class_type': 'PreviewGaussianSharp', '_meta': {'title': 'Preview Gaussian Sharp'}}}"], 'extra_pnginfo': ["{'workflow': {'id': 'sharp-basic-workflow', 'revision': 0, 'last_node_id': 11, 'last_link_id': 24, 'nodes': [{'id': 2, 'type': 'LoadSharpModel', 'pos': [-150, -400], 'size': [270, 106], 'flags': {}, 'order': 0, 'mode': 0, 'inputs': [], 'outputs': [{'name': 'model_config', 'type': 'SHARP_MODEL_CONFIG', 'links': [11]}], 'properties': {'Node name for S&R': 'LoadSharpModel', 'aux_id': 'PozzettiAndrea/ComfyUI-Sharp', 'ver': '9f1b9c726e2cfc0118866c3df783e7f6bc895226', 'cnr_id': 'comfyui-sharp'}, 'widgets_values': ['auto']}, {'id': 9, 'type': 'LoadImageWithExif', 'pos': [-208.74978010739324, -99.02778023242435], 'size': [342.28125, 420], 'flags': {}, 'order': 1, 'mode': 0, 'inputs': [], 'outputs': [{'name': 'image', 'type': 'IMAGE', 'links': [19]}, {'name': 'mask', 'type': 'MASK', 'links': None}, {'name': 'focal_length_mm', 'type': 'FLOAT', 'links': [20]}], 'properties': {'Node name for S&R': 'LoadImageWithExif', 'aux_id': 'PozzettiAndrea/ComfyUI-Sharp', 'ver': 'c69751a59b587c3b83d0e37cfe2bab3bc838574a', 'cnr_id': 'comfyui-sharp'}, 'widgets_values': ['eth3d.png', 30, 'image']}, {'id': 7, 'type': 'SharpPredict', 'pos': [226.15696022727278, -245.70490056818184], 'size': [298.890625, 236], 'flags': {}, 'order': 2, 'mode': 0, 'inputs': [{'name': 'model', 'type': 'SHARP_MODEL_CONFIG', 'link': 11}, {'name': 'image', 'type': 'IMAGE', 'link': 19}, {'name': 'extrinsics', 'shape': 7, 'type': 'EXTRINSICS', 'link': None}, {'name': 'intrinsics', 'shape': 7, 'type': 'INTRINSICS', 'link': None}, {'name': 'focal_length_mm', 'shape': 7, 'type': 'FLOAT', 'widget': {'name': 'focal_length_mm'}, 'link': 20}], 'outputs': [{'name': 'ply_path', 'type': 'STRING', 'links': [22]}, {'name': 'extrinsics', 'type': 'EXTRINSICS', 'links': [23]}, {'name': 'intrinsics', 'type': 'INTRINSICS', 'links': [24]}, {'name': 'layer_0_refined_depth', 'type': 'IMAGE', 'links': None}, {'name': 'layer_1_refined_depth', 'type': 'IMAGE', 'links': None}], 'properties': {'Node name for S&R': 'SharpPredict', 'aux_id': 'PozzettiAndrea/ComfyUI-Sharp', 'ver': 'c69751a59b587c3b83d0e37cfe2bab3bc838574a', 'cnr_id': 'comfyui-sharp'}, 'widgets_values': [30, 'sharp']}, {'id': 11, 'type': 'PreviewGaussianSharp', 'pos': [600.5860117362482, -287.72961200652577], 'size': [512, 658], 'flags': {}, 'order': 3, 'mode': 0, 'inputs': [{'name': 'ply_path', 'type': 'STRING', 'link': 22}, {'name': 'extrinsics', 'shape': 7, 'type': 'EXTRINSICS', 'link': 23}, {'name': 'intrinsics', 'shape': 7, 'type': 'INTRINSICS', 'link': 24}], 'outputs': [{'name': 'extrinsics', 'type': 'EXTRINSICS', 'links': None}, {'name': 'intrinsics', 'type': 'INTRINSICS', 'links': None}], 'properties': {'Node name for S&R': 'PreviewGaussianSharp', 'aux_id': 'PozzettiAndrea/ComfyUI-GaussianPack', 'ver': '059313a53ffa7ecc8697eddc02d2cee4727e5085'}, 'widgets_values': ['']}], 'links': [[11, 2, 0, 7, 0, 'SHARP_MODEL'], [19, 9, 0, 7, 1, 'IMAGE'], [20, 9, 2, 7, 4, 'FLOAT'], [22, 7, 0, 11, 0, 'STRING'], [23, 7, 1, 11, 1, 'EXTRINSICS'], [24, 7, 2, 11, 2, 'INTRINSICS']], 'groups': [], 'config': {}, 'extra': {'ds': {'scale': 0.9373016969629105, 'offset': [449.06099628194596, 596.4219590460706]}, 'workflowRendererVersion': 'LG', 'frontendVersion': '1.45.15'}, 'version': 0.4}}"]}, 'current_outputs': ['2', '9', '11', '7'], 'timestamp': 1780860197909}
Details: Workflow: /private/var/folders/8d/778wjbv96mq1760tv6gk374m0000gn/T/comfy-test-run-n4s1o4ug/ComfyUI-Sharp/workflows/sharp_exif.json
Node error: SharpPredict
[ComfyUI] [INFO] Using RAM pressure cache.
Capturing execution frames: sharp_exif.json
[ComfyUI] [INFO] Using RAM pressure cache.
[ComfyUI] [INFO] Using RAM pressure cache.
Frame 1 saved (frame_000.jpg, t=0.0s)
Validating workflow...
[ComfyUI] [INFO] === /validate endpoint called ===
[ComfyUI] [INFO] === /validate endpoint called ===
[ComfyUI] [INFO] Received JSON with keys: ['prompt']
[ComfyUI] [INFO] Received JSON with keys: ['prompt']
[ComfyUI] [INFO] Prompt has 4 nodes
[ComfyUI] [INFO] Prompt has 4 nodes
[ComfyUI] [INFO] Node 2: LoadSharpModel
[ComfyUI] [INFO] Node 2: LoadSharpModel
[ComfyUI] [INFO] Inputs: {"precision": "auto"}
[ComfyUI] [INFO] Inputs: {"precision": "auto"}
[ComfyUI] [INFO] Node 7: SharpPredict
[ComfyUI] [INFO] Node 7: SharpPredict
[ComfyUI] [INFO] Inputs: {"focal_length_mm": ["9", 2], "output_prefix": "sharp", "model": ["2", 0], "image": ["9", 0]}
[ComfyUI] [INFO] Inputs: {"focal_length_mm": ["9", 2], "output_prefix": "sharp", "model": ["2", 0], "image": ["9", 0]}
[ComfyUI] [INFO] Node 9: LoadImageWithExif
[ComfyUI] [INFO] Node 9: LoadImageWithExif
[ComfyUI] [INFO] Inputs: {"image": "eth3d.png", "default_focal_mm": 30}
[ComfyUI] [INFO] Inputs: {"image": "eth3d.png", "default_focal_mm": 30}
[ComfyUI] [INFO] Node 11: PreviewGaussianSharp
[ComfyUI] [INFO] Node 11: PreviewGaussianSharp
[ComfyUI] [INFO] Inputs: {"preview_gaussian": "", "ply_path": ["7", 0], "extrinsics": ["7", 1], "intrinsics": ["7", 2]}
[ComfyUI] [INFO] Inputs: {"preview_gaussian": "", "ply_path": ["7", 0], "extrinsics": ["7", 1], "intrinsics": ["7", 2]}
[ComfyUI] [INFO] Generated prompt_id: 6b486fba-b946-4cbc-8766-4743667ede94
[ComfyUI] [INFO] Generated prompt_id: 6b486fba-b946-4cbc-8766-4743667ede94
[ComfyUI] [INFO] Calling execution.validate_prompt()...
[ComfyUI] [INFO] Calling execution.validate_prompt()...
[ComfyUI] [INFO] Validation result: valid=True
[ComfyUI] [INFO] Validation result: valid=True
[ComfyUI] [INFO] === Validation PASSED ===
[ComfyUI] [INFO] === Validation PASSED ===
Queuing workflow for execution...
[ComfyUI] [INFO] got prompt
[ComfyUI] [INFO] got prompt
[ComfyUI] [comfy-env] Running LoadImageWithExif: image: 'eth3d.png', default_focal_mm: 30.0
[ComfyUI] [comfy-env] Running LoadImageWithExif: image: 'eth3d.png', default_focal_mm: 30.0
[ComfyUI] [SubprocessWorker] call_method: nodes.load_image_exif.LoadImageWithExif.EXECUTE_NORMALIZED
[ComfyUI] [SubprocessWorker] call_method: nodes.load_image_exif.LoadImageWithExif.EXECUTE_NORMALIZED
[ComfyUI] [SubprocessWorker] acquired lock, ensuring started...
[ComfyUI] [SubprocessWorker] acquired lock, ensuring started...
[ComfyUI] [ComfyUI-Sharp] Health check: ping (timeout=5.0s)...
[ComfyUI] [ComfyUI-Sharp] Health check: ping (timeout=5.0s)...
[ComfyUI] [ComfyUI-Sharp] Health check: failed
[ComfyUI] [ComfyUI-Sharp] Health check: failed
[ComfyUI] [ComfyUI-Sharp] Socket unhealthy, restarting worker...
[ComfyUI] [ComfyUI-Sharp] Socket unhealthy, restarting worker...
[ComfyUI] [ComfyUI-Sharp] python: /Users/runner/.ce/.pixi/envs/sharp-nodes/bin/python
[ComfyUI] [ComfyUI-Sharp] python: /Users/runner/.ce/.pixi/envs/sharp-nodes/bin/python
[ComfyUI] [ComfyUI-Sharp] sys_path sent to worker: ['/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI', '/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp', '/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages', '/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp']
[ComfyUI] [ComfyUI-Sharp] sys_path sent to worker: ['/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI', '/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp', '/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages', '/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp']
[ComfyUI] [SubprocessWorker] is_pixi=True, python=/Users/runner/.ce/.pixi/envs/sharp-nodes/bin/python
[ComfyUI] [SubprocessWorker] is_pixi=True, python=/Users/runner/.ce/.pixi/envs/sharp-nodes/bin/python
[ComfyUI] [SubprocessWorker] launching cmd=['/Users/runner/.pixi/bin/pixi', 'run', '--as-is', '--manifest-path', '/Users/runner/.ce/pixi.toml', '-e', 'sharp-nodes', 'python', '/var/folders/8d/778wjbv96mq1760tv6gk374m0000gn/T/comfyui_pvenv_zcu5iw2_/persistent_worker.py', 'unix:///var/folders/8d/778wjbv96mq1760tv6gk374m0000gn/T/comfy_worker_be2eddec1a40.sock']...
[ComfyUI] [SubprocessWorker] launching cmd=['/Users/runner/.pixi/bin/pixi', 'run', '--as-is', '--manifest-path', '/Users/runner/.ce/pixi.toml', '-e', 'sharp-nodes', 'python', '/var/folders/8d/778wjbv96mq1760tv6gk374m0000gn/T/comfyui_pvenv_zcu5iw2_/persistent_worker.py', 'unix:///var/folders/8d/778wjbv96mq1760tv6gk374m0000gn/T/comfy_worker_be2eddec1a40.sock']...
[ComfyUI] [SubprocessWorker] socket_addr=unix:///var/folders/8d/778wjbv96mq1760tv6gk374m0000gn/T/comfy_worker_be2eddec1a40.sock exists=True
[ComfyUI] [SubprocessWorker] socket_addr=unix:///var/folders/8d/778wjbv96mq1760tv6gk374m0000gn/T/comfy_worker_be2eddec1a40.sock exists=True
Frame 2 saved (frame_001.jpg, t=2.5s)
[capture-periodic] freeze=6ms shot=244ms unfreeze=12ms saved=True
[ComfyUI] [worker:ComfyUI-Sharp] ERROR: Failed to import comfy_kitchen, Error: No module named 'comfy_kitchen', fp8 and fp4 support will not be available.
[ComfyUI] [worker:ComfyUI-Sharp] ERROR: Failed to import comfy_kitchen, Error: No module named 'comfy_kitchen', fp8 and fp4 support will not be available.
[ComfyUI] [worker:ComfyUI-Sharp] WARNING: comfy_kitchen does not support stochastic FP8 rounding, please update comfy_kitchen.
[ComfyUI] [worker:ComfyUI-Sharp] WARNING: comfy_kitchen does not support stochastic FP8 rounding, please update comfy_kitchen.
[ComfyUI] [SubprocessWorker] worker started/confirmed
[ComfyUI] [SubprocessWorker] worker started/confirmed
[ComfyUI] [SubprocessWorker] serializing kwargs to shm...
[ComfyUI] [SubprocessWorker] serializing kwargs to shm...
[ComfyUI] [SubprocessWorker] created 0 shm blocks
[ComfyUI] [SubprocessWorker] created 0 shm blocks
[ComfyUI] [SubprocessWorker] sending request via socket...
[ComfyUI] [SubprocessWorker] sending request via socket...
[ComfyUI] [SubprocessWorker] call_id=3 sending call_method
[ComfyUI] [SubprocessWorker] call_id=3 sending call_method
Frame 3 saved (frame_002.jpg, t=4.8s)
[capture-periodic] freeze=11ms shot=1239ms unfreeze=12ms saved=True
[capture-loop] iter=50 t=7.8s state={'complete': False, 'error': None, 'executedCount': 0, 'wsState': 1} eval_ms=4 ram=2.1/7.0GB (37%) swap=5.7/8.0GB (71%) cpu=65% browser_rss=0.53GB
[ComfyUI] [SubprocessWorker] got response: ok
[ComfyUI] [SubprocessWorker] got response: ok
[ComfyUI] [comfy-env] LoadImageWithExif done (7.97s): result: NodeOutput
[ComfyUI] [comfy-env] LoadImageWithExif done (7.97s): result: NodeOutput
[ComfyUI] [comfy-env] Running LoadSharpModel: precision: 'auto'
[ComfyUI] [comfy-env] Running LoadSharpModel: precision: 'auto'
[ComfyUI] [SubprocessWorker] call_method: nodes.load_model.LoadSharpModel.EXECUTE_NORMALIZED
[ComfyUI] [SubprocessWorker] call_method: nodes.load_model.LoadSharpModel.EXECUTE_NORMALIZED
[ComfyUI] [SubprocessWorker] acquired lock, ensuring started...
[ComfyUI] [SubprocessWorker] acquired lock, ensuring started...
[ComfyUI] [ComfyUI-Sharp] Health check: ping (timeout=5.0s)...
[ComfyUI] [ComfyUI-Sharp] Health check: ping (timeout=5.0s)...
[ComfyUI] [ComfyUI-Sharp] Health check: ok
[ComfyUI] [ComfyUI-Sharp] Health check: ok
[ComfyUI] [SubprocessWorker] worker started/confirmed
[ComfyUI] [SubprocessWorker] worker started/confirmed
[ComfyUI] [SubprocessWorker] serializing kwargs to shm...
[ComfyUI] [SubprocessWorker] serializing kwargs to shm...
[ComfyUI] [SubprocessWorker] created 0 shm blocks
[ComfyUI] [SubprocessWorker] created 0 shm blocks
[ComfyUI] [SubprocessWorker] sending request via socket...
[ComfyUI] [SubprocessWorker] sending request via socket...
[ComfyUI] [SubprocessWorker] call_id=4 sending call_method
[ComfyUI] [SubprocessWorker] call_id=4 sending call_method
[ComfyUI] [worker:ComfyUI-Sharp] [LoadSharpModel] precision='auto' -> dtype=bf16 (device=mps)
[ComfyUI] [worker:ComfyUI-Sharp] [LoadSharpModel] precision='auto' -> dtype=bf16 (device=mps)
[ComfyUI] [worker:ComfyUI-Sharp] [LoadSharpModel] checkpoint already present, reusing (no download): /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/models/sharp/sharp_2572gikvuh.pt (2679.6 MB)
[ComfyUI] [worker:ComfyUI-Sharp] [LoadSharpModel] checkpoint already present, reusing (no download): /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/models/sharp/sharp_2572gikvuh.pt (2679.6 MB)
[ComfyUI] [worker:ComfyUI-Sharp] [LoadSharpModel] checkpoint ready: /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/models/sharp/sharp_2572gikvuh.pt (2679.6 MB)
[ComfyUI] [worker:ComfyUI-Sharp] [LoadSharpModel] checkpoint ready: /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/models/sharp/sharp_2572gikvuh.pt (2679.6 MB)
[ComfyUI] [worker:ComfyUI-Sharp] [LoadSharpModel] config: precision=auto -> dtype=bf16
[ComfyUI] [worker:ComfyUI-Sharp] [LoadSharpModel] config: precision=auto -> dtype=bf16
[ComfyUI] [SubprocessWorker] got response: ok
[ComfyUI] [SubprocessWorker] got response: ok
[ComfyUI] [comfy-env] LoadSharpModel done (0.05s): result: NodeOutput
[ComfyUI] [comfy-env] LoadSharpModel done (0.05s): result: NodeOutput
[ComfyUI] [comfy-env] Running SharpPredict: focal_length_mm: 20.0, output_prefix: 'sharp', model: dict, image: torch.float32 [1,2022,3040,3] cpu
[ComfyUI] [comfy-env] Running SharpPredict: focal_length_mm: 20.0, output_prefix: 'sharp', model: dict, image: torch.float32 [1,2022,3040,3] cpu
[ComfyUI] [SubprocessWorker] call_method: nodes.predict.SharpPredict.EXECUTE_NORMALIZED
[ComfyUI] [SubprocessWorker] call_method: nodes.predict.SharpPredict.EXECUTE_NORMALIZED
[ComfyUI] [SubprocessWorker] acquired lock, ensuring started...
[ComfyUI] [SubprocessWorker] acquired lock, ensuring started...
[ComfyUI] [ComfyUI-Sharp] Health check: ping (timeout=5.0s)...
[ComfyUI] [ComfyUI-Sharp] Health check: ping (timeout=5.0s)...
[ComfyUI] [ComfyUI-Sharp] Health check: ok
[ComfyUI] [ComfyUI-Sharp] Health check: ok
[ComfyUI] [SubprocessWorker] worker started/confirmed
[ComfyUI] [SubprocessWorker] worker started/confirmed
[ComfyUI] [comfy-env] PRE-SERIALIZE 'image' shape: torch.Size([1, 2022, 3040, 3])
[ComfyUI] [comfy-env] PRE-SERIALIZE 'image' shape: torch.Size([1, 2022, 3040, 3])
[ComfyUI] [SubprocessWorker] serializing kwargs to shm...
[ComfyUI] [SubprocessWorker] serializing kwargs to shm...
[ComfyUI] [SubprocessWorker] created 0 shm blocks
[ComfyUI] [SubprocessWorker] created 0 shm blocks
[ComfyUI] [SubprocessWorker] sending request via socket...
[ComfyUI] [SubprocessWorker] sending request via socket...
[ComfyUI] [SubprocessWorker] call_id=5 sending call_method
[ComfyUI] [SubprocessWorker] call_id=5 sending call_method
[ComfyUI] [worker:ComfyUI-Sharp] [comfy-env] _from_shm got dict with keys: ['__type__', 'strategy', 'manager_path', 'storage_key', 'storage_size']
[ComfyUI] [worker:ComfyUI-Sharp] [comfy-env] _from_shm got dict with keys: ['__type__', 'strategy', 'manager_path', 'storage_key', 'storage_size']
[ComfyUI] [worker:ComfyUI-Sharp] [comfy-env] DESERIALIZE TensorRef: tensor_size=[1, 2022, 3040, 3]
[ComfyUI] [worker:ComfyUI-Sharp] [comfy-env] DESERIALIZE TensorRef: tensor_size=[1, 2022, 3040, 3]
[ComfyUI] [worker:ComfyUI-Sharp] [comfy-env] DESERIALIZED tensor shape: torch.Size([1, 2022, 3040, 3])
[ComfyUI] [worker:ComfyUI-Sharp] [comfy-env] DESERIALIZED tensor shape: torch.Size([1, 2022, 3040, 3])
Frame 4 saved (frame_003.jpg, t=8.2s)
[capture-periodic] freeze=5ms shot=246ms unfreeze=6ms saved=True
Frame 5 saved (frame_004.jpg, t=10.6s)
[capture-periodic] freeze=4ms shot=122ms unfreeze=4ms saved=True
[ComfyUI] objc[20942]: Class AVFFrameReceiver is implemented in both /Users/runner/.ce/.pixi/envs/sharp-nodes/lib/libavdevice.62.3.101.dylib (0x1283447a8) and /Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/cv2/.dylibs/libavdevice.61.3.100.dylib (0x15ef303a8). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed.
[ComfyUI] objc[20942]: Class AVFFrameReceiver is implemented in both /Users/runner/.ce/.pixi/envs/sharp-nodes/lib/libavdevice.62.3.101.dylib (0x1283447a8) and /Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/cv2/.dylibs/libavdevice.61.3.100.dylib (0x15ef303a8). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed.
[ComfyUI] objc[20942]: Class AVFAudioReceiver is implemented in both /Users/runner/.ce/.pixi/envs/sharp-nodes/lib/libavdevice.62.3.101.dylib (0x1283447f8) and /Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/cv2/.dylibs/libavdevice.61.3.100.dylib (0x15ef303f8). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed.
[ComfyUI] objc[20942]: Class AVFAudioReceiver is implemented in both /Users/runner/.ce/.pixi/envs/sharp-nodes/lib/libavdevice.62.3.101.dylib (0x1283447f8) and /Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/cv2/.dylibs/libavdevice.61.3.100.dylib (0x15ef303f8). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed.
[ComfyUI] [worker:ComfyUI-Sharp] [LoadSharpModel] loading checkpoint from /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/models/sharp/sharp_2572gikvuh.pt (dtype=bf16)
[ComfyUI] [worker:ComfyUI-Sharp] [LoadSharpModel] loading checkpoint from /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/models/sharp/sharp_2572gikvuh.pt (dtype=bf16)
[capture-loop] iter=100 t=14.1s state={'complete': False, 'error': None, 'executedCount': 0, 'wsState': 1} eval_ms=72 ram=2.6/7.0GB (44%) swap=3.5/8.0GB (44%) cpu=64% browser_rss=0.54GB
Frame 6 saved (frame_005.jpg, t=17.3s)
[capture-periodic] freeze=4ms shot=458ms unfreeze=13ms saved=True
[capture-loop] iter=150 t=21.4s state={'complete': False, 'error': None, 'executedCount': 0, 'wsState': 1} eval_ms=4 ram=2.7/7.0GB (76%) swap=0.6/2.0GB (30%) cpu=88% browser_rss=0.44GB
[ComfyUI] [worker:ComfyUI-Sharp] [LoadSharpModel] initializing model on meta device...
[ComfyUI] [worker:ComfyUI-Sharp] [LoadSharpModel] initializing model on meta device...
[ComfyUI] [worker:ComfyUI-Sharp] [LoadSharpModel] model ready (torch.bfloat16)
[ComfyUI] [worker:ComfyUI-Sharp] [LoadSharpModel] model ready (torch.bfloat16)
[ComfyUI] [SubprocessWorker] callback 'request_vram_budget' from call_id=5
[ComfyUI] [SubprocessWorker] callback 'request_vram_budget' from call_id=5
[ComfyUI] [comfy-env] VRAM request: 2.81GB, free before eviction: 1.80GB
[ComfyUI] [comfy-env] VRAM request: 2.81GB, free before eviction: 1.80GB
[ComfyUI] [comfy-env] VRAM after eviction: 1.80GB
[ComfyUI] [comfy-env] VRAM after eviction: 1.80GB
Frame 7 saved (frame_006.jpg, t=26.3s)
[capture-periodic] freeze=3ms shot=260ms unfreeze=12ms saved=True
[capture-loop] iter=200 t=29.8s state={'complete': False, 'error': None, 'executedCount': 0, 'wsState': 1} eval_ms=6 ram=4.5/7.0GB (79%) swap=0.6/2.0GB (30%) cpu=73% browser_rss=0.27GB
Frame 8 saved (frame_007.jpg, t=33.8s)
[capture-periodic] freeze=2ms shot=7203ms unfreeze=24ms saved=True
Frame 9 saved (frame_008.jpg, t=43.2s)
[capture-periodic] freeze=2ms shot=3247ms unfreeze=3ms saved=True
[capture-loop] iter=250 t=46.6s state={'complete': False, 'error': None, 'executedCount': 0, 'wsState': 1} eval_ms=5 ram=5.8/7.0GB (90%) swap=0.9/2.0GB (45%) cpu=79% browser_rss=0.24GB
Frame 10 saved (frame_009.jpg, t=53.8s)
[capture-periodic] freeze=97ms shot=5990ms unfreeze=5ms saved=True
[ComfyUI] [worker:ComfyUI-Sharp]
[ComfyUI] [worker:ComfyUI-Sharp]
[ComfyUI] === WATCHDOG TICK 1 ===
[ComfyUI] === WATCHDOG TICK 1 ===
[ComfyUI] [worker:ComfyUI-Sharp] Current thread 0x000000016c1f7000 (most recent call first):
[ComfyUI] [worker:ComfyUI-Sharp] Current thread 0x000000016c1f7000 (most recent call first):
[ComfyUI] File "/var/folders/8d/778wjbv96mq1760tv6gk374m0000gn/T/comfyui_pvenv_zcu5iw2_/persistent_worker.py", line 52 in _watchdog
[ComfyUI] File "/var/folders/8d/778wjbv96mq1760tv6gk374m0000gn/T/comfyui_pvenv_zcu5iw2_/persistent_worker.py", line 52 in _watchdog
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/threading.py", line 982 in run
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/threading.py", line 982 in run
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/threading.py", line 1045 in _bootstrap_inner
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/threading.py", line 1045 in _bootstrap_inner
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/threading.py", line 1002 in _bootstrap
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/threading.py", line 1002 in _bootstrap
[ComfyUI]
[ComfyUI]
[ComfyUI] Thread 0x00000001efc5a2c0 (most recent call first):
[ComfyUI] Thread 0x00000001efc5a2c0 (most recent call first):
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/conv.py", line 548 in _conv_forward
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/conv.py", line 548 in _conv_forward
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/conv.py", line 553 in forward
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/conv.py", line 553 in forward
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/comfy/ops.py", line 537 in forward
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/comfy/ops.py", line 537 in forward
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1787 in _call_impl
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1787 in _call_impl
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1776 in _wrapped_call_impl
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1776 in _wrapped_call_impl
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/sharp/model.py", line 969 in forward
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/sharp/model.py", line 969 in forward
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1787 in _call_impl
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1787 in _call_impl
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1776 in _wrapped_call_impl
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1776 in _wrapped_call_impl
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/sharp/model.py", line 1118 in forward
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/sharp/model.py", line 1118 in forward
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1787 in _call_impl
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1787 in _call_impl
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1776 in _wrapped_call_impl
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1776 in _wrapped_call_impl
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/sharp/model.py", line 2006 in encode
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/sharp/model.py", line 2006 in encode
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/predict.py", line 348 in _predict_image_cached
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/predict.py", line 348 in _predict_image_cached
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/predict.py", line 239 in execute
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/predict.py", line 239 in execute
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 124 in decorate_context
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 124 in decorate_context
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/comfy_api/latest/_io.py", line 1892 in EXECUTE_NORMALIZED
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/comfy_api/latest/_io.py", line 1892 in EXECUTE_NORMALIZED
[ComfyUI] File "/var/folders/8d/778wjbv96mq1760tv6gk374m0000gn/T/comfyui_pvenv_zcu5iw2_/persistent_worker.py", line 1762 in main
[ComfyUI] File "/var/folders/8d/778wjbv96mq1760tv6gk374m0000gn/T/comfyui_pvenv_zcu5iw2_/persistent_worker.py", line 1762 in main
[ComfyUI] File "/var/folders/8d/778wjbv96mq1760tv6gk374m0000gn/T/comfyui_pvenv_zcu5iw2_/persistent_worker.py", line 1807 in <module>
[ComfyUI] File "/var/folders/8d/778wjbv96mq1760tv6gk374m0000gn/T/comfyui_pvenv_zcu5iw2_/persistent_worker.py", line 1807 in <module>
[ComfyUI]
[ComfyUI]
[ComfyUI] [worker:ComfyUI-Sharp] === END ===
[ComfyUI] [worker:ComfyUI-Sharp] === END ===
[ComfyUI]
[ComfyUI]
[capture-loop] iter=300 t=61.4s state={'complete': False, 'error': None, 'executedCount': 0, 'wsState': 1} eval_ms=16 ram=5.2/7.0GB (84%) swap=1.2/2.0GB (62%) cpu=77% browser_rss=0.20GB
[ComfyUI] [SubprocessWorker] got response: error
[ComfyUI] [SubprocessWorker] got response: error
[ComfyUI] [ERROR] !!! Exception during processing !!! MPS backend out of memory (MPS allocated: 6.73 GiB, other allocations: 581.22 MiB, max allowed: 7.93 GiB). Tried to allocate 576.00 MiB on private pool. Use PYTORCH_MPS_HIGH_WATERMARK_RATIO=0.0 to disable upper limit for memory allocations (may cause system failure).
[ComfyUI] [ERROR] !!! Exception during processing !!! MPS backend out of memory (MPS allocated: 6.73 GiB, other allocations: 581.22 MiB, max allowed: 7.93 GiB). Tried to allocate 576.00 MiB on private pool. Use PYTORCH_MPS_HIGH_WATERMARK_RATIO=0.0 to disable upper limit for memory allocations (may cause system failure).
[ComfyUI]
[ComfyUI]
[ComfyUI] Worker traceback:
[ComfyUI] Worker traceback:
[ComfyUI] Traceback (most recent call last):
[ComfyUI] Traceback (most recent call last):
[ComfyUI] File "/var/folders/8d/778wjbv96mq1760tv6gk374m0000gn/T/comfyui_pvenv_zcu5iw2_/persistent_worker.py", line 1762, in main
[ComfyUI] File "/var/folders/8d/778wjbv96mq1760tv6gk374m0000gn/T/comfyui_pvenv_zcu5iw2_/persistent_worker.py", line 1762, in main
[ComfyUI] result = method(**inputs)
[ComfyUI] result = method(**inputs)
[ComfyUI] ^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/comfy_api/latest/_io.py", line 1892, in EXECUTE_NORMALIZED
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/comfy_api/latest/_io.py", line 1892, in EXECUTE_NORMALIZED
[ComfyUI] to_return = cls.execute(*args, **kwargs)
[ComfyUI] to_return = cls.execute(*args, **kwargs)
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 124, in decorate_context
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 124, in decorate_context
[ComfyUI] return func(*args, **kwargs)
[ComfyUI] return func(*args, **kwargs)
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/predict.py", line 239, in execute
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/predict.py", line 239, in execute
[ComfyUI] gaussians, refined_depth_l0, refined_depth_l1 = _predict_image_cached(
[ComfyUI] gaussians, refined_depth_l0, refined_depth_l1 = _predict_image_cached(
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/predict.py", line 348, in _predict_image_cached
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/predict.py", line 348, in _predict_image_cached
[ComfyUI] monodepth_output, _ = predictor.encode(image_resized_pt)
[ComfyUI] monodepth_output, _ = predictor.encode(image_resized_pt)
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/sharp/model.py", line 2006, in encode
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/sharp/model.py", line 2006, in encode
[ComfyUI] monodepth_output = self.monodepth_model(image)
[ComfyUI] monodepth_output = self.monodepth_model(image)
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
[ComfyUI] return self._call_impl(*args, **kwargs)
[ComfyUI] return self._call_impl(*args, **kwargs)
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl
[ComfyUI] return forward_call(*args, **kwargs)
[ComfyUI] return forward_call(*args, **kwargs)
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/sharp/model.py", line 1118, in forward
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/sharp/model.py", line 1118, in forward
[ComfyUI] decoder_features = self.monodepth_predictor.decoder(encoder_features)
[ComfyUI] decoder_features = self.monodepth_predictor.decoder(encoder_features)
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
[ComfyUI] return self._call_impl(*args, **kwargs)
[ComfyUI] return self._call_impl(*args, **kwargs)
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl
[ComfyUI] return forward_call(*args, **kwargs)
[ComfyUI] return forward_call(*args, **kwargs)
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/sharp/model.py", line 970, in forward
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/sharp/model.py", line 970, in forward
[ComfyUI] features = self.fusions[i](features, features_i)
[ComfyUI] features = self.fusions[i](features, features_i)
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
[ComfyUI] return self._call_impl(*args, **kwargs)
[ComfyUI] return self._call_impl(*args, **kwargs)
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl
[ComfyUI] return forward_call(*args, **kwargs)
[ComfyUI] return forward_call(*args, **kwargs)
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/sharp/model.py", line 862, in forward
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/sharp/model.py", line 862, in forward
[ComfyUI] x = self.out_conv(x)
[ComfyUI] x = self.out_conv(x)
[ComfyUI] ^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
[ComfyUI] return self._call_impl(*args, **kwargs)
[ComfyUI] return self._call_impl(*args, **kwargs)
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl
[ComfyUI] return forward_call(*args, **kwargs)
[ComfyUI] return forward_call(*args, **kwargs)
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/comfy/ops.py", line 537, in forward
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/comfy/ops.py", line 537, in forward
[ComfyUI] return super().forward(*args, **kwargs)
[ComfyUI] return super().forward(*args, **kwargs)
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/conv.py", line 553, in forward
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/conv.py", line 553, in forward
[ComfyUI] return self._conv_forward(input, self.weight, self.bias)
[ComfyUI] return self._conv_forward(input, self.weight, self.bias)
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/conv.py", line 548, in _conv_forward
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/conv.py", line 548, in _conv_forward
[ComfyUI] return F.conv2d(
[ComfyUI] return F.conv2d(
[ComfyUI] ^^^^^^^^^
[ComfyUI] ^^^^^^^^^
[ComfyUI] RuntimeError: MPS backend out of memory (MPS allocated: 6.73 GiB, other allocations: 581.22 MiB, max allowed: 7.93 GiB). Tried to allocate 576.00 MiB on private pool. Use PYTORCH_MPS_HIGH_WATERMARK_RATIO=0.0 to disable upper limit for memory allocations (may cause system failure).
[ComfyUI] RuntimeError: MPS backend out of memory (MPS allocated: 6.73 GiB, other allocations: 581.22 MiB, max allowed: 7.93 GiB). Tried to allocate 576.00 MiB on private pool. Use PYTORCH_MPS_HIGH_WATERMARK_RATIO=0.0 to disable upper limit for memory allocations (may cause system failure).
[ComfyUI]
[ComfyUI]
[ComfyUI] [ERROR] Traceback (most recent call last):
[ComfyUI] [ERROR] Traceback (most recent call last):
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/execution.py", line 536, in execute
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/execution.py", line 536, in execute
[ComfyUI] output_data, output_ui, has_subgraph, has_pending_tasks = await get_output_data(prompt_id, unique_id, obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)
[ComfyUI] output_data, output_ui, has_subgraph, has_pending_tasks = await get_output_data(prompt_id, unique_id, obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/execution.py", line 336, in get_output_data
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/execution.py", line 336, in get_output_data
[ComfyUI] return_values = await _async_map_node_over_list(prompt_id, unique_id, obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)
[ComfyUI] return_values = await _async_map_node_over_list(prompt_id, unique_id, obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/execution.py", line 310, in _async_map_node_over_list
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/execution.py", line 310, in _async_map_node_over_list
[ComfyUI] await process_inputs(input_dict, i)
[ComfyUI] await process_inputs(input_dict, i)
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/execution.py", line 298, in process_inputs
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/execution.py", line 298, in process_inputs
[ComfyUI] result = f(**inputs)
[ComfyUI] result = f(**inputs)
[ComfyUI] ^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/venv/lib/python3.11/site-packages/comfy_env/isolation/metadata.py", line 556, in proxy
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/venv/lib/python3.11/site-packages/comfy_env/isolation/metadata.py", line 556, in proxy
[ComfyUI] result = worker.call_method(
[ComfyUI] result = worker.call_method(
[ComfyUI] ^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/venv/lib/python3.11/site-packages/comfy_env/isolation/workers/subprocess.py", line 674, in call_method
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/venv/lib/python3.11/site-packages/comfy_env/isolation/workers/subprocess.py", line 674, in call_method
[ComfyUI] raise WorkerError(
[ComfyUI] raise WorkerError(
[ComfyUI] comfy_env.isolation.workers.base.WorkerError: MPS backend out of memory (MPS allocated: 6.73 GiB, other allocations: 581.22 MiB, max allowed: 7.93 GiB). Tried to allocate 576.00 MiB on private pool. Use PYTORCH_MPS_HIGH_WATERMARK_RATIO=0.0 to disable upper limit for memory allocations (may cause system failure).
[ComfyUI] comfy_env.isolation.workers.base.WorkerError: MPS backend out of memory (MPS allocated: 6.73 GiB, other allocations: 581.22 MiB, max allowed: 7.93 GiB). Tried to allocate 576.00 MiB on private pool. Use PYTORCH_MPS_HIGH_WATERMARK_RATIO=0.0 to disable upper limit for memory allocations (may cause system failure).
[ComfyUI]
[ComfyUI]
[ComfyUI] Worker traceback:
[ComfyUI] Worker traceback:
[ComfyUI] Traceback (most recent call last):
[ComfyUI] Traceback (most recent call last):
[ComfyUI] File "/var/folders/8d/778wjbv96mq1760tv6gk374m0000gn/T/comfyui_pvenv_zcu5iw2_/persistent_worker.py", line 1762, in main
[ComfyUI] File "/var/folders/8d/778wjbv96mq1760tv6gk374m0000gn/T/comfyui_pvenv_zcu5iw2_/persistent_worker.py", line 1762, in main
[ComfyUI] result = method(**inputs)
[ComfyUI] result = method(**inputs)
[ComfyUI] ^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/comfy_api/latest/_io.py", line 1892, in EXECUTE_NORMALIZED
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/comfy_api/latest/_io.py", line 1892, in EXECUTE_NORMALIZED
[ComfyUI] to_return = cls.execute(*args, **kwargs)
[ComfyUI] to_return = cls.execute(*args, **kwargs)
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 124, in decorate_context
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 124, in decorate_context
[ComfyUI] return func(*args, **kwargs)
[ComfyUI] return func(*args, **kwargs)
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/predict.py", line 239, in execute
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/predict.py", line 239, in execute
[ComfyUI] gaussians, refined_depth_l0, refined_depth_l1 = _predict_image_cached(
[ComfyUI] gaussians, refined_depth_l0, refined_depth_l1 = _predict_image_cached(
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/predict.py", line 348, in _predict_image_cached
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/predict.py", line 348, in _predict_image_cached
[ComfyUI] monodepth_output, _ = predictor.encode(image_resized_pt)
[ComfyUI] monodepth_output, _ = predictor.encode(image_resized_pt)
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/sharp/model.py", line 2006, in encode
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/sharp/model.py", line 2006, in encode
[ComfyUI] monodepth_output = self.monodepth_model(image)
[ComfyUI] monodepth_output = self.monodepth_model(image)
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
[ComfyUI] return self._call_impl(*args, **kwargs)
[ComfyUI] return self._call_impl(*args, **kwargs)
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl
[ComfyUI] return forward_call(*args, **kwargs)
[ComfyUI] return forward_call(*args, **kwargs)
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/sharp/model.py", line 1118, in forward
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/sharp/model.py", line 1118, in forward
[ComfyUI] decoder_features = self.monodepth_predictor.decoder(encoder_features)
[ComfyUI] decoder_features = self.monodepth_predictor.decoder(encoder_features)
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
[ComfyUI] return self._call_impl(*args, **kwargs)
[ComfyUI] return self._call_impl(*args, **kwargs)
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl
[ComfyUI] return forward_call(*args, **kwargs)
[ComfyUI] return forward_call(*args, **kwargs)
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/sharp/model.py", line 970, in forward
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/sharp/model.py", line 970, in forward
[ComfyUI] features = self.fusions[i](features, features_i)
[ComfyUI] features = self.fusions[i](features, features_i)
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
[ComfyUI] return self._call_impl(*args, **kwargs)
[ComfyUI] return self._call_impl(*args, **kwargs)
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl
[ComfyUI] return forward_call(*args, **kwargs)
[ComfyUI] return forward_call(*args, **kwargs)
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/sharp/model.py", line 862, in forward
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/sharp/model.py", line 862, in forward
[ComfyUI] x = self.out_conv(x)
[ComfyUI] x = self.out_conv(x)
[ComfyUI] ^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
[ComfyUI] return self._call_impl(*args, **kwargs)
[ComfyUI] return self._call_impl(*args, **kwargs)
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl
[ComfyUI] return forward_call(*args, **kwargs)
[ComfyUI] return forward_call(*args, **kwargs)
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/comfy/ops.py", line 537, in forward
[ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/comfy/ops.py", line 537, in forward
[ComfyUI] return super().forward(*args, **kwargs)
[ComfyUI] return super().forward(*args, **kwargs)
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/conv.py", line 553, in forward
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/conv.py", line 553, in forward
[ComfyUI] return self._conv_forward(input, self.weight, self.bias)
[ComfyUI] return self._conv_forward(input, self.weight, self.bias)
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/conv.py", line 548, in _conv_forward
[ComfyUI] File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/conv.py", line 548, in _conv_forward
[ComfyUI] return F.conv2d(
[ComfyUI] return F.conv2d(
[ComfyUI] ^^^^^^^^^
[ComfyUI] ^^^^^^^^^
[ComfyUI] RuntimeError: MPS backend out of memory (MPS allocated: 6.73 GiB, other allocations: 581.22 MiB, max allowed: 7.93 GiB). Tried to allocate 576.00 MiB on private pool. Use PYTORCH_MPS_HIGH_WATERMARK_RATIO=0.0 to disable upper limit for memory allocations (may cause system failure).
[ComfyUI] RuntimeError: MPS backend out of memory (MPS allocated: 6.73 GiB, other allocations: 581.22 MiB, max allowed: 7.93 GiB). Tried to allocate 576.00 MiB on private pool. Use PYTORCH_MPS_HIGH_WATERMARK_RATIO=0.0 to disable upper limit for memory allocations (may cause system failure).
[ComfyUI]
[ComfyUI]
[ComfyUI]
[ComfyUI]
[ComfyUI] [INFO] Prompt executed in 86.82 seconds
[ComfyUI] [INFO] Prompt executed in 86.82 seconds
Execution complete (t=87.9s)
Execution error: {'prompt_id': '8a0433fa-7d52-4d8b-b620-62151ba4d2b5', 'node_id': '7', 'node_type': 'SharpPredict', 'executed': ['2', '9'], 'exception_message': 'MPS backend out of memory (MPS allocated: 6.73 GiB, other allocations: 581.22 MiB, max allowed: 7.93 GiB). Tried to allocate 576.00 MiB on private pool. Use PYTORCH_MPS_HIGH_WATERMARK_RATIO=0.0 to disable upper limit for memory allocations (may cause system failure).\n\nWorker traceback:\nTraceback (most recent call last):\n File "/var/folders/8d/778wjbv96mq1760tv6gk374m0000gn/T/comfyui_pvenv_zcu5iw2_/persistent_worker.py", line 1762, in main\n result = method(**inputs)\n ^^^^^^^^^^^^^^^^\n File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/comfy_api/latest/_io.py", line 1892, in EXECUTE_NORMALIZED\n to_return = cls.execute(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 124, in decorate_context\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/predict.py", line 239, in execute\n gaussians, refined_depth_l0, refined_depth_l1 = _predict_image_cached(\n ^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/predict.py", line 348, in _predict_image_cached\n monodepth_output, _ = predictor.encode(image_resized_pt)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/sharp/model.py", line 2006, in encode\n monodepth_output = self.monodepth_model(image)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl\n return self._call_impl(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl\n return forward_call(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/sharp/model.py", line 1118, in forward\n decoder_features = self.monodepth_predictor.decoder(encoder_features)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl\n return self._call_impl(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl\n return forward_call(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/sharp/model.py", line 970, in forward\n features = self.fusions[i](features, features_i)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl\n return self._call_impl(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl\n return forward_call(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/sharp/model.py", line 862, in forward\n x = self.out_conv(x)\n ^^^^^^^^^^^^^^^^\n File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl\n return self._call_impl(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl\n return forward_call(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/comfy/ops.py", line 537, in forward\n return super().forward(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/conv.py", line 553, in forward\n return self._conv_forward(input, self.weight, self.bias)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/conv.py", line 548, in _conv_forward\n return F.conv2d(\n ^^^^^^^^^\nRuntimeError: MPS backend out of memory (MPS allocated: 6.73 GiB, other allocations: 581.22 MiB, max allowed: 7.93 GiB). Tried to allocate 576.00 MiB on private pool. Use PYTORCH_MPS_HIGH_WATERMARK_RATIO=0.0 to disable upper limit for memory allocations (may cause system failure).\n\n', 'exception_type': 'comfy_env.isolation.workers.base.WorkerError', 'traceback': [' File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/execution.py", line 536, in execute\n output_data, output_ui, has_subgraph, has_pending_tasks = await get_output_data(prompt_id, unique_id, obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n', ' File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/execution.py", line 336, in get_output_data\n return_values = await _async_map_node_over_list(prompt_id, unique_id, obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n', ' File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/execution.py", line 310, in _async_map_node_over_list\n await process_inputs(input_dict, i)\n', ' File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/execution.py", line 298, in process_inputs\n result = f(**inputs)\n ^^^^^^^^^^^\n', ' File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/venv/lib/python3.11/site-packages/comfy_env/isolation/metadata.py", line 556, in proxy\n result = worker.call_method(\n ^^^^^^^^^^^^^^^^^^^\n', ' File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/venv/lib/python3.11/site-packages/comfy_env/isolation/workers/subprocess.py", line 674, in call_method\n raise WorkerError(\n'], 'current_inputs': {'focal_length_mm': [20], 'output_prefix': ['sharp'], 'model': ["{'model_path': '/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/models/sharp/sharp_2572gikvuh.pt', 'precision': 'auto', 'dtype': 'bf16'}"], 'image': ['tensor([[[[0.2157, 0.1647, 0.1569],\n [0.2275, 0.1608, 0.1608],\n [0.2275, 0.1647, 0.1529],\n ...,\n [0.4471, 0.4745, 0.4863],\n [0.4510, 0.4745, 0.4863],\n [0.4431, 0.4745, 0.4745]],\n\n [[0.2196, 0.1686, 0.1569],\n [0.2314, 0.1647, 0.1608],\n [0.2353, 0.1647, 0.1647],\n ...,\n [0.4549, 0.4784, 0.4706],\n [0.4549, 0.4706, 0.4784],\n [0.4431, 0.4706, 0.4667]],\n\n [[0.2314, 0.1725, 0.1569],\n [0.2510, 0.1725, 0.1608],\n [0.2392, 0.1725, 0.1686],\n ...,\n [0.4588, 0.4745, 0.4627],\n [0.4510, 0.4745, 0.4667],\n [0.4471, 0.4706, 0.4627]],\n\n ...,\n\n [[0.1569, 0.1647, 0.1843],\n [0.1647, 0.1725, 0.1882],\n [0.1490, 0.1686, 0.1922],\n ...,\n [0.1020, 0.1137, 0.1373],\n [0.1137, 0.1176, 0.1412],\n [0.0863, 0.1176, 0.1373]],\n\n [[0.1569, 0.1686, 0.1804],\n [0.1569, 0.1647, 0.1961],\n [0.1490, 0.1647, 0.1961],\n ...,\n [0.1059, 0.1216, 0.1294],\n [0.1176, 0.1216, 0.1333],\n [0.1059, 0.1137, 0.1373]],\n\n [[0.1608, 0.1686, 0.1843],\n [0.1608, 0.1686, 0.1882],\n [0.1569, 0.1686, 0.1882],\n ...,\n [0.1059, 0.1176, 0.1333],\n [0.1137, 0.1176, 0.1333],\n [0.1333, 0.1176, 0.1333]]]])'], 'prompt': ["{'2': {'inputs': {'precision': 'auto'}, 'class_type': 'LoadSharpModel', '_meta': {'title': '(Down)Load SHARP Model'}}, '7': {'inputs': {'focal_length_mm': ['9', 2], 'output_prefix': 'sharp', 'model': ['2', 0], 'image': ['9', 0]}, 'class_type': 'SharpPredict', '_meta': {'title': 'SHARP Predict (Image to PLY)'}}, '9': {'inputs': {'image': 'eth3d.png', 'default_focal_mm': 30.0}, 'class_type': 'LoadImageWithExif', '_meta': {'title': 'Load Image with EXIF (Focal Length)'}}, '11': {'inputs': {'preview_gaussian': '', 'ply_path': ['7', 0], 'extrinsics': ['7', 1], 'intrinsics': ['7', 2]}, 'class_type': 'PreviewGaussianSharp', '_meta': {'title': 'Preview Gaussian Sharp'}}}"], 'extra_pnginfo': ["{'workflow': {'id': 'sharp-basic-workflow', 'revision': 0, 'last_node_id': 11, 'last_link_id': 24, 'nodes': [{'id': 2, 'type': 'LoadSharpModel', 'pos': [-150, -400], 'size': [270, 106], 'flags': {}, 'order': 0, 'mode': 0, 'inputs': [], 'outputs': [{'name': 'model_config', 'type': 'SHARP_MODEL_CONFIG', 'links': [11]}], 'properties': {'Node name for S&R': 'LoadSharpModel', 'aux_id': 'PozzettiAndrea/ComfyUI-Sharp', 'ver': '9f1b9c726e2cfc0118866c3df783e7f6bc895226', 'cnr_id': 'comfyui-sharp'}, 'widgets_values': ['auto']}, {'id': 9, 'type': 'LoadImageWithExif', 'pos': [-208.74978010739324, -99.02778023242435], 'size': [342.28125, 420], 'flags': {}, 'order': 1, 'mode': 0, 'inputs': [], 'outputs': [{'name': 'image', 'type': 'IMAGE', 'links': [19]}, {'name': 'mask', 'type': 'MASK', 'links': None}, {'name': 'focal_length_mm', 'type': 'FLOAT', 'links': [20]}], 'properties': {'Node name for S&R': 'LoadImageWithExif', 'aux_id': 'PozzettiAndrea/ComfyUI-Sharp', 'ver': 'c69751a59b587c3b83d0e37cfe2bab3bc838574a', 'cnr_id': 'comfyui-sharp'}, 'widgets_values': ['eth3d.png', 30, 'image']}, {'id': 7, 'type': 'SharpPredict', 'pos': [226.15696022727278, -245.70490056818184], 'size': [298.890625, 236], 'flags': {}, 'order': 2, 'mode': 0, 'inputs': [{'name': 'model', 'type': 'SHARP_MODEL_CONFIG', 'link': 11}, {'name': 'image', 'type': 'IMAGE', 'link': 19}, {'name': 'extrinsics', 'shape': 7, 'type': 'EXTRINSICS', 'link': None}, {'name': 'intrinsics', 'shape': 7, 'type': 'INTRINSICS', 'link': None}, {'name': 'focal_length_mm', 'shape': 7, 'type': 'FLOAT', 'widget': {'name': 'focal_length_mm'}, 'link': 20}], 'outputs': [{'name': 'ply_path', 'type': 'STRING', 'links': [22]}, {'name': 'extrinsics', 'type': 'EXTRINSICS', 'links': [23]}, {'name': 'intrinsics', 'type': 'INTRINSICS', 'links': [24]}, {'name': 'layer_0_refined_depth', 'type': 'IMAGE', 'links': None}, {'name': 'layer_1_refined_depth', 'type': 'IMAGE', 'links': None}], 'properties': {'Node name for S&R': 'SharpPredict', 'aux_id': 'PozzettiAndrea/ComfyUI-Sharp', 'ver': 'c69751a59b587c3b83d0e37cfe2bab3bc838574a', 'cnr_id': 'comfyui-sharp'}, 'widgets_values': [30, 'sharp']}, {'id': 11, 'type': 'PreviewGaussianSharp', 'pos': [600.5860117362482, -287.72961200652577], 'size': [512, 658], 'flags': {}, 'order': 3, 'mode': 0, 'inputs': [{'name': 'ply_path', 'type': 'STRING', 'link': 22}, {'name': 'extrinsics', 'shape': 7, 'type': 'EXTRINSICS', 'link': 23}, {'name': 'intrinsics', 'shape': 7, 'type': 'INTRINSICS', 'link': 24}], 'outputs': [{'name': 'extrinsics', 'type': 'EXTRINSICS', 'links': None}, {'name': 'intrinsics', 'type': 'INTRINSICS', 'links': None}], 'properties': {'Node name for S&R': 'PreviewGaussianSharp', 'aux_id': 'PozzettiAndrea/ComfyUI-GaussianPack', 'ver': '059313a53ffa7ecc8697eddc02d2cee4727e5085'}, 'widgets_values': ['']}], 'links': [[11, 2, 0, 7, 0, 'SHARP_MODEL'], [19, 9, 0, 7, 1, 'IMAGE'], [20, 9, 2, 7, 4, 'FLOAT'], [22, 7, 0, 11, 0, 'STRING'], [23, 7, 1, 11, 1, 'EXTRINSICS'], [24, 7, 2, 11, 2, 'INTRINSICS']], 'groups': [], 'config': {}, 'extra': {'ds': {'scale': 0.9373016969629105, 'offset': [449.06099628194596, 596.4219590460706]}, 'workflowRendererVersion': 'LG', 'frontendVersion': '1.45.15'}, 'version': 0.4}}"]}, 'current_outputs': ['2', '9', '11', '7'], 'timestamp': 1780860197909}
Status: FAILED
Error: Workflow execution failed: {'prompt_id': '8a0433fa-7d52-4d8b-b620-62151ba4d2b5', 'node_id': '7', 'node_type': 'SharpPredict', 'executed': ['2', '9'], 'exception_message': 'MPS backend out of memory (MPS allocated: 6.73 GiB, other allocations: 581.22 MiB, max allowed: 7.93 GiB). Tried to allocate 576.00 MiB on private pool. Use PYTORCH_MPS_HIGH_WATERMARK_RATIO=0.0 to disable upper limit for memory allocations (may cause system failure).\n\nWorker traceback:\nTraceback (most recent call last):\n File "/var/folders/8d/778wjbv96mq1760tv6gk374m0000gn/T/comfyui_pvenv_zcu5iw2_/persistent_worker.py", line 1762, in main\n result = method(**inputs)\n ^^^^^^^^^^^^^^^^\n File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/comfy_api/latest/_io.py", line 1892, in EXECUTE_NORMALIZED\n to_return = cls.execute(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 124, in decorate_context\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/predict.py", line 239, in execute\n gaussians, refined_depth_l0, refined_depth_l1 = _predict_image_cached(\n ^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/predict.py", line 348, in _predict_image_cached\n monodepth_output, _ = predictor.encode(image_resized_pt)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/sharp/model.py", line 2006, in encode\n monodepth_output = self.monodepth_model(image)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl\n return self._call_impl(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl\n return forward_call(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/sharp/model.py", line 1118, in forward\n decoder_features = self.monodepth_predictor.decoder(encoder_features)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl\n return self._call_impl(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl\n return forward_call(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/sharp/model.py", line 970, in forward\n features = self.fusions[i](features, features_i)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl\n return self._call_impl(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl\n return forward_call(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/custom_nodes/ComfyUI-Sharp/nodes/sharp/model.py", line 862, in forward\n x = self.out_conv(x)\n ^^^^^^^^^^^^^^^^\n File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl\n return self._call_impl(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1787, in _call_impl\n return forward_call(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/comfy/ops.py", line 537, in forward\n return super().forward(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/conv.py", line 553, in forward\n return self._conv_forward(input, self.weight, self.bias)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/.ce/.pixi/envs/sharp-nodes/lib/python3.11/site-packages/torch/nn/modules/conv.py", line 548, in _conv_forward\n return F.conv2d(\n ^^^^^^^^^\nRuntimeError: MPS backend out of memory (MPS allocated: 6.73 GiB, other allocations: 581.22 MiB, max allowed: 7.93 GiB). Tried to allocate 576.00 MiB on private pool. Use PYTORCH_MPS_HIGH_WATERMARK_RATIO=0.0 to disable upper limit for memory allocations (may cause system failure).\n\n', 'exception_type': 'comfy_env.isolation.workers.base.WorkerError', 'traceback': [' File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/execution.py", line 536, in execute\n output_data, output_ui, has_subgraph, has_pending_tasks = await get_output_data(prompt_id, unique_id, obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n', ' File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/execution.py", line 336, in get_output_data\n return_values = await _async_map_node_over_list(prompt_id, unique_id, obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n', ' File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/execution.py", line 310, in _async_map_node_over_list\n await process_inputs(input_dict, i)\n', ' File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/execution.py", line 298, in process_inputs\n result = f(**inputs)\n ^^^^^^^^^^^\n', ' File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/venv/lib/python3.11/site-packages/comfy_env/isolation/metadata.py", line 556, in proxy\n result = worker.call_method(\n ^^^^^^^^^^^^^^^^^^^\n', ' File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/venv/lib/python3.11/site-packages/comfy_env/isolation/workers/subprocess.py", line 674, in call_method\n raise WorkerError(\n'], 'current_inputs': {'focal_length_mm': [20], 'output_prefix': ['sharp'], 'model': ["{'model_path': '/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/Sharp-1915/ComfyUI/models/sharp/sharp_2572gikvuh.pt', 'precision': 'auto', 'dtype': 'bf16'}"], 'image': ['tensor([[[[0.2157, 0.1647, 0.1569],\n [0.2275, 0.1608, 0.1608],\n [0.2275, 0.1647, 0.1529],\n ...,\n [0.4471, 0.4745, 0.4863],\n [0.4510, 0.4745, 0.4863],\n [0.4431, 0.4745, 0.4745]],\n\n [[0.2196, 0.1686, 0.1569],\n [0.2314, 0.1647, 0.1608],\n [0.2353, 0.1647, 0.1647],\n ...,\n [0.4549, 0.4784, 0.4706],\n [0.4549, 0.4706, 0.4784],\n [0.4431, 0.4706, 0.4667]],\n\n [[0.2314, 0.1725, 0.1569],\n [0.2510, 0.1725, 0.1608],\n [0.2392, 0.1725, 0.1686],\n ...,\n [0.4588, 0.4745, 0.4627],\n [0.4510, 0.4745, 0.4667],\n [0.4471, 0.4706, 0.4627]],\n\n ...,\n\n [[0.1569, 0.1647, 0.1843],\n [0.1647, 0.1725, 0.1882],\n [0.1490, 0.1686, 0.1922],\n ...,\n [0.1020, 0.1137, 0.1373],\n [0.1137, 0.1176, 0.1412],\n [0.0863, 0.1176, 0.1373]],\n\n [[0.1569, 0.1686, 0.1804],\n [0.1569, 0.1647, 0.1961],\n [0.1490, 0.1647, 0.1961],\n ...,\n [0.1059, 0.1216, 0.1294],\n [0.1176, 0.1216, 0.1333],\n [0.1059, 0.1137, 0.1373]],\n\n [[0.1608, 0.1686, 0.1843],\n [0.1608, 0.1686, 0.1882],\n [0.1569, 0.1686, 0.1882],\n ...,\n [0.1059, 0.1176, 0.1333],\n [0.1137, 0.1176, 0.1333],\n [0.1333, 0.1176, 0.1333]]]])'], 'prompt': ["{'2': {'inputs': {'precision': 'auto'}, 'class_type': 'LoadSharpModel', '_meta': {'title': '(Down)Load SHARP Model'}}, '7': {'inputs': {'focal_length_mm': ['9', 2], 'output_prefix': 'sharp', 'model': ['2', 0], 'image': ['9', 0]}, 'class_type': 'SharpPredict', '_meta': {'title': 'SHARP Predict (Image to PLY)'}}, '9': {'inputs': {'image': 'eth3d.png', 'default_focal_mm': 30.0}, 'class_type': 'LoadImageWithExif', '_meta': {'title': 'Load Image with EXIF (Focal Length)'}}, '11': {'inputs': {'preview_gaussian': '', 'ply_path': ['7', 0], 'extrinsics': ['7', 1], 'intrinsics': ['7', 2]}, 'class_type': 'PreviewGaussianSharp', '_meta': {'title': 'Preview Gaussian Sharp'}}}"], 'extra_pnginfo': ["{'workflow': {'id': 'sharp-basic-workflow', 'revision': 0, 'last_node_id': 11, 'last_link_id': 24, 'nodes': [{'id': 2, 'type': 'LoadSharpModel', 'pos': [-150, -400], 'size': [270, 106], 'flags': {}, 'order': 0, 'mode': 0, 'inputs': [], 'outputs': [{'name': 'model_config', 'type': 'SHARP_MODEL_CONFIG', 'links': [11]}], 'properties': {'Node name for S&R': 'LoadSharpModel', 'aux_id': 'PozzettiAndrea/ComfyUI-Sharp', 'ver': '9f1b9c726e2cfc0118866c3df783e7f6bc895226', 'cnr_id': 'comfyui-sharp'}, 'widgets_values': ['auto']}, {'id': 9, 'type': 'LoadImageWithExif', 'pos': [-208.74978010739324, -99.02778023242435], 'size': [342.28125, 420], 'flags': {}, 'order': 1, 'mode': 0, 'inputs': [], 'outputs': [{'name': 'image', 'type': 'IMAGE', 'links': [19]}, {'name': 'mask', 'type': 'MASK', 'links': None}, {'name': 'focal_length_mm', 'type': 'FLOAT', 'links': [20]}], 'properties': {'Node name for S&R': 'LoadImageWithExif', 'aux_id': 'PozzettiAndrea/ComfyUI-Sharp', 'ver': 'c69751a59b587c3b83d0e37cfe2bab3bc838574a', 'cnr_id': 'comfyui-sharp'}, 'widgets_values': ['eth3d.png', 30, 'image']}, {'id': 7, 'type': 'SharpPredict', 'pos': [226.15696022727278, -245.70490056818184], 'size': [298.890625, 236], 'flags': {}, 'order': 2, 'mode': 0, 'inputs': [{'name': 'model', 'type': 'SHARP_MODEL_CONFIG', 'link': 11}, {'name': 'image', 'type': 'IMAGE', 'link': 19}, {'name': 'extrinsics', 'shape': 7, 'type': 'EXTRINSICS', 'link': None}, {'name': 'intrinsics', 'shape': 7, 'type': 'INTRINSICS', 'link': None}, {'name': 'focal_length_mm', 'shape': 7, 'type': 'FLOAT', 'widget': {'name': 'focal_length_mm'}, 'link': 20}], 'outputs': [{'name': 'ply_path', 'type': 'STRING', 'links': [22]}, {'name': 'extrinsics', 'type': 'EXTRINSICS', 'links': [23]}, {'name': 'intrinsics', 'type': 'INTRINSICS', 'links': [24]}, {'name': 'layer_0_refined_depth', 'type': 'IMAGE', 'links': None}, {'name': 'layer_1_refined_depth', 'type': 'IMAGE', 'links': None}], 'properties': {'Node name for S&R': 'SharpPredict', 'aux_id': 'PozzettiAndrea/ComfyUI-Sharp', 'ver': 'c69751a59b587c3b83d0e37cfe2bab3bc838574a', 'cnr_id': 'comfyui-sharp'}, 'widgets_values': [30, 'sharp']}, {'id': 11, 'type': 'PreviewGaussianSharp', 'pos': [600.5860117362482, -287.72961200652577], 'size': [512, 658], 'flags': {}, 'order': 3, 'mode': 0, 'inputs': [{'name': 'ply_path', 'type': 'STRING', 'link': 22}, {'name': 'extrinsics', 'shape': 7, 'type': 'EXTRINSICS', 'link': 23}, {'name': 'intrinsics', 'shape': 7, 'type': 'INTRINSICS', 'link': 24}], 'outputs': [{'name': 'extrinsics', 'type': 'EXTRINSICS', 'links': None}, {'name': 'intrinsics', 'type': 'INTRINSICS', 'links': None}], 'properties': {'Node name for S&R': 'PreviewGaussianSharp', 'aux_id': 'PozzettiAndrea/ComfyUI-GaussianPack', 'ver': '059313a53ffa7ecc8697eddc02d2cee4727e5085'}, 'widgets_values': ['']}], 'links': [[11, 2, 0, 7, 0, 'SHARP_MODEL'], [19, 9, 0, 7, 1, 'IMAGE'], [20, 9, 2, 7, 4, 'FLOAT'], [22, 7, 0, 11, 0, 'STRING'], [23, 7, 1, 11, 1, 'EXTRINSICS'], [24, 7, 2, 11, 2, 'INTRINSICS']], 'groups': [], 'config': {}, 'extra': {'ds': {'scale': 0.9373016969629105, 'offset': [449.06099628194596, 596.4219590460706]}, 'workflowRendererVersion': 'LG', 'frontendVersion': '1.45.15'}, 'version': 0.4}}"]}, 'current_outputs': ['2', '9', '11', '7'], 'timestamp': 1780860197909}
Details: Workflow: /private/var/folders/8d/778wjbv96mq1760tv6gk374m0000gn/T/comfy-test-run-n4s1o4ug/ComfyUI-Sharp/workflows/sharp_exif.json
Node error: SharpPredict