[00:00] Crash dump logging enabled: /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/comfy-test-logs/SAM3-1659/dev/macos-cpu/crash_dump.log [00:00] [00:00] [1/5] SYNTAX [00:00] ---------------------------------------- [00:00] [DEBUG] server=None, server_url=None, api=None [00:00] Found pyproject.toml (modern format) [00:00] Found requirements.txt (legacy format) [00:00] Unicode check: OK (all characters cp1252-safe) [00:00] Forbidden patterns check: OK [00:00] [SYNTAX] PASSED [00:00] [00:00] [2/5] INSTALL [00:00] ---------------------------------------- [00:00] [DEBUG] server=None, server_url=None, api=None [00:00] Setting up ComfyUI... [00:00] Cloning ComfyUI (latest)... [00:00] Running: git clone --depth 1 https://github.com/comfyanonymous/ComfyUI.git /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI [00:01] Creating virtual environment... [00:01] Running: /Library/Frameworks/Python.framework/Versions/3.11/bin/python -m venv /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/venv [00:04] Installing uv into venv... [00:04] Running: /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/venv/bin/python -m pip install uv [00:05] Installing PyTorch... [00:05] Running: /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/venv/bin/python -m uv pip install torch torchvision torchaudio [00:13] Installing ComfyUI requirements... [00:13] Running: /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/venv/bin/python -m uv pip install -r /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/requirements.txt [00:20] Installing custom node... [00:20] Copying ComfyUI-SAM3 to custom_nodes/... [00:20] Installing node requirements... [00:20] Running: /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/venv/bin/python -m uv pip install -r /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3/requirements.txt [00:20] Running install.py... [00:20] Running: /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/venv/bin/python /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3/install.py [00:21] [00:21] Installing 2 node dependencies... [00:21] Cloning ComfyUI-VideoHelperSuite... [00:21] Installing requirements for ComfyUI-VideoHelperSuite... [00:22] Cloning ComfyUI-Multiband... [00:23] Installing requirements for ComfyUI-Multiband... [00:23] [00:23] [requirements] Re-installing main package requirements... [00:23] Installing requirements for ComfyUI-SAM3... [00:24] [comfy-env] Workspace: /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce [00:24] [comfy-env] ComfyUI: /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI [00:24] [comfy-env] Found 1 node config(s): [00:24] - sam3-nodes <- custom_nodes/ComfyUI-SAM3/nodes/comfy-env.toml (python=host) [00:24] [comfy-env] Bootstrap interpreter has python 3.11 + torch 2.11.0 (cpu). Looking for cuda-only wheels matching cpu/torch2.11/cp311. [00:24] [comfy-env] Root conda deps from ComfyUI-SAM3: ['av', 'ffmpeg'] [00:24] [comfy-env] Comfyui feature pin: torch ==2.11.0, torchvision ==0.26.*, torchaudio ==2.11.* [00:24] [comfy-env] Root conda deps: ['av', 'ffmpeg'] [00:24] [comfy-env] comfyui311: python 3.11, torch from https://download.pytorch.org/whl/cpu [00:24] Generated /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/pixi.toml [00:24] [comfy-env] Installing 1 environment(s): [00:24] - sam3-nodes (python=host, 17 pypi deps, 3 cuda wheels) [00:24] [comfy-env] Running `pixi install --all`... [00:32] WARN Skipped running the post-link scripts because `run-post-link-scripts` = `false` [00:32] - bin/.librsvg-pre-unlink.sh [00:32] To enable them, run: [00:32] pixi config set --local run-post-link-scripts insecure [00:32] More info: [00:32] https://pixi.sh/latest/reference/pixi_configuration/#run-post-link-scripts [00:35] WARN Skipped running the post-link scripts because `run-post-link-scripts` = `false` [00:35] - bin/.librsvg-pre-unlink.sh [00:35] To enable them, run: [00:35] pixi config set --local run-post-link-scripts insecure [00:35] More info: [00:35] https://pixi.sh/latest/reference/pixi_configuration/#run-post-link-scripts [00:46] ✔ The following environments have been installed: default, comfyui, sam3-nodes. [00:46] [comfy-env] Note: env `default` is on disk but no node declares it in this run. Leaving as-is. Remove via `pixi clean --environment default` if intended. [00:46] [comfy-env] sam3-nodes: deduped libomp [00:46] [comfy-env] Install log: /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/install.log [00:46] [00:46] Installation complete! [00:46] Installing 2 node dependency(ies)... [00:46] ComfyUI-VideoHelperSuite from Kosinkadink/ComfyUI-VideoHelperSuite [00:46] ComfyUI-VideoHelperSuite already exists, skipping... [00:46] ComfyUI-Multiband from PozzettiAndrea/ComfyUI-Multiband [00:46] ComfyUI-Multiband already exists, skipping... [00:46] Installing validation endpoint... [00:46] Cloning PozzettiAndrea/ComfyUI-validate-endpoint... [00:46] Running: git clone --depth 1 https://github.com/PozzettiAndrea/ComfyUI-validate-endpoint.git /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-validate-endpoint [00:47] CUDA packages absent (will mock): cc_torch, torch_generic_nms, flash_attn [00:47] [INSTALL] PASSED [00:47] [00:47] [3/5] REGISTRATION [00:47] ---------------------------------------- [00:47] [DEBUG] server=None, server_url=None, api=None [00:47] Starting ComfyUI server... [00:47] Starting ComfyUI server on port 41899... [00:47] CUDA mock packages: cc_torch, torch_generic_nms, flash_attn [00:47] Starting ComfyUI server on port 41899... [00:47] Waiting for server to be ready (timeout: 600s)... [00:48] [ComfyUI] setup plugin alembic.autogenerate.schemas [00:48] [ComfyUI] setup plugin alembic.autogenerate.tables [00:48] [ComfyUI] setup plugin alembic.autogenerate.types [00:48] [ComfyUI] setup plugin alembic.autogenerate.constraints [00:48] [ComfyUI] setup plugin alembic.autogenerate.defaults [00:48] [ComfyUI] setup plugin alembic.autogenerate.comments [00:48] [ComfyUI] Could not autodetect AIMDO implementation, assuming Nvidia [00:48] [ComfyUI] comfy-aimdo unsupported operating system: Darwin [00:48] [ComfyUI] NOTE: comfy-aimdo currently only supports Windows and Linux [00:48] [ComfyUI] [comfy-env] ComfyUI-SAM3: 1 isolation env(s): [00:48] [ComfyUI] [comfy-env] nodes -> /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes [00:50] [ComfyUI] [comfy-env] prestartup complete [00:50] [ComfyUI] [00:50] [ComfyUI] Prestartup times for custom nodes: [00:50] [ComfyUI] 2.2 seconds: /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3 [00:50] [ComfyUI] [00:50] [ComfyUI] WARNING: Potential Error in code: Torch already imported, torch should never be imported before this point. [00:53] [ComfyUI] Found comfy_kitchen backend triton: {'available': False, 'disabled': True, 'unavailable_reason': "ImportError: No module named 'triton'", 'capabilities': []} [00:53] [ComfyUI] Found comfy_kitchen backend eager: {'available': True, 'disabled': False, 'unavailable_reason': None, 'capabilities': ['apply_rope', 'apply_rope1', 'dequantize_mxfp8', 'dequantize_nvfp4', 'dequantize_per_tensor_fp8', 'quantize_mxfp8', 'quantize_nvfp4', 'quantize_per_tensor_fp8', 'scaled_mm_mxfp8', 'scaled_mm_nvfp4']} [00:53] [ComfyUI] Found comfy_kitchen backend cuda: {'available': False, 'disabled': True, 'unavailable_reason': 'Extension file not found: /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/venv/lib/python3.11/site-packages/comfy_kitchen/backends/cuda/_C.abi3.so', 'capabilities': []} [00:53] [ComfyUI] Checkpoint files will always be loaded safely. [00:53] [ComfyUI] Total VRAM 7168 MB, total RAM 7168 MB [00:53] [ComfyUI] pytorch version: 2.11.0 [00:53] [ComfyUI] Mac Version (15, 7, 4) [00:53] [ComfyUI] Set vram state to: SHARED [00:53] [ComfyUI] Device: mps [00:53] [ComfyUI] Using sub quadratic optimization for attention, if you have memory or speed issues try using: --use-split-cross-attention [00:58] [ComfyUI] objc[20398]: Class AVFFrameReceiver is implemented in both /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/venv/lib/python3.11/site-packages/cv2/.dylibs/libavdevice.61.3.100.dylib (0x11135c3a8) and /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/venv/lib/python3.11/site-packages/av/.dylibs/libavdevice.62.1.100.dylib (0x13c5083a8). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed. [00:58] [ComfyUI] objc[20398]: Class AVFAudioReceiver is implemented in both /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/venv/lib/python3.11/site-packages/cv2/.dylibs/libavdevice.61.3.100.dylib (0x11135c3f8) and /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/venv/lib/python3.11/site-packages/av/.dylibs/libavdevice.62.1.100.dylib (0x13c5083f8). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed. [01:00] [ComfyUI] Python version: 3.11.9 (v3.11.9:de54cf5be3, Apr 2 2024, 07:12:50) [Clang 13.0.0 (clang-1300.0.29.30)] [01:00] [ComfyUI] ComfyUI version: 0.20.1 [01:00] [ComfyUI] comfy-aimdo version: 0.3.0 [01:00] [ComfyUI] comfy-kitchen version: 0.2.8 [01:00] [ComfyUI] ****** User settings have been changed to be stored on the server instead of browser storage. ****** [01:00] [ComfyUI] ****** For multi-user setups add the --multi-user CLI argument to enable multiple user profiles. ****** [01:00] [ComfyUI] ComfyUI frontend version: 1.43.18 [01:00] [ComfyUI] [Prompt Server] web root: /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/venv/lib/python3.11/site-packages/comfyui_frontend_package/static [01:00] [ComfyUI] Asset seeder disabled [01:02] [ComfyUI] [comfy-env] Version: 0.3.78 [01:02] [ComfyUI] [comfy-env] register_nodes: ComfyUI-SAM3 [01:02] [ComfyUI] [comfy-env] ComfyUI source dir: /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI [01:02] [ComfyUI] [comfy-env] Found 1 isolation env(s) [01:02] [ComfyUI] [comfy-env] Scanning nodes metadata (isolation root)... [01:02] [ComfyUI] [comfy-env] Metadata scan: /Users/runner/.pixi/bin/pixi run --as-is --manifest-path /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/pixi.toml -e sam3-nodes python /var/folders/tb/y368xp_x10s3ty1b_mtl5mxr0000gn/T/comfy_meta_4t129u9w.py /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3 nodes [01:02] [ComfyUI] [comfy-env] Scan env PATH for nodes: [01:02] [ComfyUI] [comfy-env] [0] /Users/runner/hostedtoolcache/Python/3.11.9/arm64/bin [01:02] [ComfyUI] [comfy-env] [1] /Users/runner/hostedtoolcache/Python/3.11.9/arm64 [01:02] [ComfyUI] [comfy-env] [2] /opt/homebrew/lib/ruby/gems/3.3.0/bin [01:02] [ComfyUI] [comfy-env] [3] /opt/homebrew/opt/ruby@3.3/bin [01:02] [ComfyUI] [comfy-env] [4] /Users/runner/.local/bin [01:02] [ComfyUI] [comfy-env] [5] /opt/homebrew/bin [01:02] [ComfyUI] [comfy-env] [6] /opt/homebrew/sbin [01:02] [ComfyUI] [comfy-env] [7] /Users/runner/.cargo/bin [01:02] [ComfyUI] [comfy-env] [8] /usr/local/opt/curl/bin [01:02] [ComfyUI] [comfy-env] [9] /usr/local/bin [01:02] [ComfyUI] [comfy-env] [10] /usr/local/sbin [01:02] [ComfyUI] [comfy-env] [11] /Users/runner/bin [01:02] [ComfyUI] [comfy-env] [12] /Users/runner/.yarn/bin [01:02] [ComfyUI] [comfy-env] [13] /Users/runner/Library/Android/sdk/tools [01:02] [ComfyUI] [comfy-env] [14] /Users/runner/Library/Android/sdk/platform-tools [01:02] [ComfyUI] [comfy-env] [15] /Library/Frameworks/Python.framework/Versions/Current/bin [01:02] [ComfyUI] [comfy-env] [16] /Library/Frameworks/Mono.framework/Versions/Current/Commands [01:02] [ComfyUI] [comfy-env] [17] /usr/bin [01:02] [ComfyUI] [comfy-env] [18] /bin [01:02] [ComfyUI] [comfy-env] [19] /usr/sbin [01:02] [ComfyUI] [comfy-env] [20] /sbin [01:02] [ComfyUI] [comfy-env] [21] /Users/runner/.dotnet/tools [01:10] [ComfyUI] [comfy-env] Metadata scan stderr for nodes: [01:10] [ComfyUI] [comfy-env] [meta-scan] importing nodes from /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3 [01:10] [ComfyUI] [comfy-env] [meta-scan] import OK [01:10] [ComfyUI] [comfy-env] Scanned nodes: 15 nodes (7.8s) [01:10] [ComfyUI] [comfy-env] Scanned nodes root: 15 nodes (7.8s) [01:10] [ComfyUI] [comfy-env] Registered proxy route: POST /sam3/interactive_segment_one -> nodes.sam3_interactive._api_segment_one [01:10] [ComfyUI] [comfy-env] Imported nodes root: 15 nodes (isolation) [01:10] [ComfyUI] [comfy-env] Registered 15 total nodes [01:10] [ComfyUI] ComfyUI-Multiband loaded: 15 nodes registered [01:10] [ComfyUI] [01:10] [ComfyUI] Import times for custom nodes: [01:10] [ComfyUI] 0.0 seconds: /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-validate-endpoint [01:10] [ComfyUI] 0.0 seconds: /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/websocket_image_save.py [01:10] [ComfyUI] 0.1 seconds: /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-Multiband [01:10] [ComfyUI] 0.1 seconds: /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-VideoHelperSuite [01:10] [ComfyUI] 8.0 seconds: /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3 [01:10] [ComfyUI] [01:10] [ComfyUI] Context impl SQLiteImpl. [01:10] [ComfyUI] Will assume non-transactional DDL. [01:10] [ComfyUI] Context impl SQLiteImpl. [01:10] [ComfyUI] Will assume non-transactional DDL. [01:10] [ComfyUI] Running upgrade -> 0001_assets, Initial assets schema [01:10] [ComfyUI] Revision ID: 0001_assets [01:10] [ComfyUI] Revises: None [01:10] [ComfyUI] Create Date: 2025-12-10 00:00:00 [01:10] [ComfyUI] Running upgrade 0001_assets -> 0002_merge_to_asset_references, Merge AssetInfo and AssetCacheState into unified asset_references table. [01:10] [ComfyUI] Running upgrade 0002_merge_to_asset_references -> 0003_add_metadata_job_id, Add system_metadata and job_id columns to asset_references. [01:10] [ComfyUI] Change preview_id FK from assets.id to asset_references.id. [01:10] [ComfyUI] Database upgraded from None to 0003_add_metadata_job_id [01:10] [ComfyUI] Starting server [01:10] [ComfyUI] [01:10] [ComfyUI] To see the GUI go to: http://127.0.0.1:41899 [01:11] Server responding, waiting for nodes to load... [01:31] Server is ready! [01:31] Checking for import errors in server logs... [01:31] No import errors detected [01:31] Found 775 registered nodes [01:31] [REGISTRATION] PASSED [01:31] [01:31] [4/5] INSTANTIATION [01:31] ---------------------------------------- [01:31] [DEBUG] server=, server_url=None, api= [01:31] Testing node constructors... [01:36] All 15 node(s) instantiated successfully! [01:36] [INSTANTIATION] PASSED [01:36] [01:36] [5/5] EXECUTION [01:36] ---------------------------------------- [01:36] [DEBUG] server=, server_url=None, api= [01:36] CPU runner - will execute 4 workflow(s) [01:36] Running 5 workflow(s) (all with videos)... [01:36] Installing screenshot dependencies (playwright, pillow)... [01:37] Packages installed, downloading chromium browser... [01:46] Screenshot dependencies installed successfully [01:46] Starting headless browser... [01:46] macOS -- using Apple Metal GPU via ANGLE [01:47] Capturing execution frames: image_seg_point.json [01:48] [ComfyUI] [DEPRECATION WARNING] Detected import of deprecated legacy API: /extensions/core/widgetInputs.js. This is likely caused by a custom node extension using outdated APIs. Please update your extensions or contact the extension author for an updated version. [01:54] Frame 1 saved (frame_000.jpg, t=0.0s) [01:54] Validating workflow... [01:54] [ComfyUI] === /validate endpoint called === [01:54] [ComfyUI] Received JSON with keys: ['prompt'] [01:54] [ComfyUI] Prompt has 6 nodes [01:54] [ComfyUI] Node 1: LoadImage [01:54] [ComfyUI] Inputs: {"image": "example_image.jpg"} [01:54] [ComfyUI] Node 15: SAM3PointCollector [01:54] [ComfyUI] Inputs: {"points_store": "{\"positive\":[{\"x\":426.27735719657574,\"y\":321.42611685978835},{\"x\":417.13743597288953,\"y\":407.3413796907987},{\"x\":431.76130993078743,\"y\":549.9241563039649}],\"negative\" [01:54] [ComfyUI] Node 28: PreviewImage [01:54] [ComfyUI] Inputs: {"images": ["31", 2]} [01:54] [ComfyUI] Node 29: MaskPreview [01:54] [ComfyUI] Inputs: {"mask": ["31", 0]} [01:54] [ComfyUI] Node 30: LoadSAM3Model [01:54] [ComfyUI] Inputs: {"precision": "auto", "compile": false} [01:54] [ComfyUI] Node 31: SAM3Segmentation [01:54] [ComfyUI] Inputs: {"refinement_iterations": 0, "use_multimask": true, "output_best_mask": true, "sam3_model_config": ["30", 0], "image": ["1", 0], "positive_points": ["15", 0], "negative_points": ["15", 1]} [01:54] [ComfyUI] Generated prompt_id: efcf5dcc-86e8-42b0-9b0c-af08894e4ebc [01:54] [ComfyUI] Calling execution.validate_prompt()... [01:54] [ComfyUI] Validation result: valid=True [01:54] [ComfyUI] === Validation PASSED === [01:54] Queuing workflow for execution... [01:54] [ComfyUI] got prompt [01:54] [ComfyUI] [comfy-env] Running SAM3PointCollector: points_store: '{"positive":[{"x":426.27735719657574,"y":321.42611685978..., coordinates: '[{"x":426.27735719657574,"y":321.42611685978835},{"x":41..., neg_coordinates: '[]', image: torch.float32 [1,720,1280,3] cpu [01:54] [ComfyUI] [comfy-env] SubprocessWorker: /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/bin/python [01:54] [ComfyUI] [comfy-env] env_vars: {'COMFYUI_BASE': '/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI'} [01:54] [ComfyUI] [SubprocessWorker] call_method: nodes.sam3_interactive.SAM3PointCollector.EXECUTE_NORMALIZED [01:54] [ComfyUI] [SubprocessWorker] acquired lock, ensuring started... [01:54] [ComfyUI] [ComfyUI-SAM3] python: /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/bin/python [01:54] [ComfyUI] [ComfyUI-SAM3] sys_path sent to worker: ['/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI', '/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3', '/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/lib/python3.11/site-packages', '/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3'] [01:54] [ComfyUI] [SubprocessWorker] is_pixi=True, python=/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/bin/python [01:54] [ComfyUI] [SubprocessWorker] launching cmd=['/Users/runner/.pixi/bin/pixi', 'run', '--as-is', '--manifest-path', '/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/pixi.toml', '-e', 'sam3-nodes', 'python', '/var/folders/tb/y368xp_x10s3ty1b_mtl5mxr0000gn/T/comfyui_pvenv__jjhnh0w/persistent_worker.py', 'unix:///var/folders/tb/y368xp_x10s3ty1b_mtl5mxr0000gn/T/comfy_worker_474976201f3d.sock']... [01:54] [ComfyUI] [SubprocessWorker] socket_addr=unix:///var/folders/tb/y368xp_x10s3ty1b_mtl5mxr0000gn/T/comfy_worker_474976201f3d.sock exists=True [01:56] Frame 2 saved (frame_001.jpg, t=2.3s) [01:56] [capture-periodic] freeze=2ms shot=124ms unfreeze=4ms saved=True [01:57] [ComfyUI] [SubprocessWorker] worker started/confirmed [01:57] [ComfyUI] [comfy-env] PRE-SERIALIZE 'image' shape: torch.Size([1, 720, 1280, 3]) [01:57] [ComfyUI] [SubprocessWorker] serializing kwargs to shm... [01:57] [ComfyUI] [SubprocessWorker] created 0 shm blocks [01:57] [ComfyUI] [SubprocessWorker] sending request via socket... [01:57] [ComfyUI] [SubprocessWorker] call_id=1 sending call_method [01:57] [ComfyUI] [worker:ComfyUI-SAM3] [comfy-env] _from_shm got dict with keys: ['__type__', 'strategy', 'manager_path', 'storage_key', 'storage_size'] [01:57] [ComfyUI] [worker:ComfyUI-SAM3] [comfy-env] DESERIALIZE TensorRef: tensor_size=[1, 720, 1280, 3] [01:57] [ComfyUI] [worker:ComfyUI-SAM3] [comfy-env] DESERIALIZED tensor shape: torch.Size([1, 720, 1280, 3]) [01:58] [ComfyUI] [SubprocessWorker] got response: ok [01:58] [ComfyUI] [comfy-env] SAM3PointCollector done (4.13s): result: NodeOutput [01:58] [ComfyUI] [comfy-env] Running LoadSAM3Model: precision: 'auto', compile: False [01:58] [ComfyUI] [SubprocessWorker] call_method: nodes.load_model.LoadSAM3Model.EXECUTE_NORMALIZED [01:58] [ComfyUI] [SubprocessWorker] acquired lock, ensuring started... [01:58] [ComfyUI] [ComfyUI-SAM3] Health check: ping (timeout=5.0s)... [01:58] [ComfyUI] [ComfyUI-SAM3] Health check: ok [01:58] [ComfyUI] [SubprocessWorker] worker started/confirmed [01:58] [ComfyUI] [SubprocessWorker] serializing kwargs to shm... [01:58] [ComfyUI] [SubprocessWorker] created 0 shm blocks [01:58] [ComfyUI] [SubprocessWorker] sending request via socket... [01:58] [ComfyUI] [SubprocessWorker] call_id=2 sending call_method [01:58] [ComfyUI] [worker:ComfyUI-SAM3] [SAM3] Model not found at /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/models/sam3/sam3.safetensors, downloading from HuggingFace... [01:58] Node executed (1 total), capturing... [01:58] [ComfyUI] [worker:ComfyUI-SAM3] [SAM3] Downloading sam3.safetensors from apozz/sam3-safetensors... [01:58] Frame 3 saved (frame_002.jpg, t=4.7s) [01:58] [capture-node] freeze=2ms shot=103ms unfreeze=2ms saved=True [01:59] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [01:59] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [01:59] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [01:59] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [01:59] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [01:59] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:00] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:00] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:00] [capture-loop] iter=50 t=6.2s state={'complete': False, 'error': None, 'executedCount': 1, 'wsState': 1} eval_ms=12 ram=3.8/7.0GB (61%) swap=0.0/0.0GB (0%) cpu=61% browser_rss=0.52GB [02:00] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:00] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:00] [ComfyUI] [worker:ComfyUI-SAM3] [SAM3] Download progress: 10% [02:00] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:00] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:01] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:01] Frame 4 saved (frame_003.jpg, t=6.9s) [02:01] [capture-periodic] freeze=8ms shot=206ms unfreeze=5ms saved=True [02:01] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:01] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:01] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:01] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:01] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:02] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:02] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:02] [ComfyUI] [worker:ComfyUI-SAM3] [SAM3] Download progress: 20% [02:02] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:02] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:02] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:02] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:03] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:03] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:03] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:03] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:03] Frame 5 saved (frame_004.jpg, t=9.2s) [02:03] [capture-periodic] freeze=16ms shot=273ms unfreeze=24ms saved=True [02:03] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:03] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:03] [ComfyUI] [worker:ComfyUI-SAM3] [SAM3] Download progress: 30% [02:03] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:04] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:04] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:04] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:04] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:04] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:04] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:04] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:05] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:05] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:05] [ComfyUI] [worker:ComfyUI-SAM3] [SAM3] Download progress: 40% [02:05] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:05] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:05] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:05] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:05] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:05] Frame 6 saved (frame_005.jpg, t=11.6s) [02:05] [capture-periodic] freeze=10ms shot=186ms unfreeze=14ms saved=True [02:05] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:06] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:06] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:06] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:06] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:06] [ComfyUI] [worker:ComfyUI-SAM3] [SAM3] Download progress: 50% [02:06] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:06] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:06] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:07] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:07] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:07] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:07] [capture-loop] iter=100 t=13.4s state={'complete': False, 'error': None, 'executedCount': 1, 'wsState': 1} eval_ms=11 ram=3.7/7.0GB (60%) swap=0.0/0.0GB (0%) cpu=95% browser_rss=0.56GB [02:07] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:07] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:07] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:07] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:07] [ComfyUI] [worker:ComfyUI-SAM3] [SAM3] Download progress: 60% [02:07] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:08] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:08] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:08] Frame 7 saved (frame_006.jpg, t=14.0s) [02:08] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:08] [capture-periodic] freeze=18ms shot=246ms unfreeze=9ms saved=True [02:08] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:08] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:08] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:08] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:09] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:09] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:09] [ComfyUI] [worker:ComfyUI-SAM3] [SAM3] Download progress: 70% [02:09] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:09] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:09] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:09] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:10] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:10] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:10] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:10] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:10] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:10] Frame 8 saved (frame_007.jpg, t=16.4s) [02:10] [capture-periodic] freeze=10ms shot=235ms unfreeze=14ms saved=True [02:10] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:10] [ComfyUI] [worker:ComfyUI-SAM3] [SAM3] Download progress: 80% [02:10] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:11] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:11] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:11] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:11] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:11] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:11] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:11] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:12] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:12] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:12] [ComfyUI] [worker:ComfyUI-SAM3] [SAM3] Download progress: 90% [02:12] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:12] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:12] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:12] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:13] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:13] Frame 9 saved (frame_008.jpg, t=18.8s) [02:13] [capture-periodic] freeze=11ms shot=235ms unfreeze=10ms saved=True [02:13] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:13] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:13] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:13] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:14] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:14] [ComfyUI] [worker:ComfyUI-SAM3] [SAM3] Download progress: 100% [02:14] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:14] [ComfyUI] [worker:ComfyUI-SAM3] [SAM3] Downloaded sam3.safetensors [02:14] [ComfyUI] [worker:ComfyUI-SAM3] [SAM3] Model downloaded successfully [02:14] [ComfyUI] [SubprocessWorker] got response: ok [02:14] [ComfyUI] [comfy-env] LoadSAM3Model done (15.57s): result: NodeOutput [02:14] [ComfyUI] [comfy-env] Running SAM3Segmentation: refinement_iterations: 0, use_multimask: True, output_best_mask: True, sam3_model_config: dict, image: torch.float32 [1,720,1280,3] cpu, positive_points: dict, negative_points: dict [02:14] [ComfyUI] [SubprocessWorker] call_method: nodes.segmentation.SAM3Segmentation.EXECUTE_NORMALIZED [02:14] [ComfyUI] [SubprocessWorker] acquired lock, ensuring started... [02:14] [ComfyUI] [ComfyUI-SAM3] Health check: ping (timeout=5.0s)... [02:14] [ComfyUI] [ComfyUI-SAM3] Health check: ok [02:14] [ComfyUI] [SubprocessWorker] worker started/confirmed [02:14] [ComfyUI] [comfy-env] PRE-SERIALIZE 'image' shape: torch.Size([1, 720, 1280, 3]) [02:14] [ComfyUI] [SubprocessWorker] serializing kwargs to shm... [02:14] [ComfyUI] [SubprocessWorker] created 0 shm blocks [02:14] [ComfyUI] [SubprocessWorker] sending request via socket... [02:14] [ComfyUI] [SubprocessWorker] call_id=3 sending call_method [02:14] [ComfyUI] [worker:ComfyUI-SAM3] [comfy-env] _from_shm got dict with keys: ['__type__', 'strategy', 'manager_path', 'storage_key', 'storage_size'] [02:14] [ComfyUI] [worker:ComfyUI-SAM3] [comfy-env] DESERIALIZE TensorRef: tensor_size=[1, 720, 1280, 3] [02:14] [ComfyUI] [worker:ComfyUI-SAM3] [comfy-env] DESERIALIZED tensor shape: torch.Size([1, 720, 1280, 3]) [02:14] [capture-loop] iter=150 t=20.4s state={'complete': False, 'error': None, 'executedCount': 1, 'wsState': 1} eval_ms=4 ram=3.8/7.0GB (61%) swap=0.0/0.0GB (0%) cpu=87% browser_rss=0.56GB [02:15] Frame 10 saved (frame_009.jpg, t=21.1s) [02:15] [capture-periodic] freeze=2ms shot=96ms unfreeze=2ms saved=True [02:15] [ComfyUI] objc[22485]: Class AVFFrameReceiver is implemented in both /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/lib/libavdevice.62.3.101.dylib (0x11fe6c7a8) and /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/lib/python3.11/site-packages/cv2/.dylibs/libavdevice.61.3.100.dylib (0x14011c3a8). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed. [02:15] [ComfyUI] objc[22485]: Class AVFAudioReceiver is implemented in both /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/lib/libavdevice.62.3.101.dylib (0x11fe6c7f8) and /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/lib/python3.11/site-packages/cv2/.dylibs/libavdevice.61.3.100.dylib (0x14011c3f8). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed. [02:20] [capture-loop] iter=200 t=26.1s state={'complete': False, 'error': None, 'executedCount': 1, 'wsState': 1} eval_ms=2 ram=3.7/7.0GB (61%) swap=0.0/0.0GB (0%) cpu=40% browser_rss=0.55GB [02:26] [capture-loop] iter=250 t=32.0s state={'complete': False, 'error': None, 'executedCount': 1, 'wsState': 1} eval_ms=45 ram=3.3/7.0GB (67%) swap=0.0/0.0GB (0%) cpu=54% browser_rss=0.49GB [02:32] [capture-loop] iter=300 t=38.0s state={'complete': False, 'error': None, 'executedCount': 1, 'wsState': 1} eval_ms=3 ram=2.8/7.0GB (74%) swap=0.0/0.0GB (0%) cpu=50% browser_rss=0.30GB [02:37] [ComfyUI] [SubprocessWorker] callback 'request_vram_budget' from call_id=3 [02:37] [ComfyUI] [comfy-env] VRAM request: 1.84GB, free before eviction: 1.72GB [02:37] [ComfyUI] [comfy-env] VRAM after eviction: 1.72GB [02:37] [capture-loop] iter=350 t=43.8s state={'complete': False, 'error': None, 'executedCount': 1, 'wsState': 1} eval_ms=2 ram=2.6/7.0GB (77%) swap=0.0/0.0GB (0%) cpu=39% browser_rss=0.31GB [02:43] [capture-loop] iter=400 t=49.6s state={'complete': False, 'error': None, 'executedCount': 1, 'wsState': 1} eval_ms=13 ram=3.2/7.0GB (75%) swap=0.0/0.0GB (0%) cpu=62% browser_rss=0.28GB [02:53] [capture-loop] iter=450 t=59.4s state={'complete': False, 'error': None, 'executedCount': 1, 'wsState': 1} eval_ms=6 ram=4.4/7.0GB (84%) swap=0.0/0.0GB (0%) cpu=40% browser_rss=0.25GB [02:54] [ComfyUI] [worker:ComfyUI-SAM3] [02:54] [ComfyUI] === WATCHDOG TICK 1 === [02:54] [ComfyUI] [worker:ComfyUI-SAM3] Current thread 0x00000001709b3000 (most recent call first): [02:54] [ComfyUI] File "/var/folders/tb/y368xp_x10s3ty1b_mtl5mxr0000gn/T/comfyui_pvenv__jjhnh0w/persistent_worker.py", line 52 in _watchdog [02:54] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/lib/python3.11/threading.py", line 982 in run [02:54] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/lib/python3.11/threading.py", line 1045 in _bootstrap_inner [02:54] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/lib/python3.11/threading.py", line 1002 in _bootstrap [02:54] [ComfyUI] [02:54] [ComfyUI] Thread 0x00000001ffe36240 (most recent call first): [02:54] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/lib/python3.11/site-packages/torch/nn/modules/conv.py", line 548 in _conv_forward [02:54] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/comfy/ops.py", line 509 in forward_comfy_cast_weights [02:54] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/comfy/ops.py", line 516 in forward [02:54] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1790 in _call_impl [02:54] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1779 in _wrapped_call_impl [02:54] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/lib/python3.11/site-packages/torch/nn/modules/container.py", line 253 in forward [02:54] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1790 in _call_impl [02:54] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1779 in _wrapped_call_impl [02:54] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/model.py", line 2049 in forward [02:54] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/model.py", line 3321 in _forward_image_no_act_ckpt [02:54] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/model.py", line 3314 in forward_image [02:54] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/utils.py", line 1324 in set_image [02:54] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 124 in decorate_context [02:54] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/segmentation.py", line 496 in execute [02:54] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/comfy_api/latest/_io.py", line 1833 in EXECUTE_NORMALIZED [02:54] [ComfyUI] File "/var/folders/tb/y368xp_x10s3ty1b_mtl5mxr0000gn/T/comfyui_pvenv__jjhnh0w/persistent_worker.py", line 1762 in main [02:54] [ComfyUI] File "/var/folders/tb/y368xp_x10s3ty1b_mtl5mxr0000gn/T/comfyui_pvenv__jjhnh0w/persistent_worker.py", line 1807 in [02:54] [ComfyUI] [02:54] [ComfyUI] [worker:ComfyUI-SAM3] === END === [02:54] [ComfyUI] [02:59] [capture-loop] iter=500 t=65.5s state={'complete': False, 'error': None, 'executedCount': 1, 'wsState': 1} eval_ms=5 ram=2.7/7.0GB (75%) swap=0.0/0.0GB (0%) cpu=40% browser_rss=0.26GB [03:03] [ComfyUI] [SubprocessWorker] got response: ok [03:03] [ComfyUI] [comfy-env] SAM3Segmentation done (49.74s): result: NodeOutput [03:03] [ComfyUI] Prompt executed in 69.65 seconds [03:04] Node executed (3 total), capturing... [03:04] Frame 11 saved (frame_010.jpg, t=70.1s) [03:04] [capture-node] freeze=3ms shot=108ms unfreeze=0ms saved=True [03:04] Execution complete (t=70.1s) [03:04] Captured 11 unique frames over 70.27s [03:04] [debug] iframes: 0 [03:04] [timing] trigger_3d_previews: 0.0s [03:06] Saved high-quality screenshot: image_seg_point_executed.png [03:06] [timing] final_screenshot: 0.2s [03:06] Capturing execution frames: image_seg_text.json [03:12] Frame 1 saved (frame_000.jpg, t=0.0s) [03:12] Validating workflow... [03:12] [ComfyUI] === /validate endpoint called === [03:12] [ComfyUI] Received JSON with keys: ['prompt'] [03:12] [ComfyUI] Prompt has 5 nodes [03:12] [ComfyUI] Node 1: LoadImage [03:12] [ComfyUI] Inputs: {"image": "example_image.jpg"} [03:12] [ComfyUI] Node 3: PreviewImage [03:12] [ComfyUI] Inputs: {"images": ["15", 1]} [03:12] [ComfyUI] Node 4: MaskPreview [03:12] [ComfyUI] Inputs: {"mask": ["15", 0]} [03:12] [ComfyUI] Node 12: LoadSAM3Model [03:12] [ComfyUI] Inputs: {"precision": "fp32", "compile": false} [03:12] [ComfyUI] Node 15: SAM3Grounding [03:12] [ComfyUI] Inputs: {"confidence_threshold": 0.2, "text_prompt": "child", "max_detections": -1, "sam3_model_config": ["12", 0], "image": ["1", 0]} [03:12] [ComfyUI] Generated prompt_id: 7b9ad062-6796-4c93-bea1-bdc6c2980be2 [03:12] [ComfyUI] Calling execution.validate_prompt()... [03:12] [ComfyUI] Validation result: valid=True [03:12] [ComfyUI] === Validation PASSED === [03:12] Queuing workflow for execution... [03:12] [ComfyUI] got prompt [03:12] [ComfyUI] [comfy-env] Running LoadSAM3Model: precision: 'fp32', compile: False [03:12] [ComfyUI] [SubprocessWorker] call_method: nodes.load_model.LoadSAM3Model.EXECUTE_NORMALIZED [03:12] [ComfyUI] [SubprocessWorker] acquired lock, ensuring started... [03:12] [ComfyUI] [ComfyUI-SAM3] Health check: ping (timeout=5.0s)... [03:12] [ComfyUI] [ComfyUI-SAM3] Health check: ok [03:12] [ComfyUI] [SubprocessWorker] worker started/confirmed [03:12] [ComfyUI] [SubprocessWorker] serializing kwargs to shm... [03:12] [ComfyUI] [SubprocessWorker] created 0 shm blocks [03:12] [ComfyUI] [SubprocessWorker] sending request via socket... [03:12] [ComfyUI] [SubprocessWorker] call_id=4 sending call_method [03:12] [ComfyUI] [SubprocessWorker] got response: ok [03:12] [ComfyUI] [comfy-env] LoadSAM3Model done (0.01s): result: NodeOutput [03:12] [ComfyUI] [comfy-env] Running SAM3Grounding: confidence_threshold: 0.2, text_prompt: 'child', max_detections: -1, sam3_model_config: dict, image: torch.float32 [1,720,1280,3] cpu [03:12] [ComfyUI] [SubprocessWorker] call_method: nodes.segmentation.SAM3Grounding.EXECUTE_NORMALIZED [03:12] [ComfyUI] [SubprocessWorker] acquired lock, ensuring started... [03:12] [ComfyUI] [ComfyUI-SAM3] Health check: ping (timeout=5.0s)... [03:12] [ComfyUI] [ComfyUI-SAM3] Health check: ok [03:12] [ComfyUI] [SubprocessWorker] worker started/confirmed [03:12] [ComfyUI] [comfy-env] PRE-SERIALIZE 'image' shape: torch.Size([1, 720, 1280, 3]) [03:12] [ComfyUI] [SubprocessWorker] serializing kwargs to shm... [03:12] [ComfyUI] [SubprocessWorker] created 0 shm blocks [03:12] [ComfyUI] [SubprocessWorker] sending request via socket... [03:12] [ComfyUI] [SubprocessWorker] call_id=5 sending call_method [03:12] [ComfyUI] [worker:ComfyUI-SAM3] [comfy-env] _from_shm got dict with keys: ['__type__', 'strategy', 'manager_path', 'storage_key', 'storage_size'] [03:12] [ComfyUI] [worker:ComfyUI-SAM3] [comfy-env] DESERIALIZE TensorRef: tensor_size=[1, 720, 1280, 3] [03:12] [ComfyUI] [worker:ComfyUI-SAM3] [comfy-env] DESERIALIZED tensor shape: torch.Size([1, 720, 1280, 3]) [03:13] [ComfyUI] [SubprocessWorker] callback 'request_vram_budget' from call_id=5 [03:13] [ComfyUI] [comfy-env] VRAM request: 3.50GB, free before eviction: 1.79GB [03:13] [ComfyUI] [comfy-env] VRAM after eviction: 1.79GB [03:14] Frame 2 saved (frame_001.jpg, t=2.2s) [03:14] [capture-periodic] freeze=3ms shot=129ms unfreeze=3ms saved=True [03:19] Frame 3 saved (frame_002.jpg, t=4.5s) [03:19] [capture-periodic] freeze=2ms shot=2466ms unfreeze=13ms saved=True [03:20] [capture-loop] iter=50 t=8.3s state={'complete': False, 'error': None, 'executedCount': 0, 'wsState': 1} eval_ms=4 ram=3.2/7.0GB (87%) swap=0.1/1.0GB (7%) cpu=37% browser_rss=0.44GB [03:32] [capture-loop] iter=100 t=19.7s state={'complete': False, 'error': None, 'executedCount': 0, 'wsState': 1} eval_ms=15 ram=2.3/7.0GB (94%) swap=2.0/3.0GB (65%) cpu=52% browser_rss=0.22GB [03:40] [capture-loop] iter=150 t=28.5s state={'complete': False, 'error': None, 'executedCount': 0, 'wsState': 1} eval_ms=3 ram=3.0/7.0GB (66%) swap=1.7/4.0GB (42%) cpu=52% browser_rss=0.32GB [03:54] [ComfyUI] [worker:ComfyUI-SAM3] [03:54] [ComfyUI] === WATCHDOG TICK 2 === [03:54] [ComfyUI] [worker:ComfyUI-SAM3] Current thread 0x00000001709b3000 (most recent call first): [03:54] [ComfyUI] File "/var/folders/tb/y368xp_x10s3ty1b_mtl5mxr0000gn/T/comfyui_pvenv__jjhnh0w/persistent_worker.py", line 52 in _watchdog [03:54] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/lib/python3.11/threading.py", line 982 in run [03:54] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/lib/python3.11/threading.py", line 1045 in _bootstrap_inner [03:54] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/lib/python3.11/threading.py", line 1002 in _bootstrap [03:54] [ComfyUI] [03:54] [ComfyUI] Thread 0x00000001ffe36240 (most recent call first): [03:54] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/comfy/ldm/modules/sub_quadratic_attention.py", line 170 in _get_attention_scores_no_kv_chunking [03:54] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/comfy/ldm/modules/sub_quadratic_attention.py", line 269 in [03:54] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/comfy/ldm/modules/sub_quadratic_attention.py", line 268 in efficient_dot_product_attention [03:54] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/comfy/ldm/modules/attention.py", line 275 in attention_sub_quad [03:54] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/comfy/ldm/modules/attention.py", line 139 in wrapper [03:54] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/attention.py", line 120 in sam3_attention [03:54] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/model.py", line 729 in forward [03:54] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1790 in _call_impl [03:54] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1779 in _wrapped_call_impl [03:54] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/model.py", line 819 in forward [03:54] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1790 in _call_impl [03:54] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1779 in _wrapped_call_impl [03:54] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/model.py", line 983 in forward [03:54] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1790 in _call_impl [03:54] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1779 in _wrapped_call_impl [03:54] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/model.py", line 2041 in forward [03:54] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/model.py", line 3321 in _forward_image_no_act_ckpt [03:54] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/model.py", line 3314 in forward_image [03:54] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/utils.py", line 1324 in set_image [03:54] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 124 in decorate_context [03:54] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/segmentation.py", line 133 in _segment_grounding [03:54] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/segmentation.py", line 111 in execute [03:54] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/comfy_api/latest/_io.py", line 1833 in EXECUTE_NORMALIZED [03:54] [ComfyUI] File "/var/folders/tb/y368xp_x10s3ty1b_mtl5mxr0000gn/T/comfyui_pvenv__jjhnh0w/persistent_worker.py", line 1762 in main [03:54] [ComfyUI] File "/var/folders/tb/y368xp_x10s3ty1b_mtl5mxr0000gn/T/comfyui_pvenv__jjhnh0w/persistent_worker.py", line 1807 in [03:54] [ComfyUI] [03:54] [ComfyUI] [worker:ComfyUI-SAM3] === END === [03:54] [ComfyUI] [03:55] Frame 4 saved (frame_003.jpg, t=40.8s) [03:55] [capture-periodic] freeze=3ms shot=1793ms unfreeze=3ms saved=True [03:56] [capture-loop] iter=200 t=44.6s state={'complete': False, 'error': None, 'executedCount': 0, 'wsState': 1} eval_ms=3 ram=6.0/7.0GB (87%) swap=1.4/2.0GB (70%) cpu=32% browser_rss=0.26GB [04:07] [ComfyUI] [SubprocessWorker] got response: error [04:07] [capture-loop] iter=250 t=55.1s state={'complete': False, 'error': None, 'executedCount': 0, 'wsState': 1} eval_ms=2 ram=5.9/7.0GB (87%) swap=1.3/2.0GB (63%) cpu=39% browser_rss=0.26GB [04:07] [ComfyUI] !!! Exception during processing !!! The operator 'aten::_assert_async' is not currently implemented for the MPS device. If you want this op to be considered for addition please comment on https://github.com/pytorch/pytorch/issues/141287 and mention use-case, that resulted in missing op as well as commit hash 70d99e998b4955e0049d13a98d77ae1b14db1f45. As a temporary fix, you can set the environment variable `PYTORCH_ENABLE_MPS_FALLBACK=1` to use the CPU as a fallback for this op. WARNING: this will be slower than running natively on MPS. [04:07] [ComfyUI] [04:07] [ComfyUI] Worker traceback: [04:07] [ComfyUI] Traceback (most recent call last): [04:07] [ComfyUI] File "/var/folders/tb/y368xp_x10s3ty1b_mtl5mxr0000gn/T/comfyui_pvenv__jjhnh0w/persistent_worker.py", line 1762, in main [04:07] [ComfyUI] result = method(**inputs) [04:07] [ComfyUI] ^^^^^^^^^^^^^^^^ [04:07] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/comfy_api/latest/_io.py", line 1833, in EXECUTE_NORMALIZED [04:07] [ComfyUI] to_return = cls.execute(*args, **kwargs) [04:07] [ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [04:07] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/segmentation.py", line 111, in execute [04:07] [ComfyUI] result = cls._segment_grounding( [04:07] [ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^ [04:07] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/segmentation.py", line 138, in _segment_grounding [04:07] [ComfyUI] state = processor.set_text_prompt(text_prompt.strip(), state) [04:07] [ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [04:07] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 124, in decorate_context [04:07] [ComfyUI] return func(*args, **kwargs) [04:07] [ComfyUI] ^^^^^^^^^^^^^^^^^^^^^ [04:07] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/utils.py", line 1393, in set_text_prompt [04:07] [ComfyUI] return self._forward_grounding(state) [04:07] [ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [04:07] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 124, in decorate_context [04:07] [ComfyUI] return func(*args, **kwargs) [04:07] [ComfyUI] ^^^^^^^^^^^^^^^^^^^^^ [04:07] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/utils.py", line 1477, in _forward_grounding [04:07] [ComfyUI] outputs = self.model.forward_grounding( [04:07] [ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [04:07] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/model.py", line 3819, in forward_grounding [04:07] [ComfyUI] prompt, prompt_mask, backbone_out = self._encode_prompt( [04:07] [ComfyUI] ^^^^^^^^^^^^^^^^^^^^ [04:07] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/model.py", line 3561, in _encode_prompt [04:07] [ComfyUI] geo_feats, geo_masks = self.geometry_encoder( [04:07] [ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^ [04:07] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1779, in _wrapped_call_impl [04:07] [ComfyUI] return self._call_impl(*args, **kwargs) [04:07] [ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [04:07] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1790, in _call_impl [04:07] [ComfyUI] return forward_call(*args, **kwargs) [04:07] [ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [04:07] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/model.py", line 2501, in forward [04:07] [ComfyUI] final_embeds, final_mask = concat_padded_sequences( [04:07] [ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^ [04:07] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/utils.py", line 1824, in concat_padded_sequences [04:07] [ComfyUI] torch._assert_async(is_right_padded(mask1)) [04:07] [ComfyUI] NotImplementedError: The operator 'aten::_assert_async' is not currently implemented for the MPS device. If you want this op to be considered for addition please comment on https://github.com/pytorch/pytorch/issues/141287 and mention use-case, that resulted in missing op as well as commit hash 70d99e998b4955e0049d13a98d77ae1b14db1f45. As a temporary fix, you can set the environment variable `PYTORCH_ENABLE_MPS_FALLBACK=1` to use the CPU as a fallback for this op. WARNING: this will be slower than running natively on MPS. [04:07] [ComfyUI] [04:07] [ComfyUI] Traceback (most recent call last): [04:07] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/execution.py", line 535, in execute [04:07] [ComfyUI] output_data, output_ui, has_subgraph, has_pending_tasks = await get_output_data(prompt_id, unique_id, obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data) [04:07] [ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [04:07] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/execution.py", line 335, in get_output_data [04:07] [ComfyUI] return_values = await _async_map_node_over_list(prompt_id, unique_id, obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data) [04:07] [ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [04:07] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/execution.py", line 309, in _async_map_node_over_list [04:07] [ComfyUI] await process_inputs(input_dict, i) [04:07] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/execution.py", line 297, in process_inputs [04:07] [ComfyUI] result = f(**inputs) [04:07] [ComfyUI] ^^^^^^^^^^^ [04:07] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/venv/lib/python3.11/site-packages/comfy_env/isolation/metadata.py", line 536, in proxy [04:07] [ComfyUI] result = worker.call_method( [04:07] [ComfyUI] ^^^^^^^^^^^^^^^^^^^ [04:07] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/venv/lib/python3.11/site-packages/comfy_env/isolation/workers/subprocess.py", line 674, in call_method [04:07] [ComfyUI] raise WorkerError( [04:07] [ComfyUI] comfy_env.isolation.workers.base.WorkerError: The operator 'aten::_assert_async' is not currently implemented for the MPS device. If you want this op to be considered for addition please comment on https://github.com/pytorch/pytorch/issues/141287 and mention use-case, that resulted in missing op as well as commit hash 70d99e998b4955e0049d13a98d77ae1b14db1f45. As a temporary fix, you can set the environment variable `PYTORCH_ENABLE_MPS_FALLBACK=1` to use the CPU as a fallback for this op. WARNING: this will be slower than running natively on MPS. [04:07] [ComfyUI] [04:07] [ComfyUI] Worker traceback: [04:07] [ComfyUI] Traceback (most recent call last): [04:07] [ComfyUI] File "/var/folders/tb/y368xp_x10s3ty1b_mtl5mxr0000gn/T/comfyui_pvenv__jjhnh0w/persistent_worker.py", line 1762, in main [04:07] [ComfyUI] result = method(**inputs) [04:07] [ComfyUI] ^^^^^^^^^^^^^^^^ [04:07] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/comfy_api/latest/_io.py", line 1833, in EXECUTE_NORMALIZED [04:07] [ComfyUI] to_return = cls.execute(*args, **kwargs) [04:07] [ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [04:07] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/segmentation.py", line 111, in execute [04:07] [ComfyUI] result = cls._segment_grounding( [04:07] [ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^ [04:07] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/segmentation.py", line 138, in _segment_grounding [04:07] [ComfyUI] state = processor.set_text_prompt(text_prompt.strip(), state) [04:07] [ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [04:07] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 124, in decorate_context [04:07] [ComfyUI] return func(*args, **kwargs) [04:07] [ComfyUI] ^^^^^^^^^^^^^^^^^^^^^ [04:07] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/utils.py", line 1393, in set_text_prompt [04:07] [ComfyUI] return self._forward_grounding(state) [04:07] [ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [04:07] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 124, in decorate_context [04:07] [ComfyUI] return func(*args, **kwargs) [04:07] [ComfyUI] ^^^^^^^^^^^^^^^^^^^^^ [04:07] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/utils.py", line 1477, in _forward_grounding [04:07] [ComfyUI] outputs = self.model.forward_grounding( [04:07] [ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [04:07] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/model.py", line 3819, in forward_grounding [04:07] [ComfyUI] prompt, prompt_mask, backbone_out = self._encode_prompt( [04:07] [ComfyUI] ^^^^^^^^^^^^^^^^^^^^ [04:07] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/model.py", line 3561, in _encode_prompt [04:07] [ComfyUI] geo_feats, geo_masks = self.geometry_encoder( [04:07] [ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^ [04:07] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1779, in _wrapped_call_impl [04:07] [ComfyUI] return self._call_impl(*args, **kwargs) [04:07] [ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [04:07] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1790, in _call_impl [04:07] [ComfyUI] return forward_call(*args, **kwargs) [04:07] [ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [04:07] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/model.py", line 2501, in forward [04:07] [ComfyUI] final_embeds, final_mask = concat_padded_sequences( [04:07] [ComfyUI] ^^^^^^^^^^^^^^^^^^^^^^^^ [04:07] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/utils.py", line 1824, in concat_padded_sequences [04:07] [ComfyUI] torch._assert_async(is_right_padded(mask1)) [04:07] [ComfyUI] NotImplementedError: The operator 'aten::_assert_async' is not currently implemented for the MPS device. If you want this op to be considered for addition please comment on https://github.com/pytorch/pytorch/issues/141287 and mention use-case, that resulted in missing op as well as commit hash 70d99e998b4955e0049d13a98d77ae1b14db1f45. As a temporary fix, you can set the environment variable `PYTORCH_ENABLE_MPS_FALLBACK=1` to use the CPU as a fallback for this op. WARNING: this will be slower than running natively on MPS. [04:07] [ComfyUI] [04:07] [ComfyUI] [04:07] [ComfyUI] Prompt executed in 55.08 seconds [04:15] Frame 5 saved (frame_004.jpg, t=61.7s) [04:15] [capture-periodic] freeze=289ms shot=1561ms unfreeze=3ms saved=True [04:15] Execution complete (t=61.7s) [04:15] Execution error: {'prompt_id': 'f8014278-5e7d-4b45-a7dc-432b29029e0f', 'node_id': '15', 'node_type': 'SAM3Grounding', 'executed': ['1', '12'], 'exception_message': 'The operator \'aten::_assert_async\' is not currently implemented for the MPS device. If you want this op to be considered for addition please comment on https://github.com/pytorch/pytorch/issues/141287 and mention use-case, that resulted in missing op as well as commit hash 70d99e998b4955e0049d13a98d77ae1b14db1f45. As a temporary fix, you can set the environment variable `PYTORCH_ENABLE_MPS_FALLBACK=1` to use the CPU as a fallback for this op. WARNING: this will be slower than running natively on MPS.\n\nWorker traceback:\nTraceback (most recent call last):\n File "/var/folders/tb/y368xp_x10s3ty1b_mtl5mxr0000gn/T/comfyui_pvenv__jjhnh0w/persistent_worker.py", line 1762, in main\n result = method(**inputs)\n ^^^^^^^^^^^^^^^^\n File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/comfy_api/latest/_io.py", line 1833, in EXECUTE_NORMALIZED\n to_return = cls.execute(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/segmentation.py", line 111, in execute\n result = cls._segment_grounding(\n ^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/segmentation.py", line 138, in _segment_grounding\n state = processor.set_text_prompt(text_prompt.strip(), state)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 124, in decorate_context\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/utils.py", line 1393, in set_text_prompt\n return self._forward_grounding(state)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 124, in decorate_context\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/utils.py", line 1477, in _forward_grounding\n outputs = self.model.forward_grounding(\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/model.py", line 3819, in forward_grounding\n prompt, prompt_mask, backbone_out = self._encode_prompt(\n ^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/model.py", line 3561, in _encode_prompt\n geo_feats, geo_masks = self.geometry_encoder(\n ^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1779, in _wrapped_call_impl\n return self._call_impl(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1790, in _call_impl\n return forward_call(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/model.py", line 2501, in forward\n final_embeds, final_mask = concat_padded_sequences(\n ^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/utils.py", line 1824, in concat_padded_sequences\n torch._assert_async(is_right_padded(mask1))\nNotImplementedError: The operator \'aten::_assert_async\' is not currently implemented for the MPS device. If you want this op to be considered for addition please comment on https://github.com/pytorch/pytorch/issues/141287 and mention use-case, that resulted in missing op as well as commit hash 70d99e998b4955e0049d13a98d77ae1b14db1f45. As a temporary fix, you can set the environment variable `PYTORCH_ENABLE_MPS_FALLBACK=1` to use the CPU as a fallback for this op. WARNING: this will be slower than running natively on MPS.\n\n', 'exception_type': 'comfy_env.isolation.workers.base.WorkerError', 'traceback': [' File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/execution.py", line 535, in execute\n output_data, output_ui, has_subgraph, has_pending_tasks = await get_output_data(prompt_id, unique_id, obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n', ' File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/execution.py", line 335, in get_output_data\n return_values = await _async_map_node_over_list(prompt_id, unique_id, obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n', ' File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/execution.py", line 309, in _async_map_node_over_list\n await process_inputs(input_dict, i)\n', ' File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/execution.py", line 297, in process_inputs\n result = f(**inputs)\n ^^^^^^^^^^^\n', ' File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/venv/lib/python3.11/site-packages/comfy_env/isolation/metadata.py", line 536, in proxy\n result = worker.call_method(\n ^^^^^^^^^^^^^^^^^^^\n', ' File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/venv/lib/python3.11/site-packages/comfy_env/isolation/workers/subprocess.py", line 674, in call_method\n raise WorkerError(\n'], 'current_inputs': {'confidence_threshold': [0.2], 'text_prompt': ['child'], 'max_detections': [-1], 'sam3_model_config': ["{'checkpoint_path': '/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/models/sam3/sam3.safetensors', 'bpe_path': '/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/bpe_simple_vocab_16e6.txt.gz', 'precision': 'fp32', 'dtype': 'fp32', 'compile': False}"], 'image': ['tensor([[[[0.2902, 0.4667, 0.5961],\n [0.2902, 0.4667, 0.5961],\n [0.2902, 0.4667, 0.5961],\n ...,\n [0.3176, 0.5804, 0.7216],\n [0.3333, 0.6039, 0.7451],\n [0.3529, 0.6235, 0.7647]],\n\n [[0.2902, 0.4667, 0.5961],\n [0.2902, 0.4667, 0.5961],\n [0.2902, 0.4667, 0.5961],\n ...,\n [0.3137, 0.5765, 0.7176],\n [0.3137, 0.5843, 0.7255],\n [0.3255, 0.5961, 0.7373]],\n\n [[0.2902, 0.4667, 0.5961],\n [0.2902, 0.4667, 0.5961],\n [0.2902, 0.4667, 0.5961],\n ...,\n [0.3922, 0.5333, 0.6275],\n [0.3922, 0.5294, 0.6235],\n [0.3961, 0.5333, 0.6275]],\n\n ...,\n\n [[0.4941, 0.4863, 0.4157],\n [0.4941, 0.4863, 0.4157],\n [0.4980, 0.4902, 0.4196],\n ...,\n [0.5647, 0.5608, 0.5059],\n [0.5647, 0.5608, 0.5059],\n [0.5647, 0.5608, 0.5059]],\n\n [[0.5098, 0.5020, 0.4314],\n [0.5098, 0.5020, 0.4314],\n [0.5137, 0.5059, 0.4353],\n ...,\n [0.5647, 0.5608, 0.5059],\n [0.5647, 0.5608, 0.5059],\n [0.5647, 0.5608, 0.5059]],\n\n [[0.5176, 0.5098, 0.4392],\n [0.5216, 0.5137, 0.4431],\n [0.5255, 0.5176, 0.4471],\n ...,\n [0.5647, 0.5608, 0.5059],\n [0.5647, 0.5608, 0.5059],\n [0.5647, 0.5608, 0.5059]]]])']}, 'current_outputs': ['1', '15', '4', '12', '3'], 'timestamp': 1778432637946} [04:17] Capturing execution frames: scene_seg.json [04:26] Frame 1 saved (frame_000.jpg, t=0.0s) [04:26] Validating workflow... [04:26] [ComfyUI] === /validate endpoint called === [04:26] [ComfyUI] Received JSON with keys: ['prompt'] [04:26] [ComfyUI] Prompt has 10 nodes [04:26] [ComfyUI] Node 1: LoadImage [04:26] [ComfyUI] Inputs: {"image": "image.png"} [04:26] [ComfyUI] Node 3: PreviewImage [04:26] [ComfyUI] Inputs: {"images": ["27", 1]} [04:26] [ComfyUI] Node 4: MaskPreview [04:26] [ComfyUI] Inputs: {"mask": ["27", 0]} [04:26] [ComfyUI] Node 12: LoadSAM3Model [04:26] [ComfyUI] Inputs: {"precision": "fp32", "compile": false} [04:26] [ComfyUI] Node 20: SAM3MultiRegionCollector [04:26] [ComfyUI] Inputs: {"multi_prompts_store": "[{\"positive_points\":[{\"x\":5052.74953932266,\"y\":2847.0600693869637},{\"x\":4932.510591392486,\"y\":2953.9413227753366},{\"x\":4718.752461738845,\"y\":2967.301479448883},{ [04:26] [ComfyUI] Node 22: MultibandFromBatch [04:26] [ComfyUI] Inputs: {"images": ["1", 0], "masks": ["27", 0]} [04:26] [ComfyUI] Node 23: MultibandSave [04:26] [ComfyUI] Inputs: {"file_path": "output/multiband", "format": "npz", "multiband": ["26", 0]} [04:26] [ComfyUI] Node 24: MultibandPreview [04:26] [ComfyUI] Inputs: {"channel_index": 0, "multiband": ["26", 0]} [04:26] [ComfyUI] Node 26: MultibandResize [04:26] [ComfyUI] Inputs: {"upscale_method": "nearest-exact", "width": 900, "height": 600, "crop": "center", "multiband": ["22", 0]} [04:26] [ComfyUI] Node 27: SAM3MultipromptSegmentation [04:26] [ComfyUI] Inputs: {"refinement_iterations": 0, "use_multimask": false, "sam3_model_config": ["12", 0], "image": ["1", 0], "multi_prompts": ["20", 0]} [04:26] [ComfyUI] Generated prompt_id: c8448666-4ca3-4d9d-9b72-5215a16b9ef7 [04:26] [ComfyUI] Calling execution.validate_prompt()... [04:26] [ComfyUI] Validation result: valid=True [04:26] [ComfyUI] === Validation PASSED === [04:26] Queuing workflow for execution... [04:26] [ComfyUI] got prompt [04:29] Frame 2 saved (frame_001.jpg, t=2.2s) [04:29] [capture-periodic] freeze=17ms shot=543ms unfreeze=60ms saved=True [04:29] [ComfyUI] [comfy-env] Running SAM3MultiRegionCollector: multi_prompts_store: '[{"positive_points":[{"x":5052.74953932266,"y":2847.0600..., image: torch.float32 [1,4480,6720,3] cpu [04:29] [ComfyUI] [SubprocessWorker] call_method: nodes.sam3_interactive.SAM3MultiRegionCollector.EXECUTE_NORMALIZED [04:29] [ComfyUI] [SubprocessWorker] acquired lock, ensuring started... [04:29] [ComfyUI] [ComfyUI-SAM3] Health check: ping (timeout=5.0s)... [04:29] [ComfyUI] [ComfyUI-SAM3] Health check: ok [04:29] [ComfyUI] [SubprocessWorker] worker started/confirmed [04:29] [ComfyUI] [comfy-env] PRE-SERIALIZE 'image' shape: torch.Size([1, 4480, 6720, 3]) [04:29] [ComfyUI] [SubprocessWorker] serializing kwargs to shm... [04:29] [ComfyUI] [SubprocessWorker] created 0 shm blocks [04:29] [ComfyUI] [SubprocessWorker] sending request via socket... [04:29] [ComfyUI] [SubprocessWorker] call_id=6 sending call_method [04:29] [ComfyUI] [worker:ComfyUI-SAM3] [comfy-env] _from_shm got dict with keys: ['__type__', 'strategy', 'manager_path', 'storage_key', 'storage_size'] [04:29] [ComfyUI] [worker:ComfyUI-SAM3] [comfy-env] DESERIALIZE TensorRef: tensor_size=[1, 4480, 6720, 3] [04:29] [ComfyUI] [worker:ComfyUI-SAM3] [comfy-env] DESERIALIZED tensor shape: torch.Size([1, 4480, 6720, 3]) [04:30] [ComfyUI] [SubprocessWorker] got response: ok [04:30] [ComfyUI] [comfy-env] SAM3MultiRegionCollector done (1.07s): result: NodeOutput [04:30] [ComfyUI] [comfy-env] Running LoadSAM3Model: precision: 'fp32', compile: False [04:30] [ComfyUI] [SubprocessWorker] call_method: nodes.load_model.LoadSAM3Model.EXECUTE_NORMALIZED [04:30] [ComfyUI] [SubprocessWorker] acquired lock, ensuring started... [04:30] [ComfyUI] [ComfyUI-SAM3] Health check: ping (timeout=5.0s)... [04:30] [ComfyUI] [ComfyUI-SAM3] Health check: ok [04:30] [ComfyUI] [SubprocessWorker] worker started/confirmed [04:30] [ComfyUI] [SubprocessWorker] serializing kwargs to shm... [04:30] [ComfyUI] [SubprocessWorker] created 0 shm blocks [04:30] [ComfyUI] [SubprocessWorker] sending request via socket... [04:30] [ComfyUI] [SubprocessWorker] call_id=7 sending call_method [04:30] [ComfyUI] [SubprocessWorker] got response: ok [04:30] [ComfyUI] [comfy-env] LoadSAM3Model done (0.05s): result: NodeOutput [04:30] [ComfyUI] [comfy-env] Running SAM3MultipromptSegmentation: refinement_iterations: 0, use_multimask: False, sam3_model_config: dict, image: torch.float32 [1,4480,6720,3] cpu, multi_prompts: list[7] [04:30] [ComfyUI] [SubprocessWorker] call_method: nodes.segmentation.SAM3MultipromptSegmentation.EXECUTE_NORMALIZED [04:30] [ComfyUI] [SubprocessWorker] acquired lock, ensuring started... [04:30] [ComfyUI] [ComfyUI-SAM3] Health check: ping (timeout=5.0s)... [04:30] [ComfyUI] [ComfyUI-SAM3] Health check: ok [04:30] [ComfyUI] [SubprocessWorker] worker started/confirmed [04:30] [ComfyUI] [comfy-env] PRE-SERIALIZE 'image' shape: torch.Size([1, 4480, 6720, 3]) [04:30] [ComfyUI] [SubprocessWorker] serializing kwargs to shm... [04:30] [ComfyUI] [SubprocessWorker] created 0 shm blocks [04:30] [ComfyUI] [SubprocessWorker] sending request via socket... [04:30] [ComfyUI] [SubprocessWorker] call_id=8 sending call_method [04:30] [ComfyUI] [worker:ComfyUI-SAM3] [comfy-env] _from_shm got dict with keys: ['__type__', 'strategy', 'manager_path', 'storage_key', 'storage_size'] [04:30] [ComfyUI] [worker:ComfyUI-SAM3] [comfy-env] DESERIALIZE TensorRef: tensor_size=[1, 4480, 6720, 3] [04:30] [ComfyUI] [worker:ComfyUI-SAM3] [comfy-env] DESERIALIZED tensor shape: torch.Size([1, 4480, 6720, 3]) [04:30] [ComfyUI] [SubprocessWorker] callback 'request_vram_budget' from call_id=8 [04:30] [ComfyUI] [comfy-env] VRAM request: 3.50GB, free before eviction: 1.38GB [04:30] [ComfyUI] [comfy-env] VRAM after eviction: 1.38GB [04:31] Node executed (1 total), capturing... [04:32] Frame 3 saved (frame_002.jpg, t=5.3s) [04:32] [capture-node] freeze=64ms shot=528ms unfreeze=8ms saved=True [04:36] [capture-loop] iter=50 t=9.6s state={'complete': False, 'error': None, 'executedCount': 1, 'wsState': 1} eval_ms=6 ram=2.4/7.0GB (81%) swap=2.4/3.0GB (79%) cpu=58% browser_rss=0.47GB [04:55] [ComfyUI] [worker:ComfyUI-SAM3] [04:55] [ComfyUI] === WATCHDOG TICK 3 === [04:55] [ComfyUI] [worker:ComfyUI-SAM3] Current thread 0x00000001709b3000 (most recent call first): [04:55] [ComfyUI] File "/var/folders/tb/y368xp_x10s3ty1b_mtl5mxr0000gn/T/comfyui_pvenv__jjhnh0w/persistent_worker.py", line 52 in _watchdog [04:55] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/lib/python3.11/threading.py", line 982 in run [04:55] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/lib/python3.11/threading.py", line 1045 in _bootstrap_inner [04:55] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/lib/python3.11/threading.py", line 1002 in _bootstrap [04:55] [ComfyUI] [04:55] [ComfyUI] Thread 0x00000001ffe36240 (most recent call first): [04:55] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/lib/python3.11/site-packages/torch/nn/modules/activation.py", line 816 in forward [04:55] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1790 in _call_impl [04:55] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1779 in _wrapped_call_impl [04:55] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/model.py", line 145 in forward [04:55] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1790 in _call_impl [04:55] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1779 in _wrapped_call_impl [04:55] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/model.py", line 823 in forward [04:55] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1790 in _call_impl [04:55] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1779 in _wrapped_call_impl [04:55] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/model.py", line 983 in forward [04:55] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1790 in _call_impl [04:55] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1779 in _wrapped_call_impl [04:55] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/model.py", line 2041 in forward [04:55] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/model.py", line 3321 in _forward_image_no_act_ckpt [04:55] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/model.py", line 3314 in forward_image [04:55] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/utils.py", line 1324 in set_image [04:55] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 124 in decorate_context [04:55] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/segmentation.py", line 733 in execute [04:55] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/comfy_api/latest/_io.py", line 1833 in EXECUTE_NORMALIZED [04:55] [ComfyUI] File "/var/folders/tb/y368xp_x10s3ty1b_mtl5mxr0000gn/T/comfyui_pvenv__jjhnh0w/persistent_worker.py", line 1762 in main [04:55] [ComfyUI] File "/var/folders/tb/y368xp_x10s3ty1b_mtl5mxr0000gn/T/comfyui_pvenv__jjhnh0w/persistent_worker.py", line 1807 in [04:55] [ComfyUI] [04:55] [ComfyUI] [worker:ComfyUI-SAM3] === END === [04:55] [ComfyUI] [05:01] [capture-loop] iter=100 t=34.9s state={'complete': False, 'error': None, 'executedCount': 1, 'wsState': 1} eval_ms=6 ram=4.2/7.0GB (72%) swap=1.6/3.0GB (52%) cpu=47% browser_rss=0.31GB [05:07] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=8 [05:09] [capture-loop] iter=150 t=43.4s state={'complete': False, 'error': None, 'executedCount': 1, 'wsState': 1} eval_ms=6 ram=3.1/7.0GB (86%) swap=1.6/3.0GB (53%) cpu=51% browser_rss=0.32GB [05:10] Frame 4 saved (frame_003.jpg, t=43.6s) [05:10] [capture-periodic] freeze=61ms shot=456ms unfreeze=8ms saved=True [05:11] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=8 [05:12] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=8 [05:13] Frame 5 saved (frame_004.jpg, t=46.2s) [05:13] [capture-periodic] freeze=35ms shot=344ms unfreeze=11ms saved=True [05:14] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=8 [05:14] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=8 [05:15] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=8 [05:15] Frame 6 saved (frame_005.jpg, t=48.8s) [05:15] [capture-periodic] freeze=5ms shot=395ms unfreeze=7ms saved=True [05:16] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=8 [05:17] [capture-loop] iter=200 t=51.1s state={'complete': False, 'error': None, 'executedCount': 1, 'wsState': 1} eval_ms=10 ram=3.2/7.0GB (87%) swap=2.3/3.0GB (78%) cpu=72% browser_rss=0.29GB [05:18] Frame 7 saved (frame_006.jpg, t=51.3s) [05:18] [capture-periodic] freeze=5ms shot=627ms unfreeze=14ms saved=True [05:27] [capture-loop] iter=250 t=60.9s state={'complete': False, 'error': None, 'executedCount': 1, 'wsState': 1} eval_ms=8 ram=2.0/7.0GB (77%) swap=2.7/3.0GB (90%) cpu=72% browser_rss=0.25GB [05:34] [capture-loop] iter=300 t=68.1s state={'complete': False, 'error': None, 'executedCount': 1, 'wsState': 1} eval_ms=12 ram=2.4/7.0GB (80%) swap=2.7/4.0GB (67%) cpu=62% browser_rss=0.23GB [05:36] [ComfyUI] [SubprocessWorker] got response: ok [05:36] [ComfyUI] [comfy-env] SAM3MultipromptSegmentation done (66.10s): result: NodeOutput [05:38] Frame 8 saved (frame_007.jpg, t=71.9s) [05:38] [capture-periodic] freeze=7ms shot=286ms unfreeze=6ms saved=True [05:43] [capture-loop] iter=350 t=76.6s state={'complete': False, 'error': None, 'executedCount': 1, 'wsState': 1} eval_ms=5 ram=2.2/7.0GB (81%) swap=2.6/4.0GB (65%) cpu=65% browser_rss=0.29GB [05:45] Node executed (2 total), capturing... [05:47] Frame 9 saved (frame_008.jpg, t=79.2s) [05:47] [capture-node] freeze=5ms shot=1456ms unfreeze=10ms saved=True [05:48] [ComfyUI] BatchToMultiband: Processed 1 images with 3 channels each -> 3 channels [05:48] [ComfyUI] BatchToMultiband: Processed 7 masks -> 7 channels [05:48] Node executed (3 total), capturing... [05:49] Frame 10 saved (frame_009.jpg, t=82.0s) [05:49] [capture-node] freeze=92ms shot=789ms unfreeze=14ms saved=True [05:54] [ComfyUI] BatchToMultiband: Output shape torch.Size([1, 10, 4480, 6720]), channels: ['img_01_r', 'img_01_g', 'img_01_b', 'mask_01', 'mask_02', 'mask_03', 'mask_04', 'mask_05', 'mask_06', 'mask_07'] [05:54] [capture-loop] iter=400 t=87.8s state={'complete': False, 'error': None, 'executedCount': 3, 'wsState': 1} eval_ms=14 ram=2.2/7.0GB (84%) swap=2.6/4.0GB (65%) cpu=70% browser_rss=0.28GB [05:54] [ComfyUI] ResizeMultiband: 4480x6720 -> 600x900 [05:55] [ComfyUI] SaveMultibandImage: Saved to output/multiband.npz [05:55] [ComfyUI] Shape: (1, 10, 600, 900) [05:55] [ComfyUI] Format: npz [05:55] [ComfyUI] PreviewMultibandImage: Batch=1, Channels=10, Names=['img_01_r', 'img_01_g', 'img_01_b', 'mask_01', 'mask_02', 'mask_03', 'mask_04', 'mask_05', 'mask_06', 'mask_07'] [05:55] [ComfyUI] Prompt executed in 89.11 seconds [05:55] Frame 11 saved (frame_010.jpg, t=89.2s) [05:55] [capture-periodic] freeze=2ms shot=139ms unfreeze=7ms saved=True [05:55] Node executed (4 total), capturing... [05:56] Frame 12 saved (frame_011.jpg, t=89.7s) [05:56] [capture-node] freeze=3ms shot=181ms unfreeze=0ms saved=True [05:56] Execution complete (t=89.7s) [05:56] Captured 12 unique frames over 89.93s [05:56] [debug] iframes: 0 [05:56] [timing] trigger_3d_previews: 0.0s [05:58] Saved high-quality screenshot: scene_seg_executed.png [05:58] [timing] final_screenshot: 0.2s [05:58] Capturing execution frames: scene_seg_interactive.json [06:04] Frame 1 saved (frame_000.jpg, t=0.0s) [06:04] Validating workflow... [06:04] [ComfyUI] === /validate endpoint called === [06:04] [ComfyUI] Received JSON with keys: ['prompt'] [06:04] [ComfyUI] Prompt has 4 nodes [06:04] [ComfyUI] Node 1: LoadImage [06:04] [ComfyUI] Inputs: {"image": "image.png"} [06:04] [ComfyUI] Node 4: MaskPreview [06:04] [ComfyUI] Inputs: {"mask": ["26", 0]} [06:04] [ComfyUI] Node 12: LoadSAM3Model [06:04] [ComfyUI] Inputs: {"precision": "auto", "compile": false} [06:04] [ComfyUI] Node 26: SAM3InteractiveCollector [06:04] [ComfyUI] Inputs: {"multi_prompts_store": "[{\"positive_points\":[{\"x\":3348.288109906738,\"y\":2452.2208696844737}],\"negative_points\":[],\"positive_boxes\":[],\"negative_boxes\":[],\"name\":\"Prompt 1\"},{\"positiv [06:04] [ComfyUI] Generated prompt_id: 09060a63-3f30-44db-b946-a040d36b314e [06:04] [ComfyUI] Calling execution.validate_prompt()... [06:04] [ComfyUI] Validation result: valid=True [06:04] Queuing workflow for execution... [06:04] [ComfyUI] === Validation PASSED === [06:04] [ComfyUI] got prompt [06:06] [ComfyUI] [comfy-env] Running LoadSAM3Model: precision: 'auto', compile: False [06:06] [ComfyUI] [SubprocessWorker] call_method: nodes.load_model.LoadSAM3Model.EXECUTE_NORMALIZED [06:06] [ComfyUI] [SubprocessWorker] acquired lock, ensuring started... [06:06] [ComfyUI] [ComfyUI-SAM3] Health check: ping (timeout=5.0s)... [06:06] [ComfyUI] [ComfyUI-SAM3] Health check: failed [06:06] [ComfyUI] [ComfyUI-SAM3] Socket unhealthy, restarting worker... [06:06] [ComfyUI] [ComfyUI-SAM3] python: /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/bin/python [06:06] [ComfyUI] [ComfyUI-SAM3] sys_path sent to worker: ['/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI', '/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3', '/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/lib/python3.11/site-packages', '/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3'] [06:06] [ComfyUI] [SubprocessWorker] is_pixi=True, python=/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/bin/python [06:06] [ComfyUI] [SubprocessWorker] launching cmd=['/Users/runner/.pixi/bin/pixi', 'run', '--as-is', '--manifest-path', '/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/pixi.toml', '-e', 'sam3-nodes', 'python', '/var/folders/tb/y368xp_x10s3ty1b_mtl5mxr0000gn/T/comfyui_pvenv__jjhnh0w/persistent_worker.py', 'unix:///var/folders/tb/y368xp_x10s3ty1b_mtl5mxr0000gn/T/comfy_worker_2b7957c14f67.sock']... [06:06] [ComfyUI] [SubprocessWorker] socket_addr=unix:///var/folders/tb/y368xp_x10s3ty1b_mtl5mxr0000gn/T/comfy_worker_2b7957c14f67.sock exists=True [06:06] Frame 2 saved (frame_001.jpg, t=2.3s) [06:06] [capture-periodic] freeze=4ms shot=121ms unfreeze=4ms saved=True [06:08] Frame 3 saved (frame_002.jpg, t=4.4s) [06:08] [capture-periodic] freeze=5ms shot=127ms unfreeze=3ms saved=True [06:10] [capture-loop] iter=50 t=6.3s state={'complete': False, 'error': None, 'executedCount': 0, 'wsState': 1} eval_ms=5 ram=2.3/7.0GB (51%) swap=2.7/4.0GB (68%) cpu=56% browser_rss=0.73GB [06:15] [ComfyUI] [SubprocessWorker] worker started/confirmed [06:15] [ComfyUI] [SubprocessWorker] serializing kwargs to shm... [06:15] [ComfyUI] [SubprocessWorker] created 0 shm blocks [06:15] [ComfyUI] [SubprocessWorker] sending request via socket... [06:15] [ComfyUI] [SubprocessWorker] call_id=9 sending call_method [06:16] [capture-loop] iter=100 t=12.8s state={'complete': False, 'error': None, 'executedCount': 0, 'wsState': 1} eval_ms=4 ram=2.4/7.0GB (45%) swap=0.8/4.0GB (21%) cpu=62% browser_rss=0.65GB [06:17] [ComfyUI] [SubprocessWorker] got response: ok [06:17] [ComfyUI] [comfy-env] LoadSAM3Model done (11.05s): result: NodeOutput [06:17] [ComfyUI] [comfy-env] Running SAM3InteractiveCollector: multi_prompts_store: '[{"positive_points":[{"x":3348.288109906738,"y":2452.220..., sam3_model_config: dict, image: torch.float32 [1,4480,6720,3] cpu, unique_id: '26' [06:17] [ComfyUI] [SubprocessWorker] call_method: nodes.sam3_interactive.SAM3InteractiveCollector.EXECUTE_NORMALIZED [06:17] [ComfyUI] [SubprocessWorker] acquired lock, ensuring started... [06:17] [ComfyUI] [ComfyUI-SAM3] Health check: ping (timeout=5.0s)... [06:17] [ComfyUI] [ComfyUI-SAM3] Health check: ok [06:17] [ComfyUI] [SubprocessWorker] worker started/confirmed [06:17] [ComfyUI] [comfy-env] PRE-SERIALIZE 'image' shape: torch.Size([1, 4480, 6720, 3]) [06:17] [ComfyUI] [SubprocessWorker] serializing kwargs to shm... [06:17] [ComfyUI] [SubprocessWorker] created 0 shm blocks [06:17] [ComfyUI] [SubprocessWorker] sending request via socket... [06:17] [ComfyUI] [SubprocessWorker] call_id=10 sending call_method [06:17] [ComfyUI] [worker:ComfyUI-SAM3] [comfy-env] _from_shm got dict with keys: ['__type__', 'strategy', 'manager_path', 'storage_key', 'storage_size'] [06:17] [ComfyUI] [worker:ComfyUI-SAM3] [comfy-env] DESERIALIZE TensorRef: tensor_size=[1, 4480, 6720, 3] [06:17] [ComfyUI] [worker:ComfyUI-SAM3] [comfy-env] DESERIALIZED tensor shape: torch.Size([1, 4480, 6720, 3]) [06:17] Frame 4 saved (frame_003.jpg, t=13.6s) [06:17] [capture-periodic] freeze=5ms shot=161ms unfreeze=5ms saved=True [06:19] [ComfyUI] objc[28502]: Class AVFFrameReceiver is implemented in both /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/lib/libavdevice.62.3.101.dylib (0x12e1ec7a8) and /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/lib/python3.11/site-packages/cv2/.dylibs/libavdevice.61.3.100.dylib (0x1619e83a8). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed. [06:19] [ComfyUI] objc[28502]: Class AVFAudioReceiver is implemented in both /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/lib/libavdevice.62.3.101.dylib (0x12e1ec7f8) and /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/lib/python3.11/site-packages/cv2/.dylibs/libavdevice.61.3.100.dylib (0x1619e83f8). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed. [06:23] [capture-loop] iter=150 t=19.0s state={'complete': False, 'error': None, 'executedCount': 0, 'wsState': 1} eval_ms=3 ram=3.1/7.0GB (54%) swap=0.6/2.0GB (29%) cpu=46% browser_rss=0.65GB [06:29] [capture-loop] iter=200 t=25.4s state={'complete': False, 'error': None, 'executedCount': 0, 'wsState': 1} eval_ms=4 ram=3.6/7.0GB (62%) swap=0.6/2.0GB (29%) cpu=49% browser_rss=0.55GB [06:36] [capture-loop] iter=250 t=32.7s state={'complete': False, 'error': None, 'executedCount': 0, 'wsState': 1} eval_ms=5 ram=2.9/7.0GB (72%) swap=0.6/2.0GB (29%) cpu=57% browser_rss=0.42GB [06:43] [capture-loop] iter=300 t=39.2s state={'complete': False, 'error': None, 'executedCount': 0, 'wsState': 1} eval_ms=3 ram=2.6/7.0GB (77%) swap=0.6/2.0GB (29%) cpu=57% browser_rss=0.26GB [06:45] [ComfyUI] [SubprocessWorker] callback 'request_vram_budget' from call_id=10 [06:45] [ComfyUI] [comfy-env] VRAM request: 1.84GB, free before eviction: 1.66GB [06:45] [ComfyUI] [comfy-env] VRAM after eviction: 1.66GB [06:51] [capture-loop] iter=350 t=47.2s state={'complete': False, 'error': None, 'executedCount': 0, 'wsState': 1} eval_ms=4 ram=3.3/7.0GB (76%) swap=0.6/2.0GB (29%) cpu=49% browser_rss=0.27GB [06:57] [capture-loop] iter=400 t=53.3s state={'complete': False, 'error': None, 'executedCount': 0, 'wsState': 1} eval_ms=3 ram=3.3/7.0GB (67%) swap=0.6/2.0GB (28%) cpu=50% browser_rss=0.27GB [07:03] [capture-loop] iter=450 t=59.9s state={'complete': False, 'error': None, 'executedCount': 0, 'wsState': 1} eval_ms=4 ram=4.3/7.0GB (84%) swap=0.6/2.0GB (28%) cpu=46% browser_rss=0.23GB [07:07] [ComfyUI] [worker:ComfyUI-SAM3] [07:07] [ComfyUI] === WATCHDOG TICK 1 === [07:07] [ComfyUI] [worker:ComfyUI-SAM3] Current thread 0x00000001703e3000 (most recent call first): [07:07] [ComfyUI] File "/var/folders/tb/y368xp_x10s3ty1b_mtl5mxr0000gn/T/comfyui_pvenv__jjhnh0w/persistent_worker.py", line 52 in _watchdog [07:07] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/lib/python3.11/threading.py", line 982 in run [07:07] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/lib/python3.11/threading.py", line 1045 in _bootstrap_inner [07:07] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/lib/python3.11/threading.py", line 1002 in _bootstrap [07:07] [ComfyUI] [07:07] [ComfyUI] Thread 0x00000001ffe36240 (most recent call first): [07:07] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/comfy/ops.py", line 640 in forward_comfy_cast_weights [07:07] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/comfy/ops.py", line 649 in forward [07:07] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1790 in _call_impl [07:07] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1779 in _wrapped_call_impl [07:07] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/lib/python3.11/site-packages/torch/nn/modules/container.py", line 253 in forward [07:07] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1790 in _call_impl [07:07] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1779 in _wrapped_call_impl [07:07] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/model.py", line 2049 in forward [07:07] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/model.py", line 3321 in _forward_image_no_act_ckpt [07:07] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/model.py", line 3314 in forward_image [07:07] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/utils.py", line 1324 in set_image [07:07] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 124 in decorate_context [07:07] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3_interactive.py", line 687 in execute [07:07] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/comfy_api/latest/_io.py", line 1833 in EXECUTE_NORMALIZED [07:07] [ComfyUI] File "/var/folders/tb/y368xp_x10s3ty1b_mtl5mxr0000gn/T/comfyui_pvenv__jjhnh0w/persistent_worker.py", line 1762 in main [07:07] [ComfyUI] File "/var/folders/tb/y368xp_x10s3ty1b_mtl5mxr0000gn/T/comfyui_pvenv__jjhnh0w/persistent_worker.py", line 1807 in [07:07] [ComfyUI] [07:07] [ComfyUI] [worker:ComfyUI-SAM3] === END === [07:07] [ComfyUI] [07:12] [capture-loop] iter=500 t=68.8s state={'complete': False, 'error': None, 'executedCount': 0, 'wsState': 1} eval_ms=6 ram=3.7/7.0GB (76%) swap=0.5/2.0GB (27%) cpu=28% browser_rss=0.25GB [07:23] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=10 [07:24] Frame 5 saved (frame_004.jpg, t=80.4s) [07:24] [capture-periodic] freeze=2ms shot=194ms unfreeze=5ms saved=True [07:26] [capture-loop] iter=550 t=82.1s state={'complete': False, 'error': None, 'executedCount': 0, 'wsState': 1} eval_ms=5 ram=2.3/7.0GB (81%) swap=1.0/2.0GB (48%) cpu=55% browser_rss=0.20GB [07:27] [ComfyUI] [SubprocessWorker] got response: ok [07:28] [ComfyUI] [comfy-env] SAM3InteractiveCollector done (70.88s): result: NodeOutput [07:29] [ComfyUI] Prompt executed in 84.84 seconds [07:30] Node executed (1 total), capturing... [07:31] Frame 6 saved (frame_005.jpg, t=86.3s) [07:31] [capture-node] freeze=242ms shot=411ms unfreeze=2ms saved=True [07:31] Node executed (2 total), capturing... [07:32] Frame 7 saved (frame_006.jpg, t=87.2s) [07:32] [capture-node] freeze=2ms shot=682ms unfreeze=0ms saved=True [07:32] Execution complete (t=87.2s) [07:32] Captured 7 unique frames over 87.96s [07:32] [debug] iframes: 0 [07:32] [timing] trigger_3d_previews: 0.0s [07:34] Saved high-quality screenshot: scene_seg_interactive_executed.png [07:34] [timing] final_screenshot: 0.4s [07:34] [5/5] SKIPPED (not in cpu list) video_point_prompt.json [07:35] Results saved to /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/comfy-test-logs/SAM3-1659/dev/macos-cpu/results.json [07:35] Model report: 1 files, 3.2 GB -> /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/comfy-test-logs/SAM3-1659/dev/macos-cpu/models.json [07:35] Saved: /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/comfy-test-logs/SAM3-1659/dev/macos-cpu/index.html [07:35] macos: FAILED [07:35] Error: Workflow execution failed (1 error(s)) [07:35] Details: image_seg_text.json: Workflow execution failed: {'prompt_id': 'f8014278-5e7d-4b45-a7dc-432b29029e0f', 'node_id': '15', 'node_type': 'SAM3Grounding', 'executed': ['1', '12'], 'exception_message': 'The operator \'aten::_assert_async\' is not currently implemented for the MPS device. If you want this op to be considered for addition please comment on https://github.com/pytorch/pytorch/issues/141287 and mention use-case, that resulted in missing op as well as commit hash 70d99e998b4955e0049d13a98d77ae1b14db1f45. As a temporary fix, you can set the environment variable `PYTORCH_ENABLE_MPS_FALLBACK=1` to use the CPU as a fallback for this op. WARNING: this will be slower than running natively on MPS.\n\nWorker traceback:\nTraceback (most recent call last):\n File "/var/folders/tb/y368xp_x10s3ty1b_mtl5mxr0000gn/T/comfyui_pvenv__jjhnh0w/persistent_worker.py", line 1762, in main\n result = method(**inputs)\n ^^^^^^^^^^^^^^^^\n File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/comfy_api/latest/_io.py", line 1833, in EXECUTE_NORMALIZED\n to_return = cls.execute(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/segmentation.py", line 111, in execute\n result = cls._segment_grounding(\n ^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/segmentation.py", line 138, in _segment_grounding\n state = processor.set_text_prompt(text_prompt.strip(), state)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 124, in decorate_context\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/utils.py", line 1393, in set_text_prompt\n return self._forward_grounding(state)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 124, in decorate_context\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/utils.py", line 1477, in _forward_grounding\n outputs = self.model.forward_grounding(\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/model.py", line 3819, in forward_grounding\n prompt, prompt_mask, backbone_out = self._encode_prompt(\n ^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/model.py", line 3561, in _encode_prompt\n geo_feats, geo_masks = self.geometry_encoder(\n ^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1779, in _wrapped_call_impl\n return self._call_impl(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/.ce/.pixi/envs/sam3-nodes/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1790, in _call_impl\n return forward_call(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/model.py", line 2501, in forward\n final_embeds, final_mask = concat_padded_sequences(\n ^^^^^^^^^^^^^^^^^^^^^^^^\n File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/utils.py", line 1824, in concat_padded_sequences\n torch._assert_async(is_right_padded(mask1))\nNotImplementedError: The operator \'aten::_assert_async\' is not currently implemented for the MPS device. If you want this op to be considered for addition please comment on https://github.com/pytorch/pytorch/issues/141287 and mention use-case, that resulted in missing op as well as commit hash 70d99e998b4955e0049d13a98d77ae1b14db1f45. As a temporary fix, you can set the environment variable `PYTORCH_ENABLE_MPS_FALLBACK=1` to use the CPU as a fallback for this op. WARNING: this will be slower than running natively on MPS.\n\n', 'exception_type': 'comfy_env.isolation.workers.base.WorkerError', 'traceback': [' File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/execution.py", line 535, in execute\n output_data, output_ui, has_subgraph, has_pending_tasks = await get_output_data(prompt_id, unique_id, obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n', ' File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/execution.py", line 335, in get_output_data\n return_values = await _async_map_node_over_list(prompt_id, unique_id, obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n', ' File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/execution.py", line 309, in _async_map_node_over_list\n await process_inputs(input_dict, i)\n', ' File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/execution.py", line 297, in process_inputs\n result = f(**inputs)\n ^^^^^^^^^^^\n', ' File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/venv/lib/python3.11/site-packages/comfy_env/isolation/metadata.py", line 536, in proxy\n result = worker.call_method(\n ^^^^^^^^^^^^^^^^^^^\n', ' File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/venv/lib/python3.11/site-packages/comfy_env/isolation/workers/subprocess.py", line 674, in call_method\n raise WorkerError(\n'], 'current_inputs': {'confidence_threshold': [0.2], 'text_prompt': ['child'], 'max_detections': [-1], 'sam3_model_config': ["{'checkpoint_path': '/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/models/sam3/sam3.safetensors', 'bpe_path': '/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1659/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/bpe_simple_vocab_16e6.txt.gz', 'precision': 'fp32', 'dtype': 'fp32', 'compile': False}"], 'image': ['tensor([[[[0.2902, 0.4667, 0.5961],\n [0.2902, 0.4667, 0.5961],\n [0.2902, 0.4667, 0.5961],\n ...,\n [0.3176, 0.5804, 0.7216],\n [0.3333, 0.6039, 0.7451],\n [0.3529, 0.6235, 0.7647]],\n\n [[0.2902, 0.4667, 0.5961],\n [0.2902, 0.4667, 0.5961],\n [0.2902, 0.4667, 0.5961],\n ...,\n [0.3137, 0.5765, 0.7176],\n [0.3137, 0.5843, 0.7255],\n [0.3255, 0.5961, 0.7373]],\n\n [[0.2902, 0.4667, 0.5961],\n [0.2902, 0.4667, 0.5961],\n [0.2902, 0.4667, 0.5961],\n ...,\n [0.3922, 0.5333, 0.6275],\n [0.3922, 0.5294, 0.6235],\n [0.3961, 0.5333, 0.6275]],\n\n ...,\n\n [[0.4941, 0.4863, 0.4157],\n [0.4941, 0.4863, 0.4157],\n [0.4980, 0.4902, 0.4196],\n ...,\n [0.5647, 0.5608, 0.5059],\n [0.5647, 0.5608, 0.5059],\n [0.5647, 0.5608, 0.5059]],\n\n [[0.5098, 0.5020, 0.4314],\n [0.5098, 0.5020, 0.4314],\n [0.5137, 0.5059, 0.4353],\n ...,\n [0.5647, 0.5608, 0.5059],\n [0.5647, 0.5608, 0.5059],\n [0.5647, 0.5608, 0.5059]],\n\n [[0.5176, 0.5098, 0.4392],\n [0.5216, 0.5137, 0.4431],\n [0.5255, 0.5176, 0.4471],\n ...,\n [0.5647, 0.5608, 0.5059],\n [0.5647, 0.5608, 0.5059],\n [0.5647, 0.5608, 0.5059]]]])']}, 'current_outputs': ['1', '15', '4', '12', '3'], 'timestamp': 1778432637946} Details: Workflow: /private/var/folders/tb/y368xp_x10s3ty1b_mtl5mxr0000gn/T/comfy-test-run-jp0_6lcq/ComfyUI-SAM3/workflows/image_seg_text.json Node error: SAM3Grounding [07:35] Stopping ComfyUI server...