[00:00] Crash dump logging enabled: /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/comfy-test-logs/SAM3-1105/dev/macos-cpu/crash_dump.log [00:00] [00:00] [1/5] SYNTAX [00:00] ---------------------------------------- [00:00] [DEBUG] server=None, api=None [00:00] Found pyproject.toml (modern format) [00:00] Found requirements.txt (legacy format) [00:00] Unicode check: OK (all characters cp1252-safe) [00:00] Forbidden patterns check: OK [00:00] [SYNTAX] PASSED [00:00] [00:00] [2/5] INSTALL [00:00] ---------------------------------------- [00:00] [DEBUG] server=None, api=None [00:00] Setting up ComfyUI... [00:00] Cloning ComfyUI (latest)... [00:00] Running: git clone --depth 1 https://github.com/comfyanonymous/ComfyUI.git /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI [00:02] Creating virtual environment... [00:02] Running: /Library/Frameworks/Python.framework/Versions/3.13/bin/python -m venv /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/venv [00:05] Installing uv into venv... [00:05] Running: /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/venv/bin/python -m pip install uv [00:07] Installing PyTorch... [00:07] Pinning torch family: torch==2.10.0 torchvision==0.25.0 torchaudio==2.10.0 [00:07] Running: /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/venv/bin/python -m uv pip install torch==2.10.0 torchvision==0.25.0 torchaudio==2.10.0 [00:21] Installing ComfyUI requirements... [00:21] Running: /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/venv/bin/python -m uv pip install -r /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI/requirements.txt [00:39] Installing custom node... [00:39] Copying ComfyUI-SAM3 to custom_nodes/... [00:39] Installing node requirements... [00:39] Running: /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/venv/bin/python -m uv pip install -r /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI/custom_nodes/ComfyUI-SAM3/requirements.txt [00:40] Running install.py... [00:40] Running: /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/venv/bin/python /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI/custom_nodes/ComfyUI-SAM3/install.py [00:41] [00:41] Installing 2 node dependencies... [00:41] Cloning ComfyUI-VideoHelperSuite... [00:42] Installing requirements for ComfyUI-VideoHelperSuite... [00:44] Cloning ComfyUI-Multiband... [00:44] Installing requirements for ComfyUI-Multiband... [00:44] [00:44] [requirements] Re-installing main package requirements... [00:44] Installing requirements for ComfyUI-SAM3... [00:45] [comfy-env] _discover: scanning /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI/custom_nodes [00:45] [comfy-env] _discover: ComfyUI-Multiband: no comfy-env.toml found [00:46] [comfy-env] _discover: ComfyUI-SAM3 -> sam3-nodes (custom_nodes/ComfyUI-SAM3/nodes/comfy-env.toml) [00:46] [comfy-env] _discover: ComfyUI-VideoHelperSuite: no comfy-env.toml found [00:46] [comfy-env] Workspace: /Users/runner/.ce [00:46] [comfy-env] ComfyUI: /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI [00:46] [comfy-env] Found 1 node config(s): [00:46] - sam3-nodes <- custom_nodes/ComfyUI-SAM3/nodes/comfy-env.toml (python=host) [00:46] [comfy-env] Bootstrap interpreter has python 3.13 + torch 2.10.0 (cpu). Looking for cuda-only wheels matching cpu/torch2.10/cp313. [00:46] Generated /Users/runner/.ce/pixi.toml [00:46] [comfy-env] Installing 1 environment(s): [00:46] - sam3-nodes (python=host, 20 pypi deps, 3 cuda wheels) [00:46] [comfy-env] Running `pixi install -e sam3-nodes`... [00:59] WARN Skipped running the post-link scripts because `run-post-link-scripts` = `false` [00:59] - bin/.librsvg-pre-unlink.sh [00:59] To enable them, run: [00:59] pixi config set --local run-post-link-scripts insecure [00:59] More info: [00:59] https://pixi.sh/latest/reference/pixi_configuration/#run-post-link-scripts [01:15] ✔ The sam3-nodes environment has been installed. [01:15] [comfy-env] sam3-nodes: deduped libomp [01:15] [comfy-env] Recorded install hash b22362973c85 -> /Users/runner/.ce/install.hash [01:15] [comfy-env] Install log: /Users/runner/.ce/install.log [01:15] [01:15] Installation complete! [01:15] Installing 2 node dependency(ies)... [01:15] ComfyUI-VideoHelperSuite from Kosinkadink/ComfyUI-VideoHelperSuite [01:15] ComfyUI-VideoHelperSuite already exists, skipping... [01:15] ComfyUI-Multiband from PozzettiAndrea/ComfyUI-Multiband [01:15] ComfyUI-Multiband already exists, skipping... [01:15] Installing validation endpoint... [01:15] Cloning PozzettiAndrea/ComfyUI-validate-endpoint... [01:15] Running: git clone --depth 1 https://x-access-token:***@github.com/PozzettiAndrea/ComfyUI-validate-endpoint.git /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI/custom_nodes/ComfyUI-validate-endpoint [01:16] CUDA packages absent (will mock): cc_torch, torch_generic_nms, flash_attn [01:16] [INSTALL] PASSED [01:16] [01:16] [3/5] REGISTRATION [01:16] ---------------------------------------- [01:16] [DEBUG] server=None, api=None [01:16] Starting ComfyUI server... [01:16] Starting ComfyUI server on port 41889... [01:16] CUDA mock packages: cc_torch, torch_generic_nms, flash_attn [01:16] Starting ComfyUI server on port 41889... [01:16] Waiting for server to be ready (timeout: 600s)... [01:18] [ComfyUI] setup plugin alembic.autogenerate.schemas [01:18] [ComfyUI] setup plugin alembic.autogenerate.tables [01:18] [ComfyUI] setup plugin alembic.autogenerate.types [01:18] [ComfyUI] setup plugin alembic.autogenerate.constraints [01:18] [ComfyUI] setup plugin alembic.autogenerate.defaults [01:18] [ComfyUI] setup plugin alembic.autogenerate.comments [01:19] [ComfyUI] Could not autodetect AIMDO implementation, assuming Nvidia [01:19] [ComfyUI] comfy-aimdo unsupported operating system: Darwin [01:19] [ComfyUI] NOTE: comfy-aimdo currently only supports Windows and Linux [01:19] [ComfyUI] [comfy-env] ComfyUI-SAM3: 1 isolation env(s): [01:19] [ComfyUI] [comfy-env] nodes -> /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes [01:19] [ComfyUI] [comfy-env] Workspace: /Users/runner/.ce [01:19] [ComfyUI] [comfy-env] env: /Users/runner/.ce/.pixi/envs/sam3-nodes [OK] [01:24] [ComfyUI] [comfy-env] prestartup complete [01:24] [ComfyUI] [01:24] [ComfyUI] Prestartup times for custom nodes: [01:24] [ComfyUI] 5.4 seconds: /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI/custom_nodes/ComfyUI-SAM3 [01:24] [ComfyUI] [01:24] [ComfyUI] WARNING: Potential Error in code: Torch already imported, torch should never be imported before this point. [01:28] [ComfyUI] Found comfy_kitchen backend triton: {'available': False, 'disabled': True, 'unavailable_reason': "ImportError: No module named 'triton'", 'capabilities': []} [01:28] [ComfyUI] Found comfy_kitchen backend cuda: {'available': False, 'disabled': True, 'unavailable_reason': 'Extension file not found: /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/venv/lib/python3.13/site-packages/comfy_kitchen/backends/cuda/_C.abi3.so', 'capabilities': []} [01:28] [ComfyUI] Found comfy_kitchen backend eager: {'available': True, 'disabled': False, 'unavailable_reason': None, 'capabilities': ['apply_rope', 'apply_rope1', 'dequantize_mxfp8', 'dequantize_nvfp4', 'dequantize_per_tensor_fp8', 'quantize_mxfp8', 'quantize_nvfp4', 'quantize_per_tensor_fp8', 'scaled_mm_mxfp8', 'scaled_mm_nvfp4']} [01:28] [ComfyUI] Checkpoint files will always be loaded safely. [01:29] [ComfyUI] Total VRAM 7168 MB, total RAM 7168 MB [01:29] [ComfyUI] pytorch version: 2.10.0 [01:29] [ComfyUI] Mac Version (15, 7, 4) [01:29] [ComfyUI] Set vram state to: SHARED [01:29] [ComfyUI] Device: mps [01:29] [ComfyUI] Using sub quadratic optimization for attention, if you have memory or speed issues try using: --use-split-cross-attention [01:31] [ComfyUI] objc[7555]: Class AVFFrameReceiver is implemented in both /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/venv/lib/python3.13/site-packages/av/.dylibs/libavdevice.62.1.100.dylib (0x11f4dc3a8) and /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/venv/lib/python3.13/site-packages/cv2/.dylibs/libavdevice.61.3.100.dylib (0x127ff03a8). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed. [01:31] [ComfyUI] objc[7555]: Class AVFAudioReceiver is implemented in both /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/venv/lib/python3.13/site-packages/av/.dylibs/libavdevice.62.1.100.dylib (0x11f4dc3f8) and /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/venv/lib/python3.13/site-packages/cv2/.dylibs/libavdevice.61.3.100.dylib (0x127ff03f8). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed. [01:37] [ComfyUI] Python version: 3.13.13 (v3.13.13:01104ce1beb, Apr 7 2026, 14:43:30) [Clang 16.0.0 (clang-1600.0.26.6)] [01:37] [ComfyUI] ComfyUI version: 0.21.1 [01:37] [ComfyUI] comfy-aimdo version: 0.3.0 [01:37] [ComfyUI] comfy-kitchen version: 0.2.8 [01:37] [ComfyUI] ****** User settings have been changed to be stored on the server instead of browser storage. ****** [01:37] [ComfyUI] ****** For multi-user setups add the --multi-user CLI argument to enable multiple user profiles. ****** [01:37] [ComfyUI] comfyui-frontend-package version: 1.43.18 [01:37] [ComfyUI] comfyui-workflow-templates version: 0.9.77 [01:37] [ComfyUI] comfyui-embedded-docs version: 0.5.0 [01:37] [ComfyUI] comfy-kitchen version: 0.2.8 [01:37] [ComfyUI] comfy-aimdo version: 0.3.0 [01:37] [ComfyUI] [Prompt Server] web root: /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/venv/lib/python3.13/site-packages/comfyui_frontend_package/static [01:37] [ComfyUI] Asset seeder disabled [01:40] [ComfyUI] [comfy-env] Version: 0.3.89 [01:40] [ComfyUI] [comfy-env] register_nodes: ComfyUI-SAM3 [01:40] [ComfyUI] [comfy-env] ComfyUI source dir: /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI [01:40] [ComfyUI] [comfy-env] Found 1 isolation env(s) [01:40] [ComfyUI] [comfy-env] Scanning nodes metadata (isolation root)... [01:40] [ComfyUI] [comfy-env] Metadata scan: /Users/runner/.pixi/bin/pixi run --as-is --manifest-path /Users/runner/.ce/pixi.toml -e sam3-nodes python /var/folders/tb/y368xp_x10s3ty1b_mtl5mxr0000gn/T/comfy_meta_94m9rhql.py /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI/custom_nodes/ComfyUI-SAM3 nodes /var/folders/tb/y368xp_x10s3ty1b_mtl5mxr0000gn/T/comfy_meta_out_hk4hsk35.pkl [01:40] [ComfyUI] [comfy-env] Scan env PATH for nodes: [01:40] [ComfyUI] [comfy-env] [0] /Users/runner/hostedtoolcache/Python/3.13.13/arm64/bin [01:40] [ComfyUI] [comfy-env] [1] /Users/runner/hostedtoolcache/Python/3.13.13/arm64 [01:40] [ComfyUI] [comfy-env] [2] /opt/homebrew/lib/ruby/gems/3.3.0/bin [01:40] [ComfyUI] [comfy-env] [3] /opt/homebrew/opt/ruby@3.3/bin [01:40] [ComfyUI] [comfy-env] [4] /Users/runner/.local/bin [01:40] [ComfyUI] [comfy-env] [5] /opt/homebrew/bin [01:40] [ComfyUI] [comfy-env] [6] /opt/homebrew/sbin [01:40] [ComfyUI] [comfy-env] [7] /Users/runner/.cargo/bin [01:40] [ComfyUI] [comfy-env] [8] /usr/local/opt/curl/bin [01:40] [ComfyUI] [comfy-env] [9] /usr/local/bin [01:40] [ComfyUI] [comfy-env] [10] /usr/local/sbin [01:40] [ComfyUI] [comfy-env] [11] /Users/runner/bin [01:40] [ComfyUI] [comfy-env] [12] /Users/runner/.yarn/bin [01:40] [ComfyUI] [comfy-env] [13] /Users/runner/Library/Android/sdk/tools [01:40] [ComfyUI] [comfy-env] [14] /Users/runner/Library/Android/sdk/platform-tools [01:40] [ComfyUI] [comfy-env] [15] /Library/Frameworks/Python.framework/Versions/Current/bin [01:40] [ComfyUI] [comfy-env] [16] /Library/Frameworks/Mono.framework/Versions/Current/Commands [01:40] [ComfyUI] [comfy-env] [17] /usr/bin [01:40] [ComfyUI] [comfy-env] [18] /bin [01:40] [ComfyUI] [comfy-env] [19] /usr/sbin [01:40] [ComfyUI] [comfy-env] [20] /sbin [01:40] [ComfyUI] [comfy-env] [21] /Users/runner/.dotnet/tools [01:54] [ComfyUI] [comfy-env] Metadata scan stderr for nodes: [01:54] [ComfyUI] [comfy-env] [meta-scan] importing nodes from /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI/custom_nodes/ComfyUI-SAM3 [01:54] [ComfyUI] [comfy-env] [meta-scan] import OK [01:54] [ComfyUI] [comfy-env] Scanned nodes: 15 nodes (13.4s) [01:54] [ComfyUI] [comfy-env] Scanned nodes root: 15 nodes (13.4s) [01:54] [ComfyUI] [comfy-env] Registered proxy route: POST /sam3/interactive_segment_one -> nodes.sam3_interactive._api_segment_one [01:54] [ComfyUI] [comfy-env] Imported nodes root: 15 nodes (isolation) [01:54] [ComfyUI] [comfy-env] Registered 15 total nodes [01:54] [ComfyUI] ComfyUI-Multiband loaded: 15 nodes registered [01:54] [ComfyUI] [01:54] [ComfyUI] Import times for custom nodes: [01:54] [ComfyUI] 0.0 seconds: /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI/custom_nodes/ComfyUI-validate-endpoint [01:54] [ComfyUI] 0.0 seconds: /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI/custom_nodes/websocket_image_save.py [01:54] [ComfyUI] 0.2 seconds: /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI/custom_nodes/ComfyUI-VideoHelperSuite [01:54] [ComfyUI] 0.4 seconds: /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI/custom_nodes/ComfyUI-Multiband [01:54] [ComfyUI] 13.7 seconds: /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI/custom_nodes/ComfyUI-SAM3 [01:54] [ComfyUI] [01:54] [ComfyUI] Context impl SQLiteImpl. [01:54] [ComfyUI] Will assume non-transactional DDL. [01:54] [ComfyUI] Context impl SQLiteImpl. [01:54] [ComfyUI] Will assume non-transactional DDL. [01:54] [ComfyUI] Running upgrade -> 0001_assets, Initial assets schema [01:54] [ComfyUI] Revision ID: 0001_assets [01:54] [ComfyUI] Revises: None [01:54] [ComfyUI] Create Date: 2025-12-10 00:00:00 [01:54] [ComfyUI] Running upgrade 0001_assets -> 0002_merge_to_asset_references, Merge AssetInfo and AssetCacheState into unified asset_references table. [01:54] [ComfyUI] Running upgrade 0002_merge_to_asset_references -> 0003_add_metadata_job_id, Add system_metadata and job_id columns to asset_references. [01:54] [ComfyUI] Change preview_id FK from assets.id to asset_references.id. [01:54] [ComfyUI] Database upgraded from None to 0003_add_metadata_job_id [01:54] [ComfyUI] Starting server [01:54] [ComfyUI] [01:54] [ComfyUI] To see the GUI go to: http://127.0.0.1:41889 [01:54] Server responding, waiting for nodes to load... [02:14] Server is ready! [02:14] Checking for import errors in server logs... [02:14] No import errors detected [02:15] Found 790 registered nodes [02:15] [REGISTRATION] PASSED [02:15] [02:15] [4/5] INSTANTIATION [02:15] ---------------------------------------- [02:15] [DEBUG] server=, api= [02:15] Testing node constructors... [02:23] All 15 node(s) instantiated successfully! [02:23] [INSTANTIATION] PASSED [02:23] [02:23] [5/5] EXECUTION [02:23] ---------------------------------------- [02:23] [DEBUG] server=, api= [02:23] CPU runner - will execute 4 workflow(s) [02:23] Running 5 workflow(s) (all with videos)... [02:23] Installing screenshot dependencies (playwright, pillow)... [02:24] Packages installed, downloading chromium browser... [02:35] Screenshot dependencies installed successfully [02:35] Starting headless browser... [02:36] macOS -- using Apple Metal GPU via ANGLE [02:36] Capturing execution frames: image_seg_point.json [02:38] [ComfyUI] [DEPRECATION WARNING] Detected import of deprecated legacy API: /extensions/core/widgetInputs.js. This is likely caused by a custom node extension using outdated APIs. Please update your extensions or contact the extension author for an updated version. [02:44] Frame 1 saved (frame_000.jpg, t=0.0s) [02:44] Validating workflow... [02:44] [ComfyUI] === /validate endpoint called === [02:44] [ComfyUI] Received JSON with keys: ['prompt'] [02:44] [ComfyUI] Prompt has 6 nodes [02:44] [ComfyUI] Node 1: LoadImage [02:44] [ComfyUI] Inputs: {"image": "example_image.jpg"} [02:44] [ComfyUI] Node 15: SAM3PointCollector [02:44] [ComfyUI] Inputs: {"points_store": "{\"positive\":[{\"x\":426.27735719657574,\"y\":321.42611685978835},{\"x\":417.13743597288953,\"y\":407.3413796907987},{\"x\":431.76130993078743,\"y\":549.9241563039649}],\"negative\" [02:44] [ComfyUI] Node 28: PreviewImage [02:44] [ComfyUI] Inputs: {"images": ["31", 2]} [02:44] [ComfyUI] Node 29: MaskPreview [02:44] [ComfyUI] Inputs: {"mask": ["31", 0]} [02:44] [ComfyUI] Node 30: LoadSAM3Model [02:44] [ComfyUI] Inputs: {"precision": "auto", "compile": false} [02:44] [ComfyUI] Node 31: SAM3Segmentation [02:44] [ComfyUI] Inputs: {"refinement_iterations": 0, "use_multimask": true, "output_best_mask": true, "sam3_model_config": ["30", 0], "image": ["1", 0], "positive_points": ["15", 0], "negative_points": ["15", 1]} [02:44] [ComfyUI] Generated prompt_id: d089e8f1-aca2-4982-aa8b-a7be73dbc81c [02:44] [ComfyUI] Calling execution.validate_prompt()... [02:44] [ComfyUI] Validation result: valid=True [02:44] [ComfyUI] === Validation PASSED === [02:44] Queuing workflow for execution... [02:44] [ComfyUI] got prompt [02:44] [ComfyUI] [comfy-env] Running SAM3PointCollector: points_store: '{"positive":[{"x":426.27735719657574,"y":321.42611685978..., coordinates: '[{"x":426.27735719657574,"y":321.42611685978835},{"x":41..., neg_coordinates: '[]', image: torch.float32 [1,720,1280,3] cpu [02:44] [ComfyUI] [comfy-env] SubprocessWorker: /Users/runner/.ce/.pixi/envs/sam3-nodes/bin/python [02:44] [ComfyUI] [comfy-env] env_vars: {'PYTORCH_ENABLE_MPS_FALLBACK': '1', 'COMFYUI_BASE': '/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI'} [02:44] [ComfyUI] [SubprocessWorker] call_method: nodes.sam3_interactive.SAM3PointCollector.EXECUTE_NORMALIZED [02:44] [ComfyUI] [SubprocessWorker] acquired lock, ensuring started... [02:44] [ComfyUI] [ComfyUI-SAM3] python: /Users/runner/.ce/.pixi/envs/sam3-nodes/bin/python [02:44] [ComfyUI] [ComfyUI-SAM3] sys_path sent to worker: ['/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI', '/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI/custom_nodes/ComfyUI-SAM3', '/Users/runner/.ce/.pixi/envs/sam3-nodes/lib/python3.13/site-packages', '/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI/custom_nodes/ComfyUI-SAM3'] [02:44] [ComfyUI] [SubprocessWorker] is_pixi=True, python=/Users/runner/.ce/.pixi/envs/sam3-nodes/bin/python [02:44] [ComfyUI] [SubprocessWorker] launching cmd=['/Users/runner/.pixi/bin/pixi', 'run', '--as-is', '--manifest-path', '/Users/runner/.ce/pixi.toml', '-e', 'sam3-nodes', 'python', '/var/folders/tb/y368xp_x10s3ty1b_mtl5mxr0000gn/T/comfyui_pvenv_2em037f9/persistent_worker.py', 'unix:///var/folders/tb/y368xp_x10s3ty1b_mtl5mxr0000gn/T/comfy_worker_776c94d0f416.sock']... [02:44] [ComfyUI] [SubprocessWorker] socket_addr=unix:///var/folders/tb/y368xp_x10s3ty1b_mtl5mxr0000gn/T/comfy_worker_776c94d0f416.sock exists=True [02:46] Frame 2 saved (frame_001.jpg, t=2.4s) [02:46] [capture-periodic] freeze=2ms shot=147ms unfreeze=6ms saved=True [02:48] [ComfyUI] [SubprocessWorker] worker started/confirmed [02:48] [ComfyUI] [comfy-env] PRE-SERIALIZE 'image' shape: torch.Size([1, 720, 1280, 3]) [02:48] [ComfyUI] [SubprocessWorker] serializing kwargs to shm... [02:48] [ComfyUI] [SubprocessWorker] created 0 shm blocks [02:48] [ComfyUI] [SubprocessWorker] sending request via socket... [02:48] [ComfyUI] [SubprocessWorker] call_id=1 sending call_method [02:48] [ComfyUI] [worker:ComfyUI-SAM3] [comfy-env] _from_shm got dict with keys: ['__type__', 'strategy', 'manager_path', 'storage_key', 'storage_size'] [02:48] [ComfyUI] [worker:ComfyUI-SAM3] [comfy-env] DESERIALIZE TensorRef: tensor_size=[1, 720, 1280, 3] [02:48] [ComfyUI] [worker:ComfyUI-SAM3] [comfy-env] DESERIALIZED tensor shape: torch.Size([1, 720, 1280, 3]) [02:48] Frame 3 saved (frame_002.jpg, t=4.6s) [02:48] [capture-periodic] freeze=4ms shot=97ms unfreeze=3ms saved=True [02:49] [ComfyUI] [SubprocessWorker] got response: ok [02:49] [ComfyUI] [comfy-env] SAM3PointCollector done (4.81s): result: NodeOutput [02:49] [ComfyUI] [comfy-env] Running LoadSAM3Model: precision: 'auto', compile: False [02:49] [ComfyUI] [SubprocessWorker] call_method: nodes.load_model.LoadSAM3Model.EXECUTE_NORMALIZED [02:49] [ComfyUI] [SubprocessWorker] acquired lock, ensuring started... [02:49] [ComfyUI] [ComfyUI-SAM3] Health check: ping (timeout=5.0s)... [02:49] [ComfyUI] [ComfyUI-SAM3] Health check: ok [02:49] [ComfyUI] [SubprocessWorker] worker started/confirmed [02:49] [ComfyUI] [SubprocessWorker] serializing kwargs to shm... [02:49] [ComfyUI] [SubprocessWorker] created 0 shm blocks [02:49] [ComfyUI] [SubprocessWorker] sending request via socket... [02:49] [ComfyUI] [SubprocessWorker] call_id=2 sending call_method [02:49] [ComfyUI] [worker:ComfyUI-SAM3] [SAM3] Model not found at /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI/models/sam3/sam3.safetensors, downloading from HuggingFace... [02:49] Node executed (1 total), capturing... [02:49] Frame 4 saved (frame_003.jpg, t=5.4s) [02:49] [capture-node] freeze=2ms shot=101ms unfreeze=6ms saved=True [02:49] [ComfyUI] [worker:ComfyUI-SAM3] [SAM3] Downloading sam3.safetensors from apozz/sam3-safetensors... [02:50] [capture-loop] iter=50 t=6.2s state={'complete': False, 'error': None, 'executedCount': 1, 'wsState': 1} eval_ms=5 ram=3.7/7.0GB (60%) swap=0.0/0.0GB (0%) cpu=71% browser_rss=0.54GB [02:51] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:51] Frame 5 saved (frame_004.jpg, t=7.6s) [02:51] [capture-periodic] freeze=4ms shot=139ms unfreeze=2ms saved=True [02:51] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:52] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:53] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:53] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:54] Frame 6 saved (frame_005.jpg, t=9.9s) [02:54] [capture-periodic] freeze=5ms shot=154ms unfreeze=5ms saved=True [02:54] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:54] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:55] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:56] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:56] [capture-loop] iter=100 t=12.2s state={'complete': False, 'error': None, 'executedCount': 1, 'wsState': 1} eval_ms=7 ram=3.7/7.0GB (60%) swap=0.0/0.0GB (0%) cpu=87% browser_rss=0.50GB [02:56] Frame 7 saved (frame_006.jpg, t=12.2s) [02:56] [capture-periodic] freeze=5ms shot=137ms unfreeze=3ms saved=True [02:56] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:56] [ComfyUI] [worker:ComfyUI-SAM3] [SAM3] Download progress: 10% [02:57] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:57] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:58] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:58] Frame 8 saved (frame_007.jpg, t=14.4s) [02:58] [capture-periodic] freeze=2ms shot=131ms unfreeze=3ms saved=True [02:58] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [02:59] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:00] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:00] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:01] Frame 9 saved (frame_008.jpg, t=16.6s) [03:01] [capture-periodic] freeze=8ms shot=203ms unfreeze=4ms saved=True [03:01] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:01] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:02] [capture-loop] iter=150 t=18.4s state={'complete': False, 'error': None, 'executedCount': 1, 'wsState': 1} eval_ms=2 ram=3.8/7.0GB (60%) swap=0.0/0.0GB (0%) cpu=80% browser_rss=0.51GB [03:02] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:02] [ComfyUI] [worker:ComfyUI-SAM3] [SAM3] Download progress: 20% [03:03] Frame 10 saved (frame_009.jpg, t=18.9s) [03:03] [capture-periodic] freeze=3ms shot=131ms unfreeze=3ms saved=True [03:03] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:04] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:04] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:05] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:05] Frame 11 saved (frame_010.jpg, t=21.2s) [03:05] [capture-periodic] freeze=13ms shot=153ms unfreeze=2ms saved=True [03:05] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:06] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:06] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:07] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:07] Frame 12 saved (frame_011.jpg, t=23.4s) [03:07] [capture-periodic] freeze=3ms shot=136ms unfreeze=3ms saved=True [03:08] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:08] [capture-loop] iter=200 t=24.4s state={'complete': False, 'error': None, 'executedCount': 1, 'wsState': 1} eval_ms=3 ram=3.8/7.0GB (60%) swap=0.0/0.0GB (0%) cpu=74% browser_rss=0.46GB [03:08] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:08] [ComfyUI] [worker:ComfyUI-SAM3] [SAM3] Download progress: 30% [03:09] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:09] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:09] Frame 13 saved (frame_012.jpg, t=25.7s) [03:09] [capture-periodic] freeze=5ms shot=111ms unfreeze=2ms saved=True [03:10] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:10] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:11] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:12] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:12] Frame 14 saved (frame_013.jpg, t=27.9s) [03:12] [capture-periodic] freeze=5ms shot=120ms unfreeze=4ms saved=True [03:12] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:13] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:13] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:14] [capture-loop] iter=250 t=30.2s state={'complete': False, 'error': None, 'executedCount': 1, 'wsState': 1} eval_ms=3 ram=3.7/7.0GB (60%) swap=0.0/0.0GB (0%) cpu=72% browser_rss=0.46GB [03:14] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:14] [ComfyUI] [worker:ComfyUI-SAM3] [SAM3] Download progress: 40% [03:14] Frame 15 saved (frame_014.jpg, t=30.2s) [03:14] [capture-periodic] freeze=2ms shot=156ms unfreeze=5ms saved=True [03:14] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:15] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:16] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:16] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:16] Frame 16 saved (frame_015.jpg, t=32.5s) [03:16] [capture-periodic] freeze=2ms shot=108ms unfreeze=2ms saved=True [03:17] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:17] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:18] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:18] Frame 17 saved (frame_016.jpg, t=34.6s) [03:18] [capture-periodic] freeze=3ms shot=133ms unfreeze=4ms saved=True [03:19] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:19] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:20] [capture-loop] iter=300 t=36.2s state={'complete': False, 'error': None, 'executedCount': 1, 'wsState': 1} eval_ms=9 ram=3.7/7.0GB (60%) swap=0.0/0.0GB (0%) cpu=78% browser_rss=0.46GB [03:20] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:20] [ComfyUI] [worker:ComfyUI-SAM3] [SAM3] Download progress: 50% [03:21] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:21] Frame 18 saved (frame_017.jpg, t=36.9s) [03:21] [capture-periodic] freeze=2ms shot=110ms unfreeze=4ms saved=True [03:21] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:22] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:22] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:23] Frame 19 saved (frame_018.jpg, t=39.1s) [03:23] [capture-periodic] freeze=2ms shot=121ms unfreeze=4ms saved=True [03:23] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:24] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:24] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:25] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:25] Frame 20 saved (frame_019.jpg, t=41.4s) [03:25] [capture-periodic] freeze=6ms shot=131ms unfreeze=5ms saved=True [03:25] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:26] [capture-loop] iter=350 t=42.2s state={'complete': False, 'error': None, 'executedCount': 1, 'wsState': 1} eval_ms=2 ram=3.7/7.0GB (60%) swap=0.0/0.0GB (0%) cpu=77% browser_rss=0.46GB [03:26] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:26] [ComfyUI] [worker:ComfyUI-SAM3] [SAM3] Download progress: 60% [03:26] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:27] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:27] Frame 21 saved (frame_020.jpg, t=43.6s) [03:27] [capture-periodic] freeze=2ms shot=101ms unfreeze=6ms saved=True [03:28] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:28] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:29] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:29] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:30] Frame 22 saved (frame_021.jpg, t=45.8s) [03:30] [capture-periodic] freeze=2ms shot=132ms unfreeze=4ms saved=True [03:30] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:30] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:31] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:32] [capture-loop] iter=400 t=48.0s state={'complete': False, 'error': None, 'executedCount': 1, 'wsState': 1} eval_ms=3 ram=3.7/7.0GB (60%) swap=0.0/0.0GB (0%) cpu=75% browser_rss=0.46GB [03:32] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:32] [ComfyUI] [worker:ComfyUI-SAM3] [SAM3] Download progress: 70% [03:32] Frame 23 saved (frame_022.jpg, t=48.1s) [03:32] [capture-periodic] freeze=5ms shot=136ms unfreeze=5ms saved=True [03:33] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:33] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:34] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:34] Frame 24 saved (frame_023.jpg, t=50.3s) [03:34] [capture-periodic] freeze=1ms shot=94ms unfreeze=3ms saved=True [03:34] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:35] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:35] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:36] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:36] Frame 25 saved (frame_024.jpg, t=52.5s) [03:36] [capture-periodic] freeze=5ms shot=97ms unfreeze=2ms saved=True [03:37] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:37] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:38] [capture-loop] iter=450 t=53.9s state={'complete': False, 'error': None, 'executedCount': 1, 'wsState': 1} eval_ms=2 ram=3.7/7.0GB (60%) swap=0.0/0.0GB (0%) cpu=78% browser_rss=0.46GB [03:38] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:38] [ComfyUI] [worker:ComfyUI-SAM3] [SAM3] Download progress: 80% [03:38] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:39] Frame 26 saved (frame_025.jpg, t=54.8s) [03:39] [capture-periodic] freeze=4ms shot=122ms unfreeze=3ms saved=True [03:39] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:40] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:40] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:41] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:41] Frame 27 saved (frame_026.jpg, t=57.0s) [03:41] [capture-periodic] freeze=2ms shot=111ms unfreeze=4ms saved=True [03:41] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:42] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:42] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:43] Frame 28 saved (frame_027.jpg, t=59.2s) [03:43] [capture-periodic] freeze=2ms shot=82ms unfreeze=2ms saved=True [03:43] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:44] [capture-loop] iter=500 t=59.9s state={'complete': False, 'error': None, 'executedCount': 1, 'wsState': 1} eval_ms=2 ram=3.7/7.0GB (60%) swap=0.0/0.0GB (0%) cpu=71% browser_rss=0.46GB [03:44] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:44] [ComfyUI] [worker:ComfyUI-SAM3] [SAM3] Download progress: 90% [03:44] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:45] [ComfyUI] [worker:ComfyUI-SAM3] [03:45] [ComfyUI] === WATCHDOG TICK 1 === [03:45] [ComfyUI] [worker:ComfyUI-SAM3] Current thread 0x0000000170cdf000 (most recent call first): [03:45] [ComfyUI] File "/var/folders/tb/y368xp_x10s3ty1b_mtl5mxr0000gn/T/comfyui_pvenv_2em037f9/persistent_worker.py", line 52 in _watchdog [03:45] [ComfyUI] File "/Users/runner/.ce/.pixi/envs/sam3-nodes/lib/python3.13/threading.py", line 995 in run [03:45] [ComfyUI] File "/Users/runner/.ce/.pixi/envs/sam3-nodes/lib/python3.13/threading.py", line 1044 in _bootstrap_inner [03:45] [ComfyUI] File "/Users/runner/.ce/.pixi/envs/sam3-nodes/lib/python3.13/threading.py", line 1015 in _bootstrap [03:45] [ComfyUI] [03:45] [ComfyUI] Thread 0x00000001fe826240 (most recent call first): [03:45] [ComfyUI] File "/Users/runner/.ce/.pixi/envs/sam3-nodes/lib/python3.13/ssl.py", line 1138 in read [03:45] [ComfyUI] File "/Users/runner/.ce/.pixi/envs/sam3-nodes/lib/python3.13/ssl.py", line 1304 in recv_into [03:45] [ComfyUI] File "/Users/runner/.ce/.pixi/envs/sam3-nodes/lib/python3.13/socket.py", line 719 in readinto [03:45] [ComfyUI] File "/Users/runner/.ce/.pixi/envs/sam3-nodes/lib/python3.13/http/client.py", line 484 in read [03:45] [ComfyUI] File "/Users/runner/.ce/.pixi/envs/sam3-nodes/lib/python3.13/site-packages/urllib3/response.py", line 1005 in _fp_read [03:45] [ComfyUI] File "/Users/runner/.ce/.pixi/envs/sam3-nodes/lib/python3.13/site-packages/urllib3/response.py", line 1022 in _raw_read [03:45] [ComfyUI] File "/Users/runner/.ce/.pixi/envs/sam3-nodes/lib/python3.13/site-packages/urllib3/response.py", line 1110 in read [03:45] [ComfyUI] File "/Users/runner/.ce/.pixi/envs/sam3-nodes/lib/python3.13/site-packages/urllib3/response.py", line 1267 in stream [03:45] [ComfyUI] File "/Users/runner/.ce/.pixi/envs/sam3-nodes/lib/python3.13/site-packages/requests/models.py", line 937 in generate [03:45] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/load_model.py", line 128 in _download_from_huggingface [03:45] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/load_model.py", line 73 in execute [03:45] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI/comfy_api/latest/_io.py", line 1833 in EXECUTE_NORMALIZED [03:45] [ComfyUI] File "/var/folders/tb/y368xp_x10s3ty1b_mtl5mxr0000gn/T/comfyui_pvenv_2em037f9/persistent_worker.py", line 1762 in main [03:45] [ComfyUI] File "/var/folders/tb/y368xp_x10s3ty1b_mtl5mxr0000gn/T/comfyui_pvenv_2em037f9/persistent_worker.py", line 1807 in [03:45] [ComfyUI] [03:45] [ComfyUI] [worker:ComfyUI-SAM3] === END === [03:45] [ComfyUI] [03:45] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:45] Frame 29 saved (frame_028.jpg, t=61.4s) [03:45] [capture-periodic] freeze=2ms shot=86ms unfreeze=2ms saved=True [03:45] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:46] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:46] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:47] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:47] Frame 30 saved (frame_029.jpg, t=63.5s) [03:47] [capture-periodic] freeze=3ms shot=105ms unfreeze=10ms saved=True [03:48] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:48] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:49] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:49] [capture-loop] iter=550 t=65.7s state={'complete': False, 'error': None, 'executedCount': 1, 'wsState': 1} eval_ms=3 ram=3.7/7.0GB (60%) swap=0.0/0.0GB (0%) cpu=76% browser_rss=0.46GB [03:49] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:49] [ComfyUI] [worker:ComfyUI-SAM3] [SAM3] Download progress: 100% [03:49] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=2 [03:49] [ComfyUI] [worker:ComfyUI-SAM3] [SAM3] Downloaded sam3.safetensors [03:49] [ComfyUI] [worker:ComfyUI-SAM3] [SAM3] Model downloaded successfully [03:49] [ComfyUI] [SubprocessWorker] got response: ok [03:49] [ComfyUI] [comfy-env] LoadSAM3Model done (60.56s): result: NodeOutput [03:49] [ComfyUI] [comfy-env] Running SAM3Segmentation: refinement_iterations: 0, use_multimask: True, output_best_mask: True, sam3_model_config: dict, image: torch.float32 [1,720,1280,3] cpu, positive_points: dict, negative_points: dict [03:49] [ComfyUI] [SubprocessWorker] call_method: nodes.segmentation.SAM3Segmentation.EXECUTE_NORMALIZED [03:49] [ComfyUI] [SubprocessWorker] acquired lock, ensuring started... [03:49] [ComfyUI] [ComfyUI-SAM3] Health check: ping (timeout=5.0s)... [03:49] [ComfyUI] [ComfyUI-SAM3] Health check: ok [03:49] [ComfyUI] [SubprocessWorker] worker started/confirmed [03:49] [ComfyUI] [comfy-env] PRE-SERIALIZE 'image' shape: torch.Size([1, 720, 1280, 3]) [03:49] [ComfyUI] [SubprocessWorker] serializing kwargs to shm... [03:49] [ComfyUI] [SubprocessWorker] created 0 shm blocks [03:49] [ComfyUI] [SubprocessWorker] sending request via socket... [03:49] [ComfyUI] [SubprocessWorker] call_id=3 sending call_method [03:49] [ComfyUI] [worker:ComfyUI-SAM3] [comfy-env] _from_shm got dict with keys: ['__type__', 'strategy', 'manager_path', 'storage_key', 'storage_size'] [03:49] [ComfyUI] [worker:ComfyUI-SAM3] [comfy-env] DESERIALIZE TensorRef: tensor_size=[1, 720, 1280, 3] [03:49] [ComfyUI] [worker:ComfyUI-SAM3] [comfy-env] DESERIALIZED tensor shape: torch.Size([1, 720, 1280, 3]) [03:50] Frame 31 saved (frame_030.jpg, t=65.8s) [03:50] [capture-periodic] freeze=4ms shot=119ms unfreeze=8ms saved=True [03:51] [ComfyUI] objc[9901]: Class AVFFrameReceiver is implemented in both /Users/runner/.ce/.pixi/envs/sam3-nodes/lib/libavdevice.62.3.101.dylib (0x12aa647a8) and /Users/runner/.ce/.pixi/envs/sam3-nodes/lib/python3.13/site-packages/cv2/.dylibs/libavdevice.61.3.100.dylib (0x14bc1c3a8). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed. [03:51] [ComfyUI] objc[9901]: Class AVFAudioReceiver is implemented in both /Users/runner/.ce/.pixi/envs/sam3-nodes/lib/libavdevice.62.3.101.dylib (0x12aa647f8) and /Users/runner/.ce/.pixi/envs/sam3-nodes/lib/python3.13/site-packages/cv2/.dylibs/libavdevice.61.3.100.dylib (0x14bc1c3f8). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed. [03:55] [capture-loop] iter=600 t=71.5s state={'complete': False, 'error': None, 'executedCount': 1, 'wsState': 1} eval_ms=7 ram=3.7/7.0GB (60%) swap=0.0/0.0GB (0%) cpu=46% browser_rss=0.45GB [04:01] [capture-loop] iter=650 t=77.5s state={'complete': False, 'error': None, 'executedCount': 1, 'wsState': 1} eval_ms=2 ram=3.6/7.0GB (62%) swap=0.0/0.0GB (0%) cpu=54% browser_rss=0.43GB [04:07] [capture-loop] iter=700 t=83.5s state={'complete': False, 'error': None, 'executedCount': 1, 'wsState': 1} eval_ms=4 ram=3.1/7.0GB (69%) swap=0.0/0.0GB (0%) cpu=54% browser_rss=0.30GB [04:13] [capture-loop] iter=750 t=89.8s state={'complete': False, 'error': None, 'executedCount': 1, 'wsState': 1} eval_ms=2 ram=2.8/7.0GB (74%) swap=0.0/0.0GB (0%) cpu=59% browser_rss=0.26GB [04:19] [capture-loop] iter=800 t=95.8s state={'complete': False, 'error': None, 'executedCount': 1, 'wsState': 1} eval_ms=6 ram=2.6/7.0GB (77%) swap=0.0/0.0GB (0%) cpu=36% browser_rss=0.25GB [04:20] [ComfyUI] [SubprocessWorker] callback 'request_vram_budget' from call_id=3 [04:20] [ComfyUI] [comfy-env] VRAM request: 1.84GB, free before eviction: 1.72GB [04:20] [ComfyUI] [comfy-env] VRAM after eviction: 1.72GB [04:26] [capture-loop] iter=850 t=102.3s state={'complete': False, 'error': None, 'executedCount': 1, 'wsState': 1} eval_ms=4 ram=3.9/7.0GB (78%) swap=0.0/0.0GB (0%) cpu=82% browser_rss=0.25GB [04:35] [capture-loop] iter=900 t=111.0s state={'complete': False, 'error': None, 'executedCount': 1, 'wsState': 1} eval_ms=4 ram=3.0/7.0GB (85%) swap=0.0/0.0GB (0%) cpu=61% browser_rss=0.20GB [04:43] [capture-loop] iter=950 t=119.4s state={'complete': False, 'error': None, 'executedCount': 1, 'wsState': 1} eval_ms=6 ram=4.3/7.0GB (84%) swap=0.0/0.0GB (0%) cpu=49% browser_rss=0.23GB [04:45] [ComfyUI] [worker:ComfyUI-SAM3] [04:45] [ComfyUI] === WATCHDOG TICK 2 === [04:45] [ComfyUI] [worker:ComfyUI-SAM3] Current thread 0x0000000170cdf000 (most recent call first): [04:45] [ComfyUI] File "/var/folders/tb/y368xp_x10s3ty1b_mtl5mxr0000gn/T/comfyui_pvenv_2em037f9/persistent_worker.py", line 52 in _watchdog [04:45] [ComfyUI] File "/Users/runner/.ce/.pixi/envs/sam3-nodes/lib/python3.13/threading.py", line 995 in run [04:45] [ComfyUI] File "/Users/runner/.ce/.pixi/envs/sam3-nodes/lib/python3.13/threading.py", line 1044 in _bootstrap_inner [04:45] [ComfyUI] File "/Users/runner/.ce/.pixi/envs/sam3-nodes/lib/python3.13/threading.py", line 1015 in _bootstrap [04:45] [ComfyUI] [04:45] [ComfyUI] Thread 0x00000001fe826240 (most recent call first): [04:45] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/model.py", line 424 in _pe_encoding [04:45] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/model.py", line 445 in forward_with_coords [04:45] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/model.py", line 2599 in _embed_points [04:45] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/model.py", line 2686 in forward [04:45] [ComfyUI] File "/Users/runner/.ce/.pixi/envs/sam3-nodes/lib/python3.13/site-packages/torch/nn/modules/module.py", line 1787 in _call_impl [04:45] [ComfyUI] File "/Users/runner/.ce/.pixi/envs/sam3-nodes/lib/python3.13/site-packages/torch/nn/modules/module.py", line 1776 in _wrapped_call_impl [04:45] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/model.py", line 9587 in _predict [04:45] [ComfyUI] File "/Users/runner/.ce/.pixi/envs/sam3-nodes/lib/python3.13/site-packages/torch/utils/_contextlib.py", line 124 in decorate_context [04:45] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/model.py", line 9499 in predict [04:45] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/model.py", line 3989 in predict_inst [04:45] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/segmentation.py", line 556 in execute [04:45] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI/comfy_api/latest/_io.py", line 1833 in EXECUTE_NORMALIZED [04:45] [ComfyUI] File "/var/folders/tb/y368xp_x10s3ty1b_mtl5mxr0000gn/T/comfyui_pvenv_2em037f9/persistent_worker.py", line 1762 in main [04:45] [ComfyUI] File "/var/folders/tb/y368xp_x10s3ty1b_mtl5mxr0000gn/T/comfyui_pvenv_2em037f9/persistent_worker.py", line 1807 in [04:45] [ComfyUI] [04:45] [ComfyUI] [worker:ComfyUI-SAM3] === END === [04:45] [ComfyUI] [04:49] [capture-loop] iter=1000 t=125.3s state={'complete': False, 'error': None, 'executedCount': 1, 'wsState': 1} eval_ms=2 ram=3.8/7.0GB (76%) swap=0.0/0.0GB (0%) cpu=61% browser_rss=0.23GB [04:52] [ComfyUI] [SubprocessWorker] got response: ok [04:52] [ComfyUI] [comfy-env] SAM3Segmentation done (62.45s): result: NodeOutput [04:52] [ComfyUI] Prompt executed in 128.13 seconds [04:52] Node executed (3 total), capturing... [04:52] Frame 32 saved (frame_031.jpg, t=128.6s) [04:52] [capture-node] freeze=3ms shot=118ms unfreeze=0ms saved=True [04:52] Execution complete (t=128.6s) [04:52] Captured 32 unique frames over 128.85s [04:53] [debug] iframes: 0 [04:53] [timing] trigger_3d_previews: 0.0s [04:55] Saved high-quality screenshot: image_seg_point_executed.png [04:55] [timing] final_screenshot: 0.5s [04:55] Capturing execution frames: image_seg_text.json [05:02] Frame 1 saved (frame_000.jpg, t=0.0s) [05:02] Validating workflow... [05:02] [ComfyUI] === /validate endpoint called === [05:02] [ComfyUI] Received JSON with keys: ['prompt'] [05:02] [ComfyUI] Prompt has 5 nodes [05:02] [ComfyUI] Node 1: LoadImage [05:02] [ComfyUI] Inputs: {"image": "example_image.jpg"} [05:02] [ComfyUI] Node 3: PreviewImage [05:02] [ComfyUI] Inputs: {"images": ["15", 1]} [05:02] [ComfyUI] Node 4: MaskPreview [05:02] [ComfyUI] Inputs: {"mask": ["15", 0]} [05:02] [ComfyUI] Node 12: LoadSAM3Model [05:02] [ComfyUI] Inputs: {"precision": "fp32", "compile": false} [05:02] [ComfyUI] Node 15: SAM3Grounding [05:02] [ComfyUI] Inputs: {"confidence_threshold": 0.2, "text_prompt": "child", "max_detections": -1, "sam3_model_config": ["12", 0], "image": ["1", 0]} [05:02] [ComfyUI] Generated prompt_id: c77bb368-75ec-4ea1-a0b3-bbb062d97845 [05:02] [ComfyUI] Calling execution.validate_prompt()... [05:02] [ComfyUI] Validation result: valid=True [05:02] [ComfyUI] === Validation PASSED === [05:02] Queuing workflow for execution... [05:02] [ComfyUI] got prompt [05:02] [ComfyUI] [comfy-env] Running LoadSAM3Model: precision: 'fp32', compile: False [05:02] [ComfyUI] [SubprocessWorker] call_method: nodes.load_model.LoadSAM3Model.EXECUTE_NORMALIZED [05:02] [ComfyUI] [SubprocessWorker] acquired lock, ensuring started... [05:02] [ComfyUI] [ComfyUI-SAM3] Health check: ping (timeout=5.0s)... [05:02] [ComfyUI] [ComfyUI-SAM3] Health check: ok [05:02] [ComfyUI] [SubprocessWorker] worker started/confirmed [05:02] [ComfyUI] [SubprocessWorker] serializing kwargs to shm... [05:02] [ComfyUI] [SubprocessWorker] created 0 shm blocks [05:02] [ComfyUI] [SubprocessWorker] sending request via socket... [05:02] [ComfyUI] [SubprocessWorker] call_id=4 sending call_method [05:02] [ComfyUI] [SubprocessWorker] got response: ok [05:02] [ComfyUI] [comfy-env] LoadSAM3Model done (0.03s): result: NodeOutput [05:02] [ComfyUI] [comfy-env] Running SAM3Grounding: confidence_threshold: 0.2, text_prompt: 'child', max_detections: -1, sam3_model_config: dict, image: torch.float32 [1,720,1280,3] cpu [05:02] [ComfyUI] [SubprocessWorker] call_method: nodes.segmentation.SAM3Grounding.EXECUTE_NORMALIZED [05:02] [ComfyUI] [SubprocessWorker] acquired lock, ensuring started... [05:02] [ComfyUI] [ComfyUI-SAM3] Health check: ping (timeout=5.0s)... [05:02] [ComfyUI] [ComfyUI-SAM3] Health check: ok [05:02] [ComfyUI] [SubprocessWorker] worker started/confirmed [05:02] [ComfyUI] [comfy-env] PRE-SERIALIZE 'image' shape: torch.Size([1, 720, 1280, 3]) [05:02] [ComfyUI] [SubprocessWorker] serializing kwargs to shm... [05:02] [ComfyUI] [SubprocessWorker] created 0 shm blocks [05:02] [ComfyUI] [SubprocessWorker] sending request via socket... [05:02] [ComfyUI] [SubprocessWorker] call_id=5 sending call_method [05:02] [ComfyUI] [worker:ComfyUI-SAM3] [comfy-env] _from_shm got dict with keys: ['__type__', 'strategy', 'manager_path', 'storage_key', 'storage_size'] [05:02] [ComfyUI] [worker:ComfyUI-SAM3] [comfy-env] DESERIALIZE TensorRef: tensor_size=[1, 720, 1280, 3] [05:02] [ComfyUI] [worker:ComfyUI-SAM3] [comfy-env] DESERIALIZED tensor shape: torch.Size([1, 720, 1280, 3]) [05:03] [ComfyUI] [SubprocessWorker] callback 'request_vram_budget' from call_id=5 [05:03] [ComfyUI] [comfy-env] VRAM request: 3.50GB, free before eviction: 1.42GB [05:03] [ComfyUI] [comfy-env] VRAM after eviction: 1.42GB [05:04] Frame 2 saved (frame_001.jpg, t=2.2s) [05:04] [capture-periodic] freeze=2ms shot=236ms unfreeze=4ms saved=True [05:11] Frame 3 saved (frame_002.jpg, t=4.9s) [05:11] [capture-periodic] freeze=20ms shot=4880ms unfreeze=37ms saved=True [05:13] [capture-loop] iter=50 t=11.8s state={'complete': False, 'error': None, 'executedCount': 0, 'wsState': 1} eval_ms=4 ram=3.7/7.0GB (86%) swap=0.3/1.0GB (33%) cpu=66% browser_rss=0.21GB [05:14] Frame 4 saved (frame_003.jpg, t=11.9s) [05:14] [capture-periodic] freeze=3ms shot=352ms unfreeze=28ms saved=True [05:31] [capture-loop] iter=100 t=28.9s state={'complete': False, 'error': None, 'executedCount': 0, 'wsState': 1} eval_ms=351 ram=1.7/7.0GB (91%) swap=2.6/3.0GB (86%) cpu=60% browser_rss=0.28GB [05:39] [capture-loop] iter=150 t=37.1s state={'complete': False, 'error': None, 'executedCount': 0, 'wsState': 1} eval_ms=4 ram=4.3/7.0GB (73%) swap=2.0/4.0GB (50%) cpu=59% browser_rss=0.28GB [05:45] [ComfyUI] [worker:ComfyUI-SAM3] [05:45] [ComfyUI] === WATCHDOG TICK 3 === [05:45] [ComfyUI] [worker:ComfyUI-SAM3] Current thread 0x0000000170cdf000 (most recent call first): [05:45] [ComfyUI] File "/var/folders/tb/y368xp_x10s3ty1b_mtl5mxr0000gn/T/comfyui_pvenv_2em037f9/persistent_worker.py", line 52 in _watchdog [05:45] [ComfyUI] File "/Users/runner/.ce/.pixi/envs/sam3-nodes/lib/python3.13/threading.py", line 995 in run [05:45] [ComfyUI] File "/Users/runner/.ce/.pixi/envs/sam3-nodes/lib/python3.13/threading.py", line 1044 in _bootstrap_inner [05:45] [ComfyUI] File "/Users/runner/.ce/.pixi/envs/sam3-nodes/lib/python3.13/threading.py", line 1015 in _bootstrap [05:45] [ComfyUI] [05:45] [ComfyUI] Thread 0x00000001fe826240 (most recent call first): [05:45] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI/comfy/ldm/modules/sub_quadratic_attention.py", line 181 in _get_attention_scores_no_kv_chunking [05:45] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI/comfy/ldm/modules/sub_quadratic_attention.py", line 269 in efficient_dot_product_attention [05:45] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI/comfy/ldm/modules/attention.py", line 275 in attention_sub_quad [05:45] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI/comfy/ldm/modules/attention.py", line 139 in wrapper [05:45] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/attention.py", line 120 in sam3_attention [05:45] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/model.py", line 729 in forward [05:45] [ComfyUI] File "/Users/runner/.ce/.pixi/envs/sam3-nodes/lib/python3.13/site-packages/torch/nn/modules/module.py", line 1787 in _call_impl [05:45] [ComfyUI] File "/Users/runner/.ce/.pixi/envs/sam3-nodes/lib/python3.13/site-packages/torch/nn/modules/module.py", line 1776 in _wrapped_call_impl [05:45] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/model.py", line 819 in forward [05:45] [ComfyUI] File "/Users/runner/.ce/.pixi/envs/sam3-nodes/lib/python3.13/site-packages/torch/nn/modules/module.py", line 1787 in _call_impl [05:45] [ComfyUI] File "/Users/runner/.ce/.pixi/envs/sam3-nodes/lib/python3.13/site-packages/torch/nn/modules/module.py", line 1776 in _wrapped_call_impl [05:45] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/model.py", line 983 in forward [05:45] [ComfyUI] File "/Users/runner/.ce/.pixi/envs/sam3-nodes/lib/python3.13/site-packages/torch/nn/modules/module.py", line 1787 in _call_impl [05:45] [ComfyUI] File "/Users/runner/.ce/.pixi/envs/sam3-nodes/lib/python3.13/site-packages/torch/nn/modules/module.py", line 1776 in _wrapped_call_impl [05:45] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/model.py", line 2041 in forward [05:45] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/model.py", line 3321 in _forward_image_no_act_ckpt [05:45] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/model.py", line 3314 in forward_image [05:45] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/utils.py", line 1324 in set_image [05:45] [ComfyUI] File "/Users/runner/.ce/.pixi/envs/sam3-nodes/lib/python3.13/site-packages/torch/utils/_contextlib.py", line 124 in decorate_context [05:45] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/segmentation.py", line 133 in _segment_grounding [05:45] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/segmentation.py", line 111 in execute [05:45] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI/comfy_api/latest/_io.py", line 1833 in EXECUTE_NORMALIZED [05:45] [ComfyUI] File "/var/folders/tb/y368xp_x10s3ty1b_mtl5mxr0000gn/T/comfyui_pvenv_2em037f9/persistent_worker.py", line 1762 in main [05:45] [ComfyUI] File "/var/folders/tb/y368xp_x10s3ty1b_mtl5mxr0000gn/T/comfyui_pvenv_2em037f9/persistent_worker.py", line 1807 in [05:45] [ComfyUI] [05:45] [ComfyUI] [worker:ComfyUI-SAM3] === END === [05:45] [ComfyUI] [05:56] [capture-loop] iter=200 t=54.2s state={'complete': False, 'error': None, 'executedCount': 0, 'wsState': 1} eval_ms=9 ram=5.6/7.0GB (92%) swap=1.3/2.0GB (63%) cpu=50% browser_rss=0.15GB [06:05] Frame 5 saved (frame_004.jpg, t=58.2s) [06:05] [capture-periodic] freeze=3ms shot=5322ms unfreeze=45ms saved=True [06:14] Frame 6 saved (frame_005.jpg, t=65.8s) [06:14] [capture-periodic] freeze=1ms shot=6255ms unfreeze=6ms saved=True [06:15] [capture-loop] iter=250 t=73.2s state={'complete': False, 'error': None, 'executedCount': 0, 'wsState': 1} eval_ms=2 ram=3.9/7.0GB (87%) swap=1.2/2.0GB (60%) cpu=48% browser_rss=0.18GB [06:20] Frame 7 saved (frame_006.jpg, t=74.2s) [06:20] [capture-periodic] freeze=2ms shot=4245ms unfreeze=38ms saved=True [06:20] [ComfyUI] /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/utils.py:1824: UserWarning: The operator 'aten::_assert_async' is not currently supported on the MPS backend and will fall back to run on the CPU. This may have performance implications. (Triggered internally at /Users/runner/work/pytorch/pytorch/pytorch/aten/src/ATen/mps/MPSFallback.mm:15.) [06:20] [ComfyUI] torch._assert_async(is_right_padded(mask1)) [06:22] Frame 8 saved (frame_007.jpg, t=80.6s) [06:22] [capture-periodic] freeze=4ms shot=115ms unfreeze=2ms saved=True [06:25] [capture-loop] iter=300 t=83.3s state={'complete': False, 'error': None, 'executedCount': 0, 'wsState': 1} eval_ms=4 ram=2.2/7.0GB (82%) swap=1.6/2.0GB (79%) cpu=50% browser_rss=0.19GB [06:31] [capture-loop] iter=350 t=89.3s state={'complete': False, 'error': None, 'executedCount': 0, 'wsState': 1} eval_ms=2 ram=3.9/7.0GB (88%) swap=2.0/3.0GB (67%) cpu=36% browser_rss=0.19GB [06:37] [capture-loop] iter=400 t=95.0s state={'complete': False, 'error': None, 'executedCount': 0, 'wsState': 1} eval_ms=2 ram=3.4/7.0GB (82%) swap=2.0/3.0GB (65%) cpu=41% browser_rss=0.19GB [06:43] [capture-loop] iter=450 t=101.0s state={'complete': False, 'error': None, 'executedCount': 0, 'wsState': 1} eval_ms=2 ram=3.8/7.0GB (88%) swap=2.1/3.0GB (71%) cpu=53% browser_rss=0.20GB [06:44] [ComfyUI] [SubprocessWorker] got response: ok [06:44] [ComfyUI] [comfy-env] SAM3Grounding done (102.24s): result: NodeOutput [06:45] [ComfyUI] Prompt executed in 103.06 seconds [06:45] Frame 9 saved (frame_008.jpg, t=102.9s) [06:45] [capture-periodic] freeze=2ms shot=238ms unfreeze=54ms saved=True [06:45] Node executed (2 total), capturing... [06:45] Frame 10 saved (frame_009.jpg, t=103.5s) [06:45] [capture-node] freeze=2ms shot=122ms unfreeze=0ms saved=True [06:45] Execution complete (t=103.5s) [06:45] Captured 10 unique frames over 103.67s [06:45] [debug] iframes: 0 [06:45] [timing] trigger_3d_previews: 0.0s [06:47] Saved high-quality screenshot: image_seg_text_executed.png [06:48] [timing] final_screenshot: 0.2s [06:48] Capturing execution frames: scene_seg.json [06:54] Frame 1 saved (frame_000.jpg, t=0.0s) [06:54] Validating workflow... [06:54] [ComfyUI] === /validate endpoint called === [06:54] [ComfyUI] Received JSON with keys: ['prompt'] [06:54] [ComfyUI] Prompt has 10 nodes [06:54] [ComfyUI] Node 1: LoadImage [06:54] [ComfyUI] Inputs: {"image": "image.png"} [06:54] [ComfyUI] Node 3: PreviewImage [06:54] [ComfyUI] Inputs: {"images": ["27", 1]} [06:54] [ComfyUI] Node 4: MaskPreview [06:54] [ComfyUI] Inputs: {"mask": ["27", 0]} [06:54] [ComfyUI] Node 12: LoadSAM3Model [06:54] [ComfyUI] Inputs: {"precision": "fp32", "compile": false} [06:54] [ComfyUI] Node 20: SAM3MultiRegionCollector [06:54] [ComfyUI] Inputs: {"multi_prompts_store": "[{\"positive_points\":[{\"x\":5052.74953932266,\"y\":2847.0600693869637},{\"x\":4932.510591392486,\"y\":2953.9413227753366},{\"x\":4718.752461738845,\"y\":2967.301479448883},{ [06:54] [ComfyUI] Node 22: MultibandFromBatch [06:54] [ComfyUI] Inputs: {"images": ["1", 0], "masks": ["27", 0]} [06:54] [ComfyUI] Node 23: MultibandSave [06:54] [ComfyUI] Inputs: {"file_path": "output/multiband", "format": "npz", "multiband": ["26", 0]} [06:54] [ComfyUI] Node 24: MultibandPreview [06:54] [ComfyUI] Inputs: {"channel_index": 0, "multiband": ["26", 0]} [06:54] [ComfyUI] Node 26: MultibandResize [06:54] [ComfyUI] Inputs: {"upscale_method": "nearest-exact", "width": 900, "height": 600, "crop": "center", "multiband": ["22", 0]} [06:54] [ComfyUI] Node 27: SAM3MultipromptSegmentation [06:54] [ComfyUI] Inputs: {"refinement_iterations": 0, "use_multimask": false, "sam3_model_config": ["12", 0], "image": ["1", 0], "multi_prompts": ["20", 0]} [06:54] [ComfyUI] Generated prompt_id: 26973929-2148-46b6-9776-bf939a877dc1 [06:54] [ComfyUI] Calling execution.validate_prompt()... [06:54] [ComfyUI] Validation result: valid=True [06:54] [ComfyUI] === Validation PASSED === [06:54] Queuing workflow for execution... [06:54] [ComfyUI] got prompt [06:57] Frame 2 saved (frame_001.jpg, t=2.3s) [06:57] [capture-periodic] freeze=4ms shot=528ms unfreeze=18ms saved=True [06:59] [ComfyUI] [comfy-env] Running SAM3MultiRegionCollector: multi_prompts_store: '[{"positive_points":[{"x":5052.74953932266,"y":2847.0600..., image: torch.float32 [1,4480,6720,3] cpu [06:59] [ComfyUI] [SubprocessWorker] call_method: nodes.sam3_interactive.SAM3MultiRegionCollector.EXECUTE_NORMALIZED [06:59] [ComfyUI] [SubprocessWorker] acquired lock, ensuring started... [06:59] [ComfyUI] [ComfyUI-SAM3] Health check: ping (timeout=5.0s)... [06:59] [ComfyUI] [ComfyUI-SAM3] Health check: failed [06:59] [ComfyUI] [ComfyUI-SAM3] Socket unhealthy, restarting worker... [06:59] [ComfyUI] [ComfyUI-SAM3] python: /Users/runner/.ce/.pixi/envs/sam3-nodes/bin/python [06:59] [ComfyUI] [ComfyUI-SAM3] sys_path sent to worker: ['/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI', '/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI/custom_nodes/ComfyUI-SAM3', '/Users/runner/.ce/.pixi/envs/sam3-nodes/lib/python3.13/site-packages', '/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI/custom_nodes/ComfyUI-SAM3'] [06:59] [ComfyUI] [SubprocessWorker] is_pixi=True, python=/Users/runner/.ce/.pixi/envs/sam3-nodes/bin/python [06:59] [ComfyUI] [SubprocessWorker] launching cmd=['/Users/runner/.pixi/bin/pixi', 'run', '--as-is', '--manifest-path', '/Users/runner/.ce/pixi.toml', '-e', 'sam3-nodes', 'python', '/var/folders/tb/y368xp_x10s3ty1b_mtl5mxr0000gn/T/comfyui_pvenv_2em037f9/persistent_worker.py', 'unix:///var/folders/tb/y368xp_x10s3ty1b_mtl5mxr0000gn/T/comfy_worker_59e502287922.sock']... [06:59] [ComfyUI] [SubprocessWorker] socket_addr=unix:///var/folders/tb/y368xp_x10s3ty1b_mtl5mxr0000gn/T/comfy_worker_59e502287922.sock exists=True [07:00] Frame 3 saved (frame_002.jpg, t=5.3s) [07:00] [capture-periodic] freeze=27ms shot=424ms unfreeze=7ms saved=True [07:02] [capture-loop] iter=50 t=7.7s state={'complete': False, 'error': None, 'executedCount': 0, 'wsState': 1} eval_ms=3 ram=2.2/7.0GB (82%) swap=1.9/3.0GB (64%) cpu=55% browser_rss=0.48GB [07:04] Frame 4 saved (frame_003.jpg, t=7.8s) [07:04] [capture-periodic] freeze=7ms shot=1992ms unfreeze=7ms saved=True [07:06] [ComfyUI] [SubprocessWorker] worker started/confirmed [07:06] [ComfyUI] [comfy-env] PRE-SERIALIZE 'image' shape: torch.Size([1, 4480, 6720, 3]) [07:06] [ComfyUI] [SubprocessWorker] serializing kwargs to shm... [07:06] [ComfyUI] [SubprocessWorker] created 0 shm blocks [07:06] [ComfyUI] [SubprocessWorker] sending request via socket... [07:06] [ComfyUI] [SubprocessWorker] call_id=6 sending call_method [07:06] [ComfyUI] [worker:ComfyUI-SAM3] [comfy-env] _from_shm got dict with keys: ['__type__', 'strategy', 'manager_path', 'storage_key', 'storage_size'] [07:06] [ComfyUI] [worker:ComfyUI-SAM3] [comfy-env] DESERIALIZE TensorRef: tensor_size=[1, 4480, 6720, 3] [07:06] [ComfyUI] [worker:ComfyUI-SAM3] [comfy-env] DESERIALIZED tensor shape: torch.Size([1, 4480, 6720, 3]) [07:08] [ComfyUI] [SubprocessWorker] got response: ok [07:08] [ComfyUI] [comfy-env] SAM3MultiRegionCollector done (8.67s): result: NodeOutput [07:08] [ComfyUI] [comfy-env] Running LoadSAM3Model: precision: 'fp32', compile: False [07:08] [ComfyUI] [SubprocessWorker] call_method: nodes.load_model.LoadSAM3Model.EXECUTE_NORMALIZED [07:08] [ComfyUI] [SubprocessWorker] acquired lock, ensuring started... [07:08] [ComfyUI] [ComfyUI-SAM3] Health check: ping (timeout=5.0s)... [07:08] [ComfyUI] [ComfyUI-SAM3] Health check: ok [07:08] [ComfyUI] [SubprocessWorker] worker started/confirmed [07:08] [ComfyUI] [SubprocessWorker] serializing kwargs to shm... [07:08] [ComfyUI] [SubprocessWorker] created 0 shm blocks [07:08] [ComfyUI] [SubprocessWorker] sending request via socket... [07:08] [ComfyUI] [SubprocessWorker] call_id=7 sending call_method [07:08] [ComfyUI] [SubprocessWorker] got response: ok [07:08] [ComfyUI] [comfy-env] LoadSAM3Model done (0.03s): result: NodeOutput [07:08] [ComfyUI] [comfy-env] Running SAM3MultipromptSegmentation: refinement_iterations: 0, use_multimask: False, sam3_model_config: dict, image: torch.float32 [1,4480,6720,3] cpu, multi_prompts: list[7] [07:08] [ComfyUI] [SubprocessWorker] call_method: nodes.segmentation.SAM3MultipromptSegmentation.EXECUTE_NORMALIZED [07:08] [ComfyUI] [SubprocessWorker] acquired lock, ensuring started... [07:08] [ComfyUI] [ComfyUI-SAM3] Health check: ping (timeout=5.0s)... [07:08] [ComfyUI] [ComfyUI-SAM3] Health check: ok [07:08] [ComfyUI] [SubprocessWorker] worker started/confirmed [07:08] [ComfyUI] [comfy-env] PRE-SERIALIZE 'image' shape: torch.Size([1, 4480, 6720, 3]) [07:08] [ComfyUI] [SubprocessWorker] serializing kwargs to shm... [07:08] [ComfyUI] [SubprocessWorker] created 0 shm blocks [07:08] [ComfyUI] [SubprocessWorker] sending request via socket... [07:08] [ComfyUI] [SubprocessWorker] call_id=8 sending call_method [07:08] [ComfyUI] [worker:ComfyUI-SAM3] [comfy-env] _from_shm got dict with keys: ['__type__', 'strategy', 'manager_path', 'storage_key', 'storage_size'] [07:08] [ComfyUI] [worker:ComfyUI-SAM3] [comfy-env] DESERIALIZE TensorRef: tensor_size=[1, 4480, 6720, 3] [07:08] [ComfyUI] [worker:ComfyUI-SAM3] [comfy-env] DESERIALIZED tensor shape: torch.Size([1, 4480, 6720, 3]) [07:09] Node executed (1 total), capturing... [07:09] Frame 5 saved (frame_004.jpg, t=15.1s) [07:09] [capture-node] freeze=245ms shot=140ms unfreeze=4ms saved=True [07:10] [ComfyUI] objc[16658]: Class AVFFrameReceiver is implemented in both /Users/runner/.ce/.pixi/envs/sam3-nodes/lib/libavdevice.62.3.101.dylib (0x1298cc7a8) and /Users/runner/.ce/.pixi/envs/sam3-nodes/lib/python3.13/site-packages/cv2/.dylibs/libavdevice.61.3.100.dylib (0x16c6883a8). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed. [07:10] [ComfyUI] objc[16658]: Class AVFAudioReceiver is implemented in both /Users/runner/.ce/.pixi/envs/sam3-nodes/lib/libavdevice.62.3.101.dylib (0x1298cc7f8) and /Users/runner/.ce/.pixi/envs/sam3-nodes/lib/python3.13/site-packages/cv2/.dylibs/libavdevice.61.3.100.dylib (0x16c6883f8). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed. [07:11] [capture-loop] iter=100 t=16.7s state={'complete': False, 'error': None, 'executedCount': 1, 'wsState': 1} eval_ms=3 ram=2.8/7.0GB (45%) swap=0.6/2.0GB (30%) cpu=62% browser_rss=0.73GB [07:13] [ComfyUI] [SubprocessWorker] callback 'request_vram_budget' from call_id=8 [07:13] [ComfyUI] [comfy-env] VRAM request: 3.50GB, free before eviction: 4.06GB [07:13] [ComfyUI] [comfy-env] VRAM after eviction: 4.06GB [07:25] [capture-loop] iter=150 t=31.5s state={'complete': False, 'error': None, 'executedCount': 1, 'wsState': 1} eval_ms=4 ram=3.9/7.0GB (84%) swap=0.6/2.0GB (29%) cpu=60% browser_rss=0.35GB [07:47] [capture-loop] iter=200 t=53.1s state={'complete': False, 'error': None, 'executedCount': 1, 'wsState': 1} eval_ms=21 ram=2.8/7.0GB (92%) swap=2.8/4.0GB (70%) cpu=60% browser_rss=0.18GB [07:57] [capture-loop] iter=250 t=63.0s state={'complete': False, 'error': None, 'executedCount': 1, 'wsState': 1} eval_ms=3 ram=2.4/7.0GB (55%) swap=3.4/5.0GB (68%) cpu=59% browser_rss=0.19GB [08:01] [ComfyUI] [worker:ComfyUI-SAM3] [08:01] [ComfyUI] === WATCHDOG TICK 1 === [08:01] [ComfyUI] [worker:ComfyUI-SAM3] Current thread 0x000000016e80b000 (most recent call first): [08:01] [ComfyUI] File "/var/folders/tb/y368xp_x10s3ty1b_mtl5mxr0000gn/T/comfyui_pvenv_2em037f9/persistent_worker.py", line 52 in _watchdog [08:01] [ComfyUI] File "/Users/runner/.ce/.pixi/envs/sam3-nodes/lib/python3.13/threading.py", line 995 in run [08:01] [ComfyUI] File "/Users/runner/.ce/.pixi/envs/sam3-nodes/lib/python3.13/threading.py", line 1044 in _bootstrap_inner [08:01] [ComfyUI] File "/Users/runner/.ce/.pixi/envs/sam3-nodes/lib/python3.13/threading.py", line 1015 in _bootstrap [08:01] [ComfyUI] [08:01] [ComfyUI] Thread 0x00000001fe826240 (most recent call first): [08:01] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI/comfy/ldm/modules/sub_quadratic_attention.py", line 170 in _get_attention_scores_no_kv_chunking [08:01] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI/comfy/ldm/modules/sub_quadratic_attention.py", line 259 in efficient_dot_product_attention [08:01] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI/comfy/ldm/modules/attention.py", line 275 in attention_sub_quad [08:01] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI/comfy/ldm/modules/attention.py", line 139 in wrapper [08:01] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/attention.py", line 120 in sam3_attention [08:01] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/model.py", line 729 in forward [08:01] [ComfyUI] File "/Users/runner/.ce/.pixi/envs/sam3-nodes/lib/python3.13/site-packages/torch/nn/modules/module.py", line 1787 in _call_impl [08:01] [ComfyUI] File "/Users/runner/.ce/.pixi/envs/sam3-nodes/lib/python3.13/site-packages/torch/nn/modules/module.py", line 1776 in _wrapped_call_impl [08:01] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/model.py", line 819 in forward [08:01] [ComfyUI] File "/Users/runner/.ce/.pixi/envs/sam3-nodes/lib/python3.13/site-packages/torch/nn/modules/module.py", line 1787 in _call_impl [08:01] [ComfyUI] File "/Users/runner/.ce/.pixi/envs/sam3-nodes/lib/python3.13/site-packages/torch/nn/modules/module.py", line 1776 in _wrapped_call_impl [08:01] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/model.py", line 983 in forward [08:01] [ComfyUI] File "/Users/runner/.ce/.pixi/envs/sam3-nodes/lib/python3.13/site-packages/torch/nn/modules/module.py", line 1787 in _call_impl [08:01] [ComfyUI] File "/Users/runner/.ce/.pixi/envs/sam3-nodes/lib/python3.13/site-packages/torch/nn/modules/module.py", line 1776 in _wrapped_call_impl [08:01] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/model.py", line 2041 in forward [08:01] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/model.py", line 3321 in _forward_image_no_act_ckpt [08:01] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/model.py", line 3314 in forward_image [08:01] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/utils.py", line 1324 in set_image [08:01] [ComfyUI] File "/Users/runner/.ce/.pixi/envs/sam3-nodes/lib/python3.13/site-packages/torch/utils/_contextlib.py", line 124 in decorate_context [08:01] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/segmentation.py", line 733 in execute [08:01] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI/comfy_api/latest/_io.py", line 1833 in EXECUTE_NORMALIZED [08:01] [ComfyUI] File "/var/folders/tb/y368xp_x10s3ty1b_mtl5mxr0000gn/T/comfyui_pvenv_2em037f9/persistent_worker.py", line 1762 in main [08:01] [ComfyUI] File "/var/folders/tb/y368xp_x10s3ty1b_mtl5mxr0000gn/T/comfyui_pvenv_2em037f9/persistent_worker.py", line 1807 in [08:01] [ComfyUI] [08:01] [ComfyUI] [worker:ComfyUI-SAM3] === END === [08:01] [ComfyUI] [08:04] [capture-loop] iter=300 t=70.1s state={'complete': False, 'error': None, 'executedCount': 1, 'wsState': 1} eval_ms=2 ram=4.4/7.0GB (73%) swap=1.6/3.0GB (53%) cpu=45% browser_rss=0.19GB [08:22] [capture-loop] iter=350 t=87.7s state={'complete': False, 'error': None, 'executedCount': 1, 'wsState': 1} eval_ms=28 ram=3.3/7.0GB (82%) swap=1.0/2.0GB (48%) cpu=50% browser_rss=0.16GB [08:29] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=8 [08:31] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=8 [08:31] [capture-loop] iter=400 t=97.1s state={'complete': False, 'error': None, 'executedCount': 1, 'wsState': 1} eval_ms=8 ram=3.1/7.0GB (88%) swap=1.4/2.0GB (69%) cpu=58% browser_rss=0.21GB [08:31] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=8 [08:32] Frame 6 saved (frame_005.jpg, t=97.3s) [08:32] [capture-periodic] freeze=4ms shot=374ms unfreeze=15ms saved=True [08:33] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=8 [08:33] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=8 [08:34] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=8 [08:34] Frame 7 saved (frame_006.jpg, t=99.9s) [08:34] [capture-periodic] freeze=5ms shot=565ms unfreeze=9ms saved=True [08:35] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=8 [08:38] Frame 8 saved (frame_007.jpg, t=102.6s) [08:38] [capture-periodic] freeze=22ms shot=1472ms unfreeze=26ms saved=True [08:41] [capture-loop] iter=450 t=107.2s state={'complete': False, 'error': None, 'executedCount': 1, 'wsState': 1} eval_ms=2 ram=2.9/7.0GB (85%) swap=2.6/3.0GB (87%) cpu=76% browser_rss=0.19GB [08:54] Frame 9 saved (frame_008.jpg, t=114.2s) [08:54] [capture-periodic] freeze=5ms shot=5821ms unfreeze=24ms saved=True [08:56] Frame 10 saved (frame_009.jpg, t=122.1s) [08:56] [capture-periodic] freeze=4ms shot=203ms unfreeze=6ms saved=True [08:57] [ComfyUI] [SubprocessWorker] got response: ok [08:57] [ComfyUI] [comfy-env] SAM3MultipromptSegmentation done (108.69s): result: NodeOutput [08:57] [capture-loop] iter=500 t=123.0s state={'complete': False, 'error': None, 'executedCount': 1, 'wsState': 1} eval_ms=6 ram=2.6/7.0GB (54%) swap=3.7/5.0GB (73%) cpu=61% browser_rss=0.15GB [08:58] Frame 11 saved (frame_010.jpg, t=124.5s) [08:59] [capture-periodic] freeze=4ms shot=153ms unfreeze=8ms saved=True [08:59] Node executed (2 total), capturing... [08:59] Frame 12 saved (frame_011.jpg, t=125.0s) [08:59] [capture-node] freeze=6ms shot=584ms unfreeze=3ms saved=True [09:04] [capture-loop] iter=550 t=130.0s state={'complete': False, 'error': None, 'executedCount': 2, 'wsState': 1} eval_ms=5 ram=3.5/7.0GB (66%) swap=3.6/5.0GB (71%) cpu=61% browser_rss=0.39GB [09:06] [ComfyUI] BatchToMultiband: Processed 1 images with 3 channels each -> 3 channels [09:06] [ComfyUI] BatchToMultiband: Processed 7 masks -> 7 channels [09:07] Node executed (3 total), capturing... [09:07] Frame 13 saved (frame_012.jpg, t=132.9s) [09:07] [capture-node] freeze=3ms shot=575ms unfreeze=3ms saved=True [09:09] [ComfyUI] BatchToMultiband: Output shape torch.Size([1, 10, 4480, 6720]), channels: ['img_01_r', 'img_01_g', 'img_01_b', 'mask_01', 'mask_02', 'mask_03', 'mask_04', 'mask_05', 'mask_06', 'mask_07'] [09:09] [ComfyUI] ResizeMultiband: 4480x6720 -> 600x900 [09:10] Frame 14 saved (frame_013.jpg, t=135.6s) [09:10] [capture-periodic] freeze=9ms shot=217ms unfreeze=4ms saved=True [09:10] [ComfyUI] SaveMultibandImage: Saved to output/multiband.npz [09:10] [ComfyUI] Shape: (1, 10, 600, 900) [09:10] [ComfyUI] Format: npz [09:10] [ComfyUI] PreviewMultibandImage: Batch=1, Channels=10, Names=['img_01_r', 'img_01_g', 'img_01_b', 'mask_01', 'mask_02', 'mask_03', 'mask_04', 'mask_05', 'mask_06', 'mask_07'] [09:10] [ComfyUI] Prompt executed in 136.12 seconds [09:10] Node executed (4 total), capturing... [09:11] Frame 15 saved (frame_014.jpg, t=136.6s) [09:11] [capture-node] freeze=4ms shot=181ms unfreeze=0ms saved=True [09:11] Execution complete (t=136.6s) [09:11] Captured 15 unique frames over 136.78s [09:11] [debug] iframes: 0 [09:11] [timing] trigger_3d_previews: 0.0s [09:15] Saved high-quality screenshot: scene_seg_executed.png [09:15] [timing] final_screenshot: 2.8s [09:16] Capturing execution frames: scene_seg_interactive.json [09:22] Frame 1 saved (frame_000.jpg, t=0.0s) [09:22] Validating workflow... [09:22] [ComfyUI] === /validate endpoint called === [09:22] [ComfyUI] Received JSON with keys: ['prompt'] [09:22] [ComfyUI] Prompt has 4 nodes [09:22] [ComfyUI] Node 1: LoadImage [09:22] [ComfyUI] Inputs: {"image": "image.png"} [09:22] [ComfyUI] Node 4: MaskPreview [09:22] [ComfyUI] Inputs: {"mask": ["26", 0]} [09:22] [ComfyUI] Node 12: LoadSAM3Model [09:22] [ComfyUI] Inputs: {"precision": "auto", "compile": false} [09:22] [ComfyUI] Node 26: SAM3InteractiveCollector [09:22] [ComfyUI] Inputs: {"multi_prompts_store": "[{\"positive_points\":[{\"x\":3348.288109906738,\"y\":2452.2208696844737}],\"negative_points\":[],\"positive_boxes\":[],\"negative_boxes\":[],\"name\":\"Prompt 1\"},{\"positiv [09:22] [ComfyUI] Generated prompt_id: 9d31e9fc-064c-44d9-b83c-b746d12dccba [09:22] Queuing workflow for execution... [09:22] [ComfyUI] Calling execution.validate_prompt()... [09:22] [ComfyUI] Validation result: valid=True [09:22] [ComfyUI] === Validation PASSED === [09:22] [ComfyUI] got prompt [09:23] [ComfyUI] [comfy-env] Running LoadSAM3Model: precision: 'auto', compile: False [09:23] [ComfyUI] [SubprocessWorker] call_method: nodes.load_model.LoadSAM3Model.EXECUTE_NORMALIZED [09:23] [ComfyUI] [SubprocessWorker] acquired lock, ensuring started... [09:23] [ComfyUI] [ComfyUI-SAM3] Health check: ping (timeout=5.0s)... [09:23] [ComfyUI] [ComfyUI-SAM3] Health check: failed [09:23] [ComfyUI] [ComfyUI-SAM3] Socket unhealthy, restarting worker... [09:23] [ComfyUI] [ComfyUI-SAM3] python: /Users/runner/.ce/.pixi/envs/sam3-nodes/bin/python [09:23] [ComfyUI] [ComfyUI-SAM3] sys_path sent to worker: ['/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI', '/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI/custom_nodes/ComfyUI-SAM3', '/Users/runner/.ce/.pixi/envs/sam3-nodes/lib/python3.13/site-packages', '/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI/custom_nodes/ComfyUI-SAM3'] [09:23] [ComfyUI] [SubprocessWorker] is_pixi=True, python=/Users/runner/.ce/.pixi/envs/sam3-nodes/bin/python [09:23] [ComfyUI] [SubprocessWorker] launching cmd=['/Users/runner/.pixi/bin/pixi', 'run', '--as-is', '--manifest-path', '/Users/runner/.ce/pixi.toml', '-e', 'sam3-nodes', 'python', '/var/folders/tb/y368xp_x10s3ty1b_mtl5mxr0000gn/T/comfyui_pvenv_2em037f9/persistent_worker.py', 'unix:///var/folders/tb/y368xp_x10s3ty1b_mtl5mxr0000gn/T/comfy_worker_74918c732248.sock']... [09:23] [ComfyUI] [SubprocessWorker] socket_addr=unix:///var/folders/tb/y368xp_x10s3ty1b_mtl5mxr0000gn/T/comfy_worker_74918c732248.sock exists=True [09:25] Frame 2 saved (frame_001.jpg, t=2.2s) [09:25] [capture-periodic] freeze=5ms shot=944ms unfreeze=4ms saved=True [09:27] Frame 3 saved (frame_002.jpg, t=5.3s) [09:27] [capture-periodic] freeze=5ms shot=148ms unfreeze=10ms saved=True [09:29] [capture-loop] iter=50 t=6.8s state={'complete': False, 'error': None, 'executedCount': 0, 'wsState': 1} eval_ms=4 ram=2.8/7.0GB (50%) swap=1.8/4.0GB (45%) cpu=61% browser_rss=0.78GB [09:29] [ComfyUI] [SubprocessWorker] worker started/confirmed [09:29] [ComfyUI] [SubprocessWorker] serializing kwargs to shm... [09:29] [ComfyUI] [SubprocessWorker] created 0 shm blocks [09:29] [ComfyUI] [SubprocessWorker] sending request via socket... [09:29] [ComfyUI] [SubprocessWorker] call_id=9 sending call_method [09:30] [ComfyUI] [SubprocessWorker] got response: ok [09:30] [ComfyUI] [comfy-env] LoadSAM3Model done (7.11s): result: NodeOutput [09:30] [ComfyUI] [comfy-env] Running SAM3InteractiveCollector: multi_prompts_store: '[{"positive_points":[{"x":3348.288109906738,"y":2452.220..., sam3_model_config: dict, image: torch.float32 [1,4480,6720,3] cpu, unique_id: '26' [09:30] [ComfyUI] [SubprocessWorker] call_method: nodes.sam3_interactive.SAM3InteractiveCollector.EXECUTE_NORMALIZED [09:30] [ComfyUI] [SubprocessWorker] acquired lock, ensuring started... [09:30] [ComfyUI] [ComfyUI-SAM3] Health check: ping (timeout=5.0s)... [09:30] [ComfyUI] [ComfyUI-SAM3] Health check: ok [09:30] [ComfyUI] [SubprocessWorker] worker started/confirmed [09:30] [ComfyUI] [comfy-env] PRE-SERIALIZE 'image' shape: torch.Size([1, 4480, 6720, 3]) [09:30] [ComfyUI] [SubprocessWorker] serializing kwargs to shm... [09:30] [ComfyUI] [SubprocessWorker] created 0 shm blocks [09:30] [ComfyUI] [SubprocessWorker] sending request via socket... [09:30] [ComfyUI] [SubprocessWorker] call_id=10 sending call_method [09:30] [ComfyUI] [worker:ComfyUI-SAM3] [comfy-env] _from_shm got dict with keys: ['__type__', 'strategy', 'manager_path', 'storage_key', 'storage_size'] [09:30] [ComfyUI] [worker:ComfyUI-SAM3] [comfy-env] DESERIALIZE TensorRef: tensor_size=[1, 4480, 6720, 3] [09:30] [ComfyUI] [worker:ComfyUI-SAM3] [comfy-env] DESERIALIZED tensor shape: torch.Size([1, 4480, 6720, 3]) [09:32] Frame 4 saved (frame_003.jpg, t=9.8s) [09:32] [capture-periodic] freeze=15ms shot=146ms unfreeze=4ms saved=True [09:32] [ComfyUI] objc[19292]: Class AVFFrameReceiver is implemented in both /Users/runner/.ce/.pixi/envs/sam3-nodes/lib/libavdevice.62.3.101.dylib (0x121a787a8) and /Users/runner/.ce/.pixi/envs/sam3-nodes/lib/python3.13/site-packages/cv2/.dylibs/libavdevice.61.3.100.dylib (0x12e4903a8). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed. [09:32] [ComfyUI] objc[19292]: Class AVFAudioReceiver is implemented in both /Users/runner/.ce/.pixi/envs/sam3-nodes/lib/libavdevice.62.3.101.dylib (0x121a787f8) and /Users/runner/.ce/.pixi/envs/sam3-nodes/lib/python3.13/site-packages/cv2/.dylibs/libavdevice.61.3.100.dylib (0x12e4903f8). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed. [09:35] [capture-loop] iter=100 t=13.0s state={'complete': False, 'error': None, 'executedCount': 0, 'wsState': 1} eval_ms=4 ram=3.0/7.0GB (53%) swap=0.5/2.0GB (27%) cpu=54% browser_rss=0.68GB [09:41] [capture-loop] iter=150 t=19.3s state={'complete': False, 'error': None, 'executedCount': 0, 'wsState': 1} eval_ms=4 ram=3.6/7.0GB (62%) swap=0.5/2.0GB (27%) cpu=56% browser_rss=0.53GB [09:48] [capture-loop] iter=200 t=26.4s state={'complete': False, 'error': None, 'executedCount': 0, 'wsState': 1} eval_ms=8 ram=3.3/7.0GB (67%) swap=0.5/2.0GB (27%) cpu=69% browser_rss=0.48GB [09:55] [capture-loop] iter=250 t=33.2s state={'complete': False, 'error': None, 'executedCount': 0, 'wsState': 1} eval_ms=8 ram=2.8/7.0GB (73%) swap=0.5/2.0GB (27%) cpu=76% browser_rss=0.25GB [10:02] [capture-loop] iter=300 t=39.9s state={'complete': False, 'error': None, 'executedCount': 0, 'wsState': 1} eval_ms=8 ram=2.6/7.0GB (77%) swap=0.5/2.0GB (27%) cpu=59% browser_rss=0.23GB [10:06] [ComfyUI] [SubprocessWorker] callback 'request_vram_budget' from call_id=10 [10:06] [ComfyUI] [comfy-env] VRAM request: 1.84GB, free before eviction: 1.63GB [10:06] [ComfyUI] [comfy-env] VRAM after eviction: 1.63GB [10:10] [capture-loop] iter=350 t=47.9s state={'complete': False, 'error': None, 'executedCount': 0, 'wsState': 1} eval_ms=4 ram=3.1/7.0GB (85%) swap=0.5/2.0GB (27%) cpu=78% browser_rss=0.23GB [10:16] [capture-loop] iter=400 t=53.9s state={'complete': False, 'error': None, 'executedCount': 0, 'wsState': 1} eval_ms=2 ram=3.6/7.0GB (74%) swap=0.5/2.0GB (26%) cpu=90% browser_rss=0.22GB [10:23] [capture-loop] iter=450 t=60.8s state={'complete': False, 'error': None, 'executedCount': 0, 'wsState': 1} eval_ms=26 ram=2.8/7.0GB (73%) swap=0.5/2.0GB (26%) cpu=76% browser_rss=0.24GB [10:24] [ComfyUI] [worker:ComfyUI-SAM3] [10:24] [ComfyUI] === WATCHDOG TICK 1 === [10:24] [ComfyUI] [worker:ComfyUI-SAM3] Current thread 0x000000016e837000 (most recent call first): [10:24] [ComfyUI] File "/var/folders/tb/y368xp_x10s3ty1b_mtl5mxr0000gn/T/comfyui_pvenv_2em037f9/persistent_worker.py", line 52 in _watchdog [10:24] [ComfyUI] File "/Users/runner/.ce/.pixi/envs/sam3-nodes/lib/python3.13/threading.py", line 995 in run [10:24] [ComfyUI] File "/Users/runner/.ce/.pixi/envs/sam3-nodes/lib/python3.13/threading.py", line 1044 in _bootstrap_inner [10:24] [ComfyUI] File "/Users/runner/.ce/.pixi/envs/sam3-nodes/lib/python3.13/threading.py", line 1015 in _bootstrap [10:24] [ComfyUI] [10:24] [ComfyUI] Thread 0x00000001fe826240 (most recent call first): [10:24] [ComfyUI] File "/Users/runner/.ce/.pixi/envs/sam3-nodes/lib/python3.13/site-packages/PIL/Image.py", line 896 in tobytes [10:24] [ComfyUI] File "/Users/runner/.ce/.pixi/envs/sam3-nodes/lib/python3.13/site-packages/PIL/Image.py", line 820 in __array_interface__ [10:24] [ComfyUI] File "/Users/runner/.ce/.pixi/envs/sam3-nodes/lib/python3.13/site-packages/torchvision/transforms/functional.py", line 209 in pil_to_tensor [10:24] [ComfyUI] File "/Users/runner/.ce/.pixi/envs/sam3-nodes/lib/python3.13/site-packages/torchvision/transforms/v2/functional/_type_conversion.py", line 16 in to_image [10:24] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3/utils.py", line 1313 in set_image [10:24] [ComfyUI] File "/Users/runner/.ce/.pixi/envs/sam3-nodes/lib/python3.13/site-packages/torch/utils/_contextlib.py", line 124 in decorate_context [10:24] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI/custom_nodes/ComfyUI-SAM3/nodes/sam3_interactive.py", line 687 in execute [10:24] [ComfyUI] File "/Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/test_workspaces/SAM3-1105/ComfyUI/comfy_api/latest/_io.py", line 1833 in EXECUTE_NORMALIZED [10:24] [ComfyUI] File "/var/folders/tb/y368xp_x10s3ty1b_mtl5mxr0000gn/T/comfyui_pvenv_2em037f9/persistent_worker.py", line 1762 in main [10:24] [ComfyUI] File "/var/folders/tb/y368xp_x10s3ty1b_mtl5mxr0000gn/T/comfyui_pvenv_2em037f9/persistent_worker.py", line 1807 in [10:24] [ComfyUI] [10:24] [ComfyUI] [worker:ComfyUI-SAM3] === END === [10:24] [ComfyUI] [10:29] [capture-loop] iter=500 t=67.3s state={'complete': False, 'error': None, 'executedCount': 0, 'wsState': 1} eval_ms=3 ram=3.7/7.0GB (85%) swap=0.5/2.0GB (26%) cpu=76% browser_rss=0.22GB [10:39] [capture-loop] iter=550 t=76.7s state={'complete': False, 'error': None, 'executedCount': 0, 'wsState': 1} eval_ms=4 ram=3.7/7.0GB (77%) swap=0.5/2.0GB (26%) cpu=69% browser_rss=0.22GB [10:51] [ComfyUI] [SubprocessWorker] callback 'report_progress' from call_id=10 [10:53] Frame 5 saved (frame_004.jpg, t=90.5s) [10:53] [capture-periodic] freeze=6ms shot=527ms unfreeze=22ms saved=True [10:54] [capture-loop] iter=600 t=91.8s state={'complete': False, 'error': None, 'executedCount': 0, 'wsState': 1} eval_ms=14 ram=3.6/7.0GB (89%) swap=1.1/2.0GB (55%) cpu=80% browser_rss=0.20GB [11:07] [ComfyUI] [SubprocessWorker] got response: ok [11:07] [ComfyUI] [comfy-env] SAM3InteractiveCollector done (96.47s): result: NodeOutput [11:07] [capture-loop] iter=650 t=104.9s state={'complete': False, 'error': None, 'executedCount': 0, 'wsState': 1} eval_ms=5 ram=2.5/7.0GB (76%) swap=1.7/2.0GB (85%) cpu=73% browser_rss=0.18GB [11:10] Node executed (1 total), capturing... [11:10] [ComfyUI] Prompt executed in 107.48 seconds [11:11] Frame 6 saved (frame_005.jpg, t=107.7s) [11:11] [capture-node] freeze=876ms shot=300ms unfreeze=7ms saved=True [11:11] Node executed (2 total), capturing... [11:12] Frame 7 saved (frame_006.jpg, t=109.3s) [11:12] [capture-node] freeze=6ms shot=229ms unfreeze=0ms saved=True [11:12] Execution complete (t=109.3s) [11:12] Captured 7 unique frames over 109.62s [11:12] [debug] iframes: 0 [11:12] [timing] trigger_3d_previews: 0.0s [11:16] Saved high-quality screenshot: scene_seg_interactive_executed.png [11:16] [timing] final_screenshot: 2.6s [11:17] [5/5] SKIPPED (not in cpu list) video_point_prompt.json [11:18] Results saved to /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/comfy-test-logs/SAM3-1105/dev/macos-cpu/results.json [11:18] Model report: 1 files, 3.2 GB -> /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/comfy-test-logs/SAM3-1105/dev/macos-cpu/models.json [11:18] Saved: /Users/runner/work/comfy-gpu-ci/comfy-gpu-ci/comfy-test-logs/SAM3-1105/dev/macos-cpu/index.html [11:18] [EXECUTION] PASSED [11:18] macos: PASSED [11:18] Stopping ComfyUI server...