Support SAM2 models.

https://github.com/ltdrdata/ComfyUI-Impact-Pack/issues/684

README.md

@@ -8,6 +8,7 @@ This node pack helps to conveniently enhance images through Detector, Detailer,
 NOTE: The UltralyticsDetectorProvider node is not part of the ComfyUI-Impact-Pack. To use the UltralyticsDetectorProvider node, please install the ComfyUI-Impact-Subpack separately.
 
 ## NOTICE 
+* V8.18: Support [facebookresearch/sam2](https://github.com/facebookresearch/sam2) models
 * V8.0: The `Impact Subpack` is no longer installed automatically. To use `UltralyticsDetectorProvider` nodes, please install the `Impact Subpack` separately.
 * V7.6: Automatic installation is no longer supported. Please install using ComfyUI-Manager, or manually install requirements.txt and run install.py to complete the installation.
 * V7.0: Supports Switch based on Execution Model Inversion.
@@ -59,7 +60,7 @@ NOTE: The UltralyticsDetectorProvider node is not part of the ComfyUI-Impact-Pac
 
 ## Custom Nodes
 ### [Detector nodes](https://github.com/ltdrdata/ComfyUI-extension-tutorials/blob/Main/ComfyUI-Impact-Pack/tutorial/detectors.md)
-  * `SAMLoader` - Loads the SAM model.
+  * `SAMLoader (Impact)` - Loads the SAM model.
   * `ONNXDetectorProvider` - Loads the ONNX model to provide BBOX_DETECTOR.
   * `CLIPSegDetectorProvider` - Wrapper for CLIPSeg to provide BBOX_DETECTOR.
     * You need to install the ComfyUI-CLIPSeg node extension.
@@ -70,6 +71,9 @@ NOTE: The UltralyticsDetectorProvider node is not part of the ComfyUI-Impact-Pac
     * As a result, it outputs the `combined_mask`, which is a unified mask, and `batch_masks`, which are multiple masks grouped together in batch form.
     * While `batch_masks` may not be completely separated, it provides functionality to perform some level of segmentation.
   * `Simple Detector (SEGS)` - Operating primarily with `BBOX_DETECTOR`, and with the additional provision of `SAM_MODEL` or `SEGM_DETECTOR`, this node internally generates improved SEGS through mask operations on both *bbox* and *silhouette*. It serves as a convenient tool to simplify a somewhat intricate workflow.
+  * `Simple Detector for Video (SEGS)` – Performs detection on videos composed of image frames. Instead of using a single mask, it performs detection individually on each image frame and generates a SEGS object with a batch of masks. 
+  * `SAM2 Video Detector (SEGS)` – Similar to `Simple Detector for Video (SEGS)`, but utilizes SAM2’s video tracking technology to generate a SEGS object with a batch of masks. 
+      * To use this node, you must select a SAM2 model in the SAMLoader.
 
 ### ControlNet, IPAdapter
   * `ControlNetApply (SEGS)` - To apply ControlNet in SEGS, you need to use the Preprocessor Provider node from the Inspire Pack to utilize this node.
@@ -101,7 +105,7 @@ NOTE: The UltralyticsDetectorProvider node is not part of the ComfyUI-Impact-Pac
   * `DetailerDebug (SEGS)` - Refines the image based on SEGS. Additionally, it provides the ability to monitor the cropped image and the refined image of the cropped image.
     * To prevent regeneration caused by the seed that does not change every time when using 'external_seed', please disable the 'seed random generate' option in the 'Detailer...' node.
   * `MASK to SEGS` - Generates SEGS based on the mask.
-  * `MASK to SEGS For AnimateDiff` - Generates SEGS based on the mask for AnimateDiff.
+  * `MASK to SEGS For Video` - Generates SEGS based on the mask for Video. (Renamed from `MASK to SEGS For AnimateDiff`)
     * When using a single mask, convert it to SEGS to apply it to the entire frame.
     * When using a batch mask, the contour fill feature is disabled.
   * `MediaPipe FaceMesh to SEGS` - Separate each landmark from the mediapipe facemesh image to create labeled SEGS.
@@ -189,6 +193,10 @@ NOTE: The UltralyticsDetectorProvider node is not part of the ComfyUI-Impact-Pac
   * `PreviewDetailerHook` - Connecting this hook node helps provide assistance for viewing previews whenever SEGS Detailing tasks are completed. When working with a large number of SEGS, such as Make Tile SEGS, it allows for monitoring the situation as improvements progress incrementally.
     * Since this is the hook applied when pasting onto the original image, it has no effect on nodes like `SEGSDetailer`.
   * `VariationNoiseDetailerHookProvider` - Apply variation seed to the detailer. It can be applied in multiple stages through combine.
+  * `CustomSamplerDetailerHookProvider` - Apply a hook that allows you to use a custom sampler in the Detailer nodes. When using `DetailerHookCombine`, the sampler from the first hook is applied.
+  * `LamaRemoverDetailerHookProvider` – Applies Lama Remover to the upscaled image during the detailing stage. If `skip_sampling` is set to True, Lama Remover can be used alone without the detailing stage, allowing it to simply remove detected regions.
+      * Not applicable for **AnimateDiff** detailers. When using `DetailerHookCombine`, `skip_sampling` is only applied if it is set to `True` for all hooks.
+      * To use this node, the node pack at [Layer-norm/comfyui-lama-remover](https://github.com/Layer-norm/comfyui-lama-remover) must be installed.
 
 ### Iterative Upscale nodes
   * `Iterative Upscale (Latent/on Pixel Space)` - The upscaler takes the input upscaler and splits the scale_factor into steps, then iteratively performs upscaling. 
@@ -268,6 +276,7 @@ NOTE: The UltralyticsDetectorProvider node is not part of the ComfyUI-Impact-Pac
   * `Masks to Mask List`, `Mask List to Masks`, `Make Mask List`, `Make Mask Batch` - It has the same functionality as the nodes above, but uses mask as input instead of image.
   * `Flatten Mask Batch` - Flattens a Mask Batch into a single Mask. Normal operation is not guaranteed for non-binary masks. 
   * `Make List (Any)` - Create a list with arbitrary values.
+  * `Select Nth Item (Any list)` - Selects the Nth item from a list. If the index is out of range, it returns the last item in the list. 
 
 ### Logics (experimental) 
   * These nodes are experimental nodes designed to implement the logic for loops and dynamic switching.
@@ -487,3 +496,5 @@ BlenderNeok/[ComfyUI_Noise](https://github.com/BlenderNeko/ComfyUI_Noise) - The
 WASasquatch/[was-node-suite-comfyui](https://github.com/WASasquatch/was-node-suite-comfyui) - A powerful custom node extensions of ComfyUI.
 
 Trung0246/[ComfyUI-0246](https://github.com/Trung0246/ComfyUI-0246) - Nice bypass hack!
+
+Layer-norm/[comfyui-lama-remover](https://github.com/Layer-norm/comfyui-lama-remover) - Required for using `LamaRemoverDetailerHook`.

__init__.py

@@ -122,6 +122,8 @@ NODE_CLASS_MAPPINGS = {
     "UnsamplerHookProvider": UnsamplerHookProvider,
     "CoreMLDetailerHookProvider": CoreMLDetailerHookProvider,
     "PreviewDetailerHookProvider": PreviewDetailerHookProvider,
+    "CustomSamplerDetailerHookProvider": CustomSamplerDetailerHookProvider,
+    "LamaRemoverDetailerHookProvider": LamaRemoverDetailerHookProvider,
 
     "DetailerHookCombine": DetailerHookCombine,
     "NoiseInjectionDetailerHookProvider": NoiseInjectionDetailerHookProvider,
@@ -157,6 +159,7 @@ NODE_CLASS_MAPPINGS = {
     "SegmDetectorSEGS": SegmDetectorForEach,
     "ONNXDetectorSEGS": BboxDetectorForEach,
     "ImpactSimpleDetectorSEGS_for_AD": SimpleDetectorForAnimateDiff,
+    "ImpactSAM2VideoDetectorSEGS": SAM2VideoDetectorSEGS,
     "ImpactSimpleDetectorSEGS": SimpleDetectorForEach,
     "ImpactSimpleDetectorSEGSPipe": SimpleDetectorForEachPipe,
     "ImpactControlNetApplySEGS": ControlNetApplySEGS,
@@ -234,6 +237,7 @@ NODE_CLASS_MAPPINGS = {
     "ImpactMakeAnyList": MakeAnyList,
     "ImpactMakeMaskList": MakeMaskList,
     "ImpactMakeMaskBatch": MakeMaskBatch,
+    "ImpactSelectNthItemOfAnyList": NthItemOfAnyList,
 
     "RegionalSampler": RegionalSampler,
     "RegionalSamplerAdvanced": RegionalSamplerAdvanced,
@@ -301,7 +305,8 @@ NODE_DISPLAY_NAME_MAPPINGS = {
     "BboxDetectorSEGS": "BBOX Detector (SEGS)",
     "SegmDetectorSEGS": "SEGM Detector (SEGS)",
     "ONNXDetectorSEGS": "ONNX Detector (SEGS/legacy) - use BBOXDetector",
-    "ImpactSimpleDetectorSEGS_for_AD": "Simple Detector for AnimateDiff (SEGS)",
+    "ImpactSimpleDetectorSEGS_for_AD": "Simple Detector for Video (SEGS)",
+    "ImpactSAM2VideoDetectorSEGS": "SAM2 Video Detector (SEGS)",
     "ImpactSimpleDetectorSEGS": "Simple Detector (SEGS)",
     "ImpactSimpleDetectorSEGSPipe": "Simple Detector (SEGS/pipe)",
     "ImpactControlNetApplySEGS": "ControlNetApply (SEGS) - DEPRECATED",
@@ -313,7 +318,7 @@ NODE_DISPLAY_NAME_MAPPINGS = {
     "SegsToCombinedMask": "SEGS to MASK (combined)",
     "MediaPipeFaceMeshToSEGS": "MediaPipe FaceMesh to SEGS",
     "MaskToSEGS": "MASK to SEGS",
-    "MaskToSEGS_for_AnimateDiff": "MASK to SEGS for AnimateDiff",
+    "MaskToSEGS_for_AnimateDiff": "MASK to SEGS for Video",
     "BitwiseAndMaskForEach": "Pixelwise(SEGS & SEGS)",
     "SubtractMaskForEach": "Pixelwise(SEGS - SEGS)",
     "ImpactSegsAndMask": "Pixelwise(SEGS & MASK)",
@@ -328,8 +333,8 @@ NODE_DISPLAY_NAME_MAPPINGS = {
     "DetailerForEachPipe": "Detailer (SEGS/pipe)",
     "DetailerForEachDebug": "DetailerDebug (SEGS)",
     "DetailerForEachDebugPipe": "DetailerDebug (SEGS/pipe)",
-    "SEGSDetailerForAnimateDiff": "SEGSDetailer For AnimateDiff (SEGS/pipe)",
-    "DetailerForEachPipeForAnimateDiff": "Detailer For AnimateDiff (SEGS/pipe)",
+    "SEGSDetailerForAnimateDiff": "SEGSDetailer For Video (SEGS/pipe)",
+    "DetailerForEachPipeForAnimateDiff": "Detailer For Video (SEGS/pipe)",
     "SEGSUpscaler": "Upscaler (SEGS)",
     "SEGSUpscalerPipe": "Upscaler (SEGS/pipe)",
 
@@ -407,6 +412,7 @@ NODE_DISPLAY_NAME_MAPPINGS = {
     "ImpactMakeMaskList": "Make Mask List",
     "ImpactMakeMaskBatch": "Make Mask Batch",
     "ImpactMakeAnyList": "Make List (Any)",
+    "ImpactSelectNthItemOfAnyList": "Select Nth Item (Any list)",
 
     "ImpactStringSelector": "String Selector",
     "StringListToString": "String List to String",

install.py

@@ -68,7 +68,6 @@ def process_wrap(cmd_str, cwd=None, handler=None, env=None):
 
 
 try:
-    import platform
     from torchvision.datasets.utils import download_url
     import impact.config
 
@@ -99,7 +98,7 @@ try:
                 if not os.path.exists(os.path.join(sam_path, "sam_vit_b_01ec64.pth")):
                     download_url("https://dl.fbaipublicfiles.com/segment_anything/sam_vit_b_01ec64.pth", sam_path)
             except:
-                print(f"[Impact Pack] Failed to auto-download model files. Please download them manually.")
+                print("[Impact Pack] Failed to auto-download model files. Please download them manually.")
 
         if not os.path.exists(onnx_path):
             print(f"### ComfyUI-Impact-Pack: onnx model directory created ({onnx_path})")
@@ -108,18 +107,21 @@ try:
         impact.config.write_config()
 
         # Remove legacy subpack
-        subpack_path = os.path.join(os.path.dirname(__file__), 'impact_subpack')
-        if os.path.exists(subpack_path):
-            shutil.rmtree(subpack_path)
-            print(f"Legacy subpack is detected. '{subpack_path}' is removed.")
-            
-        subpack_path = os.path.join(os.path.dirname(__file__), 'subpack')
-        if os.path.exists(subpack_path):
-            shutil.rmtree(subpack_path)
-            print(f"Legacy subpack is detected. '{subpack_path}' is removed.")
+        try:
+            subpack_path = os.path.join(os.path.dirname(__file__), 'impact_subpack')
+            if os.path.exists(subpack_path):
+                shutil.rmtree(subpack_path)
+                print(f"Legacy subpack is detected. '{subpack_path}' is removed.")
+                
+            subpack_path = os.path.join(os.path.dirname(__file__), 'subpack')
+            if os.path.exists(subpack_path):
+                shutil.rmtree(subpack_path)
+                print(f"Legacy subpack is detected. '{subpack_path}' is removed.")
+        except:
+            print(f"ERROT: Failed to delete legacy subpack '{subpack_path}'\nPlease delete the folder after terminate ComfyUI.")
 
     install()
 
-except Exception as e:
+except Exception:
     print("[ERROR] ComfyUI-Impact-Pack: Dependency installation has failed. Please install manually.")
     traceback.print_exc()

js/common.js

@@ -3,6 +3,48 @@ import { app } from "../../scripts/app.js";
 
 let original_show = app.ui.dialog.show;
 
+export function customAlert(message) {
+	try {
+		app.extensionManager.toast.addAlert(message);
+	}
+	catch {
+		alert(message);
+	}
+}
+
+export function isBeforeFrontendVersion(compareVersion) {
+    try {
+        const frontendVersion = window['__COMFYUI_FRONTEND_VERSION__'];
+        if (typeof frontendVersion !== 'string') {
+            return false;
+        }
+
+        function parseVersion(versionString) {
+            const parts = versionString.split('.').map(Number);
+            return parts.length === 3 && parts.every(part => !isNaN(part)) ? parts : null;
+        }
+
+        const currentVersion = parseVersion(frontendVersion);
+        const comparisonVersion = parseVersion(compareVersion);
+
+        if (!currentVersion || !comparisonVersion) {
+            return false;
+        }
+
+        for (let i = 0; i < 3; i++) {
+            if (currentVersion[i] > comparisonVersion[i]) {
+                return false;
+            } else if (currentVersion[i] < comparisonVersion[i]) {
+                return true;
+            }
+        }
+
+        return false;
+    } catch {
+        return true;
+    }
+}
+
 function dialog_show_wrapper(html) {
 	if (typeof html === "string") {
 		if(html.includes("IMPACT-PACK-SIGNAL: STOP CONTROL BRIDGE")) {

js/impact-pack.js

@@ -1,6 +1,13 @@
 import { ComfyApp, app } from "../../scripts/app.js";
 import { ComfyDialog, $el } from "../../scripts/ui.js";
 import { api } from "../../scripts/api.js";
+import { customAlert, isBeforeFrontendVersion } from "./common.js";
+
+const is_legacy_front = () => isBeforeFrontendVersion('1.16.9');
+
+if(is_legacy_front()) {
+	customAlert("An outdated version(<1.16.9) of the `comfyui-frontend-package` is installed. It is not compatible with the current version of the Impact Pack.");
+}
 
 let wildcards_list = [];
 async function load_wildcards() {
@@ -324,6 +331,32 @@ app.registerExtension({
 			}
 		}
 
+		if(nodeData.name == "ImpactSelectNthItemOfAnyList") {
+			const onConnectionsChange = nodeType.prototype.onConnectionsChange;
+			nodeType.prototype.onConnectionsChange = function (type, index, connected, link_info) {
+				if(!link_info || this.inputs[0].type != '*')
+					return;
+
+				if(index >= 2)
+					return;
+
+				// assign type
+				let slot_type = '*';
+
+				if(type == 2) {
+					slot_type = link_info.type;
+				}
+				else {
+					const node = app.graph.getNodeById(link_info.origin_id);
+					slot_type = node.outputs[link_info.origin_slot].type;
+				}
+
+				this.inputs[0].type = slot_type;
+				this.outputs[0].type = slot_type;
+				this.outputs[0].label = slot_type;
+			}
+		}
+
 		if(nodeData.name === 'ImpactInversedSwitch') {
 			nodeData.output = ['*'];
 			nodeData.output_is_list = [false];
@@ -337,12 +370,12 @@ app.registerExtension({
 				if(type == 2) {
 					// connect output
 					if(connected){
-						if(app.graph._nodes_by_id[link_info.target_id].type == 'Reroute') {
+						if(app.graph._nodes_by_id[link_info.target_id]?.type == 'Reroute') {
 							app.graph._nodes_by_id[link_info.target_id].disconnectInput(link_info.target_slot);
 						}
 
 						if(this.outputs[0].type == '*'){
-							if(link_info.type == '*') {
+							if(link_info.type == '*' && app.graph.getNodeById(link_info.target_id).slots[link_info.target_slot].type != '*') {
 								app.graph._nodes_by_id[link_info.target_id].disconnectInput(link_info.target_slot);
 							}
 							else {
@@ -359,7 +392,7 @@ app.registerExtension({
 					}
 				}
 				else {
-					if(app.graph._nodes_by_id[link_info.origin_id].type == 'Reroute')
+					if(app.graph._nodes_by_id[link_info.origin_id]?.type == 'Reroute')
 						this.disconnectInput(link_info.target_slot);
 
 					// connect input
@@ -371,7 +404,7 @@ app.registerExtension({
 							return; // fallback
 						}
 
-						if(origin_type == '*') {
+						if(origin_type == '*' && app.graph.getNodeById(link_info.origin_id).slots[link_info.origin_slot].type != '*') {
 							this.disconnectInput(link_info.target_slot);
 							return;
 						}
@@ -395,8 +428,9 @@ app.registerExtension({
 						!stackTrace.includes('LGraphNode.prototype.connect') && // for touch device
 						!stackTrace.includes('LGraphNode.connect') && // for mouse device
 						!stackTrace.includes('loadGraphData')) {
-							if(this.outputs[link_info.origin_slot].links.length == 0)
+							if(this.outputs[link_info.origin_slot].links.length == 0) {
 								this.removeOutput(link_info.origin_slot);
+							}
 					}
 				}
 
@@ -409,9 +443,12 @@ app.registerExtension({
 					slot_i++;
 				}
 
-				let last_slot = this.outputs[this.outputs.length - 1];
-				if (last_slot.slot_index == link_info.origin_slot) {
-					this.addOutput(`output${slot_i}`, this.outputs[0].type);
+				if(connected) {
+					// NOTE: node.slot_index is different with link_info.origin_slot
+					let last_slot_index = this.outputs.length - 1;
+					if (last_slot_index == link_info.origin_slot) {
+						this.addOutput(`output${slot_i}`, this.outputs[0].type);
+					}
 				}
 
 				let select_slot = this.inputs.find(x => x.name == "select");
@@ -474,6 +511,23 @@ app.registerExtension({
 
 			const onConnectionsChange = nodeType.prototype.onConnectionsChange;
 			nodeType.prototype.onConnectionsChange = function (type, index, connected, link_info) {
+				const stackTrace = new Error().stack;
+				if(stackTrace.includes('loadGraphData')) {
+					if(this.widgets?.[0]) {
+						this.widgets[0].options.max = this.inputs.length-3;
+						this.widgets[0].value = Math.min(this.widgets[0].value, this.widgets[0].options.max);
+					}
+					return;
+				}
+
+				if(stackTrace.includes('pasteFromClipboard')) {
+					if(this.widgets?.[0]) {
+						this.widgets[0].options.max = this.inputs.length-3;
+						this.widgets[0].value = Math.min(this.widgets[0].value, this.widgets[0].options.max);
+					}
+					return;
+				}
+
 				if(!link_info)
 					return;
 
@@ -485,7 +539,7 @@ app.registerExtension({
 						}
 
 						if(this.outputs[0].type == '*'){
-							if(link_info.type == '*') {
+							if(link_info.type == '*' && app.graph.getNodeById(link_info.target_id).slots[link_info.target_slot].type != '*') {
 								app.graph._nodes_by_id[link_info.target_id].disconnectInput(link_info.target_slot);
 							}
 							else {
@@ -516,12 +570,12 @@ app.registerExtension({
 					if(this.inputs[0].type == '*'){
 						const node = app.graph.getNodeById(link_info.origin_id);
 						let origin_type = node.outputs[link_info.origin_slot]?.type;
-						if(link_info.target_slot == 0 && this.inputs.length > 1) {
+						if(link_info.target_slot == 0 && this.inputs.length > 3) {  // NOTE: widgets are regarded as input since new front
 								origin_type = this.inputs[1].type;
 								node.connect(link_info.origin_slot, node.id, 'input1');
 						}
 						
-						if(origin_type == '*') {
+						if(origin_type == '*' && app.graph.getNodeById(link_info.origin_id).slots[link_info.origin_slot].type != '*') {
 							this.disconnectInput(link_info.target_slot);
 							return;
 						}
@@ -539,15 +593,8 @@ app.registerExtension({
 				}
 
 				let select_slot = this.inputs.find(x => x.name == "select");
-				let mode_slot = this.inputs.find(x => x.name == "sel_mode");
-
-				let converted_count = 0;
-				converted_count += select_slot?1:0;
-				converted_count += mode_slot?1:0;
-
-				if (!connected && (this.inputs.length > 1+converted_count)) {
-					const stackTrace = new Error().stack;
 
+				if (!connected && (this.inputs.length > 3)) {
 					if(
 						!stackTrace.includes('LGraphNode.prototype.connect') && // for touch device
 						!stackTrace.includes('LGraphNode.connect') && // for mouse device
@@ -557,6 +604,7 @@ app.registerExtension({
 					}
 				}
 
+
 				let slot_i = 1;
 				for (let i = 0; i < this.inputs.length; i++) {
 					let input_i = this.inputs[i];
@@ -566,18 +614,13 @@ app.registerExtension({
 					}
 				}
 
-				let last_slot = this.inputs[this.inputs.length - 1];
-				if (
-					(last_slot.name == 'select' && last_slot.name != 'sel_mode' && this.inputs[this.inputs.length - 2].link != undefined)
-					|| (last_slot.name != 'select' && last_slot.name != 'sel_mode' && last_slot.link != undefined)) {
-						this.addInput(`${input_name}${slot_i}`, this.outputs[0].type);
+				if(connected) {
+					this.addInput(`${input_name}${slot_i}`, this.outputs[0].type);
 				}
 
-				if(this.widgets?.length) {
-					this.widgets[0].options.max = select_slot?this.inputs.length-1:this.inputs.length;
+				if(this.widgets?.[0]) {
+					this.widgets[0].options.max = this.inputs.length-3;
 					this.widgets[0].value = Math.min(this.widgets[0].value, this.widgets[0].options.max);
-					if(this.widgets[0].options.max > 0 && this.widgets[0].value == 0)
-						this.widgets[0].value = 1;
 				}
 			}
 		}

locales/ko/nodeDefs.json

@@ -1196,7 +1196,7 @@
 
     "ImpactWildcardEncode": {
         "description": "이 노드는 와일드카드 구문으로 작성된 텍스트 프롬프트를 처리하고 이를 조건으로 출력합니다. 또한 LoRA 구문을 지원하며, 적용된 LoRA는 모델 출력에 반영됩니다.\n\nTIP1: 워크플로가 실행되기 전에 '와일드카드 텍스트'의 처리 결과가 '채워진 텍스트'에 표시되며, 이 값은 워크플로와 함께 저장됩니다. 입력으로 변환된 시드를 사용하려면 '와일드카드 텍스트' 대신 '채워진 텍스트'에 직접 프롬프트를 작성하고, 모드를 '고정(fixed)'로 설정하세요.\nTIP2: 'Inspire Pack'이 설치되어 있으면 LBW(로라 블록 웨이트) 구문도 적용할 수 있습니다.",
-        "display_name": "와일드카드 처리기 (Impact)",
+        "display_name": "와일드카드 인코딩 (Impact)",
         "inputs": {
             "wildcard_text": {
                 "name": "와일드카드 텍스트",

modules/impact/config.py

@@ -1,7 +1,7 @@
 import configparser
 import os
 
-version_code = [8, 11]
+version_code = [8, 18]
 version = f"V{version_code[0]}.{version_code[1]}" + (f'.{version_code[2]}' if len(version_code) > 2 else '')
 
 dependency_version = 24

modules/impact/core.py

@@ -1,17 +1,15 @@
-import copy
 import os
 import warnings
 
-import numpy
 import torch
+from sam2.sam2_image_predictor import SAM2ImagePredictor
 from segment_anything import SamPredictor
 
 from comfy_extras.nodes_custom_sampler import Noise_RandomNoise
 from impact.utils import *
 from collections import namedtuple
 import numpy as np
-from skimage.measure import label
-from PIL import ImageOps
+from PIL import ImageOps, Image
 
 import nodes
 import comfy_extras.nodes_upscale_model as model_upscale
@@ -26,6 +24,9 @@ from impact import utils
 from impact import impact_sampling
 from concurrent.futures import ThreadPoolExecutor
 import inspect
+from collections import OrderedDict
+from sam2.build_sam import build_sam2, build_sam2_video_predictor
+import torch.nn.functional as F
 
 
 try:
@@ -83,7 +84,7 @@ def set_previewbridge_image(node_id, file, item):
 
 
 def erosion_mask(mask, grow_mask_by):
-    mask = make_2d_mask(mask)
+    mask = utils.make_2d_mask(mask)
 
     w = mask.shape[1]
     h = mask.shape[0]
@@ -139,7 +140,7 @@ def mix_noise(from_noise, to_noise, strength, variation_method):
 
 class REGIONAL_PROMPT:
     def __init__(self, mask, sampler, variation_seed=0, variation_strength=0.0, variation_method='linear'):
-        mask = make_2d_mask(mask)
+        mask = utils.make_2d_mask(mask)
 
         self.mask = mask
         self.sampler = sampler
@@ -199,7 +200,7 @@ def create_segmasks(results):
 
 
 def gen_detection_hints_from_mask_area(x, y, mask, threshold, use_negative):
-    mask = make_2d_mask(mask)
+    mask = utils.make_2d_mask(mask)
 
     points = []
     plabs = []
@@ -318,7 +319,10 @@ def enhance_detail(image, model, clip, vae, guide_size, guide_size_for_bbox, max
     print(f"Detailer: segment upscale for ({bbox_w, bbox_h}) | crop region {w, h} x {upscale} -> {new_w, new_h}")
 
     # upscale
-    upscaled_image = tensor_resize(image, new_w, new_h)
+    upscaled_image = utils.tensor_resize(image, new_w, new_h)
+
+    if detailer_hook is not None:
+        upscaled_image = detailer_hook.post_upscale(upscaled_image, noise_mask)
 
     cnet_pils = None
     if control_net_wrapper is not None:
@@ -327,67 +331,75 @@ def enhance_detail(image, model, clip, vae, guide_size, guide_size_for_bbox, max
         cnet_pils.extend(cnet_pils2)
 
     # prepare mask
-    if noise_mask is not None and inpaint_model:
-        imc_encode = nodes.InpaintModelConditioning().encode
-        if 'noise_mask' in inspect.signature(imc_encode).parameters:
-            positive, negative, latent_image = imc_encode(positive, negative, upscaled_image, vae, mask=noise_mask, noise_mask=True)
+    if detailer_hook is None or not detailer_hook.get_skip_sampling():
+        if noise_mask is not None and inpaint_model:
+            imc_encode = nodes.InpaintModelConditioning().encode
+            if 'noise_mask' in inspect.signature(imc_encode).parameters:
+                positive, negative, latent_image = imc_encode(positive, negative, upscaled_image, vae, mask=noise_mask, noise_mask=True)
+            else:
+                print(f"[Impact Pack] ComfyUI is an outdated version.")
+                positive, negative, latent_image = imc_encode(positive, negative, upscaled_image, vae, noise_mask)
         else:
-            print(f"[Impact Pack] ComfyUI is an outdated version.")
-            positive, negative, latent_image = imc_encode(positive, negative, upscaled_image, vae, noise_mask)
-    else:
-        latent_image = to_latent_image(upscaled_image, vae, vae_tiled_encode=vae_tiled_encode)
-        if noise_mask is not None:
-            latent_image['noise_mask'] = noise_mask
+            latent_image = utils.to_latent_image(upscaled_image, vae, vae_tiled_encode=vae_tiled_encode)
+            if noise_mask is not None:
+                latent_image['noise_mask'] = noise_mask
 
-    if detailer_hook is not None:
-        latent_image = detailer_hook.post_encode(latent_image)
-
-    refined_latent = latent_image
-
-    # ksampler
-    for i in range(0, cycle):
         if detailer_hook is not None:
+            latent_image = detailer_hook.post_encode(latent_image)
+
+        refined_latent = latent_image
+
+        sampler_opt=None
+        if detailer_hook is not None:
+            sampler_opt = detailer_hook.get_custom_sampler()
+
+        # ksampler
+        for i in range(0, cycle):
             if detailer_hook is not None:
-                detailer_hook.set_steps((i, cycle))
+                if detailer_hook is not None:
+                    detailer_hook.set_steps((i, cycle))
 
-            refined_latent = detailer_hook.cycle_latent(refined_latent)
+                refined_latent = detailer_hook.cycle_latent(refined_latent)
 
-            model2, seed2, steps2, cfg2, sampler_name2, scheduler2, positive2, negative2, upscaled_latent2, denoise2 = \
-                detailer_hook.pre_ksample(model, seed+i, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, denoise)
-            noise, is_touched = detailer_hook.get_custom_noise(seed+i, torch.zeros(latent_image['samples'].size()), is_touched=False)
-            if not is_touched:
+                model2, seed2, steps2, cfg2, sampler_name2, scheduler2, positive2, negative2, upscaled_latent2, denoise2 = \
+                    detailer_hook.pre_ksample(model, seed+i, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, denoise)
+                noise, is_touched = detailer_hook.get_custom_noise(seed+i, torch.zeros(latent_image['samples'].size()), is_touched=False)
+                if not is_touched:
+                    noise = None
+            else:
+                model2, seed2, steps2, cfg2, sampler_name2, scheduler2, positive2, negative2, upscaled_latent2, denoise2 = \
+                    model, seed + i, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, denoise
                 noise = None
+
+            refined_latent = impact_sampling.ksampler_wrapper(model2, seed2, steps2, cfg2, sampler_name2, scheduler2, positive2, negative2,
+                                                              refined_latent, denoise2, refiner_ratio, refiner_model, refiner_clip, refiner_positive, refiner_negative,
+                                                              noise=noise, scheduler_func=scheduler_func, sampler_opt=sampler_opt)
+
+        if detailer_hook is not None:
+            refined_latent = detailer_hook.pre_decode(refined_latent)
+
+        # non-latent downscale - latent downscale cause bad quality
+        start = time.time()
+        if vae_tiled_decode:
+            (refined_image,) = nodes.VAEDecodeTiled().decode(vae, refined_latent, 512) # using default settings
+            print(f"[Impact Pack] vae decoded (tiled) in {time.time() - start:.1f}s")
         else:
-            model2, seed2, steps2, cfg2, sampler_name2, scheduler2, positive2, negative2, upscaled_latent2, denoise2 = \
-                model, seed + i, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, denoise
-            noise = None
-
-        refined_latent = impact_sampling.ksampler_wrapper(model2, seed2, steps2, cfg2, sampler_name2, scheduler2, positive2, negative2,
-                                                          refined_latent, denoise2, refiner_ratio, refiner_model, refiner_clip, refiner_positive, refiner_negative,
-                                                          noise=noise, scheduler_func=scheduler_func)
-
-    if detailer_hook is not None:
-        refined_latent = detailer_hook.pre_decode(refined_latent)
-
-    # non-latent downscale - latent downscale cause bad quality
-    start = time.time()
-    if vae_tiled_decode:
-        (refined_image,) = nodes.VAEDecodeTiled().decode(vae, refined_latent, 512) # using default settings
-        print(f"[Impact Pack] vae decoded (tiled) in {time.time() - start:.1f}s")
+            try:
+                refined_image = vae.decode(refined_latent['samples'])
+            except Exception as e:
+                # usually an out-of-memory exception from the decode, so try a tiled approach
+                print(f"[Impact Pack] failed after {time.time() - start:.1f}s, doing vae.decode_tiled 64...")
+                refined_image = vae.decode_tiled(refined_latent["samples"], tile_x=64, tile_y=64, )
+            print(f"[Impact Pack] vae decoded in {time.time() - start:.1f}s")
     else:
-        try:
-            refined_image = vae.decode(refined_latent['samples'])
-        except Exception as e:
-            # usually an out-of-memory exception from the decode, so try a tiled approach
-            print(f"[Impact Pack] failed after {time.time() - start:.1f}s, doing vae.decode_tiled 64...")
-            refined_image = vae.decode_tiled(refined_latent["samples"], tile_x=64, tile_y=64, )
-        print(f"[Impact Pack] vae decoded in {time.time() - start:.1f}s")
+        # skipped
+        refined_image = upscaled_image
 
     if detailer_hook is not None:
         refined_image = detailer_hook.post_decode(refined_image)
 
     # downscale
-    refined_image = tensor_resize(refined_image, w, h)
+    refined_image = utils.tensor_resize(refined_image, w, h)
 
     # prevent mixing of device
     refined_image = refined_image.cpu()
@@ -482,10 +494,10 @@ def enhance_detail_for_animatediff(image_frames, model, clip, vae, guide_size, g
         image = torch.from_numpy(image).unsqueeze(0)
 
         # upscale
-        upscaled_image = tensor_resize(image, new_w, new_h)
+        upscaled_image = utils.tensor_resize(image, new_w, new_h)
 
         # ksampler
-        samples = to_latent_image(upscaled_image, vae)['samples']
+        samples = utils.to_latent_image(upscaled_image, vae)['samples']
 
         if latent_frames is None:
             latent_frames = samples
@@ -513,11 +525,16 @@ def enhance_detail_for_animatediff(image_frames, model, clip, vae, guide_size, g
         'samples': latent_frames
     }
 
+
+    sampler_opt=None
+    if detailer_hook is not None:
+        sampler_opt = detailer_hook.get_custom_sampler()
+
     if detailer_hook is not None:
         latent = detailer_hook.post_encode(latent)
 
     refined_latent = impact_sampling.ksampler_wrapper(model, seed, steps, cfg, sampler_name, scheduler, positive, negative,
-                                                      latent, denoise, refiner_ratio, refiner_model, refiner_clip, refiner_positive, refiner_negative, scheduler_func=scheduler_func)
+                                                      latent, denoise, refiner_ratio, refiner_model, refiner_clip, refiner_positive, refiner_negative, scheduler_func=scheduler_func, sampler_opt=sampler_opt)
 
     if detailer_hook is not None:
         refined_latent = detailer_hook.pre_decode(refined_latent)
@@ -605,6 +622,118 @@ class SAMWrapper:
         return sam_predict(predictor, points, plabs, bbox, threshold)
 
 
+class SAM2Wrapper:
+    def __init__(self, config, modelname, is_auto_mode, safe_to_gpu=None, device_mode="AUTO"):
+        self.config = config
+        self.modelname = modelname
+        self.image_predictor = None
+        self.video_predictor = None
+        self.device_mode = device_mode
+        self.safe_to_gpu = safe_to_gpu if safe_to_gpu is not None else SafeToGPU_stub()
+        self.is_auto_mode = is_auto_mode
+
+    def prepare_device(self):
+        pass
+
+    def prepare_image_device(self):
+        if self.is_auto_mode:
+            device = comfy.model_management.get_torch_device()
+            self.safe_to_gpu.to_device(self.image_predictor.model, device=device)
+
+    def prepare_video_device(self):
+        if self.is_auto_mode:
+            device = comfy.model_management.get_torch_device()
+            self.safe_to_gpu.to_device(self.video_predictor, device=device)
+
+    def release_device(self):
+        if self.is_auto_mode:
+            if self.image_predictor:
+                self.image_predictor.model.to(device="cpu")
+            if self.video_predictor:
+                self.video_predictor.to(device="cpu")
+
+    def predict(self, image, points, plabs, bbox, threshold):
+        if self.image_predictor is None:
+            self.image_predictor = SAM2ImagePredictor(build_sam2(self.config, self.modelname))
+
+        self.prepare_image_device()
+
+        self.image_predictor.set_image(image)
+
+        return sam_predict(self.image_predictor, points, plabs, bbox, threshold)
+
+    def predict_video_segs(self, image_frames, segs):
+        if self.video_predictor is None:
+            self.video_predictor = build_sam2_video_predictor(self.config, self.modelname)
+
+        self.prepare_video_device()
+
+        orig_video_height = image_frames.shape[1]
+        orig_video_width = image_frames.shape[2]
+
+        image_frames, padding = utils.resize_with_padding(image_frames, self.video_predictor.image_size, self.video_predictor.image_size)
+        image_frames = image_frames.permute(0, 3, 1, 2)
+
+        inference_state = {}
+        inference_state["images"] = image_frames
+        inference_state["num_frames"] = len(image_frames)
+        inference_state["video_height"] = self.video_predictor.image_size
+        inference_state["video_width"] = self.video_predictor.image_size
+        inference_state["offload_video_to_cpu"] = True
+        inference_state["offload_state_to_cpu"] = self.device_mode == "CPU"
+        inference_state["device"] = self.video_predictor.device
+
+        if inference_state["offload_state_to_cpu"]:
+            inference_state["storage_device"] = torch.device("cpu")
+        else:
+            inference_state["storage_device"] = self.video_predictor.device
+
+        inference_state["point_inputs_per_obj"] = {}
+        inference_state["mask_inputs_per_obj"] = {}
+        inference_state["cached_features"] = {}
+        inference_state["constants"] = {}
+
+        inference_state["obj_id_to_idx"] = OrderedDict()
+        inference_state["obj_idx_to_id"] = OrderedDict()
+        inference_state["obj_ids"] = []
+
+        inference_state["output_dict_per_obj"] = {}
+        inference_state["temp_output_dict_per_obj"] = {}
+        inference_state["frames_tracked_per_obj"] = {}
+        self.video_predictor._get_image_feature(inference_state, frame_idx=0, batch_size=1)
+
+        temp_masks = {}
+        for i in range(0, len(segs[1])):
+            bbox = segs[1][i].bbox
+
+            adjusted_bbox = utils.adjust_bbox_after_resize(
+                bbox,
+                (orig_video_height, orig_video_width),
+                (self.video_predictor.image_size, self.video_predictor.image_size),
+                padding
+            )
+
+            print(f"bbox={bbox} / adjusted_bbox={adjusted_bbox}")
+
+            points = [utils.center_of_bbox(adjusted_bbox)]
+            plabs = [1]
+            self.video_predictor.add_new_points_or_box(inference_state=inference_state, frame_idx=0, obj_id=i, points=points, labels=plabs, box=adjusted_bbox)
+            temp_masks[i] = []
+
+        for frame_idx, object_ids, masks in self.video_predictor.propagate_in_video(inference_state):
+            for i in object_ids:
+                m = masks[i]
+                m = m.permute(1, 2, 0)
+                temp_masks[i].append(m)
+
+        result = {}
+        for k, v in temp_masks.items():
+            m = torch.stack(v, dim=0)
+            m = utils.remove_padding(m, padding)
+            result[k] = utils.resize_with_padding(m, orig_video_width, orig_video_height)[0]
+
+        return result
+
 class ESAMWrapper:
     def __init__(self, model, device):
         self.model = model
@@ -630,10 +759,15 @@ class ESAMWrapper:
 def make_sam_mask(sam, segs, image, detection_hint, dilation,
                   threshold, bbox_expansion, mask_hint_threshold, mask_hint_use_negative):
 
-    if not hasattr(sam, 'sam_wrapper'):
+    if not hasattr(sam, 'sam_wrapper') and not isinstance(sam, SAM2Wrapper):
         raise Exception("[Impact Pack] Invalid SAMLoader is connected. Make sure 'SAMLoader (Impact)'.\nKnown issue: The ComfyUI-YOLO node overrides the SAMLoader (Impact), making it unusable. You need to uninstall ComfyUI-YOLO.\n\n\n")
 
-    sam_obj = sam.sam_wrapper
+
+    if isinstance(sam, SAM2Wrapper):
+        sam_obj = sam
+    else:
+        sam_obj = sam.sam_wrapper
+
     sam_obj.prepare_device()
 
     try:
@@ -651,7 +785,7 @@ def make_sam_mask(sam, segs, image, detection_hint, dilation,
 
             for i in range(len(segs)):
                 bbox = segs[i].bbox
-                center = center_of_bbox(segs[i].bbox)
+                center = utils.center_of_bbox(segs[i].bbox)
                 points.append(center)
 
                 # small point is background, big point is foreground
@@ -666,7 +800,7 @@ def make_sam_mask(sam, segs, image, detection_hint, dilation,
         else:
             for i in range(len(segs)):
                 bbox = segs[i].bbox
-                center = center_of_bbox(bbox)
+                center = utils.center_of_bbox(bbox)
 
                 x1 = max(bbox[0] - bbox_expansion, 0)
                 y1 = max(bbox[1] - bbox_expansion, 0)
@@ -712,7 +846,7 @@ def make_sam_mask(sam, segs, image, detection_hint, dilation,
                     plabs = [1, 1, 1, 1]
 
                 elif detection_hint == "mask-point-bbox":
-                    center = center_of_bbox(segs[i].bbox)
+                    center = utils.center_of_bbox(segs[i].bbox)
                     points.append(center)
                     plabs = [1]
 
@@ -733,14 +867,14 @@ def make_sam_mask(sam, segs, image, detection_hint, dilation,
                 total_masks += detected_masks
 
         # merge every collected masks
-        mask = combine_masks2(total_masks)
+        mask = utils.combine_masks2(total_masks)
 
     finally:
         sam_obj.release_device()
 
     if mask is not None:
         mask = mask.float()
-        mask = dilate_mask(mask.cpu().numpy(), dilation)
+        mask = utils.dilate_mask(mask.cpu().numpy(), dilation)
         mask = torch.from_numpy(mask)
     else:
         size = image.shape[0], image.shape[1]
@@ -791,7 +925,7 @@ def generate_detection_hints(image, seg, center, detection_hint, dilated_bbox, m
         plabs = [1, 1, 1, 1]
 
     elif detection_hint == "mask-point-bbox":
-        center = center_of_bbox(seg.bbox)
+        center = utils.center_of_bbox(seg.bbox)
         points.append(center)
         plabs = [1]
 
@@ -881,7 +1015,7 @@ def segs_scale_match(segs, target_shape):
             cropped_mask = cropped_mask.squeeze(0).squeeze(0).numpy()
 
         if cropped_image is not None:
-            cropped_image = tensor_resize(cropped_image if isinstance(cropped_image, torch.Tensor) else torch.from_numpy(cropped_image), new_w, new_h)
+            cropped_image = utils.tensor_resize(cropped_image if isinstance(cropped_image, torch.Tensor) else torch.from_numpy(cropped_image), new_w, new_h)
             cropped_image = cropped_image.numpy()
 
         new_seg = SEG(cropped_image, cropped_mask, seg.confidence, crop_region, bbox, seg.label, seg.control_net_wrapper)
@@ -921,7 +1055,7 @@ def make_sam_mask_segmented(sam, segs, image, detection_hint, dilation,
 
             for i in range(len(segs)):
                 bbox = segs[i].bbox
-                center = center_of_bbox(bbox)
+                center = utils.center_of_bbox(bbox)
                 points.append(center)
 
                 # small point is background, big point is foreground
@@ -936,7 +1070,7 @@ def make_sam_mask_segmented(sam, segs, image, detection_hint, dilation,
         else:
             for i in range(len(segs)):
                 bbox = segs[i].bbox
-                center = center_of_bbox(bbox)
+                center = utils.center_of_bbox(bbox)
                 x1 = max(bbox[0] - bbox_expansion, 0)
                 y1 = max(bbox[1] - bbox_expansion, 0)
                 x2 = min(bbox[2] + bbox_expansion, image.shape[1])
@@ -953,7 +1087,7 @@ def make_sam_mask_segmented(sam, segs, image, detection_hint, dilation,
                 total_masks += detected_masks
 
         # merge every collected masks
-        mask = combine_masks2(total_masks)
+        mask = utils.combine_masks2(total_masks)
 
     finally:
         sam_obj.release_device()
@@ -962,7 +1096,7 @@ def make_sam_mask_segmented(sam, segs, image, detection_hint, dilation,
 
     if mask is not None:
         mask = mask.float()
-        mask = dilate_mask(mask.cpu().numpy(), dilation)
+        mask = utils.dilate_mask(mask.cpu().numpy(), dilation)
         mask = torch.from_numpy(mask)
         mask = mask.to(device=mask_working_device)
     else:
@@ -979,7 +1113,7 @@ def make_sam_mask_segmented(sam, segs, image, detection_hint, dilation,
 
 
 def segs_bitwise_and_mask(segs, mask):
-    mask = make_2d_mask(mask)
+    mask = utils.make_2d_mask(mask)
 
     if mask is None:
         print("[SegsBitwiseAndMask] Cannot operate: MASK is empty.")
@@ -1005,7 +1139,7 @@ def segs_bitwise_and_mask(segs, mask):
 
 
 def segs_bitwise_subtract_mask(segs, mask):
-    mask = make_2d_mask(mask)
+    mask = utils.make_2d_mask(mask)
 
     if mask is None:
         print("[SegsBitwiseSubtractMask] Cannot operate: MASK is empty.")
@@ -1061,7 +1195,7 @@ def dilate_segs(segs, factor):
 
     new_segs = []
     for seg in segs[1]:
-        new_mask = dilate_mask(seg.cropped_mask, factor)
+        new_mask = utils.dilate_mask(seg.cropped_mask, factor)
         new_seg = SEG(seg.cropped_image, new_mask, seg.confidence, seg.crop_region, seg.bbox, seg.label, seg.control_net_wrapper)
         new_segs.append(new_seg)
 
@@ -1077,7 +1211,7 @@ class ONNXDetector:
     def detect(self, image, threshold, dilation, crop_factor, drop_size=1, detailer_hook=None):
         drop_size = max(drop_size, 1)
         try:
-            import impact.onnx as onnx
+            import impact.impact_onnx as onnx
 
             h = image.shape[1]
             w = image.shape[2]
@@ -1093,7 +1227,7 @@ class ONNXDetector:
                     x1, y1, x2, y2 = item_bbox
 
                     if x2 - x1 > drop_size and y2 - y1 > drop_size:  # minimum dimension must be (2,2) to avoid squeeze issue
-                        crop_region = make_crop_region(w, h, item_bbox, crop_factor)
+                        crop_region = utils.make_crop_region(w, h, item_bbox, crop_factor)
 
                         if detailer_hook is not None:
                             crop_region = item_bbox.post_crop_region(w, h, item_bbox, crop_region)
@@ -1103,7 +1237,7 @@ class ONNXDetector:
                         # prepare cropped mask
                         cropped_mask = np.zeros((crop_y2 - crop_y1, crop_x2 - crop_x1))
                         cropped_mask[y1 - crop_y1:y2 - crop_y1, x1 - crop_x1:x2 - crop_x1] = 1
-                        cropped_mask = dilate_mask(cropped_mask, dilation)
+                        cropped_mask = utils.dilate_mask(cropped_mask, dilation)
 
                         # make items. just convert the integer label to a string
                         item = SEG(None, cropped_mask, scores[i], crop_region, item_bbox, str(labels[i]), None)
@@ -1178,7 +1312,7 @@ def mask_to_segs(mask, combined, crop_factor, bbox_fill, drop_size=1, label='A',
                     np.max(indices[1]),
                     np.max(indices[0]),
                 )
-                crop_region = make_crop_region(
+                crop_region = utils.make_crop_region(
                     mask_i.shape[1], mask_i.shape[0], bbox, crop_factor
                 )
                 x1, y1, x2, y2 = crop_region
@@ -1212,7 +1346,7 @@ def mask_to_segs(mask, combined, crop_factor, bbox_fill, drop_size=1, label='A',
 
                 x, y, w, h = cv2.boundingRect(contour)
                 bbox = x, y, x + w, y + h
-                crop_region = make_crop_region(
+                crop_region = utils.make_crop_region(
                     mask_i.shape[1], mask_i.shape[0], bbox, crop_factor, crop_min_size
                 )
 
@@ -1286,7 +1420,7 @@ def mediapipe_facemesh_to_segs(image, crop_factor, bbox_fill, crop_min_size, dro
                 tensor = torch.from_numpy(convex_segment)
                 mask_tensor = torch.any(tensor != 0, dim=-1).float()
                 mask_tensor = mask_tensor.squeeze(0)
-                mask_tensor = torch.from_numpy(dilate_mask(mask_tensor.numpy(), dilation))
+                mask_tensor = torch.from_numpy(utils.dilate_mask(mask_tensor.numpy(), dilation))
                 mask_list.append(mask_tensor.unsqueeze(0))
 
         return mask_list
@@ -1521,7 +1655,7 @@ class TwoSamplersForMaskUpscaler:
                  hook_full_opt=None,
                  tile_size=512):
 
-        mask = make_2d_mask(mask)
+        mask = utils.make_2d_mask(mask)
 
         mask = mask.reshape((-1, 1, mask.shape[-2], mask.shape[-1]))
 
@@ -1539,7 +1673,7 @@ class TwoSamplersForMaskUpscaler:
     def upscale(self, step_info, samples, upscale_factor, save_temp_prefix=None):
         scale_method, sample_schedule, use_tiled_vae, base_sampler, mask_sampler, mask, vae = self.params
 
-        mask = make_2d_mask(mask)
+        mask = utils.make_2d_mask(mask)
 
         self.prepare_hook(step_info)
 
@@ -1569,7 +1703,7 @@ class TwoSamplersForMaskUpscaler:
     def upscale_shape(self, step_info, samples, w, h, save_temp_prefix=None):
         scale_method, sample_schedule, use_tiled_vae, base_sampler, mask_sampler, mask, vae = self.params
 
-        mask = make_2d_mask(mask)
+        mask = utils.make_2d_mask(mask)
 
         self.prepare_hook(step_info)
 
@@ -1625,7 +1759,7 @@ class TwoSamplersForMaskUpscaler:
             return cur_step % 2 == 0 or cur_step >= total_step - 1
 
     def do_samples(self, step_info, base_sampler, mask_sampler, sample_schedule, mask, upscaled_latent):
-        mask = make_2d_mask(mask)
+        mask = utils.make_2d_mask(mask)
 
         if self.is_full_sample_time(step_info, sample_schedule):
             print(f"step_info={step_info} / full time")
@@ -2069,7 +2203,7 @@ class BBoxDetectorBasedOnCLIPSeg:
     def detect(self, image, bbox_threshold, bbox_dilation, bbox_crop_factor, drop_size=1, detailer_hook=None):
         mask = self.detect_combined(image, bbox_threshold, bbox_dilation)
 
-        mask = make_2d_mask(mask)
+        mask = utils.make_2d_mask(mask)
 
         segs = mask_to_segs(mask, False, bbox_crop_factor, True, drop_size, detailer_hook=detailer_hook)
 
@@ -2099,7 +2233,7 @@ class BBoxDetectorBasedOnCLIPSeg:
         prompt = self.aux if self.prompt == '' and self.aux is not None else self.prompt
 
         mask, _, _ = CLIPSeg().segment_image(image, prompt, self.blur, threshold, dilation_factor)
-        mask = to_binary_mask(mask)
+        mask = utils.to_binary_mask(mask)
         return mask
 
     def setAux(self, x):

modules/impact/detectors.py

@@ -1,3 +1,5 @@
+import logging
+
 import impact.core as core
 from nodes import MAX_RESOLUTION
 import impact.segs_nodes as segs_nodes
@@ -163,7 +165,7 @@ class SegmDetectorCombined:
         mask = segm_detector.detect_combined(image, threshold, dilation)
 
         if mask is None:
-            mask = torch.zeros((image.shape[2], image.shape[1]), dtype=torch.float32, device="cpu")
+            mask = torch.zeros((image.shape[1], image.shape[2]), dtype=torch.float32, device="cpu")
 
         return (mask.unsqueeze(0),)
 
@@ -183,7 +185,7 @@ class BboxDetectorCombined(SegmDetectorCombined):
         mask = bbox_detector.detect_combined(image, threshold, dilation)
 
         if mask is None:
-            mask = torch.zeros((image.shape[2], image.shape[1]), dtype=torch.float32, device="cpu")
+            mask = torch.zeros((image.shape[1], image.shape[2]), dtype=torch.float32, device="cpu")
 
         return (mask.unsqueeze(0),)
 
@@ -298,6 +300,71 @@ class SimpleDetectorForEachPipe:
                                             sam_mask_hint_threshold, post_dilation=post_dilation, sam_model_opt=sam_model_opt, segm_detector_opt=segm_detector_opt,
                                             detailer_hook=detailer_hook)
 
+class SAM2VideoDetectorSEGS:
+    @classmethod
+    def INPUT_TYPES(s):
+        return {"required": {
+            "image_frames": ("IMAGE", ),
+
+            "bbox_detector": ("BBOX_DETECTOR", ),
+            "sam2_model": ("SAM_MODEL", ),
+
+            "bbox_threshold": ("FLOAT", {"default": 0.5, "min": 0.0, "max": 1.0, "step": 0.01}),
+            "sam2_threshold": ("FLOAT", {"default": 0.5, "min": 0.0, "max": 1.0, "step": 0.01}),
+
+            "crop_factor": ("FLOAT", {"default": 3.0, "min": 1.0, "max": 100, "step": 0.1}),
+            "drop_size": ("INT", {"min": 1, "max": MAX_RESOLUTION, "step": 1, "default": 10}),
+            }
+        }
+
+    RETURN_TYPES = ("SEGS", )
+    FUNCTION = "doit"
+
+    CATEGORY = "ImpactPack/Detector"
+
+    @staticmethod
+    def doit(bbox_detector, sam2_model, image_frames, bbox_threshold, sam2_threshold, crop_factor, drop_size):
+        if not isinstance(sam2_model, core.SAM2Wrapper):
+            logging.error("[Impact Pack] To use the SAM2VideoDetectorSEGS node, a SAM2 model must be provided as input to `sam2_model`.")
+            raise Exception("To use the SAM2VideoDetectorSEGS node, a SAM2 model must be provided as input to `sam2_model`.")
+
+        segs = bbox_detector.detect(image_frames[0].unsqueeze(0), bbox_threshold, 0, 0, drop_size)
+        segs_masks = sam2_model.predict_video_segs(image_frames, segs)
+
+        def get_whole_merged_mask(all_masks):
+            merged_mask = (all_masks[0] * 255).to(torch.uint8)
+            for mask in all_masks[1:]:
+                merged_mask |= (mask * 255).to(torch.uint8)
+
+            merged_mask = (merged_mask / 255.0).to(torch.float32)
+            merged_mask = utils.to_binary_mask(merged_mask, 0.1)[0]
+            return merged_mask
+
+        test_mask1 = None
+        test_mask2 = None
+        new_segs = []
+        for k, v in segs_masks.items():
+            v = v.squeeze(3)
+            m = get_whole_merged_mask(v)
+            test_mask2 = v
+            seg = segs_nodes.MaskToSEGS.doit(m, False, crop_factor, False, drop_size, contour_fill=True)[0][1]
+
+            if len(seg) == 0:
+                continue
+
+            seg = seg[0]
+
+            x1, y1, x2, y2 = seg.crop_region
+            masks = []
+            for mask in v:
+                masks.append(mask[y1:y2, x1:x2])
+            cropped_mask = torch.stack(masks)
+            cropped_mask = (cropped_mask >= (sam2_threshold*100-50)).to(torch.uint8).cpu()
+            new_seg = SEG(seg.cropped_image, cropped_mask, seg.confidence, seg.crop_region, seg.bbox, seg.label, seg.control_net_wrapper)
+            new_segs.append(new_seg)
+
+        return ((segs[0], new_segs), )
+
 
 class SimpleDetectorForAnimateDiff:
     @classmethod

modules/impact/hook_nodes.py

@@ -1,6 +1,8 @@
 import sys
 from . import hooks
 from . import defs
+from . import utils
+import nodes
 
 
 class SEGSOrderedFilterDetailerHookProvider:
@@ -83,3 +85,24 @@ class PreviewDetailerHookProvider:
     def doit(self, quality, unique_id):
         hook = hooks.PreviewDetailerHook(unique_id, quality)
         return hook, hook
+
+
+class LamaRemoverDetailerHookProvider:
+    @classmethod
+    def INPUT_TYPES(s):
+        return {
+            "required": {
+                "mask_threshold":("INT", {"default": 250, "min": 0, "max": 255, "step": 1, "display": "slider"}),
+                "gaussblur_radius": ("INT", {"default": 8, "min": 0, "max": 20, "step": 1, "display": "slider"}),
+                "skip_sampling": ("BOOLEAN", {"default": True}),
+            }
+        }
+
+    RETURN_TYPES = ("DETAILER_HOOK", )
+    FUNCTION = "doit"
+
+    CATEGORY = "ImpactPack/Util"
+
+    def doit(self, mask_threshold, gaussblur_radius, skip_sampling):
+        hook = hooks.LamaRemoverDetailerHook(mask_threshold, gaussblur_radius, skip_sampling)
+        return (hook, )

modules/impact/hooks.py

@@ -25,7 +25,7 @@ class PixelKSampleHook:
     def post_decode(self, pixels):
         return pixels
 
-    def post_upscale(self, pixels):
+    def post_upscale(self, pixels, mask=None):
         return pixels
 
     def post_encode(self, samples):
@@ -64,8 +64,8 @@ class PixelKSampleHookCombine(PixelKSampleHook):
     def post_decode(self, pixels):
         return self.hook2.post_decode(self.hook1.post_decode(pixels))
 
-    def post_upscale(self, pixels):
-        return self.hook2.post_upscale(self.hook1.post_upscale(pixels))
+    def post_upscale(self, pixels, mask=None):
+        return self.hook2.post_upscale(self.hook1.post_upscale(pixels, mask), mask)
 
     def post_encode(self, samples):
         return self.hook2.post_encode(self.hook1.post_encode(samples))
@@ -109,6 +109,15 @@ class DetailerHookCombine(PixelKSampleHookCombine):
         noise_2nd, is_touched = self.hook2.get_custom_noise(seed, noise, is_touched)
         return noise, is_touched
 
+    def get_custom_sampler(self):
+        if self.hook1.get_custom_sampler() is not None:
+            return self.hook1.get_custom_sampler()
+        else:
+            return self.hook2.get_custom_sampler()
+
+    def get_skip_sampling(self):
+        return self.hook1.get_skip_sampling() and self.hook2.get_skip_sampling()
+
 
 class SimpleCfgScheduleHook(PixelKSampleHook):
     target_cfg = 0
@@ -173,6 +182,21 @@ class DetailerHook(PixelKSampleHook):
     def get_custom_noise(self, seed, noise, is_touched):
         return noise, is_touched
 
+    def get_custom_sampler(self):
+        return None
+
+    def get_skip_sampling(self):
+        return False
+
+
+class CustomSamplerDetailerHookProvider(DetailerHook):
+    def __init__(self, sampler):
+        super().__init__()
+        self.sampler = sampler
+
+    def get_custom_sampler(self):
+        return self.sampler
+
 
 # class CustomNoiseDetailerHookProvider(DetailerHook):
 #     def __init__(self, noise):
@@ -486,6 +510,27 @@ class SEGSLabelFilterDetailerHook(DetailerHook):
         return segs_nodes.SEGSLabelFilter().doit(segs, "", self.labels)[0]
 
 
+class LamaRemoverDetailerHook(DetailerHook):
+    def __init__(self, mask_threshold, gaussblur_radius, skip_sampling):
+        super().__init__()
+        self.mask_threshold = mask_threshold
+        self.gaussblur_radius = gaussblur_radius
+        self.skip_sampling = skip_sampling
+
+    def post_upscale(self, img, mask=None):
+        if "LamaRemover" in nodes.NODE_CLASS_MAPPINGS:
+            lama_remover_obj = nodes.NODE_CLASS_MAPPINGS['LamaRemover']()
+        else:
+            utils.try_install_custom_node('https://github.com/Layer-norm/comfyui-lama-remover',
+                                          "To use 'LAMARemoverDetailerHookProvider', 'comfyui-lama-remover' nodepack is required.")
+            raise Exception("'LamaRemover' node is not installed.")
+
+        return lama_remover_obj.lama_remover(img, masks=mask, mask_threshold=self.mask_threshold, gaussblur_radius=self.gaussblur_radius, invert_mask=False)[0]
+
+    def get_skip_sampling(self):
+        return self.skip_sampling
+
+
 class PreviewDetailerHook(DetailerHook):
     def __init__(self, node_id, quality):
         super().__init__()

modules/impact/impact_pack.py

@@ -92,10 +92,21 @@ class CLIPSegDetectorProvider:
             print("[ERROR] CLIPSegToBboxDetector: CLIPSeg custom node isn't installed. You must install biegert/ComfyUI-CLIPSeg extension to use this node.")
 
 
+sam2_config_table = {
+    'sam2.1_hiera_base_plus.pt': 'configs/sam2.1/sam2.1_hiera_b+.yaml',
+    'sam2.1_hiera_large.pt': 'configs/sam2.1/sam2.1_hiera_l.yaml',
+    'sam2.1_hiera_small.pt': 'configs/sam2.1/sam2.1_hiera_s.yaml',
+    'sam2.1_hiera_tiny.pt': 'configs/sam2.1/sam2.1_hiera_t.yaml',
+    'sam2_hiera_tiny.pt': 'configs/sam2/sam2_hiera_t.yaml',
+    'sam2_hiera_small.pt': 'configs/sam2/sam2_hiera_s.yaml',
+    'sam2_hiera_base_plus.pt': 'configs/sam2/sam2_hiera_b+.yaml',
+    'sam2_hiera_large.pt': 'configs/sam2/sam2_hiera_l.yaml'
+}
+
 class SAMLoader:
     @classmethod
     def INPUT_TYPES(cls):
-        models = [x for x in folder_paths.get_filename_list("sams") if 'hq' not in x]
+        models = [x for x in folder_paths.get_filename_list("sams") if 'hq' not in x and (x.endswith('.pt') or x.endswith('.pth') or x.endswith('.safetensors'))]
 
         if 'ESAM_ModelLoader_Zho' in nodes.NODE_CLASS_MAPPINGS:
             models.append('ESAM')
@@ -136,17 +147,22 @@ class SAMLoader:
             
             print(f"Loads EfficientSAM model: (device:{device_mode})")
             return (esam, )
-
-        modelname = folder_paths.get_full_path("sams", model_name)
-
-        if 'vit_h' in model_name:
-            model_kind = 'vit_h'
-        elif 'vit_l' in model_name:
-            model_kind = 'vit_l'
+        elif model_name in sam2_config_table:
+            model_kind = 'sam2'
+            config = sam2_config_table[model_name]
+            modelname = folder_paths.get_full_path("sams", model_name)
         else:
-            model_kind = 'vit_b'
+            modelname = folder_paths.get_full_path("sams", model_name)
+
+            if 'vit_h' in model_name:
+                model_kind = 'vit_h'
+            elif 'vit_l' in model_name:
+                model_kind = 'vit_l'
+            else:
+                model_kind = 'vit_b'
+
+            sam = sam_model_registry[model_kind](checkpoint=modelname)
 
-        sam = sam_model_registry[model_kind](checkpoint=modelname)
         size = os.path.getsize(modelname)
         safe_to = core.SafeToGPU(size)
 
@@ -158,10 +174,14 @@ class SAMLoader:
 
         is_auto_mode = device_mode == "AUTO"
 
-        sam_obj = core.SAMWrapper(sam, is_auto_mode=is_auto_mode, safe_to_gpu=safe_to)
-        sam.sam_wrapper = sam_obj
+        if model_kind == 'sam2':
+            sam = core.SAM2Wrapper(config=config, modelname=modelname, is_auto_mode=is_auto_mode, safe_to_gpu=safe_to, device_mode=device_mode)
+            print(f"Loads SAM2 model: {modelname} (device:{device_mode})")
+        else:
+            sam_obj = core.SAMWrapper(sam, is_auto_mode=is_auto_mode, safe_to_gpu=safe_to)
+            sam.sam_wrapper = sam_obj
+            print(f"Loads SAM model: {modelname} (device:{device_mode})")
 
-        print(f"Loads SAM model: {modelname} (device:{device_mode})")
         return (sam, )
 
 
@@ -811,6 +831,26 @@ class CoreMLDetailerHookProvider:
         return (hook, )
 
 
+class CustomSamplerDetailerHookProvider:
+    @classmethod
+    def INPUT_TYPES(s):
+        return {"required": {
+                    "sampler": ("SAMPLER", ),
+                    },
+                }
+
+    RETURN_TYPES = ("DETAILER_HOOK",)
+    FUNCTION = "doit"
+
+    CATEGORY = "ImpactPack/Detailer"
+    
+    DESCRIPTION = "Apply a hook that allows you to use a custom sampler in the Detailer nodes. When using `DetailerHookCombine`, the sampler from the first hook is applied."
+
+    def doit(self, sampler):
+        hook = hooks.CustomSamplerDetailerHookProvider(sampler)
+        return (hook, )
+
+
 class CfgScheduleHookProvider:
     schedules = ["simple"]
 
@@ -1906,7 +1946,7 @@ class MaskRectArea:
         # search for node
         node_found = False
         for node in extra_pnginfo["workflow"]["nodes"]:
-            if node["id"] == int(unique_id):
+            if str(node["id"]) == unique_id:
                 min_x = node["properties"].get("x", 0) / 100
                 min_y = node["properties"].get("y", 0) / 100
                 width = node["properties"].get("w", 0) / 100

modules/impact/impact_sampling.py

@@ -194,7 +194,7 @@ def impact_sample(model, seed, steps, cfg, sampler_name, scheduler, positive, ne
 
 
 def ksampler_wrapper(model, seed, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, denoise,
-                     refiner_ratio=None, refiner_model=None, refiner_clip=None, refiner_positive=None, refiner_negative=None, sigma_factor=1.0, noise=None, scheduler_func=None):
+                     refiner_ratio=None, refiner_model=None, refiner_clip=None, refiner_positive=None, refiner_negative=None, sigma_factor=1.0, noise=None, scheduler_func=None, sampler_opt=None):
 
     if refiner_ratio is None or refiner_model is None or refiner_clip is None or refiner_positive is None or refiner_negative is None:
         # Use separated_sample instead of KSampler for `AYS scheduler`
@@ -206,7 +206,7 @@ def ksampler_wrapper(model, seed, steps, cfg, sampler_name, scheduler, positive,
 
         refined_latent = separated_sample(model, True, seed, advanced_steps, cfg, sampler_name, scheduler,
                                           positive, negative, latent_image, start_at_step, end_at_step, False,
-                                          sigma_ratio=sigma_factor, noise=noise, scheduler_func=scheduler_func)
+                                          sigma_ratio=sigma_factor, sampler_opt=sampler_opt, noise=noise, scheduler_func=scheduler_func)
     else:
         advanced_steps = math.floor(steps / denoise)
         start_at_step = advanced_steps - steps
@@ -215,7 +215,7 @@ def ksampler_wrapper(model, seed, steps, cfg, sampler_name, scheduler, positive,
         # print(f"pre: {start_at_step} .. {end_at_step} / {advanced_steps}")
         temp_latent = separated_sample(model, True, seed, advanced_steps, cfg, sampler_name, scheduler,
                                        positive, negative, latent_image, start_at_step, end_at_step, True,
-                                       sigma_ratio=sigma_factor, noise=noise, scheduler_func=scheduler_func)
+                                       sigma_ratio=sigma_factor, sampler_opt=sampler_opt, noise=noise, scheduler_func=scheduler_func)
 
         if 'noise_mask' in latent_image:
             # noise_latent = \
@@ -229,7 +229,7 @@ def ksampler_wrapper(model, seed, steps, cfg, sampler_name, scheduler, positive,
         # print(f"post: {end_at_step} .. {advanced_steps + 1} / {advanced_steps}")
         refined_latent = separated_sample(refiner_model, False, seed, advanced_steps, cfg, sampler_name, scheduler,
                                           refiner_positive, refiner_negative, temp_latent, end_at_step, advanced_steps + 1, False,
-                                          sigma_ratio=sigma_factor, scheduler_func=scheduler_func)
+                                          sigma_ratio=sigma_factor, sampler_opt=sampler_opt, scheduler_func=scheduler_func)
 
     return refined_latent
 

modules/impact/impact_server.py

@@ -17,7 +17,6 @@ import numpy as np
 import nodes
 from PIL import Image
 import io
-import impact.wildcards as wildcards
 import comfy
 from io import BytesIO
 import random
@@ -183,7 +182,7 @@ async def wildcards_list(request):
 @PromptServer.instance.routes.post("/impact/wildcards")
 async def populate_wildcards(request):
     data = await request.json()
-    populated = wildcards.process(data['text'], data.get('seed', None))
+    populated = impact.wildcards.process(data['text'], data.get('seed', None))
     return web.json_response({"text": populated})
 
 
@@ -512,7 +511,7 @@ def onprompt_populate_wildcards(json_data):
                 else:
                     input_seed = int(inputs['seed'])
 
-                inputs['populated_text'] = wildcards.process(inputs['wildcard_text'], input_seed)
+                inputs['populated_text'] = impact.wildcards.process(inputs['wildcard_text'], input_seed)
                 inputs['mode'] = 'reproduce'
 
                 PromptServer.instance.send_sync("impact-node-feedback", {"node_id": k, "widget_name": "populated_text", "type": "STRING", "value": inputs['populated_text']})

modules/impact/logics.py

@@ -8,7 +8,7 @@ from impact.utils import any_typ
 import impact.core as core
 import re
 import nodes
-import traceback
+
 
 class ImpactCompare:
     @classmethod
@@ -574,27 +574,6 @@ class ImpactSleep:
         return (signal,)
 
 
-error_skip_flag = False
-try:
-    import cm_global
-    def filter_message(str):
-        global error_skip_flag
-
-        if "IMPACT-PACK-SIGNAL: STOP CONTROL BRIDGE" in str:
-            return True
-        elif error_skip_flag and "ERROR:root:!!! Exception during processing !!!\n" == str:
-            error_skip_flag = False
-            return True
-        else:
-            return False
-
-    cm_global.try_call(api='cm.register_message_collapse', f=filter_message)
-
-except Exception as e:
-    print(f"[WARN] ComfyUI-Impact-Pack: `ComfyUI` or `ComfyUI-Manager` is an outdated version.")
-    pass
-
-
 def workflow_to_map(workflow):
     nodes = {}
     links = {}

modules/impact/util_nodes.py

@@ -446,6 +446,31 @@ class MakeMaskList:
         return (masks, )
 
 
+class NthItemOfAnyList:
+    @classmethod
+    def INPUT_TYPES(s):
+        return {"required":  {
+                    "any_list": (any_typ,),
+                    "index": ("INT", {"default": 0, "min": 0, "max": sys.maxsize, "step": 1, "tooltip": "The index of the item you want to select from the list."}),
+                    }
+        }
+
+    RETURN_TYPES = (any_typ,)
+    INPUT_IS_LIST = True
+    FUNCTION = "doit"
+
+    CATEGORY = "ImpactPack/Util"
+
+    DESCRIPTION = "Selects the Nth item from a list. If the index is out of range, it returns the last item in the list."
+
+    def doit(self, any_list, index):
+        i = index[0]
+        if i >= len(any_list):
+            return (any_list[-1],)
+        else:
+            return (any_list[i],)
+
+
 class MakeImageList:
     @classmethod
     def INPUT_TYPES(s):

modules/impact/utils.py

@@ -67,6 +67,54 @@ def tensor_convert_rgb(image, prefer_copy=True):
     raise ValueError(f"illegal conversion (channels: {n_channel} -> 3)")
 
 
+def resize_with_padding(image, target_w: int, target_h: int):
+    _tensor_check_image(image)
+    b, h, w, c = image.shape
+    image = image.permute(0, 3, 1, 2)  # B, C, H, W
+
+    scale = min(target_w / w, target_h / h)
+    new_w, new_h = int(w * scale), int(h * scale)
+
+    image = F.interpolate(image, size=(new_h, new_w), mode="bilinear", align_corners=False)
+
+    pad_left = (target_w - new_w) // 2
+    pad_right = target_w - new_w - pad_left
+    pad_top = (target_h - new_h) // 2
+    pad_bottom = target_h - new_h - pad_top
+
+    image = F.pad(image, (pad_left, pad_right, pad_top, pad_bottom), mode='constant', value=0)
+
+    image = image.permute(0, 2, 3, 1)  # B, H, W, C
+    return image, (pad_top, pad_bottom, pad_left, pad_right)
+
+
+def remove_padding(image, padding):
+    pad_top, pad_bottom, pad_left, pad_right = padding
+    return image[:, pad_top:image.shape[1] - pad_bottom, pad_left:image.shape[2] - pad_right, :]
+
+
+def adjust_bbox_after_resize(bbox, original_size, target_size, padding):
+    """
+    bbox: (x1, y1, x2, y2) in original image
+    original_size: (original_h, original_w)
+    target_size: (target_h, target_w)
+    padding: (pad_top, pad_bottom, pad_left, pad_right)
+    """
+    orig_h, orig_w = original_size
+    target_h, target_w = target_size
+    pad_top, pad_bottom, pad_left, pad_right = padding
+
+    scale = min(target_w / orig_w, target_h / orig_h)
+
+    # Apply scale
+    x1 = int(bbox[0] * scale + pad_left)
+    y1 = int(bbox[1] * scale + pad_top)
+    x2 = int(bbox[2] * scale + pad_left)
+    y2 = int(bbox[3] * scale + pad_top)
+
+    return x1, y1, x2, y2
+
+
 def general_tensor_resize(image, w: int, h: int):
     _tensor_check_image(image)
     image = image.permute(0, 3, 1, 2)
@@ -178,31 +226,71 @@ def tensor2numpy(image):
 
 
 def tensor_paste(image1, image2, left_top, mask):
-    """Mask and image2 has to be the same size"""
+    """
+    Pastes image2 onto image1 at position left_top using mask.
+    Supports both RGB and RGBA images.
+    """
     _tensor_check_image(image1)
     _tensor_check_image(image2)
     _tensor_check_mask(mask)
+    
     if image2.shape[1:3] != mask.shape[1:3]:
         mask = resize_mask(mask.squeeze(dim=3), image2.shape[1:3]).unsqueeze(dim=3)
-        # raise ValueError(f"Inconsistent size: Image ({image2.shape[1:3]}) != Mask ({mask.shape[1:3]})")
-
+    
     x, y = left_top
-    _, h1, w1, _ = image1.shape
-    _, h2, w2, _ = image2.shape
-
-    # calculate image patch size
+    _, h1, w1, c1 = image1.shape
+    _, h2, w2, c2 = image2.shape
+    
+    # Calculate image patch size
     w = min(w1, x + w2) - x
     h = min(h1, y + h2) - y
-
+    
     # If the patch is out of bound, nothing to do!
     if w <= 0 or h <= 0:
         return
-
+    
     mask = mask[:, :h, :w, :]
-    image1[:, y:y+h, x:x+w, :] = (
-        (1 - mask) * image1[:, y:y+h, x:x+w, :] +
-        mask * image2[:, :h, :w, :]
-    )
+    
+    # Get the region to be modified
+    region1 = image1[:, y:y+h, x:x+w, :]
+    region2 = image2[:, :h, :w, :]
+    
+    # Handle RGB and RGBA cases
+    if c1 == 3 and c2 == 3:
+        # Both RGB - simple case
+        image1[:, y:y+h, x:x+w, :] = (1 - mask) * region1 + mask * region2
+    
+    elif c1 == 4 and c2 == 4:
+        # Both RGBA - need to handle alpha channel separately
+        # RGB channels
+        image1[:, y:y+h, x:x+w, :3] = (
+            (1 - mask) * region1[:, :, :, :3] +
+            mask * region2[:, :, :, :3]
+        )
+        
+        # Alpha channel - use "over" composition
+        a1 = region1[:, :, :, 3:4]
+        a2 = region2[:, :, :, 3:4] * mask
+        new_alpha = a1 + a2 * (1 - a1)
+        image1[:, y:y+h, x:x+w, 3:4] = new_alpha
+    
+    elif c1 == 4 and c2 == 3:
+        # Target is RGBA, source is RGB - assume source is fully opaque
+        image1[:, y:y+h, x:x+w, :3] = (
+            (1 - mask) * region1[:, :, :, :3] +
+            mask * region2
+        )
+        # Alpha channel - reduce alpha where mask is applied
+        image1[:, y:y+h, x:x+w, 3:4] = region1[:, :, :, 3:4] * (1 - mask) + mask
+    
+    elif c1 == 3 and c2 == 4:
+        # Target is RGB, source is RGBA - apply source alpha to mask
+        effective_mask = mask * region2[:, :, :, 3:4]
+        image1[:, y:y+h, x:x+w, :] = (
+            (1 - effective_mask) * region1 +
+            effective_mask * region2[:, :, :, :3]
+        )
+    
     return
 
 

modules/impact/wildcards.py

@@ -8,6 +8,7 @@ import numpy as np
 import threading
 from impact import utils
 from impact import config
+import logging
 
 
 wildcards_path = os.path.abspath(os.path.join(os.path.dirname(__file__), "..", "..", "wildcards"))
@@ -65,7 +66,7 @@ def read_wildcard_dict(wildcard_path):
                     with open(file_path, 'r', encoding="UTF-8", errors="ignore") as f:
                         lines = f.read().splitlines()
                         wildcard_dict[key] = [x for x in lines if not x.strip().startswith('#')]
-            elif file.endswith('.yaml'):
+            elif file.endswith('.yaml') or file.endswith('.yml'):
                 file_path = os.path.join(root, file)
 
                 try:
@@ -211,7 +212,7 @@ def process(text, seed=None):
                 selected_items = random_gen.choice(options, p=normalized_probabilities, size=select_count, replace=False)
 
             # x may be numpy.int32, convert to string
-            selected_items2 = [re.sub(r'^\s*[0-9.]+::', '', str(x), 1) for x in selected_items]
+            selected_items2 = [re.sub(r'^\s*[0-9.]+::', '', str(x), count=1) for x in selected_items]
             replacement = select_sep.join(selected_items2)
             if '::' in replacement:
                 pass
@@ -219,7 +220,7 @@ def process(text, seed=None):
             replacements_found = True
             return replacement
 
-        pattern = r'{([^{}]*?)}'
+        pattern = r'(?<!\\)\{((?:[^{}]|(?<=\\)[{}])*?)(?<!\\)\}'
         replaced_string = re.sub(pattern, replace_option, string)
 
         return replaced_string, replacements_found
@@ -278,7 +279,7 @@ def process(text, seed=None):
 
                 normalized_probabilities = [prob / total_prob for prob in adjusted_probabilities]
                 selected_item = random_gen.choice(options, p=normalized_probabilities, replace=False)
-                replacement = re.sub(r'^\s*[0-9.]+::', '', selected_item, 1)
+                replacement = re.sub(r'^\s*[0-9.]+::', '', selected_item, count=1)
                 replacements_found = True
                 string = string.replace(f"__{match}__", replacement, 1)
             elif '*' in keyword:
@@ -358,6 +359,7 @@ def extract_lora_values(string):
         lbw = None
         lbw_a = None
         lbw_b = None
+        loader = None
 
         if len(item) > 0:
             lora = item[0]
@@ -376,6 +378,8 @@ def extract_lora_values(string):
                             lbw_b = safe_float(lbw_item[2:].strip())
                         elif lbw_item.strip() != '':
                             lbw = lbw_item
+                elif sub_item.startswith("LOADER="):
+                    loader = sub_item[7:]
 
         if a is None:
             a = 1.0
@@ -383,7 +387,7 @@ def extract_lora_values(string):
             b = a
 
         if lora is not None and lora not in added:
-            result.append((lora, a, b, lbw, lbw_a, lbw_b))
+            result.append((lora, a, b, lbw, lbw_a, lbw_b, loader))
             added.add(lora)
 
     return result
@@ -407,6 +411,8 @@ def resolve_lora_name(lora_name_cache, name):
             if x.endswith(name):
                 return x
 
+    return None
+
 
 def process_with_loras(wildcard_opt, model, clip, clip_encoder=None, seed=None, processed=None):
     """
@@ -427,7 +433,7 @@ def process_with_loras(wildcard_opt, model, clip, clip_encoder=None, seed=None,
     loras = extract_lora_values(pass1)
     pass2 = remove_lora_tags(pass1)
 
-    for lora_name, model_weight, clip_weight, lbw, lbw_a, lbw_b in loras:
+    for lora_name, model_weight, clip_weight, lbw, lbw_a, lbw_b, loader in loras:
         lora_name_ext = lora_name.split('.')
         if ('.'+lora_name_ext[-1]) not in folder_paths.supported_pt_extensions:
             lora_name = lora_name+".safetensors"
@@ -441,26 +447,36 @@ def process_with_loras(wildcard_opt, model, clip, clip_encoder=None, seed=None,
             path = None
 
         if path is not None:
-            print(f"LOAD LORA: {lora_name}: {model_weight}, {clip_weight}, LBW={lbw}, A={lbw_a}, B={lbw_b}")
+            logging.info(f"LOAD LORA: {lora_name}: {model_weight}, {clip_weight}, LBW={lbw}, A={lbw_a}, B={lbw_b}, LOADER={loader}")
 
-            def default_lora():
-                return nodes.LoraLoader().load_lora(model, clip, lora_name, model_weight, clip_weight)
-
-            if lbw is not None:
-                if 'LoraLoaderBlockWeight //Inspire' not in nodes.NODE_CLASS_MAPPINGS:
-                    utils.try_install_custom_node(
-                        'https://github.com/ltdrdata/ComfyUI-Inspire-Pack',
-                        "To use 'LBW=' syntax in wildcards, 'Inspire Pack' extension is required.")
-
-                    print(f"'LBW(Lora Block Weight)' is given, but the 'Inspire Pack' is not installed. The LBW= attribute is being ignored.")
-                    model, clip = default_lora()
+            if loader is not None:
+                if loader == 'nunchaku':
+                    if 'NunchakuFluxLoraLoader' not in nodes.NODE_CLASS_MAPPINGS:
+                        logging.warning(f"To use `LOADER=nunchaku`, 'ComfyUI-nunchaku' is required. The LOADER= attribute is being ignored.")
+                    cls = nodes.NODE_CLASS_MAPPINGS['NunchakuFluxLoraLoader']
+                    model = cls().load_lora(model, lora_name, model_weight)[0]
                 else:
-                    cls = nodes.NODE_CLASS_MAPPINGS['LoraLoaderBlockWeight //Inspire']
-                    model, clip, _ = cls().doit(model, clip, lora_name, model_weight, clip_weight, False, 0, lbw_a, lbw_b, "", lbw)
+                    logging.warning(f"LORA LOADER NOT FOUND: '{loader}'")
             else:
-                model, clip = default_lora()
+                def default_lora():
+                    return nodes.LoraLoader().load_lora(model, clip, lora_name, model_weight, clip_weight)
+
+                if lbw is not None:
+                    if 'LoraLoaderBlockWeight //Inspire' not in nodes.NODE_CLASS_MAPPINGS:
+                        utils.try_install_custom_node(
+                            'https://github.com/ltdrdata/ComfyUI-Inspire-Pack',
+                            "To use 'LBW=' syntax in wildcards, 'Inspire Pack' extension is required.")
+
+                        logging.warning(f"'LBW(Lora Block Weight)' is given, but the 'Inspire Pack' is not installed. The LBW= attribute is being ignored.")
+                        model, clip = default_lora()
+                    else:
+                        cls = nodes.NODE_CLASS_MAPPINGS['LoraLoaderBlockWeight //Inspire']
+                        model, clip, _ = cls().doit(model, clip, lora_name, model_weight, clip_weight, False, 0, lbw_a, lbw_b, "", lbw)
+
+                else:
+                    model, clip = default_lora()
         else:
-            print(f"LORA NOT FOUND: {orig_lora_name}")
+            logging.warning(f"LORA NOT FOUND: {orig_lora_name}")
 
     pass3 = [x.strip() for x in pass2.split("BREAK")]
     pass3 = [x for x in pass3 if x != '']
@@ -469,7 +485,7 @@ def process_with_loras(wildcard_opt, model, clip, clip_encoder=None, seed=None,
         pass3 = ['']
 
     pass3_str = [f'[{x}]' for x in pass3]
-    print(f"CLIP: {str.join(' + ', pass3_str)}")
+    logging.info(f"CLIP: {str.join(' + ', pass3_str)}")
 
     result = None
 

pyproject.toml

@@ -1,7 +1,7 @@
 [project]
 name = "comfyui-impact-pack"
 description = "This node pack offers various detector nodes and detailer nodes that allow you to configure a workflow that automatically enhances facial details. And provide iterative upscaler."
-version = "8.11"
+version = "8.18"
 license = { file = "LICENSE.txt" }
 dependencies = ["segment-anything", "scikit-image", "piexif", "transformers", "opencv-python-headless", "GitPython", "scipy>=1.11.4"]
 

requirements.txt

@@ -4,6 +4,7 @@ piexif
 transformers
 opencv-python-headless
 scipy>=1.11.4
-numpy<2
+numpy
 dill
-matplotlib
+matplotlib
+git+https://github.com/facebookresearch/sam2