commit ICMask and ICMaskCropBack nodes

2024-12-14 21:24:39 +08:00
parent 650cf53f08
commit 535704915f
8 changed files with 1764 additions and 1 deletions
@@ -147,6 +147,7 @@ When this error has occurred, please check the network environment.

 <font size="4">**If the dependency package error after updating,  please double clicking ```repair_dependency.bat``` (for Official ComfyUI Protable) or  ```repair_dependency_aki.bat``` (for ComfyUI-aki-v1.x) in the plugin folder to reinstall the dependency packages. </font><br />    

+* Commit [ICMask](#ICMask) and [ICMaskCropBack](#ICMaskCropBack) nodes, Used for generating In-Context image and mask, and automatic crop back. The code is from [lrzjason/Comfyui-In-Context-Lora-Utils](https://github.com/lrzjason/Comfyui-In-Context-Lora-Utils) , Thanks to the original author @小志Jason.
 * Commit [GetMainColorsV2](#GetMainColorsV2) node, add sorting by color area and output color values and proportions in the preview image. This part of the code was improved by @ HL, thanks.
 * Optimize dependency packages. Optimize some algorithms.
 * Split some nodes of the dependencies that are prone to problems into [ComfyUI_LayerStyle_Advance](#https://github.com/chflame163/ComfyUI_LayerStyle_Advance) repository. Including:
@@ -918,6 +919,35 @@ The following changes have been made based on ImageScaleByAspectRatio:
 * scale_to_length: The numerical value here serves as the length of the specified edge or the total pixels (kilo pixels) for scale_to_side.
 * background_color<sup>4</sup>: The color of the background.

+### <a id="table1">ICMask</a>
+Used for generating In-Context image and mask. The code is from [lrzjason/Comfyui-In-Context-Lora-Utils](https://github.com/lrzjason/Comfyui-In-Context-Lora-Utils) , Thanks to the original author @小志Jason.
+![image](image/icmask_example.jpg)
+
+Node Options:   
+![image](image/icmask_node.jpg)    
+
+* first_image: Images used as contextual references.
+* first_mask: Optional input, context reference image mask.
+* second_image: Used for redrawing images.
+* second_mask: Mask used for redrawing images.
+* patch_mode: There are three types of splicing modes: auto、patch_right and patch_bottom.
+* output_length: Output the long side size of the image.
+* patch_color: Fill color.
+
+Outputs:
+* image: The output image.
+* mask: The output mask.
+* icmask_data: The stitching information of the image is used for automatic cropping of subsequent nodes.
+
+### <a id="table1">ICMaskCropBack</a>
+Crop the image inference output generated by ICMask.
+
+Node Options:   
+![image](image/icmask_crop_back_node.jpg)    
+
+* image: The input image.
+* icmask_data: Splicing information output from ICMask node.
+

 ### <a id="table1">VQAPrompt</a>

@@ -128,6 +128,7 @@ os.environ['HF_ENDPOINT'] = 'https://hf-mirror.com'
 ## 更新说明
 <font size="4">**如果本插件更新后出现依赖包错误，请双击运行插件目录下的```install_requirements.bat```(官方便携包)，或 ```install_requirements_aki.bat```(秋叶整合包) 重新安装依赖包。

+* 添加 [ICMask](#ICMask) 和 [ICMaskCropBack](#ICMaskCropBack) 节点，用于生成一致性上下文图片和遮罩，以及自动回裁。代码来自[lrzjason/Comfyui-In-Context-Lora-Utils](https://github.com/lrzjason/Comfyui-In-Context-Lora-Utils) 感谢原作者@小志Jason
 * 添加 [GetMainColorsV2](#GetMainColorsV2) 节点，增加按颜色面积排序，并在预览图中输出色值和比例。这部分代码由@HL完善，感谢。
 * 优化依赖包。优化部分图形算法。
 * 分拆部分依赖易出问题的节点至[ComfyUI_LayerStyle_Advance](#https://github.com/chflame163/ComfyUI_LayerStyle_Advance)仓库。包括下列节点：    
@@ -818,6 +819,35 @@ ImageScaleByAspectRatio的V2升级版
 * scale_to_length: 这里的数值作为scale_to_side指定边的长度, 或者总像素数量(kilo pixels)。
 * background_color<sup>4</sup>: 背景色。

+### <a id="table1">ICMask</a>
+用于生成一致性上下文图片和遮罩。代码来自[lrzjason/Comfyui-In-Context-Lora-Utils](https://github.com/lrzjason/Comfyui-In-Context-Lora-Utils) 感谢原作者@小志Jason
+![image](image/icmask_example.jpg)
+
+节点选项说明:   
+![image](image/icmask_node.jpg)    
+
+* first_image: 用作上下文参考的图像。
+* first_mask: 可选输入，上下文参考图像的遮罩。
+* second_image: 用于重绘的图像。
+* second_mask: 用于重绘的图像的遮罩。
+* patch_mode: 拼接模式，有三种模式：auto、patch_right 和 patch_bottom。patch_right为左右拼接，patch_bottom为上下拼接，auto为自动模式。
+* output_length: 输出图像的长边尺寸。
+* patch_color: 填充颜色。
+
+输出:
+* image: 输出图像。
+* mask: 输出遮罩。
+* icmask_data: 图像的拼接信息，用于后续节点进行自动裁切。
+
+### <a id="table1">ICMaskCropBack</a>
+对ICMask生成的图像推理输出进行裁切。
+
+节点选项说明:   
+![image](image/icmask_crop_back_node.jpg)    
+
+* image: 图像输入。
+* icmask_data: 从ICMask输出的拼接信息。
+

 ### <a id="table1">VQAPrompt</a>
 使用blip-vqa模型进行视觉问答。本节点的部分代码参考自[celoron/ComfyUI-VisualQueryTemplate](https://github.com/celoron/ComfyUI-VisualQueryTemplate)，感谢原作者。    
@@ -0,0 +1,225 @@
+# code from https://github.com/lrzjason/Comfyui-In-Context-Lora-Utils
+
+import torch
+import numpy as np
+from PIL import Image
+import cv2
+from .imagefunc import log
+
+
+def resize_img(img, resolution, interpolation=cv2.INTER_CUBIC):
+    # print(img)
+
+    # print(resolution)
+    return cv2.resize(img, resolution, interpolation=interpolation)
+
+def create_image_from_color(width, height, color=(255, 255, 255)):
+    # OpenCV uses BGR, so convert hex color to BGR if necessary
+    if isinstance(color, str) and color.startswith('#'):
+        color = tuple(int(color[i:i + 2], 16) for i in (5, 3, 1))[::-1]
+
+    # Create a blank image with the specified color
+    blank_image = np.full((height, width, 3), color, dtype=np.uint8)
+    return blank_image
+
+def fit_image(image, mask=None, output_length=1536, patch_mode="auto"):
+    image = image.detach().cpu().numpy()
+    if mask is not None:
+        mask = mask.detach().cpu().numpy()
+
+    # print("np.all(mask == 0)",np.all(mask == 0))
+    base_length = int(output_length / 3 * 2)
+    half_length = int(output_length / 2)
+    image_height, image_width, _ = image.shape
+
+    target_width = int(half_length)
+    target_height = int(base_length)
+
+    if patch_mode == "auto":
+        if image_width > image_height:
+            patch_mode = "patch_bottom"
+            target_width = int(base_length)
+            target_height = int(half_length)
+        else:
+            patch_mode = "patch_right"
+    elif patch_mode == "patch_bottom":
+        target_width = int(base_length)
+        target_height = int(half_length)
+
+    # 等比例缩放并填充逻辑
+    scale_ratio = min(target_width / image_width, target_height / image_height)
+
+    # 计算缩放后的尺寸
+    new_width = int(image_width * scale_ratio)
+    new_height = int(image_height * scale_ratio)
+
+    # 缩放图片
+    image = resize_img(image, (new_width, new_height))
+
+    if mask is not None:
+        mask = resize_img(mask, (new_width, new_height), cv2.INTER_NEAREST_EXACT)
+
+    # 计算填充的差值
+    diff_x = target_width - new_width
+    diff_y = target_height - new_height
+
+    # 计算填充上下左右的像素
+    pad_x = diff_x // 2
+    pad_y = diff_y // 2
+
+    # 添加白色填充到图片，黑色填充到掩码
+    resized_image = cv2.copyMakeBorder(
+        image,
+        pad_y, diff_y - pad_y,
+        pad_x, diff_x - pad_x,
+        cv2.BORDER_CONSTANT, value=(255, 255, 255)
+    )
+
+    if mask is not None:
+        resized_mask = cv2.copyMakeBorder(
+            mask,
+            pad_y, diff_y - pad_y,
+            pad_x, diff_x - pad_x,
+            cv2.BORDER_CONSTANT, value=(0, 0, 0)
+        )
+
+    else:
+        resized_mask = torch.zeros((target_width, target_height))
+
+    return resized_image, resized_mask, target_width, target_height, patch_mode
+
+class ICMask_Data:
+    def __init__(self, x_offset, y_offset, target_width, target_height, total_width, total_height):
+        self.x_offset = x_offset
+        self.y_offset = y_offset
+        self.target_width = target_width
+        self.target_height = target_height
+        self.total_width = total_width
+        self.total_height = total_height
+
+
+class LS_ICMask:
+    def __init__(self):
+        self.NODE_NAME = 'IC_Mask'
+
+    @classmethod
+    def INPUT_TYPES(s):
+        return {
+            "required": {
+                "first_image": ("IMAGE",),
+                "patch_mode": (["auto", "patch_right", "patch_bottom"], {
+                    "default": "auto",
+                }),
+                "output_length": ("INT", {
+                    "default": 1536,
+                }),
+                "patch_color": (["#FF0000", "#00FF00", "#0000FF", "#FFFFFF"], {
+                    "default": "#FFFFFF",
+                }),
+            },
+            "optional": {
+                "first_mask": ("MASK",),
+                "second_image": ("IMAGE",),
+                "second_mask": ("MASK",),
+            }
+        }
+
+    RETURN_TYPES = ("IMAGE", "MASK", "ICMASK_DATA",)
+    RETURN_NAMES = ("image", "mask", "icmask_data",)
+    FUNCTION = "ic_mask"
+    CATEGORY = '😺dzNodes/LayerUtility'
+
+    def ic_mask(self, first_image, patch_mode, output_length, patch_color, first_mask=None, second_image=None,
+                 second_mask=None):
+
+        if output_length % 64 != 0:
+            output_length = output_length - (output_length % 64)
+
+        image1 = first_image[0]
+        if first_mask is None:
+            image1_mask = torch.zeros((image1.shape[0], image1.shape[1]))
+        else:
+            image1_mask = first_mask[0]
+
+        image1, image1_mask, target_width, target_height, patch_mode = fit_image(image1, image1_mask, output_length,
+                                                                                 patch_mode)
+        if second_image is not None:
+            image2 = second_image[0]
+            if second_mask is None:
+                image2_mask = torch.zeros((image2.shape[0], image2.shape[1]))
+            else:
+                image2_mask = second_mask[0]
+            image2, image2_mask, _, _, _ = fit_image(image2, image2_mask, output_length, patch_mode)
+        else:
+            image2 = create_image_from_color(target_width, target_height, color=patch_color)
+            image2 = torch.from_numpy(image2)
+            if second_mask is None:
+                image2_mask = torch.zeros((image2.shape[0], image2.shape[1]))
+            else:
+                image2_mask = second_mask[0]
+            image2, image2_mask, _, _, _ = fit_image(image2, image2_mask, output_length)
+
+        min_y = 0
+        min_x = 0
+
+        if second_mask is None or np.all(image2_mask == 0):
+            image2_mask = torch.ones((image1.shape[0], image1.shape[1]))
+
+        if patch_mode == "patch_right":
+            concatenated_image = np.hstack((image1, image2))
+            concatenated_mask = np.hstack((image1_mask, image2_mask))
+            min_x = 50
+        else:
+            concatenated_image = np.vstack((image1, image2))
+            concatenated_mask = np.vstack((image1_mask, image2_mask))
+            min_y = 50
+        min_y = int(min_y / 100.0 * concatenated_image.shape[0])
+        min_x = int(min_x / 100.0 * concatenated_image.shape[1])
+
+        return_masks = torch.from_numpy(concatenated_mask)[None,]
+
+        concatenated_image = np.clip(255. * concatenated_image, 0, 255).astype(np.float32) / 255.0
+        concatenated_image = torch.from_numpy(concatenated_image)[None,]
+
+        return_images = concatenated_image
+        icmask_data = ICMask_Data(min_x, min_y, target_width, target_height, concatenated_image.shape[1],
+                                  concatenated_image.shape[0])
+
+        return (return_images, return_masks, icmask_data)
+
+
+class LS_ICMask_CropBack:
+
+    def __init__(self):
+        self.NODE_NAME = 'IC_Mask_Crop_Back'
+
+    @classmethod
+    def INPUT_TYPES(s):
+        return {"required": { "image": ("IMAGE",),
+                              "icmask_data": ("ICMASK_DATA",),
+                              }}
+    RETURN_TYPES = ("IMAGE",)
+    FUNCTION = "crop"
+    CATEGORY = '😺dzNodes/LayerUtility'
+
+    def crop(self, image, icmask_data):
+        width = icmask_data.target_width
+        height = icmask_data.target_height
+        x = icmask_data.x_offset
+        y = icmask_data.y_offset
+        x = min(x, image.shape[2] - 1)
+        y = min(y, image.shape[1] - 1)
+        to_x = width + x
+        to_y = height + y
+        img = image[:,y:to_y, x:to_x, :]
+        return (img,)
+
+NODE_CLASS_MAPPINGS = {
+    "LayerUtility: ICMask": LS_ICMask,
+    "LayerUtility: ICMaskCropBack": LS_ICMask_CropBack,
+}
+
+NODE_DISPLAY_NAME_MAPPINGS = {
+    "LayerUtility: ICMask": "LayerUtility: IC Mask",
+    "LayerUtility: ICMaskCropBack": "LayerUtility: IC Mask Crop Back"
+}
@@ -1,7 +1,7 @@
 [project]
 name = "comfyui_layerstyle"
 description = "A set of nodes for ComfyUI it generate image like Adobe Photoshop's Layer Style. the Drop Shadow is first completed node, and follow-up work is in progress."
-version = "2.0.6"
+version = "2.0.7"
 license = "MIT"
 dependencies = ["numpy", "pillow", "torch", "matplotlib", "Scipy", "scikit_image", "scikit_learn", "opencv-contrib-python", "pymatting", "timm", "colour-science",  "transformers",  "blend_modes",  "huggingface_hub", "loguru"]