commit BatchSelector, MediapipeFacialSegment nodes, LayerUtility creates new subdirectories

2024-06-08 14:04:24 +08:00
parent 266f8a2536
commit 1f30a686d5
23 changed files with 261 additions and 24 deletions
@@ -2,7 +2,7 @@

 [中文说明点这里](./README_CN.MD)    

-如果您有定制节点、定制工作流业务，请联系email [chflame@163.com](mailto:chflame@163.com).
+商务合作请联系email [chflame@163.com](mailto:chflame@163.com).

 For business cooperation, please contact email [chflame@163.com](mailto:chflame@163.com).

@@ -80,6 +80,10 @@ When this error has occurred, please check the network environment.
 ## Update
 <font size="4">**If the dependency package error after updating, please reinstall the relevant dependency packages. </font><br />    

+
+* Commit [MediapipeFacialSegment](#MediapipeFacialSegment) node，Used to segment facial features, including left and right eyebrows, eyes, lips, and teeth.
+* Commit [BatchSelector](#BatchSelector) node，Used to retrieve specified images or masks from batch images or masks.
+* LayerUtility creates new subdirectories such as SystemIO, Data, and Prompt. Some nodes are classified into subdirectories.
 * Commit [MaskByColor](#MaskByColor) node, Generate a mask based on the selected color.
 * Commit [LoadPSD](#LoadPSD) node, It read the psd format, and output layer images. Note that this node requires the installation of the ```psd_tools``` dependency package, If error occurs during the installation of psd_tool, such as ```ModuleNotFoundError: No module named 'docopt'``` , please download [docopt's whl](https://www.piwheels.org/project/docopt/) and manual install it. 
 * Commit [SegformerB2ClothesUltra](#SegformerB2ClothesUltra) node, it used to segment character clothing. The model segmentation code is from[StartHua](https://github.com/StartHua/Comfyui_segformer_b2_clothes),  thanks to the original author.
@@ -1059,6 +1063,19 @@ Node options:
 * output: Switch output. the value is the corresponding input group. when the ```random-output``` option is True, this setting will be ignored.
 * random_output: When this is true, the ```output``` setting will be ignored and a random set will be output among all valid inputs.

+
+### <a id="table1">BatchSelector</a>
+Retrieve specified images or masks from batch images or masks.
+![image](image/batch_selector_example.jpg)    
+
+Node Options:  
+![image](image/batch_selector_node.jpg)    
+* images: Batch images input. This input is optional.
+* masks: Batch masks input. This input is optional.
+* select: Select the output image or mask at the batch index value, where 0 is the first image. Multiple values can be entered, separated by any non numeric character, including but not limited to commas, periods, semicolons, spaces or letters, and even Chinese characters.
+Note: If the value exceeds the batch size, the last image will be output. If there is no corresponding input, an empty 64x64 image or a 64x64 black mask will be output.
+
+
 ### <a id="table1">TextJoin</a>
 ![image](image/text_join_example.jpg)    
 Combine multiple paragraphs of text into one.
@@ -1387,6 +1404,19 @@ Outputs:
 * yolo_masks: For all masks identified by yolo, each individual mask is output as a mask.


+### <a id="table1">MediapipeFacialSegment</a>
+Use the Mediapipe model to detect facial features, segment left and right eyebrows, eyes, lips, and tooth.
+![image](image/mediapipe_facial_segment_example.jpg)    
+
+Node Options:  
+![image](image/mediapipe_facial_segment_node.jpg)    
+* left_eye: Recognition switch of left eye.
+* left_eyebrow: Recognition switch of left eyebrow.
+* right_eye: Recognition switch of right eye.
+* right_eyebrow: Recognition switch of right eyebrow.
+* lips: Recognition switch of lips.
+* tooth: Recognition switch of tooth.
+
 ### <a id="table1">MaskByColor</a>
 Generate a mask based on the selected color.
 ![image](image/mask_by_color_example.jpg)    
@@ -80,6 +80,9 @@ git clone https://github.com/chflame163/ComfyUI_LayerStyle.git
 ## 更新说明
 <font size="4">**如果本插件更新后出现依赖包错误，请重新安装相关依赖包。

+* 添加 [MediapipeFacialSegment](#MediapipeFacialSegment) 节点， 用于分割面部五官，包括左右眉、眼睛、嘴唇和牙齿。
+* 添加 [BatchSelector](#BatchSelector) 节点， 用于从批量图片或遮罩中获取指定的图片或遮罩。
+* LayerUtility大类新建子目录SystemIO, Data, Prompt。一部分节点被分类到子目录。
 * 添加 [MaskByColor](#MaskByColor) 节点, 根据选择的颜色生成遮罩。
 * 添加 [LoadPSD](#LoadPSD) 节点, 读取psd格式并输出图层图片。注意这个节点需要安装psd_tools依赖包，如果安装psd_tool中出现```ModuleNotFoundError: No module named 'docopt'```错误，请下载[docopt的whl](https://www.piwheels.org/project/docopt/)手动安装。
 * 添加 [SegformerB2ClothesUltra](#SegformerB2ClothesUltra)节点，用于分割人物服装。模型分割代码来自[StartHua](https://github.com/StartHua/Comfyui_segformer_b2_clothes)，感谢原作者。
@@ -1049,6 +1052,18 @@ cropped_mask: 裁切后的遮罩。
 * random_output: 当此项为True时, 将忽略```output```设置，在所有的有效输入中随机输出一组。


+### <a id="table1">BatchSelector</a>
+从批量图片或遮罩中获取指定的图片或遮罩。
+![image](image/batch_selector_example.jpg)    
+
+节点选项说明:  
+![image](image/batch_selector_node.jpg)    
+* images: 批量图片输入。此输入为可选项。
+* masks: 批量遮罩输入。此输入为可选项。
+* select: 选择输出的图片或遮罩在批量的索引值，0为第一张。可以输入多个值，中间用任意非数字字符分隔，包括不仅限于逗号，句号，分号，空格或者字母，甚至中文。
+注意:如果数值超出批量，将输出最后一张。如果没有对应的输入，将输出一个空的64x64图片或64x64黑色遮罩。
+
+
 ### <a id="table1">TextJoin</a>
 ![image](image/text_join_example.jpg)    
 将多段文字组合为一段。
@@ -1372,6 +1387,20 @@ PersonMaskUltra的V2升级版，增加了VITMatte边缘处理方法。(注意:
 * yolo_masks: yolo识别出来的所有遮罩，每个单独的遮罩输出为一个mask。


+### <a id="table1">MediapipeFacialSegment</a>
+使用Mediapipe模型检测人脸五官，分割左右眉、眼睛、嘴唇和牙齿。
+![image](image/mediapipe_facial_segment_example.jpg)    
+
+节点选项说明:  
+![image](image/mediapipe_facial_segment_node.jpg)    
+* left_eye: 左眼识别开关。
+* left_eyebrow: 左眉识别开关。
+* right_eye: 右眼识别开关。
+* right_eyebrow: 右眉识别开关。
+* lips: 嘴唇识别开关。
+* tooth: 牙齿识别开关。
+
+
 ### <a id="table1">MaskByColor</a>
 根据颜色生成遮罩。
 ![image](image/mask_by_color_example.jpg)    
@@ -94,7 +94,7 @@ class QWenImage2Prompt:
    RETURN_TYPES = ("STRING",)
    RETURN_NAMES = ("text",)
    FUNCTION = "uform_gen2_qwen_chat"
-    CATEGORY = '😺dzNodes/LayerUtility'
+    CATEGORY = '😺dzNodes/LayerUtility/Prompt'

    def uform_gen2_qwen_chat(self, image, question):
        history = []  # Example empty history
@@ -0,0 +1,65 @@
+
+from .imagefunc import *
+
+NODE_NAME = 'BatchSelector'
+
+
+
+class BatchSelector:
+
+    def __init__(self):
+        pass
+
+    @classmethod
+    def INPUT_TYPES(self):
+
+        return {
+            "required": {
+                "select": ("STRING", {"default": "0,"},),
+            },
+            "optional": {
+                "images": ("IMAGE",),  #
+                "masks": ("MASK",),  #
+            }
+        }
+
+    RETURN_TYPES = ("IMAGE", "MASK",)
+    RETURN_NAMES = ("image", "mask",)
+    FUNCTION = 'batch_selector'
+    CATEGORY = '😺dzNodes/LayerUtility/SystemIO'
+
+    def batch_selector(self, select, images=None, masks=None
+                  ):
+        ret_images = []
+        ret_masks = []
+        empty_image = pil2tensor(Image.new("RGBA", (64, 64), (0, 0, 0, 0)))
+        empty_mask = image2mask(Image.new("L", (64, 64), color="black"))
+
+        indexs = extract_numbers(select)
+        for i in indexs:
+            if images is not None:
+                if i < len(images):
+                    ret_images.append(images[i].unsqueeze(0))
+                else:
+                    ret_images.append(images[-1].unsqueeze(0))
+            if masks is not None:
+                if i < len(masks):
+                    ret_masks.append(masks[i].unsqueeze(0))
+                else:
+                    ret_masks.append(masks[-1].unsqueeze(0))
+
+        if len(ret_images) == 0:
+            ret_images.append(empty_image)
+        if len(ret_masks) == 0:
+            ret_masks.append(empty_mask)
+
+        log(f"{NODE_NAME} Processed {len(ret_images)} image(s).", message_type='finish')
+        return (torch.cat(ret_images, dim=0), torch.cat(ret_masks, dim=0),)
+
+NODE_CLASS_MAPPINGS = {
+    "LayerUtility: BatchSelector": BatchSelector
+}
+
+NODE_DISPLAY_NAME_MAPPINGS = {
+    "LayerUtility: BatchSelector": "LayerUtility: Batch Selector"
+}
@@ -22,7 +22,7 @@ class ColorValuetoRGBValue:
    RETURN_TYPES = ("INT", "INT", "INT")
    RETURN_NAMES = ("R", "G", "B")
    FUNCTION = 'color_value_to_rgb_value'
-    CATEGORY = '😺dzNodes/LayerUtility'
+    CATEGORY = '😺dzNodes/LayerUtility/Data'

    def color_value_to_rgb_value(self, color_value,):
        R, G, B = 0, 0, 0
@@ -14,7 +14,7 @@ class SeedNode:
    RETURN_TYPES = ("INT",)
    RETURN_NAMES = ("seed",)
    FUNCTION = 'seed_node'
-    CATEGORY = '😺dzNodes/LayerUtility'
+    CATEGORY = '😺dzNodes/LayerUtility/Data'

    def seed_node(self, seed):
        return (seed,)
@@ -35,7 +35,7 @@ class BooleanOperator:
    RETURN_TYPES = ("BOOLEAN",)
    RETURN_NAMES = ("output",)
    FUNCTION = 'bool_operator_node'
-    CATEGORY = '😺dzNodes/LayerUtility'
+    CATEGORY = '😺dzNodes/LayerUtility/Data'

    def bool_operator_node(self, a, b, operator):
        ret_value = False
@@ -73,7 +73,7 @@ class NumberCalculator:
    RETURN_TYPES = ("INT", "FLOAT",)
    RETURN_NAMES = ("int", "float",)
    FUNCTION = 'number_calculator_node'
-    CATEGORY = '😺dzNodes/LayerUtility'
+    CATEGORY = '😺dzNodes/LayerUtility/Data'

    def number_calculator_node(self, a, b, operator):
        ret_value = 0
@@ -106,7 +106,7 @@ class TextBoxNode:
    RETURN_TYPES = ("STRING",)
    RETURN_NAMES = ("text",)
    FUNCTION = 'text_box_node'
-    CATEGORY = '😺dzNodes/LayerUtility'
+    CATEGORY = '😺dzNodes/LayerUtility/Data'

    def text_box_node(self, text):
        return (text,)
@@ -123,7 +123,7 @@ class IntegerNode:
    RETURN_TYPES = ("INT",)
    RETURN_NAMES = ("int",)
    FUNCTION = 'integer_node'
-    CATEGORY = '😺dzNodes/LayerUtility'
+    CATEGORY = '😺dzNodes/LayerUtility/Data'

    def integer_node(self, int_value):
        return (int_value,)
@@ -140,7 +140,7 @@ class FloatNode:
    RETURN_TYPES = ("FLOAT",)
    RETURN_NAMES = ("float",)
    FUNCTION = 'float_node'
-    CATEGORY = '😺dzNodes/LayerUtility'
+    CATEGORY = '😺dzNodes/LayerUtility/Data'

    def float_node(self, float_value):
        return (float_value,)
@@ -157,7 +157,7 @@ class BooleanNode:
    RETURN_TYPES = ("BOOLEAN",)
    RETURN_NAMES = ("boolean",)
    FUNCTION = 'boolean_node'
-    CATEGORY = '😺dzNodes/LayerUtility'
+    CATEGORY = '😺dzNodes/LayerUtility/Data'

    def boolean_node(self, bool_value):
        return (bool_value,)
@@ -1777,6 +1777,10 @@ def is_contain_chinese(check_str:str) -> bool:
            return True
    return False

+# 提取字符串中的数字为列表
+def extract_numbers(string):
+    return [int(s) for s in re.findall(r'\d+', string)]
+
 def tensor_info(tensor:object) -> str:
    value = ''
    if isinstance(tensor, torch.Tensor):
@@ -31,7 +31,7 @@ class LoadPSD(LoadImage):
    RETURN_TYPES = ("IMAGE", "IMAGE", "IMAGE",)
    RETURN_NAMES = ("flat_image", "layer_image", "all_layers",)
    FUNCTION = "load_psd"
-    CATEGORY = '😺dzNodes/LayerUtility'
+    CATEGORY = '😺dzNodes/LayerUtility/SystemIO'

    def load_psd(self, image, file_path, include_hidden_layer, layer_index, find_layer_by, layer_name,):

@@ -0,0 +1,109 @@
+import numpy as np
+import mediapipe as mp
+from .imagefunc import *
+
+NODE_NAME = 'MediapipeFacialSegment'
+
+
+# 获取特征点的坐标
+def get_points(indices, face_landmarks, width, height):
+    return [(int(face_landmarks.landmark[i].x * width), int(face_landmarks.landmark[i].y * height))
+            for i in indices]
+
+# 绘制面部特征的多边形
+def draw_feature(indices, mask, face_landmarks, width, height):
+    points = get_points(indices, face_landmarks, width, height)
+    points = np.array(points, dtype=np.int32)
+    cv2.fillPoly(mask, [points], 255)
+
+class FacialFeatureSegment:
+
+    def __init__(self):
+        pass
+
+    @classmethod
+    def INPUT_TYPES(self):
+
+        return {
+            "required": {
+                "image": ("IMAGE",),  #
+                "left_eye": ("BOOLEAN", {"default": True}),
+                "left_eyebrow": ("BOOLEAN", {"default": True}),
+                "right_eye": ("BOOLEAN", {"default": True}),
+                "right_eyebrow": ("BOOLEAN", {"default": True}),
+                "lips": ("BOOLEAN", {"default": True}),
+                "tooth": ("BOOLEAN", {"default": True}),
+            },
+            "optional": {
+            }
+        }
+
+    RETURN_TYPES = ("IMAGE", "MASK",)
+    RETURN_NAMES = ("image", "mask",)
+    FUNCTION = 'facial_feature_segment'
+    CATEGORY = '😺dzNodes/LayerMask'
+
+    def facial_feature_segment(self, image,
+                              left_eye, left_eyebrow, right_eye, right_eyebrow, lips, tooth
+                  ):
+
+        # 定义面部特征索引
+        left_eye_indices = [33, 7, 163, 144, 145, 153, 154, 155, 133, 173, 157, 158, 159, 160, 161, 246]
+        right_eye_indices = [263, 249, 390, 373, 374, 380, 381, 382, 362, 398, 384, 385, 386, 387, 388, 466]
+        left_eyebrow_indices = [70, 63, 105, 66, 107, 55, 65, 52, 53, 46]
+        right_eyebrow_indices = [336, 296, 334, 293, 300, 276, 283, 282, 295, 285]
+        # upper_lip_indices = [61, 146, 91, 181, 84, 17, 314, 405, 321, 375, 291, 308, 324, 318, 402, 317, 14, 87, 178, 88, 95, 78]
+        # lower_lip_indices = [61, 185, 40, 39, 37, 0, 267, 269, 270, 409, 291, 308, 415, 310, 311, 312, 13, 82, 81, 80, 191, 78]
+        tooth_indices = [78, 95, 88, 178, 87, 14, 317, 402, 318, 324, 308, 415, 310, 311, 312, 13, 82, 81, 80, 191, 78]
+        lips_indices = [61, 76, 62, 78, 191, 80, 81, 82, 13, 312, 311, 310, 415, 308, 324, 318, 402, 317, 14, 87, 178,
+                         88, 95, 185, 40, 39, 37, 0, 267, 269, 270, 409, 291, 375, 321, 405, 314, 17, 84, 181, 91, 146,
+                         61]
+
+        ret_images = []
+        ret_masks = []
+        scale_factor = 4
+
+        for i in image:
+            face_image = tensor2pil(i.unsqueeze(0)).convert('RGB')
+            width, height = face_image.size
+            width *= scale_factor
+            height *= scale_factor
+            cv2_image = pil2cv2(face_image)
+            mp_face_mesh = mp.solutions.face_mesh
+            fase_mesh = mp_face_mesh.FaceMesh(static_image_mode=True, max_num_faces=1, min_detection_confidence=0.5)
+            results = fase_mesh.process(cv2_image)
+            mask = np.zeros((height, width), dtype=np.uint8)
+
+
+            if results.multi_face_landmarks:
+                for face_landmarks in results.multi_face_landmarks:
+                    # 绘制各个面部特征
+                    if left_eye:
+                        draw_feature(left_eye_indices, mask, face_landmarks, width, height)
+                    if right_eye:
+                        draw_feature(right_eye_indices, mask, face_landmarks, width, height)
+                    if left_eyebrow:
+                        draw_feature(left_eyebrow_indices, mask, face_landmarks, width, height)
+                    if right_eyebrow:
+                        draw_feature(right_eyebrow_indices, mask, face_landmarks, width, height)
+                    if lips:
+                        draw_feature(lips_indices, mask, face_landmarks, width, height)
+                    if tooth:
+                        draw_feature(tooth_indices, mask, face_landmarks, width, height)
+
+            mask = cv22pil(mask).convert('L')
+            mask = gaussian_blur(mask, 2)
+            mask = mask.resize(face_image.size, Image.BILINEAR)
+            ret_images.append(pil2tensor(RGB2RGBA(face_image, mask)))
+            ret_masks.append(image2mask(mask))
+
+        log(f"{NODE_NAME} Processed {len(ret_images)} image(s).", message_type='finish')
+        return (torch.cat(ret_images, dim=0), torch.cat(ret_masks, dim=0),)
+
+NODE_CLASS_MAPPINGS = {
+    "LayerMask: MediapipeFacialSegment": FacialFeatureSegment
+}
+
+NODE_DISPLAY_NAME_MAPPINGS = {
+    "LayerMask: MediapipeFacialSegment": "LayerMask: Mediapipe Facial Segment"
+}
@@ -14,10 +14,10 @@ class PrintInfo:
      },
    }

-  CATEGORY = '😺dzNodes/LayerUtility'
  RETURN_TYPES = ("STRING",)
  RETURN_NAMES = ("text",)
  FUNCTION = "print_info"
+  CATEGORY = '😺dzNodes/LayerUtility/Data'
  OUTPUT_NODE = True

  def print_info(self, anything=None):
@@ -24,7 +24,7 @@ class PromptEmbellish:
    RETURN_TYPES = ("STRING",)
    RETURN_NAMES = ("text",)
    FUNCTION = 'prompt_embellish'
-    CATEGORY = '😺dzNodes/LayerUtility'
+    CATEGORY = '😺dzNodes/LayerUtility/Prompt'

    def prompt_embellish(self, api, token_limit, describe, image=None):
        if describe == "" and image is None:
@@ -26,7 +26,7 @@ class PromptTagger:
    RETURN_TYPES = ("STRING",)
    RETURN_NAMES = ("text",)
    FUNCTION = 'prompt_tagger'
-    CATEGORY = '😺dzNodes/LayerUtility'
+    CATEGORY = '😺dzNodes/LayerUtility/Prompt'

    def prompt_tagger(self, image, api, token_limit, exclude_word, replace_with_word):
        import google.generativeai as genai
@@ -23,7 +23,7 @@ class PurgeVRAM:

    RETURN_TYPES = ()
    FUNCTION = "purge_vram"
-    CATEGORY = '😺dzNodes/LayerUtility'
+    CATEGORY = '😺dzNodes/LayerUtility/SystemIO'
    OUTPUT_NODE = True

    def purge_vram(self, anything, purge_cache, purge_models):
@@ -21,7 +21,7 @@ class CreateQRCode:
    RETURN_TYPES = ("IMAGE", )
    RETURN_NAMES = ("image", )
    FUNCTION = 'create_qrcode'
-    CATEGORY = '😺dzNodes/LayerUtility'
+    CATEGORY = '😺dzNodes/LayerUtility/SystemIO'

    def create_qrcode(self, size, border, text):
        import qrcode
@@ -58,7 +58,7 @@ class DecodeQRCode:
    RETURN_TYPES = ("STRING", )
    RETURN_NAMES = ("string", )
    FUNCTION = 'decode_qrcode'
-    CATEGORY = '😺dzNodes/LayerUtility'
+    CATEGORY = '😺dzNodes/LayerUtility/SystemIO'

    def decode_qrcode(self, image, pre_blur):
        ret_texts = []
@@ -33,7 +33,7 @@ class SaveImagePlus:
    RETURN_TYPES = ()
    FUNCTION = "save_image_plus"
    OUTPUT_NODE = True
-    CATEGORY = '😺dzNodes/LayerUtility'
+    CATEGORY = '😺dzNodes/LayerUtility/SystemIO'

    def save_image_plus(self, images, custom_path, filename_prefix, timestamp, format, quality,
                           meta_data, blind_watermark, preview, save_workflow_as_json,
@@ -24,7 +24,7 @@ class TextJoin:
    RETURN_TYPES = ("STRING",)
    RETURN_NAMES = ("text",)
    FUNCTION = "text_join"
-    CATEGORY = '😺dzNodes/LayerUtility'
+    CATEGORY = '😺dzNodes/LayerUtility/Data'

    def text_join(self, **kwargs):

@@ -22,7 +22,7 @@ class EncodeBlindWaterMark:
    RETURN_TYPES = ("IMAGE",)
    RETURN_NAMES = ("image",)
    FUNCTION = 'watermark_encode'
-    CATEGORY = '😺dzNodes/LayerUtility'
+    CATEGORY = '😺dzNodes/LayerUtility/SystemIO'

    def watermark_encode(self, image, watermark_image):

@@ -77,7 +77,7 @@ class DecodeBlindWaterMark:
    RETURN_TYPES = ("IMAGE", )
    RETURN_NAMES = ("watermark_image",)
    FUNCTION = 'watermark_decode'
-    CATEGORY = '😺dzNodes/LayerUtility'
+    CATEGORY = '😺dzNodes/LayerUtility/SystemIO'

    def watermark_decode(self, image):

@@ -22,7 +22,7 @@ class XYtoPercent:
    RETURN_TYPES = ("FLOAT", "FLOAT",)
    RETURN_NAMES = ("x_percent", "x_percent",)
    FUNCTION = 'xy_to_percent'
-    CATEGORY = '😺dzNodes/LayerUtility'
+    CATEGORY = '😺dzNodes/LayerUtility/Data'

    def xy_to_percent(self, background_image, layer_image, x, y,):

@@ -1,8 +1,8 @@
 [project]
 name = "comfyui_layerstyle"
 description = "A set of nodes for ComfyUI it generate image like Adobe Photoshop's Layer Style. the Drop Shadow is first completed node, and follow-up work is in progress."
-version = "1.0.2"
-license = "LICENSE"
+version = "1.0.3"
+license = "MIT"
 dependencies = ["numpy", "pillow", "torch", "matplotlib", "Scipy", "scikit_image", "opencv-contrib-python", "pymatting", "segment_anything", "timm", "addict", "yapf", "colour-science", "wget", "mediapipe", "loguru", "typer_config", "fastapi", "rich", "google-generativeai", "diffusers", "omegaconf", "tqdm", "transformers", "kornia", "image-reward", "ultralytics", "blend_modes", "blind-watermark", "qrcode", "pyzbar", "psd-tools"]

 [project.urls]