diff --git a/README.MD b/README.MD index 7330e0e..bdcacf4 100644 --- a/README.MD +++ b/README.MD @@ -98,7 +98,8 @@ When this error has occurred, please check the network environment. ## Update **If the dependency package error after updating, please reinstall the relevant dependency packages.
-* Commit install_requirements.bat and install_requirements_aki.bat, One click solution to install dependency packages. +* Commit [SegformerUltraV2](#SegformerUltraV2), [SegfromerFashionPipeline](#SegfromerFashionPipeline) and [SegformerClothesPipeline](#SegformerClothesPipeline) nodes, used for segmentation of clothing. please download the model file according to the instructions. +* Commit ```install_requirements.bat``` and ```install_requirements_aki.bat```, One click solution to install dependency packages. * Commit [TransparentBackgroundUltra](#TransparentBackgroundUltra) node, it remove background based on transparent-background model. * Change the VitMatte model of the [Ultra](#Ultra) node to a local call. Please download [all files of vitmatte model](https://huggingface.co/hustvl/vitmatte-small-composition-1k/tree/main) to the ```ComfyUI/models/vitmatte``` folder. * [GetColorToneV2](#GetColorToneV2) node add the ```mask``` method to the color selection option, which can accurately obtain the main color and average color within the mask. @@ -1561,7 +1562,7 @@ Generate masks for characters' faces, hair, arms, legs, and clothing, mainly use The model segmentation code is from[StartHua](https://github.com/StartHua/Comfyui_segformer_b2_clothes)，thanks to the original author. Compared to the comfyui_segformer_b2_clothes, this node has ultra-high edge details. (Note: Generating images with edges exceeding 2K in size using the VITMatte method will consume a lot of memory) -*Download all model files from [https://huggingface.co/mattmdjaga/segformer_b2_clothes](https://huggingface.co/mattmdjaga/segformer_b2_clothes) to ```ComfyUI/models/segformer_b2_clothes``` folder. +*Download all model files from [here](https://huggingface.co/mattmdjaga/segformer_b2_clothes/tree/main) to ```ComfyUI/models/segformer_b2_clothes``` folder. Node Options: ![image](image/segformer_ultra_node.jpg) @@ -1589,6 +1590,103 @@ Node Options: * device: Set whether the VitMatte to use cuda. * max_megapixels: Set the maximum size for VitMate operations. +### SegformerUltraV2 +![image](image/segformer_clothes_example.jpg) +![image](image/segformer_fashion_example.jpg) +Using the segformer model to segment clothing with ultra-high edge details. Currently supports segformer b2 clothes, segformer b3 clothes and segformer b3 fashion。 + +*from [here](https://huggingface.co/mattmdjaga/segformer_b2_clothes/tree/main) download all files to ```ComfyUI/models/segformer_b2_clothes``` folder. +*from [here](https://huggingface.co/sayeed99/segformer_b3_clothes/tree/main) download all files to ```ComfyUI/models/segformer_b3_clothes``` folder. +*from [here](https://huggingface.co/sayeed99/segformer-b3-fashion/tree/main) download all files to ```ComfyUI/models/segformer_b3_fashion``` folder. + +Node Options: +![image](image/segformer_ultra_v2_node.jpg) +* image: The input image. +* segformer_pipeline: Segformer pipeline input. The pipeline is output by SegformerClottesPipeline and SegformerFashionPipeline node. +* detail_method: Edge processing methods. provides VITMatte, VITMatte(local), PyMatting, GuidedFilter. If the model has been downloaded after the first use of VITMatte, you can use VITMatte (local) afterwards. +* detail_erode: Mask the erosion range inward from the edge. the larger the value, the larger the range of inward repair. +* detail_dilate: The edge of the mask expands outward. the larger the value, the wider the range of outward repair. +* black_point: Edge black sampling threshold. +* white_point: Edge white sampling threshold. +* process_detail: Set to false here will skip edge processing to save runtime. +* device: Set whether the VitMatte to use cuda. +* max_megapixels: Set the maximum size for VitMate operations. + +### SegformerClothesPipiline +Select the segformer clothes model and choose the segmentation content. + +Node Options: +![image](image/segformer_clothes_pipeline_node.jpg) +* model: Model selection. There are currently two models available to choose from for segformer b2 clothes and segformer b3 clothes. +* face: Facial recognition switch. +* hair: Hair recognition switch. +* hat: Hat recognition switch. +* sunglass: Sunglass recognition switch. +* left_arm: Left arm recognition switch. +* right_arm: Right arm recognition switch. +* left_leg: Left leg recognition switch. +* right_leg: Right leg recognition switch. +* left_shoe: Left shoe recognition switch. +* right_shoe: Right shoe recognition switch. +* skirt: Skirt recognition switch. +* pants: Pants recognition switch. +* dress: Dress recognition switch. +* belt: Belt recognition switch. +* bag: Bag recognition switch. +* scarf: Scarf recognition switch. + +### SegformerFashionPipiline +Select the segformer fashion model and choose the segmentation content. + +Node Options: +![image](image/segformer_fashion_pipeline_node.jpg) +* model: Model selection. Currently, there is only one model available for selection: segformer b3 fashion。 +* shirt: shirt and blouse switch. +* top: top, t-shirt, sweatshirt switch. +* sweater: sweater switch. +* cardigan: cardigan switch. +* jacket: jacket switch. +* vest: vest switch. +* pants: pants switch. +* shorts: shorts switch. +* skirt: skirt switch. +* coat: coat switch. +* dress: dress switch. +* jumpsuit: jumpsuit switch. +* cape: cape switch. +* glasses: glasses switch. +* hat: hat switch. +* hairaccessory: headband, head covering, hair accessory switch. +* tie: tie switch. +* glove: glove switch. +* watch: watch switch. +* belt: belt switch. +* legwarmer: leg warmer switch. +* tights: tights and stockings switch. +* sock: sock switch. +* shoe: shoes switch. +* bagwallet: bag and wallet switch. +* scarf: scarf switch. +* umbrella: umbrella switch. +* hood: hood switch. +* collar: collar switch. +* lapel: lapel switch. +* epaulette: epaulette switch. +* sleeve: sleeve switch. +* pocket: pocket switch. +* neckline: neckline switch. +* buckle: buckle switch. +* zipper: zipper switch. +* applique: applique switch. +* bead: bead switch. +* bow: bow switch. +* flower: flower switch. +* fringe: fringe switch. +* ribbon: ribbon switch. +* rivet: rivet switch. +* ruffle: ruffle switch. +* sequin: sequin switch. +* tassel: tassel switch. ### MaskEdgeUltraDetail Process rough masks to ultra fine edges. diff --git a/README_CN.MD b/README_CN.MD index ee09214..f41d58e 100644 --- a/README_CN.MD +++ b/README_CN.MD @@ -99,7 +99,8 @@ git clone https://github.com/chflame163/ComfyUI_LayerStyle.git ## 更新说明 **如果本插件更新后出现依赖包错误，请重新安装相关依赖包。 -* 添加 install_requirements.bat 和 install_requirements_aki.bat 文件, 一键解决安装依赖包问题。 +* 添加 [SegformerUltraV2](#SegformerUltraV2), [SegfromerFashionPipeline](#SegfromerFashionPipeline) 和 [SegformerClothesPipeline](#SegformerClothesPipeline) 节点, 用于分割服饰。请按说明下载模型文件。 +* 添加 ```install_requirements.bat``` 和 ```install_requirements_aki.bat``` 文件, 一键解决安装依赖包问题。 * 添加[TransparentBackgroundUltra](#TransparentBackgroundUltra) 节点，基于transparent-background模型，用于去除背景。 * [Ultra](#Ultra) 节点的VitMatte模型改为本地调用，请下载[所有的vitmatte模型文件](https://huggingface.co/hustvl/vitmatte-small-composition-1k/tree/main)到```ComfyUI/models/vitmatte```文件夹。 * [GetColorToneV2](#GetColorToneV2) 节点的取色选项增加```mask```方法，可精确获取遮罩内的主色和平均色。 @@ -1542,7 +1543,7 @@ PersonMaskUltra的V2升级版，增加了VITMatte边缘处理方法。 为人物生成脸、头发、手臂、腿以及服饰的遮罩，主要用于分割服装。模型分割代码来自[StartHua](https://github.com/StartHua/Comfyui_segformer_b2_clothes)，感谢原作者。 与comfyui_segformer_b2_clothes节点相比，这个节点具有超高的边缘细节。 -*从[https://huggingface.co/mattmdjaga/segformer_b2_clothes](https://huggingface.co/mattmdjaga/segformer_b2_clothes)下载全部文件至```ComfyUI/models/segformer_b2_clothes```文件夹。 +*从[这里](https://huggingface.co/mattmdjaga/segformer_b2_clothes/tree/main)下载全部文件至```ComfyUI/models/segformer_b2_clothes```文件夹。 节点选项说明: ![image](image/segformer_ultra_node.jpg) @@ -1570,6 +1571,105 @@ PersonMaskUltra的V2升级版，增加了VITMatte边缘处理方法。 * device: 设置是否使用cuda。 * max_megapixels: 设置vitmatte运算的最大尺寸。 + +### SegformerUltraV2 +![image](image/segformer_clothes_example.jpg) +![image](image/segformer_fashion_example.jpg) +使用segformer模型分割服饰，具有超高的边缘细节。目前支持segformer b2 clothes, segformer b3 clothes, segformer b3 fashion。 + +*从[这里](https://huggingface.co/mattmdjaga/segformer_b2_clothes/tree/main)下载全部文件至```ComfyUI/models/segformer_b2_clothes```文件夹。 +*从[这里](https://huggingface.co/sayeed99/segformer_b3_clothes/tree/main)下载全部文件至```ComfyUI/models/segformer_b3_clothes```文件夹。 +*从[这里](https://huggingface.co/sayeed99/segformer-b3-fashion/tree/main)下载全部文件至```ComfyUI/models/segformer_b3_fashion```文件夹。 + +节点选项说明: +![image](image/segformer_ultra_v2_node.jpg) +* image: 图像输入。 +* segformer_pipeline: segformer管线输入。管线由SegformerClothesPipeline和SegformerFashionPipeline节点输出。 +* detail_method: 边缘处理方法。提供了VITMatte, VITMatte(local), PyMatting, GuidedFilter。如果首次使用VITMatte后模型已经下载，之后可以使用VITMatte(local)。 +* detail_erode: 遮罩边缘向内侵蚀范围。数值越大，向内修复的范围越大。 +* detail_dilate: 遮罩边缘向外扩张范围。数值越大，向外修复的范围越大。 +* black_point: 边缘黑色采样阈值。 +* white_point: 边缘黑色采样阈值。 +* process_detail: 此处设为False将跳过边缘处理以节省运行时间。 +* device: 设置是否使用cuda。 +* max_megapixels: 设置vitmatte运算的最大尺寸。 + +### SegformerClothesPipiline +选择segformer clothes模型，并选择分割内容。 + +节点选项说明: +![image](image/segformer_clothes_pipeline_node.jpg) +* model: 模型选择。目前有两种模型可供选择segformer b2 clothes, segformer b3 clothes。 +* face: 脸部识别。 +* hair: 头发识别。 +* hat: 帽子识别。 +* sunglass: 墨镜识别。 +* left_arm:左手臂识别。 +* right_arm:右手臂识别。 +* left_leg:左腿识别。 +* right_leg:右腿识别。 +* left_shoe: 左鞋子识别。 +* right_shoe: 右鞋子识别。 +* skirt:短裙识别。 +* pants:裤子识别。 +* dress:连衣裙识别。 +* belt:腰带识别。 +* bag:背包识别。 +* scarf:围巾识别。 + +### SegformerFashionPipiline +选择segformer fashion模型，并选择分割内容。 + +节点选项说明: +![image](image/segformer_fashion_pipeline_node.jpg) +* model: 模型选择。目前只有一种模型可供选择segformer b3 fashion。 +* shirt: 衬衫、罩衫识别。 +* top: 上衣、t恤、运动衫识别。 +* sweater: 毛衣识别。 +* cardigan: 开襟毛衫识别。 +* jacket: 夹克识别。 +* vest: 背心识别。 +* pants: 裤子识别。 +* shorts: 短裤识别。 +* skirt: 短裙识别。 +* coat: 外套识别。 +* dress: 连衣裙识别。 +* jumpsuit: 连身裤识别。 +* cape: 斗篷识别。 +* glasses: 眼镜识别。 +* hat: 帽子识别。 +* hairaccessory: 头带、头巾、发饰识别。 +* tie: 领带识别。 +* glove: 手套识别。 +* watch: 手表识别。 +* belt: 皮带识别。 +* legwarmer: 腿套识别。 +* tights: 紧身裤和长筒袜识别。 +* sock: 袜子识别。 +* shoe: 鞋子识别。 +* bagwallet: 背包、钱包识别。 +* scarf: 围巾识别。 +* umbrella: 雨伞识别。 +* hood: 兜帽识别。 +* collar: 衣领识别。 +* lapel: 翻领识别。 +* epaulette: 肩章识别。 +* sleeve: 袖子识别。 +* pocket: 口袋识别。 +* neckline: 领口识别。 +* buckle: 带扣识别。 +* zipper: 拉链识别。 +* applique: 贴花识别。 +* bead: 珠子识别。 +* bow: 蝴蝶结识别。 +* flower: 花识别。 +* fringe: 刘海识别。 +* ribbon: 丝带识别。 +* rivet: 铆钉识别。 +* ruffle: 褶饰识别。 +* sequin: 亮片识别。 +* tassel: 流苏识别。 + ### MaskEdgeUltraDetail 处理较粗糙的遮罩使其获得超精细边缘。 ![image](image/mask_edge_ultra_detail_example.jpg) diff --git a/image/segformer_clothes_example.jpg b/image/segformer_clothes_example.jpg new file mode 100644 index 0000000..6a34981 Binary files /dev/null and b/image/segformer_clothes_example.jpg differ diff --git a/image/segformer_clothes_pipeline_node.jpg b/image/segformer_clothes_pipeline_node.jpg new file mode 100644 index 0000000..1b3febd Binary files /dev/null and b/image/segformer_clothes_pipeline_node.jpg differ diff --git a/image/segformer_fashion_example.jpg b/image/segformer_fashion_example.jpg new file mode 100644 index 0000000..28028fe Binary files /dev/null and b/image/segformer_fashion_example.jpg differ diff --git a/image/segformer_fashion_pipeline_node.jpg b/image/segformer_fashion_pipeline_node.jpg new file mode 100644 index 0000000..b0a8309 Binary files /dev/null and b/image/segformer_fashion_pipeline_node.jpg differ diff --git a/image/segformer_ultra_v2_node.jpg b/image/segformer_ultra_v2_node.jpg new file mode 100644 index 0000000..526c001 Binary files /dev/null and b/image/segformer_ultra_v2_node.jpg differ diff --git a/py/imagefunc.py b/py/imagefunc.py index 6291135..5865f58 100644 --- a/py/imagefunc.py +++ b/py/imagefunc.py @@ -1470,6 +1470,8 @@ class VITMatteModel: self.processor = processor def load_VITMatte_model(model_name:str, local_files_only:bool=False) -> object: + # if local_files_only: + # model_name = Path(os.path.join(folder_paths.models_dir, "vitmatte")) model_name = Path(os.path.join(folder_paths.models_dir, "vitmatte")) from transformers import VitMatteImageProcessor, VitMatteForImageMatting model = VitMatteForImageMatting.from_pretrained(model_name, local_files_only=local_files_only) diff --git a/py/segformer_ultra.py b/py/segformer_ultra.py index dcc3d3c..f69965f 100644 --- a/py/segformer_ultra.py +++ b/py/segformer_ultra.py @@ -6,14 +6,20 @@ from transformers import SegformerImageProcessor, AutoModelForSemanticSegmentati import torch.nn as nn from .imagefunc import * -NODE_NAME = 'SegformerB2ClothesUltra' + +class SegformerPipeline: + def __init__(self): + self.model_name = '' + self.segment_label = [] + +SegPipeline = SegformerPipeline() # 切割服装 -def get_segmentation(tensor_image): +def get_segmentation(tensor_image, model_name='segformer_b2_clothes'): cloth = tensor2pil(tensor_image) - model_folder_path = os.path.join(folder_paths.models_dir, "segformer_b2_clothes") + model_folder_path = os.path.join(folder_paths.models_dir, model_name) try: - model_folder_path = os.path.normpath(folder_paths.folder_names_and_paths['segformer_b2_clothes'][0][0]) + model_folder_path = os.path.normpath(folder_paths.folder_names_and_paths[model_name][0][0]) except: pass @@ -31,19 +37,20 @@ def get_segmentation(tensor_image): class Segformer_B2_Clothes: def __init__(self): + self.NODE_NAME = 'SegformerB2ClothesUltra' pass - + # Labels: 0: "Background", 1: "Hat", 2: "Hair", 3: "Sunglasses", 4: "Upper-clothes", 5: "Skirt", # 6: "Pants", 7: "Dress", 8: "Belt", 9: "Left-shoe", 10: "Right-shoe", 11: "Face", # 12: "Left-leg", 13: "Right-leg", 14: "Left-arm", 15: "Right-arm", 16: "Bag", 17: "Scarf" - + @classmethod def INPUT_TYPES(cls): method_list = ['VITMatte', 'VITMatte(local)', 'PyMatting', 'GuidedFilter', ] - device_list = ['cuda','cpu'] + device_list = ['cuda', 'cpu'] return {"required": - { - "image":("IMAGE",), + { + "image": ("IMAGE",), "face": ("BOOLEAN", {"default": False}), "hair": ("BOOLEAN", {"default": False}), "hat": ("BOOLEAN", {"default": False}), @@ -63,16 +70,18 @@ class Segformer_B2_Clothes: "detail_method": (method_list,), "detail_erode": ("INT", {"default": 12, "min": 1, "max": 255, "step": 1}), "detail_dilate": ("INT", {"default": 6, "min": 1, "max": 255, "step": 1}), - "black_point": ("FLOAT", {"default": 0.15, "min": 0.01, "max": 0.98, "step": 0.01, "display": "slider"}), - "white_point": ("FLOAT", {"default": 0.99, "min": 0.02, "max": 0.99, "step": 0.01, "display": "slider"}), + "black_point": ( + "FLOAT", {"default": 0.15, "min": 0.01, "max": 0.98, "step": 0.01, "display": "slider"}), + "white_point": ( + "FLOAT", {"default": 0.99, "min": 0.02, "max": 0.99, "step": 0.01, "display": "slider"}), "process_detail": ("BOOLEAN", {"default": True}), "device": (device_list,), "max_megapixels": ("FLOAT", {"default": 2.0, "min": 1, "max": 999, "step": 0.1}), - } + } } - RETURN_TYPES = ("IMAGE", "MASK", ) - RETURN_NAMES = ("image", "mask", ) + RETURN_TYPES = ("IMAGE", "MASK",) + RETURN_NAMES = ("image", "mask",) FUNCTION = "segformer_ultra" CATEGORY = '😺dzNodes/LayerMask' @@ -147,7 +156,8 @@ class Segformer_B2_Clothes: _mask = tensor2pil(mask_edge_detail(i, _mask, detail_range // 8 + 1, black_point, white_point)) else: _trimap = generate_VITMatte_trimap(_mask, detail_erode, detail_dilate) - _mask = generate_VITMatte(orig_image, _trimap, local_files_only=local_files_only, device=device, max_megapixels=max_megapixels) + _mask = generate_VITMatte(orig_image, _trimap, local_files_only=local_files_only, device=device, + max_megapixels=max_megapixels) _mask = tensor2pil(histogram_remap(pil2tensor(_mask), black_point, white_point)) else: _mask = mask2image(_mask) @@ -156,13 +166,360 @@ class Segformer_B2_Clothes: ret_images.append(pil2tensor(ret_image)) ret_masks.append(image2mask(_mask)) - log(f"{NODE_NAME} Processed {len(ret_images)} image(s).", message_type='finish') + log(f"{self.NODE_NAME} Processed {len(ret_images)} image(s).", message_type='finish') + return (torch.cat(ret_images, dim=0), torch.cat(ret_masks, dim=0),) + +class SegformerClothesPipelineLoader: + + def __init__(self): + self.NODE_NAME = 'SegformerClothesPipelineLoader' + pass + + # Labels: 0: "Background", 1: "Hat", 2: "Hair", 3: "Sunglasses", 4: "Upper-clothes", + # 5: "Skirt", 6: "Pants", 7: "Dress", 8: "Belt", 9: "Left-shoe", 10: "Right-shoe", + # 11: "Face", 12: "Left-leg", 13: "Right-leg", 14: "Left-arm", 15: "Right-arm", + # 17: "Scarf" + + @classmethod + def INPUT_TYPES(cls): + model_list = ['segformer_b3_clothes', 'segformer_b2_clothes'] + return {"required": + { "model": (model_list,), + "face": ("BOOLEAN", {"default": False, "label_on": "enabled(脸)", "label_off": "disabled(脸)"}), + "hair": ("BOOLEAN", {"default": False, "label_on": "enabled(头发)", "label_off": "disabled(头发)"}), + "hat": ("BOOLEAN", {"default": False, "label_on": "enabled(帽子)", "label_off": "disabled(帽子)"}), + "sunglass": ("BOOLEAN", {"default": False, "label_on": "enabled(墨镜)", "label_off": "disabled(墨镜)"}), + "left_arm": ("BOOLEAN", {"default": False, "label_on": "enabled(左臂)", "label_off": "disabled(左臂)"}), + "right_arm": ("BOOLEAN", {"default": False, "label_on": "enabled(右臂)", "label_off": "disabled(右臂)"}), + "left_leg": ("BOOLEAN", {"default": False, "label_on": "enabled(左腿)", "label_off": "disabled(左腿)"}), + "right_leg": ("BOOLEAN", {"default": False, "label_on": "enabled(右腿)", "label_off": "disabled(右腿)"}), + "left_shoe": ("BOOLEAN", {"default": False, "label_on": "enabled(左鞋)", "label_off": "disabled(左鞋)"}), + "right_shoe": ("BOOLEAN", {"default": False, "label_on": "enabled(右鞋)", "label_off": "disabled(右鞋)"}), + "upper_clothes": ("BOOLEAN", {"default": False, "label_on": "enabled(上衣)", "label_off": "disabled(上衣)"}), + "skirt": ("BOOLEAN", {"default": False, "label_on": "enabled(短裙)", "label_off": "disabled(短裙)"}), + "pants": ("BOOLEAN", {"default": False, "label_on": "enabled(裤子)", "label_off": "disabled(裤子)"}), + "dress": ("BOOLEAN", {"default": False, "label_on": "enabled(连衣裙)", "label_off": "disabled(连衣裙)"}), + "belt": ("BOOLEAN", {"default": False, "label_on": "enabled(腰带)", "label_off": "disabled(腰带)"}), + "bag": ("BOOLEAN", {"default": False, "label_on": "enabled(背包)", "label_off": "disabled(背包)"}), + "scarf": ("BOOLEAN", {"default": False, "label_on": "enabled(围巾)", "label_off": "disabled(围巾)"}), + } + } + + RETURN_TYPES = ("SegPipeline",) + RETURN_NAMES = ("segformer_pipeline",) + FUNCTION = "segformer_clothes_pipeline_loader" + CATEGORY = '😺dzNodes/LayerMask' + + def segformer_clothes_pipeline_loader(self, model, + face, hat, hair, sunglass, + left_leg, right_leg, left_arm, right_arm, left_shoe, right_shoe, + upper_clothes, skirt, pants, dress, belt, bag, scarf, + ): + + pipeline = SegformerPipeline() + labels_to_keep = [0] + if not hat: + labels_to_keep.append(1) + if not hair: + labels_to_keep.append(2) + if not sunglass: + labels_to_keep.append(3) + if not upper_clothes: + labels_to_keep.append(4) + if not skirt: + labels_to_keep.append(5) + if not pants: + labels_to_keep.append(6) + if not dress: + labels_to_keep.append(7) + if not belt: + labels_to_keep.append(8) + if not left_shoe: + labels_to_keep.append(9) + if not right_shoe: + labels_to_keep.append(10) + if not face: + labels_to_keep.append(11) + if not left_leg: + labels_to_keep.append(12) + if not right_leg: + labels_to_keep.append(13) + if not left_arm: + labels_to_keep.append(14) + if not right_arm: + labels_to_keep.append(15) + if not bag: + labels_to_keep.append(16) + if not scarf: + labels_to_keep.append(17) + pipeline.segment_label = labels_to_keep + pipeline.model_name = model + return (pipeline,) + +class SegformerFashionPipelineLoader: + + def __init__(self): + self.NODE_NAME = 'SegformerFashionPipelineLoader' + pass + + @classmethod + def INPUT_TYPES(cls): + model_list = ['segformer_b3_fashion'] + return {"required": + { "model": (model_list,), + "shirt": ("BOOLEAN", {"default": False, "label_on": "enabled(衬衫、罩衫)", "label_off": "disabled(衬衫、罩衫)"}), + "top": ("BOOLEAN", {"default": False, "label_on": "enabled(上衣、t恤)", "label_off": "disabled(上衣、t恤)"}), + "sweater": ("BOOLEAN", {"default": False, "label_on": "enabled(毛衣)", "label_off": "disabled(毛衣)"}), + "cardigan": ("BOOLEAN", {"default": False, "label_on": "enabled(开襟毛衫)", "label_off": "disabled(开襟毛衫)"}), + "jacket": ("BOOLEAN", {"default": False, "label_on": "enabled(夹克)", "label_off": "disabled(夹克)"}), + "vest": ("BOOLEAN", {"default": False, "label_on": "enabled(背心)", "label_off": "disabled(背心)"}), + "pants": ("BOOLEAN", {"default": False, "label_on": "enabled(裤子)", "label_off": "disabled(裤子)"}), + "shorts": ("BOOLEAN", {"default": False, "label_on": "enabled(短裤)", "label_off": "disabled(短裤)"}), + "skirt": ("BOOLEAN", {"default": False, "label_on": "enabled(裙子)", "label_off": "disabled(裙子)"}), + "coat": ("BOOLEAN", {"default": False, "label_on": "enabled(外套)", "label_off": "disabled(外套)"}), + "dress": ("BOOLEAN", {"default": False, "label_on": "enabled(连衣裙)", "label_off": "disabled(连衣裙)"}), + "jumpsuit": ("BOOLEAN", {"default": False, "label_on": "enabled(连身裤)", "label_off": "disabled(连身裤)"}), + "cape": ("BOOLEAN", {"default": False, "label_on": "enabled(斗篷)", "label_off": "disabled(斗篷)"}), + "glasses": ("BOOLEAN", {"default": False, "label_on": "enabled(眼镜)", "label_off": "disabled(眼镜)"}), + "hat": ("BOOLEAN", {"default": False, "label_on": "enabled(帽子)", "label_off": "disabled(帽子)"}), + "hairaccessory": ("BOOLEAN", {"default": False, "label_on": "enabled(头带)", "label_off": "disabled(头带)"}), + "tie": ("BOOLEAN", {"default": False, "label_on": "enabled(领带)", "label_off": "disabled(领带)"}), + "glove": ("BOOLEAN", {"default": False, "label_on": "enabled(手套)", "label_off": "disabled(手套)"}), + "watch": ("BOOLEAN", {"default": False, "label_on": "enabled(手表)", "label_off": "disabled(手表)"}), + "belt": ("BOOLEAN", {"default": False, "label_on": "enabled(皮带)", "label_off": "disabled(皮带)"}), + "legwarmer": ("BOOLEAN", {"default": False, "label_on": "enabled(腿套)", "label_off": "disabled(腿套)"}), + "tights": ("BOOLEAN", {"default": False, "label_on": "enabled(裤袜)","label_off": "disabled(裤袜)"}), + "sock": ("BOOLEAN", {"default": False, "label_on": "enabled(袜子)", "label_off": "disabled(袜子)"}), + "shoe": ("BOOLEAN", {"default": False, "label_on": "enabled(鞋子)", "label_off": "disabled(鞋子)"}), + "bagwallet": ("BOOLEAN", {"default": False, "label_on": "enabled(手包)", "label_off": "disabled(手包)"}), + "scarf": ("BOOLEAN", {"default": False, "label_on": "enabled(围巾)", "label_off": "disabled(围巾)"}), + "umbrella": ("BOOLEAN", {"default": False, "label_on": "enabled(雨伞)", "label_off": "disabled(雨伞)"}), + "hood": ("BOOLEAN", {"default": False, "label_on": "enabled(兜帽)", "label_off": "disabled(兜帽)"}), + "collar": ("BOOLEAN", {"default": False, "label_on": "enabled(衣领)", "label_off": "disabled(衣领)"}), + "lapel": ("BOOLEAN", {"default": False, "label_on": "enabled(翻领)", "label_off": "disabled(翻领)"}), + "epaulette": ("BOOLEAN", {"default": False, "label_on": "enabled(肩章)", "label_off": "disabled(肩章)"}), + "sleeve": ("BOOLEAN", {"default": False, "label_on": "enabled(袖子)", "label_off": "disabled(袖子)"}), + "pocket": ("BOOLEAN", {"default": False, "label_on": "enabled(口袋)", "label_off": "disabled(口袋)"}), + "neckline": ("BOOLEAN", {"default": False, "label_on": "enabled(领口)", "label_off": "disabled(领口)"}), + "buckle": ("BOOLEAN", {"default": False, "label_on": "enabled(带扣)", "label_off": "disabled(带扣)"}), + "zipper": ("BOOLEAN", {"default": False, "label_on": "enabled(拉链)", "label_off": "disabled(拉链)"}), + "applique": ("BOOLEAN", {"default": False, "label_on": "enabled(贴花)", "label_off": "disabled(贴花)"}), + "bead": ("BOOLEAN", {"default": False, "label_on": "enabled(珠子)", "label_off": "disabled(珠子)"}), + "bow": ("BOOLEAN", {"default": False, "label_on": "enabled(蝴蝶结)", "label_off": "disabled(蝴蝶结)"}), + "flower": ("BOOLEAN", {"default": False, "label_on": "enabled(花)", "label_off": "disabled(花)"}), + "fringe": ("BOOLEAN", {"default": False, "label_on": "enabled(刘海)", "label_off": "disabled(刘海)"}), + "ribbon": ("BOOLEAN", {"default": False, "label_on": "enabled(丝带)", "label_off": "disabled(丝带)"}), + "rivet": ("BOOLEAN", {"default": False, "label_on": "enabled(铆钉)", "label_off": "disabled(铆钉)"}), + "ruffle": ("BOOLEAN", {"default": False, "label_on": "enabled(褶饰)", "label_off": "disabled(褶饰)"}), + "sequin": ("BOOLEAN", {"default": False, "label_on": "enabled(亮片)", "label_off": "disabled(亮片)"}), + "tassel": ("BOOLEAN", {"default": False, "label_on": "enabled(流苏)", "label_off": "disabled(流苏)"}), + } + } + + RETURN_TYPES = ("SegPipeline",) + RETURN_NAMES = ("segformer_pipeline",) + FUNCTION = "segformer_fashion_pipeline_loader" + CATEGORY = '😺dzNodes/LayerMask' + + def segformer_fashion_pipeline_loader(self, model, + shirt, top, sweater, cardigan, jacket, vest, pants, + shorts, skirt, coat, dress, jumpsuit, cape, glasses, + hat, hairaccessory, tie, glove, watch, belt, legwarmer, + tights, sock, shoe, bagwallet, scarf, umbrella, hood, + collar, lapel, epaulette, sleeve, pocket, neckline, + buckle, zipper, applique, bead, bow, flower, fringe, + ribbon, rivet, ruffle, sequin, tassel + ): + + pipeline = SegformerPipeline() + labels_to_keep = [0] + if not shirt: + labels_to_keep.append(1) + if not top: + labels_to_keep.append(2) + if not sweater: + labels_to_keep.append(3) + if not cardigan: + labels_to_keep.append(4) + if not jacket: + labels_to_keep.append(5) + if not vest: + labels_to_keep.append(6) + if not pants: + labels_to_keep.append(7) + if not shorts: + labels_to_keep.append(8) + if not skirt: + labels_to_keep.append(9) + if not coat: + labels_to_keep.append(10) + if not dress: + labels_to_keep.append(11) + if not jumpsuit: + labels_to_keep.append(12) + if not cape: + labels_to_keep.append(13) + if not glasses: + labels_to_keep.append(14) + if not hat: + labels_to_keep.append(15) + if not hairaccessory: + labels_to_keep.append(16) + if not tie: + labels_to_keep.append(17) + if not glove: + labels_to_keep.append(18) + if not watch: + labels_to_keep.append(19) + if not belt: + labels_to_keep.append(20) + if not legwarmer: + labels_to_keep.append(21) + if not tights: + labels_to_keep.append(22) + if not sock: + labels_to_keep.append(23) + if not shoe: + labels_to_keep.append(24) + if not bagwallet: + labels_to_keep.append(25) + if not scarf: + labels_to_keep.append(26) + if not umbrella: + labels_to_keep.append(27) + if not hood: + labels_to_keep.append(28) + if not collar: + labels_to_keep.append(29) + if not lapel: + labels_to_keep.append(30) + if not epaulette: + labels_to_keep.append(31) + if not sleeve: + labels_to_keep.append(32) + if not pocket: + labels_to_keep.append(33) + if not neckline: + labels_to_keep.append(34) + if not buckle: + labels_to_keep.append(35) + if not zipper: + labels_to_keep.append(36) + if not applique: + labels_to_keep.append(37) + if not bead: + labels_to_keep.append(38) + if not bow: + labels_to_keep.append(39) + if not flower: + labels_to_keep.append(40) + if not fringe: + labels_to_keep.append(41) + if not ribbon: + labels_to_keep.append(42) + if not rivet: + labels_to_keep.append(43) + if not ruffle: + labels_to_keep.append(44) + if not sequin: + labels_to_keep.append(45) + if not tassel: + labels_to_keep.append(46) + + pipeline.segment_label = labels_to_keep + pipeline.model_name = model + return (pipeline,) + +class SegformerUltraV2: + + def __init__(self): + self.NODE_NAME = 'SegformerUltraV2' + pass + + @classmethod + def INPUT_TYPES(cls): + method_list = ['VITMatte', 'VITMatte(local)', 'PyMatting', 'GuidedFilter', ] + device_list = ['cuda', 'cpu'] + return {"required": + { + "image": ("IMAGE",), + "segformer_pipeline": ("SegPipeline",), + "detail_method": (method_list,), + "detail_erode": ("INT", {"default": 8, "min": 1, "max": 255, "step": 1}), + "detail_dilate": ("INT", {"default": 6, "min": 1, "max": 255, "step": 1}), + "black_point": ("FLOAT", {"default": 0.01, "min": 0.01, "max": 0.98, "step": 0.01, "display": "slider"}), + "white_point": ("FLOAT", {"default": 0.99, "min": 0.02, "max": 0.99, "step": 0.01, "display": "slider"}), + "process_detail": ("BOOLEAN", {"default": True}), + "device": (device_list,), + "max_megapixels": ("FLOAT", {"default": 2.0, "min": 1, "max": 999, "step": 0.1}), + } + } + + RETURN_TYPES = ("IMAGE", "MASK",) + RETURN_NAMES = ("image", "mask",) + FUNCTION = "segformer_ultra_v2" + CATEGORY = '😺dzNodes/LayerMask' + + def segformer_ultra_v2(self, image, segformer_pipeline, + detail_method, detail_erode, detail_dilate, black_point, white_point, + process_detail, device, max_megapixels, + ): + model = segformer_pipeline.model_name + labels_to_keep = segformer_pipeline.segment_label + ret_images = [] + ret_masks = [] + + if detail_method == 'VITMatte(local)': + local_files_only = True + else: + local_files_only = False + + for i in image: + pred_seg, cloth = get_segmentation(i, model_name=model) + i = torch.unsqueeze(i, 0) + i = pil2tensor(tensor2pil(i).convert('RGB')) + orig_image = tensor2pil(i).convert('RGB') + + mask = np.isin(pred_seg, labels_to_keep).astype(np.uint8) + + # 创建agnostic-mask图像 + mask_image = Image.fromarray((1 - mask) * 255) + mask_image = mask_image.convert("L") + _mask = pil2tensor(mask_image) + + detail_range = detail_erode + detail_dilate + if process_detail: + if detail_method == 'GuidedFilter': + _mask = guided_filter_alpha(i, _mask, detail_range // 6 + 1) + _mask = tensor2pil(histogram_remap(_mask, black_point, white_point)) + elif detail_method == 'PyMatting': + _mask = tensor2pil(mask_edge_detail(i, _mask, detail_range // 8 + 1, black_point, white_point)) + else: + _trimap = generate_VITMatte_trimap(_mask, detail_erode, detail_dilate) + _mask = generate_VITMatte(orig_image, _trimap, local_files_only=local_files_only, device=device, + max_megapixels=max_megapixels) + _mask = tensor2pil(histogram_remap(pil2tensor(_mask), black_point, white_point)) + else: + _mask = mask2image(_mask) + + ret_image = RGB2RGBA(orig_image, _mask.convert('L')) + ret_images.append(pil2tensor(ret_image)) + ret_masks.append(image2mask(_mask)) + + log(f"{self.NODE_NAME} Processed {len(ret_images)} image(s).", message_type='finish') return (torch.cat(ret_images, dim=0), torch.cat(ret_masks, dim=0),) NODE_CLASS_MAPPINGS = { - "LayerMask: SegformerB2ClothesUltra": Segformer_B2_Clothes + "LayerMask: SegformerB2ClothesUltra": Segformer_B2_Clothes, + "LayerMask: SegformerUltraV2": SegformerUltraV2, + "LayerMask: SegformerClothesPipelineLoader": SegformerClothesPipelineLoader, + "LayerMask: SegformerFashionPipelineLoader": SegformerFashionPipelineLoader, } NODE_DISPLAY_NAME_MAPPINGS = { - "LayerMask: SegformerB2ClothesUltra": "LayerMask: Segformer B2 Clothes Ultra" + "LayerMask: SegformerB2ClothesUltra": "LayerMask: Segformer B2 Clothes Ultra", + "LayerMask: SegformerUltraV2": "LayerMask: Segformer Ultra V2", + "LayerMask: SegformerClothesPipelineLoader": "LayerMask: Segformer Clothes Pipeline", + "LayerMask: SegformerFashionPipelineLoader": "LayerMask: Segformer Fashion Pipeline" } + diff --git a/pyproject.toml b/pyproject.toml index c1eed51..f35eabb 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,7 +1,7 @@ [project] name = "comfyui_layerstyle" description = "A set of nodes for ComfyUI it generate image like Adobe Photoshop's Layer Style. the Drop Shadow is first completed node, and follow-up work is in progress." -version = "1.0.17" +version = "1.0.18" license = "MIT" dependencies = ["numpy", "pillow", "torch", "matplotlib", "Scipy", "scikit_image", "opencv-contrib-python", "pymatting", "segment_anything", "timm", "addict", "yapf", "colour-science", "wget", "mediapipe", "loguru", "typer_config", "fastapi", "rich", "google-generativeai", "diffusers", "omegaconf", "tqdm", "transformers", "kornia", "image-reward", "ultralytics", "blend_modes", "blind-watermark", "qrcode", "pyzbar", "psd-tools"] diff --git a/workflow/segformet_clothes_example.json b/workflow/segformet_clothes_example.json new file mode 100644 index 0000000..70fe346 --- /dev/null +++ b/workflow/segformet_clothes_example.json @@ -0,0 +1,251 @@ +{ + "last_node_id": 13, + "last_link_id": 21, + "nodes": [ + { + "id": 11, + "type": "LayerMask: SegformerClothesPipelineLoader", + "pos": [ + -1510, + -370 + ], + "size": { + "0": 315, + "1": 466 + }, + "flags": {}, + "order": 0, + "mode": 0, + "outputs": [ + { + "name": "segformer_pipeline", + "type": "SegPipeline", + "links": [ + 16 + ], + "shape": 3, + "slot_index": 0 + } + ], + "title": "LayerMask: Segformer Clothes Pipeline", + "properties": { + "Node name for S&R": "LayerMask: SegformerClothesPipelineLoader" + }, + "widgets_values": [ + "segformer_b3_clothes", + true, + true, + true, + true, + true, + true, + true, + true, + true, + true, + true, + true, + true, + true, + true, + true, + true + ] + }, + { + "id": 9, + "type": "LayerMask: SegformerUltraV2", + "pos": [ + -1160, + -240 + ], + "size": { + "0": 315, + "1": 246 + }, + "flags": {}, + "order": 2, + "mode": 0, + "inputs": [ + { + "name": "image", + "type": "IMAGE", + "link": 13 + }, + { + "name": "segformer_pipeline", + "type": "SegPipeline", + "link": 16 + } + ], + "outputs": [ + { + "name": "image", + "type": "IMAGE", + "links": [ + 14 + ], + "shape": 3, + "slot_index": 0 + }, + { + "name": "mask", + "type": "MASK", + "links": [ + 15 + ], + "shape": 3, + "slot_index": 1 + } + ], + "properties": { + "Node name for S&R": "LayerMask: SegformerUltraV2" + }, + "widgets_values": [ + "VITMatte", + 44, + 6, + 0.01, + 0.99, + true, + "cuda", + 2 + ] + }, + { + "id": 4, + "type": "PreviewImage", + "pos": [ + -798, + -614 + ], + "size": [ + 230.18093750785056, + 422.02388956433197 + ], + "flags": {}, + "order": 3, + "mode": 0, + "inputs": [ + { + "name": "images", + "type": "IMAGE", + "link": 14 + } + ], + "properties": { + "Node name for S&R": "PreviewImage" + } + }, + { + "id": 6, + "type": "LayerMask: MaskPreview", + "pos": [ + -800, + -140 + ], + "size": [ + 236.40463640741564, + 438.79732236746304 + ], + "flags": {}, + "order": 4, + "mode": 0, + "inputs": [ + { + "name": "mask", + "type": "MASK", + "link": 15 + } + ], + "properties": { + "Node name for S&R": "LayerMask: MaskPreview" + } + }, + { + "id": 3, + "type": "LoadImage", + "pos": [ + -1919, + -513 + ], + "size": [ + 330.2955113657599, + 658.6634932172642 + ], + "flags": {}, + "order": 1, + "mode": 0, + "outputs": [ + { + "name": "IMAGE", + "type": "IMAGE", + "links": [ + 13 + ], + "shape": 3, + "slot_index": 0 + }, + { + "name": "MASK", + "type": "MASK", + "links": null, + "shape": 3 + } + ], + "properties": { + "Node name for S&R": "LoadImage" + }, + "widgets_values": [ + "768x1344_dress.png", + "image" + ] + } + ], + "links": [ + [ + 13, + 3, + 0, + 9, + 0, + "IMAGE" + ], + [ + 14, + 9, + 0, + 4, + 0, + "IMAGE" + ], + [ + 15, + 9, + 1, + 6, + 0, + "MASK" + ], + [ + 16, + 11, + 0, + 9, + 1, + "SegPipeline" + ] + ], + "groups": [], + "config": {}, + "extra": { + "ds": { + "scale": 0.5445000000000026, + "offset": [ + 3409.5755571201344, + 1368.2914043226942 + ] + } + }, + "version": 0.4 +} \ No newline at end of file diff --git a/workflow/segformet_fashion_example.json b/workflow/segformet_fashion_example.json new file mode 100644 index 0000000..b228863 --- /dev/null +++ b/workflow/segformet_fashion_example.json @@ -0,0 +1,280 @@ +{ + "last_node_id": 13, + "last_link_id": 21, + "nodes": [ + { + "id": 5, + "type": "PreviewImage", + "pos": [ + -770, + -460 + ], + "size": { + "0": 337.95587158203125, + "1": 447.5793151855469 + }, + "flags": {}, + "order": 3, + "mode": 0, + "inputs": [ + { + "name": "images", + "type": "IMAGE", + "link": 18 + } + ], + "properties": { + "Node name for S&R": "PreviewImage" + } + }, + { + "id": 7, + "type": "LayerMask: MaskPreview", + "pos": [ + -770, + 40 + ], + "size": { + "0": 333.88427734375, + "1": 474.7603759765625 + }, + "flags": {}, + "order": 4, + "mode": 0, + "inputs": [ + { + "name": "mask", + "type": "MASK", + "link": 19 + } + ], + "properties": { + "Node name for S&R": "LayerMask: MaskPreview" + } + }, + { + "id": 12, + "type": "LayerMask: SegformerUltraV2", + "pos": [ + -1160, + -90 + ], + "size": { + "0": 315, + "1": 246 + }, + "flags": {}, + "order": 2, + "mode": 0, + "inputs": [ + { + "name": "image", + "type": "IMAGE", + "link": 21 + }, + { + "name": "segformer_pipeline", + "type": "SegPipeline", + "link": 20 + } + ], + "outputs": [ + { + "name": "image", + "type": "IMAGE", + "links": [ + 18 + ], + "shape": 3, + "slot_index": 0 + }, + { + "name": "mask", + "type": "MASK", + "links": [ + 19 + ], + "shape": 3, + "slot_index": 1 + } + ], + "properties": { + "Node name for S&R": "LayerMask: SegformerUltraV2" + }, + "widgets_values": [ + "VITMatte", + 8, + 6, + 0.01, + 0.99, + true, + "cuda", + 2 + ] + }, + { + "id": 10, + "type": "LayerMask: SegformerFashionPipelineLoader", + "pos": [ + -1580, + -540 + ], + "size": { + "0": 315, + "1": 1162 + }, + "flags": {}, + "order": 0, + "mode": 0, + "outputs": [ + { + "name": "segformer_pipeline", + "type": "SegPipeline", + "links": [ + 20 + ], + "shape": 3, + "slot_index": 0 + } + ], + "title": "LayerMask: Segformer Fashion Pipeline", + "properties": { + "Node name for S&R": "LayerMask: SegformerFashionPipelineLoader" + }, + "widgets_values": [ + "segformer_b3_fashion", + true, + true, + true, + true, + true, + true, + true, + true, + true, + true, + true, + true, + true, + true, + true, + true, + true, + true, + true, + true, + true, + true, + true, + true, + true, + true, + true, + true, + true, + true, + true, + true, + true, + true, + true, + true, + true, + true, + true, + true, + true, + true, + true, + true, + true, + true + ] + }, + { + "id": 13, + "type": "LoadImage", + "pos": [ + -1990, + -200 + ], + "size": { + "0": 311.8735656738281, + "1": 470.3501892089844 + }, + "flags": {}, + "order": 1, + "mode": 0, + "outputs": [ + { + "name": "IMAGE", + "type": "IMAGE", + "links": [ + 21 + ], + "shape": 3, + "slot_index": 0 + }, + { + "name": "MASK", + "type": "MASK", + "links": null, + "shape": 3 + } + ], + "properties": { + "Node name for S&R": "LoadImage" + }, + "widgets_values": [ + "768x1344_dress.png", + "image" + ] + } + ], + "links": [ + [ + 18, + 12, + 0, + 5, + 0, + "IMAGE" + ], + [ + 19, + 12, + 1, + 7, + 0, + "MASK" + ], + [ + 20, + 10, + 0, + 12, + 1, + "SegPipeline" + ], + [ + 21, + 13, + 0, + 12, + 0, + "IMAGE" + ] + ], + "groups": [], + "config": {}, + "extra": { + "ds": { + "scale": 0.7972024500000043, + "offset": [ + 2634.915464714221, + 818.4797865982553 + ] + } + }, + "version": 0.4 +} \ No newline at end of file