nodes/chflame163-ComfyUI_LayerStyle
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

ObjectDetector nodes add bboxes sort option

Browse Source
This commit is contained in:
chflame163
2024-09-03 18:21:33 +08:00
parent be649864ff
commit f62e577217
12 changed files with 787 additions and 694 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.MD
+5
View File
@@ -103,6 +103,7 @@ When this error has occurred, please check the network environment.
## Update
<font size="4">**If the dependency package error after updating, please double clicking ```repair_dependency.bat``` (for Official ComfyUI Protable) or ```repair_dependency_aki.bat``` (for ComfyUI-aki-v1.x) in the plugin folder to reinstall the dependency packages. </font><br />
* The Object Detector nodes added sort bbox option, which allows sorting from left to right, top to bottom, and large to small, making object selection more intuitive and convenient. The nodes released yesterday has been abandoned, please manually replace it with the new version node (sorry).
* Commit [SAM2Ultra](#SAM2Ultra), [SAM2VideoUltra](#SAM2VideoUltra), [ObjectDetectorFL2](#ObjectDetectorFL2), [ObjectDetectorYOLOWorld](#ObjectDetectorYOLOWorld), [ObjectDetectorYOLO8](#ObjectDetectorYOLO8), [ObjectDetectorMask](#ObjectDetectorMask) and [BBoxJoin](#BBoxJoin) nodes.
Download models from [BaiduNetdisk](https://pan.baidu.com/s/1xaQYBA6ktxvAxm310HXweQ?pwd=auki) or [huggingface.co/Kijai/sam2-safetensors](https://huggingface.co/Kijai/sam2-safetensors/tree/main) and copy to ```ComfyUI/models/sam2``` folder,
Download models from [BaiduNetdisk](https://pan.baidu.com/s/1QpjajeTA37vEAU2OQnbDcQ?pwd=nqsk) or [GoogleDrive](https://drive.google.com/drive/folders/1nrsfq4S-yk9ewJgwrhXAoNVqIFLZ1at7?usp=sharing) and copy to ```ComfyUI/models/yolo-world``` folder.
@@ -1656,6 +1657,7 @@ Node Options:
* image: The image to segment.
* florence2_model: Florence2 model, it from [LoadFlorence2Model](#LoadFlorence2Model) node.
* prompt: Describe the object that needs to be identified.
* sort_method: The selection box sorting method has 3 options: "left_to_right", "top_to_bottom" and "big_to_small".
* bbox_select: Select the input box data. There are three options: "all" to select all, "first" to select the box with the highest confidence, and "by_index" to specify the index of the box.
* select_index: This option is valid when bbox_delect is 'by_index'. 0 is the first one. Multiple values can be entered, separated by any non numeric character, including but not limited to commas, periods, semicolons, spaces or letters, and even Chinese.
@@ -1669,6 +1671,7 @@ Node Options:
* confidence_threshold: The threshold of confidence.
* nms_iou_threshold: The threshold of Non-Maximum Suppression.
* prompt: Describe the object that needs to be identified.
* sort_method: The selection box sorting method has 3 options: "left_to_right", "top_to_bottom" and "big_to_small".
* bbox_select: Select the input box data. There are three options: "all" to select all, "first" to select the box with the highest confidence, and "by_index" to specify the index of the box.
* select_index: This option is valid when bbox_delect is 'by_index'. 0 is the first one. Multiple values can be entered, separated by any non numeric character, including but not limited to commas, periods, semicolons, spaces or letters, and even Chinese.
@@ -1680,6 +1683,7 @@ Node Options:
![image](image/object_detector_yolo8_node.jpg)
* image: The image to segment.
* yolo_model: Choose the yolo model.
* sort_method: The selection box sorting method has 3 options: "left_to_right", "top_to_bottom" and "big_to_small".
* bbox_select: Select the input box data. There are three options: "all" to select all, "first" to select the box with the highest confidence, and "by_index" to specify the index of the box.
* select_index: This option is valid when bbox_delect is 'by_index'. 0 is the first one. Multiple values can be entered, separated by any non numeric character, including but not limited to commas, periods, semicolons, spaces or letters, and even Chinese.
@@ -1689,6 +1693,7 @@ Use mask as recognition box data. All areas surrounded by white areas on the mas
Node Options:
![image](image/object_detector_mask_node.jpg)
* object_mask: The mask input.
* sort_method: The selection box sorting method has 3 options: "left_to_right", "top_to_bottom" and "big_to_small".
* bbox_select: Select the input box data. There are three options: "all" to select all, "first" to select the box with the highest confidence, and "by_index" to specify the index of the box.
* select_index: This option is valid when bbox_delect is 'by_index'. 0 is the first one. Multiple values can be entered, separated by any non numeric character, including but not limited to commas, periods, semicolons, spaces or letters, and even Chinese.
README_CN.MD
+5
View File
@@ -105,6 +105,7 @@ git clone https://github.com/chflame163/ComfyUI_LayerStyle.git
## 更新说明
<font size="4">**如果本插件更新后出现依赖包错误，请双击运行插件目录下的```install_requirements.bat```(官方便携包)，或 ```install_requirements_aki.bat```(秋叶整合包) 重新安装依赖包。
* ObjectDectector节点组增加sort bbox功能， 可按从左到右、从上到下、从大到小排序，选择物体更直观方便。昨天发布的节点已放弃，请手动更换为新版节点(对不起)。
* 添加 [SAM2Ultra](#SAM2Ultra), [SAM2VideoUltra](#SAM2VideoUltra), [ObjectDetectorFL2](#ObjectDetectorFL2), [ObjectDetectorYOLOWorld](#ObjectDetectorYOLOWorld), [ObjectDetectorYOLO8](#ObjectDetectorYOLO8), [ObjectDetectorMask](#ObjectDetectorMask) 和 [BBoxJoin](#BBoxJoin)节点。
请从[百度网盘](https://pan.baidu.com/s/1xaQYBA6ktxvAxm310HXweQ?pwd=auki) 或者 [huggingface.co/Kijai/sam2-safetensors](https://huggingface.co/Kijai/sam2-safetensors/tree/main)下载全部模型文件并复制到```ComfyUI/models/sam2```文件夹;
从 [百度网盘](https://pan.baidu.com/s/1QpjajeTA37vEAU2OQnbDcQ?pwd=nqsk) 或[GoogleDrive](https://drive.google.com/drive/folders/1nrsfq4S-yk9ewJgwrhXAoNVqIFLZ1at7?usp=sharing)下载模型文件并复制到```ComfyUI/models/yolo-world```文件夹。
@@ -1633,6 +1634,7 @@ https://github.com/user-attachments/assets/b2a45c96-4be1-4470-8ceb-addaf301b0cb
* image: 图片输入。
* florence2_model: Florence2模型。从[Florence2模型加载器](#LoadFlorence2Model)输入。
* prompt: 描述需要识别的对象。
* sort_method: 选择框排序方法， 有3个选项："left_to_right"为从左到右排序，"top_to_bottom"为从上到下排序，"big_to_small"为从大到小排序。
* bbox_select: 选择输入的框数据。有3个选项："all"为全部选择，"first"为选择置信度最高的框，"by_index"可以指定框的索引。
* select_index: 当bbox_select为"by_index"时，此选项有效。0为第一张。可以输入多个值，中间用任意非数字字符分隔，包括不仅限于逗号，句号，分号，空格或者字母，甚至中文。
@@ -1647,6 +1649,7 @@ https://github.com/user-attachments/assets/b2a45c96-4be1-4470-8ceb-addaf301b0cb
* confidence_threshold: 置信度阈值。
* nms_iou_threshold: 非极大值抑制阈值。
* prompt: 描述需要识别的对象。
* sort_method: 选择框排序方法， 有3个选项："left_to_right"为从左到右排序，"top_to_bottom"为从上到下排序，"big_to_small"为从大到小排序。
* bbox_select: 选择输入的框数据。有3个选项："all"为全部选择，"first"为选择置信度最高的框，"by_index"可以指定框的索引。
* select_index: 当bbox_select为"by_index"时，此选项有效。0为第一张。可以输入多个值，中间用任意非数字字符分隔，包括不仅限于逗号，句号，分号，空格或者字母，甚至中文。
@@ -1658,6 +1661,7 @@ https://github.com/user-attachments/assets/b2a45c96-4be1-4470-8ceb-addaf301b0cb
![image](image/object_detector_yolo8_node.jpg)
* image: 图片输入。
* yolo_model: 选择yolo模型。
* sort_method: 选择框排序方法， 有3个选项："left_to_right"为从左到右排序，"top_to_bottom"为从上到下排序，"big_to_small"为从大到小排序。
* bbox_select: 选择输入的框数据。有3个选项："all"为全部选择，"first"为选择置信度最高的框，"by_index"可以指定框的索引。
* select_index: 当bbox_select为"by_index"时，此选项有效。0为第一张。可以输入多个值，中间用任意非数字字符分隔，包括不仅限于逗号，句号，分号，空格或者字母，甚至中文。
@@ -1667,6 +1671,7 @@ https://github.com/user-attachments/assets/b2a45c96-4be1-4470-8ceb-addaf301b0cb
节点选项说明:
![image](image/object_detector_mask_node.jpg)
* object_mask: 遮罩输入。
* sort_method: 选择框排序方法， 有3个选项："left_to_right"为从左到右排序，"top_to_bottom"为从上到下排序，"big_to_small"为从大到小排序。
* bbox_select: 选择输入的框数据。有3个选项："all"为全部选择，"first"为选择置信度最高的框，"by_index"可以指定框的索引。
* select_index: 当bbox_select为"by_index"时，此选项有效。0为第一张。可以输入多个值，中间用任意非数字字符分隔，包括不仅限于逗号，句号，分号，空格或者字母，甚至中文。
image/object_detector_fl2_node.jpg
BIN
View File
Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 51 KiB

After

Width:  |  Height:  |  Size: 69 KiB

image/object_detector_mask_node.jpg
BIN
View File
Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 57 KiB

After

Width:  |  Height:  |  Size: 66 KiB

image/object_detector_yolo8_node.jpg
BIN
View File
Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 64 KiB

After

Width:  |  Height:  |  Size: 74 KiB

image/object_detector_yolo_world_node.jpg
BIN
View File
Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 88 KiB

After

Width:  |  Height:  |  Size: 106 KiB

image/sam2_example.jpg
BIN
View File
Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 657 KiB

After

Width:  |  Height:  |  Size: 637 KiB

py/imagefunc.py
+34 -19
View File
@@ -157,25 +157,6 @@ def mask2image(mask:torch.Tensor) -> Image:
'''Image Functions'''
def draw_bounding_boxes(image:Image, bboxes:list, color:str="#FF0000", line_width:int=5) -> Image:
"""
Draw bounding boxes on the image using the coordinates provided in the bboxes dictionary.
"""
if len(bboxes) > 0:
draw = ImageDraw.Draw(image)
width, height = image.size
if line_width < 0: #auto line width
line_width = (image.width + image.height) // 300
for box in bboxes:
xmin = min(box[0],box[2])
xmax = max(box[0],box[2])
ymin = min(box[1],box[3])
ymax = max(box[1], box[3])
draw.rectangle([xmin, ymin, xmax, ymax], outline=color, width=line_width)
return image
# 颜色加深
def blend_color_burn(background_image:Image, layer_image:Image) -> Image:
img_1 = cv22ski(pil2cv2(background_image))
@@ -1947,6 +1928,14 @@ def is_contain_chinese(check_str:str) -> bool:
return True
return False
# 生成随机颜色
def generate_random_color():
"""
Generate a random color in hexadecimal format.
"""
# random.seed(int(time.time()))
return "#{:06x}".format(random.randint(0x101010, 0xFFFFFF))
# 提取字符串中的int数为列表
def extract_numbers(string):
return [int(s) for s in re.findall(r'\d+', string)]
@@ -2234,6 +2223,32 @@ FONT_LIST = list(FONT_DICT.keys())
log(f'Find {len(FONT_LIST)} Fonts in {default_font_dir}')
def draw_bounding_boxes(image: Image, bboxes: list, color: str = "#FF0000", line_width: int = 5) -> Image:
"""
Draw bounding boxes on the image using the coordinates provided in the bboxes dictionary.
"""
font_size = 25
font = ImageFont.truetype(list(FONT_DICT.items())[0][1], font_size)
if len(bboxes) > 0:
draw = ImageDraw.Draw(image)
width, height = image.size
if line_width < 0: # auto line width
line_width = (image.width + image.height) // 1000
for index, box in enumerate(bboxes):
random_color = generate_random_color()
if color != "random":
random_color = color
xmin = min(box[0], box[2])
xmax = max(box[0], box[2])
ymin = min(box[1], box[3])
ymax = max(box[1], box[3])
draw.rectangle([xmin, ymin, xmax, ymax], outline=random_color, width=line_width)
draw.text((xmin, ymin - font_size*1.2), str(index), font=font, fill=random_color)
return image
gemini_generate_config = {
"temperature": 0,
py/object_detector.py
+26 -9
View File
@@ -2,7 +2,17 @@
from .imagefunc import *
select_list = ["all", "first", "by_index"]
sort_method_list = ["left_to_right", "top_to_bottom", "big_to_small"]
def sort_bboxes(bboxes:list, method:str) -> list:
sorted_bboxes = []
if method == "left_to_right":
sorted_bboxes = sorted(bboxes, key=lambda bbox: bbox[0])
elif method == "top_to_bottom":
sorted_bboxes = sorted(bboxes, key=lambda bbox: bbox[1])
else:# bit_to_small
sorted_bboxes = sorted(bboxes, key=lambda bbox: (bbox[2] - bbox[0]) * (bbox[3] - bbox[1]), reverse=True)
return sorted_bboxes
def select_bboxes(bboxes:list, bbox_select:str, select_index:str) -> list:
indexs = extract_numbers(select_index)
@@ -66,6 +76,7 @@ class LS_OBJECT_DETECTOR_FL2:
"image": ("IMAGE", ), #
"prompt": ("STRING", {"default": "subject"}),
"florence2_model": ("FLORENCE2",),
"sort_method": (sort_method_list,),
"bbox_select": (select_list,),
"select_index": ("STRING", {"default": "0,"},),
},
@@ -78,7 +89,7 @@ class LS_OBJECT_DETECTOR_FL2:
FUNCTION = 'object_detector_fl2'
CATEGORY = '😺dzNodes/LayerMask'
def object_detector_fl2(self, image, prompt, florence2_model, bbox_select, select_index):
def object_detector_fl2(self, image, prompt, florence2_model, sort_method, bbox_select, select_index):
bboxes = []
ret_previews = []
@@ -102,8 +113,9 @@ class LS_OBJECT_DETECTOR_FL2:
results["height"] = img.height
bboxes = self.fbboxes_to_list(results)
bboxes = sort_bboxes(bboxes, sort_method)
bboxes = select_bboxes(bboxes, bbox_select, select_index)
preview = draw_bounding_boxes(img, bboxes, color="#FF0000", line_width=-1)
preview = draw_bounding_boxes(img, bboxes, color="random", line_width=-1)
ret_previews.append(pil2tensor(preview))
if len(bboxes) == 0:
log(f"{self.NODE_NAME} no object found", message_type='warning')
@@ -169,6 +181,7 @@ class LS_OBJECT_DETECTOR_MASK:
return {
"required": {
"object_mask": ("MASK",),
"sort_method": (sort_method_list,),
"bbox_select": (select_list,),
"select_index": ("STRING", {"default": "0,"},),
},
@@ -181,7 +194,7 @@ class LS_OBJECT_DETECTOR_MASK:
FUNCTION = 'object_detector_mask'
CATEGORY = '😺dzNodes/LayerMask'
def object_detector_mask(self, object_mask, bbox_select, select_index):
def object_detector_mask(self, object_mask, sort_method, bbox_select, select_index):
bboxes = []
if object_mask.dim() == 2:
@@ -196,9 +209,10 @@ class LS_OBJECT_DETECTOR_MASK:
for contour in contours:
x, y, w, h = cv2.boundingRect(contour)
bboxes.append([x, y, x + w, y + h])
bboxes = sort_bboxes(bboxes, sort_method)
bboxes = select_bboxes(bboxes, bbox_select, select_index)
ret_previews = []
preview = draw_bounding_boxes(tensor2pil(object_mask[0]).convert("RGB"), bboxes, color="#FF0000", line_width=-1)
preview = draw_bounding_boxes(tensor2pil(object_mask[0]).convert("RGB"), bboxes, color="random", line_width=-1)
ret_previews.append(pil2tensor(preview))
if len(bboxes) == 0:
@@ -224,6 +238,7 @@ class LS_OBJECT_DETECTOR_YOLO8:
"required": {
"image": ("IMAGE", ),
"yolo_model": (FILE_LIST,),
"sort_method": (sort_method_list,),
"bbox_select": (select_list,),
"select_index": ("STRING", {"default": "0,"},),
},
@@ -236,7 +251,7 @@ class LS_OBJECT_DETECTOR_YOLO8:
FUNCTION = 'object_detector_yolo8'
CATEGORY = '😺dzNodes/LayerMask'
def object_detector_yolo8(self, image, yolo_model, bbox_select, select_index):
def object_detector_yolo8(self, image, yolo_model, sort_method, bbox_select, select_index):
from ultralytics import YOLO
model_path = os.path.join(folder_paths.models_dir, 'yolo')
@@ -256,9 +271,9 @@ class LS_OBJECT_DETECTOR_YOLO8:
for box in result.boxes:
x1, y1, x2, y2 = box.xyxy[0].cpu().numpy()
bboxes.append([x1, y1, x2, y2])
bboxes = sort_bboxes(bboxes, sort_method)
bboxes = select_bboxes(bboxes, bbox_select, select_index)
preview = draw_bounding_boxes(_image.convert("RGB"), bboxes, color="#FF0000", line_width=-1)
preview = draw_bounding_boxes(_image.convert("RGB"), bboxes, color="random", line_width=-1)
ret_previews.append(pil2tensor(preview))
if len(bboxes) == 0:
@@ -287,6 +302,7 @@ class LS_OBJECT_DETECTOR_YOLOWORLD:
"confidence_threshold": ("FLOAT", {"default": 0.05, "min": 0, "max": 1, "step": 0.01}),
"nms_iou_threshold": ("FLOAT", {"default": 0.3, "min": 0, "max": 1, "step": 0.01}),
"prompt": ("STRING", {"default": "subject"}),
"sort_method": (sort_method_list,),
"bbox_select": (select_list,),
"select_index": ("STRING", {"default": "0,"},),
},
@@ -301,7 +317,7 @@ class LS_OBJECT_DETECTOR_YOLOWORLD:
def object_detector_yoloworld(self, image, yolo_world_model,
confidence_threshold, nms_iou_threshold, prompt,
bbox_select, select_index):
sort_method, bbox_select, select_index):
import supervision as sv
model=self.load_yolo_world_model(yolo_world_model, prompt)
@@ -317,9 +333,10 @@ class LS_OBJECT_DETECTOR_YOLOWORLD:
infer_outputs.append(detections)
bboxes = infer_outputs[0].xyxy.tolist()
bboxes = [[int(value) for value in sublist] for sublist in bboxes]
bboxes = sort_bboxes(bboxes, sort_method)
bboxes = select_bboxes(bboxes, bbox_select, select_index)
ret_previews = []
preview = draw_bounding_boxes(tensor2pil(image[0]).convert('RGB'), bboxes, color="#FF0000", line_width=-1)
preview = draw_bounding_boxes(tensor2pil(image[0]).convert('RGB'), bboxes, color="random", line_width=-1)
ret_previews.append(pil2tensor(preview))
if len(bboxes) == 0:
pyproject.toml
+1 -1
View File
@@ -1,7 +1,7 @@
[project]
name = "comfyui_layerstyle"
description = "A set of nodes for ComfyUI it generate image like Adobe Photoshop's Layer Style. the Drop Shadow is first completed node, and follow-up work is in progress."
version = "1.0.41"
version = "1.0.42"
license = "MIT"
dependencies = ["numpy", "pillow", "torch", "matplotlib", "Scipy", "scikit_image", "opencv-contrib-python", "pymatting", "segment_anything", "timm", "addict", "yapf", "colour-science", "wget", "mediapipe", "loguru", "typer_config", "fastapi", "rich", "google-generativeai", "diffusers", "omegaconf", "tqdm", "transformers", "kornia", "image-reward", "ultralytics", "blend_modes", "blind-watermark", "qrcode", "pyzbar", "transparent-background", "huggingface_hub", "accelerate", "bitsandbytes", "torchscale", "wandb", "hydra-core", "psd-tools", "inference-cli[yolo-world]", "inference-gpu[yolo-world]", "onnxruntime"]
workflow/sam2_ultra_example.json
+396 -391
View File
@@ -1,278 +1,13 @@
{
"last_node_id": 25,
"last_link_id": 37,
"last_node_id": 29,
"last_link_id": 46,
"nodes": [
{
"id": 2,
"type": "LoadImage",
"pos": {
"0": 60,
"1": 180,
"2": 0,
"3": 0,
"4": 0,
"5": 0,
"6": 0,
"7": 0,
"8": 0,
"9": 0
},
"size": {
"0": 315,
"1": 314
},
"flags": {},
"order": 0,
"mode": 0,
"inputs": [],
"outputs": [
{
"name": "IMAGE",
"type": "IMAGE",
"links": [
1,
6,
10,
18,
22,
25
],
"slot_index": 0,
"shape": 3
},
{
"name": "MASK",
"type": "MASK",
"links": null,
"shape": 3
}
],
"properties": {
"Node name for S&R": "LoadImage"
},
"widgets_values": [
"seven_persons_1280x720.jpg",
"image"
]
},
{
"id": 6,
"type": "LayerMask: ObjectDetectorYOLOWorld",
"pos": {
"0": 491,
"1": 540,
"2": 0,
"3": 0,
"4": 0,
"5": 0,
"6": 0,
"7": 0,
"8": 0,
"9": 0
},
"size": {
"0": 310.79998779296875,
"1": 206.62655639648438
},
"flags": {},
"order": 3,
"mode": 0,
"inputs": [
{
"name": "image",
"type": "IMAGE",
"link": 6
}
],
"outputs": [
{
"name": "bboxes",
"type": "BBOXES",
"links": [
36
],
"slot_index": 0,
"shape": 3
},
{
"name": "preview",
"type": "IMAGE",
"links": null,
"shape": 3
}
],
"properties": {
"Node name for S&R": "LayerMask: ObjectDetectorYOLOWorld"
},
"widgets_values": [
"yolo_world/v2-x",
0.05,
0.3,
"person",
"by_index",
"0,3,4"
]
},
{
"id": 4,
"type": "LayerMask: LoadFlorence2Model",
"pos": {
"0": 502,
"1": 132,
"2": 0,
"3": 0,
"4": 0,
"5": 0,
"6": 0,
"7": 0,
"8": 0,
"9": 0
},
"size": {
"0": 301.9246520996094,
"1": 65.07791900634766
},
"flags": {},
"order": 1,
"mode": 0,
"inputs": [],
"outputs": [
{
"name": "florence2_model",
"type": "FLORENCE2",
"links": [
2
],
"shape": 3
}
],
"properties": {
"Node name for S&R": "LayerMask: LoadFlorence2Model"
},
"widgets_values": [
"base"
]
},
{
"id": 3,
"type": "LayerMask: ObjectDetectorFL2",
"pos": {
"0": 502,
"1": 247,
"2": 0,
"3": 0,
"4": 0,
"5": 0,
"6": 0,
"7": 0,
"8": 0,
"9": 0
},
"size": {
"0": 302.3999938964844,
"1": 129.92591857910156
},
"flags": {},
"order": 5,
"mode": 0,
"inputs": [
{
"name": "image",
"type": "IMAGE",
"link": 1
},
{
"name": "florence2_model",
"type": "FLORENCE2",
"link": 2
}
],
"outputs": [
{
"name": "bboxes",
"type": "BBOXES",
"links": [
19
],
"slot_index": 0,
"shape": 3
},
{
"name": "preview",
"type": "IMAGE",
"links": null,
"shape": 3
}
],
"properties": {
"Node name for S&R": "LayerMask: ObjectDetectorFL2"
},
"widgets_values": [
"women",
"all",
"0,"
]
},
{
"id": 12,
"type": "LayerMask: ObjectDetectorMask",
"pos": {
"0": 497,
"1": 815,
"2": 0,
"3": 0,
"4": 0,
"5": 0,
"6": 0,
"7": 0,
"8": 0,
"9": 0
},
"size": {
"0": 298.78375244140625,
"1": 102
},
"flags": {},
"order": 6,
"mode": 0,
"inputs": [
{
"name": "object_mask",
"type": "MASK",
"link": 16
}
],
"outputs": [
{
"name": "bboxes",
"type": "BBOXES",
"links": [
37
],
"slot_index": 0,
"shape": 3
},
{
"name": "preview",
"type": "IMAGE",
"links": null,
"shape": 3
}
],
"properties": {
"Node name for S&R": "LayerMask: ObjectDetectorMask"
},
"widgets_values": [
"all",
"0,"
]
},
{
"id": 25,
"type": "LayerMask: BBoxJoin",
"pos": {
"0": 858,
"1": 687,
"0": 1010,
"1": 860,
"2": 0,
"3": 0,
"4": 0,
@@ -287,18 +22,18 @@
"1": 86
},
"flags": {},
"order": 9,
"order": 8,
"mode": 0,
"inputs": [
{
"name": "bboxes_1",
"type": "BBOXES",
"link": 36
"link": 42
},
{
"name": "bboxes_2",
"type": "BBOXES",
"link": 37
"link": 44
},
{
"name": "bboxes_3",
@@ -318,8 +53,8 @@
"links": [
33
],
"shape": 3,
"slot_index": 0
"slot_index": 0,
"shape": 3
}
],
"properties": {
@@ -330,8 +65,8 @@
"id": 22,
"type": "LayerMask: SAM2Ultra",
"pos": {
"0": 1170,
"1": 550,
"0": 1320,
"1": 610,
"2": 0,
"3": 0,
"4": 0,
@@ -346,7 +81,7 @@
"1": 366
},
"flags": {},
"order": 12,
"order": 11,
"mode": 0,
"inputs": [
{
@@ -400,8 +135,8 @@
"id": 23,
"type": "LayerMask: SAM2Ultra",
"pos": {
"0": 1170,
"1": 980,
"0": 1320,
"1": 1040,
"2": 0,
"3": 0,
"4": 0,
@@ -416,7 +151,7 @@
"1": 366
},
"flags": {},
"order": 7,
"order": 9,
"mode": 0,
"inputs": [
{
@@ -427,7 +162,7 @@
{
"name": "bboxes",
"type": "BBOXES",
"link": 26
"link": 46
}
],
"outputs": [
@@ -470,8 +205,8 @@
"id": 5,
"type": "PreviewImage",
"pos": {
"0": 1540,
"1": 130,
"0": 1690,
"1": 190,
"2": 0,
"3": 0,
"4": 0,
@@ -486,7 +221,7 @@
"1": 368.59552001953125
},
"flags": {},
"order": 11,
"order": 10,
"mode": 0,
"inputs": [
{
@@ -504,8 +239,8 @@
"id": 8,
"type": "PreviewImage",
"pos": {
"0": 1540,
"1": 550,
"0": 1690,
"1": 610,
"2": 0,
"3": 0,
"4": 0,
@@ -538,8 +273,8 @@
"id": 11,
"type": "PreviewImage",
"pos": {
"0": 1540,
"1": 970,
"0": 1690,
"1": 1030,
"2": 0,
"3": 0,
"4": 0,
@@ -554,7 +289,7 @@
"1": 368.59552001953125
},
"flags": {},
"order": 10,
"order": 12,
"mode": 0,
"inputs": [
{
@@ -569,11 +304,11 @@
}
},
{
"id": 9,
"type": "LayerMask: ObjectDetectorYOLO8",
"id": 4,
"type": "LayerMask: LoadFlorence2Model",
"pos": {
"0": 513,
"1": 1103,
"0": 670,
"1": 210,
"2": 0,
"3": 0,
"4": 0,
@@ -584,51 +319,37 @@
"9": 0
},
"size": {
"0": 301.3816833496094,
"1": 157.36773681640625
"0": 301.9246520996094,
"1": 65.07791900634766
},
"flags": {},
"order": 4,
"order": 0,
"mode": 0,
"inputs": [
{
"name": "image",
"type": "IMAGE",
"link": 10
}
],
"inputs": [],
"outputs": [
{
"name": "bboxes",
"type": "BBOXES",
"name": "florence2_model",
"type": "FLORENCE2",
"links": [
26
38
],
"slot_index": 0,
"shape": 3
},
{
"name": "preview",
"type": "IMAGE",
"links": null,
"shape": 3
"shape": 3,
"slot_index": 0
}
],
"properties": {
"Node name for S&R": "LayerMask: ObjectDetectorYOLO8"
"Node name for S&R": "LayerMask: LoadFlorence2Model"
},
"widgets_values": [
"face_yolov8m.pt",
"by_index",
"1,2,3,5,6"
"base"
]
},
{
"id": 15,
"type": "LoadImage",
"pos": {
"0": 57,
"1": 792,
"0": 210,
"1": 960,
"2": 0,
"3": 0,
"4": 0,
@@ -639,11 +360,11 @@
"9": 0
},
"size": [
320.16066817973604,
310.2105949521583
320.1606750488281,
314
],
"flags": {},
"order": 2,
"order": 1,
"mode": 0,
"inputs": [],
"outputs": [
@@ -658,7 +379,7 @@
"name": "MASK",
"type": "MASK",
"links": [
16
43
],
"slot_index": 1,
"shape": 3
@@ -672,12 +393,120 @@
"image"
]
},
{
"id": 28,
"type": "LayerMask: ObjectDetectorMask",
"pos": {
"0": 670,
"1": 880,
"2": 0,
"3": 0,
"4": 0,
"5": 0,
"6": 0,
"7": 0,
"8": 0,
"9": 0
},
"size": [
305.4603283920758,
126
],
"flags": {},
"order": 3,
"mode": 0,
"inputs": [
{
"name": "object_mask",
"type": "MASK",
"link": 43
}
],
"outputs": [
{
"name": "bboxes",
"type": "BBOXES",
"links": [
44
],
"shape": 3,
"slot_index": 0
},
{
"name": "preview",
"type": "IMAGE",
"links": null,
"shape": 3
}
],
"properties": {
"Node name for S&R": "LayerMask: ObjectDetectorMask"
},
"widgets_values": [
"left_to_right",
"all",
"0,"
]
},
{
"id": 2,
"type": "LoadImage",
"pos": {
"0": 210,
"1": 350,
"2": 0,
"3": 0,
"4": 0,
"5": 0,
"6": 0,
"7": 0,
"8": 0,
"9": 0
},
"size": {
"0": 315,
"1": 314
},
"flags": {},
"order": 2,
"mode": 0,
"inputs": [],
"outputs": [
{
"name": "IMAGE",
"type": "IMAGE",
"links": [
18,
22,
25,
39,
41,
45
],
"slot_index": 0,
"shape": 3
},
{
"name": "MASK",
"type": "MASK",
"links": null,
"shape": 3
}
],
"properties": {
"Node name for S&R": "LoadImage"
},
"widgets_values": [
"seven_persons_1280x720.jpg",
"image"
]
},
{
"id": 21,
"type": "LayerMask: SAM2Ultra",
"pos": {
"0": 1170,
"1": 120,
"0": 1330,
"1": 180,
"2": 0,
"3": 0,
"4": 0,
@@ -692,7 +521,7 @@
"1": 366
},
"flags": {},
"order": 8,
"order": 7,
"mode": 0,
"inputs": [
{
@@ -703,7 +532,7 @@
{
"name": "bboxes",
"type": "BBOXES",
"link": 19
"link": 40
}
],
"outputs": [
@@ -741,49 +570,185 @@
"cuda",
2
]
},
{
"id": 26,
"type": "LayerMask: ObjectDetectorFL2",
"pos": {
"0": 670,
"1": 340,
"2": 0,
"3": 0,
"4": 0,
"5": 0,
"6": 0,
"7": 0,
"8": 0,
"9": 0
},
"size": [
302.3999938964844,
150
],
"flags": {},
"order": 4,
"mode": 0,
"inputs": [
{
"name": "image",
"type": "IMAGE",
"link": 39
},
{
"name": "florence2_model",
"type": "FLORENCE2",
"link": 38
}
],
"outputs": [
{
"name": "bboxes",
"type": "BBOXES",
"links": [
40
],
"shape": 3,
"slot_index": 0
},
{
"name": "preview",
"type": "IMAGE",
"links": null,
"shape": 3
}
],
"properties": {
"Node name for S&R": "LayerMask: ObjectDetectorFL2"
},
"widgets_values": [
"woman",
"left_to_right",
"all",
"0,"
]
},
{
"id": 27,
"type": "LayerMask: ObjectDetectorYOLOWorld",
"pos": {
"0": 670,
"1": 590,
"2": 0,
"3": 0,
"4": 0,
"5": 0,
"6": 0,
"7": 0,
"8": 0,
"9": 0
},
"size": [
310.79998779296875,
222
],
"flags": {},
"order": 5,
"mode": 0,
"inputs": [
{
"name": "image",
"type": "IMAGE",
"link": 41
}
],
"outputs": [
{
"name": "bboxes",
"type": "BBOXES",
"links": [
42
],
"shape": 3,
"slot_index": 0
},
{
"name": "preview",
"type": "IMAGE",
"links": null,
"shape": 3
}
],
"properties": {
"Node name for S&R": "LayerMask: ObjectDetectorYOLOWorld"
},
"widgets_values": [
"yolo_world/v2-x",
0.05,
0.3,
"person",
"left_to_right",
"by_index",
"2,4,5"
]
},
{
"id": 29,
"type": "LayerMask: ObjectDetectorYOLO8",
"pos": {
"0": 670,
"1": 1120,
"2": 0,
"3": 0,
"4": 0,
"5": 0,
"6": 0,
"7": 0,
"8": 0,
"9": 0
},
"size": [
312.30031362156797,
150
],
"flags": {},
"order": 6,
"mode": 0,
"inputs": [
{
"name": "image",
"type": "IMAGE",
"link": 45
}
],
"outputs": [
{
"name": "bboxes",
"type": "BBOXES",
"links": [
46
],
"shape": 3,
"slot_index": 0
},
{
"name": "preview",
"type": "IMAGE",
"links": null,
"shape": 3
}
],
"properties": {
"Node name for S&R": "LayerMask: ObjectDetectorYOLO8"
},
"widgets_values": [
"face_yolov8m.pt",
"left_to_right",
"by_index",
"0,1,2,5,6"
]
}
],
"links": [
[
1,
2,
0,
3,
0,
"IMAGE"
],
[
2,
4,
0,
3,
1,
"FLORENCE2"
],
[
6,
2,
0,
6,
0,
"IMAGE"
],
[
10,
2,
0,
9,
0,
"IMAGE"
],
[
16,
15,
1,
12,
0,
"MASK"
],
[
18,
2,
@@ -792,14 +757,6 @@
0,
"IMAGE"
],
[
19,
3,
0,
21,
1,
"BBOXES"
],
[
21,
21,
@@ -832,14 +789,6 @@
0,
"IMAGE"
],
[
26,
9,
0,
23,
1,
"BBOXES"
],
[
27,
23,
@@ -857,30 +806,86 @@
"BBOXES"
],
[
36,
6,
38,
4,
0,
26,
1,
"FLORENCE2"
],
[
39,
2,
0,
26,
0,
"IMAGE"
],
[
40,
26,
0,
21,
1,
"BBOXES"
],
[
41,
2,
0,
27,
0,
"IMAGE"
],
[
42,
27,
0,
25,
0,
"BBOXES"
],
[
37,
12,
43,
15,
1,
28,
0,
"MASK"
],
[
44,
28,
0,
25,
1,
"BBOXES"
],
[
45,
2,
0,
29,
0,
"IMAGE"
],
[
46,
29,
0,
23,
1,
"BBOXES"
]
],
"groups": [],
"config": {},
"extra": {
"ds": {
"scale": 0.751314800901578,
"scale": 0.683013455365071,
"offset": [
412.8676235805176,
29.36872079491155
412.29938543495007,
64.95539321714692
]
}
},
workflow/sam2_video_ultra_example.json
+320 -274
View File
@@ -1,148 +1,7 @@
{
"last_node_id": 137,
"last_link_id": 86,
"last_node_id": 139,
"last_link_id": 90,
"nodes": [
{
"id": 126,
"type": "LayerMask: SAM2VideoUltra",
"pos": {
"0": 797,
"1": 181,
"2": 0,
"3": 0,
"4": 0,
"5": 0,
"6": 0,
"7": 0,
"8": 0,
"9": 0
},
"size": {
"0": 340.20001220703125,
"1": 406
},
"flags": {},
"order": 6,
"mode": 0,
"inputs": [
{
"name": "image",
"type": "IMAGE",
"link": 86
},
{
"name": "bboxes",
"type": "BBOXES",
"link": 83
},
{
"name": "pre_mask",
"type": "MASK",
"link": null
},
{
"name": "first_frame_mask",
"type": "MASK",
"link": null
}
],
"outputs": [
{
"name": "mask",
"type": "MASK",
"links": [
78
],
"slot_index": 0,
"shape": 3
},
{
"name": "preview",
"type": "IMAGE",
"links": [
71
],
"slot_index": 1,
"shape": 3
}
],
"properties": {
"Node name for S&R": "LayerMask: SAM2VideoUltra"
},
"widgets_values": [
"sam2_hiera_base_plus.safetensors",
"fp16",
false,
false,
"#FF0080",
"VITMatte",
6,
6,
0.15,
0.99,
true,
"cuda",
0.5
]
},
{
"id": 135,
"type": "LayerUtility: BatchSelector",
"pos": {
"0": 96,
"1": 288,
"2": 0,
"3": 0,
"4": 0,
"5": 0,
"6": 0,
"7": 0,
"8": 0,
"9": 0
},
"size": {
"0": 263.46063232421875,
"1": 79.99906921386719
},
"flags": {},
"order": 3,
"mode": 0,
"inputs": [
{
"name": "images",
"type": "IMAGE",
"link": 80
},
{
"name": "masks",
"type": "MASK",
"link": null
}
],
"outputs": [
{
"name": "image",
"type": "IMAGE",
"links": [
81
],
"slot_index": 0,
"shape": 3
},
{
"name": "mask",
"type": "MASK",
"links": null,
"shape": 3
}
],
"properties": {
"Node name for S&R": "LayerUtility: BatchSelector"
},
"widgets_values": [
"0,"
]
},
{
"id": 116,
"type": "VHS_LoadVideoPath",
@@ -280,107 +139,6 @@
"horizontal": false
}
},
{
"id": 136,
"type": "LayerMask: LoadFlorence2Model",
"pos": {
"0": 93,
"1": 439,
"2": 0,
"3": 0,
"4": 0,
"5": 0,
"6": 0,
"7": 0,
"8": 0,
"9": 0
},
"size": {
"0": 269.74298095703125,
"1": 58.20575714111328
},
"flags": {},
"order": 1,
"mode": 0,
"inputs": [],
"outputs": [
{
"name": "florence2_model",
"type": "FLORENCE2",
"links": [
82
],
"shape": 3
}
],
"properties": {
"Node name for S&R": "LayerMask: LoadFlorence2Model"
},
"widgets_values": [
"base"
]
},
{
"id": 134,
"type": "LayerMask: ObjectDetectorFL2",
"pos": {
"0": 404,
"1": 337,
"2": 0,
"3": 0,
"4": 0,
"5": 0,
"6": 0,
"7": 0,
"8": 0,
"9": 0
},
"size": {
"0": 343.99267578125,
"1": 126
},
"flags": {},
"order": 5,
"mode": 0,
"inputs": [
{
"name": "image",
"type": "IMAGE",
"link": 81
},
{
"name": "florence2_model",
"type": "FLORENCE2",
"link": 82
}
],
"outputs": [
{
"name": "bboxes",
"type": "BBOXES",
"links": [
83
],
"slot_index": 0,
"shape": 3
},
{
"name": "preview",
"type": "IMAGE",
"links": [],
"slot_index": 1,
"shape": 3
}
],
"properties": {
"Node name for S&R": "LayerMask: ObjectDetectorFL2"
},
"widgets_values": [
"main subject",
"by_index",
"0, 1"
]
},
{
"id": 127,
"type": "VHS_VideoCombine",
@@ -481,7 +239,7 @@
638.2624397277832
],
"flags": {},
"order": 8,
"order": 9,
"mode": 0,
"inputs": [
{
@@ -561,7 +319,7 @@
"1": 26
},
"flags": {},
"order": 7,
"order": 8,
"mode": 0,
"inputs": [
{
@@ -605,7 +363,7 @@
634.2687377929688
],
"flags": {},
"order": 9,
"order": 10,
"mode": 0,
"inputs": [
{
@@ -664,6 +422,286 @@
"muted": false
}
}
},
{
"id": 135,
"type": "LayerUtility: BatchSelector",
"pos": {
"0": 99,
"1": 275,
"2": 0,
"3": 0,
"4": 0,
"5": 0,
"6": 0,
"7": 0,
"8": 0,
"9": 0
},
"size": {
"0": 263.46063232421875,
"1": 79.99906921386719
},
"flags": {},
"order": 3,
"mode": 0,
"inputs": [
{
"name": "images",
"type": "IMAGE",
"link": 80
},
{
"name": "masks",
"type": "MASK",
"link": null
}
],
"outputs": [
{
"name": "image",
"type": "IMAGE",
"links": [
87
],
"slot_index": 0,
"shape": 3
},
{
"name": "mask",
"type": "MASK",
"links": null,
"shape": 3
}
],
"properties": {
"Node name for S&R": "LayerUtility: BatchSelector"
},
"widgets_values": [
"0,"
]
},
{
"id": 136,
"type": "LayerMask: LoadFlorence2Model",
"pos": {
"0": 93,
"1": 439,
"2": 0,
"3": 0,
"4": 0,
"5": 0,
"6": 0,
"7": 0,
"8": 0,
"9": 0
},
"size": {
"0": 269.74298095703125,
"1": 58.20575714111328
},
"flags": {},
"order": 1,
"mode": 0,
"inputs": [],
"outputs": [
{
"name": "florence2_model",
"type": "FLORENCE2",
"links": [
88
],
"shape": 3,
"slot_index": 0
}
],
"properties": {
"Node name for S&R": "LayerMask: LoadFlorence2Model"
},
"widgets_values": [
"base"
]
},
{
"id": 139,
"type": "PreviewImage",
"pos": {
"0": 798,
"1": -111,
"2": 0,
"3": 0,
"4": 0,
"5": 0,
"6": 0,
"7": 0,
"8": 0,
"9": 0
},
"size": [
343.80730263450573,
225.6595587048399
],
"flags": {},
"order": 7,
"mode": 0,
"inputs": [
{
"name": "images",
"type": "IMAGE",
"link": 90
}
],
"outputs": [],
"properties": {
"Node name for S&R": "PreviewImage"
}
},
{
"id": 126,
"type": "LayerMask: SAM2VideoUltra",
"pos": {
"0": 798,
"1": 166,
"2": 0,
"3": 0,
"4": 0,
"5": 0,
"6": 0,
"7": 0,
"8": 0,
"9": 0
},
"size": {
"0": 340.20001220703125,
"1": 406
},
"flags": {},
"order": 6,
"mode": 0,
"inputs": [
{
"name": "image",
"type": "IMAGE",
"link": 86
},
{
"name": "bboxes",
"type": "BBOXES",
"link": 89
},
{
"name": "pre_mask",
"type": "MASK",
"link": null
},
{
"name": "first_frame_mask",
"type": "MASK",
"link": null
}
],
"outputs": [
{
"name": "mask",
"type": "MASK",
"links": [
78
],
"slot_index": 0,
"shape": 3
},
{
"name": "preview",
"type": "IMAGE",
"links": [
71
],
"slot_index": 1,
"shape": 3
}
],
"properties": {
"Node name for S&R": "LayerMask: SAM2VideoUltra"
},
"widgets_values": [
"sam2_hiera_base_plus.safetensors",
"fp16",
false,
false,
"#FF0080",
"VITMatte",
6,
6,
0.15,
0.99,
true,
"cuda",
0.5
]
},
{
"id": 138,
"type": "LayerMask: ObjectDetectorFL2",
"pos": {
"0": 421,
"1": 326,
"2": 0,
"3": 0,
"4": 0,
"5": 0,
"6": 0,
"7": 0,
"8": 0,
"9": 0
},
"size": [
318.99998647724385,
150
],
"flags": {},
"order": 5,
"mode": 0,
"inputs": [
{
"name": "image",
"type": "IMAGE",
"link": 87
},
{
"name": "florence2_model",
"type": "FLORENCE2",
"link": 88
}
],
"outputs": [
{
"name": "bboxes",
"type": "BBOXES",
"links": [
89
],
"shape": 3,
"slot_index": 0
},
{
"name": "preview",
"type": "IMAGE",
"links": [
90
],
"shape": 3,
"slot_index": 1
}
],
"properties": {
"Node name for S&R": "LayerMask: ObjectDetectorFL2"
},
"widgets_values": [
"main subject",
"top_to_bottom",
"by_index",
"1,2"
]
}
],
"links": [
@@ -707,30 +745,6 @@
0,
"IMAGE"
],
[
81,
135,
0,
134,
0,
"IMAGE"
],
[
82,
136,
0,
134,
1,
"FLORENCE2"
],
[
83,
134,
0,
126,
1,
"BBOXES"
],
[
85,
116,
@@ -746,16 +760,48 @@
126,
0,
"IMAGE"
],
[
87,
135,
0,
138,
0,
"IMAGE"
],
[
88,
136,
0,
138,
1,
"FLORENCE2"
],
[
89,
138,
0,
126,
1,
"BBOXES"
],
[
90,
138,
1,
139,
0,
"IMAGE"
]
],
"groups": [],
"config": {},
"extra": {
"ds": {
"scale": 0.7513148009015782,
"scale": 0.7513148009015777,
"offset": [
838.6779586252102,
87.72197853201384
996.1918474342962,
133.300244699258
]
}
},