V3.0: subpack features
This commit is contained in:
Dr.Lt.Data
@@ -0,0 +1,3 @@
|
||||
[submodule "subpack"]
|
||||
path = subpack
|
||||
url = https://github.com/ltdrdata/ComfyUI-Impact-Subpack
|
||||
@@ -7,6 +7,12 @@ This custom node helps to conveniently enhance images through Detector, Detailer
|
||||
|
||||
|
||||
## NOTICE
|
||||
<<<<<<< HEAD
|
||||
* Starting from V3.0, nodes related to `mmdet` are optional nodes that are activated only based on the configuration settings.
|
||||
- Through ComfyUI-Impact-Subpack, you can utilize UltralysticsDetectorProvider to access various detection models.
|
||||
=======
|
||||
* Starting from V3.0, nodes related to mmdet are optional nodes that are activated only based on the configuration settings.
|
||||
>>>>>>> fb0f901 (V3.0: subpack features)
|
||||
* Between versions 2.22 and 2.21, there is partial compatibility loss regarding the Detailer workflow. If you continue to use the existing workflow, errors may occur during execution. An additional output called "enhanced_alpha_list" has been added to Detailer-related nodes.
|
||||
* The permission error related to cv2 that occurred during the installation of Impact Pack has been patched in version 2.21.4. However, please note that the latest versions of ComfyUI and ComfyUI-Manager are required.
|
||||
* The "PreviewBridge" feature may not function correctly on ComfyUI versions released before July 1, 2023.
|
||||
@@ -16,7 +22,9 @@ This custom node helps to conveniently enhance images through Detector, Detailer
|
||||
|
||||
## Custom Nodes
|
||||
* SAMLoader - Loads the SAM model.
|
||||
* MMDetDetectorProvider - Loads the MMDet model to provide BBOX_DETECTOR and SEGM_DETECTOR.
|
||||
* UltralysticsDetectorProvider - Loads the Ultralystics model to provide SEGM_DETECTOR, BBOX_DETECTOR.
|
||||
- Unlike `MMDetDetectorProvider`, for segm models, `BBOX_DETECTOR` is also provided.
|
||||
- The various models available in UltralysticsDetectorProvider can be downloaded through **ComfyUI-Manager**.
|
||||
* ONNXDetectorProvider - Loads the ONNX model to provide SEGM_DETECTOR.
|
||||
* CLIPSegDetectorProvider - Wrapper for CLIPSeg to provide BBOX_DETECTOR.
|
||||
* You need to install the ComfyUI-CLIPSeg node extension.
|
||||
@@ -102,6 +110,14 @@ This takes latent as input and outputs latent as the result.
|
||||
* RegionalSampler, CombineRegionalPrompts, RegionalPrompt - experimental feature
|
||||
- multiple region version of TwoAdvancedSamplersForMask
|
||||
|
||||
|
||||
<<<<<<< HEAD
|
||||
## MMDet nodes
|
||||
* MMDetDetectorProvider - Loads the MMDet model to provide BBOX_DETECTOR and SEGM_DETECTOR.
|
||||
* To use the existing MMDetDetectorProvider, you need to enable the MMDet usage configuration.
|
||||
|
||||
=======
|
||||
>>>>>>> fb0f901 (V3.0: subpack features)
|
||||
## Feature
|
||||
* Interactive SAM Detector (Clipspace) - When you right-click on a node that has 'MASK' and 'IMAGE' outputs, a context menu will open. From this menu, you can either open a dialog to create a SAM Mask using 'Open in SAM Detector', or copy the content (likely mask data) using 'Copy (Clipspace)' and generate a mask using 'Impact SAM Detector' from the clipspace menu, and then paste it using 'Paste (Clipspace)'.
|
||||
|
||||
@@ -115,15 +131,38 @@ This takes latent as input and outputs latent as the result.
|
||||
* BboxDetectorCombined -> BBOX Detector (combined)
|
||||
* SegmDetectorCombined -> SEGM Detector (combined)
|
||||
* MaskPainter -> PreviewBridge
|
||||
* To use the existing deprecated legacy nodes, you need to enable the MMDet usage configuration.
|
||||
|
||||
|
||||
## How to activate 'MMDet usage'
|
||||
* Upon the initial execution, an `impact-pack.ini` file will be generated in the custom_nodes/ComfyUI-Impact-Pack directory.
|
||||
```
|
||||
[default]
|
||||
dependency_version = 2
|
||||
mmdet_skip = True
|
||||
```
|
||||
* Change `mmdet_skip = True` to `mmdet_skip = False`
|
||||
```
|
||||
[default]
|
||||
dependency_version = 2
|
||||
mmdet_skip = False
|
||||
```
|
||||
* Restart ComfyUI
|
||||
|
||||
|
||||
## Installation
|
||||
|
||||
1. cd custom_nodes
|
||||
1. git clone https://github.com/ltdrdata/ComfyUI-Impact-Pack.git
|
||||
3. cd ComfyUI-Impact-Pack
|
||||
4. (optional) python install.py
|
||||
1. `cd custom_nodes`
|
||||
1. `git clone https://github.com/ltdrdata/ComfyUI-Impact-Pack.git`
|
||||
3. `cd ComfyUI-Impact-Pack`
|
||||
4. (optional) `git submodule update --init --recursive`
|
||||
* Impact Pack will automatically download subpack during its initial launch.
|
||||
5. (optional) `python install.py`
|
||||
* Impact Pack will automatically install its dependencies during its initial launch.
|
||||
5. Restart ComfyUI
|
||||
* For the portable version, you should execute the command `..\..\..\python_embedded\python.exe install.py` to run the installation script.
|
||||
|
||||
|
||||
6. Restart ComfyUI
|
||||
|
||||
* NOTE: If an error occurs during the installation process, please refer to [Troubleshooting Page](troubleshooting/TROUBLESHOOTING.md) for assistance.
|
||||
* You can use this colab notebook [colab notebook](https://colab.research.google.com/github/ltdrdata/ComfyUI-Impact-Pack/blob/Main/notebook/comfyui_colab_impact_pack.ipynb) to launch it. This notebook automatically downloads the impact pack to the custom_nodes directory, installs the tested dependencies, and runs it.
|
||||
@@ -133,10 +172,13 @@ This takes latent as input and outputs latent as the result.
|
||||
* pip install
|
||||
* openmim
|
||||
* segment-anything
|
||||
* pycocotools
|
||||
* onnxruntime
|
||||
* ultralytics
|
||||
* scikit-image
|
||||
* piexif
|
||||
* (optional) pycocotools
|
||||
* (optional) onnxruntime
|
||||
|
||||
* mim install
|
||||
* mim install (optional)
|
||||
* mmcv==2.0.0, mmdet==3.0.0, mmengine==0.7.2
|
||||
|
||||
* linux packages (ubuntu)
|
||||
|
||||
+46
-22
@@ -2,45 +2,54 @@ import shutil
|
||||
import folder_paths
|
||||
import os
|
||||
import sys
|
||||
import importlib
|
||||
|
||||
comfy_path = os.path.dirname(folder_paths.__file__)
|
||||
impact_path = os.path.join(os.path.dirname(__file__))
|
||||
subpack_path = os.path.join(os.path.dirname(__file__), "subpack")
|
||||
modules_path = os.path.join(os.path.dirname(__file__), "modules")
|
||||
wildcards_path = os.path.join(os.path.dirname(__file__), "wildcards")
|
||||
custom_wildcards_path = os.path.join(os.path.dirname(__file__), "custom_wildcards")
|
||||
|
||||
sys.path.append(modules_path)
|
||||
sys.path.append(subpack_path)
|
||||
|
||||
|
||||
import impact.config
|
||||
print(f"### Loading: ComfyUI-Impact-Pack ({impact.config.version})")
|
||||
|
||||
|
||||
def do_install():
|
||||
import importlib
|
||||
spec = importlib.util.spec_from_file_location('impact_install', os.path.join(os.path.dirname(__file__), 'install.py'))
|
||||
impact_install = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(impact_install)
|
||||
|
||||
|
||||
# ensure dependency
|
||||
if impact.config.read_config()[1] < impact.config.dependency_version:
|
||||
if impact.config.get_config()['dependency_version'] < impact.config.dependency_version:
|
||||
print(f"## ComfyUI-Impact-Pack: Updating dependencies")
|
||||
do_install()
|
||||
|
||||
|
||||
# Core
|
||||
# recheck dependencies for colab
|
||||
try:
|
||||
import folder_paths
|
||||
import torch
|
||||
import cv2
|
||||
import mmcv
|
||||
import numpy as np
|
||||
from mmdet.apis import (inference_detector, init_detector)
|
||||
import comfy.samplers
|
||||
import comfy.sd
|
||||
import warnings
|
||||
from PIL import Image, ImageFilter
|
||||
from mmdet.evaluation import get_classes
|
||||
from skimage.measure import label, regionprops
|
||||
from collections import namedtuple
|
||||
import piexif
|
||||
|
||||
if not impact.config.get_config()['mmdet_skip']:
|
||||
import mmcv
|
||||
from mmdet.apis import (inference_detector, init_detector)
|
||||
from mmdet.evaluation import get_classes
|
||||
except:
|
||||
import importlib
|
||||
print("### ComfyUI-Impact-Pack: Reinstall dependencies (several dependencies are missing.)")
|
||||
@@ -77,7 +86,6 @@ impact.wildcards.read_wildcard_dict(custom_wildcards_path)
|
||||
|
||||
NODE_CLASS_MAPPINGS = {
|
||||
"SAMLoader": SAMLoader,
|
||||
"MMDetDetectorProvider": MMDetDetectorProvider,
|
||||
"CLIPSegDetectorProvider": CLIPSegDetectorProvider,
|
||||
"ONNXDetectorProvider": ONNXDetectorProvider,
|
||||
|
||||
@@ -171,16 +179,9 @@ NODE_CLASS_MAPPINGS = {
|
||||
"RegionalSampler": RegionalSampler,
|
||||
"CombineRegionalPrompts": CombineRegionalPrompts,
|
||||
"RegionalPrompt": RegionalPrompt,
|
||||
|
||||
"MaskPainter": impact.legacy_nodes.MaskPainter,
|
||||
"MMDetLoader": impact.legacy_nodes.MMDetLoader,
|
||||
"SegsMaskCombine": impact.legacy_nodes.SegsMaskCombine,
|
||||
"BboxDetectorForEach": impact.legacy_nodes.BboxDetectorForEach,
|
||||
"SegmDetectorForEach": impact.legacy_nodes.SegmDetectorForEach,
|
||||
"BboxDetectorCombined": impact.legacy_nodes.BboxDetectorCombined,
|
||||
"SegmDetectorCombined": impact.legacy_nodes.SegmDetectorCombined,
|
||||
}
|
||||
|
||||
|
||||
NODE_DISPLAY_NAME_MAPPINGS = {
|
||||
"BboxDetectorSEGS": "BBOX Detector (SEGS)",
|
||||
"SegmDetectorSEGS": "SEGM Detector (SEGS)",
|
||||
@@ -223,14 +224,37 @@ NODE_DISPLAY_NAME_MAPPINGS = {
|
||||
"SEGSSwitch": "Switch (SEGS)",
|
||||
|
||||
"MasksToMaskList": "Masks to Mask List",
|
||||
|
||||
"MaskPainter": "MaskPainter (Deprecated)",
|
||||
"MMDetLoader": "MMDetLoader (Legacy)",
|
||||
"SegsMaskCombine": "SegsMaskCombine (Legacy)",
|
||||
"BboxDetectorForEach": "BboxDetectorForEach (Legacy)",
|
||||
"SegmDetectorForEach": "SegmDetectorForEach (Legacy)",
|
||||
"BboxDetectorCombined": "BboxDetectorCombined (Legacy)",
|
||||
"SegmDetectorCombined": "SegmDetectorCombined (Legacy)",
|
||||
}
|
||||
|
||||
if not impact.config.get_config()['mmdet_skip']:
|
||||
NODE_CLASS_MAPPINGS.update({
|
||||
"MMDetDetectorProvider": MMDetDetectorProvider,
|
||||
"MMDetLoader": impact.legacy_nodes.MMDetLoader,
|
||||
"MaskPainter": impact.legacy_nodes.MaskPainter,
|
||||
"SegsMaskCombine": impact.legacy_nodes.SegsMaskCombine,
|
||||
"BboxDetectorForEach": impact.legacy_nodes.BboxDetectorForEach,
|
||||
"SegmDetectorForEach": impact.legacy_nodes.SegmDetectorForEach,
|
||||
"BboxDetectorCombined": impact.legacy_nodes.BboxDetectorCombined,
|
||||
"SegmDetectorCombined": impact.legacy_nodes.SegmDetectorCombined,
|
||||
})
|
||||
|
||||
NODE_DISPLAY_NAME_MAPPINGS.update({
|
||||
"MaskPainter": "MaskPainter (Deprecated)",
|
||||
"MMDetLoader": "MMDetLoader (Legacy)",
|
||||
"SegsMaskCombine": "SegsMaskCombine (Legacy)",
|
||||
"BboxDetectorForEach": "BboxDetectorForEach (Legacy)",
|
||||
"SegmDetectorForEach": "SegmDetectorForEach (Legacy)",
|
||||
"BboxDetectorCombined": "BboxDetectorCombined (Legacy)",
|
||||
"SegmDetectorCombined": "SegmDetectorCombined (Legacy)",
|
||||
})
|
||||
|
||||
try:
|
||||
import impact.subpack_nodes
|
||||
|
||||
NODE_CLASS_MAPPINGS.update(impact.subpack_nodes.NODE_CLASS_MAPPINGS)
|
||||
NODE_DISPLAY_NAME_MAPPINGS.update(impact.subpack_nodes.NODE_DISPLAY_NAME_MAPPINGS)
|
||||
|
||||
except:
|
||||
pass
|
||||
|
||||
__all__ = ['NODE_CLASS_MAPPINGS', 'NODE_DISPLAY_NAME_MAPPINGS']
|
||||
|
||||
+70
-26
@@ -8,6 +8,9 @@ if sys.argv[0] == 'install.py':
|
||||
sys.path.append('.') # for portable version
|
||||
|
||||
impact_path = os.path.join(os.path.dirname(__file__), "modules")
|
||||
subpack_path = os.path.join(os.path.dirname(__file__), "subpack")
|
||||
subpack_repo = ""
|
||||
|
||||
|
||||
sys.path.append(impact_path)
|
||||
sys.path.append(comfy_path)
|
||||
@@ -28,12 +31,26 @@ else:
|
||||
mim_install = [sys.executable, '-m', 'mim', 'install']
|
||||
|
||||
|
||||
def ensure_subpack():
|
||||
import git
|
||||
repo = git.Repo(os.path.dirname(__file__))
|
||||
origin = repo.remote(name='origin')
|
||||
origin.pull()
|
||||
repo.git.submodule('update', '--init', '--recursive')
|
||||
|
||||
|
||||
def remove_olds():
|
||||
global comfy_path
|
||||
|
||||
comfy_path = os.path.dirname(folder_paths.__file__)
|
||||
custom_nodes_path = os.path.join(comfy_path, "custom_nodes")
|
||||
old_ini_path = os.path.join(custom_nodes_path, "impact-pack.ini")
|
||||
old_py_path = os.path.join(custom_nodes_path, "comfyui-impact-pack.py")
|
||||
|
||||
if os.path.exists(impact.config.old_config_path):
|
||||
impact.config.get_config()['mmdet_skip'] = False
|
||||
os.remove(impact.config.old_config_path)
|
||||
|
||||
if os.path.exists(old_ini_path):
|
||||
print(f"Delete legacy file: {old_ini_path}")
|
||||
os.remove(old_ini_path)
|
||||
@@ -44,24 +61,29 @@ def remove_olds():
|
||||
|
||||
|
||||
def ensure_pip_packages_first():
|
||||
try:
|
||||
import pycocotools
|
||||
except Exception:
|
||||
if platform.system() not in ["Windows"] or platform.machine() not in ["AMD64", "x86_64"]:
|
||||
print(f"Your system is {platform.system()}; !! You need to install 'libpython3-dev' for this step. !!")
|
||||
subpack_req = os.path.join(subpack_path, "requirements.txt")
|
||||
if os.path.exists(subpack_req):
|
||||
subprocess.run(pip_install + ['-r', 'requirements.txt'], cwd=subpack_path)
|
||||
|
||||
subprocess.check_call(pip_install + ['pycocotools'])
|
||||
else:
|
||||
pycocotools = {
|
||||
(3, 8): "https://github.com/Bing-su/dddetailer/releases/download/pycocotools/pycocotools-2.0.6-cp38-cp38-win_amd64.whl",
|
||||
(3, 9): "https://github.com/Bing-su/dddetailer/releases/download/pycocotools/pycocotools-2.0.6-cp39-cp39-win_amd64.whl",
|
||||
(3, 10): "https://github.com/Bing-su/dddetailer/releases/download/pycocotools/pycocotools-2.0.6-cp310-cp310-win_amd64.whl",
|
||||
(3, 11): "https://github.com/Bing-su/dddetailer/releases/download/pycocotools/pycocotools-2.0.6-cp311-cp311-win_amd64.whl",
|
||||
}
|
||||
if not impact.config.get_config()['mmdet_skip']:
|
||||
try:
|
||||
import pycocotools
|
||||
except Exception:
|
||||
if platform.system() not in ["Windows"] or platform.machine() not in ["AMD64", "x86_64"]:
|
||||
print(f"Your system is {platform.system()}; !! You need to install 'libpython3-dev' for this step. !!")
|
||||
|
||||
version = sys.version_info[:2]
|
||||
url = pycocotools[version]
|
||||
subprocess.check_call(pip_install + [url])
|
||||
subprocess.check_call(pip_install + ['pycocotools'])
|
||||
else:
|
||||
pycocotools = {
|
||||
(3, 8): "https://github.com/Bing-su/dddetailer/releases/download/pycocotools/pycocotools-2.0.6-cp38-cp38-win_amd64.whl",
|
||||
(3, 9): "https://github.com/Bing-su/dddetailer/releases/download/pycocotools/pycocotools-2.0.6-cp39-cp39-win_amd64.whl",
|
||||
(3, 10): "https://github.com/Bing-su/dddetailer/releases/download/pycocotools/pycocotools-2.0.6-cp310-cp310-win_amd64.whl",
|
||||
(3, 11): "https://github.com/Bing-su/dddetailer/releases/download/pycocotools/pycocotools-2.0.6-cp311-cp311-win_amd64.whl",
|
||||
}
|
||||
|
||||
version = sys.version_info[:2]
|
||||
url = pycocotools[version]
|
||||
subprocess.check_call(pip_install + [url])
|
||||
|
||||
|
||||
def ensure_pip_packages_last():
|
||||
@@ -83,6 +105,11 @@ def ensure_pip_packages_last():
|
||||
except:
|
||||
print(f"ComfyUI-Impact-Pack: failed to install 'opencv-python'. Please, install manually.")
|
||||
|
||||
try:
|
||||
import git
|
||||
except Exception:
|
||||
subprocess.check_call(pip_install + ['gitpython'])
|
||||
|
||||
|
||||
def ensure_mmdet_package():
|
||||
try:
|
||||
@@ -99,7 +126,10 @@ def ensure_mmdet_package():
|
||||
def install():
|
||||
remove_olds()
|
||||
ensure_pip_packages_first()
|
||||
ensure_mmdet_package()
|
||||
|
||||
if not impact.config.get_config()['mmdet_skip']:
|
||||
ensure_mmdet_package()
|
||||
|
||||
ensure_pip_packages_last()
|
||||
|
||||
# Download model
|
||||
@@ -108,24 +138,38 @@ def install():
|
||||
model_path = folder_paths.models_dir
|
||||
|
||||
bbox_path = os.path.join(model_path, "mmdets", "bbox")
|
||||
#segm_path = os.path.join(model_path, "mmdets", "segm") -- deprecated
|
||||
sam_path = os.path.join(model_path, "sams")
|
||||
onnx_path = os.path.join(model_path, "onnx")
|
||||
|
||||
if not os.path.exists(os.path.join(bbox_path, "mmdet_anime-face_yolov3.pth")):
|
||||
download_url("https://huggingface.co/dustysys/ddetailer/resolve/main/mmdet/bbox/mmdet_anime-face_yolov3.pth", bbox_path)
|
||||
if not os.path.exists(bbox_path):
|
||||
os.makedirs(bbox_path)
|
||||
|
||||
if not os.path.exists(os.path.join(bbox_path, "mmdet_anime-face_yolov3.py")):
|
||||
download_url("https://raw.githubusercontent.com/Bing-su/dddetailer/master/config/mmdet_anime-face_yolov3.py", bbox_path)
|
||||
if not impact.config.get_config()['mmdet_skip']:
|
||||
if not os.path.exists(os.path.join(bbox_path, "mmdet_anime-face_yolov3.pth")):
|
||||
download_url("https://huggingface.co/dustysys/ddetailer/resolve/main/mmdet/bbox/mmdet_anime-face_yolov3.pth", bbox_path)
|
||||
|
||||
if not os.path.exists(os.path.join(sam_path, "sam_vit_b_01ec64.pth")):
|
||||
download_url("https://dl.fbaipublicfiles.com/segment_anything/sam_vit_b_01ec64.pth", sam_path)
|
||||
if not os.path.exists(os.path.join(bbox_path, "mmdet_anime-face_yolov3.py")):
|
||||
download_url("https://raw.githubusercontent.com/Bing-su/dddetailer/master/config/mmdet_anime-face_yolov3.py", bbox_path)
|
||||
|
||||
if not os.path.exists(os.path.join(sam_path, "sam_vit_b_01ec64.pth")):
|
||||
download_url("https://dl.fbaipublicfiles.com/segment_anything/sam_vit_b_01ec64.pth", sam_path)
|
||||
|
||||
subpack_install_script = os.path.join(subpack_path, "install.py")
|
||||
|
||||
if not os.path.exists(subpack_install_script):
|
||||
print(f"### ComfyUI-Impact-Pack: Downloading subpack")
|
||||
ensure_subpack()
|
||||
|
||||
if os.path.exists(subpack_install_script):
|
||||
subprocess.run([sys.executable, 'install.py'], cwd=subpack_path)
|
||||
subprocess.run(pip_install + ['-r', 'requirements.txt'], cwd=subpack_path)
|
||||
|
||||
if not os.path.exists(onnx_path):
|
||||
print(f"### ComfyUI-Impact-Pack: onnx model directory created ({onnx_path})")
|
||||
os.mkdir(onnx_path)
|
||||
|
||||
impact.config.write_config(comfy_path)
|
||||
impact.config.write_config()
|
||||
|
||||
|
||||
install()
|
||||
install()
|
||||
|
||||
|
||||
@@ -1,21 +1,24 @@
|
||||
import configparser
|
||||
import os
|
||||
|
||||
version = "V2.23.1"
|
||||
|
||||
dependency_version = 1
|
||||
version = "V3.0"
|
||||
|
||||
dependency_version = 2
|
||||
|
||||
my_path = os.path.dirname(__file__)
|
||||
config_path = os.path.join(my_path, "impact-pack.ini")
|
||||
old_config_path = os.path.join(my_path, "impact-pack.ini")
|
||||
config_path = os.path.join(my_path, "..", "..", "impact-pack.ini")
|
||||
latent_letter_path = os.path.join(my_path, "..", "..", "latent.png")
|
||||
|
||||
MAX_RESOLUTION = 8192
|
||||
|
||||
def write_config(comfy_path):
|
||||
|
||||
def write_config():
|
||||
config = configparser.ConfigParser()
|
||||
config['default'] = {
|
||||
'dependency_version': dependency_version,
|
||||
'comfy_path': comfy_path
|
||||
'mmdet_skip': get_config()['mmdet_skip'],
|
||||
}
|
||||
with open(config_path, 'w') as configfile:
|
||||
config.write(configfile)
|
||||
@@ -27,6 +30,22 @@ def read_config():
|
||||
config.read(config_path)
|
||||
default_conf = config['default']
|
||||
|
||||
return default_conf['comfy_path'], int(default_conf['dependency_version'])
|
||||
return {
|
||||
'dependency_version': int(default_conf['dependency_version']),
|
||||
'mmdet_skip': default_conf['mmdet_skip'].lower() == 'true'
|
||||
}
|
||||
|
||||
except Exception:
|
||||
return "", 0
|
||||
return {'dependency_version': 0, 'mmdet_skip': True}
|
||||
|
||||
|
||||
cached_config = None
|
||||
|
||||
|
||||
def get_config():
|
||||
global cached_config
|
||||
|
||||
if cached_config is None:
|
||||
cached_config = read_config()
|
||||
|
||||
return cached_config
|
||||
|
||||
+2
-191
@@ -1,7 +1,4 @@
|
||||
import os
|
||||
import mmcv
|
||||
from mmdet.apis import (inference_detector, init_detector)
|
||||
from mmdet.evaluation import get_classes
|
||||
from segment_anything import SamPredictor
|
||||
import torch.nn.functional as F
|
||||
|
||||
@@ -62,12 +59,6 @@ class NO_SEGM_DETECTOR:
|
||||
pass
|
||||
|
||||
|
||||
def load_mmdet(model_path):
|
||||
model_config = os.path.splitext(model_path)[0] + ".py"
|
||||
model = init_detector(model_config, model_path, device="cpu")
|
||||
return model
|
||||
|
||||
|
||||
def create_segmasks(results):
|
||||
bboxs = results[1]
|
||||
segms = results[2]
|
||||
@@ -80,90 +71,6 @@ def create_segmasks(results):
|
||||
return results
|
||||
|
||||
|
||||
def inference_segm_old(model, image, conf_threshold):
|
||||
image = image.numpy()[0] * 255
|
||||
mmdet_results = inference_detector(model, image)
|
||||
|
||||
bbox_results, segm_results = mmdet_results
|
||||
label = "A"
|
||||
|
||||
classes = get_classes("coco")
|
||||
labels = [
|
||||
np.full(bbox.shape[0], i, dtype=np.int32)
|
||||
for i, bbox in enumerate(bbox_results)
|
||||
]
|
||||
n, m = bbox_results[0].shape
|
||||
if n == 0:
|
||||
return [[], [], []]
|
||||
labels = np.concatenate(labels)
|
||||
bboxes = np.vstack(bbox_results)
|
||||
segms = mmcv.concat_list(segm_results)
|
||||
filter_idxs = np.where(bboxes[:, -1] > conf_threshold)[0]
|
||||
results = [[], [], []]
|
||||
for i in filter_idxs:
|
||||
results[0].append(label + "-" + classes[labels[i]])
|
||||
results[1].append(bboxes[i])
|
||||
results[2].append(segms[i])
|
||||
|
||||
return results
|
||||
|
||||
|
||||
def inference_segm(image, modelname, conf_thres, lab="A"):
|
||||
image = image.numpy()[0] * 255
|
||||
mmdet_results = inference_detector(modelname, image).pred_instances
|
||||
bboxes = mmdet_results.bboxes.numpy()
|
||||
segms = mmdet_results.masks.numpy()
|
||||
scores = mmdet_results.scores.numpy()
|
||||
|
||||
classes = get_classes("coco")
|
||||
|
||||
n, m = bboxes.shape
|
||||
if n == 0:
|
||||
return [[], [], [], []]
|
||||
labels = mmdet_results.labels
|
||||
filter_inds = np.where(mmdet_results.scores > conf_thres)[0]
|
||||
results = [[], [], [], []]
|
||||
for i in filter_inds:
|
||||
results[0].append(lab + "-" + classes[labels[i]])
|
||||
results[1].append(bboxes[i])
|
||||
results[2].append(segms[i])
|
||||
results[3].append(scores[i])
|
||||
|
||||
return results
|
||||
|
||||
|
||||
def inference_bbox(modelname, image, conf_threshold):
|
||||
image = image.numpy()[0] * 255
|
||||
label = "A"
|
||||
output = inference_detector(modelname, image).pred_instances
|
||||
cv2_image = np.array(image)
|
||||
cv2_image = cv2_image[:, :, ::-1].copy()
|
||||
cv2_gray = cv2.cvtColor(cv2_image, cv2.COLOR_BGR2GRAY)
|
||||
|
||||
segms = []
|
||||
for x0, y0, x1, y1 in output.bboxes:
|
||||
cv2_mask = np.zeros(cv2_gray.shape, np.uint8)
|
||||
cv2.rectangle(cv2_mask, (int(x0), int(y0)), (int(x1), int(y1)), 255, -1)
|
||||
cv2_mask_bool = cv2_mask.astype(bool)
|
||||
segms.append(cv2_mask_bool)
|
||||
|
||||
n, m = output.bboxes.shape
|
||||
if n == 0:
|
||||
return [[], [], [], []]
|
||||
|
||||
bboxes = output.bboxes.numpy()
|
||||
scores = output.scores.numpy()
|
||||
filter_idxs = np.where(scores > conf_threshold)[0]
|
||||
results = [[], [], [], []]
|
||||
for i in filter_idxs:
|
||||
results[0].append(label)
|
||||
results[1].append(bboxes[i])
|
||||
results[2].append(segms[i])
|
||||
results[3].append(scores[i])
|
||||
|
||||
return results
|
||||
|
||||
|
||||
def gen_detection_hints_from_mask_area(x, y, mask, threshold, use_negative):
|
||||
points = []
|
||||
plabs = []
|
||||
@@ -669,57 +576,7 @@ def apply_mask_to_each_seg(segs, masks):
|
||||
return segs[0], items
|
||||
|
||||
|
||||
class BBoxDetector:
|
||||
bbox_model = None
|
||||
|
||||
def __init__(self, bbox_model):
|
||||
self.bbox_model = bbox_model
|
||||
|
||||
def detect(self, image, threshold, dilation, crop_factor, drop_size=1):
|
||||
drop_size = max(drop_size, 1)
|
||||
mmdet_results = inference_bbox(self.bbox_model, image, threshold)
|
||||
segmasks = create_segmasks(mmdet_results)
|
||||
|
||||
if dilation > 0:
|
||||
segmasks = dilate_masks(segmasks, dilation)
|
||||
|
||||
items = []
|
||||
h = image.shape[1]
|
||||
w = image.shape[2]
|
||||
|
||||
for x in segmasks:
|
||||
item_bbox = x[0]
|
||||
item_mask = x[1]
|
||||
|
||||
y1, x1, y2, x2 = item_bbox
|
||||
|
||||
if x2 - x1 > drop_size and y2 - y1 > drop_size: # minimum dimension must be (2,2) to avoid squeeze issue
|
||||
crop_region = make_crop_region(w, h, item_bbox, crop_factor)
|
||||
cropped_image = crop_image(image, crop_region)
|
||||
cropped_mask = crop_ndarray2(item_mask, crop_region)
|
||||
confidence = x[2]
|
||||
# bbox_size = (item_bbox[2]-item_bbox[0],item_bbox[3]-item_bbox[1]) # (w,h)
|
||||
|
||||
item = SEG(cropped_image, cropped_mask, confidence, crop_region, item_bbox)
|
||||
|
||||
items.append(item)
|
||||
|
||||
shape = image.shape[1], image.shape[2]
|
||||
return shape, items
|
||||
|
||||
def detect_combined(self, image, threshold, dilation):
|
||||
mmdet_results = inference_bbox(self.bbox_model, image, threshold)
|
||||
segmasks = create_segmasks(mmdet_results)
|
||||
if dilation > 0:
|
||||
segmasks = dilate_masks(segmasks, dilation)
|
||||
|
||||
return combine_masks(segmasks)
|
||||
|
||||
def setAux(self, x):
|
||||
pass
|
||||
|
||||
|
||||
class ONNXDetector(BBoxDetector):
|
||||
class ONNXDetector:
|
||||
onnx_model = None
|
||||
|
||||
def __init__(self, onnx_model):
|
||||
@@ -769,52 +626,6 @@ class ONNXDetector(BBoxDetector):
|
||||
pass
|
||||
|
||||
|
||||
class SegmDetector(BBoxDetector):
|
||||
segm_model = None
|
||||
|
||||
def __init__(self, segm_model):
|
||||
self.segm_model = segm_model
|
||||
|
||||
def detect(self, image, threshold, dilation, crop_factor, drop_size=1):
|
||||
drop_size = max(drop_size, 1)
|
||||
mmdet_results = inference_segm(image, self.segm_model, threshold)
|
||||
segmasks = create_segmasks(mmdet_results)
|
||||
|
||||
if dilation > 0:
|
||||
segmasks = dilate_masks(segmasks, dilation)
|
||||
|
||||
items = []
|
||||
h = image.shape[1]
|
||||
w = image.shape[2]
|
||||
for x in segmasks:
|
||||
item_bbox = x[0]
|
||||
item_mask = x[1]
|
||||
|
||||
y1, x1, y2, x2 = item_bbox
|
||||
|
||||
if x2 - x1 > drop_size and y2 - y1 > drop_size: # minimum dimension must be (2,2) to avoid squeeze issue
|
||||
crop_region = make_crop_region(w, h, item_bbox, crop_factor)
|
||||
cropped_image = crop_image(image, crop_region)
|
||||
cropped_mask = crop_ndarray2(item_mask, crop_region)
|
||||
confidence = x[2]
|
||||
|
||||
item = SEG(cropped_image, cropped_mask, confidence, crop_region, item_bbox)
|
||||
items.append(item)
|
||||
|
||||
return image.shape, items
|
||||
|
||||
def detect_combined(self, image, threshold, dilation):
|
||||
mmdet_results = inference_bbox(self.bbox_model, image, threshold)
|
||||
segmasks = create_segmasks(mmdet_results)
|
||||
if dilation > 0:
|
||||
segmasks = dilate_masks(segmasks, dilation)
|
||||
|
||||
return combine_masks(segmasks)
|
||||
|
||||
def setAux(self, x):
|
||||
pass
|
||||
|
||||
|
||||
def mask_to_segs(mask, combined, crop_factor, bbox_fill, drop_size=1):
|
||||
drop_size = max(drop_size, 1)
|
||||
if mask is None:
|
||||
@@ -1408,7 +1219,7 @@ except:
|
||||
|
||||
# REQUIREMENTS: biegert/ComfyUI-CLIPSeg
|
||||
try:
|
||||
class BBoxDetectorBasedOnCLIPSeg(BBoxDetector):
|
||||
class BBoxDetectorBasedOnCLIPSeg:
|
||||
prompt = None
|
||||
blur = None
|
||||
threshold = None
|
||||
|
||||
@@ -335,17 +335,17 @@ class SEGSToImageList:
|
||||
|
||||
for seg in segs[1]:
|
||||
if seg.cropped_image is not None:
|
||||
cropped_image = pil2tensor(seg.cropped_image)
|
||||
cropped_image = torch.from_numpy(seg.cropped_image)
|
||||
elif fallback_image_opt is not None:
|
||||
# take from original image
|
||||
cropped_image = torch.from_numpy(crop_image(fallback_image_opt, seg.crop_region))
|
||||
else:
|
||||
cropped_image = empty_pil()
|
||||
cropped_image = empty_pil_tensor()
|
||||
|
||||
results.append(cropped_image)
|
||||
|
||||
if len(results) == 0:
|
||||
results.append(empty_pil())
|
||||
results.append(empty_pil_tensor())
|
||||
|
||||
return (results,)
|
||||
|
||||
@@ -810,10 +810,10 @@ class FaceDetailer:
|
||||
mask = core.segs_to_combined_mask(segs)
|
||||
|
||||
if len(cropped_enhanced) == 0:
|
||||
cropped_enhanced = [empty_pil()]
|
||||
cropped_enhanced = [empty_pil_tensor()]
|
||||
|
||||
if len(cropped_enhanced_alpha) == 0:
|
||||
cropped_enhanced_alpha = [empty_pil()]
|
||||
cropped_enhanced_alpha = [empty_pil_tensor()]
|
||||
|
||||
return enhanced_img, cropped_enhanced, cropped_enhanced_alpha, mask,
|
||||
|
||||
@@ -1351,10 +1351,10 @@ class FaceDetailerPipe:
|
||||
sam_mask_hint_use_negative, drop_size, bbox_detector, wildcard, sam_model_opt)
|
||||
|
||||
if len(cropped_enhanced) == 0:
|
||||
cropped_enhanced = [empty_pil()]
|
||||
cropped_enhanced = [empty_pil_tensor()]
|
||||
|
||||
if len(cropped_enhanced_alpha) == 0:
|
||||
cropped_enhanced_alpha = [empty_pil()]
|
||||
cropped_enhanced_alpha = [empty_pil_tensor()]
|
||||
|
||||
return enhanced_img, cropped_enhanced, cropped_enhanced_alpha, mask, detailer_pipe
|
||||
|
||||
@@ -1378,13 +1378,13 @@ class DetailerForEachTest(DetailerForEach):
|
||||
|
||||
# set fallback image
|
||||
if len(cropped) == 0:
|
||||
cropped = [empty_pil()]
|
||||
cropped = [empty_pil_tensor()]
|
||||
|
||||
if len(cropped_enhanced) == 0:
|
||||
cropped_enhanced = [empty_pil()]
|
||||
cropped_enhanced = [empty_pil_tensor()]
|
||||
|
||||
if len(cropped_enhanced_alpha) == 0:
|
||||
cropped_enhanced_alpha = [empty_pil()]
|
||||
cropped_enhanced_alpha = [empty_pil_tensor()]
|
||||
|
||||
return enhanced_img, cropped, cropped_enhanced, cropped_enhanced_alpha
|
||||
|
||||
@@ -1409,13 +1409,13 @@ class DetailerForEachTestPipe(DetailerForEachPipe):
|
||||
|
||||
# set fallback image
|
||||
if len(cropped) == 0:
|
||||
cropped = [empty_pil()]
|
||||
cropped = [empty_pil_tensor()]
|
||||
|
||||
if len(cropped_enhanced) == 0:
|
||||
cropped_enhanced = [empty_pil()]
|
||||
cropped_enhanced = [empty_pil_tensor()]
|
||||
|
||||
if len(cropped_enhanced_alpha) == 0:
|
||||
cropped_enhanced_alpha = [empty_pil()]
|
||||
cropped_enhanced_alpha = [empty_pil_tensor()]
|
||||
|
||||
return enhanced_img, cropped, cropped_enhanced, cropped_enhanced_alpha
|
||||
|
||||
|
||||
@@ -210,7 +210,7 @@ def scale_tensor_and_to_pil(w, h, image):
|
||||
return image.resize((w, h), resample=LANCZOS)
|
||||
|
||||
|
||||
def empty_pil(w=64, h=64):
|
||||
def empty_pil_tensor(w=64, h=64):
|
||||
image = Image.new("RGB", (w, h))
|
||||
draw = ImageDraw.Draw(image)
|
||||
draw.rectangle((0, 0, w-1, h-1), fill=(0, 0, 0))
|
||||
|
||||
Submodule
+1
Submodule subpack added at da645b1cf3
Reference in New Issue
Block a user