Compare commits

..
Author SHA1 Message Date
Dr.Lt.Data c2824aa943 wip 2024-02-09 13:22:07 +09:00
Dr.Lt.Data 1b86633194 wip 2024-02-08 13:10:25 +09:00
15 changed files with 183 additions and 406 deletions
+6 -11
View File
@@ -6,8 +6,7 @@
This custom node helps to conveniently enhance images through Detector, Detailer, Upscaler, Pipe, and more.
## NOTICE
* V4.77: Compatibility patch applied. Requires ComfyUI version (Oct. 8th) or later.
## NOTICE
* V4.73.3: ControlNetApply (SEGS) supports AnimateDiff
* V4.20.1: Due to the feature update in `RegionalSampler`, the parameter order has changed, causing malfunctions in previously created `RegionalSamplers`. Please adjust the parameters accordingly.
* V4.12: `MASKS` is changed to `MASK`.
@@ -41,13 +40,12 @@ This custom node helps to conveniently enhance images through Detector, Detailer
* While `batch_masks` may not be completely separated, it provides functionality to perform some level of segmentation.
* Simple Detector (SEGS) - Operating primarily with `BBOX_DETECTOR`, and with the additional provision of `SAM_MODEL` or `SEGM_DETECTOR`, this node internally generates improved SEGS through mask operations on both *bbox* and *silhouette*. It serves as a convenient tool to simplify a somewhat intricate workflow.
* ControlNet, IPAdapter
* ControlNet
* ControlNetApply (SEGS) - To apply ControlNet in SEGS, you need to use the Preprocessor Provider node from the Inspire Pack to utilize this node.
* `segs_preprocessor` and `control_image` can be selectively applied. If an `control_image` is given, `segs_preprocessor` will be ignored.
* If set to `control_image`, you can preview the cropped cnet image through `SEGSPreview (CNET Image)`. Images generated by `segs_preprocessor` should be verified through the `cnet_images` output of each Detailer.
* The `segs_preprocessor` operates by applying preprocessing on-the-fly based on the cropped image during the detailing process, while `control_image` will be cropped and used as input to `ControlNetApply (SEGS)`.
* ControlNetClear (SEGS) - Clear applied ControlNet in SEGS
* IPAdapterApply (SEGS) - To apply IPAdapter in SEGS, you need to use the Preprocessor Provider node from the Inspire Pack to utilize this node.
* ControlNetClear (SEGS) - Clear applied ControlNet in SEGS
* Bitwise(SEGS & SEGS) - Performs a 'bitwise and' operation between two SEGS.
* Bitwise(SEGS - SEGS) - Subtracts one SEGS from another.
@@ -93,7 +91,6 @@ This custom node helps to conveniently enhance images through Detector, Detailer
* SEGS Filter (label) - This node filters SEGS based on the label of the detected areas.
* SEGS Filter (ordered) - This node sorts SEGS based on size and position and retrieves SEGs within a certain range.
* SEGS Filter (range) - This node retrieves only SEGs from SEGS that have a size and position within a certain range.
* SEGS Assign (label) - Assign labels sequentially to SEGS. This node is useful when used with `[LAB]` of FaceDetailer.
* SEGSConcat - Concatenate segs1 and segs2. If source shape of segs1 and segs2 are different from segs2 will be ignored.
* Picker (SEGS) - Among the input SEGS, you can select a specific SEG through a dialog. If no SEG is selected, it outputs an empty SEGS. Increasing the batch_size of SEGSDetailer can be used for the purpose of selecting from the candidates.
* Set Default Image For SEGS - Set a default image for SEGS. SEGS with images set this way do not need to have a fallback image set. When override is set to false, the original image is preserved.
@@ -127,9 +124,8 @@ This custom node helps to conveniently enhance images through Detector, Detailer
* You need to install the [BlenderNeko/ComfyUI_TiledKSampler](https://github.com/BlenderNeko/ComfyUI_TiledKSampler) node extension.
* PK_HOOK
* DenoiseScheduleHookProvider - IterativeUpscale provides a hook that gradually changes the denoise to target_denoise as the iterative-step progresses.
* CfgScheduleHookProvider - IterativeUpscale provides a hook that gradually changes the cfg to target_cfg as the iterative-step progresses.
* StepsScheduleHookProvider - IterativeUpscale provides a hook that gradually changes the sampling-steps to target_steps as the iterative-step progresses.
* DenoiseScheduleHookProvider - IterativeUpscale provides a hook that gradually changes the denoise to target_denoise as the step progresses.
* CfgScheduleHookProvider - IterativeUpscale provides a hook that gradually changes the cfg to target_cfg as the step progresses.
* NoiseInjectionHookProvider - During each iteration of IterativeUpscale, noise is injected into the latent space while varying the strength according to a schedule.
* You need to install the [BlenderNeko/ComfyUI_Noise](https://github.com/BlenderNeko/ComfyUI_Noise) node extension.
* The seed serves as the initial value required for generating noise, and it increments by 1 with each iteration as the process unfolds.
@@ -161,8 +157,7 @@ This takes latent as input and outputs latent as the result.
* You need to install the [BlenderNeko/ComfyUI_TiledKSampler](https://github.com/BlenderNeko/ComfyUI_TiledKSampler) node extension.
* TwoAdvancedSamplersForMask - TwoSamplersForMask is similar to TwoAdvancedSamplersForMask, but they differ in their operation. TwoSamplersForMask performs sampling in the mask area only after all the samples in the base area are finished. On the other hand, TwoAdvancedSamplersForMask performs sampling in both the base area and the mask area sequentially at each step.
* KSamplerAdvancedProvider - This is a wrapper that enables KSampler to be used in TwoAdvancedSamplersForMask, RegionalSampler.
* sigma_factor: By multiplying the denoise schedule by the sigma_factor, you can adjust the amount of denoising based on the configured denoise.
* KSamplerAdvancedProvider - This is a wrapper that enables KSampler to be used in TwoAdvancedSamplersForMask.
* TwoSamplersForMaskUpscalerProvider - This is an Upscaler that extends TwoSamplersForMask to be used in Iterative Upscale.
* TwoSamplersForMaskUpscalerProviderPipe - pipe version of TwoSamplersForMaskUpscalerProvider.
+1 -6
View File
@@ -176,13 +176,11 @@ NODE_CLASS_MAPPINGS = {
"PixelKSampleHookCombine": PixelKSampleHookCombine,
"DenoiseScheduleHookProvider": DenoiseScheduleHookProvider,
"StepsScheduleHookProvider": StepsScheduleHookProvider,
"CfgScheduleHookProvider": CfgScheduleHookProvider,
"NoiseInjectionHookProvider": NoiseInjectionHookProvider,
"UnsamplerHookProvider": UnsamplerHookProvider,
"CoreMLDetailerHookProvider": CoreMLDetailerHookProvider,
"PreviewDetailerHookProvider": PreviewDetailerHookProvider,
"StableCascade_DetailerHookProvider": StableCascade_DetailerHookProvider,
"DetailerHookCombine": DetailerHookCombine,
"NoiseInjectionDetailerHookProvider": NoiseInjectionDetailerHookProvider,
@@ -218,7 +216,6 @@ NODE_CLASS_MAPPINGS = {
"ImpactControlNetApplySEGS": ControlNetApplySEGS,
"ImpactControlNetApplyAdvancedSEGS": ControlNetApplyAdvancedSEGS,
"ImpactControlNetClearSEGS": ControlNetClearSEGS,
"ImpactIPAdapterApplySEGS": IPAdapterApplySEGS,
"ImpactDecomposeSEGS": DecomposeSEGS,
"ImpactAssembleSEGS": AssembleSEGS,
@@ -230,6 +227,7 @@ NODE_CLASS_MAPPINGS = {
"ImpactDilateMaskInSEGS": DilateMaskInSEGS,
"ImpactGaussianBlurMaskInSEGS": GaussianBlurMaskInSEGS,
"ImpactScaleBy_BBOX_SEG_ELT": SEG_ELT_BBOX_ScaleBy,
"ImpactSEGSLatentComposite": SEGSLatentComposite,
"BboxDetectorCombined_v2": BboxDetectorCombined,
"SegmDetectorCombined_v2": SegmDetectorCombined,
@@ -288,7 +286,6 @@ NODE_CLASS_MAPPINGS = {
"ImpactCombineConditionings": CombineConditionings,
"ImpactConcatConditionings": ConcatConditionings,
"ImpactSEGSLabelAssign": SEGSLabelAssign,
"ImpactSEGSLabelFilter": SEGSLabelFilter,
"ImpactSEGSRangeFilter": SEGSRangeFilter,
"ImpactSEGSOrderedFilter": SEGSOrderedFilter,
@@ -341,7 +338,6 @@ NODE_DISPLAY_NAME_MAPPINGS = {
"ImpactSimpleDetectorSEGSPipe": "Simple Detector (SEGS/pipe)",
"ImpactControlNetApplySEGS": "ControlNetApply (SEGS)",
"ImpactControlNetApplyAdvancedSEGS": "ControlNetApplyAdvanced (SEGS)",
"ImpactIPAdapterApplySEGS": "IPAdapterApply (SEGS)",
"BboxDetectorCombined_v2": "BBOX Detector (combined)",
"SegmDetectorCombined_v2": "SEGM Detector (combined)",
@@ -389,7 +385,6 @@ NODE_DISPLAY_NAME_MAPPINGS = {
"ImpactKSamplerBasicPipe": "KSampler (pipe)",
"ImpactKSamplerAdvancedBasicPipe": "KSampler (Advanced/pipe)",
"ImpactSEGSLabelAssign": "SEGS Assign (label)",
"ImpactSEGSLabelFilter": "SEGS Filter (label)",
"ImpactSEGSRangeFilter": "SEGS Filter (range)",
"ImpactSEGSOrderedFilter": "SEGS Filter (ordered)",
+2 -2
View File
@@ -65,7 +65,7 @@ class SEGSDetailerForAnimateDiff:
else:
cropped_image_frames = torch.concat((cropped_image_frames, cropped_image), dim=0)
cropped_image_frames = cropped_image_frames.cpu().numpy()
cropped_image_frames = cropped_image_frames.numpy()
enhanced_image_tensor, cnet_images = core.enhance_detail_for_animatediff(cropped_image_frames, model, clip, vae, guide_size, guide_size_for, max_size,
seg.bbox, seed, steps, cfg, sampler_name, scheduler,
positive, negative, denoise, seg.cropped_mask,
@@ -79,7 +79,7 @@ class SEGSDetailerForAnimateDiff:
if enhanced_image_tensor is None:
new_cropped_image = cropped_image_frames
else:
new_cropped_image = enhanced_image_tensor.cpu().numpy()
new_cropped_image = enhanced_image_tensor.numpy()
new_seg = SEG(new_cropped_image, seg.cropped_mask, seg.confidence, seg.crop_region, seg.bbox, seg.label, None)
new_segs.append(new_seg)
+1 -1
View File
@@ -2,7 +2,7 @@ import configparser
import os
version_code = [4, 81]
version_code = [4, 74]
version = f"V{version_code[0]}.{version_code[1]}" + (f'.{version_code[2]}' if len(version_code) > 2 else '')
dependency_version = 20
+15 -89
View File
@@ -18,7 +18,6 @@ from comfy import model_management
from impact import utils
from impact import impact_sampling
from concurrent.futures import ThreadPoolExecutor
from comfy.ldm.cascade.stage_c_coder import StageC_coder
SEG = namedtuple("SEG",
@@ -215,20 +214,6 @@ def enhance_detail(image, model, clip, vae, guide_size, guide_size_for_bbox, max
new_w = w
new_h = h
is_stable_cascade_mode = isinstance(vae.first_stage_model, StageC_coder)
if is_stable_cascade_mode:
dw = new_w % 8
dh = new_h % 8
# preserve aspect ratio as possible
if dw > 3 or dh > 3:
new_w += 8 - dw
new_h += 8 - dh
elif dw > 0 or dh > 0:
new_w -= dw
new_h -= dh
if detailer_hook is not None:
new_w, new_h = detailer_hook.touch_scaled_size(new_w, new_h)
@@ -240,21 +225,12 @@ def enhance_detail(image, model, clip, vae, guide_size, guide_size_for_bbox, max
cnet_pils = None
if control_net_wrapper is not None:
positive, negative, cnet_pils = control_net_wrapper.apply(positive, negative, upscaled_image, noise_mask)
model, cnet_pils2 = control_net_wrapper.doit_ipadapter(model)
cnet_pils.extend(cnet_pils2)
# prepare mask
if noise_mask is not None and inpaint_model:
positive, negative, latent_image = nodes.InpaintModelConditioning().encode(positive, negative, upscaled_image, vae, noise_mask)
else:
if is_stable_cascade_mode:
latent_image = detailer_hook.stable_cascade_vae_encode(vae, upscaled_image)
if latent_image is None:
print(f"[Impact Pack] When using the StableCascade model, it is necessary to connect the StableCascade_DetailerHook.")
raise Exception("StableCascade_DetailerHook is not provided.")
else:
latent_image = to_latent_image(upscaled_image, vae)
latent_image = to_latent_image(upscaled_image, vae)
if noise_mask is not None:
latent_image['noise_mask'] = noise_mask
@@ -280,17 +256,11 @@ def enhance_detail(image, model, clip, vae, guide_size, guide_size_for_bbox, max
refined_latent = impact_sampling.ksampler_wrapper(model2, seed2, steps2, cfg2, sampler_name2, scheduler2, positive2, negative2,
refined_latent, denoise2, refiner_ratio, refiner_model, refiner_clip, refiner_positive, refiner_negative)
# non-latent downscale - latent downscale cause bad quality
if detailer_hook is not None:
refined_latent = detailer_hook.pre_decode(refined_latent)
stage_b = detailer_hook.stable_cascade_stage_b(image, positive, negative, refined_latent)
else:
stage_b = None
if stage_b is None:
refined_image = vae.decode(refined_latent['samples'])
else:
refined_image = stage_b
# non-latent downscale - latent downscale cause bad quality
refined_image = vae.decode(refined_latent['samples'])
if detailer_hook is not None:
refined_image = detailer_hook.post_decode(refined_image)
@@ -1521,57 +1491,9 @@ class PixelKSampleUpscaler:
return refined_latent
class IPAdapterWrapper:
def __init__(self, ipadapter_pipe, weight, noise, weight_type, start_at, end_at, unfold_batch, faceid_v2, weight_v2, reference_image, prev_control_net=None):
self.reference_image = reference_image
self.ipadapter_pipe = ipadapter_pipe
self.weight = weight
self.weight_type = weight_type
self.noise = noise
self.start_at = start_at
self.end_at = end_at
self.unfold_batch = unfold_batch
self.prev_control_net = prev_control_net
self.faceid_v2 = faceid_v2
self.weight_v2 = weight_v2
self.image = reference_image
# name 'apply_ipadapter' isn't allowed
def doit_ipadapter(self, model):
cnet_image_list = [self.image]
prev_cnet_images = []
if 'IPAdapterApply' not in nodes.NODE_CLASS_MAPPINGS:
utils.try_install_custom_node('https://github.com/cubiq/ComfyUI_IPAdapter_plus',
"To use 'IPAdapterApplySEGS' node, 'ComfyUI IPAdapter Plus' extension is required.")
raise Exception(f"[ERROR] To use IPAdapterApplySEGS, you need to install 'ComfyUI IPAdapter Plus'")
obj = nodes.NODE_CLASS_MAPPINGS['IPAdapterApply']
ipadapter, _, clip_vision, insightface, lora_loader = self.ipadapter_pipe
model = lora_loader(model)
if self.prev_control_net is not None:
model, prev_cnet_images = self.prev_control_net.doit_ipadapter(model)
model = obj().apply_ipadapter(ipadapter, model, self.weight, clip_vision=clip_vision, image=self.image,
embeds=None, weight_type=self.weight_type, noise=self.noise,
attn_mask=None, start_at=self.start_at, end_at=self.end_at,
unfold_batch=self.unfold_batch, insightface=insightface, faceid_v2=self.faceid_v2, weight_v2=self.weight_v2)[0]
cnet_image_list.extend(prev_cnet_images)
return model, cnet_image_list
def apply(self, positive, negative, image, mask=None, use_acn=False):
if self.prev_control_net is not None:
return self.prev_control_net.apply(positive, negative, image, mask, use_acn=use_acn)
else:
return positive, negative, []
class ControlNetWrapper:
def __init__(self, control_net, strength, preprocessor, prev_control_net=None, original_size=None, crop_region=None, control_image=None):
def __init__(self, control_net, strength, preprocessor, prev_control_net=None,
original_size=None, crop_region=None, control_image=None):
self.control_net = control_net
self.strength = strength
self.preprocessor = preprocessor
@@ -1614,12 +1536,6 @@ class ControlNetWrapper:
return positive, negative, cnet_image_list
def doit_ipadapter(self, model):
if self.prev_control_net is not None:
return self.prev_control_net.doit_ipadapter(model)
else:
return model, []
class ControlNetAdvancedWrapper:
def __init__(self, control_net, strength, start_percent, end_percent, preprocessor, prev_control_net=None,
@@ -1823,6 +1739,16 @@ class BBoxDetectorBasedOnCLIPSeg:
self.aux = x
def get_seg_size(seg):
x1, y1, x2, y2 = seg.crop_region
return x2-x1, y2-y1
def get_bbox_size(seg):
x1, y1, x2, y2 = seg.bbox
return x2-x1, y2-y1
def update_node_status(node, text, progress=None):
if PromptServer.instance.client_id is None:
return
+9 -13
View File
@@ -209,20 +209,16 @@ class SimpleDetectorForEach:
if len(image) > 1:
raise Exception('[Impact Pack] ERROR: SimpleDetectorForEach does not allow image batches.\nPlease refer to https://github.com/ltdrdata/ComfyUI-extension-tutorials/blob/Main/ComfyUI-Impact-Pack/tutorial/batching-detailer.md for more information.')
if segm_detector_opt is not None and hasattr(segm_detector_opt, 'bbox_detector') and segm_detector_opt.bbox_detector == bbox_detector:
# Better segm support for YOLO-World detector
segs = segm_detector_opt.detect(image, sub_threshold, sub_dilation, crop_factor, drop_size, detailer_hook=detailer_hook)
else:
segs = bbox_detector.detect(image, bbox_threshold, bbox_dilation, crop_factor, drop_size, detailer_hook=detailer_hook)
segs = bbox_detector.detect(image, bbox_threshold, bbox_dilation, crop_factor, drop_size, detailer_hook=detailer_hook)
if sam_model_opt is not None:
mask = core.make_sam_mask(sam_model_opt, segs, image, "center-1", sub_dilation,
sub_threshold, sub_bbox_expansion, sam_mask_hint_threshold, False)
segs = core.segs_bitwise_and_mask(segs, mask)
elif segm_detector_opt is not None:
segm_segs = segm_detector_opt.detect(image, sub_threshold, sub_dilation, crop_factor, drop_size, detailer_hook=detailer_hook)
mask = core.segs_to_combined_mask(segm_segs)
segs = core.segs_bitwise_and_mask(segs, mask)
if sam_model_opt is not None:
mask = core.make_sam_mask(sam_model_opt, segs, image, "center-1", sub_dilation,
sub_threshold, sub_bbox_expansion, sam_mask_hint_threshold, False)
segs = core.segs_bitwise_and_mask(segs, mask)
elif segm_detector_opt is not None:
segm_segs = segm_detector_opt.detect(image, sub_threshold, sub_dilation, crop_factor, drop_size, detailer_hook=detailer_hook)
mask = core.segs_to_combined_mask(segm_segs)
segs = core.segs_bitwise_and_mask(segs, mask)
segs = core.dilate_segs(segs, post_dilation)
-28
View File
@@ -1,7 +1,6 @@
import sys
from . import hooks
from . import defs
import comfy
class SEGSOrderedFilterDetailerHookProvider:
@@ -82,30 +81,3 @@ class PreviewDetailerHookProvider:
def doit(self, quality, unique_id):
hook = hooks.PreviewDetailerHook(unique_id, quality)
return (hook, )
class StableCascade_DetailerHookProvider:
@classmethod
def INPUT_TYPES(s):
return {
"required": {
"b_model": ("MODEL",),
"b_vae": ("VAE",),
"b_seed": ("INT", {"default": 0, "min": 0, "max": 0xffffffffffffffff}),
"b_steps": ("INT", {"default": 5, "min": 1, "max": 10000}),
"b_cfg": ("FLOAT", {"default": 1.1, "min": 0.0, "max": 100.0, "step": 0.1, "round": 0.01}),
"b_sampler_name": (comfy.samplers.KSampler.SAMPLERS,),
"b_scheduler": (comfy.samplers.KSampler.SCHEDULERS,),
"c_compression": ("INT", {"default": 42, "min": 4, "max": 128, "step": 1}),
},
}
RETURN_TYPES = ("DETAILER_HOOK", )
FUNCTION = "doit"
CATEGORY = "ImpactPack/Util"
def doit(self, b_model, b_vae, b_seed, b_steps, b_cfg, b_sampler_name, b_scheduler, c_compression):
hook = hooks.StableCascade_DetailerHook(b_model, b_vae, b_seed, b_steps, b_cfg, b_sampler_name, b_scheduler, c_compression)
return (hook, )
+13 -96
View File
@@ -1,6 +1,4 @@
import copy
import comfy_extras.nodes_stable_cascade
import nodes
from impact import utils
@@ -10,8 +8,6 @@ from server import PromptServer
import asyncio
import folder_paths
import os
from impact import impact_sampling
class PixelKSampleHook:
cur_step = 0
@@ -105,20 +101,6 @@ class DetailerHookCombine(PixelKSampleHookCombine):
image = self.hook2.post_paste(image)
return image
def stable_cascade_vae_encode(self, vae, pixels):
latent = self.hook1.stable_cascade_vae_encode(vae, pixels)
if latent is not None:
return latent
return self.hook2.stable_cascade_vae_encode(vae, pixels)
def stable_cascade_stage_b(self, image, positive, negative, latent):
image = self.hook1.stable_cascade_stage_b(image, positive, negative, latent)
if image is not None:
return image
return self.hook2.stable_cascade_stage_b(image, positive, negative, latent)
class SimpleCfgScheduleHook(PixelKSampleHook):
target_cfg = 0
@@ -127,14 +109,11 @@ class SimpleCfgScheduleHook(PixelKSampleHook):
super().__init__()
self.target_cfg = target_cfg
def pre_ksample(self, model, seed, steps, cfg, sampler_name, scheduler, positive, negative, upscaled_latent, denoise):
if self.total_step > 1:
progress = self.cur_step / (self.total_step - 1)
gap = self.target_cfg - cfg
current_cfg = int(cfg + gap * progress)
else:
current_cfg = self.target_cfg
def pre_ksample(self, model, seed, steps, cfg, sampler_name, scheduler, positive, negative, upscaled_latent,
denoise):
progress = self.cur_step / self.total_step
gap = self.target_cfg - cfg
current_cfg = cfg + gap * progress
return model, seed, steps, current_cfg, sampler_name, scheduler, positive, negative, upscaled_latent, denoise
@@ -143,33 +122,14 @@ class SimpleDenoiseScheduleHook(PixelKSampleHook):
super().__init__()
self.target_denoise = target_denoise
def pre_ksample(self, model, seed, steps, cfg, sampler_name, scheduler, positive, negative, upscaled_latent, denoise):
if self.total_step > 1:
progress = self.cur_step / (self.total_step - 1)
gap = self.target_denoise - denoise
current_denoise = denoise + gap * progress
else:
current_denoise = self.target_denoise
def pre_ksample(self, model, seed, steps, cfg, sampler_name, scheduler, positive, negative, upscaled_latent,
denoise):
progress = self.cur_step / self.total_step
gap = self.target_denoise - denoise
current_denoise = denoise + gap * progress
return model, seed, steps, cfg, sampler_name, scheduler, positive, negative, upscaled_latent, current_denoise
class SimpleStepsScheduleHook(PixelKSampleHook):
def __init__(self, target_steps):
super().__init__()
self.target_steps = target_steps
def pre_ksample(self, model, seed, steps, cfg, sampler_name, scheduler, positive, negative, upscaled_latent, denoise):
if self.total_step > 1:
progress = self.cur_step / (self.total_step - 1)
gap = self.target_steps - steps
current_steps = int(steps + gap * progress)
else:
current_steps = self.target_steps
return model, seed, current_steps, cfg, sampler_name, scheduler, positive, negative, upscaled_latent, denoise
class DetailerHook(PixelKSampleHook):
def cycle_latent(self, latent):
return latent
@@ -180,44 +140,6 @@ class DetailerHook(PixelKSampleHook):
def post_paste(self, image):
return image
def stable_cascade_vae_encode(self, vae, pixels):
return None
def stable_cascade_stage_b(self, image, positive, negative, latent):
return None
class StableCascade_DetailerHook(DetailerHook):
def __init__(self, b_model, b_vae, b_seed, b_steps, b_cfg, b_sampler_name, b_scheduler, c_compression):
super().__init__()
self.b_model = b_model
self.b_vae = b_vae
self.b_seed = b_seed
self.b_steps = b_steps
self.b_cfg = b_cfg
self.b_sampler_name = b_sampler_name
self.b_scheduler = b_scheduler
self.c_compression = c_compression
self.b_latent = None
def stable_cascade_vae_encode(self, vae, pixels):
obj = comfy_extras.nodes_stable_cascade.StableCascade_StageC_VAEEncode()
stage_c, stage_b = obj.generate(pixels, vae, compression=self.c_compression)
self.b_latent = stage_b
return stage_c
def stable_cascade_stage_b(self, image, positive, negative, latent):
# prepare stage_b
# self.b_latent['noise_mask'] = latent['noise_mask']
b_positive = comfy_extras.nodes_stable_cascade.StableCascade_StageB_Conditioning().set_prior(positive, latent)[0]
# stage_b sampling
b_latent = impact_sampling.ksampler_wrapper(self.b_model, self.b_seed, self.b_steps, self.b_cfg, self.b_sampler_name, self.b_scheduler, b_positive, negative, self.b_latent, 1.0)
# stage_b decoding
self.b_latent = None
return self.b_vae.decode(b_latent['samples'])
class SimpleDetailerDenoiseSchedulerHook(DetailerHook):
def __init__(self, target_denoise):
@@ -225,14 +147,9 @@ class SimpleDetailerDenoiseSchedulerHook(DetailerHook):
self.target_denoise = target_denoise
def pre_ksample(self, model, seed, steps, cfg, sampler_name, scheduler, positive, negative, latent, denoise):
if self.total_step > 1:
progress = self.cur_step / (self.total_step - 1)
gap = self.target_denoise - denoise
current_denoise = denoise + gap * progress
else:
# ignore hook if total cycle <= 1
current_denoise = denoise
progress = self.cur_step / self.total_step
gap = self.target_denoise - denoise
current_denoise = denoise + gap * progress
return model, seed, steps, cfg, sampler_name, scheduler, positive, negative, latent, current_denoise
+6 -32
View File
@@ -237,10 +237,8 @@ class DetailerForEach:
else:
cropped_mask = None
if wildcard_chooser is not None and wmode != "LAB":
if wildcard_chooser is not None:
seg_seed, wildcard_item = wildcard_chooser.get(seg)
elif wildcard_chooser is not None and wmode == "LAB":
seg_seed, wildcard_item = None, wildcard_chooser.get(seg)
else:
seg_seed, wildcard_item = None, None
@@ -274,7 +272,7 @@ class DetailerForEach:
# Convert enhanced_pil_alpha to RGBA mode
enhanced_image_alpha = tensor_convert_rgba(enhanced_image)
new_seg_image = enhanced_image.numpy() # alpha should not be applied to seg_image
# Apply the mask
mask = tensor_resize(mask, *tensor_get_size(enhanced_image))
tensor_putalpha(enhanced_image_alpha, mask)
@@ -782,30 +780,6 @@ class DenoiseScheduleHookProvider:
return (hook, )
class StepsScheduleHookProvider:
schedules = ["simple"]
@classmethod
def INPUT_TYPES(s):
return {"required": {
"schedule_for_iteration": (s.schedules,),
"target_steps": ("INT", {"default": 20, "min": 1, "max": 10000}),
},
}
RETURN_TYPES = ("PK_HOOK",)
FUNCTION = "doit"
CATEGORY = "ImpactPack/Upscale"
def doit(self, schedule_for_iteration, target_steps):
hook = None
if schedule_for_iteration == "simple":
hook = hooks.SimpleStepsScheduleHook(target_steps)
return (hook, )
class DetailerHookCombine:
@classmethod
def INPUT_TYPES(s):
@@ -1130,7 +1104,7 @@ class IterativeLatentUpscale:
new_h = h*upscale_factor
core.update_node_status(unique_id, f"Final step | x{upscale_factor:.2f}", 1.0)
print(f"IterativeLatentUpscale[Final]: {new_w:.1f}x{new_h:.1f} (scale:{upscale_factor:.2f}) ")
step_info = steps-1, steps
step_info = steps, steps
current_latent = upscaler.upscale_shape(step_info, current_latent, new_w, new_h, temp_prefix)
core.update_node_status(unique_id, "", None)
@@ -1477,7 +1451,7 @@ class SegsBitwiseAndMask:
def doit(self, segs, mask):
return (core.segs_bitwise_and_mask(segs, mask), )
class SegsBitwiseAndMaskForEach:
@classmethod
@@ -1652,7 +1626,7 @@ class SubtractMask:
"mask2": ("MASK", ),
}
}
RETURN_TYPES = ("MASK",)
FUNCTION = "doit"
@@ -1783,7 +1757,7 @@ class ImageReceiver:
return hash(image_data)
else:
return hash(image)
from server import PromptServer
+12 -13
View File
@@ -119,18 +119,18 @@ def separated_sample(model, add_noise, seed, steps, cfg, sampler_name, scheduler
def ksampler_wrapper(model, seed, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, denoise,
refiner_ratio=None, refiner_model=None, refiner_clip=None, refiner_positive=None, refiner_negative=None, sigma_factor=1.0):
refiner_ratio=None, refiner_model=None, refiner_clip=None, refiner_positive=None, refiner_negative=None):
if refiner_ratio is None or refiner_model is None or refiner_clip is None or refiner_positive is None or refiner_negative is None:
refined_latent = nodes.KSampler().sample(model, seed, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, denoise * sigma_factor)[0]
refined_latent = nodes.KSampler().sample(model, seed, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, denoise)[0]
else:
advanced_steps = math.floor(steps / denoise)
start_at_step = advanced_steps - steps
end_at_step = start_at_step + math.floor(steps * (1.0 - refiner_ratio))
# print(f"pre: {start_at_step} .. {end_at_step} / {advanced_steps}")
print(f"pre: {start_at_step} .. {end_at_step} / {advanced_steps}")
temp_latent = separated_sample(model, True, seed, advanced_steps, cfg, sampler_name, scheduler,
positive, negative, latent_image, start_at_step, end_at_step, True, sigma_ratio=sigma_factor)
positive, negative, latent_image, start_at_step, end_at_step, True)
if 'noise_mask' in latent_image:
# noise_latent = \
@@ -141,9 +141,9 @@ def ksampler_wrapper(model, seed, steps, cfg, sampler_name, scheduler, positive,
latent_compositor = nodes.NODE_CLASS_MAPPINGS['LatentCompositeMasked']()
temp_latent = latent_compositor.composite(latent_image, temp_latent, 0, 0, False, latent_image['noise_mask'])[0]
# print(f"post: {end_at_step} .. {advanced_steps + 1} / {advanced_steps}")
print(f"post: {end_at_step} .. {advanced_steps + 1} / {advanced_steps}")
refined_latent = separated_sample(refiner_model, False, seed, advanced_steps, cfg, sampler_name, scheduler,
refiner_positive, refiner_negative, temp_latent, end_at_step, advanced_steps + 1, False, sigma_ratio=sigma_factor)
refiner_positive, refiner_negative, temp_latent, end_at_step, advanced_steps + 1, False)
return refined_latent
@@ -151,19 +151,18 @@ def ksampler_wrapper(model, seed, steps, cfg, sampler_name, scheduler, positive,
class KSamplerAdvancedWrapper:
params = None
def __init__(self, model, cfg, sampler_name, scheduler, positive, negative, sampler_opt=None, sigma_factor=1.0):
self.params = model, cfg, sampler_name, scheduler, positive, negative, sigma_factor
def __init__(self, model, cfg, sampler_name, scheduler, positive, negative, sampler_opt=None):
self.params = model, cfg, sampler_name, scheduler, positive, negative
self.sampler_opt = sampler_opt
def clone_with_conditionings(self, positive, negative):
model, cfg, sampler_name, scheduler, _, _, _ = self.params
model, cfg, sampler_name, scheduler, _, _ = self.params
return KSamplerAdvancedWrapper(model, cfg, sampler_name, scheduler, positive, negative, self.sampler_opt)
def sample_advanced(self, add_noise, seed, steps, latent_image, start_at_step, end_at_step, return_with_leftover_noise, hook=None,
recovery_mode="ratio additional", recovery_sampler="AUTO", recovery_sigma_ratio=1.0):
model, cfg, sampler_name, scheduler, positive, negative, sigma_factor = self.params
# steps, start_at_step, end_at_step = self.compensate_denoise(steps, start_at_step, end_at_step)
model, cfg, sampler_name, scheduler, positive, negative = self.params
if hook is not None:
model, seed, steps, cfg, sampler_name, scheduler, positive, negative, upscaled_latent = hook.pre_ksample_advanced(model, add_noise, seed, steps, cfg, sampler_name, scheduler,
@@ -184,7 +183,7 @@ class KSamplerAdvancedWrapper:
if sigma_ratio > 0:
latent_image = separated_sample(model, add_noise, seed, steps, cfg, sampler_name, scheduler,
positive, negative, latent_image, start_at_step, end_at_step,
return_with_leftover_noise, sigma_ratio=sigma_ratio * sigma_factor, sampler_opt=self.sampler_opt)
return_with_leftover_noise, sigma_ratio=sigma_ratio, sampler_opt=self.sampler_opt)
except ValueError as e:
if str(e) == 'sigma_min and sigma_max must not be 0':
print(f"\nWARN: sampling skipped - sigma_min and sigma_max are 0")
@@ -208,7 +207,7 @@ class KSamplerAdvancedWrapper:
try:
latent_image = separated_sample(model, add_noise, seed, steps, cfg, recovery_sampler, scheduler,
positive, negative, latent_image, start_at_step-compensate, end_at_step,
return_with_leftover_noise, sigma_ratio=recovery_sigma_ratio * sigma_factor, sampler_opt=self.sampler_opt)
return_with_leftover_noise, sigma_ratio=recovery_sigma_ratio, sampler_opt=self.sampler_opt)
except ValueError as e:
if str(e) == 'sigma_min and sigma_max must not be 0':
print(f"\nWARN: sampling skipped - sigma_min and sigma_max are 0")
+9 -10
View File
@@ -424,11 +424,11 @@ class ImpactQueueTriggerCountdown:
@classmethod
def INPUT_TYPES(cls):
return {"required": {
"signal": (any_typ,),
"count": ("INT", {"default": 0, "min": 0, "max": 0xffffffffffffffff}),
"total": ("INT", {"default": 10, "min": 1, "max": 0xffffffffffffffff}),
"mode": ("BOOLEAN", {"default": True, "label_on": "Trigger", "label_off": "Don't trigger"}),
},
"optional": {"signal": (any_typ,),},
"hidden": {"unique_id": "UNIQUE_ID"}
}
@@ -439,15 +439,14 @@ class ImpactQueueTriggerCountdown:
RETURN_NAMES = ("signal_opt", "count", "total")
OUTPUT_NODE = True
def doit(self, count, total, mode, unique_id, signal=None):
if (mode):
if count < total - 1:
PromptServer.instance.send_sync("impact-node-feedback",
{"node_id": unique_id, "widget_name": "count", "type": "int", "value": count+1})
PromptServer.instance.send_sync("impact-add-queue", {})
if count >= total - 1:
PromptServer.instance.send_sync("impact-node-feedback",
{"node_id": unique_id, "widget_name": "count", "type": "int", "value": 0})
def doit(self, signal, count, total, mode, unique_id):
if count < total - 1 and (mode):
PromptServer.instance.send_sync("impact-node-feedback",
{"node_id": unique_id, "widget_name": "count", "type": "int", "value": count+1})
PromptServer.instance.send_sync("impact-add-queue", {})
if count >= total - 1:
PromptServer.instance.send_sync("impact-node-feedback",
{"node_id": unique_id, "widget_name": "count", "type": "int", "value": 0})
return (signal, count, total)
-1
View File
@@ -1,6 +1,5 @@
import folder_paths
from impact.core import *
import os
import mmcv
from mmdet.apis import (inference_detector, init_detector)
+92 -88
View File
@@ -1,7 +1,10 @@
import os
import sys
import torch
import impact.impact_server
import nodes
from nodes import MAX_RESOLUTION
from impact.utils import *
@@ -123,6 +126,94 @@ class SEGSDetailer:
return (segs, cnet_pil_list)
class SEGSLatentComposite:
@classmethod
def INPUT_TYPES(s):
return {"required": {
"src_segs": ("SEGS", ),
"dest_segs": ("SEGS", ),
"target_latent": ("LATENT", ),
"vae": ("VAE", ),
},
"optional": {"ref_image_opt": ("IMAGE", ), }
}
RETURN_TYPES = ("LATENT", )
FUNCTION = "doit"
CATEGORY = "ImpactPack/__for_testing"
@staticmethod
def doit(src_segs, dest_segs, target_latent, vae, ref_image_opt=None):
apply_count = min(len(src_segs[1]), len(dest_segs[1]))
if apply_count == 0:
print(f"[Impact Pack] SEGSLatentComposite: src_segs or dest_segs is empty")
return (target_latent, )
if src_segs[1][0].cropped_image is None:
if ref_image_opt is None:
print(f"[Impact Pack] SEGSLatentComposite: there is no cropped_image nor ref_image_opt")
return (target_latent, )
src_segs = DefaultImageForSEGS().doit(src_segs, ref_image_opt, False)[0]
target_latent = target_latent.copy()
target_latent['samples'] = torch.clone(target_latent['samples'])
for i in range(0, apply_count):
seg1 = src_segs[1][i]
seg2 = dest_segs[1][i]
w1, h1 = core.get_bbox_size(seg1)
w2, h2 = core.get_bbox_size(seg2)
scale_factor = 1
if w1-w2 < h1-h2:
# fit to vertical
target_h = h2
scale_factor = target_h/h1
target_w = int(w1 * scale_factor)
dy = 0
dx = int((w2-target_w)/2//8)
elif w1-w2 > h1-h2:
# fit to horizontal
target_w = w2
scale_factor = target_w/w1
target_h = int(h1*scale_factor)
dx = 0
dy = int((h2-target_h)/2//8)
else:
# same ratio
target_h, target_w = w2, h2
scale_factor = target_w/w1
dx, dy = 0, 0
ax1, ay1, ax2, ay2 = seg1.bbox
bx1, by1, _, _ = seg1.crop_region
ax1, ax2, ay1, ay2 = (ax1-bx1), (ax2-bx1), (ay1-by1), (ay2-by1)
seg1_bbox_image = seg1.cropped_image[:, ay1:ay2, ax1:ax2, :]
seg1_image = utils.tensor_resize(seg1_bbox_image, target_w, target_h)
seg1_samples = nodes.VAEEncode().encode(vae, seg1_image)[0]['samples']
seg1_mask = utils.make_3d_mask(torch.from_numpy(seg1.cropped_mask))
seg1_mask = seg1_mask[ :, ay1:ay2, ax1:ax2]
seg1_mask = utils.resize_mask(seg1_mask, tuple(seg1_samples.shape[2:4]))
seg1_mask = seg1_mask.unsqueeze(0)
x1, y1, _, _ = seg2.bbox
x1 = int(x1//8 + dx)
y1 = int(y1//8 + dy)
x2 = x1 + seg1_samples.shape[3]
y2 = y1 + seg1_samples.shape[2]
target_samples = target_latent['samples'][:, :, y1:y2, x1:x2]
target_samples = seg1_samples*seg1_mask + target_samples*(1.0-seg1_mask)
target_latent['samples'][:, :, y1:y2, x1:x2] = target_samples
return (target_latent,)
class SEGSPaste:
@classmethod
def INPUT_TYPES(s):
@@ -415,44 +506,6 @@ class SEGSLabelFilter:
return SEGSLabelFilter.filter(segs, labels)
class SEGSLabelAssign:
@classmethod
def INPUT_TYPES(s):
return {"required": {
"segs": ("SEGS", ),
"labels": ("STRING", {"multiline": True, "placeholder": "List the label to be assigned in order of segs, separated by commas"}),
},
}
RETURN_TYPES = ("SEGS",)
RETURN_NAMES = ("SEGS",)
FUNCTION = "doit"
CATEGORY = "ImpactPack/Util"
@staticmethod
def assign(segs, labels):
labels = [label.strip() for label in labels]
if len(labels) != len(segs[1]):
print(f'Warning (SEGSLabelAssign): length of labels ({len(labels)}) != length of segs ({len(segs[1])})')
labeled_segs = []
idx = 0
for x in segs[1]:
if len(labels) > idx:
x = x._replace(label=labels[idx])
labeled_segs.append(x)
idx += 1
return ((segs[0], labeled_segs), )
def doit(self, segs, labels):
labels = labels.split(',')
return SEGSLabelAssign.assign(segs, labels)
class SEGSOrderedFilter:
@classmethod
def INPUT_TYPES(s):
@@ -1121,55 +1174,6 @@ class MaskToSEGS_for_AnimateDiff:
return MaskToSEGS().doit(result_mask, False, crop_factor, False, drop_size, contour_fill)
class IPAdapterApplySEGS:
@classmethod
def INPUT_TYPES(s):
return {"required": {
"segs": ("SEGS",),
"ipadapter_pipe": ("IPADAPTER_PIPE",),
"weight": ("FLOAT", {"default": 0.7, "min": -1, "max": 3, "step": 0.05}),
"noise": ("FLOAT", {"default": 0.4, "min": 0.0, "max": 1.0, "step": 0.01}),
"weight_type": (["original", "linear", "channel penalty"], {"default": 'channel penalty'}),
"start_at": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 1.0, "step": 0.001}),
"end_at": ("FLOAT", {"default": 0.9, "min": 0.0, "max": 1.0, "step": 0.001}),
"unfold_batch": ("BOOLEAN", {"default": False}),
"faceid_v2": ("BOOLEAN", {"default": False}),
"weight_v2": ("FLOAT", {"default": 1.0, "min": -1, "max": 3, "step": 0.05}),
"context_crop_factor": ("FLOAT", {"default": 1.2, "min": 1.0, "max": 100, "step": 0.1}),
"reference_image": ("IMAGE",),
}
}
RETURN_TYPES = ("SEGS",)
FUNCTION = "doit"
CATEGORY = "ImpactPack/Util"
def doit(self, segs, ipadapter_pipe, weight, noise, weight_type, start_at, end_at, unfold_batch, faceid_v2, weight_v2, context_crop_factor, reference_image):
if len(ipadapter_pipe) == 4:
print(f"[Impact Pack] IPAdapterApplySEGS: Installed Inspire Pack is outdated.")
raise Exception("Inspire Pack is outdated.")
new_segs = []
h, w = segs[0]
if reference_image.shape[2] != w or reference_image.shape[1] != h:
reference_image = tensor_resize(reference_image, w, h)
for seg in segs[1]:
# The context_crop_region sets how much wider the IPAdapter context will reflect compared to the crop_region, not the bbox
context_crop_region = make_crop_region(w, h, seg.crop_region, context_crop_factor)
cropped_image = crop_image(reference_image, context_crop_region)
control_net_wrapper = core.IPAdapterWrapper(ipadapter_pipe, weight, noise, weight_type, start_at, end_at, unfold_batch, faceid_v2, weight_v2, cropped_image, prev_control_net=seg.control_net_wrapper)
new_seg = SEG(seg.cropped_image, seg.cropped_mask, seg.confidence, seg.crop_region, seg.bbox, seg.label, control_net_wrapper)
new_segs.append(new_seg)
return ((segs[0], new_segs), )
class ControlNetApplySEGS:
@classmethod
def INPUT_TYPES(s):
@@ -1437,7 +1441,7 @@ class MakeTileSEGS:
FUNCTION = "doit"
CATEGORY = "ImpactPack/__for_testing"
CATEGORY = "ImpactPack/Util"
def doit(self, images, bbox_size, crop_factor, min_overlap, filter_segs_dilation, mask_irregularity=0, irregular_mask_mode="Reuse fast", filter_in_segs_opt=None, filter_out_segs_opt=None):
if bbox_size <= 2*min_overlap:
+2 -3
View File
@@ -67,7 +67,6 @@ class KSamplerAdvancedProvider:
"cfg": ("FLOAT", {"default": 8.0, "min": 0.0, "max": 100.0}),
"sampler_name": (comfy.samplers.KSampler.SAMPLERS, ),
"scheduler": (comfy.samplers.KSampler.SCHEDULERS, ),
"sigma_factor": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 10.0, "step": 0.01}),
"basic_pipe": ("BASIC_PIPE", )
},
"optional": {
@@ -80,9 +79,9 @@ class KSamplerAdvancedProvider:
CATEGORY = "ImpactPack/Sampler"
def doit(self, cfg, sampler_name, scheduler, basic_pipe, sigma_factor=1.0, sampler_opt=None):
def doit(self, cfg, sampler_name, scheduler, basic_pipe, sampler_opt=None):
model, _, _, positive, negative = basic_pipe
sampler = KSamplerAdvancedWrapper(model, cfg, sampler_name, scheduler, positive, negative, sampler_opt=sampler_opt, sigma_factor=sigma_factor)
sampler = KSamplerAdvancedWrapper(model, cfg, sampler_name, scheduler, positive, negative, sampler_opt=sampler_opt)
return (sampler, )
+15 -13
View File
@@ -8,8 +8,6 @@ from . import config
from PIL import Image, ImageFilter
from scipy.ndimage import zoom
import comfy
import comfy.ldm.cascade as cascade
from comfy_extras import nodes_stable_cascade
class TensorBatchBuilder:
@@ -492,21 +490,14 @@ def crop_image(image, crop_region):
return crop_tensor4(image, crop_region)
def to_latent_image(pixels, vae, compression=None):
def to_latent_image(pixels, vae):
x = pixels.shape[1]
y = pixels.shape[2]
if pixels.shape[1] != x or pixels.shape[2] != y:
pixels = pixels[:, :x, :y, :]
vae_encode = nodes.VAEEncode()
if hasattr(nodes.VAEEncode, "vae_encode_crop_pixels"):
# backward compatibility
print(f"[Impact Pack] ComfyUI is outdated.")
pixels = vae_encode.vae_encode_crop_pixels(pixels)
t = vae.encode(pixels[:, :, :, :3])
return {"samples": t}
return vae_encode.encode(vae, pixels)[0]
pixels = nodes.VAEEncode.vae_encode_crop_pixels(pixels)
t = vae.encode(pixels[:, :, :, :3])
return {"samples": t}
def empty_pil_tensor(w=64, h=64):
@@ -533,6 +524,17 @@ def make_3d_mask(mask):
return mask
def get_mask_size(mask):
if len(mask.shape) == 2:
return mask.shape[1], mask.shape[0]
elif len(mask.shape) == 3:
return mask.shape[2], mask.shape[1]
elif len(mask.shape) == 4:
return mask.shape[3], mask.shape[2]
raise Exception("unexpected mask dimension")
def is_same_device(a, b):
a_device = torch.device(a) if isinstance(a, str) else a
b_device = torch.device(b) if isinstance(b, str) else b