Added combinatorial detailer

This commit is contained in:
curious.jp
2024-05-05 16:06:07 +10:00
parent 5e3d1a9814
commit 6858d6f889
3 changed files with 74 additions and 9 deletions
+18 -6
View File
@@ -1,7 +1,19 @@
# ComfyUI-MaskBatchPermutations
Permutes a mask batch to present possible additive combinations. Passing a mask batch (e.g. out of [SEGS to Mask Batch](https://github.com/ltdrdata/ComfyUI-Impact-Pack)) will return a new mask batch representing all the possible combinations of the included masks. So, a mask batch with two mask sections, "A" and "B, will return a batch containing an empty mask, an empty mask & A, an empty mask & B, and an empty mask & A & B.
## example workflow
![An image embedding a workflow showing this node being used with a mask batch with three items.](workflow_example.png)
This image contains an embedded workflow.
## why?
"Automatic" face detailing without direct operator intervention usually works well, but occasionally it wrecks an otherwise good face. Instead of having to hand compose them back together in something like GIMP and fix the metadata, I decided to create this instead.
Provides two nodes, Permute Mask Batch and Combinatorial Detailer.
## Permute Mask Batch
Passing a mask batch (e.g. out of [SEGS to Mask Batch](https://github.com/ltdrdata/ComfyUI-Impact-Pack)) will return a new mask batch representing all the possible combinations of the included masks. So, a mask batch with two masks, "A" and "B, will return a new batch containing an empty mask, an empty mask & A, an empty mask & B, and an empty mask & A & B.
### example workflow
![An image embedding a workflow showing this node being used with a mask batch with three items.](workflow_example.png)
This image contains an embedded workflow.
## Combinatorial Detailer
Similar to Permute Mask Batch but accepts a mask batch, a base image, and then a batch of candidate images (for example, the batched outputs of several separate detailer passes using different prompts or seeds). Provides a batch of images representing the possible combinations of the base image, masks and candidates. Be advised that this can create very large batches - a set of masks representing three regions and with two candidate images will generate (2 + 1)<sup>3</sup> combinations (27) as each mask area will present with either the base image, or one of the two candidates.
### example workflow
In this example, I have given both detailers deliberately divergent prompts to make it clearer in the example output what is going on.
![An image embedding a workflow showing this node being used with a mask batch with three items and two candidates.](workflow_combi.png)
# why?
"Automatic" face detailing without direct operator intervention usually works well, but occasionally it wrecks an otherwise good face. Instead of having to hand compose them back together in something like GIMP and fix the metadata, I decided to create this instead.
+56 -3
View File
@@ -35,14 +35,67 @@ class PermuteMaskBatch:
output[i] = combined
return (output,)
class CombinatorialDetailer:
# no internal state
def __init__(self):
pass
@classmethod
def INPUT_TYPES(s):
"""
Input: mask
"""
return {
"required": {
"masks": ("MASK",),
"base_image": ("IMAGE",),
"candidates": ("IMAGE",),
},
}
RETURN_TYPES = ("IMAGE",)
RETURN_NAMES = ("IMAGE",)
FUNCTION = "combinatorialDetailer"
OUTPUT_NODE = False
CATEGORY = "image"
def combinatorialDetailer(self, masks, base_image, candidates):
candidate_count, height, width, _ = candidates.shape
mask_count = masks.shape[0]
expanded_masks = [x.unsqueeze(-1) for x in masks]
# Each mask area can be in one of `n + 1` states (all candidates + base)
num_combinations = (candidate_count + 1) ** mask_count
output_images = torch.zeros((num_combinations, height, width, 3), dtype=base_image.dtype)
output_images[0] = base_image[0]
# Iterate over all other possible combinations
for i in range(1, num_combinations):
combined_image = base_image[0].clone()
current_combination = i
for mask_index in range(mask_count):
selected_candidate = current_combination % (candidate_count + 1)
# print("out image", i, "mask index", mask_index, "selected candidate", selected_candidate)
if selected_candidate != 0:
combined_image = torch.where(expanded_masks[mask_index] == 1, candidates[selected_candidate - 1], combined_image)
current_combination //= (candidate_count + 1)
output_images[i] = combined_image
return (output_images,)
NODE_CLASS_MAPPINGS= {
"PermuteMaskBatch": PermuteMaskBatch,
"CombinatorialDetailer": CombinatorialDetailer,
}
NODE_DISPLAY_NAME_MAPPINGS = {
"PermuteMaskBatch": 'Permute Mask Batch'
"PermuteMaskBatch": "Permute Mask Batch",
"CombinatorialDetailer": "Combinatorial Detailer",
}
__version__ = '1.0.0'
__all__ = ['NODE_CLASS_MAPPINGS', 'NODE_DISPLAY_NAME_MAPPINGS']
__version__ = "1.1.0"
__all__ = ["NODE_CLASS_MAPPINGS", "NODE_DISPLAY_NAME_MAPPINGS"]
Binary file not shown.

After

Width:  |  Height:  |  Size: 4.0 MiB