diff --git a/README.md b/README.md index 971f040..63a66d5 100644 --- a/README.md +++ b/README.md @@ -40,10 +40,15 @@ Useful editor actions: - `Replace grid`: replace the whole layout with an even grid. - `Grid in T#`: subdivide the selected tile without rebuilding the whole layout. +- `Show masks` / `Hide masks`: toggle a colored overlay for tile object masks in the editor. +- `Stitch mask` / `Raw masks`: preview the hardened and feathered replace-paste mask shape without changing saved masks. - `Mask to Tile`: turn painted regions into object tiles. +- `Merge masks`: Shift/Cmd/Ctrl-select multiple tiles, then union their cropped object masks back onto the selected tiles. +- `Merge tiles`: Shift/Cmd/Ctrl-select multiple tiles, then replace them with one larger tile; existing masks are unioned into the merged tile when available. - `Refresh masks`: after manually moving sub-tiles from a masked tile, re-crop the inherited masks to the current tile boxes. - `Fill gaps`: add background tiles for uncovered areas. - `Auto Tile`: run SAM3.1 or QwenVL3 detection and write the detected layout back into the editor. +- `Auto SAM`: run SAM3.1 detection and keep only detected object tiles without automatic gap filling. When subdividing a masked tile with `Grid in`, the Mask mode can crop the original object mask into child tiles or skip empty mask children. The child tiles keep a parent mask source, so `Refresh masks` can update them after manual edits. @@ -55,6 +60,9 @@ When subdividing a masked tile with `Grid in`, the Mask mode can crop the origin | `TTP Smart Tile Set Preview` | Preview a tile set as a contact sheet or one selected tile. | | `TTP QwenVL3 Local Loader` | Load a local QwenVL tagging model from `ComfyUI/models/text_encoders`. | | `TTP Smart Tile QwenVL Prompt Set Builder` | Generate per-tile prompts before loop processing. | +| `TTP Smart Tile Prompt Override` | Filter selected tiles and replace, prepend, append, or find/replace per-tile prompts before sampling. | +| `TTP Smart Tile Composite Override` | Filter selected tiles and set composite mode, replace shape, layer, priority, occlusion priority, blend, and importance before assembly. | +| `TTP Smart Tile Stack Order` | Put tiles such as `T1,T3,T2` at the front in that order, leaving all unlisted tiles behind. | | `TTP Smart Tile Semantic Rank` | Classify tiles and write recommended layer, priority, scale weight, and composite metadata. | | `TTP Smart Tile Loop Source` | Output one tile at a time for VAE Encode / sampler / VAE Decode. | | `TTP Smart Tile Loop Collect` | Collect each processed tile back into the same tile set. | @@ -68,11 +76,14 @@ When subdividing a masked tile with `Grid in`, the Mask mode can crop the origin ```text TTP Smart Tile Interactive Crop -> TTP Smart Tile QwenVL Prompt Set Builder (optional) + -> TTP Smart Tile Prompt Override (optional) + -> TTP Smart Tile Composite Override (optional) -> TTP Smart Tile Semantic Rank (optional) -> TTP Smart Tile Loop Source -> VAE Encode / Sampler / VAE Decode -> TTP Smart Tile Loop Collect -> TTP Smart Tile Semantic Rank (optional final refresh) + -> TTP Smart Tile Stack Order (optional final manual order) -> TTP Smart Tile Output Size Estimate (optional) -> TTP Smart Tile Assemble -> TTP Smart Tile Save Final Image @@ -111,9 +122,12 @@ This workflow is the recommended full Smart Tile loop. It uses `TTP Smart Tile I | Node / 节点 | Example setting / 示例设置 | Notes / 说明 | |---|---|---| -| `Interactive Crop` | `auto_detect_mode=sam3.1`, `default_pad=32`, `default_blend=32`, `auto_object_padding=64`, `auto_max_tiles=16` | Use `Auto Tile` to create an object-aware layout, then set `auto_detect_mode=none` if you want to freeze manual edits. / 用 `Auto Tile` 生成语义分块；如果之后要固定手动编辑结果，可以把 `auto_detect_mode` 改回 `none`。 | +| `Interactive Crop` | `auto_detect_mode=sam3.1`, `default_pad=32`, `default_blend=32`, `auto_object_padding=64`, `auto_mask_expand=16`, `auto_max_tiles=16` | Use `Auto Tile` to create an object-aware layout, then set `auto_detect_mode=none` if you want to freeze manual edits. `auto_mask_expand` dilates object masks for edit pasteback; set it to `0` for exact masks. / 用 `Auto Tile` 生成语义分块；如果之后要固定手动编辑结果，可以把 `auto_detect_mode` 改回 `none`。`auto_mask_expand` 会膨胀对象蒙版，适合 edit 贴回；设为 `0` 则保留精确蒙版。 | | `QwenVL3 Local Loader` | `qwen3vl_4b_fp8_scaled.safetensors` | Replace this with the QwenVL model installed in `ComfyUI/models/text_encoders`. / 按你本机 `models/text_encoders` 里的 QwenVL 模型替换。 | | `QwenVL Prompt Set Builder` | `reference_image_mode=first_message`, `prompt_preset=tile_img2img_prompt`, `output_language=chinese` | Builds prompts before the tile loop, then `Loop Source.prompt` feeds the text encoder. / 在循环前一次性生成每块提示词，然后由 `Loop Source.prompt` 送入文本编码。 | +| `Prompt Override` | `selector_type=label`, `selector=face`, `prompt_mode=replace` | Rewrites only matched tile prompts. Use `drop_unmatched` when only selected regions should run through an edit sampler and Assemble should keep the rest from `source_image`. / 只改写匹配 tile 的提示词；局部 edit 时可用 `drop_unmatched`，其余区域由 Assemble 的 `source_image` 底图保留。 | +| `Composite Override` | `selector_type=index`, `selector=T3`, `composite_mode=replace_tile`, `replace_tile_shape=mask_first`, `occlusion_priority_mode=max` | Puts selected tiles above lower layers and makes them hard-replace the canvas. `mask_first` uses an object mask when present and hardens the mask interior like an inpaint stitch; `tile_box_first` ignores the mask and replaces the full tile box. / 把指定 tile 放到更高层并硬替换画布；`mask_first` 有 mask 就按 mask 替换，并像 inpaint stitch 一样硬化 mask 主体；`tile_box_first` 即使有 mask 也按完整 tile 方块替换。 | +| `Stack Order` | `tile_order=T1,T3,T2`, `top_order=first_on_top`, `composite_mode=replace_tile` | Makes the listed tiles the front stack in the typed order while all unlisted tiles stay behind. Place it after the final `Semantic Rank` so manual order wins. The tiles keep their own `blend` and mask feathering. / 按填写顺序把这些 tile 放到最前面，未填写的 tile 都在后面；建议放在最终 `Semantic Rank` 后面，让手动顺序最后生效；tile 仍使用各自的 `blend` 和 mask 羽化。 | | `Image Upscale Prep` | `scale=2.5`, `round_to=8`, `max_megapixels=1.5`, `use_upscale_model=true` | Enlarges each tile before img2img while keeping each tile under the megapixel cap. / 采样前放大每块 tile，同时用百万像素上限控制显存。 | | `Loop Source` / `Loop Collect` | `Process All Tiles` workflow | The tile count can be 4, 8, 16, or any layout count; no manual sampler duplication is needed. / 分块数量可以变化，不需要手动复制多套 sampler。 | | `Output Size Estimate` | `focus_weighted` | Estimates the final canvas from processed tile sizes, giving more influence to focus/detail tiles. / 根据处理后的 tile 尺寸估算最终画布，并更重视细节块。 | @@ -172,6 +186,8 @@ Assembly options worth starting with: | `assemble_device` | `auto` or `gpu` | Uses GPU paste/weight accumulation when available. | | `base_canvas_mode` | `black` when you do not want source pixels underneath | Prevents the original image from showing through uncovered or low-weight areas. | | `auto_composite_policy` | `safe_auto` | Keeps background/context low and promotes focus/detail tiles. | +| `auto_composite_policy` | `replace_tile` | For edit workflows, hard-replaces selected tile regions over the base canvas. Combine with `replace_tile_shape=mask_first` for mask priority, or `tile_box_first` for full tile-box replacement. | +| `replace_tile_shape` | `mask_first` or `tile_box_first` | Controls replace-tile shape priority. `mask_first` uses object masks when available and hardens soft mask interiors so only the edge blends; `tile_box_first` ignores masks and pastes the full tile box. | | `small_tile_on_top` | `true` for detail workflows | Helps small face/eye/text tiles win overlaps against larger body/context tiles. | | `mask_blend_mode` | `auto` or `mask_feather` | Uses object masks when present and feathered rectangular masks otherwise. | | `color_correction` | `off` first, then `reinhard_lab`, `mkl_lab`, or `histogram` as needed | Uses ComfyUI's official Transfer Color logic; reference defaults to the source image. | diff --git a/TTP_toolsets.py b/TTP_toolsets.py index 9355cbe..0b2ab31 100644 --- a/TTP_toolsets.py +++ b/TTP_toolsets.py @@ -454,7 +454,7 @@ def _ttp_normalize_tile_box(tile, image_width, image_height, defaults): normalized["object_mask"] = dict(tile["object_mask"]) if isinstance(tile.get("object_mask_source"), dict): normalized["object_mask_source"] = dict(tile["object_mask_source"]) - for key in ("caption", "prompt", "negative", "semantic_category", "semantic_role", "recommended_composite_mode"): + for key in ("caption", "prompt", "negative", "semantic_category", "semantic_role", "recommended_composite_mode", "composite_mode", "replace_tile_shape"): if key in tile: normalized[key] = str(tile.get(key, "")) for key in ( @@ -463,6 +463,7 @@ def _ttp_normalize_tile_box(tile, image_width, image_height, defaults): "recommended_layer", "recommended_priority", "recommended_occlusion_priority", + "object_mask_expand", ): if key in tile: normalized[key] = _ttp_safe_float(tile.get(key), 0.0) @@ -691,7 +692,20 @@ def _ttp_mask_tensor_to_pil_list(masks, image_width, image_height): return result -def _ttp_encode_object_mask_data(mask_images, mask_indices, box): +def _ttp_expand_l_mask(mask, expand_px): + expand_px = max(0, min(2048, int(round(float(expand_px or 0))))) + if expand_px <= 0: + return mask.convert("L") + expanded = mask.convert("L") + remaining = expand_px + while remaining > 0: + step = min(remaining, 127) + expanded = expanded.filter(ImageFilter.MaxFilter(step * 2 + 1)) + remaining -= step + return expanded + + +def _ttp_encode_object_mask_data(mask_images, mask_indices, box, mask_expand=0): if not mask_images or not mask_indices: return None x0, y0, x1, y1 = [int(round(value)) for value in box] @@ -705,6 +719,9 @@ def _ttp_encode_object_mask_data(mask_images, mask_indices, box): if cropped.getbbox() is None: continue combined = ImageChops.lighter(combined, cropped) + if combined.getbbox() is None: + return None + combined = _ttp_expand_l_mask(combined, mask_expand) if combined.getbbox() is None: return None buffer = BytesIO() @@ -728,12 +745,36 @@ def _ttp_decode_object_mask_data(mask_data): return None -def _ttp_tile_object_mask_array(tile, out_width, out_height, mask_blend_mode="mask_feather", feather=0): +def _ttp_harden_stitch_mask_array(mask_array, feather=0, threshold=0.05): + mask_2d = np.clip(np.asarray(mask_array, dtype=np.float32), 0.0, 1.0) + hard = (mask_2d > float(threshold)).astype(np.float32) + if float(hard.max()) <= 0.0: + return hard[:, :, None] + + feather = max(0, int(round(float(feather)))) + if feather <= 0: + return hard[:, :, None] + + hard_pil = Image.fromarray((hard * 255.0).astype(np.uint8), mode="L") + expand_px = max(1, min(64, int(round(float(feather) * 0.25)))) + hard_pil = hard_pil.filter(ImageFilter.MaxFilter(expand_px * 2 + 1)) + blur_radius = max(0.75, min(64.0, float(feather) * 0.25)) + softened = hard_pil.filter(ImageFilter.GaussianBlur(radius=blur_radius)) + softened_array = np.array(softened).astype(np.float32) / 255.0 + return np.maximum(hard, softened_array)[:, :, None] + + +def _ttp_tile_object_mask_array(tile, out_width, out_height, mask_blend_mode="mask_feather", feather=0, hard_stitch=False, prefer_own_mask=False): if str(mask_blend_mode) == "off": return None - mask_data = tile.get("object_mask_source") - if not isinstance(mask_data, dict) or mask_data.get("format") != "png_base64": + if bool(prefer_own_mask): mask_data = tile.get("object_mask") + if not isinstance(mask_data, dict) or mask_data.get("format") != "png_base64": + mask_data = tile.get("object_mask_source") + else: + mask_data = tile.get("object_mask_source") + if not isinstance(mask_data, dict) or mask_data.get("format") != "png_base64": + mask_data = tile.get("object_mask") mask_pil = _ttp_decode_object_mask_data(mask_data) if mask_pil is None: return None @@ -757,6 +798,10 @@ def _ttp_tile_object_mask_array(tile, out_width, out_height, mask_blend_mode="ma (paste_x + clip_left, paste_y + clip_top), ) source_mask = source_mask.resize((out_width, out_height), Image.Resampling.BILINEAR) + if hard_stitch: + source_array = np.array(source_mask).astype(np.float32) / 255.0 + stitch_feather = feather if str(mask_blend_mode) in ("auto", "mask_feather") else 0 + return _ttp_harden_stitch_mask_array(source_array, stitch_feather) if str(mask_blend_mode) in ("auto", "mask_feather"): min_radius = min(12.0, max(0.75, min(int(out_width), int(out_height)) * 0.006)) radius = max(min_radius, float(feather) * 0.35) @@ -820,6 +865,7 @@ def _ttp_tile_semantic_text(tile): "prompt", "prompt_tag", "prompt_source", + "composite_mode", "object_id", "semantic_category", "semantic_role", @@ -999,12 +1045,39 @@ def _ttp_weighted_percentile(values, weights=None, percentile=0.7): return float(value_array[index]) +def _ttp_tile_composite_mode(tile): + return str(tile.get("composite_mode", tile.get("recommended_composite_mode", "")) or "").strip().lower() + + +def _ttp_is_replace_tile_mode(mode): + return str(mode or "").strip().lower() in ("replace_tile", "hard_replace", "hard_replace_tile") + + +def _ttp_normalize_replace_tile_shape(value, default="mask_first"): + default = str(default or "mask_first").strip().lower() + if default not in ("mask_first", "tile_box_first"): + default = "mask_first" + mode = str(value or default).strip().lower() + if mode in ("tile_box_first", "box_first", "rectangle_first", "rect_first", "square_first", "tile_box", "rectangle", "rect", "box", "square"): + return "tile_box_first" + if mode in ("mask_first", "mask", "object_mask", "mask_priority"): + return "mask_first" + return default + + +def _ttp_tile_replace_shape(tile, default="mask_first"): + mode = str(tile.get("replace_tile_shape", "") or "").strip().lower() + if not mode or mode == "keep": + return _ttp_normalize_replace_tile_shape(default) + return _ttp_normalize_replace_tile_shape(mode, default) + + def _ttp_should_soft_overlay_tile(tile, policy, is_large_tile, tile_area_ratio, large_tile_area_threshold): policy = str(policy or "safe_auto") - if policy in ("strict_layer", "replace_object") or is_large_tile: + tile_mode = _ttp_tile_composite_mode(tile) + if policy in ("strict_layer", "replace_object", "replace_tile") or _ttp_is_replace_tile_mode(tile_mode) or is_large_tile: return False - recommended_mode = str(tile.get("recommended_composite_mode", "") or "").lower() - if recommended_mode == "soft_overlay": + if tile_mode == "soft_overlay": return True text = _ttp_tile_semantic_text(tile) if _ttp_is_context_tile_text(text): @@ -1558,6 +1631,7 @@ def _ttp_boxes_to_auto_layout( default_pad=128, default_blend=64, object_padding=96, + mask_expand=16, max_tiles=16, include_background=True, allow_object_overlap=True, @@ -1636,9 +1710,11 @@ def _ttp_boxes_to_auto_layout( "recommended_occlusion_priority": 0.0, "recommended_composite_mode": "context", }) - object_mask = _ttp_encode_object_mask_data(mask_images, item.get("mask_indices", []), box) + object_mask = _ttp_encode_object_mask_data(mask_images, item.get("mask_indices", []), box, mask_expand=mask_expand) if object_mask is not None: tile["object_mask"] = object_mask + if int(mask_expand) > 0: + tile["object_mask_expand"] = int(mask_expand) tiles.append(tile) if len(tiles) == (1 if include_background else 0): @@ -2095,6 +2171,8 @@ def _ttp_tile_set_fingerprint(tile_set): "recommended_priority": tile.get("recommended_priority", ""), "recommended_occlusion_priority": tile.get("recommended_occlusion_priority", ""), "recommended_composite_mode": tile.get("recommended_composite_mode", ""), + "composite_mode": tile.get("composite_mode", ""), + "replace_tile_shape": tile.get("replace_tile_shape", ""), "object_mask": _ttp_mask_data_signature(tile.get("object_mask")), "object_mask_source": _ttp_mask_data_signature(tile.get("object_mask_source")), "prompt": tile.get("prompt", ""), @@ -2970,6 +3048,7 @@ def _ttp_run_sam3_auto_layout( default_pad, default_blend, object_padding, + mask_expand, max_tiles, allow_object_overlap, paint_mask_payload=None, @@ -3012,6 +3091,7 @@ def _ttp_run_sam3_auto_layout( default_pad=int(default_pad), default_blend=int(default_blend), object_padding=int(object_padding), + mask_expand=int(mask_expand), max_tiles=int(max_tiles), include_background=False, allow_object_overlap=bool(allow_object_overlap), @@ -3147,6 +3227,7 @@ def _ttp_run_qwenvl3_auto_layout( default_pad, default_blend, object_padding, + mask_expand, max_tiles, allow_object_overlap, paint_mask_payload=None, @@ -3184,6 +3265,7 @@ def _ttp_run_qwenvl3_auto_layout( default_pad=int(default_pad), default_blend=int(default_blend), object_padding=int(object_padding), + mask_expand=int(mask_expand), max_tiles=int(max_tiles), include_background=False, allow_object_overlap=bool(allow_object_overlap), @@ -3200,6 +3282,7 @@ def _ttp_run_paint_mask_auto_layout( default_pad, default_blend, object_padding, + mask_expand, max_tiles, allow_object_overlap, ): @@ -3215,6 +3298,7 @@ def _ttp_run_paint_mask_auto_layout( default_pad=int(default_pad), default_blend=int(default_blend), object_padding=int(object_padding), + mask_expand=int(mask_expand), max_tiles=int(max_tiles), include_background=False, allow_object_overlap=bool(allow_object_overlap), @@ -3490,6 +3574,7 @@ class TTP_Smart_Tile_Interactive_Crop_Experimental: }), "allow_object_overlap": ("BOOLEAN", {"default": True}), "auto_object_padding": ("INT", {"default": 96, "min": 0, "max": 2048, "step": 8}), + "auto_mask_expand": ("INT", {"default": 16, "min": 0, "max": 2048, "step": 4}), "auto_max_tiles": ("INT", {"default": 16, "min": 1, "max": 64, "step": 1}), "auto_paint_mask": ("STRING", {"default": "", "multiline": False}), }, @@ -3526,6 +3611,7 @@ class TTP_Smart_Tile_Interactive_Crop_Experimental: "auto_prompt": kwargs.get("auto_prompt", ""), "allow_object_overlap": kwargs.get("allow_object_overlap", True), "auto_object_padding": kwargs.get("auto_object_padding", 96), + "auto_mask_expand": kwargs.get("auto_mask_expand", 16), "auto_max_tiles": kwargs.get("auto_max_tiles", 16), "auto_paint_mask": hashlib.sha256(str(kwargs.get("auto_paint_mask", "") or "").encode("utf-8")).hexdigest()[:16], "source_image": type(kwargs.get("source_image")).__name__ if kwargs.get("source_image") is not None else None, @@ -3560,6 +3646,7 @@ class TTP_Smart_Tile_Interactive_Crop_Experimental: auto_prompt="person, face, hands, eyes, text, foreground object, important object", allow_object_overlap=True, auto_object_padding=96, + auto_mask_expand=16, auto_max_tiles=16, auto_paint_mask="", source_image=None, @@ -3585,6 +3672,7 @@ class TTP_Smart_Tile_Interactive_Crop_Experimental: int(default_pad), int(default_blend), int(auto_object_padding), + int(auto_mask_expand), int(auto_max_tiles), bool(allow_object_overlap), auto_paint_mask, @@ -3598,6 +3686,7 @@ class TTP_Smart_Tile_Interactive_Crop_Experimental: int(default_pad), int(default_blend), int(auto_object_padding), + int(auto_mask_expand), int(auto_max_tiles), bool(allow_object_overlap), auto_paint_mask, @@ -3610,6 +3699,7 @@ class TTP_Smart_Tile_Interactive_Crop_Experimental: int(default_pad), int(default_blend), int(auto_object_padding), + int(auto_mask_expand), int(auto_max_tiles), bool(allow_object_overlap), ) @@ -4087,6 +4177,564 @@ class TTP_Smart_Tile_QwenVL_Prompt_Set_Builder_Experimental: return (next_tile_set, prompt_set_json, summary) +class TTP_Smart_Tile_Prompt_Override_Experimental: + @classmethod + def INPUT_TYPES(cls): + text_modes = ["keep", "replace", "prepend", "append", "find_replace", "clear"] + return { + "required": { + "tile_set": ("TTP_SMART_TILE_SET", {"forceInput": True}), + "selector_type": ([ + "all", + "index", + "name", + "label", + "prompt_tag", + "source", + "semantic_category", + "semantic_role", + "prompt_contains", + "regex", + ], {"default": "label"}), + "selector": ("STRING", {"default": "", "multiline": False}), + "unmatched_mode": (["keep", "drop_unmatched"], {"default": "keep"}), + "prompt_mode": (text_modes, {"default": "replace"}), + "prompt_text": ("STRING", {"default": "", "multiline": True, "dynamicPrompts": False}), + "negative_mode": (text_modes, {"default": "keep"}), + "negative_text": ("STRING", {"default": "", "multiline": True, "dynamicPrompts": False}), + "find_text": ("STRING", {"default": "", "multiline": False, "dynamicPrompts": False}), + } + } + + RETURN_TYPES = ("TTP_SMART_TILE_SET", "STRING", "STRING") + RETURN_NAMES = ("tile_set", "override_json", "summary") + FUNCTION = "override_prompts" + CATEGORY = "TTP/Smart Tile" + + @staticmethod + def _selector_terms(selector): + return [ + part.strip() + for part in re.split(r"[,;，、\n\r]+", str(selector or "")) + if part.strip() + ] + + @staticmethod + def _selector_index_order(selector, count): + indexes = [] + seen = set() + for term in TTP_Smart_Tile_Prompt_Override_Experimental._selector_terms(selector): + cleaned = term.strip() + t_range = re.match(r"^[tT]\s*(\d+)\s*-\s*[tT]?\s*(\d+)$", cleaned) + zero_range = re.match(r"^(\d+)\s*-\s*(\d+)$", cleaned) + t_single = re.match(r"^[tT]\s*(\d+)$", cleaned) + zero_single = re.match(r"^\d+$", cleaned) + if t_range: + start = int(t_range.group(1)) - 1 + end = int(t_range.group(2)) - 1 + elif zero_range: + start = int(zero_range.group(1)) + end = int(zero_range.group(2)) + elif t_single: + start = end = int(t_single.group(1)) - 1 + elif zero_single: + start = end = int(cleaned) + else: + continue + if end < start: + start, end = end, start + for index in range(start, end + 1): + if 0 <= index < count and index not in seen: + indexes.append(index) + seen.add(index) + return indexes + + @staticmethod + def _selector_index_set(selector, count): + indexes = set() + for index in TTP_Smart_Tile_Prompt_Override_Experimental._selector_index_order(selector, count): + indexes.add(index) + return indexes + + @staticmethod + def _match_tile(tile, index, count, selector_type, selector): + selector_type = str(selector_type or "label") + if selector_type == "all": + return True + if selector_type == "index": + return index in TTP_Smart_Tile_Prompt_Override_Experimental._selector_index_set(selector, count) + + terms = [term.lower() for term in TTP_Smart_Tile_Prompt_Override_Experimental._selector_terms(selector)] + if not terms: + return False + + fields = { + "name": [tile.get("name", "")], + "label": [tile.get("label", "")], + "prompt_tag": [tile.get("prompt_tag", "")], + "source": [tile.get("source", "")], + "semantic_category": [tile.get("semantic_category", "")], + "semantic_role": [tile.get("semantic_role", "")], + "prompt_contains": [tile.get("prompt", ""), tile.get("negative", ""), tile.get("caption", "")], + } + searchable = fields.get(selector_type) + if selector_type == "regex": + searchable = [ + str(index), + str(index + 1), + tile.get("name", ""), + tile.get("label", ""), + tile.get("prompt_tag", ""), + tile.get("source", ""), + tile.get("semantic_category", ""), + tile.get("semantic_role", ""), + tile.get("caption", ""), + tile.get("prompt", ""), + tile.get("negative", ""), + ] + text = "\n".join(str(value or "") for value in searchable) + try: + return bool(re.search(str(selector or ""), text, flags=re.IGNORECASE)) + except re.error as exc: + raise ValueError(f"Invalid Smart Tile prompt override regex: {exc}") from exc + if searchable is None: + searchable = [ + tile.get("name", ""), + tile.get("label", ""), + tile.get("prompt_tag", ""), + tile.get("source", ""), + tile.get("caption", ""), + tile.get("prompt", ""), + ] + text = "\n".join(str(value or "").lower() for value in searchable) + return any(term in text for term in terms) + + @staticmethod + def _join_text(prefix, suffix): + prefix = str(prefix or "").strip() + suffix = str(suffix or "").strip() + if prefix and suffix: + return f"{prefix}\n{suffix}" + return prefix or suffix + + @staticmethod + def _apply_text_mode(current, mode, text, find_text): + current = str(current or "") + mode = str(mode or "keep") + text = str(text or "").strip() + find_text = str(find_text or "") + if mode == "keep": + return current + if mode == "clear": + return "" + if mode == "replace": + return text + if mode == "prepend": + return TTP_Smart_Tile_Prompt_Override_Experimental._join_text(text, current) + if mode == "append": + return TTP_Smart_Tile_Prompt_Override_Experimental._join_text(current, text) + if mode == "find_replace": + return current.replace(find_text, text) if find_text else current + return current + + @staticmethod + def _position_for_tile(tile): + box = tile.get("paste_box", tile.get("core_box", [0, 0, 1, 1])) + x, y, w, h = [int(round(float(value))) for value in box[:4]] + return (x, y, x + max(1, w), y + max(1, h)) + + def override_prompts( + self, + tile_set, + selector_type="label", + selector="", + unmatched_mode="keep", + prompt_mode="replace", + prompt_text="", + negative_mode="keep", + negative_text="", + find_text="", + ): + tile_images, _tile_meta, tiles_info = _ttp_validate_tile_set(tile_set) + next_tile_set = _ttp_clone_tile_set(tile_set) + next_tiles = next_tile_set["tile_meta"]["tiles"] + next_images = list(next_tile_set.get("tile_images", [])) + next_positions = list(next_tile_set.get("positions", [])) + + kept_tiles = [] + kept_images = [] + kept_positions = [] + records = [] + matched_count = 0 + drop_unmatched = str(unmatched_mode or "keep") == "drop_unmatched" + count = len(tiles_info) + + for index, tile in enumerate(next_tiles): + matched = self._match_tile(tile, index, count, selector_type, selector) + if not matched and drop_unmatched: + continue + + updated = dict(tile) + if matched: + matched_count += 1 + before_prompt = str(updated.get("prompt", "")) + before_negative = str(updated.get("negative", "")) + updated["prompt"] = self._apply_text_mode(before_prompt, prompt_mode, prompt_text, find_text) + updated["negative"] = self._apply_text_mode(before_negative, negative_mode, negative_text, find_text) + existing_source = str(updated.get("prompt_source", "") or "") + updated["prompt_source"] = f"{existing_source}+override" if existing_source else "override" + updated["prompt_override"] = True + updated["prompt_override_selector_type"] = str(selector_type or "") + updated["prompt_override_selector"] = str(selector or "") + records.append({ + "index": int(index), + "name": updated.get("name", f"tile_{index}"), + "label": updated.get("label", ""), + "prompt_before": before_prompt, + "prompt_after": updated["prompt"], + "negative_before": before_negative, + "negative_after": updated["negative"], + }) + + kept_tiles.append(updated) + kept_images.append(next_images[index] if index < len(next_images) else tile_images[index]) + if index < len(next_positions): + kept_positions.append(next_positions[index]) + else: + kept_positions.append(self._position_for_tile(updated)) + + if not kept_tiles: + raise ValueError("Smart Tile prompt override kept no tiles. Change selector or unmatched_mode.") + + for new_index, tile in enumerate(kept_tiles): + tile["id"] = new_index + + next_tile_set["tile_meta"]["tiles"] = kept_tiles + next_tile_set["tile_images"] = kept_images + next_tile_set["positions"] = kept_positions + + override_json = json.dumps({ + "type": "ttp_smart_tile_prompt_override", + "selector_type": str(selector_type or ""), + "selector": str(selector or ""), + "unmatched_mode": str(unmatched_mode or "keep"), + "prompt_mode": str(prompt_mode or "keep"), + "negative_mode": str(negative_mode or "keep"), + "matched": int(matched_count), + "kept": int(len(kept_tiles)), + "dropped": int(count - len(kept_tiles)), + "tiles": records, + }, ensure_ascii=False, indent=2) + summary_lines = [ + f"matched={matched_count} kept={len(kept_tiles)} dropped={count - len(kept_tiles)} selector={selector_type}:{selector}" + ] + summary_lines.extend( + f'{record["index"]}:{record["name"]} -> {_ttp_compact_text(record["prompt_after"], 160)}' + for record in records + ) + return (next_tile_set, override_json, "\n".join(summary_lines)) + + +class TTP_Smart_Tile_Composite_Override_Experimental: + @classmethod + def INPUT_TYPES(cls): + number_modes = ["keep", "set", "add", "max", "min"] + return { + "required": { + "tile_set": ("TTP_SMART_TILE_SET", {"forceInput": True}), + "selector_type": ([ + "all", + "index", + "name", + "label", + "prompt_tag", + "source", + "semantic_category", + "semantic_role", + "prompt_contains", + "regex", + ], {"default": "index"}), + "selector": ("STRING", {"default": "T1", "multiline": False}), + "unmatched_mode": (["keep", "drop_unmatched"], {"default": "keep"}), + "composite_mode": (["keep", "replace_tile", "soft_overlay", "blend", "replace", "context"], {"default": "replace_tile"}), + "replace_tile_shape": (["keep", "mask_first", "tile_box_first"], {"default": "keep"}), + "layer_mode": (number_modes, {"default": "max"}), + "layer": ("FLOAT", {"default": 10.0, "min": -1000000.0, "max": 1000000.0, "step": 1.0}), + "priority_mode": (number_modes, {"default": "max"}), + "priority": ("FLOAT", {"default": 200.0, "min": -1000000.0, "max": 1000000.0, "step": 1.0}), + "occlusion_priority_mode": (number_modes, {"default": "max"}), + "occlusion_priority": ("FLOAT", {"default": 50000.0, "min": -1000000.0, "max": 1000000.0, "step": 10.0}), + "blend_mode": (number_modes, {"default": "keep"}), + "blend": ("INT", {"default": 32, "min": 0, "max": 2048, "step": 1}), + "importance_mode": (number_modes, {"default": "keep"}), + "importance": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 10.0, "step": 0.05}), + } + } + + RETURN_TYPES = ("TTP_SMART_TILE_SET", "STRING", "STRING") + RETURN_NAMES = ("tile_set", "composite_json", "summary") + FUNCTION = "override_composite" + CATEGORY = "TTP/Smart Tile" + + @staticmethod + def _apply_number_mode(current, mode, value): + mode = str(mode or "keep") + current = _ttp_safe_float(current, 0.0) + value = _ttp_safe_float(value, 0.0) + if mode == "keep": + return current + if mode == "set": + return value + if mode == "add": + return current + value + if mode == "max": + return max(current, value) + if mode == "min": + return min(current, value) + return current + + @staticmethod + def _position_for_tile(tile): + return TTP_Smart_Tile_Prompt_Override_Experimental._position_for_tile(tile) + + def override_composite( + self, + tile_set, + selector_type="index", + selector="T1", + unmatched_mode="keep", + composite_mode="replace_tile", + replace_tile_shape="keep", + layer_mode="max", + layer=10.0, + priority_mode="max", + priority=200.0, + occlusion_priority_mode="max", + occlusion_priority=50000.0, + blend_mode="keep", + blend=32, + importance_mode="keep", + importance=1.0, + ): + tile_images, _tile_meta, tiles_info = _ttp_validate_tile_set(tile_set) + next_tile_set = _ttp_clone_tile_set(tile_set) + next_tiles = next_tile_set["tile_meta"]["tiles"] + next_images = list(next_tile_set.get("tile_images", [])) + next_positions = list(next_tile_set.get("positions", [])) + + kept_tiles = [] + kept_images = [] + kept_positions = [] + records = [] + matched_count = 0 + drop_unmatched = str(unmatched_mode or "keep") == "drop_unmatched" + count = len(tiles_info) + composite_mode = str(composite_mode or "keep") + replace_tile_shape = str(replace_tile_shape or "keep") + + for index, tile in enumerate(next_tiles): + matched = TTP_Smart_Tile_Prompt_Override_Experimental._match_tile(tile, index, count, selector_type, selector) + if not matched and drop_unmatched: + continue + + updated = dict(tile) + if matched: + matched_count += 1 + before = { + "composite_mode": updated.get("composite_mode", updated.get("recommended_composite_mode", "")), + "replace_tile_shape": updated.get("replace_tile_shape", ""), + "layer": updated.get("layer", 0.0), + "priority": updated.get("priority", 0.0), + "occlusion_priority": updated.get("occlusion_priority", 0.0), + "blend": updated.get("blend", 0), + "importance": updated.get("importance", 1.0), + } + if composite_mode != "keep": + updated["composite_mode"] = composite_mode + updated["recommended_composite_mode"] = composite_mode + if replace_tile_shape != "keep": + updated["replace_tile_shape"] = _ttp_normalize_replace_tile_shape(replace_tile_shape) + updated["layer"] = self._apply_number_mode(updated.get("layer", 0.0), layer_mode, layer) + updated["priority"] = self._apply_number_mode(updated.get("priority", 0.0), priority_mode, priority) + updated["occlusion_priority"] = self._apply_number_mode(updated.get("occlusion_priority", 0.0), occlusion_priority_mode, occlusion_priority) + updated["blend"] = max(0, int(round(self._apply_number_mode(updated.get("blend", 0), blend_mode, blend)))) + updated["importance"] = max(0.0, self._apply_number_mode(updated.get("importance", 1.0), importance_mode, importance)) + updated["composite_override"] = True + updated["composite_override_selector_type"] = str(selector_type or "") + updated["composite_override_selector"] = str(selector or "") + records.append({ + "index": int(index), + "name": updated.get("name", f"tile_{index}"), + "label": updated.get("label", ""), + "before": before, + "after": { + "composite_mode": updated.get("composite_mode", ""), + "replace_tile_shape": updated.get("replace_tile_shape", ""), + "layer": updated.get("layer", 0.0), + "priority": updated.get("priority", 0.0), + "occlusion_priority": updated.get("occlusion_priority", 0.0), + "blend": updated.get("blend", 0), + "importance": updated.get("importance", 1.0), + }, + }) + + kept_tiles.append(updated) + kept_images.append(next_images[index] if index < len(next_images) else tile_images[index]) + if index < len(next_positions): + kept_positions.append(next_positions[index]) + else: + kept_positions.append(self._position_for_tile(updated)) + + if not kept_tiles: + raise ValueError("Smart Tile composite override kept no tiles. Change selector or unmatched_mode.") + + for new_index, tile in enumerate(kept_tiles): + tile["id"] = new_index + + next_tile_set["tile_meta"]["tiles"] = kept_tiles + next_tile_set["tile_images"] = kept_images + next_tile_set["positions"] = kept_positions + + composite_json = json.dumps({ + "type": "ttp_smart_tile_composite_override", + "selector_type": str(selector_type or ""), + "selector": str(selector or ""), + "unmatched_mode": str(unmatched_mode or "keep"), + "matched": int(matched_count), + "kept": int(len(kept_tiles)), + "dropped": int(count - len(kept_tiles)), + "tiles": records, + }, ensure_ascii=False, indent=2) + summary_lines = [ + f"matched={matched_count} kept={len(kept_tiles)} dropped={count - len(kept_tiles)} selector={selector_type}:{selector}" + ] + summary_lines.extend( + f'{record["index"]}:{record["name"]} mode={record["after"]["composite_mode"]} shape={record["after"]["replace_tile_shape"] or "default"} layer={record["after"]["layer"]} occ={record["after"]["occlusion_priority"]}' + for record in records + ) + return (next_tile_set, composite_json, "\n".join(summary_lines)) + + +class TTP_Smart_Tile_Stack_Order_Experimental: + @classmethod + def INPUT_TYPES(cls): + return { + "required": { + "tile_set": ("TTP_SMART_TILE_SET", {"forceInput": True}), + "tile_order": ("STRING", {"default": "T1,T3,T2", "multiline": False}), + "top_order": (["first_on_top", "last_on_top"], {"default": "first_on_top"}), + "composite_mode": (["replace_tile", "keep"], {"default": "replace_tile"}), + "replace_tile_shape": (["keep", "mask_first", "tile_box_first"], {"default": "keep"}), + } + } + + RETURN_TYPES = ("TTP_SMART_TILE_SET", "STRING", "STRING") + RETURN_NAMES = ("tile_set", "order_json", "summary") + FUNCTION = "apply_stack_order" + CATEGORY = "TTP/Smart Tile" + + @staticmethod + def _position_for_tile(tile): + return TTP_Smart_Tile_Prompt_Override_Experimental._position_for_tile(tile) + + def apply_stack_order( + self, + tile_set, + tile_order="T1,T3,T2", + top_order="first_on_top", + composite_mode="replace_tile", + replace_tile_shape="keep", + ): + tile_images, _tile_meta, tiles_info = _ttp_validate_tile_set(tile_set) + count = len(tiles_info) + ordered_indexes = TTP_Smart_Tile_Prompt_Override_Experimental._selector_index_order(tile_order, count) + if not ordered_indexes: + raise ValueError("Smart Tile stack order matched no tiles. Use indexes like T1,T3,T2.") + + next_tile_set = _ttp_clone_tile_set(tile_set) + next_tiles = next_tile_set["tile_meta"]["tiles"] + next_images = list(next_tile_set.get("tile_images", [])) + next_positions = list(next_tile_set.get("positions", [])) + + max_layer = max((_ttp_safe_float(tile.get("layer", 0.0), 0.0) for tile in next_tiles), default=0.0) + max_occlusion = max((_ttp_safe_float(tile.get("occlusion_priority", 0.0), 0.0) for tile in next_tiles), default=0.0) + step = 1000.0 + first_on_top = str(top_order or "first_on_top") == "first_on_top" + composite_mode = str(composite_mode or "replace_tile") + replace_tile_shape = str(replace_tile_shape or "keep") + order_rank = {} + order_position = {} + ordered_count = len(ordered_indexes) + for rank, index in enumerate(ordered_indexes): + stack_rank = ordered_count - rank if first_on_top else rank + 1 + order_rank[index] = stack_rank + order_position[index] = rank + + records = [] + for index, tile in enumerate(next_tiles): + if index not in order_rank: + continue + stack_rank = order_rank[index] + before = { + "layer": tile.get("layer", 0.0), + "occlusion_priority": tile.get("occlusion_priority", 0.0), + "composite_mode": tile.get("composite_mode", tile.get("recommended_composite_mode", "")), + "replace_tile_shape": tile.get("replace_tile_shape", ""), + } + tile["layer"] = max_layer + float(stack_rank) + tile["occlusion_priority"] = max_occlusion + float(stack_rank) * step + if composite_mode != "keep": + tile["composite_mode"] = composite_mode + tile["recommended_composite_mode"] = composite_mode + if replace_tile_shape != "keep": + tile["replace_tile_shape"] = _ttp_normalize_replace_tile_shape(replace_tile_shape) + tile["stack_order_override"] = True + tile["stack_order_selector"] = str(tile_order or "") + tile["stack_order_position"] = int(ordered_indexes.index(index) + 1) + tile["stack_order_top_rank"] = float(stack_rank) + records.append({ + "index": int(index), + "selector": f"T{index + 1}", + "name": tile.get("name", f"tile_{index}"), + "before": before, + "after": { + "layer": tile.get("layer", 0.0), + "occlusion_priority": tile.get("occlusion_priority", 0.0), + "composite_mode": tile.get("composite_mode", tile.get("recommended_composite_mode", "")), + "replace_tile_shape": tile.get("replace_tile_shape", ""), + "top_rank": float(stack_rank), + }, + }) + records.sort(key=lambda record: order_position.get(int(record["index"]), count)) + + if len(next_positions) < len(next_tiles): + next_tile_set["positions"] = [ + next_positions[index] if index < len(next_positions) else self._position_for_tile(tile) + for index, tile in enumerate(next_tiles) + ] + else: + next_tile_set["positions"] = next_positions + next_tile_set["tile_images"] = next_images if next_images else list(tile_images) + + order_json = json.dumps({ + "type": "ttp_smart_tile_stack_order", + "tile_order": str(tile_order or ""), + "top_order": str(top_order or "first_on_top"), + "composite_mode": composite_mode, + "replace_tile_shape": replace_tile_shape, + "matched": int(len(records)), + "tiles": records, + }, ensure_ascii=False, indent=2) + summary_lines = [ + f"matched={len(records)} order={tile_order} top_order={top_order} mode={composite_mode}" + ] + summary_lines.extend( + f'{record["selector"]}:{record["name"]} layer={record["after"]["layer"]} occ={record["after"]["occlusion_priority"]}' + for record in records + ) + return (next_tile_set, order_json, "\n".join(summary_lines)) + + class TTP_Smart_Tile_Loop_Source_Experimental: @classmethod def INPUT_TYPES(cls): @@ -4579,7 +5227,8 @@ class TTP_Smart_Tile_Assemble_Experimental: "base_canvas_mode": (["auto", "black", "base_image", "source_image"], {"default": "auto"}), "weight_preview_mode": (["raw_weight", "coverage"], {"default": "raw_weight"}), "small_tile_on_top": ("BOOLEAN", {"default": False}), - "auto_composite_policy": (["safe_auto", "strict_layer", "soft_detail", "replace_object"], {"default": "safe_auto"}), + "auto_composite_policy": (["safe_auto", "strict_layer", "soft_detail", "replace_object", "replace_tile"], {"default": "safe_auto"}), + "replace_tile_shape": (["mask_first", "tile_box_first"], {"default": "mask_first"}), }, "optional": { "sampled_tiles": ("IMAGE",), @@ -4620,6 +5269,7 @@ class TTP_Smart_Tile_Assemble_Experimental: weight_preview_mode="raw_weight", small_tile_on_top=False, auto_composite_policy="safe_auto", + replace_tile_shape="mask_first", sampled_tiles=None, tile_meta=None, tile_set=None, @@ -4737,6 +5387,7 @@ class TTP_Smart_Tile_Assemble_Experimental: "weight_preview_mode": preview_mode, "small_tile_on_top": _ttp_safe_bool(small_tile_on_top, False), "auto_composite_policy": str(auto_composite_policy or "safe_auto"), + "replace_tile_shape": str(replace_tile_shape or "mask_first"), "mask_blend_mode": str(mask_blend_mode), "large_tile_policy": str(large_tile_policy), }, @@ -4767,10 +5418,15 @@ class TTP_Smart_Tile_Assemble_Experimental: small_tile_on_top = _ttp_safe_bool(small_tile_on_top, False) composite_policy = str(auto_composite_policy or "safe_auto") - if composite_policy not in ("safe_auto", "strict_layer", "soft_detail", "replace_object"): + if composite_policy not in ("safe_auto", "strict_layer", "soft_detail", "replace_object", "replace_tile"): composite_policy = "safe_auto" - auto_policy_ranking = composite_policy in ("safe_auto", "soft_detail", "replace_object") - destructive_occlusion = composite_policy in ("strict_layer", "replace_object") + replace_tile_shape_default = _ttp_normalize_replace_tile_shape(replace_tile_shape) + auto_policy_ranking = composite_policy in ("safe_auto", "soft_detail", "replace_object", "replace_tile") + destructive_occlusion = composite_policy in ("strict_layer", "replace_object", "replace_tile") + replace_tile_policy_active = composite_policy == "replace_tile" or any( + _ttp_is_replace_tile_mode(_ttp_tile_composite_mode(tile)) + for tile in tiles_info + ) auto_policy_candidates = auto_policy_ranking and any( not _ttp_is_context_tile_text(_ttp_tile_search_text(tile)) and ( _ttp_is_detail_tile_text(_ttp_tile_search_text(tile)) @@ -4780,7 +5436,7 @@ class TTP_Smart_Tile_Assemble_Experimental: ) for tile in tiles_info ) - use_occlusion = small_tile_on_top or auto_policy_candidates or any( + use_occlusion = replace_tile_policy_active or small_tile_on_top or auto_policy_candidates or any( float(tile.get("occlusion_priority", 0.0)) != 0.0 or int(tile.get("layer", 0)) != 0 for tile in tiles_info ) @@ -4856,6 +5512,9 @@ class TTP_Smart_Tile_Assemble_Experimental: tile_area_ratio, large_tile_area_threshold, ) + tile_mode = _ttp_tile_composite_mode(tile) + replace_tile_mode = composite_policy == "replace_tile" or _ttp_is_replace_tile_mode(tile_mode) + tile_replace_shape = _ttp_tile_replace_shape(tile, replace_tile_shape_default) low_resolution_large_tile = ( has_base_canvas and is_large_tile @@ -4909,8 +5568,24 @@ class TTP_Smart_Tile_Assemble_Experimental: 1.0, 1.0, ) - object_mask = _ttp_tile_object_mask_array(tile, out_w, out_h, str(mask_blend_mode), blend) - if object_mask is not None: + mask_mode = "mask_feather" if replace_tile_mode else str(mask_blend_mode) + object_mask = _ttp_tile_object_mask_array( + tile, + out_w, + out_h, + mask_mode, + blend, + hard_stitch=replace_tile_mode and tile_replace_shape != "tile_box_first", + prefer_own_mask=replace_tile_mode, + ) + if replace_tile_mode: + if tile_replace_shape == "tile_box_first": + mask = _ttp_rect_feather_mask(out_w, out_h, blend) + elif object_mask is not None: + mask = object_mask + else: + mask = _ttp_rect_feather_mask(out_w, out_h, blend) + elif object_mask is not None: if soft_detail_overlay and str(mask_blend_mode) in ("auto", "mask_feather", "mask_only"): blur_px = max(1.0, min(8.0, max(float(blend) * 0.08, min(out_w, out_h) * 0.012))) object_halo = _ttp_soften_detail_mask_array( @@ -4928,7 +5603,7 @@ class TTP_Smart_Tile_Assemble_Experimental: mask = mask * object_mask elif _ttp_should_rect_feather_tile(tile, is_large_tile, str(mask_blend_mode)): mask = mask * _ttp_rect_feather_mask(out_w, out_h, blend) - tile_destructive_occlusion = destructive_occlusion or ( + tile_destructive_occlusion = replace_tile_mode or destructive_occlusion or ( composite_policy == "safe_auto" and object_mask is not None and not soft_detail_overlay @@ -4947,6 +5622,9 @@ class TTP_Smart_Tile_Assemble_Experimental: rank_occlusion = float(tile.get("occlusion_priority", 0.0)) rank_layer = float(tile.get("layer", 0.0)) rank_priority = priority + rank_tiebreaker = (float(index) + 1.0) * 0.001 if replace_tile_mode else 0.0 + if replace_tile_mode: + rank_occlusion = max(rank_occlusion, 1.0) if auto_policy_ranking and not is_large_tile: text = _ttp_tile_search_text(tile) if _ttp_is_detail_tile_text(text) or soft_detail_overlay: @@ -4961,7 +5639,9 @@ class TTP_Smart_Tile_Assemble_Experimental: rank_layer = max(rank_layer, 1.0) if soft_detail_overlay: mask = mask * (1.0 + coverage_mask * coverage_mask) - if is_large_tile and large_tile_policy == "context_only" and has_base_canvas: + if replace_tile_mode: + pass + elif is_large_tile and large_tile_policy == "context_only" and has_base_canvas: mask = mask * context_tile_weight rank_occlusion = min(rank_occlusion, 0.0) rank_layer = min(rank_layer, 0.0) @@ -5081,7 +5761,7 @@ class TTP_Smart_Tile_Assemble_Experimental: mask_t = torch.as_tensor(mask, dtype=torch.float32, device=assemble_torch_device) clear_t = torch.as_tensor(clear_mask, dtype=torch.float32, device=assemble_torch_device) if use_occlusion: - rank = rank_occlusion * 1000000.0 + rank_layer * 10000.0 + (rank_priority if use_priority else 0.0) + rank = rank_occlusion * 1000000.0 + rank_layer * 10000.0 + (rank_priority if use_priority else 0.0) + rank_tiebreaker rank_view = priority_ranks[paste_y:y_end, paste_x:x_end] mask_active = coverage_t > 1e-6 higher = (rank > rank_view + 1e-6) & mask_active @@ -5110,7 +5790,7 @@ class TTP_Smart_Tile_Assemble_Experimental: weights[paste_y:y_end, paste_x:x_end].add_(mask_t) preview_weights[paste_y:y_end, paste_x:x_end].add_(coverage_t) elif use_occlusion: - rank = rank_occlusion * 1000000.0 + rank_layer * 10000.0 + (rank_priority if use_priority else 0.0) + rank = rank_occlusion * 1000000.0 + rank_layer * 10000.0 + (rank_priority if use_priority else 0.0) + rank_tiebreaker rank_view = priority_ranks[paste_y:y_end, paste_x:x_end] mask_active = coverage_mask > 1e-6 higher = (rank > rank_view + 1e-6) & mask_active @@ -6092,6 +6772,9 @@ NODE_CLASS_MAPPINGS = { "TTP_Smart_Tile_Set_Preview_Experimental": TTP_Smart_Tile_Set_Preview_Experimental, "TTP_QwenVL3_Local_Loader_Experimental": TTP_QwenVL3_Local_Loader_Experimental, "TTP_Smart_Tile_QwenVL_Prompt_Set_Builder_Experimental": TTP_Smart_Tile_QwenVL_Prompt_Set_Builder_Experimental, + "TTP_Smart_Tile_Prompt_Override_Experimental": TTP_Smart_Tile_Prompt_Override_Experimental, + "TTP_Smart_Tile_Composite_Override_Experimental": TTP_Smart_Tile_Composite_Override_Experimental, + "TTP_Smart_Tile_Stack_Order_Experimental": TTP_Smart_Tile_Stack_Order_Experimental, "TTP_Smart_Tile_Loop_Source_Experimental": TTP_Smart_Tile_Loop_Source_Experimental, "TTP_Smart_Tile_Loop_Collect_Experimental": TTP_Smart_Tile_Loop_Collect_Experimental, "TTP_Smart_Tile_Image_Upscale_Prep_Experimental": TTP_Smart_Tile_Image_Upscale_Prep_Experimental, @@ -6117,6 +6800,9 @@ NODE_DISPLAY_NAME_MAPPINGS = { "TTP_Smart_Tile_Set_Preview_Experimental": "TTP Smart Tile Set Preview", "TTP_QwenVL3_Local_Loader_Experimental": "TTP QwenVL3 Local Loader", "TTP_Smart_Tile_QwenVL_Prompt_Set_Builder_Experimental": "TTP Smart Tile QwenVL Prompt Set Builder", + "TTP_Smart_Tile_Prompt_Override_Experimental": "TTP Smart Tile Prompt Override", + "TTP_Smart_Tile_Composite_Override_Experimental": "TTP Smart Tile Composite Override", + "TTP_Smart_Tile_Stack_Order_Experimental": "TTP Smart Tile Stack Order", "TTP_Smart_Tile_Loop_Source_Experimental": "TTP Smart Tile Loop Source", "TTP_Smart_Tile_Loop_Collect_Experimental": "TTP Smart Tile Loop Collect", "TTP_Smart_Tile_Image_Upscale_Prep_Experimental": "TTP Smart Tile Image Upscale Prep", diff --git a/scripts/smoke_smart_tile_grid_crop.py b/scripts/smoke_smart_tile_grid_crop.py index e6207f1..e92213a 100644 --- a/scripts/smoke_smart_tile_grid_crop.py +++ b/scripts/smoke_smart_tile_grid_crop.py @@ -153,6 +153,7 @@ assert_equal("auto_detect_request" in required_inputs, True, "interactive crop s assert_equal("auto_prompt" in required_inputs, True, "interactive crop should expose a visual prompt") assert_equal("allow_object_overlap" in required_inputs, True, "interactive crop should expose object overlap control") assert_equal("auto_object_padding" in required_inputs, True, "interactive crop should expose object padding control") +assert_equal("auto_mask_expand" in required_inputs, True, "interactive crop should expose auto mask expansion control") assert_equal("auto_max_tiles" in required_inputs, True, "interactive crop should expose auto max tiles control") assert_equal("auto_paint_mask" in required_inputs, True, "interactive crop should expose hidden painted mask input") assert_equal("vision_model" in optional_inputs, True, "interactive crop should expose a vision model input") @@ -184,6 +185,7 @@ expected_assemble_required_order = [ "weight_preview_mode", "small_tile_on_top", "auto_composite_policy", + "replace_tile_shape", ] assert_equal( list(assemble_inputs["required"].keys()), @@ -207,8 +209,10 @@ assert_equal(assemble_inputs["required"]["base_canvas_mode"][0], ["auto", "black assert_equal(assemble_inputs["required"]["weight_preview_mode"][0], ["raw_weight", "coverage"], "assemble should expose raw weight and coverage preview modes") assert_equal(assemble_inputs["required"]["weight_preview_mode"][1]["default"], "raw_weight", "assemble should keep raw overlap weight preview as the default") assert_equal("small_tile_on_top" in assemble_inputs["required"], True, "assemble should expose small tile top stacking") -assert_equal(assemble_inputs["required"]["auto_composite_policy"][0], ["safe_auto", "strict_layer", "soft_detail", "replace_object"], "assemble should expose automatic compositing policy") +assert_equal(assemble_inputs["required"]["auto_composite_policy"][0], ["safe_auto", "strict_layer", "soft_detail", "replace_object", "replace_tile"], "assemble should expose automatic compositing policy") assert_equal(assemble_inputs["required"]["auto_composite_policy"][1]["default"], "safe_auto", "assemble should default to safe auto compositing") +assert_equal(assemble_inputs["required"]["replace_tile_shape"][0], ["mask_first", "tile_box_first"], "assemble should expose replace tile shape priority") +assert_equal(assemble_inputs["required"]["replace_tile_shape"][1]["default"], "mask_first", "assemble should default replace tile to mask priority") assert_equal(assemble_inputs["optional"]["done"][0], "BOOLEAN", "assemble should accept loop done gate") preview_inputs = ttp.TTP_Smart_Tile_Set_Preview_Experimental.INPUT_TYPES() assert_equal(preview_inputs["required"]["tile_set"][0], "TTP_SMART_TILE_SET", "tile set preview should accept Smart Tile Set") @@ -257,6 +261,160 @@ assert_equal( ttp.TTP_Smart_Tile_QwenVL_Prompt_Set_Builder_Experimental, "prompt builder should be registered", ) +prompt_override_inputs = ttp.TTP_Smart_Tile_Prompt_Override_Experimental.INPUT_TYPES() +assert_equal(prompt_override_inputs["required"]["tile_set"][0], "TTP_SMART_TILE_SET", "prompt override should accept Smart Tile Set") +assert_equal(prompt_override_inputs["required"]["tile_set"][1].get("forceInput"), True, "prompt override tile_set should be a forceInput connection") +assert_equal("regex" in prompt_override_inputs["required"]["selector_type"][0], True, "prompt override should support regex selectors") +assert_equal(prompt_override_inputs["required"]["unmatched_mode"][0], ["keep", "drop_unmatched"], "prompt override should allow dropping unmatched tiles") +assert_equal(prompt_override_inputs["required"]["prompt_text"][1]["dynamicPrompts"], False, "prompt override text should not use dynamic prompt parsing") +assert_equal( + ttp.NODE_CLASS_MAPPINGS["TTP_Smart_Tile_Prompt_Override_Experimental"], + ttp.TTP_Smart_Tile_Prompt_Override_Experimental, + "prompt override should be registered", +) + +override_tile_set = { + "version": 1, + "type": "ttp_smart_tile_set", + "original_size": [100, 100], + "tile_meta": { + "version": 3, + "type": "ttp_smart_tile", + "storage": "tile_set", + "original_size": [100, 100], + "tiles": [ + { + "id": 0, + "name": "tile_1", + "label": "face", + "source": "paint_mask", + "core_box": [0, 0, 50, 50], + "sample_box": [0, 0, 50, 50], + "paste_box": [0, 0, 50, 50], + "prompt": "old face prompt", + "negative": "old face negative", + }, + { + "id": 1, + "name": "tile_2", + "label": "background", + "source": "manual", + "core_box": [50, 0, 50, 50], + "sample_box": [50, 0, 50, 50], + "paste_box": [50, 0, 50, 50], + "prompt": "old background prompt", + "negative": "old background negative", + }, + ], + }, + "tile_images": ["face_image", "background_image"], + "positions": [(0, 0, 50, 50), (50, 0, 100, 50)], +} +override_node = ttp.TTP_Smart_Tile_Prompt_Override_Experimental() +overridden_set, override_json, override_summary = override_node.override_prompts( + override_tile_set, + selector_type="label", + selector="face", + unmatched_mode="keep", + prompt_mode="replace", + prompt_text="edit only the face", + negative_mode="append", + negative_text="avoid changing identity", +) +assert_equal(overridden_set["tile_meta"]["tiles"][0]["prompt"], "edit only the face", "prompt override should replace matched tile prompt") +assert_equal(overridden_set["tile_meta"]["tiles"][0]["negative"], "old face negative\navoid changing identity", "prompt override should append matched tile negative") +assert_equal(overridden_set["tile_meta"]["tiles"][1]["prompt"], "old background prompt", "prompt override should keep unmatched tile prompt") +assert_equal(json.loads(override_json)["matched"], 1, "prompt override JSON should report matched tile count") +assert_equal("matched=1" in override_summary, True, "prompt override summary should report matched count") +dropped_set, _dropped_json, _dropped_summary = override_node.override_prompts( + override_tile_set, + selector_type="label", + selector="face", + unmatched_mode="drop_unmatched", + prompt_mode="replace", + prompt_text="edit only the face", +) +assert_equal(len(dropped_set["tile_meta"]["tiles"]), 1, "prompt override should drop unmatched tile metadata") +assert_equal(len(dropped_set["tile_images"]), 1, "prompt override should drop unmatched tile images") +assert_equal(dropped_set["tile_meta"]["tiles"][0]["id"], 0, "prompt override should reindex kept tile ids") +composite_override_inputs = ttp.TTP_Smart_Tile_Composite_Override_Experimental.INPUT_TYPES() +assert_equal(composite_override_inputs["required"]["tile_set"][0], "TTP_SMART_TILE_SET", "composite override should accept Smart Tile Set") +assert_equal(composite_override_inputs["required"]["tile_set"][1].get("forceInput"), True, "composite override tile_set should be a forceInput connection") +assert_equal("replace_tile" in composite_override_inputs["required"]["composite_mode"][0], True, "composite override should expose replace_tile mode") +assert_equal(composite_override_inputs["required"]["replace_tile_shape"][0], ["keep", "mask_first", "tile_box_first"], "composite override should expose replace tile shape priority") +assert_equal(composite_override_inputs["required"]["selector_type"][0], prompt_override_inputs["required"]["selector_type"][0], "composite override selectors should match prompt override selectors") +assert_equal( + ttp.NODE_CLASS_MAPPINGS["TTP_Smart_Tile_Composite_Override_Experimental"], + ttp.TTP_Smart_Tile_Composite_Override_Experimental, + "composite override should be registered", +) +composite_node = ttp.TTP_Smart_Tile_Composite_Override_Experimental() +composited_set, composite_json, composite_summary = composite_node.override_composite( + override_tile_set, + selector_type="label", + selector="face", + composite_mode="replace_tile", + replace_tile_shape="tile_box_first", + layer_mode="max", + layer=12, + priority_mode="max", + priority=220, + occlusion_priority_mode="max", + occlusion_priority=60000, + blend_mode="set", + blend=24, +) +composited_tile = composited_set["tile_meta"]["tiles"][0] +assert_equal(composited_tile["composite_mode"], "replace_tile", "composite override should set replace_tile mode") +assert_equal(composited_tile["recommended_composite_mode"], "replace_tile", "composite override should update recommended composite mode") +assert_equal(composited_tile["replace_tile_shape"], "tile_box_first", "composite override should set replace tile shape priority") +assert_equal(composited_tile["layer"], 12, "composite override should raise the selected tile layer") +assert_equal(composited_tile["priority"], 220, "composite override should raise the selected tile priority") +assert_equal(composited_tile["occlusion_priority"], 60000, "composite override should raise the selected tile occlusion priority") +assert_equal(composited_tile["blend"], 24, "composite override should set selected tile blend") +assert_equal(json.loads(composite_json)["matched"], 1, "composite override JSON should report matched tile count") +assert_equal("matched=1" in composite_summary, True, "composite override summary should report matched count") +stack_order_inputs = ttp.TTP_Smart_Tile_Stack_Order_Experimental.INPUT_TYPES() +assert_equal(stack_order_inputs["required"]["tile_set"][0], "TTP_SMART_TILE_SET", "stack order should accept Smart Tile Set") +assert_equal(stack_order_inputs["required"]["tile_set"][1].get("forceInput"), True, "stack order tile_set should be a forceInput connection") +assert_equal(stack_order_inputs["required"]["top_order"][0], ["first_on_top", "last_on_top"], "stack order should expose top order direction") +assert_equal(stack_order_inputs["required"]["composite_mode"][1]["default"], "replace_tile", "stack order should default to replace-tile compositing") +assert_equal( + ttp.NODE_CLASS_MAPPINGS["TTP_Smart_Tile_Stack_Order_Experimental"], + ttp.TTP_Smart_Tile_Stack_Order_Experimental, + "stack order should be registered", +) +stack_order_tile_set = { + "version": 1, + "type": "ttp_smart_tile_set", + "original_size": [100, 100], + "tile_meta": { + "version": 3, + "type": "ttp_smart_tile", + "storage": "tile_set", + "original_size": [100, 100], + "tiles": [ + {"id": 0, "name": "tile_1", "core_box": [0, 0, 50, 50], "sample_box": [0, 0, 50, 50], "layer": 0, "occlusion_priority": 0}, + {"id": 1, "name": "tile_2", "core_box": [0, 0, 50, 50], "sample_box": [0, 0, 50, 50], "layer": 0, "occlusion_priority": 0}, + {"id": 2, "name": "tile_3", "core_box": [0, 0, 50, 50], "sample_box": [0, 0, 50, 50], "layer": 0, "occlusion_priority": 0}, + {"id": 3, "name": "tile_4", "core_box": [0, 0, 50, 50], "sample_box": [0, 0, 50, 50], "layer": 1, "occlusion_priority": 80000}, + ], + }, + "tile_images": ["tile_1", "tile_2", "tile_3", "tile_4"], +} +stack_node = ttp.TTP_Smart_Tile_Stack_Order_Experimental() +stacked_set, stack_json, stack_summary = stack_node.apply_stack_order( + stack_order_tile_set, + tile_order="T1，T3，T2", +) +stacked_tiles = stacked_set["tile_meta"]["tiles"] +assert_equal(stacked_tiles[0]["occlusion_priority"] > stacked_tiles[2]["occlusion_priority"], True, "stack order should make the first listed tile topmost by default") +assert_equal(stacked_tiles[2]["occlusion_priority"] > stacked_tiles[1]["occlusion_priority"], True, "stack order should preserve listed ordering between selected tiles") +assert_equal(stacked_tiles[1]["occlusion_priority"] > stacked_tiles[3]["occlusion_priority"], True, "stack order should place listed tiles above unlisted tiles") +assert_equal(stacked_tiles[0]["composite_mode"], "replace_tile", "stack order should default selected tiles to replace_tile mode") +assert_equal("composite_mode" in stacked_tiles[3], False, "stack order should not change unlisted tile composite mode") +assert_equal(json.loads(stack_json)["tiles"][1]["selector"], "T3", "stack order JSON should preserve selector order") +assert_equal("T1" in stack_summary and "T3" in stack_summary and "T2" in stack_summary, True, "stack order summary should include selected tiles") loop_source_inputs = ttp.TTP_Smart_Tile_Loop_Source_Experimental.INPUT_TYPES() assert_equal(loop_source_inputs["required"]["tile_set"][0], "TTP_SMART_TILE_SET", "loop source should accept Smart Tile Set") assert_equal(loop_source_inputs["optional"]["clip"][0], "CLIP", "loop source should optionally encode tile prompts with CLIP") @@ -1145,6 +1303,37 @@ mask_array = ttp._ttp_tile_object_mask_array(masked_auto_meta[0], masked_auto_me assert_equal(mask_array.shape[2], 1, "decoded object mask should be a single-channel weight map") assert_equal(float(mask_array.max()) > 0.5, True, "decoded object mask should keep foreground weight") +small_expand_mask = Image.new("L", (32, 32), 0) +for x in range(10, 14): + for y in range(10, 14): + small_expand_mask.putpixel((x, y), 255) +small_mask_layout = ttp._ttp_boxes_to_auto_layout( + [{"x": 10, "y": 10, "width": 4, "height": 4, "label": "eye", "score": 0.9}], + 32, + 32, + object_padding=4, + mask_expand=0, + max_tiles=1, + include_background=False, + masks=[small_expand_mask], +) +expanded_mask_layout = ttp._ttp_boxes_to_auto_layout( + [{"x": 10, "y": 10, "width": 4, "height": 4, "label": "eye", "score": 0.9}], + 32, + 32, + object_padding=4, + mask_expand=2, + max_tiles=1, + include_background=False, + masks=[small_expand_mask], +) +small_mask_tile = ttp._ttp_parse_smart_tile_layout(small_mask_layout, 32, 32)[0] +expanded_mask_tile = ttp._ttp_parse_smart_tile_layout(expanded_mask_layout, 32, 32)[0] +small_mask_pixels = np.count_nonzero(np.array(ttp._ttp_decode_object_mask_data(small_mask_tile["object_mask"])) > 0) +expanded_mask_pixels = np.count_nonzero(np.array(ttp._ttp_decode_object_mask_data(expanded_mask_tile["object_mask"])) > 0) +assert_equal(expanded_mask_pixels > small_mask_pixels, True, "auto mask expand should dilate object mask foreground pixels") +assert_equal(expanded_mask_tile.get("object_mask_expand"), 2.0, "auto mask expand should be preserved in tile metadata") + large_body_mask = Image.new("L", (900, 600), 255) large_body_layout = ttp._ttp_boxes_to_auto_layout( [{"x": 0, "y": 0, "width": 900, "height": 600, "label": "person body", "score": 0.93}], @@ -1215,6 +1404,7 @@ paint_layout, paint_message = ttp._ttp_run_paint_mask_auto_layout( default_pad=8, default_blend=4, object_padding=2, + mask_expand=2, max_tiles=8, allow_object_overlap=True, ) @@ -1226,6 +1416,51 @@ assert_equal("paint mask" in paint_message.lower(), True, "paint-only auto tile half_mask = Image.new("L", (8, 8), 128) half_mask_data = ttp._ttp_encode_object_mask_data([half_mask], [0], [0, 0, 8, 8]) assemble_node = ttp.TTP_Smart_Tile_Assemble_Experimental() +stack_assemble_tile_set = { + "type": "ttp_smart_tile_set", + "original_size": [8, 8], + "tile_meta": { + "type": "ttp_smart_tile", + "original_size": [8, 8], + "tiles": [ + { + "name": f"stack_tile_{index + 1}", + "core_box": [0, 0, 8, 8], + "sample_box": [0, 0, 8, 8], + "tile_canvas_size": [8, 8], + "tile_canvas_box": [0, 0, 8, 8], + "overlap_edges_px_source": {"left": 0, "right": 0, "top": 0, "bottom": 0}, + "blend": 0, + "importance": 1.0, + "priority": 0.0, + "layer": 1 if index == 3 else 0, + "occlusion_priority": 80000 if index == 3 else 0, + } + for index in range(4) + ], + }, + "tile_images": [ + ttp.pil2tensor(Image.new("RGB", (8, 8), (220, 20, 20)))[0], + ttp.pil2tensor(Image.new("RGB", (8, 8), (20, 220, 20)))[0], + ttp.pil2tensor(Image.new("RGB", (8, 8), (20, 20, 220)))[0], + ttp.pil2tensor(Image.new("RGB", (8, 8), (220, 220, 20)))[0], + ], +} +stack_assemble_set, _stack_assemble_json, _stack_assemble_summary = stack_node.apply_stack_order( + stack_assemble_tile_set, + tile_order="T1,T3,T2", +) +stack_assembled, _stack_assembled_weights = assemble_node.assemble_tiles( + blend_multiplier=1.0, + output_scale=1.0, + use_priority=True, + base_canvas_mode="black", + auto_composite_policy="safe_auto", + tile_set=stack_assemble_set, +) +stack_center = stack_assembled.array[0, 4, 4] +assert_equal(float(stack_center[0]) > float(stack_center[2]), True, "stack order assemble should put the first listed tile on top by default") +assert_equal(float(stack_center[0]) > float(stack_center[1]), True, "stack order assemble should keep unlisted and later listed tiles behind the first listed tile") soft_tile_meta = { "type": "ttp_smart_tile", "original_size": [8, 8], @@ -1545,6 +1780,166 @@ safe_eye_center = safe_eye.array[0, 8, 8] assert_equal(float(safe_eye_edge[0]) > float(strict_eye_edge[0]), True, "safe auto detail overlay should preserve more face color at eye mask edges") assert_equal(float(safe_eye_center[2]) > float(safe_eye_center[0]), True, "safe auto detail overlay should keep eye details stronger at the mask center") +replace_mask_full = Image.new("L", (16, 16), 0) +ImageDraw.Draw(replace_mask_full).rectangle((7, 7, 8, 8), fill=255) +replace_mask_data = ttp._ttp_encode_object_mask_data([replace_mask_full], [0], [4, 4, 12, 12]) +replace_mask_tile_set = { + "type": "ttp_smart_tile_set", + "original_size": [16, 16], + "tile_meta": { + "type": "ttp_smart_tile", + "original_size": [16, 16], + "tiles": [ + { + "name": "masked_edit", + "label": "paint mask edit", + "core_box": [4, 4, 8, 8], + "sample_box": [4, 4, 8, 8], + "tile_canvas_size": [8, 8], + "tile_canvas_box": [0, 0, 8, 8], + "overlap_edges_px_source": {"left": 0, "right": 0, "top": 0, "bottom": 0}, + "blend": 0, + "importance": 1.0, + "priority": 0.0, + "layer": 0, + "occlusion_priority": 0, + "object_mask": replace_mask_data, + "composite_mode": "replace_tile", + }, + ], + }, + "tile_images": [ + ttp.pil2tensor(Image.new("RGB", (8, 8), (20, 20, 220)))[0], + ], +} +replace_source = ttp.pil2tensor(Image.new("RGB", (16, 16), (220, 20, 20))) +replace_masked, _replace_masked_weights = assemble_node.assemble_tiles( + blend_multiplier=1.0, + output_scale=1.0, + use_priority=True, + base_canvas_mode="source_image", + mask_blend_mode="off", + auto_composite_policy="replace_tile", + source_image=replace_source, + tile_set=replace_mask_tile_set, +) +replace_mask_corner = replace_masked.array[0, 4, 4] +replace_mask_center = replace_masked.array[0, 8, 8] +assert_equal(float(replace_mask_corner[0]) > float(replace_mask_corner[2]), True, "replace_tile must preserve source pixels outside an object mask instead of pasting a rectangle") +assert_equal(float(replace_mask_center[2]) > float(replace_mask_center[0]), True, "replace_tile must hard replace pixels inside an object mask") +replace_boxed, _replace_boxed_weights = assemble_node.assemble_tiles( + blend_multiplier=1.0, + output_scale=1.0, + use_priority=True, + base_canvas_mode="source_image", + mask_blend_mode="off", + auto_composite_policy="replace_tile", + replace_tile_shape="tile_box_first", + source_image=replace_source, + tile_set=replace_mask_tile_set, +) +replace_box_corner = replace_boxed.array[0, 4, 4] +replace_box_center = replace_boxed.array[0, 8, 8] +assert_equal(float(replace_box_corner[2]) > float(replace_box_corner[0]), True, "replace_tile tile_box_first should paste the full tile rectangle even when an object mask exists") +assert_equal(float(replace_box_center[2]) > float(replace_box_center[0]), True, "replace_tile tile_box_first should still replace the object center") + +replace_soft_mask_full = Image.new("L", (16, 16), 0) +ImageDraw.Draw(replace_soft_mask_full).rectangle((4, 4, 11, 11), fill=128) +replace_soft_mask_data = ttp._ttp_encode_object_mask_data([replace_soft_mask_full], [0], [4, 4, 12, 12]) +replace_soft_mask_tile_set = { + "type": "ttp_smart_tile_set", + "original_size": [16, 16], + "tile_meta": { + "type": "ttp_smart_tile", + "original_size": [16, 16], + "tiles": [ + { + "name": "soft_masked_edit", + "label": "paint mask edit", + "core_box": [4, 4, 8, 8], + "sample_box": [4, 4, 8, 8], + "tile_canvas_size": [8, 8], + "tile_canvas_box": [0, 0, 8, 8], + "overlap_edges_px_source": {"left": 0, "right": 0, "top": 0, "bottom": 0}, + "blend": 0, + "importance": 1.0, + "priority": 0.0, + "layer": 0, + "occlusion_priority": 0, + "object_mask": replace_soft_mask_data, + "composite_mode": "replace_tile", + }, + ], + }, + "tile_images": [ + ttp.pil2tensor(Image.new("RGB", (8, 8), (20, 20, 220)))[0], + ], +} +replace_soft_masked, _replace_soft_masked_weights = assemble_node.assemble_tiles( + blend_multiplier=1.0, + output_scale=1.0, + use_priority=True, + base_canvas_mode="source_image", + mask_blend_mode="off", + auto_composite_policy="replace_tile", + source_image=replace_source, + tile_set=replace_soft_mask_tile_set, +) +replace_soft_center = replace_soft_masked.array[0, 8, 8] +assert_equal(float(replace_soft_center[2]) > 0.8, True, "replace_tile mask_first should harden soft mask interiors instead of making the whole edit translucent") +assert_equal(float(replace_soft_center[0]) < 0.2, True, "replace_tile mask_first should remove source pixels from soft mask interiors") + +replace_own_mask_full = Image.new("L", (16, 16), 0) +ImageDraw.Draw(replace_own_mask_full).rectangle((7, 7, 8, 8), fill=255) +replace_source_mask_full = Image.new("L", (16, 16), 0) +ImageDraw.Draw(replace_source_mask_full).rectangle((4, 4, 11, 11), fill=255) +replace_own_mask_data = ttp._ttp_encode_object_mask_data([replace_own_mask_full], [0], [4, 4, 12, 12]) +replace_source_mask_data = ttp._ttp_encode_object_mask_data([replace_source_mask_full], [0], [4, 4, 12, 12]) +replace_own_mask_tile_set = { + "type": "ttp_smart_tile_set", + "original_size": [16, 16], + "tile_meta": { + "type": "ttp_smart_tile", + "original_size": [16, 16], + "tiles": [ + { + "name": "own_mask_edit", + "label": "paint mask edit", + "core_box": [4, 4, 8, 8], + "sample_box": [4, 4, 8, 8], + "tile_canvas_size": [8, 8], + "tile_canvas_box": [0, 0, 8, 8], + "overlap_edges_px_source": {"left": 0, "right": 0, "top": 0, "bottom": 0}, + "blend": 0, + "importance": 1.0, + "priority": 0.0, + "layer": 0, + "occlusion_priority": 0, + "object_mask": replace_own_mask_data, + "object_mask_source": replace_source_mask_data, + "composite_mode": "replace_tile", + }, + ], + }, + "tile_images": [ + ttp.pil2tensor(Image.new("RGB", (8, 8), (20, 20, 220)))[0], + ], +} +replace_own_masked, _replace_own_masked_weights = assemble_node.assemble_tiles( + blend_multiplier=1.0, + output_scale=1.0, + use_priority=True, + base_canvas_mode="source_image", + mask_blend_mode="off", + auto_composite_policy="replace_tile", + source_image=replace_source, + tile_set=replace_own_mask_tile_set, +) +replace_source_only_pixel = replace_own_masked.array[0, 4, 4] +replace_own_center_pixel = replace_own_masked.array[0, 8, 8] +assert_equal(float(replace_source_only_pixel[0]) > float(replace_source_only_pixel[2]), True, "replace_tile mask_first should prefer the tile's own cropped object mask over a broader source mask") +assert_equal(float(replace_own_center_pixel[2]) > float(replace_own_center_pixel[0]), True, "replace_tile mask_first should still replace pixels inside the tile's own mask") + subdivided_mask_full = Image.new("L", (64, 64), 0) ImageDraw.Draw(subdivided_mask_full).rectangle((24, 24, 39, 39), fill=255) subdivided_mask_data = ttp._ttp_encode_object_mask_data([subdivided_mask_full], [0], [24, 24, 40, 40]) diff --git a/scripts/smoke_smart_tile_interactive.mjs b/scripts/smoke_smart_tile_interactive.mjs index 79bbb27..b9e3576 100644 --- a/scripts/smoke_smart_tile_interactive.mjs +++ b/scripts/smoke_smart_tile_interactive.mjs @@ -34,6 +34,8 @@ assert(/async function inferSmartTileLayout/.test(source), "tile editor should e assert(/app\.queuePrompt\(0,\s*1,\s*\[String\(node\.id\)\]\)/.test(source), "Infer action should queue only the current Smart Tile node"); assert(/ttp-smart-tile-layout/.test(source), "Infer action should listen for backend layout updates"); assert(/auto_detect_request/.test(source), "Infer action should increment the node inference request widget"); +assert(/options\.fillGaps !== false/.test(source), "Infer action should allow disabling automatic gap filling"); +assert(/modeOverride: "sam3\.1"/.test(source), "Auto SAM should force SAM3.1 detection"); assert(!/\/ttp\/smart_tile\/analyze/.test(source), "Infer action should not use the old analysis route"); assert(/occlusion_priority/.test(source), "interactive layout should preserve auto tile priority metadata"); assert(/TTP_Smart_Tile_Loop_Source_Experimental/.test(source), "frontend should know the Smart Tile Loop Source node"); @@ -54,6 +56,8 @@ assert(/snapCreatedTile/.test(source), "tile editor should snap newly drawn tile assert(/function fillTileGaps/.test(source), "tile editor should share automatic gap filling logic"); assert(/function autoMaxTiles/.test(source), "tile editor should read the Auto Tile max tile limit"); assert(/const filled = fillTileGaps\(node, layout\.tiles, autoMaxTiles\(node\)\);/.test(source), "Auto Tile gap fill should respect auto_max_tiles"); +assert(/if \(fillGaps\) \{\s*const filled = fillTileGaps\(node, layout\.tiles, autoMaxTiles\(node\)\);/s.test(source), "Auto Tile should only fill gaps when requested"); +assert(/writeLayout\(node, layout\.tiles, 0\);\s*const coverage = analyzeCoverage\(layout\.tiles\);/s.test(source), "Auto SAM should write detected tiles without gap filling"); assert(/source: "auto_gap"/.test(source), "auto-filled gap tiles should be marked as background gap tiles"); assert(/function editorStageSize/.test(source), "tile editor should compute a fixed source-ratio stage size"); assert(!/aspect-ratio:/.test(source), "tile editor should not rely on CSS aspect-ratio when the node is resized"); @@ -73,16 +77,50 @@ assert(/crop_mask_skip_empty/.test(source), "Grid in should allow skipping empty assert(/createButton\(`Grid in T\$\{selectedIndex \+ 1\}`, async \(\) =>/.test(source), "Grid in should run asynchronously for mask cropping"); assert(/ensurePaintMaskCanvas/.test(source), "tile editor should create a paint mask canvas"); assert(/auto_paint_mask/.test(source), "tile editor should sync painted masks into the hidden backend input"); +assert(/MASK_OVERLAY_COLORS/.test(source), "tile editor should define distinct colors for object-mask overlays"); +assert(/function maskOverlayEnabled/.test(source), "tile editor should expose a mask overlay toggle state"); +assert(/function stitchMaskPreviewEnabled/.test(source), "tile editor should expose a stitch mask preview toggle state"); +assert(/async function renderMaskOverlay/.test(source), "tile editor should render object masks as a colored overlay"); +assert(/tintedMaskCanvas/.test(source), "tile editor should tint mask pixels before showing overlays"); +assert(/stitchedMaskValues/.test(source), "tile editor should preview hardened replace stitch masks"); +assert(!/canvas\.isConnected/.test(source), "mask overlay rendering should not require the staged canvas to be connected before renderEditor appends it"); +assert(/Show masks/.test(source), "tile editor should expose a show masks toggle"); +assert(/Hide masks/.test(source), "tile editor should expose a hide masks toggle"); +assert(/Stitch mask/.test(source), "tile editor should expose a stitch mask preview toggle"); +assert(/Raw masks/.test(source), "tile editor should expose a raw mask preview toggle"); +assert(/ttpSmartTileShowMasks/.test(source), "mask overlay visibility should be stored on the editor node instance"); +assert(/ttpSmartTilePreviewStitchMask/.test(source), "stitch mask preview should be stored on the editor node instance"); +assert(/renderMaskOverlay\(node, maskOverlay, tiles\)/.test(source), "renderEditor should draw the colored mask overlay when enabled"); +assert(/tintedMaskCanvas\(maskImage, color, \{/.test(source), "renderMaskOverlay should pass preview options into mask tinting"); +assert(/node\.ttpSmartTileShowMasks = true;/.test(source), "stitch mask preview should auto-enable colored mask overlays"); +assert(/Move/.test(source), "tile editor should expose an explicit move mode"); +assert(/node\.ttpSmartTilePaintMode = "off";\s*renderEditor\(node\);/s.test(source), "Move should restore tile drag and resize mode"); +assert(/pointer-events:" \+ \(paintMode === "off" \? "auto" : "none"\)/.test(source), "tile regions should only block dragging while Brush or Erase is active"); assert(/Brush/.test(source), "tile editor should expose a paint brush action"); assert(/Erase/.test(source), "tile editor should expose an erase action"); assert(/Mask to Tile/.test(source), "tile editor should expose a button that commits painted masks into layout tiles"); +assert(/Mask Replace/.test(source), "tile editor should expose a button that replaces the layout with painted mask tiles"); assert(/Refresh masks/.test(source), "tile editor should expose a refresh masks action"); assert(/refreshInheritedMasks\(node, tiles, gridMaskMode\(node\)\)/.test(source), "Refresh masks should use the current grid mask mode"); +assert(/function selectedTileIndexes/.test(source), "tile editor should track multi-selected tiles"); +assert(/function setTileSelection/.test(source), "tile editor should support modifier-click multi-selection"); +assert(/boolEventModifier\(event\)/.test(beginDrag), "modifier-clicking an existing tile should toggle multi-selection"); +assert(/async function mergedObjectMaskForTiles/.test(source), "tile editor should union selected tile masks"); +assert(/async function mergeSelectedMasks/.test(source), "tile editor should expose selected mask merging logic"); +assert(/async function mergeSelectedTiles/.test(source), "tile editor should expose selected tile merging logic"); +assert(/Merge masks/.test(source), "tile editor should expose a merge masks action"); +assert(/Merge tiles/.test(source), "tile editor should expose a merge tiles action"); +assert(/ownObjectMaskData/.test(source), "mask merging should prefer each tile's cropped mask over parent source masks"); assert(/function addPaintMaskTiles/.test(source), "tile editor should convert painted masks into persisted tiles"); +assert(/auto_mask_expand/.test(source), "paint-created tiles should read the auto mask expansion widget"); +assert(/expandMaskImageData/.test(source), "paint-created masks should support foreground expansion"); +assert(/maskCropData\(canvas, box, maskExpand\)/.test(source), "paint-created masks should apply the requested mask expansion"); assert(/source: "paint_mask"/.test(source), "paint-created tiles should be marked as paint mask tiles"); assert(/object_mask:\s*\{/.test(source), "paint-created tiles should persist object masks in layout metadata"); assert(/Clear mask/.test(source), "tile editor should expose a clear mask action"); assert(/syncPaintMaskWidget\(node\);\s*inferSmartTileLayout\(node\);/s.test(source), "Auto Tile should submit the latest painted mask before inference"); +assert(/Auto SAM/.test(source), "tile editor should expose an Auto SAM action"); +assert(/syncPaintMaskWidget\(node\);\s*inferSmartTileLayout\(node, \{\s*fillGaps: false,/s.test(source), "Auto SAM should submit masks and disable automatic gap filling"); assert(!/PROMPT_BUILDER_NAME/.test(source), "frontend must not hook the QwenVL prompt builder node"); assert(!/repairPromptBuilderWidgets/.test(source), "prompt builder values must be left to native ComfyUI widget serialization"); assert(!/ttp_prompt_builder_values/.test(source), "prompt builder must not restore stale named widget values from properties"); diff --git a/web/js/ttp_smart_tile_interactive.js b/web/js/ttp_smart_tile_interactive.js index 8470008..ebeabc9 100644 --- a/web/js/ttp_smart_tile_interactive.js +++ b/web/js/ttp_smart_tile_interactive.js @@ -10,10 +10,20 @@ const snapGuideThresholdPx = 10; const largeContextAreaRatio = 0.9; const STORAGE_PREFIX = "ttp_smart_tile_interactive_layout"; const GRID_MASK_MODES = ["off", "crop_mask", "crop_mask_skip_empty"]; +const MASK_OVERLAY_COLORS = [ + [248, 113, 113], + [52, 211, 153], + [96, 165, 250], + [251, 191, 36], + [216, 180, 254], + [45, 212, 191], + [251, 113, 133], + [163, 230, 53], +]; const TILE_METADATA_KEYS = [ "name", "source", "label", "score", "layer", "object_id", "occlusion_priority", "priority", "importance", "pad", "blend", "object_mask", "object_mask_source", - "semantic_category", "semantic_role", "semantic_score", "recommended_scale_weight", + "object_mask_expand", "semantic_category", "semantic_role", "semantic_score", "recommended_scale_weight", "recommended_layer", "recommended_priority", "recommended_occlusion_priority", "recommended_composite_mode", ]; @@ -191,6 +201,16 @@ function sourceObjectMaskData(tile) { return null; } +function ownObjectMaskData(tile) { + if (tile?.object_mask?.format === "png_base64" && tile.object_mask?.data) { + return tile.object_mask; + } + if (tile?.object_mask_source?.format === "png_base64" && tile.object_mask_source?.data) { + return tile.object_mask_source; + } + return null; +} + function loadObjectMaskImage(maskData) { return new Promise((resolve, reject) => { if (!maskData || maskData.format !== "png_base64" || !maskData.data) { @@ -204,6 +224,188 @@ function loadObjectMaskImage(maskData) { }); } +function maskOverlayEnabled(node) { + return Boolean(node?.ttpSmartTileShowMasks); +} + +function stitchMaskPreviewEnabled(node) { + return Boolean(node?.ttpSmartTilePreviewStitchMask); +} + +function maskValuesFromImageData(imageData) { + const data = imageData.data; + let maxRgb = 0; + let minAlpha = 255; + let maxAlpha = 0; + for (let index = 0; index < data.length; index += 4) { + maxRgb = Math.max(maxRgb, data[index], data[index + 1], data[index + 2]); + minAlpha = Math.min(minAlpha, data[index + 3]); + maxAlpha = Math.max(maxAlpha, data[index + 3]); + } + const useAlphaOnly = maxRgb === 0 && maxAlpha > minAlpha; + const values = new Uint8Array(Math.floor(data.length / 4)); + let hasForeground = false; + for (let index = 0, pixel = 0; index < data.length; index += 4, pixel += 1) { + const maskValue = useAlphaOnly + ? data[index + 3] + : Math.max(data[index], data[index + 1], data[index + 2]); + values[pixel] = maskValue; + if (maskValue > 0) { + hasForeground = true; + } + } + return { values, hasForeground }; +} + +function boxBlurHorizontal(values, width, height, radius) { + const output = new Uint8Array(values.length); + const blurRadius = Math.max(1, Math.round(Number(radius) || 1)); + for (let y = 0; y < height; y += 1) { + const rowOffset = y * width; + const prefix = new Uint32Array(width + 1); + for (let x = 0; x < width; x += 1) { + prefix[x + 1] = prefix[x] + values[rowOffset + x]; + } + for (let x = 0; x < width; x += 1) { + const left = Math.max(0, x - blurRadius); + const right = Math.min(width - 1, x + blurRadius); + const count = Math.max(1, right - left + 1); + output[rowOffset + x] = Math.round((prefix[right + 1] - prefix[left]) / count); + } + } + return output; +} + +function boxBlurVertical(values, width, height, radius) { + const output = new Uint8Array(values.length); + const blurRadius = Math.max(1, Math.round(Number(radius) || 1)); + for (let x = 0; x < width; x += 1) { + const prefix = new Uint32Array(height + 1); + for (let y = 0; y < height; y += 1) { + prefix[y + 1] = prefix[y] + values[y * width + x]; + } + for (let y = 0; y < height; y += 1) { + const top = Math.max(0, y - blurRadius); + const bottom = Math.min(height - 1, y + blurRadius); + const count = Math.max(1, bottom - top + 1); + output[y * width + x] = Math.round((prefix[bottom + 1] - prefix[top]) / count); + } + } + return output; +} + +function blurMaskValues(values, width, height, radius) { + const horizontal = boxBlurHorizontal(values, width, height, radius); + return boxBlurVertical(horizontal, width, height, radius); +} + +function stitchedMaskValues(values, width, height, feather) { + const hard = new Uint8Array(values.length); + const threshold = Math.floor(255 * 0.05); + let hasForeground = false; + for (let index = 0; index < values.length; index += 1) { + if (values[index] > threshold) { + hard[index] = 255; + hasForeground = true; + } + } + if (!hasForeground) { + return hard; + } + const radius = Math.max(0, Math.round(Number(feather) || 0)); + if (radius <= 0) { + return hard; + } + const stitchRadius = Math.max(1, Math.min(64, Math.round(radius * 0.25))); + const expanded = slidingMaxVertical( + slidingMaxHorizontal(hard, width, height, stitchRadius), + width, + height, + stitchRadius + ); + const softened = blurMaskValues(expanded, width, height, stitchRadius); + const output = new Uint8Array(values.length); + for (let index = 0; index < values.length; index += 1) { + output[index] = Math.max(hard[index], softened[index]); + } + return output; +} + +function tintedMaskCanvas(maskImage, color, options = {}) { + const width = Math.max(1, Math.round(maskImage?.naturalWidth || maskImage?.width || 1)); + const height = Math.max(1, Math.round(maskImage?.naturalHeight || maskImage?.height || 1)); + const canvas = document.createElement("canvas"); + canvas.width = width; + canvas.height = height; + const context = canvas.getContext("2d", { willReadFrequently: true }); + if (!context) { + return null; + } + context.clearRect(0, 0, width, height); + context.drawImage(maskImage, 0, 0, width, height); + const imageData = context.getImageData(0, 0, width, height); + const data = imageData.data; + const extracted = maskValuesFromImageData(imageData); + if (!extracted.hasForeground) { + return null; + } + const values = options.stitch + ? stitchedMaskValues(extracted.values, width, height, options.feather) + : extracted.values; + for (let index = 0, pixel = 0; index < data.length; index += 4, pixel += 1) { + const maskValue = values[pixel]; + data[index] = color[0]; + data[index + 1] = color[1]; + data[index + 2] = color[2]; + data[index + 3] = Math.round(maskValue * 0.52); + } + context.putImageData(imageData, 0, 0); + return canvas; +} + +async function renderMaskOverlay(node, canvas, tiles) { + const sourceSize = imageSourceSize(node); + const context = canvas?.getContext?.("2d", { willReadFrequently: true }); + if (!sourceSize || !context) { + return; + } + const token = Symbol("mask-overlay"); + node.ttpSmartTileMaskOverlayToken = token; + canvas.width = sourceSize.width; + canvas.height = sourceSize.height; + context.clearRect(0, 0, canvas.width, canvas.height); + + for (const [index, tile] of tiles.entries()) { + if (node.ttpSmartTileMaskOverlayToken !== token) { + return; + } + const maskData = ownObjectMaskData(tile); + const maskBox = objectMaskPixelBox(maskData); + if (!maskData?.data || !maskBox) { + continue; + } + const maskImage = await loadObjectMaskImage(maskData); + if (node.ttpSmartTileMaskOverlayToken !== token || !maskImage) { + return; + } + const color = MASK_OVERLAY_COLORS[index % MASK_OVERLAY_COLORS.length]; + const tintedMask = tintedMaskCanvas(maskImage, color, { + stitch: stitchMaskPreviewEnabled(node), + feather: Number(tile?.blend ?? layoutDefaults(node).blend ?? 0), + }); + if (!tintedMask) { + continue; + } + context.drawImage( + tintedMask, + maskBox.x0, + maskBox.y0, + maskBox.width, + maskBox.height + ); + } +} + function maskCanvasHasForeground(canvas) { const context = canvas?.getContext?.("2d", { willReadFrequently: true }); if (!context || !canvas.width || !canvas.height) { @@ -460,6 +662,157 @@ async function refreshInheritedMasks(node, tiles, mode) { return { tiles: result, refreshed, skipped, emptied, unchanged }; } +async function mergedObjectMaskForTiles(node, tiles, selectedIndexes) { + const entries = []; + for (const index of selectedIndexes) { + const maskData = ownObjectMaskData(tiles[index]); + const maskBox = objectMaskPixelBox(maskData); + if (!maskData?.data || !maskBox) { + continue; + } + const maskImage = await loadObjectMaskImage(maskData); + if (!maskImage) { + continue; + } + entries.push({ maskImage, maskBox }); + } + if (!entries.length) { + return null; + } + + const x0 = Math.min(...entries.map((entry) => entry.maskBox.x0)); + const y0 = Math.min(...entries.map((entry) => entry.maskBox.y0)); + const x1 = Math.max(...entries.map((entry) => entry.maskBox.x1)); + const y1 = Math.max(...entries.map((entry) => entry.maskBox.y1)); + const width = Math.max(1, x1 - x0); + const height = Math.max(1, y1 - y0); + const canvas = document.createElement("canvas"); + canvas.width = width; + canvas.height = height; + const context = canvas.getContext("2d", { willReadFrequently: true }); + if (!context) { + return null; + } + context.clearRect(0, 0, width, height); + context.globalCompositeOperation = "lighter"; + for (const entry of entries) { + context.drawImage( + entry.maskImage, + entry.maskBox.x0 - x0, + entry.maskBox.y0 - y0, + entry.maskBox.width, + entry.maskBox.height + ); + } + context.globalCompositeOperation = "source-over"; + if (!maskCanvasHasForeground(canvas)) { + return null; + } + const maskData = { + format: "png_base64", + box: [Math.round(x0), Math.round(y0), Math.round(width), Math.round(height)], + width: Math.round(width), + height: Math.round(height), + data: canvas.toDataURL("image/png").split(",", 2)[1], + }; + return { + maskData, + maskBox: { x0, y0, x1, y1, width, height }, + maskImage: canvas, + count: entries.length, + }; +} + +async function mergeSelectedMasks(node, tiles, selectedIndexes, selectedIndex) { + if (selectedIndexes.length < 2) { + node.ttpSmartTileStatus = "Select at least two tiles."; + return false; + } + const merged = await mergedObjectMaskForTiles(node, tiles, selectedIndexes); + if (!merged) { + node.ttpSmartTileStatus = "Selected tiles have no masks to merge."; + return false; + } + const selectedSet = new Set(selectedIndexes); + let updated = 0; + const nextTiles = tiles.map((tile, index) => { + if (!selectedSet.has(index)) { + return tile; + } + const nextTile = { ...tile }; + const cropped = cropObjectMaskForTile(node, merged.maskImage, merged.maskBox, nextTile); + if (cropped) { + nextTile.object_mask = cropped; + nextTile.object_mask_source = merged.maskData; + updated += 1; + } else { + delete nextTile.object_mask; + delete nextTile.object_mask_source; + } + return normalizeTile(node, nextTile); + }); + if (!updated) { + node.ttpSmartTileStatus = "Merged mask does not overlap selected tiles."; + return false; + } + writeLayout(node, nextTiles, selectedIndex, selectedIndexes); + node.ttpSmartTileStatus = `Merged ${merged.count} mask(s) across ${updated} tile(s).`; + return true; +} + +async function mergeSelectedTiles(node, tiles, selectedIndexes, selectedIndex) { + if (selectedIndexes.length < 2) { + node.ttpSmartTileStatus = "Select at least two tiles."; + return false; + } + const selectedSet = new Set(selectedIndexes); + const selectedTiles = selectedIndexes.map((index) => tiles[index]).filter(Boolean); + if (selectedTiles.length < 2) { + return false; + } + const primaryIndex = selectedSet.has(selectedIndex) ? selectedIndex : selectedIndexes[0]; + const primaryTile = tiles[primaryIndex] ?? selectedTiles[0]; + const x0 = Math.min(...selectedTiles.map((tile) => Number(tile.x0))); + const y0 = Math.min(...selectedTiles.map((tile) => Number(tile.y0))); + const x1 = Math.max(...selectedTiles.map((tile) => Number(tile.x1))); + const y1 = Math.max(...selectedTiles.map((tile) => Number(tile.y1))); + const selectorText = selectedIndexes.map((index) => `T${index + 1}`).join("+"); + const mergedTile = normalizeTile(node, { + ...primaryTile, + x0, + y0, + x1, + y1, + name: `merged_${selectorText.toLowerCase()}`, + source: "manual_merge", + label: `merged ${selectorText}`, + }); + + const mergedMask = await mergedObjectMaskForTiles(node, tiles, selectedIndexes); + if (mergedMask) { + const cropped = cropObjectMaskForTile(node, mergedMask.maskImage, mergedMask.maskBox, mergedTile); + if (cropped) { + mergedTile.object_mask = cropped; + mergedTile.object_mask_source = mergedMask.maskData; + } else { + delete mergedTile.object_mask; + delete mergedTile.object_mask_source; + } + } else { + delete mergedTile.object_mask; + delete mergedTile.object_mask_source; + } + + const insertIndex = Math.min(...selectedIndexes); + const remaining = tiles.filter((_tile, index) => !selectedSet.has(index)); + remaining.splice(insertIndex, 0, mergedTile); + writeLayout(node, remaining, insertIndex, [insertIndex]); + node.ttpSmartTileStatus = mergedMask + ? `Merged ${selectedIndexes.length} tile(s) with ${mergedMask.count} mask(s).` + : `Merged ${selectedIndexes.length} tile(s).`; + return true; +} + function sourceStorageKey(node) { const size = imageSourceSize(node); const sizeKey = size ? `${Math.round(size.width)}x${Math.round(size.height)}` : "unknown"; @@ -634,18 +987,100 @@ function paintMaskComponents(canvas, minArea = 24) { return components; } -function maskCropData(canvas, box) { +function slidingMaxHorizontal(values, width, height, radius) { + const output = new Uint8Array(values.length); + for (let y = 0; y < height; y += 1) { + const rowOffset = y * width; + const deque = []; + let head = 0; + let added = -1; + for (let x = 0; x < width; x += 1) { + const limit = Math.min(width - 1, x + radius); + while (added < limit) { + added += 1; + const value = values[rowOffset + added]; + while (deque.length > head && values[rowOffset + deque[deque.length - 1]] <= value) { + deque.pop(); + } + deque.push(added); + } + const minX = x - radius; + while (deque.length > head && deque[head] < minX) { + head += 1; + } + output[rowOffset + x] = deque.length > head ? values[rowOffset + deque[head]] : 0; + } + } + return output; +} + +function slidingMaxVertical(values, width, height, radius) { + const output = new Uint8Array(values.length); + for (let x = 0; x < width; x += 1) { + const deque = []; + let head = 0; + let added = -1; + for (let y = 0; y < height; y += 1) { + const limit = Math.min(height - 1, y + radius); + while (added < limit) { + added += 1; + const value = values[added * width + x]; + while (deque.length > head && values[deque[deque.length - 1] * width + x] <= value) { + deque.pop(); + } + deque.push(added); + } + const minY = y - radius; + while (deque.length > head && deque[head] < minY) { + head += 1; + } + output[y * width + x] = deque.length > head ? values[deque[head] * width + x] : 0; + } + } + return output; +} + +function expandMaskImageData(imageData, width, height, expandPx) { + const radius = Math.max(0, Math.min(2048, Math.round(Number(expandPx) || 0))); + if (radius <= 0) { + return imageData; + } + const source = new Uint8Array(width * height); + const data = imageData.data; + for (let index = 0; index < source.length; index += 1) { + const offset = index * 4; + source[index] = Math.max(data[offset], data[offset + 1], data[offset + 2], data[offset + 3]); + } + const horizontal = slidingMaxHorizontal(source, width, height, radius); + const expanded = slidingMaxVertical(horizontal, width, height, radius); + for (let index = 0; index < expanded.length; index += 1) { + const value = expanded[index]; + const offset = index * 4; + data[offset] = value; + data[offset + 1] = value; + data[offset + 2] = value; + data[offset + 3] = 255; + } + return imageData; +} + +function maskCropData(canvas, box, expandPx = 0) { const width = Math.max(1, Math.round(box.x1 - box.x0)); const height = Math.max(1, Math.round(box.y1 - box.y0)); const crop = document.createElement("canvas"); crop.width = width; crop.height = height; - const context = crop.getContext("2d"); + const context = crop.getContext("2d", { willReadFrequently: true }); context.drawImage(canvas, box.x0, box.y0, width, height, 0, 0, width, height); + const radius = Math.max(0, Math.round(Number(expandPx) || 0)); + if (radius > 0) { + const imageData = context.getImageData(0, 0, width, height); + context.putImageData(expandMaskImageData(imageData, width, height, radius), 0, 0); + } return crop.toDataURL("image/png").split(",", 2)[1]; } -function addPaintMaskTiles(node, tiles, selectedIndex) { +function addPaintMaskTiles(node, tiles, selectedIndex, replaceExisting = false) { const canvas = ensurePaintMaskCanvas(node); const size = imageSourceSize(node); if (!canvas || !size || !paintMaskHasPixels(canvas)) { @@ -658,8 +1093,9 @@ function addPaintMaskTiles(node, tiles, selectedIndex) { return false; } const padding = Math.max(0, Math.round(Number(widgetByName(node, "auto_object_padding")?.value ?? 96))); + const maskExpand = Math.max(0, Math.round(Number(widgetByName(node, "auto_mask_expand")?.value ?? 16))); const defaults = layoutDefaults(node); - const nextTiles = [...tiles]; + const nextTiles = replaceExisting ? [] : [...tiles]; let added = 0; for (const component of components) { if (nextTiles.length >= MAX_TILES) { @@ -686,12 +1122,13 @@ function addPaintMaskTiles(node, tiles, selectedIndex) { importance: 1.0, layer: 5, occlusion_priority: 3200 + nextTiles.length, + object_mask_expand: maskExpand, object_mask: { format: "png_base64", box: [Math.round(box.x0), Math.round(box.y0), Math.round(width), Math.round(height)], width: Math.round(width), height: Math.round(height), - data: maskCropData(canvas, box), + data: maskCropData(canvas, box, maskExpand), }, })); added += 1; @@ -700,14 +1137,63 @@ function addPaintMaskTiles(node, tiles, selectedIndex) { node.ttpSmartTileStatus = `Mask has ${components.length} region(s), but max ${MAX_TILES} tiles is reached.`; return false; } - writeLayout(node, nextTiles, Math.min(nextTiles.length - 1, Math.max(0, selectedIndex + added))); + const nextSelected = replaceExisting + ? Math.max(0, added - 1) + : Math.min(nextTiles.length - 1, Math.max(0, selectedIndex + added)); + writeLayout(node, nextTiles, nextSelected); clearPaintMask(node); node.ttpSmartTilePaintMode = "off"; - node.ttpSmartTileStatus = `Mask to Tile added ${added} tile(s).`; + node.ttpSmartTileStatus = replaceExisting + ? `Mask Replace created ${added} tile(s).` + : `Mask to Tile added ${added} tile(s).`; return true; } -function writeLayout(node, tiles, selectedIndex = 0) { +function selectedTileIndexes(node, count, fallbackIndex = 0) { + const fallback = Math.max(0, Math.min(Math.max(0, count - 1), Number(fallbackIndex ?? 0))); + const raw = Array.isArray(node.ttpSmartTileSelectedIndexes) + ? node.ttpSmartTileSelectedIndexes + : [node.ttpSmartTileSelectedIndex ?? fallback]; + const result = []; + const seen = new Set(); + for (const value of raw) { + const index = Math.round(Number(value)); + if (Number.isFinite(index) && index >= 0 && index < count && !seen.has(index)) { + result.push(index); + seen.add(index); + } + } + if (!result.length && count > 0) { + result.push(fallback); + } + return result.sort((a, b) => a - b); +} + +function setTileSelection(node, index, count, event = null) { + const safeIndex = Math.max(0, Math.min(Math.max(0, count - 1), Math.round(Number(index) || 0))); + const multi = boolEventModifier(event); + if (!multi) { + node.ttpSmartTileSelectedIndex = safeIndex; + node.ttpSmartTileSelectedIndexes = [safeIndex]; + return; + } + const current = selectedTileIndexes(node, count, safeIndex); + const selected = new Set(current); + if (selected.has(safeIndex) && selected.size > 1) { + selected.delete(safeIndex); + } else { + selected.add(safeIndex); + } + const next = [...selected].sort((a, b) => a - b); + node.ttpSmartTileSelectedIndexes = next; + node.ttpSmartTileSelectedIndex = safeIndex; +} + +function boolEventModifier(event) { + return Boolean(event?.shiftKey) || Boolean(event?.metaKey) || Boolean(event?.ctrlKey); +} + +function writeLayout(node, tiles, selectedIndex = 0, selectedIndexes = null) { const normalizedTiles = (Array.isArray(tiles) && tiles.length ? tiles : defaultTiles(node)) .slice(0, MAX_TILES) .map((tile, index) => ({ @@ -729,6 +1215,9 @@ function writeLayout(node, tiles, selectedIndex = 0) { } node.ttpSmartTileLayout = layout; node.ttpSmartTileSelectedIndex = Math.max(0, Math.min(normalizedTiles.length - 1, Number(selectedIndex ?? 0))); + node.ttpSmartTileSelectedIndexes = selectedIndexes === null + ? [node.ttpSmartTileSelectedIndex] + : selectedTileIndexes({ ttpSmartTileSelectedIndexes: selectedIndexes }, normalizedTiles.length, node.ttpSmartTileSelectedIndex); storeLayout(node, normalizedTiles); } @@ -989,18 +1478,26 @@ async function loadSelectedInputImage(node, force = false) { renderEditor(node); } -async function inferSmartTileLayout(node) { - const mode = String(widgetByName(node, "auto_detect_mode")?.value ?? "none"); +async function inferSmartTileLayout(node, options = {}) { + const modeWidget = widgetByName(node, "auto_detect_mode"); + if (options.modeOverride && modeWidget) { + modeWidget.value = String(options.modeOverride); + } + const mode = String(modeWidget?.value ?? "none"); if (mode === "none") { node.ttpSmartTileStatus = "Set auto detect mode to sam3.1 or qwenvl3 before inference."; renderEditor(node); return; } + const fillGaps = options.fillGaps !== false; + const actionLabel = String(options.label || (fillGaps ? "Auto Tile" : "Auto SAM")); const requestWidget = widgetByName(node, "auto_detect_request"); if (requestWidget) { requestWidget.value = Number(requestWidget.value ?? 0) + 1; } - node.ttpSmartTileStatus = `Queued ${mode} inference for this node only...`; + node.ttpSmartTileAutoFillGaps = fillGaps; + node.ttpSmartTileInferenceLabel = actionLabel; + node.ttpSmartTileStatus = `Queued ${actionLabel} (${mode}) inference for this node only...`; renderEditor(node); try { await app.queuePrompt(0, 1, [String(node.id)]); @@ -1025,6 +1522,8 @@ function applyInferenceResult(detail) { if (requestWidget) { requestWidget.value = 0; } + const fillGaps = node.ttpSmartTileAutoFillGaps !== false; + const actionLabel = String(node.ttpSmartTileInferenceLabel || (fillGaps ? "Auto Tile" : "Auto SAM")); const statusParts = []; if (detail.message) { statusParts.push(detail.message); @@ -1035,13 +1534,22 @@ function applyInferenceResult(detail) { try { const layout = JSON.parse(detail.layout_json); if (Array.isArray(layout?.tiles) && layout.tiles.length) { - const filled = fillTileGaps(node, layout.tiles, autoMaxTiles(node)); - writeLayout(node, filled.tiles, 0); - if (filled.added > 0) { - statusParts.push(`Auto Tile added ${filled.added} gap tile(s).`); - } - if (filled.coverage.gaps.length > 0) { - statusParts.push(`${filled.coverage.gaps.length} gap(s) remain.`); + if (fillGaps) { + const filled = fillTileGaps(node, layout.tiles, autoMaxTiles(node)); + writeLayout(node, filled.tiles, 0); + if (filled.added > 0) { + statusParts.push(`Auto Tile added ${filled.added} gap tile(s).`); + } + if (filled.coverage.gaps.length > 0) { + statusParts.push(`${filled.coverage.gaps.length} gap(s) remain.`); + } + } else { + writeLayout(node, layout.tiles, 0); + const coverage = analyzeCoverage(layout.tiles); + statusParts.push(`${actionLabel} kept detected tile(s) without gap filling.`); + if (coverage.gaps.length > 0) { + statusParts.push(`${coverage.gaps.length} gap(s) remain.`); + } } } } catch (error) { @@ -1225,7 +1733,16 @@ function renderEditor(node) { const currentTiles = node.ttpSmartTileLayout?.tiles ?? parseLayout(node); const tiles = currentTiles.map((tile) => normalizeTile(node, tile)); const selectedIndex = Math.max(0, Math.min(tiles.length - 1, Number(node.ttpSmartTileSelectedIndex ?? 0))); - writeLayout(node, tiles, selectedIndex); + const selectedIndexes = selectedTileIndexes(node, tiles.length, selectedIndex); + const selectedSet = new Set(selectedIndexes); + const hasTileMasks = tiles.some((tile) => Boolean(ownObjectMaskData(tile)?.data)); + if (!hasTileMasks && maskOverlayEnabled(node)) { + node.ttpSmartTileShowMasks = false; + } + if (!hasTileMasks && stitchMaskPreviewEnabled(node)) { + node.ttpSmartTilePreviewStitchMask = false; + } + writeLayout(node, tiles, selectedIndex, selectedIndexes); const sourceSize = imageSourceSize(node); const selectedRect = tilePixelRect(node, tiles[selectedIndex]); @@ -1288,6 +1805,26 @@ function renderEditor(node) { stage.append(empty); } + if (sourceSize && hasTileMasks && maskOverlayEnabled(node)) { + const maskOverlay = document.createElement("canvas"); + maskOverlay.title = "Tile object masks"; + maskOverlay.style.cssText = [ + "position:absolute", + "inset:0", + "width:100%", + "height:100%", + "pointer-events:none", + "z-index:8", + ].join(";"); + stage.append(maskOverlay); + renderMaskOverlay(node, maskOverlay, tiles).catch((error) => { + node.ttpSmartTileStatus = error?.message || "Mask overlay failed."; + scheduleCanvas(node); + }); + } else { + node.ttpSmartTileMaskOverlayToken = null; + } + const paintCanvas = ensurePaintMaskCanvas(node); if (paintCanvas) { paintCanvas.style.cssText = [ @@ -1469,7 +2006,10 @@ function renderEditor(node) { y1: nearestSnap(tile.y1, "y", skipIndex), }); - const renderOrder = tiles.map((_tile, index) => index).filter((index) => index !== selectedIndex); + const renderOrder = tiles + .map((_tile, index) => index) + .filter((index) => !selectedSet.has(index)); + renderOrder.push(...selectedIndexes.filter((index) => index !== selectedIndex)); renderOrder.push(selectedIndex); const regionElements = new Map(); @@ -1481,7 +2021,12 @@ function renderEditor(node) { const index = Math.max(0, Math.min(tiles.length - 1, selected)); const dragMode = resizeHit(tiles[index], point) ? "resize" : mode; const targetRegion = regionElements.get(index) ?? region; - node.ttpSmartTileSelectedIndex = index; + if (boolEventModifier(event) && dragMode !== "resize") { + setTileSelection(node, index, tiles.length, event); + renderEditor(node); + return; + } + setTileSelection(node, index, tiles.length); const startPoint = pointFromEvent(event); const startClientX = event.clientX; const startClientY = event.clientY; @@ -1544,10 +2089,11 @@ function renderEditor(node) { for (const index of renderOrder) { const tile = tiles[index]; - const selected = index === selectedIndex; + const selected = selectedSet.has(index); + const primarySelected = index === selectedIndex; const region = document.createElement("div"); region.textContent = String(index + 1); - region.title = `Tile ${index + 1}`; + region.title = `Tile ${index + 1}${selected && !primarySelected ? " selected" : ""}`; region.style.cssText = [ "position:absolute", "box-sizing:border-box", @@ -1558,13 +2104,13 @@ function renderEditor(node) { `top:${tile.y0 * 100}%`, `width:${Math.max(0.1, (tile.x1 - tile.x0) * 100)}%`, `height:${Math.max(0.1, (tile.y1 - tile.y0) * 100)}%`, - "border:" + (selected ? "2px solid #f8fafc" : "1px solid rgba(226,232,240,.55)"), - "background:" + (selected ? "rgba(14,165,233,.28)" : "rgba(15,23,42,.20)"), + "border:" + (primarySelected ? "2px solid #f8fafc" : selected ? "2px solid #38bdf8" : "1px solid rgba(226,232,240,.55)"), + "background:" + (primarySelected ? "rgba(14,165,233,.28)" : selected ? "rgba(14,165,233,.18)" : "rgba(15,23,42,.20)"), "color:#f8fafc", "font-weight:800", "letter-spacing:0", - "box-shadow:" + (selected ? "0 0 0 1px rgba(15,23,42,.85),0 0 16px rgba(14,165,233,.55)" : "none"), - "z-index:" + (selected ? "100" : String(10 + index)), + "box-shadow:" + (primarySelected ? "0 0 0 1px rgba(15,23,42,.85),0 0 16px rgba(14,165,233,.55)" : selected ? "0 0 0 1px rgba(15,23,42,.65)" : "none"), + "z-index:" + (primarySelected ? "100" : selected ? "90" : String(10 + index)), "cursor:move", "user-select:none", "pointer-events:" + (paintMode === "off" ? "auto" : "none"), @@ -1598,7 +2144,7 @@ function renderEditor(node) { const point = pointFromEvent(event); const hits = renderOrder.filter((index) => tileContainsPoint(tiles[index], point)); if (hits.length || tiles.length >= MAX_TILES) { - node.ttpSmartTileSelectedIndex = hits.pop() ?? selectedIndex; + setTileSelection(node, hits.pop() ?? selectedIndex, tiles.length, event); renderEditor(node); return; } @@ -1779,6 +2325,16 @@ function renderEditor(node) { node.ttpSmartTilePaintMode = node.ttpSmartTilePaintMode === "erase" ? "off" : "erase"; renderEditor(node); }); + const moveButton = createButton("Move", () => { + node.ttpSmartTilePaintMode = "off"; + renderEditor(node); + }); + moveButton.title = "Return to tile move and resize mode"; + if (paintMode === "off") { + moveButton.style.background = "#a7f3d0"; + moveButton.style.borderColor = "#34d399"; + moveButton.style.color = "#064e3b"; + } if (paintMode === "paint") { paintButton.style.background = "#38bdf8"; paintButton.style.borderColor = "#7dd3fc"; @@ -1789,8 +2345,34 @@ function renderEditor(node) { eraseButton.style.borderColor = "#e2e8f0"; eraseButton.style.color = "#0f172a"; } + const maskOverlayButton = createButton(maskOverlayEnabled(node) ? "Hide masks" : "Show masks", () => { + node.ttpSmartTileShowMasks = !maskOverlayEnabled(node); + renderEditor(node); + }, !hasTileMasks); + maskOverlayButton.title = "Toggle colored object-mask overlay"; + if (maskOverlayEnabled(node)) { + maskOverlayButton.style.background = "#a7f3d0"; + maskOverlayButton.style.borderColor = "#34d399"; + maskOverlayButton.style.color = "#064e3b"; + } + const stitchMaskButton = createButton(stitchMaskPreviewEnabled(node) ? "Raw masks" : "Stitch mask", () => { + node.ttpSmartTilePreviewStitchMask = !stitchMaskPreviewEnabled(node); + if (node.ttpSmartTilePreviewStitchMask) { + node.ttpSmartTileShowMasks = true; + } + renderEditor(node); + }, !hasTileMasks); + stitchMaskButton.title = "Preview the hardened and feathered mask shape used by replace pasteback"; + if (stitchMaskPreviewEnabled(node)) { + stitchMaskButton.style.background = "#fde68a"; + stitchMaskButton.style.borderColor = "#f59e0b"; + stitchMaskButton.style.color = "#78350f"; + } paintControls.append( paintLabel, + moveButton, + maskOverlayButton, + stitchMaskButton, paintButton, eraseButton, document.createTextNode("px"), @@ -1802,6 +2384,13 @@ function renderEditor(node) { renderEditor(node); } }, !paintMaskHasPixels(ensurePaintMaskCanvas(node)) || tiles.length >= MAX_TILES), + createButton("Mask Replace", () => { + if (addPaintMaskTiles(node, tiles, 0, true)) { + renderEditor(node); + } else { + renderEditor(node); + } + }, !paintMaskHasPixels(ensurePaintMaskCanvas(node))), createButton("Refresh masks", async () => { updateGridValues(); const refreshed = await refreshInheritedMasks(node, tiles, gridMaskMode(node)).catch((error) => { @@ -1833,6 +2422,13 @@ function renderEditor(node) { writeLayout(node, refreshed.tiles, Math.min(selectedIndex, refreshed.tiles.length - 1)); renderEditor(node); }, !tiles.some((tile) => tile?.object_mask_source?.data)), + createButton("Merge masks", async () => { + await mergeSelectedMasks(node, tiles, selectedIndexes, selectedIndex).catch((error) => { + node.ttpSmartTileStatus = error?.message || "Merge masks failed."; + return false; + }); + renderEditor(node); + }, selectedIndexes.length < 2), createButton("Clear mask", () => { clearPaintMask(node); node.ttpSmartTilePaintMode = "off"; @@ -1842,13 +2438,13 @@ function renderEditor(node) { controls.append(paintControls); for (const [index] of tiles.entries()) { - const button = createButton(`T${index + 1}`, () => { - node.ttpSmartTileSelectedIndex = index; + const button = createButton(`T${index + 1}`, (event) => { + setTileSelection(node, index, tiles.length, event); renderEditor(node); }); - if (index === selectedIndex) { - button.style.background = "#38bdf8"; - button.style.borderColor = "#7dd3fc"; + if (selectedSet.has(index)) { + button.style.background = index === selectedIndex ? "#38bdf8" : "#bae6fd"; + button.style.borderColor = index === selectedIndex ? "#7dd3fc" : "#38bdf8"; button.style.color = "#082f49"; } actions.append(button); @@ -1859,6 +2455,14 @@ function renderEditor(node) { syncPaintMaskWidget(node); inferSmartTileLayout(node); }), + createButton("Auto SAM", () => { + syncPaintMaskWidget(node); + inferSmartTileLayout(node, { + fillGaps: false, + label: "Auto SAM", + modeOverride: "sam3.1", + }); + }), createButton("Add tile", () => { const base = tiles[selectedIndex] ?? { x0: 0.25, y0: 0.25, x1: 0.75, y1: 0.75 }; const width = Math.max(0.18, Math.min(0.48, base.x1 - base.x0)); @@ -1884,6 +2488,13 @@ function renderEditor(node) { writeLayout(node, tiles.filter((_tile, index) => index !== selectedIndex), Math.max(0, selectedIndex - 1)); renderEditor(node); }, tiles.length <= 1), + createButton("Merge tiles", async () => { + await mergeSelectedTiles(node, tiles, selectedIndexes, selectedIndex).catch((error) => { + node.ttpSmartTileStatus = error?.message || "Merge tiles failed."; + return false; + }); + renderEditor(node); + }, selectedIndexes.length < 2), createButton("Fill gaps", () => { const filled = fillTileGaps(node, tiles); node.ttpSmartTileStatus = filled.coverage.gaps.length @@ -1902,7 +2513,9 @@ function renderEditor(node) { const status = document.createElement("div"); status.style.cssText = "display:flex;gap:10px;flex-wrap:wrap;opacity:.76;"; const selectedLabel = selectedRect - ? `T${selectedIndex + 1}: x${selectedRect.x0}, y${selectedRect.y0}, ${selectedRect.width}x${selectedRect.height}` + ? selectedIndexes.length > 1 + ? `${selectedIndexes.map((index) => `T${index + 1}`).join(",")} selected / active T${selectedIndex + 1}` + : `T${selectedIndex + 1}: x${selectedRect.x0}, y${selectedRect.y0}, ${selectedRect.width}x${selectedRect.height}` : `T${selectedIndex + 1}`; const coverageLabel = coverage.gaps.length ? `${coverage.gaps.length} gap(s), ${(coverage.uncoveredArea * 100).toFixed(2)}%`