Merge smart_tile into main

This commit is contained in:
ttplanet
2026-07-07 16:27:39 +08:00
5 changed files with 1805 additions and 57 deletions
+17 -1
View File
@@ -40,10 +40,15 @@ Useful editor actions:
- `Replace grid`: replace the whole layout with an even grid.
- `Grid in T#`: subdivide the selected tile without rebuilding the whole layout.
- `Show masks` / `Hide masks`: toggle a colored overlay for tile object masks in the editor.
- `Stitch mask` / `Raw masks`: preview the hardened and feathered replace-paste mask shape without changing saved masks.
- `Mask to Tile`: turn painted regions into object tiles.
- `Merge masks`: Shift/Cmd/Ctrl-select multiple tiles, then union their cropped object masks back onto the selected tiles.
- `Merge tiles`: Shift/Cmd/Ctrl-select multiple tiles, then replace them with one larger tile; existing masks are unioned into the merged tile when available.
- `Refresh masks`: after manually moving sub-tiles from a masked tile, re-crop the inherited masks to the current tile boxes.
- `Fill gaps`: add background tiles for uncovered areas.
- `Auto Tile`: run SAM3.1 or QwenVL3 detection and write the detected layout back into the editor.
- `Auto SAM`: run SAM3.1 detection and keep only detected object tiles without automatic gap filling.
When subdividing a masked tile with `Grid in`, the Mask mode can crop the original object mask into child tiles or skip empty mask children. The child tiles keep a parent mask source, so `Refresh masks` can update them after manual edits.
@@ -55,6 +60,9 @@ When subdividing a masked tile with `Grid in`, the Mask mode can crop the origin
| `TTP Smart Tile Set Preview` | Preview a tile set as a contact sheet or one selected tile. |
| `TTP QwenVL3 Local Loader` | Load a local QwenVL tagging model from `ComfyUI/models/text_encoders`. |
| `TTP Smart Tile QwenVL Prompt Set Builder` | Generate per-tile prompts before loop processing. |
| `TTP Smart Tile Prompt Override` | Filter selected tiles and replace, prepend, append, or find/replace per-tile prompts before sampling. |
| `TTP Smart Tile Composite Override` | Filter selected tiles and set composite mode, replace shape, layer, priority, occlusion priority, blend, and importance before assembly. |
| `TTP Smart Tile Stack Order` | Put tiles such as `T1,T3,T2` at the front in that order, leaving all unlisted tiles behind. |
| `TTP Smart Tile Semantic Rank` | Classify tiles and write recommended layer, priority, scale weight, and composite metadata. |
| `TTP Smart Tile Loop Source` | Output one tile at a time for VAE Encode / sampler / VAE Decode. |
| `TTP Smart Tile Loop Collect` | Collect each processed tile back into the same tile set. |
@@ -68,11 +76,14 @@ When subdividing a masked tile with `Grid in`, the Mask mode can crop the origin
```text
TTP Smart Tile Interactive Crop
-> TTP Smart Tile QwenVL Prompt Set Builder (optional)
-> TTP Smart Tile Prompt Override (optional)
-> TTP Smart Tile Composite Override (optional)
-> TTP Smart Tile Semantic Rank (optional)
-> TTP Smart Tile Loop Source
-> VAE Encode / Sampler / VAE Decode
-> TTP Smart Tile Loop Collect
-> TTP Smart Tile Semantic Rank (optional final refresh)
-> TTP Smart Tile Stack Order (optional final manual order)
-> TTP Smart Tile Output Size Estimate (optional)
-> TTP Smart Tile Assemble
-> TTP Smart Tile Save Final Image
@@ -111,9 +122,12 @@ This workflow is the recommended full Smart Tile loop. It uses `TTP Smart Tile I
| Node / 节点 | Example setting / 示例设置 | Notes / 说明 |
|---|---|---|
| `Interactive Crop` | `auto_detect_mode=sam3.1`, `default_pad=32`, `default_blend=32`, `auto_object_padding=64`, `auto_max_tiles=16` | Use `Auto Tile` to create an object-aware layout, then set `auto_detect_mode=none` if you want to freeze manual edits. / 用 `Auto Tile` 生成语义分块;如果之后要固定手动编辑结果,可以把 `auto_detect_mode` 改回 `none`。 |
| `Interactive Crop` | `auto_detect_mode=sam3.1`, `default_pad=32`, `default_blend=32`, `auto_object_padding=64`, `auto_mask_expand=16`, `auto_max_tiles=16` | Use `Auto Tile` to create an object-aware layout, then set `auto_detect_mode=none` if you want to freeze manual edits. `auto_mask_expand` dilates object masks for edit pasteback; set it to `0` for exact masks. / 用 `Auto Tile` 生成语义分块;如果之后要固定手动编辑结果,可以把 `auto_detect_mode` 改回 `none`。`auto_mask_expand` 会膨胀对象蒙版,适合 edit 贴回;设为 `0` 则保留精确蒙版。 |
| `QwenVL3 Local Loader` | `qwen3vl_4b_fp8_scaled.safetensors` | Replace this with the QwenVL model installed in `ComfyUI/models/text_encoders`. / 按你本机 `models/text_encoders` 里的 QwenVL 模型替换。 |
| `QwenVL Prompt Set Builder` | `reference_image_mode=first_message`, `prompt_preset=tile_img2img_prompt`, `output_language=chinese` | Builds prompts before the tile loop, then `Loop Source.prompt` feeds the text encoder. / 在循环前一次性生成每块提示词,然后由 `Loop Source.prompt` 送入文本编码。 |
| `Prompt Override` | `selector_type=label`, `selector=face`, `prompt_mode=replace` | Rewrites only matched tile prompts. Use `drop_unmatched` when only selected regions should run through an edit sampler and Assemble should keep the rest from `source_image`. / 只改写匹配 tile 的提示词;局部 edit 时可用 `drop_unmatched`,其余区域由 Assemble 的 `source_image` 底图保留。 |
| `Composite Override` | `selector_type=index`, `selector=T3`, `composite_mode=replace_tile`, `replace_tile_shape=mask_first`, `occlusion_priority_mode=max` | Puts selected tiles above lower layers and makes them hard-replace the canvas. `mask_first` uses an object mask when present and hardens the mask interior like an inpaint stitch; `tile_box_first` ignores the mask and replaces the full tile box. / 把指定 tile 放到更高层并硬替换画布;`mask_first` 有 mask 就按 mask 替换,并像 inpaint stitch 一样硬化 mask 主体;`tile_box_first` 即使有 mask 也按完整 tile 方块替换。 |
| `Stack Order` | `tile_order=T1,T3,T2`, `top_order=first_on_top`, `composite_mode=replace_tile` | Makes the listed tiles the front stack in the typed order while all unlisted tiles stay behind. Place it after the final `Semantic Rank` so manual order wins. The tiles keep their own `blend` and mask feathering. / 按填写顺序把这些 tile 放到最前面,未填写的 tile 都在后面;建议放在最终 `Semantic Rank` 后面,让手动顺序最后生效;tile 仍使用各自的 `blend` 和 mask 羽化。 |
| `Image Upscale Prep` | `scale=2.5`, `round_to=8`, `max_megapixels=1.5`, `use_upscale_model=true` | Enlarges each tile before img2img while keeping each tile under the megapixel cap. / 采样前放大每块 tile,同时用百万像素上限控制显存。 |
| `Loop Source` / `Loop Collect` | `Process All Tiles` workflow | The tile count can be 4, 8, 16, or any layout count; no manual sampler duplication is needed. / 分块数量可以变化,不需要手动复制多套 sampler。 |
| `Output Size Estimate` | `focus_weighted` | Estimates the final canvas from processed tile sizes, giving more influence to focus/detail tiles. / 根据处理后的 tile 尺寸估算最终画布,并更重视细节块。 |
@@ -172,6 +186,8 @@ Assembly options worth starting with:
| `assemble_device` | `auto` or `gpu` | Uses GPU paste/weight accumulation when available. |
| `base_canvas_mode` | `black` when you do not want source pixels underneath | Prevents the original image from showing through uncovered or low-weight areas. |
| `auto_composite_policy` | `safe_auto` | Keeps background/context low and promotes focus/detail tiles. |
| `auto_composite_policy` | `replace_tile` | For edit workflows, hard-replaces selected tile regions over the base canvas. Combine with `replace_tile_shape=mask_first` for mask priority, or `tile_box_first` for full tile-box replacement. |
| `replace_tile_shape` | `mask_first` or `tile_box_first` | Controls replace-tile shape priority. `mask_first` uses object masks when available and hardens soft mask interiors so only the edge blends; `tile_box_first` ignores masks and pastes the full tile box. |
| `small_tile_on_top` | `true` for detail workflows | Helps small face/eye/text tiles win overlaps against larger body/context tiles. |
| `mask_blend_mode` | `auto` or `mask_feather` | Uses object masks when present and feathered rectangular masks otherwise. |
| `color_correction` | `off` first, then `reinhard_lab`, `mkl_lab`, or `histogram` as needed | Uses ComfyUI's official Transfer Color logic; reference defaults to the source image. |
+706 -20
View File
@@ -454,7 +454,7 @@ def _ttp_normalize_tile_box(tile, image_width, image_height, defaults):
normalized["object_mask"] = dict(tile["object_mask"])
if isinstance(tile.get("object_mask_source"), dict):
normalized["object_mask_source"] = dict(tile["object_mask_source"])
for key in ("caption", "prompt", "negative", "semantic_category", "semantic_role", "recommended_composite_mode"):
for key in ("caption", "prompt", "negative", "semantic_category", "semantic_role", "recommended_composite_mode", "composite_mode", "replace_tile_shape"):
if key in tile:
normalized[key] = str(tile.get(key, ""))
for key in (
@@ -463,6 +463,7 @@ def _ttp_normalize_tile_box(tile, image_width, image_height, defaults):
"recommended_layer",
"recommended_priority",
"recommended_occlusion_priority",
"object_mask_expand",
):
if key in tile:
normalized[key] = _ttp_safe_float(tile.get(key), 0.0)
@@ -691,7 +692,20 @@ def _ttp_mask_tensor_to_pil_list(masks, image_width, image_height):
return result
def _ttp_encode_object_mask_data(mask_images, mask_indices, box):
def _ttp_expand_l_mask(mask, expand_px):
expand_px = max(0, min(2048, int(round(float(expand_px or 0)))))
if expand_px <= 0:
return mask.convert("L")
expanded = mask.convert("L")
remaining = expand_px
while remaining > 0:
step = min(remaining, 127)
expanded = expanded.filter(ImageFilter.MaxFilter(step * 2 + 1))
remaining -= step
return expanded
def _ttp_encode_object_mask_data(mask_images, mask_indices, box, mask_expand=0):
if not mask_images or not mask_indices:
return None
x0, y0, x1, y1 = [int(round(value)) for value in box]
@@ -705,6 +719,9 @@ def _ttp_encode_object_mask_data(mask_images, mask_indices, box):
if cropped.getbbox() is None:
continue
combined = ImageChops.lighter(combined, cropped)
if combined.getbbox() is None:
return None
combined = _ttp_expand_l_mask(combined, mask_expand)
if combined.getbbox() is None:
return None
buffer = BytesIO()
@@ -728,12 +745,36 @@ def _ttp_decode_object_mask_data(mask_data):
return None
def _ttp_tile_object_mask_array(tile, out_width, out_height, mask_blend_mode="mask_feather", feather=0):
def _ttp_harden_stitch_mask_array(mask_array, feather=0, threshold=0.05):
mask_2d = np.clip(np.asarray(mask_array, dtype=np.float32), 0.0, 1.0)
hard = (mask_2d > float(threshold)).astype(np.float32)
if float(hard.max()) <= 0.0:
return hard[:, :, None]
feather = max(0, int(round(float(feather))))
if feather <= 0:
return hard[:, :, None]
hard_pil = Image.fromarray((hard * 255.0).astype(np.uint8), mode="L")
expand_px = max(1, min(64, int(round(float(feather) * 0.25))))
hard_pil = hard_pil.filter(ImageFilter.MaxFilter(expand_px * 2 + 1))
blur_radius = max(0.75, min(64.0, float(feather) * 0.25))
softened = hard_pil.filter(ImageFilter.GaussianBlur(radius=blur_radius))
softened_array = np.array(softened).astype(np.float32) / 255.0
return np.maximum(hard, softened_array)[:, :, None]
def _ttp_tile_object_mask_array(tile, out_width, out_height, mask_blend_mode="mask_feather", feather=0, hard_stitch=False, prefer_own_mask=False):
if str(mask_blend_mode) == "off":
return None
mask_data = tile.get("object_mask_source")
if not isinstance(mask_data, dict) or mask_data.get("format") != "png_base64":
if bool(prefer_own_mask):
mask_data = tile.get("object_mask")
if not isinstance(mask_data, dict) or mask_data.get("format") != "png_base64":
mask_data = tile.get("object_mask_source")
else:
mask_data = tile.get("object_mask_source")
if not isinstance(mask_data, dict) or mask_data.get("format") != "png_base64":
mask_data = tile.get("object_mask")
mask_pil = _ttp_decode_object_mask_data(mask_data)
if mask_pil is None:
return None
@@ -757,6 +798,10 @@ def _ttp_tile_object_mask_array(tile, out_width, out_height, mask_blend_mode="ma
(paste_x + clip_left, paste_y + clip_top),
)
source_mask = source_mask.resize((out_width, out_height), Image.Resampling.BILINEAR)
if hard_stitch:
source_array = np.array(source_mask).astype(np.float32) / 255.0
stitch_feather = feather if str(mask_blend_mode) in ("auto", "mask_feather") else 0
return _ttp_harden_stitch_mask_array(source_array, stitch_feather)
if str(mask_blend_mode) in ("auto", "mask_feather"):
min_radius = min(12.0, max(0.75, min(int(out_width), int(out_height)) * 0.006))
radius = max(min_radius, float(feather) * 0.35)
@@ -820,6 +865,7 @@ def _ttp_tile_semantic_text(tile):
"prompt",
"prompt_tag",
"prompt_source",
"composite_mode",
"object_id",
"semantic_category",
"semantic_role",
@@ -999,12 +1045,39 @@ def _ttp_weighted_percentile(values, weights=None, percentile=0.7):
return float(value_array[index])
def _ttp_tile_composite_mode(tile):
return str(tile.get("composite_mode", tile.get("recommended_composite_mode", "")) or "").strip().lower()
def _ttp_is_replace_tile_mode(mode):
return str(mode or "").strip().lower() in ("replace_tile", "hard_replace", "hard_replace_tile")
def _ttp_normalize_replace_tile_shape(value, default="mask_first"):
default = str(default or "mask_first").strip().lower()
if default not in ("mask_first", "tile_box_first"):
default = "mask_first"
mode = str(value or default).strip().lower()
if mode in ("tile_box_first", "box_first", "rectangle_first", "rect_first", "square_first", "tile_box", "rectangle", "rect", "box", "square"):
return "tile_box_first"
if mode in ("mask_first", "mask", "object_mask", "mask_priority"):
return "mask_first"
return default
def _ttp_tile_replace_shape(tile, default="mask_first"):
mode = str(tile.get("replace_tile_shape", "") or "").strip().lower()
if not mode or mode == "keep":
return _ttp_normalize_replace_tile_shape(default)
return _ttp_normalize_replace_tile_shape(mode, default)
def _ttp_should_soft_overlay_tile(tile, policy, is_large_tile, tile_area_ratio, large_tile_area_threshold):
policy = str(policy or "safe_auto")
if policy in ("strict_layer", "replace_object") or is_large_tile:
tile_mode = _ttp_tile_composite_mode(tile)
if policy in ("strict_layer", "replace_object", "replace_tile") or _ttp_is_replace_tile_mode(tile_mode) or is_large_tile:
return False
recommended_mode = str(tile.get("recommended_composite_mode", "") or "").lower()
if recommended_mode == "soft_overlay":
if tile_mode == "soft_overlay":
return True
text = _ttp_tile_semantic_text(tile)
if _ttp_is_context_tile_text(text):
@@ -1558,6 +1631,7 @@ def _ttp_boxes_to_auto_layout(
default_pad=128,
default_blend=64,
object_padding=96,
mask_expand=16,
max_tiles=16,
include_background=True,
allow_object_overlap=True,
@@ -1636,9 +1710,11 @@ def _ttp_boxes_to_auto_layout(
"recommended_occlusion_priority": 0.0,
"recommended_composite_mode": "context",
})
object_mask = _ttp_encode_object_mask_data(mask_images, item.get("mask_indices", []), box)
object_mask = _ttp_encode_object_mask_data(mask_images, item.get("mask_indices", []), box, mask_expand=mask_expand)
if object_mask is not None:
tile["object_mask"] = object_mask
if int(mask_expand) > 0:
tile["object_mask_expand"] = int(mask_expand)
tiles.append(tile)
if len(tiles) == (1 if include_background else 0):
@@ -2095,6 +2171,8 @@ def _ttp_tile_set_fingerprint(tile_set):
"recommended_priority": tile.get("recommended_priority", ""),
"recommended_occlusion_priority": tile.get("recommended_occlusion_priority", ""),
"recommended_composite_mode": tile.get("recommended_composite_mode", ""),
"composite_mode": tile.get("composite_mode", ""),
"replace_tile_shape": tile.get("replace_tile_shape", ""),
"object_mask": _ttp_mask_data_signature(tile.get("object_mask")),
"object_mask_source": _ttp_mask_data_signature(tile.get("object_mask_source")),
"prompt": tile.get("prompt", ""),
@@ -2970,6 +3048,7 @@ def _ttp_run_sam3_auto_layout(
default_pad,
default_blend,
object_padding,
mask_expand,
max_tiles,
allow_object_overlap,
paint_mask_payload=None,
@@ -3012,6 +3091,7 @@ def _ttp_run_sam3_auto_layout(
default_pad=int(default_pad),
default_blend=int(default_blend),
object_padding=int(object_padding),
mask_expand=int(mask_expand),
max_tiles=int(max_tiles),
include_background=False,
allow_object_overlap=bool(allow_object_overlap),
@@ -3147,6 +3227,7 @@ def _ttp_run_qwenvl3_auto_layout(
default_pad,
default_blend,
object_padding,
mask_expand,
max_tiles,
allow_object_overlap,
paint_mask_payload=None,
@@ -3184,6 +3265,7 @@ def _ttp_run_qwenvl3_auto_layout(
default_pad=int(default_pad),
default_blend=int(default_blend),
object_padding=int(object_padding),
mask_expand=int(mask_expand),
max_tiles=int(max_tiles),
include_background=False,
allow_object_overlap=bool(allow_object_overlap),
@@ -3200,6 +3282,7 @@ def _ttp_run_paint_mask_auto_layout(
default_pad,
default_blend,
object_padding,
mask_expand,
max_tiles,
allow_object_overlap,
):
@@ -3215,6 +3298,7 @@ def _ttp_run_paint_mask_auto_layout(
default_pad=int(default_pad),
default_blend=int(default_blend),
object_padding=int(object_padding),
mask_expand=int(mask_expand),
max_tiles=int(max_tiles),
include_background=False,
allow_object_overlap=bool(allow_object_overlap),
@@ -3490,6 +3574,7 @@ class TTP_Smart_Tile_Interactive_Crop_Experimental:
}),
"allow_object_overlap": ("BOOLEAN", {"default": True}),
"auto_object_padding": ("INT", {"default": 96, "min": 0, "max": 2048, "step": 8}),
"auto_mask_expand": ("INT", {"default": 16, "min": 0, "max": 2048, "step": 4}),
"auto_max_tiles": ("INT", {"default": 16, "min": 1, "max": 64, "step": 1}),
"auto_paint_mask": ("STRING", {"default": "", "multiline": False}),
},
@@ -3526,6 +3611,7 @@ class TTP_Smart_Tile_Interactive_Crop_Experimental:
"auto_prompt": kwargs.get("auto_prompt", ""),
"allow_object_overlap": kwargs.get("allow_object_overlap", True),
"auto_object_padding": kwargs.get("auto_object_padding", 96),
"auto_mask_expand": kwargs.get("auto_mask_expand", 16),
"auto_max_tiles": kwargs.get("auto_max_tiles", 16),
"auto_paint_mask": hashlib.sha256(str(kwargs.get("auto_paint_mask", "") or "").encode("utf-8")).hexdigest()[:16],
"source_image": type(kwargs.get("source_image")).__name__ if kwargs.get("source_image") is not None else None,
@@ -3560,6 +3646,7 @@ class TTP_Smart_Tile_Interactive_Crop_Experimental:
auto_prompt="person, face, hands, eyes, text, foreground object, important object",
allow_object_overlap=True,
auto_object_padding=96,
auto_mask_expand=16,
auto_max_tiles=16,
auto_paint_mask="",
source_image=None,
@@ -3585,6 +3672,7 @@ class TTP_Smart_Tile_Interactive_Crop_Experimental:
int(default_pad),
int(default_blend),
int(auto_object_padding),
int(auto_mask_expand),
int(auto_max_tiles),
bool(allow_object_overlap),
auto_paint_mask,
@@ -3598,6 +3686,7 @@ class TTP_Smart_Tile_Interactive_Crop_Experimental:
int(default_pad),
int(default_blend),
int(auto_object_padding),
int(auto_mask_expand),
int(auto_max_tiles),
bool(allow_object_overlap),
auto_paint_mask,
@@ -3610,6 +3699,7 @@ class TTP_Smart_Tile_Interactive_Crop_Experimental:
int(default_pad),
int(default_blend),
int(auto_object_padding),
int(auto_mask_expand),
int(auto_max_tiles),
bool(allow_object_overlap),
)
@@ -4087,6 +4177,564 @@ class TTP_Smart_Tile_QwenVL_Prompt_Set_Builder_Experimental:
return (next_tile_set, prompt_set_json, summary)
class TTP_Smart_Tile_Prompt_Override_Experimental:
@classmethod
def INPUT_TYPES(cls):
text_modes = ["keep", "replace", "prepend", "append", "find_replace", "clear"]
return {
"required": {
"tile_set": ("TTP_SMART_TILE_SET", {"forceInput": True}),
"selector_type": ([
"all",
"index",
"name",
"label",
"prompt_tag",
"source",
"semantic_category",
"semantic_role",
"prompt_contains",
"regex",
], {"default": "label"}),
"selector": ("STRING", {"default": "", "multiline": False}),
"unmatched_mode": (["keep", "drop_unmatched"], {"default": "keep"}),
"prompt_mode": (text_modes, {"default": "replace"}),
"prompt_text": ("STRING", {"default": "", "multiline": True, "dynamicPrompts": False}),
"negative_mode": (text_modes, {"default": "keep"}),
"negative_text": ("STRING", {"default": "", "multiline": True, "dynamicPrompts": False}),
"find_text": ("STRING", {"default": "", "multiline": False, "dynamicPrompts": False}),
}
}
RETURN_TYPES = ("TTP_SMART_TILE_SET", "STRING", "STRING")
RETURN_NAMES = ("tile_set", "override_json", "summary")
FUNCTION = "override_prompts"
CATEGORY = "TTP/Smart Tile"
@staticmethod
def _selector_terms(selector):
return [
part.strip()
for part in re.split(r"[,;,、\n\r]+", str(selector or ""))
if part.strip()
]
@staticmethod
def _selector_index_order(selector, count):
indexes = []
seen = set()
for term in TTP_Smart_Tile_Prompt_Override_Experimental._selector_terms(selector):
cleaned = term.strip()
t_range = re.match(r"^[tT]\s*(\d+)\s*-\s*[tT]?\s*(\d+)$", cleaned)
zero_range = re.match(r"^(\d+)\s*-\s*(\d+)$", cleaned)
t_single = re.match(r"^[tT]\s*(\d+)$", cleaned)
zero_single = re.match(r"^\d+$", cleaned)
if t_range:
start = int(t_range.group(1)) - 1
end = int(t_range.group(2)) - 1
elif zero_range:
start = int(zero_range.group(1))
end = int(zero_range.group(2))
elif t_single:
start = end = int(t_single.group(1)) - 1
elif zero_single:
start = end = int(cleaned)
else:
continue
if end < start:
start, end = end, start
for index in range(start, end + 1):
if 0 <= index < count and index not in seen:
indexes.append(index)
seen.add(index)
return indexes
@staticmethod
def _selector_index_set(selector, count):
indexes = set()
for index in TTP_Smart_Tile_Prompt_Override_Experimental._selector_index_order(selector, count):
indexes.add(index)
return indexes
@staticmethod
def _match_tile(tile, index, count, selector_type, selector):
selector_type = str(selector_type or "label")
if selector_type == "all":
return True
if selector_type == "index":
return index in TTP_Smart_Tile_Prompt_Override_Experimental._selector_index_set(selector, count)
terms = [term.lower() for term in TTP_Smart_Tile_Prompt_Override_Experimental._selector_terms(selector)]
if not terms:
return False
fields = {
"name": [tile.get("name", "")],
"label": [tile.get("label", "")],
"prompt_tag": [tile.get("prompt_tag", "")],
"source": [tile.get("source", "")],
"semantic_category": [tile.get("semantic_category", "")],
"semantic_role": [tile.get("semantic_role", "")],
"prompt_contains": [tile.get("prompt", ""), tile.get("negative", ""), tile.get("caption", "")],
}
searchable = fields.get(selector_type)
if selector_type == "regex":
searchable = [
str(index),
str(index + 1),
tile.get("name", ""),
tile.get("label", ""),
tile.get("prompt_tag", ""),
tile.get("source", ""),
tile.get("semantic_category", ""),
tile.get("semantic_role", ""),
tile.get("caption", ""),
tile.get("prompt", ""),
tile.get("negative", ""),
]
text = "\n".join(str(value or "") for value in searchable)
try:
return bool(re.search(str(selector or ""), text, flags=re.IGNORECASE))
except re.error as exc:
raise ValueError(f"Invalid Smart Tile prompt override regex: {exc}") from exc
if searchable is None:
searchable = [
tile.get("name", ""),
tile.get("label", ""),
tile.get("prompt_tag", ""),
tile.get("source", ""),
tile.get("caption", ""),
tile.get("prompt", ""),
]
text = "\n".join(str(value or "").lower() for value in searchable)
return any(term in text for term in terms)
@staticmethod
def _join_text(prefix, suffix):
prefix = str(prefix or "").strip()
suffix = str(suffix or "").strip()
if prefix and suffix:
return f"{prefix}\n{suffix}"
return prefix or suffix
@staticmethod
def _apply_text_mode(current, mode, text, find_text):
current = str(current or "")
mode = str(mode or "keep")
text = str(text or "").strip()
find_text = str(find_text or "")
if mode == "keep":
return current
if mode == "clear":
return ""
if mode == "replace":
return text
if mode == "prepend":
return TTP_Smart_Tile_Prompt_Override_Experimental._join_text(text, current)
if mode == "append":
return TTP_Smart_Tile_Prompt_Override_Experimental._join_text(current, text)
if mode == "find_replace":
return current.replace(find_text, text) if find_text else current
return current
@staticmethod
def _position_for_tile(tile):
box = tile.get("paste_box", tile.get("core_box", [0, 0, 1, 1]))
x, y, w, h = [int(round(float(value))) for value in box[:4]]
return (x, y, x + max(1, w), y + max(1, h))
def override_prompts(
self,
tile_set,
selector_type="label",
selector="",
unmatched_mode="keep",
prompt_mode="replace",
prompt_text="",
negative_mode="keep",
negative_text="",
find_text="",
):
tile_images, _tile_meta, tiles_info = _ttp_validate_tile_set(tile_set)
next_tile_set = _ttp_clone_tile_set(tile_set)
next_tiles = next_tile_set["tile_meta"]["tiles"]
next_images = list(next_tile_set.get("tile_images", []))
next_positions = list(next_tile_set.get("positions", []))
kept_tiles = []
kept_images = []
kept_positions = []
records = []
matched_count = 0
drop_unmatched = str(unmatched_mode or "keep") == "drop_unmatched"
count = len(tiles_info)
for index, tile in enumerate(next_tiles):
matched = self._match_tile(tile, index, count, selector_type, selector)
if not matched and drop_unmatched:
continue
updated = dict(tile)
if matched:
matched_count += 1
before_prompt = str(updated.get("prompt", ""))
before_negative = str(updated.get("negative", ""))
updated["prompt"] = self._apply_text_mode(before_prompt, prompt_mode, prompt_text, find_text)
updated["negative"] = self._apply_text_mode(before_negative, negative_mode, negative_text, find_text)
existing_source = str(updated.get("prompt_source", "") or "")
updated["prompt_source"] = f"{existing_source}+override" if existing_source else "override"
updated["prompt_override"] = True
updated["prompt_override_selector_type"] = str(selector_type or "")
updated["prompt_override_selector"] = str(selector or "")
records.append({
"index": int(index),
"name": updated.get("name", f"tile_{index}"),
"label": updated.get("label", ""),
"prompt_before": before_prompt,
"prompt_after": updated["prompt"],
"negative_before": before_negative,
"negative_after": updated["negative"],
})
kept_tiles.append(updated)
kept_images.append(next_images[index] if index < len(next_images) else tile_images[index])
if index < len(next_positions):
kept_positions.append(next_positions[index])
else:
kept_positions.append(self._position_for_tile(updated))
if not kept_tiles:
raise ValueError("Smart Tile prompt override kept no tiles. Change selector or unmatched_mode.")
for new_index, tile in enumerate(kept_tiles):
tile["id"] = new_index
next_tile_set["tile_meta"]["tiles"] = kept_tiles
next_tile_set["tile_images"] = kept_images
next_tile_set["positions"] = kept_positions
override_json = json.dumps({
"type": "ttp_smart_tile_prompt_override",
"selector_type": str(selector_type or ""),
"selector": str(selector or ""),
"unmatched_mode": str(unmatched_mode or "keep"),
"prompt_mode": str(prompt_mode or "keep"),
"negative_mode": str(negative_mode or "keep"),
"matched": int(matched_count),
"kept": int(len(kept_tiles)),
"dropped": int(count - len(kept_tiles)),
"tiles": records,
}, ensure_ascii=False, indent=2)
summary_lines = [
f"matched={matched_count} kept={len(kept_tiles)} dropped={count - len(kept_tiles)} selector={selector_type}:{selector}"
]
summary_lines.extend(
f'{record["index"]}:{record["name"]} -> {_ttp_compact_text(record["prompt_after"], 160)}'
for record in records
)
return (next_tile_set, override_json, "\n".join(summary_lines))
class TTP_Smart_Tile_Composite_Override_Experimental:
@classmethod
def INPUT_TYPES(cls):
number_modes = ["keep", "set", "add", "max", "min"]
return {
"required": {
"tile_set": ("TTP_SMART_TILE_SET", {"forceInput": True}),
"selector_type": ([
"all",
"index",
"name",
"label",
"prompt_tag",
"source",
"semantic_category",
"semantic_role",
"prompt_contains",
"regex",
], {"default": "index"}),
"selector": ("STRING", {"default": "T1", "multiline": False}),
"unmatched_mode": (["keep", "drop_unmatched"], {"default": "keep"}),
"composite_mode": (["keep", "replace_tile", "soft_overlay", "blend", "replace", "context"], {"default": "replace_tile"}),
"replace_tile_shape": (["keep", "mask_first", "tile_box_first"], {"default": "keep"}),
"layer_mode": (number_modes, {"default": "max"}),
"layer": ("FLOAT", {"default": 10.0, "min": -1000000.0, "max": 1000000.0, "step": 1.0}),
"priority_mode": (number_modes, {"default": "max"}),
"priority": ("FLOAT", {"default": 200.0, "min": -1000000.0, "max": 1000000.0, "step": 1.0}),
"occlusion_priority_mode": (number_modes, {"default": "max"}),
"occlusion_priority": ("FLOAT", {"default": 50000.0, "min": -1000000.0, "max": 1000000.0, "step": 10.0}),
"blend_mode": (number_modes, {"default": "keep"}),
"blend": ("INT", {"default": 32, "min": 0, "max": 2048, "step": 1}),
"importance_mode": (number_modes, {"default": "keep"}),
"importance": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 10.0, "step": 0.05}),
}
}
RETURN_TYPES = ("TTP_SMART_TILE_SET", "STRING", "STRING")
RETURN_NAMES = ("tile_set", "composite_json", "summary")
FUNCTION = "override_composite"
CATEGORY = "TTP/Smart Tile"
@staticmethod
def _apply_number_mode(current, mode, value):
mode = str(mode or "keep")
current = _ttp_safe_float(current, 0.0)
value = _ttp_safe_float(value, 0.0)
if mode == "keep":
return current
if mode == "set":
return value
if mode == "add":
return current + value
if mode == "max":
return max(current, value)
if mode == "min":
return min(current, value)
return current
@staticmethod
def _position_for_tile(tile):
return TTP_Smart_Tile_Prompt_Override_Experimental._position_for_tile(tile)
def override_composite(
self,
tile_set,
selector_type="index",
selector="T1",
unmatched_mode="keep",
composite_mode="replace_tile",
replace_tile_shape="keep",
layer_mode="max",
layer=10.0,
priority_mode="max",
priority=200.0,
occlusion_priority_mode="max",
occlusion_priority=50000.0,
blend_mode="keep",
blend=32,
importance_mode="keep",
importance=1.0,
):
tile_images, _tile_meta, tiles_info = _ttp_validate_tile_set(tile_set)
next_tile_set = _ttp_clone_tile_set(tile_set)
next_tiles = next_tile_set["tile_meta"]["tiles"]
next_images = list(next_tile_set.get("tile_images", []))
next_positions = list(next_tile_set.get("positions", []))
kept_tiles = []
kept_images = []
kept_positions = []
records = []
matched_count = 0
drop_unmatched = str(unmatched_mode or "keep") == "drop_unmatched"
count = len(tiles_info)
composite_mode = str(composite_mode or "keep")
replace_tile_shape = str(replace_tile_shape or "keep")
for index, tile in enumerate(next_tiles):
matched = TTP_Smart_Tile_Prompt_Override_Experimental._match_tile(tile, index, count, selector_type, selector)
if not matched and drop_unmatched:
continue
updated = dict(tile)
if matched:
matched_count += 1
before = {
"composite_mode": updated.get("composite_mode", updated.get("recommended_composite_mode", "")),
"replace_tile_shape": updated.get("replace_tile_shape", ""),
"layer": updated.get("layer", 0.0),
"priority": updated.get("priority", 0.0),
"occlusion_priority": updated.get("occlusion_priority", 0.0),
"blend": updated.get("blend", 0),
"importance": updated.get("importance", 1.0),
}
if composite_mode != "keep":
updated["composite_mode"] = composite_mode
updated["recommended_composite_mode"] = composite_mode
if replace_tile_shape != "keep":
updated["replace_tile_shape"] = _ttp_normalize_replace_tile_shape(replace_tile_shape)
updated["layer"] = self._apply_number_mode(updated.get("layer", 0.0), layer_mode, layer)
updated["priority"] = self._apply_number_mode(updated.get("priority", 0.0), priority_mode, priority)
updated["occlusion_priority"] = self._apply_number_mode(updated.get("occlusion_priority", 0.0), occlusion_priority_mode, occlusion_priority)
updated["blend"] = max(0, int(round(self._apply_number_mode(updated.get("blend", 0), blend_mode, blend))))
updated["importance"] = max(0.0, self._apply_number_mode(updated.get("importance", 1.0), importance_mode, importance))
updated["composite_override"] = True
updated["composite_override_selector_type"] = str(selector_type or "")
updated["composite_override_selector"] = str(selector or "")
records.append({
"index": int(index),
"name": updated.get("name", f"tile_{index}"),
"label": updated.get("label", ""),
"before": before,
"after": {
"composite_mode": updated.get("composite_mode", ""),
"replace_tile_shape": updated.get("replace_tile_shape", ""),
"layer": updated.get("layer", 0.0),
"priority": updated.get("priority", 0.0),
"occlusion_priority": updated.get("occlusion_priority", 0.0),
"blend": updated.get("blend", 0),
"importance": updated.get("importance", 1.0),
},
})
kept_tiles.append(updated)
kept_images.append(next_images[index] if index < len(next_images) else tile_images[index])
if index < len(next_positions):
kept_positions.append(next_positions[index])
else:
kept_positions.append(self._position_for_tile(updated))
if not kept_tiles:
raise ValueError("Smart Tile composite override kept no tiles. Change selector or unmatched_mode.")
for new_index, tile in enumerate(kept_tiles):
tile["id"] = new_index
next_tile_set["tile_meta"]["tiles"] = kept_tiles
next_tile_set["tile_images"] = kept_images
next_tile_set["positions"] = kept_positions
composite_json = json.dumps({
"type": "ttp_smart_tile_composite_override",
"selector_type": str(selector_type or ""),
"selector": str(selector or ""),
"unmatched_mode": str(unmatched_mode or "keep"),
"matched": int(matched_count),
"kept": int(len(kept_tiles)),
"dropped": int(count - len(kept_tiles)),
"tiles": records,
}, ensure_ascii=False, indent=2)
summary_lines = [
f"matched={matched_count} kept={len(kept_tiles)} dropped={count - len(kept_tiles)} selector={selector_type}:{selector}"
]
summary_lines.extend(
f'{record["index"]}:{record["name"]} mode={record["after"]["composite_mode"]} shape={record["after"]["replace_tile_shape"] or "default"} layer={record["after"]["layer"]} occ={record["after"]["occlusion_priority"]}'
for record in records
)
return (next_tile_set, composite_json, "\n".join(summary_lines))
class TTP_Smart_Tile_Stack_Order_Experimental:
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"tile_set": ("TTP_SMART_TILE_SET", {"forceInput": True}),
"tile_order": ("STRING", {"default": "T1,T3,T2", "multiline": False}),
"top_order": (["first_on_top", "last_on_top"], {"default": "first_on_top"}),
"composite_mode": (["replace_tile", "keep"], {"default": "replace_tile"}),
"replace_tile_shape": (["keep", "mask_first", "tile_box_first"], {"default": "keep"}),
}
}
RETURN_TYPES = ("TTP_SMART_TILE_SET", "STRING", "STRING")
RETURN_NAMES = ("tile_set", "order_json", "summary")
FUNCTION = "apply_stack_order"
CATEGORY = "TTP/Smart Tile"
@staticmethod
def _position_for_tile(tile):
return TTP_Smart_Tile_Prompt_Override_Experimental._position_for_tile(tile)
def apply_stack_order(
self,
tile_set,
tile_order="T1,T3,T2",
top_order="first_on_top",
composite_mode="replace_tile",
replace_tile_shape="keep",
):
tile_images, _tile_meta, tiles_info = _ttp_validate_tile_set(tile_set)
count = len(tiles_info)
ordered_indexes = TTP_Smart_Tile_Prompt_Override_Experimental._selector_index_order(tile_order, count)
if not ordered_indexes:
raise ValueError("Smart Tile stack order matched no tiles. Use indexes like T1,T3,T2.")
next_tile_set = _ttp_clone_tile_set(tile_set)
next_tiles = next_tile_set["tile_meta"]["tiles"]
next_images = list(next_tile_set.get("tile_images", []))
next_positions = list(next_tile_set.get("positions", []))
max_layer = max((_ttp_safe_float(tile.get("layer", 0.0), 0.0) for tile in next_tiles), default=0.0)
max_occlusion = max((_ttp_safe_float(tile.get("occlusion_priority", 0.0), 0.0) for tile in next_tiles), default=0.0)
step = 1000.0
first_on_top = str(top_order or "first_on_top") == "first_on_top"
composite_mode = str(composite_mode or "replace_tile")
replace_tile_shape = str(replace_tile_shape or "keep")
order_rank = {}
order_position = {}
ordered_count = len(ordered_indexes)
for rank, index in enumerate(ordered_indexes):
stack_rank = ordered_count - rank if first_on_top else rank + 1
order_rank[index] = stack_rank
order_position[index] = rank
records = []
for index, tile in enumerate(next_tiles):
if index not in order_rank:
continue
stack_rank = order_rank[index]
before = {
"layer": tile.get("layer", 0.0),
"occlusion_priority": tile.get("occlusion_priority", 0.0),
"composite_mode": tile.get("composite_mode", tile.get("recommended_composite_mode", "")),
"replace_tile_shape": tile.get("replace_tile_shape", ""),
}
tile["layer"] = max_layer + float(stack_rank)
tile["occlusion_priority"] = max_occlusion + float(stack_rank) * step
if composite_mode != "keep":
tile["composite_mode"] = composite_mode
tile["recommended_composite_mode"] = composite_mode
if replace_tile_shape != "keep":
tile["replace_tile_shape"] = _ttp_normalize_replace_tile_shape(replace_tile_shape)
tile["stack_order_override"] = True
tile["stack_order_selector"] = str(tile_order or "")
tile["stack_order_position"] = int(ordered_indexes.index(index) + 1)
tile["stack_order_top_rank"] = float(stack_rank)
records.append({
"index": int(index),
"selector": f"T{index + 1}",
"name": tile.get("name", f"tile_{index}"),
"before": before,
"after": {
"layer": tile.get("layer", 0.0),
"occlusion_priority": tile.get("occlusion_priority", 0.0),
"composite_mode": tile.get("composite_mode", tile.get("recommended_composite_mode", "")),
"replace_tile_shape": tile.get("replace_tile_shape", ""),
"top_rank": float(stack_rank),
},
})
records.sort(key=lambda record: order_position.get(int(record["index"]), count))
if len(next_positions) < len(next_tiles):
next_tile_set["positions"] = [
next_positions[index] if index < len(next_positions) else self._position_for_tile(tile)
for index, tile in enumerate(next_tiles)
]
else:
next_tile_set["positions"] = next_positions
next_tile_set["tile_images"] = next_images if next_images else list(tile_images)
order_json = json.dumps({
"type": "ttp_smart_tile_stack_order",
"tile_order": str(tile_order or ""),
"top_order": str(top_order or "first_on_top"),
"composite_mode": composite_mode,
"replace_tile_shape": replace_tile_shape,
"matched": int(len(records)),
"tiles": records,
}, ensure_ascii=False, indent=2)
summary_lines = [
f"matched={len(records)} order={tile_order} top_order={top_order} mode={composite_mode}"
]
summary_lines.extend(
f'{record["selector"]}:{record["name"]} layer={record["after"]["layer"]} occ={record["after"]["occlusion_priority"]}'
for record in records
)
return (next_tile_set, order_json, "\n".join(summary_lines))
class TTP_Smart_Tile_Loop_Source_Experimental:
@classmethod
def INPUT_TYPES(cls):
@@ -4579,7 +5227,8 @@ class TTP_Smart_Tile_Assemble_Experimental:
"base_canvas_mode": (["auto", "black", "base_image", "source_image"], {"default": "auto"}),
"weight_preview_mode": (["raw_weight", "coverage"], {"default": "raw_weight"}),
"small_tile_on_top": ("BOOLEAN", {"default": False}),
"auto_composite_policy": (["safe_auto", "strict_layer", "soft_detail", "replace_object"], {"default": "safe_auto"}),
"auto_composite_policy": (["safe_auto", "strict_layer", "soft_detail", "replace_object", "replace_tile"], {"default": "safe_auto"}),
"replace_tile_shape": (["mask_first", "tile_box_first"], {"default": "mask_first"}),
},
"optional": {
"sampled_tiles": ("IMAGE",),
@@ -4620,6 +5269,7 @@ class TTP_Smart_Tile_Assemble_Experimental:
weight_preview_mode="raw_weight",
small_tile_on_top=False,
auto_composite_policy="safe_auto",
replace_tile_shape="mask_first",
sampled_tiles=None,
tile_meta=None,
tile_set=None,
@@ -4737,6 +5387,7 @@ class TTP_Smart_Tile_Assemble_Experimental:
"weight_preview_mode": preview_mode,
"small_tile_on_top": _ttp_safe_bool(small_tile_on_top, False),
"auto_composite_policy": str(auto_composite_policy or "safe_auto"),
"replace_tile_shape": str(replace_tile_shape or "mask_first"),
"mask_blend_mode": str(mask_blend_mode),
"large_tile_policy": str(large_tile_policy),
},
@@ -4767,10 +5418,15 @@ class TTP_Smart_Tile_Assemble_Experimental:
small_tile_on_top = _ttp_safe_bool(small_tile_on_top, False)
composite_policy = str(auto_composite_policy or "safe_auto")
if composite_policy not in ("safe_auto", "strict_layer", "soft_detail", "replace_object"):
if composite_policy not in ("safe_auto", "strict_layer", "soft_detail", "replace_object", "replace_tile"):
composite_policy = "safe_auto"
auto_policy_ranking = composite_policy in ("safe_auto", "soft_detail", "replace_object")
destructive_occlusion = composite_policy in ("strict_layer", "replace_object")
replace_tile_shape_default = _ttp_normalize_replace_tile_shape(replace_tile_shape)
auto_policy_ranking = composite_policy in ("safe_auto", "soft_detail", "replace_object", "replace_tile")
destructive_occlusion = composite_policy in ("strict_layer", "replace_object", "replace_tile")
replace_tile_policy_active = composite_policy == "replace_tile" or any(
_ttp_is_replace_tile_mode(_ttp_tile_composite_mode(tile))
for tile in tiles_info
)
auto_policy_candidates = auto_policy_ranking and any(
not _ttp_is_context_tile_text(_ttp_tile_search_text(tile)) and (
_ttp_is_detail_tile_text(_ttp_tile_search_text(tile))
@@ -4780,7 +5436,7 @@ class TTP_Smart_Tile_Assemble_Experimental:
)
for tile in tiles_info
)
use_occlusion = small_tile_on_top or auto_policy_candidates or any(
use_occlusion = replace_tile_policy_active or small_tile_on_top or auto_policy_candidates or any(
float(tile.get("occlusion_priority", 0.0)) != 0.0 or int(tile.get("layer", 0)) != 0
for tile in tiles_info
)
@@ -4856,6 +5512,9 @@ class TTP_Smart_Tile_Assemble_Experimental:
tile_area_ratio,
large_tile_area_threshold,
)
tile_mode = _ttp_tile_composite_mode(tile)
replace_tile_mode = composite_policy == "replace_tile" or _ttp_is_replace_tile_mode(tile_mode)
tile_replace_shape = _ttp_tile_replace_shape(tile, replace_tile_shape_default)
low_resolution_large_tile = (
has_base_canvas
and is_large_tile
@@ -4909,8 +5568,24 @@ class TTP_Smart_Tile_Assemble_Experimental:
1.0,
1.0,
)
object_mask = _ttp_tile_object_mask_array(tile, out_w, out_h, str(mask_blend_mode), blend)
if object_mask is not None:
mask_mode = "mask_feather" if replace_tile_mode else str(mask_blend_mode)
object_mask = _ttp_tile_object_mask_array(
tile,
out_w,
out_h,
mask_mode,
blend,
hard_stitch=replace_tile_mode and tile_replace_shape != "tile_box_first",
prefer_own_mask=replace_tile_mode,
)
if replace_tile_mode:
if tile_replace_shape == "tile_box_first":
mask = _ttp_rect_feather_mask(out_w, out_h, blend)
elif object_mask is not None:
mask = object_mask
else:
mask = _ttp_rect_feather_mask(out_w, out_h, blend)
elif object_mask is not None:
if soft_detail_overlay and str(mask_blend_mode) in ("auto", "mask_feather", "mask_only"):
blur_px = max(1.0, min(8.0, max(float(blend) * 0.08, min(out_w, out_h) * 0.012)))
object_halo = _ttp_soften_detail_mask_array(
@@ -4928,7 +5603,7 @@ class TTP_Smart_Tile_Assemble_Experimental:
mask = mask * object_mask
elif _ttp_should_rect_feather_tile(tile, is_large_tile, str(mask_blend_mode)):
mask = mask * _ttp_rect_feather_mask(out_w, out_h, blend)
tile_destructive_occlusion = destructive_occlusion or (
tile_destructive_occlusion = replace_tile_mode or destructive_occlusion or (
composite_policy == "safe_auto"
and object_mask is not None
and not soft_detail_overlay
@@ -4947,6 +5622,9 @@ class TTP_Smart_Tile_Assemble_Experimental:
rank_occlusion = float(tile.get("occlusion_priority", 0.0))
rank_layer = float(tile.get("layer", 0.0))
rank_priority = priority
rank_tiebreaker = (float(index) + 1.0) * 0.001 if replace_tile_mode else 0.0
if replace_tile_mode:
rank_occlusion = max(rank_occlusion, 1.0)
if auto_policy_ranking and not is_large_tile:
text = _ttp_tile_search_text(tile)
if _ttp_is_detail_tile_text(text) or soft_detail_overlay:
@@ -4961,7 +5639,9 @@ class TTP_Smart_Tile_Assemble_Experimental:
rank_layer = max(rank_layer, 1.0)
if soft_detail_overlay:
mask = mask * (1.0 + coverage_mask * coverage_mask)
if is_large_tile and large_tile_policy == "context_only" and has_base_canvas:
if replace_tile_mode:
pass
elif is_large_tile and large_tile_policy == "context_only" and has_base_canvas:
mask = mask * context_tile_weight
rank_occlusion = min(rank_occlusion, 0.0)
rank_layer = min(rank_layer, 0.0)
@@ -5081,7 +5761,7 @@ class TTP_Smart_Tile_Assemble_Experimental:
mask_t = torch.as_tensor(mask, dtype=torch.float32, device=assemble_torch_device)
clear_t = torch.as_tensor(clear_mask, dtype=torch.float32, device=assemble_torch_device)
if use_occlusion:
rank = rank_occlusion * 1000000.0 + rank_layer * 10000.0 + (rank_priority if use_priority else 0.0)
rank = rank_occlusion * 1000000.0 + rank_layer * 10000.0 + (rank_priority if use_priority else 0.0) + rank_tiebreaker
rank_view = priority_ranks[paste_y:y_end, paste_x:x_end]
mask_active = coverage_t > 1e-6
higher = (rank > rank_view + 1e-6) & mask_active
@@ -5110,7 +5790,7 @@ class TTP_Smart_Tile_Assemble_Experimental:
weights[paste_y:y_end, paste_x:x_end].add_(mask_t)
preview_weights[paste_y:y_end, paste_x:x_end].add_(coverage_t)
elif use_occlusion:
rank = rank_occlusion * 1000000.0 + rank_layer * 10000.0 + (rank_priority if use_priority else 0.0)
rank = rank_occlusion * 1000000.0 + rank_layer * 10000.0 + (rank_priority if use_priority else 0.0) + rank_tiebreaker
rank_view = priority_ranks[paste_y:y_end, paste_x:x_end]
mask_active = coverage_mask > 1e-6
higher = (rank > rank_view + 1e-6) & mask_active
@@ -6092,6 +6772,9 @@ NODE_CLASS_MAPPINGS = {
"TTP_Smart_Tile_Set_Preview_Experimental": TTP_Smart_Tile_Set_Preview_Experimental,
"TTP_QwenVL3_Local_Loader_Experimental": TTP_QwenVL3_Local_Loader_Experimental,
"TTP_Smart_Tile_QwenVL_Prompt_Set_Builder_Experimental": TTP_Smart_Tile_QwenVL_Prompt_Set_Builder_Experimental,
"TTP_Smart_Tile_Prompt_Override_Experimental": TTP_Smart_Tile_Prompt_Override_Experimental,
"TTP_Smart_Tile_Composite_Override_Experimental": TTP_Smart_Tile_Composite_Override_Experimental,
"TTP_Smart_Tile_Stack_Order_Experimental": TTP_Smart_Tile_Stack_Order_Experimental,
"TTP_Smart_Tile_Loop_Source_Experimental": TTP_Smart_Tile_Loop_Source_Experimental,
"TTP_Smart_Tile_Loop_Collect_Experimental": TTP_Smart_Tile_Loop_Collect_Experimental,
"TTP_Smart_Tile_Image_Upscale_Prep_Experimental": TTP_Smart_Tile_Image_Upscale_Prep_Experimental,
@@ -6117,6 +6800,9 @@ NODE_DISPLAY_NAME_MAPPINGS = {
"TTP_Smart_Tile_Set_Preview_Experimental": "TTP Smart Tile Set Preview",
"TTP_QwenVL3_Local_Loader_Experimental": "TTP QwenVL3 Local Loader",
"TTP_Smart_Tile_QwenVL_Prompt_Set_Builder_Experimental": "TTP Smart Tile QwenVL Prompt Set Builder",
"TTP_Smart_Tile_Prompt_Override_Experimental": "TTP Smart Tile Prompt Override",
"TTP_Smart_Tile_Composite_Override_Experimental": "TTP Smart Tile Composite Override",
"TTP_Smart_Tile_Stack_Order_Experimental": "TTP Smart Tile Stack Order",
"TTP_Smart_Tile_Loop_Source_Experimental": "TTP Smart Tile Loop Source",
"TTP_Smart_Tile_Loop_Collect_Experimental": "TTP Smart Tile Loop Collect",
"TTP_Smart_Tile_Image_Upscale_Prep_Experimental": "TTP Smart Tile Image Upscale Prep",
+396 -1
View File
@@ -153,6 +153,7 @@ assert_equal("auto_detect_request" in required_inputs, True, "interactive crop s
assert_equal("auto_prompt" in required_inputs, True, "interactive crop should expose a visual prompt")
assert_equal("allow_object_overlap" in required_inputs, True, "interactive crop should expose object overlap control")
assert_equal("auto_object_padding" in required_inputs, True, "interactive crop should expose object padding control")
assert_equal("auto_mask_expand" in required_inputs, True, "interactive crop should expose auto mask expansion control")
assert_equal("auto_max_tiles" in required_inputs, True, "interactive crop should expose auto max tiles control")
assert_equal("auto_paint_mask" in required_inputs, True, "interactive crop should expose hidden painted mask input")
assert_equal("vision_model" in optional_inputs, True, "interactive crop should expose a vision model input")
@@ -184,6 +185,7 @@ expected_assemble_required_order = [
"weight_preview_mode",
"small_tile_on_top",
"auto_composite_policy",
"replace_tile_shape",
]
assert_equal(
list(assemble_inputs["required"].keys()),
@@ -207,8 +209,10 @@ assert_equal(assemble_inputs["required"]["base_canvas_mode"][0], ["auto", "black
assert_equal(assemble_inputs["required"]["weight_preview_mode"][0], ["raw_weight", "coverage"], "assemble should expose raw weight and coverage preview modes")
assert_equal(assemble_inputs["required"]["weight_preview_mode"][1]["default"], "raw_weight", "assemble should keep raw overlap weight preview as the default")
assert_equal("small_tile_on_top" in assemble_inputs["required"], True, "assemble should expose small tile top stacking")
assert_equal(assemble_inputs["required"]["auto_composite_policy"][0], ["safe_auto", "strict_layer", "soft_detail", "replace_object"], "assemble should expose automatic compositing policy")
assert_equal(assemble_inputs["required"]["auto_composite_policy"][0], ["safe_auto", "strict_layer", "soft_detail", "replace_object", "replace_tile"], "assemble should expose automatic compositing policy")
assert_equal(assemble_inputs["required"]["auto_composite_policy"][1]["default"], "safe_auto", "assemble should default to safe auto compositing")
assert_equal(assemble_inputs["required"]["replace_tile_shape"][0], ["mask_first", "tile_box_first"], "assemble should expose replace tile shape priority")
assert_equal(assemble_inputs["required"]["replace_tile_shape"][1]["default"], "mask_first", "assemble should default replace tile to mask priority")
assert_equal(assemble_inputs["optional"]["done"][0], "BOOLEAN", "assemble should accept loop done gate")
preview_inputs = ttp.TTP_Smart_Tile_Set_Preview_Experimental.INPUT_TYPES()
assert_equal(preview_inputs["required"]["tile_set"][0], "TTP_SMART_TILE_SET", "tile set preview should accept Smart Tile Set")
@@ -257,6 +261,160 @@ assert_equal(
ttp.TTP_Smart_Tile_QwenVL_Prompt_Set_Builder_Experimental,
"prompt builder should be registered",
)
prompt_override_inputs = ttp.TTP_Smart_Tile_Prompt_Override_Experimental.INPUT_TYPES()
assert_equal(prompt_override_inputs["required"]["tile_set"][0], "TTP_SMART_TILE_SET", "prompt override should accept Smart Tile Set")
assert_equal(prompt_override_inputs["required"]["tile_set"][1].get("forceInput"), True, "prompt override tile_set should be a forceInput connection")
assert_equal("regex" in prompt_override_inputs["required"]["selector_type"][0], True, "prompt override should support regex selectors")
assert_equal(prompt_override_inputs["required"]["unmatched_mode"][0], ["keep", "drop_unmatched"], "prompt override should allow dropping unmatched tiles")
assert_equal(prompt_override_inputs["required"]["prompt_text"][1]["dynamicPrompts"], False, "prompt override text should not use dynamic prompt parsing")
assert_equal(
ttp.NODE_CLASS_MAPPINGS["TTP_Smart_Tile_Prompt_Override_Experimental"],
ttp.TTP_Smart_Tile_Prompt_Override_Experimental,
"prompt override should be registered",
)
override_tile_set = {
"version": 1,
"type": "ttp_smart_tile_set",
"original_size": [100, 100],
"tile_meta": {
"version": 3,
"type": "ttp_smart_tile",
"storage": "tile_set",
"original_size": [100, 100],
"tiles": [
{
"id": 0,
"name": "tile_1",
"label": "face",
"source": "paint_mask",
"core_box": [0, 0, 50, 50],
"sample_box": [0, 0, 50, 50],
"paste_box": [0, 0, 50, 50],
"prompt": "old face prompt",
"negative": "old face negative",
},
{
"id": 1,
"name": "tile_2",
"label": "background",
"source": "manual",
"core_box": [50, 0, 50, 50],
"sample_box": [50, 0, 50, 50],
"paste_box": [50, 0, 50, 50],
"prompt": "old background prompt",
"negative": "old background negative",
},
],
},
"tile_images": ["face_image", "background_image"],
"positions": [(0, 0, 50, 50), (50, 0, 100, 50)],
}
override_node = ttp.TTP_Smart_Tile_Prompt_Override_Experimental()
overridden_set, override_json, override_summary = override_node.override_prompts(
override_tile_set,
selector_type="label",
selector="face",
unmatched_mode="keep",
prompt_mode="replace",
prompt_text="edit only the face",
negative_mode="append",
negative_text="avoid changing identity",
)
assert_equal(overridden_set["tile_meta"]["tiles"][0]["prompt"], "edit only the face", "prompt override should replace matched tile prompt")
assert_equal(overridden_set["tile_meta"]["tiles"][0]["negative"], "old face negative\navoid changing identity", "prompt override should append matched tile negative")
assert_equal(overridden_set["tile_meta"]["tiles"][1]["prompt"], "old background prompt", "prompt override should keep unmatched tile prompt")
assert_equal(json.loads(override_json)["matched"], 1, "prompt override JSON should report matched tile count")
assert_equal("matched=1" in override_summary, True, "prompt override summary should report matched count")
dropped_set, _dropped_json, _dropped_summary = override_node.override_prompts(
override_tile_set,
selector_type="label",
selector="face",
unmatched_mode="drop_unmatched",
prompt_mode="replace",
prompt_text="edit only the face",
)
assert_equal(len(dropped_set["tile_meta"]["tiles"]), 1, "prompt override should drop unmatched tile metadata")
assert_equal(len(dropped_set["tile_images"]), 1, "prompt override should drop unmatched tile images")
assert_equal(dropped_set["tile_meta"]["tiles"][0]["id"], 0, "prompt override should reindex kept tile ids")
composite_override_inputs = ttp.TTP_Smart_Tile_Composite_Override_Experimental.INPUT_TYPES()
assert_equal(composite_override_inputs["required"]["tile_set"][0], "TTP_SMART_TILE_SET", "composite override should accept Smart Tile Set")
assert_equal(composite_override_inputs["required"]["tile_set"][1].get("forceInput"), True, "composite override tile_set should be a forceInput connection")
assert_equal("replace_tile" in composite_override_inputs["required"]["composite_mode"][0], True, "composite override should expose replace_tile mode")
assert_equal(composite_override_inputs["required"]["replace_tile_shape"][0], ["keep", "mask_first", "tile_box_first"], "composite override should expose replace tile shape priority")
assert_equal(composite_override_inputs["required"]["selector_type"][0], prompt_override_inputs["required"]["selector_type"][0], "composite override selectors should match prompt override selectors")
assert_equal(
ttp.NODE_CLASS_MAPPINGS["TTP_Smart_Tile_Composite_Override_Experimental"],
ttp.TTP_Smart_Tile_Composite_Override_Experimental,
"composite override should be registered",
)
composite_node = ttp.TTP_Smart_Tile_Composite_Override_Experimental()
composited_set, composite_json, composite_summary = composite_node.override_composite(
override_tile_set,
selector_type="label",
selector="face",
composite_mode="replace_tile",
replace_tile_shape="tile_box_first",
layer_mode="max",
layer=12,
priority_mode="max",
priority=220,
occlusion_priority_mode="max",
occlusion_priority=60000,
blend_mode="set",
blend=24,
)
composited_tile = composited_set["tile_meta"]["tiles"][0]
assert_equal(composited_tile["composite_mode"], "replace_tile", "composite override should set replace_tile mode")
assert_equal(composited_tile["recommended_composite_mode"], "replace_tile", "composite override should update recommended composite mode")
assert_equal(composited_tile["replace_tile_shape"], "tile_box_first", "composite override should set replace tile shape priority")
assert_equal(composited_tile["layer"], 12, "composite override should raise the selected tile layer")
assert_equal(composited_tile["priority"], 220, "composite override should raise the selected tile priority")
assert_equal(composited_tile["occlusion_priority"], 60000, "composite override should raise the selected tile occlusion priority")
assert_equal(composited_tile["blend"], 24, "composite override should set selected tile blend")
assert_equal(json.loads(composite_json)["matched"], 1, "composite override JSON should report matched tile count")
assert_equal("matched=1" in composite_summary, True, "composite override summary should report matched count")
stack_order_inputs = ttp.TTP_Smart_Tile_Stack_Order_Experimental.INPUT_TYPES()
assert_equal(stack_order_inputs["required"]["tile_set"][0], "TTP_SMART_TILE_SET", "stack order should accept Smart Tile Set")
assert_equal(stack_order_inputs["required"]["tile_set"][1].get("forceInput"), True, "stack order tile_set should be a forceInput connection")
assert_equal(stack_order_inputs["required"]["top_order"][0], ["first_on_top", "last_on_top"], "stack order should expose top order direction")
assert_equal(stack_order_inputs["required"]["composite_mode"][1]["default"], "replace_tile", "stack order should default to replace-tile compositing")
assert_equal(
ttp.NODE_CLASS_MAPPINGS["TTP_Smart_Tile_Stack_Order_Experimental"],
ttp.TTP_Smart_Tile_Stack_Order_Experimental,
"stack order should be registered",
)
stack_order_tile_set = {
"version": 1,
"type": "ttp_smart_tile_set",
"original_size": [100, 100],
"tile_meta": {
"version": 3,
"type": "ttp_smart_tile",
"storage": "tile_set",
"original_size": [100, 100],
"tiles": [
{"id": 0, "name": "tile_1", "core_box": [0, 0, 50, 50], "sample_box": [0, 0, 50, 50], "layer": 0, "occlusion_priority": 0},
{"id": 1, "name": "tile_2", "core_box": [0, 0, 50, 50], "sample_box": [0, 0, 50, 50], "layer": 0, "occlusion_priority": 0},
{"id": 2, "name": "tile_3", "core_box": [0, 0, 50, 50], "sample_box": [0, 0, 50, 50], "layer": 0, "occlusion_priority": 0},
{"id": 3, "name": "tile_4", "core_box": [0, 0, 50, 50], "sample_box": [0, 0, 50, 50], "layer": 1, "occlusion_priority": 80000},
],
},
"tile_images": ["tile_1", "tile_2", "tile_3", "tile_4"],
}
stack_node = ttp.TTP_Smart_Tile_Stack_Order_Experimental()
stacked_set, stack_json, stack_summary = stack_node.apply_stack_order(
stack_order_tile_set,
tile_order="T1,T3,T2",
)
stacked_tiles = stacked_set["tile_meta"]["tiles"]
assert_equal(stacked_tiles[0]["occlusion_priority"] > stacked_tiles[2]["occlusion_priority"], True, "stack order should make the first listed tile topmost by default")
assert_equal(stacked_tiles[2]["occlusion_priority"] > stacked_tiles[1]["occlusion_priority"], True, "stack order should preserve listed ordering between selected tiles")
assert_equal(stacked_tiles[1]["occlusion_priority"] > stacked_tiles[3]["occlusion_priority"], True, "stack order should place listed tiles above unlisted tiles")
assert_equal(stacked_tiles[0]["composite_mode"], "replace_tile", "stack order should default selected tiles to replace_tile mode")
assert_equal("composite_mode" in stacked_tiles[3], False, "stack order should not change unlisted tile composite mode")
assert_equal(json.loads(stack_json)["tiles"][1]["selector"], "T3", "stack order JSON should preserve selector order")
assert_equal("T1" in stack_summary and "T3" in stack_summary and "T2" in stack_summary, True, "stack order summary should include selected tiles")
loop_source_inputs = ttp.TTP_Smart_Tile_Loop_Source_Experimental.INPUT_TYPES()
assert_equal(loop_source_inputs["required"]["tile_set"][0], "TTP_SMART_TILE_SET", "loop source should accept Smart Tile Set")
assert_equal(loop_source_inputs["optional"]["clip"][0], "CLIP", "loop source should optionally encode tile prompts with CLIP")
@@ -1145,6 +1303,37 @@ mask_array = ttp._ttp_tile_object_mask_array(masked_auto_meta[0], masked_auto_me
assert_equal(mask_array.shape[2], 1, "decoded object mask should be a single-channel weight map")
assert_equal(float(mask_array.max()) > 0.5, True, "decoded object mask should keep foreground weight")
small_expand_mask = Image.new("L", (32, 32), 0)
for x in range(10, 14):
for y in range(10, 14):
small_expand_mask.putpixel((x, y), 255)
small_mask_layout = ttp._ttp_boxes_to_auto_layout(
[{"x": 10, "y": 10, "width": 4, "height": 4, "label": "eye", "score": 0.9}],
32,
32,
object_padding=4,
mask_expand=0,
max_tiles=1,
include_background=False,
masks=[small_expand_mask],
)
expanded_mask_layout = ttp._ttp_boxes_to_auto_layout(
[{"x": 10, "y": 10, "width": 4, "height": 4, "label": "eye", "score": 0.9}],
32,
32,
object_padding=4,
mask_expand=2,
max_tiles=1,
include_background=False,
masks=[small_expand_mask],
)
small_mask_tile = ttp._ttp_parse_smart_tile_layout(small_mask_layout, 32, 32)[0]
expanded_mask_tile = ttp._ttp_parse_smart_tile_layout(expanded_mask_layout, 32, 32)[0]
small_mask_pixels = np.count_nonzero(np.array(ttp._ttp_decode_object_mask_data(small_mask_tile["object_mask"])) > 0)
expanded_mask_pixels = np.count_nonzero(np.array(ttp._ttp_decode_object_mask_data(expanded_mask_tile["object_mask"])) > 0)
assert_equal(expanded_mask_pixels > small_mask_pixels, True, "auto mask expand should dilate object mask foreground pixels")
assert_equal(expanded_mask_tile.get("object_mask_expand"), 2.0, "auto mask expand should be preserved in tile metadata")
large_body_mask = Image.new("L", (900, 600), 255)
large_body_layout = ttp._ttp_boxes_to_auto_layout(
[{"x": 0, "y": 0, "width": 900, "height": 600, "label": "person body", "score": 0.93}],
@@ -1215,6 +1404,7 @@ paint_layout, paint_message = ttp._ttp_run_paint_mask_auto_layout(
default_pad=8,
default_blend=4,
object_padding=2,
mask_expand=2,
max_tiles=8,
allow_object_overlap=True,
)
@@ -1226,6 +1416,51 @@ assert_equal("paint mask" in paint_message.lower(), True, "paint-only auto tile
half_mask = Image.new("L", (8, 8), 128)
half_mask_data = ttp._ttp_encode_object_mask_data([half_mask], [0], [0, 0, 8, 8])
assemble_node = ttp.TTP_Smart_Tile_Assemble_Experimental()
stack_assemble_tile_set = {
"type": "ttp_smart_tile_set",
"original_size": [8, 8],
"tile_meta": {
"type": "ttp_smart_tile",
"original_size": [8, 8],
"tiles": [
{
"name": f"stack_tile_{index + 1}",
"core_box": [0, 0, 8, 8],
"sample_box": [0, 0, 8, 8],
"tile_canvas_size": [8, 8],
"tile_canvas_box": [0, 0, 8, 8],
"overlap_edges_px_source": {"left": 0, "right": 0, "top": 0, "bottom": 0},
"blend": 0,
"importance": 1.0,
"priority": 0.0,
"layer": 1 if index == 3 else 0,
"occlusion_priority": 80000 if index == 3 else 0,
}
for index in range(4)
],
},
"tile_images": [
ttp.pil2tensor(Image.new("RGB", (8, 8), (220, 20, 20)))[0],
ttp.pil2tensor(Image.new("RGB", (8, 8), (20, 220, 20)))[0],
ttp.pil2tensor(Image.new("RGB", (8, 8), (20, 20, 220)))[0],
ttp.pil2tensor(Image.new("RGB", (8, 8), (220, 220, 20)))[0],
],
}
stack_assemble_set, _stack_assemble_json, _stack_assemble_summary = stack_node.apply_stack_order(
stack_assemble_tile_set,
tile_order="T1,T3,T2",
)
stack_assembled, _stack_assembled_weights = assemble_node.assemble_tiles(
blend_multiplier=1.0,
output_scale=1.0,
use_priority=True,
base_canvas_mode="black",
auto_composite_policy="safe_auto",
tile_set=stack_assemble_set,
)
stack_center = stack_assembled.array[0, 4, 4]
assert_equal(float(stack_center[0]) > float(stack_center[2]), True, "stack order assemble should put the first listed tile on top by default")
assert_equal(float(stack_center[0]) > float(stack_center[1]), True, "stack order assemble should keep unlisted and later listed tiles behind the first listed tile")
soft_tile_meta = {
"type": "ttp_smart_tile",
"original_size": [8, 8],
@@ -1545,6 +1780,166 @@ safe_eye_center = safe_eye.array[0, 8, 8]
assert_equal(float(safe_eye_edge[0]) > float(strict_eye_edge[0]), True, "safe auto detail overlay should preserve more face color at eye mask edges")
assert_equal(float(safe_eye_center[2]) > float(safe_eye_center[0]), True, "safe auto detail overlay should keep eye details stronger at the mask center")
replace_mask_full = Image.new("L", (16, 16), 0)
ImageDraw.Draw(replace_mask_full).rectangle((7, 7, 8, 8), fill=255)
replace_mask_data = ttp._ttp_encode_object_mask_data([replace_mask_full], [0], [4, 4, 12, 12])
replace_mask_tile_set = {
"type": "ttp_smart_tile_set",
"original_size": [16, 16],
"tile_meta": {
"type": "ttp_smart_tile",
"original_size": [16, 16],
"tiles": [
{
"name": "masked_edit",
"label": "paint mask edit",
"core_box": [4, 4, 8, 8],
"sample_box": [4, 4, 8, 8],
"tile_canvas_size": [8, 8],
"tile_canvas_box": [0, 0, 8, 8],
"overlap_edges_px_source": {"left": 0, "right": 0, "top": 0, "bottom": 0},
"blend": 0,
"importance": 1.0,
"priority": 0.0,
"layer": 0,
"occlusion_priority": 0,
"object_mask": replace_mask_data,
"composite_mode": "replace_tile",
},
],
},
"tile_images": [
ttp.pil2tensor(Image.new("RGB", (8, 8), (20, 20, 220)))[0],
],
}
replace_source = ttp.pil2tensor(Image.new("RGB", (16, 16), (220, 20, 20)))
replace_masked, _replace_masked_weights = assemble_node.assemble_tiles(
blend_multiplier=1.0,
output_scale=1.0,
use_priority=True,
base_canvas_mode="source_image",
mask_blend_mode="off",
auto_composite_policy="replace_tile",
source_image=replace_source,
tile_set=replace_mask_tile_set,
)
replace_mask_corner = replace_masked.array[0, 4, 4]
replace_mask_center = replace_masked.array[0, 8, 8]
assert_equal(float(replace_mask_corner[0]) > float(replace_mask_corner[2]), True, "replace_tile must preserve source pixels outside an object mask instead of pasting a rectangle")
assert_equal(float(replace_mask_center[2]) > float(replace_mask_center[0]), True, "replace_tile must hard replace pixels inside an object mask")
replace_boxed, _replace_boxed_weights = assemble_node.assemble_tiles(
blend_multiplier=1.0,
output_scale=1.0,
use_priority=True,
base_canvas_mode="source_image",
mask_blend_mode="off",
auto_composite_policy="replace_tile",
replace_tile_shape="tile_box_first",
source_image=replace_source,
tile_set=replace_mask_tile_set,
)
replace_box_corner = replace_boxed.array[0, 4, 4]
replace_box_center = replace_boxed.array[0, 8, 8]
assert_equal(float(replace_box_corner[2]) > float(replace_box_corner[0]), True, "replace_tile tile_box_first should paste the full tile rectangle even when an object mask exists")
assert_equal(float(replace_box_center[2]) > float(replace_box_center[0]), True, "replace_tile tile_box_first should still replace the object center")
replace_soft_mask_full = Image.new("L", (16, 16), 0)
ImageDraw.Draw(replace_soft_mask_full).rectangle((4, 4, 11, 11), fill=128)
replace_soft_mask_data = ttp._ttp_encode_object_mask_data([replace_soft_mask_full], [0], [4, 4, 12, 12])
replace_soft_mask_tile_set = {
"type": "ttp_smart_tile_set",
"original_size": [16, 16],
"tile_meta": {
"type": "ttp_smart_tile",
"original_size": [16, 16],
"tiles": [
{
"name": "soft_masked_edit",
"label": "paint mask edit",
"core_box": [4, 4, 8, 8],
"sample_box": [4, 4, 8, 8],
"tile_canvas_size": [8, 8],
"tile_canvas_box": [0, 0, 8, 8],
"overlap_edges_px_source": {"left": 0, "right": 0, "top": 0, "bottom": 0},
"blend": 0,
"importance": 1.0,
"priority": 0.0,
"layer": 0,
"occlusion_priority": 0,
"object_mask": replace_soft_mask_data,
"composite_mode": "replace_tile",
},
],
},
"tile_images": [
ttp.pil2tensor(Image.new("RGB", (8, 8), (20, 20, 220)))[0],
],
}
replace_soft_masked, _replace_soft_masked_weights = assemble_node.assemble_tiles(
blend_multiplier=1.0,
output_scale=1.0,
use_priority=True,
base_canvas_mode="source_image",
mask_blend_mode="off",
auto_composite_policy="replace_tile",
source_image=replace_source,
tile_set=replace_soft_mask_tile_set,
)
replace_soft_center = replace_soft_masked.array[0, 8, 8]
assert_equal(float(replace_soft_center[2]) > 0.8, True, "replace_tile mask_first should harden soft mask interiors instead of making the whole edit translucent")
assert_equal(float(replace_soft_center[0]) < 0.2, True, "replace_tile mask_first should remove source pixels from soft mask interiors")
replace_own_mask_full = Image.new("L", (16, 16), 0)
ImageDraw.Draw(replace_own_mask_full).rectangle((7, 7, 8, 8), fill=255)
replace_source_mask_full = Image.new("L", (16, 16), 0)
ImageDraw.Draw(replace_source_mask_full).rectangle((4, 4, 11, 11), fill=255)
replace_own_mask_data = ttp._ttp_encode_object_mask_data([replace_own_mask_full], [0], [4, 4, 12, 12])
replace_source_mask_data = ttp._ttp_encode_object_mask_data([replace_source_mask_full], [0], [4, 4, 12, 12])
replace_own_mask_tile_set = {
"type": "ttp_smart_tile_set",
"original_size": [16, 16],
"tile_meta": {
"type": "ttp_smart_tile",
"original_size": [16, 16],
"tiles": [
{
"name": "own_mask_edit",
"label": "paint mask edit",
"core_box": [4, 4, 8, 8],
"sample_box": [4, 4, 8, 8],
"tile_canvas_size": [8, 8],
"tile_canvas_box": [0, 0, 8, 8],
"overlap_edges_px_source": {"left": 0, "right": 0, "top": 0, "bottom": 0},
"blend": 0,
"importance": 1.0,
"priority": 0.0,
"layer": 0,
"occlusion_priority": 0,
"object_mask": replace_own_mask_data,
"object_mask_source": replace_source_mask_data,
"composite_mode": "replace_tile",
},
],
},
"tile_images": [
ttp.pil2tensor(Image.new("RGB", (8, 8), (20, 20, 220)))[0],
],
}
replace_own_masked, _replace_own_masked_weights = assemble_node.assemble_tiles(
blend_multiplier=1.0,
output_scale=1.0,
use_priority=True,
base_canvas_mode="source_image",
mask_blend_mode="off",
auto_composite_policy="replace_tile",
source_image=replace_source,
tile_set=replace_own_mask_tile_set,
)
replace_source_only_pixel = replace_own_masked.array[0, 4, 4]
replace_own_center_pixel = replace_own_masked.array[0, 8, 8]
assert_equal(float(replace_source_only_pixel[0]) > float(replace_source_only_pixel[2]), True, "replace_tile mask_first should prefer the tile's own cropped object mask over a broader source mask")
assert_equal(float(replace_own_center_pixel[2]) > float(replace_own_center_pixel[0]), True, "replace_tile mask_first should still replace pixels inside the tile's own mask")
subdivided_mask_full = Image.new("L", (64, 64), 0)
ImageDraw.Draw(subdivided_mask_full).rectangle((24, 24, 39, 39), fill=255)
subdivided_mask_data = ttp._ttp_encode_object_mask_data([subdivided_mask_full], [0], [24, 24, 40, 40])
+38
View File
@@ -34,6 +34,8 @@ assert(/async function inferSmartTileLayout/.test(source), "tile editor should e
assert(/app\.queuePrompt\(0,\s*1,\s*\[String\(node\.id\)\]\)/.test(source), "Infer action should queue only the current Smart Tile node");
assert(/ttp-smart-tile-layout/.test(source), "Infer action should listen for backend layout updates");
assert(/auto_detect_request/.test(source), "Infer action should increment the node inference request widget");
assert(/options\.fillGaps !== false/.test(source), "Infer action should allow disabling automatic gap filling");
assert(/modeOverride: "sam3\.1"/.test(source), "Auto SAM should force SAM3.1 detection");
assert(!/\/ttp\/smart_tile\/analyze/.test(source), "Infer action should not use the old analysis route");
assert(/occlusion_priority/.test(source), "interactive layout should preserve auto tile priority metadata");
assert(/TTP_Smart_Tile_Loop_Source_Experimental/.test(source), "frontend should know the Smart Tile Loop Source node");
@@ -54,6 +56,8 @@ assert(/snapCreatedTile/.test(source), "tile editor should snap newly drawn tile
assert(/function fillTileGaps/.test(source), "tile editor should share automatic gap filling logic");
assert(/function autoMaxTiles/.test(source), "tile editor should read the Auto Tile max tile limit");
assert(/const filled = fillTileGaps\(node, layout\.tiles, autoMaxTiles\(node\)\);/.test(source), "Auto Tile gap fill should respect auto_max_tiles");
assert(/if \(fillGaps\) \{\s*const filled = fillTileGaps\(node, layout\.tiles, autoMaxTiles\(node\)\);/s.test(source), "Auto Tile should only fill gaps when requested");
assert(/writeLayout\(node, layout\.tiles, 0\);\s*const coverage = analyzeCoverage\(layout\.tiles\);/s.test(source), "Auto SAM should write detected tiles without gap filling");
assert(/source: "auto_gap"/.test(source), "auto-filled gap tiles should be marked as background gap tiles");
assert(/function editorStageSize/.test(source), "tile editor should compute a fixed source-ratio stage size");
assert(!/aspect-ratio:/.test(source), "tile editor should not rely on CSS aspect-ratio when the node is resized");
@@ -73,16 +77,50 @@ assert(/crop_mask_skip_empty/.test(source), "Grid in should allow skipping empty
assert(/createButton\(`Grid in T\$\{selectedIndex \+ 1\}`, async \(\) =>/.test(source), "Grid in should run asynchronously for mask cropping");
assert(/ensurePaintMaskCanvas/.test(source), "tile editor should create a paint mask canvas");
assert(/auto_paint_mask/.test(source), "tile editor should sync painted masks into the hidden backend input");
assert(/MASK_OVERLAY_COLORS/.test(source), "tile editor should define distinct colors for object-mask overlays");
assert(/function maskOverlayEnabled/.test(source), "tile editor should expose a mask overlay toggle state");
assert(/function stitchMaskPreviewEnabled/.test(source), "tile editor should expose a stitch mask preview toggle state");
assert(/async function renderMaskOverlay/.test(source), "tile editor should render object masks as a colored overlay");
assert(/tintedMaskCanvas/.test(source), "tile editor should tint mask pixels before showing overlays");
assert(/stitchedMaskValues/.test(source), "tile editor should preview hardened replace stitch masks");
assert(!/canvas\.isConnected/.test(source), "mask overlay rendering should not require the staged canvas to be connected before renderEditor appends it");
assert(/Show masks/.test(source), "tile editor should expose a show masks toggle");
assert(/Hide masks/.test(source), "tile editor should expose a hide masks toggle");
assert(/Stitch mask/.test(source), "tile editor should expose a stitch mask preview toggle");
assert(/Raw masks/.test(source), "tile editor should expose a raw mask preview toggle");
assert(/ttpSmartTileShowMasks/.test(source), "mask overlay visibility should be stored on the editor node instance");
assert(/ttpSmartTilePreviewStitchMask/.test(source), "stitch mask preview should be stored on the editor node instance");
assert(/renderMaskOverlay\(node, maskOverlay, tiles\)/.test(source), "renderEditor should draw the colored mask overlay when enabled");
assert(/tintedMaskCanvas\(maskImage, color, \{/.test(source), "renderMaskOverlay should pass preview options into mask tinting");
assert(/node\.ttpSmartTileShowMasks = true;/.test(source), "stitch mask preview should auto-enable colored mask overlays");
assert(/Move/.test(source), "tile editor should expose an explicit move mode");
assert(/node\.ttpSmartTilePaintMode = "off";\s*renderEditor\(node\);/s.test(source), "Move should restore tile drag and resize mode");
assert(/pointer-events:" \+ \(paintMode === "off" \? "auto" : "none"\)/.test(source), "tile regions should only block dragging while Brush or Erase is active");
assert(/Brush/.test(source), "tile editor should expose a paint brush action");
assert(/Erase/.test(source), "tile editor should expose an erase action");
assert(/Mask to Tile/.test(source), "tile editor should expose a button that commits painted masks into layout tiles");
assert(/Mask Replace/.test(source), "tile editor should expose a button that replaces the layout with painted mask tiles");
assert(/Refresh masks/.test(source), "tile editor should expose a refresh masks action");
assert(/refreshInheritedMasks\(node, tiles, gridMaskMode\(node\)\)/.test(source), "Refresh masks should use the current grid mask mode");
assert(/function selectedTileIndexes/.test(source), "tile editor should track multi-selected tiles");
assert(/function setTileSelection/.test(source), "tile editor should support modifier-click multi-selection");
assert(/boolEventModifier\(event\)/.test(beginDrag), "modifier-clicking an existing tile should toggle multi-selection");
assert(/async function mergedObjectMaskForTiles/.test(source), "tile editor should union selected tile masks");
assert(/async function mergeSelectedMasks/.test(source), "tile editor should expose selected mask merging logic");
assert(/async function mergeSelectedTiles/.test(source), "tile editor should expose selected tile merging logic");
assert(/Merge masks/.test(source), "tile editor should expose a merge masks action");
assert(/Merge tiles/.test(source), "tile editor should expose a merge tiles action");
assert(/ownObjectMaskData/.test(source), "mask merging should prefer each tile's cropped mask over parent source masks");
assert(/function addPaintMaskTiles/.test(source), "tile editor should convert painted masks into persisted tiles");
assert(/auto_mask_expand/.test(source), "paint-created tiles should read the auto mask expansion widget");
assert(/expandMaskImageData/.test(source), "paint-created masks should support foreground expansion");
assert(/maskCropData\(canvas, box, maskExpand\)/.test(source), "paint-created masks should apply the requested mask expansion");
assert(/source: "paint_mask"/.test(source), "paint-created tiles should be marked as paint mask tiles");
assert(/object_mask:\s*\{/.test(source), "paint-created tiles should persist object masks in layout metadata");
assert(/Clear mask/.test(source), "tile editor should expose a clear mask action");
assert(/syncPaintMaskWidget\(node\);\s*inferSmartTileLayout\(node\);/s.test(source), "Auto Tile should submit the latest painted mask before inference");
assert(/Auto SAM/.test(source), "tile editor should expose an Auto SAM action");
assert(/syncPaintMaskWidget\(node\);\s*inferSmartTileLayout\(node, \{\s*fillGaps: false,/s.test(source), "Auto SAM should submit masks and disable automatic gap filling");
assert(!/PROMPT_BUILDER_NAME/.test(source), "frontend must not hook the QwenVL prompt builder node");
assert(!/repairPromptBuilderWidgets/.test(source), "prompt builder values must be left to native ComfyUI widget serialization");
assert(!/ttp_prompt_builder_values/.test(source), "prompt builder must not restore stale named widget values from properties");
+648 -35
View File
@@ -10,10 +10,20 @@ const snapGuideThresholdPx = 10;
const largeContextAreaRatio = 0.9;
const STORAGE_PREFIX = "ttp_smart_tile_interactive_layout";
const GRID_MASK_MODES = ["off", "crop_mask", "crop_mask_skip_empty"];
const MASK_OVERLAY_COLORS = [
[248, 113, 113],
[52, 211, 153],
[96, 165, 250],
[251, 191, 36],
[216, 180, 254],
[45, 212, 191],
[251, 113, 133],
[163, 230, 53],
];
const TILE_METADATA_KEYS = [
"name", "source", "label", "score", "layer", "object_id", "occlusion_priority",
"priority", "importance", "pad", "blend", "object_mask", "object_mask_source",
"semantic_category", "semantic_role", "semantic_score", "recommended_scale_weight",
"object_mask_expand", "semantic_category", "semantic_role", "semantic_score", "recommended_scale_weight",
"recommended_layer", "recommended_priority", "recommended_occlusion_priority", "recommended_composite_mode",
];
@@ -191,6 +201,16 @@ function sourceObjectMaskData(tile) {
return null;
}
function ownObjectMaskData(tile) {
if (tile?.object_mask?.format === "png_base64" && tile.object_mask?.data) {
return tile.object_mask;
}
if (tile?.object_mask_source?.format === "png_base64" && tile.object_mask_source?.data) {
return tile.object_mask_source;
}
return null;
}
function loadObjectMaskImage(maskData) {
return new Promise((resolve, reject) => {
if (!maskData || maskData.format !== "png_base64" || !maskData.data) {
@@ -204,6 +224,188 @@ function loadObjectMaskImage(maskData) {
});
}
function maskOverlayEnabled(node) {
return Boolean(node?.ttpSmartTileShowMasks);
}
function stitchMaskPreviewEnabled(node) {
return Boolean(node?.ttpSmartTilePreviewStitchMask);
}
function maskValuesFromImageData(imageData) {
const data = imageData.data;
let maxRgb = 0;
let minAlpha = 255;
let maxAlpha = 0;
for (let index = 0; index < data.length; index += 4) {
maxRgb = Math.max(maxRgb, data[index], data[index + 1], data[index + 2]);
minAlpha = Math.min(minAlpha, data[index + 3]);
maxAlpha = Math.max(maxAlpha, data[index + 3]);
}
const useAlphaOnly = maxRgb === 0 && maxAlpha > minAlpha;
const values = new Uint8Array(Math.floor(data.length / 4));
let hasForeground = false;
for (let index = 0, pixel = 0; index < data.length; index += 4, pixel += 1) {
const maskValue = useAlphaOnly
? data[index + 3]
: Math.max(data[index], data[index + 1], data[index + 2]);
values[pixel] = maskValue;
if (maskValue > 0) {
hasForeground = true;
}
}
return { values, hasForeground };
}
function boxBlurHorizontal(values, width, height, radius) {
const output = new Uint8Array(values.length);
const blurRadius = Math.max(1, Math.round(Number(radius) || 1));
for (let y = 0; y < height; y += 1) {
const rowOffset = y * width;
const prefix = new Uint32Array(width + 1);
for (let x = 0; x < width; x += 1) {
prefix[x + 1] = prefix[x] + values[rowOffset + x];
}
for (let x = 0; x < width; x += 1) {
const left = Math.max(0, x - blurRadius);
const right = Math.min(width - 1, x + blurRadius);
const count = Math.max(1, right - left + 1);
output[rowOffset + x] = Math.round((prefix[right + 1] - prefix[left]) / count);
}
}
return output;
}
function boxBlurVertical(values, width, height, radius) {
const output = new Uint8Array(values.length);
const blurRadius = Math.max(1, Math.round(Number(radius) || 1));
for (let x = 0; x < width; x += 1) {
const prefix = new Uint32Array(height + 1);
for (let y = 0; y < height; y += 1) {
prefix[y + 1] = prefix[y] + values[y * width + x];
}
for (let y = 0; y < height; y += 1) {
const top = Math.max(0, y - blurRadius);
const bottom = Math.min(height - 1, y + blurRadius);
const count = Math.max(1, bottom - top + 1);
output[y * width + x] = Math.round((prefix[bottom + 1] - prefix[top]) / count);
}
}
return output;
}
function blurMaskValues(values, width, height, radius) {
const horizontal = boxBlurHorizontal(values, width, height, radius);
return boxBlurVertical(horizontal, width, height, radius);
}
function stitchedMaskValues(values, width, height, feather) {
const hard = new Uint8Array(values.length);
const threshold = Math.floor(255 * 0.05);
let hasForeground = false;
for (let index = 0; index < values.length; index += 1) {
if (values[index] > threshold) {
hard[index] = 255;
hasForeground = true;
}
}
if (!hasForeground) {
return hard;
}
const radius = Math.max(0, Math.round(Number(feather) || 0));
if (radius <= 0) {
return hard;
}
const stitchRadius = Math.max(1, Math.min(64, Math.round(radius * 0.25)));
const expanded = slidingMaxVertical(
slidingMaxHorizontal(hard, width, height, stitchRadius),
width,
height,
stitchRadius
);
const softened = blurMaskValues(expanded, width, height, stitchRadius);
const output = new Uint8Array(values.length);
for (let index = 0; index < values.length; index += 1) {
output[index] = Math.max(hard[index], softened[index]);
}
return output;
}
function tintedMaskCanvas(maskImage, color, options = {}) {
const width = Math.max(1, Math.round(maskImage?.naturalWidth || maskImage?.width || 1));
const height = Math.max(1, Math.round(maskImage?.naturalHeight || maskImage?.height || 1));
const canvas = document.createElement("canvas");
canvas.width = width;
canvas.height = height;
const context = canvas.getContext("2d", { willReadFrequently: true });
if (!context) {
return null;
}
context.clearRect(0, 0, width, height);
context.drawImage(maskImage, 0, 0, width, height);
const imageData = context.getImageData(0, 0, width, height);
const data = imageData.data;
const extracted = maskValuesFromImageData(imageData);
if (!extracted.hasForeground) {
return null;
}
const values = options.stitch
? stitchedMaskValues(extracted.values, width, height, options.feather)
: extracted.values;
for (let index = 0, pixel = 0; index < data.length; index += 4, pixel += 1) {
const maskValue = values[pixel];
data[index] = color[0];
data[index + 1] = color[1];
data[index + 2] = color[2];
data[index + 3] = Math.round(maskValue * 0.52);
}
context.putImageData(imageData, 0, 0);
return canvas;
}
async function renderMaskOverlay(node, canvas, tiles) {
const sourceSize = imageSourceSize(node);
const context = canvas?.getContext?.("2d", { willReadFrequently: true });
if (!sourceSize || !context) {
return;
}
const token = Symbol("mask-overlay");
node.ttpSmartTileMaskOverlayToken = token;
canvas.width = sourceSize.width;
canvas.height = sourceSize.height;
context.clearRect(0, 0, canvas.width, canvas.height);
for (const [index, tile] of tiles.entries()) {
if (node.ttpSmartTileMaskOverlayToken !== token) {
return;
}
const maskData = ownObjectMaskData(tile);
const maskBox = objectMaskPixelBox(maskData);
if (!maskData?.data || !maskBox) {
continue;
}
const maskImage = await loadObjectMaskImage(maskData);
if (node.ttpSmartTileMaskOverlayToken !== token || !maskImage) {
return;
}
const color = MASK_OVERLAY_COLORS[index % MASK_OVERLAY_COLORS.length];
const tintedMask = tintedMaskCanvas(maskImage, color, {
stitch: stitchMaskPreviewEnabled(node),
feather: Number(tile?.blend ?? layoutDefaults(node).blend ?? 0),
});
if (!tintedMask) {
continue;
}
context.drawImage(
tintedMask,
maskBox.x0,
maskBox.y0,
maskBox.width,
maskBox.height
);
}
}
function maskCanvasHasForeground(canvas) {
const context = canvas?.getContext?.("2d", { willReadFrequently: true });
if (!context || !canvas.width || !canvas.height) {
@@ -460,6 +662,157 @@ async function refreshInheritedMasks(node, tiles, mode) {
return { tiles: result, refreshed, skipped, emptied, unchanged };
}
async function mergedObjectMaskForTiles(node, tiles, selectedIndexes) {
const entries = [];
for (const index of selectedIndexes) {
const maskData = ownObjectMaskData(tiles[index]);
const maskBox = objectMaskPixelBox(maskData);
if (!maskData?.data || !maskBox) {
continue;
}
const maskImage = await loadObjectMaskImage(maskData);
if (!maskImage) {
continue;
}
entries.push({ maskImage, maskBox });
}
if (!entries.length) {
return null;
}
const x0 = Math.min(...entries.map((entry) => entry.maskBox.x0));
const y0 = Math.min(...entries.map((entry) => entry.maskBox.y0));
const x1 = Math.max(...entries.map((entry) => entry.maskBox.x1));
const y1 = Math.max(...entries.map((entry) => entry.maskBox.y1));
const width = Math.max(1, x1 - x0);
const height = Math.max(1, y1 - y0);
const canvas = document.createElement("canvas");
canvas.width = width;
canvas.height = height;
const context = canvas.getContext("2d", { willReadFrequently: true });
if (!context) {
return null;
}
context.clearRect(0, 0, width, height);
context.globalCompositeOperation = "lighter";
for (const entry of entries) {
context.drawImage(
entry.maskImage,
entry.maskBox.x0 - x0,
entry.maskBox.y0 - y0,
entry.maskBox.width,
entry.maskBox.height
);
}
context.globalCompositeOperation = "source-over";
if (!maskCanvasHasForeground(canvas)) {
return null;
}
const maskData = {
format: "png_base64",
box: [Math.round(x0), Math.round(y0), Math.round(width), Math.round(height)],
width: Math.round(width),
height: Math.round(height),
data: canvas.toDataURL("image/png").split(",", 2)[1],
};
return {
maskData,
maskBox: { x0, y0, x1, y1, width, height },
maskImage: canvas,
count: entries.length,
};
}
async function mergeSelectedMasks(node, tiles, selectedIndexes, selectedIndex) {
if (selectedIndexes.length < 2) {
node.ttpSmartTileStatus = "Select at least two tiles.";
return false;
}
const merged = await mergedObjectMaskForTiles(node, tiles, selectedIndexes);
if (!merged) {
node.ttpSmartTileStatus = "Selected tiles have no masks to merge.";
return false;
}
const selectedSet = new Set(selectedIndexes);
let updated = 0;
const nextTiles = tiles.map((tile, index) => {
if (!selectedSet.has(index)) {
return tile;
}
const nextTile = { ...tile };
const cropped = cropObjectMaskForTile(node, merged.maskImage, merged.maskBox, nextTile);
if (cropped) {
nextTile.object_mask = cropped;
nextTile.object_mask_source = merged.maskData;
updated += 1;
} else {
delete nextTile.object_mask;
delete nextTile.object_mask_source;
}
return normalizeTile(node, nextTile);
});
if (!updated) {
node.ttpSmartTileStatus = "Merged mask does not overlap selected tiles.";
return false;
}
writeLayout(node, nextTiles, selectedIndex, selectedIndexes);
node.ttpSmartTileStatus = `Merged ${merged.count} mask(s) across ${updated} tile(s).`;
return true;
}
async function mergeSelectedTiles(node, tiles, selectedIndexes, selectedIndex) {
if (selectedIndexes.length < 2) {
node.ttpSmartTileStatus = "Select at least two tiles.";
return false;
}
const selectedSet = new Set(selectedIndexes);
const selectedTiles = selectedIndexes.map((index) => tiles[index]).filter(Boolean);
if (selectedTiles.length < 2) {
return false;
}
const primaryIndex = selectedSet.has(selectedIndex) ? selectedIndex : selectedIndexes[0];
const primaryTile = tiles[primaryIndex] ?? selectedTiles[0];
const x0 = Math.min(...selectedTiles.map((tile) => Number(tile.x0)));
const y0 = Math.min(...selectedTiles.map((tile) => Number(tile.y0)));
const x1 = Math.max(...selectedTiles.map((tile) => Number(tile.x1)));
const y1 = Math.max(...selectedTiles.map((tile) => Number(tile.y1)));
const selectorText = selectedIndexes.map((index) => `T${index + 1}`).join("+");
const mergedTile = normalizeTile(node, {
...primaryTile,
x0,
y0,
x1,
y1,
name: `merged_${selectorText.toLowerCase()}`,
source: "manual_merge",
label: `merged ${selectorText}`,
});
const mergedMask = await mergedObjectMaskForTiles(node, tiles, selectedIndexes);
if (mergedMask) {
const cropped = cropObjectMaskForTile(node, mergedMask.maskImage, mergedMask.maskBox, mergedTile);
if (cropped) {
mergedTile.object_mask = cropped;
mergedTile.object_mask_source = mergedMask.maskData;
} else {
delete mergedTile.object_mask;
delete mergedTile.object_mask_source;
}
} else {
delete mergedTile.object_mask;
delete mergedTile.object_mask_source;
}
const insertIndex = Math.min(...selectedIndexes);
const remaining = tiles.filter((_tile, index) => !selectedSet.has(index));
remaining.splice(insertIndex, 0, mergedTile);
writeLayout(node, remaining, insertIndex, [insertIndex]);
node.ttpSmartTileStatus = mergedMask
? `Merged ${selectedIndexes.length} tile(s) with ${mergedMask.count} mask(s).`
: `Merged ${selectedIndexes.length} tile(s).`;
return true;
}
function sourceStorageKey(node) {
const size = imageSourceSize(node);
const sizeKey = size ? `${Math.round(size.width)}x${Math.round(size.height)}` : "unknown";
@@ -634,18 +987,100 @@ function paintMaskComponents(canvas, minArea = 24) {
return components;
}
function maskCropData(canvas, box) {
function slidingMaxHorizontal(values, width, height, radius) {
const output = new Uint8Array(values.length);
for (let y = 0; y < height; y += 1) {
const rowOffset = y * width;
const deque = [];
let head = 0;
let added = -1;
for (let x = 0; x < width; x += 1) {
const limit = Math.min(width - 1, x + radius);
while (added < limit) {
added += 1;
const value = values[rowOffset + added];
while (deque.length > head && values[rowOffset + deque[deque.length - 1]] <= value) {
deque.pop();
}
deque.push(added);
}
const minX = x - radius;
while (deque.length > head && deque[head] < minX) {
head += 1;
}
output[rowOffset + x] = deque.length > head ? values[rowOffset + deque[head]] : 0;
}
}
return output;
}
function slidingMaxVertical(values, width, height, radius) {
const output = new Uint8Array(values.length);
for (let x = 0; x < width; x += 1) {
const deque = [];
let head = 0;
let added = -1;
for (let y = 0; y < height; y += 1) {
const limit = Math.min(height - 1, y + radius);
while (added < limit) {
added += 1;
const value = values[added * width + x];
while (deque.length > head && values[deque[deque.length - 1] * width + x] <= value) {
deque.pop();
}
deque.push(added);
}
const minY = y - radius;
while (deque.length > head && deque[head] < minY) {
head += 1;
}
output[y * width + x] = deque.length > head ? values[deque[head] * width + x] : 0;
}
}
return output;
}
function expandMaskImageData(imageData, width, height, expandPx) {
const radius = Math.max(0, Math.min(2048, Math.round(Number(expandPx) || 0)));
if (radius <= 0) {
return imageData;
}
const source = new Uint8Array(width * height);
const data = imageData.data;
for (let index = 0; index < source.length; index += 1) {
const offset = index * 4;
source[index] = Math.max(data[offset], data[offset + 1], data[offset + 2], data[offset + 3]);
}
const horizontal = slidingMaxHorizontal(source, width, height, radius);
const expanded = slidingMaxVertical(horizontal, width, height, radius);
for (let index = 0; index < expanded.length; index += 1) {
const value = expanded[index];
const offset = index * 4;
data[offset] = value;
data[offset + 1] = value;
data[offset + 2] = value;
data[offset + 3] = 255;
}
return imageData;
}
function maskCropData(canvas, box, expandPx = 0) {
const width = Math.max(1, Math.round(box.x1 - box.x0));
const height = Math.max(1, Math.round(box.y1 - box.y0));
const crop = document.createElement("canvas");
crop.width = width;
crop.height = height;
const context = crop.getContext("2d");
const context = crop.getContext("2d", { willReadFrequently: true });
context.drawImage(canvas, box.x0, box.y0, width, height, 0, 0, width, height);
const radius = Math.max(0, Math.round(Number(expandPx) || 0));
if (radius > 0) {
const imageData = context.getImageData(0, 0, width, height);
context.putImageData(expandMaskImageData(imageData, width, height, radius), 0, 0);
}
return crop.toDataURL("image/png").split(",", 2)[1];
}
function addPaintMaskTiles(node, tiles, selectedIndex) {
function addPaintMaskTiles(node, tiles, selectedIndex, replaceExisting = false) {
const canvas = ensurePaintMaskCanvas(node);
const size = imageSourceSize(node);
if (!canvas || !size || !paintMaskHasPixels(canvas)) {
@@ -658,8 +1093,9 @@ function addPaintMaskTiles(node, tiles, selectedIndex) {
return false;
}
const padding = Math.max(0, Math.round(Number(widgetByName(node, "auto_object_padding")?.value ?? 96)));
const maskExpand = Math.max(0, Math.round(Number(widgetByName(node, "auto_mask_expand")?.value ?? 16)));
const defaults = layoutDefaults(node);
const nextTiles = [...tiles];
const nextTiles = replaceExisting ? [] : [...tiles];
let added = 0;
for (const component of components) {
if (nextTiles.length >= MAX_TILES) {
@@ -686,12 +1122,13 @@ function addPaintMaskTiles(node, tiles, selectedIndex) {
importance: 1.0,
layer: 5,
occlusion_priority: 3200 + nextTiles.length,
object_mask_expand: maskExpand,
object_mask: {
format: "png_base64",
box: [Math.round(box.x0), Math.round(box.y0), Math.round(width), Math.round(height)],
width: Math.round(width),
height: Math.round(height),
data: maskCropData(canvas, box),
data: maskCropData(canvas, box, maskExpand),
},
}));
added += 1;
@@ -700,14 +1137,63 @@ function addPaintMaskTiles(node, tiles, selectedIndex) {
node.ttpSmartTileStatus = `Mask has ${components.length} region(s), but max ${MAX_TILES} tiles is reached.`;
return false;
}
writeLayout(node, nextTiles, Math.min(nextTiles.length - 1, Math.max(0, selectedIndex + added)));
const nextSelected = replaceExisting
? Math.max(0, added - 1)
: Math.min(nextTiles.length - 1, Math.max(0, selectedIndex + added));
writeLayout(node, nextTiles, nextSelected);
clearPaintMask(node);
node.ttpSmartTilePaintMode = "off";
node.ttpSmartTileStatus = `Mask to Tile added ${added} tile(s).`;
node.ttpSmartTileStatus = replaceExisting
? `Mask Replace created ${added} tile(s).`
: `Mask to Tile added ${added} tile(s).`;
return true;
}
function writeLayout(node, tiles, selectedIndex = 0) {
function selectedTileIndexes(node, count, fallbackIndex = 0) {
const fallback = Math.max(0, Math.min(Math.max(0, count - 1), Number(fallbackIndex ?? 0)));
const raw = Array.isArray(node.ttpSmartTileSelectedIndexes)
? node.ttpSmartTileSelectedIndexes
: [node.ttpSmartTileSelectedIndex ?? fallback];
const result = [];
const seen = new Set();
for (const value of raw) {
const index = Math.round(Number(value));
if (Number.isFinite(index) && index >= 0 && index < count && !seen.has(index)) {
result.push(index);
seen.add(index);
}
}
if (!result.length && count > 0) {
result.push(fallback);
}
return result.sort((a, b) => a - b);
}
function setTileSelection(node, index, count, event = null) {
const safeIndex = Math.max(0, Math.min(Math.max(0, count - 1), Math.round(Number(index) || 0)));
const multi = boolEventModifier(event);
if (!multi) {
node.ttpSmartTileSelectedIndex = safeIndex;
node.ttpSmartTileSelectedIndexes = [safeIndex];
return;
}
const current = selectedTileIndexes(node, count, safeIndex);
const selected = new Set(current);
if (selected.has(safeIndex) && selected.size > 1) {
selected.delete(safeIndex);
} else {
selected.add(safeIndex);
}
const next = [...selected].sort((a, b) => a - b);
node.ttpSmartTileSelectedIndexes = next;
node.ttpSmartTileSelectedIndex = safeIndex;
}
function boolEventModifier(event) {
return Boolean(event?.shiftKey) || Boolean(event?.metaKey) || Boolean(event?.ctrlKey);
}
function writeLayout(node, tiles, selectedIndex = 0, selectedIndexes = null) {
const normalizedTiles = (Array.isArray(tiles) && tiles.length ? tiles : defaultTiles(node))
.slice(0, MAX_TILES)
.map((tile, index) => ({
@@ -729,6 +1215,9 @@ function writeLayout(node, tiles, selectedIndex = 0) {
}
node.ttpSmartTileLayout = layout;
node.ttpSmartTileSelectedIndex = Math.max(0, Math.min(normalizedTiles.length - 1, Number(selectedIndex ?? 0)));
node.ttpSmartTileSelectedIndexes = selectedIndexes === null
? [node.ttpSmartTileSelectedIndex]
: selectedTileIndexes({ ttpSmartTileSelectedIndexes: selectedIndexes }, normalizedTiles.length, node.ttpSmartTileSelectedIndex);
storeLayout(node, normalizedTiles);
}
@@ -989,18 +1478,26 @@ async function loadSelectedInputImage(node, force = false) {
renderEditor(node);
}
async function inferSmartTileLayout(node) {
const mode = String(widgetByName(node, "auto_detect_mode")?.value ?? "none");
async function inferSmartTileLayout(node, options = {}) {
const modeWidget = widgetByName(node, "auto_detect_mode");
if (options.modeOverride && modeWidget) {
modeWidget.value = String(options.modeOverride);
}
const mode = String(modeWidget?.value ?? "none");
if (mode === "none") {
node.ttpSmartTileStatus = "Set auto detect mode to sam3.1 or qwenvl3 before inference.";
renderEditor(node);
return;
}
const fillGaps = options.fillGaps !== false;
const actionLabel = String(options.label || (fillGaps ? "Auto Tile" : "Auto SAM"));
const requestWidget = widgetByName(node, "auto_detect_request");
if (requestWidget) {
requestWidget.value = Number(requestWidget.value ?? 0) + 1;
}
node.ttpSmartTileStatus = `Queued ${mode} inference for this node only...`;
node.ttpSmartTileAutoFillGaps = fillGaps;
node.ttpSmartTileInferenceLabel = actionLabel;
node.ttpSmartTileStatus = `Queued ${actionLabel} (${mode}) inference for this node only...`;
renderEditor(node);
try {
await app.queuePrompt(0, 1, [String(node.id)]);
@@ -1025,6 +1522,8 @@ function applyInferenceResult(detail) {
if (requestWidget) {
requestWidget.value = 0;
}
const fillGaps = node.ttpSmartTileAutoFillGaps !== false;
const actionLabel = String(node.ttpSmartTileInferenceLabel || (fillGaps ? "Auto Tile" : "Auto SAM"));
const statusParts = [];
if (detail.message) {
statusParts.push(detail.message);
@@ -1035,13 +1534,22 @@ function applyInferenceResult(detail) {
try {
const layout = JSON.parse(detail.layout_json);
if (Array.isArray(layout?.tiles) && layout.tiles.length) {
const filled = fillTileGaps(node, layout.tiles, autoMaxTiles(node));
writeLayout(node, filled.tiles, 0);
if (filled.added > 0) {
statusParts.push(`Auto Tile added ${filled.added} gap tile(s).`);
}
if (filled.coverage.gaps.length > 0) {
statusParts.push(`${filled.coverage.gaps.length} gap(s) remain.`);
if (fillGaps) {
const filled = fillTileGaps(node, layout.tiles, autoMaxTiles(node));
writeLayout(node, filled.tiles, 0);
if (filled.added > 0) {
statusParts.push(`Auto Tile added ${filled.added} gap tile(s).`);
}
if (filled.coverage.gaps.length > 0) {
statusParts.push(`${filled.coverage.gaps.length} gap(s) remain.`);
}
} else {
writeLayout(node, layout.tiles, 0);
const coverage = analyzeCoverage(layout.tiles);
statusParts.push(`${actionLabel} kept detected tile(s) without gap filling.`);
if (coverage.gaps.length > 0) {
statusParts.push(`${coverage.gaps.length} gap(s) remain.`);
}
}
}
} catch (error) {
@@ -1225,7 +1733,16 @@ function renderEditor(node) {
const currentTiles = node.ttpSmartTileLayout?.tiles ?? parseLayout(node);
const tiles = currentTiles.map((tile) => normalizeTile(node, tile));
const selectedIndex = Math.max(0, Math.min(tiles.length - 1, Number(node.ttpSmartTileSelectedIndex ?? 0)));
writeLayout(node, tiles, selectedIndex);
const selectedIndexes = selectedTileIndexes(node, tiles.length, selectedIndex);
const selectedSet = new Set(selectedIndexes);
const hasTileMasks = tiles.some((tile) => Boolean(ownObjectMaskData(tile)?.data));
if (!hasTileMasks && maskOverlayEnabled(node)) {
node.ttpSmartTileShowMasks = false;
}
if (!hasTileMasks && stitchMaskPreviewEnabled(node)) {
node.ttpSmartTilePreviewStitchMask = false;
}
writeLayout(node, tiles, selectedIndex, selectedIndexes);
const sourceSize = imageSourceSize(node);
const selectedRect = tilePixelRect(node, tiles[selectedIndex]);
@@ -1288,6 +1805,26 @@ function renderEditor(node) {
stage.append(empty);
}
if (sourceSize && hasTileMasks && maskOverlayEnabled(node)) {
const maskOverlay = document.createElement("canvas");
maskOverlay.title = "Tile object masks";
maskOverlay.style.cssText = [
"position:absolute",
"inset:0",
"width:100%",
"height:100%",
"pointer-events:none",
"z-index:8",
].join(";");
stage.append(maskOverlay);
renderMaskOverlay(node, maskOverlay, tiles).catch((error) => {
node.ttpSmartTileStatus = error?.message || "Mask overlay failed.";
scheduleCanvas(node);
});
} else {
node.ttpSmartTileMaskOverlayToken = null;
}
const paintCanvas = ensurePaintMaskCanvas(node);
if (paintCanvas) {
paintCanvas.style.cssText = [
@@ -1469,7 +2006,10 @@ function renderEditor(node) {
y1: nearestSnap(tile.y1, "y", skipIndex),
});
const renderOrder = tiles.map((_tile, index) => index).filter((index) => index !== selectedIndex);
const renderOrder = tiles
.map((_tile, index) => index)
.filter((index) => !selectedSet.has(index));
renderOrder.push(...selectedIndexes.filter((index) => index !== selectedIndex));
renderOrder.push(selectedIndex);
const regionElements = new Map();
@@ -1481,7 +2021,12 @@ function renderEditor(node) {
const index = Math.max(0, Math.min(tiles.length - 1, selected));
const dragMode = resizeHit(tiles[index], point) ? "resize" : mode;
const targetRegion = regionElements.get(index) ?? region;
node.ttpSmartTileSelectedIndex = index;
if (boolEventModifier(event) && dragMode !== "resize") {
setTileSelection(node, index, tiles.length, event);
renderEditor(node);
return;
}
setTileSelection(node, index, tiles.length);
const startPoint = pointFromEvent(event);
const startClientX = event.clientX;
const startClientY = event.clientY;
@@ -1544,10 +2089,11 @@ function renderEditor(node) {
for (const index of renderOrder) {
const tile = tiles[index];
const selected = index === selectedIndex;
const selected = selectedSet.has(index);
const primarySelected = index === selectedIndex;
const region = document.createElement("div");
region.textContent = String(index + 1);
region.title = `Tile ${index + 1}`;
region.title = `Tile ${index + 1}${selected && !primarySelected ? " selected" : ""}`;
region.style.cssText = [
"position:absolute",
"box-sizing:border-box",
@@ -1558,13 +2104,13 @@ function renderEditor(node) {
`top:${tile.y0 * 100}%`,
`width:${Math.max(0.1, (tile.x1 - tile.x0) * 100)}%`,
`height:${Math.max(0.1, (tile.y1 - tile.y0) * 100)}%`,
"border:" + (selected ? "2px solid #f8fafc" : "1px solid rgba(226,232,240,.55)"),
"background:" + (selected ? "rgba(14,165,233,.28)" : "rgba(15,23,42,.20)"),
"border:" + (primarySelected ? "2px solid #f8fafc" : selected ? "2px solid #38bdf8" : "1px solid rgba(226,232,240,.55)"),
"background:" + (primarySelected ? "rgba(14,165,233,.28)" : selected ? "rgba(14,165,233,.18)" : "rgba(15,23,42,.20)"),
"color:#f8fafc",
"font-weight:800",
"letter-spacing:0",
"box-shadow:" + (selected ? "0 0 0 1px rgba(15,23,42,.85),0 0 16px rgba(14,165,233,.55)" : "none"),
"z-index:" + (selected ? "100" : String(10 + index)),
"box-shadow:" + (primarySelected ? "0 0 0 1px rgba(15,23,42,.85),0 0 16px rgba(14,165,233,.55)" : selected ? "0 0 0 1px rgba(15,23,42,.65)" : "none"),
"z-index:" + (primarySelected ? "100" : selected ? "90" : String(10 + index)),
"cursor:move",
"user-select:none",
"pointer-events:" + (paintMode === "off" ? "auto" : "none"),
@@ -1598,7 +2144,7 @@ function renderEditor(node) {
const point = pointFromEvent(event);
const hits = renderOrder.filter((index) => tileContainsPoint(tiles[index], point));
if (hits.length || tiles.length >= MAX_TILES) {
node.ttpSmartTileSelectedIndex = hits.pop() ?? selectedIndex;
setTileSelection(node, hits.pop() ?? selectedIndex, tiles.length, event);
renderEditor(node);
return;
}
@@ -1779,6 +2325,16 @@ function renderEditor(node) {
node.ttpSmartTilePaintMode = node.ttpSmartTilePaintMode === "erase" ? "off" : "erase";
renderEditor(node);
});
const moveButton = createButton("Move", () => {
node.ttpSmartTilePaintMode = "off";
renderEditor(node);
});
moveButton.title = "Return to tile move and resize mode";
if (paintMode === "off") {
moveButton.style.background = "#a7f3d0";
moveButton.style.borderColor = "#34d399";
moveButton.style.color = "#064e3b";
}
if (paintMode === "paint") {
paintButton.style.background = "#38bdf8";
paintButton.style.borderColor = "#7dd3fc";
@@ -1789,8 +2345,34 @@ function renderEditor(node) {
eraseButton.style.borderColor = "#e2e8f0";
eraseButton.style.color = "#0f172a";
}
const maskOverlayButton = createButton(maskOverlayEnabled(node) ? "Hide masks" : "Show masks", () => {
node.ttpSmartTileShowMasks = !maskOverlayEnabled(node);
renderEditor(node);
}, !hasTileMasks);
maskOverlayButton.title = "Toggle colored object-mask overlay";
if (maskOverlayEnabled(node)) {
maskOverlayButton.style.background = "#a7f3d0";
maskOverlayButton.style.borderColor = "#34d399";
maskOverlayButton.style.color = "#064e3b";
}
const stitchMaskButton = createButton(stitchMaskPreviewEnabled(node) ? "Raw masks" : "Stitch mask", () => {
node.ttpSmartTilePreviewStitchMask = !stitchMaskPreviewEnabled(node);
if (node.ttpSmartTilePreviewStitchMask) {
node.ttpSmartTileShowMasks = true;
}
renderEditor(node);
}, !hasTileMasks);
stitchMaskButton.title = "Preview the hardened and feathered mask shape used by replace pasteback";
if (stitchMaskPreviewEnabled(node)) {
stitchMaskButton.style.background = "#fde68a";
stitchMaskButton.style.borderColor = "#f59e0b";
stitchMaskButton.style.color = "#78350f";
}
paintControls.append(
paintLabel,
moveButton,
maskOverlayButton,
stitchMaskButton,
paintButton,
eraseButton,
document.createTextNode("px"),
@@ -1802,6 +2384,13 @@ function renderEditor(node) {
renderEditor(node);
}
}, !paintMaskHasPixels(ensurePaintMaskCanvas(node)) || tiles.length >= MAX_TILES),
createButton("Mask Replace", () => {
if (addPaintMaskTiles(node, tiles, 0, true)) {
renderEditor(node);
} else {
renderEditor(node);
}
}, !paintMaskHasPixels(ensurePaintMaskCanvas(node))),
createButton("Refresh masks", async () => {
updateGridValues();
const refreshed = await refreshInheritedMasks(node, tiles, gridMaskMode(node)).catch((error) => {
@@ -1833,6 +2422,13 @@ function renderEditor(node) {
writeLayout(node, refreshed.tiles, Math.min(selectedIndex, refreshed.tiles.length - 1));
renderEditor(node);
}, !tiles.some((tile) => tile?.object_mask_source?.data)),
createButton("Merge masks", async () => {
await mergeSelectedMasks(node, tiles, selectedIndexes, selectedIndex).catch((error) => {
node.ttpSmartTileStatus = error?.message || "Merge masks failed.";
return false;
});
renderEditor(node);
}, selectedIndexes.length < 2),
createButton("Clear mask", () => {
clearPaintMask(node);
node.ttpSmartTilePaintMode = "off";
@@ -1842,13 +2438,13 @@ function renderEditor(node) {
controls.append(paintControls);
for (const [index] of tiles.entries()) {
const button = createButton(`T${index + 1}`, () => {
node.ttpSmartTileSelectedIndex = index;
const button = createButton(`T${index + 1}`, (event) => {
setTileSelection(node, index, tiles.length, event);
renderEditor(node);
});
if (index === selectedIndex) {
button.style.background = "#38bdf8";
button.style.borderColor = "#7dd3fc";
if (selectedSet.has(index)) {
button.style.background = index === selectedIndex ? "#38bdf8" : "#bae6fd";
button.style.borderColor = index === selectedIndex ? "#7dd3fc" : "#38bdf8";
button.style.color = "#082f49";
}
actions.append(button);
@@ -1859,6 +2455,14 @@ function renderEditor(node) {
syncPaintMaskWidget(node);
inferSmartTileLayout(node);
}),
createButton("Auto SAM", () => {
syncPaintMaskWidget(node);
inferSmartTileLayout(node, {
fillGaps: false,
label: "Auto SAM",
modeOverride: "sam3.1",
});
}),
createButton("Add tile", () => {
const base = tiles[selectedIndex] ?? { x0: 0.25, y0: 0.25, x1: 0.75, y1: 0.75 };
const width = Math.max(0.18, Math.min(0.48, base.x1 - base.x0));
@@ -1884,6 +2488,13 @@ function renderEditor(node) {
writeLayout(node, tiles.filter((_tile, index) => index !== selectedIndex), Math.max(0, selectedIndex - 1));
renderEditor(node);
}, tiles.length <= 1),
createButton("Merge tiles", async () => {
await mergeSelectedTiles(node, tiles, selectedIndexes, selectedIndex).catch((error) => {
node.ttpSmartTileStatus = error?.message || "Merge tiles failed.";
return false;
});
renderEditor(node);
}, selectedIndexes.length < 2),
createButton("Fill gaps", () => {
const filled = fillTileGaps(node, tiles);
node.ttpSmartTileStatus = filled.coverage.gaps.length
@@ -1902,7 +2513,9 @@ function renderEditor(node) {
const status = document.createElement("div");
status.style.cssText = "display:flex;gap:10px;flex-wrap:wrap;opacity:.76;";
const selectedLabel = selectedRect
? `T${selectedIndex + 1}: x${selectedRect.x0}, y${selectedRect.y0}, ${selectedRect.width}x${selectedRect.height}`
? selectedIndexes.length > 1
? `${selectedIndexes.map((index) => `T${index + 1}`).join(",")} selected / active T${selectedIndex + 1}`
: `T${selectedIndex + 1}: x${selectedRect.x0}, y${selectedRect.y0}, ${selectedRect.width}x${selectedRect.height}`
: `T${selectedIndex + 1}`;
const coverageLabel = coverage.gaps.length
? `${coverage.gaps.length} gap(s), ${(coverage.uncoveredArea * 100).toFixed(2)}%`