nodes/TTPlanetPig-Comfyui_TTP_Toolset
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Update Smart Tile README and workflow example

Browse Source
This commit is contained in:
TTPlanetPIG
2026-07-05 12:25:23 +08:00
parent e6c5bd4f2e
commit c317bf8553
11 changed files with 3319 additions and 53 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+41
View File
@@ -100,6 +100,47 @@ Use `TTP Smart Tile Loop Source` and `TTP Smart Tile Loop Collect` instead of ma
By default, `TTP Smart Tile Assemble` uses `assemble_mode=final_only`. Connect `TTP Smart Tile Loop Collect.done` to `TTP Smart Tile Assemble.done` so unfinished loop runs return a lightweight preview while `done=false`, then perform the full assemble once after the last tile. Switch `assemble_mode` to `always` only when you really want a full recomposite after every tile. If pixel alignment is enabled, unfinished loop runs are automatically treated as final-only to avoid repeated expensive alignment passes.
#### **Complete Example Workflow / 完整范例工作流**
Example workflow / 示例工作流: [`examples/smart_tile_qwen_sam_loop_example_v2.json`](examples/smart_tile_qwen_sam_loop_example_v2.json)
这份工作流展示了当前推荐的 Smart Tile 完整链路：先用 `TTP Smart Tile Interactive Crop` 和 `sam3.1` 做 Auto Tile 分块，再用 `TTP Smart Tile QwenVL Prompt Set Builder` 生成每块提示词。随后 `Semantic Rank` 写入语义优先级，`Loop Source` 把 tile 一张一张送入 img2img，`Image Upscale Prep` 负责采样前放大，`Loop Collect` 收集结果，最后由 `Output Size Estimate` 估算画布尺寸，`Assemble` 在 `final_only` 模式下一次拼合，`Save Final Image` 只保存最终图。
This workflow is the recommended full Smart Tile loop. It uses `TTP Smart Tile Interactive Crop` with `sam3.1` Auto Tile, prepares per-tile prompts with `TTP Smart Tile QwenVL Prompt Set Builder`, ranks tiles with `Semantic Rank`, sends one tile at a time through img2img with `Loop Source`, upscales each tile before sampling with `Image Upscale Prep`, collects results with `Loop Collect`, estimates the final canvas with `Output Size Estimate`, assembles once in `final_only` mode, and saves only the completed result.
**Workflow overview / 工作流全景**
<img src="docs/images/smart_tile_workflow_chrome_overview.png" alt="Smart Tile complete workflow in ComfyUI" width="960">
**Key views / 关键视图**
<table>
<tr>
<td width="50%"><strong>Interactive editor / 交互分块编辑器</strong><br><img src="docs/images/smart_tile_chrome_interactive_editor.png" alt="Smart Tile interactive editor" width="420"></td>
<td width="50%"><strong>Tile set preview / 分块预览</strong><br><img src="docs/images/smart_tile_chrome_tile_set_preview.png" alt="Smart Tile tile set preview" width="420"></td>
</tr>
<tr>
<td width="50%"><strong>Weight preview / 权重预览</strong><br><img src="docs/images/smart_tile_chrome_weight_preview.png" alt="Smart Tile weight preview" width="420"></td>
<td width="50%"><strong>Final preview / 最终预览</strong><br><img src="docs/images/smart_tile_chrome_final_preview.png" alt="Smart Tile final preview" width="420"></td>
</tr>
</table>
**Key settings / 关键设置**
| Node / 节点 | Example setting / 示例设置 | Notes / 说明 |
|---|---|---|
| `Interactive Crop` | `auto_detect_mode=sam3.1`, `default_pad=32`, `default_blend=32`, `auto_object_padding=64`, `auto_max_tiles=16` | Use `Auto Tile` to create an object-aware layout, then set `auto_detect_mode=none` if you want to freeze manual edits. / 用 `Auto Tile` 生成语义分块；如果之后要固定手动编辑结果，可以把 `auto_detect_mode` 改回 `none`。 |
| `QwenVL3 Local Loader` | `qwen3vl_4b_fp8_scaled.safetensors` | Replace this with the QwenVL model installed in `ComfyUI/models/text_encoders`. / 按你本机 `models/text_encoders` 里的 QwenVL 模型替换。 |
| `QwenVL Prompt Set Builder` | `reference_image_mode=first_message`, `prompt_preset=tile_img2img_prompt`, `output_language=chinese` | Builds prompts before the tile loop, then `Loop Source.prompt` feeds the text encoder. / 在循环前一次性生成每块提示词，然后由 `Loop Source.prompt` 送入文本编码。 |
| `Image Upscale Prep` | `scale=2.5`, `round_to=8`, `max_megapixels=1.5`, `use_upscale_model=true` | Enlarges each tile before img2img while keeping each tile under the megapixel cap. / 采样前放大每块 tile，同时用百万像素上限控制显存。 |
| `Loop Source` / `Loop Collect` | `Process All Tiles` workflow | The tile count can be 4, 8, 16, or any layout count; no manual sampler duplication is needed. / 分块数量可以变化，不需要手动复制多套 sampler。 |
| `Output Size Estimate` | `focus_weighted` | Estimates the final canvas from processed tile sizes, giving more influence to focus/detail tiles. / 根据处理后的 tile 尺寸估算最终画布，并更重视细节块。 |
| `Assemble` | `assemble_mode=final_only`, `mask_blend_mode=mask_feather`, `color_correction=histogram`, `small_tile_on_top=true`, `auto_composite_policy=safe_auto` | Assembles only after the loop is done, uses mask-aware feathering, and keeps small detail tiles above larger context tiles. / 循环结束后只拼合一次，使用蒙版羽化，并让小细节块优先于大背景块。 |
Before running the example, replace model filenames such as the diffusion model, CLIP/text encoder, VAE, LoRA, and upscale model with files installed on your machine. The workflow is meant as a wiring reference; model choices can be changed freely.
运行示例前，请把 diffusion model、CLIP/text encoder、VAE、LoRA、upscale model 等模型名替换成你本机已经安装的文件。这份工作流主要作为连线参考，具体模型可以自由替换。
#### **Auto Tile: SAM3.1 or QwenVL3**
![Smart Tile automatic detection modes](docs/images/smart_tile_auto_modes.svg)
TTP_toolsets.py
+360 -46
View File
@@ -731,7 +731,9 @@ def _ttp_decode_object_mask_data(mask_data):
def _ttp_tile_object_mask_array(tile, out_width, out_height, mask_blend_mode="mask_feather", feather=0):
if str(mask_blend_mode) == "off":
return None
mask_data = tile.get("object_mask")
mask_data = tile.get("object_mask_source")
if not isinstance(mask_data, dict) or mask_data.get("format") != "png_base64":
mask_data = tile.get("object_mask")
mask_pil = _ttp_decode_object_mask_data(mask_data)
if mask_pil is None:
return None
@@ -756,7 +758,9 @@ def _ttp_tile_object_mask_array(tile, out_width, out_height, mask_blend_mode="ma
)
source_mask = source_mask.resize((out_width, out_height), Image.Resampling.BILINEAR)
if str(mask_blend_mode) in ("auto", "mask_feather"):
radius = max(0.0, min(64.0, float(feather) * 0.35))
min_radius = min(12.0, max(0.75, min(int(out_width), int(out_height)) * 0.006))
radius = max(min_radius, float(feather) * 0.35)
radius = max(0.0, min(64.0, radius))
if radius > 0:
source_mask = source_mask.filter(ImageFilter.GaussianBlur(radius=radius))
array = np.array(source_mask).astype(np.float32) / 255.0
@@ -849,10 +853,10 @@ def _ttp_is_detail_tile_text(text):
def _ttp_tile_area_ratio(tile, original_area):
sample_box = tile.get("sample_box", [0, 0, 0, 0])
if len(sample_box) < 4:
box = tile.get("core_box", tile.get("sample_box", [0, 0, 0, 0]))
if len(box) < 4:
return 0.0
area = max(0.0, float(sample_box[2])) * max(0.0, float(sample_box[3]))
area = max(0.0, float(box[2])) * max(0.0, float(box[3]))
return _ttp_clamp(area / max(1.0, float(original_area)), 0.0, 1.0)
@@ -1002,7 +1006,7 @@ def _ttp_should_soft_overlay_tile(tile, policy, is_large_tile, tile_area_ratio,
recommended_mode = str(tile.get("recommended_composite_mode", "") or "").lower()
if recommended_mode == "soft_overlay":
return True
text = _ttp_tile_search_text(tile)
text = _ttp_tile_semantic_text(tile)
if _ttp_is_context_tile_text(text):
return False
if policy == "soft_detail":
@@ -1013,7 +1017,11 @@ def _ttp_should_soft_overlay_tile(tile, policy, is_large_tile, tile_area_ratio,
return True
small_threshold = max(0.02, min(0.18, float(large_tile_area_threshold) * 0.35))
has_rank = float(tile.get("occlusion_priority", 0.0)) > 0.0 or float(tile.get("layer", 0.0)) > 0.0
return has_rank and float(tile_area_ratio) <= small_threshold
if tile.get("object_mask") and has_rank:
mask_overlay_threshold = max(small_threshold, min(0.32, float(large_tile_area_threshold) * 0.6))
if float(tile_area_ratio) <= mask_overlay_threshold:
return True
return False
def _ttp_soften_detail_mask_array(mask, grow_px=1, blur_px=1.0):
@@ -1532,11 +1540,17 @@ def _ttp_auto_tile_rank(label, object_area, image_area, object_index):
)
area_ratio = max(0.0, min(1.0, float(object_area) / max(1.0, float(image_area))))
small_bonus = min(450, int(round((1.0 - area_ratio) * 450.0)))
if _ttp_is_large_context_auto_tile(label_text, area_ratio):
return 0, 0
if any(keyword in label_text for keyword in detail_keywords):
return 4, 2000 + small_bonus + int(object_index)
return 2, 300 + small_bonus + int(object_index)
def _ttp_is_large_context_auto_tile(label, area_ratio):
return float(area_ratio) >= 0.9 and not _ttp_is_detail_tile_text(str(label or "").lower())
def _ttp_boxes_to_auto_layout(
bboxes,
image_width,
@@ -1590,10 +1604,14 @@ def _ttp_boxes_to_auto_layout(
x0, y0, x1, y1 = box
object_area = (x1 - x0) * (y1 - y0)
image_area = max(1, image_width * image_height)
layer, occlusion_priority = _ttp_auto_tile_rank(item.get("label", f"object_{object_index}"), object_area, image_area, object_index)
priority = 70 + min(80, int(100 * object_area / image_area)) + int(float(item.get("score", 1.0)) * 20)
area_ratio = max(0.0, min(1.0, float(object_area) / float(image_area)))
raw_label = str(item.get("label", f"object_{object_index}") or f"object_{object_index}")
is_large_context = _ttp_is_large_context_auto_tile(raw_label, area_ratio)
layer, occlusion_priority = _ttp_auto_tile_rank(raw_label, object_area, image_area, object_index)
priority = 5 if is_large_context else 70 + min(80, int(100 * area_ratio)) + int(float(item.get("score", 1.0)) * 20)
label = f"{raw_label} large context" if is_large_context and "context" not in raw_label.lower() else raw_label
tile = {
"name": f"auto_{item.get('label', 'object')}_{object_index}",
"name": f"{'auto_large_context' if is_large_context else 'auto_' + raw_label}_{object_index}",
"x0": x0 / image_width,
"y0": y0 / image_height,
"x1": x1 / image_width,
@@ -1601,14 +1619,23 @@ def _ttp_boxes_to_auto_layout(
"pad": int(default_pad) if allow_object_overlap else max(0, int(default_pad) // 2),
"blend": int(default_blend),
"priority": priority,
"importance": 1.0,
"source": "auto",
"label": item.get("label", f"object_{object_index}"),
"importance": 0.35 if is_large_context else 1.0,
"source": "auto_context" if is_large_context else "auto",
"label": label,
"score": float(item.get("score", 1.0)),
"layer": layer,
"object_id": object_index,
"object_id": 0 if is_large_context else object_index,
"occlusion_priority": occlusion_priority,
}
if is_large_context:
tile.update({
"semantic_category": "background",
"semantic_role": "context",
"recommended_layer": 0.0,
"recommended_priority": 10.0,
"recommended_occlusion_priority": 0.0,
"recommended_composite_mode": "context",
})
object_mask = _ttp_encode_object_mask_data(mask_images, item.get("mask_indices", []), box)
if object_mask is not None:
tile["object_mask"] = object_mask
@@ -1720,6 +1747,37 @@ def _ttp_round_down_to(value, multiple):
return max(multiple, rounded)
def _ttp_scaled_source_box(box, scale):
x, y, width, height = [float(value) for value in box]
scale = max(0.01, float(scale))
left = int(round(x * scale))
top = int(round(y * scale))
right = int(round((x + width) * scale))
bottom = int(round((y + height) * scale))
return left, top, max(1, right - left), max(1, bottom - top)
def _ttp_scaled_overlap_edges(sample_box, overlap_edges, scale, out_width, out_height):
sx, sy, sw, sh = [float(value) for value in sample_box]
scale = max(0.01, float(scale))
def edge_before(start, amount, limit):
amount = _ttp_clamp(float(amount), 0.0, float(limit))
return max(0, int(round((start + amount) * scale)) - int(round(start * scale)))
def edge_after(start, length, amount, limit):
amount = _ttp_clamp(float(amount), 0.0, float(limit))
end = start + length
return max(0, int(round(end * scale)) - int(round((end - amount) * scale)))
return {
"left": _ttp_clamp(edge_before(sx, overlap_edges.get("left", 0), sw), 0, int(out_width)),
"right": _ttp_clamp(edge_after(sx, sw, overlap_edges.get("right", 0), sw), 0, int(out_width)),
"top": _ttp_clamp(edge_before(sy, overlap_edges.get("top", 0), sh), 0, int(out_height)),
"bottom": _ttp_clamp(edge_after(sy, sh, overlap_edges.get("bottom", 0), sh), 0, int(out_height)),
}
def _ttp_smart_upscale_target_size(width, height, scale=2.0, round_to=8, max_megapixels=0.0):
width = max(1, int(width))
height = max(1, int(height))
@@ -1953,27 +2011,104 @@ def _ttp_clone_tile_set(tile_set):
}
def _ttp_mask_data_signature(mask_data):
if not isinstance(mask_data, dict):
return None
data = str(mask_data.get("data", "") or "")
digest = hashlib.sha1(data.encode("utf-8")).hexdigest()[:16] if data else ""
return {
"format": mask_data.get("format", ""),
"box": mask_data.get("box", []),
"width": mask_data.get("width", 0),
"height": mask_data.get("height", 0),
"hash": digest,
}
def _ttp_debug_enabled():
return str(os.environ.get("TTP_SMART_TILE_DEBUG", "")).strip().lower() in ("1", "true", "yes", "on")
def _ttp_debug_tile_set(label, tile_set=None, tile_meta=None, tile_images=None, extra=None):
if not _ttp_debug_enabled():
return
tile_meta = tile_meta if isinstance(tile_meta, dict) else (tile_set or {}).get("tile_meta", {})
tile_images = tile_images if tile_images is not None else (tile_set or {}).get("tile_images", [])
tiles = tile_meta.get("tiles", []) if isinstance(tile_meta, dict) else []
original_size = tile_meta.get("original_size", (tile_set or {}).get("original_size", [])) if isinstance(tile_meta, dict) else []
fingerprint = ""
try:
source = tile_set if isinstance(tile_set, dict) else {"tile_meta": tile_meta, "tile_images": tile_images, "original_size": original_size}
fingerprint = hashlib.sha1(_ttp_tile_set_fingerprint(source).encode("utf-8")).hexdigest()[:12]
except Exception:
fingerprint = "unavailable"
print(
f"[TTP Smart Tile Debug] {label}: original_size={original_size} "
f"tiles={len(tiles)} images={len(tile_images or [])} fingerprint={fingerprint} extra={extra or {}}"
)
for index, tile in enumerate(tiles[:64]):
image_size = None
if tile_images is not None and index < len(tile_images):
try:
image_size = _ttp_image_tensor_size(tile_images[index])
except Exception:
image_size = None
print(
"[TTP Smart Tile Debug] "
f" tile[{index}] name={tile.get('name')} label={tile.get('label')} source={tile.get('source')} "
f"core={tile.get('core_box')} sample={tile.get('sample_box')} canvas={tile.get('tile_canvas_size')}/{tile.get('tile_canvas_box')} "
f"edges={tile.get('overlap_edges_px_source')} image_size={image_size} "
f"priority={tile.get('priority')} importance={tile.get('importance')} layer={tile.get('layer')} occ={tile.get('occlusion_priority')} "
f"mode={tile.get('recommended_composite_mode')} category={tile.get('semantic_category')} role={tile.get('semantic_role')} "
f"mask={_ttp_mask_data_signature(tile.get('object_mask'))} "
f"mask_source={_ttp_mask_data_signature(tile.get('object_mask_source'))}"
)
if len(tiles) > 64:
print(f"[TTP Smart Tile Debug] ... {len(tiles) - 64} more tile(s)")
def _ttp_tile_set_fingerprint(tile_set):
tile_meta = tile_set.get("tile_meta", {})
tiles = tile_meta.get("tiles", [])
def tile_signature(tile):
return {
"name": tile.get("name", ""),
"label": tile.get("label", ""),
"source": tile.get("source", ""),
"core_box": tile.get("core_box"),
"sample_box": tile.get("sample_box"),
"paste_box": tile.get("paste_box"),
"tile_canvas_size": tile.get("tile_canvas_size"),
"tile_canvas_box": tile.get("tile_canvas_box"),
"overlap_edges_px_source": tile.get("overlap_edges_px_source", {}),
"pad": tile.get("pad", 0),
"blend": tile.get("blend", 0),
"priority": tile.get("priority", 0),
"importance": tile.get("importance", 0),
"layer": tile.get("layer", 0),
"object_id": tile.get("object_id", 0),
"occlusion_priority": tile.get("occlusion_priority", 0),
"semantic_category": tile.get("semantic_category", ""),
"semantic_role": tile.get("semantic_role", ""),
"recommended_layer": tile.get("recommended_layer", ""),
"recommended_priority": tile.get("recommended_priority", ""),
"recommended_occlusion_priority": tile.get("recommended_occlusion_priority", ""),
"recommended_composite_mode": tile.get("recommended_composite_mode", ""),
"object_mask": _ttp_mask_data_signature(tile.get("object_mask")),
"object_mask_source": _ttp_mask_data_signature(tile.get("object_mask_source")),
"prompt": tile.get("prompt", ""),
"negative": tile.get("negative", ""),
"caption": tile.get("caption", ""),
"prompt_tag": tile.get("prompt_tag", ""),
"prompt_source": tile.get("prompt_source", ""),
"tile_hash": tile.get("tile_hash", ""),
}
return json.dumps({
"size": tile_set.get("original_size", tile_meta.get("original_size")),
"count": len(tile_set.get("tile_images", [])),
"boxes": [tile.get("sample_box") for tile in tiles],
"names": [tile.get("name") for tile in tiles],
"masks": [bool(tile.get("object_mask")) for tile in tiles],
"prompts": [
{
"prompt": tile.get("prompt", ""),
"negative": tile.get("negative", ""),
"caption": tile.get("caption", ""),
"label": tile.get("label", ""),
"prompt_tag": tile.get("prompt_tag", ""),
"prompt_source": tile.get("prompt_source", ""),
"tile_hash": tile.get("tile_hash", ""),
}
for tile in tiles
],
"tiles": [tile_signature(tile) for tile in tiles],
}, sort_keys=True)
@@ -3374,6 +3509,7 @@ class TTP_Smart_Tile_Interactive_Crop_Experimental:
RETURN_NAMES = ("source_image", "tiles", "tile_set", "tile_meta", "positions", "preview", "layout_json")
FUNCTION = "interactive_crop_tiles"
CATEGORY = "TTP/Smart Tile"
OUTPUT_NODE = True
@classmethod
def IS_CHANGED(cls, **kwargs):
@@ -4001,6 +4137,19 @@ class TTP_Smart_Tile_Loop_Source_Experimental:
or int(session.get("restart_request", -1)) != int(restart_request)
)
if should_restart:
if _ttp_debug_enabled():
reason = []
if session is None:
reason.append("new_session")
elif session.get("fingerprint") != fingerprint:
reason.append("fingerprint_changed")
if session is not None and int(session.get("restart_request", -1)) != int(restart_request):
reason.append("restart_request_changed")
print(
"[TTP Smart Tile Debug] loop_source restart "
f"session={session_key} reason={','.join(reason) or 'unknown'} "
f"restart_request={int(restart_request)}"
)
session = {
"fingerprint": fingerprint,
"restart_request": int(restart_request),
@@ -4011,6 +4160,16 @@ class TTP_Smart_Tile_Loop_Source_Experimental:
"done": False,
}
_TTP_SMART_TILE_LOOP_SESSIONS[session_key] = session
elif _ttp_debug_enabled():
print(
"[TTP Smart Tile Debug] loop_source reuse "
f"session={session_key} index={session.get('index', 0)} done={session.get('done', False)}"
)
_ttp_debug_tile_set(
"loop_source input",
tile_set=tile_set,
extra={"session": session_key, "restart": bool(should_restart), "restart_request": int(restart_request)},
)
count = len(tile_images)
index = _ttp_clamp(int(session.get("index", 0)), 0, max(0, count - 1))
@@ -4090,6 +4249,17 @@ class TTP_Smart_Tile_Loop_Collect_Experimental:
index = _ttp_clamp(int(tile_task.get("index", 0)), 0, max(0, count - 1))
if not bool(tile_task.get("done", False)) and count > 0:
tile_images[index] = _ttp_first_image_tensor(processed_image)
if _ttp_debug_enabled():
processed_size = None
try:
processed_size = _ttp_image_tensor_size(_ttp_first_image_tensor(processed_image))
except Exception:
processed_size = None
print(
"[TTP Smart Tile Debug] loop_collect update "
f"session={session_id} index={index} count={count} task_done={bool(tile_task.get('done', False))} "
f"processed_size={processed_size}"
)
next_index = index + 1
done = next_index >= count
@@ -4099,6 +4269,11 @@ class TTP_Smart_Tile_Loop_Collect_Experimental:
event_task = dict(tile_task)
event_task["index"] = min(next_index, max(0, count - 1))
_ttp_send_smart_tile_loop_event(event_task, done=done, message=status)
_ttp_debug_tile_set(
"loop_collect output",
tile_set=processed_tile_set,
extra={"session": session_id, "next_index": int(next_index), "done": bool(done), "status": status},
)
return (_ttp_clone_tile_set(processed_tile_set), bool(done), int(next_index), status)
@@ -4317,18 +4492,66 @@ class TTP_Smart_Tile_Output_Size_Estimate_Experimental:
def estimate_output_size(self, tile_set, scale_strategy="median", source_image=None, done=None):
if done is not None and not bool(done):
info = "deferred: waiting for done=true before estimating final Smart Tile output size"
return (0.0, 0, 0, 0.0, 0.0, info)
pending_scale = 1.0
pending_scale_x = 1.0
pending_scale_y = 1.0
pending_width = 1
pending_height = 1
tile_meta = tile_set.get("tile_meta") if isinstance(tile_set, dict) else None
if isinstance(tile_meta, dict):
original_width, original_height = tile_meta.get("original_size", [1, 1])
original_width = max(1, int(round(original_width)))
original_height = max(1, int(round(original_height)))
pending_width = original_width
pending_height = original_height
if source_image is not None:
source_width, source_height = _ttp_image_tensor_size(source_image[0])
pending_width = max(1, int(source_width))
pending_height = max(1, int(source_height))
pending_scale_x = pending_width / max(1, original_width)
pending_scale_y = pending_height / max(1, original_height)
pending_scale = float(np.median([pending_scale_x, pending_scale_y]))
info = (
"deferred: waiting for done=true before estimating final Smart Tile output size; "
"returning safe pending size for downstream ImageScale nodes"
)
_ttp_debug_tile_set(
"size_estimate skipped",
tile_set=tile_set,
extra={
"done": bool(done),
"strategy": str(scale_strategy),
"output_scale": float(pending_scale),
"output_size": [int(pending_width), int(pending_height)],
"scale_x": float(pending_scale_x),
"scale_y": float(pending_scale_y),
"info": info,
},
)
return (float(pending_scale), int(pending_width), int(pending_height), float(pending_scale_x), float(pending_scale_y), info)
if not isinstance(tile_set, dict) or tile_set.get("type") != "ttp_smart_tile_set":
raise ValueError("tile_set must come from TTP Smart Tile Interactive Crop or Loop Collect.")
tile_meta = tile_set.get("tile_meta")
tile_images = tile_set.get("tile_images", [])
return _ttp_estimate_smart_tile_output_size(
result = _ttp_estimate_smart_tile_output_size(
tile_meta,
tile_images,
scale_strategy=str(scale_strategy),
source_image=source_image,
)
_ttp_debug_tile_set(
"size_estimate input",
tile_set=tile_set,
extra={
"done": None if done is None else bool(done),
"strategy": str(scale_strategy),
"output_scale": float(result[0]),
"output_size": [int(result[1]), int(result[2])],
"scale_x": float(result[3]),
"scale_y": float(result[4]),
},
)
return result
class TTP_Smart_Tile_Assemble_Experimental:
@@ -4354,6 +4577,7 @@ class TTP_Smart_Tile_Assemble_Experimental:
"assemble_device": (["auto", "cpu", "gpu"], {"default": "auto"}),
"assemble_mode": (["final_only", "always"], {"default": "final_only"}),
"base_canvas_mode": (["auto", "black", "base_image", "source_image"], {"default": "auto"}),
"weight_preview_mode": (["raw_weight", "coverage"], {"default": "raw_weight"}),
"small_tile_on_top": ("BOOLEAN", {"default": False}),
"auto_composite_policy": (["safe_auto", "strict_layer", "soft_detail", "replace_object"], {"default": "safe_auto"}),
},
@@ -4393,6 +4617,7 @@ class TTP_Smart_Tile_Assemble_Experimental:
assemble_device="auto",
assemble_mode="final_only",
base_canvas_mode="auto",
weight_preview_mode="raw_weight",
small_tile_on_top=False,
auto_composite_policy="safe_auto",
sampled_tiles=None,
@@ -4429,6 +4654,9 @@ class TTP_Smart_Tile_Assemble_Experimental:
effective_assemble_mode = "final_only"
canvas_mode = str(base_canvas_mode or "auto")
preview_mode = str(weight_preview_mode or "raw_weight")
if preview_mode not in ("raw_weight", "coverage"):
preview_mode = "raw_weight"
if canvas_mode == "base_image":
base_source_image = base_image
elif canvas_mode == "source_image":
@@ -4440,6 +4668,18 @@ class TTP_Smart_Tile_Assemble_Experimental:
base_source_image = base_image if base_image is not None else source_image
if effective_assemble_mode == "final_only" and done is not None and not bool(done):
_ttp_debug_tile_set(
"assemble skipped",
tile_set=tile_set,
tile_meta=tile_meta,
tile_images=tile_images,
extra={
"done": bool(done),
"requested_mode": requested_assemble_mode,
"effective_mode": effective_assemble_mode,
"base_canvas_mode": canvas_mode,
},
)
return _ttp_assemble_placeholder_output(tile_images, base_image=base_source_image)
original_width, original_height = tile_meta["original_size"]
@@ -4480,6 +4720,27 @@ class TTP_Smart_Tile_Assemble_Experimental:
if assemble_torch_device is None and assemble_device_mode == "gpu":
assemble_fallback_reason = "gpu_unavailable"
use_gpu_assemble = assemble_torch_device is not None
_ttp_debug_tile_set(
"assemble input",
tile_set=tile_set,
tile_meta=tile_meta,
tile_images=tile_images,
extra={
"done": None if done is None else bool(done),
"output_scale": float(output_scale),
"output_size": [int(output_width), int(output_height)],
"requested_device": assemble_device_mode,
"actual_device": str(assemble_torch_device) if use_gpu_assemble else "cpu",
"requested_mode": requested_assemble_mode,
"effective_mode": effective_assemble_mode,
"base_canvas_mode": canvas_mode,
"weight_preview_mode": preview_mode,
"small_tile_on_top": _ttp_safe_bool(small_tile_on_top, False),
"auto_composite_policy": str(auto_composite_policy or "safe_auto"),
"mask_blend_mode": str(mask_blend_mode),
"large_tile_policy": str(large_tile_policy),
},
)
if base_source_image is not None:
base_pil = tensor2pil(base_source_image[0].unsqueeze(0)).convert("RGB")
@@ -4489,22 +4750,27 @@ class TTP_Smart_Tile_Assemble_Experimental:
if use_gpu_assemble:
canvas = torch.as_tensor(base_array, dtype=torch.float32, device=assemble_torch_device)
weights = torch.ones((output_height, output_width, 1), dtype=torch.float32, device=assemble_torch_device)
preview_weights = torch.zeros((output_height, output_width, 1), dtype=torch.float32, device=assemble_torch_device)
else:
canvas = base_array
weights = np.ones((output_height, output_width, 1), dtype=np.float32)
preview_weights = np.zeros((output_height, output_width, 1), dtype=np.float32)
else:
if use_gpu_assemble:
canvas = torch.zeros((output_height, output_width, 3), dtype=torch.float32, device=assemble_torch_device)
weights = torch.zeros((output_height, output_width, 1), dtype=torch.float32, device=assemble_torch_device)
preview_weights = torch.zeros((output_height, output_width, 1), dtype=torch.float32, device=assemble_torch_device)
else:
canvas = np.zeros((output_height, output_width, 3), dtype=np.float32)
weights = np.zeros((output_height, output_width, 1), dtype=np.float32)
preview_weights = np.zeros((output_height, output_width, 1), dtype=np.float32)
small_tile_on_top = bool(small_tile_on_top)
small_tile_on_top = _ttp_safe_bool(small_tile_on_top, False)
composite_policy = str(auto_composite_policy or "safe_auto")
if composite_policy not in ("safe_auto", "strict_layer", "soft_detail", "replace_object"):
composite_policy = "safe_auto"
auto_policy_ranking = composite_policy in ("safe_auto", "soft_detail", "replace_object")
destructive_occlusion = composite_policy in ("strict_layer", "replace_object")
auto_policy_candidates = auto_policy_ranking and any(
not _ttp_is_context_tile_text(_ttp_tile_search_text(tile)) and (
_ttp_is_detail_tile_text(_ttp_tile_search_text(tile))
@@ -4576,7 +4842,10 @@ class TTP_Smart_Tile_Assemble_Experimental:
tile_scale_x = content_width / max(1, sw)
tile_scale_y = content_height / max(1, sh)
tile_effective_scale = min(tile_scale_x, tile_scale_y)
tile_area_ratio = (float(sw) * float(sh)) / original_area
core_box = tile.get("core_box", [sx, sy, sw, sh])
core_w = float(core_box[2]) if len(core_box) >= 4 else float(sw)
core_h = float(core_box[3]) if len(core_box) >= 4 else float(sh)
tile_area_ratio = (max(1.0, core_w) * max(1.0, core_h)) / original_area
tile_label = str(tile.get("label", tile.get("source", ""))).lower()
is_context_label = any(word in tile_label for word in ("background", "context", "full image", "full_image", "large context"))
is_large_tile = tile_area_ratio >= large_tile_area_threshold or is_context_label
@@ -4598,10 +4867,7 @@ class TTP_Smart_Tile_Assemble_Experimental:
local_bottom = _ttp_clamp(content_top + content_height, local_top + 1, tile_pil.height)
paste_region = tile_pil.crop((local_left, local_top, local_right, local_bottom))
out_x = int(round(sx * output_scale))
out_y = int(round(sy * output_scale))
out_w = max(1, int(round(sw * output_scale)))
out_h = max(1, int(round(sh * output_scale)))
out_x, out_y, out_w, out_h = _ttp_scaled_source_box([sx, sy, sw, sh], output_scale)
crop_px = max(0, int(edge_crop_px))
if crop_px > 0 and paste_region.width > crop_px * 2 and paste_region.height > crop_px * 2:
paste_region = paste_region.crop((
@@ -4628,25 +4894,51 @@ class TTP_Smart_Tile_Assemble_Experimental:
paste_region = tensor2pil(corrected[0].unsqueeze(0)).convert("RGB")
blend = int(round(tile.get("blend", 0) * output_scale * blend_multiplier))
scaled_overlap_edges = _ttp_scaled_overlap_edges(
[sx, sy, sw, sh],
tile.get("overlap_edges_px_source", {}),
output_scale,
out_w,
out_h,
)
mask = _ttp_create_sample_blend_mask(
out_w,
out_h,
tile.get("overlap_edges_px_source", {}),
scaled_overlap_edges,
blend,
output_scale,
output_scale,
1.0,
1.0,
)
object_mask = _ttp_tile_object_mask_array(tile, out_w, out_h, str(mask_blend_mode), blend)
if object_mask is not None:
if soft_detail_overlay and str(mask_blend_mode) in ("auto", "mask_feather", "mask_only"):
blur_px = max(1.0, min(8.0, max(float(blend) * 0.08, min(out_w, out_h) * 0.012)))
object_halo = _ttp_soften_detail_mask_array(
object_mask,
grow_px=max(1, int(round(min(out_w, out_h) * 0.018))),
blur_px=max(blur_px, min(out_w, out_h) * 0.018),
)
object_mask = _ttp_soften_detail_mask_array(object_mask, grow_px=0, blur_px=blur_px)
if str(mask_blend_mode) in ("auto", "mask_feather"):
halo_strength = _ttp_clamp(max(0.12, float(context_tile_weight) * 0.65), 0.05, 0.22)
object_mask = np.maximum(object_mask, object_halo * halo_strength)
if str(mask_blend_mode) == "mask_only":
mask = object_mask
else:
mask = mask * object_mask
elif _ttp_should_rect_feather_tile(tile, is_large_tile, str(mask_blend_mode)):
mask = mask * _ttp_rect_feather_mask(out_w, out_h, blend)
tile_destructive_occlusion = destructive_occlusion or (
composite_policy == "safe_auto"
and object_mask is not None
and not soft_detail_overlay
) or (
composite_policy == "safe_auto"
and has_base_canvas
and is_large_tile
and not low_resolution_large_tile
and large_tile_policy != "context_only"
)
coverage_mask = np.clip(mask.copy(), 0.0, 1.0)
importance = max(0.0, float(tile.get("importance", 1.0)))
priority = max(0.0, float(tile.get("priority", 50.0)))
@@ -4679,6 +4971,7 @@ class TTP_Smart_Tile_Assemble_Experimental:
rank_occlusion = min(rank_occlusion, 0.0)
rank_layer = min(rank_layer, 0.0)
rank_priority = 0.0
clear_mask = coverage_mask if destructive_occlusion else np.clip(mask, 0.0, 1.0)
region = np.array(paste_region).astype(np.float32) / 255.0
if str(pixel_alignment) != "off":
@@ -4767,6 +5060,7 @@ class TTP_Smart_Tile_Assemble_Experimental:
region = region[region_top:region_top + region_h, region_left:region_left + region_w]
mask = mask[region_top:region_top + region_h, region_left:region_left + region_w]
coverage_mask = coverage_mask[region_top:region_top + region_h, region_left:region_left + region_w]
clear_mask = clear_mask[region_top:region_top + region_h, region_left:region_left + region_w]
if str(color_correction) == "local_mean_std":
ref = color_reference_pil
if ref.size != (output_width, output_height):
@@ -4785,6 +5079,7 @@ class TTP_Smart_Tile_Assemble_Experimental:
region_t = torch.as_tensor(region, dtype=torch.float32, device=assemble_torch_device)
coverage_t = torch.as_tensor(coverage_mask, dtype=torch.float32, device=assemble_torch_device)
mask_t = torch.as_tensor(mask, dtype=torch.float32, device=assemble_torch_device)
clear_t = torch.as_tensor(clear_mask, dtype=torch.float32, device=assemble_torch_device)
if use_occlusion:
rank = rank_occlusion * 1000000.0 + rank_layer * 10000.0 + (rank_priority if use_priority else 0.0)
rank_view = priority_ranks[paste_y:y_end, paste_x:x_end]
@@ -4793,21 +5088,27 @@ class TTP_Smart_Tile_Assemble_Experimental:
eligible = (rank >= rank_view - 1e-6) & mask_active
canvas_view = canvas[paste_y:y_end, paste_x:x_end]
weights_view = weights[paste_y:y_end, paste_x:x_end]
preview_view = preview_weights[paste_y:y_end, paste_x:x_end]
if soft_detail_overlay:
protect = higher & (coverage_t >= 0.5)
rank_view[protect] = rank
elif not tile_destructive_occlusion:
pass
else:
higher_alpha = coverage_t * higher.to(dtype=torch.float32)
higher_alpha = clear_t * higher.to(dtype=torch.float32)
keep = 1.0 - higher_alpha
canvas_view.mul_(keep)
weights_view.mul_(keep)
preview_view.mul_(keep)
rank_view[higher] = rank
eligible_f = eligible.to(dtype=torch.float32)
canvas_view.add_(region_t * mask_t * eligible_f)
weights_view.add_(mask_t * eligible_f)
preview_view.add_(coverage_t * eligible_f)
else:
canvas[paste_y:y_end, paste_x:x_end].add_(region_t * mask_t)
weights[paste_y:y_end, paste_x:x_end].add_(mask_t)
preview_weights[paste_y:y_end, paste_x:x_end].add_(coverage_t)
elif use_occlusion:
rank = rank_occlusion * 1000000.0 + rank_layer * 10000.0 + (rank_priority if use_priority else 0.0)
rank_view = priority_ranks[paste_y:y_end, paste_x:x_end]
@@ -4816,21 +5117,27 @@ class TTP_Smart_Tile_Assemble_Experimental:
eligible = (rank >= rank_view - 1e-6) & mask_active
canvas_view = canvas[paste_y:y_end, paste_x:x_end]
weights_view = weights[paste_y:y_end, paste_x:x_end]
preview_view = preview_weights[paste_y:y_end, paste_x:x_end]
if np.any(higher):
if soft_detail_overlay:
protect = higher & (coverage_mask >= 0.5)
rank_view[protect[:, :, 0], :] = rank
elif not tile_destructive_occlusion:
pass
else:
higher_alpha = coverage_mask * higher.astype(np.float32)
higher_alpha = clear_mask * higher.astype(np.float32)
keep = 1.0 - higher_alpha
canvas_view *= keep
weights_view *= keep
preview_view *= keep
rank_view[higher[:, :, 0], :] = rank
canvas_view += region * mask * eligible
weights_view += mask * eligible
preview_view += coverage_mask * eligible
else:
canvas[paste_y:y_end, paste_x:x_end] += region * mask
weights[paste_y:y_end, paste_x:x_end] += mask
preview_weights[paste_y:y_end, paste_x:x_end] += coverage_mask
if str(pixel_alignment) != "off":
devices = ",".join(sorted(alignment_stats["devices"])) or ("cpu" if alignment_stats["cpu"] else "none")
@@ -4855,14 +5162,21 @@ class TTP_Smart_Tile_Assemble_Experimental:
if use_gpu_assemble:
safe_weights = torch.clamp(weights, min=1e-6)
output = torch.clamp(canvas / safe_weights, 0.0, 1.0).detach().cpu()
max_weight = torch.clamp(weights.max(), min=1e-6)
weight_preview = torch.clamp(weights / max_weight, 0.0, 1.0).repeat(1, 1, 3).detach().cpu()
if preview_mode == "coverage":
weight_preview = (preview_weights > 1e-6).to(dtype=torch.float32)
else:
max_weight = torch.clamp(preview_weights.max(), min=1e-6)
weight_preview = torch.clamp(preview_weights / max_weight, 0.0, 1.0)
weight_preview = weight_preview.repeat(1, 1, 3).detach().cpu()
return (output.unsqueeze(0), weight_preview.unsqueeze(0))
safe_weights = np.maximum(weights, 1e-6)
output = canvas / safe_weights
output = np.clip(output, 0.0, 1.0)
weight_preview = np.clip(weights / max(1e-6, weights.max()), 0.0, 1.0)
if preview_mode == "coverage":
weight_preview = (preview_weights > 1e-6).astype(np.float32)
else:
weight_preview = np.clip(preview_weights / max(float(preview_weights.max()), 1e-6), 0.0, 1.0).astype(np.float32)
weight_preview = np.repeat(weight_preview, 3, axis=2)
return (
docs/images/smart_tile_chrome_final_preview.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 133 KiB

docs/images/smart_tile_chrome_interactive_editor.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 118 KiB

docs/images/smart_tile_chrome_tile_set_preview.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 40 KiB

docs/images/smart_tile_chrome_weight_preview.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 63 KiB

docs/images/smart_tile_workflow_chrome_overview.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 151 KiB

examples/smart_tile_qwen_sam_loop_example_v2.json
+2411
View File
File diff suppressed because one or more lines are too long
scripts/smoke_smart_tile_grid_crop.py
+420 -2
View File
@@ -78,6 +78,27 @@ def assert_equal(actual, expected, message):
raise AssertionError(f"{message}: expected {expected}, got {actual}")
def assert_image_close(actual, expected, message, tolerance=1e-6):
actual_array = getattr(actual, "array", actual)
expected_array = getattr(expected, "array", expected)
if not np.allclose(actual_array, expected_array, atol=tolerance, rtol=0.0):
max_delta = float(np.max(np.abs(actual_array - expected_array)))
raise AssertionError(f"{message}: max delta {max_delta} > {tolerance}")
def call_assemble_like_comfy(node, widget_values, **linked_inputs):
kwargs = {}
required = node.INPUT_TYPES()["required"]
for index, (name, spec) in enumerate(required.items()):
if index < len(widget_values):
kwargs[name] = widget_values[index]
continue
if isinstance(spec, tuple) and len(spec) > 1 and isinstance(spec[1], dict) and "default" in spec[1]:
kwargs[name] = spec[1]["default"]
kwargs.update(linked_inputs)
return node.assemble_tiles(**kwargs)
install_comfy_stubs()
import TTP_toolsets as ttp # noqa: E402
@@ -141,6 +162,34 @@ assert_equal("qwen_vl_model" in optional_inputs, True, "interactive crop should
assert_equal(hidden_inputs["unique_id"], "UNIQUE_ID", "interactive crop should receive its ComfyUI node id")
assemble_inputs = ttp.TTP_Smart_Tile_Assemble_Experimental.INPUT_TYPES()
expected_assemble_required_order = [
"blend_multiplier",
"output_scale",
"use_priority",
"tile_alignment",
"edge_crop_px",
"color_correction",
"color_strength",
"mask_blend_mode",
"pixel_alignment",
"pixel_alignment_radius",
"pixel_alignment_device",
"large_tile_policy",
"large_tile_area_threshold",
"min_tile_scale_ratio",
"context_tile_weight",
"assemble_device",
"assemble_mode",
"base_canvas_mode",
"weight_preview_mode",
"small_tile_on_top",
"auto_composite_policy",
]
assert_equal(
list(assemble_inputs["required"].keys()),
expected_assemble_required_order,
"assemble required widgets should keep the same order ComfyUI stores in workflow widget values",
)
assert_equal("sampled_tiles" in assemble_inputs["optional"], True, "assemble should keep batch tiles as an optional compatibility input")
assert_equal("tile_meta" in assemble_inputs["optional"], True, "assemble should keep batch metadata as an optional compatibility input")
assert_equal("tile_set" in assemble_inputs["optional"], True, "assemble should accept true variable-size tile sets")
@@ -155,6 +204,8 @@ assert_equal(assemble_inputs["required"]["assemble_device"][0], ["auto", "cpu",
assert_equal(assemble_inputs["required"]["assemble_mode"][0], ["final_only", "always"], "assemble should expose final-only mode first")
assert_equal(assemble_inputs["required"]["assemble_mode"][1]["default"], "final_only", "assemble should default to final-only loop compositing")
assert_equal(assemble_inputs["required"]["base_canvas_mode"][0], ["auto", "black", "base_image", "source_image"], "assemble should expose base canvas source selection")
assert_equal(assemble_inputs["required"]["weight_preview_mode"][0], ["raw_weight", "coverage"], "assemble should expose raw weight and coverage preview modes")
assert_equal(assemble_inputs["required"]["weight_preview_mode"][1]["default"], "raw_weight", "assemble should keep raw overlap weight preview as the default")
assert_equal("small_tile_on_top" in assemble_inputs["required"], True, "assemble should expose small tile top stacking")
assert_equal(assemble_inputs["required"]["auto_composite_policy"][0], ["safe_auto", "strict_layer", "soft_detail", "replace_object"], "assemble should expose automatic compositing policy")
assert_equal(assemble_inputs["required"]["auto_composite_policy"][1]["default"], "safe_auto", "assemble should default to safe auto compositing")
@@ -409,6 +460,120 @@ selected_tile, selected_info = preview_node.preview_tile_set(
assert_equal(list(selected_tile.shape[1:3]), [328, 432], "selected tile preview should preserve the selected tile size")
assert_equal("4:" in selected_info, True, "selected tile preview should include tile index info")
grid_assemble_node = ttp.TTP_Smart_Tile_Assemble_Experimental()
_grid_output_raw, grid_raw_weight_preview = grid_assemble_node.assemble_tiles(
blend_multiplier=1.0,
output_scale=1.0,
use_priority=True,
base_canvas_mode="black",
mask_blend_mode="mask_feather",
tile_set=tile_set,
)
assert_equal(float(grid_raw_weight_preview.array.max()), 1.0, "raw 3x3 weight preview should normalize the highest overlap to white")
assert_equal(float(grid_raw_weight_preview.array.min()) < 1.0, True, "raw 3x3 weight preview should show lower-weight single tile areas darker than overlap zones")
_grid_output, grid_coverage_preview = grid_assemble_node.assemble_tiles(
blend_multiplier=1.0,
output_scale=1.0,
use_priority=True,
base_canvas_mode="black",
weight_preview_mode="coverage",
mask_blend_mode="mask_feather",
tile_set=tile_set,
)
assert_equal(float(grid_coverage_preview.array.min()), 1.0, "black canvas 3x3 coverage preview should be uniformly covered")
assert_equal(float(grid_coverage_preview.array.max()), 1.0, "black canvas 3x3 coverage preview should not show raw overlap weight grid lines")
comfy_assemble_widget_values = [
1.0,
0.0,
True,
"resize",
0,
"off",
0.35,
"mask_feather",
"off",
0,
"cpu",
"use_if_higher_resolution",
0.55,
0.95,
0.25,
"cpu",
"final_only",
"black",
"raw_weight",
False,
"safe_auto",
]
direct_comfy_output, direct_comfy_weights = grid_assemble_node.assemble_tiles(
blend_multiplier=1.0,
output_scale=1.0,
use_priority=True,
tile_alignment="resize",
edge_crop_px=0,
color_correction="off",
color_strength=0.35,
mask_blend_mode="mask_feather",
pixel_alignment="off",
pixel_alignment_radius=0,
pixel_alignment_device="cpu",
large_tile_policy="use_if_higher_resolution",
large_tile_area_threshold=0.55,
min_tile_scale_ratio=0.95,
context_tile_weight=0.25,
assemble_device="cpu",
assemble_mode="final_only",
base_canvas_mode="black",
weight_preview_mode="raw_weight",
small_tile_on_top=False,
auto_composite_policy="safe_auto",
tile_set=tile_set,
done=True,
)
replayed_comfy_output, replayed_comfy_weights = call_assemble_like_comfy(
grid_assemble_node,
comfy_assemble_widget_values,
output_scale=1.0,
tile_set=tile_set,
done=True,
)
assert_image_close(replayed_comfy_output, direct_comfy_output, "ComfyUI widget-order assemble replay should match direct assemble output")
assert_image_close(replayed_comfy_weights, direct_comfy_weights, "ComfyUI widget-order assemble replay should match direct assemble weight preview")
ranked_grid_meta = dict(tile_set["tile_meta"])
ranked_grid_tiles = []
for tile_index, tile in enumerate(tile_set["tile_meta"]["tiles"]):
ranked_tile = dict(tile)
ranked_tile["layer"] = tile_index
ranked_tile["occlusion_priority"] = tile_index
ranked_grid_tiles.append(ranked_tile)
ranked_grid_meta["tiles"] = ranked_grid_tiles
ranked_grid_tile_set = dict(tile_set)
ranked_grid_tile_set["tile_meta"] = ranked_grid_meta
_ranked_grid_output, ranked_grid_weight_preview = grid_assemble_node.assemble_tiles(
blend_multiplier=1.0,
output_scale=1.0,
use_priority=True,
base_canvas_mode="black",
mask_blend_mode="mask_feather",
auto_composite_policy="safe_auto",
tile_set=ranked_grid_tile_set,
)
ranked_width, ranked_height = ranked_grid_meta["original_size"]
internal_x = sorted({tile["core_box"][0] for tile in ranked_grid_tiles if tile["core_box"][0] > 0})
internal_y = sorted({tile["core_box"][1] for tile in ranked_grid_tiles if tile["core_box"][1] > 0})
column_centers = sorted({min(ranked_width - 1, tile["core_box"][0] + tile["core_box"][2] // 2) for tile in ranked_grid_tiles[:3]})
row_centers = sorted({min(ranked_height - 1, ranked_grid_tiles[row * 3]["core_box"][1] + ranked_grid_tiles[row * 3]["core_box"][3] // 2) for row in range(3)})
ranked_seam_values = []
for seam_x in internal_x:
for center_y in row_centers:
ranked_seam_values.append(float(ranked_grid_weight_preview.array[0, center_y, seam_x, 0]))
for seam_y in internal_y:
for center_x in column_centers:
ranked_seam_values.append(float(ranked_grid_weight_preview.array[0, seam_y, center_x, 0]))
assert_equal(min(ranked_seam_values) > 0.45, True, "safe auto ranked 3x3 grid seams should keep both tile contributions instead of showing dark split lines")
prompt_builder = ttp.TTP_Smart_Tile_QwenVL_Prompt_Set_Builder_Experimental()
prompt_tile_set, prompt_set_json, prompt_summary = prompt_builder.build_prompt_set(
tile_set,
@@ -777,7 +942,7 @@ mixed_scale_tile_set = {
],
}
deferred_scale, deferred_w, deferred_h, deferred_x, deferred_y, deferred_info = size_node.estimate_output_size(mixed_scale_tile_set, done=False)
assert_equal([deferred_scale, deferred_w, deferred_h, deferred_x, deferred_y], [0.0, 0, 0, 0.0, 0.0], "output size estimate should skip work before loop completion")
assert_equal([deferred_scale, deferred_w, deferred_h, deferred_x, deferred_y], [1.0, 100, 50, 1.0, 1.0], "output size estimate should return safe pending dimensions before loop completion")
assert_equal("deferred" in deferred_info, True, "output size estimate should report deferred status")
size_scale, size_w, size_h, size_scale_x, size_scale_y, size_info = size_node.estimate_output_size(mixed_scale_tile_set)
assert_equal(round(size_scale, 4), 3.0, "output size estimate should match assemble median scale inference")
@@ -979,6 +1144,30 @@ assert_equal("object_mask" in masked_auto_meta[0], True, "auto layout should pre
mask_array = ttp._ttp_tile_object_mask_array(masked_auto_meta[0], masked_auto_meta[0]["sample_box"][2], masked_auto_meta[0]["sample_box"][3], "mask_feather", 32)
assert_equal(mask_array.shape[2], 1, "decoded object mask should be a single-channel weight map")
assert_equal(float(mask_array.max()) > 0.5, True, "decoded object mask should keep foreground weight")
large_body_mask = Image.new("L", (900, 600), 255)
large_body_layout = ttp._ttp_boxes_to_auto_layout(
[{"x": 0, "y": 0, "width": 900, "height": 600, "label": "person body", "score": 0.93}],
900,
600,
default_pad=64,
default_blend=32,
object_padding=0,
max_tiles=4,
include_background=False,
allow_object_overlap=True,
masks=[large_body_mask],
)
large_body_meta = ttp._ttp_parse_smart_tile_layout(large_body_layout, 900, 600)
large_body_tile = large_body_meta[0]
assert_equal("large context" in large_body_tile["label"], True, "huge non-detail auto body tiles should be marked as large context")
assert_equal(large_body_tile["layer"], 0, "huge non-detail auto body tiles should stay on the lowest layer")
assert_equal(large_body_tile["occlusion_priority"], 0, "huge non-detail auto body tiles should not outrank face/detail masks")
assert_equal(float(large_body_tile["priority"]) <= 10.0, True, "huge non-detail auto body tiles should keep low composite priority")
assert_equal(float(large_body_tile["importance"]) <= 0.35, True, "huge non-detail auto body tiles should remain a low-weight context layer")
assert_equal("object_mask" in large_body_tile, True, "huge body context tiles should still preserve their body mask")
assert_equal(large_body_tile["recommended_composite_mode"], "context", "huge body context tiles should advertise context compositing")
semantic_mask_layout = json.dumps({
"tiles": [{
"name": "person_grid_1",
@@ -1138,7 +1327,7 @@ stack_meta = {
{
"name": "small_face",
"label": "local detail",
"core_box": [2, 2, 6, 6],
"core_box": [2, 2, 4, 4],
"sample_box": [2, 2, 4, 4],
"tile_canvas_size": [4, 4],
"tile_canvas_box": [0, 0, 4, 4],
@@ -1181,6 +1370,54 @@ small_above, _small_above_weights = assemble_node.assemble_tiles(
small_above_pixel = small_above.array[0, 4, 4]
assert_equal(float(small_above_pixel[2]) > float(small_above_pixel[0]), True, "small_tile_on_top should stack smaller tiles above larger context tiles")
legacy_bad_bool, _legacy_bad_bool_weights = assemble_node.assemble_tiles(
blend_multiplier=1.0,
output_scale=1.0,
use_priority=True,
base_canvas_mode="black",
small_tile_on_top="safe_auto",
auto_composite_policy="strict_layer",
tile_set=stack_tile_set,
)
legacy_bad_bool_pixel = legacy_bad_bool.array[0, 4, 4]
assert_equal(float(legacy_bad_bool_pixel[0]) > float(legacy_bad_bool_pixel[2]), True, "legacy string values should not be treated as small_tile_on_top=true")
fingerprint_mask_a = Image.new("L", (8, 8), 0)
ImageDraw.Draw(fingerprint_mask_a).rectangle((0, 0, 3, 7), fill=255)
fingerprint_mask_b = Image.new("L", (8, 8), 0)
ImageDraw.Draw(fingerprint_mask_b).rectangle((4, 0, 7, 7), fill=255)
fingerprint_source_a = ttp._ttp_encode_object_mask_data([fingerprint_mask_a], [0], [0, 0, 8, 8])
fingerprint_source_b = ttp._ttp_encode_object_mask_data([fingerprint_mask_b], [0], [0, 0, 8, 8])
fingerprint_tile = {
"name": "context_grid_1",
"label": "large context",
"core_box": [0, 0, 8, 8],
"sample_box": [0, 0, 8, 8],
"object_mask": fingerprint_source_a,
"object_mask_source": fingerprint_source_a,
"layer": 0,
"occlusion_priority": 0,
}
fingerprint_tile_set_a = {
"type": "ttp_smart_tile_set",
"original_size": [8, 8],
"tile_meta": {"type": "ttp_smart_tile", "original_size": [8, 8], "tiles": [fingerprint_tile]},
"tile_images": [None],
}
fingerprint_tile_b = dict(fingerprint_tile)
fingerprint_tile_b["object_mask_source"] = fingerprint_source_b
fingerprint_tile_set_b = {
"type": "ttp_smart_tile_set",
"original_size": [8, 8],
"tile_meta": {"type": "ttp_smart_tile", "original_size": [8, 8], "tiles": [fingerprint_tile_b]},
"tile_images": [None],
}
assert_equal(
ttp._ttp_tile_set_fingerprint(fingerprint_tile_set_a) != ttp._ttp_tile_set_fingerprint(fingerprint_tile_set_b),
True,
"loop fingerprint should change when object_mask_source changes",
)
feather_stack_meta = {
"type": "ttp_smart_tile",
"original_size": [16, 16],
@@ -1308,6 +1545,187 @@ safe_eye_center = safe_eye.array[0, 8, 8]
assert_equal(float(safe_eye_edge[0]) > float(strict_eye_edge[0]), True, "safe auto detail overlay should preserve more face color at eye mask edges")
assert_equal(float(safe_eye_center[2]) > float(safe_eye_center[0]), True, "safe auto detail overlay should keep eye details stronger at the mask center")
subdivided_mask_full = Image.new("L", (64, 64), 0)
ImageDraw.Draw(subdivided_mask_full).rectangle((24, 24, 39, 39), fill=255)
subdivided_mask_data = ttp._ttp_encode_object_mask_data([subdivided_mask_full], [0], [24, 24, 40, 40])
subdivided_detail_tile = {
"name": "object_5_second_grid_child",
"label": "object_5",
"core_box": [24, 24, 16, 16],
"sample_box": [8, 8, 48, 48],
"tile_canvas_size": [48, 48],
"tile_canvas_box": [0, 0, 48, 48],
"overlap_edges_px_source": {"left": 16, "right": 16, "top": 16, "bottom": 16},
"blend": 0,
"importance": 1.0,
"priority": 0.0,
"layer": 4,
"occlusion_priority": 2000,
"object_mask": subdivided_mask_data,
"object_mask_source": subdivided_mask_data,
}
subdivided_meta = {
"type": "ttp_smart_tile",
"original_size": [64, 64],
"tiles": [
{
"name": "face_base",
"label": "face",
"core_box": [0, 0, 64, 64],
"sample_box": [0, 0, 64, 64],
"tile_canvas_size": [64, 64],
"tile_canvas_box": [0, 0, 64, 64],
"overlap_edges_px_source": {"left": 0, "right": 0, "top": 0, "bottom": 0},
"blend": 0,
"importance": 1.0,
"priority": 0.0,
"layer": 1,
"occlusion_priority": 100,
},
subdivided_detail_tile,
],
}
subdivided_tile_set = {
"type": "ttp_smart_tile_set",
"tile_meta": subdivided_meta,
"tile_images": [
ttp.pil2tensor(Image.new("RGB", (64, 64), (220, 20, 20)))[0],
ttp.pil2tensor(Image.new("RGB", (48, 48), (20, 20, 220)))[0],
],
}
assert_equal(ttp._ttp_tile_area_ratio(subdivided_detail_tile, 64 * 64) < 0.07, True, "subdivided detail ranking should use core area instead of padded sample area")
subdivided_mask_array = ttp._ttp_tile_object_mask_array(subdivided_detail_tile, 48, 48, "mask_feather", 0)
assert_equal(0.0 < float(subdivided_mask_array[16, 15, 0]) < 1.0, True, "mask_feather should soften subdivided object masks even when blend is zero")
strict_subdivided, _strict_subdivided_weights = assemble_node.assemble_tiles(
blend_multiplier=1.0,
output_scale=1.0,
use_priority=True,
base_canvas_mode="black",
mask_blend_mode="mask_feather",
auto_composite_policy="strict_layer",
tile_set=subdivided_tile_set,
)
safe_subdivided, _safe_subdivided_weights = assemble_node.assemble_tiles(
blend_multiplier=1.0,
output_scale=1.0,
use_priority=True,
base_canvas_mode="black",
mask_blend_mode="mask_feather",
auto_composite_policy="safe_auto",
tile_set=subdivided_tile_set,
)
strict_subdivided_edge = strict_subdivided.array[0, 32, 24]
safe_subdivided_edge = safe_subdivided.array[0, 32, 24]
safe_subdivided_center = safe_subdivided.array[0, 32, 32]
assert_equal(float(safe_subdivided_edge[0]) > float(strict_subdivided_edge[0]), True, "safe auto should not let a repeatedly subdivided masked tile cut a hard hole in the lower face tile")
assert_equal(float(safe_subdivided_center[2]) > float(safe_subdivided_center[0]), True, "safe auto should still keep the subdivided detail strong at the mask center")
split_body_mask = Image.new("L", (64, 32), 255)
split_body_source_mask = ttp._ttp_encode_object_mask_data([split_body_mask], [0], [0, 0, 64, 32])
split_body_left_mask = ttp._ttp_encode_object_mask_data([split_body_mask], [0], [0, 0, 32, 32])
split_body_right_mask = ttp._ttp_encode_object_mask_data([split_body_mask], [0], [32, 0, 64, 32])
split_body_left_tile = {
"name": "body_context_grid_1",
"label": "person body large context",
"source": "auto_context",
"core_box": [0, 0, 32, 32],
"sample_box": [0, 0, 40, 32],
"tile_canvas_size": [40, 32],
"tile_canvas_box": [0, 0, 40, 32],
"overlap_edges_px_source": {"left": 0, "right": 8, "top": 0, "bottom": 0},
"blend": 8,
"importance": 0.35,
"priority": 10.0,
"layer": 0,
"occlusion_priority": 0,
"recommended_composite_mode": "context",
"object_mask": split_body_left_mask,
"object_mask_source": split_body_source_mask,
}
split_body_right_tile = {
"name": "body_context_grid_2",
"label": "person body large context",
"source": "auto_context",
"core_box": [32, 0, 32, 32],
"sample_box": [24, 0, 40, 32],
"tile_canvas_size": [40, 32],
"tile_canvas_box": [0, 0, 40, 32],
"overlap_edges_px_source": {"left": 8, "right": 0, "top": 0, "bottom": 0},
"blend": 8,
"importance": 0.35,
"priority": 10.0,
"layer": 0,
"occlusion_priority": 0,
"recommended_composite_mode": "context",
"object_mask": split_body_right_mask,
"object_mask_source": split_body_source_mask,
}
split_source_mask_array = ttp._ttp_tile_object_mask_array(split_body_left_tile, 40, 32, "mask_feather", 8)
assert_equal(float(split_source_mask_array[16, 36, 0]) > 0.5, True, "subdivided context masks should use source masks so overlap regions stay available")
split_body_tile_set = {
"type": "ttp_smart_tile_set",
"tile_meta": {
"type": "ttp_smart_tile",
"original_size": [64, 32],
"tiles": [split_body_left_tile, split_body_right_tile],
},
"tile_images": [
ttp.pil2tensor(Image.new("RGB", (40, 32), (90, 90, 90)))[0],
ttp.pil2tensor(Image.new("RGB", (40, 32), (90, 90, 90)))[0],
],
}
_split_body_output, split_body_weights = assemble_node.assemble_tiles(
blend_multiplier=1.0,
output_scale=1.0,
use_priority=True,
base_canvas_mode="black",
mask_blend_mode="mask_feather",
auto_composite_policy="safe_auto",
tile_set=split_body_tile_set,
)
split_body_single_weight = float(split_body_weights.array[0, 16, 16, 0])
split_body_seam_weight = float(split_body_weights.array[0, 16, 32, 0])
assert_equal(split_body_seam_weight > split_body_single_weight + 0.2, True, "subdivided context mask seams should show real overlap blending")
fractional_scale_tile_set = {
"type": "ttp_smart_tile_set",
"tile_meta": {
"type": "ttp_smart_tile",
"original_size": [3, 1],
"tiles": [
{
"name": f"column_{index}",
"label": "grid",
"core_box": [index, 0, 1, 1],
"sample_box": [index, 0, 1, 1],
"tile_canvas_size": [1, 1],
"tile_canvas_box": [0, 0, 1, 1],
"overlap_edges_px_source": {"left": 0, "right": 0, "top": 0, "bottom": 0},
"blend": 0,
"importance": 1.0,
"priority": 0.0,
}
for index in range(3)
],
},
"tile_images": [
ttp.pil2tensor(Image.new("RGB", (1, 1), (index * 80, index * 80, index * 80)))[0]
for index in range(3)
],
}
_fractional_output, fractional_weights = assemble_node.assemble_tiles(
blend_multiplier=1.0,
output_scale=1.4,
use_priority=False,
base_canvas_mode="black",
weight_preview_mode="coverage",
tile_set=fractional_scale_tile_set,
)
assert_equal(list(fractional_weights.shape), [1, 1, 4, 3], "fractional output scale should round the full canvas size")
assert_equal(float(fractional_weights.array.min()), 1.0, "fractional output scale should not leave rounded tile boundary gaps")
fractional_edges = ttp._ttp_scaled_overlap_edges([1, 0, 3, 1], {"left": 1, "right": 0, "top": 0, "bottom": 0}, 1.4, 5, 1)
assert_equal(fractional_edges["left"], 2, "fractional overlap edges should use source endpoints instead of independent length rounding")
deferred_output, deferred_weights = assemble_node.assemble_tiles(
blend_multiplier=1.0,
output_scale=1.0,
scripts/smoke_smart_tile_interactive.mjs
+6 -1
View File
@@ -31,7 +31,7 @@ assert(!/releasePointerCapture/.test(beginDrag), "existing tile drag should not
assert(!/writeLayout\(node, nextTiles, newIndex\);/.test(createTileBeforeMove), "empty-stage pointerdown should not create a tile before dragging");
assert(/if \(!hasDragged && movedPx < dragThresholdPx\)/.test(createTile), "new tile creation should wait until movement crosses the threshold");
assert(/async function inferSmartTileLayout/.test(source), "tile editor should expose an Infer action");
assert(/app\.queuePrompt\(0\)/.test(source), "Infer action should queue the current ComfyUI graph");
assert(/app\.queuePrompt\(0,\s*1,\s*\[String\(node\.id\)\]\)/.test(source), "Infer action should queue only the current Smart Tile node");
assert(/ttp-smart-tile-layout/.test(source), "Infer action should listen for backend layout updates");
assert(/auto_detect_request/.test(source), "Infer action should increment the node inference request widget");
assert(!/\/ttp\/smart_tile\/analyze/.test(source), "Infer action should not use the old analysis route");
@@ -64,6 +64,11 @@ assert(/GRID_MASK_MODES/.test(source), "tile editor should expose grid mask inhe
assert(/async function gridTilesWithInheritedMask/.test(source), "Grid in should be able to inherit and crop object masks");
assert(/async function refreshInheritedMasks/.test(source), "tile editor should refresh inherited masks after manual edits");
assert(/function cropObjectMaskForTile/.test(source), "Grid in should crop object masks per child tile");
assert(/function isLargeContextTile/.test(source), "Grid in should detect huge context tiles");
assert(/function contextGridTileMetadata/.test(source), "Grid in should keep huge context children on the lowest layer");
assert(/tileAreaRatio\(tile\) >= largeContextAreaRatio/.test(source), "huge non-detail tiles should be treated as context tiles");
assert(/const maskData = sourceObjectMaskData\(sourceTile\);/.test(source), "huge context Grid in should still preserve and crop the body mask");
assert(!/sourceIsLargeContext \? null : sourceObjectMaskData/.test(source), "huge context Grid in must not drop the source body mask");
assert(/crop_mask_skip_empty/.test(source), "Grid in should allow skipping empty mask child tiles");
assert(/createButton\(`Grid in T\$\{selectedIndex \+ 1\}`, async \(\) =>/.test(source), "Grid in should run asynchronously for mask cropping");
assert(/ensurePaintMaskCanvas/.test(source), "tile editor should create a paint mask canvas");
web/js/ttp_smart_tile_interactive.js
+81 -4
View File
@@ -7,6 +7,7 @@ const MAX_TILES = 64;
const MAX_GRID_AXIS = 8;
const dragThresholdPx = 3;
const snapGuideThresholdPx = 10;
const largeContextAreaRatio = 0.9;
const STORAGE_PREFIX = "ttp_smart_tile_interactive_layout";
const GRID_MASK_MODES = ["off", "crop_mask", "crop_mask_skip_empty"];
const TILE_METADATA_KEYS = [
@@ -217,7 +218,73 @@ function maskCanvasHasForeground(canvas) {
return false;
}
function tileAreaRatio(tile) {
const width = Math.max(0, Number(tile?.x1 ?? 0) - Number(tile?.x0 ?? 0));
const height = Math.max(0, Number(tile?.y1 ?? 0) - Number(tile?.y0 ?? 0));
return Math.max(0, Math.min(1, width * height));
}
function tileSearchText(tile) {
return [
tile?.label,
tile?.name,
tile?.source,
tile?.semantic_category,
tile?.semantic_role,
tile?.recommended_composite_mode,
].filter(Boolean).join(" ").toLowerCase();
}
function isDetailTileText(text) {
return [
"face", "head", "eye", "eyes", "eyelash", "eyebrow", "glasses",
"mouth", "lip", "lips", "teeth", "nose", "ear", "ears",
"hand", "hands", "finger", "fingers", "text", "letter", "logo",
"jewelry", "ornament", "detail", "focus",
].some((word) => text.includes(word));
}
function isLargeContextTile(tile) {
const text = tileSearchText(tile);
if (/(background|context|full image|full_image|large context)/.test(text)) {
return true;
}
return tileAreaRatio(tile) >= largeContextAreaRatio && !isDetailTileText(text);
}
function contextGridTileMetadata(sourceTile, childIndex) {
const baseName = String(sourceTile?.name || sourceTile?.label || "large_context").replace(/\s+/g, "_");
const rawLabel = String(sourceTile?.label || "large context");
const label = /context|background/i.test(rawLabel) ? rawLabel : `${rawLabel} large context`;
const metadata = {
name: `${baseName}_context_grid_${childIndex + 1}`,
source: sourceTile?.source || "auto_context",
label,
priority: Math.min(10, Number(sourceTile?.priority ?? 10) || 10),
importance: Math.min(0.35, Number(sourceTile?.importance ?? 0.35) || 0.35),
layer: 0,
object_id: 0,
occlusion_priority: 0,
semantic_category: "background",
semantic_role: "context",
recommended_layer: 0,
recommended_priority: 10,
recommended_occlusion_priority: 0,
recommended_composite_mode: "context",
};
if (sourceTile?.pad !== undefined) {
metadata.pad = sourceTile.pad;
}
if (sourceTile?.blend !== undefined) {
metadata.blend = sourceTile.blend;
}
return metadata;
}
function inheritedGridTileMetadata(sourceTile, childIndex) {
if (isLargeContextTile(sourceTile)) {
return contextGridTileMetadata(sourceTile, childIndex);
}
const metadata = {};
for (const key of TILE_METADATA_KEYS) {
if (key !== "name" && key !== "object_mask" && sourceTile?.[key] !== undefined) {
@@ -297,7 +364,12 @@ function cropObjectMaskForTile(node, maskImage, maskBox, tile) {
async function gridTilesWithInheritedMask(node, sourceTile, columns, rows, mode) {
const children = gridTiles(node, columns, rows, sourceTile);
if (mode === "off") {
return { tiles: children, cropped: 0, skipped: 0, emptied: 0 };
return {
tiles: children.map((child, index) => normalizeTile(node, { ...child, ...inheritedGridTileMetadata(sourceTile, index) })),
cropped: 0,
skipped: 0,
emptied: 0,
};
}
const maskData = sourceObjectMaskData(sourceTile);
@@ -355,7 +427,7 @@ async function refreshInheritedMasks(node, tiles, mode) {
};
for (const [index, tile] of tiles.entries()) {
const sourceMask = tile?.object_mask_source;
const sourceMask = sourceObjectMaskData(tile);
if (!sourceMask?.data) {
unchanged += 1;
result.push(tile);
@@ -363,6 +435,11 @@ async function refreshInheritedMasks(node, tiles, mode) {
}
const { maskBox, maskImage } = await loadCachedMask(sourceMask);
const nextTile = { ...tile };
if (isLargeContextTile(tile)) {
const contextMetadata = contextGridTileMetadata(tile, index);
contextMetadata.name = tile?.name || contextMetadata.name;
Object.assign(nextTile, contextMetadata);
}
const refreshedMask = cropObjectMaskForTile(node, maskImage, maskBox, tile);
if (refreshedMask) {
nextTile.object_mask = refreshedMask;
@@ -923,10 +1000,10 @@ async function inferSmartTileLayout(node) {
if (requestWidget) {
requestWidget.value = Number(requestWidget.value ?? 0) + 1;
}
node.ttpSmartTileStatus = `Queued ${mode} inference...`;
node.ttpSmartTileStatus = `Queued ${mode} inference for this node only...`;
renderEditor(node);
try {
await app.queuePrompt(0);
await app.queuePrompt(0, 1, [String(node.id)]);
} catch (error) {
if (requestWidget) {
requestWidget.value = 0;