Add refining only the character or only the background with another model
Two more ways under "Redraw with another model" (want: character, background). The character is cut out (InSPyReNet, made solid, grown 12 px); the character way blurs that mask so the second model's redraw fades out at the edge (Differential Diffusion), the background way redraws its inverse; both paste the redraw back through the mask, so the rest of the picture stays the same to the pixel. Denoise 0.45 from GPT's comparison of 2026-11-10. Both built through the real page on the test ComfyUI on Anima (Harem 2.9B, RealSkin, Kirazuri), 1440x1024: outside the soft edge and outside the background nothing changed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
DemonGatanjieu
co-authored by
Claude Opus 5.5
Claude Opus 5.5
parent
9e0062f331
commit
f0a026142a
@@ -32,6 +32,8 @@
|
||||
- When the canvas's model reads English only (SD 1.5, SDXL, Pony, Illustrious, Flux and the like) and the prompt is Chinese, Japanese or Korean, AMB says so before running: such pictures follow nothing of the prompt and show whatever the model leans to. An AI's run is stopped once and told to write the prompt in English (called again unchanged, it runs as written); the job sheet's Start warns once and runs on the second press. Models that read Chinese (Anima, Qwen-Image, Z-Image) are not stopped.
|
||||
- 现成做法接在别的步骤后面时，不再接上一步模型线上的 Tile LLLite、ControlNet 这类跟图片尺寸绑定的补丁：那一步的补丁只属于那一步。以前在 Tile 放大后面接脸手细化或再放大，可能出错甚至让 ComfyUI 整个崩掉（测试里崩过一次）。
|
||||
- A ready-made way going on from another step no longer takes the size-bound patches on that step's model line (a Tile LLLite, a ControlNet): they belong to their own pass. Face and hand detailing or another enlargement after a Tile redraw could fail, or crash ComfyUI (it did once in a test).
|
||||
- 换模型精修多了两种：**只改人物**和**只改场景**。AMB 先自动抠出人物，人物那一种只让第二个模型重画人物，边缘慢慢减弱，场景一个像素都不动；场景那一种只重画背景，人物和它边上一圈原样保留。这样可以先用一个模型把整张图的构图和动作定好（人坐不坐得住椅子，要两样一起画才对），再让擅长人物或擅长场景的模型只改它擅长的那一部分。强度默认 0.45，是这次对照测试里姿势保得住、变化又看得出的一档。重画的是哪一块，会在预览里显示（白色的部分）。自动抠图常会把人坐着的椅子、手里拿的伞和包也算进人物：只改人物时它们跟着人一起重画，接触的地方还对得上；只改场景时它们跟人一起原样保留。原来的整张重画保持不变。实测 RTX 4050 6GB、Anima（Harem 出图，RealSkin 只改人物、Kirazuri 只改场景），1440×1024，一次跑完三张约 326 秒。
|
||||
- Refining with another model has two more kinds: **only the character** and **only the background**. AMB cuts the character out first; the character kind has the second model redraw only the character, fading out at its edge, the scene not changed by a single pixel; the background kind redraws only the background, the character and a band around it left as they were. So one model can settle the whole picture's layout and pose first (a character only sits on a bench when both are drawn together), and a model good at characters or at scenery changes only its part. The strength is 0.45, in a comparison the one that kept the pose and still showed the change. The preview shows what is redrawn (white). The cut-out often takes the bench the character sits on and the umbrella or bag it holds along: redrawn with the character in the first kind, so where they touch still fits, and kept with it in the second. The whole-picture refine is unchanged. Tried on an RTX 4050 6 GB with Anima (Harem draws, RealSkin only the character, Kirazuri only the background), 1440×1024: about 326 s for the three pictures in one run.
|
||||
|
||||
## v1.63.0 — unreleased
|
||||
|
||||
|
||||
@@ -60,7 +60,7 @@
|
||||
Most "AI + image" tools either hide ComfyUI behind a cloud generator or ask an LLM to guess raw node connections from scratch—often breaking your open workflow. **AI Workbench** takes a practical middle route built around your own ComfyUI library:
|
||||
|
||||
- **Your open workflow stays safe.** By default (**AI edits a copy**), the first canvas action duplicates your workflow into a dedicated task tab. Your own tab is never touched, and every step on the timeline has **Undo** and **Find on canvas**.
|
||||
- **Tested building blocks instead of guesswork.** Rather than inventing fragile graphs, the AI opens **your saved workflows**, **any gallery picture's embedded workflow**, or **Official examples**, and extends them with **Ways** tested here (two-pass Tile upscale, mid-way model switch, crop-and-stitch or LanPaint area repaint, hand repair, background swap, pose/depth control…) plus on-demand **Skills** (such as matching pose complexity to your model's parameter scale, or planning multi-stage passes). You can also click **Use it this time** on any Way to lock the AI to your preferred method.
|
||||
- **Tested building blocks instead of guesswork.** Rather than inventing fragile graphs, the AI opens **your saved workflows**, **any gallery picture's embedded workflow**, or **Official examples**, and extends them with **Ways** tested here (two-pass Tile upscale, mid-way model switch, another model on only the character or only the background, crop-and-stitch or LanPaint area repaint, hand repair, background swap, pose/depth control…) plus on-demand **Skills** (such as matching pose complexity to your model's parameter scale, or planning multi-stage passes). You can also click **Use it this time** on any Way to lock the AI to your preferred method.
|
||||
- **Steer before committing GPU time.** On multi-pass workflows, the AI can run a few fast first-pass **base previews** first; **Pick a base** opens on the right-hand **Stage**, you click **Take this one**, and the second pass sharpens that exact image without redrawing the base. Finished runs appear with a **before/after comparison slider**, **Turned out well / Not quite** feedback buttons, and a `✎` brush tool to mark regions for local repainting.
|
||||
- **Clear boundaries (`Gates`).** You decide what the AI may **Go ahead** with, must **Ask me first** about, or may **Never** do: changing the canvas, using the GPU, downloading models, scanning models online, moving model files. Two more settings cover nodes it places by itself: any custom node pack or only those of the Ways, and whether an untried route of its own waits for your OK (with an optional per-task `🔒` privacy switch when you want prompt text and pictures kept local as `[P1]` / `[img1]` tokens instead of sent to the AI). Works locally with Claude, Cursor, Antigravity, Cherry Studio, VS Code and other MCP-capable apps—copy one sentence from **Connect an AI** to set it up, or see [docs/guides/mcp.md](docs/guides/mcp.md).
|
||||
|
||||
@@ -249,7 +249,7 @@ Open the browser with **Shift + B** (press again to close), the cube button at t
|
||||
很多人一听“在 ComfyUI 里接 AI”，第一反应往往是两个顾虑：要么怕它像云端生图软件一样绕开自己的本地模型库，要么怕大模型瞎猜节点连线，把自己辛辛苦苦调好的画布搅得一团糟。**AI 工作台**走的是另一条路——**让 AI 当你的操作助手，规矩由 AMB 守住**：
|
||||
|
||||
- **默认只改“副本”，绝不弄乱你开着的画布。** 任务默认开启 **AI 改副本**：AI 第一次动手前，AMB 自动把工作流复制到专属的任务标签页里操作，你原来的标签页一笔不动；时间线上的每一步都能点 **撤销** 或 **在画布上找到**。
|
||||
- **不靠瞎猜连线，优先用你的工作流和实测“做法”。** AI 会直接调用**你存好的工作流及预设**、**图库某张图自带的工作流**，或是开箱即用的 **官方示例**；需要加功能时，由 AMB 按你已装的节点包搭上实测过的 **做法**（Tile 锁图二段放大、半途换模型接着画、裁切缝合 / LanPaint 局部重绘、修手先放大再修、换背景、姿势/深度控制……），并配合按需 **Skill**（按主模型参数量给动作建议、分阶段出图思路）。你还能在做法面板点 **这次用它**，直接指定本次任务必须走哪套做法。
|
||||
- **不靠瞎猜连线，优先用你的工作流和实测“做法”。** AI 会直接调用**你存好的工作流及预设**、**图库某张图自带的工作流**，或是开箱即用的 **官方示例**；需要加功能时，由 AMB 按你已装的节点包搭上实测过的 **做法**（Tile 锁图二段放大、半途换模型接着画、换模型只改人物或只改场景、裁切缝合 / LanPaint 局部重绘、修手先放大再修、换背景、姿势/深度控制……），并配合按需 **Skill**（按主模型参数量给动作建议、分阶段出图思路）。你还能在做法面板点 **这次用它**，直接指定本次任务必须走哪套做法。
|
||||
- **先挑快底图，再跑慢精修，展台拉帘对比。** 跑两段放大这类耗时流程时，AI 可以先快速跑几张第一段底图预览（不污染正式输出目录）；右侧**展台**弹出挑选面板，你点 **就选这张**，第二段直接接着这张图做细节重绘，省掉重跑底图的时间。出图后展台提供**前后拉帘对比**、**效果好 / 不太行** 按钮，哪里不满意还能点图片右上角 `✎` 用画笔涂出区域让它只重画那一块。
|
||||
- **权限全由“关卡”把关。** 改画布、用显卡出图、下载模型、联网扫描模型、挪动模型文件，都能在 **关卡** 里设为“直接做 / 先问我 / 不许做”；AI 自己往画布上加节点时，能不能用任意第三方节点包、没验证过的新路线要不要先问你，也在这里设（任务标题栏还带可选的 `🔒` 隐私开关，开启后提示词与图片名会换成 `[P1]`、`[img1]` 代号，不把原图和原文发给 AI）。支持连接 Antigravity、Claude、Cursor、Cherry Studio、VS Code 等支持 MCP 的本机软件——点工作台里的 **连接 AI** 复制一句话发给它就能配好，详见 [docs/guides/mcp.md](docs/guides/mcp.md)。
|
||||
|
||||
|
||||
@@ -0,0 +1,257 @@
|
||||
{
|
||||
"job": "refine",
|
||||
"want": "background",
|
||||
"rank": 9,
|
||||
"short": "background only",
|
||||
"title": {
|
||||
"zh": "换模型精修·只改场景",
|
||||
"en": "Refine only the background"
|
||||
},
|
||||
"about": {
|
||||
"zh": "自动抠出人物，换另一个模型只重画背景，人物和它边上一圈原样不动，另存一张。",
|
||||
"en": "Cuts the character out; another model redraws only the background, the character and a band around it left as they were; saved beside it."
|
||||
},
|
||||
"source": {
|
||||
"zh": "常见的“抠图换背景”做法，只是背景不换新、只轻轻重画，强度按 2026-11-10 的场景重画对照定",
|
||||
"en": "The common cut-out-and-replace-the-background way, the background only lightly redrawn, not replaced; its strength from a background redraw comparison of 2026-11-10"
|
||||
},
|
||||
"packs": {
|
||||
"InspyrenetRembg": "comfyui-inspyrenet-rembg"
|
||||
},
|
||||
"nodes": {
|
||||
"second_model": {
|
||||
"class_type": "CheckpointLoaderSimple",
|
||||
"like": "$loader",
|
||||
"inputs": {}
|
||||
},
|
||||
"second_positive": {
|
||||
"class_type": "CLIPTextEncode",
|
||||
"like": "$sampler.positive",
|
||||
"inputs": {
|
||||
"clip": ["second_model", 1]
|
||||
}
|
||||
},
|
||||
"second_negative": {
|
||||
"class_type": "CLIPTextEncode",
|
||||
"like": "$sampler.negative",
|
||||
"inputs": {
|
||||
"clip": ["second_model", 1]
|
||||
}
|
||||
},
|
||||
"cut": {
|
||||
"class_type": "InspyrenetRembg",
|
||||
"inputs": {
|
||||
"image": ["$save.images", "*"],
|
||||
"torchscript_jit": "default"
|
||||
}
|
||||
},
|
||||
"solid": {
|
||||
"class_type": "ThresholdMask",
|
||||
"inputs": {
|
||||
"mask": ["cut", 1],
|
||||
"value": 0.5
|
||||
}
|
||||
},
|
||||
"edge": {
|
||||
"class_type": "GrowMask",
|
||||
"inputs": {
|
||||
"mask": ["solid", 0],
|
||||
"expand": 12,
|
||||
"tapered_corners": true
|
||||
}
|
||||
},
|
||||
"background": {
|
||||
"class_type": "InvertMask",
|
||||
"inputs": {
|
||||
"mask": ["edge", 0]
|
||||
}
|
||||
},
|
||||
"encode": {
|
||||
"class_type": "VAEEncode",
|
||||
"inputs": {
|
||||
"pixels": ["$save.images", "*"],
|
||||
"vae": ["second_model", 2]
|
||||
}
|
||||
},
|
||||
"area": {
|
||||
"class_type": "SetLatentNoiseMask",
|
||||
"inputs": {
|
||||
"samples": ["encode", 0],
|
||||
"mask": ["background", 0]
|
||||
}
|
||||
},
|
||||
"redraw": {
|
||||
"class_type": "KSampler",
|
||||
"like": "$sampler",
|
||||
"inputs": {
|
||||
"model": ["second_model", 0],
|
||||
"positive": ["second_positive", 0],
|
||||
"negative": ["second_negative", 0],
|
||||
"latent_image": ["area", 0],
|
||||
"denoise": 0.45
|
||||
}
|
||||
},
|
||||
"decode": {
|
||||
"class_type": "VAEDecode",
|
||||
"inputs": {
|
||||
"samples": ["redraw", 0],
|
||||
"vae": ["second_model", 2]
|
||||
}
|
||||
},
|
||||
"paste": {
|
||||
"class_type": "ImageCompositeMasked",
|
||||
"inputs": {
|
||||
"destination": ["$save.images", "*"],
|
||||
"source": ["decode", 0],
|
||||
"x": 0,
|
||||
"y": 0,
|
||||
"resize_source": false,
|
||||
"mask": ["background", 0]
|
||||
}
|
||||
},
|
||||
"save": {
|
||||
"class_type": "SaveImage",
|
||||
"inputs": {
|
||||
"images": ["paste", 0],
|
||||
"filename_prefix": "refined_background"
|
||||
}
|
||||
},
|
||||
"as_picture": {
|
||||
"class_type": "MaskToImage",
|
||||
"inputs": {
|
||||
"mask": ["background", 0]
|
||||
}
|
||||
},
|
||||
"look": {
|
||||
"class_type": "PreviewImage",
|
||||
"inputs": {
|
||||
"images": ["as_picture", 0]
|
||||
}
|
||||
}
|
||||
},
|
||||
"after": "The redraw's loader (second_model) starts as the same checkpoint: set_model it to the model the user wants for the background (ask_user, search_models), add_lora a style LoRA onto it if they want one, and write its prompts (second_positive, second_negative, copied from the first) in that model's style. Then run_workflow; the picture is saved beside it (prefix refined_background), the character the same to the pixel. Denoise 0.45: textures and light take the second model's look and the layout stays; at 0.6 the ground gets heavier and the edge round the character may show (dark chair legs most). The preview shows what is redrawn (white): part of the character in it, say so.",
|
||||
"gives": {
|
||||
"sampler": "redraw",
|
||||
"save": "save"
|
||||
},
|
||||
"own_model": {
|
||||
"nodes": ["second_model"],
|
||||
"copies_loras": false
|
||||
},
|
||||
"notes": "The background is the inverse of the character's mask grown 12 px, so the character and a 12 px band around it, where it meets a bench or the ground, stay as they were; the paste-back keeps them to the pixel. In the 2026-11-10 comparison (an Anima picture, Kirazuri redrawing the background) this way at 0.45 changed the scene about 3.6% on average and the character not at all; the bench still held the character up. The cut-out takes what stands out in front, often with what the character sits on or holds (a bench, an umbrella, a bag): kept with it, so where they touch stays as it was.",
|
||||
"models": [],
|
||||
"tested": {
|
||||
"gpu": "RTX 4050 Laptop",
|
||||
"vram_gb": 6,
|
||||
"date": "2026-11-10",
|
||||
"runs": [
|
||||
{
|
||||
"model": "Anima (Harem 2.9B, then Kirazuri v4.0 on the background, denoise 0.45)",
|
||||
"size": "1440×1024",
|
||||
"seconds": 240
|
||||
}
|
||||
]
|
||||
},
|
||||
"variants": {
|
||||
"anima": {
|
||||
"nodes": {
|
||||
"second_model": {
|
||||
"class_type": "UNETLoader",
|
||||
"like": "$loader",
|
||||
"inputs": {}
|
||||
},
|
||||
"cut": {
|
||||
"class_type": "InspyrenetRembg",
|
||||
"inputs": {
|
||||
"image": ["$save.images", "*"],
|
||||
"torchscript_jit": "default"
|
||||
}
|
||||
},
|
||||
"solid": {
|
||||
"class_type": "ThresholdMask",
|
||||
"inputs": {
|
||||
"mask": ["cut", 1],
|
||||
"value": 0.5
|
||||
}
|
||||
},
|
||||
"edge": {
|
||||
"class_type": "GrowMask",
|
||||
"inputs": {
|
||||
"mask": ["solid", 0],
|
||||
"expand": 12,
|
||||
"tapered_corners": true
|
||||
}
|
||||
},
|
||||
"background": {
|
||||
"class_type": "InvertMask",
|
||||
"inputs": {
|
||||
"mask": ["edge", 0]
|
||||
}
|
||||
},
|
||||
"encode": {
|
||||
"class_type": "VAEEncode",
|
||||
"inputs": {
|
||||
"pixels": ["$save.images", "*"],
|
||||
"vae": ["$decode.vae", "*"]
|
||||
}
|
||||
},
|
||||
"area": {
|
||||
"class_type": "SetLatentNoiseMask",
|
||||
"inputs": {
|
||||
"samples": ["encode", 0],
|
||||
"mask": ["background", 0]
|
||||
}
|
||||
},
|
||||
"redraw": {
|
||||
"class_type": "KSampler",
|
||||
"like": "$sampler",
|
||||
"inputs": {
|
||||
"model": ["second_model", 0],
|
||||
"positive": ["$sampler.positive", "*"],
|
||||
"negative": ["$sampler.negative", "*"],
|
||||
"latent_image": ["area", 0],
|
||||
"denoise": 0.45
|
||||
}
|
||||
},
|
||||
"decode": {
|
||||
"class_type": "VAEDecode",
|
||||
"inputs": {
|
||||
"samples": ["redraw", 0],
|
||||
"vae": ["$decode.vae", "*"]
|
||||
}
|
||||
},
|
||||
"paste": {
|
||||
"class_type": "ImageCompositeMasked",
|
||||
"inputs": {
|
||||
"destination": ["$save.images", "*"],
|
||||
"source": ["decode", 0],
|
||||
"x": 0,
|
||||
"y": 0,
|
||||
"resize_source": false,
|
||||
"mask": ["background", 0]
|
||||
}
|
||||
},
|
||||
"save": {
|
||||
"class_type": "SaveImage",
|
||||
"inputs": {
|
||||
"images": ["paste", 0],
|
||||
"filename_prefix": "refined_background"
|
||||
}
|
||||
},
|
||||
"as_picture": {
|
||||
"class_type": "MaskToImage",
|
||||
"inputs": {
|
||||
"mask": ["background", 0]
|
||||
}
|
||||
},
|
||||
"look": {
|
||||
"class_type": "PreviewImage",
|
||||
"inputs": {
|
||||
"images": ["as_picture", 0]
|
||||
}
|
||||
}
|
||||
},
|
||||
"after": "On this Anima model the redraw's loader (second_model) is a UNET loader that starts as the canvas's model; it reuses the canvas's text encoder, VAE and prompts. set_model it to the other Anima model the user wants for the background (ask_user, search_models); a LoRA on the first model is not on it: add_lora it onto second_model when the user wants that look kept. Then run_workflow; the picture is saved beside it (prefix refined_background), the character the same to the pixel. Denoise 0.45: textures and light take the second model's look and the layout stays; at 0.6 the ground gets heavier and the edge round the character may show (dark chair legs most). The preview shows what is redrawn (white): part of the character in it, say so."
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,287 @@
|
||||
{
|
||||
"job": "refine",
|
||||
"want": "character",
|
||||
"rank": 8,
|
||||
"short": "character only",
|
||||
"title": {
|
||||
"zh": "换模型精修·只改人物",
|
||||
"en": "Refine only the character"
|
||||
},
|
||||
"about": {
|
||||
"zh": "自动抠出人物，换另一个模型只重画人物，边缘慢慢减弱，场景原样不动，另存一张。",
|
||||
"en": "Cuts the character out; another model redraws only it, fading out at its edge, the scene left as it was; saved beside it."
|
||||
},
|
||||
"source": {
|
||||
"zh": "常见的抠图加渐变遮罩局部重画（ComfyUI 自带的 Differential Diffusion），强度按 2026-11-10 的人物重画对照定",
|
||||
"en": "The common cut-out with a graded-mask repaint (ComfyUI's own Differential Diffusion), its strength from a character redraw comparison of 2026-11-10"
|
||||
},
|
||||
"packs": {
|
||||
"InspyrenetRembg": "comfyui-inspyrenet-rembg"
|
||||
},
|
||||
"nodes": {
|
||||
"second_model": {
|
||||
"class_type": "CheckpointLoaderSimple",
|
||||
"like": "$loader",
|
||||
"inputs": {}
|
||||
},
|
||||
"second_positive": {
|
||||
"class_type": "CLIPTextEncode",
|
||||
"like": "$sampler.positive",
|
||||
"inputs": {
|
||||
"clip": ["second_model", 1]
|
||||
}
|
||||
},
|
||||
"second_negative": {
|
||||
"class_type": "CLIPTextEncode",
|
||||
"like": "$sampler.negative",
|
||||
"inputs": {
|
||||
"clip": ["second_model", 1]
|
||||
}
|
||||
},
|
||||
"cut": {
|
||||
"class_type": "InspyrenetRembg",
|
||||
"inputs": {
|
||||
"image": ["$save.images", "*"],
|
||||
"torchscript_jit": "default"
|
||||
}
|
||||
},
|
||||
"solid": {
|
||||
"class_type": "ThresholdMask",
|
||||
"inputs": {
|
||||
"mask": ["cut", 1],
|
||||
"value": 0.5
|
||||
}
|
||||
},
|
||||
"edge": {
|
||||
"class_type": "GrowMask",
|
||||
"inputs": {
|
||||
"mask": ["solid", 0],
|
||||
"expand": 12,
|
||||
"tapered_corners": true
|
||||
}
|
||||
},
|
||||
"as_picture": {
|
||||
"class_type": "MaskToImage",
|
||||
"inputs": {
|
||||
"mask": ["edge", 0]
|
||||
}
|
||||
},
|
||||
"fade": {
|
||||
"class_type": "ImageBlur",
|
||||
"inputs": {
|
||||
"image": ["as_picture", 0],
|
||||
"blur_radius": 31,
|
||||
"sigma": 0.5
|
||||
}
|
||||
},
|
||||
"area_mask": {
|
||||
"class_type": "ImageToMask",
|
||||
"inputs": {
|
||||
"image": ["fade", 0],
|
||||
"channel": "red"
|
||||
}
|
||||
},
|
||||
"soft": {
|
||||
"class_type": "DifferentialDiffusion",
|
||||
"inputs": {
|
||||
"model": ["second_model", 0]
|
||||
}
|
||||
},
|
||||
"encode": {
|
||||
"class_type": "VAEEncode",
|
||||
"inputs": {
|
||||
"pixels": ["$save.images", "*"],
|
||||
"vae": ["second_model", 2]
|
||||
}
|
||||
},
|
||||
"area": {
|
||||
"class_type": "SetLatentNoiseMask",
|
||||
"inputs": {
|
||||
"samples": ["encode", 0],
|
||||
"mask": ["area_mask", 0]
|
||||
}
|
||||
},
|
||||
"redraw": {
|
||||
"class_type": "KSampler",
|
||||
"like": "$sampler",
|
||||
"inputs": {
|
||||
"model": ["soft", 0],
|
||||
"positive": ["second_positive", 0],
|
||||
"negative": ["second_negative", 0],
|
||||
"latent_image": ["area", 0],
|
||||
"denoise": 0.45
|
||||
}
|
||||
},
|
||||
"decode": {
|
||||
"class_type": "VAEDecode",
|
||||
"inputs": {
|
||||
"samples": ["redraw", 0],
|
||||
"vae": ["second_model", 2]
|
||||
}
|
||||
},
|
||||
"paste": {
|
||||
"class_type": "ImageCompositeMasked",
|
||||
"inputs": {
|
||||
"destination": ["$save.images", "*"],
|
||||
"source": ["decode", 0],
|
||||
"x": 0,
|
||||
"y": 0,
|
||||
"resize_source": false,
|
||||
"mask": ["area_mask", 0]
|
||||
}
|
||||
},
|
||||
"save": {
|
||||
"class_type": "SaveImage",
|
||||
"inputs": {
|
||||
"images": ["paste", 0],
|
||||
"filename_prefix": "refined_character"
|
||||
}
|
||||
},
|
||||
"look": {
|
||||
"class_type": "PreviewImage",
|
||||
"inputs": {
|
||||
"images": ["fade", 0]
|
||||
}
|
||||
}
|
||||
},
|
||||
"after": "The redraw's loader (second_model) starts as the same checkpoint: set_model it to the model the user wants for the character (ask_user, search_models), add_lora a character or style LoRA onto it if they want one, and write its prompts (second_positive, second_negative, copied from the first) in that model's style. Then run_workflow; the picture is saved beside it (prefix refined_character), the scene the same to the pixel. Denoise 0.45: face, hair and clothes take the second model's look and the pose stays; 0.3 changes little; at 0.6 the face changes more and colours fade. The preview shows what is redrawn (white, fading at the edge): part of the character missing, or the background in it, say so.",
|
||||
"gives": {
|
||||
"sampler": "redraw",
|
||||
"save": "save"
|
||||
},
|
||||
"own_model": {
|
||||
"nodes": ["second_model"],
|
||||
"copies_loras": false
|
||||
},
|
||||
"notes": "The character's mask grows 12 px and is blurred about 15 px, so the redraw fades out at its edge; Differential Diffusion makes the mask's grey a weaker redraw, and the paste-back keeps everything outside it as it was. In the 2026-11-10 comparison (an Anima picture, RealSkin redrawing the character) this way at 0.45 changed the character about 5.5% on average and the scene 0.03%; the coat stayed on the bench. A small character in a wide picture gets few pixels for its face: repaint it (inpaint_crop_stitch) instead. The cut-out takes what stands out in front, often with what the character sits on or holds (a bench, an umbrella, a bag): redrawn with it, so where they touch still fits.",
|
||||
"models": [],
|
||||
"tested": {
|
||||
"gpu": "RTX 4050 Laptop",
|
||||
"vram_gb": 6,
|
||||
"date": "2026-11-10",
|
||||
"runs": [
|
||||
{
|
||||
"model": "Anima (Harem 2.9B, then RealSkin 1.3 on the character, denoise 0.45)",
|
||||
"size": "1440×1024",
|
||||
"seconds": 242
|
||||
}
|
||||
]
|
||||
},
|
||||
"variants": {
|
||||
"anima": {
|
||||
"nodes": {
|
||||
"second_model": {
|
||||
"class_type": "UNETLoader",
|
||||
"like": "$loader",
|
||||
"inputs": {}
|
||||
},
|
||||
"cut": {
|
||||
"class_type": "InspyrenetRembg",
|
||||
"inputs": {
|
||||
"image": ["$save.images", "*"],
|
||||
"torchscript_jit": "default"
|
||||
}
|
||||
},
|
||||
"solid": {
|
||||
"class_type": "ThresholdMask",
|
||||
"inputs": {
|
||||
"mask": ["cut", 1],
|
||||
"value": 0.5
|
||||
}
|
||||
},
|
||||
"edge": {
|
||||
"class_type": "GrowMask",
|
||||
"inputs": {
|
||||
"mask": ["solid", 0],
|
||||
"expand": 12,
|
||||
"tapered_corners": true
|
||||
}
|
||||
},
|
||||
"as_picture": {
|
||||
"class_type": "MaskToImage",
|
||||
"inputs": {
|
||||
"mask": ["edge", 0]
|
||||
}
|
||||
},
|
||||
"fade": {
|
||||
"class_type": "ImageBlur",
|
||||
"inputs": {
|
||||
"image": ["as_picture", 0],
|
||||
"blur_radius": 31,
|
||||
"sigma": 0.5
|
||||
}
|
||||
},
|
||||
"area_mask": {
|
||||
"class_type": "ImageToMask",
|
||||
"inputs": {
|
||||
"image": ["fade", 0],
|
||||
"channel": "red"
|
||||
}
|
||||
},
|
||||
"soft": {
|
||||
"class_type": "DifferentialDiffusion",
|
||||
"inputs": {
|
||||
"model": ["second_model", 0]
|
||||
}
|
||||
},
|
||||
"encode": {
|
||||
"class_type": "VAEEncode",
|
||||
"inputs": {
|
||||
"pixels": ["$save.images", "*"],
|
||||
"vae": ["$decode.vae", "*"]
|
||||
}
|
||||
},
|
||||
"area": {
|
||||
"class_type": "SetLatentNoiseMask",
|
||||
"inputs": {
|
||||
"samples": ["encode", 0],
|
||||
"mask": ["area_mask", 0]
|
||||
}
|
||||
},
|
||||
"redraw": {
|
||||
"class_type": "KSampler",
|
||||
"like": "$sampler",
|
||||
"inputs": {
|
||||
"model": ["soft", 0],
|
||||
"positive": ["$sampler.positive", "*"],
|
||||
"negative": ["$sampler.negative", "*"],
|
||||
"latent_image": ["area", 0],
|
||||
"denoise": 0.45
|
||||
}
|
||||
},
|
||||
"decode": {
|
||||
"class_type": "VAEDecode",
|
||||
"inputs": {
|
||||
"samples": ["redraw", 0],
|
||||
"vae": ["$decode.vae", "*"]
|
||||
}
|
||||
},
|
||||
"paste": {
|
||||
"class_type": "ImageCompositeMasked",
|
||||
"inputs": {
|
||||
"destination": ["$save.images", "*"],
|
||||
"source": ["decode", 0],
|
||||
"x": 0,
|
||||
"y": 0,
|
||||
"resize_source": false,
|
||||
"mask": ["area_mask", 0]
|
||||
}
|
||||
},
|
||||
"save": {
|
||||
"class_type": "SaveImage",
|
||||
"inputs": {
|
||||
"images": ["paste", 0],
|
||||
"filename_prefix": "refined_character"
|
||||
}
|
||||
},
|
||||
"look": {
|
||||
"class_type": "PreviewImage",
|
||||
"inputs": {
|
||||
"images": ["fade", 0]
|
||||
}
|
||||
}
|
||||
},
|
||||
"after": "On this Anima model the redraw's loader (second_model) is a UNET loader that starts as the canvas's model; it reuses the canvas's text encoder, VAE and prompts. set_model it to the other Anima model the user wants for the character (ask_user, search_models); a LoRA on the first model is not on it: add_lora it onto second_model when the user wants that look kept. Then run_workflow; the picture is saved beside it (prefix refined_character), the scene the same to the pixel. Denoise 0.45: face, hair and clothes take the second model's look and the pose stays; 0.3 changes little; at 0.6 the face changes more and colours fade. The preview shows what is redrawn (white, fading at the edge): part of the character missing, or the background in it, say so."
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -203,6 +203,26 @@ class BlockTests(unittest.TestCase):
|
||||
with self.assertRaises(reference_blocks.MissingModel):
|
||||
reference_blocks.build("upscale_tile", missing, main_base=lambda: "Anima")
|
||||
|
||||
def test_refining_only_the_character_or_the_background_pastes_the_rest_back(self):
|
||||
self.assertEqual(reference_blocks.directory(EVERYTHING)["refine"], "Redraw with another model (want: midway, character, background)")
|
||||
self.assertEqual(reference_blocks.pick("refine", None, EVERYTHING), "refine_other_model", "no want: the whole picture")
|
||||
for want, kept_by in (("character", "area_mask"), ("background", "background")):
|
||||
with self.subTest(want=want):
|
||||
name = reference_blocks.pick("refine", want, EVERYTHING)
|
||||
nodes = reference_blocks.build(name, EVERYTHING, main_base=lambda: "Anima")["nodes"]
|
||||
self.assertEqual(nodes["second_model"]["class_type"], "UNETLoader")
|
||||
self.assertEqual(nodes["cut"]["inputs"]["image"], ["$save.images", "*"], "the step's picture is cut out")
|
||||
self.assertEqual((nodes["area"]["inputs"]["mask"], nodes["paste"]["inputs"]["mask"]), ([kept_by, 0], [kept_by, 0]),
|
||||
"redrawn where the mask is, and only there pasted onto the picture")
|
||||
self.assertEqual((nodes["paste"]["inputs"]["destination"], nodes["save"]["inputs"]["images"]), (["$save.images", "*"], ["paste", 0]))
|
||||
self.assertEqual(nodes["redraw"]["inputs"]["denoise"], 0.45, "the tested strength")
|
||||
sdxl = reference_blocks.build(name, EVERYTHING, main_base=lambda: "Illustrious")["nodes"]
|
||||
self.assertEqual((sdxl["second_model"]["class_type"], sdxl["encode"]["inputs"]["vae"]), ("CheckpointLoaderSimple", ["second_model", 2]))
|
||||
character = reference_blocks.build("refine_character", EVERYTHING, main_base=lambda: "Anima")["nodes"]
|
||||
self.assertEqual(character["redraw"]["inputs"]["model"], ["soft", 0], "the mask's grey a weaker redraw: the edge fades")
|
||||
with self.assertRaisesRegex(reference_blocks.BlockError, "needs comfyui-inspyrenet-rembg"):
|
||||
reference_blocks.build("refine_background", BUILT_IN)
|
||||
|
||||
def test_a_picture_from_words_is_built_by_the_model_family_on_the_model_picked(self):
|
||||
anima = {**EVERYTHING, "CLIPLoader": {"input": {"required": {"clip_name": [["clip_l.safetensors", "Anima/qwen_3_06b_base.safetensors"]]}}},
|
||||
"VAELoader": {"input": {"required": {"vae_name": [["sdxl_vae.safetensors", "Anima/qwenImageGGUF_vae.safetensors"]]}}}}
|
||||
|
||||
Reference in New Issue
Block a user