From 8b51dffd2b658b815fd679d7fd7dbf181a2b753d Mon Sep 17 00:00:00 2001 From: DemonGatanjieu <207458273+DemonGatanjieu@users.noreply.github.com> Date: Tue, 27 Oct 2026 15:59:55 +0800 Subject: [PATCH] Follow a pose reads it with DWPose; pose and depth maps left as previews OpenPose misread anime characters; DWPose (torch detector: the onnx one found nobody with this onnxruntime-gpu) reads a single anime character down to face and fingers. A crowded, heavily stylised picture can still come out empty, so both control blocks keep their map as a PreviewImage the AI sees in run_status's previews, and their next step says what to do when it is empty. Live on the isolated 8199 ComfyUI: DWPose followed a hand-behind-head pose (64 s); the three-elf picture stays unreadable for both readers. Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 4 +-- api/blocks/control_depth.json | 22 +++++++++++++++-- api/blocks/control_pose.json | 46 ++++++++++++++++++++++++++--------- 3 files changed, 57 insertions(+), 15 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 09ced83..9dda2da 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -15,7 +15,7 @@ - **背景移除**：把一张图的背景抠掉，存成透明 PNG（文件名以 nobg 开头）。需要节点包 comfyui-inspyrenet-rembg。 - **读图出标签**：你递给 AI 一张图，WD14 打标器读出它的 Danbooru 标签，AI 照着写提示词。需要节点包 ComfyUI-WD14-Tagger，第一次用时会下载打标模型（约 1.2GB）。AI 运行后，节点输出的文字（比如这些标签）现在也会带回给它。 - 装了 KJNodes 时，几种放大都会在最后把颜色对齐到原图（参考工作流的做法），防止放大后颜色跑偏；没装也照常能用，面板上写着"装了 KJNodes 会更好"。 - - 照着姿势画用的 OpenPose 认真人照片和 3D 图准，认二次元人物常认错；面板和给 AI 的说明里都写明了，二次元图用照着构图画。 + - 照着姿势画改用 DWPose 读姿势：二次元人物也认得准，连脸和手指都标（原来的 OpenPose 认二次元常认错）。人物挤成一团、画风很浓的图可能认不出，读出的骨架图或深度图会作为预览给 AI 看，空的它就知道换照着构图画。第一次用时会下载 DWPose 的模型（约 350MB）。 - 做法会按画布上的底模换搭法：画布上是 Anima 模型时，照着构图画、照着姿势画改用 Anima 专用的 LLLite 控制网，局部重绘·裁切缝合会加上 Anima 的重绘 LLLite。AI 照样只说用哪块。需要节点包 ComfyUI-Anima-LLLite 和 models/controlnet 里的 anima-lllite 模型。Anima 的 LLLite 比 SDXL 的控制网温和，提示词要和参考图说的一致（几个人、在干什么），AI 会被这样提醒。 - 画布上有好几个采样器时，AI 要指明用哪一个，AMB 不乱猜。 - AI 工作台右边多了**做法**面板：按"要做的事"分组列出所有做法，每块写着能干什么、从哪学来的。缺节点包或缺模型的变灰，写明缺什么，点缺的节点包会打开 ComfyUI-Manager 去搜它。每块有个开关：关掉的，AI 就看不到，硬要用也会被拒绝。AI 开工时只收到开着又能用的做法，其余的只附一行"装了什么还能用什么"。 @@ -33,7 +33,7 @@ - **Remove the background**: saves a picture with its background cut away as a transparent PNG (prefix nobg). Needs comfyui-inspyrenet-rembg. - **Tags from a picture**: from a picture you hand the AI, the WD14 tagger reads its Danbooru tags for the AI to write the prompt from. Needs ComfyUI-WD14-Tagger; the first time it downloads its model (about 1.2 GB). What a run's nodes show as words (these tags) now reaches the AI with the run. - With KJNodes, the upscales match their colours to the original at the end (as the source workflow does), so they don't drift; without it they work as before and the panel says they're better with KJNodes. - - OpenPose (Follow a pose) reads photos and 3D well and anime characters often wrongly; the panel and the AI are told so, and to use Follow a layout for anime. + - Follow a pose reads the pose with DWPose: anime characters too, face and fingers included (OpenPose often misread them). A crowded or very stylised picture may come out empty; the pose skeleton or depth map is left as a preview for the AI, which then turns to Follow a layout. The first time, DWPose downloads its models (about 350 MB). - A way is built for the canvas's model: on an Anima model, Follow a layout and Follow a pose use Anima's ControlNet-LLLite, and Repaint: crop and stitch adds Anima's repaint LLLite; the AI still only names the way. Needs ComfyUI-Anima-LLLite and anima-lllite models in models/controlnet. Anima's LLLite steers more gently than an SDXL ControlNet, so the prompt has to agree with the picture, and the AI is told so. - With several samplers on the canvas the AI has to say which one; nothing is guessed. - A **Ways** panel on the right of the AI workbench lists every way by job, what it does and where it came from. Ways missing a node pack or a model are greyed with what they lack; a missing pack opens ComfyUI-Manager searched for it. Each has a switch: one turned off is not offered to the AI, and building it is refused. When it begins, the AI gets only the ways that are on and can run here, with one line on what installing which pack would add. diff --git a/api/blocks/control_depth.json b/api/blocks/control_depth.json index 897943d..bfe132e 100644 --- a/api/blocks/control_depth.json +++ b/api/blocks/control_depth.json @@ -71,9 +71,18 @@ "end_percent": 0.7 }, "wire_to": "$sampler" + }, + "look": { + "class_type": "PreviewImage", + "inputs": { + "images": [ + "depth", + 0 + ] + } } }, - "after": "run_workflow: the picture now keeps that layout (strength 0.7, for the first 70% of the steps; lower either to let the model change more). The prompt still says what is drawn.", + "after": "run_workflow: the picture now keeps that layout (strength 0.7, for the first 70% of the steps; lower either to let the model change more). The prompt still says what is drawn. Its depth map is among the run's previews: if it is empty or wrong (a crowded or very stylised picture), the picture is too hard to read.", "variants": { "anima": { "packs": { @@ -118,9 +127,18 @@ "preserve_wrapper": true }, "wire_to": "$sampler" + }, + "look": { + "class_type": "PreviewImage", + "inputs": { + "images": [ + "depth", + 0 + ] + } } }, - "after": "On this Anima model an Anima ControlNet-LLLite steers it, more gently than an SDXL ControlNet: say in the prompt what the picture shows (how many people, what they do), or the prompt wins. Strength 1.0 for all the steps; up to 1.5 holds it closer. run_workflow." + "after": "On this Anima model an Anima ControlNet-LLLite steers it, more gently than an SDXL ControlNet: say in the prompt what the picture shows (how many people, what they do), or the prompt wins. Strength 1.0 for all the steps; up to 1.5 holds it closer. run_workflow. Its depth map is among the run's previews: if it is empty or wrong (a crowded or very stylised picture), the picture is too hard to read." } } } diff --git a/api/blocks/control_pose.json b/api/blocks/control_pose.json index b7c0c84..73b497a 100644 --- a/api/blocks/control_pose.json +++ b/api/blocks/control_pose.json @@ -1,21 +1,21 @@ { "job": "follow", "rank": 1, - "short": "its pose, for photos and 3D", + "short": "its pose", "title": { "zh": "照着姿势画", "en": "Follow a pose" }, "about": { - "zh": "你递给 AI 一张图，AMB 读出人物姿势（OpenPose），用和你底模对得上的控制网让新图摆出这个姿势。 OpenPose 认真人照片和 3D 图准，认二次元人物常常认错，二次元图用「照着构图画」更稳（参考工作流作者也这么说）。 画布上是 Anima 模型时，自动改用 Anima 专用的 LLLite 控制网（需要 ComfyUI-Anima-LLLite）。", - "en": "From a picture you hand the AI, Anomalous reads the pose (OpenPose) and steers the picture with a ControlNet that fits your model. OpenPose reads photos and 3D well and anime characters often wrongly; for anime, Follow a layout is steadier (as the source's author says). On an Anima model it uses an Anima ControlNet-LLLite instead (needs ComfyUI-Anima-LLLite)." + "zh": "你递给 AI 一张图，AMB 读出人物姿势（DWPose），用和你底模对得上的控制网让新图摆出这个姿势。 DWPose 认二次元人物也准，连脸和手指都标；人物挤成一团、画风很浓的图可能认不出，这时用「照着构图画」。 画布上是 Anima 模型时，自动改用 Anima 专用的 LLLite 控制网（需要 ComfyUI-Anima-LLLite）。", + "en": "From a picture you hand the AI, Anomalous reads the pose (DWPose) and steers the picture with a ControlNet that fits your model. DWPose reads anime characters too, face and fingers included; a crowded or very stylised picture may come out empty, then use Follow a layout. On an Anima model it uses an Anima ControlNet-LLLite instead (needs ComfyUI-Anima-LLLite)." }, "source": { "zh": "参考工作流的「底图控制网」组", "en": "The reference workflow's ControlNet group" }, "packs": { - "OpenposePreprocessor": "comfyui_controlnet_aux" + "DWPreprocessor": "comfyui_controlnet_aux" }, "takes": "reference", "control": "pose", @@ -27,7 +27,7 @@ } }, "pose": { - "class_type": "OpenposePreprocessor", + "class_type": "DWPreprocessor", "inputs": { "image": [ "reference", @@ -36,7 +36,10 @@ "detect_hand": "enable", "detect_body": "enable", "detect_face": "enable", - "resolution": 1024 + "resolution": 1024, + "bbox_detector": "yolox_l.torchscript.pt", + "pose_estimator": "dw-ll_ucoco_384_bs5.torchscript.pt", + "scale_stick_for_xinsr_cn": "disable" } }, "controlnet": { @@ -73,13 +76,22 @@ "end_percent": 0.8 }, "wire_to": "$sampler" + }, + "look": { + "class_type": "PreviewImage", + "inputs": { + "images": [ + "pose", + 0 + ] + } } }, - "after": "run_workflow: the picture now takes that pose (strength 0.8, for the first 80% of the steps; lower either for a looser pose). The prompt still says who and what: change it if it fights the pose. If the picture came out ignoring the pose, OpenPose likely misread an anime picture: try control_depth.", + "after": "run_workflow: the picture now takes that pose (strength 0.8, for the first 80% of the steps; lower either for a looser pose). The prompt still says who and what: change it if it fights the pose. Its pose skeleton is among the run's previews: if it is empty or wrong (a crowded or very stylised picture), try control_depth.", "variants": { "anima": { "packs": { - "OpenposePreprocessor": "comfyui_controlnet_aux", + "DWPreprocessor": "comfyui_controlnet_aux", "AnimaLLLiteApply_sdscripts": "ComfyUI-Anima-LLLite" }, "control": null, @@ -92,7 +104,7 @@ } }, "pose": { - "class_type": "OpenposePreprocessor", + "class_type": "DWPreprocessor", "inputs": { "image": [ "reference", @@ -101,7 +113,10 @@ "detect_hand": "enable", "detect_body": "enable", "detect_face": "enable", - "resolution": 1024 + "resolution": 1024, + "bbox_detector": "yolox_l.torchscript.pt", + "pose_estimator": "dw-ll_ucoco_384_bs5.torchscript.pt", + "scale_stick_for_xinsr_cn": "disable" } }, "steer": { @@ -122,9 +137,18 @@ "preserve_wrapper": true }, "wire_to": "$sampler" + }, + "look": { + "class_type": "PreviewImage", + "inputs": { + "images": [ + "pose", + 0 + ] + } } }, - "after": "On this Anima model an Anima ControlNet-LLLite steers it, more gently than an SDXL ControlNet: say in the prompt what the picture shows (how many people, what they do), or the prompt wins. Strength 1.0 for all the steps; up to 1.5 holds it closer. run_workflow." + "after": "On this Anima model an Anima ControlNet-LLLite steers it, more gently than an SDXL ControlNet: say in the prompt what the picture shows (how many people, what they do), or the prompt wins. Strength 1.0 for all the steps; up to 1.5 holds it closer. run_workflow. Its pose skeleton is among the run's previews: if it is empty or wrong (a crowded or very stylised picture), try control_depth." } } }