diff --git a/README.md b/README.md index 0318279..58628f7 100644 --- a/README.md +++ b/README.md @@ -2,13 +2,13 @@ > 中文 | [English](#english) -把**最多 6 张参考图**一次性注入 conditioning 的单节点,专为 **FLUX.2 dev / klein** 这类吃 `reference_latents` 的图像编辑模型设计。每张图有独立的 `strength`,**`strength = 0` 就跳过那张图** —— 于是一个静态工作流就能覆盖 0~6 张参考图,不用 bypass、不用动态改线、不用为不同图数准备多个工作流。 +把**最多 6 张参考图**一次性注入 conditioning 的单节点,专为 **FLUX.2 dev / klein** 这类吃 `reference_latents` 的图像编辑模型设计。每张图有独立的 `strength`,**`strength = 0` 就跳过那张图** —— 于是一个静态工作流就能覆盖 0~6 张参考图,不用 bypass、不用动态改线、不用为不同图数准备多个工作流。程序化提交还可用 `reference_list` 文本(图数=行数,无占位图)。 ## 它解决什么 ComfyUI 原生 `ReferenceLatent` 一次只喂一张参考图,要多张就得手动把多个 `ReferenceLatent` 串起来,而且没有逐图强度控制。本节点: -- **一个节点喂多张**(≤6),内部按原生 `reference_latents` append 语义逐张拼接; +- **一个节点喂多张**(≤6 socket,或 reference_list 不限),内部按原生 `reference_latents` append 语义逐张拼接; - **每张独立 strength**(范围 `0 ~ 5`),`0 = 跳过`,`1.0 = 原生强度`,`>1 增强`; - **内置 VAE 编码**,直接接 `IMAGE`,不用每张图外挂一个 `VAEEncode`; - **全 optional**:一张都不接 → conditioning 原样透传(等于纯文生图)。 @@ -41,6 +41,23 @@ VAELoader ─►│ vae │ 把 6 个图槽留着,想用几张就接几张(或者把不想要的那张 `strength` 调 0)。提交时不参与的槽不产生任何 latent,**云端/本地行为完全一致**,无需为不同图数维护多份工作流。 +## 程序化喂图:`reference_list`(无占位图,图数=行数) + +画布手动用 `image_N` socket;但**程序化/API 提交**时,socket 会留下"空槽"(要么占位图、要么动态改线)。为此节点提供另一条路 —— `reference_list` 多行文本: + +``` +2.0 iVBORw0KGgoAAAANS... ← 第 1 张,strength=2.0 +iVBORw0KGgoAAAANS... ← 第 2 张,strength 省略=1.0 +0.5 data:image/png;base64,iVBOR... ← 第 3 张,支持 data URI 前缀 +``` + +- **每行一张图**,格式 `[strength ]`,strength 可省(=1.0); +- **图数 = 非空行数**,完全由你填的文本决定 —— graph 结构恒定,**没有空槽、没有占位图、没有动态改线、没有多份 JSON**; +- 空行忽略,`strength=0` 的行跳过; +- 可与 `image_N` socket 同时用(socket 在前)。 + +后端只需把 N 张图拼成 N 行塞进这个字段,一个静态工作流适配任意张数。这就是"**静态的是 graph,动态的是数据**"。 + ## strength 怎么作用(以及它的局限) `reference_latents` 在 conditioning 里只是一串 latent,模型对其中每个 latent **没有独立的权重输入**。所以在 conditioning 这一层,"调强度"唯一能做的就是缩放这张图自己的 latent 数值。这是**近似**,不是干净的"影响力旋钮"——缩放会顺带改变 latent 在 VAE 空间的分布: @@ -79,13 +96,13 @@ MIT。基于 ComfyUI 公开的原生 `ReferenceLatent` 机制独立实现。 > [中文](#comfyui-multi-reference-latent) | English -A single node that injects **up to 6 reference images** into conditioning, built for **FLUX.2 dev / klein** and other edit models that consume `reference_latents`. Each image has its own `strength`, and **`strength = 0` skips that image** — so one static workflow covers 0–6 references without bypass, rewiring, or multiple workflows. +A single node that injects **up to 6 reference images** into conditioning, built for **FLUX.2 dev / klein** and other edit models that consume `reference_latents`. Each image has its own `strength`, and **`strength = 0` skips that image** — so one static workflow covers 0–6 references without bypass, rewiring, or multiple workflows. For programmatic submission there is also a `reference_list` text field (image count = number of lines, no placeholder images). ## What it solves Native `ReferenceLatent` feeds one reference at a time and has no per-image strength. This node: -- **Feeds many in one node** (≤6), chaining them with the native `reference_latents` append semantics; +- **Feeds many in one node** (≤6 sockets, or unlimited via reference_list), chaining them with the native `reference_latents` append semantics; - **Per-image strength** (range `0–5`): `0 = skip`, `1.0 = native`, `>1` stronger; - **Built-in VAE encode** — wire `IMAGE` directly, no per-image `VAEEncode`; - **All optional** — connect nothing and conditioning passes through unchanged (plain text-to-image). @@ -106,6 +123,23 @@ Native `ReferenceLatent` feeds one reference at a time and has no per-image stre Keep all 6 slots; wire as many as you need (or set an unwanted image's `strength` to 0). Disabled slots produce no latent at all, so behavior is identical locally and on any backend — no need to maintain separate workflows per image count. +## Programmatic input: `reference_list` (no placeholder images) + +`image_N` sockets are for the canvas; but for **programmatic / API submission**, sockets leave "empty slots" (needing placeholder images or graph rewiring). So the node offers another path — the `reference_list` multiline text: + +``` +2.0 iVBORw0KGgoAAAANS... ← image 1, strength 2.0 +iVBORw0KGgoAAAANS... ← image 2, strength omitted = 1.0 +0.5 data:image/png;base64,iVBOR... ← image 3, data URI prefix supported +``` + +- **one image per line**, format `[strength ]`, strength optional (=1.0); +- **image count = number of non-empty lines**, fully data-driven — the graph structure is fixed: **no empty slots, no placeholder images, no rewiring, no multiple JSONs**; +- blank lines ignored, lines with `strength=0` skipped; +- works alongside `image_N` sockets (sockets first). + +Your backend just joins N images into N lines and fills this field; one static workflow adapts to any count. The graph stays static, the data drives the image count. + ## How strength works (and its limits) `reference_latents` is just a list of latents in the conditioning; the model has **no per-latent weight input**. So at the conditioning level the only thing "strength" can do is scale that image's own latent values. This is an **approximation**, not a clean influence knob — scaling also shifts the latent's distribution in VAE space: diff --git a/nodes.py b/nodes.py index 4810140..f6aea35 100644 --- a/nodes.py +++ b/nodes.py @@ -1,9 +1,15 @@ """ comfyui_multi_referencelatent — MultiReferenceLatent -把若干张参考图(本节点上限 6)一次性挂进同一条 conditioning,供 FLUX.2 dev / klein +把若干张参考图一次性挂进同一条 conditioning,供 FLUX.2 dev / klein 等"读 reference_latents 的编辑模型"使用。等价于把多个原生 ReferenceLatent 串成一串, -外加内置 VAE 编码与逐图开关,省掉重复连线。 +外加内置 VAE 编码与逐图强度,省掉重复连线。 + +两种喂图方式,可单用也可并用: + A) image_1..6 这些 IMAGE socket —— 画布手动用,连 LoadImage、能看预览。 + B) reference_list 多行文本(每行一张图的 base64) —— 程序化/后端用。 + 图数 = 文本有几行,完全由数据决定,graph 结构恒定,无"空槽"、无占位图、无动态改线。 + 一个静态工作流即可适配任意张数(0..N),后端只往这个文本框填几行就喂几张。 实现只依赖 ComfyUI 公开的两个东西: 1. vae.encode(pixels) —— 与内置 VAEEncode 同款,把像素图编码成 latent; @@ -19,17 +25,12 @@ comfyui_multi_referencelatent — MultiReferenceLatent - 0 < strength < 1 : 弱化该图(近似,可能伴随轻微的内容呈现变化) - strength > 1 : 放大该图(谨慎,容易过曝/失真) -输入设计: - conditioning 是唯一必填项。vae 与全部 image 均为可选。 - - 一张参考图都不接 → conditioning 原样返回(此时节点是个直通,等于纯文生图); - - 接了 N 张有效图 → 逐张编码并追加,得到含 N 个 reference_latents 的 conditioning。 - 好处:同一个静态工作流,接几张就生效几张,不必为不同参考图数量另存工作流, - 也不必动用 bypass 或改连线。 +conditioning 是唯一必填项;一张参考图都没有时,节点原样返回(直通,等于纯文生图)。 """ import node_helpers -# 节点暴露的参考图槽位数量。要扩到更多,改这里即可,UI 与逻辑都按它生成。 +# image socket 的槽位数量(画布手动用)。reference_list 文本方式不受这个上限约束。 NUM_SLOTS = 6 @@ -49,6 +50,15 @@ class MultiReferenceLatent: f"<1 弱化、>1 放大均为近似,见节点说明。" ), }) + slots["reference_list"] = ("STRING", { + "multiline": True, + "default": "", + "tooltip": ( + "程序化/后端用:每行一张参考图,格式「strength base64」(strength 可省=1.0)," + "如「1.0 iVBOR...」或直接「iVBOR...」。图数=非空行数,无槽、无占位图。" + "可与上面的 image_N socket 同时用(socket 在前)。支持 data:image/...;base64, 前缀。" + ), + }) return { "required": { "conditioning": ("CONDITIONING",), @@ -70,7 +80,8 @@ class MultiReferenceLatent: FUNCTION = "inject_references" CATEGORY = "advanced/conditioning/edit_models" DESCRIPTION = ( - "把至多 6 张参考图挂进 conditioning(FLUX.2 dev/klein 等编辑模型)。" + "把多张参考图挂进 conditioning(FLUX.2 dev/klein 等编辑模型)。" + "两种喂图:image_N socket(画布)或 reference_list 多行 base64(程序化,图数=行数,无占位图)。" "每张图独立 strength,0=跳过;一张不接则直通(纯文生图)。内置 VAE 编码。" ) @@ -80,19 +91,68 @@ class MultiReferenceLatent: rgb = image[..., :3] return vae.encode(rgb) - def inject_references(self, conditioning, strength_mode="manual", vae=None, **slots): - # 先收集"有效"的参考图(接了图且 strength>0),按槽位序 1..N,暂不编码。 - # 推迟编码是为了让 normalize 能先看到全部权重、算完总和再缩放,只编码一次。 - active = [] # [(image, weight), ...] + @staticmethod + def _parse_list_line(line): + """ + 解析 reference_list 的一行 → (weight, base64) 或 None(空行)。 + 行格式:「strength base64」或「base64」。strength 缺省为 1.0。 + base64 字符集不含空格,所以用首个空白切一刀:第一段能转 float 就是 strength。 + """ + s = line.strip() + if not s: + return None + head, _, tail = s.partition(" ") + if tail: + try: + return (float(head), tail.strip()) + except ValueError: + pass # 第一段不是数 → 整行都是 base64 + return (1.0, s) + + @staticmethod + def _b64_to_image(b64): + """base64(可带 data URI 前缀)→ ComfyUI IMAGE 张量 [1, H, W, 3], float 0..1。""" + import base64 as _b64m + import io as _io + import numpy as _np + import torch as _torch + from PIL import Image as _Image + + s = b64.strip() + if s.startswith("data:"): + s = s.split(",", 1)[1] if "," in s else s + raw = _b64m.b64decode(s) + pil = _Image.open(_io.BytesIO(raw)).convert("RGB") + arr = _np.asarray(pil, dtype=_np.float32) / 255.0 + return _torch.from_numpy(arr)[None, ] + + def inject_references(self, conditioning, strength_mode="manual", + vae=None, reference_list="", **slots): + # 统一收集"有效"参考图为 (image_tensor, weight),先不编码。 + # 推迟编码:让 normalize 能先看到全部权重、算完总和再缩放,且只编码一次。 + active = [] # [(image_tensor, weight), ...] + + # 来源 A:image_N socket(画布手动),按槽位序 1..N。 for n in range(1, NUM_SLOTS + 1): image = slots.get(f"image_{n}") weight = slots.get(f"strength_{n}", 1.0) if image is None or weight <= 0.0: - continue # 没接图 / strength=0 → 跳过(两种模式下都成立) - if vae is None: + continue # 没接图 / strength=0 → 跳过 + active.append((image, weight)) + + # 来源 B:reference_list 多行 base64(程序化),图数 = 非空行数。 + for lineno, line in enumerate(reference_list.splitlines(), 1): + parsed = self._parse_list_line(line) + if parsed is None: + continue # 空行 + weight, b64 = parsed + if weight <= 0.0: + continue # strength=0 → 跳过该行 + try: + image = self._b64_to_image(b64) + except Exception as e: raise ValueError( - f"image_{n} 接了参考图,但 vae 输入是空的。" - f"请把 VAELoader 连到本节点的 vae。" + f"reference_list 第 {lineno} 行 base64 解码失败:{e}" ) active.append((image, weight)) @@ -100,6 +160,9 @@ class MultiReferenceLatent: if not active: return (conditioning,) + if vae is None: + raise ValueError("有参考图但 vae 输入是空的。请把 VAELoader 连到本节点的 vae。") + # normalize:按总和把各权重归一化到和=1,保留相对比例;全相等时即均分 1/N。 # 总和必为正(只有 weight>0 的图才进 active),不会除零。 if strength_mode == "normalize": diff --git a/pyproject.toml b/pyproject.toml index 022eb6d..99a2ca9 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,7 +1,7 @@ [project] name = "comfyui_multi_referencelatent" description = "Inject up to 6 reference images into conditioning for FLUX.2 (and other edit models) in a single node, each with its own strength (0 = skip). Built-in VAE encode." -version = "1.1.0" +version = "1.2.0" license = { file = "LICENSE" } readme = "README.md" requires-python = ">=3.9"