feat: reference_list input — data-driven image count, no placeholders (v1.2.0)
Optional multiline reference_list: one base64 image per line, "[strength ]<base64>". Image count = non-empty lines, so a single static graph adapts to any number of references with no empty slots, no placeholder images, no rewiring. Decodes base64->IMAGE internally. Works alongside image_N sockets; normalize/strength=0 apply across both. README updated, version 1.2.0.
This commit is contained in:
lynclee
@@ -2,13 +2,13 @@
|
||||
|
||||
> 中文 | [English](#english)
|
||||
|
||||
把**最多 6 张参考图**一次性注入 conditioning 的单节点,专为 **FLUX.2 dev / klein** 这类吃 `reference_latents` 的图像编辑模型设计。每张图有独立的 `strength`,**`strength = 0` 就跳过那张图** —— 于是一个静态工作流就能覆盖 0~6 张参考图,不用 bypass、不用动态改线、不用为不同图数准备多个工作流。
|
||||
把**最多 6 张参考图**一次性注入 conditioning 的单节点,专为 **FLUX.2 dev / klein** 这类吃 `reference_latents` 的图像编辑模型设计。每张图有独立的 `strength`,**`strength = 0` 就跳过那张图** —— 于是一个静态工作流就能覆盖 0~6 张参考图,不用 bypass、不用动态改线、不用为不同图数准备多个工作流。程序化提交还可用 `reference_list` 文本(图数=行数,无占位图)。
|
||||
|
||||
## 它解决什么
|
||||
|
||||
ComfyUI 原生 `ReferenceLatent` 一次只喂一张参考图,要多张就得手动把多个 `ReferenceLatent` 串起来,而且没有逐图强度控制。本节点:
|
||||
|
||||
- **一个节点喂多张**(≤6),内部按原生 `reference_latents` append 语义逐张拼接;
|
||||
- **一个节点喂多张**(≤6 socket,或 reference_list 不限),内部按原生 `reference_latents` append 语义逐张拼接;
|
||||
- **每张独立 strength**(范围 `0 ~ 5`),`0 = 跳过`,`1.0 = 原生强度`,`>1 增强`;
|
||||
- **内置 VAE 编码**,直接接 `IMAGE`,不用每张图外挂一个 `VAEEncode`;
|
||||
- **全 optional**:一张都不接 → conditioning 原样透传(等于纯文生图)。
|
||||
@@ -41,6 +41,23 @@ VAELoader ─►│ vae │
|
||||
|
||||
把 6 个图槽留着,想用几张就接几张(或者把不想要的那张 `strength` 调 0)。提交时不参与的槽不产生任何 latent,**云端/本地行为完全一致**,无需为不同图数维护多份工作流。
|
||||
|
||||
## 程序化喂图:`reference_list`(无占位图,图数=行数)
|
||||
|
||||
画布手动用 `image_N` socket;但**程序化/API 提交**时,socket 会留下"空槽"(要么占位图、要么动态改线)。为此节点提供另一条路 —— `reference_list` 多行文本:
|
||||
|
||||
```
|
||||
2.0 iVBORw0KGgoAAAANS... ← 第 1 张,strength=2.0
|
||||
iVBORw0KGgoAAAANS... ← 第 2 张,strength 省略=1.0
|
||||
0.5 data:image/png;base64,iVBOR... ← 第 3 张,支持 data URI 前缀
|
||||
```
|
||||
|
||||
- **每行一张图**,格式 `[strength ]<base64>`,strength 可省(=1.0);
|
||||
- **图数 = 非空行数**,完全由你填的文本决定 —— graph 结构恒定,**没有空槽、没有占位图、没有动态改线、没有多份 JSON**;
|
||||
- 空行忽略,`strength=0` 的行跳过;
|
||||
- 可与 `image_N` socket 同时用(socket 在前)。
|
||||
|
||||
后端只需把 N 张图拼成 N 行塞进这个字段,一个静态工作流适配任意张数。这就是"**静态的是 graph,动态的是数据**"。
|
||||
|
||||
## strength 怎么作用(以及它的局限)
|
||||
|
||||
`reference_latents` 在 conditioning 里只是一串 latent,模型对其中每个 latent **没有独立的权重输入**。所以在 conditioning 这一层,"调强度"唯一能做的就是缩放这张图自己的 latent 数值。这是**近似**,不是干净的"影响力旋钮"——缩放会顺带改变 latent 在 VAE 空间的分布:
|
||||
@@ -79,13 +96,13 @@ MIT。基于 ComfyUI 公开的原生 `ReferenceLatent` 机制独立实现。
|
||||
|
||||
> [中文](#comfyui-multi-reference-latent) | English
|
||||
|
||||
A single node that injects **up to 6 reference images** into conditioning, built for **FLUX.2 dev / klein** and other edit models that consume `reference_latents`. Each image has its own `strength`, and **`strength = 0` skips that image** — so one static workflow covers 0–6 references without bypass, rewiring, or multiple workflows.
|
||||
A single node that injects **up to 6 reference images** into conditioning, built for **FLUX.2 dev / klein** and other edit models that consume `reference_latents`. Each image has its own `strength`, and **`strength = 0` skips that image** — so one static workflow covers 0–6 references without bypass, rewiring, or multiple workflows. For programmatic submission there is also a `reference_list` text field (image count = number of lines, no placeholder images).
|
||||
|
||||
## What it solves
|
||||
|
||||
Native `ReferenceLatent` feeds one reference at a time and has no per-image strength. This node:
|
||||
|
||||
- **Feeds many in one node** (≤6), chaining them with the native `reference_latents` append semantics;
|
||||
- **Feeds many in one node** (≤6 sockets, or unlimited via reference_list), chaining them with the native `reference_latents` append semantics;
|
||||
- **Per-image strength** (range `0–5`): `0 = skip`, `1.0 = native`, `>1` stronger;
|
||||
- **Built-in VAE encode** — wire `IMAGE` directly, no per-image `VAEEncode`;
|
||||
- **All optional** — connect nothing and conditioning passes through unchanged (plain text-to-image).
|
||||
@@ -106,6 +123,23 @@ Native `ReferenceLatent` feeds one reference at a time and has no per-image stre
|
||||
|
||||
Keep all 6 slots; wire as many as you need (or set an unwanted image's `strength` to 0). Disabled slots produce no latent at all, so behavior is identical locally and on any backend — no need to maintain separate workflows per image count.
|
||||
|
||||
## Programmatic input: `reference_list` (no placeholder images)
|
||||
|
||||
`image_N` sockets are for the canvas; but for **programmatic / API submission**, sockets leave "empty slots" (needing placeholder images or graph rewiring). So the node offers another path — the `reference_list` multiline text:
|
||||
|
||||
```
|
||||
2.0 iVBORw0KGgoAAAANS... ← image 1, strength 2.0
|
||||
iVBORw0KGgoAAAANS... ← image 2, strength omitted = 1.0
|
||||
0.5 data:image/png;base64,iVBOR... ← image 3, data URI prefix supported
|
||||
```
|
||||
|
||||
- **one image per line**, format `[strength ]<base64>`, strength optional (=1.0);
|
||||
- **image count = number of non-empty lines**, fully data-driven — the graph structure is fixed: **no empty slots, no placeholder images, no rewiring, no multiple JSONs**;
|
||||
- blank lines ignored, lines with `strength=0` skipped;
|
||||
- works alongside `image_N` sockets (sockets first).
|
||||
|
||||
Your backend just joins N images into N lines and fills this field; one static workflow adapts to any count. The graph stays static, the data drives the image count.
|
||||
|
||||
## How strength works (and its limits)
|
||||
|
||||
`reference_latents` is just a list of latents in the conditioning; the model has **no per-latent weight input**. So at the conditioning level the only thing "strength" can do is scale that image's own latent values. This is an **approximation**, not a clean influence knob — scaling also shifts the latent's distribution in VAE space:
|
||||
|
||||
@@ -1,9 +1,15 @@
|
||||
"""
|
||||
comfyui_multi_referencelatent — MultiReferenceLatent
|
||||
|
||||
把若干张参考图(本节点上限 6)一次性挂进同一条 conditioning,供 FLUX.2 dev / klein
|
||||
把若干张参考图一次性挂进同一条 conditioning,供 FLUX.2 dev / klein
|
||||
等"读 reference_latents 的编辑模型"使用。等价于把多个原生 ReferenceLatent 串成一串,
|
||||
外加内置 VAE 编码与逐图开关,省掉重复连线。
|
||||
外加内置 VAE 编码与逐图强度,省掉重复连线。
|
||||
|
||||
两种喂图方式,可单用也可并用:
|
||||
A) image_1..6 这些 IMAGE socket —— 画布手动用,连 LoadImage、能看预览。
|
||||
B) reference_list 多行文本(每行一张图的 base64) —— 程序化/后端用。
|
||||
图数 = 文本有几行,完全由数据决定,graph 结构恒定,无"空槽"、无占位图、无动态改线。
|
||||
一个静态工作流即可适配任意张数(0..N),后端只往这个文本框填几行就喂几张。
|
||||
|
||||
实现只依赖 ComfyUI 公开的两个东西:
|
||||
1. vae.encode(pixels) —— 与内置 VAEEncode 同款,把像素图编码成 latent;
|
||||
@@ -19,17 +25,12 @@ comfyui_multi_referencelatent — MultiReferenceLatent
|
||||
- 0 < strength < 1 : 弱化该图(近似,可能伴随轻微的内容呈现变化)
|
||||
- strength > 1 : 放大该图(谨慎,容易过曝/失真)
|
||||
|
||||
输入设计:
|
||||
conditioning 是唯一必填项。vae 与全部 image 均为可选。
|
||||
- 一张参考图都不接 → conditioning 原样返回(此时节点是个直通,等于纯文生图);
|
||||
- 接了 N 张有效图 → 逐张编码并追加,得到含 N 个 reference_latents 的 conditioning。
|
||||
好处:同一个静态工作流,接几张就生效几张,不必为不同参考图数量另存工作流,
|
||||
也不必动用 bypass 或改连线。
|
||||
conditioning 是唯一必填项;一张参考图都没有时,节点原样返回(直通,等于纯文生图)。
|
||||
"""
|
||||
|
||||
import node_helpers
|
||||
|
||||
# 节点暴露的参考图槽位数量。要扩到更多,改这里即可,UI 与逻辑都按它生成。
|
||||
# image socket 的槽位数量(画布手动用)。reference_list 文本方式不受这个上限约束。
|
||||
NUM_SLOTS = 6
|
||||
|
||||
|
||||
@@ -49,6 +50,15 @@ class MultiReferenceLatent:
|
||||
f"<1 弱化、>1 放大均为近似,见节点说明。"
|
||||
),
|
||||
})
|
||||
slots["reference_list"] = ("STRING", {
|
||||
"multiline": True,
|
||||
"default": "",
|
||||
"tooltip": (
|
||||
"程序化/后端用:每行一张参考图,格式「strength base64」(strength 可省=1.0),"
|
||||
"如「1.0 iVBOR...」或直接「iVBOR...」。图数=非空行数,无槽、无占位图。"
|
||||
"可与上面的 image_N socket 同时用(socket 在前)。支持 data:image/...;base64, 前缀。"
|
||||
),
|
||||
})
|
||||
return {
|
||||
"required": {
|
||||
"conditioning": ("CONDITIONING",),
|
||||
@@ -70,7 +80,8 @@ class MultiReferenceLatent:
|
||||
FUNCTION = "inject_references"
|
||||
CATEGORY = "advanced/conditioning/edit_models"
|
||||
DESCRIPTION = (
|
||||
"把至多 6 张参考图挂进 conditioning(FLUX.2 dev/klein 等编辑模型)。"
|
||||
"把多张参考图挂进 conditioning(FLUX.2 dev/klein 等编辑模型)。"
|
||||
"两种喂图:image_N socket(画布)或 reference_list 多行 base64(程序化,图数=行数,无占位图)。"
|
||||
"每张图独立 strength,0=跳过;一张不接则直通(纯文生图)。内置 VAE 编码。"
|
||||
)
|
||||
|
||||
@@ -80,19 +91,68 @@ class MultiReferenceLatent:
|
||||
rgb = image[..., :3]
|
||||
return vae.encode(rgb)
|
||||
|
||||
def inject_references(self, conditioning, strength_mode="manual", vae=None, **slots):
|
||||
# 先收集"有效"的参考图(接了图且 strength>0),按槽位序 1..N,暂不编码。
|
||||
# 推迟编码是为了让 normalize 能先看到全部权重、算完总和再缩放,只编码一次。
|
||||
active = [] # [(image, weight), ...]
|
||||
@staticmethod
|
||||
def _parse_list_line(line):
|
||||
"""
|
||||
解析 reference_list 的一行 → (weight, base64) 或 None(空行)。
|
||||
行格式:「strength base64」或「base64」。strength 缺省为 1.0。
|
||||
base64 字符集不含空格,所以用首个空白切一刀:第一段能转 float 就是 strength。
|
||||
"""
|
||||
s = line.strip()
|
||||
if not s:
|
||||
return None
|
||||
head, _, tail = s.partition(" ")
|
||||
if tail:
|
||||
try:
|
||||
return (float(head), tail.strip())
|
||||
except ValueError:
|
||||
pass # 第一段不是数 → 整行都是 base64
|
||||
return (1.0, s)
|
||||
|
||||
@staticmethod
|
||||
def _b64_to_image(b64):
|
||||
"""base64(可带 data URI 前缀)→ ComfyUI IMAGE 张量 [1, H, W, 3], float 0..1。"""
|
||||
import base64 as _b64m
|
||||
import io as _io
|
||||
import numpy as _np
|
||||
import torch as _torch
|
||||
from PIL import Image as _Image
|
||||
|
||||
s = b64.strip()
|
||||
if s.startswith("data:"):
|
||||
s = s.split(",", 1)[1] if "," in s else s
|
||||
raw = _b64m.b64decode(s)
|
||||
pil = _Image.open(_io.BytesIO(raw)).convert("RGB")
|
||||
arr = _np.asarray(pil, dtype=_np.float32) / 255.0
|
||||
return _torch.from_numpy(arr)[None, ]
|
||||
|
||||
def inject_references(self, conditioning, strength_mode="manual",
|
||||
vae=None, reference_list="", **slots):
|
||||
# 统一收集"有效"参考图为 (image_tensor, weight),先不编码。
|
||||
# 推迟编码:让 normalize 能先看到全部权重、算完总和再缩放,且只编码一次。
|
||||
active = [] # [(image_tensor, weight), ...]
|
||||
|
||||
# 来源 A:image_N socket(画布手动),按槽位序 1..N。
|
||||
for n in range(1, NUM_SLOTS + 1):
|
||||
image = slots.get(f"image_{n}")
|
||||
weight = slots.get(f"strength_{n}", 1.0)
|
||||
if image is None or weight <= 0.0:
|
||||
continue # 没接图 / strength=0 → 跳过(两种模式下都成立)
|
||||
if vae is None:
|
||||
continue # 没接图 / strength=0 → 跳过
|
||||
active.append((image, weight))
|
||||
|
||||
# 来源 B:reference_list 多行 base64(程序化),图数 = 非空行数。
|
||||
for lineno, line in enumerate(reference_list.splitlines(), 1):
|
||||
parsed = self._parse_list_line(line)
|
||||
if parsed is None:
|
||||
continue # 空行
|
||||
weight, b64 = parsed
|
||||
if weight <= 0.0:
|
||||
continue # strength=0 → 跳过该行
|
||||
try:
|
||||
image = self._b64_to_image(b64)
|
||||
except Exception as e:
|
||||
raise ValueError(
|
||||
f"image_{n} 接了参考图,但 vae 输入是空的。"
|
||||
f"请把 VAELoader 连到本节点的 vae。"
|
||||
f"reference_list 第 {lineno} 行 base64 解码失败:{e}"
|
||||
)
|
||||
active.append((image, weight))
|
||||
|
||||
@@ -100,6 +160,9 @@ class MultiReferenceLatent:
|
||||
if not active:
|
||||
return (conditioning,)
|
||||
|
||||
if vae is None:
|
||||
raise ValueError("有参考图但 vae 输入是空的。请把 VAELoader 连到本节点的 vae。")
|
||||
|
||||
# normalize:按总和把各权重归一化到和=1,保留相对比例;全相等时即均分 1/N。
|
||||
# 总和必为正(只有 weight>0 的图才进 active),不会除零。
|
||||
if strength_mode == "normalize":
|
||||
|
||||
+1
-1
@@ -1,7 +1,7 @@
|
||||
[project]
|
||||
name = "comfyui_multi_referencelatent"
|
||||
description = "Inject up to 6 reference images into conditioning for FLUX.2 (and other edit models) in a single node, each with its own strength (0 = skip). Built-in VAE encode."
|
||||
version = "1.1.0"
|
||||
version = "1.2.0"
|
||||
license = { file = "LICENSE" }
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.9"
|
||||
|
||||
Reference in New Issue
Block a user