initial public release

This commit is contained in:
XinPeng76
2026-07-08 14:05:13 +08:00
commit 02a0b7082c
8 changed files with 1164 additions and 0 deletions
+10
View File
@@ -0,0 +1,10 @@
__pycache__/
*.py[cod]
*$py.class
.env
.venv/
venv/
.DS_Store
Thumbs.db
+155
View File
@@ -0,0 +1,155 @@
# ComfyUI_TJS - Truncated Jump Sampling
中文 | [English](README_EN.md)
ComfyUI_TJS 是一个用于 ComfyUI 的自定义节点插件,用来实验
**TJS(Truncated Jump Sampling,截断跳步采样)**。
该插件用于验证一种基于 endpoint / denoised 预测的训练无关采样加速思路。
## 功能简介
TJS 的核心思想是:不一定要把扩散/流匹配采样轨迹完整跑完。模型在中间步已经能给出一个
`x0` / `denoised` 端点预测,因此可以先运行一部分采样步骤,再直接输出端点估计,从而减少 NFE。
简化流程如下:
```text
k* = ceil(gamma * K)
run sampler from sigma[0] to sigma[k*]
return x0_hat(xt, sigma[k*]) with one endpoint model call
```
其中:
- `K` 是原始完整采样步数。
- `gamma` 是早退比例,例如 `0.6` 表示只跑大约 60% 的轨迹。
- `k*` 是实际早退步。
- `x0_hat` 是模型在中间 latent 上预测的干净端点。
在 ComfyUI 中,endpoint estimate 对应模型包装器给出的原生 `denoised` latent,因此理论上可以兼容 diffusion、flow matching、Flux/SD3 类模型包装。
当 `gamma = 1.0` 时,节点会运行完整采样 schedule,并跳过额外 endpoint call,等价于普通 KSampler 的边界情况。
## 安装
把本文件夹复制到 ComfyUI 的自定义节点目录:
```text
ComfyUI/custom_nodes/ComfyUI_TJS/
```
然后重启 ComfyUI。
## 节点说明
### TJS Sampler (Truncated Jump Sampling)
这是主要可用节点,可以作为普通文生图采样器的替代节点使用。
输入:
| Input | 中文说明 |
|---|---|
| `model` | ComfyUI 加载的模型 |
| `total_steps` | 原始完整采样步数 `K`,例如 `30` |
| `early_exit_gamma` | 早退比例,例如 `0.6` |
| `cfg` | CFG scale |
| `sampler_name` | 截断采样阶段使用的 ComfyUI sampler |
| `scheduler` | ComfyUI sigma scheduler |
| `positive` / `negative` | 正向/反向条件 |
| `latent_image` | 空 latent 或输入 latent |
| `seed` | 随机种子 |
| `model_type` | 信息标记:`auto`、`diffusion` 或 `flow` |
| `denoise` | 可选 denoise 强度 |
输出:
| Output | 中文说明 |
|---|---|
| `latent_x0` | endpoint decoded latent,接到 VAE Decode |
| `latent_xt` | 早退时刻的中间 noisy latent |
| `k_star` | 实际早退步 |
| `nfe_used` | 早退时为 `k_star + 1`;`gamma = 1.0` 时为 `K` |
| `nfe_saving_pct` | 相对完整步数节省的 NFE 百分比 |
| `sigma_at_exit` | endpoint decode 使用的 sigma |
### TJS Decode (Endpoint / Advanced KSampler)
状态:实验性 / TODO。
这个节点目前还没有完全写对,暂时不要把它当成可靠正式功能。它的目标用法是配合
KSampler Advanced:先让 KSampler Advanced 跑到某个中间 `end_at_step`,保留 leftover noisy latent,
再用 TJS Decode 把这个中间状态转换成 endpoint `x0` 预测。
后续需要继续修复和验证的问题:
- 如何严格对齐 KSampler Advanced 的中间 latent 与对应 sigma。
- 如何保证不同 sampler / scheduler 下 endpoint decode 的输入缩放完全正确。
- 如何在工作流里可靠传递中间步信息,而不是让用户手动猜连续时间 `t`。
当前节点设计为:用户不需要知道连续时间 `t` 或 sigma,只需要填入和 KSampler Advanced 相同的
`steps`、`sampler_name`、`scheduler`、`end_at_step`,节点内部尝试重建:
```text
sigma* = sigmas[end_at_step]
```
注意:在 KSampler Advanced 中,需要设置:
```text
return_with_leftover_noise = enable
```
如果关闭 leftover noise,高级采样器会强制采到 sigma=0,输出已经是完整去噪 latent,此时就没有可供 TJS decode 的中间 noisy endpoint state。
### TJS Decode (Manual Sigma)
调试节点。只有当你已经明确知道某个中间 latent 对应的精确 sigma 时才建议使用。
普通 Advanced KSampler 工作流不要优先用这个节点。
## 使用示例
### 直接使用 TJS Sampler
例如设置:
```text
total_steps = 30
early_exit_gamma = 0.6
```
节点会运行到:
```text
k* = ceil(0.6 * 30) = 18
```
然后执行一次 endpoint call。总成本约为:
```text
NFE = 18 + 1 = 19
```
相对 30 步完整采样,约节省 `36.7%` NFE。
### 计划中的 Advanced KSampler 用法
这部分仍是 TODO,当前 TJS Decode 节点还需要修复。
目标工作流如下:
1. 使用 KSampler Advanced。
2. 设置 `steps = 30`,`end_at_step = 18`。
3. 设置 `return_with_leftover_noise = enable`。
4. 把 KSampler Advanced 输出的中间 latent 接到 `TJS Decode (Endpoint / Advanced KSampler)`。
5. TJS Decode 使用相同的 `steps`、`end_at_step`、`sampler_name`、`scheduler`。
6. 把 `latent_x0` 接到 VAE Decode。
## 当前限制
- `TJS Sampler` 是当前主要实现。
- `TJS Decode (Endpoint / Advanced KSampler)` 仍是实验节点,需要后续修复。
- 不同模型、sampler、scheduler 对 latent 缩放和 denoised 语义可能不同,实际效果需要逐模型验证。
- 当前插件更适合作为算法验证和 ComfyUI 原型实验,不建议直接用于严肃生产流程。
+171
View File
@@ -0,0 +1,171 @@
# ComfyUI_TJS - Truncated Jump Sampling
[中文](README.md) | English
ComfyUI_TJS is a custom node plugin for ComfyUI, built to experiment with
**TJS (Truncated Jump Sampling)**.
This plugin explores a training-free sampling acceleration idea based on
endpoint / denoised prediction.
## What It Does
TJS is based on a simple idea: diffusion and flow-matching samplers may not
need to traverse the full trajectory. At an intermediate step, the model
already produces an `x0` / `denoised` endpoint estimate. TJS runs only part of
the sampling trajectory, then uses that endpoint prediction as the final latent.
Simplified algorithm:
```text
k* = ceil(gamma * K)
run sampler from sigma[0] to sigma[k*]
return x0_hat(xt, sigma[k*]) with one endpoint model call
```
Where:
- `K` is the full sampling step budget.
- `gamma` is the early-exit ratio, for example `0.6`.
- `k*` is the actual early-exit step.
- `x0_hat` is the clean endpoint estimate predicted from the intermediate latent.
In ComfyUI, this endpoint estimate corresponds to the model wrapper's native
`denoised` latent. In principle, this makes the approach usable with diffusion,
flow-matching, Flux/SD3-style model wrappers, and related models.
When `gamma = 1.0`, the node runs the full sampling schedule and skips the
extra endpoint call, matching the ordinary KSampler boundary case.
## Installation
Copy this folder into ComfyUI's custom node directory:
```text
ComfyUI/custom_nodes/ComfyUI_TJS/
```
Then restart ComfyUI.
## Nodes
### TJS Sampler (Truncated Jump Sampling)
This is the main usable node. It can be used as an experimental replacement for
a normal text-to-image sampler.
Inputs:
| Input | Description |
|---|---|
| `model` | Loaded ComfyUI model |
| `total_steps` | Full sampling step budget `K`, for example `30` |
| `early_exit_gamma` | Early-exit ratio, for example `0.6` |
| `cfg` | CFG scale |
| `sampler_name` | ComfyUI sampler used for the truncated trajectory |
| `scheduler` | ComfyUI sigma scheduler |
| `positive` / `negative` | Positive / negative conditioning |
| `latent_image` | Empty latent or input latent |
| `seed` | Random seed |
| `model_type` | Informational selector: `auto`, `diffusion`, or `flow` |
| `denoise` | Optional denoise strength |
Outputs:
| Output | Description |
|---|---|
| `latent_x0` | Endpoint decoded latent; connect this to VAE Decode |
| `latent_xt` | Intermediate noisy latent at the early-exit step |
| `k_star` | Actual early-exit step |
| `nfe_used` | `k_star + 1` for early exit, or `K` when `gamma = 1.0` |
| `nfe_saving_pct` | NFE saving percentage relative to full sampling |
| `sigma_at_exit` | Sigma used by the endpoint decode |
### TJS Decode (Endpoint / Advanced KSampler)
Status: experimental / TODO.
This node is not fully correct yet. Do not treat it as a reliable production
feature. The intended future workflow is to pair it with KSampler Advanced:
run KSampler Advanced to an intermediate `end_at_step`, keep the leftover noisy
latent, then use TJS Decode to convert that intermediate state into an endpoint
`x0` prediction.
Open issues to fix:
- Strictly align the KSampler Advanced intermediate latent with the correct sigma.
- Ensure the input scaling for endpoint decode is correct across samplers and schedulers.
- Pass intermediate step information reliably through the workflow instead of asking users to guess continuous time `t`.
The current node attempts to infer:
```text
sigma* = sigmas[end_at_step]
```
from the same `steps`, `sampler_name`, `scheduler`, and `end_at_step` used by
KSampler Advanced.
Important: in KSampler Advanced, use:
```text
return_with_leftover_noise = enable
```
If leftover noise is disabled, KSampler Advanced forces the final sigma to zero,
so the output is already fully denoised and there is no intermediate noisy state
left for TJS Decode.
### TJS Decode (Manual Sigma)
Debug-only node. Use it only when you already know the exact sigma for an
intermediate latent.
For normal Advanced KSampler workflows, prefer the planned
`TJS Decode (Endpoint / Advanced KSampler)` workflow after it is fixed.
## Example
### Direct TJS Sampler Usage
Example settings:
```text
total_steps = 30
early_exit_gamma = 0.6
```
The node runs to:
```text
k* = ceil(0.6 * 30) = 18
```
Then it performs one endpoint call:
```text
NFE = 18 + 1 = 19
```
Compared with full 30-step sampling, this saves about `36.7%` NFE.
### Planned Advanced KSampler Workflow
This part is still TODO because the current TJS Decode node needs further
repair and validation.
Target workflow:
1. Use KSampler Advanced.
2. Set `steps = 30` and `end_at_step = 18`.
3. Set `return_with_leftover_noise = enable`.
4. Feed the intermediate latent into `TJS Decode (Endpoint / Advanced KSampler)`.
5. Use the same `steps`, `end_at_step`, `sampler_name`, and `scheduler` in TJS Decode.
6. Connect `latent_x0` to VAE Decode.
## Current Limitations
- `TJS Sampler` is the main implemented path.
- `TJS Decode (Endpoint / Advanced KSampler)` is still experimental and needs future repair.
- Latent scaling and `denoised` semantics may differ across models, samplers, and schedulers.
- This plugin is intended for research prototyping and ComfyUI experiments, not production workflows.
+310
View File
File diff suppressed because one or more lines are too long
+1
View File
File diff suppressed because one or more lines are too long
+5
View File
@@ -0,0 +1,5 @@
"""ComfyUI_TJS custom node package."""
from .tjs_nodes import NODE_CLASS_MAPPINGS, NODE_DISPLAY_NAME_MAPPINGS
__all__ = ["NODE_CLASS_MAPPINGS", "NODE_DISPLAY_NAME_MAPPINGS"]
+31
View File
@@ -0,0 +1,31 @@
"""Compatibility wrapper for older imports.
ComfyUI loads this custom node through __init__.py, which imports tjs_nodes.
This file re-exports the same mappings in case a local workflow imports
ComfyUI_TJS.nodes directly.
"""
try:
from .tjs_nodes import (
NODE_CLASS_MAPPINGS,
NODE_DISPLAY_NAME_MAPPINGS,
TJSDecode,
TJSDecodeManualSigma,
TJSSampler,
)
except ImportError:
from tjs_nodes import ( # type: ignore
NODE_CLASS_MAPPINGS,
NODE_DISPLAY_NAME_MAPPINGS,
TJSDecode,
TJSDecodeManualSigma,
TJSSampler,
)
__all__ = [
"NODE_CLASS_MAPPINGS",
"NODE_DISPLAY_NAME_MAPPINGS",
"TJSDecode",
"TJSDecodeManualSigma",
"TJSSampler",
]
+481
View File
@@ -0,0 +1,481 @@
"""
ComfyUI nodes for Truncated Jump Sampling (TJS).
TJS runs the ordinary sampler for k* early steps, then asks the model for the
endpoint prediction at the early-exit state. In ComfyUI/k-diffusion terms that
endpoint prediction is the model's `denoised` latent at sigma*.
"""
import math
import torch
import comfy.model_management
import comfy.sample
import comfy.samplers
try:
import comfy.utils
except Exception: # pragma: no cover - older ComfyUI builds may differ.
pass
import latent_preview
def _progress_enabled():
utils = getattr(comfy, "utils", None)
return bool(getattr(utils, "PROGRESS_BAR_ENABLED", True))
def _sampler_object(name):
if hasattr(comfy.samplers, "sampler_object"):
return comfy.samplers.sampler_object(name)
return comfy.samplers.ksampler(name)
def _calculate_sigmas(model, steps, sampler_name, scheduler, denoise):
device = comfy.model_management.get_torch_device()
if hasattr(comfy.samplers, "KSampler"):
sampler = comfy.samplers.KSampler(
model,
steps,
device,
sampler=sampler_name,
scheduler=scheduler,
denoise=denoise,
)
return sampler.sigmas.to(device)
from comfy_extras.nodes_custom_sampler import BasicScheduler
return BasicScheduler().get_sigmas(model, scheduler, steps, denoise)[0].to(device)
def _fix_latent(model, latent):
fixed = latent.copy()
fixed["samples"] = comfy.sample.fix_empty_latent_channels(model, latent["samples"])
return fixed
def _prepare_noise(latent, seed):
samples = latent["samples"]
batch_inds = latent.get("batch_index")
if hasattr(comfy.sample, "prepare_noise"):
return comfy.sample.prepare_noise(samples, seed, batch_inds)
generator = torch.Generator("cpu").manual_seed(seed)
return torch.randn(
samples.size(),
dtype=samples.dtype,
layout=samples.layout,
generator=generator,
device="cpu",
)
def _zeros_like_noise(latent_samples):
return torch.zeros(
latent_samples.size(),
dtype=latent_samples.dtype,
layout=latent_samples.layout,
device="cpu",
)
def _sample_custom(
model,
noise,
cfg,
sampler,
sigmas,
positive,
negative,
latent_image,
noise_mask=None,
callback=None,
seed=0,
):
disable_pbar = not _progress_enabled()
if hasattr(comfy.sample, "sample_custom"):
try:
return comfy.sample.sample_custom(
model,
noise,
cfg,
sampler,
sigmas,
positive,
negative,
latent_image,
noise_mask=noise_mask,
callback=callback,
disable_pbar=disable_pbar,
seed=seed,
)
except TypeError as exc:
if "noise_mask" not in str(exc):
raise
return comfy.sample.sample_custom(
model,
noise,
cfg,
sampler,
sigmas,
positive,
negative,
latent_image,
denoise_mask=noise_mask,
callback=callback,
disable_pbar=disable_pbar,
seed=seed,
)
device = comfy.model_management.get_torch_device()
return comfy.samplers.sample(
model,
noise,
positive,
negative,
cfg,
device,
sampler,
sigmas,
latent_image=latent_image,
denoise_mask=noise_mask,
callback=callback,
disable_pbar=disable_pbar,
seed=seed,
)
def _callback_x0_to_latent(model, x0_output, fallback_samples):
if "x0" not in x0_output:
return fallback_samples
return model.model.process_latent_out(x0_output["x0"].detach().cpu())
def _endpoint_decode(model, latent_xt, sigma, cfg, positive, negative, seed=0):
"""One model evaluation at sigma, returning ComfyUI's denoised endpoint."""
sigma = float(sigma)
if sigma <= 0.0:
return latent_xt
device = comfy.model_management.get_torch_device()
sigmas = torch.tensor([sigma, 0.0], dtype=torch.float32, device=device)
sampler = _sampler_object("euler")
noise = _zeros_like_noise(latent_xt)
x0_output = {}
callback = latent_preview.prepare_callback(model, 1, x0_output)
samples = _sample_custom(
model,
noise,
cfg,
sampler,
sigmas,
positive,
negative,
latent_xt,
noise_mask=None,
callback=callback,
seed=seed,
)
return _callback_x0_to_latent(model, x0_output, samples)
class TJSSampler:
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"model": ("MODEL",),
"total_steps": ("INT", {"default": 30, "min": 2, "max": 10000}),
"early_exit_gamma": (
"FLOAT",
{
"default": 0.6,
"min": 0.05,
"max": 1.0,
"step": 0.05,
"display": "slider",
},
),
"cfg": (
"FLOAT",
{"default": 7.5, "min": 0.0, "max": 100.0, "step": 0.1},
),
"sampler_name": (comfy.samplers.KSampler.SAMPLERS,),
"scheduler": (comfy.samplers.KSampler.SCHEDULERS,),
"positive": ("CONDITIONING",),
"negative": ("CONDITIONING",),
"latent_image": ("LATENT",),
"seed": (
"INT",
{"default": 0, "min": 0, "max": 0xffffffffffffffff},
),
"model_type": (
["auto", "diffusion", "flow"],
{"default": "auto"},
),
},
"optional": {
"denoise": (
"FLOAT",
{"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.01},
),
},
}
RETURN_TYPES = ("LATENT", "LATENT", "INT", "INT", "FLOAT", "FLOAT")
RETURN_NAMES = (
"latent_x0",
"latent_xt",
"k_star",
"nfe_used",
"nfe_saving_pct",
"sigma_at_exit",
)
FUNCTION = "sample"
CATEGORY = "sampling/TJS"
def sample(
self,
model,
total_steps,
early_exit_gamma,
cfg,
sampler_name,
scheduler,
positive,
negative,
latent_image,
seed,
model_type="auto",
denoise=1.0,
):
latent = _fix_latent(model, latent_image)
latent_samples = latent["samples"]
noise_mask = latent.get("noise_mask")
sigmas = _calculate_sigmas(model, total_steps, sampler_name, scheduler, denoise)
actual_steps = max(1, int(sigmas.shape[-1]) - 1)
gamma = float(early_exit_gamma)
k_star = max(1, math.ceil(gamma * total_steps))
k_star = min(k_star, actual_steps)
sigma_star = float(sigmas[k_star].detach().cpu())
trunc_sigmas = sigmas[: k_star + 1].clone()
noise = _prepare_noise(latent, seed)
sampler = _sampler_object(sampler_name)
callback = latent_preview.prepare_callback(model, k_star)
latent_xt = _sample_custom(
model,
noise,
cfg,
sampler,
trunc_sigmas,
positive,
negative,
latent_samples,
noise_mask=noise_mask,
callback=callback,
seed=seed,
)
if sigma_star <= 0.0 or k_star >= actual_steps:
latent_x0 = latent_xt
nfe_used = k_star
else:
latent_x0 = _endpoint_decode(
model,
latent_xt,
sigma_star,
cfg,
positive,
negative,
seed=seed,
)
nfe_used = k_star + 1
nfe_saving = max(0.0, (1.0 - nfe_used / float(total_steps)) * 100.0)
out_x0 = latent.copy()
out_x0["samples"] = latent_x0
out_xt = latent.copy()
out_xt["samples"] = latent_xt
print(
"[ComfyUI_TJS] "
f"K={total_steps} gamma={early_exit_gamma:.3f} "
f"k*={k_star} sigma*={sigma_star:.6g} "
f"NFE={nfe_used} saving={nfe_saving:.1f}% mode={model_type}"
)
return (out_x0, out_xt, k_star, nfe_used, round(nfe_saving, 1), sigma_star)
class TJSDecode:
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"model": ("MODEL",),
"latent": ("LATENT",),
"steps": ("INT", {"default": 30, "min": 1, "max": 10000}),
"end_at_step": (
"INT",
{
"default": 18,
"min": 0,
"max": 10000,
"tooltip": "Use the same end_at_step as KSampler Advanced.",
},
),
"cfg": (
"FLOAT",
{"default": 7.5, "min": 0.0, "max": 100.0, "step": 0.1},
),
"sampler_name": (comfy.samplers.KSampler.SAMPLERS,),
"scheduler": (comfy.samplers.KSampler.SCHEDULERS,),
"positive": ("CONDITIONING",),
"negative": ("CONDITIONING",),
"model_type": (
["auto", "diffusion", "flow"],
{"default": "auto"},
),
},
"optional": {
"seed": (
"INT",
{"default": 0, "min": 0, "max": 0xffffffffffffffff},
),
"denoise": (
"FLOAT",
{"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.01},
),
},
}
RETURN_TYPES = ("LATENT", "FLOAT", "INT")
RETURN_NAMES = ("latent_x0", "sigma_at_decode", "decode_step")
FUNCTION = "decode"
CATEGORY = "sampling/TJS"
def decode(
self,
model,
latent,
steps,
end_at_step,
cfg,
sampler_name,
scheduler,
positive,
negative,
model_type="auto",
seed=0,
denoise=1.0,
):
fixed = _fix_latent(model, latent)
sigmas = _calculate_sigmas(model, steps, sampler_name, scheduler, denoise)
actual_steps = max(1, int(sigmas.shape[-1]) - 1)
decode_step = max(0, min(int(end_at_step), actual_steps))
sigma = float(sigmas[decode_step].detach().cpu())
latent_x0 = _endpoint_decode(
model,
fixed["samples"],
sigma,
cfg,
positive,
negative,
seed=seed,
)
out = fixed.copy()
out["samples"] = latent_x0
print(
"[ComfyUI_TJS] advanced decode "
f"steps={steps} end_at_step={end_at_step} "
f"decode_step={decode_step} sigma={sigma:.6g} mode={model_type}"
)
return (out, sigma, decode_step)
class TJSDecodeManualSigma:
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"model": ("MODEL",),
"latent": ("LATENT",),
"sigma": (
"FLOAT",
{"default": 1.0, "min": 0.0, "max": 1000.0, "step": 0.01},
),
"cfg": (
"FLOAT",
{"default": 7.5, "min": 0.0, "max": 100.0, "step": 0.1},
),
"positive": ("CONDITIONING",),
"negative": ("CONDITIONING",),
"model_type": (
["auto", "diffusion", "flow"],
{"default": "auto"},
),
},
"optional": {
"seed": (
"INT",
{"default": 0, "min": 0, "max": 0xffffffffffffffff},
),
},
}
RETURN_TYPES = ("LATENT",)
RETURN_NAMES = ("latent_x0",)
FUNCTION = "decode"
CATEGORY = "sampling/TJS"
def decode(
self,
model,
latent,
sigma,
cfg,
positive,
negative,
model_type="auto",
seed=0,
):
fixed = _fix_latent(model, latent)
latent_x0 = _endpoint_decode(
model,
fixed["samples"],
sigma,
cfg,
positive,
negative,
seed=seed,
)
out = fixed.copy()
out["samples"] = latent_x0
print(
f"[ComfyUI_TJS] manual decode sigma={float(sigma):.6g} "
f"mode={model_type}"
)
return (out,)
NODE_CLASS_MAPPINGS = {
"TJSSampler": TJSSampler,
"TJSDecode": TJSDecode,
"TJSDecodeManualSigma": TJSDecodeManualSigma,
}
NODE_DISPLAY_NAME_MAPPINGS = {
"TJSSampler": "TJS Sampler (Truncated Jump Sampling)",
"TJSDecode": "TJS Decode (Endpoint / Advanced KSampler)",
"TJSDecodeManualSigma": "TJS Decode (Manual Sigma)",
}