nodes/olivv-cs-ComfyUI-FunPack
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Add audio-safe motion pulses to hybrid sampler

Browse Source
This commit is contained in:
digital-garbage
2026-05-06 12:21:07 +03:00
parent 7e12c64c52
commit b414652f99
2 changed files with 231 additions and 162 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/FunPackHybridEuler2SSampler.md
+32 -20
View File
@@ -11,13 +11,9 @@ It is designed as a quality/speed compromise between:
This sampler keeps classic Euler ancestral for the early "structure building" stage where motion, anatomy and main composition are forming. On the late denoise steps it switches to a deterministic Euler / DPM-Solver++(2S) ODE refinement path where fine detail and cleanup usually matter most.
It can also build an optional paper-style Restart schedule at the progress point you choose:
It can also apply optional early/mid **motion transition pulses** for single-clip image-to-video workflows. These pulses add monotonic noise kicks at selected normal denoise steps instead of inserting upward sigma jumps. This is intended to push LTX2.3 away from stale frame-1 reference behavior while avoiding the audio damage caused by Restart replay.
- detect a restart anchor across the full sigma schedule
- insert upward sigma jumps back to a higher sigma
- replay the chosen sigma interval one or more times
Unlike the older implementation, Restart is now encoded directly into the outgoing `SIGMAS` schedule, so the visible schedule passed into `SamplerCustomAdvanced` is the actual schedule that gets executed.
Restart replay is disabled. Legacy restart widgets may still appear for workflow compatibility, but they are ignored.
## Recommended wiring
@@ -43,15 +39,30 @@ Example: `0.35` means only the last 35% of steps use the ODE refinement path.
- `0.0` = pure late-step Euler ODE
- `1.0` = full late-step DPM++(2S)-style correction
**restart_steps**: Target size of the restarted sigma interval. The node interprets this against the actual sigma curve and expands the outgoing `SIGMAS` schedule accordingly.
**restart_steps**: Legacy compatibility control. Ignored.
**restart_repeats**: How many Restart loops to run. `0` disables Restart.
**restart_repeats**: Legacy compatibility control. Ignored.
**restart_trigger_pct**: Sampling progress point where Restart is triggered across the full schedule. This is independent from `high_quality_pct`, so values such as `0.30` can trigger an early restart instead of being clamped to the Euler-to-2S transition point.
**restart_trigger_pct**: Legacy compatibility control. Ignored.
**restart_noise**: Noise strength used when re-noising the latent up to the restart interval's higher sigma.
**restart_noise**: Legacy compatibility control. Ignored.
**sigmas**: Optional incoming sigma schedule. If connected, the node outputs a restart-expanded sigma schedule that matches the sampler settings. If not connected, the sampler output still works, but there is no schedule for the node to rewrite.
**transition_mode**: Motion pulse preset:
- `off`: preserve legacy sampler behavior.
- `balanced`: one moderate early/mid pulse.
- `aggressive`: at least two stronger early/mid pulses for stale image-to-video generations.
- `custom`: use the transition count, spacing, and strength exactly as configured.
**transition_start_pct**: Sampling progress point where the first motion pulse is applied.
**transition_count**: Number of requested early/mid motion pulses. Pulses that would land in the late quality phase are skipped.
**transition_spacing_pct**: Progress spacing between motion pulses.
**transition_strength**: Strength of the monotonic noise kick. Higher values push harder against stale image references, with more drift risk.
**sigmas**: Optional incoming sigma schedule. If connected, the node returns the same monotonic schedule plus sampler-side metadata for motion pulses. If not connected, the sampler computes pulse positions from the runtime schedule.
## Recommended starting values
@@ -59,18 +70,19 @@ Example: `0.35` means only the last 35% of steps use the ODE refinement path.
- `s_noise = 1.0`
- `high_quality_pct = 0.30` to `0.40`
- `correction_blend = 1.0`
- `restart_repeats = 0` for baseline testing
- `transition_mode = off` for baseline testing
For a first Restart test, try:
For an aggressive LTX2.3 image-to-video motion test, try:
- `high_quality_pct = 0.35`
- `correction_blend = 1.0`
- `restart_steps = 3`
- `restart_repeats = 1`
- `restart_trigger_pct = 0.85`
- `restart_noise = 1.0`
- `transition_mode = aggressive`
- `transition_start_pct = 0.30`
- `transition_count = 2`
- `transition_spacing_pct = 0.22`
- `transition_strength = 0.85`
Example: with `8` base sampling steps, `restart_steps = 3`, and `restart_repeats = 1`, a normal evenly spaced schedule will usually expand into a schedule with about `3` extra restart transitions. With custom sigmas, the exact replay step count can shift slightly because Restart is anchored to the real sigma values rather than raw step indices.
Keep the prompt/refiner conditioning explicit about both action and camera change, for example `orbiting camera`, `dolly in`, `zoom out`, `new side angle`, `turning`, or `dynamic pose change`.
## Expected behavior
@@ -81,9 +93,9 @@ Compared to plain `euler_ancestral`, this sampler should usually:
- keep the early motion/anatomy formation more lively
- cost less than running a heavier deterministic solver for the whole schedule
With Restart enabled, it may further improve late cleanup and detail consistency, but it will increase runtime and may be too aggressive for some video workflows if overused.
With motion pulses enabled, it should more strongly encourage action, camera movement, and viewpoint changes inside a single clip. Stronger settings can increase subject drift or visual instability.
Because Restart is now represented in the outgoing `SIGMAS`, the total number of executed transitions should match what the expanded schedule shows in the downstream custom sampler.
The outgoing `SIGMAS` remain monotonic. Motion pulses happen inside the sampler at selected denoise steps rather than by expanding the schedule.
## Limitation
samplers.py
+199 -142
View File
@@ -8,6 +8,9 @@ import comfy.samplers
import comfy.utils
TRANSITION_MODES = ["off", "balanced", "aggressive", "custom"]
def _sigma_fn(t):
return t.neg().exp()
@@ -45,148 +48,161 @@ def _hybrid_ode_step(model, x, sigma, sigma_next, s_in, extra_args, correction_b
return x_euler.lerp(x_2s, correction_blend), denoised
def _renoise_to_sigma(x, current_sigma, target_sigma, restart_noise, noise_sampler):
if target_sigma <= current_sigma or restart_noise <= 0.0:
def _apply_motion_pulse(x, sigma, sigma_next, pulse_noise, noise_sampler):
if pulse_noise <= 0.0 or sigma_next >= sigma:
return x
sigma_delta_sq = max(0.0, float(target_sigma * target_sigma - current_sigma * current_sigma))
sigma_delta_sq = max(0.0, float(sigma * sigma - sigma_next * sigma_next))
if sigma_delta_sq <= 0.0:
return x
sigma_delta = math.sqrt(sigma_delta_sq)
return x + noise_sampler(current_sigma, target_sigma) * (restart_noise * sigma_delta)
return x + noise_sampler(sigma, sigma_next) * (pulse_noise * sigma_delta)
def _find_restart_anchor_index(sigmas, total_steps, restart_trigger_pct):
schedule_sigmas = sigmas[:total_steps]
if schedule_sigmas.numel() <= 1:
def _find_schedule_anchor_index(sigmas, total_steps, schedule_progress):
if sigmas is None or total_steps <= 1:
return 0
high_sigma = float(schedule_sigmas[0].item())
low_sigma = None
for idx in range(int(schedule_sigmas.shape[0]) - 1, -1, -1):
value = float(schedule_sigmas[idx].item())
if value > 0.0:
low_sigma = value
break
if low_sigma is None:
return total_steps - 1
schedule_progress = max(0.0, min(1.0, restart_trigger_pct))
if high_sigma <= 0.0 or low_sigma <= 0.0 or abs(high_sigma - low_sigma) <= 1e-12:
return min(total_steps - 1, max(0, int(round(schedule_progress * max(0, total_steps - 1)))))
log_high = math.log(high_sigma)
log_low = math.log(low_sigma)
target_log_sigma = log_high + (log_low - log_high) * schedule_progress
best_index = 0
best_distance = None
for idx in range(total_steps):
sigma_value = float(sigmas[idx].item())
if sigma_value <= 0.0:
continue
distance = abs(math.log(sigma_value) - target_log_sigma)
if best_distance is None or distance < best_distance:
best_distance = distance
best_index = idx
return best_index
schedule_progress = max(0.0, min(1.0, schedule_progress))
return min(total_steps - 1, max(0, int(round(schedule_progress * max(0, total_steps - 1)))))
def _build_restart_sigmas(sigmas, total_steps, restart_trigger_pct, restart_steps):
if total_steps <= 0:
return sigmas[:0], 0
def _resolve_transition_options(transition_mode, transition_start_pct,
transition_count, transition_spacing_pct,
transition_strength):
mode = (transition_mode or "off").lower()
if mode not in TRANSITION_MODES:
mode = "off"
anchor_index = _find_restart_anchor_index(sigmas, total_steps, restart_trigger_pct)
anchor_index = min(total_steps - 1, max(0, anchor_index))
if mode == "off":
return {
"enabled": False,
"start_pct": 0.30,
"count": 0,
"spacing_pct": 0.22,
"strength": 0.0,
"noise": 0.0,
}
if restart_steps <= 1:
return sigmas[anchor_index:anchor_index + 1], anchor_index
start_pct = 0.30 if transition_start_pct is None else float(transition_start_pct)
spacing_pct = 0.22 if transition_spacing_pct is None else float(transition_spacing_pct)
strength = 0.85 if transition_strength is None else float(transition_strength)
count = 2 if transition_count is None else int(transition_count)
schedule_sigmas = sigmas[:total_steps]
if schedule_sigmas.numel() <= 1:
return sigmas[anchor_index:anchor_index + 1], anchor_index
start_pct = max(0.02, min(0.90, start_pct))
spacing_pct = max(0.04, min(0.45, spacing_pct))
strength = max(0.0, min(1.0, strength))
count = max(1, min(6, count))
positive_logs = []
for idx in range(int(schedule_sigmas.shape[0])):
sigma_value = float(schedule_sigmas[idx].item())
if sigma_value > 0.0:
positive_logs.append(math.log(sigma_value))
if mode == "balanced":
count = min(count, 1)
strength = 0.55 if transition_strength is None else min(strength, 0.70)
elif mode == "aggressive":
count = max(2, count)
strength = max(strength, 0.85)
if len(positive_logs) >= 2:
total_log_span = abs(positive_logs[0] - positive_logs[-1])
avg_log_step = total_log_span / max(1, len(positive_logs) - 1)
target_log_span = avg_log_step * max(1, restart_steps - 1)
else:
target_log_span = 0.0
noise = 0.10 + strength * 0.55
restart_from = anchor_index
accumulated_span = 0.0
for idx in range(anchor_index, 0, -1):
sigma_hi = float(sigmas[idx - 1].item())
sigma_lo = float(sigmas[idx].item())
restart_from = idx - 1
if sigma_hi > 0.0 and sigma_lo > 0.0:
accumulated_span += abs(math.log(sigma_hi) - math.log(sigma_lo))
if (anchor_index - restart_from) >= 1 and accumulated_span >= target_log_span:
break
restart_from = max(0, restart_from)
return sigmas[restart_from:anchor_index + 1], anchor_index
return {
"enabled": True,
"start_pct": start_pct,
"count": count,
"spacing_pct": spacing_pct,
"strength": strength,
"noise": max(0.0, min(0.80, noise)),
}
def _expand_restart_sigmas(sigmas, high_quality_pct, restart_steps, restart_repeats, restart_trigger_pct):
if sigmas is None or not isinstance(sigmas, torch.Tensor):
return None, None
expanded_sigmas = sigmas.detach().clone()
total_steps = max(0, int(expanded_sigmas.shape[0]) - 1)
if total_steps <= 0:
return expanded_sigmas, None
def _get_late_start_index(total_steps, high_quality_pct):
high_quality_pct = max(0.0, min(1.0, float(high_quality_pct)))
restart_steps = max(2, int(restart_steps))
restart_repeats = max(0, int(restart_repeats))
restart_trigger_pct = max(0.0, min(1.0, float(restart_trigger_pct)))
late_steps = max(1, int(math.ceil(total_steps * high_quality_pct))) if high_quality_pct > 0.0 else 0
late_start = max(0, total_steps - late_steps)
return max(0, total_steps - late_steps)
quality_sigma_start = None
if late_start < expanded_sigmas.shape[0]:
quality_sigma_start = float(expanded_sigmas[late_start].item())
if restart_repeats <= 0:
return expanded_sigmas, quality_sigma_start
restart_sigmas, anchor_index = _build_restart_sigmas(
expanded_sigmas,
total_steps,
restart_trigger_pct,
restart_steps,
def _build_transition_pulse_steps(sigmas, total_steps, high_quality_pct,
transition_mode, transition_start_pct,
transition_count, transition_spacing_pct,
transition_strength):
options = _resolve_transition_options(
transition_mode,
transition_start_pct,
transition_count,
transition_spacing_pct,
transition_strength,
)
if restart_sigmas.numel() <= 1:
return expanded_sigmas, quality_sigma_start
if not options["enabled"] or total_steps <= 2:
return [], options
prefix = expanded_sigmas[:anchor_index + 1]
suffix = expanded_sigmas[anchor_index + 1:]
repeated = [restart_sigmas.clone() for _ in range(restart_repeats)]
expanded_sigmas = torch.cat([prefix] + repeated + [suffix], dim=0)
return expanded_sigmas, quality_sigma_start
late_start = _get_late_start_index(total_steps, high_quality_pct)
latest_anchor_pct = max(0.04, min(0.92, (late_start - 1) / max(1, total_steps - 1)))
pulse_steps = []
used_anchors = set()
for pulse_index in range(options["count"]):
trigger_pct = options["start_pct"] + options["spacing_pct"] * pulse_index
if trigger_pct >= latest_anchor_pct:
break
anchor_index = _find_schedule_anchor_index(sigmas, total_steps, trigger_pct)
anchor_index = min(total_steps - 1, max(0, anchor_index))
if anchor_index in used_anchors or anchor_index >= late_start:
continue
used_anchors.add(anchor_index)
pulse_steps.append({
"step_index": anchor_index,
"noise": options["noise"],
})
return pulse_steps, options
def _prepare_dynamic_sigmas(sigmas, high_quality_pct, transition_mode="off",
transition_start_pct=0.30, transition_count=2,
transition_spacing_pct=0.22, transition_strength=0.85):
if sigmas is None or not isinstance(sigmas, torch.Tensor):
return None, None, [], 0.0
base_sigmas = sigmas.detach().clone()
total_steps = max(0, int(base_sigmas.shape[0]) - 1)
if total_steps <= 0:
return base_sigmas, None, [], 0.0
late_start = _get_late_start_index(total_steps, high_quality_pct)
quality_sigma_start = None
if late_start < base_sigmas.shape[0]:
quality_sigma_start = float(base_sigmas[late_start].item())
pulse_steps, transition_options = _build_transition_pulse_steps(
base_sigmas,
total_steps,
high_quality_pct,
transition_mode,
transition_start_pct,
transition_count,
transition_spacing_pct,
transition_strength,
)
return base_sigmas, quality_sigma_start, pulse_steps, transition_options["noise"]
def sample_funpack_hybrid_euler_2s(model, x, sigmas, extra_args=None, callback=None,
disable=None, eta=1.0, s_noise=1.0,
high_quality_pct=0.35, correction_blend=1.0,
restart_steps=3, restart_repeats=0,
restart_trigger_pct=0.85, restart_noise=1.0,
quality_sigma_start=None):
quality_sigma_start=None,
transition_mode="off",
transition_start_pct=0.30,
transition_count=2,
transition_spacing_pct=0.22,
transition_strength=0.85,
transition_noise=0.0,
transition_pulse_steps=None):
"""
Hybrid sampler:
- Early schedule: Euler ancestral for motion/anatomy buildup.
- Late schedule: deterministic Euler / DPM-Solver++(2S) ODE refinement for detail.
- Restart: paper-style re-noise and short replay at the user-selected schedule point.
- Motion pulses: optional monotonic noise kicks before the late quality phase.
"""
if isinstance(model.inner_model.inner_model.model_sampling, comfy.model_sampling.CONST):
return k_diffusion_sampling.sample_euler_ancestral(
@@ -204,10 +220,23 @@ def sample_funpack_hybrid_euler_2s(model, x, sigmas, extra_args=None, callback=N
high_quality_pct = max(0.0, min(1.0, float(high_quality_pct)))
correction_blend = max(0.0, min(1.0, float(correction_blend)))
restart_steps = max(2, int(restart_steps))
restart_repeats = max(0, int(restart_repeats))
restart_trigger_pct = max(0.0, min(1.0, float(restart_trigger_pct)))
restart_noise = max(0.0, float(restart_noise))
if not transition_pulse_steps:
_, _, transition_pulse_steps, computed_transition_noise = _prepare_dynamic_sigmas(
sigmas,
high_quality_pct,
transition_mode,
transition_start_pct,
transition_count,
transition_spacing_pct,
transition_strength,
)
transition_noise = computed_transition_noise
transition_noise = max(0.0, float(transition_noise))
transition_step_noise = {
int(item.get("step_index", -1)): max(0.0, float(item.get("noise", transition_noise)))
for item in (transition_pulse_steps or [])
if isinstance(item, dict)
}
s_in = x.new_ones([x.shape[0]])
callback_step = 0
@@ -222,25 +251,13 @@ def sample_funpack_hybrid_euler_2s(model, x, sigmas, extra_args=None, callback=N
for i in comfy.utils.model_trange(total_steps, disable=disable):
sigma = sigmas[i]
sigma_next = sigmas[i + 1]
is_restart_jump = sigma_next > sigma
in_quality_phase = quality_sigma_start is not None and float(sigma.item()) <= quality_sigma_start
if is_restart_jump:
denoised = None
if callback is not None:
denoised = model(x, sigma * s_in, **extra_args)
callback({
"x": x,
"i": callback_step,
"sigma": sigma,
"sigma_hat": sigma,
"denoised": denoised,
})
callback_step += 1
x = _renoise_to_sigma(x, sigma, sigma_next, restart_noise, noise_sampler)
continue
if not in_quality_phase:
pulse_noise = transition_step_noise.get(int(i), 0.0)
if pulse_noise > 0.0:
x = _apply_motion_pulse(x, sigma, sigma_next, pulse_noise, noise_sampler)
denoised = model(x, sigma * s_in, **extra_args)
if callback is not None:
@@ -329,28 +346,60 @@ class FunPackHybridEuler2SSampler:
"min": 2,
"max": 32,
"step": 1,
"tooltip": "How many previous sigma steps are revisited during Restart replay."
"tooltip": "Legacy Restart control. Ignored because Restart replay is disabled for LTX audio safety."
}),
"restart_repeats": ("INT", {
"default": 0,
"min": 0,
"max": 8,
"step": 1,
"tooltip": "How many paper-style re-noise/replay loops to run at the chosen restart point. Set to 0 to disable Restart."
"tooltip": "Legacy Restart control. Ignored because Restart replay is disabled for LTX audio safety."
}),
"restart_trigger_pct": ("FLOAT", {
"default": 0.85,
"min": 0.0,
"max": 1.0,
"step": 0.01,
"tooltip": "Sampling progress point where Restart is triggered across the full schedule."
"tooltip": "Legacy Restart control. Ignored because Restart replay is disabled for LTX audio safety."
}),
"restart_noise": ("FLOAT", {
"default": 1.0,
"min": 0.0,
"max": 2.0,
"step": 0.01,
"tooltip": "Noise strength used when re-noising up to the Restart interval's higher sigma."
"tooltip": "Legacy Restart control. Ignored because Restart replay is disabled for LTX audio safety."
}),
"transition_mode": (TRANSITION_MODES, {
"default": "off",
"tooltip": "Adds early/mid motion pulses for single-clip image-to-video. Off preserves legacy sampler behavior."
}),
"transition_start_pct": ("FLOAT", {
"default": 0.30,
"min": 0.0,
"max": 0.95,
"step": 0.01,
"tooltip": "Sampling progress where the first motion pulse starts."
}),
"transition_count": ("INT", {
"default": 2,
"min": 1,
"max": 6,
"step": 1,
"tooltip": "How many early/mid motion pulses to request before the late quality phase."
}),
"transition_spacing_pct": ("FLOAT", {
"default": 0.22,
"min": 0.04,
"max": 0.45,
"step": 0.01,
"tooltip": "Progress spacing between motion pulses."
}),
"transition_strength": ("FLOAT", {
"default": 0.85,
"min": 0.0,
"max": 1.0,
"step": 0.01,
"tooltip": "How strongly motion pulses add monotonic noise kicks. Higher values push harder against stale image references."
}),
},
"optional": {
@@ -362,16 +411,21 @@ class FunPackHybridEuler2SSampler:
RETURN_NAMES = ("sampler", "sigmas")
FUNCTION = "get_sampler"
CATEGORY = "FunPack/Sampling"
DESCRIPTION = "Hybrid sampler: early Euler ancestral for motion, late DPM-Solver++(2S) ODE refinement for quality, with optional sigma-driven Restart replay."
DESCRIPTION = "Hybrid sampler: early Euler ancestral for motion, late DPM-Solver++(2S) ODE refinement for quality, with optional audio-safe motion pulses."
def get_sampler(self, eta, s_noise, high_quality_pct, correction_blend,
restart_steps, restart_repeats, restart_trigger_pct, restart_noise, sigmas=None):
expanded_sigmas, quality_sigma_start = _expand_restart_sigmas(
restart_steps, restart_repeats, restart_trigger_pct, restart_noise,
transition_mode="off", transition_start_pct=0.30,
transition_count=2, transition_spacing_pct=0.22,
transition_strength=0.85, sigmas=None):
prepared_sigmas, quality_sigma_start, transition_pulse_steps, transition_noise = _prepare_dynamic_sigmas(
sigmas,
high_quality_pct,
restart_steps,
restart_repeats,
restart_trigger_pct,
transition_mode,
transition_start_pct,
transition_count,
transition_spacing_pct,
transition_strength,
)
sampler = comfy.samplers.KSAMPLER(
sample_funpack_hybrid_euler_2s,
@@ -380,11 +434,14 @@ class FunPackHybridEuler2SSampler:
"s_noise": s_noise,
"high_quality_pct": high_quality_pct,
"correction_blend": correction_blend,
"restart_steps": restart_steps,
"restart_repeats": restart_repeats,
"restart_trigger_pct": restart_trigger_pct,
"restart_noise": restart_noise,
"quality_sigma_start": quality_sigma_start,
"transition_mode": transition_mode,
"transition_start_pct": transition_start_pct,
"transition_count": transition_count,
"transition_spacing_pct": transition_spacing_pct,
"transition_strength": transition_strength,
"transition_noise": transition_noise,
"transition_pulse_steps": transition_pulse_steps,
}
)
return (sampler, expanded_sigmas)
return (sampler, prepared_sigmas)