Add audio-safe motion pulses to hybrid sampler
This commit is contained in:
@@ -11,13 +11,9 @@ It is designed as a quality/speed compromise between:
|
||||
|
||||
This sampler keeps classic Euler ancestral for the early "structure building" stage where motion, anatomy and main composition are forming. On the late denoise steps it switches to a deterministic Euler / DPM-Solver++(2S) ODE refinement path where fine detail and cleanup usually matter most.
|
||||
|
||||
It can also build an optional paper-style Restart schedule at the progress point you choose:
|
||||
It can also apply optional early/mid **motion transition pulses** for single-clip image-to-video workflows. These pulses add monotonic noise kicks at selected normal denoise steps instead of inserting upward sigma jumps. This is intended to push LTX2.3 away from stale frame-1 reference behavior while avoiding the audio damage caused by Restart replay.
|
||||
|
||||
- detect a restart anchor across the full sigma schedule
|
||||
- insert upward sigma jumps back to a higher sigma
|
||||
- replay the chosen sigma interval one or more times
|
||||
|
||||
Unlike the older implementation, Restart is now encoded directly into the outgoing `SIGMAS` schedule, so the visible schedule passed into `SamplerCustomAdvanced` is the actual schedule that gets executed.
|
||||
Restart replay is disabled. Legacy restart widgets may still appear for workflow compatibility, but they are ignored.
|
||||
|
||||
## Recommended wiring
|
||||
|
||||
@@ -43,15 +39,30 @@ Example: `0.35` means only the last 35% of steps use the ODE refinement path.
|
||||
- `0.0` = pure late-step Euler ODE
|
||||
- `1.0` = full late-step DPM++(2S)-style correction
|
||||
|
||||
**restart_steps**: Target size of the restarted sigma interval. The node interprets this against the actual sigma curve and expands the outgoing `SIGMAS` schedule accordingly.
|
||||
**restart_steps**: Legacy compatibility control. Ignored.
|
||||
|
||||
**restart_repeats**: How many Restart loops to run. `0` disables Restart.
|
||||
**restart_repeats**: Legacy compatibility control. Ignored.
|
||||
|
||||
**restart_trigger_pct**: Sampling progress point where Restart is triggered across the full schedule. This is independent from `high_quality_pct`, so values such as `0.30` can trigger an early restart instead of being clamped to the Euler-to-2S transition point.
|
||||
**restart_trigger_pct**: Legacy compatibility control. Ignored.
|
||||
|
||||
**restart_noise**: Noise strength used when re-noising the latent up to the restart interval's higher sigma.
|
||||
**restart_noise**: Legacy compatibility control. Ignored.
|
||||
|
||||
**sigmas**: Optional incoming sigma schedule. If connected, the node outputs a restart-expanded sigma schedule that matches the sampler settings. If not connected, the sampler output still works, but there is no schedule for the node to rewrite.
|
||||
**transition_mode**: Motion pulse preset:
|
||||
|
||||
- `off`: preserve legacy sampler behavior.
|
||||
- `balanced`: one moderate early/mid pulse.
|
||||
- `aggressive`: at least two stronger early/mid pulses for stale image-to-video generations.
|
||||
- `custom`: use the transition count, spacing, and strength exactly as configured.
|
||||
|
||||
**transition_start_pct**: Sampling progress point where the first motion pulse is applied.
|
||||
|
||||
**transition_count**: Number of requested early/mid motion pulses. Pulses that would land in the late quality phase are skipped.
|
||||
|
||||
**transition_spacing_pct**: Progress spacing between motion pulses.
|
||||
|
||||
**transition_strength**: Strength of the monotonic noise kick. Higher values push harder against stale image references, with more drift risk.
|
||||
|
||||
**sigmas**: Optional incoming sigma schedule. If connected, the node returns the same monotonic schedule plus sampler-side metadata for motion pulses. If not connected, the sampler computes pulse positions from the runtime schedule.
|
||||
|
||||
## Recommended starting values
|
||||
|
||||
@@ -59,18 +70,19 @@ Example: `0.35` means only the last 35% of steps use the ODE refinement path.
|
||||
- `s_noise = 1.0`
|
||||
- `high_quality_pct = 0.30` to `0.40`
|
||||
- `correction_blend = 1.0`
|
||||
- `restart_repeats = 0` for baseline testing
|
||||
- `transition_mode = off` for baseline testing
|
||||
|
||||
For a first Restart test, try:
|
||||
For an aggressive LTX2.3 image-to-video motion test, try:
|
||||
|
||||
- `high_quality_pct = 0.35`
|
||||
- `correction_blend = 1.0`
|
||||
- `restart_steps = 3`
|
||||
- `restart_repeats = 1`
|
||||
- `restart_trigger_pct = 0.85`
|
||||
- `restart_noise = 1.0`
|
||||
- `transition_mode = aggressive`
|
||||
- `transition_start_pct = 0.30`
|
||||
- `transition_count = 2`
|
||||
- `transition_spacing_pct = 0.22`
|
||||
- `transition_strength = 0.85`
|
||||
|
||||
Example: with `8` base sampling steps, `restart_steps = 3`, and `restart_repeats = 1`, a normal evenly spaced schedule will usually expand into a schedule with about `3` extra restart transitions. With custom sigmas, the exact replay step count can shift slightly because Restart is anchored to the real sigma values rather than raw step indices.
|
||||
Keep the prompt/refiner conditioning explicit about both action and camera change, for example `orbiting camera`, `dolly in`, `zoom out`, `new side angle`, `turning`, or `dynamic pose change`.
|
||||
|
||||
## Expected behavior
|
||||
|
||||
@@ -81,9 +93,9 @@ Compared to plain `euler_ancestral`, this sampler should usually:
|
||||
- keep the early motion/anatomy formation more lively
|
||||
- cost less than running a heavier deterministic solver for the whole schedule
|
||||
|
||||
With Restart enabled, it may further improve late cleanup and detail consistency, but it will increase runtime and may be too aggressive for some video workflows if overused.
|
||||
With motion pulses enabled, it should more strongly encourage action, camera movement, and viewpoint changes inside a single clip. Stronger settings can increase subject drift or visual instability.
|
||||
|
||||
Because Restart is now represented in the outgoing `SIGMAS`, the total number of executed transitions should match what the expanded schedule shows in the downstream custom sampler.
|
||||
The outgoing `SIGMAS` remain monotonic. Motion pulses happen inside the sampler at selected denoise steps rather than by expanding the schedule.
|
||||
|
||||
## Limitation
|
||||
|
||||
|
||||
Reference in New Issue
Block a user