nodes/olivv-cs-ComfyUI-FunPack
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

fix: cut_opening_frames cuts decoded pixels, not latents (noisy first frame)

Browse Source 
The opening frame came out noisy because the cut was made in latent space. The
LTX video VAE is causal: latent frame 0 is the temporal origin and decodes to a
single pixel frame, while every later latent frame was generated as a
CONTINUATION of the frames before it. Slicing the front off the latent promoted
a continuation frame to position 0, and the decoder then applied the
temporal-origin handling that frame was never generated for. H3 already cut
decoded images for unrelated reasons and so never had the bug; LTX now does the
same thing.

The count was wrong too, quietly. A latent cut could only remove whole latent
frames, and the promoted frame changed from covering 1 pixel to covering
`scale`, so dropping k latent frames shortened the clip by k*scale rather than
the (k-1)*scale+1 it actually spanned. The value was also rounded up to a whole
latent frame on the way in. It is now exact: N means N, matching H3.

Each eligible scene records a (start, count) pixel span instead of cutting; the
spans are removed after decode, later ones first so earlier removals cannot
shift their indices. Mid-chain hard-cut scenes keep working — the span is
computed from the scene's position in the chain, which is exact because anchor
scenes append with zero blend overlap. The latent's audio stream loses the same
stretches of TIME, mapped by proportion on its own axis, or sound would run
ahead of picture with no error. The latent's VIDEO stream is deliberately left
whole (cutting it is the bug), so video comes from the IMAGES output — which the
Editor already wires. A run with IMAGES unconnected now skips with a reason
instead of silently doing nothing.

_cut_opening_latent is removed rather than left dead, since its docstring
documented the broken approach as correct.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
digital-garbage
2026-08-11 23:47:51 +03:00
co-authored by Claude Opus 5
parent e7b624d684
commit 98d48e78e3
2 changed files with 259 additions and 109 deletions
Download Patch File Download Diff File Expand all files Collapse all files
samplers.py
+155 -56
View File
@@ -2820,7 +2820,7 @@ class FunPackLTXAVSceneChainSampler:
}),
"cut_opening_frames": ("INT", {
"default": 0, "min": 0, "max": 512, "step": 8,
"tooltip": "Let the i2v anchor do its work, then cut it out of the clip: generate the scene exactly as normal (anchor pinned at full strength the whole way, nothing weakened, no extra sampling), then drop this many frames off the FRONT of the finished clip. The anchor is a pinned latent frame at position 0 — it transfers identity, style and composition better than anything that softens it on the way in (ALG blurs it and loses character detail; Best-FaceID tokens approximate it and lose some too), but it is also literally the first frame you see, so every i2v scene opens on the exact reference still. Cutting it afterwards keeps the transfer and removes the tell: an i2v generation that reads as t2v. 0 (default) = off. The value is in REAL frames and is converted with the VAE's own time scale; one latent frame (8 real frames at the standard scale) removes just the anchor itself, which is usually NOT enough — the anchor is followed by a settling-in stretch where the shot is still leaving the reference still and little is happening yet, and on a prompt that asks for immediate action that dead time is exactly what you want gone (48 was the value that worked on a 768x768x305@30 i2v chain with a quick-cut prompt — a starting point for this pipeline, not a universal default). NOTHING IS REGROWN: the scene comes out that much SHORTER than the length you asked for, and the audio is cropped to match. That is the trade — every surviving frame was generated as part of one continuous shot, with no invented ending. Needs a pinned i2v anchor; skipped with the reason in the scene report on continuation scenes and on scenes carrying guide frames or JoyAI audio memory. ON MINIMAX H3 the same setting works differently, because H3's anchor is a keyframe condition row rather than a pinned latent frame and its video latent lives on a 5k+2 grid an arbitrary cut cannot land on: the frames are dropped off the front of the DECODED image batch instead (exact to the frame, no rounding), and the latent's audio stream is cropped to match so sound and picture still start together. The LATENT output keeps its full video stream there — take video from IMAGES on a cut H3 run — and only the chain's opening is cut, not each scene's.",
"tooltip": "Let the i2v anchor do its work, then cut it out of the clip: generate the scene exactly as normal (anchor pinned at full strength the whole way, nothing weakened, no extra sampling), then drop this many frames off the FRONT of the finished clip. The anchor is a pinned latent frame at position 0 — it transfers identity, style and composition better than anything that softens it on the way in (ALG blurs it and loses character detail; Best-FaceID tokens approximate it and lose some too), but it is also literally the first frame you see, so every i2v scene opens on the exact reference still. Cutting it afterwards keeps the transfer and removes the tell: an i2v generation that reads as t2v. 0 (default) = off. The value is in REAL frames and is EXACT — N means N, with no rounding to the latent grid. The anchor itself is only the first ~8 real frames, which is usually NOT enough: it is followed by a settling-in stretch where the shot is still leaving the reference still and little is happening yet, and on a prompt that asks for immediate action that dead time is exactly what you want gone (48 was the value that worked on a 768x768x305@30 i2v chain with a quick-cut prompt — a starting point for this pipeline, not a universal default). NOTHING IS REGROWN: the scene comes out that much SHORTER than the length you asked for, and the audio is cropped to match. That is the trade — every surviving frame was generated as part of one continuous shot, with no invented ending. HOW IT IS CUT: on the DECODED frames, never on the latent, on LTX and MiniMax H3 alike. The video VAE is causal — latent frame 0 is the temporal origin — so slicing the front off the latent promoted a continuation frame to position 0 and it decoded with origin handling it was never generated for, which came out as a noisy first frame. Decoding everything first and dropping pixels afterwards leaves every surviving frame in the context it was sampled in. Consequence: the LATENT output keeps its FULL video stream, so take video from the IMAGES output on a cut run (audio from the latent as usual — its audio stream IS cropped, so sound and picture still start together). The IMAGES output must be connected or there is nothing to crop, and the run says so. Needs a pinned i2v anchor; skipped with the reason in the scene report on continuation scenes and on scenes carrying guide frames or JoyAI audio memory. On MiniMax H3 only the chain's opening is cut, not each scene's, because H3's anchor is a keyframe condition row rather than a pinned latent prefix.",
}),
"context_windows": ("BOOLEAN", {
"default": False,
@@ -3780,50 +3780,99 @@ class FunPackLTXAVSceneChainSampler:
idx[dim] = slice(n - keep, n)
return tensor[tuple(idx)]
def _cut_opening_latent(self, latent, drop):
"""Cut the first `drop` latent frames off a FINISHED scene latent.
def _scene_pixel_start(self, latent_start, time_scale):
"""First DECODED pixel index of a scene that begins at `latent_start` in the chain.
The i2v anchor is a pinned frame at position 0: it transfers identity, style and
composition at full strength all the way down the schedule, and it is also literally
the first frame of the clip, so every i2v scene opens on the exact reference still.
Cutting it (plus the settling-in stretch behind it, where the shot is still leaving
that still) afterwards keeps the transfer and removes the tell.
Nothing is regrown, so the scene simply comes out `drop` latent frames SHORTER than
the length that was asked for. That is the whole trade: every surviving frame was
generated as part of one continuous shot, with no invented ending and no joint.
Nothing downstream constrains the new size: LTXAV's video patchifier has temporal
patch size 1 (SymmetricPatchifier(1) -> no divisibility rule on latent frames), the
streams reach the transformer as independent tensors with no enforced video:audio
ratio, and VAE decode is (f-1)*scale+1 for any f >= 1.
What DOES matter is proportion. Audio timings come from the audio stream's own index
and video's from its own RoPE, so cropping the two by different amounts of TIME
desynchronises them silently — no error, just sound that drifts. Each extra stream's
kept length is therefore derived from the new VIDEO length directly, never by
subtracting a separately-rounded drop.
Returns (latent, dropped).
The causal VAE decodes f latent frames to (f-1)*scale+1 pixels: latent frame 0 is
the temporal origin and covers ONE pixel, every later latent frame covers `scale`.
So latent frame i starts at pixel (i-1)*scale+1, and only frame 0 starts at 0.
"""
i = max(0, int(latent_start))
if i == 0:
return 0
return (i - 1) * max(1, int(time_scale)) + 1
def _cut_opening_pixel_spans(self, images, spans):
"""Remove `spans` — (start, count) pixel ranges — from a decoded image batch.
Cutting the OPENING has to happen here, on pixels, and never on the latent. The LTX
VAE is causal: latent frame 0 is the temporal origin, and every later latent frame
was generated as a CONTINUATION of the frames before it. Slice the front off the
latent and the survivor promoted to position 0 gets decoded with the origin handling
it was never generated for, which is what produced the noisy opening frame. Decoding
the whole latent first and dropping pixels afterwards leaves the origin intact, so
every surviving frame decodes in the context it was sampled in.
It is also exact. A latent cut could only remove whole latent frames, and because
the promoted frame changes from covering 1 pixel to covering `scale`, dropping k
latent frames actually shortened the clip by k*scale pixels rather than the
(k-1)*scale+1 it spanned — so the count never matched what was asked for either.
Here N means N, the way it already does on H3.
Later spans are removed first so earlier removals cannot shift their indices.
"""
if not isinstance(images, torch.Tensor) or not spans:
return images, 0
total = int(images.shape[0])
keep = torch.ones(total, dtype=torch.bool)
for start, count in sorted(spans, key=lambda s: int(s[0]), reverse=True):
start = max(0, min(int(start), total))
end = max(start, min(start + max(0, int(count)), total))
keep[start:end] = False
if bool(keep.all()):
return images, 0
if not bool(keep.any()):
# Never hand back an empty batch: a cut that would remove the whole clip is a
# misconfiguration, and returning nothing turns it into an obscure downstream
# crash instead of a visibly-too-short video.
keep[-1] = True
return images[keep], total - int(keep.sum())
def _remove_latent_time_spans(self, latent, pixel_spans, pixel_total):
"""Remove the same stretches of TIME from every non-video stream of `latent`.
`pixel_spans` are indices into the decoded VIDEO. Audio runs at its own rate on its
own axis (LTXAV puts time on dim 2; MiniMax H3 puts it last), so each span is mapped
by PROPORTION of the clip rather than by a frame count — cropping the two streams by
different amounts of time desynchronises them with no error, just sound that drifts.
The video stream is deliberately untouched: it is the thing that must keep its
temporal origin, and the caller takes video from the decoded IMAGES instead.
"""
if not pixel_spans or int(pixel_total) <= 0:
return latent
result = self._clone_latent(latent)
tensors = self._latent_tensors(result)
video = tensors[0]
frames = self._tensor_frames(video)
drop = max(0, min(int(drop), frames - 1))
if drop <= 0:
return result, 0
tensors[0] = video[:, :, drop:]
kept = frames - drop
for _idx in range(1, len(tensors)):
tensors[_idx] = self._crop_stream_head_to(tensors[_idx], _idx, kept, frames)
changed = False
for idx in range(1, len(tensors)):
tensor = tensors[idx]
axis = self._time_dims[idx] if idx < len(self._time_dims) else 2
try:
length = int(tensor.shape[axis])
except Exception:
continue
if length <= 1:
continue
keep = torch.ones(length, dtype=torch.bool)
for start, count in pixel_spans:
a = int(round(int(start) / float(pixel_total) * length))
b = int(round((int(start) + int(count)) / float(pixel_total) * length))
a = max(0, min(a, length))
b = max(a, min(b, length))
keep[a:b] = False
if bool(keep.all()):
continue
if not bool(keep.any()):
keep[-1] = True
tensors[idx] = tensor.index_select(axis, torch.nonzero(keep, as_tuple=True)[0].to(tensor.device))
changed = True
if not changed:
return latent
if self._is_nested(result.get("samples")):
result["samples"] = comfy.nested_tensor.NestedTensor(tensors)
else:
result["samples"] = tensors[0]
# The pin belongs to a frame that no longer exists.
result.pop("noise_mask", None)
return result, drop
return result
def _cut_opening_pixels(self, images, latent, pixel_frames):
"""MiniMax H3's cut: drop the opening off the DECODED frames, not off the latent.
@@ -6234,6 +6283,14 @@ class FunPackLTXAVSceneChainSampler:
carried_guide_frames = 0
boundary_entries = []
cumulative_latent_frames = 0
# (start_pixel, count) ranges cut_opening_frames will remove from the DECODED video,
# collected per eligible scene and applied once after decode. Latents stay whole so
# every frame decodes with the temporal origin it was sampled against.
_cut_spans = []
# Read before the scene loop, not just before the decode: cut_opening_frames is a
# crop on the decoded frames now, so scene eligibility depends on knowing whether
# anything is being decoded at all.
want_image = self._output_connected(prompt, unique_id, 1)
# Load liked direction once for embed_guidance. Source selects which learned direction:
# 'relative' = this prompt's key; 'absolute' = the global, prompt-agnostic taste store
@@ -6710,8 +6767,10 @@ class FunPackLTXAVSceneChainSampler:
# the eligibility below can even be evaluated.
_cut_drop = 0
if int(cut_opening_frames) > 0 and not self._is_h3:
_cut_drop = self._expected_latent_frames(
int(cut_opening_frames) + 1, time_scale) - 1
# The cut is measured in REAL frames and applied to decoded pixels, so
# there is no rounding to a latent grid any more and no minimum: 1 means
# 1. This is only a "does this scene qualify" flag now.
_cut_drop = 1
if carried + soft_carried > 0:
_reason = ("continuation scene — the opening is carried frames from the "
"previous scene, not an anchor, and cutting them would break "
@@ -6723,9 +6782,11 @@ class FunPackLTXAVSceneChainSampler:
"which those extra frames would make wrong")
elif self._context_scene_latent_frames(chunk) is None:
_reason = "scene latent could not be read"
elif _cut_drop <= 0:
_reason = (f"cut_opening_frames={cut_opening_frames} is under one latent "
f"frame at this VAE time scale")
elif not want_image:
# The cut is a crop on the decoded frames now, so with nothing
# decoding there is nothing to crop — the same rule H3 already has.
_reason = ("the IMAGES output is not connected — the cut is a crop on "
"the decoded frames, so there is nothing to crop")
elif self._anchor_pinned_frames(chunk) <= 0:
# No pinned prefix = a genuine t2v scene (no anchor image attached).
# Cutting here would throw away real generated frames for nothing —
@@ -6862,24 +6923,33 @@ class FunPackLTXAVSceneChainSampler:
else:
model.model_options.pop("model_function_wrapper", None)
if _cut_drop > 0:
# The finished clip, minus its opening. Nothing is regrown, so the scene is
# now shorter than the length that was asked for — say so in the report
# rather than leaving it to be inferred from the output.
_before_cut = self._tensor_frames(self._latent_tensors(sampled)[0])
sampled, _dropped = self._cut_opening_latent(sampled, _cut_drop)
_after_cut = self._tensor_frames(self._latent_tensors(sampled)[0])
# Recorded now, applied after decode. The latent itself is deliberately left
# whole: cutting it here is what made the opening frame noisy, because the
# frame promoted to position 0 gets decoded as a temporal origin it was never
# generated as (see _cut_opening_pixel_spans). The scene's position in the
# chain is read BEFORE it is appended, so the span is an index into the final
# decoded video, and anchor scenes append with zero blend overlap, which is
# what makes that index exact.
_cut_latent_start = (0 if output is None
else self._tensor_frames(self._latent_tensors(output)[0]))
_cut_spans.append((self._scene_pixel_start(_cut_latent_start, time_scale),
int(cut_opening_frames)))
_pinned_n = self._anchor_pinned_frames(chunk)
# What the anchor actually spans in PIXELS, which is what the cut is now
# measured in: the pinned prefix is at the scene's origin, so it covers
# (pinned - 1) * scale + 1 real frames.
_anchor_px = (_pinned_n - 1) * max(1, int(time_scale)) + 1 if _pinned_n > 0 else 0
run_mechanisms.append(
f"cut_opening_frames(dropped {_dropped} of {_pinned_n} pinned latent "
f"frames — clip CROPPED to {_after_cut} of {_before_cut} latent frames, "
f"~{int(cut_opening_frames)} real frames shorter)")
if _dropped < _pinned_n:
f"cut_opening_frames(scheduled: {int(cut_opening_frames)} real frames off "
f"the front of this scene, cut after decode so the opening frame keeps a "
f"clean origin — the scene is that much SHORTER)")
if int(cut_opening_frames) < _anchor_px:
# Part of the anchor is still visible — the one outcome this exists to
# prevent, so name the fix.
run_mechanisms.append(
f"cut_opening_frames WARNING: {_pinned_n - _dropped} anchor latent "
f"frame(s) still in the clip — raise cut_opening_frames to at least "
f"{_pinned_n * max(1, int(time_scale))}")
f"cut_opening_frames WARNING: the pinned anchor spans {_anchor_px} real "
f"frames and only {int(cut_opening_frames)} are being cut — raise "
f"cut_opening_frames to at least {_anchor_px}")
if carried + soft_carried > 0:
sampled = self._crop_video_head(sampled, carried + soft_carried)
if guide_tail > 0:
@@ -6968,7 +7038,7 @@ class FunPackLTXAVSceneChainSampler:
# RETURN_TYPES slot indices: 0=latent, 1=images, 2=status...
# Sampling is fully complete. Latent is untouched and returned as-is.
# IMAGES: decode the whole latent in one pass, then apply transition effects.
want_image = self._output_connected(prompt, unique_id, 1)
# (want_image was read before the scene loop — cut_opening_frames eligibility needs it.)
images = None
if want_image:
@@ -6988,6 +7058,35 @@ class FunPackLTXAVSceneChainSampler:
images = self._apply_transitions_pixel(decoded, boundary_entries, transition_duration)
_phase_decode = _time.perf_counter() - _t_dec0
# cut_opening_frames, LTX path. Applied HERE, after a decode that saw every latent
# frame in its sampled context, rather than on the latent inside the loop — a latent
# cut promoted a continuation frame to position 0 and the causal VAE decoded it as an
# origin, which is what made the first frame noisy. Transitions run first so
# boundary_entries still index the uncut video.
if _cut_spans and not self._is_h3:
_px_total = int(images.shape[0]) if isinstance(images, torch.Tensor) else 0
if _px_total > 0:
images, _px_dropped = self._cut_opening_pixel_spans(images, _cut_spans)
if _px_dropped > 0:
# The audio stream has to lose the same stretches of TIME or the sound
# runs ahead of the picture — silently, with no error, exactly the trap
# _crop_stream_head_to exists for on the carried-frame path.
output = self._remove_latent_time_spans(output, _cut_spans, _px_total)
report_lines.append(
f"cut_opening_frames(dropped {_px_dropped} of {_px_total} decoded "
f"frames across {len(_cut_spans)} scene opening(s) — cut after decode, "
f"so the first surviving frame keeps a clean origin; the audio stream "
f"was cropped by the same amount of time)")
report_lines.append(
"cut_opening_frames NOTE: the LATENT output keeps its FULL video "
"stream (cutting it is what produced a noisy opening frame) — take "
"video from the IMAGES output on a cut run, audio from the latent.")
if transition_duration and boundary_entries:
report_lines.append(
"cut_opening_frames NOTE: transitions were rendered before this "
"cut, so a transition running INTO a cut scene loses the frames "
"that were removed with the anchor.")
# cut_opening_frames, H3 path. The LTX path cut each scene's finished LATENT inside
# the loop above; H3 has no pinned latent prefix to cut and an off-grid video latent
# cannot be decoded, so the cut lands here instead — exactly N frames off the head of
tests/test_cut_opening.py
+104 -53
View File
@@ -4,15 +4,26 @@ The point of the feature is that the anchor is NOT weakened on the way in (ALG b
loses character detail; Best-FaceID overlap tokens approximate it and lose some too) — it is
pinned at full strength for the whole schedule and the opening is then cut off the FINISHED
clip. Nothing is regrown to replace it (regrowing was tried and the invented ending came out
with worse or missing movement), so the invariants worth testing are:
with worse or missing movement).
The cut happens on DECODED PIXELS, never on the latent — on both LTX and H3 now. Cutting the
latent promoted a continuation frame to position 0, and the causal VAE decoded it with the
temporal-origin handling it was never generated for, which showed up as a noisy first frame.
It also could not be exact: a latent cut removed whole latent frames, and since the promoted
frame went from covering 1 pixel to covering `scale`, the clip shortened by a different
amount than the span that was removed.
Invariants worth testing:
1. The clip really does get SHORTER — no tail is invented, no joint is blended, and every
surviving frame is the original untouched one.
2. Video and audio still describe the same duration. Audio timing comes from the audio
2. Exactly N frames go, from the right place, including for a mid-chain scene opening, and
multiple spans do not shift each other's indices.
3. Video and audio still describe the same duration. Audio timing comes from the audio
stream's own index, so cropping the two by different amounts desyncs the clip silently.
3. The pin goes with the frame it belonged to.
4. The cut can never consume the whole clip, and never mutates its input.
5. The t2v guard: no pinned anchor means there is nothing to cut out.
4. The latent's VIDEO stream is left whole — that is what keeps the origin intact.
5. The cut can never consume the whole clip, and never mutates its input.
6. The t2v guard: no pinned anchor means there is nothing to cut out.
"""
import sys
import types
@@ -79,80 +90,120 @@ def _av_latent(video_frames=10, audio_frames=20):
return {"samples": _FakeNested([video, audio]), "noise_mask": torch.zeros_like(video)}
# ── the cut ─────────────────────────────────────────────────────────────────
# ── the cut: pixels, never latents ──────────────────────────────────────────
def test_cut_shortens_the_clip_instead_of_regrowing_a_tail():
cut, dropped = _node()._cut_opening_latent(_latent(frames=10), 3)
assert dropped == 3
# 10 - 3, not 10: nothing is invented at the end, which is the whole point.
assert cut["samples"].shape[2] == 7
# And what survives is the ORIGINAL frames 3..9, untouched — no blend, no joint.
assert [int(v) for v in cut["samples"][0, 0, :, 0, 0]] == [3, 4, 5, 6, 7, 8, 9]
def test_scene_pixel_start_accounts_for_the_causal_origin():
"""The VAE decodes f latent frames to (f-1)*scale+1 pixels: latent frame 0 is the
temporal origin and covers ONE pixel, every later frame covers `scale`. A scene starting
at latent frame i therefore starts at pixel (i-1)*scale+1, not i*scale."""
n = _node()
assert n._scene_pixel_start(0, 8) == 0
assert n._scene_pixel_start(1, 8) == 1
assert n._scene_pixel_start(2, 8) == 9
assert n._scene_pixel_start(5, 8) == 33
def test_cut_drops_the_pin_with_the_frame_it_belonged_to():
latent = _latent(frames=10)
assert "noise_mask" in latent
cut, _ = _node()._cut_opening_latent(latent, 1)
assert "noise_mask" not in cut
def test_cut_removes_exactly_the_requested_pixel_frames():
"""N means N. The old latent cut could only remove whole latent frames, and because the
promoted frame changed from covering 1 pixel to covering `scale`, the clip actually
shortened by a different amount than the span it removed."""
images = torch.arange(20, dtype=torch.float32).view(20, 1, 1, 1)
cut, dropped = _node()._cut_opening_pixel_spans(images, [(0, 6)])
assert dropped == 6
assert cut.shape[0] == 14
# The survivors are the ORIGINAL frames 6..19, untouched — no blend, no joint.
assert [int(v) for v in cut[:, 0, 0, 0]] == list(range(6, 20))
def test_cut_handles_a_mid_chain_scene_opening():
"""A hard-cut scene later in the chain has its own anchor to remove, so the cut is a
span, not just a head crop."""
images = torch.arange(20, dtype=torch.float32).view(20, 1, 1, 1)
cut, dropped = _node()._cut_opening_pixel_spans(images, [(10, 4)])
assert dropped == 4
assert [int(v) for v in cut[:, 0, 0, 0]] == list(range(10)) + list(range(14, 20))
def test_multiple_spans_do_not_shift_each_other():
"""Removing an earlier span first would move every later index — the classic
delete-while-iterating bug. Later spans must be removed first."""
images = torch.arange(20, dtype=torch.float32).view(20, 1, 1, 1)
cut, dropped = _node()._cut_opening_pixel_spans(images, [(0, 3), (10, 2)])
assert dropped == 5
assert [int(v) for v in cut[:, 0, 0, 0]] == list(range(3, 10)) + list(range(12, 20))
def test_cut_never_consumes_the_whole_clip():
cut, dropped = _node()._cut_opening_latent(_latent(frames=4), 99)
assert dropped == 3 # clamped to frames - 1
assert cut["samples"].shape[2] == 1
"""A cut longer than the clip is a misconfiguration; an empty batch turns it into an
obscure downstream crash instead of a visibly-too-short video."""
images = torch.arange(5, dtype=torch.float32).view(5, 1, 1, 1)
cut, dropped = _node()._cut_opening_pixel_spans(images, [(0, 99)])
assert cut.shape[0] == 1
assert dropped == 4
def test_zero_drop_is_a_no_op():
cut, dropped = _node()._cut_opening_latent(_latent(frames=6), 0)
def test_no_spans_is_a_no_op():
images = torch.arange(6, dtype=torch.float32).view(6, 1, 1, 1)
cut, dropped = _node()._cut_opening_pixel_spans(images, [])
assert dropped == 0
assert torch.equal(cut["samples"], _latent(frames=6)["samples"])
assert cut is images
def test_source_latent_is_never_mutated():
latent = _latent(frames=8)
before = latent["samples"].clone()
_node()._cut_opening_latent(latent, 3)
assert torch.equal(latent["samples"], before)
def test_source_images_are_never_mutated():
images = torch.arange(8, dtype=torch.float32).view(8, 1, 1, 1)
before = images.clone()
_node()._cut_opening_pixel_spans(images, [(0, 3)])
assert torch.equal(images, before)
# ── audio stays in sync ─────────────────────────────────────────────────────
def test_cut_keeps_audio_and_video_describing_the_same_duration(nested_stub):
def test_audio_loses_the_same_proportion_of_time(nested_stub):
"""Audio timing comes from the audio stream's own index and video's from its own RoPE,
so cropping them by different amounts of TIME desyncs the clip silently — no error,
just drifting sound. The kept lengths must stay in proportion."""
cut, _ = _node()._cut_opening_latent(_av_latent(10, 20), 3)
video, audio = cut["samples"].unbind()
assert video.shape[2] == 7 and audio.shape[2] == 14 # 7/10 == 14/20
# Cropped from the HEAD, matching the video — the tail is what survives.
assert [int(v) for v in audio[0, 0]] == list(range(6, 20))
so cropping them by different amounts of TIME desyncs the clip silently — no error, just
drifting sound. Spans are mapped by proportion, not by frame count."""
latent = _av_latent(10, 20)
out = _node()._remove_latent_time_spans(latent, [(0, 40)], 100)
_v, audio = out["samples"].unbind()
assert audio.shape[2] == 12 # 40% of 20 removed from the head
assert [int(v) for v in audio[0, 0]] == list(range(8, 20))
def test_cut_keeps_proportion_when_the_rate_does_not_divide_evenly(nested_stub):
"""The rounding trap: deriving audio's KEPT length from the new video length keeps the
two in proportion; subtracting a separately-rounded drop can leave them a frame apart."""
for v_frames, a_frames, drop in ((10, 15, 3), (9, 14, 4), (13, 7, 5), (10, 21, 7)):
cut, _ = _node()._cut_opening_latent(_av_latent(v_frames, a_frames), drop)
video, audio = cut["samples"].unbind()
assert audio.shape[2] == max(1, round(a_frames * video.shape[2] / v_frames))
assert audio.shape[2] >= 1 and video.shape[2] >= 1
def test_video_stream_is_left_whole(nested_stub):
"""The video latent must keep its temporal origin — that is the entire point of moving
the cut to pixels. Video comes from the decoded IMAGES instead."""
latent = _av_latent(10, 20)
out = _node()._remove_latent_time_spans(latent, [(0, 40)], 100)
video, _a = out["samples"].unbind()
assert video.shape[2] == 10
def test_cut_uses_each_streams_own_time_axis(nested_stub):
def test_audio_span_removal_never_empties_the_stream(nested_stub):
out = _node()._remove_latent_time_spans(_av_latent(10, 20), [(0, 100)], 100)
_v, audio = out["samples"].unbind()
assert audio.shape[2] >= 1
def test_source_latent_is_never_mutated(nested_stub):
latent = _av_latent(10, 20)
before = latent["samples"].unbind()[1].clone()
_node()._remove_latent_time_spans(latent, [(0, 40)], 100)
assert torch.equal(latent["samples"].unbind()[1], before)
def test_audio_uses_its_own_time_axis(nested_stub):
"""MiniMax H3 puts the STEREO CHANNEL on dim 2 and time last ([B, 32, 2, T]); LTXAV puts
time on dim 2 for both streams. Slicing an H3 audio stream on dim 2 would crop the stereo
pair down to mono and leave the duration untouched — silent, and wrong both ways."""
time on dim 2. Slicing an H3 audio stream on dim 2 would crop the stereo pair down to
mono and leave the duration untouched — silent, and wrong both ways."""
n = _node()
video = torch.zeros(1, 4, 10, 2, 2)
audio = torch.arange(20, dtype=torch.float32).view(1, 1, 1, 20).repeat(1, 32, 2, 1)
n._time_dims = (2, 3, 3, 3) # what _set_stream_axes records on H3
cut, _ = n._cut_opening_latent(
{"samples": _FakeNested([video, audio])}, 3)
_v, a = cut["samples"].unbind()
out = n._remove_latent_time_spans({"samples": _FakeNested([video, audio])}, [(0, 40)], 100)
_v, a = out["samples"].unbind()
assert a.shape[2] == 2 # both stereo channels survive
assert a.shape[3] == 14 # 7/10 of the duration, cropped from the head
assert [int(v) for v in a[0, 0, 0]] == list(range(6, 20))
assert a.shape[3] == 12 # 40% of the duration, removed from the head
assert [int(v) for v in a[0, 0, 0]] == list(range(8, 20))
# ── H3: the cut lands on the decoded frames, not the latent ─────────────────