fix: H3 image conditioning reaches the model
Two independent, silent breaks made an H3 i2v scene generate from the prompt alone. 1. The Editor always runs split_by_transitions, and the per-scene conditionings REPLACE the single entry Studio built — but they were re-encoded with no reference_image and no h3_references. Both the frame-0 keyframe pin and the Qwen vision presentation were gone before the sampler looked, even for a one-scene project. The split now carries the anchor onto scene 0 and the references onto every scene. Gated to H3: on LTX the anchor rides in the latent, and adding Gemma3 vision to scene 0 would change existing output. 2. A MiniMax H3 Image to Video node hands its first/last frame pins out on its CONDITIONING output, which had nowhere to go — the sampler's positive comes from Studio. New optional h3_keyframes input on the Chain Sampler salvages them: first frame onto the opening scene, last frame re-indexed onto the closing scene's own final frame (the source node's frame_count is its own, and a stale index would land mid-clip where PackedLayout refuses it). The editor auto-wires it for H3 projects. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5
parent
cd6639e7a7
commit
fdf39bbd03
@@ -2,6 +2,29 @@
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
### Fixed
|
||||
- **H3 image conditioning actually reaches the model.** Two independent breaks, both silent,
|
||||
both ending with a generation that ignored the input image entirely and left the prompt to
|
||||
carry the whole shot. The Movie Editor always splits a run into per-scene conditionings,
|
||||
and that split re-encoded every scene *without* the anchor image or the ref2va references —
|
||||
so neither the keyframe pin nor the `<Picture i>` presentation ever reached the sampler,
|
||||
even for a single-scene project. The split now carries the anchor onto the opening scene
|
||||
and the references onto every scene. LTX projects are untouched: there the anchor travels
|
||||
in the latent, and adding vision to scene 1 would change existing output.
|
||||
- **A wired `MiniMax H3 Image to Video` node no longer throws its image away.** That node
|
||||
carries its first/last frame pins on its CONDITIONING output, which this pipeline dropped
|
||||
because the sampler's positive comes from Studio. New optional **Chain Sampler ·
|
||||
h3_keyframes** input (auto-wired for H3 projects) salvages the pins — first frame onto the
|
||||
opening scene, last frame re-indexed onto the closing scene's own final frame.
|
||||
|
||||
### Changed
|
||||
- **Guide strengths span the full 0..1 range.** Mid-scene guide, identity pin, prior-scene
|
||||
guide, the per-guide strength field, and the JoyAI memory floor were clamped to 0.25–0.5.
|
||||
That band is the measured audio-safe sweet spot, not a physical limit; it stays in the
|
||||
tooltips and out of the code.
|
||||
- The H3 `v2a_grad_scale` warning chip is gone. The knob does nothing without JoyAI audio
|
||||
memory on any model, so flagging it as an H3 limitation blamed the wrong thing.
|
||||
|
||||
## [3.5.0] - 2026-08-04
|
||||
|
||||
A second model family. FunPack was built around LTX-2 / LTXAV and now supports MiniMax H3
|
||||
|
||||
+34
-2
@@ -6938,7 +6938,25 @@ class FunPackVideoRefinerV2(FunPackVideoRefiner):
|
||||
scene_effects.append(effect)
|
||||
return scene_texts, scene_effects
|
||||
|
||||
def _v2_transition_scene_conditionings(self, clip, scene_texts, scene_effects=None, encode_cache=None):
|
||||
def _v2_transition_scene_conditionings(self, clip, scene_texts, scene_effects=None, encode_cache=None,
|
||||
reference_image=None, h3_references=None):
|
||||
"""Per-scene conditionings for the Chain Sampler, one entry per scene.
|
||||
|
||||
These REPLACE the single-entry conditioning built above, so anything that lived on
|
||||
that entry has to be re-established here or it never reaches the sampler. That is
|
||||
why the visual conditioning travels in:
|
||||
|
||||
* ``reference_image`` — the i2v anchor, H3 only. Scene 0 only, matching how the chain
|
||||
reads it (later scenes continue from the previous scene's output unless they carry
|
||||
their own anchor). It has to be re-encoded, not copied: on H3 the anchor is presented
|
||||
to Qwen inside the tokenize call, so a scene encoded without it is textually blind to
|
||||
the image no matter what metadata is attached afterwards. LTX is left alone — there
|
||||
the anchor reaches the model through the latent, and adding Gemma3 vision to scene 0
|
||||
would change what every existing multi-scene LTX project generates.
|
||||
* ``h3_references`` — ref2va reference media. EVERY scene: a reference identity holds
|
||||
across the whole chain, and the ``<Picture i>`` ordinals in a scene's prompt only
|
||||
resolve if that scene's own encode presented them.
|
||||
"""
|
||||
if clip is None:
|
||||
return None
|
||||
scene_texts = list(scene_texts or [])
|
||||
@@ -6946,6 +6964,12 @@ class FunPackVideoRefinerV2(FunPackVideoRefiner):
|
||||
return None
|
||||
scene_effects = list(scene_effects or [])
|
||||
|
||||
try:
|
||||
from . import minimax_h3 as h3mod
|
||||
except ImportError:
|
||||
import minimax_h3 as h3mod
|
||||
anchor_image = reference_image if h3mod.is_h3_clip(clip) else None
|
||||
|
||||
conditionings = []
|
||||
scene_count = len(scene_texts)
|
||||
seen_texts = {}
|
||||
@@ -6953,7 +6977,11 @@ class FunPackVideoRefinerV2(FunPackVideoRefiner):
|
||||
seen_count = seen_texts.get(scene_text, 0)
|
||||
seen_texts[scene_text] = seen_count + 1
|
||||
encode_text = f"Returning to an earlier scene: {scene_text}" if seen_count > 0 else scene_text
|
||||
cond, meta, _ = self._v2_encode_prompt(clip, encode_text, encode_cache=encode_cache)
|
||||
scene_image = anchor_image if scene_index == 0 else None
|
||||
cond, meta, _ = self._v2_encode_prompt(
|
||||
clip, encode_text, encode_cache=encode_cache,
|
||||
reference_image=scene_image, h3_references=h3_references,
|
||||
)
|
||||
if not isinstance(cond, torch.Tensor):
|
||||
return None
|
||||
scene_meta = dict(meta) if isinstance(meta, dict) else {"pooled_output": None}
|
||||
@@ -12866,6 +12894,10 @@ class FunPackVideoRefinerV2(FunPackVideoRefiner):
|
||||
if scene_texts:
|
||||
scene_conditionings = self._v2_transition_scene_conditionings(
|
||||
clip, scene_texts, scene_effects=split_scene_effects, encode_cache=encode_cache,
|
||||
# The per-scene entries replace `output_conditioning` wholesale, so H3's
|
||||
# visual conditioning has to be re-established here or the anchor image
|
||||
# and the ref2va references never reach the sampler at all.
|
||||
reference_image=source_image, h3_references=_h3_references,
|
||||
)
|
||||
if scene_conditionings is not None:
|
||||
output_conditioning = self._v2_apply_scene_seed_metadata(
|
||||
|
||||
+18
-2
@@ -681,8 +681,24 @@ without complaint: `minimax_h3_fl2va_*` for text-to-video and anchor/keyframe sc
|
||||
using both at once in one scene means one of them is untrained conditioning. Text encoder
|
||||
and both VAEs are shared. Needs ComfyUI v0.30.0 or newer.
|
||||
|
||||
**Inert on H3:** Best-FaceID identity transfer, Bounded Attention, segmented detailing, the
|
||||
and the second pass's latent op all depend on LTX transformer structure H3 does not have.
|
||||
**Getting an image into an H3 scene — two routes, both supported:**
|
||||
|
||||
1. **Timeline image → `Studio · source_image`** (the default wiring for an image_processing
|
||||
loader). Studio presents it to Qwen and hands the pixels to the Chain Sampler, which pins
|
||||
it as the opening scene's frame-0 keyframe. This is the route the Editor's own image
|
||||
scenes use.
|
||||
2. **A `MiniMax H3 Image to Video` node's `first_frame` / `last_frame`.** That node emits its
|
||||
pins on its **CONDITIONING** output — which used to go nowhere, because the sampler's
|
||||
positive comes from Studio, so the image was silently discarded. Wire that output to
|
||||
**Chain Sampler · h3_keyframes** (auto-wired for new projects) and the pins survive: a
|
||||
first_frame lands on scene 1, a last_frame on the last scene's final frame. Only the pins
|
||||
are read — write the prompt in Studio as usual.
|
||||
|
||||
The run prints `h3_keyframe_anchor(...)` or `h3_wired_keyframes(...)` in its scene mechanics.
|
||||
If neither appears, the model never saw the image.
|
||||
|
||||
**Inert on H3:** Best-FaceID identity transfer, Bounded Attention, segmented detailing, and
|
||||
the second pass's latent op all depend on LTX transformer structure H3 does not have.
|
||||
The Chain Sampler switches them off and says why; the main window warns before the run.
|
||||
|
||||
**Wrong:** Install stuck → **Cancel install**. Pip security policy errors → install packs manually in terminal. After Manager install, restart before pack install.
|
||||
|
||||
@@ -136,6 +136,10 @@ FAMILIES: dict[str, dict] = {
|
||||
("audiodec", "vae", "VAE", True),
|
||||
# optional: only needed to encode AUDIO ref2va references
|
||||
("sampler", "audio_vae", "VAE", False),
|
||||
# optional: a MiniMax H3 Image to Video node emits its first/last frame pins
|
||||
# on its CONDITIONING output, which this pipeline otherwise drops (the
|
||||
# sampler's positive comes from Studio). Wiring it here keeps the pins.
|
||||
("sampler", "h3_keyframes", "CONDITIONING", False),
|
||||
),
|
||||
},
|
||||
},
|
||||
|
||||
@@ -103,6 +103,10 @@ PORT_TO_OPEN_CORE: dict[str, tuple[str, str]] = {
|
||||
#
|
||||
# Everything not listed is inherited, so a rule added for LTXAV reaches H3 too.
|
||||
_H3_LATENT_PORT = "FunPackLTXAVSceneChainSampler.latent_template"
|
||||
# A MiniMax H3 Image to Video node in the latent role also emits CONDITIONING carrying its
|
||||
# first_frame / last_frame pins. Studio owns the sampler's positive, so that conditioning has
|
||||
# nowhere to go and the image is silently lost; this port is where the pins are salvaged.
|
||||
_H3_KEYFRAME_PORT = "FunPackLTXAVSceneChainSampler.h3_keyframes"
|
||||
|
||||
FAMILY_WIRING: dict[str, dict] = {
|
||||
"ltxav": {},
|
||||
@@ -111,19 +115,27 @@ FAMILY_WIRING: dict[str, dict] = {
|
||||
"audio_vae": [("VAE", None, "VAEDecodeAudio.vae"),
|
||||
("VAE", None, "FunPackLTXAVSceneChainSampler.audio_vae")],
|
||||
"audio_encoder": [],
|
||||
"empty_latent": [("LATENT", None, _H3_LATENT_PORT)],
|
||||
"video_latent": [("LATENT", None, _H3_LATENT_PORT)],
|
||||
"empty_latent": [("LATENT", None, _H3_LATENT_PORT),
|
||||
("CONDITIONING", None, _H3_KEYFRAME_PORT)],
|
||||
"video_latent": [("LATENT", None, _H3_LATENT_PORT),
|
||||
("CONDITIONING", None, _H3_KEYFRAME_PORT)],
|
||||
"image_processing": [("IMAGE", None, "FunPackStudio.source_image")],
|
||||
},
|
||||
"type_chain_terminals": {"LATENT": [_H3_LATENT_PORT]},
|
||||
"type_chain_terminals": {"LATENT": [_H3_LATENT_PORT],
|
||||
"CONDITIONING": [_H3_KEYFRAME_PORT]},
|
||||
"default_wires": {
|
||||
"audio_vae": {"VAE": "port:VAEDecodeAudio.vae"},
|
||||
"audio_encoder": {},
|
||||
"empty_latent": {"LATENT": "port:" + _H3_LATENT_PORT},
|
||||
"video_latent": {"LATENT": "port:" + _H3_LATENT_PORT},
|
||||
# both outputs of the H3 latent node are wired by default: the AV latent, and the
|
||||
# keyframe pins that come with it when the node has a first/last frame image
|
||||
"empty_latent": {"LATENT": "port:" + _H3_LATENT_PORT,
|
||||
"CONDITIONING": "port:" + _H3_KEYFRAME_PORT},
|
||||
"video_latent": {"LATENT": "port:" + _H3_LATENT_PORT,
|
||||
"CONDITIONING": "port:" + _H3_KEYFRAME_PORT},
|
||||
},
|
||||
"port_labels": {
|
||||
_H3_LATENT_PORT: "Chain Sampler · latent_template (H3 AV latent)",
|
||||
_H3_KEYFRAME_PORT: "Chain Sampler · h3_keyframes (first/last frame pins)",
|
||||
"VAEDecodeAudio.vae": "VAE Decode Audio · vae",
|
||||
"FunPackLTXAVSceneChainSampler.audio_vae": "Chain Sampler · audio_vae (ref2va audio)",
|
||||
},
|
||||
@@ -134,6 +146,7 @@ FAMILY_WIRING: dict[str, dict] = {
|
||||
"FunPackStudio.source_image": ("studio", "source_image"),
|
||||
"FunPackLTXAVSceneChainSampler.vae": ("sampler", "vae"),
|
||||
_H3_LATENT_PORT: ("sampler", "latent_template"),
|
||||
_H3_KEYFRAME_PORT: ("sampler", "h3_keyframes"),
|
||||
"FunPackLTXAVSceneChainSampler.audio_vae": ("sampler", "audio_vae"),
|
||||
"VAEDecodeAudio.vae": ("audiodec", "vae"),
|
||||
},
|
||||
|
||||
+72
-1
@@ -2351,6 +2351,9 @@ class FunPackLTXAVSceneChainSampler:
|
||||
"audio_vae": ("VAE", {
|
||||
"tooltip": "MiniMax H3 only: the audio VAE, needed to encode AUDIO reference media for ref2va (a voice or ambience clip the generation should sound like). Studio lists the references and bakes their <Audio j> labels into the prompt; this VAE turns them into the latent blocks the DiT packs. Image references need only the main vae and work without this. Ignored entirely on LTX.",
|
||||
}),
|
||||
"h3_keyframes": ("CONDITIONING", {
|
||||
"tooltip": "MiniMax H3 only: the CONDITIONING output of a MiniMax H3 Image to Video node, wired here purely so its first_frame / last_frame pins survive. That node's conditioning is otherwise discarded (the sampler's positive comes from Studio), which silently drops the image. Only the keyframe pins are read — the prompt encoded by that node is ignored, so write your prompt in Studio as usual. A first-frame pin lands on scene 1, a last-frame pin on the last scene's final frame. Ignored entirely on LTX.",
|
||||
}),
|
||||
"refinement_key_input": ("STRING", {
|
||||
"default": "",
|
||||
"multiline": False,
|
||||
@@ -4267,6 +4270,59 @@ class FunPackLTXAVSceneChainSampler:
|
||||
aug=None if aug >= 1.0 else aug)
|
||||
return positive, True
|
||||
|
||||
def _h3_external_pins(self, conditioning):
|
||||
"""Rescue the keyframe pins from a MiniMax H3 Image to Video node's conditioning.
|
||||
|
||||
That node encodes first_frame/last_frame into `minimax_keyframes` and hands them out
|
||||
on its CONDITIONING output — which this pipeline drops, because the sampler's positive
|
||||
comes from Studio. Wiring the node's conditioning into `h3_keyframes` lets the pins
|
||||
travel anyway. Only the pins are taken; the node's own prompt encode is not used.
|
||||
|
||||
Returns ``{"first": [...], "last": [...], "aug": float|None}``. The node writes a
|
||||
first_frame at index 0 and a last_frame at ITS frame_count - 1, which is a per-node
|
||||
length that need not match this run's scene length — so pins are classified here and
|
||||
re-indexed against the scene they land on, never trusted as absolute positions.
|
||||
"""
|
||||
pins = self._conditioning_value(conditioning, "minimax_keyframes") or []
|
||||
if not pins:
|
||||
return None
|
||||
source_count = self._conditioning_value(conditioning, "minimax_frame_count")
|
||||
first, last = [], []
|
||||
for pin in pins:
|
||||
try:
|
||||
index = int(pin["resolved_frame_index"])
|
||||
except (KeyError, TypeError, ValueError):
|
||||
continue
|
||||
(first if index == 0 else last).append(pin)
|
||||
if not first and not last:
|
||||
return None
|
||||
aug = self._conditioning_value(conditioning, "minimax_visual_cond_noise_aug")
|
||||
return {
|
||||
"first": first,
|
||||
"last": last,
|
||||
"aug": float(aug) if aug is not None else None,
|
||||
"source_count": int(source_count) if source_count else None,
|
||||
}
|
||||
|
||||
def _apply_h3_external_pins(self, positive, pins, scene_index, scene_count):
|
||||
"""Place rescued pins on the scene they belong to: first on the opening scene, last
|
||||
on the closing one. A one-scene run gets both, which is the plain single-clip case.
|
||||
|
||||
Returns (positive, applied_labels)."""
|
||||
frame_count = max(1, int(self._h3_frame_count))
|
||||
placed, labels = [], []
|
||||
if scene_index == 0 and pins["first"]:
|
||||
placed.extend({**p, "resolved_frame_index": 0} for p in pins["first"])
|
||||
labels.append("first")
|
||||
if scene_index == scene_count - 1 and pins["last"]:
|
||||
# re-indexed onto THIS scene's last frame — the source node's frame_count is its
|
||||
# own, and a pin at a stale index would land mid-clip, where the layout refuses it
|
||||
placed.extend({**p, "resolved_frame_index": frame_count - 1} for p in pins["last"])
|
||||
labels.append("last")
|
||||
if not placed:
|
||||
return positive, []
|
||||
return self._h3_add_keyframes(positive, placed, frame_count, aug=pins["aug"]), labels
|
||||
|
||||
def _append_h3_keyframe(self, guide_frame, apply_at, strength, positive, negative):
|
||||
"""H3's equivalent of an LTX guide: a keyframe pin carried on the conditioning.
|
||||
|
||||
@@ -5490,7 +5546,7 @@ class FunPackLTXAVSceneChainSampler:
|
||||
context_window_freenoise=True, context_window_retain_first=False,
|
||||
cut_opening_frames=0,
|
||||
second_pass=False, second_pass_op="none", second_pass_sigmas=None,
|
||||
audio_vae=None,
|
||||
audio_vae=None, h3_keyframes=None,
|
||||
unique_id=None, prompt=None):
|
||||
if not isinstance(positive, list) or not positive:
|
||||
raise ValueError("positive conditioning must contain at least one scene entry.")
|
||||
@@ -5645,6 +5701,13 @@ class FunPackLTXAVSceneChainSampler:
|
||||
max_scene_count = max(1, int(max_scenes))
|
||||
scene_conditionings = positive[:max_scene_count]
|
||||
scene_count = len(scene_conditionings)
|
||||
# Keyframe pins wired in from a MiniMax H3 Image to Video node, read once for the run.
|
||||
_h3_wired_pins = self._h3_external_pins(h3_keyframes) if self._is_h3 else None
|
||||
if h3_keyframes and not self._is_h3:
|
||||
print("[FunPackSceneChain] h3_keyframes is wired but this model is not MiniMax H3 — ignored.")
|
||||
elif h3_keyframes and _h3_wired_pins is None:
|
||||
print("[FunPackSceneChain] h3_keyframes is wired but carries no keyframe pins — the "
|
||||
"source node had no first_frame/last_frame image.")
|
||||
time_scale = self._time_scale(vae)
|
||||
video_frames = self._validate_template_length(latent_template, num_frames_per_scene, time_scale, vae=vae)
|
||||
video_overlap = self._overlap_frames(latent_template, frame_overlap, time_scale, vae=vae)
|
||||
@@ -5916,6 +5979,14 @@ class FunPackLTXAVSceneChainSampler:
|
||||
scene_positive, chunk, vae, _anchor_image, _anchor_strength)
|
||||
if _pinned:
|
||||
run_mechanisms.append(f"h3_keyframe_anchor(strength={_anchor_strength:g})")
|
||||
# Pins rescued from a wired MiniMax H3 Image to Video node. Applied AFTER the
|
||||
# anchor so an explicitly wired first_frame wins the frame-0 slot — the user
|
||||
# drew that wire, the anchor is inferred from the timeline.
|
||||
if _h3_wired_pins:
|
||||
scene_positive, _pin_labels = self._apply_h3_external_pins(
|
||||
scene_positive, _h3_wired_pins, scene_index, scene_count)
|
||||
if _pin_labels:
|
||||
run_mechanisms.append(f"h3_wired_keyframes({'+'.join(_pin_labels)})")
|
||||
|
||||
# ref2va reference blocks. Applied after the chunk is final because the blocks are
|
||||
# sized against this scene's canvas, and to every scene branch (fresh / anchored /
|
||||
|
||||
@@ -479,3 +479,80 @@ def test_mixing_pins_and_references_is_called_out(capsys):
|
||||
}]])
|
||||
out = capsys.readouterr().out
|
||||
assert "BOTH" in out and "fl2va" in out and "ref2va" in out
|
||||
|
||||
|
||||
# ── pins rescued from a wired MiniMax H3 Image to Video node ─────────────────
|
||||
# That node hands its first_frame/last_frame pins out on its CONDITIONING output, which this
|
||||
# pipeline drops (Studio owns the sampler's positive). Wiring it into h3_keyframes is the
|
||||
# difference between the image conditioning the generation and being silently discarded.
|
||||
|
||||
def _wired_conditioning(indices, frame_count=124, aug=None):
|
||||
pins = [{"resolved_frame_index": i, "latent": torch.full((1, 24, 1, 48, 84), float(i))}
|
||||
for i in indices]
|
||||
meta = {"minimax_keyframes": pins, "minimax_frame_count": frame_count}
|
||||
if aug is not None:
|
||||
meta["minimax_visual_cond_noise_aug"] = aug
|
||||
return [[torch.zeros(1, 12, 5120), meta]]
|
||||
|
||||
|
||||
def test_wired_pins_are_split_into_first_and_last():
|
||||
node = h3_node()
|
||||
pins = node._h3_external_pins(_wired_conditioning([0, 123], aug=0.8))
|
||||
assert [p["resolved_frame_index"] for p in pins["first"]] == [0]
|
||||
assert [p["resolved_frame_index"] for p in pins["last"]] == [123]
|
||||
assert pins["aug"] == pytest.approx(0.8)
|
||||
assert pins["source_count"] == 124
|
||||
|
||||
|
||||
def test_conditioning_without_pins_is_reported_as_nothing_to_rescue():
|
||||
node = h3_node()
|
||||
assert node._h3_external_pins([[torch.zeros(1, 12, 5120), {}]]) is None
|
||||
assert node._h3_external_pins(None) is None
|
||||
|
||||
|
||||
def test_a_first_frame_pin_lands_on_the_opening_scene_only():
|
||||
node = h3_node(frame_count=124)
|
||||
pins = node._h3_external_pins(_wired_conditioning([0]))
|
||||
positive = [[torch.zeros(1, 12, 5120), {}]]
|
||||
|
||||
out, labels = node._apply_h3_external_pins(positive, pins, scene_index=0, scene_count=3)
|
||||
assert labels == ["first"]
|
||||
assert [kf["resolved_frame_index"] for kf in out[0][1]["minimax_keyframes"]] == [0]
|
||||
|
||||
out, labels = node._apply_h3_external_pins(positive, pins, scene_index=1, scene_count=3)
|
||||
assert labels == [] and out is positive # untouched mid-chain
|
||||
|
||||
|
||||
def test_a_last_frame_pin_is_reindexed_onto_the_closing_scenes_own_length():
|
||||
"""The source node's frame_count is its own — a stale index would land mid-clip, where
|
||||
PackedLayout refuses it."""
|
||||
node = h3_node(frame_count=209)
|
||||
pins = node._h3_external_pins(_wired_conditioning([0, 123], frame_count=124))
|
||||
|
||||
out, labels = node._apply_h3_external_pins(
|
||||
[[torch.zeros(1, 12, 5120), {}]], pins, scene_index=2, scene_count=3)
|
||||
assert labels == ["last"]
|
||||
assert [kf["resolved_frame_index"] for kf in out[0][1]["minimax_keyframes"]] == [208]
|
||||
|
||||
|
||||
def test_a_single_scene_run_gets_both_pins():
|
||||
node = h3_node(frame_count=124)
|
||||
pins = node._h3_external_pins(_wired_conditioning([0, 123]))
|
||||
out, labels = node._apply_h3_external_pins(
|
||||
[[torch.zeros(1, 12, 5120), {}]], pins, scene_index=0, scene_count=1)
|
||||
assert labels == ["first", "last"]
|
||||
assert [kf["resolved_frame_index"] for kf in out[0][1]["minimax_keyframes"]] == [0, 123]
|
||||
|
||||
|
||||
def test_a_wired_first_frame_overrides_the_timeline_anchor_on_frame_zero():
|
||||
"""Both target frame 0; the explicit wire is the one the user drew."""
|
||||
node = h3_node(frame_count=124)
|
||||
anchor = torch.zeros(1, 24, 1, 48, 84)
|
||||
positive = node._h3_add_keyframes(
|
||||
[[torch.zeros(1, 12, 5120), {}]],
|
||||
[{"resolved_frame_index": 0, "latent": anchor}], 124)
|
||||
|
||||
pins = node._h3_external_pins(_wired_conditioning([0]))
|
||||
out, _labels = node._apply_h3_external_pins(positive, pins, scene_index=0, scene_count=1)
|
||||
kfs = out[0][1]["minimax_keyframes"]
|
||||
assert len(kfs) == 1 and kfs[0]["latent"] is not anchor
|
||||
|
||||
@@ -113,3 +113,46 @@ def test_the_reference_size_mode_reaches_the_sampler(h3_media):
|
||||
H3Clip(), "a shot",
|
||||
h3_references=[{"kind": "image", "filename": "face.png", "size": "max"}])
|
||||
assert meta["funpack_h3_refs"][0]["size"] == "max"
|
||||
|
||||
|
||||
# ── the multi-scene split must not drop the visual conditioning ──────────────
|
||||
# The Movie Editor always runs split_by_transitions, and the per-scene conditionings REPLACE
|
||||
# the single entry Studio built. Anything that lived on that entry — the anchor, the resolved
|
||||
# references — has to be re-established per scene or it never reaches the sampler at all.
|
||||
|
||||
def test_the_anchor_survives_the_multi_scene_split_on_the_opening_scene():
|
||||
node = FunPackVideoRefinerV2()
|
||||
clip = H3Clip()
|
||||
scenes = node._v2_transition_scene_conditionings(
|
||||
clip, ["a room", "a street", "a roof"], reference_image=IMAGE)
|
||||
|
||||
assert [m["funpack_scene_index"] for _c, m in scenes] == [0, 1, 2]
|
||||
# scene 0 carries the pin AND was encoded with the image in front of Qwen
|
||||
assert scenes[0][1]["funpack_h3_anchor"]["image"] is IMAGE
|
||||
assert clip.tokenize_kwargs[0]["images"] == [IMAGE]
|
||||
# later scenes continue from the previous scene's output, so no anchor and no vision
|
||||
assert all("funpack_h3_anchor" not in m for _c, m in scenes[1:])
|
||||
assert all(not kw.get("images") for kw in clip.tokenize_kwargs[1:])
|
||||
|
||||
|
||||
def test_references_survive_the_split_on_every_scene(h3_media):
|
||||
"""A reference identity holds across the whole chain, and <Picture 1> only resolves in a
|
||||
scene whose own encode presented it."""
|
||||
node = FunPackVideoRefinerV2()
|
||||
clip = H3Clip()
|
||||
scenes = node._v2_transition_scene_conditionings(
|
||||
clip, ["a room", "a street"],
|
||||
h3_references=[{"kind": "image", "filename": "face.png"}])
|
||||
|
||||
assert all([r["filename"] for r in m["funpack_h3_refs"]] == ["face.png"] for _c, m in scenes)
|
||||
assert all(kw.get("minimax_ref_items") for kw in clip.tokenize_kwargs)
|
||||
|
||||
|
||||
def test_an_ltx_split_is_left_alone():
|
||||
"""On LTX the anchor reaches the model through the latent. Adding Gemma3 vision to scene 0
|
||||
here would change what every existing multi-scene LTX project generates."""
|
||||
node = FunPackVideoRefinerV2()
|
||||
node._gemma3_has_vision = lambda clip: True
|
||||
scenes = node._v2_transition_scene_conditionings(LTXClip(), ["a room", "a street"],
|
||||
reference_image=IMAGE)
|
||||
assert all("funpack_h3_anchor" not in m for _c, m in scenes)
|
||||
|
||||
Reference in New Issue
Block a user