latent: a one-axis latent cannot be derived, and the sweep said more than it checked
Two corrections to the commit before this one. A rank-1 latent's length is in the NODE that builds it and never in the format: core turns seconds into samples with a 44100 rate and a 2048 hop that latent_format does not carry, and the same rank means tokens of geometry for Hunyuan3D. Deriving `length` samples gave StableAudio a latent of 1 where core's own default is 1024 -- three orders of magnitude short, reported as success. Refused now, with the reason and the way out: a module for that model. The sweep claimed to skip H3 and did not. H3 has a rank-3 latent, so it was derived and passed at 32 channels for a video branch that has 24 -- a test passing on a model it is not correct for. So the previous message's "produces exactly what core's nodes compute for every single-tensor model" was wrong on both counts, and the sweep now says what it actually proves: the derivation agrees with what a model PUBLISHES, which is a floor and not a certificate. Rejected from the same report: an absent model module letting H3 fall through to the generic shape. Core cannot detect that without naming models, which is the one thing it must not do; a module that fails to import is already reported as a Warning, which is the level that can see it.
This commit is contained in:
@@ -8,8 +8,15 @@ Two paths, and the order matters:
|
||||
reports, and its length snaps to a model-specific frame grid. So a model module
|
||||
may provide `empty_latent` and build it itself.
|
||||
2. **Otherwise it is derived** from `latent_format`, which is correct for every
|
||||
single-tensor model: SDXL (4ch, /8), Wan21 (16ch, /8, /4) and LTXV (128ch,
|
||||
/32, /8) all come out right, including models that ship after this is written.
|
||||
single-tensor model with a spatial latent: SDXL (4ch, /8), Wan21 (16ch, /8,
|
||||
/4) and LTXV (128ch, /32, /8) all come out right, including models that ship
|
||||
after this is written.
|
||||
|
||||
A one-axis latent is neither: its length is not published anywhere, and the same
|
||||
rank means seconds of audio for StableAudio and tokens of geometry for
|
||||
Hunyuan3D. That is refused here rather than guessed, on the same reasoning as
|
||||
point 1 -- deriving it produced a latent a thousandth of the length asked for and
|
||||
reported success.
|
||||
|
||||
The provider protocol, and it matters: **return None for "not my model", and let
|
||||
anything else raise.** A provider decides whether it recognises the model FIRST,
|
||||
@@ -62,7 +69,17 @@ def derive(model, width, height, length, batch_size):
|
||||
frames = ((max(1, length) - 1) // temporal) + 1
|
||||
samples = torch.zeros([batch_size, channels, frames, h, w], device=device)
|
||||
elif rank == 1:
|
||||
samples = torch.zeros([batch_size, channels, max(1, length)], device=device)
|
||||
# Refused, not guessed. A one-axis latent's length is not derivable from
|
||||
# anything the format publishes: core's own EmptyLatentAudio turns
|
||||
# SECONDS into samples with a rate and a hop that live in the node
|
||||
# (44100 and 2048), not in latent_format, and EmptyLatentHunyuan3Dv2
|
||||
# counts tokens for a resolution -- the same rank meaning two unrelated
|
||||
# things. Building `length` samples produced a latent three orders of
|
||||
# magnitude short of a soundtrack and called it a success.
|
||||
raise RuntimeError(
|
||||
"This model's latent has a single axis whose length is not derivable "
|
||||
"from what the model publishes -- seconds, sample rate and hop are "
|
||||
"not in its latent format. A module for this model can supply it.")
|
||||
else:
|
||||
samples = torch.zeros([batch_size, channels, h, w], device=device)
|
||||
|
||||
@@ -87,7 +104,9 @@ class FunPackEmptyLatent(io.ComfyNode):
|
||||
io.Int.Input("width", default=768, min=16, max=16384, step=16),
|
||||
io.Int.Input("height", default=512, min=16, max=16384, step=16),
|
||||
io.Int.Input("length", default=1, min=1, max=16384,
|
||||
tooltip="Frames. Ignored by models whose latent has no time axis."),
|
||||
tooltip="Frames. Ignored by models whose latent has no "
|
||||
"time axis, and it is frames -- not seconds and "
|
||||
"not samples."),
|
||||
io.Int.Input("batch_size", default=1, min=1, max=4096, optional=True),
|
||||
],
|
||||
outputs=[
|
||||
|
||||
@@ -230,15 +230,16 @@ def test_wan22_matches_what_comfys_own_node_produces():
|
||||
assert tuple(ours["samples"].shape) == tuple(theirs.shape)
|
||||
|
||||
|
||||
def test_every_model_comfyui_ships_gets_a_latent_of_the_rank_it_asked_for():
|
||||
def test_every_spatial_model_comfyui_ships_gets_a_latent_matching_its_own_format():
|
||||
"""The whole catalogue, not a list somebody maintained.
|
||||
|
||||
A model family is supported here when its latent comes out right, and the
|
||||
derivation reads the format rather than knowing the family -- so the honest
|
||||
test is every config upstream ships, including the ones added after this was
|
||||
written. A family needing more than its format publishes has to say so with
|
||||
a module of its own; H3 is the only one that does, and it is skipped here
|
||||
because its latent is not a single tensor.
|
||||
What this proves and what it does not, said plainly. It proves the
|
||||
derivation agrees with what each model PUBLISHES, for every config upstream
|
||||
ships including ones added after this was written. It does not prove the
|
||||
published format is the whole truth: MiniMax H3 reports 32 channels for a
|
||||
video branch that has 24, and passes this sweep while being wrong. A model
|
||||
whose real latent differs from its format needs a module, and H3's own tests
|
||||
are where that is checked -- so passing here is a floor, not a certificate.
|
||||
"""
|
||||
import comfy.supported_models as sm
|
||||
from modules.latent.empty.nodes import derive
|
||||
@@ -249,7 +250,7 @@ def test_every_model_comfyui_ships_gets_a_latent_of_the_rank_it_asked_for():
|
||||
if fmt is None:
|
||||
continue
|
||||
rank = getattr(fmt, "latent_dimensions", 2)
|
||||
if rank not in (1, 2, 3):
|
||||
if rank not in (2, 3):
|
||||
continue
|
||||
out = derive(_Model(cfg), 832, 480, 81, 1)
|
||||
shape = tuple(out["samples"].shape)
|
||||
@@ -261,3 +262,27 @@ def test_every_model_comfyui_ships_gets_a_latent_of_the_rank_it_asked_for():
|
||||
checked += 1
|
||||
|
||||
assert checked > 80, f"only {checked} model configs were reached; upstream moved"
|
||||
|
||||
|
||||
def test_a_one_axis_latent_is_refused_rather_than_guessed():
|
||||
"""Every rank-1 family upstream ships, by name at the time of writing and by
|
||||
enumeration after it.
|
||||
|
||||
The length of a single-axis latent is in the NODE that builds it, never in
|
||||
the format: core turns seconds into samples with a 44100 rate and a 2048 hop
|
||||
that latent_format does not carry, and the same rank means tokens of geometry
|
||||
for Hunyuan3D. Deriving `length` samples gave a StableAudio latent of 1 where
|
||||
core's own default is 1024 -- three orders of magnitude short, reported as
|
||||
success, which is the shape of failure this node exists to refuse.
|
||||
"""
|
||||
import comfy.supported_models as sm
|
||||
from modules.latent.empty.nodes import derive
|
||||
|
||||
families = [cfg for cfg in sm.models
|
||||
if getattr(getattr(cfg, "latent_format", None), "latent_dimensions", 2) == 1]
|
||||
assert {c.__name__ for c in families} >= {"StableAudio", "ACEStep15", "Hunyuan3Dv2"}, (
|
||||
[c.__name__ for c in families])
|
||||
|
||||
for cfg in families:
|
||||
with pytest.raises(RuntimeError, match="not derivable"):
|
||||
derive(_Model(cfg), 832, 480, 81, 1)
|
||||
|
||||
Reference in New Issue
Block a user