Prompt Markup strips it before the encoder; Apply Prompt Markup puts weights
and timed windows on attention and blends on the conditioning. Where the words
sit is the H3 module's (text_tokenizer, prompt_rows, video_time_rows). Wired
into both H3 presets.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
H3's latent upscaler loads through the H3 module (ComfyUI's loader has no
branch for it); an upscale resizes H3's anchor pin so pass 2 stays pinned.
FunPack Sampler takes an optional wired schedule, run exactly as written.
New preset: H3 Reference to Video + second pass.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The H3 module offers anchor_pin, so ALG blurs the pin's latent without
learning H3's payload layout.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Block repeat's video-only mode now really keeps audio single-pass: H3 blocks
write in place, so v4 restored audio from an already-repeated stream.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The port plan calls for every module under modules/ to declare a
CATEGORY so Settings/Engine panels can group by it. Only alg, sharpen,
appearance, and full_control had one; the other 12 (loaders, empty
latent, media load, minimax_h3, output decode, sampling modifiers/
sampler, system pipeline) are plain infrastructure with no user-facing
settings dial, so they land in "system" except output_decode (post,
matching its STAGE) and the two sampling modules (sampling). Verified
against core/tests (548 passed) -- category validation and the
registry's field-mapping catch a typo here, per the earlier fixture-gap
fix.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
keep_proportion/extra_padding/invert_crop (ImageTransformKJ) and format
(SaveVideo) were wrapped in a redundant {"field": "value"} dict instead of
the plain string ComfyUI's own DynamicCombo expansion expects at the wire
level -- every one of them silently failed graph validation and fell back
to the node's own default, undermining the "total_pixels" resolution-fit
behavior this preset exists for. Found by loading the preset in a real
browser against real node schemas; the dev server never had comfy_extras
or KJNodes registered before now, so this was invisible locally until
tools/devserver.py also gained real node registration (init_extra_nodes +
KJNodes' load_custom_node, matching what the pytest `registered` fixture
already did).
The two pipeline_preset tests that were supposed to catch this had too
broad a filter -- it excluded ANY "is not one of" message as an assumed
"combo not chosen yet" state, which in this codebase actually shows up as
"needs X and nothing fills it" instead. Tightened to exempt only the three
loader-file combos genuinely left unset on a fresh install.
Verified live: preset loads with zero combo-validation problems, only the
three expected unset-loader-file messages remain, and Generate correctly
refuses with an honest reason rather than a false success.
Pieces B and C: Scene gains source_image (aspect-ratio reference only --
resolution itself stays a project.video setting, ImageTransformKJ's
"total_pixels" keep_proportion fits the picture's aspect into that budget)
and references (media ids, in order). Inspector's Scene tab gets pick/clear/
remove controls; picking routes through the existing Media bin via a small
pending-pick handshake in boot.js (one bin, one onPick, so "which field
asked" has to be tracked between the click that asked and the click that
answered).
References wire into MiniMaxH3ReferenceToVideo's ref_images autogrow input
dynamically, per run, only for what a scene actually has -- an unwired slot,
not an empty image, is what "not in use" means to that node. The target is
data on the role itself (wireTo), not a slot id boot.js would otherwise have
to know, keeping the "core never names an implementation" contract intact
one layer up into the app.
Verifying this exposed a second real core gap (after the MatchType one):
core/graph.py had no idea what an Autogrow (V3's io.Autogrow) input even
is -- "ref_images" reported as one opaque COMFY_AUTOGROW_V3 slot, its real
per-instance sockets (ref_image_0, ref_image_1, ...) invisible to
declared()/build(), so a wire naming one was refused as an input the node
does not have. Fixed generically in comfy_types.py (autogrow_instances())
and graph.py, confirmed against ComfyUI's own _io.py (finalize_prefix's
dotted join, build_nested_inputs's runtime un-nesting) -- not guessed, and
proven end to end with a real override()+build() test using the exact edit
boot.js makes.
Piece A of the H3 workflow port: the loader/SigmaShift/ImageTransformKJ/
R2V/sampler-at-CFG-1/decode/video-mux chain the H3 user runs by hand today,
offered as a preset (GET /api/pipeline/presets) loadable into the same
editable pipeline core/graph.py already supports -- not a second "default",
which module-load order would make ambiguous.
References are deliberately NOT pre-wired into R2V's ref_images autogrow;
an unwired optional slot is what "not in use" means to that node (absent
from its ref_images dict), not an empty image degrading its <Picture i>
numbering. boot.js wires them per-run based on what's actually picked --
next commit.
Building this against a real ComfyUI+KJNodes registration (not a fake
schema) surfaced a genuine core bug: core/graph.py's from_comfyui() had no
idea what to do with a V3 MatchType socket (ImageTransformKJ's own image
input) -- its bare type name "COMFY_MATCHTYPE_V3" matched nothing, so a
completely legal IMAGE wire into it was refused. Fixed generically in
comfy_types.py (match_type_union()) and graph.py, for any node using
MatchType, not special-cased to this one.
Every node's own OUTPUTS were invisible on its own panel in the pipeline
editor -- wiring an input already let you pick a source, but opening the
node that PRODUCES something told you nothing about what it makes or
whether anything reads it. Found live, by a real user just looking at a
node, after two full review rounds that only ever checked the consuming
side. Fixed with a read-only "Outputs" section per node, naming each
output, its type, and who (if anyone) currently wires from it.
Also: model family detection. core/probe.py reads a safetensors header
(tensor names only, no weights) and asks every installed model module,
through the same capability-lookup mechanism already used for
empty_latent/decode, whether the file is its own -- core never names a
family itself. A real, adversarially-found bug in the ported signature:
one of MiniMax H3's three candidate prefixes doesn't correspond to any
prefix ComfyUI's own loader actually resolves to, traced through
comfy/model_detection.py and comfy/sd.py -- the probe could have called a
file H3 that ComfyUI's real loader would refuse. Removed, and pinned with
a dedicated test module. A second finding narrowed the folder search to
only the folders FunPack's own loaders actually read from.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Decode now takes the MODEL, so a provider answers "is this mine" by identity
instead of by shape. H3 was claiming any latent that happened to have two
parts -- right today because it is the only thing producing one, wrong the
moment a second AV model exists, and wrong in the way that reads as an
unrelated fault rather than an error. It declines when the model is not
wired, and the node says that is what is missing.
That completes the shape a model module should have: one folder saying how
this model behaves at each stage -- what it is, how its latent is built, how
its output is decoded -- instead of that knowledge spreading through the
conditioning and sampling code as special cases.
Added a modifier that runs on SD1.5, because a feature nobody can watch is a
feature nobody can check. It declares no traits, so it applies everywhere,
which is the other half of the rule ALG demonstrates by narrowing. Measured
on this machine at a fixed seed: off and on differ, 0.4 gives a visibly
richer picture that still reads correctly, and 1.2 blows the highlights --
so the range is -1..1, set from what was measured rather than from a round
number. A slider whose top half destroys the image lies about its range.
Two things found while building it. ComfyUI has a real circular import
between nodes_latent and nodes_post_processing that resolves only because
init_extra_nodes reaches one first; a custom node has to break it the same
way. And the accumulation bug returned through a third door: stripping
handled keyed wrappers but not the anonymous function lists on
model_options, so chaining doubled the effect. Tagging is now generic -- it
looks for our mark rather than for a list of hook names that would need
keeping current.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Core already turns images into a video and writes the file, and handles
containers, codecs and muxing properly. Writing our own would be a worse
copy of tested code, so there is no FunPack save node and no reason for VHS
either.
What core cannot generalise is decoding a latent that is not one tensor: an
AV latent has two branches wanting different VAEs, and their arrangement is
the model's business. Same rule as the empty latent -- a model module may
claim it, and H3 now does. A nested latent nobody claims is refused rather
than handed to vae.decode, which would raise somewhere confusing or quietly
decode one branch as the picture. A missing audio VAE is named, because
returning no audio looks exactly like a model that makes none.
With this the pipeline closes end to end using FunPack's eight nodes and
core's samplers and video nodes -- verified by loading both together. That
matters more than the node count: until now nothing FunPack produced could
be measured, only reasoned about.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Derived from latent_format where that is possible, which is every
single-tensor model: SDXL, Wan21 and LTXV all come out matching the numbers
core's own nodes compute, including models that ship after this.
H3 cannot be derived and is the reason modules exist. Its latent is a
NestedTensor of a video and an audio branch, its video channel count is not
the latent_channels its format reports, and its length snaps to a frame
grid. So modules/models/minimax_h3 claims it and builds it, importing the
grid from ComfyUI rather than copying a second definition of it.
Nothing outside that folder names a model, and nothing here assumes video:
a 2-D latent format yields an image latent with no branch anywhere.
Modules reach core through modules/_core.py. ComfyUI imports a pack by file
location, so an absolute `import core` in a module resolves under pytest and
not under ComfyUI -- the node then never registers, silently. That cost the
same bug twice, so there is now a test that loads the pack the way ComfyUI
does, in an isolated subprocess whose cwd is NOT the repo. The first version
of that test passed with the bug reintroduced, because cwd was on sys.path.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
