Prompt Markup strips it before the encoder; Apply Prompt Markup puts weights
and timed windows on attention and blends on the conditioning. Where the words
sit is the H3 module's (text_tokenizer, prompt_rows, video_time_rows). Wired
into both H3 presets.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Both replace what a block allocates, which crashes aimdo's compiler (seen on a
rental with v4's shadow negative). Attached to the model, so any sampler.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
H3's latent upscaler loads through the H3 module (ComfyUI's loader has no
branch for it); an upscale resizes H3's anchor pin so pass 2 stays pinned.
FunPack Sampler takes an optional wired schedule, run exactly as written.
New preset: H3 Reference to Video + second pass.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The H3 module offers anchor_pin, so ALG blurs the pin's latent without
learning H3's payload layout.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
core.patching.strip now also removes our object patches, so a feature switched
off does not keep its patch on a model that goes round again.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
v4 read the prompt length from transformer_options['c_crossattn'], which
ComfyUI never sets, so every block fell back to stock. It also stood down on
any i2v scene (per-token mod rows). Both fixed; tested on a real tiny H3.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Block repeat's video-only mode now really keeps audio single-pass: H3 blocks
write in place, so v4 restored audio from an already-repeated stream.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Round 2 of extensive_testing on 548d99a found no exploitable defect --
has_block's "contains, not is" semantics are shared with minimax_h3's
own equally consequential is_h3() gate, and nothing in this codebase
nests diffusion_model in a wrapper that could trigger a false positive
today. Recorded as a one-line limit rather than left implicit, per the
round's only (non-blocking) suggestion.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Round 1 of extensive_testing on 79ca8cf found is_h3_model() matched on
diffusion_model's bare class name ("MiniMaxH3Model"), inherited
unchanged from v4. This codebase's own core/traits.py.has_block()
documents exactly why that's unsafe for a name this generic: a
same-named class from an unrelated custom node or a future FunPack
model would be silently treated as H3 and sparsified at 0.90 with
H3-specific audio-prefix pinning -- the identical LTX-collision risk
this file's own docstring already warns about, just not guarded
against strongly enough. modules/models/minimax_h3 already qualifies
its own identical check (MODEL_CLASS = the dotted path); this port
carried over the private, weaker duplicate instead of reusing that
pattern.
Switched to core/traits.py's has_block() with the same qualified dotted
name, kept as a local string (not an import of modules/models/minimax_h3
itself, which pulls in comfy_extras at import time -- this file
deliberately stays importable without ComfyUI on the path). Added a
test exercising the exact collision the bare match would have missed:
a class named MiniMaxH3Model from an unrelated module.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Loader-level attention option: mean-pool Q and a smoothed K into
blocks, score them with one small matmul, attend only the top
1-sparsity fraction of key blocks per query block, via a vendored
Triton kernel. Runs alongside the chosen attention backend rather than
replacing it (one override slot; the backend becomes SLA's dense
fall-through for the text refiner, masked calls, and trailing dense
steps), and refuses cleanly -- naming why -- on anything that is not
MiniMax H3 or a machine without CUDA+Triton.
modules/loaders/sla_block_map.py and sla_kernel.py are near-verbatim
ports of v4's vendored files (no FunPack-specific imports, so nothing
to adapt); sla_attention.py is the loader-facing glue (H3 gate,
override composition, per-step wrapper), same content as v4's with a
"not a module" note added to match gguf_support.py's convention.
Wired into FunPackDiffusionModelLoader as six new optional widgets,
following v4's loaders.py wiring but expressed in v5's io.Schema style.
The Triton kernel itself is unexercisable on this Mac (no CUDA) --
matches how real generation is verified in this project (rented GPU,
not local). Ported the CPU-testable half of v4's test suite (the
override contract, the H3 gate, the per-step wrapper); the three
kernel-correctness tests carry the same CUDA+Triton skip guard v4 used.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The port plan calls for every module under modules/ to declare a
CATEGORY so Settings/Engine panels can group by it. Only alg, sharpen,
appearance, and full_control had one; the other 12 (loaders, empty
latent, media load, minimax_h3, output decode, sampling modifiers/
sampler, system pipeline) are plain infrastructure with no user-facing
settings dial, so they land in "system" except output_decode (post,
matching its STAGE) and the two sampling modules (sampling). Verified
against core/tests (548 passed) -- category validation and the
registry's field-mapping catch a typo here, per the earlier fixture-gap
fix.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Two pieces of the "port v4's exact UI onto v5's backend" plan:
- core/contract.py: new optional ModuleSpec.category field (closed set:
continuity, guidance, conditioning, sampling, post, system), validated in
core/schema.py, threaded through core/registry.py's FIELDS and
to_manifest(). Presentation grouping, independent of STAGE (execution
order) -- a module can be stage="sampling" and category="guidance" at
once. Only the 4 existing modules that actually render a settings panel
(alg, sharpen, appearance, full_control) got a category assigned; the
other 12 have none to categorize (no SETTINGS block) and stay
uncategorised, which is valid.
- v4's frontend (movie_editor/frontend/*, ~29k lines, 61 files) copied
verbatim into app/legacy/. core/routes.py's "/" route now serves
legacy/index.html instead of the composer entry point; a <base> tag in
that file resolves its own relative script/link tags against
/funpack/app/legacy/ regardless of what URL it's served at, so every
file moved as a unit with zero per-file path edits. Verified live: all
61 assets load 200 OK, the real v4 onboarding screen renders pixel-for-
pixel (it's the actual DOM/CSS, not a reimplementation), and several v4
API calls (/api/media, /api/projects, /api/git/status) already succeed
against v5's real backend by coincidence of shared route naming -- the
rest 404/405 as expected, since api.js's ~90 functions haven't been
rewired to v5's actual routes yet (next piece of this plan).
The old composer entry point (app/index.html, app/composer/**) stays in
the tree, unreferenced, until a later cleanup pass once nothing imports it.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
GGUF: port v4's gguf_support.py (backend selection, content-based .gguf
detection, dequantize fallback) and wire it into both the diffusion-model
and CLIP loaders.
Pipeline editor: reopening the Models-and-pipeline settings window was
silently discarding the whole edited pipeline (a loaded preset, every
loader file, every sampler setting) because GET /api/pipeline is
deliberately stateless server-side and boot.js never fed its own
already-saved slots back into a freshly remounted editor -- found live
while walking through the MiniMax-H3 setup flow. Fixed by re-validating
the saved slots through check() on reopen instead of always replaying
the raw stateless default.
Also fixes, found via the extensive_testing adversarial loop over both
of the above: a stale file-listing cache in the pipeline window (no way
to see a newly-added model file without a full page reload), and a
chain of races/leaks in its own refresh/commit error-banner handling
that surfaced across five review rounds -- a rejection from a superseded
request overwriting a fresher success, a failure banner never clearing
on success, banners bleeding across groups on navigation, and a
commit()'s refusal landing on the wrong view after the user backs out
before it answers.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
keep_proportion/extra_padding/invert_crop (ImageTransformKJ) and format
(SaveVideo) were wrapped in a redundant {"field": "value"} dict instead of
the plain string ComfyUI's own DynamicCombo expansion expects at the wire
level -- every one of them silently failed graph validation and fell back
to the node's own default, undermining the "total_pixels" resolution-fit
behavior this preset exists for. Found by loading the preset in a real
browser against real node schemas; the dev server never had comfy_extras
or KJNodes registered before now, so this was invisible locally until
tools/devserver.py also gained real node registration (init_extra_nodes +
KJNodes' load_custom_node, matching what the pytest `registered` fixture
already did).
The two pipeline_preset tests that were supposed to catch this had too
broad a filter -- it excluded ANY "is not one of" message as an assumed
"combo not chosen yet" state, which in this codebase actually shows up as
"needs X and nothing fills it" instead. Tightened to exempt only the three
loader-file combos genuinely left unset on a fresh install.
Verified live: preset loads with zero combo-validation problems, only the
three expected unset-loader-file messages remain, and Generate correctly
refuses with an honest reason rather than a false success.
Pieces B and C: Scene gains source_image (aspect-ratio reference only --
resolution itself stays a project.video setting, ImageTransformKJ's
"total_pixels" keep_proportion fits the picture's aspect into that budget)
and references (media ids, in order). Inspector's Scene tab gets pick/clear/
remove controls; picking routes through the existing Media bin via a small
pending-pick handshake in boot.js (one bin, one onPick, so "which field
asked" has to be tracked between the click that asked and the click that
answered).
References wire into MiniMaxH3ReferenceToVideo's ref_images autogrow input
dynamically, per run, only for what a scene actually has -- an unwired slot,
not an empty image, is what "not in use" means to that node. The target is
data on the role itself (wireTo), not a slot id boot.js would otherwise have
to know, keeping the "core never names an implementation" contract intact
one layer up into the app.
Verifying this exposed a second real core gap (after the MatchType one):
core/graph.py had no idea what an Autogrow (V3's io.Autogrow) input even
is -- "ref_images" reported as one opaque COMFY_AUTOGROW_V3 slot, its real
per-instance sockets (ref_image_0, ref_image_1, ...) invisible to
declared()/build(), so a wire naming one was refused as an input the node
does not have. Fixed generically in comfy_types.py (autogrow_instances())
and graph.py, confirmed against ComfyUI's own _io.py (finalize_prefix's
dotted join, build_nested_inputs's runtime un-nesting) -- not guessed, and
proven end to end with a real override()+build() test using the exact edit
boot.js makes.
Piece A of the H3 workflow port: the loader/SigmaShift/ImageTransformKJ/
R2V/sampler-at-CFG-1/decode/video-mux chain the H3 user runs by hand today,
offered as a preset (GET /api/pipeline/presets) loadable into the same
editable pipeline core/graph.py already supports -- not a second "default",
which module-load order would make ambiguous.
References are deliberately NOT pre-wired into R2V's ref_images autogrow;
an unwired optional slot is what "not in use" means to that node (absent
from its ref_images dict), not an empty image degrading its <Picture i>
numbering. boot.js wires them per-run based on what's actually picked --
next commit.
Building this against a real ComfyUI+KJNodes registration (not a fake
schema) surfaced a genuine core bug: core/graph.py's from_comfyui() had no
idea what to do with a V3 MatchType socket (ImageTransformKJ's own image
input) -- its bare type name "COMFY_MATCHTYPE_V3" matched nothing, so a
completely legal IMAGE wire into it was refused. Fixed generically in
comfy_types.py (match_type_union()) and graph.py, for any node using
MatchType, not special-cased to this one.
The bridge piece B and C (resolution source, reference images) both need:
Media bin picks have no way into a ComfyUI graph today. One node, media_id
in, IMAGE out, backed by core/media.py's existing store.
Also fixes a real .gitignore bug found along the way: a bare "media/"
pattern matched modules/media/ (this node's own source) as well as the
intended root-level user-data directory. Rooted both media/ and projects/.
Every node's own OUTPUTS were invisible on its own panel in the pipeline
editor -- wiring an input already let you pick a source, but opening the
node that PRODUCES something told you nothing about what it makes or
whether anything reads it. Found live, by a real user just looking at a
node, after two full review rounds that only ever checked the consuming
side. Fixed with a read-only "Outputs" section per node, naming each
output, its type, and who (if anyone) currently wires from it.
Also: model family detection. core/probe.py reads a safetensors header
(tensor names only, no weights) and asks every installed model module,
through the same capability-lookup mechanism already used for
empty_latent/decode, whether the file is its own -- core never names a
family itself. A real, adversarially-found bug in the ported signature:
one of MiniMax H3's three candidate prefixes doesn't correspond to any
prefix ComfyUI's own loader actually resolves to, traced through
comfy/model_detection.py and comfy/sd.py -- the probe could have called a
file H3 that ComfyUI's real loader would refuse. Removed, and pinned with
a dedicated test module. A second finding narrowed the folder search to
only the folders FunPack's own loaders actually read from.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Most FunPack pipelines run at CFG 1, where a negative prompt does
nothing -- it was one more box beside the prompt every Generate reads,
for a field only worth touching occasionally. The reference pipeline's
negative role now targets "project.negative" instead of
"generation.prompt"; the inspector's Project tab hosts it, in a STABLE
node outside the part draw() tears down and rebuilds on every scene
selection, so a field mounted there survives normal use.
