Prompt Markup strips it before the encoder; Apply Prompt Markup puts weights
and timed windows on attention and blends on the conditioning. Where the words
sit is the H3 module's (text_tokenizer, prompt_rows, video_time_rows). Wired
into both H3 presets.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Both replace what a block allocates, which crashes aimdo's compiler (seen on a
rental with v4's shadow negative). Attached to the model, so any sampler.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
H3's latent upscaler loads through the H3 module (ComfyUI's loader has no
branch for it); an upscale resizes H3's anchor pin so pass 2 stays pinned.
FunPack Sampler takes an optional wired schedule, run exactly as written.
New preset: H3 Reference to Video + second pass.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The H3 module offers anchor_pin, so ALG blurs the pin's latent without
learning H3's payload layout.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
core.patching.strip now also removes our object patches, so a feature switched
off does not keep its patch on a model that goes round again.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
v4 read the prompt length from transformer_options['c_crossattn'], which
ComfyUI never sets, so every block fell back to stock. It also stood down on
any i2v scene (per-token mod rows). Both fixed; tested on a real tiny H3.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Block repeat's video-only mode now really keeps audio single-pass: H3 blocks
write in place, so v4 restored audio from an already-repeated stream.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Round 2 of extensive_testing on 548d99a found no exploitable defect --
has_block's "contains, not is" semantics are shared with minimax_h3's
own equally consequential is_h3() gate, and nothing in this codebase
nests diffusion_model in a wrapper that could trigger a false positive
today. Recorded as a one-line limit rather than left implicit, per the
round's only (non-blocking) suggestion.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Round 1 of extensive_testing on 79ca8cf found is_h3_model() matched on
diffusion_model's bare class name ("MiniMaxH3Model"), inherited
unchanged from v4. This codebase's own core/traits.py.has_block()
documents exactly why that's unsafe for a name this generic: a
same-named class from an unrelated custom node or a future FunPack
model would be silently treated as H3 and sparsified at 0.90 with
H3-specific audio-prefix pinning -- the identical LTX-collision risk
this file's own docstring already warns about, just not guarded
against strongly enough. modules/models/minimax_h3 already qualifies
its own identical check (MODEL_CLASS = the dotted path); this port
carried over the private, weaker duplicate instead of reusing that
pattern.
Switched to core/traits.py's has_block() with the same qualified dotted
name, kept as a local string (not an import of modules/models/minimax_h3
itself, which pulls in comfy_extras at import time -- this file
deliberately stays importable without ComfyUI on the path). Added a
test exercising the exact collision the bare match would have missed:
a class named MiniMaxH3Model from an unrelated module.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Loader-level attention option: mean-pool Q and a smoothed K into
blocks, score them with one small matmul, attend only the top
1-sparsity fraction of key blocks per query block, via a vendored
Triton kernel. Runs alongside the chosen attention backend rather than
replacing it (one override slot; the backend becomes SLA's dense
fall-through for the text refiner, masked calls, and trailing dense
steps), and refuses cleanly -- naming why -- on anything that is not
MiniMax H3 or a machine without CUDA+Triton.
modules/loaders/sla_block_map.py and sla_kernel.py are near-verbatim
ports of v4's vendored files (no FunPack-specific imports, so nothing
to adapt); sla_attention.py is the loader-facing glue (H3 gate,
override composition, per-step wrapper), same content as v4's with a
"not a module" note added to match gguf_support.py's convention.
Wired into FunPackDiffusionModelLoader as six new optional widgets,
following v4's loaders.py wiring but expressed in v5's io.Schema style.
The Triton kernel itself is unexercisable on this Mac (no CUDA) --
matches how real generation is verified in this project (rented GPU,
not local). Ported the CPU-testable half of v4's test suite (the
override contract, the H3 gate, the per-step wrapper); the three
kernel-correctness tests carry the same CUDA+Triton skip guard v4 used.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The port plan calls for every module under modules/ to declare a
CATEGORY so Settings/Engine panels can group by it. Only alg, sharpen,
appearance, and full_control had one; the other 12 (loaders, empty
latent, media load, minimax_h3, output decode, sampling modifiers/
sampler, system pipeline) are plain infrastructure with no user-facing
settings dial, so they land in "system" except output_decode (post,
matching its STAGE) and the two sampling modules (sampling). Verified
against core/tests (548 passed) -- category validation and the
registry's field-mapping catch a typo here, per the earlier fixture-gap
fix.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Two pieces of the "port v4's exact UI onto v5's backend" plan:
- core/contract.py: new optional ModuleSpec.category field (closed set:
continuity, guidance, conditioning, sampling, post, system), validated in
core/schema.py, threaded through core/registry.py's FIELDS and
to_manifest(). Presentation grouping, independent of STAGE (execution
order) -- a module can be stage="sampling" and category="guidance" at
once. Only the 4 existing modules that actually render a settings panel
(alg, sharpen, appearance, full_control) got a category assigned; the
other 12 have none to categorize (no SETTINGS block) and stay
uncategorised, which is valid.
- v4's frontend (movie_editor/frontend/*, ~29k lines, 61 files) copied
verbatim into app/legacy/. core/routes.py's "/" route now serves
legacy/index.html instead of the composer entry point; a <base> tag in
that file resolves its own relative script/link tags against
/funpack/app/legacy/ regardless of what URL it's served at, so every
file moved as a unit with zero per-file path edits. Verified live: all
61 assets load 200 OK, the real v4 onboarding screen renders pixel-for-
pixel (it's the actual DOM/CSS, not a reimplementation), and several v4
API calls (/api/media, /api/projects, /api/git/status) already succeed
against v5's real backend by coincidence of shared route naming -- the
rest 404/405 as expected, since api.js's ~90 functions haven't been
rewired to v5's actual routes yet (next piece of this plan).
The old composer entry point (app/index.html, app/composer/**) stays in
the tree, unreferenced, until a later cleanup pass once nothing imports it.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
GGUF: port v4's gguf_support.py (backend selection, content-based .gguf
detection, dequantize fallback) and wire it into both the diffusion-model
and CLIP loaders.
Pipeline editor: reopening the Models-and-pipeline settings window was
silently discarding the whole edited pipeline (a loaded preset, every
loader file, every sampler setting) because GET /api/pipeline is
deliberately stateless server-side and boot.js never fed its own
already-saved slots back into a freshly remounted editor -- found live
while walking through the MiniMax-H3 setup flow. Fixed by re-validating
the saved slots through check() on reopen instead of always replaying
the raw stateless default.
Also fixes, found via the extensive_testing adversarial loop over both
of the above: a stale file-listing cache in the pipeline window (no way
to see a newly-added model file without a full page reload), and a
chain of races/leaks in its own refresh/commit error-banner handling
that surfaced across five review rounds -- a rejection from a superseded
request overwriting a fresher success, a failure banner never clearing
on success, banners bleeding across groups on navigation, and a
commit()'s refusal landing on the wrong view after the user backs out
before it answers.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
keep_proportion/extra_padding/invert_crop (ImageTransformKJ) and format
(SaveVideo) were wrapped in a redundant {"field": "value"} dict instead of
the plain string ComfyUI's own DynamicCombo expansion expects at the wire
level -- every one of them silently failed graph validation and fell back
to the node's own default, undermining the "total_pixels" resolution-fit
behavior this preset exists for. Found by loading the preset in a real
browser against real node schemas; the dev server never had comfy_extras
or KJNodes registered before now, so this was invisible locally until
tools/devserver.py also gained real node registration (init_extra_nodes +
KJNodes' load_custom_node, matching what the pytest `registered` fixture
already did).
The two pipeline_preset tests that were supposed to catch this had too
broad a filter -- it excluded ANY "is not one of" message as an assumed
"combo not chosen yet" state, which in this codebase actually shows up as
"needs X and nothing fills it" instead. Tightened to exempt only the three
loader-file combos genuinely left unset on a fresh install.
Verified live: preset loads with zero combo-validation problems, only the
three expected unset-loader-file messages remain, and Generate correctly
refuses with an honest reason rather than a false success.
Pieces B and C: Scene gains source_image (aspect-ratio reference only --
resolution itself stays a project.video setting, ImageTransformKJ's
"total_pixels" keep_proportion fits the picture's aspect into that budget)
and references (media ids, in order). Inspector's Scene tab gets pick/clear/
remove controls; picking routes through the existing Media bin via a small
pending-pick handshake in boot.js (one bin, one onPick, so "which field
asked" has to be tracked between the click that asked and the click that
answered).
References wire into MiniMaxH3ReferenceToVideo's ref_images autogrow input
dynamically, per run, only for what a scene actually has -- an unwired slot,
not an empty image, is what "not in use" means to that node. The target is
data on the role itself (wireTo), not a slot id boot.js would otherwise have
to know, keeping the "core never names an implementation" contract intact
one layer up into the app.
Verifying this exposed a second real core gap (after the MatchType one):
core/graph.py had no idea what an Autogrow (V3's io.Autogrow) input even
is -- "ref_images" reported as one opaque COMFY_AUTOGROW_V3 slot, its real
per-instance sockets (ref_image_0, ref_image_1, ...) invisible to
declared()/build(), so a wire naming one was refused as an input the node
does not have. Fixed generically in comfy_types.py (autogrow_instances())
and graph.py, confirmed against ComfyUI's own _io.py (finalize_prefix's
dotted join, build_nested_inputs's runtime un-nesting) -- not guessed, and
proven end to end with a real override()+build() test using the exact edit
boot.js makes.
Piece A of the H3 workflow port: the loader/SigmaShift/ImageTransformKJ/
R2V/sampler-at-CFG-1/decode/video-mux chain the H3 user runs by hand today,
offered as a preset (GET /api/pipeline/presets) loadable into the same
editable pipeline core/graph.py already supports -- not a second "default",
which module-load order would make ambiguous.
References are deliberately NOT pre-wired into R2V's ref_images autogrow;
an unwired optional slot is what "not in use" means to that node (absent
from its ref_images dict), not an empty image degrading its <Picture i>
numbering. boot.js wires them per-run based on what's actually picked --
next commit.
Building this against a real ComfyUI+KJNodes registration (not a fake
schema) surfaced a genuine core bug: core/graph.py's from_comfyui() had no
idea what to do with a V3 MatchType socket (ImageTransformKJ's own image
input) -- its bare type name "COMFY_MATCHTYPE_V3" matched nothing, so a
completely legal IMAGE wire into it was refused. Fixed generically in
comfy_types.py (match_type_union()) and graph.py, for any node using
MatchType, not special-cased to this one.
The bridge piece B and C (resolution source, reference images) both need:
Media bin picks have no way into a ComfyUI graph today. One node, media_id
in, IMAGE out, backed by core/media.py's existing store.
Also fixes a real .gitignore bug found along the way: a bare "media/"
pattern matched modules/media/ (this node's own source) as well as the
intended root-level user-data directory. Rooted both media/ and projects/.
Every node's own OUTPUTS were invisible on its own panel in the pipeline
editor -- wiring an input already let you pick a source, but opening the
node that PRODUCES something told you nothing about what it makes or
whether anything reads it. Found live, by a real user just looking at a
node, after two full review rounds that only ever checked the consuming
side. Fixed with a read-only "Outputs" section per node, naming each
output, its type, and who (if anyone) currently wires from it.
Also: model family detection. core/probe.py reads a safetensors header
(tensor names only, no weights) and asks every installed model module,
through the same capability-lookup mechanism already used for
empty_latent/decode, whether the file is its own -- core never names a
family itself. A real, adversarially-found bug in the ported signature:
one of MiniMax H3's three candidate prefixes doesn't correspond to any
prefix ComfyUI's own loader actually resolves to, traced through
comfy/model_detection.py and comfy/sd.py -- the probe could have called a
file H3 that ComfyUI's real loader would refuse. Removed, and pinned with
a dedicated test module. A second finding narrowed the folder search to
only the folders FunPack's own loaders actually read from.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Most FunPack pipelines run at CFG 1, where a negative prompt does
nothing -- it was one more box beside the prompt every Generate reads,
for a field only worth touching occasionally. The reference pipeline's
negative role now targets "project.negative" instead of
"generation.prompt"; the inspector's Project tab hosts it, in a STABLE
node outside the part draw() tears down and rebuilds on every scene
selection, so a field mounted there survives normal use.
Steps, Sampler and Scheduler are on the main window -- the three v4 buried
several clicks deep in Engine Settings. No new UI: the sampler slot declares
roles the way the prompt and the size already do.
What made that nearly free is that the controls now come from the NODE's own
description, through the settingFor/rendererFor pair the pipeline window has
always used. So a combo is a dropdown of the real choices and a number obeys
the real bounds, and prompt.js lost both of its hand-rolled branches: it no
longer decides what a control looks like, it asks.
File: New project, and every project listed with the open one ticked. The
store has been able to do this since the day it arrived and there was no way
in -- 20 lines of menu.
Three faults found on the way, all in the shared input helper:
* ENTER COULD NOT BE TYPED IN A TEXTAREA. `wire()` bound Enter to commit for
every text control, textareas included, so a multi-line value was
impossible -- the prompt above all, which is the longest multi-line thing
in the app. It has been that way in the pipeline window since it was
written; nobody had typed a second line into one.
* A control committed on blur alone, so autofill, a password manager and any
script setting a value went unheard. It listens to `change` as well now,
and commit() only acts on a real difference so the pair costs nothing.
* A control commits on blur, which means text still being typed is not in
the project yet -- and a tab closing is exactly when that is lost. boot
blurs before it saves.
Width, height and length belong to the PROJECT, not to a scene: a scene
cropped on the timeline and regenerated comes back at the project's size,
because the crop was a timeline decision and a regenerate is a new scene.
They sit in the Constructor beside the prompt -- the two things settled in
one sitting -- and travel to the graph on the path the prompt already uses.
No new mechanism: the pipeline declares roles at `project.video` the same
way it declares the prompt, so core still names no node and no setting. A
role whose value is a number gets a number control instead of a box; that
is the whole of the UI change. What the pipeline does not offer is not
shown, so a graph with no length input simply has no Length.
Bounds are the NODE's and stay there -- it already refuses a value outside
them with a reason, beside Generate. A project file is the one thing here
that outlives the code that wrote it, so everything read back out of one is
checked: a whole positive number, or gone.
Also, four faults the browser suite was already red with before this:
* Six tests still clicked a rail toggle that moved into the timeline head.
* At 420px the panel OVERLAYS the centre -- and the named toggles are in
the centre, so the panel covered the control that closes it. That is the
v4 trap this project forbids. They are not offered at that width now; the
View menu, which nothing can cover, is the way in and out.
* A 768px window laid out 860px wide and scrolled sideways. Root cause, and
it was never about one component: a flex item's automatic minimum is its
CONTENT, so every stack, split, panel body and frame region in the app
handed its content's width upward. A scroller that cannot shrink below
what it holds is not a scroller -- the strip of scenes made the WINDOW as
wide as the strip. Floored at the five places every layout routes through.
* A bin test counted `.cx-cell-glyph` across the whole page, so the timeline
drawing cells with the same glyph in them broke a test about the bin.
The layout was assembled by hand and read like it. Measured at 1440x900:
the three columns started at three different heights, a panel head was 41px
when it held a control and 34 when it did not, the gap left of the centre
was 7px and right of it 19, the properties column sat 11px further in than
the others inside a border that meant nothing, and five controls in one
list had four different widths -- a <select> takes the width of its widest
option, so the row listing checkpoint filenames was 124px wider than its
neighbours.
Now: --gutter is the space between regions, between two panels stacked in
a column, and inside the bands; the head band height is the panel head
height is the collapsible head height; --control-col gives every settings
control the same two edges. The frame's bands run edge to edge and the
content is inset by the gutter, so the app has an outline instead of a
margin that changed per region.
Also here, all of it visible in the same screenshot:
* The status sat against the Generate button with 1300px of empty bar to
its left -- everything was in one right-aligned group. What the app says
is at the start of the row now, what you press is at the end.
* A panel body is a column with a gap: two things put in it by two parts
of the app had whatever space their line boxes happened to leave.
* A container query adds no specificity, so the narrow-panel rule written
above the rules it overrides lost -- and flex-basis in a column is
HEIGHT, which turned four settings into a panel two screens long.
* The theme you picked was thrown away on every load: the appearance
module applied its own declared default over the choice the page had
already applied from storage. It adopts the live value now, and a panel
is built after setup so its control shows what setup settled on.
The prompt was a slot input on a CLIPTextEncode, reachable only by opening
the pipeline window and finding the node. A slot can now say that one of
its inputs belongs on the app's surface -- a place and a label -- and the
shell offers a region with that name or does not, exactly as a module's
panel works. Core validates that a role is names and nothing else.
Values for those inputs travel addressed by slot rather than as a whole
pipeline, so the window that owns the structure and a box that owns one
value cannot disagree about the slots between them. An address that is
not there is refused: a prompt typed into a box for a removed slot has
gone nowhere, and a run that starts anyway is not the run that was asked
for.
Region stand-ins are empty states now, taken down once something mounts --
'modules appear here' left above the modules that appeared explained
nothing to nobody.
Every module panel was decoration. The values lived in the browser and
nothing sent them, so a modifier switched on was one the run never heard
about. They now travel with the pipeline on every check, and core places
them wherever a module says they belong -- announced, not known, so the
node that carries them is not privileged.
Settings with nowhere to go are said beside Generate rather than
swallowed, and the warning survives the run it is about: the status line
is redrawn by every message, so a warning put there was gone the moment
the run said Queued.
Also: a settings row in a docked panel squeezed its label to one word per
line, because the control keeps a 140px floor and took it out of the
label. Below the width where they fit side by side, they stack.
One wide modal, a card per group, and a group that opens into a node list
beside that node/'s parameters, saved a group at a time. Groups travel on the
slot, so a pipeline keeps its own arrangement and a new group is a slot
claiming a name nobody used.
Values are drafted until Save; adding, removing, replacing and moving a node
go to the server as they happen, because only it can say whether the result
still wires up.
An unset dropdown reads "not set" rather than showing the first file as though
it had been chosen -- v4 recorded that trap, and picking the entry the box
already shows fires no change, so the one click that looks like the fix does
nothing.
New: region.stack, a frame header, a modal footer that can be refilled.
Not yet: wiring. A socket says so where a control would be.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The wording fix a commit ago was written by hand in two files and the same
mislabel was still sitting in four others -- traits, chain, modifier install and
node schema all reported a thing that loaded, recognised its work and then
raised as one that "did not load", sending a reader after an import error that
never happened. Three hand-written strings would have drifted, so log.broke()
now says it once and every site calls it.
Two of the new tests were vacuous when first written -- "nothing said did not
load" is also true where nothing ran at all, and picking the first provider
found lands on one whose traits exclude the test's model. Both assert the line
is present before asserting the wrong line is not.
One finding, confirmed, plus four sites it did not name.
`log.failed` means "did not load", and both the latent and the decode paths used
it for a provider that loaded, claimed the model and then broke while working --
"did not load -- RuntimeError: ..." sends whoever reads the log looking for an
import error that never happened. The levels were separated for exactly this.
One finding, confirmed.
Two corrections to the commit before this one.
A rank-1 latent's length is in the NODE that builds it and never in the format:
core turns seconds into samples with a 44100 rate and a 2048 hop that
latent_format does not carry, and the same rank means tokens of geometry for
Hunyuan3D. Deriving `length` samples gave StableAudio a latent of 1 where core's
own default is 1024 -- three orders of magnitude short, reported as success.
Refused now, with the reason and the way out: a module for that model.
The sweep claimed to skip H3 and did not. H3 has a rank-3 latent, so it was
derived and passed at 32 channels for a video branch that has 24 -- a test
passing on a model it is not correct for. So the previous message's "produces
exactly what core's nodes compute for every single-tensor model" was wrong on
both counts, and the sweep now says what it actually proves: the derivation
agrees with what a model PUBLISHES, which is a floor and not a certificate.
Rejected from the same report: an absent model module letting H3 fall through to
the generic shape. Core cannot detect that without naming models, which is the
one thing it must not do; a module that fails to import is already reported as a
Warning, which is the level that can see it.
LTX and WAN needed no module: both publish everything the derivation reads, and
it produces exactly what EmptyLTXVLatentVideo and Wan22ImageToVideoLatent
compute, by a different route. Pinned, along with a sweep over all 101 configs
ComfyUI ships -- rank, channel count and no empty axis -- so a family that
arrives after this was written is covered or fails here rather than at the first
sampling step.
H3 remains the only model needing a module of its own, because its latent is not
a single tensor.
Detail declared no traits, which read as "works on everything" and meant "is
offered on everything and drops itself on half of them". Core's unsharp mask
pads in two dimensions and raises on anything with more, so on a model with a
time axis the toggle was present, switchable, and inert -- a knob that does
nothing, which is the fault this project has shipped once already. It now
declares spatial_latent and is absent where it cannot work.
The guard's regression case survives, because the guard is not about a module
being wrong about itself: a hook meets the tensor it was handed. The stub model
now announces a latent rank, without which every trait-declaring modifier was
filtered out and those tests passed by installing nothing at all.
Round seven, one finding, confirmed in part -- the maths is deliberately not
being fixed, the advertisement was the defect.
