One conditioning delta is 586x3584 floats — several megabytes of base64 inside
a document that is parsed and rewritten IN FULL on every generation. That is
the cost that grows with everything ever learned, and it is paid whether or not
any of those tensors are used.
Tensors now go to <key>.json.tensors.pt and the JSON keeps a named reference
whose id is the tensor's path in the tree ("state.global.liked_dir.direction"),
so the key stays readable and a reader can see what a reference points at.
No migration: a key written before this carries inline blobs, is read exactly
as it stands, and moves to the sidecar on its next save. If the sidecar cannot
be written the tensors stay inline — a slow key is fine, a JSON pointing at a
sidecar that does not exist is not. A reference the sidecar has lost reads as
no tensor, so a direction goes unapplied rather than breaking the run.
Deletion already sweeps it: delete_refinement_key globs every <key>.* sidecar
for exactly this reason. Session Reset removes it explicitly.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The key is read and rewritten IN FULL on every run and holds base64
conditioning tensors, so its cost grows with everything ever learned — which
is what "2.0s, then 2.4s" looks like from outside. Timing the read and the
write, with the file size and the entry counts of the collections that can
grow without bound, says whether that is where the time goes and which
collection to bound first.
Measurement only. Nothing about what is stored or how changes here.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A phrase's vector came from encoding that phrase alone — a full pass of the
text encoder per phrase, and on H3 that encoder is Qwen3-VL-32B. The phrases
are encoded together now, as one joined text, and each vector is the mean of
the rows its own tokens occupy; the tokenizer's offset mapping identifies those
rows exactly, so this is a lookup rather than an approximation.
Each phrase is located at the position it was joined at rather than by
searching the text, because one phrase can be a substring of another and a
search would hand both the same rows.
Falls back to one encode per phrase, unchanged, when the tokenizer offers no
offsets or the spans do not line up with the conditioning that came back. The
cosine step is shared by both paths so a phrase scores the same either way, and
the batched pass is labelled separately in the per-run encoder tally.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
CATEGORY_DESCRIPTIONS is a class attribute of eight fixed strings, so the
vectors depend only on the encoder. They were re-embedded every run — eight
full passes of Qwen3-VL-32B on H3 — to arrive at the same eight vectors.
Held for the process, keyed by the encoder, behind a WEAK reference: pinning a
text encoder in VRAM to save eight passes would be the wrong trade. An identity
check sits on top of the id() key because ids are reused once a model is freed,
and a stale hit would classify phrases against another encoder's geometry.
Not a cache of generated content — nothing user-typed goes into it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The encoder is the most expensive thing Studio touches — on H3 it is
Qwen3-VL-32B — and most calls to it are bookkeeping rather than the prompt:
embedding eight category descriptions to classify a phrase by cosine
similarity, embedding a memory to ask whether it is about this scene.
Every call now records its purpose, its cost, and whether the per-run cache
served it. One line per run says where the time went and what share of Studio
it was, so "26 seconds before generation starts" becomes a list of candidates
rather than a guess. The tally is per run and never persisted.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Locating a phrase's token range runs the text encoder once per phrase — on H3
that is the same multi-billion-parameter encoder the prompt goes through, up
to eight times. The ranges exist only to feed the attn2 direction patch, and
H3 has no cross-attention for it to hook, so the patch declined on the very
next line and the results were dropped.
The same question is now asked before the work instead of after it. LTX is
unaffected: it reads attn2, so the ranges are still computed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
291 lines: _lora_words, _ensure_lora_memory, _score_lora_prompt_relation and
_update_lora_weight_suggestions. The scorer matched prompt words against each
LoRA's filename; the writer was the v1 half of the suggestion pair, superseded
by _v2_update_lora_suggestions, and unreachable since the v1 refiner entry
point was removed on this branch.
_lora_state_id stays — the live v2 writer still uses it.
Checked: no refs anywhere (grep -a, all file types, node_modules excluded),
not a route handler, not in NODE_CLASS_MAPPINGS, not an INPUT_TYPES widget,
no getattr dispatch. Tests: 1379 passed, baseline plus this branch's two new
ones.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The suggester split each LoRA's FILE NAME into words and matched them against
the prompt; the overlap scaled how far that LoRA's weight moved. That guessed
what a LoRA does from what someone happened to call it, which is a different
question from how much weight it has earned.
Relevance now comes from the type set on the LoRA and the categories present
in the prompt. Weight tuning itself is unchanged — ratings, axis feedback and
the learned offset all behave as before. Behaviour change: a LoRA whose name
echoes the prompt will move less per rating than it used to, and one with an
opaque name will move more.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
32 methods, 1317 lines: the co-occurrence memory, token/pair banks, phrase
placements, embedding canvas and prompt composer. Every entry point was
called only from the v1 refine() removed in the previous commit, or from
another method in the cluster — verified individually after that removal, all
clean. Lucky and Void are one commit because they share the token banks and
call into each other; splitting them would leave a commit referencing a
method that no longer exists.
Into the Void's toggle only ever existed on the unregistered v1 node, so no
UI could reach it. The Lucky TOGGLE stays: the Studio's Lucky is a separate
56-line composer over phrase_memory (_v2_compose_lucky_prompt), unrelated to
this code and still live.
The v2 state schema declares none of these keys, so refinement-key training
never touched them. Tests: 1377 passed, same as baseline.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
FunPackVideoRefiner is not in NODE_CLASS_MAPPINGS; only FunPackVideoRefinerV2
is, and it overrides FUNCTION, INPUT_TYPES and IS_CHANGED with no call to its
parent. Nothing in the repo calls refine(), so the v1 node contract and its
1263-line body could not execute.
The class itself stays — V2 inherits ~200 live helpers from it. What goes is
the node contract that ComfyUI never reads and the entry point nothing reaches.
Checked: no refs (grep -a, incl. the NUL-byte frontend files), not registered,
not a route handler, no getattr dispatch (the file's only two getattr sites
read literal flags), no JS or workflow-template reference, and the v2 state
schema declares none of its keys. Tests: 1377 passed, same as baseline.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
"conditioning memory failed (size of tensor a (492) must match tensor b
(499))" — the compatibility gate checks the channel width and the batch
and deliberately not the sequence length, so the payload passed and the
lerp threw. On LTX the encoder pads to a fixed length and this never
happened; Qwen does not pad, so on H3 the length moves with every prompt
edit and the entire learned direction was dropped for that run.
_resize_conditioning_sequence_like was already in the file for exactly
this, with zero callers. Both the blend and the repel now use it, and say
once when a payload had to be resampled.
An exact-length payload is untouched, so the common case — same prompt,
re-rolled — stays bit-exact.
Two bugs, one line apart.
Phrase memory accumulates everything the session has rated, including
phrases from prompts since edited away. The cap to 8 was applied BEFORE
checking whether a phrase is in the current prompt, so those could take
all eight slots on rank alone, match nothing, and leave phrases actually
in the prompt unweighted — while appearing in the status line as
"weighted", which is where they were seen.
And there was no reason for a cap. Locating a phrase here is a substring
find and a scan of the tokenizer's offset map. The 8 was carried over
from _v2_find_phrase_token_ranges, which locates a phrase by RE-ENCODING
it through the text encoder — eight of those is eight Qwen3-VL-32B
forward passes, and the cap is the whole point there.
The status line now names what was APPLIED, and counts how many learned
phrases are in this prompt rather than how many exist.
"<Picture n>: " and "<Audio n>: " are emitted by the H3 tokenizer, not
written by anyone. An AUDIO reference is the awkward one: it contributes
its label and NO vision block, so it lands inside the same run of text
tags as the prompt.
Placement is now anchored at the END of that run rather than requiring
the run to nearly equal the prompt, so any number of label tokens in
front no longer shifts the bias — or refuses it outright, which is what
the slack cap did.
Protection follows the same boundary: the whole tensor is stashed and
everything before the verified prompt start is restored, so a label is no
more steerable than the picture is. Without a tokenizer it falls back to
the end of the vision block — a label may stay steerable, the reference
never does.
An r2v conditioning is laid out reference-first: the "<Picture n>: "
label, the vision block, then the encoded prompt. So the boundary is the
only thing Studio needs — rows before it are the picture and are not
Studio's to touch, rows from it on are the prompt and are.
Protecting by the image tags alone left the label steerable, since it is
text sitting between the two.
prompt_region() is now the single place that walk lives; prompt_base and
choose_encoded_text both read it instead of repeating it.
Skipping was not the answer — Studio has the text. What it did not have
was a way to know WHICH text: it keeps the raw prompt on its own port and
expands per scene, while the editor hands a linked node the already
expanded string. So its copy and the encoded one can differ.
server._expanded_link_texts is exactly what a node outside Studio
received, and the builder now forwards it. And the choice is CHECKED, not
guessed: minimax_token_tags says how many text positions the tensor
holds, so each candidate is tokenized and the one that fits wins, which
also yields the exact base offset. No arithmetic fallback, no assumption
about whose string it was.
Emphasis is only skipped when no candidate tokenizes to the tensor's text
run at all, which means the encoded string genuinely is not here.
"placed from the conditioning tail" means the tags were not usable and
the bias fell back to cond_len minus the tokens measured here. That holds
only when Studio encoded the text. On a wired conditioning another node
did, from a string this one never saw — the editor hands a linked node
the pre-expanded prompt while Studio keeps the raw one — so the
arithmetic lands the bias on a window of the wrong words with nothing
saying so.
The modality tags are the only trustworthy placement there. Without them
the emphasis is skipped and says why. Studio's own encodes still use the
tail, where the measurement is its own.
512 -> 197 positions inside finalize. _v2_apply_bounded_attention is
always-on: it re-encodes the prompt through CLIP, concatenates the two
sentence halves and REPLACES the tensor with the result. That result has
no reference image in it, because this node never received one.
So on a reference run the picture and the prompt were never both in the
conditioning the sampler got — which half survived depended on which
step ran last, and the only visible symptom was a tag-count mismatch
three functions away that read like housekeeping.
Entries that arrived on a wire now carry funpack_conditioning_owner, and
no step that rebuilds a conditioning by encoding text will touch one.
Same rule the scene split got. Bounded Attention still runs normally on
Studio's own encodes.
Every stage of _v2_finalize_conditioning is length-checked and names
itself in the log if it changes one, so the next such bug reports itself
instead of surfacing as a trimmed tag count.
An r2v node emits the reference block and the encoded prompt as separate
entries. _v2_extract_conditioning reads entry 0 and refine_v2 rebuilt a
single-entry list around it, so everything after the first entry was
dropped: the tensor kept the vision block and lost the prompt. 197
positions of a 512-position tag map, of which 191 are image marks — the
right character, speaking invented syllables.
The extra entries now pass through unchanged. Steering is fitted to the
entry Studio measured; applying it blind to a companion entry of another
length and purpose would be a guess, and discarding them is the bug.
The tag-surplus message said the reference was missing. It is the
opposite — a leading run of image marks IS the reference, and what gets
trimmed is the prompt.
197 positions against 512 modality tags is not the one spare tag a vision
block leaves, and "trimmed 315 spare tags" reads like housekeeping. The
ownership line now reports the conditioning's size, its tag count and its
entry count as RECEIVED, so it is possible to tell whether the prompt
arrived short or was cut inside this node. An outsized surplus also
reports the surviving text/image split, and any length change between
finalize's input and output is called a bug in the log.
Also resolves $variables in the phrase list before the status line, not
only before matching — the log printed raw phrases, which made it look
like `$style` was reaching the encoder.
The split re-encodes each scene from text and its entries replace
output_conditioning WHOLESALE. It re-establishes H3's visual conditioning
from source_image / h3_references — and nothing in the editor pipeline
sets either, because the reference lives on the R2V node, not on Studio.
So a wired reference conditioning was replaced by text-only encodes: not
appearance drift, a different person.
_v2_transition_scene_conditionings returns None when clip is None, so the
wired conditioning survived with CLIP disconnected. That is why
disconnecting CLIP "fixed" it, and why every fix upstream of the split
changed nothing.
Studio cannot split a conditioning it did not build. When one is wired it
keeps it, for every scene, and says so.
H3 tokenizes a reference INTO the conditioning, and comfy's
token_tags_from_embeds_info marks the whole vision block (plus its
flanking vision_start/vision_end) as 0 with text at 1. Those rows are
Qwen's encoding of the picture.
Every steering path is a lerp or add over the whole tensor — learned
directions, taste pull, concept deltas, the absolute store,
negative_erase — so the encoded picture was being pulled toward a
direction learned from TEXT. That is the character appearance drift, and
it only happens with CLIP connected because that is what builds the
directions.
The tag map existed and nothing read it. The vision rows are now stashed
before any steering and put back after all of it. Text rows still steer.
The spans were measured on funpack_scene_text — the RAW prompt, still
holding `$style` and unrolled shortcuts — while the conditioning was
built from the resolved string. Different string, so every span lands on
the wrong words or the placement guard rejects the run. funpack_encode_text
is the one that was encoded; use it.
Phrase memory also stores raw text, so a phrase could carry `$style`
while the prompt had it resolved and match nothing. Variables are a plain
substitution, so both sides are put in the same form. Shortcuts are not
peeked: a /trigger rolls per seed and peeking would give a different roll
from the encoded one, so an unrolled phrase failing to match is correct.
A wired positive CONDITIONING means Studio does not encode the prompt.
That is not a preference to configure; it is what wiring the input says,
and it is what a wired negative CONDITIONING has always done. Wanting
CLIP to own the prompt means not wiring one.
Removes clip_owns_prompt from the node, both settings panels and the
settings JSON.
A new scale was invented where two calibrated ones existed.
The ceiling is now 1.25 — the exact boost _v2_build_attn2_patch has
applied to an emphasised phrase's K/V on LTX since long before this, on
a path that works. A different mechanism for the same intent should not
push harder.
The ramp toward it is _v2_auto_strength, which already answers "how much
evidence is there": base 0.030, scaled by bad_streak and reward EMA,
clamped to [0.008, 0.085]. So one Awful reaches about x1.13 and a
sustained bad streak x1.19, instead of the top phrase jumping straight to
the maximum off a single rating.
Two defects behind a reported x1.9.
Spans overlap — a phrase and one of its own words match the same tokens
and their biases ADD. Intended, so the word reinforces the phrase, but
unbounded three overlapping x1.5 spans reach an effective x3.4 on one
token. Bounded against the strongest SINGLE span, so an explicit
(word:2.0) still means exactly 2.0.
And it shipped on by default. It changes what the model sees on every
rated run, and it is gated on is_h3_clip — which is why disconnecting
CLIP made generations normal again. Now a toggle, off.
Also replaces prefer_wired_conditioning with clip_owns_prompt: a wired
conditioning owns the prompt by default, and the old order is the opt-in.
CLIP won whenever both were connected, so Studio replaced an R2V/I2V
node's conditioning with its own text-only encode. On H3 that discards
the reference outright: Qwen3-VL is causal and the vision block precedes
the prompt, so every prompt row is a hidden state that already read the
image. Studio cannot reproduce that — nothing wires h3_references or
source_image in the editor pipeline, so its encode never sees a picture.
Detected from minimax_token_tags (0 marks a vision block) rather than
guessed, so a text-only or non-H3 graph keeps the old precedence.
Studio reads studio_settings.refiner.negative_prompt whenever the wired
negative_prompt is blank. The editor writes the wired one and has no
field for the stored one, so clearing the visible box handed control to
a negative saved in an earlier session, invisibly — a 353-character
negative encoding on a run whose Negative prompt field was empty.
The editor's box now writes both, empty included. The log says which
source supplied the text instead of only how long it was.
Three problems, all visible in the first live run.
x1.03 on every phrase: category scores are 0..1 confidences and a phrase
rarely carries much of one axis, so `1 + strength * score` was applied
and inert. Weights are now relative to the strongest candidate, making
strength mean "how much the best phrase gets".
94 candidates capped to 8 by LENGTH kept an arbitrary eight. Ranking is
by weight; longest-first is now only the application order, so a phrase
still lands before one of its own words adds to it.
The prompt was assumed to be the tail of the conditioning. The same run
reported 367 conditioning rows against 368 modality tags, which is
exactly how that assumption breaks. minimax_token_tags marks text as 1
and vision blocks as 0, and references precede the prompt, so the prompt
is the trailing run of 1s — read it instead of inferring it. The status
line says which of the two placed it.
Studio has known which words to weight since 2.0 —
_v2_concept_units_for_run splits every run into phrases, single words and
n-grams; _v2_update_phrase_memory trains each against the rating's
missing/wrong axes. The only consumer was _v2_build_attn2_patch, and H3
skips it: one packed self-attention stream, no cross-attention. So the
learning ran on every generation and reached nothing.
It now becomes an attention-logit bias on the prompt's token positions in
the packed stream, which is the same statement the K/V x1.25 emphasis was
making, at a place H3 has. Boost only by default.
Located with the tokenizer's offset mapping (verified against ComfyUI's
H3 vocabulary), not by re-encoding each phrase through Qwen3-VL-32B the
way _v2_find_phrase_token_ranges does.
The bias is a mask and SLA routes masked calls to dense, so a weighted
run is a dense run. Said once, at install.
MiniMax H3 encodes with Qwen3-VL; the stand-in path fetched a 12B Gemma
tokenizer regardless. Local HF cache is now tried before any download,
and a download announces itself — it blocks the run with no timeout.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
On H3 it is a Qwen3-VL-32B forward whose output CFG 1.0 never reads.
It still has to happen, but it must be visible when accounting for
what a run loaded and when.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Erase the negative's pooled direction from the positive conditioning
(projection by default, never the vision span). Plus an option to let a
wired positive CONDITIONING own the prompt while CLIP keeps its jobs.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Pass 2 can run a different algorithm. Unwired it reuses pass 1; in the
Editor it is behind Second pass > Own sampler, and Studio mirrors the
high pass into that output when off, so existing projects are unchanged.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Lift the unsharp out of the sampler loop through a denoiser proxy, the
same way ALG does. Window is a fraction of the schedule; audio excluded.
Off by default; an unwrappable sampler keeps sampling and says so.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
_refine_latent rejects nested latents at the door and every other caller
returns the clone unmodified, so the alias is unreachable. Noted so the next
audit does not re-open it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Unreferenced, and negatives do not exist at CFG=1 where most generation happens.
Drops _v2_negative_prompt_advisor and its two private helpers (131 lines);
_v2_negative_text_conflicts_with_request keeps 2 live callers and stays.
Tests: 968 passed, same as baseline.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
14 more silent no-ops made audible: conditioning memory blend/repel/rollback,
Bounded Attention, ALG blur, guide image load, identity install, wired pins,
shortcut expansion. Both hook-removal leaks now name the restart.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
One mechanism for both: failed() names the attempt, the error and the effect on
the output, once per run; note_on_change states a standing condition once and
again only when it changes. 13 silent no-ops made audible, 4 repeats silenced.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
CLIP wins whenever it is connected and the wired CONDITIONING is never read;
the ownership label that knew this was computed and thrown away.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
LTX-2.5 encodes with Gemma4 where 2.3 used Gemma3. Concept steering finds a phrase
by matching its token ids inside the prompt's, so a tokenizer from the wrong family
still returns spans — pointing at the wrong tokens. Nothing raises; steering just
stops working.
The tokenizer now comes from the connected CLIP, which is by construction the one
that produced the conditioning. The download stays as the fallback for runs with no
CLIP, and says so rather than claiming to be Gemma3.
Learned state is stamped with its encoder and cleared when that changes, on both the
V1 and V2 paths. Clearing reuses Session Reset so the value function, x0 snapshots
and the keyless Absolute store go with it instead of being orphaned. An unstamped
key is adopted, never wiped: every key predating this has no stamp.
The vision probe no longer depends on the submodule being called gemma3_12b — it
still is on 2.5, where it holds Gemma4.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
LTX 2.5 reuses LTXAVModel/LTXBaseModel behind config flags, so the samplers,
the identity-transfer wrappers and the guide path all bind to it unchanged.
Two places assumed 2.3 specifics, and both would have gone wrong quietly.
The AV channel split was a hardcoded width table. From 2.5 on comfy reads the
video/audio projection widths off the checkpoint rather than assuming them, so
a checkpoint can ship a pair the table has never seen — and an unrecognised
width made ltxav_video_channels return the FULL width, which silently steers
the audio text context along with the video one. The only symptom would have
been degraded audio. The split is now measured off the live model when one is
on hand; the table stays as the fallback.
Decode-time noise only ever reached VideoVAE.decode. 2.5's diffusion decoder
takes no timestep and seeds itself, but it is an nn.Module, so stamping the
attributes onto it succeeded and was then ignored — a knob that reads as live
and does nothing. It is now capability-tested against the real contract
(hasattr decode_timestep, not a class name) and says so when it cannot apply.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Three things that were each locked to one implementation detail:
- steps + schedule were a KSampler-branch feature, though every sampler here
walks a sigma list handed in from outside. They are now pass-level, so
Distilled Flow and Hybrid Euler 2S can run a named schedule (and the second
pass can be computed rather than typed) in both frontends and on the node.
- the project's Frames field snapped to the model's grid on every family. On
H3 that is required, but its arrows moved by 1 and the snap put the value
straight back; on LTX the builder rounds up by itself, so the snap only
fought the user. H3 now steps a whole 17 frames per arrow, LTX keeps what
you type and walks 8k+1 with its arrows.
- ALG lived inside the Distilled Flow step loop, so it was unreachable with a
stock KSampler. Its blurred/sharp swap is decided by the step's sigma, and
sigma is an argument of every model call, so a denoiser proxy drives it from
outside any sampler. New alg_anchor toggle on the Chain Sampler drives the
in-loop version too, so there is one control wherever you are.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
An image prompt comes back one tag long: comfy builds minimax_token_tags from
the tokenizer's INPUT sequence (embeds.shape[1]) while the conditioning is what
the encoder returned, and the vision block's expansion does not survive 1:1.
The reconcile treated any surplus as corruption and dropped the whole vector —
which on exactly those prompts discards the modality-0 marks for the image span
and leaves the DiT modulating the picture as if it were text.
Position i still means position i, so the tail is trimmed instead. Growing
(appended tokens) is unchanged.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two independent, silent breaks made an H3 i2v scene generate from the prompt
alone.
1. The Editor always runs split_by_transitions, and the per-scene conditionings
REPLACE the single entry Studio built — but they were re-encoded with no
reference_image and no h3_references. Both the frame-0 keyframe pin and the
Qwen vision presentation were gone before the sampler looked, even for a
one-scene project. The split now carries the anchor onto scene 0 and the
references onto every scene. Gated to H3: on LTX the anchor rides in the
latent, and adding Gemma3 vision to scene 0 would change existing output.
2. A MiniMax H3 Image to Video node hands its first/last frame pins out on its
CONDITIONING output, which had nowhere to go — the sampler's positive comes
from Studio. New optional h3_keyframes input on the Chain Sampler salvages
them: first frame onto the opening scene, last frame re-indexed onto the
closing scene's own final frame (the source node's frame_count is its own,
and a stale index would land mid-clip where PackedLayout refuses it). The
editor auto-wires it for H3 projects.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A SAMPLER object is only the step function, so picking KSampler used to leave
the pass with no schedule at all unless a sigma list was typed by hand.
steps + schedule now build one from the loaded model's model_sampling, exactly
as ComfyUI's BasicScheduler does. The schedule dropdown is the switch between
the two sources rather than a silent precedence rule: "use_user_sigmas" — the
default, so nothing changes for an existing config — runs the Sigmas field as
typed, and any real scheduler computes instead, leaving the typed schedule
parked in the field to switch back to without retyping it. Studio logs the fact
that the field is being ignored, and steps is only shown while it applies.
Both frontends, same change.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
minimax_h3.py shipped tags_match / token_tags_length / extend_token_tags with a
docstring saying a mismatch is a hard IndexError mid-forward — and nothing ever
called them. Studio edits conditioning in many places, and H3 tokenizes reference
images into the same span (ref_image_size "match" scales them to the output
resolution), so the token count moves with the prompt AND with the resolution.
That is why the crash looked intermittent and why editing either one made it come
and go, reported only as a bare "list index out of range" at step 0.
Studio now reconciles before returning: extend the tags when tokens were appended,
drop them when tokens were removed (absent tags the DiT handles, wrong tags it does
not), and print which happened — losing the tags changes how the text span is
modulated, so it must not be silent.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
H3 support was written against ComfyUI's PR; it has since merged (57500fc5, shipped in
v0.30.0) and the weights are out. Everything the adapter assumed still holds — audio time
axis, 17k+5 grid, batch-1 forward, the imported optimized_attention symbol, every payload
key — so this is what the release itself changed, not a correction of the reading. LTX-2 /
LTXAV is untouched throughout: a project picks a family and both are first-class.
* i2v anchors now work on H3. LTX writes the image into the starting latent
(LTXVImgToVideoInplace); H3 has no such path — its anchor is a condition row packed
beside the text — so a scene's anchor image becomes a frame-0 keyframe pin instead, and
the LTX node is no longer run against an H3 latent. Studio hands the source image's
pixels to the sampler (it owns the CLIP, the sampler owns the VAE). Pins now MERGE, so
a first-frame anchor and a last-frame guide coexist: H3's own first-and-last mode.
* Frame geometry follows the model: frames per scene snap to the family's grid (LTX 8k+1,
H3 17k+5) in both frontends, in the graph builder, and in the sampler, and H3 renders at
its fixed 24 fps. Picking a family moves the project onto its geometry and says so.
* The two checkpoints are named. fl2va (anchors) and ref2va (references) load identically
and never reject each other's conditioning, so the run reports which mode it is in and
the setup panel spells out the split.
* Pipeline setup is no longer a placeholder: released, with the published filenames.
* Reference images can be encoded at the reference pipeline's 2048px short edge.
Fixes found on the way:
* A video reference lost its soundtrack between Studio and the sampler — Studio had
already announced it to the text encoder as <Audio 1>, so every later audio ordinal in
the prompt pointed one reference earlier than written.
* Easy Gen never knew it was on H3: its model config lives on the project, which the
shared capability check did not read, so every H3 warning silently reported LTX.
* An off-grid scene length failed with arithmetic instead of an explanation.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The graph builder is now parameterized by model family instead of assuming LTXAV. H3
keeps the core's shape but three of its nodes do not apply: LTXVConditioning (stamps LTX's
frame_rate onto conditioning, which H3 has no field for), LTXVConcatAVLatent (H3's own
EmptyMiniMaxH3LatentAV emits both streams, so the latent feeds the sampler directly) and
LTXVAudioVAEDecode (reads an output_sample_rate H3's audio VAE does not define — it would
raise). Requirements, wirable ports, role targets, guided-wiring hints and the port picker
all follow the family, so a loader can no longer be wired into a node the graph never emits.
The family is a Models setting, asked and never guessed from a checkpoint filename: a wrong
guess fails deep inside ComfyUI at generate time instead of in the panel that can fix it.
Video references now work, rather than being refused. FunPack owns the pipeline and already
requires ffmpeg for preview/export, so minimax_h3 decodes reference clips itself — once, at
24 fps — and derives both views from that single decode: the 2 fps timestamped frames Qwen
sees, and the full-rate latent block the DiT packs. A clip's own soundtrack rides with it,
presented as its <Audio j> label immediately before its <Video k>, which is what ties the
two together. Capped at 15s because reference tokens ride every sampling step.
Editor UI: a Reference media view (H3 only — there is no LTX equivalent to grey out) with
per-type 1-based tags shown exactly as the prompt must spell them, reorder controls that
renumber both sides at once, and a soundtrack picker on video entries. Warning chips beside
Generate for the LTX-only sampler settings H3 switches off, so that is known before a
generation is spent rather than after; Easy Gen gets the same warnings on its strip and the
same family selector (it already loads models.js and pipeline_caps.js).
Verified live in the Cutting Room: the family toggle flips requirements, ports and hints
both ways and persists per project; tags renumber on reorder; the H3 chips replace the
Best-FaceID chip on H3 and hand back to it on LTXAV.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
H3 packs reference blocks into the sequence itself (own position grid, never denoised,
never rendered), which is the thing Best-FaceID approximated on LTX with overlap tokens
plus a trained ArcFace projector. On H3 there is nothing to project and nothing to slice
back off, so this is the model's own feature, not a port of that one.
A reference has to reach the model twice and in the same order both times: as
presentation (Qwen sees "<Picture 1>: <vision block>" so the prompt can name it) and as
a packed latent block. Those encodes happen in different nodes, so the ordered spec
travels between them:
- Studio owns the CLIP: resolves the reference list, bakes the presentation via the
tokenizer's minimax_ref_items, and records what it presented as funpack_h3_refs on
the conditioning. Reference media and a first-frame anchor are separate conditioning
modes, so source_image steps aside (loudly) when references are present.
- Chain Sampler owns the VAE: encodes exactly that list, in that order, sized against
the scene's canvas, and attaches minimax_refs. Applied on every scene branch so an
identity holds across the whole chain, and cached per run so it is encoded once
rather than once per scene.
Order is the contract — drop or reorder an entry and "<Picture 2>" points at a
different reference than the text encoder saw. Anything that cannot be encoded is
dropped AND named, and an audio reference lost for want of an audio_vae says out loud
that later <Audio j> labels have shifted.
New optional audio_vae input on the Chain Sampler (audio references only; image
references need just the main vae). Video references are refused with a reason rather
than half-implemented — they need 2 fps frame sampling and a video loader FunPack does
not own.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
