Not used on H3, not used on LTX any more. Two widgets, the append method, the
Editor's continuity controls for it, and the auto-enable that turned it on for
every multi-scene carry chain.
JoyAI memory and carried i2v guides keep the guide-attention machinery they
share with it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Six widgets (three write gains, prompt loudness, taste push, learned/manual
mode) and the whole rating loop behind them. Never proved out on the rental:
picture gain past +-0.1 adds distortion and latent noise, the rest changed
nothing legible, and the audio gain was never worth testing.
The final-layer picture detail SURVIVES and now reads its own widget — it was
the only one of the set that showed a real effect. Relabelled experimental.
Two things fall out with them. The refinement key no longer holds any sampler
value, so key scoping has nothing left to scope and goes too. And the card's
ownership table now reads dependsVals as well as dependsValue: the Studio rows
use the plural form, so absolute_strength was printing under steer_mode
"relative", where it does nothing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Seven rental runs across its whole range: below 0.5 the picture breaks up,
above 0.5 it reproduces the reference outright, and nothing in between is
worth having. It never did the job it was built for either.
Kept: the final-layer video scale it shipped alongside, which does work.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Measured on the rental and it never worked as intended. Two widgets, the
wrapper, the context-windows exclusion it needed, and its two tests.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
On the card from the rental: Chain Sampler 35 rows -> 4, Studio 13 -> 6,
Studio inputs 2 -> 0.
Three additions. A key the node does not DECLARE is now dropped rather than
kept: ComfyUI builds a node's arguments from its own INPUT_TYPES, so
`block_steer` (removed from this pack entirely) and a `steps` key the Chain
Sampler has never had reached nothing, while printing beside live settings as
if they had. An empty value decided nothing either. And the studio_settings
refiner block is filtered against the Editor's own defaults — it has no node
to ask, so an unrecognised key there is still kept: unjudgeable is not inert.
identity_projector was printing a model filename under a disabled feature. It
is rendered by hand in the Editor so it carries no dependsOn, hence
_OWNED_EXTRA, kept apart so the generated half stays generated.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two rules: skip a value equal to the widget's own default, and skip a value
whose feature is switched off. A region-sharpening prompt printed under a
disabled sharpener read as if it had run, and enough of that buries the few
values that did decide the render. On a realistic project this cuts the Chain
Sampler section from 8 rows to 4.
Defaults are read off the node's own INPUT_TYPES rather than copied here.
Ownership mirrors the Editor's dependsOn wiring, and a test regenerates the
table from engine_settings.js so the two cannot drift. Owner chains are walked
to the end — v2a_grad_scale belongs to joyai_audio_memory, which belongs to
joyai_memory. A section left with nothing to say is dropped.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Measured: 0.4 loose and inventive, 0.6-0.75 several versions of the shot
attempted at once with the strongest winning, 0.75 and up simply the
reference. No usable band anywhere in the range.
The timestep is how the model knows what stage everything is at, so a prompt
that disagrees with the picture about the stage costs it the ability to settle
on one reading — which is what "multiple variants in one video" is. Structure
is chosen early, so the edit now waits until it has been: ramped in through
_make_steer_ramp, the same late-step gate the rating-driven wrappers share,
interpolating from the prompt's own row so a closed gate is byte-identical to
not being installed.
Third hypothesis for this dial. If it still just reproduces the reference, the
feature does not earn its place and comes out.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Measured, after the label fix: at 1.0 it still reproduces the reference
almost exactly, so the <Picture N> label was never the cause. In H3
"finished" is what marks a row as reference material — content that was given
and should be reproduced. Marking the prompt rows that way moves them into
that category, and the reference takes over the frame.
Measured range: 1.0 copies the reference; ~0.4 follows the prompt but invents
things in neither; below 0.3 the picture breaks up. It does not improve
prompt adherence, which is what it was built for.
The widget NAME stays h3_prompt_time — it is in saved projects and saved
workflows, and renaming it would drop the setting from both.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Measured on a rental: 0.8 gave more contrast and 1.2 less, and 1.15 was
nearly inert.
The layer computes norm(x) * (1 + scale) + shift, and the first build
multiplied `scale` itself. H3's scale is negative, so raising the dial shrank
(1 + scale) and the control inverted. Scaling the whole (1 + scale) term
instead is monotone in magnitude whichever sign a checkpoint carries, so
"above 1.0 is more" holds by construction rather than by luck. shift is still
left alone — it is the bias.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Measured on a rental at 0.9-1.0: it reproduced the reference image almost
untouched in the frame, overriding a prompt that said in words the reference
was not where the scene begins.
An r2v conditioning is laid out `<Picture n>: ` label / vision block / prompt,
and the LABEL is text — the vision block is not. The first version re-timed by
modality tag, so it hit the label too. That label already reads as "compose
like this"; made maximally authoritative, the model composed like it.
The prompt is the LAST text run, and nothing outside the text span carries the
text tag, so that identifies it with no help from the conditioning. Row ranges
live only in mod_segments, which AdalnProj never sees — so this moves onto
ComfyUI's own per-block replace hook, which is handed them.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Reported from a rental: typing in the Composer, focus was lost, typing
continued, and a clip was cut in half by an `s`. Focus loss lands on <body>,
which the handler read as "nothing focused, so this is for the timeline".
While a writing surface is open — the Composer, any floating window — letters
belong to it whether or not it holds focus. And with "Show timeline on hover"
on and the timeline collapsed, clip edits are refused outright: a selection
nobody can see gives no sign anything happened. Transport keys stay live in
that case; the player is on screen either way.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
They stayed in the project: inert (learned mode reads the key, never the
dials) and invisible (the dials hide in learned mode), until the next time the
project was opened. Correct output, state nobody could see — which is what
this rule exists to prevent.
A knob whose default hands ownership back to something else now takes the
values it owned with it (clearsOnDefault). Choosing manual clears nothing.
The JS test suite also pins its extracted copy against the real renderer, and
the cleared list against the backend's key-scoped set.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The key-scoping strip lived in Project.from_dict, which parses the browser's
SAVE payload as well as the file on disk. So the Editor sent a value, got a
project back without it, and redrew the default — reported from the rental as
the mode switching itself back to learned. Every one of these dials was
unsettable, not just the mode.
The strip now runs only when a project is read from disk. h3_gain_mode and
h3_prompt_time leave the set entirely — a mode is a choice, and prompt_time is
never learned. Manual mode strips nothing: those values are hand-typed, and
there is no learned state to reset.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
fp16 and fp8_e4m3fn_fast both wreck H3's audio before anything looks wrong,
and the loader's own notes go into a status string the Editor never renders —
so the only symptom was a bad soundtrack. One console line, once per run, on
H3 only. Names the fix in the first sentence, which is the half that survives
the console trim.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
h3_prompt_time is not one of the learned strengths, but it was gated behind
h3_gain_mode=manual like the ones that are — so using it meant switching
rating-learned gains off. It now reads its widget in both modes, and shows in
both.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
h3_prompt_time: the prompt is told how finished it is independently of the
picture, so the model leans on it from the first step instead of treating it
as unreliable while the picture is noise. One extra row through each block's
AdaLN projection, dropped before the result returns. Manual dial only — its
off value sits at the end of its range, not the middle, so the rating loop
cannot explore around it safely.
h3_video_detail: crisper or softer picture with the audio bit-for-bit
unchanged. The final layer is past the last attention pass, so unlike every
other picture control it has no path to the soundtrack.
apply_adaln_gains -> apply_adaln_edits; both block edits share one wrapper.
video_detail joins the rating-learned gains.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Validated on the rental and it does not earn its place: 3.5 minutes against 1
for the same clip, worse picture, garbage audio. Some of that was mine — no
weight prefetch across the 50 blocks, a host-offloaded K/V cache, and an extra
commit forward per chunk — but fixing those only buys parity on speed, and the
technique's real win is unbounded length, which H3 already covers natively at
the lengths in use here.
Gone: h3_causal.py, the four h3_causal_* sampler widgets and their Engine
Settings rows, `chunk_causal` on the diffusion loader, and five test files.
KEPT, because it is not about this at all: the LoRA key RENAME table. Stripping
a wrapper head leaves `blocks.0...`, comfy's key map has no bare form, and the
file silently applies nothing — 532 keys, zero applied, no error. That was
breaking any PEFT-style adapter named this way, not just the one that found it.
Its tests moved to test_lora_shape_mismatch.py alongside the MATCHED NOTHING
diagnostics.
Two INPUT_TYPES widgets go with this (h3_causal_* on the sampler, chunk_causal
on the loader). A saved workflow carrying them loads fine — both were optional
and are matched by name — but the values are ignored from here.
Tests: 1700 python, 68 JS.
It was trimmed at "adaLN reads a compact…", which cut every actionable word:
how many dropped, that the rest of the LoRA still applies, and what to do. The
verdict is now the first sentence, and the line asks for its full text — this
only prints when a LoRA is half-landing, which is the one moment the whole
explanation earns its line.
Tests: 1830 python (+1), 68 JS.
Two things, both from the rental.
THE LORA. Its keys are ComfyUI's module paths exactly, under a different head:
base_model.model.dit.audio_patch_proj.lora_A.weight
diffusion_model.audio_patch_proj <- what comfy's map holds
Every candidate the loader tried was a STRIP, which left `audio_patch_proj...`
— and comfy's key map has no bare form to land on, so 532 keys matched nothing
and the file did nothing at all. LORA_KEY_RENAMES adds (prefix, replacement)
candidates, scored against the real key map like the strips, so a rename that
lands beats a strip that does not without either being ordered first.
THE PROGRESS. The causal lane replaces comfy's sampler rather than wrapping it,
so nothing ticked the progress bar and nothing answered an interrupt: a long
generation looked exactly like a hung one, and Cancel did nothing until the
whole clip finished. The rollout now reports every model call, and the sampler
maps that onto the budget this scene was allotted — the bar was sized in
scenes x steps and a chunked scene runs that many steps PER CHUNK, so ticking
raw would have overrun it. The phase readout says "scene 1/3 · chunk 7/22", and
the interrupt is checked before every step, not just every chunk.
Tests: 1829 python (+5), 68 JS.
The rental found it: the RAVEN LoRA reports MATCHED NOTHING. Every chunk-causal
result so far was base H3 driving a cache it has never been trained for, which
is exactly the failure mode the loader warns about — and the warning was
invisible, in a status string the Editor shows nowhere.
Two fixes so the next one is diagnosable:
- MATCHED NOTHING now prints BOTH namings, three keys each: what the file has
and what the model wants. "Matched no weights" is not actionable; the names
are, because they say whether the file is for another architecture (nothing
alike) or the same one under different module names (a rename rule away).
Every entry in LORA_KEY_PREFIXES was found by hand for want of this line.
- a LoRA that matched nothing no longer logs as Active. Loaded is not applied,
and that was the same lie the status string used to tell in private.
funpack_log gains `full=True` for the rare message whose PAYLOAD is the detail,
where the first-clause trim removes the only reason to print it. Not for
emphasis — a feature's state and its reason stay one clause.
Also: the H3 checkpoint-mode note no longer asserts a mismatch it never checked.
Nothing in the state dict identifies fl2va from ref2va, so it cannot know which
file is loaded, yet it said "fl2va weights will read it, but were not trained on
it" on EVERY run with references — including on the ref2va and hybrid
checkpoints where nothing is wrong. It now names the conditioning and stops.
Pins-and-references together stays a warning; that one is a real finding.
Tests: 1824 python (+4), 68 JS.
`keys=384 fmt=stripped base_model.model.dit.` only ever went into the node's
`status` output, and the Editor's fixed graph shows that nowhere. So a LoRA that
matched nothing looked exactly like one that applied — and on the chunk-causal
lane, where the adapter is what the whole mode depends on, there was no way to
tell which had happened.
[FunPack] LoRA raven_streaming_4nfe.safetensors: Active | keys=384 … at +1.000
[FunPack] LoRA stack: Inactive | No LoRA is selected
Tests: 1818 python (+1), 68 JS.
One bug reported twice on the rental. `estimate_memory` calls
`extra_conds_shapes(**cond)`, so prepare_sampling wants ComfyUI's INTERNAL cond
dicts — I passed the [tensor, meta] pairs a node receives, it raised "argument
after ** must be a mapping, not tuple", the load never happened, and the
conditioning then failed on a model still half on the CPU. The second line was
the first line's consequence.
convert_cond now runs first, which is what the dense path does.
A failed load at BUILD time refuses the lane instead of warning: everything
after it assumes the model is resident, so a warning there just resurfaces as a
device mismatch three lines later. Mid-rollout it still only warns — by then the
run is under way and whatever is resident is better than a dead scene.
A ComfyUI without comfy.sampler_helpers falls back to load_models_gpu rather
than refusing; only a real load failure is fatal.
The tests for this now swap prepare_sampling with monkeypatch instead of
install_module. A raising stub merged onto the shared module object poisoned
every test that ran after it — the same parent-attribute trap the suite already
has a note about.
Tests: 1817 python, 68 JS.
[FunPack] K/V conditioning patch: Inactive | Non-LTX model loaded
[FunPack] Chunk cache: Active | 22 chunks, 4 steps each, consistency
`feature(tag, name, active, reason)` is the shape, and `skipped` renders
identically — a deliberate skip and an inactive feature are the same fact and
should not have two spellings. `failed` becomes `Name: Failed | <effect>`,
keeping the effect rather than the exception text: "that explains what I am
looking at" is the half worth the line, and an exception message rarely says it.
Everything funpack_log prints is now cut at the first sentence end, so the call
sites keep their reasoning without spending it on every run. SENTENCE ENDS ONLY
— an em-dash introduces the reason as often as the essay, and a renderer that
guesses wrong eats the one part of the line worth reading. A single long
sentence is hard-trimmed instead of left to run.
FUNPACK_LOG=verbose (or set_verbose) prints the full text, exception included.
Converted to the new shape: the chunk cache, region locks, interior pins, the
taste push, the token-refiner edit, AdaLN gains, rating-learned gains, phrase
emphasis, the K/V conditioning patch, the scene split, Bounded Attention,
per-block LoRA weights, LTX enhancements. The rest of the log gets the trim for
free and keeps its own phrasing until converted.
Tests: 1815 python (+9), 68 JS.
Every chunk restarting from the input image on the rental. The prompt and the
anchor shared one cache chunk, so pinning the prompt pinned the anchor with it —
for the whole clip, in a context of three visible blocks. In a dense render the
anchor is one pin among hundreds of frames' worth of rows; there it was a third
of everything the model could see, and the model did what it was trained to do
with a conditioning row: compose from it.
They are now two chunks. The prompt is 0 and always pinned — it is every moment
of the clip. The conditioning is 1, so `sink` decides how long it stays: 1
releases the anchor after the opening, 2 keeps it (today's behaviour), 3 also
pins the opening shot. Media chunks moved to i + 2, and the conditioning slot is
reserved even on a t2v run so `sink` counts the same either way.
The prefill is two forwards, and the conditioning pass reads the text already in
the cache — so the anchor is still represented WITH the prompt rather than in
isolation.
Note the widget's meaning shifted by one: sink 2 was prompt+anchor+opening shot
and is now prompt+anchor.
Tests: 1806 python, 68 JS. The upstream check now prefills a real keyframe and
confirms sink 1 releases it while keeping the prompt.
extra_conds runs H3's token refiner, which is a partial forward — so preparing
the conditioning before the model is resident gave `mat1 is on cuda:0,
different from other tensors on cpu` and refused the lane.
The load moved ahead of it, and it now goes through ComfyUI's own
comfy.sampler_helpers.prepare_sampling rather than a bare load_models_gpu with
memory_required=0, so the estimate, the wrappers and the placement are the ones
the dense path would have made for this clip. Both call sites share one helper:
a session can be built and run much later, and another model may have evicted
this one in between.
Tests: 1805 python, 68 JS.
The rental is on 0.34.0, whose comfy.ldm.minimax.model has no time_shift_slope
— that function is newer than stable. It came in through the same all-or-
nothing import as the classes the lane cannot exist without, so one absent name
raised ImportError and make_causal reported "this ComfyUI has no MiniMax H3 to
extend". Which then read as a broken install, and before that as a switch the
user had not flipped. They had.
The import is now split. The classes and helpers with no substitute stay hard,
so a genuinely too-old ComfyUI is still named precisely by the ImportError.
The ones that arrived later are read with getattr and fall back to what THAT
ComfyUI's own dense forward does.
For time_shift_slope that means not scaling the audio velocity at all: 0.34.0's
_forward returns `-audio_out` unscaled (verified against the tagged source), so
applying the correction here would put the chunked lane on a different audio
grid than the dense path on the same install. Where the function exists, it is
applied exactly as before.
Tests: 1804 python, 68 JS. The upstream check now deletes time_shift_slope and
drives a chunk through the lane without it.
The refusal asserted a cause it had never checked — "switch on chunk_causal" —
which was wrong, and sent you looking at a switch that was already on.
Three changes, none of them a guess:
- build_session no longer refuses a stock H3. Re-classing adds no parameter,
renames no key and costs nothing, and with no cache passed the model is
bit-identical to stock — so a mode that is already switched ON has no
reason to refuse itself over a switch on another node. It does the work.
- when it does that, it says so, naming the loader as the end that did not
take. That is the line that tells us whether `chunk_causal` is off or its
value never reaches the node.
- the loader PRINTS its chunk-causal outcome. It only ever went into the
`status` string, which the Editor's fixed graph shows nowhere, so a
"REQUESTED BUT NOT APPLIED" was invisible and the sampler downstream had
nothing to go on but a guess.
If make_causal genuinely cannot run, the message now names the diffusion
model's actual class instead of asserting what the user did.
Tests: 1804 python (+3), 68 JS.
The chunk-causal switch replaced a `raven_lora` combo on the diffusion loader
with a boolean plus an ordinary LoRA, but the Chain Sampler tooltip and the
Engine Settings detail still told the user to go and set the old widget. Both
now name what is actually there. A test refuses the string anywhere a user can
read it.
Tests: 1801 python (+1), 68 JS.
The two entry points the rollout was waiting on — prefill_text and forward_chunk
— are written, and with them the whole lane is ours. raven_causal.py is gone:
its pure parts (step rules, the audio sigma remap, the rollout loop) moved into
h3_causal.py, and its bridge to ComfyUI-MiniMax-H3-RAVEN-Streaming is deleted
along with the probe that hunted for that package in custom_nodes. Nothing has
to be installed for the mode to exist.
How a chunk is cut: ONE full-clip PackedLayout is built by ComfyUI's own
extra_conds, and every chunk is a SELECTION of its rows. So a chunk's RoPE
angles, modality tags and timestep classes are the ones a dense forward would
have given those same rows, and the keyframe pins and reference blocks keep the
rows upstream packed for them — which is what makes i2v and r2v work here
rather than be refused. Those rows are the prefix: cached once as chunk 0, so
media chunk i is cache chunk i + 1 and a sink of 2 pins the prompt plus the
opening shot.
A chunk is committed CLEAN — one extra forward at sigma 0 after its last step.
Caching the last denoising step's K/V would hand every later chunk a context
that still carries that step's noise.
Two failures that used to be a crash fifty blocks deep are now refusals by
name: a clip whose soundtrack does not cover its picture, and a clip too short
to have anything for a cache to remember.
Tests: 1800 python (+1 net; the RAVEN bridge's tests are gone, the plan,
session and real-model checks are new), 68 JS. The new upstream check drives
the REAL DiT at toy width through both lanes and confirms the safety property
holds: with no cache passed, the causal model is bit-identical to stock H3.
Not yet run on the rental.
Untying them from the project file was the ask; taking the dials away was not.
The six rows are back in Engine Settings exactly as they were.
What stays from the previous commit is the part that answers the ask: they are
stripped from sampler_inputs on load, so a dial moved in a session applies to
that session and the next open reads the key — or the defaults when no key is
active or the key files have been deleted.
Tests: 1777 python, 68 JS.
Asked twice, and the first fix answered the wrong question. Clearing a dial on
its default stopped the value being STORED, but it was still a project field —
so deleting every refinement key left the old behaviour applying, and the only
way back was typing the neutral value in by hand.
These knobs are learned from ratings and live in the key's own state. A copy in
the project file was a second source of truth that outlived the key it came
from. So:
- h3_gain_mode / h3_gain_video / h3_gain_prompt / h3_gain_audio /
h3_prompt_scale / h3_taste_bias are stripped from sampler_inputs on project
load, so a project that already carries them heals itself on first open
rather than needing a hand-edited JSON;
- they are no longer offered in Engine Settings at all, because offering one
writes it back the moment it is touched — which is how the second copy got
there;
- with no key active the sampler already returns neutral, so deleting the key
files now genuinely resets the behaviour.
Still widgets on the NODE: a raw ComfyUI graph with no Refiner in it drives
them by hand, which is what h3_gain_mode 'manual' exists for. Key-scoped means
not in the project, not removed.
Tests: 1771 python (+15), 68 JS.
Every message about loading the chunk-causal adapter named FunPack Apply LoRA
Weights. That is the wrong node: it is the optional stack producer for
prompt-specific trained strengths. FunPack LoRA Loader loads AND applies from
its own file list, and always has — so the instruction sent the user to add a
node they do not need.
Its own description read the same way ("Loads LoRAs from a FunPack LoRA
stack"), which is where the confusion came from. Both it and the docstring now
say the stack is the narrow case, not the way in.
Tests: 1767 passed (+5).
Reported from a rental: the H3 gain dials could not be cleared. Once touched, a
value was written into the project file and stayed; typing the default back in
stored the default rather than removing the key, so the value outlived the
refinement key it came from and read as learned state that would not clear.
The learned values themselves have always lived only in the refinement key —
what leaked into the project was the MANUAL side of the same dials, which from
the user's chair is the same knob.
Writing a value equal to the knob's default now REMOVES it, so "back to
default" really is back to nothing and a project only records what was
deliberately changed. Applies to every sampler knob, not just the H3 ones: the
same trap was there for all of them. Keyed on equality with the declared
default, not on truthiness, because h3_taste_bias is centred on 0 and a
truthiness rule would never clear it.
Tests: 1751 python, 68 JS (+9).
Both halves now go through nodes we already own, so nothing third-party has to
be installed for the feature to exist.
The diffusion loader gains `chunk_causal`, which RE-CLASSES an already-loaded
H3 in place. That is what removes the need for a separate loader entirely: the
alternative is reimplementing ComfyUI's whole diffusion-model load path so the
adapter can be attached before the ModelPatcher exists and its memory ledger
counts it. Re-classing adds no parameter, renames no key and copies nothing —
only which forward runs changes — so the ledger and offload accounting are
untouched and the switch is idempotent.
The adapter is then an ORDINARY LoRA through FunPack Apply LoRA Weights. Our
loader already scores prefix variants against the model's real key map; it just
did not know this one. RAVEN's keys are `base_model.model.dit.<path>`, so
stripping only `base_model.model.` left `dit.` in front of every key and the
whole file matched nothing, silently. One entry in LORA_KEY_PREFIXES fixes it.
A ComfyUI too old for H3, or a non-H3 model, declines with a reason rather than
raising — one optional switch must not turn the loader into a dead node.
Tests: 1751 passed.
First half of owning the causal lane outright, so the feature needs no
third-party package installed.
h3_causal.py: a ChunkKVCache with sink/window eviction and host offload, the
chunk cut, and CausalAttention / CausalDiTBlock / CausalMiniMaxH3Model as
subclasses of ComfyUI's own H3 classes — same module names, same state dict, so
weights load with no remapping and a PEFT LoRA attaches through ComfyUI's
ordinary adapter (which already reads lora_A/lora_B keys).
The safety property: with cache=None every forward defers to its parent, so the
dense lane is bit-identical to stock H3 and nothing here can degrade an
ordinary generation.
Geometry is BORROWED, not re-derived. A chunk's RoPE positions must be the ones
the dense layout would have given those rows, so the full clip's grid is built
with upstream's own _frame_grid / _video_grid / _audio_grid and sliced per
chunk. An upstream grid change is inherited rather than silently disagreed with.
Two traps handled explicitly:
- pack_audio is channel-major over the WHOLE clip, so a chunk's audio rows are
TWO spans, not one. A contiguous slice would hand the model the wrong half
of the stereo field, quietly.
- blocks are RE-CLASSED in place, adopting submodules by reference, never
wrapped. Wrapping renames the state-dict keys of everything inside, which
has broken unpatching in this project before.
Tests: 1751 passed (+21). prefill_text/forward_chunk still to come.
FunPack owns its loading pipeline, so the causal lane must not need a
third-party loader node in the graph. `raven_lora` on FunPack Diffusion Model
Loader turns an ordinary H3 checkpoint into the chunk-causal DiT with the
adapter attached. Same checkpoint either way — it is a LoRA, there is no
separate RAVEN model file, and every message now says so.
The weight load itself is DELEGATED to the RAVEN package rather than
reimplemented, and not for convenience: the adapter is an FP32 activation
residual registered on the base Linear leaves and must be attached before the
ModelPatcher is built. ModelPatcher.model_size() memoises, so an adapter
attached afterwards is invisible to Comfy's memory ledger and never moved by
partial CPU offload — the model would report itself smaller than it is and the
residual would end up on the wrong device.
Selecting a LoRA when the causal class is unavailable RAISES with the fix in
the message, rather than quietly loading a plain model: a model without the
adapter would leave the causal lane reading an attention pattern nothing was
trained for, which reads as a quality problem with no visible cause.
Strength is fixed at 1.0 — the adapter is what teaches the chunked attention
pattern, so "partly on" is not a useful state.
Tests: 1730 passed (+12).
Found by asking how CFG=1.0 reaches the model. It reaches it through
comfy.sample.sample_custom -> sampling_function, which sets uncond_ = None at
1.0 so the negative branch is never evaluated. Nothing wrong there.
But the causal lane bypasses sample_custom entirely, and sample_custom's
prepare_sampling is also what calls load_models_gpu — which is in turn what
applies a patcher's OBJECT PATCHES (partially_load -> patch_model). So on that
path the weights could still be offloaded AND every object patch FunPack
installs would silently not be there: the per-modality AdaLN gains and the
token-refiner edit (prompt scale + learned taste bias) would all have been
absent, with nothing reporting it.
Loaded once for the whole rollout, before the first forward — not per chunk,
which would fight Comfy's own offload decision all the way down the clip. A
failed load degrades to whatever was already resident and says the patches may
be missing, rather than turning a placement problem into a dead run.
Tests: 1719 passed (+5).
A mode on the existing sampler, not a new node. h3_causal_chunks generates the
clip in time chunks that each attend to a KV cache of the ones before them, so
a long clip stays one continuous shot with real memory instead of separate
scenes stitched together. Sink and window are exposed as what they actually
are: chunks pinned from the start, and recent chunks still remembered.
Two defects found while auditing the bridge I had sketched earlier (which a
blanket `git add -A` had swept into an unrelated commit — it was dead code, but
it should not have gone in under that message):
- it read request.video_channels / audio_channels, which do not exist on
RAVEN's LatentRequest. First real call would have been an AttributeError.
- it handed BOTH streams the same sigma. H3 denoises audio on its own shifted
grid (3 against video's 12), so that is exactly the error h3_audio_clock
exists to correct, and it bites hardest on the few-step schedules this lane
is built for. Audio now goes through the same remap, transcribed from
comfy.ldm.minimax.model.time_shift_sigma and verified equal to the live
function across the range.
The lane REPLACES the sampler rather than wrapping it, so every reason it
cannot run falls through loudly: not H3, pack not installed, stock
bidirectional model. A rollout that fails mid-way raises rather than quietly
falling back to ordinary sampling, which would return a clip under a mode the
user believes ran. The run states that the wired sampler and CFG are unused on
this path, and that there is no per-chunk redo.
Editor: Settings > Engine > Sampler > Experimental, hidden entirely off H3.
Tests: 1714 passed (+18). Never run on hardware.
81 uses of --font-mono, two stray var(--mono, ...) fallbacks that were always
resolving to raw monospace because that token was never defined, and the boot
error banner's inline font. All now Inter. The --font-mono token is deleted
rather than repointed at a sans stack — a name saying "mono" that isn't would
be a trap for whoever reads it next.
JetBrains Mono is also dropped from the Google Fonts request. It was still
being fetched on every load for a family nothing used.
Kept the one thing monospace was earning: digits that do not jitter. A timecode
or countdown re-laying itself out every frame is why readouts reached for a
mono stack at all, so those selectors get font-variant-numeric: tabular-nums.
Inter carries tabular figures, so the alignment survives without the typewriter.
Courier New stays in the video text-overlay font picker. That is a creative
choice burned into the output, not interface chrome.
Verified in a browser against the real stylesheet: every sampled element
resolves to Inter, numeric readouts report tabular-nums.
The Movie Editor's builder forces split_by_transitions on for EVERY project, and
the per-scene conditionings replace the base entry — so enhancing the base
prompt as well was a second full generation whose result was then discarded. At
the ~59s per call measured on the rental that is a minute of pure waste per run,
and it looked like thinking mode being stuck on.
It was not thinking: that defaults off and is passed explicitly, so it can never
inherit the repair advisor's own default of True. It was two code paths both
doing the work.
Now the base is enhanced only when the run does NOT split, and the skip says so
on the console rather than being silent. One generation per prompt that is
actually encoded.
Tests: 1696 passed (+6).
The 2000-character cap was wrong. H3 has no padding, so every prompt token is a
real sequence row and a long prompt is a legitimate thing to want — an enhanced
one especially. Cutting at a fixed length contradicted the token limit the user
had just set.
Length cannot tell "long and detailed" from "stuck". What distinguishes them is
that a model which has stopped writing starts repeating itself, so that is what
is detected now: three occurrences of the same sentence and the prompt is cut
at the first repeat. A long prompt that never repeats is never touched.
The only remaining length bound is the one the user's own token limit already
implies (~4 chars/token), so there is one control rather than two fighting.
Short recurring lines are exempt — "She smiles." legitimately recurs in a shot
list.
Tests: 1690 passed.
Reported from a rental: with the length limit raised to 8192 the enhancer used
all of it. The H3 encoder has no trained decoder head — ComfyUI falls back to
tied word embeddings — so it may never emit a stop token, and max_length stops
being a ceiling and becomes the actual length. Nothing downstream noticed: it
just encoded thousands of tokens of rambling as the prompt.
Now capped at 2000 characters and cut back to the last COMPLETE sentence, never
mid-word, and the status says the model did not stop on its own.
Also reported: it seems to run on CPU. Every generation now reports the device
it executed on, how long it took and the cap that applied — a 32B encoder
sharing a card with the DiT can be placed on the CPU by ComfyUI's loader, which
is minutes per call and invisible from the returned text.
Tests: 1685 passed (+9 enhancer).
RAVEN's contribution is not a sampler, it is a way of shaping the sequence: cut
the clip into time chunks, carry each to completion, commit it into a KV cache
as clean context, generate the next one attending to that cache. The model gets
real memory of what it already drew, so a long clip stays one continuous shot
instead of a stitched chain.
That shaping is separable from the 4-step consistency sampler it ships with,
and this keeps them apart so each can be dissected:
- the chunk cut and the cache are RAVEN's (prefill_text / forward_chunk);
- the SCHEDULE is ours — any scheduler, any step count, bong_tangent included;
- the STEP RULE is a knob: consistency (what the LoRA is distilled for),
plain flow euler, or rectified-flow ancestral. So a chunked run can be
compared against an unchunked one without two variables moving.
Audio chunk boundaries are derived from the shared clock (40 latents/s against
a fixed 24 fps, 17 frames per chunk), not from a hard-coded 28/29 cadence — the
cadence comes out as a consequence and stays right if the grid moves. Verified
against the real H3 grid: 192 frames -> 12 chunks, 320 audio latents, matching
RAVEN's own validated run.
How i2v/r2v survives: `known_chunks` leading chunks are taken as given rather
than sampled, but are still COMMITTED. Chunk 0 comes from FunPack's ordinary
dense path — where pins, references and region locks already work — and only
then enters the cache. The causal lane is never asked to model conditioning
rows it has no layout for, which is exactly what RAVEN's own sampler refuses.
Every model interaction arrives as a callable, so the loop is tested with no
ComfyUI, no RAVEN package and no weights.
Core + 39 tests. Sampler wiring and the rental run are NOT done yet.
Runs after shortcuts and after $variables resolve, before encoding — the model
is handed exactly the text that would otherwise have been encoded, and the
result is what ComfyUI generates from, so nothing downstream knows it happened.
Not the repair advisor: that one is triggered by a rating and fixes a named
fault. This runs on every generation, needs no key, no rating and no previous
run, and only elaborates. Uses advisor_clip when one is wired (the Studio node
already has the socket), the generation CLIP otherwise.
Multi-scene enhances each scene separately. The editor's scene list is
authoritative on count, so rewriting the scenes as one paragraph and
re-splitting could come back with a different number and desync every clip from
its anchor. One generation per distinct text, cached within the run.
Every failure path returns the ORIGINAL prompt — an enhancer that empties the
prompt on a bad generation would render a blank video and blame the model.
Thinking traces, code fences, preambles and whole-answer quotes are stripped;
quotes inside the prompt survive, because speech is quoted on purpose.
Settings > Engine > Studio: toggle, editable system prompt (new textarea field
kind), length limit, temperature, top-p, thinking. Off by default.
Tests: 1637 passed (+24).
Reported from a rental: manual gain mode with the refinement keys deleted died
with "'TokenRefiner' object has no attribute 'inner'". Learned mode was fine,
because with no key the gains are neutral and the installer returns before
building anything — so only manual, which ignores keys, ever built the wrapper.
Wrapping a module in an nn.Module renames every weight beneath it
(token_refiner.blocks.* -> token_refiner.inner.blocks.*). patch_model applies
object patches BEFORE loading weights, so the names were recorded WITH .inner;
unpatch_model restores weights BEFORE unwrapping, and with several patchers
alive off one shared model — one clone per scene, plus one for the AdaLN gains
— another patcher unwraps first and the restore walks .inner on a real
TokenRefiner.
Both wrappers now patch `<module>.forward` through the same object-patch
mechanism instead of replacing the module, so the tree and the state dict stay
exactly as the checkpoint built them. Neither is an nn.Module any more, and
each captures the original bound method rather than re-reading `.forward` at
call time — ComfyUI restores a patch by writing the previous value back, which
can be a live wrapper, so re-reading would nest a scale per scene.
Reproduced against faithful patch semantics before and after. AdaLN gains had
the identical latent hazard and are fixed the same way.
dependsVals (an array) is the Studio-field convention; sampler knobs read a
singular dependsValue and fall back to a plain truthy check when it's absent —
which a non-empty combo string always passes. All five dials showed regardless
of h3_gain_mode. Switched to dependsValue: "manual" to match the mechanism
that's actually there.
The prior wording made the frame-index/anchor-eviction issue read as the
reason JoyAI is blocked on H3. It isn't: no JoyAI-Echo LoRA exists for H3, so
it's inaccessible by design regardless of interior pins. The frame-index issue
is real but secondary — it's what would also go wrong if someone forced it on.
h3_gain_mode and its five dials existed only as Chain Sampler widgets, so the
Editor could turn the learning on but never take it manual. They now sit under
Sampler > Experimental, hidden off H3 by the same rule as h3_audio_clock, and
the five dials depend on manual mode so learned mode shows one control.
Audited every Chain Sampler and Studio widget against Engine Settings while
here. One other real gap: carry_overlap_through_anchor was reachable from no
frontend at all. Added under Continuity.
Everything else unexposed is owned elsewhere by design (timeline, rating
picker, Models window, Batch Training) or is data plumbing, not a setting.
Region lock: a keyframe pin can cover part of a frame. Transparent areas of an
i2v anchor are dropped from the packed sequence entirely — not blanked, not
noised, absent — so the model invents them. Row deletion re-derives the layout
from upstream's own output; two patches share one keep-mask so they cannot
disagree. Surfaced through the anchor's alpha, which needs no new wire and works
in the Editor. Excluded pixels are filled with the kept area's mean before the
VAE so no hard edge bleeds in. EXPERIMENTAL: fl2va saw whole condition frames.
Taste bias: the liked direction Studio already learns is now also added AFTER
H3's token refiner, where it lands as sent instead of being mixed across tokens
and normalized away. Relative to the span's own row norm, so one strength means
the same thing on every prompt. How far to push is a fifth learned gain,
centred on 0 and signed; h3_taste_bias is the manual override. Needs 3 liked
runs.
Gain state generalized to per-key centres, bounds and exploration width.
Tests: 1608 passed (+46). Untested on a rental.
