Reaches inside the forward pass instead of around it: captures the
video rows of one block's hidden state per generation, and once a
refinement key has 3+ liked and 3+ disliked runs, adds their mean
difference back into that block's output on every later run. No
gradient, no MLP -- a mean-difference of vectors is the whole model.
Masked to video rows only via the block's own mod_segments tags,
never audio or text -- the one documented failure of this class of
intervention (project_self_consistency_failed.md) happened on a
different model (LTXAV) via joint audio-video attention.
Off by default, H3-only, unvalidated.
0 = only cemented phrases get extra attention. 1 = that bias
collapses to nothing, letting untrained content compete evenly.
Same attention-bias channel as the existing toggle, scaled down.
Settings ▸ Learning ▸ "Start a fresh measurement". Deleting the refinement key
was not enough: the probe log lives outside the <key>.* sidecar convention on
purpose, so it survives a key clear, and the per-bucket heads are REBUILT from
it on every rating — the next rating restored them from runs still on disk.
So the clear takes both, and leaves the switch alone: "throw away what was
measured" and "stop measuring" are different requests.
Verified against a running ComfyUI: 12 runs and the heads file removed, count
back to 0.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
ComfyUI executes a node under torch.inference_mode(), so a descriptor read out of
the probe log becomes an inference tensor and autograd refuses it:
Trajectory probe intake failed: Inference tensors do not track version counter.
Every head stayed empty and the run reported "nothing learned yet" on every
generation — so the feature was OFF while looking merely untrained, which is the
one failure invisible from the outside: "not steering" is also what correct
behaviour looks like before enough ratings. Seen on a live rental with 73 runs
logged and zero of them trained.
The output value function's trainer already carries this guard; this one did not.
Also: the bucket-isolation test was measuring TRAINING accuracy, where a
few-hundred-parameter MLP fits 20 noise vectors perfectly and every bucket scores
1.0. It now holds rows back, which is the only version that can tell a head that
learned from one that memorised.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Every rating-driven mechanism shares one gate that opens only over the last half
of the schedule — after motion and layout are decided — so a rating about
movement has never had a window to act in.
This learns one value function per schedule quarter, each trained on that
quarter's own predicted output, each steering inside its own window, with no
shared late ramp. It reads the probe's log, so it starts trained on whatever has
already been rated.
Basis: 51 rated runs across two independent sessions, different prompts and
balances. The early window measured 88% and 89% of the late window's effect,
direction unanimous across all 8 bucket-by-session results, sessions combining
to p 0.043. Cross-validated on the clean session the heads rank held-out runs at
AUC 0.73-0.82.
Beside output_guidance, not replacing it, and off by default: the honest test is
the same seed with it off and on. A window with under 10 samples is byte
identical to the feature being off, so "nothing changed" stays readable, and the
run says which windows can steer rather than only that it is on.
Not included: axis routing (quality late, action early). It is the next step and
it would muddy this A/B.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Runs of one prompt share most of their content: on a real 21-run session, 86% of
each descriptor was the part every run had. Comparing raw vectors measured that
shared part, so every distance came out near zero and near equal — mean cosine
0.97, and no separation at ANY bucket including the late ones where steering
demonstrably works.
Centring first flips the same data from -0.002 (backwards) to +0.03 and p 0.83
to 0.11, consistent across all four buckets. Not significant at 5 good / 16 bad,
but pointing the right way and now limited by sample size rather than by the
statistic.
The descriptor was not at fault. adaptive_avg_pool1d over the flat latent keeps
512 REGIONAL means, not one global mean, and it distinguishes clips fine.
Also: the prompt id is quantised before hashing. Raw bytes answered "are these
bit-identical", so any value crossing a rounding boundary changed the id — one
prompt produced 20 distinct ids across 21 runs. Stable now for jitter of the
size a GPU produces; no exact hash of floats can do better.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Settings ▸ Learning gains Download and Load. A rental is replaced whenever
something breaks, refinements/ is not in git, and the reading needs ~16 rated
runs — so without a way off the box the count restarts on every instance and
the answer is never reached. 16 runs is ~85 KB.
Importing the same export twice adds nothing: the permutation test reads n
directly, so duplicated runs would report a separation as far more significant
than the evidence supports.
An uploaded .pt is a pickle, so the restore parses it with weights_only — every
field in a run is a tensor or a primitive, and anything else is refused rather
than executed. Nothing but numbers, a rating key, a timestamp and a one-way
prompt hash is stored; no prompt text has ever been written to these files.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Settings ▸ Learning ▸ Trajectory probe: a switch to record, the run count,
and a Read button that gives the answer in plain language. The switch is the
same environment variable the sampler reads and the editor sets it in the
running process, so it takes effect on the next generation with no restart —
and it is written to disk, so a restarted box comes back recording instead of
having quietly stopped.
tools/trajectory_separation.py stays as the terminal door for a headless box.
Verified against a running ComfyUI: toggle both ways, persistence across a
real restart, and the reading recovering a planted early signal.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Every rating-driven mechanism shares one gate (_make_steer_ramp) that opens
only over the last half of the schedule, so a "Missing action" rating has no
window in which it could act. Banking x0_hat per schedule position would give
it one — if the early half carries any signal. This measures that.
Off unless FUNPACK_TRAJECTORY_PROBE=1. Records, never steers. Built for H3:
buckets clamp to few-step runs, batched predictions are refused rather than
averaged with their unconditional row, and each pass binds its own schedule.
Read the result with tools/trajectory_separation.py.
fix: reference numbering counts within its own kind
A reference slot resolves per kind, so marking an image then a video labelled
the video R2 while the only slot reaching it was "Reference video 1" — there
was no way to have an R1 video and an R1 image at once.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Prompt repair lost most of itself in an earlier stage; what was left was a
fallback that appended missing phrases back onto the prompt, reachable only
through a list nothing fills any more. Gone. The axes stay — active_repair_axes
is the "what's missing" signal the advisor reads, and it never edits a prompt
by itself.
prompt_repair and prefer_wired_conditioning are read by nothing at all and only
survive in old saved settings, so the card stops printing them.
Bounded attention is built and has never been run, and its left/right split is
fixed. Both labels now say so.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Never validated on a real run, so 4.0.0 does not offer it. The widgets stay
for a raw ComfyUI graph and the module is marked FOR FUTURE REFERENCE — the
CLIPSeg tube, the upsampler pass and the feathered paste-back are the hard
parts and are worth having if it is picked up again.
The card still hides its settings when it is off, so a graph that does tick it
still reads correctly.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Did not work. The widget, the per-step correction, the denoiser proxy for
samplers whose loop we do not own, the sampler-suitability tables, and its
test file.
The step helper was identity without a clock, so every call site in the
distilled-flow and hybrid loops becomes the value it was passing through.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Not used on H3, not used on LTX any more. Two widgets, the append method, the
Editor's continuity controls for it, and the auto-enable that turned it on for
every multi-scene carry chain.
JoyAI memory and carried i2v guides keep the guide-attention machinery they
share with it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Six widgets (three write gains, prompt loudness, taste push, learned/manual
mode) and the whole rating loop behind them. Never proved out on the rental:
picture gain past +-0.1 adds distortion and latent noise, the rest changed
nothing legible, and the audio gain was never worth testing.
The final-layer picture detail SURVIVES and now reads its own widget — it was
the only one of the set that showed a real effect. Relabelled experimental.
Two things fall out with them. The refinement key no longer holds any sampler
value, so key scoping has nothing left to scope and goes too. And the card's
ownership table now reads dependsVals as well as dependsValue: the Studio rows
use the plural form, so absolute_strength was printing under steer_mode
"relative", where it does nothing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Seven rental runs across its whole range: below 0.5 the picture breaks up,
above 0.5 it reproduces the reference outright, and nothing in between is
worth having. It never did the job it was built for either.
Kept: the final-layer video scale it shipped alongside, which does work.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Measured on the rental and it never worked as intended. Two widgets, the
wrapper, the context-windows exclusion it needed, and its two tests.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
On the card from the rental: Chain Sampler 35 rows -> 4, Studio 13 -> 6,
Studio inputs 2 -> 0.
Three additions. A key the node does not DECLARE is now dropped rather than
kept: ComfyUI builds a node's arguments from its own INPUT_TYPES, so
`block_steer` (removed from this pack entirely) and a `steps` key the Chain
Sampler has never had reached nothing, while printing beside live settings as
if they had. An empty value decided nothing either. And the studio_settings
refiner block is filtered against the Editor's own defaults — it has no node
to ask, so an unrecognised key there is still kept: unjudgeable is not inert.
identity_projector was printing a model filename under a disabled feature. It
is rendered by hand in the Editor so it carries no dependsOn, hence
_OWNED_EXTRA, kept apart so the generated half stays generated.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two rules: skip a value equal to the widget's own default, and skip a value
whose feature is switched off. A region-sharpening prompt printed under a
disabled sharpener read as if it had run, and enough of that buries the few
values that did decide the render. On a realistic project this cuts the Chain
Sampler section from 8 rows to 4.
Defaults are read off the node's own INPUT_TYPES rather than copied here.
Ownership mirrors the Editor's dependsOn wiring, and a test regenerates the
table from engine_settings.js so the two cannot drift. Owner chains are walked
to the end — v2a_grad_scale belongs to joyai_audio_memory, which belongs to
joyai_memory. A section left with nothing to say is dropped.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Measured: 0.4 loose and inventive, 0.6-0.75 several versions of the shot
attempted at once with the strongest winning, 0.75 and up simply the
reference. No usable band anywhere in the range.
The timestep is how the model knows what stage everything is at, so a prompt
that disagrees with the picture about the stage costs it the ability to settle
on one reading — which is what "multiple variants in one video" is. Structure
is chosen early, so the edit now waits until it has been: ramped in through
_make_steer_ramp, the same late-step gate the rating-driven wrappers share,
interpolating from the prompt's own row so a closed gate is byte-identical to
not being installed.
Third hypothesis for this dial. If it still just reproduces the reference, the
feature does not earn its place and comes out.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Measured, after the label fix: at 1.0 it still reproduces the reference
almost exactly, so the <Picture N> label was never the cause. In H3
"finished" is what marks a row as reference material — content that was given
and should be reproduced. Marking the prompt rows that way moves them into
that category, and the reference takes over the frame.
Measured range: 1.0 copies the reference; ~0.4 follows the prompt but invents
things in neither; below 0.3 the picture breaks up. It does not improve
prompt adherence, which is what it was built for.
The widget NAME stays h3_prompt_time — it is in saved projects and saved
workflows, and renaming it would drop the setting from both.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Measured on a rental: 0.8 gave more contrast and 1.2 less, and 1.15 was
nearly inert.
The layer computes norm(x) * (1 + scale) + shift, and the first build
multiplied `scale` itself. H3's scale is negative, so raising the dial shrank
(1 + scale) and the control inverted. Scaling the whole (1 + scale) term
instead is monotone in magnitude whichever sign a checkpoint carries, so
"above 1.0 is more" holds by construction rather than by luck. shift is still
left alone — it is the bias.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Measured on a rental at 0.9-1.0: it reproduced the reference image almost
untouched in the frame, overriding a prompt that said in words the reference
was not where the scene begins.
An r2v conditioning is laid out `<Picture n>: ` label / vision block / prompt,
and the LABEL is text — the vision block is not. The first version re-timed by
modality tag, so it hit the label too. That label already reads as "compose
like this"; made maximally authoritative, the model composed like it.
The prompt is the LAST text run, and nothing outside the text span carries the
text tag, so that identifies it with no help from the conditioning. Row ranges
live only in mod_segments, which AdalnProj never sees — so this moves onto
ComfyUI's own per-block replace hook, which is handed them.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Reported from a rental: typing in the Composer, focus was lost, typing
continued, and a clip was cut in half by an `s`. Focus loss lands on <body>,
which the handler read as "nothing focused, so this is for the timeline".
While a writing surface is open — the Composer, any floating window — letters
belong to it whether or not it holds focus. And with "Show timeline on hover"
on and the timeline collapsed, clip edits are refused outright: a selection
nobody can see gives no sign anything happened. Transport keys stay live in
that case; the player is on screen either way.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The key-scoping strip lived in Project.from_dict, which parses the browser's
SAVE payload as well as the file on disk. So the Editor sent a value, got a
project back without it, and redrew the default — reported from the rental as
the mode switching itself back to learned. Every one of these dials was
unsettable, not just the mode.
The strip now runs only when a project is read from disk. h3_gain_mode and
h3_prompt_time leave the set entirely — a mode is a choice, and prompt_time is
never learned. Manual mode strips nothing: those values are hand-typed, and
there is no learned state to reset.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
fp16 and fp8_e4m3fn_fast both wreck H3's audio before anything looks wrong,
and the loader's own notes go into a status string the Editor never renders —
so the only symptom was a bad soundtrack. One console line, once per run, on
H3 only. Names the fix in the first sentence, which is the half that survives
the console trim.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
h3_prompt_time is not one of the learned strengths, but it was gated behind
h3_gain_mode=manual like the ones that are — so using it meant switching
rating-learned gains off. It now reads its widget in both modes, and shows in
both.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
h3_prompt_time: the prompt is told how finished it is independently of the
picture, so the model leans on it from the first step instead of treating it
as unreliable while the picture is noise. One extra row through each block's
AdaLN projection, dropped before the result returns. Manual dial only — its
off value sits at the end of its range, not the middle, so the rating loop
cannot explore around it safely.
h3_video_detail: crisper or softer picture with the audio bit-for-bit
unchanged. The final layer is past the last attention pass, so unlike every
other picture control it has no path to the soundtrack.
apply_adaln_gains -> apply_adaln_edits; both block edits share one wrapper.
video_detail joins the rating-learned gains.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Validated on the rental and it does not earn its place: 3.5 minutes against 1
for the same clip, worse picture, garbage audio. Some of that was mine — no
weight prefetch across the 50 blocks, a host-offloaded K/V cache, and an extra
commit forward per chunk — but fixing those only buys parity on speed, and the
technique's real win is unbounded length, which H3 already covers natively at
the lengths in use here.
Gone: h3_causal.py, the four h3_causal_* sampler widgets and their Engine
Settings rows, `chunk_causal` on the diffusion loader, and five test files.
KEPT, because it is not about this at all: the LoRA key RENAME table. Stripping
a wrapper head leaves `blocks.0...`, comfy's key map has no bare form, and the
file silently applies nothing — 532 keys, zero applied, no error. That was
breaking any PEFT-style adapter named this way, not just the one that found it.
Its tests moved to test_lora_shape_mismatch.py alongside the MATCHED NOTHING
diagnostics.
Two INPUT_TYPES widgets go with this (h3_causal_* on the sampler, chunk_causal
on the loader). A saved workflow carrying them loads fine — both were optional
and are matched by name — but the values are ignored from here.
Tests: 1700 python, 68 JS.
It was trimmed at "adaLN reads a compact…", which cut every actionable word:
how many dropped, that the rest of the LoRA still applies, and what to do. The
verdict is now the first sentence, and the line asks for its full text — this
only prints when a LoRA is half-landing, which is the one moment the whole
explanation earns its line.
Tests: 1830 python (+1), 68 JS.
Two things, both from the rental.
THE LORA. Its keys are ComfyUI's module paths exactly, under a different head:
base_model.model.dit.audio_patch_proj.lora_A.weight
diffusion_model.audio_patch_proj <- what comfy's map holds
Every candidate the loader tried was a STRIP, which left `audio_patch_proj...`
— and comfy's key map has no bare form to land on, so 532 keys matched nothing
and the file did nothing at all. LORA_KEY_RENAMES adds (prefix, replacement)
candidates, scored against the real key map like the strips, so a rename that
lands beats a strip that does not without either being ordered first.
THE PROGRESS. The causal lane replaces comfy's sampler rather than wrapping it,
so nothing ticked the progress bar and nothing answered an interrupt: a long
generation looked exactly like a hung one, and Cancel did nothing until the
whole clip finished. The rollout now reports every model call, and the sampler
maps that onto the budget this scene was allotted — the bar was sized in
scenes x steps and a chunked scene runs that many steps PER CHUNK, so ticking
raw would have overrun it. The phase readout says "scene 1/3 · chunk 7/22", and
the interrupt is checked before every step, not just every chunk.
Tests: 1829 python (+5), 68 JS.
The rental found it: the RAVEN LoRA reports MATCHED NOTHING. Every chunk-causal
result so far was base H3 driving a cache it has never been trained for, which
is exactly the failure mode the loader warns about — and the warning was
invisible, in a status string the Editor shows nowhere.
Two fixes so the next one is diagnosable:
- MATCHED NOTHING now prints BOTH namings, three keys each: what the file has
and what the model wants. "Matched no weights" is not actionable; the names
are, because they say whether the file is for another architecture (nothing
alike) or the same one under different module names (a rename rule away).
Every entry in LORA_KEY_PREFIXES was found by hand for want of this line.
- a LoRA that matched nothing no longer logs as Active. Loaded is not applied,
and that was the same lie the status string used to tell in private.
funpack_log gains `full=True` for the rare message whose PAYLOAD is the detail,
where the first-clause trim removes the only reason to print it. Not for
emphasis — a feature's state and its reason stay one clause.
Also: the H3 checkpoint-mode note no longer asserts a mismatch it never checked.
Nothing in the state dict identifies fl2va from ref2va, so it cannot know which
file is loaded, yet it said "fl2va weights will read it, but were not trained on
it" on EVERY run with references — including on the ref2va and hybrid
checkpoints where nothing is wrong. It now names the conditioning and stops.
Pins-and-references together stays a warning; that one is a real finding.
Tests: 1824 python (+4), 68 JS.
`keys=384 fmt=stripped base_model.model.dit.` only ever went into the node's
`status` output, and the Editor's fixed graph shows that nowhere. So a LoRA that
matched nothing looked exactly like one that applied — and on the chunk-causal
lane, where the adapter is what the whole mode depends on, there was no way to
tell which had happened.
[FunPack] LoRA raven_streaming_4nfe.safetensors: Active | keys=384 … at +1.000
[FunPack] LoRA stack: Inactive | No LoRA is selected
Tests: 1818 python (+1), 68 JS.
One bug reported twice on the rental. `estimate_memory` calls
`extra_conds_shapes(**cond)`, so prepare_sampling wants ComfyUI's INTERNAL cond
dicts — I passed the [tensor, meta] pairs a node receives, it raised "argument
after ** must be a mapping, not tuple", the load never happened, and the
conditioning then failed on a model still half on the CPU. The second line was
the first line's consequence.
convert_cond now runs first, which is what the dense path does.
A failed load at BUILD time refuses the lane instead of warning: everything
after it assumes the model is resident, so a warning there just resurfaces as a
device mismatch three lines later. Mid-rollout it still only warns — by then the
run is under way and whatever is resident is better than a dead scene.
A ComfyUI without comfy.sampler_helpers falls back to load_models_gpu rather
than refusing; only a real load failure is fatal.
The tests for this now swap prepare_sampling with monkeypatch instead of
install_module. A raising stub merged onto the shared module object poisoned
every test that ran after it — the same parent-attribute trap the suite already
has a note about.
Tests: 1817 python, 68 JS.
[FunPack] K/V conditioning patch: Inactive | Non-LTX model loaded
[FunPack] Chunk cache: Active | 22 chunks, 4 steps each, consistency
`feature(tag, name, active, reason)` is the shape, and `skipped` renders
identically — a deliberate skip and an inactive feature are the same fact and
should not have two spellings. `failed` becomes `Name: Failed | <effect>`,
keeping the effect rather than the exception text: "that explains what I am
looking at" is the half worth the line, and an exception message rarely says it.
Everything funpack_log prints is now cut at the first sentence end, so the call
sites keep their reasoning without spending it on every run. SENTENCE ENDS ONLY
— an em-dash introduces the reason as often as the essay, and a renderer that
guesses wrong eats the one part of the line worth reading. A single long
sentence is hard-trimmed instead of left to run.
FUNPACK_LOG=verbose (or set_verbose) prints the full text, exception included.
Converted to the new shape: the chunk cache, region locks, interior pins, the
taste push, the token-refiner edit, AdaLN gains, rating-learned gains, phrase
emphasis, the K/V conditioning patch, the scene split, Bounded Attention,
per-block LoRA weights, LTX enhancements. The rest of the log gets the trim for
free and keeps its own phrasing until converted.
Tests: 1815 python (+9), 68 JS.
Every chunk restarting from the input image on the rental. The prompt and the
anchor shared one cache chunk, so pinning the prompt pinned the anchor with it —
for the whole clip, in a context of three visible blocks. In a dense render the
anchor is one pin among hundreds of frames' worth of rows; there it was a third
of everything the model could see, and the model did what it was trained to do
with a conditioning row: compose from it.
They are now two chunks. The prompt is 0 and always pinned — it is every moment
of the clip. The conditioning is 1, so `sink` decides how long it stays: 1
releases the anchor after the opening, 2 keeps it (today's behaviour), 3 also
pins the opening shot. Media chunks moved to i + 2, and the conditioning slot is
reserved even on a t2v run so `sink` counts the same either way.
The prefill is two forwards, and the conditioning pass reads the text already in
the cache — so the anchor is still represented WITH the prompt rather than in
isolation.
Note the widget's meaning shifted by one: sink 2 was prompt+anchor+opening shot
and is now prompt+anchor.
Tests: 1806 python, 68 JS. The upstream check now prefills a real keyframe and
confirms sink 1 releases it while keeping the prompt.
extra_conds runs H3's token refiner, which is a partial forward — so preparing
the conditioning before the model is resident gave `mat1 is on cuda:0,
different from other tensors on cpu` and refused the lane.
The load moved ahead of it, and it now goes through ComfyUI's own
comfy.sampler_helpers.prepare_sampling rather than a bare load_models_gpu with
memory_required=0, so the estimate, the wrappers and the placement are the ones
the dense path would have made for this clip. Both call sites share one helper:
a session can be built and run much later, and another model may have evicted
this one in between.
Tests: 1805 python, 68 JS.
The rental is on 0.34.0, whose comfy.ldm.minimax.model has no time_shift_slope
— that function is newer than stable. It came in through the same all-or-
nothing import as the classes the lane cannot exist without, so one absent name
raised ImportError and make_causal reported "this ComfyUI has no MiniMax H3 to
extend". Which then read as a broken install, and before that as a switch the
user had not flipped. They had.
The import is now split. The classes and helpers with no substitute stay hard,
so a genuinely too-old ComfyUI is still named precisely by the ImportError.
The ones that arrived later are read with getattr and fall back to what THAT
ComfyUI's own dense forward does.
For time_shift_slope that means not scaling the audio velocity at all: 0.34.0's
_forward returns `-audio_out` unscaled (verified against the tagged source), so
applying the correction here would put the chunked lane on a different audio
grid than the dense path on the same install. Where the function exists, it is
applied exactly as before.
Tests: 1804 python, 68 JS. The upstream check now deletes time_shift_slope and
drives a chunk through the lane without it.
The refusal asserted a cause it had never checked — "switch on chunk_causal" —
which was wrong, and sent you looking at a switch that was already on.
Three changes, none of them a guess:
- build_session no longer refuses a stock H3. Re-classing adds no parameter,
renames no key and costs nothing, and with no cache passed the model is
bit-identical to stock — so a mode that is already switched ON has no
reason to refuse itself over a switch on another node. It does the work.
- when it does that, it says so, naming the loader as the end that did not
take. That is the line that tells us whether `chunk_causal` is off or its
value never reaches the node.
- the loader PRINTS its chunk-causal outcome. It only ever went into the
`status` string, which the Editor's fixed graph shows nowhere, so a
"REQUESTED BUT NOT APPLIED" was invisible and the sampler downstream had
nothing to go on but a guess.
If make_causal genuinely cannot run, the message now names the diffusion
model's actual class instead of asserting what the user did.
Tests: 1804 python (+3), 68 JS.
The chunk-causal switch replaced a `raven_lora` combo on the diffusion loader
with a boolean plus an ordinary LoRA, but the Chain Sampler tooltip and the
Engine Settings detail still told the user to go and set the old widget. Both
now name what is actually there. A test refuses the string anywhere a user can
read it.
Tests: 1801 python (+1), 68 JS.
The two entry points the rollout was waiting on — prefill_text and forward_chunk
— are written, and with them the whole lane is ours. raven_causal.py is gone:
its pure parts (step rules, the audio sigma remap, the rollout loop) moved into
h3_causal.py, and its bridge to ComfyUI-MiniMax-H3-RAVEN-Streaming is deleted
along with the probe that hunted for that package in custom_nodes. Nothing has
to be installed for the mode to exist.
How a chunk is cut: ONE full-clip PackedLayout is built by ComfyUI's own
extra_conds, and every chunk is a SELECTION of its rows. So a chunk's RoPE
angles, modality tags and timestep classes are the ones a dense forward would
have given those same rows, and the keyframe pins and reference blocks keep the
rows upstream packed for them — which is what makes i2v and r2v work here
rather than be refused. Those rows are the prefix: cached once as chunk 0, so
media chunk i is cache chunk i + 1 and a sink of 2 pins the prompt plus the
opening shot.
A chunk is committed CLEAN — one extra forward at sigma 0 after its last step.
Caching the last denoising step's K/V would hand every later chunk a context
that still carries that step's noise.
Two failures that used to be a crash fifty blocks deep are now refusals by
name: a clip whose soundtrack does not cover its picture, and a clip too short
to have anything for a cache to remember.
Tests: 1800 python (+1 net; the RAVEN bridge's tests are gone, the plan,
session and real-model checks are new), 68 JS. The new upstream check drives
the REAL DiT at toy width through both lanes and confirms the safety property
holds: with no cache passed, the causal model is bit-identical to stock H3.
Not yet run on the rental.
Untying them from the project file was the ask; taking the dials away was not.
The six rows are back in Engine Settings exactly as they were.
What stays from the previous commit is the part that answers the ask: they are
stripped from sampler_inputs on load, so a dial moved in a session applies to
that session and the next open reads the key — or the defaults when no key is
active or the key files have been deleted.
Tests: 1777 python, 68 JS.
Asked twice, and the first fix answered the wrong question. Clearing a dial on
its default stopped the value being STORED, but it was still a project field —
so deleting every refinement key left the old behaviour applying, and the only
way back was typing the neutral value in by hand.
These knobs are learned from ratings and live in the key's own state. A copy in
the project file was a second source of truth that outlived the key it came
from. So:
- h3_gain_mode / h3_gain_video / h3_gain_prompt / h3_gain_audio /
h3_prompt_scale / h3_taste_bias are stripped from sampler_inputs on project
load, so a project that already carries them heals itself on first open
rather than needing a hand-edited JSON;
- they are no longer offered in Engine Settings at all, because offering one
writes it back the moment it is touched — which is how the second copy got
there;
- with no key active the sampler already returns neutral, so deleting the key
files now genuinely resets the behaviour.
Still widgets on the NODE: a raw ComfyUI graph with no Refiner in it drives
them by hand, which is what h3_gain_mode 'manual' exists for. Key-scoped means
not in the project, not removed.
Tests: 1771 python (+15), 68 JS.
Every message about loading the chunk-causal adapter named FunPack Apply LoRA
Weights. That is the wrong node: it is the optional stack producer for
prompt-specific trained strengths. FunPack LoRA Loader loads AND applies from
its own file list, and always has — so the instruction sent the user to add a
node they do not need.
Its own description read the same way ("Loads LoRAs from a FunPack LoRA
stack"), which is where the confusion came from. Both it and the docstring now
say the stack is the narrow case, not the way in.
Tests: 1767 passed (+5).
Both halves now go through nodes we already own, so nothing third-party has to
be installed for the feature to exist.
The diffusion loader gains `chunk_causal`, which RE-CLASSES an already-loaded
H3 in place. That is what removes the need for a separate loader entirely: the
alternative is reimplementing ComfyUI's whole diffusion-model load path so the
adapter can be attached before the ModelPatcher exists and its memory ledger
counts it. Re-classing adds no parameter, renames no key and copies nothing —
only which forward runs changes — so the ledger and offload accounting are
untouched and the switch is idempotent.
The adapter is then an ORDINARY LoRA through FunPack Apply LoRA Weights. Our
loader already scores prefix variants against the model's real key map; it just
did not know this one. RAVEN's keys are `base_model.model.dit.<path>`, so
stripping only `base_model.model.` left `dit.` in front of every key and the
whole file matched nothing, silently. One entry in LORA_KEY_PREFIXES fixes it.
A ComfyUI too old for H3, or a non-H3 model, declines with a reason rather than
raising — one optional switch must not turn the loader into a dead node.
Tests: 1751 passed.
First half of owning the causal lane outright, so the feature needs no
third-party package installed.
h3_causal.py: a ChunkKVCache with sink/window eviction and host offload, the
chunk cut, and CausalAttention / CausalDiTBlock / CausalMiniMaxH3Model as
subclasses of ComfyUI's own H3 classes — same module names, same state dict, so
weights load with no remapping and a PEFT LoRA attaches through ComfyUI's
ordinary adapter (which already reads lora_A/lora_B keys).
The safety property: with cache=None every forward defers to its parent, so the
dense lane is bit-identical to stock H3 and nothing here can degrade an
ordinary generation.
Geometry is BORROWED, not re-derived. A chunk's RoPE positions must be the ones
the dense layout would have given those rows, so the full clip's grid is built
with upstream's own _frame_grid / _video_grid / _audio_grid and sliced per
chunk. An upstream grid change is inherited rather than silently disagreed with.
Two traps handled explicitly:
- pack_audio is channel-major over the WHOLE clip, so a chunk's audio rows are
TWO spans, not one. A contiguous slice would hand the model the wrong half
of the stereo field, quietly.
- blocks are RE-CLASSED in place, adopting submodules by reference, never
wrapped. Wrapping renames the state-dict keys of everything inside, which
has broken unpatching in this project before.
Tests: 1751 passed (+21). prefill_text/forward_chunk still to come.
FunPack owns its loading pipeline, so the causal lane must not need a
third-party loader node in the graph. `raven_lora` on FunPack Diffusion Model
Loader turns an ordinary H3 checkpoint into the chunk-causal DiT with the
adapter attached. Same checkpoint either way — it is a LoRA, there is no
separate RAVEN model file, and every message now says so.
The weight load itself is DELEGATED to the RAVEN package rather than
reimplemented, and not for convenience: the adapter is an FP32 activation
residual registered on the base Linear leaves and must be attached before the
ModelPatcher is built. ModelPatcher.model_size() memoises, so an adapter
attached afterwards is invisible to Comfy's memory ledger and never moved by
partial CPU offload — the model would report itself smaller than it is and the
residual would end up on the wrong device.
Selecting a LoRA when the causal class is unavailable RAISES with the fix in
the message, rather than quietly loading a plain model: a model without the
adapter would leave the causal lane reading an attention pattern nothing was
trained for, which reads as a quality problem with no visible cause.
Strength is fixed at 1.0 — the adapter is what teaches the chunked attention
pattern, so "partly on" is not a useful state.
Tests: 1730 passed (+12).
Found by asking how CFG=1.0 reaches the model. It reaches it through
comfy.sample.sample_custom -> sampling_function, which sets uncond_ = None at
1.0 so the negative branch is never evaluated. Nothing wrong there.
But the causal lane bypasses sample_custom entirely, and sample_custom's
prepare_sampling is also what calls load_models_gpu — which is in turn what
applies a patcher's OBJECT PATCHES (partially_load -> patch_model). So on that
path the weights could still be offloaded AND every object patch FunPack
installs would silently not be there: the per-modality AdaLN gains and the
token-refiner edit (prompt scale + learned taste bias) would all have been
absent, with nothing reporting it.
Loaded once for the whole rollout, before the first forward — not per chunk,
which would fight Comfy's own offload decision all the way down the clip. A
failed load degrades to whatever was already resident and says the patches may
be missing, rather than turning a placement problem into a dead run.
Tests: 1719 passed (+5).
A mode on the existing sampler, not a new node. h3_causal_chunks generates the
clip in time chunks that each attend to a KV cache of the ones before them, so
a long clip stays one continuous shot with real memory instead of separate
scenes stitched together. Sink and window are exposed as what they actually
are: chunks pinned from the start, and recent chunks still remembered.
Two defects found while auditing the bridge I had sketched earlier (which a
blanket `git add -A` had swept into an unrelated commit — it was dead code, but
it should not have gone in under that message):
- it read request.video_channels / audio_channels, which do not exist on
RAVEN's LatentRequest. First real call would have been an AttributeError.
- it handed BOTH streams the same sigma. H3 denoises audio on its own shifted
grid (3 against video's 12), so that is exactly the error h3_audio_clock
exists to correct, and it bites hardest on the few-step schedules this lane
is built for. Audio now goes through the same remap, transcribed from
comfy.ldm.minimax.model.time_shift_sigma and verified equal to the live
function across the range.
The lane REPLACES the sampler rather than wrapping it, so every reason it
cannot run falls through loudly: not H3, pack not installed, stock
bidirectional model. A rollout that fails mid-way raises rather than quietly
falling back to ordinary sampling, which would return a clip under a mode the
user believes ran. The run states that the wired sampler and CFG are unused on
this path, and that there is no per-chunk redo.
Editor: Settings > Engine > Sampler > Experimental, hidden entirely off H3.
Tests: 1714 passed (+18). Never run on hardware.
The Movie Editor's builder forces split_by_transitions on for EVERY project, and
the per-scene conditionings replace the base entry — so enhancing the base
prompt as well was a second full generation whose result was then discarded. At
the ~59s per call measured on the rental that is a minute of pure waste per run,
and it looked like thinking mode being stuck on.
It was not thinking: that defaults off and is passed explicitly, so it can never
inherit the repair advisor's own default of True. It was two code paths both
doing the work.
Now the base is enhanced only when the run does NOT split, and the skip says so
on the console rather than being silent. One generation per prompt that is
actually encoded.
Tests: 1696 passed (+6).
The 2000-character cap was wrong. H3 has no padding, so every prompt token is a
real sequence row and a long prompt is a legitimate thing to want — an enhanced
one especially. Cutting at a fixed length contradicted the token limit the user
had just set.
Length cannot tell "long and detailed" from "stuck". What distinguishes them is
that a model which has stopped writing starts repeating itself, so that is what
is detected now: three occurrences of the same sentence and the prompt is cut
at the first repeat. A long prompt that never repeats is never touched.
The only remaining length bound is the one the user's own token limit already
implies (~4 chars/token), so there is one control rather than two fighting.
Short recurring lines are exempt — "She smiles." legitimately recurs in a shot
list.
Tests: 1690 passed.
