Studio gains an enhanced_prompt output; the link wires to it. Enhancer off or failed sends prompt + postfix unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ComfyUI tokenizers drop an unknown system_prompt kwarg, and Gemma skips its chat template by default. Instructions now go in the user turn of the model's own template, which also places the image.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- region lock: layout patch matched the old PackedLayout signature and crashed every H3 run; interior pins now use upstream's native any-frame placement
- REINS/Q-steer/AV decouple/probes: masks cover the target video/audio rows only, not pins, refs or vision pads
- video detail: works with the PDD FinalLayer signature; now in Engine Settings
- Q-steer: steers queries before RoPE, not after (old banks are not reused)
- ALG anchor: blurs the frame-0 pin on H3
- temporal style: hidden on H3
- shadow negative: main-window chip names what it switches off without compose
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Own tab with its full sampling settings (temperature, top-p, top-k, min-p,
repetition/presence penalty, seed, greedy) and a last-run before/after view.
Studio's source_image now reaches the enhancer on the per-scene path too.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The media bin's grid always requested the FULL original file for a
thumbnail (an <img>/video-frame decode of the whole thing, only
cropped down visually via CSS/canvas afterward). A 4K photo or a
multi-MB clip got fully downloaded and decoded just to show a 96px
square, every render.
Added GET /api/media/{mid}/thumb: generates a small JPEG on first
request (Pillow resize for images, one ffmpeg frame-grab for video),
caches it under media/thumbnails/, and serves that instead. Deleting
a media item removes its cached thumbnail too. mediabrowser.js now
loads from this endpoint for both image and video kinds -- the same
canvas-drawing path as before, just fed a file that's already small
instead of the original. Measured live: a 1.57MB test clip's cached
thumbnail is 1.6KB.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
_select_best_seed called value_fn.forward()/compress() directly on a
candidate's captured prediction with no device handling -- the value
function loads CPU-only (map_location='cpu'), but the candidate tensor
lives on whatever device the model ran on (cuda in production). Every
candidate failed to score on every real GPU generation since this
feature shipped; _select_best_seed always fell back to the unchanged
seed, silently a no-op the entire time despite passing all tests
(the suite is CPU-only, so a mismatch never had two devices to hit).
Every other value_function.py call site (gradient/ascend/search) moves
its input onto next(self.parameters()).device first -- this was the
one path that skipped that. Mirrored it: compress on the candidate's
own device (cheap reduction), then move the small compressed vector
onto the value function's device before scoring.
Added a regression test using a real second torch device (mps on this
box, cuda in CI/production) to reproduce the exact cross-device
failure, and fixed three pre-existing value_fn test doubles that were
missing parameters() and had silently started hitting this same
crash branch once the fix started calling it.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Compose-mode message and Editor tooltip were stating a false remedy
(turn REINS down to passive-capture-only/strength 0 to free blocks) --
REINS claims all 50 blocks the moment it's on at all, strength only
gates injection not installation. Corrected to say only fully
disabling REINS frees any blocks.
_select_best_seed had three silent failure paths (no usable
prediction, a scoring exception, zero candidates scored) that burned
real GPU compute and reported nothing. Added prints to all three.
Added _output_value_fn_sample_count so "not ready yet" messages show
the actual banked sample count instead of leaving the user blind on
how close they are to the threshold.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Adversarial review of 77cf37e found a real, cheaply-closable test gap: this
file already calls node.sample() end-to-end with a monkeypatched value
function for the explore_first_step tests, so the same pattern (plus
spoofing minimax_h3.is_h3_model) proves the snapshot save is no longer
skipped for H3.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
h3_shadow_negative's non-composing install was silently starving REINS/
Q-steer of their own per-block capture, so ratings never trained them while
shadow negative was on (0/0 liked/disliked no matter how much was rated).
Add h3_shadow_negative_compose ("Allow when REINS/Q-steer are active"): when
on, shadow negative skips any block another dit-patch mechanism already
claimed, leaving it to keep capturing/steering there, and only pushes on the
rest. Structural coexistence, not per-block math blending -- disclosed as
such in the docstring and tooltip. Off by default, unchanged behavior.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Native port of the community "H3 Shadow Negative" pack's Basic scope: runs the
negative CONDITIONING (already threaded through the sampler, otherwise dead
weight since H3's CFG is fixed at 1.0) through H3's DiT blocks as a second
shadow stream and pushes the real stream's attention output away from it,
NAG-style. Non-composing with REINS/Q-steer/av_decouple/block-repeat/
attn_temperature by design -- replaces a block's forward outright and prints
which existing patches it superseded.
Full Editor (Cutting Room) UI exposure lands in this same change, along with
explore_first_step's own missing Editor UI from an earlier session.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Tries N candidate seeds with one throwaway denoising step each, scores
each with the trained output-space value function, commits to the
best-scoring seed for the real full pass. Candidates are scored fully
isolated from the scene's own guidance stack (model_function_wrapper
chain, context_windows, v2a_grad_scale, identity_overlap -- each torn
down and reinstalled around the candidate-scoring window) against the
scene's fully-finalized conditioning. Any failure in the value
function, schedule, or the isolation cycle itself degrades to a
silent no-op, declared in the run report, never a crash.
Also hardens _install_context_windows/_install_v2a_scale to roll back
partial state on failure (a leaked context_handler key was silently
re-enabling windowing for later scenes) and fixes the original
non-explore v2a install call site to degrade like its sibling
mechanisms instead of aborting the render.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Corrected the actual design: encode two FULL conditionings ($style
SHORTCUT1 $postfix and $style SHORTCUT2 $postfix) and average them row by
row -- (1-strength)*A + strength*B per position, strength=0.5 giving
exactly (A+B)/2 as originally specified. The prior mean-vector version
collapsed phraseB's whole sentence into one flat direction applied
identically everywhere, discarding its own word-by-word content.
Resamples the shorter side (linear interpolation along the token axis)
when the two shortcuts tokenize to different lengths, so both sides line
up for the average without changing the output's token count.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Confirmed on the rental: the previous version (phrase-scoped shift) showed
zero visible effect even at strength 1.5. Qwen's rows are contextual, so
every word in "a cat sits on a mat" already carries "sits", not just the
verb's own row -- editing 1-4 rows out of the sentence left the rest still
pointing at phraseA's action. Now shifts the whole prompt-token range toward
the alt sentence's own whole-prompt mean.
Multiple [a|b] spans in one prompt are additive: each span's delta is
measured against the original conditioning, not against a prior span's
already-shifted result, so they don't compound order-dependently.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Default 0.5 shift left phraseA dominating the video. [phraseA|phraseB:N]
overrides strength per blend. Fixed a shared regex gap along the way: every
weight/strength markup in this file (not just blend) silently rejected
leading-dot decimals like ".8" -- fell through to literal text with no
error, reproducing the exact "override did nothing" bug this feature exists
to fix.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
A real shortcut expanding to "wide shot: character walks" made [a|b]
never match at all, no error, because the previous fix banned ':' and
'@' anywhere in either phrase to stop one dangerous shape (an alt
phrase ending in ":1.5"). Two narrower shape-guards were tried and each
let a different shape through (colon-then-@, an em dash) while still
blocking ordinary prose. Reverted to fully permissive: a markup-shaped
alt phrase just reaches Qwen as literal punctuation in its own separate
re-encode, the same accepted trade-off this file already makes for a
weighted phrase's own stray parentheses -- a quality footnote, not a
leak, confirmed by tracing the encode path and its existing exception
handling.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Adversarial review of the previous commit found two real leaks and fixed
them: a colon in an alt phrase reached the real text encoder unstripped
(no bracket survives to re-catch it after substitution), and a wired
conditioning re-blended a second time because the re-tag guard only
checked for tag absence. Also reports per-span placement failures instead
of only counting successes.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Measurement only, opt-in from Settings > Refinement & Taste. Reading map is free
(attention share on the phrase's keys per block); response map costs one extra
forward per phrase per model call. Phrases are numbered, never quoted.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A latent unsharp on H3's 24-channel latent does not translate to picture
sharpness. Wrong mechanism, not wrong number; removed rather than tuned.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The parser existed but nothing called it; the syntax went to Qwen as text.
One pass strips both markups so timed and weighted spans share one clean text;
both encode paths carry them.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The editor strips the brackets before the encoding node sees them and hands the
windows to Studio as char spans on the text it gave that node.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Query rows are attended in chunks split at window edges, each with its own key
bias: exact, no SxS mask, no extra model call. Outside the window the phrase is
masked from video rows (H3-World single-egress); audio and text rows never see a
window. Composer lists the windows under the prompt.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
h3_explore_temperature was positive-only (flattens the softmax -- more
exploratory, pulled away from the prompt's own weighting). Same knob now
also takes negative values (widget min -0.9) to do the opposite: sharpen
attention so it commits harder to what the prompt already made confident,
instead of only ever being pulled away from it. Same code path (temperature
= 1 + strength, dividing Q before the dot product), sign decides direction.
Floored at 0.05 so an out-of-widget-range strength can't drive temperature
to zero or negative. Fixed a NaN gap the floor doesn't cover: `!= 0.0`
(needed so negative values install at all) is also true for NaN, and
max(1+nan, 0.05) is nan, not 0.05 -- excluded explicitly.
Editor UI widget (engine_settings.js) updated to match.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
aimdo's malloc-graph compiler assumes every block-scope forward allocates the
same pattern call to call. REINS while actually injecting (not passive
capture) and block repeat both patch what a block's forward does, which
crashed on a real rental with "aimdo memory compile error" -- confirmed via
instance log: identical workflow succeeds in passive-capture-only mode
(injection branch never entered) and fails the moment strength > 0. This is
a known upstream conflict class (Comfy-Org/comfy-aimdo #16144, #16342), not
specific to REINS; their fix is --disable-comfy-compiler, read live per
malloc-graph scope rather than cached at startup, so toggling it in Python
around just the affected sample_custom call works without touching the
(templated, rental-fixed) launch command.
Gated on actual install success (model identity before/after each
installer), not on the raw request flags -- a silently-refused install
(non-contiguous span, no block list, zero strength) leaves a block's
forward genuinely unpatched, so forcing the compiler off there would be a
pure, needless perf/VRAM regression instead of a fix.
Also: FUNPACK_AIMDO_LOG env var to raise comfy_aimdo's own native log level
to DEBUG, for diagnosing this class of failure directly from aimdo's side
when needed.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The previous caching test only exercised the zero-direction torch.zeros()
fallback -- round 1's actual bug lived in the real-direction branch
(direction.to(out.dtype).to(out.device)), which a torch.zeros count can't
see at all. Adds a second test using a deliberately non-matching dtype
(float64 direction against a float32 harness) so .to() is never a
coincidental no-op, then counts .to() calls on the direction tensor itself.
Verified by mutation: temporarily reverted dir_t_cache to round-1's buggy
shape (real-direction path uncached) and confirmed this new test fails
(3 calls instead of 1) while the old zero-direction-only test still passes
-- proving the old test alone would have let that exact regression through.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The caching logic itself (a block's second+ call reusing the built
direction/zero tensor instead of reallocating) had no regression test --
only the single-call math was covered. Counts torch.zeros calls across
three calls to the same block; only the first call's mask-cache-miss and
dir_t-cache-miss should allocate, the rest should be pure cache hits.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Adversarial review of the leak fix found the new tests only exercised REINS
stacked on itself, leaving 6 of 7 install mechanisms and the entire
optimized_attention_override chain (av_decouple/attn_temperature/q_steering)
untested. Add a test stacking REINS -> av_decouple -> q_steering -- different
mechanisms chaining onto each other, the realistic multi-run leak shape -- and
confirm one strip call unwinds both the per-block dit dict and the single-slot
attention override chain.
patches_replace["dit"] and optimized_attention_override installs had no tag+strip
protection, unlike the scene-wrapper and block-hook mechanisms fixed for the same bug
class earlier. An interrupted run left these hooks on the shared model; the next run
chained a new layer onto the leaked one instead of starting clean, growing unbounded
across interrupted runs -- the likely cause of the live "potential memory leak...
circular references" warnings and slow full-GC pauses.
REINS ("h3_repr_steering") only ever captured while its own toggle was on --
turning it off also stopped it from learning, with no way to bank rating
history separately. New h3_repr_steering_passive_capture keeps the existing
unconditional per-block capture running (forcing strength to 0 so nothing is
ever injected) even while the main toggle is off. Exposed a real dormant bug
in REINS' own console print: it claimed "applying learned direction(s)"
whenever a direction existed, regardless of strength -- harmless before
because the toggle and strength widget always moved together, not once
passive capture legitimately runs at strength 0 with real data available.
Fixed to report "capturing only" in that case, mirroring h3_q_steering's own
already-fixed version of the same print.
Also: h3_q_steer_block/h3_q_steer_strength (from the H3 query steering
feature) were never wired into the Movie Editor's Settings UI, only exposed
as raw ComfyUI node widgets -- missing this project's own rule that every
new sampler toggle lands in the Editor in the same change. Added both,
plus the new passive-capture checkbox, to Engine Settings > Chain Guidance
alongside REINS' existing controls.
Two real faults found by a third adversarial review round on 4154746:
- _install_h3_av_decouple's own bail check (`_strength <= 0.0`) disagreed with
_sample_chunk's install guard (`h3_av_decouple > 0.0`) for a NaN strength:
every ordered comparison is False for NaN under IEEE 754, so `<= 0.0` let a
NaN through as "on" while the guard treated the same value as "off" --
reproducing the exact av_decouple-defeats-q_steering bug the last fix
closed, just reached through a different value. Both now use the same
`not (x > 0.0)` predicate so they can't disagree.
- h3_q_steering was installed before h3_explore_temperature, so if both
target the same block, temperature's Q-scaling reached q_steering's
capture before it ran -- banking a temperature-distorted "natural Q"
descriptor, contradicting the mechanism's own documented ordering promise
(capture reflects the network's natural Q, same claim h3_repr_steering
makes for hidden state). q_steering is now installed last (outermost),
after both av_decouple and attn_temperature, so it always sees Q first.
av_decouple still can't compose with q_steering at all regardless of
order (unrelated masking issue, already handled by skipping install).
Regression tests for both, plus a NaN case added to h3_av_decouple's own
existing zero-strength no-op test.
Two real faults found by a second adversarial review round on 608a97a:
- h3_av_decouple, installed after q_steering, becomes the OUTER
optimized_attention_override and always calls its inner override with a
non-None mask for the video-row partition -- exactly the case q_steering's
own override treats as "already handled, back off". With both enabled,
q_steering captured and injected nothing while still printing "applying"
and paying the clone/hook cost. The two can't currently compose, so
q_steering is now skipped entirely (with a declared reason) whenever
h3_av_decouple is also active, rather than silently no-op.
- conditioning.py's q_steer commit call was silent on no_pending/no_key,
justified by a comment claiming REINS' own commit is silent there too --
that was wrong, REINS is loud on exactly those outcomes. Fixed to match.
Also tightened the shape-mismatch warning's wording: it checks total element
count, not the heads/head_dim split, so a same-total-different-split
mismatch would still reshape onto the wrong head boundaries undetected --
documented as a known narrow gap rather than fixed, since a real fix would
mean changing the persistence format REINS itself also depends on.
Added regression tests for the av_decouple interaction (both directions:
guard fires when it should, stays quiet when it shouldn't).
Three real faults found by adversarial review of 299941a:
- sample()'s own default for h3_q_steer_block was "0,1" while INPUT_TYPES
declared "" -- any saved workflow predating this widget omits the key
entirely, so ComfyUI falls back to sample()'s default, silently turning on
capture/clone/hook for every H3 generation with a refinement key. Now both
default to blank, matching every sibling knob in this file.
- The "applying learned direction(s)" console line fired whenever a direction
was available, not when it was actually applied -- strength=0 still printed
it, which is the exact silent-success-message trap this project has already
been burned by. Now split into an "applying" line (strength>0) and a
"capturing only" line (strength=0).
- A direction banked under a different heads*head_dim layout (model swap on
the same refinement_key) hit an unguarded .view() and would hard-crash
generation instead of degrading. Now checked and skipped with a one-time
notice, same pattern as the existing batch>1 guard.
Added regression tests for all three, plus two coverage-gap tests the review
flagged (kind isolation against REAL persistence, not mocks; two named blocks
steering with independent directions in one run).
Same liked-minus-disliked mean-difference as h3_repr_steering, captured off Q inside
optimized_attention_override instead of a block's hidden state, own kind="q_steer" sidecar
so the two vector spaces never collide. Only Q is biased: a uniform additive bias on K is a
provable softmax no-op, and on V it's mathematically identical to h3_repr_steering's own
mechanism, so neither is a genuinely different lever. Off by default (blank block spec).
h3_repr_steering.py's persistence functions take an optional kind= param (default unchanged)
so this reuses capture()/direction()/commit() verbatim instead of a parallel implementation.
optimized_attention_override was a single dict key that h3_av_decouple claimed
outright -- a second mechanism on the same slot would silently replace it with
no error, the exact bug class dit_patches chaining already guards against
elsewhere. Fixed so any install chains through whatever's already there.
New h3_explore_temperature (+ block spec): flattens attention's own softmax at
named blocks instead of adding ungated latent noise -- variety stays bounded
by the prompt since it can't attend to anything not already in context.
Deliberately NOT wired to REINS or any rating data: neither mechanism's
behavior is understood well enough yet to make coupling them debuggable. A
plain manual dial, same as block repeat. Off by default.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
No way to read H3's actual trained attention-logit scale from this machine
(checkpoint isn't here) -- so normalizing the knob to 0-1 against a hidden
made-up multiplier (12.0) was a second guess wearing a calibrated costume,
and exactly the kind of thing that produces another flat live-tested curve.
The knob is now the raw softmax logit penalty directly (0-50, step 0.5), so
a live sweep searches the actual unit instead of my conversion factor.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Live 0-1 strength sweep came back flat -- confirmed the mechanism, not just the
value, was wrong. H3's Q/K get RMSNorm'd before attention, so scaling a key
toward zero pushes its logit toward the *mean* of an already near-zero-centered
distribution, not toward excluded. Switched to what actually suppresses a
softmax entry: an additive negative bias on the targeted logits, still passed
through the same per-query-group split (still no S^2 mask, still same total
FLOPs -- the bias is a per-key vector broadcast over the group's queries).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
H3 has no cross-attn to mask (unlike LTXAV's Bounded Attention), so a dense
attention mask would be GB-scale over its ~37k-token packed sequence. Uses
comfy's own optimized_attention_override extension point instead: runs the
same joint attention as separate video-query/audio-query passes against a K
where the other modality's rows are scaled down, at the same total FLOPs as
one pass. New h3_av_decouple sampler input + Engine Settings entry, off by
default (0 installs nothing).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The sweep's rating feature was wired against REINS' single 'pending'
capture (one slot per key, overwritten by whatever generates next),
which meant only the MOST RECENT result in a whole batch could ever be
rated -- the opposite of what batch training actually does: rate every
result you generated, independently, in any order.
New h3_repr_capture_slot sampler input (blank by default = unchanged
ordinary one-at-a-time flow) saves a run's capture under an explicit
slot instead of the single pending. New h3_repr_steering.save_capture_slot
/ commit_slot / discard_slot mirror save_pending/commit but keyed, so a
whole batch stays independently rateable the same way batch_training.py's
per-item .cond.pt files already do for value-function training.
New server routes commit/discard a slot directly -- no extra generation
needed, unlike the old 'fire another render with rating attached'
workaround that existed only because of the single-slot limitation.
Every gallery card is now rateable (Like/Dislike/Skip), not just the
last one.
h3_repr_steering_block goes from a fixed combo (18 candidate blocks) to
free text -- a single block, a range, or a comma list, same syntax as
Repeat block(s) (31-40 / 4,5,6). Each named block steers with its OWN
learned direction from its OWN liked/disliked history at that block --
there is no shared 'the direction', just N independent ones applied at
their own positions in the same forward pass. A block short on rated
data at itself just captures, same as any unnamed candidate block.
CANDIDATE_BLOCKS widened from 18 curated blocks to all 50 -- every block
is now both capturable and steerable, not just the ones on the old grid.
Fixed a pre-existing sync gap: h3_block_repeat_last_steps was never added
to settings_card.py's ownership/defaults tables, breaking two golden-file
tests as soon as they ran (test_settings_card_noise.py).
