When temporal variance of the latent drops below 5% of overall variance,
applies a normalised deviation gradient that pushes frames apart from each
other. Fires only at sigma < 0.5 (same window as other latent guidance),
uses embed_guidance_strength as scale. Active whenever embed_guidance is on
and liked_dir is available — no user input needed.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Unconstrained ascent was stripping action content from conditioning to reach
a "safe" static-character attractor. Added 8% displacement cap relative to
original conditioning norm — quality direction is still steered but the
prompt content can't be erased. Also raised MIN_SAMPLES from 5 to 10 to
reduce overfitting risk before the value function activates.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
closePicker() didn't remove the mousedown listener, so clicking the button
to re-open the picker would immediately close it. Now stores the cleanup
in module scope and calls it from closePicker().
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Trains a second OnlineValueFunction on rated video latents (compressed via
channel mean-pool). At each denoising step, computes ∂reward/∂latent through
the lightweight MLP and applies it as an in-place drift to the noisy latent —
gradients never touch the 22B transformer. Controlled by embed_guidance_strength
alongside the existing conditioning guidance. Studio toggle: "Latent guidance".
Cleared on session reset alongside the conditioning VF.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Heart modifier now applies full axis pressure (not 50% softened) — meaning
"fix this specific issue but the quality direction was great." The quality
signal comes from boosted reward and conditioning delta, not from reducing
axis pressure. Nailed it + heart removed (that's just Perfect). Positive
grid changed to 1fr 1fr to fill width with 2 items.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The ♥ button on any rating covers the same semantic. Removed from picker
UI and V2_RATING_LABELS; profile kept in Python for backward compat.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Active class was static (set at creation). Now all option elements are
tracked and active state is updated dynamically on each pick.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
ComfyUI rejects widget values not in the allowed list. Generate all
valid label|loved combinations at definition time.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Each rating option now has a small ♥ button on its right. Clicking the main
label applies the base rating; clicking ♥ applies label|loved. No toggle
state — simpler and more direct.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Any rating can now be combined with a "Loved it" toggle. The suffix |loved
is appended to the rating string and parsed in normalize_refiner_v2_rating.
When active: reward boosted to max(base, 0.85), axis pressure softened 50%,
conditioning direction delta raised to 0.45, preferred context applied at
0.30 base_delta. Disabled for Perfect (redundant) and -Just forget it-.
Button label shows ♥ when modifier is active.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
addDOMWidget clicks were swallowed by the canvas. Switched to addWidget
button which ComfyUI routes correctly. Picker now positions at mouse coords
from the widget callback event.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replaces the rating dropdown on Studio and Refiner V2 nodes with a button
that opens a categorized floating picker. Each option shows its reward value
and a one-line hint. Rating auto-resets to -Just forget it- after execution
so forgotten ratings don't silently apply to the next run.
Also adds Wrong action + quality (reward -0.40, wrong_axes action, missing
quality) and drops Wrong action reward from +0.10 to -0.10.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Restores the old "Loved it" learning behavior (reward 0.75, intent delta
+0.03, preferred context 0.65, axis conditioning 0.35, LoRA 0.44/0.24)
as a distinct rating. Sits between Perfect and Loved it on the ladder:
prompt adherence is rewarded but not at full strength.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Previously a weaker "Perfect". Now means: output looks great, prompt
adherence is irrelevant. Skips missing_intent_phrases learning, skips
preferred context lock-in, zeroes intent phrase reinforcement. Keeps
conditioning direction delta and raises value function reward to 0.85.
Quality/style LoRAs rewarded at Perfect level (0.55x).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Two server routes (GET /funpack/value_function/export, POST /funpack/value_function/import)
serve and receive the .pt binary. Studio Session tab gains Export and Import buttons
below the value guidance toggle, keyed to the active refinement key.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
torch.load under inference_mode creates inference tensors; MLP params then
can't participate in backward. Same fix as the earlier value function loader
in samplers.py.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
reset_session now deletes the value function .pt file alongside the
enhancement files. Previously it was orphaned across resets.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Before returning modified_positive, ascend each scene's conditioning tensor
toward higher predicted reward via the value function. Covers both the
standalone Refiner V2 node and FunPack Studio (which delegates to refine_v2).
No-op until value function reaches MIN_SAMPLES; handles transition-split
multi-scene conditioning correctly.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Before each denoising run, when the value function is ready, gradient-ascend
the text conditioning until predicted reward plateaus (self-terminating: step
size = 0.5% of conditioning norm, stops when delta < 1e-3, caps at 50 steps).
No user-facing params — fully automated. Per-step wrapper still runs on top.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Model weights loaded via torch.load inside ComfyUI's inference_mode become
inference tensors and can't participate in backward. Wrapping the load in
inference_mode(False) ensures parameters are normal autograd tensors.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
.clone() on an inference tensor inherits the inference flag even inside
inference_mode(False). torch.empty() inside that context creates a true
normal tensor; copy_() fills it from the inference tensor without flag
inheritance.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
MLP lives on CPU, conditioning on GPU — device mismatch silently fell back
to fixed direction. Now moves conditioning to MLP device for the forward/
backward, then returns gradient to original device.
Also surface value function fallback errors instead of swallowing them.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Splitting now only fires on user-defined custom triggers (from templates)
plus the generic 'scene N' numeric pattern. The 40+ predefined English
phrases were causing unwanted splits in user-authored prompts.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
nn.Linear weights are inference tensors if instantiated under ComfyUI's
inference_mode context. Moving the entire create+train block inside
inference_mode(False) ensures model weights are normal autograd tensors.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Buffer tensors were created inside inference_mode — detach().cpu() doesn't
escape the inference flag. Wrapping the whole method ensures the clone and
all subsequent tensor ops produce normal autograd-tracked tensors.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
torch.enable_grad() cannot override torch.inference_mode() in older PyTorch.
Add inference_mode(False) as outer wrapper in both train_on and gradient.
Also .clone() tensors from buffer/input before grad tracking to avoid
inference_mode tensor view restrictions.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
ComfyUI execution context may have torch.no_grad() active, which kills
gradient tracking needed for training and gradient computation.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
value_function.py: OnlineValueFunction MLP that trains incrementally on
(conditioning, reward) pairs. Uses replay buffer of 100 samples, 20
gradient steps per update. save/load to .pt file via refinement_state_path.
Studio popup: new "Value guidance" toggle. When on, trains the value function
after every rated generation (good and bad). Falls back to no-op if conditioning
deserialization fails.
chain sampler: _load_value_function reads the trained MLP if ready (5+
samples). _build_embed_guidance_wrapper uses value function gradient
(∂reward/∂conditioning) when available, fixed liked direction as fallback.
Log reports which mode is active and sample count.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Extends the Refiner's one-shot conditioning steering to apply at every
denoising step. Reads the liked conditioning direction from the Refiner's
state file via refinement_key_input. At each step, nudges c_crossattn toward
the learned liked direction with sigma-scaling (no effect at sigma=1, full
effect below sigma=0.5 where detail refinement happens).
Requires 3+ liked generations to have a meaningful direction. No-ops silently
if direction is unavailable.
New inputs: embed_guidance (BOOLEAN), embed_guidance_strength (FLOAT 0.005-0.1).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Below 0.25: audio degrades to noise (guide frame in latent confuses
cross-modal attention without enough video coherence), character appearance
drifts while background over-anchors to previous scene.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Empirical testing: 0.25 is the sweet spot — locks character position and
action pixel-perfect while allowing background to drift freely toward the
new scene's prompt. Above 0.35 causes character-space conflicts when scene
composition shifts significantly.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
_prepend_soft_continuation creates the noise mask with ones_like(video_tensor)
giving it 128 channels instead of 1. Using torch.full_like(masks[0][:,:,:1])
copies the exact shape signature from the existing mask so the cat always works
regardless of channel count.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
guide_mask was [B,1,1,1,1] but noise_mask is [B,1,F,H,W] — cat on dim=2
requires matching H,W. Now uses masks[0].shape[3:] for spatial dims.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Previous approach (post-block hidden state injection) was fundamentally
broken for LTXAV: modifying video tokens after a block contaminates audio
through joint audio-video attention, causing audio noise and latent bleed.
New approach: append the middle frame of the previous scene as a guide
token to the END of the current chunk latent, with keyframe_idxs pointing
to F//2 of the current scene in pixel space, plus guide_attention_entries
at the specified strength. The model's existing _attention_with_guide_mask
mechanism attenuates attention weights inside the attention computation —
audio tokens never see modified video tokens.
After sampling, _crop_video_tail removes the guide frame.
New inputs: mid_scene_guide (BOOLEAN), mid_scene_guide_strength (FLOAT 0.1-0.8).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Reads settings.refiner.vision_conditioning (default true) and gates
source_image passed as reference_image to Gemma3 encoding. Toggle lives
in the popup UI next to Reference injection, not on the node.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Old: average all tokens in middle third → one scalar direction → spread to
surrounding zones. Loses all spatial structure, can't preserve positioning.
New: take the exact spatial token map of the single middle frame (F//2),
inject it position-for-position into every other frame. Token at (h,w) in
the anchor frame maps to (h,w) in all other frames. Character layout and
positioning stays spatially consistent across the scene. Anchor frame
itself is left unchanged.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Re-encoding the next scene's conditioning with the previous scene's
last frame is useless for fade-to-black workflows — the last frame
is black, so it's conditioning on nothing. Removing the whole path.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Middle-frame tokens are the anchor source — blending them with their
own mean was pointless self-contamination and the main cause of
collapse at higher strengths. Now only the surrounding frames
(before mid_start and after mid_end) receive the injection; the
anchor zone is left untouched each step.
Max strength capped at 0.5 in the UI; tooltip warns against going high.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sampler vision re-encoding: prefix scene text with
"Continuing from this scene: {text}" so the model treats the
previous frame as temporal context, not an unrelated subject.
Studio per-scene reference conditioning: re-enable reference_image
with prefix "The character shown performs the following: {encode_text}"
so the model knows the image is a character reference, not something
to describe. Only applies when source_image is connected.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Vision encoding rewrites the full conditioning embedding — with a 512-token
limit, a poor generic description from the template displaces the actual
scene prompt content. Reverting until the template can be properly designed.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Previous: capture middle-frame tokens once at first sigma < 0.95, reuse
for all remaining steps — injected noisy step-6 states into cleaner
step-7/8 passes, causing latent noise flooding.
Now: stateless per-step operation. At every step below sigma=0.95,
compute the mean of the current step's middle-frame tokens and blend
it into all token positions at the given strength. Anchor is always
from the current denoising state so no high-sigma artefacts persist
beyond the step they were produced in.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
_v2_transition_scene_conditionings was encoding per-scene texts without
the reference image — only the main single-scene conditioning path passed
it through. Now passes reference_image=source_image to all per-scene
encodings so every scene conditioning includes the character reference
via Gemma3 vision.
Removes the redundant reference_conditioning option from the chain sampler
(which required a wasteful VAE decode mid-sampling to get the pixel image
that Studio already has).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
