Detects when a denoising trajectory diverges from the learned
good-trajectory (velocity-bias) memory and rotates the prediction
back toward it on that step only. Reactive, read-only on the memory,
audio-safe (direction-only, magnitude-preserved on the denoised latent).
New sampler widgets: rescue_mode / rescue_threshold / rescue_strength,
threaded through get_sampler extra_options. Shares velocity_refinement_key
and aspect bucket with velocity bias; no-op until that memory is captured.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
New order: positive signals (broad→specific) → negative signals
(broad→specific) → delta cap → VF gate.
Concept-in-context delta moved before bad conditioning repulsion so
positive phrase signal isn't added after negatives are removed.
Pair repulsion stays last among negatives as most targeted.
VF final gate: after all patches + delta cap, if net effect lowered
VF predicted reward by >0.05, linearly roll back toward original
proportionally to how much worse it got (max 75% rollback). Visible
as vf-rollback:X.XX in axis adjustments.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
search() generates 16 random unit-direction perturbations (2.5% of
conditioning norm each), scores each with the VF, returns the best.
Runs after ascend() in _v2_ascend_conditioning — ascent exploits the
known gradient direction, search explores the neighbourhood around it
for any better nearby region. Total cost: 16 VF forward passes ≈ free.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Learns which phrase combinations in the final encoded prompt correlate with
anatomy/quality failures (reward < -0.10, wrong_appearance, wrong_action+quality).
For each pair of phrases (up to top 8) present in a bad generation, accumulates
the conditioning direction as a per-pair bad slot using the existing
_v2_store_direction mechanism.
At generation time: for each phrase pair in the current final prompt with ≥3
observations, applies a repulsion from the stored bad direction (max 3 pairs,
strength capped at 3.0% of conditioning norm). Visible in axis_actions as
pair-repulsion:N. Prunes to 200 pairs when exceeding 300.
Trigger includes wrong_appearance explicitly (reward=0.0 but signals anatomy
artifacts), missing quality, and all outcomes below -0.10 reward.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The freeze locked prompt text to the previous Perfect output, overriding
intentional prompt changes when intent similarity stayed above 0.50. With
VF + conditioning memory maintaining quality direction, the freeze is no
longer needed and only causes unwanted video repetition.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Learns which phrases in the final encoded prompt (not intent, not raw input —
the actual text that went into CLIP) correlate with reward changes when added
or removed between consecutive generations.
Storage: concept_delta_memory per phrase text — add_sum/count, remove_sum/count.
Learning: on rating, diffs previous_run.final_phrase_texts vs prev_final_phrase_texts;
credits reward to added phrases, records it for removed.
Application: before conditioning memory is applied, computes a weighted direction
from phrase embeddings (via phrase_memory) for phrases new to this final prompt
that have ≥3 add observations and avg_reward > 0.10. Nudge capped at 3.5% of
conditioning norm alongside all other deltas.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Used refinement_key_input/refinement_key directly instead of eff_key
which is assigned ~340 lines later in refine_v2.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Two new VF-driven behaviours in _v2_apply_conditioning_memory:
1. Global confidence scalar — VF predicted reward scales all conditioning memory
modifications: confidence_scale = (vf_score+1)/2 clamped to [0.4, 1.0].
High score → full strength (direction is good). Low score → dampened
(be cautious, current conditioning direction looks bad). Applies regardless
of which prompt is active — purely VF-signal driven.
2. Gradient-aligned phrase boosting — VF gradient direction (∂reward/∂conditioning)
is computed and compared to each axis direction slot via cosine similarity.
Boost factor [0.5, 1.5] applied to direction strength: phrases/axes pointing
with the reward gradient are amplified, those pointing against are dampened.
Applies to liked_dir, missing axis repair, and satisfied axis preserve.
VF score and boost factors visible in adaptation status output.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Attention weight accumulation:
- Capture per-token output norm at identity blocks [14,20,21,30,33] during every
generation via the optimized_attention patch (same path as temperature)
- EMA-merge into blessed file on liked ratings (0.8 old + 0.2 new)
- Inject as V-scaling bias on future runs: important tokens get amplified V,
pushing attention toward historically attended positions
- Temp/blessed files cleared on session reset
VF-driven temperature:
- build_enhancements accepts conditioning=None, loads value function, scores it
- _derive_temperature_map(vf_score=) continuously modulates block temperatures:
high VF score → sharpen (0.80×), low → warm/explore (1.20×), regardless of rating
- Rating-driven temperature still applies when reward < 0.8; VF blends with it
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
LTX packs all spatial/temporal dims into a single token [1, 1, C*T*H*W]
during sampling. Per-step latent steering requires hooking into the model
internals, not the k-diffusion callback. Removed: compress_latent(),
_load_latent_value_function(), latent VF training, motion floor callback,
latent_value_guidance toggle in Studio.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Previously latent_vf_strength was 0 when no latent VF existed, making
motion floor a no-op. Now embed_guidance_strength is always passed so
motion floor applies at the correct strength regardless of latent VF state.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
LTX samples in [B, seq, C] token format during sampling, not [B, C, T, H, W].
Use sequence variance as temporal variance proxy — works for both 3D and 5D.
Replace fixed 0.5 sigma threshold with 1-sigma scale so motion floor and
latent VF activate across LTX's distilled high-sigma schedule naturally.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
OUTPUT_NODE = True forces ComfyUI to run the node even with no downstream
connection on the LATENT output. Motion floor now activates as soon as
embed_guidance is on and a refinement key is set, independent of liked_dir.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When temporal variance of the latent drops below 5% of overall variance,
applies a normalised deviation gradient that pushes frames apart from each
other. Fires only at sigma < 0.5 (same window as other latent guidance),
uses embed_guidance_strength as scale. Active whenever embed_guidance is on
and liked_dir is available — no user input needed.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Unconstrained ascent was stripping action content from conditioning to reach
a "safe" static-character attractor. Added 8% displacement cap relative to
original conditioning norm — quality direction is still steered but the
prompt content can't be erased. Also raised MIN_SAMPLES from 5 to 10 to
reduce overfitting risk before the value function activates.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
closePicker() didn't remove the mousedown listener, so clicking the button
to re-open the picker would immediately close it. Now stores the cleanup
in module scope and calls it from closePicker().
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Trains a second OnlineValueFunction on rated video latents (compressed via
channel mean-pool). At each denoising step, computes ∂reward/∂latent through
the lightweight MLP and applies it as an in-place drift to the noisy latent —
gradients never touch the 22B transformer. Controlled by embed_guidance_strength
alongside the existing conditioning guidance. Studio toggle: "Latent guidance".
Cleared on session reset alongside the conditioning VF.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Heart modifier now applies full axis pressure (not 50% softened) — meaning
"fix this specific issue but the quality direction was great." The quality
signal comes from boosted reward and conditioning delta, not from reducing
axis pressure. Nailed it + heart removed (that's just Perfect). Positive
grid changed to 1fr 1fr to fill width with 2 items.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The ♥ button on any rating covers the same semantic. Removed from picker
UI and V2_RATING_LABELS; profile kept in Python for backward compat.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Active class was static (set at creation). Now all option elements are
tracked and active state is updated dynamically on each pick.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
ComfyUI rejects widget values not in the allowed list. Generate all
valid label|loved combinations at definition time.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Each rating option now has a small ♥ button on its right. Clicking the main
label applies the base rating; clicking ♥ applies label|loved. No toggle
state — simpler and more direct.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Any rating can now be combined with a "Loved it" toggle. The suffix |loved
is appended to the rating string and parsed in normalize_refiner_v2_rating.
When active: reward boosted to max(base, 0.85), axis pressure softened 50%,
conditioning direction delta raised to 0.45, preferred context applied at
0.30 base_delta. Disabled for Perfect (redundant) and -Just forget it-.
Button label shows ♥ when modifier is active.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
addDOMWidget clicks were swallowed by the canvas. Switched to addWidget
button which ComfyUI routes correctly. Picker now positions at mouse coords
from the widget callback event.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replaces the rating dropdown on Studio and Refiner V2 nodes with a button
that opens a categorized floating picker. Each option shows its reward value
and a one-line hint. Rating auto-resets to -Just forget it- after execution
so forgotten ratings don't silently apply to the next run.
Also adds Wrong action + quality (reward -0.40, wrong_axes action, missing
quality) and drops Wrong action reward from +0.10 to -0.10.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Restores the old "Loved it" learning behavior (reward 0.75, intent delta
+0.03, preferred context 0.65, axis conditioning 0.35, LoRA 0.44/0.24)
as a distinct rating. Sits between Perfect and Loved it on the ladder:
prompt adherence is rewarded but not at full strength.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Previously a weaker "Perfect". Now means: output looks great, prompt
adherence is irrelevant. Skips missing_intent_phrases learning, skips
preferred context lock-in, zeroes intent phrase reinforcement. Keeps
conditioning direction delta and raises value function reward to 0.85.
Quality/style LoRAs rewarded at Perfect level (0.55x).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Two server routes (GET /funpack/value_function/export, POST /funpack/value_function/import)
serve and receive the .pt binary. Studio Session tab gains Export and Import buttons
below the value guidance toggle, keyed to the active refinement key.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
torch.load under inference_mode creates inference tensors; MLP params then
can't participate in backward. Same fix as the earlier value function loader
in samplers.py.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
reset_session now deletes the value function .pt file alongside the
enhancement files. Previously it was orphaned across resets.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Before returning modified_positive, ascend each scene's conditioning tensor
toward higher predicted reward via the value function. Covers both the
standalone Refiner V2 node and FunPack Studio (which delegates to refine_v2).
No-op until value function reaches MIN_SAMPLES; handles transition-split
multi-scene conditioning correctly.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Before each denoising run, when the value function is ready, gradient-ascend
the text conditioning until predicted reward plateaus (self-terminating: step
size = 0.5% of conditioning norm, stops when delta < 1e-3, caps at 50 steps).
No user-facing params — fully automated. Per-step wrapper still runs on top.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Before each denoising run, when the value function is ready, gradient-ascend
the text conditioning until predicted reward plateaus (self-terminating: step
size = 0.5% of conditioning norm, stops when delta < 1e-3, caps at 50 steps).
No user-facing params — fully automated. Per-step wrapper still runs on top.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- embed_guidance: per-step conditioning nudge toward learned quality direction
- Online value function: trains incrementally on rated generations, provides
gradient-based steering (fixed direction fallback when not ready)
- mid_scene_guide: previous scene's middle frame as guide via LTX attention
- Self-consistency replaced with proper guide attention mechanism
- Predefined transition phrases removed — custom triggers only
- Vision conditioning toggle in Studio popup
- Value guidance toggle in Studio popup
- ComfyUI updated to current master
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Model weights loaded via torch.load inside ComfyUI's inference_mode become
inference tensors and can't participate in backward. Wrapping the load in
inference_mode(False) ensures parameters are normal autograd tensors.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
.clone() on an inference tensor inherits the inference flag even inside
inference_mode(False). torch.empty() inside that context creates a true
normal tensor; copy_() fills it from the inference tensor without flag
inheritance.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
MLP lives on CPU, conditioning on GPU — device mismatch silently fell back
to fixed direction. Now moves conditioning to MLP device for the forward/
backward, then returns gradient to original device.
Also surface value function fallback errors instead of swallowing them.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Splitting now only fires on user-defined custom triggers (from templates)
plus the generic 'scene N' numeric pattern. The 40+ predefined English
phrases were causing unwanted splits in user-authored prompts.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
