All configuration is now auto-computed at sampling time - no user-facing
settings except the optional total_frames INT. Window count is derived
from the number of conditioning entries Refiner/Studio outputs. Window
size is total_frames (or actual latent size) / num_windows. Boundaries
are placed at equal intervals. split_conds_to_windows always on.
Studio: move split_by_transitions from direct node input to the Refiner
tab in the popup editor. Removed direct input, added toggle + hint text.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Studio now exposes split_by_transitions as an optional BOOLEAN
input on the node face, giving direct control without editing
the raw settings JSON. The direct input takes precedence over
the popup JSON setting when connected.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Refiner V2 and Studio: new split_by_transitions option detects
transition words in the final prompt (then, suddenly, cut to, etc.),
extracts the character description (text before the first comma),
and encodes one conditioning per scene window with the character
description prepended to each segment. Returns a multi-entry
conditioning list that routes per-window via split_conds_to_windows.
Context Transition Windows: new total_frames optional input.
When > 0 and split_conds_to_windows is enabled, context_length
is auto-computed at sampling time as total_frames / num_cond_entries
so windows size themselves to match Refiner's segment count.
Also fixes pre-existing bugs: early error return in refine_v2 was
missing video_latent (7th return value), and test unpacking was stale.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Three bugs causing i2v latent to be lost:
1. eff_key and prev_reward were defined inside 'if patched_model is not
None' but used outside it. NameError was silently caught but could
cause unexpected behavior. Moved both definitions before the block.
2. load_and_apply_creativity_mask returned bare {'samples': ...} dropping
all other keys from the original latent (noise_mask, type, batch_index,
etc.). Now copies the full original dict and only replaces 'samples'.
3. Modified samples were left on CPU (detached earlier) while the original
latent samples may be on CUDA. Now moves modified tensor to match the
original device and dtype before returning.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Two bugs:
1. 'LTXV' in 'LTXAV' == False - substring search silently missed LTXAV.
Fixed with explicit set membership: cls_name in {'LTXV', 'LTXAV'}.
2. model_config is on model.model (BaseModel), not on the ModelPatcher
itself - so the unet_config fallback path also never fired.
Fixed by reading inner.model_config instead of model.model_config.
LTX-2.3 (22b) loads as LTXAV. Both LTXV and LTXAV now correctly detected.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
refine_v2: initialise video_latent = latent instead of None so the connected
input passes through even when no key is set or mask computation is skipped.
Mask result only overwrites when it returns something non-None.
load_and_apply_creativity_mask: track explicit_latent separately from the
disk-loaded source. All early-exit and error paths return explicit_latent
(the connected input unchanged) instead of None, preserving the i2v or
previous-generation latent through the node when mask has nothing to add.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sits between Perfect (1.0) and Missing details (0.35). For when the output
looks great and you'd keep it, but it took creative liberty with the prompt
(man grabbed woman's hand instead of the other way - still a hand grab, still
loved, but not word-for-word correct).
Behaviour vs Perfect:
- axis_feedback: same - no missing/wrong axes, repair cleared
- good_streak: increments (same as Perfect)
- liked_dir: updated (quality direction was good)
- liked_conditioning average: NOT updated (intent drift - only Perfect locks in)
- phrase memory: delta 0.35x vs Perfect's 0.55x; category weights +0.08 vs +0.13
- preferred context: base_delta 0.65 vs 1.0
- phrase forgiven_count: -0.10 vs -0.18
- LoRA weights: quality/style +0.44x, action/character +0.24x (vs Perfect's 0.55x flat)
- Attention anchors: blessed (composition was good regardless of prompt fidelity)
- Advisor: not treated as perfect execution - repair signal still possible
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Removed reward >= 0.8 early returns from build_creativity_mask and
load_and_apply_creativity_mask. The global floor formula already handles
high reward correctly: at 1.0 it produces 0.0 floor, leaving only gentle
spatial variance (0-8% of latent std). A Perfect-rated latent is the best
possible reference - discarding it was wrong.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
clear_refinement_data() removes temp attention maps, blessed attention
maps, and creativity latent for the refinement key. Called from
_v2_load_state() at the start of a reset before fresh state is written.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Action LoRAs can carry micro-movement details tied to the action itself
(subtle motion blur, fluid dynamics, small camera adjustments). Previously
they only responded to the action axis, so 'Missing details' ratings never
adjusted their weight. Now mirrors character type which already had both.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Removed from LORA_BLOCK_TYPE_PROFILES, LORA_TYPE_BLOCK_ZONES,
LORA_REFINER_TYPE_PROFILES, ACTION_LORA_TYPES, the validation
special-case, and the Studio JS LORA_TYPES dropdown.
normalize_lora_type() keeps the concept->action mapping for silent
backwards compatibility with any existing saved stacks.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- _is_ltx_model() checks inner model class name, unet_config image_model
key (ltxv/ltxav), and model_type string. build_enhancements() returns
the model unchanged with a console note when a non-LTX model is detected
(e.g. WAN2.2) - generation continues normally, only the LTX patches skip.
- Update model input tooltip in FunPackVideoRefinerV2 to state LTX-2/LTXAV
family requirement and pass-through behavior for other models.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
apply_model takes (x, t, **c) not (args_dict). Both wrapper call
sites were passing the whole dict as x, leaving t unfilled.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add ltx_enhancements.py with four automatic generation improvements
driven by the refiner rating system:
Technique 2 - Per-block attention temperature: sharper (colder) attention
on semantic anchor blocks 14/19 when quality/concept is missing; softer
on early blocks when details are missing. Capped at 0.72-1.22x range.
Technique 3 - Temporal RoPE style: frame_rate manipulation via
model_function_wrapper. Selector exposed in FunPackVideoRefinerV2 node
(optional input) and in Studio popup Refiner tab.
natural=no change, accelerate=+35% fps, decelerate=-28% fps,
loop/freeze=experimental coordinate tricks.
Technique 4 - Denoise creativity mask: spatial noise weighting derived
from latent variance; high-variance regions get more denoising freedom
scaled by reward. Built but applied by downstream latent consumers.
Technique 5 - Attention anchor transfer: hidden states at anchor blocks
14 and 19 are captured via patches_replace["dit"] on every run and saved
to refinements/attn_maps/temp_{key}.pt. When the user rates a run
Perfect, bless_attention_maps() promotes the temp file to a blessed file.
Subsequent runs inject the blessed maps at 5-28% strength scaled by
reward, transferring semantic structure from good outputs onto new runs.
All enhancements are composable with existing attn2 direction patches
and STG/PAG blocks via patches_replace composition.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add LORA_TYPE_BLOCK_ZONES: type-aware block zones that suppress bad-fit
blocks to 0.0 and mildly boost good-zone blocks per LoRA type
- Add SEMANTIC_ANCHOR_BLOCKS {14, 19}: confirmed semantic focal points
(PAG default=14, STG defaults=14,19) protected from all suppression
- Add _semantic_anchor_quality: uses LoRA energy share in blocks 14+19
as a quality signal, scaling all block weights 0.92-1.08x accordingly
- Stack priority: weaker LoRA keeps base scale on anchor blocks instead
of being damped, letting the dominant signal carry semantic content
- Fix range display: status line now shows effective weights (model_weight
* scale) instead of raw multipliers; adds zeroed= and anchor_q= fields
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Event delegation on the panel root catches every input/change event
from any field inside the popup and schedules a debounced save to the
studio_settings and adjustments widgets. Changes survive page refresh
without needing to click Close.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
renderSampler() now clears the body before rebuilding so changing the
sampler type replaces the settings instead of appending to them.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Two new output pairs between seed and loss_graph:
- high_pass_sampler (SAMPLER) + high_pass_sigmas (SIGMAS)
- low_pass_sampler (SAMPLER) + low_pass_sigmas (SIGMAS)
Sampler tab in popup: per-pass selection of Hybrid Euler 2S, Distilled
Flow, or KSampler (standard ComfyUI fallback). Each type shows its own
settings fields. Motion pulse and velocity bias sections collapse when
mode is "off". Sigmas entered as comma-separated floats; empty means
rely on external sigmas input.
Python: _build_one_sampler() and _build_samplers() class methods handle
the instantiation and sigma parsing. Each sampler is built from the
same get_sampler() call as the standalone node.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When Scene Builder mode is not Pass-through, the Refiner tab shows a
blue banner explaining the positive prompt is built in the Scene tab
with a clickable link that switches to it. The intent override field is
grayed out and disabled since intent derives from the scene prompt.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add negative_prompt STRING input. Priority chain for negative output:
1. negative_conditioning (pre-encoded CONDITIONING) - pass through as-is
2. negative_prompt (text) encoded via CLIP - removes need for external CLIPTextEncode
3. Popup default negative_prompt field in Refiner tab
4. Empty list if nothing is provided
Negative prompt field added to Refiner tab in popup with override toggle.
Override system extended with negative_prompt key.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Inputs reordered per spec: model, clip, advisor_clip, positive_conditioning,
negative_conditioning, clip_vision_output, source_image, lora_stack,
positive_prompt, user_intent_prompt, feedback_prompt, refinement_key_input.
Outputs reordered: model, modified_positive, negative, seed, loss_graph,
status, training_info, encoded_prompts.
negative_conditioning: CONDITIONING input passed straight through to the
negative output unchanged. Empty list when not connected.
advisor_clip: external CLIP input now accepted. Takes precedence over the
LLM configured in the Advisor popup tab. Same pattern as other overridable
inputs.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
LoRA pipeline is now fully internal to Studio:
1. FunPackApplyLoraWeights reads session weight suggestions for the
current prompt and builds a stack with adjusted model weights
2. FunPackLoraLoader applies the stack to model and clip
3. refine_v2 receives the LoRA-patched model and applies the attn2
direction patch on top
Removes the lora_stack return value - MODEL output already contains
everything (LoRA patches + direction injection). External lora_stack
input still accepted to bypass internal management entirely.
Adds mode (ltx2/wan) and per_block settings to the LoRA tab in popup,
stored in loras_config dict in studio_settings.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Moved from required widget to optional forceInput so it appears as a
connector like all other inputs, not as a text area widget.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add lora_stack (FUNPACK_LORA_STACK) as the 8th return value so the
active stack flows out for downstream use
- Remove positive_prompt from hidden widgets so the text area is visible
on the node face and can be connected as a link input
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Each input that has both a connected link and a popup value now has an
"Override" toggle below it (default off). Off = external input wins,
popup value is a fallback. On = popup value wins regardless.
Applies to: refinement_key (Session tab), feedback_prompt and
user_intent_prompt (Refiner tab). All other settings are popup-only
and unaffected.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Connected value overrides whatever is typed in the Refiner popup.
Useful for wiring dynamic feedback from another node.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
FunPackStudio combines Refinement Key, Scene Builder, Refiner V2,
Advisor LLM, LoRA loading, and Conditioning Adjust under a single node
with a tabbed popup editor. Rating widget stays on the node face;
everything else is in the popup.
Python:
- FunPackStudio.run() chains all subsystems: scene builder prompt
construction, advisor LLM load (using shared cache), LoRA loading via
FunPackLoraLoader, FunPackVideoRefinerV2.refine_v2(), and
FunPackConditioningAdjust.adjust(). No code duplication.
- Added optional _seed parameter to FunPackVideoRefinerV2.refine_v2()
so Studio can generate a seed and output it for wiring to samplers.
- Added /funpack/available_loras endpoint for the LoRA picker.
JS (funpack_studio.js):
- Tabbed popup: Session, Scene, Refiner, Advisor, LoRA, Adjustments.
- Session: refinement key, reset toggle, scene builder mode selector.
- Scene: scene load/save, phrase bank from memory, positive prompt
editor (active when scene builder is not Pass-through).
- Refiner: all V2 settings, feedback prompt, intent override.
- Advisor: LLM enable toggle, model path, dtype.
- LoRA: dynamic list of LoRAs with name/type/model-weight/clip-weight,
loaded from /funpack/available_loras.
- Adjustments: phrase/strength pairs with session phrase bank.
- Settings serialized as JSON in hidden studio_settings widget.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The node was extracting pooled_output from the conditioning meta dict,
which is None for LTX/Gemma3 text encoders. Adjustments were silently
skipped for every phrase.
Use conditioning.mean(dim=(0,1)) on the sequence tensor itself - same
approach V2 uses for all its conditioning memory operations. Phrase
vectors are encoded the same way (mean of their sequence tensor). The
accumulated direction shift is then applied as a uniform broadcast
across all sequence positions so the tensor shape stays intact.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
addCustomWidget failed silently in this ComfyUI version. Switch to
node.addWidget("button",...) which is natively supported. Also add the
same hideWidget retry timing pattern Scene Builder uses (rAF + two
setTimeout calls) so the adjustments text widget reliably disappears
after the node is created or reconfigured.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When refinement_key_input is connected (from FunPack Refinement Key
Loader), the popup fetches phrase_memory from that session via a new
/funpack/phrase_memory endpoint and shows all learned phrases as
clickable chips grouped by category. Click a chip to add it to the
adjustment list with a default strength of 0.1.
Search field filters phrases live. Phrases are sorted by evidence count
so the most-seen ones appear first within each category.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Inserts between Refiner V2 and the sampler. Takes CONDITIONING + CLIP,
lets the user specify phrase/strength pairs in a floating popup panel.
For each active pair: encode the phrase, compute unit-norm direction from
base conditioning toward phrase conditioning (in pooled space), apply at
user-set strength. Positive = push toward that phrase, negative = away.
Bypasses V2's category classification entirely - adjustments target the
exact phrase the user typed, no mis-categorization possible.
Popup hides the raw JSON widget, shows "Edit Adjustments" button on the
node face with a compact summary of active adjustments. Close saves.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Two issues prevented diagnostics from reliably appearing in Full mode:
1. The analysis prompt asked for "root cause of recurring patterns" which
is too vague for early sessions with no history. Now adapts: simple
per-run analysis when no session history exists, pattern analysis when
history is available.
2. When the LLM fails or produces empty output after thinking-token
stripping, analysis was silently "" and nothing was stored. Added a
rule-based fallback that builds a diagnostic from axis_feedback
(regressed/missing/wrong axes) so the history always accumulates
even when the LLM produces nothing useful.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Both _v2_apply_perfect_repair_phrases (injects phrases from previously
Perfect-rated runs) and _v2_emphasized_prompt (repeats existing phrases
matching missing axes) were running unconditionally regardless of the
prompt_repair toggle. With prompt_repair=False they should be no-ops -
the user has explicitly said they do not want the node modifying the
prompt text from memory.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When Pass 1 generates a diagnostic, append it to the system prompt of
Pass 2 so the LLM has the session root-cause analysis at the highest
priority level rather than only as a user-message field. The analysis
remains in the user message too for reference.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The diagnostic analysis pass now reasons over session history instead of
just the current prompt. It receives the axis feedback (what's missing,
regressed, resolved, wrong), current rating, and the accumulated session
history, and its output asks for root cause identification across runs
rather than per-run "what to fix" instructions.
Diagnostics are now recorded for all advisor modes that generate them
(Full and Only diagnostics), not only "Only diagnostics". Previously the
Full mode ran analysis but discarded the result without storing it, so
the history never built up and subsequent repairs had no session context
to draw on.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Added explicit rules against metaphors, prose, and emotional language.
Requires every output phrase to describe something a camera can directly
capture. Targets the tendency of instruction-tuned LLMs to produce
poetic rewrites that video diffusion models cannot interpret.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Qwen3 encodes <think> and </think> as special token IDs (151648/151649).
With skip_special_tokens=True they are stripped silently before the regex
runs, leaving raw reasoning content concatenated with the response and no
way to separate them. The thinking content then bleeds into the repaired
prompt.
Fix: always decode with skip_special_tokens=False so the thinking delimiters
are present in the string. Strip complete <think>...</think> blocks first,
then truncated blocks (no closing tag, model ran out of budget), then clean
remaining special tokens like <|im_end|> with a separate pattern.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Removed three tests that exclusively tested negative conditioning/prompt
functionality which was removed in 2.5.0:
- test_refiner_v2_negative_advisor_repairs_negative_prompt_and_exports_one_string
- test_refiner_v2_returns_empty_negative_conditioning_when_negative_blank
- test_negative_prompt_repair_adds_poorly_rated_tags_persistently
Fixed six tests with stale assertions caused by API changes since they
were written: updated return-value unpack positions after negative output
was removed, dropped CONDITIONING-owned/CLIP-owned status checks that no
longer appear in any output, dropped Gemma3 tokenizer status check that
is only present in the error path now, updated Learning section header
name, and updated RETURN_NAMES assertion to check by value not position.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When pad_token == eos_token the model cannot infer the attention mask
from input_ids, so it warns and attends to all positions including
padding. In practice this causes the model to echo the input prompt
unchanged, randomly drop spaces, or produce garbled output depending
on sampling temperature. Passing an all-ones attention_mask explicitly
tells the model exactly which positions are real input tokens.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
apply_chat_template(..., tokenize=True, return_tensors="pt") returns a
BatchEncoding in newer transformers versions instead of a plain tensor.
Accessing .shape[1] on a BatchEncoding raises AttributeError because it
looks up "shape" in the data dict and finds nothing.
Extract input_ids from the result with hasattr check before calling
.to(device) and computing prompt_length. Applied to both the normal
chat-template path and the skip_template encode() path.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Perfect is not a ceiling. If the user has an active advisor mode and
provides feedback_prompt, the advisor must honor it regardless of the
rating. Two guards were blocking this:
- Analysis pass (pass 1): was skipped when is_perfect=True even with no
feedback, so Full/Only-diagnostics mode produced no analysis for
Perfect runs. Removed the not-is_perfect restriction so analysis runs
whenever mode requires it and no explicit feedback was given.
- Repair guard (pass 2): had an explicit is_perfect-and-not-has_feedback
branch that silently blocked repair for every Perfect run with no user
text. Removed it entirely. The only remaining guard is the existing
allow_prompt_change check (Learning mode / advisor mode Off).
Now for Perfect + user feedback: advisor honors the feedback fully.
For Perfect + no feedback + Full mode: analysis runs, repair runs.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Strip <think>...</think> blocks in decode so Qwen3 chain-of-thought
doesn't pollute the repaired prompt and cause validation to reject it
- Pass enable_thinking kwarg to apply_chat_template (with TypeError
fallback for models that don't support it); propagate thinking flag
through tokenize -> generate so the token budget is expanded by 2048
when thinking mode is active, giving room for reasoning + response
- Return 1.0 from _v2_text_semantic_similarity when the clip has no
encode_from_tokens_scheduled - generation-only clips can't compute
similarity so they shouldn't be penalized with a 0.0 semantic gate
- Add FunPackAdvisorLLM to standalone (non-package) imports in
__init__.py so tests can run without the full ComfyUI runtime
- Update three stale advisor tests: 5-value _v2_prompt_advisor unpack,
"Encoded as:" prefix change, 6-return-value unpacking after negative
conditioning was removed
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When a phrase injected by the perfect-repair system is present during a
generation rated Awful or Wrong Appearance, it is now hard-deleted from
perfect_repairs in the intent family slot. One bad rating is enough to
stop the re-injection permanently. Evicted phrase names are reported in
the intent family status line.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
