- Add LORA_TYPE_BLOCK_ZONES: type-aware block zones that suppress bad-fit
blocks to 0.0 and mildly boost good-zone blocks per LoRA type
- Add SEMANTIC_ANCHOR_BLOCKS {14, 19}: confirmed semantic focal points
(PAG default=14, STG defaults=14,19) protected from all suppression
- Add _semantic_anchor_quality: uses LoRA energy share in blocks 14+19
as a quality signal, scaling all block weights 0.92-1.08x accordingly
- Stack priority: weaker LoRA keeps base scale on anchor blocks instead
of being damped, letting the dominant signal carry semantic content
- Fix range display: status line now shows effective weights (model_weight
* scale) instead of raw multipliers; adds zeroed= and anchor_q= fields
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Event delegation on the panel root catches every input/change event
from any field inside the popup and schedules a debounced save to the
studio_settings and adjustments widgets. Changes survive page refresh
without needing to click Close.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
renderSampler() now clears the body before rebuilding so changing the
sampler type replaces the settings instead of appending to them.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Two new output pairs between seed and loss_graph:
- high_pass_sampler (SAMPLER) + high_pass_sigmas (SIGMAS)
- low_pass_sampler (SAMPLER) + low_pass_sigmas (SIGMAS)
Sampler tab in popup: per-pass selection of Hybrid Euler 2S, Distilled
Flow, or KSampler (standard ComfyUI fallback). Each type shows its own
settings fields. Motion pulse and velocity bias sections collapse when
mode is "off". Sigmas entered as comma-separated floats; empty means
rely on external sigmas input.
Python: _build_one_sampler() and _build_samplers() class methods handle
the instantiation and sigma parsing. Each sampler is built from the
same get_sampler() call as the standalone node.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When Scene Builder mode is not Pass-through, the Refiner tab shows a
blue banner explaining the positive prompt is built in the Scene tab
with a clickable link that switches to it. The intent override field is
grayed out and disabled since intent derives from the scene prompt.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add negative_prompt STRING input. Priority chain for negative output:
1. negative_conditioning (pre-encoded CONDITIONING) - pass through as-is
2. negative_prompt (text) encoded via CLIP - removes need for external CLIPTextEncode
3. Popup default negative_prompt field in Refiner tab
4. Empty list if nothing is provided
Negative prompt field added to Refiner tab in popup with override toggle.
Override system extended with negative_prompt key.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Inputs reordered per spec: model, clip, advisor_clip, positive_conditioning,
negative_conditioning, clip_vision_output, source_image, lora_stack,
positive_prompt, user_intent_prompt, feedback_prompt, refinement_key_input.
Outputs reordered: model, modified_positive, negative, seed, loss_graph,
status, training_info, encoded_prompts.
negative_conditioning: CONDITIONING input passed straight through to the
negative output unchanged. Empty list when not connected.
advisor_clip: external CLIP input now accepted. Takes precedence over the
LLM configured in the Advisor popup tab. Same pattern as other overridable
inputs.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
LoRA pipeline is now fully internal to Studio:
1. FunPackApplyLoraWeights reads session weight suggestions for the
current prompt and builds a stack with adjusted model weights
2. FunPackLoraLoader applies the stack to model and clip
3. refine_v2 receives the LoRA-patched model and applies the attn2
direction patch on top
Removes the lora_stack return value - MODEL output already contains
everything (LoRA patches + direction injection). External lora_stack
input still accepted to bypass internal management entirely.
Adds mode (ltx2/wan) and per_block settings to the LoRA tab in popup,
stored in loras_config dict in studio_settings.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Moved from required widget to optional forceInput so it appears as a
connector like all other inputs, not as a text area widget.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add lora_stack (FUNPACK_LORA_STACK) as the 8th return value so the
active stack flows out for downstream use
- Remove positive_prompt from hidden widgets so the text area is visible
on the node face and can be connected as a link input
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Each input that has both a connected link and a popup value now has an
"Override" toggle below it (default off). Off = external input wins,
popup value is a fallback. On = popup value wins regardless.
Applies to: refinement_key (Session tab), feedback_prompt and
user_intent_prompt (Refiner tab). All other settings are popup-only
and unaffected.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Connected value overrides whatever is typed in the Refiner popup.
Useful for wiring dynamic feedback from another node.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
FunPackStudio combines Refinement Key, Scene Builder, Refiner V2,
Advisor LLM, LoRA loading, and Conditioning Adjust under a single node
with a tabbed popup editor. Rating widget stays on the node face;
everything else is in the popup.
Python:
- FunPackStudio.run() chains all subsystems: scene builder prompt
construction, advisor LLM load (using shared cache), LoRA loading via
FunPackLoraLoader, FunPackVideoRefinerV2.refine_v2(), and
FunPackConditioningAdjust.adjust(). No code duplication.
- Added optional _seed parameter to FunPackVideoRefinerV2.refine_v2()
so Studio can generate a seed and output it for wiring to samplers.
- Added /funpack/available_loras endpoint for the LoRA picker.
JS (funpack_studio.js):
- Tabbed popup: Session, Scene, Refiner, Advisor, LoRA, Adjustments.
- Session: refinement key, reset toggle, scene builder mode selector.
- Scene: scene load/save, phrase bank from memory, positive prompt
editor (active when scene builder is not Pass-through).
- Refiner: all V2 settings, feedback prompt, intent override.
- Advisor: LLM enable toggle, model path, dtype.
- LoRA: dynamic list of LoRAs with name/type/model-weight/clip-weight,
loaded from /funpack/available_loras.
- Adjustments: phrase/strength pairs with session phrase bank.
- Settings serialized as JSON in hidden studio_settings widget.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The node was extracting pooled_output from the conditioning meta dict,
which is None for LTX/Gemma3 text encoders. Adjustments were silently
skipped for every phrase.
Use conditioning.mean(dim=(0,1)) on the sequence tensor itself - same
approach V2 uses for all its conditioning memory operations. Phrase
vectors are encoded the same way (mean of their sequence tensor). The
accumulated direction shift is then applied as a uniform broadcast
across all sequence positions so the tensor shape stays intact.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
addCustomWidget failed silently in this ComfyUI version. Switch to
node.addWidget("button",...) which is natively supported. Also add the
same hideWidget retry timing pattern Scene Builder uses (rAF + two
setTimeout calls) so the adjustments text widget reliably disappears
after the node is created or reconfigured.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When refinement_key_input is connected (from FunPack Refinement Key
Loader), the popup fetches phrase_memory from that session via a new
/funpack/phrase_memory endpoint and shows all learned phrases as
clickable chips grouped by category. Click a chip to add it to the
adjustment list with a default strength of 0.1.
Search field filters phrases live. Phrases are sorted by evidence count
so the most-seen ones appear first within each category.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Inserts between Refiner V2 and the sampler. Takes CONDITIONING + CLIP,
lets the user specify phrase/strength pairs in a floating popup panel.
For each active pair: encode the phrase, compute unit-norm direction from
base conditioning toward phrase conditioning (in pooled space), apply at
user-set strength. Positive = push toward that phrase, negative = away.
Bypasses V2's category classification entirely - adjustments target the
exact phrase the user typed, no mis-categorization possible.
Popup hides the raw JSON widget, shows "Edit Adjustments" button on the
node face with a compact summary of active adjustments. Close saves.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Two issues prevented diagnostics from reliably appearing in Full mode:
1. The analysis prompt asked for "root cause of recurring patterns" which
is too vague for early sessions with no history. Now adapts: simple
per-run analysis when no session history exists, pattern analysis when
history is available.
2. When the LLM fails or produces empty output after thinking-token
stripping, analysis was silently "" and nothing was stored. Added a
rule-based fallback that builds a diagnostic from axis_feedback
(regressed/missing/wrong axes) so the history always accumulates
even when the LLM produces nothing useful.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Both _v2_apply_perfect_repair_phrases (injects phrases from previously
Perfect-rated runs) and _v2_emphasized_prompt (repeats existing phrases
matching missing axes) were running unconditionally regardless of the
prompt_repair toggle. With prompt_repair=False they should be no-ops -
the user has explicitly said they do not want the node modifying the
prompt text from memory.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When Pass 1 generates a diagnostic, append it to the system prompt of
Pass 2 so the LLM has the session root-cause analysis at the highest
priority level rather than only as a user-message field. The analysis
remains in the user message too for reference.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The diagnostic analysis pass now reasons over session history instead of
just the current prompt. It receives the axis feedback (what's missing,
regressed, resolved, wrong), current rating, and the accumulated session
history, and its output asks for root cause identification across runs
rather than per-run "what to fix" instructions.
Diagnostics are now recorded for all advisor modes that generate them
(Full and Only diagnostics), not only "Only diagnostics". Previously the
Full mode ran analysis but discarded the result without storing it, so
the history never built up and subsequent repairs had no session context
to draw on.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Added explicit rules against metaphors, prose, and emotional language.
Requires every output phrase to describe something a camera can directly
capture. Targets the tendency of instruction-tuned LLMs to produce
poetic rewrites that video diffusion models cannot interpret.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Qwen3 encodes <think> and </think> as special token IDs (151648/151649).
With skip_special_tokens=True they are stripped silently before the regex
runs, leaving raw reasoning content concatenated with the response and no
way to separate them. The thinking content then bleeds into the repaired
prompt.
Fix: always decode with skip_special_tokens=False so the thinking delimiters
are present in the string. Strip complete <think>...</think> blocks first,
then truncated blocks (no closing tag, model ran out of budget), then clean
remaining special tokens like <|im_end|> with a separate pattern.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Removed three tests that exclusively tested negative conditioning/prompt
functionality which was removed in 2.5.0:
- test_refiner_v2_negative_advisor_repairs_negative_prompt_and_exports_one_string
- test_refiner_v2_returns_empty_negative_conditioning_when_negative_blank
- test_negative_prompt_repair_adds_poorly_rated_tags_persistently
Fixed six tests with stale assertions caused by API changes since they
were written: updated return-value unpack positions after negative output
was removed, dropped CONDITIONING-owned/CLIP-owned status checks that no
longer appear in any output, dropped Gemma3 tokenizer status check that
is only present in the error path now, updated Learning section header
name, and updated RETURN_NAMES assertion to check by value not position.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When pad_token == eos_token the model cannot infer the attention mask
from input_ids, so it warns and attends to all positions including
padding. In practice this causes the model to echo the input prompt
unchanged, randomly drop spaces, or produce garbled output depending
on sampling temperature. Passing an all-ones attention_mask explicitly
tells the model exactly which positions are real input tokens.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
apply_chat_template(..., tokenize=True, return_tensors="pt") returns a
BatchEncoding in newer transformers versions instead of a plain tensor.
Accessing .shape[1] on a BatchEncoding raises AttributeError because it
looks up "shape" in the data dict and finds nothing.
Extract input_ids from the result with hasattr check before calling
.to(device) and computing prompt_length. Applied to both the normal
chat-template path and the skip_template encode() path.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Perfect is not a ceiling. If the user has an active advisor mode and
provides feedback_prompt, the advisor must honor it regardless of the
rating. Two guards were blocking this:
- Analysis pass (pass 1): was skipped when is_perfect=True even with no
feedback, so Full/Only-diagnostics mode produced no analysis for
Perfect runs. Removed the not-is_perfect restriction so analysis runs
whenever mode requires it and no explicit feedback was given.
- Repair guard (pass 2): had an explicit is_perfect-and-not-has_feedback
branch that silently blocked repair for every Perfect run with no user
text. Removed it entirely. The only remaining guard is the existing
allow_prompt_change check (Learning mode / advisor mode Off).
Now for Perfect + user feedback: advisor honors the feedback fully.
For Perfect + no feedback + Full mode: analysis runs, repair runs.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Strip <think>...</think> blocks in decode so Qwen3 chain-of-thought
doesn't pollute the repaired prompt and cause validation to reject it
- Pass enable_thinking kwarg to apply_chat_template (with TypeError
fallback for models that don't support it); propagate thinking flag
through tokenize -> generate so the token budget is expanded by 2048
when thinking mode is active, giving room for reasoning + response
- Return 1.0 from _v2_text_semantic_similarity when the clip has no
encode_from_tokens_scheduled - generation-only clips can't compute
similarity so they shouldn't be penalized with a 0.0 semantic gate
- Add FunPackAdvisorLLM to standalone (non-package) imports in
__init__.py so tests can run without the full ComfyUI runtime
- Update three stale advisor tests: 5-value _v2_prompt_advisor unpack,
"Encoded as:" prefix change, 6-return-value unpacking after negative
conditioning was removed
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When a phrase injected by the perfect-repair system is present during a
generation rated Awful or Wrong Appearance, it is now hard-deleted from
perfect_repairs in the intent family slot. One bad rating is enough to
stop the re-injection permanently. Evicted phrase names are reported in
the intent family status line.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- _v2_direction_readout: new helper showing each direction slot's status
in plain language ("warming up 2/3 runs", "active 8 runs, mag 0.412",
"away from learned pattern") for liked, bad, and all three axes.
- _v2_apply_conditioning_memory: return replaced with multi-line readable
block showing strength, reward trend, streak, which mode each slot is
using, and what axis adjustments were applied.
- training_info Adaptation section: now includes the full direction
readout so users can see exactly what memory state the refiner is in.
- Model patch status: expanded from one line to a readable block showing
which direction slots are active with their run counts, and lists the
phrase texts being emphasized (up to 4) alongside position range count.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
TextGenerate calls tokenize(prompt, skip_template=bool, image=..., thinking=...)
and generate(..., min_p=..., presence_penalty=...). Wrapper now handles all of:
- skip_template=True: raw encode without chat template (for nodes like
TextGenerateLTX2Prompt that pre-format with role tokens)
- skip_template=False (default): apply model's own chat template via
apply_chat_template, with optional system_prompt kwarg
- min_p, presence_penalty: forwarded to model.generate() with progressive
fallback stripping unsupported params via TypeError catch
- All other kwargs absorbed silently so future TextGenerate params don't break
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
FunPackAdvisorLLM:
- Loads any HuggingFace CausalLM via AutoModelForCausalLM.from_pretrained
- Wraps it in _FunPackAdvisorLLMWrapper which exposes tokenize/generate/decode
matching the interface _v2_generate_advisor_text already uses
- tokenize() accepts system_prompt kwarg, applies the model's own chat
template via apply_chat_template, absorbs unsupported kwargs silently
- generate() uses max_new_tokens (not max_length) so prompt length does
not eat into the generation budget
- Model cached in _FUNPACK_ADVISOR_LLM_CACHE by (path, dtype) - no
reload on repeated runs
- Returns CLIP type so it plugs directly into advisor_clip input
- Default: huihui-ai/Huihui-Qwen3-8B-abliterated-v2
Session reset fix:
- _v2_empty_state was missing intent_expansion_memory, session_source_mean_count,
liked_dir, bad_dir - these survived reset via setdefault on next run.
Added all four to the empty state so reset is complete.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Both _v2_advisor_prompt and _v2_advisor_analysis_prompt now build natural
language user messages instead of structured FIELD: value format that only
instruction-following models can interpret.
System prompt reduced to one line: "You are a video prompt editor. Rewrite
the prompt to fix the specific issue. Output only the corrected prompt."
Simple enough to not conflict with enhancement models' own system prompts.
Repair user message: Issue description, user note, scene context, past
feedback, suggestions, analysis, then "Prompt to fix: ... Write the
corrected prompt." - reads as a natural enhancement request.
Analysis user message: same context then "What specifically needs to
change? One or two sentences, reference actual words from the prompt."
This works for:
- Enhancement models (Sulphur/Qwen): see a natural prompt improvement
request they were trained on, produce useful output
- Instruction models: follow the clear natural language request
- The label-free output is already handled by the raw-text fallback
in _v2_parse_advisor_response
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
generate/decode/return were inside the except TypeError block so they
only ran when layer 1 failed. When layer 1 succeeded (system_prompt kwarg
accepted), tokens was set but the function fell off the end returning None,
causing 'cannot unpack non-iterable NoneType' at the call site.
Restructure: tokenize into tokens=None, try each layer in order using
if tokens is None as the gate, then generate/decode/return always runs
after tokenization regardless of which layer succeeded.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
CLIP.tokenize() passes **kwargs to tokenize_with_weights, so wrappers
that natively handle system/user separation (like Sulphur/Qwen) just
work by passing system_prompt= as a kwarg. Three-layer fallback:
1. clip.tokenize(user, system_prompt=system) - native kwarg support
2. apply_chat_template via BFS-found HF tokenizer - manual formatting
3. flat string + "Output:" anchor - last resort for bare wrappers
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The advisor was sending one flat string to clip.tokenize() so models
expecting ChatML/Gemma role tokens treated the whole thing as a user
message and echoed instructions back into the output.
- _v2_find_chat_tokenizer: BFS through CLIP wrapper hierarchy looking
for a HuggingFace tokenizer exposing apply_chat_template (depth ≤ 5,
checks tokenizer/processor/cond_stage_model/patcher/model attrs).
- _v2_advisor_analysis_prompt and _v2_advisor_prompt now return
(system_str, user_str) tuples instead of one combined string.
- _v2_generate_advisor_text now accepts (system_prompt, user_prompt):
- If chat tokenizer found: applies apply_chat_template with proper
system/user roles and add_generation_prompt=True before tokenizing.
- Fallback (no template): flat concatenation + "Output:" anchor so
the model generates output instead of echoing instructions.
- Updated V2_PROMPT_ADVISOR_SYSTEM_PROMPT to state that inputs arrive
in the user message, so the model knows the structure.
- Updated both call sites in _v2_prompt_advisor.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
1.05 repetition_penalty was negligible for a 12B model generating long
sequences. 1.3 is meaningfully aggressive. no_repeat_ngram_size=5 hard-
blocks any 5-gram from repeating, which stops loops that penalty alone
misses. Temperature raised 0.5->0.7 so the peaked distribution doesn't
trap the model in local loops even with penalty applied. Falls back to
no no_repeat_ngram_size if the wrapper doesn't forward that kwarg.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
encode_cache was per-run only - every new rating caused all phrase
encodings to redo from scratch even with unchanged CLIP and text.
_V2_PERSISTENT_ENCODE_CACHE is a module-level dict keyed by
(id(clip), prompt_text), capped at 4096 entries. _v2_encode_prompt
checks persistent cache before encoding, writes to both caches on
miss. If CLIP model is swapped, id(clip) changes so stale entries
are never used (just unused memory until cap evicts them).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
MODEL input/output:
- Optional MODEL input; patched MODEL is the 6th output alongside existing 5.
- _v2_apply_model_patches: clones model and installs attn2_patch for
layer-level cross-attention K/V injection using stored direction vectors.
Directions are in text-encoder space [dim] which matches context_attn2
before projection - compatible across all models using attn2 cross-attention.
- Per-layer scale 0.02 x NORM_SCALE(0.3) x avg_active_norm so effect
accumulates gradually across layers without blowing up.
Phrase emphasis via K/V scaling:
- _v2_find_phrase_token_ranges: locates repair candidate phrases in the
full conditioning sequence using cosine similarity between mean-pooled
phrase encoding and each token position. Model-agnostic, no tokenizer
format dependency.
- _v2_build_attn2_patch: scales K and V at matched positions by 1.25,
boosting cross-attention to concepts added by prompt repair/advisor.
Status and training_info cleanup:
- Filter out lines that just say "none", "off", "skipped", "idle" - these
added noise without signal.
- Status now shows only what actually happened: adaptation summary,
repair/advisor actions, flags for special modes.
- training_info reorganized: Run, Memory, Prompt Analysis, Adaptation,
Guidance, LoRA - sections only shown when non-empty.
- Removed negative_memory_status from training_info (negative removed).
- Added model_patch_status to Adaptation section when model connected.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The existing full-tensor conditioning storage resets when prompt lengths
change, making the memory useless when prompts change every run. This
adds a parallel direction-based mechanism that is shape-invariant:
- _v2_pool_conditioning: mean-pools [batch, seq, dim] -> [dim]
- _v2_update_session_mean: running mean of raw source conditioning
across all runs, pooled to [dim]. Represents the session baseline.
- _v2_store_direction: stores unit-normalized delta (payload - session_mean)
alongside existing averaged full-tensor. Accumulated per slot as a
running-averaged unit direction + magnitude scalar.
- _v2_apply_direction: applies stored direction to active tokens only
(norm > 1% of max), scaled by strength * NORM_SCALE(0.3) * avg_token_norm.
Active-token masking prevents adjusting padding/near-zero tokens.
Applied everywhere conditioning is stored and applied:
- liked_conditioning: switches to direction mode after 3+ perfect runs
- bad_conditioning: switches to direction repulsion after 3+ bad runs
- axis_conditioning_memory[axis][positive/negative]: per-axis directions
Before 3 runs accumulate per slot, falls back to existing lerp/repel
approach. Both mechanisms are fully backward compatible - old state
keys are preserved alongside new direction keys.
Advisor RATING description updated: Missing details now correctly
describes small actions and micro-interactions, not visual quality.
Missing quality covers visual quality/artifacts.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
