Attribution was re-splitting the raw prompt independently and comparing
scene counts, then guessing on mismatch (all-keys union -> cross-training,
or empty -> keys not detected, incl. single-scene-with-keys). Both the
80MB cross-training and Darki's "custom uploaded keys not detected" trace
to that.
resolve_scene_refinement_keys now reads each scene directly: split with
the shortcut-aware split_timeline_verbatim (same scenes generation uses)
and collect, via new refinement_keys_in_text(), the keys whose shortcuts
fired in each scene (anchor keys apply to all). No scene-count compare, no
union/empty fallback except a genuine multi-scene misalignment. Single
scene with a custom key is detected; each scene gets exactly its keys.
Reverted the original-text scan added to prompt_scene_shortcut_keys (it
desynced that splitter from parse_timeline/generation and caused the
detection regression); it's now used only for session-reset all_keys.
Training law: a scene owned by custom key(s) trains ONLY them; the project
default key is left untouched and learns solely from scenes with no custom
key. Tests added for all three behaviors.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Wrong-* ratings (appearance/details/action/combos) are prompt-REPAIR
signals — "good gen, wrong words/identity" — not quality rewards. They
were feeding their 0.0/low reward into both value-function training and
the keyless Absolute taste store, stamping visually-good conditioning as
a low-value valley. The VF ascent (now always the default key, applied to
every scene) then pushed conditioning away from good regions into drift /
wrong-character / distorted gens; Absolute stored good gens as global
bad-taste.
Wrong-* now carry skip_value_function: still drive prompt repair + the
relative per-key direction/category memory, but train no reward asset.
Quality ratings (Missing-*/Perfect/Nailed/Awful) unchanged. Gated all 3
relative VF call sites + _v2_learn_absolute. Preventative only — keys
poisoned before this fix still need reset/retrain.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Per-scene key attribution diverged from the generation split whenever the
advisor rewrote prompt_to_encode away from the raw prompt, so
resolve_scene_refinement_keys hit its all-keys UNION fallback and assigned
every key to every scene. That union got persisted into the run and read
back at rating time, training every key on every clip (value_fn bloat).
- prompt_scene_shortcut_keys now scans both expanded and original text
(mirrors split_timeline_verbatim), so identical structure -> matching
scene count -> precise attribution far more often.
- Divergence fallback is now default-key only (empty sets), not the
all-keys union; a diverged run under-trains instead of cross-training.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A replacement is a prose phrase and may contain commas (e.g. "The video
rapidly cuts, showing the next view."). Both the Movie Editor's splitLines
and the backend _shortcut_replacements string fallback split on commas, so
one phrase became several variants: the editor showed it across lines,
expansion picked one half at random (never matching the comma-form transition
trigger), and a comma edit reverted on save because it re-split every time.
Replacements are now one-per-line only (commas preserved); triggers still
accept comma-separation per their UI. Existing torn replacements self-heal on
the next edit + save.
- mediabrowser.js: separate splitTriggers (\n or ,) and splitReplacements (\n).
- templates.py: _shortcut_replacements string fallback splits on \n only.
- tests: comma phrase stays one replacement (list + string + normalize paths).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
split_timeline_verbatim expanded shortcuts before detecting transitions, so
a cut marker that is ALSO a shortcut (e.g. qcut/cut, common now that cut
markers carry refinement keys) had its trigger word replaced before
detection -> the split was lost or misplaced ("scene 2 removed, scene 1
shows scene 2's prompt"). It now ALSO scans the original verbatim text for
triggers and merges with the expanded scan; trigger-shortcuts are caught at
their true offset, shortcut-driven splits and plain triggers are unchanged,
and the dedup collapses duplicates. Lossless round-trip preserved.
- conditioning.py: original-text trigger scan in split_timeline_verbatim.
- tests: user's qcut/cut case (2 scenes, actions 10|20,30, lossless) +
shortcut-driven split still works.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The shortcut-internal-cut guard disabled the supported 'shortcut whose
expansion resolves to a transition' splitting (see project prompt model),
so scenes stopped splitting at all. Reverts split_timeline_verbatim to the
prior behavior; the real drop/shift bug needs a targeted fix with the
user's actual prompt + shortcut definition.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
split_timeline_verbatim detected scene cuts in EXPANDED text and projected
them back onto the original. A cut living inside a shortcut's replacement
got projected onto the shortcut's edge; colliding/early boundaries were then
deduped away, dropping a scene and shifting the rest ("scene 2 removed,
scene 1 shows scene 2's prompt") until reload. This surfaced more once
shortcuts could carry refinement keys (more key-bound shortcuts in prompts).
The editor treats shortcuts as opaque (they only expand for preview/
generation), so the verbatim splitter now ignores any transition match that
starts inside a shortcut expansion. Literal user-typed markers still split;
generation still expands and splits unchanged. Result round-trips losslessly.
- conditioning.py: _starts_in_shortcut guard in split_timeline_verbatim.
- tests: shortcut-internal cut is ignored + lossless; literal cuts still split.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Per-scene multi-key learning trains every refinement key whose shortcut
fired in the prompt, so a reset must clear them all — not just the
project/default key, which left stale per-key state behind.
- conditioning.py: _v2_reset_prompt_keys() — on reset_session, union the
non-default keys from the raw positive + intent prompts and reset/save
each (primary key excluded; main flow already handles it). Reported in
the state status string.
- store.js: "Reset Studio session" now confirms before arming, listing
every key it will wipe (default + non-default keys in the prompt, from
the preview's scene_refinement_keys). Disarming does not re-prompt.
- tests: union/primary-exclusion + no-op cases.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Surface, before generation, which refinement key(s) each scene will steer
with — exactly matching the generation-time resolution.
- conditioning.py: extract resolve_scene_refinement_keys() as the single
source of truth; FunPackVideoRefinerV2._v2_scene_refinement_keys now
delegates to it (no behavior change).
- bridge.scene_refinement_keys(): preview payload per scene — explicit
non-default keys, or the project default key when none fired.
- server /preview: attach scene_refinement_keys (both gen and non-gen
branches), resolved against the project's refinement_key.
- inspector.js + styles.css: per-scene key chip in the timeline preview
(avg when >1; greyed "(default)" fallback).
- tests: resolver/Studio parity + bridge preview payload.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Shortcuts carry an optional refinement_key (templates model + expansion);
editor gains a "Use non-default refinement key" checkbox + field.
- Project-level Refinement key field (Studio engine card) wired through
server -> builder into FunPackRefinementKeyLoader (settable in any mode).
- Per-scene key attribution (prompt_scene_shortcut_keys): keys whose shortcuts
fired in a scene; anchor keys count for every scene; union fallback on
rewrite/count mismatch.
- Per-scene conditioning merge: steer with each participating key's state and
average (one key = substitute, none = default), replacing the default-key
relative steering for keyed scenes.
- Multi-key rating: full per-scene learning suite + value fn trained into each
participating key's state.
- Tests for shortcut key surfacing, expansion tagging, per-scene attribution,
fallback, and keyloader injection.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Use sampler boundary pixel positions for preview/export offsets instead of subtracting frame_overlap on the client, which misaligned i2v-guided multi-scene runs.
Introduces a new temporal_style option that cycles motion→hold ramps
across each scene's denoise trajectory via a segmented frame_rate wrapper.
Wired through Studio, engine settings, chain sampler conditioning tags,
and build_enhancements for single-scene workflows.
Each mixed scene is its own queue request (max_scenes=1), not continuous
chain overlap. Scene 2+ uses its anchor via graph Img2Video and borrows
hidden guides from the previous scene. Sampler applies guide stacks on
the first chunk for solo mixed runs.
Mixed i2v anchors no longer blend the previous scene tail into the
starting chunk or the stitched output. Dragging an image onto a
continuation scene auto-sets mixed so montage stays one queue request.
Preview in-points skip overlap before mixed scenes.
Scene chain sampler scene_boundaries JSON now reports pixel/latent overlap
zones, boundary contamination ranges, per-scene mechanisms, and global steering
notes. Pytest conftest stubs the parent FunPack package so light tests run in
the ComfyUI venv without importing folder_paths; align timeline/builder/nodes
tests with current prompt and report formats.
The previous commit accidentally swept in untracked junk (__pycache__,
.DS_Store, .claude/, refinements/*.pt runtime artifacts, research PDF) via
git add -A. Untrack them and ignore going forward. No source changes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Generate now builds the fixed FunPack path in code (builder.py) instead of
loading a runtime template:
- emits the fixed core (Studio -> LTXVConditioning -> Chain Sampler ->
Separate AV -> Audio decode -> Normalize -> VHS Combine, plus Concat AV,
SaveRefinement, KeyLoader, fps/frames primitives) with internal links and
widget values seeded from the reference workflow
- splices in the user's configured loader/image-proc/empty-latent slots,
resolves explicit 'Wire to' links, then auto-wires remaining typed inputs
from the unique producer of that type; ambiguous/unsatisfied are reported
- forces VHS save_output so results are retrievable
- generate route uses builder + blocks on unsatisfied config; drops the
template injector (workflow.py removed)
Fix 'ComfyUI offline' when adding loader nodes: object_info is now built
in-process from NODE_CLASS_MAPPINGS (no loopback HTTP, no port guessing;
tuples normalized via JSON), HTTP kept as fallback. Refresh clears
folder_paths.filename_list_cache like ComfyUI's R key.
Tests: +test_builder.py (4), removed obsolete workflow-injection tests; 19 pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Extends the attention anchor system with three representative blocks from
the concept-formation/identity zone (20-35) which handles character
appearance, body proportions, and distinguishing visual traits.
- IDENTITY_BLOCKS = [21, 26, 31]: evenly spaced across the identity zone
- sigma gate (0.2, 0.55): mid-sigma window where identity is being formed,
slightly later than semantic anchors (0.2, 0.5)
- max inject strength 12% (vs 28% for semantic anchors): lighter touch to
preserve appearance without overriding scene-specific action/composition
- Same blessed map file, same capture/bless/inject infrastructure
- _sigma_gated_strength now takes configurable sigma_high/sigma_low params
- _build_block_replacement takes sigma_gate tuple per block
Fix pre-existing test failures from single-word transition removal:
"suddenly" replaced with "cut to" in transition-related test prompts.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
New placement option 'silent': the transition phrase marks the scene
boundary but is stripped from the output entirely. Neither the previous
nor the next segment contains it. Available globally and as a per-entry
override on custom transitions.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Trailing \b after a non-word character (like '.') never matches since
\b requires a word char on one side. Custom triggers ending with
punctuation now use (?=\s|$) as the end boundary instead, so
"Scene cut." correctly splits prompts.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Global setting in Studio Refiner tab: "Transition placement" (start/end).
start = transition opens the new segment (existing default behavior).
end = transition closes the previous segment instead.
- Per-entry placement override in Transitions tab (global/start/end).
Custom transitions can pin their placement regardless of the global setting.
- transition items gain a placement field (global/start/end, defaults to global).
- load_custom_transition_triggers returns {trigger_lower: placement|None} so
the split function can do per-trigger lookups efficiently.
- refine_v2 and Studio.run() wired with split_transition_placement parameter.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The [a-z][\w-]* alternative matched any word after "scene", turning
"scene proceeds", "scene features", "scene shows" etc. into false
transition split points. Removed the over-broad alternative - the
pattern now only matches numbered scenes (scene 1, scene -2) and
"scene minus <label>" forms.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When the stacking fix merges a pure-transition segment into the next one,
it now uses a space instead of ", " if the segment ends with an article
(a/an/the). This prevents "the scene then cuts to the, next view" when
"next" is also a transition word and the segments get merged mid-phrase.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
If a prompt ends with a transition phrase plus nothing (or just a lone
article like "the"), the last segment is now dropped rather than kept
as a near-empty scene. Covers cases like "...K/DA Kai'Sa, the scene
then cuts to the" where the user did not complete the transition.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Shortcuts: allow empty string as a valid replacement to remove matched phrases
from prompts; post-pass cleans up orphaned commas/spaces left by removals
- Shortcuts: add cross-shortcut longer-phrase-wins test confirming ordered
alternation handles multiple shortcuts with overlapping triggers correctly
- Transitions: new user-editable transition word list stored in transitions.json;
custom triggers extend the built-in split pattern at runtime, keyed by name
- Transitions: Studio JS gets a dedicated Transitions tab (name, trigger, enabled)
with Add/Refresh/Import/Export toolbar
- Transitions: four API routes mirroring shortcuts (GET, export, save/delete, import)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Multi-entry conditioning without a working per-window routing mechanism
accumulates all scene conditionings onto every frame, which confuses the
model. The model already places scene cuts naturally from transition words
in the full prompt - splitting the conditioning breaks this.
split_by_transitions now:
- Detects scene segments and shows them in encoded_prompts (transparency)
- Outputs a SINGLE full-prompt conditioning (model uses natural cuts)
- Status shows "Transition split: N scenes detected"
- encoded_prompts shows "Detected scenes" for use with FunPack Scene Noise
FunPackSceneCutWindows still works independently: it reads num_scenes from
conditioning entries, so with single conditioning it stays inactive (safe).
FunPackSceneNoise still seeds noise per temporal region from the seed.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Anchor fix: use the ENTIRE first segment (before the first transition word)
as the character anchor for all windows. Previous code split at the first
comma, which broke prompts like "In this anime video, Character is a..."
by using "In this anime video" as the anchor and losing the character desc
from all subsequent windows.
i2v fix: in FunPackSceneCutWindows, scene N>0 tokens now have access to
scene 0's tokens as key/value (in addition to their own scene). The i2v
guide frame lives in the early temporal positions (scene 0 region), so
blocking it from later scenes caused the reference image to be ignored.
Scenes 1 and 2+ remain isolated from each other.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Refiner V2 and Studio: new split_by_transitions option detects
transition words in the final prompt (then, suddenly, cut to, etc.),
extracts the character description (text before the first comma),
and encodes one conditioning per scene window with the character
description prepended to each segment. Returns a multi-entry
conditioning list that routes per-window via split_conds_to_windows.
Context Transition Windows: new total_frames optional input.
When > 0 and split_conds_to_windows is enabled, context_length
is auto-computed at sampling time as total_frames / num_cond_entries
so windows size themselves to match Refiner's segment count.
Also fixes pre-existing bugs: early error return in refine_v2 was
missing video_latent (7th return value), and test unpacking was stale.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Two issues prevented diagnostics from reliably appearing in Full mode:
1. The analysis prompt asked for "root cause of recurring patterns" which
is too vague for early sessions with no history. Now adapts: simple
per-run analysis when no session history exists, pattern analysis when
history is available.
2. When the LLM fails or produces empty output after thinking-token
stripping, analysis was silently "" and nothing was stored. Added a
rule-based fallback that builds a diagnostic from axis_feedback
(regressed/missing/wrong axes) so the history always accumulates
even when the LLM produces nothing useful.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The diagnostic analysis pass now reasons over session history instead of
just the current prompt. It receives the axis feedback (what's missing,
regressed, resolved, wrong), current rating, and the accumulated session
history, and its output asks for root cause identification across runs
rather than per-run "what to fix" instructions.
Diagnostics are now recorded for all advisor modes that generate them
(Full and Only diagnostics), not only "Only diagnostics". Previously the
Full mode ran analysis but discarded the result without storing it, so
the history never built up and subsequent repairs had no session context
to draw on.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Removed three tests that exclusively tested negative conditioning/prompt
functionality which was removed in 2.5.0:
- test_refiner_v2_negative_advisor_repairs_negative_prompt_and_exports_one_string
- test_refiner_v2_returns_empty_negative_conditioning_when_negative_blank
- test_negative_prompt_repair_adds_poorly_rated_tags_persistently
Fixed six tests with stale assertions caused by API changes since they
were written: updated return-value unpack positions after negative output
was removed, dropped CONDITIONING-owned/CLIP-owned status checks that no
longer appear in any output, dropped Gemma3 tokenizer status check that
is only present in the error path now, updated Learning section header
name, and updated RETURN_NAMES assertion to check by value not position.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Strip <think>...</think> blocks in decode so Qwen3 chain-of-thought
doesn't pollute the repaired prompt and cause validation to reject it
- Pass enable_thinking kwarg to apply_chat_template (with TypeError
fallback for models that don't support it); propagate thinking flag
through tokenize -> generate so the token budget is expanded by 2048
when thinking mode is active, giving room for reasoning + response
- Return 1.0 from _v2_text_semantic_similarity when the clip has no
encode_from_tokens_scheduled - generation-only clips can't compute
similarity so they shouldn't be penalized with a 0.0 semantic gate
- Add FunPackAdvisorLLM to standalone (non-package) imports in
__init__.py so tests can run without the full ComfyUI runtime
- Update three stale advisor tests: 5-value _v2_prompt_advisor unpack,
"Encoded as:" prefix change, 6-return-value unpacking after negative
conditioning was removed
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Full mode now runs two sequential LLM calls per generation:
Pass 1 (analysis): a focused prompt asks the model to identify exactly
what needs to change — what to add, what to remove, what to fix — and
output a single DIAGNOSTIC line. Both user feedback and feedback history
are present. Max tokens is halved since only one line is needed.
Pass 2 (repair): builds on pass 1 by including the analysis result as
"Analysis of what needs to change:" before the task instruction. The
model applies the analysis rather than having to simultaneously diagnose
and rewrite in a single response.
Only diagnostics mode runs only pass 1, stores the analysis in
advisor_feedback_history as an advisor-sourced entry (labelled
"Advisor note:" in future history context), and returns without
modifying the prompt — making it useful as a genuine preview step
before switching to Full mode.
Only prompt mode runs only pass 2 directly (no analysis overhead),
unchanged from before.
Both passes receive feedback_prompt and feedback_history.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Advisor context is now minimal and feedback-first:
- User feedback (current run, placed first when present)
- Current prompt
- User intent
- Previous encoded prompt (anchor for feedback — so 'he held her head'
has a referent)
- Suggestions from memory (repair candidates, max 5)
- Past feedback history (last 10 entries, most recent first)
Removed from context: rating label, axis codes, plain-English axis
translation, encoded-vs-original prompt duplication, advisor-poisoning
note, previous advisor diagnostic, perfect example, V2 recommendations,
source image text note (image is fed directly to vision-capable models
via the existing tokenizer image= path)
New persistent store: global_state["advisor_feedback_history"] — up to
10 entries, each {feedback, rating, iteration}. Only populated when
feedback_prompt is non-empty. Updated at the end of each run so the
current run's feedback is available for future runs, not the current.
History format: "Rating: user's exact feedback text" e.g.
"1. Missing details: I asked for smoke but it didn't appear."
"2. Wrong action: he was supposed to hold her hand, not her head."
Removed dead helpers: _v2_find_perfect_example_for_intent,
_v2_build_v2_recommendations, _v2_axes_to_plain_english,
_AXIS_DESCRIPTIONS — none used by advisor context any more.
Cleaned up _v2_prompt_advisor and _v2_negative_prompt_advisor
signatures: removed rating_label, phrases, perfect_example,
v2_recommendations, has_image params; added feedback_history.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The advisor prompt was flooded with internal refiner state that means
nothing to a language model: phrase category tags, raw axis code labels,
redundant recommendations, and duplicate encoded/original prompt fields.
With 15+ fields of noisy context, explicit user feedback buried at the
bottom was competing with everything else for the model's attention.
Changes:
- Phrase analysis removed entirely from advisor context
- Axis labels translated to plain English descriptions via _v2_axes_to_
plain_english: "missing action axis" → "The previous output was missing:
main action, movement, motion direction"
- V2 recommendations field removed (was a third copy of the same axis/
candidate information)
- Encoded prompt only shown when it actually differs from the original
- Repair candidates capped to 5 and framed as "Consider adding:" rather
than a raw list
- Negative advisor prompt receives the same treatment
- Negative prompt length rejection raised from 900 to 1200 chars
- feedback_prompt placed first in user context in both advisors, before
all other fields
- System prompt OVERRIDE block injected when feedback is present
- Protected-category and intent-distance validation bypassed when user
feedback is provided
- is_perfect analysis-only block and intent-change guard also bypassed
when feedback is present
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Remove hardcoded Gemma chat tokens; advisor prompts now use plain
structured text compatible with any CLIP text generator
- Raise advisor generation limit from 384 to 800 tokens (internal)
- Repair candidates are now passed as suggestions to the advisor before
being applied; rule-based repair applies only if the advisor is off or
produces no valid output
- Add optional feedback_prompt input for user-provided natural-language
feedback, injected at highest priority in the advisor system prompt
- Add Prompt only execution mode: full prompt shaping with conditioning
vectors passed through unchanged
- Rename advisor modes to Off / Only diagnostics / Only prompt / Full;
Only prompt suppresses the diagnostic line for clean output
- Advisor detects when its own previous output caused a bad rating and
notes this explicitly in the next system prompt
- Inject perfect-rated prompt example into advisor system prompt when
one exists for the current intent family
- Inject V2 phrase-memory recommendations into advisor system prompt;
user feedback takes priority over these
- Guard against stale rating attribution when user intent changes between
runs (body similarity < 0.50 skips repair application)
- Advisor no longer skips on Perfect rating; runs in analysis-only mode
to capture what worked
- Previous advisor diagnostic is fed back into the next run's system
prompt, closing the learning loop
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
