Runs after shortcuts and after $variables resolve, before encoding — the model
is handed exactly the text that would otherwise have been encoded, and the
result is what ComfyUI generates from, so nothing downstream knows it happened.
Not the repair advisor: that one is triggered by a rating and fixes a named
fault. This runs on every generation, needs no key, no rating and no previous
run, and only elaborates. Uses advisor_clip when one is wired (the Studio node
already has the socket), the generation CLIP otherwise.
Multi-scene enhances each scene separately. The editor's scene list is
authoritative on count, so rewriting the scenes as one paragraph and
re-splitting could come back with a different number and desync every clip from
its anchor. One generation per distinct text, cached within the run.
Every failure path returns the ORIGINAL prompt — an enhancer that empties the
prompt on a bad generation would render a blank video and blame the model.
Thinking traces, code fences, preambles and whole-answer quotes are stripped;
quotes inside the prompt survive, because speech is quoted on purpose.
Settings > Engine > Studio: toggle, editable system prompt (new textarea field
kind), length limit, temperature, top-p, thinking. Off by default.
Tests: 1637 passed (+24).
The suggester split each LoRA's FILE NAME into words and matched them against
the prompt; the overlap scaled how far that LoRA's weight moved. That guessed
what a LoRA does from what someone happened to call it, which is a different
question from how much weight it has earned.
Relevance now comes from the type set on the LoRA and the categories present
in the prompt. Weight tuning itself is unchanged — ratings, axis feedback and
the learned offset all behave as before. Behaviour change: a LoRA whose name
echoes the prompt will move less per rating than it used to, and one with an
opaque name will move more.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two defects behind a reported x1.9.
Spans overlap — a phrase and one of its own words match the same tokens
and their biases ADD. Intended, so the word reinforces the phrase, but
unbounded three overlapping x1.5 spans reach an effective x3.4 on one
token. Bounded against the strongest SINGLE span, so an explicit
(word:2.0) still means exactly 2.0.
And it shipped on by default. It changes what the model sees on every
rated run, and it is gated on is_h3_clip — which is why disconnecting
CLIP made generations normal again. Now a toggle, off.
Also replaces prefer_wired_conditioning with clip_owns_prompt: a wired
conditioning owns the prompt by default, and the old order is the opt-in.
Every one of them was a real test of a live feature. 18 failed only in a full-suite run and
passed alone, 4 asserted behaviour that was deliberately changed and never got updated.
Ordering (18): each test module built its own partial `comfy` stub and registered it with
`sys.modules.setdefault(...)`, so the first module to import won and every later module's
richer stub was silently discarded — the scene chain sampler then died on `comfy` having no
`.sample`, and the mask-safe attention install bailed at its first guard because
`import comfy.ldm.modules.attention` resolves through the PARENT's attribute, which one
module set and another did not. tests/_comfy_stubs.py now builds that package tree once,
wiring every parent attribute, and test modules attach their fakes as attributes on the
shared modules (per-test, via fixtures, where the fake is only theirs — the idiom
test_detailing.py already used). Same file also stubs `server.PromptServer`, whose absence
made every module that imports templates.py (i.e. anything importing conditioning) fail on
`No module named 'server'`.
Stale assertions (4):
* the "every learning rating" loop excluded non-learning ratings by hardcoded LABEL, so the
later-added `Wrong appearance|loved` and the two control sentinels were asserted to train
phrase memory. Now filtered by profile key, with the reason per key.
* three split_by_transitions tests still used "scene ten" / "cut to" as cut markers. The
unified splitter cuts on a real transition trigger or the generic `scene <N>` label, on
purpose — an incidental phrase must never split a prompt. Rewritten on the current
contract, and they now also pin that the delimiter is dropped rather than encoded.
Verified: 584 pass, and every file passes alone, in reverse order, and with movie_editor's
suite standalone. Mutation-checked the rewritten split tests (disabling the scene-label
pattern fails 4 of 5) so they are not passing vacuously.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Route the conditioning variant from per-config outcome history, via the one
charter-safe surface: the INTERNAL value-function ascend+search (not user knobs,
so no WYSIWYG / hidden-settings violation). Sampler-owned visual ops stay on the
sampler (boundary law kept).
Directly serves the original complaint (Perfect not locking, Awful reproducing):
- _v2_path_conditioning_plan: matching-arm Beta posteriors -> liked-rate mean ->
"lock" (>=0.6: hold the liked conditioning, skip the stochastic search),
"explore" (<=0.4: widen the search to flee a disliked region), else "normal"
- value_function.search gains eps_scale to widen/tighten the search radius
- threaded conditioning_plan through ascend -> apply_scene_refinement_keys ->
finalize; computed once per run from path_outcomes + prompt concepts; logged
Only active once the per-key VF is ready, since it rides the existing ascend path.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Upgrade the seed explore/exploit gate from a flat reuse chance to a proper
bandit over the path-outcome arms:
- _v2_path_arm_beta: Beta posterior over an arm's liked-rate (liked>=0.3 success,
<0.0 failure, neutral ignored; Laplace prior)
- _v2_path_explore_decision: Thompson-sample theta per concept-matching arm vs a
fresh Beta(1,1) explore arm; good arms exploit, dislike-dominated arms route to
a fresh seed ("do something else")
- wired into _v2_choose_successful_seed (falls back to the flat gate with no data)
- shared _v2_path_matching_arms helper; _V2_KEY_REWARD + liked/disliked consts
5 tests incl. a statistical one: a strongly-liked arm rarely explores, a hated
arm almost always does.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Make path_outcomes ACT via the one Studio->sampler lever wired today: the seed
(FunPackStudio.run -> _v2_choose_successful_seed -> seed output -> sampler). No
new data paths invented.
The existing seed chooser only exploited liked seeds with zero avoidance — the
headline gap. Now:
- recorder tags each seed sample avoid = reward<0 and not skip_value_function
(Wrong-* excluded: the seed was fine, the words/identity were off)
- _v2_path_disliked_seeds: seeds that gave a low-quality result for matching concepts
- _v2_choose_successful_seed redraws a disliked fallback, drops liked-then-later-
disliked seeds from reuse, and reports how many it avoided
Broader config levers (guidance/lora) need new wiring and are deferred pending
graph-topology confirmation.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Audited every learning sink in the V2 rating flow. Rating-learning and the
Absolute store already honored skip_learning, but two feedback sinks ran gated
only by feedback_prompt — so a 'Just ignore it' run with typed feedback still
wrote advisor_feedback_history + intent_expansion_memory.
Add _v2_rating_is_discard (forget rating or auto-discarded refusal) and gate
both feedback sinks on it, without touching the active continue/fresh_prompt
modes that legitimately keep feedback.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Reframe ratings from local steering into global routing: every rating —
not just successful ones — is recorded as a labeled outcome on the per-scene
config "arm" that produced it, so a later planner can route away from arms
that keep generating disliked results.
- _v2_path_signature: arm = family + concepts + guidance config, seed EXCLUDED
(seed is the random draw within an arm); md5 arm_id collapses identical configs
- _v2_update_path_outcomes: learns from EVERY rating, decaying counts so recent
behavior dominates; records seed-level samples for later seed routing
- hooked at the shared per-scene learning hub + single-run path; stored in
per-key state under "path_outcomes"
Pure instrumentation — no generation behavior change yet (Phase 2 = Studio planner).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Deletes the FunPack Scene Builder node and the Studio Scene Builder feature
— the per-key scene database (scene_db / universal_memory / saved scenes),
the Studio "Scene" tab, scene-phrase authority over classification, and the
scene-database wildcard cleanup. It was redundant: helpful on paper but
effectively unused, undrivable from the Cutting Room, and a risk to shortcuts.
Removed:
- node registration (__init__.py, funpack.py) + web/funpack_scene_builder.js
- templates.py: scene-DB storage layer (load/save/normalize/empty_scene_db,
SCENE_DB_VERSION/SCENE_STATE_FIELD/SCENE_CATEGORY_SOURCES, scene_store_path),
all scene_* helpers, resolve_scene_database_wildcards, the FunPackSceneBuilder
class, and the /funpack/scenes* routes
- conditioning.py: _v2_scene_builder_db / _v2_scene_memory_item /
_v2_apply_scene_builder_authority / _v2_scene_builder_category_for_entry /
_v2_sync_scene_builder_memory / _v2_resolve_scene_builder_wildcards (+ wildcard
helpers + _v2_scene_key), the scene_db threading through classify/learning,
the scene_category_locked phrase fields + their training branch, the
state["scene_builder"] preservation, and the Studio scene_builder prompt mode
- web/funpack_studio.js: Scene tab, SB_MODES, scene_builder settings, renderScene,
fetch/saveScene, the Session Scene-Builder section, and the Refiner banner/notice
- tests/test_scene_builder.py (whole file) + scene-builder tests in
test_v2_category_alignment.py; the SceneBuilder example workflow + its doc;
README link; Scene Builder mentions in docs/FunPackStudio.md
Kept intact: shortcuts (apply_prompt_shortcuts / shortcut DB), transitions and
the canonical split_scenes splitter, and FunPackRefinementKeyLoader. Studio now
always uses the connected positive_prompt. No new test failures (the 16/97
remaining in tests/ are pre-existing env/sampler failures); movie_editor 140 pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Reworks the "Wrong appearance" rating from a (now-removed) prompt-repair
signal into a conditioning-level consistency anchor (option b). "Wrong
appearance" now means "character consistency wasn't preserved" and steers
the next generation back toward the established character.
Capture (_v2_update_appearance_anchor, wired into both the single-run
learning path and the shared per-scene suite _v2_learn_scene_into_state):
- good rating (like/nailed_it/loved_it/loved) → store the rated run's
conditioning as the per-key "blessed appearance" (overwrite = last good)
- wrong_appearance → store the rejected run's conditioning as the "drift"
global_state is already per-refinement-key, so both live with that key.
Apply (_v2_apply_conditioning_memory_impl, after the intent-family delta):
when the user rated the previous gen wrong_appearance, gently pull this gen
toward the blessed appearance (lerp 0.05) and repel from the drift (0.04),
reusing _v2_apply_conditioning_payload / _v2_repel_conditioning_payload. No
ramping; the existing 7.5% delta cap bounds the net effect. No-op when
nothing is stored or shapes mismatch.
Drops the dead wrong_categories field (zero consumers after Part A) and
keeps skip_value_function (the anchor is a conditioning nudge, not a reward).
Tests: 7 new unit tests (storage + overwrite + skip + pull/repel + idle +
non-trigger); updated test_wrong_appearance_rating_is_available for the new
semantics. Zero new failures vs HEAD (the 10/31 remaining are pre-existing
env failures from missing server/comfy modules).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Removes the now-unreferenced repair-only machinery left behind after
Stage 2 (1/3) neutralized the application:
- _v2_prompt_repair_text_allowed
- _v2_emphasized_prompt
- _v2_repair_prompt_for_missing_axes and the repair-candidate helpers
- perfect_repairs / rejected_repairs storage + perfect-repair eviction
note in the intent-family learn path
Shared helpers still used by intent-alignment (e.g. _v2_repair_intent_roots,
_v2_repair_candidate_matches_intent, _v2_prompt_contains_text) are kept, as
is the repair_feedback / active_repair_axes "what's missing" signal the
advisor reads.
Deletes the 11 category-alignment tests that exercised the removed methods
directly. No new test failures (the 10/31 remaining are pre-existing env
failures from missing server/comfy modules).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Neutralize the repair application in refine(): the intent-aligned prompt is
encoded as-is — perfect-repair phrase injection, missing-axis repair, and the
emphasis pass are gone. The advisor remains the only prompt-rewriting path, and
repair_feedback/active_repair_axes is KEPT (it's the 'what's missing' signal the
advisor still reads). Drop the prompt_repair param from refine(), INPUT_TYPES,
FunPackStudio, and both UIs (web + engine_settings).
The dead repair-only methods (_v2_repair_prompt_for_missing_axes, perfect-repair
phrases, etc.) come out in the next commit; shared helpers like
_v2_repair_intent_roots (used by intent alignment) stay.
Deleted the now-obsolete test_repair_candidates_saved_as_json_lists (tested the
removed injection). No new test failures vs dev.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Extends the attention anchor system with three representative blocks from
the concept-formation/identity zone (20-35) which handles character
appearance, body proportions, and distinguishing visual traits.
- IDENTITY_BLOCKS = [21, 26, 31]: evenly spaced across the identity zone
- sigma gate (0.2, 0.55): mid-sigma window where identity is being formed,
slightly later than semantic anchors (0.2, 0.5)
- max inject strength 12% (vs 28% for semantic anchors): lighter touch to
preserve appearance without overriding scene-specific action/composition
- Same blessed map file, same capture/bless/inject infrastructure
- _sigma_gated_strength now takes configurable sigma_high/sigma_low params
- _build_block_replacement takes sigma_gate tuple per block
Fix pre-existing test failures from single-word transition removal:
"suddenly" replaced with "cut to" in transition-related test prompts.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Multi-entry conditioning without a working per-window routing mechanism
accumulates all scene conditionings onto every frame, which confuses the
model. The model already places scene cuts naturally from transition words
in the full prompt - splitting the conditioning breaks this.
split_by_transitions now:
- Detects scene segments and shows them in encoded_prompts (transparency)
- Outputs a SINGLE full-prompt conditioning (model uses natural cuts)
- Status shows "Transition split: N scenes detected"
- encoded_prompts shows "Detected scenes" for use with FunPack Scene Noise
FunPackSceneCutWindows still works independently: it reads num_scenes from
conditioning entries, so with single conditioning it stays inactive (safe).
FunPackSceneNoise still seeds noise per temporal region from the seed.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Anchor fix: use the ENTIRE first segment (before the first transition word)
as the character anchor for all windows. Previous code split at the first
comma, which broke prompts like "In this anime video, Character is a..."
by using "In this anime video" as the anchor and losing the character desc
from all subsequent windows.
i2v fix: in FunPackSceneCutWindows, scene N>0 tokens now have access to
scene 0's tokens as key/value (in addition to their own scene). The i2v
guide frame lives in the early temporal positions (scene 0 region), so
blocking it from later scenes caused the reference image to be ignored.
Scenes 1 and 2+ remain isolated from each other.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Refiner V2 and Studio: new split_by_transitions option detects
transition words in the final prompt (then, suddenly, cut to, etc.),
extracts the character description (text before the first comma),
and encodes one conditioning per scene window with the character
description prepended to each segment. Returns a multi-entry
conditioning list that routes per-window via split_conds_to_windows.
Context Transition Windows: new total_frames optional input.
When > 0 and split_conds_to_windows is enabled, context_length
is auto-computed at sampling time as total_frames / num_cond_entries
so windows size themselves to match Refiner's segment count.
Also fixes pre-existing bugs: early error return in refine_v2 was
missing video_latent (7th return value), and test unpacking was stale.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Two issues prevented diagnostics from reliably appearing in Full mode:
1. The analysis prompt asked for "root cause of recurring patterns" which
is too vague for early sessions with no history. Now adapts: simple
per-run analysis when no session history exists, pattern analysis when
history is available.
2. When the LLM fails or produces empty output after thinking-token
stripping, analysis was silently "" and nothing was stored. Added a
rule-based fallback that builds a diagnostic from axis_feedback
(regressed/missing/wrong axes) so the history always accumulates
even when the LLM produces nothing useful.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The diagnostic analysis pass now reasons over session history instead of
just the current prompt. It receives the axis feedback (what's missing,
regressed, resolved, wrong), current rating, and the accumulated session
history, and its output asks for root cause identification across runs
rather than per-run "what to fix" instructions.
Diagnostics are now recorded for all advisor modes that generate them
(Full and Only diagnostics), not only "Only diagnostics". Previously the
Full mode ran analysis but discarded the result without storing it, so
the history never built up and subsequent repairs had no session context
to draw on.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Removed three tests that exclusively tested negative conditioning/prompt
functionality which was removed in 2.5.0:
- test_refiner_v2_negative_advisor_repairs_negative_prompt_and_exports_one_string
- test_refiner_v2_returns_empty_negative_conditioning_when_negative_blank
- test_negative_prompt_repair_adds_poorly_rated_tags_persistently
Fixed six tests with stale assertions caused by API changes since they
were written: updated return-value unpack positions after negative output
was removed, dropped CONDITIONING-owned/CLIP-owned status checks that no
longer appear in any output, dropped Gemma3 tokenizer status check that
is only present in the error path now, updated Learning section header
name, and updated RETURN_NAMES assertion to check by value not position.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Strip <think>...</think> blocks in decode so Qwen3 chain-of-thought
doesn't pollute the repaired prompt and cause validation to reject it
- Pass enable_thinking kwarg to apply_chat_template (with TypeError
fallback for models that don't support it); propagate thinking flag
through tokenize -> generate so the token budget is expanded by 2048
when thinking mode is active, giving room for reasoning + response
- Return 1.0 from _v2_text_semantic_similarity when the clip has no
encode_from_tokens_scheduled - generation-only clips can't compute
similarity so they shouldn't be penalized with a 0.0 semantic gate
- Add FunPackAdvisorLLM to standalone (non-package) imports in
__init__.py so tests can run without the full ComfyUI runtime
- Update three stale advisor tests: 5-value _v2_prompt_advisor unpack,
"Encoded as:" prefix change, 6-return-value unpacking after negative
conditioning was removed
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Full mode now runs two sequential LLM calls per generation:
Pass 1 (analysis): a focused prompt asks the model to identify exactly
what needs to change — what to add, what to remove, what to fix — and
output a single DIAGNOSTIC line. Both user feedback and feedback history
are present. Max tokens is halved since only one line is needed.
Pass 2 (repair): builds on pass 1 by including the analysis result as
"Analysis of what needs to change:" before the task instruction. The
model applies the analysis rather than having to simultaneously diagnose
and rewrite in a single response.
Only diagnostics mode runs only pass 1, stores the analysis in
advisor_feedback_history as an advisor-sourced entry (labelled
"Advisor note:" in future history context), and returns without
modifying the prompt — making it useful as a genuine preview step
before switching to Full mode.
Only prompt mode runs only pass 2 directly (no analysis overhead),
unchanged from before.
Both passes receive feedback_prompt and feedback_history.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Advisor context is now minimal and feedback-first:
- User feedback (current run, placed first when present)
- Current prompt
- User intent
- Previous encoded prompt (anchor for feedback — so 'he held her head'
has a referent)
- Suggestions from memory (repair candidates, max 5)
- Past feedback history (last 10 entries, most recent first)
Removed from context: rating label, axis codes, plain-English axis
translation, encoded-vs-original prompt duplication, advisor-poisoning
note, previous advisor diagnostic, perfect example, V2 recommendations,
source image text note (image is fed directly to vision-capable models
via the existing tokenizer image= path)
New persistent store: global_state["advisor_feedback_history"] — up to
10 entries, each {feedback, rating, iteration}. Only populated when
feedback_prompt is non-empty. Updated at the end of each run so the
current run's feedback is available for future runs, not the current.
History format: "Rating: user's exact feedback text" e.g.
"1. Missing details: I asked for smoke but it didn't appear."
"2. Wrong action: he was supposed to hold her hand, not her head."
Removed dead helpers: _v2_find_perfect_example_for_intent,
_v2_build_v2_recommendations, _v2_axes_to_plain_english,
_AXIS_DESCRIPTIONS — none used by advisor context any more.
Cleaned up _v2_prompt_advisor and _v2_negative_prompt_advisor
signatures: removed rating_label, phrases, perfect_example,
v2_recommendations, has_image params; added feedback_history.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The advisor prompt was flooded with internal refiner state that means
nothing to a language model: phrase category tags, raw axis code labels,
redundant recommendations, and duplicate encoded/original prompt fields.
With 15+ fields of noisy context, explicit user feedback buried at the
bottom was competing with everything else for the model's attention.
Changes:
- Phrase analysis removed entirely from advisor context
- Axis labels translated to plain English descriptions via _v2_axes_to_
plain_english: "missing action axis" → "The previous output was missing:
main action, movement, motion direction"
- V2 recommendations field removed (was a third copy of the same axis/
candidate information)
- Encoded prompt only shown when it actually differs from the original
- Repair candidates capped to 5 and framed as "Consider adding:" rather
than a raw list
- Negative advisor prompt receives the same treatment
- Negative prompt length rejection raised from 900 to 1200 chars
- feedback_prompt placed first in user context in both advisors, before
all other fields
- System prompt OVERRIDE block injected when feedback is present
- Protected-category and intent-distance validation bypassed when user
feedback is provided
- is_perfect analysis-only block and intent-change guard also bypassed
when feedback is present
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Remove hardcoded Gemma chat tokens; advisor prompts now use plain
structured text compatible with any CLIP text generator
- Raise advisor generation limit from 384 to 800 tokens (internal)
- Repair candidates are now passed as suggestions to the advisor before
being applied; rule-based repair applies only if the advisor is off or
produces no valid output
- Add optional feedback_prompt input for user-provided natural-language
feedback, injected at highest priority in the advisor system prompt
- Add Prompt only execution mode: full prompt shaping with conditioning
vectors passed through unchanged
- Rename advisor modes to Off / Only diagnostics / Only prompt / Full;
Only prompt suppresses the diagnostic line for clean output
- Advisor detects when its own previous output caused a bad rating and
notes this explicitly in the next system prompt
- Inject perfect-rated prompt example into advisor system prompt when
one exists for the current intent family
- Inject V2 phrase-memory recommendations into advisor system prompt;
user feedback takes priority over these
- Guard against stale rating attribution when user intent changes between
runs (body similarity < 0.50 skips repair application)
- Advisor no longer skips on Perfect rating; runs in analysis-only mode
to capture what worked
- Previous advisor diagnostic is fed back into the next run's system
prompt, closing the learning loop
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
