olivv-cs-ComfyUI-FunPack/docs/FunPackVideoRefinerV2.md

FunPack Video Refiner V2

Video Refiner V2 is a prompt-owned refiner. It takes prompt text and a connected CLIP, encodes the prompt inside the node, learns from ratings, and returns refined positive conditioning.

V2 replaces the old FunPack Video Refiner public node. It uses separate fresh state and does not migrate old histories.

Inputs

positive_prompt: Prompt text to encode and refine. When an LLM enhancer is in the workflow this is the enhancer output. V2 treats it as the "suggested prompt" when building advisor context.

rating: Feedback for the previous V2 generation:

• Perfect
• Missing details
• Missing action
• Missing quality
• Missing details + action
• Wrong details
• Wrong action
• Wrong details + action
• Wrong appearance
• Missing details + quality
• Missing action + quality
• Awful
• -Just forget it-

Older Missing concept labels are accepted as aliases for Missing action.

Use Wrong appearance when the video is polluted by remembered clothing, character traits, subject identity, or background concepts that were not requested for the current image.

mode: Execution mode:

• Refine: applies learned prompt shaping and conditioning vector adaptation. Default.
• Prompt only: applies all prompt shaping but passes conditioning vectors through unchanged. Learning still runs. Use when you want to pause conditioning adaptation while prompt refinement continues.
• Learning: observes ratings and updates all memory, but passes both prompt and conditioning through unchanged.

advisor_mode: CLIP text-generation advisor:

• Off: no advisor call.
• Only diagnostics: runs the analysis pass only, stores the finding in feedback history as an Advisor note: entry, and returns the prompt unchanged. Use this to preview what the advisor would say before committing to Full.
• Only prompt: runs the repair pass only (no analysis). Changes the prompt silently; outputs a plain prompt string with no labels.
• Full: runs both passes. Pass 1 identifies what specifically needs to change. Pass 2 applies those findings. The advisor's suggestion, analysis text, and pre-repair prompt are all visible in encoded_prompts.

The advisor receives four explicit inputs: ORIGINAL_USER_INTENT (the user's intent prompt), LAST_PROMPT (the previous encoded prompt), RATING (the current rating label), and OPTIONAL_NOTE (feedback prompt, memory suggestions, history, and analysis combined). Token budget: 1200 for the analysis pass, 1600 for the repair pass.

clip: Text encoder. V2 uses this for prompt encoding, phrase and category similarity checks, and advisor text generation when the model exposes generate/decode.

advisor_clip: Optional separate generative CLIP/Gemma model for the advisor. When connected, the advisor uses this model for both analysis and repair passes while clip continues handling encoding and similarity. When disconnected, the advisor falls back to clip.

advisor_thinking: Enables thinking mode for compatible Gemma CLIP text generators.

feedback_prompt: Optional natural-language description of what was wrong with the previous output, e.g. he was supposed to hold her hand not her head. When connected it is placed first in both advisor passes and the system follows it exactly, bypassing axis-based repair logic, intent distance checks, and protected category rules.

prompt_repair: Enables or disables the rule-based phrase injection from phrase memory for missing axes (default on). Turn off early in a session before enough context has accumulated, or when memory suggestions are disrupting the generation.

refinement_key: V2 session key. All memory and history is stored under this key.

refinement_key_input: Optional linked key input, usually from FunPack Refinement Key Loader. When connected, overrides the text widget.

seed: Optional seed for deterministic exploration paths. Studio passes its generated seed here and can store successful sampler seeds when its seed output is connected.

reset_session: Clears all V2 session state for the selected key. Scene Builder memory is preserved.

lora_stack: Optional stack from FunPack Apply LoRA Weights / FunPack LoRA Loader. V2 writes prompt-specific suggested model weights.

clip_vision_output: Optional CLIP Vision output for the source image. Stored as advisory context and diagnostics only; not blended into positive conditioning.

source_image: Optional source image or frame batch. V2 stores width, height, aspect bucket, and a fingerprint to report when the input image changed between runs. Also passed to vision-capable advisor models.

user_intent_prompt: Optional raw user request before any LLM enhancement. V2 compares it with positive_prompt, stores intent-enhance pairs, and can restore omitted original phrases or suppress repeatedly rejected enhancer-only additions.

im_feeling_lucky: Composes a learned prompt from rated phrase memory and encodes it through the connected CLIP.

split_by_transitions: Detects scene transition phrases in positive_prompt. When disabled, modified_positive is one normal conditioning entry for the full prompt. When enabled, modified_positive becomes one conditioning entry per detected scene, intended for FunPack LTXAV Scene Chain Sampler.

Scene splitting is order-only. Written labels such as scene ten, scene -999999, or scene minus infinity are treated as transition text, not as real scene numbers. Text before the first transition is used as a shared character/global anchor and prepended to every detected scene.

In Studio split mode, each scene conditioning entry also receives funpack_scene_seed metadata. Scene Chain uses those seeds when present.

Feedback History

V2 maintains a rolling history of the last ten user-provided feedback_prompt entries in session state. Each entry is labelled with its rating, for example:

1. Missing action: he was supposed to hold her hand not her head
2. Wrong details: the smoke was rising instead of drifting sideways
3. Advisor note: motion keywords are underrepresented in the suggested prompt

Entries from Only diagnostics runs are stored as Advisor note: and carry forward into Full mode runs. Both advisor passes see the full history so patterns across runs influence the analysis and repair.

Outputs

modified_positive: Refined positive conditioning. With split_by_transitions=True, this is a multi-entry scene conditioning list for FunPack LTXAV Scene Chain Sampler.

status: Short runtime summary including mode, advisor status, and repair actions taken.

training_info: Full diagnostics including encoded prompt, category classification, adaptation strength, Lucky memory status, and LoRA suggestions.

loss_graph: V2 learning loss graph.

encoded_prompts: The exact prompt text used for encoding, plus advisor context when the advisor ran:

Positive prompt: ...

Advisor suggestion (applied): ...

Advisor analysis: ...

Pre-advisor prompt: ...

Advisor suggestion is labelled (applied) when validation passed and the prompt was changed, or (rejected) when validation blocked it. Pre-advisor prompt only appears when the suggestion was applied and the prompt changed. Advisor analysis only appears when the analysis pass ran (Full mode).

When split_by_transitions=True, encoded_prompts also shows a Detected scenes section. Multi-entry conditioning from this mode should go to FunPack LTXAV Scene Chain Sampler; normal samplers may mix all scene conditionings together.

Appearance Safety

Refiner V2 treats image-to-video appearance as image-owned. Learned appearance, subject/character, and environment/background phrases are not auto-added by Prompt Repair or Lucky unless already present in the current prompt.

When feedback_prompt is connected, appearance-related changes requested by the user are applied without restriction.

Refinement Key Loader

FunPack Refinement Key Loader lists existing V2 keys, creates a typed key when none is selected, and exposes Import, Export, and Refresh buttons in the browser UI.

Connect its refinement_key output to Refiner V2's refinement_key_input and to FunPack Apply LoRA Weights' refinement_key_input so both nodes share the same memory.

Removed From V2

V2 does not include sigma refinement, latent refinement, manual scheduler modes, or feedback questions.

Scheduler behaviour is automatic: good streaks make V2 gentler, bad streaks make it push harder away from bad directions. In Prompt only mode, streaks and reward EMA are not updated so prompt-only ratings do not affect conditioning adaptation strength.