50 lines
3.0 KiB
Markdown
50 lines
3.0 KiB
Markdown
# FunPack LoRA Weight Workflow
|
|
|
|
This workflow lets `FunPack Video Refiner` suggest prompt-specific LoRA weights while keeping LoRA loading deterministic.
|
|
|
|
## Node Order
|
|
|
|
Use this pattern:
|
|
|
|
`FunPack Apply LoRA Weights` -> `FunPack LoRA Loader` -> `FunPack Video Refiner`
|
|
|
|
The loader does not learn. The refiner owns prompt parsing, concept matching, rating interpretation, and saving suggested weights.
|
|
|
|
## FunPack Apply LoRA Weights
|
|
|
|
This node defines the selected LoRAs and their base weights.
|
|
|
|
Inputs:
|
|
|
|
- **positive_prompt**: Exact prompt text for lookup.
|
|
- **refinement_key**: Same key used by `FunPack Video Refiner`.
|
|
- **mode**: Same tokenizer mode as the refiner.
|
|
- **per_block**: If enabled for `ltx2` stacks, the loader derives hidden transformer block scales from each LoRA's own patch magnitudes. The UI still exposes only the regular base weights.
|
|
- **lora_N**: LoRA file.
|
|
- **lora_N_type**: `general`, `concept`, `style`, `quality`, or `character`.
|
|
- **lora_N_base_weight**: Trainer-recommended model base weight.
|
|
|
|
Use **+ Add LoRA** to add more LoRA rows as needed.
|
|
|
|
On the first run for an exact prompt, no saved suggestion exists, so base weights are used. On later runs, the node reads the refiner JSON and applies saved suggestions for that exact prompt only.
|
|
|
|
## FunPack LoRA Loader
|
|
|
|
This node only loads the LoRA stack. It applies the model weights prepared by `FunPack Apply LoRA Weights`, then passes the same stack forward so the refiner can learn from it. The `clip` input is optional and uses zero CLIP strength when omitted.
|
|
|
|
When `per_block` is enabled on an `ltx2` stack, the loader keeps the user-facing global LoRA weight but redistributes it across detected `transformer_blocks.N` patches automatically. Blocks with stronger LoRA magnitude get a bit more weight, weaker blocks get a bit less, and non-block patches stay at the global weight. If the loader cannot detect a usable LTX transformer block layout, it falls back to the normal global application path.
|
|
|
|
## Video Refiner Integration
|
|
|
|
Connect the `lora_stack` output from `FunPack LoRA Loader` into the optional `lora_stack` input on `FunPack Video Refiner`.
|
|
|
|
The refiner compares each LoRA's declared type and filename with the concepts it already extracted from the prompt. After processing the rating category, it saves `lora_weight_suggestions` into the same prompt entry in the refiner JSON.
|
|
|
|
Examples:
|
|
|
|
- `I like it` boosts matching LoRAs gently and can stabilize repeated good offsets.
|
|
- `Missing details` keeps the main direction mostly positive while nudging related concept, character, and general LoRAs upward.
|
|
- `Missing concept` boosts the best-matching concept or character LoRA, while reducing weaker competing concept LoRAs that may be crowding it out.
|
|
- `Missing quality` can boost quality LoRAs and the best concept match, while reducing unrelated or competing concept LoRAs.
|
|
- `I don't like it` treats matching concept LoRAs more aggressively than broad LoRAs, so repeated failures can mark a LoRA as a likely culprit, push its automatic weight down to `0.0`, or invert it with a negative weight.
|