88 lines
5.3 KiB
Markdown
88 lines
5.3 KiB
Markdown
# FreeU Advanced Plus (And Post-CFG SHIFT)
|
||
Let's say you and I grab dinner, and movie after lunch? 🌃📺😏
|
||
|
||
|
||
|
||
### Exmaple of default node settings applied across blocks.
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
## Input Parameters
|
||
|
||
- `model` (`MODEL`): Model to patch
|
||
- `target_block` (`COMBO`): Which block to target; `input_block`, `middle_block`, and `output_block`
|
||
- `multiscale_mode` (`COMBO`): A list of available multiscale modes:
|
||
- `["Default", "Bandpass", "Low-Pass", "High-Pass", "Pass-Through", "Gaussian-Blur", "Edge-Enhancement", "Sharpen", "Multi-Bandpass", "Multi-Low-Pass", "Multi-High-Pass", "Multi-Pass-Through", "Multi-Gaussian-Blur", "Multi-Edge-Enhancement", "Multi-Sharpen"]`
|
||
- `multiscale_strength` (`FLOAT`, Default: 1.0, Range: [0.0, 1.0], Step: 0.001): Strength of scaling
|
||
- `b1_slice` (`INT`, Default: 640, Range: [64, 1280], Step: 1): The size of the array slice for b1 operation
|
||
- `b2_slice` (`INT`, Default: 640, Range: [64, 640], Step: 1): The size of the array slice for b2 operation
|
||
- `b1` (`FLOAT`, Default: 1.1, Range: [0.0, 10.0], Step: 0.001): `b1` output multiplier
|
||
- `b2` (`FLOAT`, Default: 1.2, Range: [0.0, 10.0], Step: 0.001): `b2` output multiplier
|
||
- `s1` (`FLOAT`, Default: 0.9, Range: [0.0, 10.0], Step: 0.001): `s1` Fourier transform scale strength
|
||
- `s2` (`FLOAT`, Default: 0.2, Range: [0.0, 10.0], Step: 0.001): `s2` Fourier transform scale strength
|
||
|
||
### Optional Parameters
|
||
|
||
- `b1_mode` (`COMBO`): Blending modes for `b1` multiplied result.
|
||
- `['bislerp', 'colorize', 'cosine interp', 'cuberp', 'hslerp', 'inject', 'lerp', 'linear dodge', 'slerp']`
|
||
- `b1_blend` (`FLOAT`, Default: 1.0, Range: [0.0, 100], Step: 0.001): Blending strength for `b1`.
|
||
- `b2_mode` (`COMBO`): Blending modes for `b2` multiplied result.
|
||
- `['bislerp', 'colorize', 'cosine interp', 'cuberp', 'hslerp', 'inject', 'lerp', 'linear dodge', 'slerp']`
|
||
- `b2_blend` (`FLOAT`, Default: 1.0, Range: [0.0, 100], Step: 0.001): Blending strength for `b2`.
|
||
- `threshold` (`INT`, Default: 1.0, Range: [1, 10], Step: 1): The exposed threshold value of the Fourier transform function.
|
||
- `use_override_scales` (`COMBO`): "true", or "false" on whether to use `override_scales`
|
||
- `override_scales` (`STRING`, Default: [Multiline String]): Override scales. Create custom scales and experiment with results.
|
||
- Example `10, 1.5` would create the `multiscale_mode` effect `Sharpen`
|
||
- You can use `#`, `//` and `!` to comment out lines.
|
||
|
||
### FreeU BibTex
|
||
```
|
||
@article{Si2023FreeU,
|
||
author = {Chenyang Si, Ziqi Huang, Yuming Jiang, Ziwei Liu},
|
||
title = {FreeU: Free Lunch in Diffusion U-Net},
|
||
journal = {arXiv},
|
||
year = {2023},
|
||
}
|
||
```
|
||
## :newspaper_roll: License
|
||
|
||
Distributed under the MIT License. See `LICENSE` for more information.
|
||
|
||
---
|
||
|
||
## Post-CFG SHIFT (Flux)
|
||
|
||
Post-CFG Stepwise Hybrid Inject + Fourier Tuning.
|
||
|
||
- Runs after classifier-free guidance (CFG) merges cond/uncond each sampler step.
|
||
- Modifies the sampler’s current denoised tensor (in VAE latent space in typical pipelines), not model weights.
|
||
- Applies a hybrid blend between the denoised tensor and a scaled version, with optional frequency-domain shaping.
|
||
|
||
### How it works
|
||
1) Model predicts noise; CFG produces a denoised tensor for the current step.
|
||
2) SHIFT blends `denoised` with `denoised * b` using the chosen `mode` and `blend`.
|
||
3) Optionally applies `Fourier_filter` with per-scale controls.
|
||
4) Applies a final `force_gain` multiplier.
|
||
|
||
### Parameters
|
||
- `mode` (combo): Blend strategy for `denoised` vs `denoised*b`.
|
||
- Useful: `inject` (strong), `stable_slerp` (smooth), `lerp` (linear), etc.
|
||
- `blend` (float): Blend amount between base and scaled tensors.
|
||
- `b` (float): Scale factor for the injected path. Higher = stronger effect.
|
||
- `apply_fourier` (bool): Enable frequency-domain shaping.
|
||
- `multiscale_mode` (combo): Preset shaping curves. Use stable options (e.g., Default, Pass-Through, Sharpen).
|
||
- `multiscale_strength` (float): Intensity of multi-scale shaping.
|
||
- `threshold` (int): Base radius in frequency mask.
|
||
- `s` (float): Base scale value applied at `threshold` radius.
|
||
- `force_gain` (float): Final multiplier to boost or attenuate the overall effect.
|
||
- `debug_log` (bool): Prints one-time registration and periodic fire logs.
|
||
|
||
### Notes
|
||
- SHIFT is always-on in Flux; attention/forward-timestep/wrapper paths are disabled for stability.
|
||
- If a multiscale preset yields flat/gray output, switch to a stable preset (e.g., Sharpen, Pass-Through) or tune `threshold`/`s`.
|