6.2 KiB
Stable-Diffusion-temperature-settings
Provides the ability to set the temperature for both UNET and CLIP. For ComfyUI.
The nodes
Specifities
- The CLIP patch ignores the connections and patches the model within the memory. Simply disconnecting it does not revert the behavior. To revert to default behavior set it at 1 or reload the model without the node connected. It is the only node ignoring the connections and does not modify anything but the connected CLIP model.
- For SD1 and SDXL nodes: settings the temperature at zero will use a dynamic scale proportional to the resolution. It is good for lower resolutions but not on point for higher.
Requirements
Requires pytorch 2.3 and above.
Usage
Like any other model patch:
- Pay attention to not use SD1 nodes on SDXL and vice-versa or you will get a key not found error.
CLIP temperature at 0.75, 1 and 1.25. Prompt "a bio-organic living plant spaceship"
It basically either refine or dilutes the precision.
Most things below relates to the relation with resolution.
Resolution unlocking side-effect:
Changing the UNET temperature allows to obtain better results a different resolutions.
It needs to be higher for smaller resolutions and lower for higher resolutions.
To this effect I wrote a dynamic scaling temperature function. It sets itself relatively to the selected resolution.
TO USE IT: Select the patch corresponding to your model (currently SD1.x or SDXL). Set the temperature slider at 0.
It is imperfect for resolutions higher than 1.5x as the amplification becomes too strong. But it is very effective for smaller. For higher resolutions you can always set the temperature manually or do a double-pass or use the "light" intensity.
There seem to be a proportional trend. I would however need faster hardware to be able to generate more samples and become able to detect it. I opened the discussions for this repository if you feel like sharing your best settings.
Examples
Using SDXL, 512x512, 256x256, 128x128 with/without:
SDXL 1536x1536, single pass, 12 steps. I let you guess which is which.
Here using SDXL at a resolution of 328x328. First row temperature at 1, second row using dynamic scaled attention:
Non cherry-picked SD v1-5-pruned-emaonly at 512*1024, first row with the dynamic scale, second without:
Lost workflows for these as they were done during testing but proves the idea of making SD better at different resolutions:
The temperature was applied to all layers except input 1 and 2, output 9, 10 and 11. At 0.71. Only on self-attention. Using SD v1-5-pruned-emaonly. Resolution at 1024*512. The temperature matches 1/2**2 and the surface is double the normal resolution.
Workflow:
Troubleshooting / Q and A
- KeyError: 'input_1': You mixed up the SDXL/SD1 nodes
- Something the 'scale' argument: You need to update pytorch
- Windows error message when starting ComfyUI: You updated pytorch but not xformers (that update goes fast)
- Images are more burned than my pizza when I code: temperature too low
- Images are blurry like I removed my glasses: temperature too high
- Does it work for "..." : I have no idea. If it's another model then the dynamic scale is not set yet so you can not set it at 0. You can however use the node named "Unet Temperature any model" and set the temperature manually!
- It is compatible with "...": I can only say that if you want to use the Automatic-CFG with it, you should plug these nodes BEFORE it. For the other patches I do not know. Some may replace the attention function if they do patch the transformer layers.
- Which are the best values? This is uncharted territory. The CLIP temperature is more forgetful. The UNET a lot less, yet something near 0.71 seems overall
- Out of memory error: It uses pytorch attention. It comes from you ;)
Patreon
Give an incentive to contributors: