Stable-Diffusion-temperature-settings
Provides the ability to set the temperature for both UNET and CLIP. For ComfyUI.
Usage
Like any other model patch:
CLIP temperature at 0.75, 1 and 1.25. Prompt "a bio-organic living plant spaceship"
It basically either refine or dilutes the precision.
All the nodes
-
Model specific nodes only have for difference a custom auto-scaling when set at 0
-
The CLIP temperature patches the attention within the VRAM/RAM. The modification ignores connections but is only linked to what is loaded. To get default behavior set the temperature at one.
Requirements
Requires pytorch 2.3 and above.
Most things below relates to the relation with resolution.
Resolution unlocking side-effect:
Changing the UNET temperature allows to obtain better results a different resolutions.
It needs to be higher for smaller resolutions and lower for higher resolutions.
To this effect I wrote a dynamic scaling temperature function. It sets itself relatively to the selected resolution.
TO USE IT: Select the patch corresponding to your model (currently SD1.x or SDXL). Set the temperature slider at 0.
It is imperfect for resolutions higher than 1.5x as the amplification becomes too strong. But it is very effective for smaller. For higher resolutions you can always set the temperature manually or do a double-pass or use the "light" intensity.
There seem to be a proportional trend. I would however need faster hardware to be able to generate more samples and become able to detect it. I opened the discussions for this repository if you feel like sharing your best settings.
Examples
Using SDXL, 512x512, 256x256, 128x128 with/without:
SDXL 1536x1536, single pass, 12 steps. I let you guess which is which.
Here using SDXL at a resolution of 328x328. First row temperature at 1, second row using dynamic scaled attention:
Non cherry-picked SD v1-5-pruned-emaonly at 512*1024, first row with the dynamic scale, second without:
Lost workflows for these as they were done during testing but proves the idea of making SD better at different resolutions:
The temperature was applied to all layers except input 1 and 2, output 9, 10 and 11. At 0.71. Only on self-attention. Using SD v1-5-pruned-emaonly. Resolution at 1024*512. The temperature matches 1/2**2 and the surface is double the normal resolution.
Workflow:
Troubleshooting / Q and A
- KeyError: 'input_1': You mixed up the SDXL/SD1 nodes
- Something the 'scale' argument: You need to update pytorch
- Windows error message when starting ComfyUI: You updated pytorch but not xformers (that update goes fast)
- Images are more burned than my pizza when I code: temperature too low
- Images are blurry like I removed my glasses: temperature too high
- Does it work for such model: Probably! There is a node for all models. The model-specific only benefit from an automatic scale if you set them at zero.
- It is compatible with such extension: I can only say that if you want to use the Automatic-CFG with it, you should plug these nodes BEFORE it. For the other patches I do not know. Some may replace the attention function if they do patch the transformer layers.
- Which are the best values? This is uncharted territory. The CLIP temperature is more forgetful. The UNET a lot less, yet something near 0.71 seems overall
- Out of memory error: It uses pytorch attention. It comes from you ;)
Patreon
Give an incentive to contributors: