nodes/blepping-ComfyUI-sonar
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Merge pull request #11 from blepping/nov2024update

Browse Source 
November 2024 mega update
This commit is contained in:
blepping
2024-11-30 01:01:33 -07:00
committed by GitHub
parent 4f1934345a 6369627e99
commit f7cbbfcbda
8 changed files with 1020 additions and 165 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+7 -3
View File
@@ -107,11 +107,15 @@ Original Sonar Sampler implementation (for A1111): https://github.com/Kahsolt/st
My version was initially based on this Sonar sampler implementation for Diffusers: https://github.com/alexblattner/modified-euler-samplers-for-sonar-diffusers/
Many noise generation functions copied from https://github.com/Clybius/ComfyUI-Extra-Samplers with only minor modifications. I may have broken some of them in the process _or_ they may not have been suitable for use and I took them anyway. If they don't work it is not a reflection on the original source.
* Many noise generation functions copied from https://github.com/Clybius/ComfyUI-Extra-Samplers with only minor modifications. I may have broken some of them in the process _or_ they may not have been suitable for use and I took them anyway. If they don't work it is not a reflection on the original source.
* New pyramid noise based on implementation in [Jonathan Whitaker](https://wandb.ai/johnowhitaker/multires_noise/reports/Multi-Resolution-Noise-for-Diffusion-Model-Training--VmlldzozNjYyOTU2)'s article on multi-resolution noise.
* Original `SonarPowerNoise` contributed by [elias-gaeros](https://github.com/elias-gaeros/). Additionally, he provided a lot of guidance with refactoring it to allow separate filtering and other enhancements and answered a multitude of dumb questions. To say those changes are only co-authored is probably giving myself too much credit. Thank you! Your patience and help is very much appreciated.
* New 1/f (onef) and power law (white, grey, violet, velvet) noise types referenced from https://github.com/WASasquatch/PowerNoiseSuite
New pyramid noise based on implementation in [Jonathan Whitaker](https://wandb.ai/johnowhitaker/multires_noise/reports/Multi-Resolution-Noise-for-Diffusion-Model-Training--VmlldzozNjYyOTU2)'s article on multi-resolution noise.
## Errata
Original `SonarPowerNoise` contributed by [elias-gaeros](https://github.com/elias-gaeros/). Additionally, he provided a lot of guidance with refactoring it to allow separate filtering and other enhancements and answered a multitude of dumb questions. To say those changes are only co-authored is probably giving myself too much credit. Thank you! Your patience and help is very much appreciated.
* The noise types might not actually do what they claim. In that, I mean something I called "pink" noise might not be what is technically known as "pink noise". My implementations are best-effort. Bug reports and contributions to improve this repo are always welcome!
* Whether noise gets generated on GPU or CPU is probably inconsistent. This means changing GPU types may change seeds, also when this eventually gets fixed it will probably also change seeds.
## Sonar Examples
changelog.md
+15
View File
@@ -2,6 +2,21 @@
Note, only relatively significant changes to user-visible functionality will be included here. Most recent changes at the top.
## 20241129
*Note*: Contains some potentially workflow-breaking changes.
* `pink` noise type renamed to `pink_old` - the implementation was incorrect.
* `power` noise type renamed to `power_old` - the implementation was incorrect.
* Added `onef_pinkish` (higher frequencye) and `onef_greenish` (lower frequency) noise types.
* Added `SonarAdvanced1fNoise` node and `onef_pinkish`, `onef_greenish`, `onef_pinkish_mix`, `onef_greenish_mix`, and `onef_pinkishgreenish` noise types.
* Added `SonarAdvancedPowerLawNoise` node and `grey`, `white`, `violet` and `velvet` noise types.
* The `SonarAdvancedPyramidNoise` node can now use upscale methods from my [ComfyUI-bleh](https://github.com/blepping/ComfyUI-bleh) node pack if it is available.
* Added the `SonarChannelNoise` and `SonarBlendedNoise` nodes.
* Added the `SonarBlehOpsNoise` node.
* Added advanced parameter input to the SampleConfigOverride node, you can now pass options directly to the wrapped sampler function.
* Custom noise inputs now are semi-wildcard and will accept `OCS_NOISE` or `SONAR_CUSTOM_NOISE` interchangeably.
## 20240823
* Added descriptions and tooltips for most nodes.
docs/advanced_noise_nodes.md
+48
View File
@@ -72,6 +72,8 @@ If you want to create noise for initial sampling, connect model and sigmas to th
This node can be used to override configuration settings for other samplers, including the noise type. For example, you could force `euler_ancestral` to use a different noise type. It's also possible to override other settings like `s_noise`, etc. *Note*: The wrapper inspects the sampling function's arguments to see what it supports, so you should connect the sampler directly to this rather than having other nodes (like a different sampler wrapper) in between.
You can enter YAML parameters in the text input, these arguments are passed directly to the sampler function without any error checking. If the same key exists in the node itself (i.e. `s_noise`) the one in the text input will take precedence. Note that these are based on the internal sampler function so the names of the arguments won't necessarily be the same as the sampler node (but they often are). You may need to check the source code for the sampler.
***
### `SONAR_CUSTOM_NOISE to NOISE`
@@ -84,6 +86,26 @@ This node can be used to convert Sonar custom noise to the `NOISE` type used by
Allows setting some parameters for the pyramid noise variants (`pyramid`, `highres_pyramid` and `pyramid_old`). `discount` further from zero generally results in a more extreme colorful effect (can also be set to negative values). Higher `iterations` also tends to make the effect more extreme - zero iterations will just return normal Gaussian noise. You can also experiment with the `upscale_mode` for different effects.
### `SonarAdvanced1fNoise`
More extensive documentation TBD (hopefully). For now, a few recipes:
These differ differ only in alpha. For the other parameters, use `k=1, vf=1, hf=1, use_sqrt=true` to start.
* `blue`: `alpha=1`
* `green`: `alpha=0.75`
* `pink`: `alpha=0.5`
*
### `SonarAdvancedPowerLawNoise`
More extensive documentation TBD (hopefully). For now, a few recipes:
* `white`: `alpha=0, use_sign=true, div_max_dims=none`
* `grey`: `alpha=0, use_sign=false, div_max_dims=none`
* `velvet`: `alpha=1, use_sign=true, div_max_dims=all, use_div_max_abs=true`
* `violet`: `alpha=0.5, use_sign=true, div_max_dims=all, use_div_max_abs=true`
***
### `SonarModulatedNoise`
@@ -326,3 +348,29 @@ Light to dark (negative strength):
Randomly chooses between the noise types in the chain connected to it each time the noise sampler is called.
You generally do not want to use `rescale` here. You can also set `mix_count` to choose and combine multiple
types.
### `SonarChannelNoise`
Allows using a different noise generator per channel. The custom noise items attached to this node are treated as a list where the furthest item from the node will correspond to channel 0. For example where CN is a custom noise node and SCN is the `SonarChannelNoise` node:
```plaintext
CN (channel 0) -> CN (channel 1) -> SCN
```
Don't enable `rescale` in the custom noise nodes attached to `SonarChannelNoise`. If you want a blend of noise types for a channel, you can use something like `SonarBlendedNoise`.
### `SonarBlendedNoise`
Allows blending two noise generators. If [ComfyUI-bleh](https://github.com/blepping/ComfyUI-bleh) is available, you will have access to many more blending modes.
### `SonarBlehOpsNoise`
Only provided if [ComfyUI-bleh](https://github.com/blepping/ComfyUI-bleh) is available. Allows transforming/manipulating noise with bleh blockops expressions. For instance, you can do something like:
```yaml
- ops:
- [multiply, -1]
- [roll, -2, 0.5]
```
to flip the sign on the noise and then roll dimension -2 (height) by 50%.
docs/base_noise_types.md
+18 -2
View File
@@ -8,6 +8,18 @@ noise of that type. However you can either schedule the noise type to kick in at
(as in these examples) and/or mix it with something a bit more run of the mill. See
[advanced_noise_nodes](advanced_noise_nodes.md).
## Documentation TBD
* `grey`
* `onef_greenish_mix` (50/50 mix of positive/negative noise.)
* `onef_greenish`
* `onef_pinkish_mix` (50/50 mix of positive/negative noise.)
* `onef_pinkish`
* `onef_pinkishgreenish` (50/50 mix of `onef_pinkish` and `onef_greenish`.)
* `velvet`
* `violet`
* `white`
## Brownian
This is the default noise type for SDE samplers.
@@ -62,13 +74,17 @@ Variation using bislerp scaling:
***
## Pink
## Pink Old
Previously known as `pink`. The implementation isn't correct, though in terms of results it's fine.
![Pink](../assets/example_images/noise_base_types/noise_pink.png)
***
## Power Builtin
## Power Old
Previously known as `power`. The implementation isn't correct, though in terms of results it's fine.
![PowerBuiltin](../assets/example_images/noise_base_types/noise_power_builtin.png)
py/nodes.py
+463 -69
View File
@@ -8,6 +8,7 @@ from typing import Any, Callable
import numpy as np
import torch
import yaml
from comfy import samplers
from . import external, noise
@@ -24,6 +25,44 @@ from .sonar import (
SonarGuidanceMixin,
)
if "bleh" in external.MODULES:
bleh_latent_utils = external.MODULES["bleh"].py.latent_utils
BLEND_MODES = bleh_latent_utils.BLENDING_MODES
UPSCALE_METHODS = bleh_latent_utils.UPSCALE_METHODS
del bleh_latent_utils
else:
BLEND_MODES = {"lerp": torch.lerp}
UPSCALE_METHODS = (
"bilinear",
"nearest-exact",
"nearest",
"area",
"bicubic",
"bislerp",
)
class Wildcard(str): # noqa: FURB189
__slots__ = ("whitelist",)
@classmethod
def __new__(cls, s, *args: list, whitelist=None, **kwargs: dict):
result = super().__new__(s, *args, **kwargs)
result.whitelist = whitelist
return result
def __ne__(self, other): # noqa: D105
return False if self.whitelist is None else other not in self.whitelist
WILDCARD_NOISE = Wildcard(
"*",
whitelist=frozenset((
"SONAR_CUSTOM_NOISE",
"OCS_NOISE",
)),
)
class NoisyLatentLikeNode:
DESCRIPTION = "Allows generating noise (and optionally adding it) based on a reference latent. Note: For img2img workflows, you will generally want to enable add_to_latent as well as connecting the model and sigmas inputs."
@@ -97,7 +136,7 @@ class NoisyLatentLikeNode:
},
"optional": {
"custom_noise_opt": (
"SONAR_CUSTOM_NOISE",
WILDCARD_NOISE,
{
"tooltip": "Allows connecting a custom noise chain. When connected, noise_type has no effect.",
},
@@ -122,7 +161,7 @@ class NoisyLatentLikeNode:
cls,
*,
noise_type: str,
seed: None | int,
seed: int | None,
latent: dict,
multiplier: float = 1.0,
add_to_latent=False,
@@ -130,7 +169,7 @@ class NoisyLatentLikeNode:
cpu_noise=True,
normalize=True,
custom_noise_opt: object | None = None,
mul_by_sigmas_opt: None | torch.Tensor = None,
mul_by_sigmas_opt: torch.Tensor | None = None,
model_opt: object | None = None,
):
model, sigmas = model_opt, mul_by_sigmas_opt
@@ -213,8 +252,8 @@ class SonarCustomNoiseNodeBase(abc.ABC):
"FLOAT",
{
"default": 1.0,
"min": -100.0,
"max": 100.0,
"min": -10000.0,
"max": 10000.0,
"step": 0.001,
"round": False,
"tooltip": "Scaling factor for the generated noise of this type.",
@@ -230,7 +269,7 @@ class SonarCustomNoiseNodeBase(abc.ABC):
{
"default": 0.0,
"min": 0.0,
"max": 100.0,
"max": 10000.0,
"step": 0.001,
"round": False,
"tooltip": "When non-zero, this custom noise item and other custom noise items items connected to it will have their factor scaled to add up to the specified rescale value. When set to 0, rescaling is disabled.",
@@ -240,7 +279,7 @@ class SonarCustomNoiseNodeBase(abc.ABC):
if include_chain:
result["optional"] |= {
"sonar_custom_noise_opt": (
"SONAR_CUSTOM_NOISE",
WILDCARD_NOISE,
{
"tooltip": "Optional input for more custom noise items.",
},
@@ -286,7 +325,7 @@ class SonarCustomNoiseNode(SonarCustomNoiseNodeBase):
class SonarNormalizeNoiseNodeMixin:
@staticmethod
def get_normalize(val: str) -> None | bool:
def get_normalize(val: str) -> bool | None:
return None if val == "default" else val == "forced"
@@ -298,7 +337,7 @@ class SonarModulatedNoiseNode(SonarCustomNoiseNodeBase, SonarNormalizeNoiseNodeM
result = super().INPUT_TYPES(include_rescale=False, include_chain=False)
result["required"] |= {
"sonar_custom_noise": (
"SONAR_CUSTOM_NOISE",
WILDCARD_NOISE,
{
"tooltip": "Input custom noise to modulate.",
},
@@ -395,7 +434,7 @@ class SonarRepeatedNoiseNode(SonarCustomNoiseNodeBase, SonarNormalizeNoiseNodeMi
result = super().INPUT_TYPES(include_rescale=False, include_chain=False)
result["required"] |= {
"sonar_custom_noise": (
"SONAR_CUSTOM_NOISE",
WILDCARD_NOISE,
{
"tooltip": "Custom noise input for items to repeat. Note: Unlike most other custom noise nodes, this is treated like a list.",
},
@@ -471,7 +510,7 @@ class SonarScheduledNoiseNode(SonarCustomNoiseNodeBase, SonarNormalizeNoiseNodeM
},
),
"sonar_custom_noise": (
"SONAR_CUSTOM_NOISE",
WILDCARD_NOISE,
{
"tooltip": "Custom noise to use when start_percent and end_percent matches.",
},
@@ -503,7 +542,7 @@ class SonarScheduledNoiseNode(SonarCustomNoiseNodeBase, SonarNormalizeNoiseNodeM
}
result["optional"] |= {
"fallback_sonar_custom_noise": (
"SONAR_CUSTOM_NOISE",
WILDCARD_NOISE,
{
"tooltip": "Optional input for noise to use when outside of the start_percent, end_percent range. NOTE: When not connected, defaults to NO NOISE which is probably not what you want.",
},
@@ -547,13 +586,13 @@ class SonarCompositeNoiseNode(SonarCustomNoiseNodeBase, SonarNormalizeNoiseNodeM
result = super().INPUT_TYPES(include_rescale=False, include_chain=False)
result["required"] |= {
"sonar_custom_noise_dst": (
"SONAR_CUSTOM_NOISE",
WILDCARD_NOISE,
{
"tooltip": "Custom noise input for noise where the mask is not set.",
},
),
"sonar_custom_noise_src": (
"SONAR_CUSTOM_NOISE",
WILDCARD_NOISE,
{
"tooltip": "Custom noise input for noise where the mask is set..",
},
@@ -625,7 +664,7 @@ class SonarGuidedNoiseNode(SonarCustomNoiseNodeBase, SonarNormalizeNoiseNodeMixi
},
),
"sonar_custom_noise": (
"SONAR_CUSTOM_NOISE",
WILDCARD_NOISE,
{
"tooltip": "Custom noise input to combine with the guidance.",
},
@@ -707,7 +746,7 @@ class SonarRandomNoiseNode(SonarCustomNoiseNodeBase, SonarNormalizeNoiseNodeMixi
result = super().INPUT_TYPES(include_rescale=False, include_chain=False)
result["required"] |= {
"sonar_custom_noise": (
"SONAR_CUSTOM_NOISE",
WILDCARD_NOISE,
{
"tooltip": "Custom noise input for noise items to randomize. Note: Unlike most other custom noise nodes, this is treated like a list.",
},
@@ -750,6 +789,131 @@ class SonarRandomNoiseNode(SonarCustomNoiseNodeBase, SonarNormalizeNoiseNodeMixi
)
class SonarChannelNoiseNode(SonarCustomNoiseNodeBase, SonarNormalizeNoiseNodeMixin):
DESCRIPTION = "Custom noise type that uses a different noise generator for each channel. Note: The connected noise items are treated as a list. If you want to blend noise types, you can use something like a SonarBlendedNoise node."
@classmethod
def INPUT_TYPES(cls):
result = super().INPUT_TYPES(include_rescale=False, include_chain=False)
result["required"] |= {
"sonar_custom_noise": (
WILDCARD_NOISE,
{
"tooltip": "Custom noise input for noise items corresponding to each channel. SD1/2x and SDXL use 4 channels, Flux and SD3 use 16. Note: Unlike most other custom noise nodes, this is treated like a list where the noise item furthest from the node corresponds to channel 0.",
},
),
"insufficient_channels_mode": (
("wrap", "repeat", "zero"),
{
"default": "wrap",
"tooltip": "Controls behavior for when there are less noise items connected than channels in the latent. wrap - wraps back to the first noise item, repeat - repeats the last item, zero - fills the channel with zeros (generally not recommended).",
},
),
"normalize": (
("default", "forced", "disabled"),
{
"tooltip": "Controls whether the generated noise is normalized to 1.0 strength.",
},
),
}
return result
@classmethod
def get_item_class(cls):
return noise.ChannelNoise
def go(
self,
factor,
*,
sonar_custom_noise,
insufficient_channels_mode,
normalize,
):
return super().go(
factor,
noise=sonar_custom_noise,
insufficient_channels_mode=insufficient_channels_mode,
normalize=self.get_normalize(normalize),
)
class SonarBlendedNoiseNode(SonarCustomNoiseNodeBase, SonarNormalizeNoiseNodeMixin):
DESCRIPTION = "Custom noise type that allows blending two other noise items."
@classmethod
def INPUT_TYPES(cls):
result = super().INPUT_TYPES()
result["required"] |= {
"noise_2_percent": (
"FLOAT",
{
"default": 0.5,
"tooltip": "Blend strength for custom_noise_2. Note that if set to 0 then custom_noise_2 is optional (and will not be called to generate noise) and if set to 1 then custom_noise_1 will not be called to generate noise. This is worth mentioning since going from a strength of 0.000000001 to 0 could make a big difference.",
},
),
"blend_mode": (
tuple(BLEND_MODES.keys()),
{
"default": "lerp",
"tooltip": "Mode used for blending the two noise types. More modes will be available if ComfyUI-bleh is installed.",
},
),
"normalize": (
("default", "forced", "disabled"),
{
"tooltip": "Controls whether the generated noise is normalized to 1.0 strength. For weird blend modes, you may want to set this to forced.",
},
),
}
result["optional"] |= {
"custom_noise_1": (
WILDCARD_NOISE,
{
"tooltip": "Custom noise. Optional if noise_2 percent is 1.",
},
),
"custom_noise_2": (
WILDCARD_NOISE,
{
"tooltip": "Custom noise. Optional if noise_2_percent is 0.",
},
),
}
return result
@classmethod
def get_item_class(cls):
return noise.BlendedNoise
def go(
self,
*,
factor,
rescale,
sonar_custom_noise_opt=None,
normalize,
noise_2_percent,
custom_noise_1=None,
custom_noise_2=None,
blend_mode="lerp",
):
blend_function = BLEND_MODES.get(blend_mode)
if blend_function is None:
raise ValueError("Unknown blend mode")
return super().go(
factor,
rescale=rescale,
sonar_custom_noise_opt=sonar_custom_noise_opt,
blend_function=blend_function,
normalize=self.get_normalize(normalize),
custom_noise_1=custom_noise_1,
custom_noise_2=custom_noise_2,
noise_2_percent=noise_2_percent,
)
class SonarAdvancedPyramidNoiseNode(SonarCustomNoiseNodeBase):
DESCRIPTION = (
"Custom noise type that allows specifying parameters for Pyramid variants."
@@ -787,15 +951,7 @@ class SonarAdvancedPyramidNoiseNode(SonarCustomNoiseNodeBase):
},
),
"upscale_mode": (
(
"default",
"bilinear",
"nearest-exact",
"nearest",
"area",
"bicubic",
"bislerp",
),
("default", *UPSCALE_METHODS),
{
"tooltip": "Allows setting the scaling mode for Pyramid noise. Leave on default to use the variant default.",
"default": "default",
@@ -830,6 +986,163 @@ class SonarAdvancedPyramidNoiseNode(SonarCustomNoiseNodeBase):
)
class SonarAdvanced1fNoiseNode(SonarCustomNoiseNodeBase):
DESCRIPTION = "Custom noise type that allows specifying parameters for 1f (pink, green, etc) variants."
@classmethod
def INPUT_TYPES(cls):
result = super().INPUT_TYPES()
result["required"] |= {
"alpha": (
"FLOAT",
{
"default": 0.25,
"tooltip": "Similar to the advanced power noise node, positive values increase low frequencies (with colorful effects), negative values increase high frequencies.",
},
),
"k": (
"FLOAT",
{
"default": 1.0,
"tooltip": "Currently no description of exactly what it does, it's just another knob you can try turning for a different effect.",
},
),
"vertical_factor": (
"FLOAT",
{
"default": 1.0,
"tooltip": "Vertical frequency scaling factor.",
},
),
"horizontal_factor": (
"FLOAT",
{
"default": 1.0,
"tooltip": "Horizontal frequency scaling factor.",
},
),
"use_sqrt": (
"BOOLEAN",
{
"default": True,
"tooltip": "Controls whether to sqrt when dividing the FFT. Negative hfac/wfac won't work when enabled. Turning it off seems to make the parameters have a much stronger effect.",
},
),
}
return result
@classmethod
def get_item_class(cls):
return noise.Advanced1fNoise
def go(
self,
*,
factor,
rescale,
alpha,
k,
vertical_factor,
horizontal_factor,
use_sqrt,
sonar_custom_noise_opt=None,
):
return super().go(
factor,
rescale=rescale,
sonar_custom_noise_opt=sonar_custom_noise_opt,
alpha=alpha,
k=k,
hfac=vertical_factor,
wfac=horizontal_factor,
use_sqrt=use_sqrt,
)
class SonarAdvancedPowerLawNoiseNode(SonarCustomNoiseNodeBase):
DESCRIPTION = "Custom noise type that allows specifying parameters for power law (grey, violet, etc) variants. "
@classmethod
def INPUT_TYPES(cls):
result = super().INPUT_TYPES()
result["required"] |= {
"alpha": (
"FLOAT",
{
"default": 0.5,
"tooltip": "Alpha parameter of the generated noise. Positive values (low frequency noise) tend to produce colorful results.",
},
),
"div_max_dims": (
(
"none",
"non-batch",
"spatial",
"all",
"batch",
"channel",
"height",
"width",
),
{
"default": "non-batch",
"tooltip": "If non-none, the noise gets divide by the maxmimu over this dimension.",
},
),
"use_div_max_abs": (
"BOOLEAN",
{
"default": True,
"tooltip": "Only has an effect when div_max_dims is not none. Controls whether maximization is done with the absolute values or raw values.",
},
),
"use_sign": (
"BOOLEAN",
{
"default": False,
"tooltip": "When set, only the sign of the initial noise is used, so -0.5, -0.2 all turn into -1, 0.5, 2, etc all turn into 1.",
},
),
}
return result
@classmethod
def get_item_class(cls):
return noise.AdvancedPowerLawNoise
MAX_DIMS_MAP = { # noqa: RUF012
"none": None,
"non-batch": (-3, -2, -1),
"spatial": (-2, -1),
"all": (),
"batch": 0,
"channel": 1,
"height": 2,
"width": 3,
}
def go(
self,
*,
factor,
rescale,
alpha,
div_max_dims,
use_sign,
use_div_max_abs,
sonar_custom_noise_opt=None,
):
return super().go(
factor,
rescale=rescale,
sonar_custom_noise_opt=sonar_custom_noise_opt,
alpha=alpha,
div_max_dims=self.MAX_DIMS_MAP.get(div_max_dims),
use_sign=use_sign,
use_div_max_abs=use_div_max_abs,
)
class CustomNOISE:
def __init__(
self,
@@ -903,7 +1216,7 @@ class SonarToComfyNOISENode:
return {
"required": {
"custom_noise": (
"SONAR_CUSTOM_NOISE",
WILDCARD_NOISE,
{
"tooltip": "Custom noise type to convert.",
},
@@ -1180,7 +1493,7 @@ class SamplerNodeSonarEulerAncestral(SamplerNodeSonarEuler):
result["optional"].update(
{
"custom_noise_opt": (
"SONAR_CUSTOM_NOISE",
WILDCARD_NOISE,
{
"tooltip": "Optional input for custom noise used during ancestral or SDE sampling. When connected, the built-in noise_type selector is ignored.",
},
@@ -1254,7 +1567,7 @@ class SamplerNodeSonarDPMPPSDE(SamplerNodeSonarEuler):
result["optional"].update(
{
"custom_noise_opt": (
"SONAR_CUSTOM_NOISE",
WILDCARD_NOISE,
{
"tooltip": "Optional input for custom noise used during ancestral or SDE sampling. When connected, the built-in noise_type selector is ignored.",
},
@@ -1302,7 +1615,7 @@ class SamplerNodeSonarDPMPPSDE(SamplerNodeSonarEuler):
class SamplerNodeConfigOverride:
DESCRIPTION = "Allows overriding paramaters for a SAMPLER. Only parameters that particular sampler supports will be applied, so for example setting ETA will have no effect for non-ancestral Euler."
KWARG_OVERRIDES = ("s_noise", "eta", "s_churn", "r", "solver_type")
# KWARG_OVERRIDES = ("s_noise", "eta", "s_churn", "r", "solver_type")
@classmethod
def INPUT_TYPES(cls):
@@ -1369,17 +1682,28 @@ class SamplerNodeConfigOverride:
},
"optional": {
"noise_type": (
tuple(NoiseType.get_names()),
("DEFAULT", *NoiseType.get_names()),
{
"tooltip": "Noise type used during ancestral or SDE sampling. Only used when the custom noise input is not connected.",
"default": "DEFAULT",
"tooltip": "Noise type used during ancestral or SDE sampling. Leave blank to use the default for the attached sampler. Only used when the custom noise input is not connected.",
},
),
"custom_noise_opt": (
"SONAR_CUSTOM_NOISE",
WILDCARD_NOISE,
{
"tooltip": "Optional input for custom noise used during ancestral or SDE sampling. When connected, the built-in noise_type selector is ignored.",
},
),
"yaml_parameters": (
"STRING",
{
"tooltip": "Allows specifying custom parameters via YAML. This input can be converted to a multiline text widget. Note: When specifying paramaters this way, there is no error checking.",
"placeholder": "# YAML or JSON here",
"dynamicPrompts": False,
"multiline": True,
"defaultInput": True,
},
),
},
}
@@ -1401,7 +1725,25 @@ class SamplerNodeConfigOverride:
noise_type=None,
custom_noise_opt=None,
normalize=True,
yaml_parameters="",
):
sampler_kwargs = {
"s_noise": s_noise,
"eta": eta,
"s_churn": s_churn,
"r": r,
"solver_type": sde_solver,
}
if yaml_parameters:
extra_params = yaml.safe_load(yaml_parameters)
if extra_params is None:
pass
elif not isinstance(extra_params, dict):
raise ValueError(
"SamplerConfigOverride: yaml_parameters must either be null or an object",
)
else:
sampler_kwargs |= extra_params
return (
samplers.KSAMPLER(
self.sampler_function,
@@ -1410,14 +1752,10 @@ class SamplerNodeConfigOverride:
"override_sampler_cfg": {
"sampler": sampler,
"noise_type": NoiseType[noise_type.upper()]
if noise_type is not None
if noise_type not in {None, "DEFAULT"}
else None,
"custom_noise": custom_noise_opt,
"s_noise": s_noise,
"eta": eta,
"s_churn": s_churn,
"r": r,
"solver_type": sde_solver,
"sampler_kwargs": sampler_kwargs,
"cpu_noise": cpu_noise,
"normalize": normalize,
},
@@ -1444,43 +1782,41 @@ class SamplerNodeConfigOverride:
if extra_args is None:
extra_args = {}
cfg = override_sampler_cfg
sampler, noise_type, custom_noise, cpu, normalize = (
sampler, sampler_kwargs, noise_type, custom_noise, cpu, normalize = (
cfg["sampler"],
cfg["sampler_kwargs"],
cfg.get("noise_type"),
cfg.get("custom_noise"),
cfg.get("cpu_noise", True),
cfg.get("normalize", True),
)
sigma_min, sigma_max = sigmas[sigmas > 0].min(), sigmas.max()
seed = extra_args.get("seed")
if custom_noise is not None:
noise_sampler = custom_noise.make_noise_sampler(
x,
sigma_min,
sigma_max,
seed=seed,
cpu=cpu,
normalized=normalize,
)
elif noise_type is not None:
noise_sampler = noise.get_noise_sampler(
noise_type,
x,
sigma_min,
sigma_max,
seed=seed,
cpu=cpu,
normalized=normalize,
)
sig = inspect.signature(sampler.sampler_function)
params = sig.parameters
kwargs = kwargs.copy()
if "noise_sampler" in params:
seed = extra_args.get("seed")
if custom_noise is not None:
noise_sampler = custom_noise.make_noise_sampler(
x,
sigma_min,
sigma_max,
seed=seed,
cpu=cpu,
normalized=normalize,
)
elif noise_type is not None:
noise_sampler = noise.get_noise_sampler(
noise_type,
x,
sigma_min,
sigma_max,
seed=seed,
cpu=cpu,
normalized=normalize,
)
kwargs |= {k: v for k, v in sampler_kwargs.items() if k in params}
if "noise_sampler" in params:
kwargs["noise_sampler"] = noise_sampler
for k in cls.KWARG_OVERRIDES:
if k not in params or cfg.get(k) is None:
continue
kwargs[k] = cfg[k]
return sampler.sampler_function(
model,
x,
@@ -1499,6 +1835,8 @@ NODE_CLASS_MAPPINGS = {
"SamplerConfigOverride": SamplerNodeConfigOverride,
"NoisyLatentLike": NoisyLatentLikeNode,
"SonarAdvancedPyramidNoise": SonarAdvancedPyramidNoiseNode,
"SonarAdvanced1fNoise": SonarAdvanced1fNoiseNode,
"SonarAdvancedPowerLawNoise": SonarAdvancedPowerLawNoiseNode,
"SonarCustomNoise": SonarCustomNoiseNode,
"SonarCompositeNoise": SonarCompositeNoiseNode,
"SonarModulatedNoise": SonarModulatedNoiseNode,
@@ -1506,6 +1844,8 @@ NODE_CLASS_MAPPINGS = {
"SonarScheduledNoise": SonarScheduledNoiseNode,
"SonarGuidedNoise": SonarGuidedNoiseNode,
"SonarRandomNoise": SonarRandomNoiseNode,
"SonarChannelNoise": SonarChannelNoiseNode,
"SonarBlendedNoise": SonarBlendedNoiseNode,
"SONAR_CUSTOM_NOISE to NOISE": SonarToComfyNOISENode,
}
@@ -1517,18 +1857,19 @@ if "bleh" in external.MODULES:
bleh = external.MODULES["bleh"]
bleh_latentutils = bleh.py.latent_utils
bleh_ops = bleh.py.nodes.ops
class SonarBlendFilterNoiseNode(
SonarCustomNoiseNodeBase,
SonarNormalizeNoiseNodeMixin,
):
DESCRIPTION = "Custom noise type that allows blending and filtering the output of another noise generator."
DESCRIPTION = "Custom noise type that allows blending and filtering the output of another noise generator using ComfyUI-bleh."
@classmethod
def INPUT_TYPES(cls):
result = super().INPUT_TYPES(include_rescale=False, include_chain=False)
result["required"] |= {
"sonar_custom_noise": ("SONAR_CUSTOM_NOISE",),
"sonar_custom_noise": (WILDCARD_NOISE,),
"blend_mode": (
("simple_add", *bleh_latentutils.BLENDING_MODES.keys()),
),
@@ -1604,7 +1945,60 @@ if "bleh" in external.MODULES:
normalize_result=self.get_normalize(normalize_result),
)
NODE_CLASS_MAPPINGS["SonarBlendFilterNoise"] = SonarBlendFilterNoiseNode
class SonarBlehOpsNoiseNode(
SonarCustomNoiseNodeBase,
SonarNormalizeNoiseNodeMixin,
):
DESCRIPTION = (
"Custom noise type that allows manipulating noise with ComfyUI-bleh ops."
)
@classmethod
def INPUT_TYPES(cls):
result = super().INPUT_TYPES(include_rescale=False, include_chain=False)
result["required"] |= {
"sonar_custom_noise": (WILDCARD_NOISE,),
"normalize": (
("default", "forced", "disabled"),
{
"tooltip": "Controls whether the generated noise is normalized to 1.0 strength.",
},
),
"rules": (
"STRING",
{
"tooltip": "Enter rules in the bleh block ops format here.",
"placeholder": "# YAML ops here",
"dynamicPrompts": False,
"multiline": True,
},
),
}
return result
@classmethod
def get_item_class(cls):
return noise.BlehOpsNoise
def go(
self,
*,
factor,
sonar_custom_noise,
rules,
normalize,
):
return super().go(
factor,
noise=sonar_custom_noise.clone(),
rules=bleh_ops.RuleGroup.from_yaml(rules),
normalize=normalize,
)
NODE_CLASS_MAPPINGS |= {
"SonarBlendFilterNoise": SonarBlendFilterNoiseNode,
"SonarBlehOpsNoise": SonarBlehOpsNoiseNode,
}
if "restart" in external.MODULES:
rs = external.MODULES["restart"]
@@ -1648,7 +2042,7 @@ if "restart" in external.MODULES:
"chunked_mode": ("BOOLEAN", {"default": True}),
},
"optional": {
"custom_noise_opt": ("SONAR_CUSTOM_NOISE",),
"custom_noise_opt": (WILDCARD_NOISE,),
},
}
@@ -1716,7 +2110,7 @@ if "restart" in external.MODULES:
"chunked_mode": ("BOOLEAN", {"default": True}),
},
"optional": {
"custom_noise_opt": ("SONAR_CUSTOM_NOISE",),
"custom_noise_opt": (WILDCARD_NOISE,),
},
}
py/noise.py
+292 -20
View File
@@ -199,28 +199,60 @@ class NoiseSampler:
return noise
class AdvancedPyramidNoise(CustomNoiseItemBase):
class AdvancedNoiseBase(CustomNoiseItemBase):
ns_factory_arg_keys = ()
# This has to be done as a property for some reason.
@property
def ns_factory(self):
raise NotImplementedError
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
noise_types = {
"pyramid": pyramid_noise_like,
"pyramid_old": pyramid_old_noise_like,
"highres_pyramid": highres_pyramid_noise_like,
}
if self.ns_factory is None:
raise NotImplementedError("ns_factory not implemented")
noise_sampler_kwargs = {}
if self.discount is not None:
noise_sampler_kwargs["discount"] = self.discount
if self.iterations is not None:
noise_sampler_kwargs["iterations"] = self.iterations
if self.upscale_mode is not None:
noise_sampler_kwargs["upscale_mode"] = self.upscale_mode
self.sampler_function = NoiseSampler.simple(
partial(noise_types[self.variant], **noise_sampler_kwargs),
for k in self.ns_factory_arg_keys:
v = getattr(self, k, None)
if v is not None:
noise_sampler_kwargs[k] = v
self.sampler_factory = NoiseSampler.simple(
partial(self.ns_factory, **noise_sampler_kwargs),
)
@torch.no_grad()
def make_noise_sampler(self, *args, **kwargs):
return self.sampler_function(*args, factor=self.factor, **kwargs)
return self.sampler_factory(*args, factor=self.factor, **kwargs)
class AdvancedPyramidNoise(AdvancedNoiseBase):
ns_factory_arg_keys = ("discount", "iterations", "upscale_mode")
pyramid_variants_map = { # noqa: RUF012
"pyramid": pyramid_noise_like,
"pyramid_old": pyramid_old_noise_like,
"highres_pyramid": highres_pyramid_noise_like,
}
@property
def ns_factory(self):
return self.pyramid_variants_map[self.variant]
class Advanced1fNoise(AdvancedNoiseBase):
ns_factory_arg_keys = ("alpha", "hfac", "wfac", "k", "use_sqrt", "base_power")
@property
def ns_factory(self):
return onef_noise_like
class AdvancedPowerLawNoise(AdvancedNoiseBase):
ns_factory_arg_keys = ("alpha", "div_max_dims", "use_sign")
@property
def ns_factory(self):
return powerlaw_noise_like
class CompositeNoise(CustomNoiseItemBase):
@@ -833,9 +865,142 @@ class RandomNoise(CustomNoiseItemBase):
return noise_sampler
class ChannelNoise(CustomNoiseItemBase):
def __init__(self, factor, *, noise, insufficient_channels_mode, normalize):
if len(noise.items) == 0:
raise ValueError("ChannelNoise requires at least one noise item")
if insufficient_channels_mode not in {"wrap", "repeat", "zero"}:
raise ValueError("Bad insufficient_channels_mode")
super().__init__(
factor,
noise=noise.clone(),
insufficient_channels_mode=insufficient_channels_mode,
normalize=normalize,
)
def clone_key(self, k):
if k == "noise":
return self.noise.clone()
return super().clone_key(k)
def make_noise_sampler(self, x, *args, normalized=True, **kwargs):
factor = self.factor
icmode = self.insufficient_channels_mode
c = x.shape[1]
noise_items = self.noise.items[:c]
num_samplers = len(noise_items)
def make_zero_noise_sampler(x, *_args, **_kwargs):
return lambda *_args, **_kwargs: torch.zeros_like(x)
make_zero_noise_sampler.make_noise_sampler = make_zero_noise_sampler
while len(noise_items) < c:
if icmode == "wrap":
item = noise_items[len(noise_items) % num_samplers]
elif icmode == "repeat":
item = noise_items[num_samplers - 1]
elif icmode == "zero":
item = make_zero_noise_sampler
else:
raise ValueError("Bad insufficient_channels_mode")
noise_items.append(item)
noise_samplers = tuple(
ni.make_noise_sampler(
x[:, ni_channel : ni_channel + 1, ...],
*args,
normalized=False,
**kwargs,
)
for ni_channel, ni in enumerate(noise_items)
)
normalize = self.get_normalize("normalize", normalized)
def noise_sampler(s, sn):
noise = torch.cat(tuple(ns(s, sn) for ns in noise_samplers), dim=1)
return scale_noise(noise, factor, normalized=normalize)
return noise_sampler
class BlendedNoise(CustomNoiseItemBase):
def __init__(
self,
factor,
*,
normalize,
blend_function,
custom_noise_1=None,
custom_noise_2=None,
noise_2_percent=0.5,
):
if custom_noise_1 is None and noise_2_percent != 1:
raise ValueError(
"When custom_noise_1 is not attached noise_2_percent must be set to 1",
)
if custom_noise_2 is None and noise_2_percent != 0:
raise ValueError(
"When custom_noise_2 is not attached noise_2_percent must be set to 0",
)
if noise_2_percent == 1:
custom_noise_1, custom_noise_2 = custom_noise_2, None
noise_2_percent = 0.0
super().__init__(
factor,
noise_2_percent=noise_2_percent,
blend_function=blend_function,
custom_noise_1=custom_noise_1.clone(),
custom_noise_2=None if custom_noise_2 is None else custom_noise_2.clone(),
normalize=normalize,
)
def clone_key(self, k):
if k == "custom_noise_1":
return self.custom_noise_1.clone()
if k == "custom_noise_2":
return None if self.custom_noise_2 is None else self.custom_noise_2.clone()
return super().clone_key(k)
def make_noise_sampler(self, x, *args, normalized=True, **kwargs):
factor = self.factor
normalize = self.get_normalize("normalize", normalized)
blend_function = self.blend_function
n2_blend = self.noise_2_percent
n2_blend_tensor = x.new_full((1,), n2_blend)
ns_1 = self.custom_noise_1.make_noise_sampler(
x,
*args,
normalized=False,
**kwargs,
)
ns_2 = (
None
if self.custom_noise_2 is None
else self.custom_noise_2.make_noise_sampler(
x,
*args,
normalized=False,
**kwargs,
)
)
def noise_sampler(s, sn):
noise_1 = ns_1(s, sn)
noise = (
noise_1
if n2_blend == 0 or ns_2 is None
else blend_function(noise_1, ns_2(s, sn), n2_blend_tensor)
)
return scale_noise(noise, factor, normalized=normalize)
return noise_sampler
if "bleh" in external.MODULES:
bleh = external.MODULES["bleh"]
BLU = bleh.py.latent_utils
BOPS = bleh.py.nodes.ops
class BlendFilterNoise(CustomNoiseItemBase):
def __init__(
@@ -855,7 +1020,7 @@ if "bleh" in external.MODULES:
normalize_noise,
):
if len(noise.items) == 0:
raise ValueError("BlendFilterNoise requires ta least one noise item")
raise ValueError("BlendFilterNoise requires at least one noise item")
super().__init__(
factor,
noise=noise.clone(),
@@ -935,6 +1100,58 @@ if "bleh" in external.MODULES:
return noise_sampler
class BlehOpsNoise(CustomNoiseItemBase):
def __init__(
self,
factor,
*,
noise,
rules,
normalize,
):
if len(noise.items) == 0:
raise ValueError("BlehOpsNoise requires at least one noise item")
super().__init__(
factor,
noise=noise.clone(),
rules=rules,
normalize=normalize,
)
def clone_key(self, k):
if k == "noise":
return self.noise.clone()
return super().clone_key(k)
def make_noise_sampler(self, x, *args, normalized=True, **kwargs):
factor = self.factor
normalize = self.get_normalize("normalize", normalized)
rulegroup = self.rules
internal_ns = self.noise.make_noise_sampler(
x,
*args,
normalized=False,
**kwargs,
)
def noise_sampler(s, sn):
noise = internal_ns(s, sn)
if len(rulegroup.rules):
state = {
BOPS.CondType.TYPE: BOPS.PatchType.LATENT,
BOPS.CondType.PERCENT: 0.0,
BOPS.CondType.BLOCK: -1,
BOPS.CondType.STAGE: -1,
"sigma": None if s is None else s,
"h": noise,
"hsp": x.detach().clone(),
"target": "h",
}
noise = rulegroup.eval(state, toplevel=True)["h"]
return scale_noise(noise, factor, normalized=normalize)
return noise_sampler
NOISE_SAMPLERS: dict[NoiseType, Callable] = {
NoiseType.BROWNIAN: NoiseSampler.wrap(sampling.BrownianTreeNoiseSampler),
@@ -942,17 +1159,72 @@ NOISE_SAMPLERS: dict[NoiseType, Callable] = {
NoiseType.UNIFORM: NoiseSampler.simple(uniform_noise_like),
NoiseType.PERLIN: NoiseSampler.simple(rand_perlin_like),
NoiseType.STUDENTT: NoiseSampler.simple(studentt_noise_like),
NoiseType.PINK: NoiseSampler.simple(pink_noise_like),
NoiseType.ONEF_PINKISH: NoiseSampler.simple(partial(onef_noise_like, alpha=-0.5)),
NoiseType.ONEF_GREENISH: NoiseSampler.simple(partial(onef_noise_like, alpha=0.5)),
NoiseType.ONEF_PINKISHGREENISH: NoiseSampler.simple(
lambda x: onef_noise_like(x, alpha=0.5)
.add_(onef_noise_like(x, alpha=-0.5))
.mul_(0.5),
),
NoiseType.ONEF_PINKISH_MIX: NoiseSampler.simple(
lambda x: onef_noise_like(x, alpha=-0.5)
.mul_(-1.0)
.add_(onef_noise_like(x, alpha=-0.5))
.mul_(0.5),
),
NoiseType.ONEF_GREENISH_MIX: NoiseSampler.simple(
lambda x: onef_noise_like(x, alpha=0.5)
.mul_(-1.0)
.add_(onef_noise_like(x, alpha=0.5))
.mul_(0.5),
),
NoiseType.WHITE: NoiseSampler.simple(
partial(
powerlaw_noise_like,
alpha=0.0,
use_sign=True,
),
),
NoiseType.GREY: NoiseSampler.simple(
partial(
powerlaw_noise_like,
alpha=0.0,
use_sign=False,
),
),
NoiseType.VELVET: NoiseSampler.simple(
partial(
powerlaw_noise_like,
alpha=1.0,
use_sign=True,
div_max_dims=(-3, -2, -1),
),
),
NoiseType.VIOLET: NoiseSampler.simple(
partial(
powerlaw_noise_like,
alpha=0.5,
use_sign=True,
div_max_dims=(-3, -2, -1),
),
),
NoiseType.PINK_OLD: NoiseSampler.simple(pink_noise_old_like),
NoiseType.HIGHRES_PYRAMID: NoiseSampler.simple(highres_pyramid_noise_like),
NoiseType.PYRAMID: NoiseSampler.simple(pyramid_noise_like),
NoiseType.RAINBOW_MILD: NoiseSampler.simple(
lambda x: (green_noise_like(x) * 0.55 + rand_perlin_like(x) * 0.7) * 1.15,
lambda x: green_noise_like(x)
.mul_(0.55)
.add_(rand_perlin_like(x).mul_(0.7))
.mul_(1.15),
),
NoiseType.RAINBOW_INTENSE: NoiseSampler.simple(
lambda x: (green_noise_like(x) * 0.75 + rand_perlin_like(x) * 0.5) * 1.15,
lambda x: green_noise_like(x)
.mul_(0.75)
.add_(rand_perlin_like(x).mul_(0.5))
.mul_(1.15),
),
NoiseType.LAPLACIAN: NoiseSampler.simple(laplacian_noise_like),
NoiseType.POWER: NoiseSampler.simple(power_noise_like),
NoiseType.POWER_OLD: NoiseSampler.simple(power_noise_old_like),
NoiseType.GREEN_TEST: NoiseSampler.simple(green_noise_like),
NoiseType.PYRAMID_OLD: NoiseSampler.simple(pyramid_old_noise_like),
NoiseType.PYRAMID_BISLERP: NoiseSampler.simple(
py/noise_generation.py
+170 -68
View File
@@ -6,42 +6,56 @@ from enum import Enum, auto
from typing import Callable
import torch
from comfy.model_management import device_supports_non_blocking
from comfy.utils import common_upscale
from torch import FloatTensor, Generator, Tensor
from torch.distributions import Laplace, StudentT
from .external import MODULES as EXT
# ruff: noqa: D412, D413, D417, D212, D407, ANN002, ANN003, FBT001, FBT002, S311
class NoiseType(Enum):
GAUSSIAN = auto()
UNIFORM = auto()
BROWNIAN = auto()
PERLIN = auto()
STUDENTT = auto()
HIGHRES_PYRAMID = auto()
PYRAMID = auto()
PYRAMID_MIX = auto()
PINK = auto()
LAPLACIAN = auto()
POWER = auto()
RAINBOW_MILD = auto()
RAINBOW_INTENSE = auto()
GAUSSIAN = auto()
GREEN_TEST = auto()
PYRAMID_OLD = auto()
PYRAMID_BISLERP = auto()
HIGHRES_PYRAMID_BISLERP = auto()
PYRAMID_OLD_BISLERP = auto()
PYRAMID_OLD_AREA = auto()
PYRAMID_AREA = auto()
GREY = auto()
HIGHRES_PYRAMID = auto()
HIGHRES_PYRAMID_AREA = auto()
HIGHRES_PYRAMID_BISLERP = auto()
LAPLACIAN = auto()
ONEF_GREENISH = auto()
ONEF_GREENISH_MIX = auto()
ONEF_PINKISH = auto()
ONEF_PINKISH_MIX = auto()
ONEF_PINKISHGREENISH = auto()
PERLIN = auto()
PINK_OLD = auto()
POWER_OLD = auto()
PYRAMID = auto()
PYRAMID_AREA = auto()
PYRAMID_BISLERP = auto()
PYRAMID_DISCOUNT5 = auto()
PYRAMID_MIX_BISLERP = auto()
PYRAMID_MIX = auto()
PYRAMID_MIX_AREA = auto()
PYRAMID_MIX_BISLERP = auto()
PYRAMID_OLD = auto()
PYRAMID_OLD_AREA = auto()
PYRAMID_OLD_BISLERP = auto()
RAINBOW_INTENSE = auto()
RAINBOW_MILD = auto()
STUDENTT = auto()
UNIFORM = auto()
VELVET = auto()
VIOLET = auto()
WHITE = auto()
@classmethod
def get_names(cls, default=None, skip=None):
def get_names(cls, default=GAUSSIAN, skip=None):
if default is not None:
if isinstance(default, int):
default = cls(default)
yield default.name.lower()
for nt in cls:
if nt == default or (skip and nt in skip):
@@ -65,6 +79,32 @@ def scale_noise(noise, factor=1.0, *, normalized=True, threshold_std_devs=2.5):
return noise.mul_(factor) if factor != 1 else noise
if "bleh" in EXT:
scale_samples = EXT["bleh"].py.latent_utils.scale_samples
else:
def scale_samples(
samples,
width,
height,
*,
mode="bicubic",
):
return common_upscale(samples, width, height, mode, None)
CAN_NONBLOCK = {}
def tensor_to(tensor, dest):
device = dest.device if isinstance(dest, torch.Tensor) else dest
non_blocking = CAN_NONBLOCK.get(device)
if non_blocking is None:
non_blocking = device_supports_non_blocking(device)
CAN_NONBLOCK[device] = non_blocking
return tensor.to(dest, non_blocking=non_blocking)
def get_positions(block_shape: tuple[int, int]) -> Tensor:
"""
Generate position tensor.
@@ -233,7 +273,7 @@ def perlin_noise(
# random vectors on grid points
vectors = unfold_grid(torch.stack((torch.cos(angle), torch.sin(angle)), dim=1))
# positions inside grid cells [0, 1)
positions = get_positions((bh, bw)).to(vectors)
positions = tensor_to(get_positions((bh, bw)), vectors)
return perlin_noise_tensor(vectors, positions).squeeze(0)
@@ -251,11 +291,14 @@ def rand_perlin_like(x, *, generator=None):
noise_height = noise.size(dim=2)
noise_width = noise.size(dim=3)
for _ in range(2):
noise += perlin_noise(
(noise_height, noise_width),
(noise_height, noise_width),
batch_size=x.shape[1], # This should be the number of channels.
).to(x.device)
noise += tensor_to(
perlin_noise(
(noise_height, noise_width),
(noise_height, noise_width),
batch_size=x.shape[1], # This should be the number of channels.
),
x.device,
)
return scale_noise(noise)
@@ -267,9 +310,8 @@ def uniform_noise_like(x, *, generator=None):
device=x.device,
layout=x.layout,
generator=generator,
)
- 0.5
) * 3.46
).sub_(0.5)
).mul_(3.46)
def highres_pyramid_noise_like(
@@ -292,12 +334,11 @@ def highres_pyramid_noise_like(
for i in range(iterations):
r = rs[i].item()
h, w = min(orig_h * 15, int(h * (r**i))), min(orig_w * 15, int(w * (r**i)))
noise += common_upscale(
torch.randn(b, c, h, w, generator=generator).to(x),
noise += scale_samples(
tensor_to(torch.randn(b, c, h, w, generator=generator), x),
orig_w,
orig_h,
upscale_mode,
None,
mode=upscale_mode,
).mul_(discount**i)
if h >= orig_h * 15 or w >= orig_w * 15:
break # Lowest resolution is 1x1
@@ -313,14 +354,14 @@ def pyramid_old_noise_like(
iterations=5,
upscale_mode="nearest-exact",
):
size = x.size()
size = x.shape
b, c, h, w = size
orig_h, orig_w = h, w
noise = torch.zeros(size=size, dtype=x.dtype, layout=x.layout, device=device)
r = 1
for i in range(iterations):
r *= 2
noise += common_upscale(
noise += scale_samples(
torch.normal(
mean=0,
std=0.5**i,
@@ -332,10 +373,9 @@ def pyramid_old_noise_like(
),
orig_w,
orig_h,
upscale_mode,
None,
mode=upscale_mode,
).mul_(discount**i)
return noise.to(device=x.device)
return tensor_to(noise, x.device)
# Copied from https://wandb.ai/johnowhitaker/multires_noise/reports/Multi-Resolution-Noise-for-Diffusion-Model-Training--VmlldzozNjYyOTU2
@@ -357,12 +397,11 @@ def pyramid_noise_like(
torch.rand(1, generator=generator).cpu().item() * 2 + 2
) # Rather than always going 2x,
w, h = max(1, int(w / (r**i))), max(1, int(h / (r**i)))
noise += common_upscale(
torch.randn(b, c, w, h).to(x),
noise += scale_samples(
tensor_to(torch.randn(b, c, w, h), x),
orig_h,
orig_w,
upscale_mode,
None,
mode=upscale_mode,
).mul_(
discount**i,
)
@@ -372,7 +411,7 @@ def pyramid_noise_like(
def studentt_noise_like(x):
noise = StudentT(loc=0, scale=0.2, df=1).rsample(x.size())
noise = StudentT(loc=0, scale=0.2, df=1).rsample(x.shape)
s: FloatTensor = torch.quantile(noise.flatten(start_dim=1).abs(), 0.75, dim=-1)
s = s.reshape(*s.shape, 1, 1, 1)
noise = noise.clamp(-s, s)
@@ -391,39 +430,100 @@ def green_noise_like(x, *, generator=None): # noqa: ARG001
power[0, 0] = 1
noise = torch.fft.ifft2(torch.fft.fft2(noise) / torch.sqrt(power))
noise *= scale / noise.std()
noise = torch.real(noise).to(x.device)
noise = tensor_to(torch.real(noise), x.device)
return scale_noise(noise)
def generate_1f_noise(tensor, alpha, k, generator=None):
"""Generate 1/f noise for a given tensor.
Args:
tensor: The tensor to add noise to.
alpha: The parameter that determines the slope of the spectrum.
k: A constant.
Returns:
A tensor with the same shape as `tensor` containing 1/f noise.
"""
batch = tensor.shape[0]
fft = torch.fft.fft2(tensor)
freq = torch.arange(1, len(fft) + 1, dtype=torch.float).view(batch, 1, 1, 1)
# Completely wrong implementation here.
def generate_1f_noise_old(tensor, alpha, k, generator=None):
freq = 1.0
spectral_density = k / freq**alpha
return torch.randn(tensor.shape, generator=generator) * spectral_density
def pink_noise_like(x, *, generator=None):
return scale_noise(generate_1f_noise(x, 2.0, 1.0, generator=generator)).to(x.device)
def pink_noise_old_like(x, *, generator=None):
return tensor_to(
scale_noise(generate_1f_noise_old(x, 2.0, 1.0, generator=generator)),
x.device,
)
# Referenced from: https://github.com/WASasquatch/PowerNoiseSuite
def generate_1f_noise(
tensor,
*,
alpha=-2.0,
k=1.0,
hfac=1.0,
wfac=1.0,
base_power=1.0,
use_sqrt=True,
generator=None,
):
batch, _channels, height, width = tensor.shape
noise = torch.randn(tensor.shape, generator=generator)
freq_x = torch.fft.fftfreq(height, hfac)
freq_y = torch.fft.fftfreq(width, wfac)
fx, fy = torch.meshgrid(freq_x, freq_y, indexing="ij")
power = (fx**2 + fy**2) ** (-alpha / 2.0)
if k != 0:
power = k / power
power[0, 0] = base_power
power = power.unsqueeze(0).expand(batch, 1, height, width)
noise_fft = torch.fft.fftn(noise)
noise_fft /= (
torch.sqrt(power.to(noise_fft.dtype)) if use_sqrt else power.to(noise_fft.dtype)
)
return torch.fft.ifftn(noise_fft).real
def onef_noise_like(x, *, generator=None, **kwargs):
return tensor_to(
scale_noise(generate_1f_noise(x, generator=generator, **kwargs)),
x.device,
)
# Referenced from: https://github.com/WASasquatch/PowerNoiseSuite
def generate_powerlaw_noise(
tensor: torch.Tensor,
*,
alpha=1.0,
div_max_dims=None,
use_sign=False,
use_div_max_abs=True,
generator=None,
) -> torch.Tensor:
noise = torch.randn(tensor.shape, generator=generator)
modulation = torch.abs(noise) ** alpha
noise = (torch.sign(noise) if use_sign else noise).mul_(modulation)
if div_max_dims is not None:
noise /= torch.amax(
torch.abs(noise) if use_div_max_abs else noise,
keepdim=True,
dim=div_max_dims,
)
return noise
def powerlaw_noise_like(x, *, generator=None, **kwargs):
return tensor_to(
scale_noise(generate_powerlaw_noise(x, generator=generator, **kwargs)),
x.device,
)
def laplacian_noise_like(x):
noise = torch.randn_like(x).div_(4.0)
noise += Laplace(loc=0, scale=1.0).rsample(x.size()).to(noise.device)
noise += tensor_to(Laplace(loc=0, scale=1.0).rsample(x.shape), noise.device)
return scale_noise(noise)
def power_noise_like(tensor, alpha=2, k=1): # This doesn't work properly right now
def power_noise_old_like(tensor, alpha=2, k=1): # This doesn't work properly right now
"""Generate 1/f noise for a given tensor.
Args:
@@ -440,10 +540,10 @@ def power_noise_like(tensor, alpha=2, k=1): # This doesn't work properly right
(len(fft),) + (1,) * (tensor.dim() - 1),
)
spectral_density = k / freq**alpha
noise = torch.rand(tensor.shape).mul_(spectral_density)
mean = torch.mean(noise, dim=(-2, -1), keepdim=True).to(tensor.device)
std = torch.std(noise, dim=(-2, -1), keepdim=True).to(tensor.device)
return noise.to(tensor.device).sub_(mean).div_(std)
noise = tensor_to(torch.rand(tensor.shape).mul_(spectral_density), tensor.device)
mean = torch.mean(noise, dim=(-2, -1), keepdim=True)
std = torch.std(noise, dim=(-2, -1), keepdim=True)
return noise.sub_(mean).div_(std)
__all__ = (
@@ -452,8 +552,10 @@ __all__ = (
"green_noise_like",
"highres_pyramid_noise_like",
"laplacian_noise_like",
"pink_noise_like",
"power_noise_like",
"onef_noise_like",
"pink_noise_old_like",
"power_noise_old_like",
"powerlaw_noise_like",
"pyramid_noise_like",
"pyramid_old_noise_like",
"rand_perlin_like",
py/powernoise.py
+7 -3
View File
@@ -16,7 +16,11 @@ from comfy.k_diffusion.sampling import BrownianTreeNoiseSampler
from PIL import Image
from torch import Tensor
from .nodes import SonarCustomNoiseNodeBase, SonarNormalizeNoiseNodeMixin
from .nodes import (
WILDCARD_NOISE,
SonarCustomNoiseNodeBase,
SonarNormalizeNoiseNodeMixin,
)
from .noise import CustomNoiseItemBase
from .noise_generation import scale_noise
@@ -112,7 +116,7 @@ class PowerFilter:
scale=1.0,
rel_bw=0.125,
oversample=4,
compose_with: None | PowerFilter = None,
compose_with: PowerFilter | None = None,
compose_mode="max",
):
self.min_freq = min_freq
@@ -688,7 +692,7 @@ class SonarPowerFilterNoiseNode(SonarPowerNoiseNode, SonarNormalizeNoiseNodeMixi
del result["required"][k]
result["required"] |= {
"sonar_custom_noise": (
"SONAR_CUSTOM_NOISE",
WILDCARD_NOISE,
{
"tooltip": "Custom noise type to filter.",
},