The other four generators evaluate a procedural field per pixel per channel, so a wide latent costs one render per channel and a long video multiplies that by the frame count. `spectral` draws the whole channel stack's Fourier coefficients at once, shapes their amplitude by a radial envelope and takes one inverse real FFT. Against domain_warp on the same latents: 0.02s against 1.16s at H3's 608x352/56 frames, 0.13s against 5.31s at 1344x768/124 frames, 0.01s against 0.61s at LTXV's 128 channels -- where it also spans more of them, 115 of 128 against 84. It is a different instrument, not a faster domain_warp. A shaped-Gaussian field is a cloud: no filaments, no swirls, and colour schemes mean nothing to it because there is no vector field to map onto a palette. What it has instead is direct control over the one property the model actually reads. HANDOFF's measurements found the model settles composition from the noise's large-scale structure and that the shader's effect is mostly large-scale structure; the video-diffusion literature arrives at the same place from the other side. Here that band is a parameter rather than a side effect: noise_scale sets it, octaves sets the roll-off, warp_strength the anisotropy. Temporal coherence falls out for free. Holding the seed and advancing time turns each mode at its own rate instead of redrawing the field, measured as 0.27 frame-to-frame correlation against temporal_coherent's 0.26 -- and domain_warp's 0.00, which is worth knowing on its own. Two things that are not obvious and are commented where they live: - The envelope is a soft shelf, `(1 + (|k|/corner)^2)^(-beta/2)`, not a `|k|^-beta` power law. A power law has a pole at DC, so the zero-frequency coefficient swamps everything and every channel comes out a flat wash. - Bandwidth and channel rank are the same dial. Every channel is an independent draw but they all live in whatever subspace the envelope leaves open, so a band narrow enough to look good can hand the sampler a draw spanning far fewer directions than it has channels -- the exact failure the channel-rank work was about. `_corner_floor` keeps the band wide enough to carry CHANNEL_BASIS directions. It takes the grid size and *not* the requested channel count: an envelope that varied with target_channels would draw channel 0 of a wide request differently from a one-channel request at the same seed, and every travel-mode basis is built from one-channel draws. Each channel is seeded separately, the way shaders/base.py seeds its extra channels. Drawing the whole stack from one generator looks equivalent and is not: torch's CPU normal fill works in blocks of 16, so the first channel of a wide draw only matches a one-channel draw when the per-channel element count is a multiple of 16. It was for the coefficients and was not for the phase drift rates, which made channel 0 agree on frame 0 and diverge after it. Two golden fixtures, one image and one video, and tests/test_spectral.py for the properties the fixtures cannot express -- that the field is large-scale rather than white, that noise_scale moves the band, that the rank holds at five latent shapes, and that it stays far cheaper than drawing per channel. Also added to ALL_SHADERS, so it runs the whole generator battery. Its strengths are not calibrated against real prompts the way the other four are, and the tooltip says so. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
215 lines
7.0 KiB
Python
215 lines
7.0 KiB
Python
"""
|
|
ComfyUI-ShaderNoiseKSampler
|
|
|
|
A custom KSampler node that uses shader-based noise patterns
|
|
for creative image generation.
|
|
"""
|
|
|
|
# Import node classes from nodes package
|
|
from .nodes import (
|
|
ShaderNoiseKSampler,
|
|
DirectShaderNoiseKSampler,
|
|
AdvancedImageComparer,
|
|
VideoComparer,
|
|
)
|
|
from .shader_noise_walk import ShaderNoiseWalk
|
|
from .shader_to_tensor import ShaderToTensor
|
|
|
|
# Import shader registry
|
|
from .shaders.registry import (
|
|
ShaderRegistry,
|
|
register_shader,
|
|
get_shader,
|
|
list_shaders,
|
|
)
|
|
|
|
# Importing the generator modules runs their @shader_generator decorators,
|
|
# which register "domain_warp", "tensor_field", "curl_noise" and
|
|
# "temporal_coherent". Only the aliases below are registered here.
|
|
from .shaders.domain_warp import (
|
|
DomainWarpGenerator,
|
|
generate_domain_warp_tensor,
|
|
)
|
|
from .shaders.tensor_field import (
|
|
TensorFieldGenerator,
|
|
generate_tensor_field_tensor,
|
|
)
|
|
from .shaders.curl_noise import (
|
|
CurlNoiseGenerator,
|
|
generate_curl_noise_tensor,
|
|
)
|
|
from .shaders.temporal_coherent_noise import (
|
|
TemporalCoherentNoiseGenerator,
|
|
generate_temporal_coherent_noise_tensor,
|
|
)
|
|
from .shaders.spectral import (
|
|
SpectralNoiseGenerator,
|
|
generate_spectral_tensor,
|
|
)
|
|
|
|
register_shader("curl", CurlNoiseGenerator, {
|
|
"description": "Curl/fluid noise patterns (alias)",
|
|
"supports_temporal": True,
|
|
})
|
|
register_shader("temporal_coherent_noise", TemporalCoherentNoiseGenerator, {
|
|
"description": "Temporally coherent noise (alias)",
|
|
"supports_temporal": True,
|
|
})
|
|
|
|
# Register API routes for server-side parameter saving
|
|
try:
|
|
from server import PromptServer
|
|
from .api_routes import setup_routes
|
|
setup_routes(PromptServer.instance)
|
|
except ImportError:
|
|
# PromptServer not available (e.g., running tests without ComfyUI)
|
|
pass
|
|
except Exception as e:
|
|
print(f"[ShaderNoiseKSampler] Warning: Could not register API routes: {e}")
|
|
|
|
# Legacy SHADER_GENERATORS dict for backward compatibility
|
|
# Maps shader type names to generator functions
|
|
SHADER_GENERATORS = {
|
|
"domain_warp": generate_domain_warp_tensor,
|
|
"tensor_field": generate_tensor_field_tensor,
|
|
"curl": generate_curl_noise_tensor,
|
|
"curl_noise": generate_curl_noise_tensor,
|
|
"temporal_coherent": generate_temporal_coherent_noise_tensor,
|
|
"temporal_coherent_noise": generate_temporal_coherent_noise_tensor,
|
|
}
|
|
|
|
|
|
def _wrap_legacy_generator(legacy_func):
|
|
"""
|
|
Wrap a legacy generator function to accept the new 'params' keyword argument.
|
|
|
|
Legacy functions expect 'shader_params' as a dict, but the new convention uses
|
|
'params' which may be a ShaderParams instance. This wrapper translates between
|
|
the two conventions and converts ShaderParams to dict.
|
|
|
|
Args:
|
|
legacy_func: Legacy generator function expecting shader_params as dict
|
|
|
|
Returns:
|
|
Wrapped function accepting params (ShaderParams or dict)
|
|
"""
|
|
def wrapper(**kwargs):
|
|
# If 'params' is provided but not 'shader_params', translate it
|
|
if 'params' in kwargs and 'shader_params' not in kwargs:
|
|
params = kwargs.pop('params')
|
|
# Convert ShaderParams to dict if needed for legacy function
|
|
if hasattr(params, 'to_dict'):
|
|
shader_params = params.to_dict()
|
|
elif hasattr(params, '__iter__'):
|
|
shader_params = dict(params)
|
|
else:
|
|
shader_params = params
|
|
kwargs['shader_params'] = shader_params
|
|
return legacy_func(**kwargs)
|
|
return wrapper
|
|
|
|
|
|
def get_shader_generator(shader_type: str):
|
|
"""
|
|
Get the appropriate shader generator function based on shader type.
|
|
|
|
This function provides backward compatibility with the old API
|
|
while using the new registry system internally. The returned function
|
|
accepts both 'params' (new convention) and 'shader_params' (legacy convention).
|
|
|
|
Args:
|
|
shader_type: Name of the shader type
|
|
|
|
Returns:
|
|
Generator function for the shader type. Falls back to generate_noise_tensor
|
|
if not found (consistent with shader_noise_ksampler.py behavior).
|
|
"""
|
|
# Import here to avoid circular imports
|
|
from .shader_params_reader import generate_noise_tensor
|
|
|
|
# First try the legacy dict for backward compatibility
|
|
# Wrap legacy functions to accept 'params' keyword argument
|
|
if shader_type in SHADER_GENERATORS:
|
|
return _wrap_legacy_generator(SHADER_GENERATORS[shader_type])
|
|
|
|
# Fall back to registry - return the static generate method
|
|
generator_class = get_shader(shader_type)
|
|
if generator_class is not None:
|
|
# Return the static generate method directly (consistent with shader_noise_ksampler.py)
|
|
return generator_class.generate
|
|
|
|
# Fallback: wrap generate_noise_tensor to translate params -> shader_params
|
|
# This matches the behavior in shader_noise_ksampler.py
|
|
def fallback_wrapper(params, height, width, batch_size, device, seed, target_channels, **kwargs):
|
|
# Convert ShaderParams to dict if needed for legacy function
|
|
if hasattr(params, 'to_dict'):
|
|
shader_params = params.to_dict()
|
|
elif hasattr(params, '__iter__'):
|
|
shader_params = dict(params)
|
|
else:
|
|
shader_params = {}
|
|
return generate_noise_tensor(
|
|
shader_params=shader_params,
|
|
height=height,
|
|
width=width,
|
|
batch_size=batch_size,
|
|
device=device,
|
|
seed=seed,
|
|
target_channels=target_channels,
|
|
**kwargs
|
|
)
|
|
return fallback_wrapper
|
|
|
|
|
|
def register_shader_generator(shader_type: str, generator_function):
|
|
"""
|
|
Register a shader generator function.
|
|
|
|
This function provides backward compatibility with the old API.
|
|
Registers to both the legacy SHADER_GENERATORS dict and the new registry.
|
|
|
|
Args:
|
|
shader_type: Name of the shader type
|
|
generator_function: Generator function or class to register
|
|
"""
|
|
# Add to legacy dict for backward compatibility
|
|
SHADER_GENERATORS[shader_type] = generator_function
|
|
# Also register to the new registry so shader_noise_ksampler.py can find it
|
|
register_shader(shader_type, generator_function)
|
|
|
|
|
|
# Node class mappings
|
|
NODE_CLASS_MAPPINGS = {
|
|
"ShaderNoiseKSampler": ShaderNoiseKSampler,
|
|
"ShaderNoiseKSamplerDirect": DirectShaderNoiseKSampler,
|
|
"ShaderNoiseWalk": ShaderNoiseWalk,
|
|
"AdvancedImageComparer": AdvancedImageComparer,
|
|
"Video Comparer": VideoComparer,
|
|
}
|
|
|
|
# Display name mappings
|
|
NODE_DISPLAY_NAME_MAPPINGS = {
|
|
"ShaderNoiseKSampler": "Shader Noise KSampler",
|
|
"ShaderNoiseKSamplerDirect": "Shader Noise KSampler (Direct)",
|
|
"ShaderNoiseWalk": "Shader Noise Walk",
|
|
"AdvancedImageComparer": "Advanced Image Comparer",
|
|
"Video Comparer": "Video Comparer",
|
|
}
|
|
|
|
# Add web directory for UI components
|
|
WEB_DIRECTORY = "./web"
|
|
|
|
# List of exported elements
|
|
__all__ = [
|
|
"NODE_CLASS_MAPPINGS",
|
|
"NODE_DISPLAY_NAME_MAPPINGS",
|
|
"WEB_DIRECTORY",
|
|
"SHADER_GENERATORS",
|
|
"get_shader_generator",
|
|
"register_shader_generator",
|
|
"ShaderRegistry",
|
|
"register_shader",
|
|
"get_shader",
|
|
"list_shaders",
|
|
]
|