Merge branch 'master'
This commit is contained in:
Æmotion
@@ -0,0 +1,302 @@
|
||||
# ComfyUI-ShaderNoiseKSampler
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
**Transform AI image generation from random sampling into deliberate navigation through mathematical topologies.**
|
||||
|
||||
ComfyUI-ShaderNoiseKSampler is an advanced custom KSampler node that replaces standard noise generation with a multi-stage shader-based system. Unlike traditional sampling that teleports randomly between different seeds, this tool provides a structured vehicle for exploring the neighborhood around a chosen seed, using mathematical patterns to navigate along coherent paths through latent space.
|
||||
|
||||
> [!TIP]
|
||||
> Perfect for artists and researchers looking to explore the latent space with more intention, sculpt image features with mathematical precision, and achieve unique visual styles through deliberate exploration rather than random discovery.
|
||||
|
||||
|
||||
*(Suggestion: Replace with a WEBP/GIF showcasing the ShaderNoiseKSampler in action, perhaps cycling through different noise types or parameters.)*
|
||||
|
||||
## 🧭 Navigating the Seed Universe
|
||||
|
||||
Think of standard image generation, where you try different seeds, as driving from one town to another. Each new seed takes you to a completely different town.
|
||||
|
||||
The ShaderNoiseKSampler works differently. It's like picking one town (one specific seed) that you find particularly interesting, and instead of just driving through to the next one, you decide to stop, get out of the car, and really explore *that specific town*. You can wander down its hidden alleyways, check out its diverse neighborhoods, and discover all the unique details and variations it holds.
|
||||
|
||||
The shader noise is your map and the shader parameters (like Noise Scale, Octaves, Warp Strength, etc.) are your compass for this in-depth local exploration:
|
||||
|
||||
- **🔍 Noise Scale: The Zoom Control** - Determines how "zoomed in" or "zoomed out" you are in latent space
|
||||
- **🔬 Octaves: The Detail Slider** - Controls the level of detail and complexity in your noise pattern
|
||||
- **🌀 Warp Strength: The Non-Linear Navigator** - Creates non-linear paths through latent space
|
||||
- **🔄 Phase Shift: The Perspective Shifter** - Reveals different "facets" of the same core elements
|
||||
|
||||
The core innovation is treating latent space as a territory to be explored rather than a lottery to be played - turning the act of AI image generation into a journey of deliberate artistic discovery guided by the elegant language of mathematical patterns.
|
||||
|
||||
## 🔮 The Innovation: Controllable Noise
|
||||
|
||||
While traditional samplers rely on pure randomness, **ShaderNoiseKSampler** introduces a paradigm shift: mathematically controllable noise patterns derived from shader technology (shader noise). This isn't just navigation - it's the invention of a new vehicle.
|
||||
|
||||
By replacing the standard random noise distribution with structured shader-generated patterns (shader noise), we transform the diffusion process from a random walk into a deliberate journey. The noise itself becomes an artistic medium you can sculpt through mathematical parameters:
|
||||
|
||||
- Scale, octaves, and warp create precise noise topographies
|
||||
- Transformations and blending modes give you noise "vocabulary"
|
||||
- Shape masks and color schemes provide spatial and frequency control
|
||||
|
||||
This controlled noise approach bridges the gap between the deterministic world of procedural generation and the probabilistic nature of diffusion models, offering creative control without sacrificing the generative AI's creative potential.
|
||||
|
||||
## 📖 The Shader Matrix: In-Depth Documentation
|
||||
|
||||
A core feature of this project is the **"📊 Show Shader Matrix"** button available on the `ShaderNoiseKSampler` (and related) nodes. Clicking this button opens an extensive, self-contained modal dialog—The Shader Matrix—which provides:
|
||||
|
||||
- Detailed explanations of all shader noise types, mathematical foundations, and parameters.
|
||||
- Interactive visualizations of noise patterns and shape masks.
|
||||
- Python code examples for noise generation.
|
||||
- A comprehensive guide to the philosophy and usage of the node.
|
||||
|
||||
This README provides an overview, but the Shader Matrix is your ultimate guide for deep dives!
|
||||
|
||||
## ✨ Features
|
||||
|
||||
|
||||
*(Suggestion: Replace with a screenshot of the ShaderNoiseKSampler node interface.)*
|
||||
|
||||
- **🚀 Advanced KSampler Replacement**: Integrates directly into your workflow as a KSampler.
|
||||
- **🔬 Multi-Stage Shader Application**:
|
||||
- **Sequential Stages**: Apply shader noise over segments of the diffusion process.
|
||||
- **Injection Stages**: Apply shader noise at specific, discrete steps.
|
||||
- **🎨 Twelve Shader Noise Archetypes**: Explore a vast range of patterns:
|
||||
- Tensor Field, Cellular, Domain Warp, Fractal (FBM), Perlin, Waves, Gaussian, Heterogeneous FBM, Interference, Spectral, 3D Projection, Curl Noise.
|
||||
- **🎭 Sophisticated Blending & Transformations**:
|
||||
- **Blend Modes**: Combine shader noise with base noise using modes like Multiply, Add, Overlay, Screen, Soft Light, Hard Light, Difference.
|
||||
- **Noise Transformations**: Apply mathematical operations (Absolute, Sin, Square Root, etc.) to shader noise before blending.
|
||||
- **💠 Shape Masks**: Spatially modulate noise with geometric overlays (Radial, Linear, Grid, Vignette, Spiral, Hexgrid, etc.) with adjustable strength.
|
||||
- **🌈 Color Schemes Integration**: Apply color transformations (Inferno, Magma, Viridis, Jet, Turbo, etc.) to the noise *before* it influences the diffusion model, subtly guiding structure and aesthetics. Adjustable intensity.
|
||||
- **⏳ Temporal Coherence**:
|
||||
- Generate frame-consistent evolving noise for animations.
|
||||
- Ensure consistent base noise for predictable exploration when tweaking parameters for still images.
|
||||
- **🎛️ Granular Control**:
|
||||
- Global `shader_strength` and per-stage strength tuning.
|
||||
- Adjust `noise_scale`, `octaves`, `warp_strength`, `phase_shift`, and more for each stage.
|
||||
- **💾 Parameter Management**: Save your shader parameter configurations.
|
||||
- **📊 "Show Shader Matrix" Button**: Access comprehensive, interactive documentation and visualizations directly within ComfyUI (Alt+M shortcut).
|
||||
- **🤝 Compatibility**: Works with various models including SD 1.5, SDXL, WAN2.1, Hunyuan, and more (see Model Compatibility section).
|
||||
- **🧠 Latent Space Cartography**: Create a map of the territory surrounding your seed, developing an intuitive understanding of how to navigate to specific effects.
|
||||
- **🔄 Persistent Identities in Variation**: Observe how similar elements persist across parameter adjustments, revealing how the model encodes concepts and their relationships.
|
||||
- **💎 Discovery of "Hidden Gems"**: Find interesting variations that exist in the spaces "between" seeds that random sampling might statistically miss.
|
||||
|
||||
## 📥 Installation
|
||||
|
||||
### Option 1: Using ComfyUI Manager
|
||||
|
||||
1. Install [ComfyUI Manager](https://github.com/ltdrdata/ComfyUI-Manager) if you don't have it already.
|
||||
2. Open ComfyUI, go to the **Manager** tab.
|
||||
3. Click on **Install Custom Nodes**.
|
||||
4. Search for "**ComfyUI-ShaderNoiseKSampler**" and click **Install**.
|
||||
5. Restart ComfyUI.
|
||||
|
||||
### Option 2: Manual Installation
|
||||
|
||||
```bash
|
||||
cd /path/to/ComfyUI/custom_nodes
|
||||
git clone https://github.com/AEmotionStudio/ComfyUI-ShaderNoiseKsampler.git
|
||||
```
|
||||
|
||||
Restart ComfyUI after installation. No additional `pip install` steps are required for the core functionality.
|
||||
|
||||
## 🚀 Usage
|
||||
|
||||
1. **Add Node**: Add the `ShaderNoiseKSampler` node to your ComfyUI graph.
|
||||
2. **Connect Inputs**:
|
||||
- `model`: Your primary AI model.
|
||||
- `positive`, `negative`: Your conditioning prompts.
|
||||
- `latent_image`: The input latent (e.g., from an Empty Latent Image node).
|
||||
3. **Basic Sampling Parameters**:
|
||||
- Set `seed`, `steps`, `cfg`, `sampler_name`, `scheduler`, and `denoise` as you would for a standard KSampler.
|
||||
4. **Configure Shader Noise**: This is where the exploration begins!
|
||||
- **Stages**: Define `sequential_stages` and `injection_stages`.
|
||||
- **Global Controls**: Set `shader_strength` (0.0 to disable shaders), `blend_mode`, and `noise_transform`.
|
||||
- **Per-Stage Controls**: For each stage, configure:
|
||||
- `shader_noise_type` (e.g., perlin, cellular)
|
||||
- `noise_scale` (zoom control), `noise_octaves` (detail level), `noise_warp_strength` (non-linear navigation), `noise_phase_shift` (perspective shift)
|
||||
- `shape_mask_type` and `shape_mask_strength`
|
||||
- `color_scheme` and `color_intensity`
|
||||
- Stage-specific `strength_multiplier`
|
||||
- **Temporal Coherence**: Enable `use_temporal_coherence` for animations or consistent exploration.
|
||||
5. **Explore**: Use the "📊 Show Shader Matrix" button (or Alt+M) to better understand the noise patterns you're creating.
|
||||
6. **Generate**: Queue your prompt and witness the shader-guided generation!
|
||||
|
||||
> [!TIP]
|
||||
> Start with a low `shader_strength` (e.g., 0.1-0.3) and a single `sequential_stage` to understand the impact of different noise types and parameters. Gradually increase complexity as you become more familiar with how each parameter navigates the latent space.
|
||||
|
||||
## 🧠 Latent Space Navigation
|
||||
|
||||
Unlike random seed exploration, the ShaderNoiseKSampler provides a methodical way to navigate the latent space:
|
||||
|
||||
- **Creative Control vs. Serendipity**: Find a balance between intentional direction and unexpected discovery. You're not precisely controlling the output, but you're not completely at the mercy of randomness either - you're steering through possibility space.
|
||||
|
||||
- **Frequency Domain Exploration**: The mathematical nature of this tool allows you to explore how different frequency patterns map to semantic features in the generated images, revealing fundamental patterns in how visual information is encoded in the model.
|
||||
|
||||
- **Methodical Discovery**: As you experiment with different parameter combinations, you'll develop an intuitive understanding of how they affect the output, allowing for more deliberate creative choices.
|
||||
|
||||
- **Fine-Grained Control**: You can introduce a wide spectrum of changes, from barely noticeable subtleties to more significant transformations. This gives you the flexibility to choose your desired degree of alteration, all while preserving the core meaning, motion, and recognizable objects (semantic elements) of the image.
|
||||
|
||||
## ⚙️ Configuration Options
|
||||
|
||||
The `ShaderNoiseKSampler` offers extensive control. Key parameters are listed below with their navigational significance. For an exhaustive list and explanations, please refer to the **"📊 Show Shader Matrix"** documentation within ComfyUI.
|
||||
|
||||
| Option | Description & Navigational Significance | Default (Example) |
|
||||
|------------------------------|-------------------------------------------------------------------------------------------------------------|-------------------|
|
||||
| **`seed`** | Master seed for reproducibility. This is your "town" in the latent space universe. | `8888` |
|
||||
| **`steps`** | Number of sampling iterations. | `20` |
|
||||
| **`cfg`** | Classifier-Free Guidance scale. | `7.0` |
|
||||
| **`sampler_name`** | E.g., `euler_ancestral`, `dpm_2_ancestral`. | `euler_ancestral` |
|
||||
| **`scheduler`** | E.g., `normal`, `beta`, `simple`. | `beta` |
|
||||
| **`denoise`** | Denoising strength. | `1.0` |
|
||||
| **`sequential_stages`** | Number of shader stages applied sequentially. | `1` |
|
||||
| **`injection_stages`** | Number of shader stages injected at specific steps. | `0` |
|
||||
| **`shader_strength`** | Global strength of shader noise influence (0.0 to disable). | `0.3` |
|
||||
| **`blend_mode`** | How shader noise combines with base noise (e.g., `multiply`, `add`). | `multiply` |
|
||||
| **`noise_transform`** | Math operation on shader noise (e.g., `none`, `absolute`, `sin`). | `none` |
|
||||
| **`use_temporal_coherence`** | For consistent noise in animations or exploration. | `false` |
|
||||
| **`shader_noise_type (per stage)`** | The base pattern (e.g., `domain_warp`, `tensor_field`, `curl_noise`). | `domain_warp` |
|
||||
| **`noise_scale (per stage)`** | The "Zoom Control" - determines how "zoomed in" or "zoomed out" you are in latent space. | `1.0` |
|
||||
| **`noise_octaves (per stage)`** | The "Detail Slider" - controls the level of detail and complexity in your noise pattern. | `1` |
|
||||
| **`noise_warp_strength (per stage)`** | The "Non-Linear Navigator" - creates non-linear paths through latent space. | `0.5` |
|
||||
| **`noise_phase_shift (per stage)`** | The "Perspective Shifter" - reveals different "facets" of the same core elements. | `0.5` |
|
||||
| **`shape_mask_type (per stage)`** | Geometric mask overlay (e.g., `radial`, `grid`). | `none` |
|
||||
| **`shape_mask_strength (per stage)`**| Intensity of the shape mask. | `0.5` |
|
||||
| **`color_scheme (per stage)`** | Color mapping for noise (e.g., `viridis`, `jet`). | `none` |
|
||||
| **`color_intensity (per stage)`**| Strength of the color scheme influence. | `0.8` |
|
||||
|
||||
### Sampler & Scheduler Compatibility
|
||||
|
||||
Generally, `ShaderNoiseKSampler` aims for broad compatibility. The following are often good starting points (refer to the Shader Matrix for more details):
|
||||
|
||||
- **Recommended Samplers**: `euler_ancestral`, `dpm_2_ancestral`, `dpmppm_2_ancestral`, `lcm`
|
||||
- **Recommended Schedulers**: `beta` (often preferred), `normal`, `simple`, `kl_optimal`
|
||||
|
||||
### 🧱 Model Compatibility
|
||||
|
||||
The `ShaderNoiseKSampler` also aims to be compatible with a variety of models. Compatibility often depends on the model's latent channel structure. The node attempts to automatically detect the channel count for many common models.
|
||||
|
||||
| Model Family / Name | Category | Status | Channels (Expected) | Notes |
|
||||
| ---------------------------------- | ------------ | ------------------------------------ | ------------------- | --------------------------------------------------------------------- |
|
||||
| SD 1.5 / SD_X4 | Image | Compatible | 4 | Standard 4-channel models. |
|
||||
| SDXL / SDXL Refiner | Image | Compatible | 4 | Standard 4-channel models. |
|
||||
| Stable Cascade B | Image | Compatible | 4 | Formerly "Stable Cascade (Decoder/Non-Prior)". |
|
||||
| Stable Cascade (Prior) | Image | Compatible | 16 | Higher channel count. |
|
||||
| SD3 | Image | Compatible | 16 | Higher channel count. |
|
||||
| Flux.1 / Flux | Image | Compatible | 16 | Higher channel count. |
|
||||
| HunyuanDiT | Image | Compatible | 4 | Standard 4-channel model, similar to SD1.5/SDXL. |
|
||||
| Chroma | Image | Compatible | 4 | Assumed 4-channel VAE based on related models. |
|
||||
| HiDream / Flow | Image | Compatible | 16 | Higher channel count. |
|
||||
| WAN / Warp / Pixel / Anime (WAN2.1) | Video (3D) | Compatible | 16 | Higher channel count. |
|
||||
| Mochi | Video (3D) | Compatible | 12 | May require specific handling. |
|
||||
| LTXV | Video (3D) | Compatible | 128 | Very high channel count. Special configuration or specific noise handling might be beneficial. |
|
||||
| CosmosVideo / Cosmos1CV8x8x8 | Video (3D) | Compatible | 16 | Channel count confirmed. |
|
||||
| HunyuanVideo | Video (3D) | Compatible | 16 | Official docs mention a 16x VAE channel compression. Effective channels for noise may differ or require specific handling. |
|
||||
| AnimateDiff (on SD 1.5) | Video (3D) | Compatible | 4 | Uses 4 channels from the base SD 1.5 model. |
|
||||
| Stable Video Diffusion (SVD) | Video (3D) | Compatible | 4 | Latent diffusion model, likely 4 channels. |
|
||||
| ACEStep / ACE | Audio (1D) | Compatible | 8 | Compatibility with 2D noise may vary. |
|
||||
| StableAudio1 | Audio (1D) | Untested | 64 | Compatibility with 2D noise is uncertain. |
|
||||
| Hunyuan3Dv2 | 3D Model | Untested | 64 | 1D channels. Compatibility with 2D noise is uncertain. |
|
||||
| Hunyuan3Dv2mini | 3D Model | Untested | 64 | 1D channels. Compatibility with 2D noise is uncertain. |
|
||||
| **Other Models** | | | | |
|
||||
| Generic 4-channel models | Image | Compatible (by default) | 4 | Most other standard diffusion models. |
|
||||
| Models with >16 channels (unlisted)| Various | Untested / May Require Configuration | Varies | Models with very high channel counts may require specific configurations or might exhibit different performance characteristics. |
|
||||
| Other non-standard architecture | Various | Untested | Varies | Results may vary. |
|
||||
|
||||
**Note on Channel Detection:** The `ShaderNoiseKSampler` includes logic to infer model channel counts (see `get_model_channel_count` in `shader_noise_ksampler.py`). If you encounter issues with a new or untested model, understanding its latent channel structure is important. For models with very high channel counts, performance or compatibility might vary, and specific noise generation approaches could be more suitable.
|
||||
|
||||
## 🔬 Shader Noise Deep Dive (Brief Overview)
|
||||
|
||||
The true depth of `ShaderNoiseKSampler` lies in its components. The "Shader Matrix" covers these extensively.
|
||||
|
||||
- **Shader Noise Types**: Start your exploration with three powerful, freely available noise archetypes: `Domain Warp` for intricate, flowing distortions, `Tensor Field` for structured and directional patterns, and `Curl Noise` for smooth, fluid dynamics. Each offers a **unique visual lens** 🔭 for navigating latent space. ✨ **Additional advanced noise types** are available to supporters on [Ko-fi](https://ko-fi.com/aemotionstudio).
|
||||
* `[Image: Domain Warp Noise Preview]`
|
||||
* `[Image: Tensor Field Noise Preview]`
|
||||
* `[Image: Curl Noise Preview]`
|
||||
- **Blend Modes**: Determine how the crafted shader noise interacts with the underlying base noise. `Multiply` can create depth, `Add` can introduce highlights, and `Overlay` can enhance contrast.
|
||||
- **Noise Transformations**: Apply mathematical functions like `absolute` (creates ridges), `sin` (creates bands), or `sqrt` (compresses highlights) to the raw shader noise before blending, dramatically altering its characteristics.
|
||||
- **Shape Masks**: Impose geometric forms onto your noise. A `radial` mask can create focus, a `grid` can introduce blocky structures. Strength is key.
|
||||
- **Color Schemes**: More than just a visual flair for the noise preview, these schemes (`viridis`, `inferno`, `jet`, etc.) transform the noise data itself. This "colored" noise can then guide the diffusion model in unique ways, influencing texture, features, and mood by altering how the model "perceives" the noise structure.
|
||||
|
||||
## ❓ Troubleshooting
|
||||
|
||||
- **Shader Effects Not Visible**:
|
||||
- Ensure `shader_strength` is greater than `0.0`.
|
||||
|
||||
- **Unexpected Results**: Small parameter changes can sometimes lead to large visual shifts. Use the shader visualizer to understand the noise before generating. Experimentation is encouraged.
|
||||
|
||||
- **Consult the Shader Matrix**: The in-app documentation is your best friend for detailed troubleshooting and understanding.
|
||||
|
||||
### Known Issues
|
||||
|
||||
- **Optimal Use Cases**: The `ShaderNoiseKSampler` currently demonstrates its strongest capabilities in text-to-video and text-to-image generation. Performance with image-to-video or video-to-video workflows may not be as pronounced.
|
||||
|
||||
- **Parameter Queuing with Non-Direct KSampler**: The standard (non-direct) `ShaderNoiseKSampler` node does not support queuing of differing parameter sets for consecutive runs i.e. you can't save over older parameters because their read at runtime. For rapid iteration and experimentation with varied parameters without needing to save each configuration, the `(direct) ShaderNoiseSampler` variant is recommended.
|
||||
|
||||
- > [!WARNING]
|
||||
> **Potential for Visual Instability**: Certain parameter explorations, particularly with high intensity or complex interactions, may result in visually disruptive outputs such as flashing images or harsh artifacts. Users are advised to iterate with caution.
|
||||
|
||||
## 🌱 Research & Development Nature
|
||||
|
||||
ComfyUI-ShaderNoiseKSampler is born from dedicated personal research and investigation. While it introduces exciting ways to navigate latent space, please consider it an active exploration. This means there's a vast potential for further improvements, new discoveries, and community-driven enhancements as the project evolves. Your understanding of its current research-driven phase is appreciated!
|
||||
|
||||
### 🗺️ Roadmap: The Next Evolution
|
||||
|
||||
The journey into mathematically guided latent space exploration is just beginning. Here's a glimpse of what's on the horizon:
|
||||
|
||||
| Area | Focus | Technologies Involved (Examples) |
|
||||
| :---------------------------------------------- | :-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :---------------------------------------- |
|
||||
| **🌌 Advanced Latent Space Cartography** | Evolve current research into an intuitive "Visual Intent Engine". This system will allow users to express desired visual outcomes semantically (e.g., "enhance fabric texture," "shift lighting ambiance"). The engine will then intelligently translate these intents into optimal shader parameter configurations, fostering a more direct and expressive artistic workflow by deeply mapping the interplay between parameters and visual impact. | Semantic AI, Parameter Response Modeling, Machine Learning |
|
||||
| **🎨 Expanded Shader Noise Palettes** | Introduce a richer vocabulary of shader noise types, each with unique navigational properties and aesthetic fingerprints. Explore new dimensions of textural complexity, flow dynamics, and structural organization. | Advanced Noise Algorithms, GLSL concepts |
|
||||
| **🧭 Precision Navigation Tools** | Create more granular and predictable tools for manipulating latent pathways. Research direct correlations between mathematical noise constructs and emergent visual features for greater artistic intent. | Mathematical Modelling, Latent Space Analysis |
|
||||
| **🔮 Cross-Modal Exploration** | Investigate applying structured noise principles to modalities beyond 2D images, such as 3D and audio, opening new avenues for creative exploration. | Signal Processing, Generative Models for Audio/3D |
|
||||
|
||||
Stay tuned to unlock new ways to interact with and understand the vast potential of generative models through the lens of controlled, mathematical noise.
|
||||
|
||||
## 🤝 Contributing
|
||||
|
||||
Contributions are welcome! Whether it's bug reports, feature suggestions, or pull requests, your help is appreciated.
|
||||
|
||||
1. Fork the repository.
|
||||
2. Create a new branch (`git checkout -b feature/AmazingFeature`).
|
||||
3. Make your changes.
|
||||
4. Commit your changes (`git commit -m 'Add some AmazingFeature'`).
|
||||
5. Push to the branch (`git push origin feature/AmazingFeature`).
|
||||
6. Open a Pull Request.
|
||||
|
||||
## 🙏 Acknowledgements
|
||||
|
||||
- The ComfyUI team for creating such a flexible and powerful platform.
|
||||
- The developers of libraries and concepts that inspired aspects of this work (e.g., GLSL, various noise algorithms).
|
||||
- The ComfyUI community for their continuous innovation and support.
|
||||
- Users and contributors who provide feedback and suggestions.
|
||||
|
||||
## 🔗 Connect with Æmotion (Developer)
|
||||
|
||||
- YouTube: [AEmotionStudio](https://www.youtube.com/@aemotionstudio/videos)
|
||||
- GitHub: [AEmotionStudio](https://github.com/AEmotionStudio)
|
||||
- Discord: [Join our community](https://discord.gg/UzC9353mfp)
|
||||
- Website: [aemotionstudio.org](https://aemotionstudio.org/)
|
||||
|
||||
## ☕ Support
|
||||
|
||||
If you find ComfyUI-ShaderNoiseKSampler useful and wish to support its development, consider:
|
||||
|
||||
[](https://ko-fi.com/aemotionstudio)
|
||||
|
||||
Your support helps dedicate more time to maintaining and improving this project, developing new features, and creating better documentation and tutorials.
|
||||
|
||||
### 💖 Additional Ways to Support
|
||||
|
||||
- ⭐ Star the repository on GitHub.
|
||||
- 📢 Share it with others in the AI art community.
|
||||
- 🛠️ Contribute to its development (see Contributing section).
|
||||
- 💡 Provide feedback and feature requests.
|
||||
|
||||
For business inquiries or professional support, please contact me through my [website](https://aemotionstudio.org/) or join my [Discord server](https://discord.gg/UzC9353mfp).
|
||||
|
||||
## 📜 License
|
||||
|
||||
This project is licensed under the **GNU General Public License v3.0**.
|
||||
See the `LICENSE` file for details (You may need to create this file if it doesn't exist, containing the GPL-3.0 license text).
|
||||
+76
@@ -0,0 +1,76 @@
|
||||
# First import components from shader noise ksampler
|
||||
from .shader_noise_ksampler import (
|
||||
ShaderNoiseKSampler,
|
||||
register_shader_generator,
|
||||
SHADER_GENERATORS
|
||||
)
|
||||
|
||||
# Import DirectShaderNoiseKSampler
|
||||
from .direct_shader_ksampler import DirectShaderNoiseKSampler
|
||||
|
||||
# Import the stripped version of ShaderNoiseKSampler
|
||||
# Commented out due to missing file
|
||||
# from .shader_noise_ksampler_stripped import ShaderNoiseKSamplerStripped
|
||||
|
||||
# Next import tensor class
|
||||
from .shader_to_tensor import ShaderToTensor
|
||||
|
||||
# Import and integrate domain warp
|
||||
from .shaders.domain_warp import add_domain_warp_to_tensor, generate_domain_warp_tensor
|
||||
|
||||
# Apply domain warp integration
|
||||
print("=== INTEGRATING DOMAIN WARP TO SHADER SYSTEM ===")
|
||||
add_domain_warp_to_tensor(ShaderToTensor)
|
||||
register_shader_generator("domain_warp", generate_domain_warp_tensor)
|
||||
print(f"Domain warp integration complete - has domain_warp: {'domain_warp' in dir(ShaderToTensor)}")
|
||||
|
||||
# Import and integrate tensor field
|
||||
from .shaders.tensor_field import add_tensor_field_to_tensor, generate_tensor_field_tensor
|
||||
|
||||
# Apply tensor field integration
|
||||
print("=== INTEGRATING TENSOR FIELD TO SHADER SYSTEM ===")
|
||||
add_tensor_field_to_tensor(ShaderToTensor)
|
||||
register_shader_generator("tensor_field", generate_tensor_field_tensor)
|
||||
print(f"Tensor field integration complete - has tensor_field: {'tensor_field' in dir(ShaderToTensor)}")
|
||||
|
||||
# Import and integrate Curl Noise
|
||||
from .shaders.curl_noise import add_curl_noise_to_tensor, generate_curl_noise_tensor
|
||||
|
||||
# Apply Curl Noise integration
|
||||
print("=== INTEGRATING CURL NOISE TO SHADER SYSTEM ===")
|
||||
add_curl_noise_to_tensor(ShaderToTensor)
|
||||
register_shader_generator("curl", generate_curl_noise_tensor)
|
||||
register_shader_generator("curl_noise", generate_curl_noise_tensor)
|
||||
print(f"Curl noise integration complete - has curl_noise: {'curl_noise' in dir(ShaderToTensor)}")
|
||||
|
||||
# Import and integrate temporal coherent noise
|
||||
from .shaders.temporal_coherent_noise import integrate_temporal_coherent_noise, generate_temporal_coherent_noise_tensor
|
||||
|
||||
# Apply temporal coherent noise integration
|
||||
print("=== INTEGRATING TEMPORAL COHERENT NOISE TO SHADER SYSTEM ===")
|
||||
integrate_temporal_coherent_noise()
|
||||
register_shader_generator("temporal_coherent", generate_temporal_coherent_noise_tensor)
|
||||
register_shader_generator("temporal_coherent_noise", generate_temporal_coherent_noise_tensor)
|
||||
print(f"Temporal coherent noise integration complete - has temporal_coherent_noise: {'temporal_coherent_noise' in dir(ShaderToTensor)}")
|
||||
|
||||
# Node class mappings
|
||||
NODE_CLASS_MAPPINGS = {
|
||||
"ShaderNoiseKSampler": ShaderNoiseKSampler,
|
||||
"ShaderNoiseKSamplerDirect": DirectShaderNoiseKSampler,
|
||||
}
|
||||
|
||||
# Display name mappings
|
||||
NODE_DISPLAY_NAME_MAPPINGS = {
|
||||
"ShaderNoiseKSampler": "Shader Noise KSampler",
|
||||
"ShaderNoiseKSamplerDirect": "Shader Noise KSampler (Direct)",
|
||||
}
|
||||
|
||||
# Add web directory for UI components
|
||||
WEB_DIRECTORY = "./web"
|
||||
|
||||
# List of JS files to be loaded
|
||||
__js_files__ = ["gradient_title.js", "shader_renderer.js", "shader_params_save_button.js", "shader_info_button.js", "noise_visualizer.js"]
|
||||
|
||||
|
||||
# List of exported elements
|
||||
__all__ = ["NODE_CLASS_MAPPINGS", "NODE_DISPLAY_NAME_MAPPINGS", "WEB_DIRECTORY", "__js_files__", "__css_files__", "SHADER_GENERATORS"]
|
||||
@@ -0,0 +1,11 @@
|
||||
{
|
||||
"shaderType": "curl_noise",
|
||||
"shaderScale": 1,
|
||||
"shaderOctaves": 1,
|
||||
"shaderWarpStrength": 0.5,
|
||||
"shaderShapeType": "none",
|
||||
"shaderShapeStrength": 1,
|
||||
"shaderPhaseShift": 0.5,
|
||||
"colorScheme": "none",
|
||||
"shaderColorIntensity": 0.8
|
||||
}
|
||||
@@ -0,0 +1,299 @@
|
||||
import comfy.sample
|
||||
from .shader_params_reader import get_shader_params
|
||||
from .shader_noise_ksampler import ShaderNoiseKSampler, get_visualizer, set_debug_level
|
||||
|
||||
# Define a simple parameter response mapper if needed
|
||||
class ParameterResponseMapper:
|
||||
"""Simplified version of ParameterResponseMapper for direct shader parameters"""
|
||||
def __init__(self, model_type="SD1.5"):
|
||||
self.model_type = model_type
|
||||
|
||||
def get_recommended_adjustments(self, target_attrs, current_params, max_params=2):
|
||||
"""Return recommended parameter adjustments based on target attributes"""
|
||||
# This is a simple implementation that could be expanded
|
||||
adjustments = {}
|
||||
|
||||
if not target_attrs or not isinstance(target_attrs, dict):
|
||||
return adjustments
|
||||
|
||||
# Simple mapping from attributes to shader parameters
|
||||
for attr, value in target_attrs.items():
|
||||
# Only adjust if the attribute is strongly present (value > 0.5)
|
||||
if value < 0.5:
|
||||
continue
|
||||
|
||||
# Map common attributes to shader parameters
|
||||
if attr == "detailed" and "octaves" in current_params:
|
||||
adjustments["octaves"] = min(current_params["octaves"] + 1, 8)
|
||||
elif attr == "smooth" and "octaves" in current_params:
|
||||
adjustments["octaves"] = max(current_params["octaves"] - 1, 1)
|
||||
elif attr == "twisted" and "warp_strength" in current_params:
|
||||
adjustments["warp_strength"] = min(current_params["warp_strength"] + 0.2, 5.0)
|
||||
elif attr == "flowing" and "phase_shift" in current_params:
|
||||
adjustments["phase_shift"] = (current_params["phase_shift"] + 0.5) % 6.28
|
||||
elif attr == "large_scale" and "scale" in current_params:
|
||||
adjustments["scale"] = min(current_params["scale"] + 0.5, 10.0)
|
||||
elif attr == "small_scale" and "scale" in current_params:
|
||||
adjustments["scale"] = max(current_params["scale"] - 0.5, 0.1)
|
||||
|
||||
# Limit number of adjustments
|
||||
if len(adjustments) >= max_params:
|
||||
break
|
||||
|
||||
return adjustments
|
||||
|
||||
class DirectShaderNoiseKSampler(ShaderNoiseKSampler):
|
||||
@classmethod
|
||||
def INPUT_TYPES(s):
|
||||
return {
|
||||
"required": {
|
||||
"model": ("MODEL", {"tooltip": "The AI model used for image generation"}),
|
||||
"seed": ("INT", {"default": 8888, "min": 0, "max": 0xffffffffffffffff, "tooltip": "Random seed for generation. Same seed with same parameters will generate the same image."}),
|
||||
"steps": ("INT", {"default": 20, "min": 1, "max": 10000, "tooltip": "Number of sampling steps. Higher values can produce better results but take longer"}),
|
||||
"cfg": ("FLOAT", {"default": 7.0, "min": 0.0, "max": 100.0, "step": 0.1, "tooltip": "Classifier-free guidance scale. Higher values follow the prompt more closely"}),
|
||||
"sampler_name": (comfy.samplers.KSampler.SAMPLERS, {"default": "euler_ancestral", "tooltip": "Algorithm used for the sampling process"}),
|
||||
"scheduler": (comfy.samplers.KSampler.SCHEDULERS, {"default": "beta", "tooltip": "Scheduler used to determine noise level at each step"}),
|
||||
"positive": ("CONDITIONING", {"tooltip": "Positive conditioning/prompts that guide what to include in the image"}),
|
||||
"negative": ("CONDITIONING", {"tooltip": "Negative conditioning/prompts that guide what to exclude from the image"}),
|
||||
"latent_image": ("LATENT", {"tooltip": "Input latent image to be processed"}),
|
||||
"denoise": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.01, "tooltip": "Denoising strength. Lower values preserve more of the original image"}),
|
||||
"sequential_stages": ("INT", {"default": 1, "min": 0, "max": 10, "step": 1, "tooltip": "Number of sequential shader stages to apply before injection stages"}),
|
||||
"injection_stages": ("INT", {"default": 0, "min": 0, "max": 10, "step": 1, "tooltip": "Number of injection shader stages to apply after sequential stages"}),
|
||||
"shader_strength": ("FLOAT", {"default": 0.3, "min": 0.0, "max": 1.0, "step": 0.01, "tooltip": "Strength of the shader noise influence for both stage types. Set to 0.0 to disable and use only base noise."}),
|
||||
"blend_mode": (["normal", "add", "multiply", "screen", "overlay", "soft_light", "hard_light", "difference"], {"default": "multiply", "tooltip": "Method used to blend the shader noise with the base noise"}),
|
||||
"noise_transform": (["none", "reverse", "inverse", "absolute", "square", "sqrt", "log", "sin", "cos"], {"default": "none", "tooltip": "Apply mathematical transformations to the noise for creative effects"}),
|
||||
"use_temporal_coherence": ("BOOLEAN", {"default": False, "tooltip": "Ensures consistent noise patterns. For sequences like video frames, it helps maintain frame-to-frame consistency (e.g., using the same base seed and 4D noise). For single image generations, it ensures the base noise is derived consistently from the main seed."}),
|
||||
|
||||
# New direct shader parameters
|
||||
"shader_type": (["domain_warp", "tensor_field", "curl_noise"], {"default": "domain_warp", "tooltip": "Select the type of shader noise pattern to use in the visualization [different types have different characteristic outputs]"}),
|
||||
"shape_type": (["none", "radial", "linear", "spiral", "checkerboard", "spots", "hexgrid", "stripes", "gradient", "vignette", "cross", "stars", "triangles", "concentric", "rays", "zigzag"], {"default": "none", "tooltip": "Apply a shape mask to the shader noise pattern to create more complex structures [not post processing - is applied to shader noise pattern before rendering]"}),
|
||||
"color_scheme": (["none", "blue_red", "viridis", "plasma", "inferno", "magma", "turbo", "jet", "rainbow", "cool", "hot", "parula", "hsv", "autumn", "winter", "spring", "summer", "copper", "pink", "bone", "ocean", "terrain", "neon", "fire"], {"default": "none", "tooltip": "Choose a color palette to apply to the shader noise visualization [not post processing - is applied to the shader noise pattern before rendering]"}),
|
||||
"noise_scale": ("FLOAT", {"default": 1.0, "min": 0.1, "max": 10.0, "step": 0.001, "tooltip": "Adjust the scale of the shader noise pattern - lower values create larger, zoomed-in features; higher values create smaller, zoomed-out features [small value shifts can lead to larger variations]"}),
|
||||
"octaves": ("FLOAT", {"default": 1.0, "min": 1.0, "max": 8.0, "step": 0.1, "tooltip": "Number of shader noise layers to combine - higher values add more detail and complexity [small value shifts can lead to larger variations]"}),
|
||||
"warp_strength": ("FLOAT", {"default": 0.5, "min": 0.0, "max": 5.0, "step": 0.001, "tooltip": "Control how much the shader noise pattern warps and distorts - higher values create more swirling or complex transformations [small adjustments are good for subtle variations]"}),
|
||||
"shape_mask_strength": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 2.0, "step": 0.0001, "tooltip": "Adjust the intensity of the shape mask\'s effect on the shader noise pattern - higher values make the shape more prominent [small adjustments are good for subtle variations - not effective without shape mask]"}),
|
||||
"phase_shift": ("FLOAT", {"default": 0.5, "min": 0.0, "max": 2.0, "step": 0.0001, "tooltip": "Shift the phase of the shader noise pattern to create different variations or animate patterns over time [small adjustments are good for subtle variations]"}),
|
||||
"color_intensity": ("FLOAT", {"default": 0.8, "min": 0.0, "max": 1.0, "step": 0.0001, "tooltip": "Adjust the intensity of the color scheme application - lower values are more desaturated, higher values are more vibrant [small adjustments are good for subtle variations - not effective without color scheme]"}),
|
||||
},
|
||||
"optional": {
|
||||
"custom_sigmas": ("SIGMAS", {"tooltip": "Optional custom sigma schedule to override the model's default schedule"}),
|
||||
},
|
||||
"hidden": {
|
||||
"debug_level": (["0-Off", "1-Basic", "2-Detailed", "3-Verbose"], {"default": "0-Off", "tooltip": "Enable debugging at specified level to understand what's happening during shader generation and sampling."}),
|
||||
"fast_high_channel_noise": ("BOOLEAN", {"default": False, "tooltip": "Use a faster, simplified noise generation method for models with many channels (>16), like LTXV."}),
|
||||
"sequential_distribution": (["uniform", "linear_decrease", "linear_increase", "gaussian", "first_stronger", "last_stronger"], {"default": "linear_decrease", "tooltip": "How shader strength is distributed across sequential stages"}),
|
||||
"injection_distribution": (["uniform", "linear_decrease", "linear_increase", "gaussian", "first_stronger", "last_stronger"], {"default": "linear_decrease", "tooltip": "How shader strength is distributed across injection stages"}),
|
||||
"denoise_visualization_frequency": (["Every step", "25% intervals", "10% intervals", "4 steps", "2 steps"], {"default": "Every step", "tooltip": "How often to save images during the denoising process. Higher frequency means more images but slower generation."}),
|
||||
"target_attribute_changes": ("STRING", {"forceInput": True, "tooltip": "Connect output from ParameterResponseMapperNode here"}),
|
||||
}
|
||||
}
|
||||
|
||||
RETURN_TYPES = ("LATENT",)
|
||||
FUNCTION = "sample"
|
||||
CATEGORY = "sampling"
|
||||
|
||||
@classmethod
|
||||
def REGISTER_MATRIX_BUTTON(s):
|
||||
return True
|
||||
|
||||
@classmethod
|
||||
def IS_CHANGED(s, model, seed, steps, cfg, sampler_name, scheduler, positive, negative, latent_image,
|
||||
denoise=1.0, sequential_stages=2, injection_stages=3, shader_strength=0.3, blend_mode="multiply",
|
||||
noise_transform="none", sequential_distribution="linear_decrease", injection_distribution="linear_decrease",
|
||||
use_temporal_coherence=False, debug_level="0-Off", fast_high_channel_noise=False,
|
||||
denoise_visualization_frequency="25% intervals", custom_sigmas=None, target_attribute_changes="",
|
||||
shader_type="domain_warp", shape_type="none", color_scheme="none", noise_scale=1.0, octaves=1,
|
||||
warp_strength=0.5, shape_mask_strength=1.0, phase_shift=0.5, color_intensity=0.8):
|
||||
# Ensure all direct parameters also trigger re-execution when changed
|
||||
return (seed, steps, cfg, sampler_name, scheduler, denoise, sequential_stages,
|
||||
injection_stages, shader_strength, blend_mode, noise_transform,
|
||||
sequential_distribution, injection_distribution, use_temporal_coherence,
|
||||
debug_level, denoise_visualization_frequency, custom_sigmas,
|
||||
target_attribute_changes, fast_high_channel_noise,
|
||||
# Include direct shader parameters
|
||||
shader_type, shape_type, color_scheme, noise_scale, octaves,
|
||||
warp_strength, shape_mask_strength, phase_shift, color_intensity)
|
||||
|
||||
def sample(self, model, seed, steps, cfg, sampler_name, scheduler, positive, negative, latent_image,
|
||||
denoise=1.0, sequential_stages=2, injection_stages=3, shader_strength=0.3, blend_mode="multiply",
|
||||
noise_transform="none", sequential_distribution="linear_decrease", injection_distribution="linear_decrease",
|
||||
use_temporal_coherence=False, debug_level="0-Off", fast_high_channel_noise=False,
|
||||
denoise_visualization_frequency="25% intervals", custom_sigmas=None, target_attribute_changes="",
|
||||
shader_type="tensor_field", shape_type="none", color_scheme="none", noise_scale=1.0, octaves=3.0,
|
||||
warp_strength=0.5, shape_mask_strength=1.0, phase_shift=0.0, color_intensity=0.8):
|
||||
"""
|
||||
Run the multi-stage shader noise k-sampler with direct parameter inputs
|
||||
"""
|
||||
# Parse debug level from the selected option
|
||||
debug_level_value = int(debug_level.split("-")[0])
|
||||
|
||||
# Set the debug level in the shader debugger
|
||||
debugger = set_debug_level(debug_level_value)
|
||||
|
||||
# Get the visualizer
|
||||
visualizer = get_visualizer()
|
||||
|
||||
# Get device early from latent_image
|
||||
device = latent_image["samples"].device
|
||||
if debugger.enabled:
|
||||
print(f"ℹ️ Using device: {device}")
|
||||
|
||||
# Get the shader parameters, but we'll modify them with our direct inputs
|
||||
shader_params = get_shader_params()
|
||||
|
||||
# Override shader parameters with direct inputs - ensure all naming variants are set
|
||||
# Shader Type (set all variants)
|
||||
shader_params["shader_type"] = shader_type
|
||||
shader_params["shaderType"] = shader_type
|
||||
|
||||
# Shape Type (set all variants)
|
||||
shader_params["shape_type"] = shape_type
|
||||
shader_params["shaderShapeType"] = shape_type
|
||||
|
||||
# Color Scheme
|
||||
shader_params["colorScheme"] = color_scheme
|
||||
shader_params["color_scheme"] = color_scheme
|
||||
|
||||
# Noise Scale (set all variants)
|
||||
shader_params["scale"] = noise_scale
|
||||
shader_params["shaderScale"] = noise_scale
|
||||
|
||||
# Octaves (set all variants)
|
||||
shader_params["octaves"] = float(octaves) # Ensure float for compatibility
|
||||
shader_params["shaderOctaves"] = float(octaves)
|
||||
|
||||
# Warp Strength (set all variants)
|
||||
shader_params["warp_strength"] = warp_strength
|
||||
shader_params["shaderWarpStrength"] = warp_strength
|
||||
|
||||
# Shape Mask Strength (set all variants)
|
||||
shader_params["shapemaskstrength"] = shape_mask_strength
|
||||
shader_params["shaderShapeStrength"] = shape_mask_strength
|
||||
shader_params["shapeMaskStrength"] = shape_mask_strength # Added capital M version
|
||||
shader_params["shape_mask_strength"] = shape_mask_strength # Added underscore version
|
||||
shader_params["shape_strength"] = shape_mask_strength # Added alternative name checked in shader_to_tensor.py
|
||||
|
||||
# Phase Shift (set all variants)
|
||||
shader_params["phase_shift"] = phase_shift
|
||||
shader_params["shaderPhaseShift"] = phase_shift
|
||||
|
||||
# Color Intensity (set all variants)
|
||||
shader_params["intensity"] = color_intensity
|
||||
shader_params["shaderColorIntensity"] = color_intensity
|
||||
|
||||
# Other parameters
|
||||
shader_params["time"] = shader_params.get("time", 0.0) # Keep existing time or set to 0
|
||||
shader_params["base_seed"] = seed # Set base seed for temporal coherence
|
||||
shader_params["useTemporalCoherence"] = use_temporal_coherence
|
||||
shader_params["temporal_coherence"] = use_temporal_coherence # Add alternative name
|
||||
shader_params["fast_high_channel_noise"] = fast_high_channel_noise
|
||||
|
||||
# Visualization type (default to 3/ellipses as in the default parameters)
|
||||
shader_params["visualization_type"] = shader_params.get("visualization_type", 3)
|
||||
|
||||
# --- Handle Parameter Response Mapper Integration ---
|
||||
if target_attribute_changes and target_attribute_changes.strip():
|
||||
if debugger.enabled:
|
||||
print("🧠 Applying Parameter Response Mapper adjustments...")
|
||||
try:
|
||||
import json
|
||||
target_attrs = json.loads(target_attribute_changes)
|
||||
|
||||
if isinstance(target_attrs, dict) and target_attrs:
|
||||
# Determine model type for mapper (simple inference)
|
||||
model_name_lower = getattr(model, 'model_name', "").lower()
|
||||
if "sdxl" in model_name_lower:
|
||||
mapper_model_type = "SDXL"
|
||||
else:
|
||||
mapper_model_type = "SD1.5" # Default
|
||||
|
||||
# Use our local ParameterResponseMapper
|
||||
mapper = ParameterResponseMapper(model_type=mapper_model_type)
|
||||
|
||||
# Extract current shader parameters relevant to the mapper
|
||||
current_mapper_params = {
|
||||
"scale": noise_scale,
|
||||
"octaves": float(octaves),
|
||||
"warp_strength": warp_strength,
|
||||
"phase_shift": phase_shift
|
||||
}
|
||||
|
||||
if debugger.enabled and debugger.debug_level >= 2:
|
||||
print(f" Mapper using model_type: {mapper_model_type}")
|
||||
print(f" Target Attributes: {target_attrs}")
|
||||
print(f" Current Params for Mapper: {current_mapper_params}")
|
||||
|
||||
# Get recommendations
|
||||
recommendations = mapper.get_recommended_adjustments(
|
||||
target_attrs,
|
||||
current_mapper_params,
|
||||
max_params=2
|
||||
)
|
||||
|
||||
if recommendations:
|
||||
if debugger.enabled:
|
||||
print(f"✅ Mapper recommended adjustments: {recommendations}")
|
||||
# Apply recommendations to the shader_params - set ALL variants
|
||||
for param, value in recommendations.items():
|
||||
# Update all possible parameter naming variants
|
||||
if param == "scale":
|
||||
shader_params["scale"] = value
|
||||
shader_params["shaderScale"] = value
|
||||
elif param == "octaves":
|
||||
shader_params["octaves"] = float(value)
|
||||
shader_params["shaderOctaves"] = float(value)
|
||||
elif param == "warp_strength":
|
||||
shader_params["warp_strength"] = value
|
||||
shader_params["shaderWarpStrength"] = value
|
||||
elif param == "phase_shift":
|
||||
shader_params["phase_shift"] = value
|
||||
shader_params["shaderPhaseShift"] = value
|
||||
else:
|
||||
# For any other parameters
|
||||
shader_params[param] = value
|
||||
except Exception as e:
|
||||
print(f"❌ Error processing target_attribute_changes: {e}. Skipping mapper adjustments.")
|
||||
|
||||
if debugger.enabled:
|
||||
# Debug output for direct parameters
|
||||
print(f"🔧 Direct Shader Parameters:")
|
||||
print(f" Shader Type: {shader_type}")
|
||||
print(f" Shape Type: {shape_type}")
|
||||
print(f" Color Scheme: {color_scheme}")
|
||||
print(f" Noise Scale: {noise_scale}")
|
||||
print(f" Octaves: {octaves}")
|
||||
print(f" Warp Strength: {warp_strength}")
|
||||
print(f" Shape Mask Strength: {shape_mask_strength}")
|
||||
print(f" Phase Shift: {phase_shift}")
|
||||
print(f" Color Intensity: {color_intensity}")
|
||||
|
||||
# Call parent class sample method with the modified shader params
|
||||
# Use the parent class implementation from ShaderNoiseKSampler
|
||||
result = super().sample(
|
||||
model=model,
|
||||
seed=seed,
|
||||
steps=steps,
|
||||
cfg=cfg,
|
||||
sampler_name=sampler_name,
|
||||
scheduler=scheduler,
|
||||
positive=positive,
|
||||
negative=negative,
|
||||
latent_image=latent_image,
|
||||
denoise=denoise,
|
||||
sequential_stages=sequential_stages,
|
||||
injection_stages=injection_stages,
|
||||
shader_strength=shader_strength,
|
||||
blend_mode=blend_mode,
|
||||
noise_transform=noise_transform,
|
||||
sequential_distribution=sequential_distribution,
|
||||
injection_distribution=injection_distribution,
|
||||
use_temporal_coherence=use_temporal_coherence,
|
||||
debug_level=debug_level,
|
||||
fast_high_channel_noise=fast_high_channel_noise,
|
||||
denoise_visualization_frequency=denoise_visualization_frequency,
|
||||
custom_sigmas=custom_sigmas,
|
||||
target_attribute_changes=target_attribute_changes,
|
||||
shader_params_override=shader_params, # Pass our modified shader params to parent
|
||||
)
|
||||
|
||||
return result
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,609 @@
|
||||
import torch
|
||||
import math
|
||||
|
||||
class ShaderToTensor:
|
||||
"""
|
||||
Class for converting shader operations to PyTorch tensors
|
||||
This class serves as the bridge between WebGL shaders and PyTorch tensor operations
|
||||
|
||||
Methods are designed to closely match the shader implementations in the web UI
|
||||
"""
|
||||
|
||||
@classmethod
|
||||
def shader_noise_to_tensor(cls, batch_size=1, height=64, width=64, shader_type="tensor_field",
|
||||
visualization_type=3, scale=1.0, phase_shift=0.0,
|
||||
warp_strength=0.0, time=0.0, device="cuda", seed=0, octaves=3.0,
|
||||
shape_type="none", shape_mask_strength=1.0, shader_params=None):
|
||||
"""
|
||||
Generate noise tensors from shader parameters
|
||||
|
||||
Args:
|
||||
batch_size: Number of images in batch
|
||||
height: Height of tensor
|
||||
width: Width of tensor
|
||||
shader_type: Type of shader ("tensor_field", "heterogeneous_fbm", "projection_3d")
|
||||
visualization_type: Type of visualization for tensor field (0-3)
|
||||
scale: Scale factor
|
||||
phase_shift: Contrast adjustment
|
||||
warp_strength: Warp amount
|
||||
time: Animation time
|
||||
device: Device to create tensor on
|
||||
seed: Random seed
|
||||
octaves: Number of octaves for fractal noise
|
||||
shape_type: Type of shape mask to apply
|
||||
shape_mask_strength: Strength of shape mask application
|
||||
shader_params: Full shader parameters dictionary (if provided)
|
||||
|
||||
Returns:
|
||||
Tensor with shape [batch_size, 4, height, width]
|
||||
"""
|
||||
# If shader_params is provided, use the dedicated method instead
|
||||
if shader_params is not None:
|
||||
return cls.shader_noise_to_tensor_with_params(
|
||||
batch_size, height, width, shader_type,
|
||||
shader_params, time, device, seed
|
||||
)
|
||||
|
||||
# Create coordinate grid
|
||||
y, x = torch.meshgrid(
|
||||
torch.linspace(-1, 1, height, device=device),
|
||||
torch.linspace(-1, 1, width, device=device),
|
||||
indexing='ij'
|
||||
)
|
||||
|
||||
# Combine into coordinate tensor [batch, height, width, 2]
|
||||
p = torch.stack([x, y], dim=-1).unsqueeze(0).repeat(batch_size, 1, 1, 1)
|
||||
|
||||
# Generate different shader patterns
|
||||
if shader_type == "tensor_field":
|
||||
# Simple tensor field pattern
|
||||
shader_noise = cls.tensor_field(
|
||||
p, visualization_type, scale, warp_strength, phase_shift, time, device
|
||||
)
|
||||
elif shader_type == "heterogeneous_fbm":
|
||||
# Heterogeneous FBM noise
|
||||
shader_noise = cls.fbm_noise(
|
||||
p, scale, warp_strength, phase_shift, octaves, time, device
|
||||
)
|
||||
elif shader_type == "projection_3d":
|
||||
# 3D projection noise
|
||||
shader_noise = cls.projection3d(
|
||||
p, scale, warp_strength, phase_shift, octaves, time, device
|
||||
)
|
||||
else:
|
||||
# Raise error instead of falling back to random noise
|
||||
raise ValueError(f"Shader type '{shader_type}' is not directly supported in shader_noise_to_tensor. Please use shader_noise_to_tensor_with_params or ensure the shader type is registered.")
|
||||
|
||||
# Convert from [B, H, W, 1] to [B, 1, H, W]
|
||||
shader_noise = shader_noise.permute(0, 3, 1, 2)
|
||||
|
||||
# Normalize to have mean 0 and std 1 (like typical noise)
|
||||
shader_noise = (shader_noise - shader_noise.mean()) / (shader_noise.std() + 1e-8)
|
||||
|
||||
# Expand channels if needed
|
||||
if shader_noise.shape[1] == 1:
|
||||
# Expand to 9 channels for latent space operations instead of just 4
|
||||
shader_noise = shader_noise.expand(-1, 9, -1, -1) # [B, 1, H, W] -> [B, 9, H, W]
|
||||
# If we have 4 channels but need 9 for latent operations
|
||||
elif shader_noise.shape[1] == 4:
|
||||
# Get existing channels
|
||||
r, g, b, a = shader_noise.chunk(4, dim=1)
|
||||
|
||||
# Create 5 more channels as variations of the existing ones
|
||||
c5 = (r + g) / 2.0 # Average of red and green
|
||||
c6 = (g + b) / 2.0 # Average of green and blue
|
||||
c7 = (b + a) / 2.0 # Average of blue and alpha
|
||||
c8 = (r + b) / 2.0 # Average of red and blue
|
||||
c9 = (r + g + b + a) / 4.0 # Average of all channels
|
||||
|
||||
# Combine all 9 channels
|
||||
shader_noise = torch.cat([r, g, b, a, c5, c6, c7, c8, c9], dim=1)
|
||||
|
||||
return shader_noise
|
||||
|
||||
@classmethod
|
||||
def tensor_field(cls, p, viz_type, scale, warp_strength, phase_shift, time, device):
|
||||
"""
|
||||
Simple implementation of tensor field visualization
|
||||
|
||||
Args:
|
||||
p: Coordinate tensor [batch, height, width, 2]
|
||||
viz_type: Visualization type (0-3)
|
||||
scale: Scale factor
|
||||
warp_strength: Warp amount
|
||||
phase_shift: Contrast adjustment
|
||||
time: Animation parameter
|
||||
device: Torch device
|
||||
|
||||
Returns:
|
||||
Field tensor [batch, height, width, 1]
|
||||
"""
|
||||
batch, height, width, _ = p.shape
|
||||
|
||||
# Scale coordinates by user parameter
|
||||
p = p * scale
|
||||
|
||||
# Apply domain warping if warp strength > 0
|
||||
if warp_strength > 0.0:
|
||||
# Create a smooth flow field for warping
|
||||
warp_coords = p * 0.3 + torch.tensor([time * 0.05, 0.0], device=device).reshape(1, 1, 1, 2)
|
||||
warp_noise = cls.simplex_noise(warp_coords)
|
||||
|
||||
# Apply warp to coordinates
|
||||
p = p + warp_noise * warp_strength
|
||||
|
||||
# Calculate angle field (similar to atan2 in shaders)
|
||||
angle = torch.atan2(p[:, :, :, 1], p[:, :, :, 0])
|
||||
|
||||
# Calculate sines and cosines with 2 offset frequencies
|
||||
sin1 = torch.sin(angle + time * 0.1)
|
||||
cos1 = torch.cos(angle + time * 0.1)
|
||||
sin2 = torch.sin(angle * 2.0 + time * 0.2)
|
||||
cos2 = torch.cos(angle * 2.0 + time * 0.2)
|
||||
|
||||
# Different visualization types
|
||||
if viz_type == 0:
|
||||
# Simple sine wave
|
||||
result = sin1
|
||||
elif viz_type == 1:
|
||||
# Combined sine waves
|
||||
result = sin1 * cos2
|
||||
elif viz_type == 2:
|
||||
# Squared combination
|
||||
result = sin1 * sin1 + cos2 * cos2
|
||||
else: # viz_type == 3 (default - ellipses)
|
||||
# Create elliptical patterns
|
||||
radius = torch.sqrt(torch.sum(p * p, dim=-1, keepdim=True))
|
||||
radius = radius + 0.01 # Avoid division by zero
|
||||
result = (sin1 + cos2) / radius
|
||||
|
||||
# Apply phase shift (contrast adjustment)
|
||||
contrast = 1.0 + phase_shift
|
||||
result = result * contrast
|
||||
|
||||
# Ensure output is in valid [-1, 1] range
|
||||
result = torch.clamp(result, -1.0, 1.0)
|
||||
|
||||
return result
|
||||
|
||||
@classmethod
|
||||
def fbm_noise(cls, p, scale, warp_strength, phase_shift, octaves, time, device):
|
||||
"""
|
||||
Heterogeneous FBM noise implementation
|
||||
|
||||
Args:
|
||||
p: Coordinate tensor [batch, height, width, 2]
|
||||
scale: Scale factor
|
||||
warp_strength: Warp amount
|
||||
phase_shift: Contrast adjustment
|
||||
octaves: Number of detail levels
|
||||
time: Animation time
|
||||
device: Torch device
|
||||
|
||||
Returns:
|
||||
Noise tensor [batch, height, width, 1]
|
||||
"""
|
||||
batch, height, width, _ = p.shape
|
||||
|
||||
# Scale coordinates by user parameter
|
||||
p = p * scale
|
||||
|
||||
# Apply domain warping if warp strength > 0
|
||||
if warp_strength > 0.0:
|
||||
# Create a smooth flow field for warping
|
||||
warp_coords = p * 0.3 + torch.tensor([time * 0.05, 0.0], device=device).reshape(1, 1, 1, 2)
|
||||
warp_noise = cls.simplex_noise(warp_coords)
|
||||
|
||||
# Apply warp to coordinates
|
||||
p = p + warp_noise * warp_strength
|
||||
|
||||
# Generate multi-octave FBM noise
|
||||
result = torch.zeros(batch, height, width, 1, device=device)
|
||||
amplitude = 1.0
|
||||
frequency = 1.0
|
||||
|
||||
# Loop through octaves
|
||||
for i in range(min(int(octaves), 8)):
|
||||
current_p = p * frequency + torch.tensor([i * 2.0, time * 0.1], device=device).reshape(1, 1, 1, 2)
|
||||
value = cls.simplex_noise(current_p)
|
||||
|
||||
# Calculate heterogeneous weight
|
||||
if i > 0:
|
||||
# This creates areas with more or less detail
|
||||
weight = torch.clamp(cls.simplex_noise(p * 0.1 + i * 0.5), 0.0, 1.0)
|
||||
value = value * weight
|
||||
|
||||
# Add weighted noise at current frequency
|
||||
result += amplitude * value
|
||||
|
||||
# Prepare for next octave
|
||||
frequency *= 2.0
|
||||
amplitude *= 0.5
|
||||
|
||||
# Apply phase shift (contrast adjustment)
|
||||
contrast = 1.0 + phase_shift
|
||||
result = result * contrast
|
||||
|
||||
# Ensure output is in valid [-1, 1] range
|
||||
result = torch.clamp(result, -1.0, 1.0)
|
||||
|
||||
return result
|
||||
|
||||
@classmethod
|
||||
def projection3d(cls, p, scale, warp_strength, phase_shift, octaves, time, device):
|
||||
"""
|
||||
3D projection noise implementation
|
||||
|
||||
Args:
|
||||
p: Coordinate tensor [batch, height, width, 2]
|
||||
scale: Scale factor
|
||||
warp_strength: Warp amount
|
||||
phase_shift: Contrast adjustment
|
||||
octaves: Number of detail levels
|
||||
time: Animation time
|
||||
device: Torch device
|
||||
|
||||
Returns:
|
||||
Noise tensor [batch, height, width, 1]
|
||||
"""
|
||||
batch, height, width, _ = p.shape
|
||||
|
||||
# Scale coordinates by user parameter
|
||||
p = p * scale
|
||||
|
||||
# Apply domain warping if warp strength > 0
|
||||
if warp_strength > 0.0:
|
||||
# Create a smooth flow field for warping
|
||||
warp_coords1 = p * 0.3 + torch.tensor([0.0, time * 0.05], device=device).reshape(1, 1, 1, 2)
|
||||
warp_coords2 = p * 0.3 + torch.tensor([time * 0.05, 0.0], device=device).reshape(1, 1, 1, 2)
|
||||
|
||||
# Generate noise for warping
|
||||
warp_noise1 = cls.simplex_noise(warp_coords1)
|
||||
warp_noise2 = cls.simplex_noise(warp_coords2, seed=13.5)
|
||||
|
||||
# Apply warp to coordinates
|
||||
p = p + torch.cat([warp_noise1, warp_noise2], dim=-1) * warp_strength
|
||||
|
||||
# Create 3D coordinates for projection
|
||||
z = torch.ones(batch, height, width, 1, device=device) * (time * 0.1)
|
||||
coords_3d = torch.cat([p, z], dim=-1)
|
||||
|
||||
# Generate multi-octave 3D noise
|
||||
result = torch.zeros(batch, height, width, 1, device=device)
|
||||
amplitude = 1.0
|
||||
frequency = 1.0
|
||||
|
||||
# Loop through octaves
|
||||
for i in range(min(int(octaves), 8)):
|
||||
# Scale and add phase offset for each octave
|
||||
current_coords = coords_3d * frequency
|
||||
value = cls.simplex_noise_3d(current_coords, seed=i * 13.5)
|
||||
|
||||
# Add to result with decreasing amplitude
|
||||
result += amplitude * value
|
||||
|
||||
# Prepare for next octave
|
||||
frequency *= 2.0
|
||||
amplitude *= 0.5
|
||||
|
||||
# Apply phase shift (contrast adjustment)
|
||||
contrast = 1.0 + phase_shift
|
||||
result = result * contrast
|
||||
|
||||
# Ensure output is in valid [-1, 1] range
|
||||
result = torch.clamp(result, -1.0, 1.0)
|
||||
|
||||
return result
|
||||
|
||||
@classmethod
|
||||
def simplex_noise(cls, coords, seed=0):
|
||||
"""
|
||||
Simplex-like noise implementation in PyTorch
|
||||
|
||||
Args:
|
||||
coords: Coordinate tensor [batch, height, width, 2]
|
||||
seed: Random seed value
|
||||
|
||||
Returns:
|
||||
Noise tensor [batch, height, width, 1]
|
||||
"""
|
||||
batch, height, width, _ = coords.shape
|
||||
device = coords.device
|
||||
|
||||
# Integer and fractional parts
|
||||
i0 = torch.floor(coords)
|
||||
i1 = i0 + 1.0
|
||||
f0 = coords - i0
|
||||
|
||||
# Dot products with random gradients
|
||||
def random_gradient(p, seed_val=seed):
|
||||
# Simple hash function
|
||||
h = p[:, :, :, 0] * 15.0 + p[:, :, :, 1] * 37.0 + seed_val
|
||||
h = torch.sin(h) * 43758.5453
|
||||
angle = h * 2.0 * math.pi
|
||||
return torch.stack([torch.cos(angle), torch.sin(angle)], dim=-1)
|
||||
|
||||
# Get gradients at corners
|
||||
g00 = random_gradient(i0)
|
||||
g10 = random_gradient(torch.cat([i1[:, :, :, 0:1], i0[:, :, :, 1:2]], dim=-1))
|
||||
g01 = random_gradient(torch.cat([i0[:, :, :, 0:1], i1[:, :, :, 1:2]], dim=-1))
|
||||
g11 = random_gradient(i1)
|
||||
|
||||
# Calculate contributions from each corner
|
||||
def contribution(grad, pos, point):
|
||||
# Calculate falloff
|
||||
t = 0.5 - torch.sum((point - pos) ** 2, dim=-1, keepdim=True)
|
||||
t = torch.maximum(t, torch.zeros_like(t))
|
||||
t = t * t * t * t # Quintic interpolation
|
||||
|
||||
# Dot product of gradient and offset
|
||||
offset = point - pos
|
||||
return t * torch.sum(grad * offset, dim=-1, keepdim=True)
|
||||
|
||||
# Get contributions
|
||||
n00 = contribution(g00, i0, coords)
|
||||
n10 = contribution(g10, torch.cat([i1[:, :, :, 0:1], i0[:, :, :, 1:2]], dim=-1), coords)
|
||||
n01 = contribution(g01, torch.cat([i0[:, :, :, 0:1], i1[:, :, :, 1:2]], dim=-1), coords)
|
||||
n11 = contribution(g11, i1, coords)
|
||||
|
||||
# Smoothstep for interpolation
|
||||
t = f0 * f0 * f0 * (f0 * (f0 * 6.0 - 15.0) + 10.0)
|
||||
|
||||
# Bilinear interpolation with smoothstep
|
||||
n_x0 = n00 * (1.0 - t[:, :, :, 0:1]) + n10 * t[:, :, :, 0:1]
|
||||
n_x1 = n01 * (1.0 - t[:, :, :, 0:1]) + n11 * t[:, :, :, 0:1]
|
||||
noise = n_x0 * (1.0 - t[:, :, :, 1:2]) + n_x1 * t[:, :, :, 1:2]
|
||||
|
||||
# Scale to match WebGL shader output range
|
||||
return 2.0 * noise
|
||||
|
||||
@classmethod
|
||||
def simplex_noise_3d(cls, coords, seed=0):
|
||||
"""
|
||||
Simplified 3D noise implementation
|
||||
This is a basic approximation for the 3D projection
|
||||
|
||||
Args:
|
||||
coords: 3D coordinate tensor [batch, height, width, 3]
|
||||
seed: Random seed value
|
||||
|
||||
Returns:
|
||||
Noise tensor [batch, height, width, 1]
|
||||
"""
|
||||
batch, height, width, _ = coords.shape
|
||||
device = coords.device
|
||||
|
||||
# Slice the 3D space in multiple 2D planes
|
||||
xy_coords = coords[:, :, :, 0:2]
|
||||
yz_coords = coords[:, :, :, 1:3]
|
||||
xz_coords = torch.cat([coords[:, :, :, 0:1], coords[:, :, :, 2:3]], dim=-1)
|
||||
|
||||
# Get 2D noise for each plane
|
||||
xy_noise = cls.simplex_noise(xy_coords, seed=seed)
|
||||
yz_noise = cls.simplex_noise(yz_coords, seed=seed+1)
|
||||
xz_noise = cls.simplex_noise(xz_coords, seed=seed+2)
|
||||
|
||||
# Combine the noises
|
||||
noise = (xy_noise + yz_noise + xz_noise) / 3.0
|
||||
|
||||
return noise
|
||||
|
||||
@classmethod
|
||||
def shader_noise_to_tensor_with_params(cls, batch_size=1, height=64, width=64, shader_type="tensor_field",
|
||||
shader_params=None, time=0.0, device="cuda", seed=0):
|
||||
"""
|
||||
Generate noise tensors using the full shader_params dictionary
|
||||
|
||||
Args:
|
||||
batch_size: Number of images in batch
|
||||
height: Height of tensor
|
||||
width: Width of tensor
|
||||
shader_type: Type of shader ("tensor_field", "heterogeneous_fbm", "projection_3d", "cellular", "fractal", "perlin", "waves", "gaussian")
|
||||
shader_params: Full dictionary of shader parameters from JSON
|
||||
time: Animation time
|
||||
device: Device to create tensor on
|
||||
seed: Random seed
|
||||
|
||||
Returns:
|
||||
Tensor with shape [batch_size, height, width, 1] in BHWC format
|
||||
"""
|
||||
# Fallback to empty dict if None
|
||||
if shader_params is None:
|
||||
shader_params = {}
|
||||
|
||||
# Extract common parameters with fallbacks
|
||||
scale = shader_params.get("shaderScale", shader_params.get("scale", 1.0))
|
||||
phase_shift = shader_params.get("shaderPhaseShift", shader_params.get("phase_shift", 0.0))
|
||||
warp_strength = shader_params.get("shaderWarpStrength", shader_params.get("warp_strength", 0.0))
|
||||
octaves = shader_params.get("shaderOctaves", shader_params.get("octaves", 3.0))
|
||||
visualization_type = shader_params.get("visualization_type", 3)
|
||||
|
||||
# Extract shape parameters for all noise types
|
||||
shape_type = shader_params.get("shaderShapeType", shader_params.get("shape_type", "none"))
|
||||
shape_strength = shader_params.get("shaderShapeStrength", shader_params.get("shapemaskstrength", shader_params.get("shape_strength", 1.0)))
|
||||
|
||||
# Ensure consistent parameter naming in shader_params
|
||||
shader_params["shaderShapeType"] = shape_type
|
||||
shader_params["shaderShapeStrength"] = shape_strength
|
||||
|
||||
# Extract temporal coherence parameters
|
||||
use_temporal_coherence = shader_params.get("useTemporalCoherence", shader_params.get("temporal_coherence", False))
|
||||
base_seed = shader_params.get("base_seed", seed)
|
||||
|
||||
# Make sure we have a proper time parameter for animation/temporal coherence
|
||||
if "time" not in shader_params or shader_params["time"] is None:
|
||||
shader_params["time"] = time
|
||||
else:
|
||||
# If time is already in shader_params, ensure it's used
|
||||
time = shader_params["time"]
|
||||
|
||||
# Extract color parameters for debugging
|
||||
color_scheme = shader_params.get("colorScheme", "none")
|
||||
color_intensity = shader_params.get("shaderColorIntensity", 0.8)
|
||||
|
||||
# Set deterministic seed for this operation
|
||||
# Use base_seed if temporal coherence is enabled
|
||||
effective_seed = base_seed if use_temporal_coherence else seed
|
||||
torch.manual_seed(effective_seed)
|
||||
|
||||
# Use the specialized methods for specific shader types
|
||||
if shader_type == "cellular" and hasattr(cls, 'cellular_noise_with_params'):
|
||||
noise_bhwc = cls.cellular_noise_with_params(batch_size, height, width, shader_params, time, device, effective_seed)
|
||||
elif shader_type == "domain_warp" and hasattr(cls, 'domain_warp_with_params'):
|
||||
noise_bhwc = cls.domain_warp_with_params(batch_size, height, width, shader_params, time, device, effective_seed)
|
||||
elif shader_type == "fractal" and hasattr(cls, 'fractal_noise_with_params'):
|
||||
noise_bhwc = cls.fractal_noise_with_params(batch_size, height, width, shader_params, time, device, effective_seed)
|
||||
elif shader_type == "perlin" and hasattr(cls, 'perlin_noise_with_params'):
|
||||
noise_bhwc = cls.perlin_noise_with_params(batch_size, height, width, shader_params, time, device, effective_seed)
|
||||
elif shader_type == "waves" and hasattr(cls, 'waves_noise_with_params'):
|
||||
noise_bhwc = cls.waves_noise_with_params(batch_size, height, width, shader_params, time, device, effective_seed)
|
||||
elif shader_type == "gaussian" and hasattr(cls, 'gaussian_noise_with_params'):
|
||||
noise_bhwc = cls.gaussian_noise_with_params(batch_size, height, width, shader_params, time, device, effective_seed)
|
||||
elif shader_type == "tensor_field" and hasattr(cls, 'tensor_field_with_params'):
|
||||
noise_bhwc = cls.tensor_field_with_params(batch_size, height, width, shader_params, time, device, effective_seed)
|
||||
elif shader_type == "heterogeneous_fbm" and hasattr(cls, 'heterogeneous_fbm_with_params'):
|
||||
noise_bhwc = cls.heterogeneous_fbm_with_params(batch_size, height, width, shader_params, time, device, effective_seed)
|
||||
elif shader_type == "interference_patterns" and hasattr(cls, 'interference_patterns_with_params'):
|
||||
noise_bhwc = cls.interference_patterns_with_params(batch_size, height, width, shader_params, time, device, effective_seed)
|
||||
elif (shader_type == "spectral" or shader_type == "spectral_noise") and hasattr(cls, 'spectral_noise_with_params'):
|
||||
noise_bhwc = cls.spectral_noise_with_params(batch_size, height, width, shader_params, time, device, effective_seed)
|
||||
elif (shader_type == "projection_3d" or shader_type == "3d_projection") and hasattr(cls, 'projection_3d_with_params'):
|
||||
noise_bhwc = cls.projection_3d_with_params(batch_size, height, width, shader_params, time, device, effective_seed)
|
||||
elif (shader_type == "curl" or shader_type == "curl_noise") and hasattr(cls, 'curl_noise_with_params'):
|
||||
noise_bhwc = cls.curl_noise_with_params(batch_size, height, width, shader_params, time, device, effective_seed)
|
||||
else:
|
||||
# Use the standard method for other shader types
|
||||
noise_bhwc = cls.shader_noise_to_tensor(
|
||||
batch_size=batch_size,
|
||||
height=height,
|
||||
width=width,
|
||||
shader_type=shader_type,
|
||||
visualization_type=visualization_type,
|
||||
scale=scale,
|
||||
phase_shift=phase_shift,
|
||||
warp_strength=warp_strength,
|
||||
time=time,
|
||||
device=device,
|
||||
seed=effective_seed,
|
||||
octaves=octaves
|
||||
)
|
||||
|
||||
# Convert from BHWC to BCHW for color processing
|
||||
noise_bchw = noise_bhwc.permute(0, 3, 1, 2)
|
||||
|
||||
# Expand channels if needed for latent space operations
|
||||
if noise_bchw.shape[1] == 1:
|
||||
# Expand to 9 channels for latent space operations instead of just 4
|
||||
noise_bchw = noise_bchw.expand(-1, 9, -1, -1)
|
||||
# If we have 4 channels but need 9 for latent operations
|
||||
elif noise_bchw.shape[1] == 4:
|
||||
# Get existing channels
|
||||
r, g, b, a = noise_bchw.chunk(4, dim=1)
|
||||
|
||||
# Create 5 more channels as variations of the existing ones
|
||||
c5 = (r + g) / 2.0 # Average of red and green
|
||||
c6 = (g + b) / 2.0 # Average of green and blue
|
||||
c7 = (b + a) / 2.0 # Average of blue and alpha
|
||||
c8 = (r + b) / 2.0 # Average of red and blue
|
||||
c9 = (r + g + b + a) / 4.0 # Average of all channels
|
||||
|
||||
# Combine all 9 channels
|
||||
noise_bchw = torch.cat([r, g, b, a, c5, c6, c7, c8, c9], dim=1)
|
||||
|
||||
# Apply color scheme if we have one
|
||||
if color_scheme != "none" and color_intensity > 0:
|
||||
# Import here to avoid circular import
|
||||
from .shader_params_reader import ShaderParamsReader
|
||||
|
||||
# Get stats before color application
|
||||
before_min = noise_bchw.min().item()
|
||||
before_max = noise_bchw.max().item()
|
||||
before_mean = noise_bchw.mean().item()
|
||||
before_std = noise_bchw.std().item()
|
||||
|
||||
# Apply color transformation
|
||||
noise_bchw = ShaderParamsReader.apply_color_scheme(noise_bchw, shader_params)
|
||||
|
||||
# Get stats after color application
|
||||
after_min = noise_bchw.min().item()
|
||||
after_max = noise_bchw.max().item()
|
||||
after_mean = noise_bchw.mean().item()
|
||||
after_std = noise_bchw.std().item()
|
||||
|
||||
# Check if the stats changed significantly (which would indicate the color was applied)
|
||||
if abs(before_std - after_std) < 0.001 and abs(before_mean - after_mean) < 0.001:
|
||||
pass
|
||||
else:
|
||||
pass
|
||||
|
||||
# Convert back to BHWC format for return
|
||||
noise_bhwc = noise_bchw.permute(0, 2, 3, 1)
|
||||
|
||||
return noise_bhwc
|
||||
|
||||
@classmethod
|
||||
def curl_noise_with_params(cls, batch_size, height, width, shader_params, time, device, seed):
|
||||
"""
|
||||
Method that directly takes the full shader_params dictionary
|
||||
|
||||
Args:
|
||||
batch_size: Number of images in batch
|
||||
height: Height of tensor
|
||||
width: Width of tensor
|
||||
shader_params: Full parameter dictionary from JSON
|
||||
time: Animation time
|
||||
device: Device to create tensor on
|
||||
seed: Random seed
|
||||
|
||||
Returns:
|
||||
Noise tensor [batch, height, width, 1]
|
||||
"""
|
||||
# Add time to shader params if not already present
|
||||
if "time" not in shader_params:
|
||||
shader_params["time"] = time
|
||||
|
||||
# Ensure shape parameters are properly extracted and included
|
||||
shape_type = shader_params.get("shaderShapeType", shader_params.get("shape_type", "none"))
|
||||
shape_mask_strength = shader_params.get("shaderShapeStrength", shader_params.get("shapemaskstrength", 1.0))
|
||||
|
||||
# Ensure phase shift is properly extracted and used with the correct parameter name
|
||||
phase_shift = shader_params.get("shaderPhaseShift", shader_params.get("phase_shift", 0.5))
|
||||
|
||||
# Extract temporal coherence parameters
|
||||
base_seed = shader_params.get("base_seed", seed) # Use provided seed as default base_seed
|
||||
use_temporal_coherence = shader_params.get("temporal_coherence", shader_params.get("useTemporalCoherence", False))
|
||||
|
||||
# Set deterministic seed for this operation
|
||||
effective_seed = base_seed if use_temporal_coherence else seed
|
||||
|
||||
# Make sure the correct parameter names are set for consistency
|
||||
shader_params["shaderShapeType"] = shape_type
|
||||
shader_params["shaderShapeStrength"] = shape_mask_strength
|
||||
shader_params["shaderPhaseShift"] = phase_shift # Ensure it's consistently named
|
||||
shader_params["base_seed"] = base_seed # Ensure base_seed is set
|
||||
shader_params["temporal_coherence"] = use_temporal_coherence # Ensure temporal_coherence is set
|
||||
|
||||
# Determine target channels for the generator call
|
||||
# Prioritize from shader_params, then ACEStep/ACE logic, then default.
|
||||
gen_target_channels = 4 # Default
|
||||
if "target_channels" in shader_params:
|
||||
gen_target_channels = int(shader_params["target_channels"])
|
||||
elif shader_params.get("inner_model_class") in ["ACEStep", "ACE"]:
|
||||
gen_target_channels = 8
|
||||
|
||||
# Use the CurlNoiseGenerator directly with the full params
|
||||
from .shaders.curl_noise import CurlNoiseGenerator
|
||||
curl_noise = CurlNoiseGenerator.get_curl_noise(
|
||||
batch_size=batch_size,
|
||||
height=height,
|
||||
width=width,
|
||||
shader_params=shader_params, # Pass the full parameter dictionary
|
||||
device=device,
|
||||
seed=effective_seed, # Use base_seed if temporal coherence is enabled
|
||||
target_channels=gen_target_channels # Explicitly pass determined target channels
|
||||
)
|
||||
|
||||
# Convert to BHWC format to match expected output
|
||||
result = curl_noise.permute(0, 2, 3, 1)[:, :, :, 0:1]
|
||||
|
||||
return result
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,562 @@
|
||||
import torch
|
||||
import math
|
||||
from ..shader_params_reader import ShaderParamsReader
|
||||
|
||||
class TemporalCoherentNoiseGenerator:
|
||||
"""
|
||||
Implementation of temporally coherent noise that maintains consistency
|
||||
between animation frames by treating time as a proper 4th dimension
|
||||
rather than using different seeds per frame.
|
||||
|
||||
This class generates noise that smoothly transitions between frames
|
||||
by implementing true 4D (x,y,z,time) noise functions.
|
||||
"""
|
||||
|
||||
@staticmethod
|
||||
def get_temporal_noise(batch_size, height, width, shader_params, device="cuda", base_seed=0):
|
||||
"""
|
||||
Generate temporally coherent noise tensor
|
||||
|
||||
Args:
|
||||
batch_size: Number of images in batch
|
||||
height: Height of tensor
|
||||
width: Width of tensor
|
||||
shader_params: Dictionary containing parameters from shader_params.json
|
||||
device: Device to create tensor on
|
||||
base_seed: Base seed value for initialization (NOT changed per frame)
|
||||
|
||||
Returns:
|
||||
Tensor with shape [batch_size, 4, height, width]
|
||||
"""
|
||||
# Extract parameters from shader_params
|
||||
scale = shader_params.get("shaderScale", shader_params.get("scale", 1.0))
|
||||
warp_strength = shader_params.get("shaderWarpStrength", shader_params.get("warp_strength", 0.5))
|
||||
phase_shift = shader_params.get("shaderPhaseShift", shader_params.get("phase_shift", 0.5))
|
||||
octaves = shader_params.get("shaderOctaves", shader_params.get("octaves", 3))
|
||||
time = shader_params.get("time", 0.0) # Critical for temporal coherence
|
||||
frequency_range = shader_params.get("frequencyRange", shader_params.get("frequency_range", 0))
|
||||
shape_type = shader_params.get("shaderShapeType", shader_params.get("shape_type", "none"))
|
||||
shape_mask_strength = shader_params.get("shaderShapeStrength", shader_params.get("shapemaskstrength", 1.0))
|
||||
|
||||
# Create coordinate grid (normalized to [-1, 1])
|
||||
# Important: make sure we create the grid with the correct orientation for height/width
|
||||
# In PyTorch, meshgrid with ij indexing puts height first, width second
|
||||
y, x = torch.meshgrid(
|
||||
torch.linspace(-1, 1, height, device=device),
|
||||
torch.linspace(-1, 1, width, device=device),
|
||||
indexing='ij'
|
||||
)
|
||||
|
||||
# Combine into coordinate tensor [batch, height, width, 2]
|
||||
p = torch.stack([x, y], dim=-1).unsqueeze(0).repeat(batch_size, 1, 1, 1)
|
||||
|
||||
# Verify p has the correct shape before proceeding
|
||||
if p.shape[1:3] != (height, width):
|
||||
# Try to fix by transposing or reshaping if needed
|
||||
if p.shape[1] == width and p.shape[2] == height:
|
||||
p = p.permute(0, 2, 1, 3)
|
||||
|
||||
# Set consistent seed for initialization
|
||||
torch.manual_seed(base_seed)
|
||||
|
||||
# Generate temporal coherent noise
|
||||
result = TemporalCoherentNoiseGenerator.temporal_spectral_noise(
|
||||
p, scale, warp_strength, phase_shift, octaves,
|
||||
frequency_range, time, device, base_seed
|
||||
)
|
||||
|
||||
# Apply shape mask if requested
|
||||
if shape_type not in ["none", "0"] and shape_mask_strength > 0:
|
||||
shape_mask = ShaderParamsReader.apply_shape_mask(p, shape_type)
|
||||
result = torch.lerp(result, result * shape_mask, shape_mask_strength)
|
||||
|
||||
# Ensure output is in valid [-1, 1] range
|
||||
result = torch.clamp(result, -1.0, 1.0)
|
||||
|
||||
# Convert from [batch, height, width, 1] to [batch, 4, height, width]
|
||||
result = result.permute(0, 3, 1, 2) # [batch, 1, height, width]
|
||||
result = result.expand(-1, 4, -1, -1) # [batch, 4, height, width]
|
||||
|
||||
# Final verification of output dimensions
|
||||
if result.shape[2:] != (height, width):
|
||||
pass # Added pass statement to fix linter error
|
||||
|
||||
return result
|
||||
|
||||
@staticmethod
|
||||
def temporal_spectral_noise(p, scale, warp_strength, phase_shift, octaves, frequency_range, time, device, base_seed):
|
||||
"""
|
||||
Generate spectral noise with true temporal coherence across frames
|
||||
|
||||
Args:
|
||||
p: Coordinate tensor [batch, height, width, 2]
|
||||
scale: Scale factor
|
||||
warp_strength: Amount of warping
|
||||
phase_shift: Phase/rotation adjustment
|
||||
octaves: Number of detail levels (1-8)
|
||||
frequency_range: Type of frequency filtering (1-4)
|
||||
time: Animation time parameter (critical for temporal coherence)
|
||||
device: Device to use for computation
|
||||
base_seed: Base random seed value (consistent across frames)
|
||||
|
||||
Returns:
|
||||
Noise tensor [batch, height, width, 1]
|
||||
"""
|
||||
batch, height, width, _ = p.shape
|
||||
|
||||
# Scale by user parameter
|
||||
p = p * scale
|
||||
|
||||
# Extend 2D coordinates to 3D by adding time as third dimension
|
||||
# This is the key to temporal coherence
|
||||
p_temporal = torch.cat([
|
||||
p,
|
||||
torch.ones_like(p[:, :, :, :1]) * time
|
||||
], dim=-1) # [batch, height, width, 3] where third dimension is time
|
||||
|
||||
# Apply 3D warp if enabled
|
||||
if warp_strength > 0.0:
|
||||
# Create a smooth 3D flow field for warping
|
||||
# Note: Using 3D noise for warping instead of 2D noise with time offset
|
||||
warp_p = p_temporal * 0.4 # Scale for warp coords
|
||||
|
||||
# Generate 3D noise for each component of the warp
|
||||
# Add seed dimension for 3D simplex noise
|
||||
warp_seed1 = torch.ones_like(warp_p[:, :, :, :1]) * base_seed
|
||||
warp_p_with_seed1 = torch.cat([warp_p, warp_seed1], dim=-1)
|
||||
|
||||
warp_seed2 = torch.ones_like(warp_p[:, :, :, :1]) * (base_seed + 1)
|
||||
warp_p_with_seed2 = torch.cat([warp_p + 5.0, warp_seed2], dim=-1)
|
||||
|
||||
# Get noise for x and y dimensions only
|
||||
warp_noise1 = TemporalCoherentNoiseGenerator.simplex_noise_3d(warp_p_with_seed1)
|
||||
warp_noise2 = TemporalCoherentNoiseGenerator.simplex_noise_3d(warp_p_with_seed2)
|
||||
|
||||
# Create warp vectors - shape [batch, height, width, 1]
|
||||
# Now properly expand to match p_temporal's shape
|
||||
zeros = torch.zeros_like(warp_noise1)
|
||||
|
||||
# Create spatial warp vector with explicit shape control
|
||||
# Make sure dimensions match exactly with p_temporal
|
||||
x_warp = warp_noise1 # [batch, height, width, 1]
|
||||
y_warp = warp_noise2 # [batch, height, width, 1]
|
||||
t_warp = zeros # [batch, height, width, 1] - no warp in time dimension
|
||||
|
||||
# Apply the warp to coordinates, component-wise
|
||||
# Make sure we're doing the addition with properly aligned tensors
|
||||
try:
|
||||
p_temporal_x = p_temporal[:, :, :, 0:1] + x_warp * warp_strength
|
||||
p_temporal_y = p_temporal[:, :, :, 1:2] + y_warp * warp_strength
|
||||
p_temporal_t = p_temporal[:, :, :, 2:3] # Time dimension unchanged
|
||||
|
||||
# Recombine the warped components
|
||||
p_temporal = torch.cat([p_temporal_x, p_temporal_y, p_temporal_t], dim=-1)
|
||||
except RuntimeError as e:
|
||||
# Alternative approach - reshape noise to match p_temporal dimensions
|
||||
# This ensures that height and width dimensions match exactly
|
||||
if x_warp.shape[1:3] != p_temporal.shape[1:3]:
|
||||
# Transpose the height and width dimensions if needed
|
||||
if (x_warp.shape[1] == p_temporal.shape[2] and
|
||||
x_warp.shape[2] == p_temporal.shape[1]):
|
||||
# Need to transpose dimensions 1 and 2
|
||||
x_warp = x_warp.permute(0, 2, 1, 3)
|
||||
y_warp = y_warp.permute(0, 2, 1, 3)
|
||||
t_warp = t_warp.permute(0, 2, 1, 3)
|
||||
else:
|
||||
# Try to resize the noise tensors to match
|
||||
target_height, target_width = p_temporal.shape[1:3]
|
||||
x_warp = torch.nn.functional.interpolate(
|
||||
x_warp.permute(0, 3, 1, 2), # [B, C, H, W]
|
||||
size=(target_height, target_width),
|
||||
mode='bilinear'
|
||||
).permute(0, 2, 3, 1) # Back to [B, H, W, C]
|
||||
|
||||
y_warp = torch.nn.functional.interpolate(
|
||||
y_warp.permute(0, 3, 1, 2),
|
||||
size=(target_height, target_width),
|
||||
mode='bilinear'
|
||||
).permute(0, 2, 3, 1)
|
||||
|
||||
t_warp = torch.zeros(batch, target_height, target_width, 1, device=device)
|
||||
|
||||
# Try again with the reshaped tensors
|
||||
p_temporal_x = p_temporal[:, :, :, 0:1] + x_warp * warp_strength
|
||||
p_temporal_y = p_temporal[:, :, :, 1:2] + y_warp * warp_strength
|
||||
p_temporal_t = p_temporal[:, :, :, 2:3] # Time dimension unchanged
|
||||
|
||||
# Recombine the warped components
|
||||
p_temporal = torch.cat([p_temporal_x, p_temporal_y, p_temporal_t], dim=-1)
|
||||
|
||||
# Calculate frequency domain coordinates
|
||||
uv = p_temporal[:, :, :, :2] # Use spatial dimensions for UV mapping
|
||||
freq = (p_temporal[:, :, :, :2] - 0.5) * 2.0
|
||||
|
||||
# Calculate radius and angle in frequency space
|
||||
radius = torch.sqrt(freq[:, :, :, 0]**2 + freq[:, :, :, 1]**2)
|
||||
angle = torch.atan2(freq[:, :, :, 1], freq[:, :, :, 0])
|
||||
|
||||
# Apply phase shift with time component for smooth rotation
|
||||
# This creates smooth transitions between frames
|
||||
angle = angle + (phase_shift + time * 0.1) * math.pi
|
||||
|
||||
# Recalculate frequency coordinates after rotation
|
||||
freq_rotated_x = radius * torch.cos(angle)
|
||||
freq_rotated_y = radius * torch.sin(angle)
|
||||
freq = torch.stack([freq_rotated_x, freq_rotated_y], dim=-1)
|
||||
|
||||
# Initialize filter based on frequency range
|
||||
filter_tensor = torch.ones_like(radius)
|
||||
|
||||
# Radial filtering based on frequency range (with time modulation)
|
||||
if frequency_range == 1: # Temporal low-pass
|
||||
cutoff = 0.25 + 0.05 * torch.sin(torch.tensor(time * 0.2, device=device))
|
||||
filter_tensor = torch.sigmoid((1.0 - radius - cutoff) * 10.0)
|
||||
elif frequency_range == 2: # Temporal band-pass
|
||||
center = 0.5 + 0.1 * torch.sin(torch.tensor(time * 0.3, device=device))
|
||||
width = 0.2 + 0.05 * torch.cos(torch.tensor(time * 0.25, device=device))
|
||||
low, high = center - width/2, center + width/2
|
||||
low_pass = torch.sigmoid((radius - low) * 10.0)
|
||||
high_pass = torch.sigmoid((high - radius) * 10.0)
|
||||
filter_tensor = low_pass * high_pass
|
||||
elif frequency_range == 3: # Temporal high-pass
|
||||
cutoff = 0.6 + 0.05 * torch.sin(torch.tensor(time * 0.15, device=device))
|
||||
filter_tensor = torch.sigmoid((radius - cutoff) * 10.0)
|
||||
elif frequency_range == 4: # Directional filter with temporal rotation
|
||||
# Directional filtering - rotates smoothly over time
|
||||
dir_strength = 0.8
|
||||
num_dir = 4 # Number of directions/lobes
|
||||
angle_mod = (angle + time * 0.2) % (2.0 * math.pi)
|
||||
dir_filter = 0.5 + 0.5 * torch.cos(torch.tensor(float(num_dir), device=device) * angle_mod)
|
||||
filter_tensor = torch.lerp(torch.ones_like(dir_filter), dir_filter, dir_strength)
|
||||
|
||||
# Generate base noise with different frequency components using 3D noise
|
||||
noise = torch.zeros_like(radius).unsqueeze(-1)
|
||||
|
||||
# Determine number of octaves based on user control
|
||||
max_octaves = min(int(octaves), 8)
|
||||
|
||||
# Sum multiple octaves with different frequencies
|
||||
for i in range(max_octaves):
|
||||
# Scale frequency based on octave
|
||||
freq_scale = 2.0 ** i
|
||||
|
||||
# Scale amplitude based on frequency (1/f noise)
|
||||
amp = 1.0 / freq_scale
|
||||
|
||||
# Use 3D simplex noise to get temporally coherent noise
|
||||
# We add a different phase offset per octave
|
||||
octave_p = p_temporal * freq_scale + torch.tensor([0.0, 0.0, i * 1.5], device=device)
|
||||
|
||||
# Get 3D noise value
|
||||
noise_val = TemporalCoherentNoiseGenerator.simplex_noise_3d(
|
||||
torch.cat([octave_p, torch.ones_like(octave_p[:, :, :, :1]) * (base_seed + i)], dim=-1)
|
||||
)
|
||||
|
||||
# Apply frequency-based filtering
|
||||
freq_factor = i / max(max_octaves - 1, 1) # 0-1 range for frequency
|
||||
freq_filter = torch.ones_like(noise_val)
|
||||
|
||||
if frequency_range == 1: # Low-pass
|
||||
freq_filter = 1.0 - freq_factor
|
||||
elif frequency_range == 2: # Band-pass
|
||||
freq_filter = 1.0 - torch.abs(torch.tensor(freq_factor, device=device) - 0.5) * 2.0
|
||||
elif frequency_range == 3: # High-pass
|
||||
freq_filter = freq_factor
|
||||
|
||||
noise = noise + (noise_val * amp * freq_filter)
|
||||
|
||||
# Apply filter and normalize
|
||||
noise = noise * filter_tensor.unsqueeze(-1)
|
||||
|
||||
# Add smooth temporal modulations
|
||||
time_factor = torch.sin(torch.tensor(time * 0.3, device=device))
|
||||
noise = noise * (1.0 + 0.1 * time_factor)
|
||||
|
||||
# Normalize to [-1, 1] range
|
||||
noise = torch.clamp(noise * 1.5, -1.0, 1.0)
|
||||
|
||||
return noise
|
||||
|
||||
@staticmethod
|
||||
def simplex_noise_3d(coords):
|
||||
"""
|
||||
3D simplex noise implementation in PyTorch
|
||||
This is the key to temporal coherence as it treats time as a full dimension
|
||||
|
||||
Args:
|
||||
coords: Coordinate tensor [batch, height, width, 4]
|
||||
where dims are [x, y, time, seed]
|
||||
|
||||
Returns:
|
||||
Noise tensor [batch, height, width, 1]
|
||||
"""
|
||||
batch, height, width, _ = coords.shape
|
||||
device = coords.device
|
||||
|
||||
# Extract x, y, z (time), and seed components
|
||||
x, y, z, seed = coords[:, :, :, 0], coords[:, :, :, 1], coords[:, :, :, 2], coords[:, :, :, 3]
|
||||
|
||||
# 3D Simplex noise constants
|
||||
F3 = 1.0 / 3.0
|
||||
G3 = 1.0 / 6.0
|
||||
|
||||
# Skew input space to determine which simplex cell contains point
|
||||
s = (x + y + z) * F3
|
||||
i = torch.floor(x + s)
|
||||
j = torch.floor(y + s)
|
||||
k = torch.floor(z + s)
|
||||
|
||||
t = (i + j + k) * G3
|
||||
X0 = i - t
|
||||
Y0 = j - t
|
||||
Z0 = k - t
|
||||
|
||||
# Unskewed distances from cell origin
|
||||
x0 = x - X0
|
||||
y0 = y - Y0
|
||||
z0 = z - Z0
|
||||
|
||||
# Determine which simplex we are in
|
||||
# Offsets for second corner of simplex
|
||||
i1 = torch.zeros_like(x0)
|
||||
j1 = torch.zeros_like(y0)
|
||||
k1 = torch.zeros_like(z0)
|
||||
|
||||
# Offsets for third corner of simplex
|
||||
i2 = torch.zeros_like(x0)
|
||||
j2 = torch.zeros_like(y0)
|
||||
k2 = torch.zeros_like(z0)
|
||||
|
||||
# This logic is hard to vectorize, but possible
|
||||
# Using temporary masks to determine the simplex case
|
||||
x_ge_y = (x0 >= y0).float()
|
||||
y_ge_z = (y0 >= z0).float()
|
||||
x_ge_z = (x0 >= z0).float()
|
||||
|
||||
# i1
|
||||
i1 = x_ge_y * x_ge_z
|
||||
# j1
|
||||
j1 = (1 - x_ge_y) * y_ge_z
|
||||
# k1
|
||||
k1 = (1 - x_ge_z) * (1 - y_ge_z)
|
||||
|
||||
# i2
|
||||
i2 = x_ge_y + (1 - x_ge_y) * x_ge_z
|
||||
# j2
|
||||
j2 = x_ge_y * (1 - x_ge_z) + (1 - x_ge_y)
|
||||
# k2
|
||||
k2 = (1 - x_ge_z) + x_ge_z * (1 - x_ge_y)
|
||||
|
||||
# Calculate noise contributions from each corner
|
||||
# 3D noise contributions from simplex corners
|
||||
noise = torch.zeros_like(x0)
|
||||
|
||||
# Corner 1 - origin of simplex
|
||||
t0 = 0.6 - x0*x0 - y0*y0 - z0*z0
|
||||
mask0 = (t0 >= 0).float()
|
||||
t0 = t0 * t0
|
||||
n0 = mask0 * t0 * t0 * TemporalCoherentNoiseGenerator.grad3d(i, j, k, x0, y0, z0, seed)
|
||||
|
||||
# Corner 2
|
||||
x1 = x0 - i1 + G3
|
||||
y1 = y0 - j1 + G3
|
||||
z1 = z0 - k1 + G3
|
||||
t1 = 0.6 - x1*x1 - y1*y1 - z1*z1
|
||||
mask1 = (t1 >= 0).float()
|
||||
t1 = t1 * t1
|
||||
n1 = mask1 * t1 * t1 * TemporalCoherentNoiseGenerator.grad3d(i + i1, j + j1, k + k1, x1, y1, z1, seed)
|
||||
|
||||
# Corner 3
|
||||
x2 = x0 - i2 + 2.0 * G3
|
||||
y2 = y0 - j2 + 2.0 * G3
|
||||
z2 = z0 - k2 + 2.0 * G3
|
||||
t2 = 0.6 - x2*x2 - y2*y2 - z2*z2
|
||||
mask2 = (t2 >= 0).float()
|
||||
t2 = t2 * t2
|
||||
n2 = mask2 * t2 * t2 * TemporalCoherentNoiseGenerator.grad3d(i + i2, j + j2, k + k2, x2, y2, z2, seed)
|
||||
|
||||
# Corner 4 (last corner of simplex)
|
||||
x3 = x0 - 1.0 + 3.0 * G3
|
||||
y3 = y0 - 1.0 + 3.0 * G3
|
||||
z3 = z0 - 1.0 + 3.0 * G3
|
||||
t3 = 0.6 - x3*x3 - y3*y3 - z3*z3
|
||||
mask3 = (t3 >= 0).float()
|
||||
t3 = t3 * t3
|
||||
n3 = mask3 * t3 * t3 * TemporalCoherentNoiseGenerator.grad3d(i + 1, j + 1, k + 1, x3, y3, z3, seed)
|
||||
|
||||
# Sum up noise contributions
|
||||
# Scale to stay within [-1,1]
|
||||
result = (n0 + n1 + n2 + n3) * 32.0
|
||||
|
||||
# Ensure we maintain the input height/width dimensions
|
||||
if result.shape[1:3] != (height, width):
|
||||
# Reshape to match the input dimensions
|
||||
result = result.reshape(batch, height, width).unsqueeze(-1)
|
||||
|
||||
return result.unsqueeze(-1) if len(result.shape) == 3 else result
|
||||
|
||||
@staticmethod
|
||||
def grad3d(ix, iy, iz, x, y, z, seed):
|
||||
"""
|
||||
3D gradient function for simplex noise
|
||||
Computes the dot product of a random gradient vector and a distance vector
|
||||
|
||||
Args:
|
||||
ix, iy, iz: Grid cell coordinates
|
||||
x, y, z: Position relative to grid cell
|
||||
seed: Seed value
|
||||
|
||||
Returns:
|
||||
Dot product of gradient and distance vectors
|
||||
"""
|
||||
# Simple hash for deterministic, repeatable patterns
|
||||
# Convert to integers before applying bitwise operations
|
||||
# Need to use mod instead of bitwise & for floats
|
||||
ix_int = torch.floor(ix).to(torch.int32)
|
||||
iy_int = torch.floor(iy).to(torch.int32)
|
||||
iz_int = torch.floor(iz).to(torch.int32)
|
||||
seed_int = torch.floor(seed).to(torch.int32)
|
||||
|
||||
# Compute hash (now with integer tensors)
|
||||
h = (ix_int * 1619 + iy_int * 31337 + iz_int * 6971 + seed_int * 1013) & 15
|
||||
|
||||
# Convert back to original dtype for the rest of the calculations
|
||||
h = h.to(x.dtype)
|
||||
|
||||
# Convert lower 4 bits of hash to 12 gradient directions
|
||||
u = torch.where(h < 8, x, y)
|
||||
v = torch.where(h < 4, y, torch.where((h == 12) | (h == 14), x, z))
|
||||
|
||||
# Convert hash to 12 gradient directions
|
||||
# 1 or -1 based on hash bits - avoiding bitwise operations on floats
|
||||
# The & operation is replaced with modulo division which is safer for floats
|
||||
h_mod_2 = torch.remainder(h, 2)
|
||||
h_mod_4 = torch.remainder(h, 4)
|
||||
|
||||
u_sign = (h_mod_2 == 0).float() * 2.0 - 1.0
|
||||
v_sign = (h_mod_4 < 2).float() * 2.0 - 1.0
|
||||
|
||||
return u_sign * u + v_sign * v
|
||||
|
||||
def add_temporal_noise_to_shader_params(shader_params, frame_number, base_seed, fps=24, duration=5):
|
||||
"""
|
||||
Prepare shader parameters for temporal coherent noise
|
||||
|
||||
Args:
|
||||
shader_params: Original shader parameters dictionary
|
||||
frame_number: Current frame number
|
||||
base_seed: Base seed for the entire animation
|
||||
fps: Frames per second
|
||||
duration: Animation duration in seconds
|
||||
|
||||
Returns:
|
||||
Modified shader parameters with temporal settings
|
||||
"""
|
||||
# Create a copy to avoid modifying the original
|
||||
params = shader_params.copy()
|
||||
|
||||
# Calculate time value based on frame number (0.0 to 1.0 over duration)
|
||||
time = (frame_number / (fps * duration)) % 1.0
|
||||
|
||||
# Store time parameter for the shader
|
||||
params["time"] = time
|
||||
|
||||
# Store the base seed (constant across frames)
|
||||
params["base_seed"] = base_seed
|
||||
|
||||
# Add subtle phase progression for temporal interest
|
||||
# This creates a smooth progression that's continuous
|
||||
base_phase = params.get("shaderPhaseShift", 0.5)
|
||||
frame_phase = base_phase + time * 0.4 # Subtle phase change over time
|
||||
params["shaderPhaseShift"] = frame_phase
|
||||
|
||||
return params
|
||||
|
||||
def generate_temporal_coherent_noise_tensor(shader_params, height, width, batch_size=1, device="cuda", seed=0):
|
||||
"""
|
||||
Generate temporal coherent noise tensor directly using shader_params
|
||||
|
||||
Args:
|
||||
shader_params: Dictionary containing parameters from shader_params.json
|
||||
height: Height of tensor
|
||||
width: Width of tensor
|
||||
batch_size: Number of images in batch
|
||||
device: Device to create tensor on
|
||||
seed: Random seed to ensure consistent noise with same seed
|
||||
|
||||
Returns:
|
||||
Noise tensor with shape [batch_size, 4, height, width]
|
||||
"""
|
||||
# Extract temporal coherence parameters if present
|
||||
time = shader_params.get("time", 0.0)
|
||||
base_seed = shader_params.get("base_seed", seed)
|
||||
use_temporal_coherence = shader_params.get("useTemporalCoherence", True) # Default is True for this type
|
||||
|
||||
# Set deterministic seed for this operation
|
||||
torch.manual_seed(base_seed if use_temporal_coherence else seed)
|
||||
|
||||
# Generate temporal coherent noise using the TemporalCoherentNoiseGenerator
|
||||
noise_bhwc = TemporalCoherentNoiseGenerator.get_temporal_noise(
|
||||
batch_size=batch_size,
|
||||
height=height,
|
||||
width=width,
|
||||
shader_params=shader_params,
|
||||
device=device,
|
||||
base_seed=base_seed if use_temporal_coherence else seed
|
||||
)
|
||||
|
||||
# Reset random seed state to not affect other operations
|
||||
torch.manual_seed(torch.seed())
|
||||
|
||||
# Ensure we have the right format - should already be [batch, 4, height, width]
|
||||
if noise_bhwc.shape[1] == 1:
|
||||
noise_bhwc = noise_bhwc.expand(-1, 4, -1, -1)
|
||||
|
||||
# Normalize the noise to match typical noise distribution
|
||||
noise_bhwc = (noise_bhwc - noise_bhwc.mean()) / (noise_bhwc.std() + 1e-8)
|
||||
|
||||
return noise_bhwc
|
||||
|
||||
def integrate_temporal_coherent_noise():
|
||||
"""
|
||||
Function to integrate temporal coherent noise into shader_to_tensor.py
|
||||
|
||||
This adds the temporal_coherent_noise method to the ShaderToTensor class
|
||||
"""
|
||||
from ..shader_to_tensor import ShaderToTensor
|
||||
|
||||
def temporal_coherent_noise(cls, batch_size, height, width, shader_params, time, device, base_seed):
|
||||
"""
|
||||
Generate temporally coherent noise for animation frames
|
||||
|
||||
Args:
|
||||
batch_size: Number of images in batch
|
||||
height: Height of tensor
|
||||
width: Width of tensor
|
||||
shader_params: Dictionary containing parameters
|
||||
time: Animation time
|
||||
device: Device to create tensor on
|
||||
base_seed: Base seed value (consistent across animation)
|
||||
|
||||
Returns:
|
||||
Noise tensor [batch_size, 4, height, width]
|
||||
"""
|
||||
# Update shader parameters with time value
|
||||
updated_params = shader_params.copy()
|
||||
updated_params["time"] = time
|
||||
|
||||
# Call the temporal coherent noise generator
|
||||
noise = TemporalCoherentNoiseGenerator.get_temporal_noise(
|
||||
batch_size, height, width, updated_params, device, base_seed
|
||||
)
|
||||
|
||||
return noise
|
||||
|
||||
# Add the method to ShaderToTensor class
|
||||
setattr(ShaderToTensor, 'temporal_coherent_noise', classmethod(temporal_coherent_noise))
|
||||
|
||||
# Register in shader type handlers
|
||||
if hasattr(ShaderToTensor, 'shader_handlers') and isinstance(ShaderToTensor.shader_handlers, dict):
|
||||
ShaderToTensor.shader_handlers['temporal_coherent'] = ShaderToTensor.temporal_coherent_noise
|
||||
|
||||
return True
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,245 @@
|
||||
// gradient_title.js - Adds a custom gradient title to ShaderNoiseKSampler node
|
||||
|
||||
import { app } from "../../scripts/app.js";
|
||||
|
||||
// Cache for rendering optimization
|
||||
const CACHE = {
|
||||
titleCanvas: null,
|
||||
titleCtx: null,
|
||||
lastWidth: 0,
|
||||
lastHeight: 0,
|
||||
lastTime: 0,
|
||||
frameCount: 0,
|
||||
frameSkip: 2, // Only update animation every X frames
|
||||
collapsed: {
|
||||
canvas: null,
|
||||
ctx: null,
|
||||
lastWidth: 0
|
||||
}
|
||||
};
|
||||
|
||||
// Register a callback to run when ComfyUI is fully loaded
|
||||
app.registerExtension({
|
||||
name: "ShaderNoiseKSampler.GradientTitle",
|
||||
async beforeRegisterNodeDef(nodeType, nodeData, app) {
|
||||
// Apply to both shader noise ksampler nodes
|
||||
if (nodeData.name === "ShaderNoiseKSampler" || nodeData.name === "ShaderNoiseKSamplerDirect") {
|
||||
// Store the original onDrawForeground function if it exists
|
||||
const origOnDrawForeground = nodeType.prototype.onDrawForeground;
|
||||
|
||||
// Add our own onDrawForeground function
|
||||
nodeType.prototype.onDrawForeground = function(ctx) {
|
||||
// Call the original onDrawForeground if it exists
|
||||
if (origOnDrawForeground) {
|
||||
origOnDrawForeground.apply(this, arguments);
|
||||
}
|
||||
|
||||
// Draw a custom gradient title
|
||||
drawGradientTitle(this, ctx);
|
||||
};
|
||||
|
||||
// Clean up resources when node is removed
|
||||
const origOnRemoved = nodeType.prototype.onRemoved;
|
||||
nodeType.prototype.onRemoved = function() {
|
||||
if (origOnRemoved) {
|
||||
origOnRemoved.apply(this, arguments);
|
||||
}
|
||||
|
||||
// Clear cached canvases to prevent memory leaks
|
||||
CACHE.titleCanvas = null;
|
||||
CACHE.titleCtx = null;
|
||||
CACHE.collapsed.canvas = null;
|
||||
CACHE.collapsed.ctx = null;
|
||||
};
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
/**
|
||||
* Draws a gradient title directly on the canvas
|
||||
* @param {LGraphNode} node - The node to apply the gradient to
|
||||
* @param {CanvasRenderingContext2D} ctx - The canvas context
|
||||
*/
|
||||
function drawGradientTitle(node, ctx) {
|
||||
// Get title area dimensions
|
||||
const titleHeight = node.flags.collapsed ? 20 : 30; // Smaller height when collapsed
|
||||
const width = node.flags.collapsed ? 190 : node.size[0]; // Smaller width when collapsed
|
||||
const fullHeight = node.size[1]; // Get actual node height
|
||||
const equationY = 45; // Y position for the equation, moved lower
|
||||
|
||||
// Choose appropriate equation based on node type
|
||||
let equation, collapsedEquation;
|
||||
|
||||
if (node.type === "ShaderNoiseKSamplerDirect") {
|
||||
equation = "Lt = Sα(N) ∘ Kβ(t) ⟿";
|
||||
collapsedEquation = "Lt = Sα(N) ∘ Kβ(t) ⟿";
|
||||
} else {
|
||||
equation = "Lt = Sα(N) ∘ Kβ(t)";
|
||||
collapsedEquation = "Lt = Sα(N) ∘ Kβ(t)";
|
||||
}
|
||||
|
||||
// Update animation frame counter (performance optimization)
|
||||
CACHE.frameCount = (CACHE.frameCount + 1) % (CACHE.frameSkip + 1);
|
||||
const shouldUpdateAnimation = CACHE.frameCount === 0;
|
||||
|
||||
// Save current state
|
||||
ctx.save();
|
||||
|
||||
// Reset shadow properties for gradient drawing
|
||||
ctx.shadowColor = "transparent";
|
||||
ctx.shadowBlur = 0;
|
||||
ctx.shadowOffsetX = 0;
|
||||
ctx.shadowOffsetY = 0;
|
||||
|
||||
// Create vertical black gradient for entire background that fills node height
|
||||
const gradient = ctx.createLinearGradient(0, 0, 0, fullHeight);
|
||||
gradient.addColorStop(0, "#000000"); // Pure black at top
|
||||
gradient.addColorStop(0.2, "#101010"); // Transition to very dark gray
|
||||
gradient.addColorStop(1, "#101010"); // Very dark gray at bottom
|
||||
|
||||
// Create smooth shimmer effect for mathematical formula - only calculate if animation should update
|
||||
let shimmerPosition = 0.5; // Default middle position
|
||||
if (shouldUpdateAnimation) {
|
||||
const time = Date.now() / 3000; // Faster time factor (changed from 4000)
|
||||
shimmerPosition = (Math.sin(time) + 1) / 2; // Changed from cos to sin for left-to-right only
|
||||
// Store for later use if needed
|
||||
CACHE.lastTime = time;
|
||||
} else {
|
||||
// Reuse last calculation for animation frames we're skipping
|
||||
const time = CACHE.lastTime || Date.now() / 3000;
|
||||
shimmerPosition = (Math.sin(time) + 1) / 2;
|
||||
}
|
||||
|
||||
// Add collapse button handler
|
||||
if (node.flags.collapsed) {
|
||||
// If node is collapsed, adjust the title rendering
|
||||
ctx.fillStyle = gradient;
|
||||
ctx.fillRect(0, 0, width, titleHeight);
|
||||
|
||||
// Draw etched shadow for collapsed version
|
||||
ctx.fillStyle = "rgba(0,0,0,0.3)";
|
||||
ctx.font = "italic 11px Arial"; // Smaller font for collapsed state
|
||||
ctx.textAlign = "center";
|
||||
ctx.textBaseline = "middle";
|
||||
ctx.fillText(collapsedEquation, width / 2 + 1, titleHeight / 2 + 1);
|
||||
|
||||
// Create base golden gradient for collapsed version
|
||||
const baseGradient = ctx.createLinearGradient(0, titleHeight/2 - 5, 0, titleHeight/2 + 5);
|
||||
baseGradient.addColorStop(0, "#B8860B"); // Darker gold
|
||||
baseGradient.addColorStop(0.5, "#FFD700"); // Bright gold
|
||||
baseGradient.addColorStop(1, "#B8860B"); // Darker gold
|
||||
|
||||
// Draw base golden text
|
||||
ctx.fillStyle = baseGradient;
|
||||
ctx.font = "italic 11px Arial";
|
||||
ctx.textAlign = "center";
|
||||
ctx.textBaseline = "middle";
|
||||
ctx.fillText(collapsedEquation, width / 2, titleHeight / 2);
|
||||
|
||||
// Create moving highlight effect for collapsed version
|
||||
const highlightWidth = width * 0.4; // Width of the highlight
|
||||
const highlightX = -highlightWidth + (width + highlightWidth) * shimmerPosition; // Adjusted range
|
||||
|
||||
const shimmerGradient = ctx.createLinearGradient(
|
||||
highlightX - highlightWidth/2, 0,
|
||||
highlightX + highlightWidth/2, 0
|
||||
);
|
||||
|
||||
// Create smooth highlight transition
|
||||
shimmerGradient.addColorStop(0, "rgba(255, 255, 200, 0)");
|
||||
shimmerGradient.addColorStop(0.1, "rgba(255, 255, 200, 0)");
|
||||
shimmerGradient.addColorStop(0.5, "rgba(255, 255, 200, 0.3)");
|
||||
shimmerGradient.addColorStop(0.9, "rgba(255, 255, 200, 0)");
|
||||
shimmerGradient.addColorStop(1, "rgba(255, 255, 200, 0)");
|
||||
|
||||
// Apply highlight
|
||||
ctx.fillStyle = shimmerGradient;
|
||||
ctx.fillText(collapsedEquation, width / 2, titleHeight / 2);
|
||||
|
||||
// Add outline glow that follows the highlight
|
||||
const glowIntensity = Math.max(0, 1 - Math.abs(width/2 - highlightX)/(width/4));
|
||||
ctx.shadowColor = `rgba(255, 255, 200, ${glowIntensity * 0.3})`;
|
||||
ctx.shadowBlur = 4; // Less blur for collapsed version
|
||||
ctx.shadowOffsetX = 0;
|
||||
ctx.shadowOffsetY = 0;
|
||||
ctx.fillText(collapsedEquation, width / 2, titleHeight / 2);
|
||||
|
||||
// Skip the rest of the rendering when collapsed
|
||||
ctx.restore();
|
||||
return;
|
||||
}
|
||||
|
||||
// Draw background that fills the entire node
|
||||
ctx.fillStyle = gradient;
|
||||
|
||||
// Use rounded rectangle for the background with rounded corners at the bottom
|
||||
if (!node.flags.collapsed) {
|
||||
const cornerRadius = 8; // Adjust radius as needed
|
||||
|
||||
// Create path for rounded rectangle
|
||||
ctx.beginPath();
|
||||
ctx.moveTo(0, 0); // Top-left corner (no rounding)
|
||||
ctx.lineTo(width, 0); // Top-right corner (no rounding)
|
||||
ctx.lineTo(width, fullHeight - cornerRadius); // Right edge before bottom-right corner
|
||||
ctx.arcTo(width, fullHeight, width - cornerRadius, fullHeight, cornerRadius); // Bottom-right rounded corner
|
||||
ctx.lineTo(cornerRadius, fullHeight); // Bottom edge before bottom-left corner
|
||||
ctx.arcTo(0, fullHeight, 0, fullHeight - cornerRadius, cornerRadius); // Bottom-left rounded corner
|
||||
ctx.lineTo(0, 0); // Left edge back to top
|
||||
ctx.closePath();
|
||||
ctx.fill();
|
||||
} else {
|
||||
// Keep regular rectangle for collapsed state
|
||||
ctx.fillRect(0, 0, width, fullHeight); // No extra padding
|
||||
}
|
||||
|
||||
// Draw etched shadow
|
||||
ctx.fillStyle = "rgba(0,0,0,0.3)";
|
||||
ctx.font = "italic 14px Arial"; // Smaller font for equation
|
||||
ctx.textAlign = "center";
|
||||
ctx.textBaseline = "middle";
|
||||
ctx.fillText(equation, width / 2 + 2, equationY + 2);
|
||||
|
||||
// Create base golden gradient
|
||||
const baseGradient = ctx.createLinearGradient(0, equationY - 7, 0, equationY + 7);
|
||||
baseGradient.addColorStop(0, "#B8860B"); // Darker gold
|
||||
baseGradient.addColorStop(0.5, "#FFD700"); // Bright gold
|
||||
baseGradient.addColorStop(1, "#B8860B"); // Darker gold
|
||||
|
||||
// Draw base golden text
|
||||
ctx.fillStyle = baseGradient;
|
||||
ctx.font = "italic 14px Arial"; // Smaller font for equation
|
||||
ctx.textAlign = "center";
|
||||
ctx.textBaseline = "middle";
|
||||
ctx.fillText(equation, width / 2, equationY);
|
||||
|
||||
// Create moving highlight effect
|
||||
const highlightWidth = width * 0.4; // Width of the highlight
|
||||
const highlightX = -highlightWidth + (width + highlightWidth) * shimmerPosition; // Adjusted range
|
||||
|
||||
const shimmerGradient = ctx.createLinearGradient(
|
||||
highlightX - highlightWidth/2, 0,
|
||||
highlightX + highlightWidth/2, 0
|
||||
);
|
||||
|
||||
// Create smooth highlight transition
|
||||
shimmerGradient.addColorStop(0, "rgba(255, 255, 200, 0)");
|
||||
shimmerGradient.addColorStop(0.1, "rgba(255, 255, 200, 0)");
|
||||
shimmerGradient.addColorStop(0.5, "rgba(255, 255, 200, 0.3)");
|
||||
shimmerGradient.addColorStop(0.9, "rgba(255, 255, 200, 0)");
|
||||
shimmerGradient.addColorStop(1, "rgba(255, 255, 200, 0)");
|
||||
|
||||
// Apply highlight
|
||||
ctx.fillStyle = shimmerGradient;
|
||||
ctx.fillText(equation, width / 2, equationY);
|
||||
|
||||
// Add outline glow that follows the highlight
|
||||
const glowIntensity = Math.max(0, 1 - Math.abs(width/2 - highlightX)/(width/4));
|
||||
ctx.shadowColor = `rgba(255, 255, 200, ${glowIntensity * 0.3})`;
|
||||
ctx.shadowBlur = 8;
|
||||
ctx.shadowOffsetX = 0;
|
||||
ctx.shadowOffsetY = 0;
|
||||
ctx.fillText(equation, width / 2, equationY);
|
||||
|
||||
// Restore context state
|
||||
ctx.restore();
|
||||
}
|
||||
@@ -0,0 +1,13 @@
|
||||
<svg width="241" height="194" viewBox="0 0 241 194" fill="none" xmlns="http://www.w3.org/2000/svg">
|
||||
<mask id="mask0_1_219" style="mask-type:luminance" maskUnits="userSpaceOnUse" x="-1" y="0" width="242" height="194">
|
||||
<path d="M240.469 0.958984H-0.00585938V193.918H240.469V0.958984Z" fill="white"/>
|
||||
</mask>
|
||||
<g mask="url(#mask0_1_219)">
|
||||
<path d="M96.1344 193.911C61.1312 193.911 32.6597 178.256 15.9721 149.829C1.19788 124.912 -0.00585938 97.9229 -0.00585938 67.7662C-0.00585938 49.8876 5.37293 34.3215 15.5413 22.7466C24.8861 12.1157 38.1271 5.22907 52.8317 3.35378C70.2858 1.14271 91.9848 0.958984 114.545 0.958984C151.259 0.958984 161.63 1.4088 176.075 2.85328C195.29 4.76026 211.458 11.932 222.824 23.5955C234.368 35.4428 240.469 51.2624 240.469 69.3627V72.9994C240.469 103.885 219.821 129.733 191.046 136.759C188.898 141.827 186.237 146.871 183.089 151.837L183.006 151.964C172.869 167.632 149.042 193.918 103.401 193.918H96.1281L96.1344 193.911Z" fill="white"/>
|
||||
<path d="M174.568 17.9772C160.927 16.6151 151.38 16.1589 114.552 16.1589C90.908 16.1589 70.9008 16.387 54.7644 18.4334C33.3949 21.164 15.2058 37.5285 15.2058 67.7674C15.2058 98.0066 16.796 121.422 29.0741 142.107C42.9425 165.751 66.1302 178.707 96.1412 178.707H103.414C140.242 178.707 160.25 159.156 170.253 143.698C174.574 136.874 177.754 130.058 179.801 123.234C205.947 120.96 225.27 99.3624 225.27 72.9941V69.3577C225.27 40.9432 206.631 21.164 174.574 17.9772H174.568Z" fill="white"/>
|
||||
<path d="M15.1975 67.7674C15.1975 37.5285 33.3866 21.164 54.7559 18.4334C70.8987 16.387 90.906 16.1589 114.544 16.1589C151.372 16.1589 160.919 16.6151 174.559 17.9772C206.617 21.1576 225.255 40.937 225.255 69.3577V72.9941C225.255 99.3687 205.932 120.966 179.786 123.234C177.74 130.058 174.559 136.874 170.238 143.698C160.235 159.156 140.228 178.707 103.4 178.707H96.1264C66.1155 178.707 42.9277 165.751 29.0595 142.107C16.7814 121.422 15.1912 98.4563 15.1912 67.7674" fill="#202020"/>
|
||||
<path d="M32.2469 67.9899C32.2469 97.3168 34.0654 116.184 43.6127 133.689C54.5225 153.924 74.3018 161.653 96.8117 161.653H103.857C133.411 161.653 147.736 147.329 155.693 134.829C159.558 128.462 162.966 121.417 164.784 112.547L166.147 106.864H174.332C192.521 106.864 208.208 92.09 208.208 73.2166V69.8082C208.208 48.6669 195.024 37.5228 172.058 34.7987C159.102 33.6646 151.372 33.2084 114.538 33.2084C89.7602 33.2084 72.0272 33.4364 58.6152 35.4828C39.7483 38.2134 32.2407 48.8951 32.2407 67.9899" fill="white"/>
|
||||
<path d="M166.158 83.6801C166.158 86.4107 168.204 88.4572 171.841 88.4572C183.435 88.4572 189.802 81.8619 189.802 70.9523C189.802 60.0427 183.435 53.2195 171.841 53.2195C168.204 53.2195 166.158 55.2657 166.158 57.9963V83.6866V83.6801Z" fill="#202020"/>
|
||||
<path d="M54.5321 82.3198C54.5321 95.732 62.0332 107.326 71.5807 116.424C77.9478 122.562 87.9515 128.93 94.7685 133.022C96.8147 134.157 98.8611 134.841 101.136 134.841C103.866 134.841 106.134 134.157 107.959 133.022C114.782 128.93 124.779 122.562 130.919 116.424C140.694 107.332 148.195 95.7383 148.195 82.3198C148.195 67.7673 137.286 54.8115 121.599 54.8115C112.28 54.8115 105.912 59.5882 101.136 66.1772C96.8147 59.582 90.2259 54.8115 80.9001 54.8115C64.9855 54.8115 54.5256 67.7673 54.5256 82.3198" fill="#FF5A16"/>
|
||||
</g>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 3.2 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 24 KiB |
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,420 @@
|
||||
// Add a button to manually save shader parameters to file
|
||||
import { app } from "../../scripts/app.js";
|
||||
|
||||
app.registerExtension({
|
||||
name: "ShaderParamsSaveButton",
|
||||
async beforeRegisterNodeDef(nodeType, nodeData, app) {
|
||||
if (nodeData.name === "ShaderNoiseKSampler") {
|
||||
// Store original onNodeCreated to maintain the node's behavior
|
||||
const origOnNodeCreated = nodeType.prototype.onNodeCreated;
|
||||
|
||||
// Add our button to the node
|
||||
nodeType.prototype.onNodeCreated = function() {
|
||||
// Call original function to preserve existing behavior
|
||||
if (origOnNodeCreated) {
|
||||
origOnNodeCreated.apply(this, arguments);
|
||||
}
|
||||
|
||||
// Keep track of the node instance
|
||||
const node = this;
|
||||
|
||||
// Track if we're currently saving (to prevent double-clicks)
|
||||
let isSaving = false;
|
||||
|
||||
// Add indicator widget that will show when parameters need saving
|
||||
// Create a custom widget object instead of using the built-in text widget
|
||||
const indicatorWidget = {
|
||||
name: "⚠️ Parameters with 🔄 must be saved ⚠️",
|
||||
type: "custom_indicator",
|
||||
value: "",
|
||||
options: { className: "shader-params-indicator" },
|
||||
tooltip: "parameters marked with 🔄 require saving to take effect in the generation process",
|
||||
disabled: true,
|
||||
// Add computeSize method to properly handle resizing
|
||||
computeSize: function() {
|
||||
// Return fixed height but variable width based on parent node width
|
||||
if (this.parent && this.parent.size) {
|
||||
// Adjust width to match node width with some padding
|
||||
return [this.parent.size[0] - 30, 20];
|
||||
}
|
||||
return [220, 28]; // Default size if parent not available
|
||||
},
|
||||
// Custom draw method for the widget
|
||||
draw: function(ctx, node, widget_width, y, widget_height) {
|
||||
if (!ctx) return;
|
||||
|
||||
// Draw background
|
||||
ctx.fillStyle = "rgba(255, 119, 0, 0.1)";
|
||||
ctx.strokeStyle = "#ff7700";
|
||||
ctx.lineWidth = 1;
|
||||
|
||||
// Draw rounded rectangle for the widget background
|
||||
const radius = 4;
|
||||
const x = 15; // Padding from left edge
|
||||
const width = widget_width - 30; // Subtract padding from both sides
|
||||
|
||||
ctx.beginPath();
|
||||
ctx.moveTo(x + radius, y);
|
||||
ctx.lineTo(x + width - radius, y);
|
||||
ctx.quadraticCurveTo(x + width, y, x + width, y + radius);
|
||||
ctx.lineTo(x + width, y + widget_height - radius);
|
||||
ctx.quadraticCurveTo(x + width, y + widget_height, x + width - radius, y + widget_height);
|
||||
ctx.lineTo(x + radius, y + widget_height);
|
||||
ctx.quadraticCurveTo(x, y + widget_height, x, y + widget_height - radius);
|
||||
ctx.lineTo(x, y + radius);
|
||||
ctx.quadraticCurveTo(x, y, x + radius, y);
|
||||
ctx.closePath();
|
||||
|
||||
ctx.fill();
|
||||
ctx.stroke();
|
||||
|
||||
// Draw text
|
||||
ctx.fillStyle = "#ff7700";
|
||||
ctx.font = "bold 12px Arial";
|
||||
ctx.textAlign = "center";
|
||||
ctx.fillText(this.name, x + width / 2, y + widget_height / 2 + 4);
|
||||
|
||||
return widget_height;
|
||||
}
|
||||
};
|
||||
|
||||
// Add widget reference to node
|
||||
indicatorWidget.parent = this;
|
||||
this.widgets.push(indicatorWidget);
|
||||
|
||||
// --- Refactored Save Function ---
|
||||
const saveParameters = () => {
|
||||
if (isSaving) return;
|
||||
isSaving = true;
|
||||
|
||||
// Change button text to indicate saving
|
||||
if (saveButtonWidget) saveButtonWidget.name = "Saving...";
|
||||
console.log("Starting save process (triggered)...");
|
||||
|
||||
// Print each widget and its properties for deep debugging
|
||||
if (node.widgets) {
|
||||
console.log("Widget details:");
|
||||
node.widgets.forEach((widget, index) => {
|
||||
console.log(`Widget ${index}: name="${widget.name}", type=${widget.type}, value=${widget.value}`);
|
||||
});
|
||||
}
|
||||
|
||||
// Find the widgets that contain our values
|
||||
// Default values
|
||||
let shaderScale = 1.0;
|
||||
let shaderOctaves = 1;
|
||||
let shaderWarpStrength = 0.5;
|
||||
let shaderShapeStrength = 1.0;
|
||||
let shaderPhaseShift = 0.5;
|
||||
let shaderColorIntensity = 0.8;
|
||||
let shaderType = "tensor_field";
|
||||
let shaderShapeType = "none";
|
||||
let colorScheme = "none";
|
||||
|
||||
// Try to find widgets by partial name match
|
||||
if (node.widgets) {
|
||||
for (const widget of node.widgets) {
|
||||
// Ensure widget and widget.name exist before accessing
|
||||
if (!widget || typeof widget.name !== 'string') continue;
|
||||
|
||||
const name = widget.name.toLowerCase();
|
||||
|
||||
// Use partial name matching for more flexibility
|
||||
if (name.includes("scale") && !name.includes("color")) {
|
||||
shaderScale = widget.value;
|
||||
console.log(`Found shader scale: ${widget.value}`);
|
||||
}
|
||||
else if (name.includes("octaves")) {
|
||||
shaderOctaves = widget.value;
|
||||
console.log(`Found octaves: ${widget.value}`);
|
||||
}
|
||||
else if (name.includes("warp")) {
|
||||
shaderWarpStrength = widget.value;
|
||||
console.log(`Found warp strength: ${widget.value}`);
|
||||
}
|
||||
else if (name.includes("shape") && name.includes("strength")) {
|
||||
shaderShapeStrength = widget.value;
|
||||
console.log(`Found shape strength: ${widget.value}`);
|
||||
}
|
||||
else if (name.includes("phase")) {
|
||||
shaderPhaseShift = widget.value;
|
||||
console.log(`Found phase: ${widget.value}`);
|
||||
}
|
||||
else if (name.includes("color") && name.includes("intensity")) {
|
||||
shaderColorIntensity = widget.value;
|
||||
console.log(`Found color intensity: ${widget.value}`);
|
||||
}
|
||||
else if (name.includes("shader") && name.includes("type")) {
|
||||
shaderType = widget.value;
|
||||
console.log(`Found shader type: ${widget.value}`);
|
||||
}
|
||||
else if (name.includes("shape") && name.includes("type")) {
|
||||
shaderShapeType = widget.value;
|
||||
console.log(`Found shape type: ${widget.value}`);
|
||||
}
|
||||
else if (name.includes("color") && name.includes("scheme")) {
|
||||
colorScheme = widget.value;
|
||||
console.log(`Found color scheme: ${widget.value}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Create the properties object with the gathered values
|
||||
const currentProps = {
|
||||
shaderType: shaderType,
|
||||
shaderScale: shaderScale,
|
||||
shaderOctaves: shaderOctaves,
|
||||
shaderWarpStrength: shaderWarpStrength,
|
||||
shaderShapeType: shaderShapeType,
|
||||
shaderShapeStrength: shaderShapeStrength,
|
||||
shaderPhaseShift: shaderPhaseShift,
|
||||
colorScheme: colorScheme,
|
||||
shaderColorIntensity: shaderColorIntensity
|
||||
};
|
||||
|
||||
console.log("Saving shader properties:", currentProps);
|
||||
|
||||
try {
|
||||
// Serialize with pretty printing
|
||||
const jsonData = JSON.stringify(currentProps, null, 2);
|
||||
|
||||
// Save to localStorage as well for redundancy
|
||||
try {
|
||||
localStorage.setItem('shader_params', jsonData);
|
||||
console.log("Saved to localStorage successfully");
|
||||
} catch (localErr) {
|
||||
console.error("Failed to save to localStorage:", localErr);
|
||||
}
|
||||
|
||||
// Send the file directly to the data directory - most reliable method
|
||||
// Create a simple download link to trigger the file save
|
||||
const blob = new Blob([jsonData], {type: 'application/json'});
|
||||
const url = URL.createObjectURL(blob);
|
||||
const a = document.createElement('a');
|
||||
a.href = url;
|
||||
a.download = 'shader_params.json';
|
||||
document.body.appendChild(a);
|
||||
a.click();
|
||||
|
||||
// Show instructions in console AFTER download is triggered
|
||||
console.log("%cIMPORTANT: Please save the downloaded file to the following location (overwrite if exists):", "color: red; font-weight: bold");
|
||||
console.log("%ccustom_nodes/ComfyUI-ShaderNoiseKsampler/data/shader_params.json", "color: blue; font-weight: bold");
|
||||
|
||||
// Clean up and update button state on success
|
||||
setTimeout(() => {
|
||||
try {
|
||||
document.body.removeChild(a);
|
||||
URL.revokeObjectURL(url);
|
||||
console.log("Shader params JSON download link removed.");
|
||||
} catch (cleanupError) {
|
||||
console.warn("Could not clean up download link:", cleanupError);
|
||||
}
|
||||
|
||||
// if (saveButtonWidget) saveButtonWidget.name = "✗ Parameters Not Saved";
|
||||
setTimeout(() => {
|
||||
if (saveButtonWidget) saveButtonWidget.name = "💾 Save Shader Parameters";
|
||||
isSaving = false;
|
||||
}, 2000);
|
||||
}, 500);
|
||||
|
||||
} catch (error) {
|
||||
console.error("Error saving shader parameters:", error);
|
||||
if (saveButtonWidget) saveButtonWidget.name = "Error Saving!";
|
||||
setTimeout(() => {
|
||||
if (saveButtonWidget) saveButtonWidget.name = "💾 Save Shader Parameters";
|
||||
isSaving = false;
|
||||
}, 3000);
|
||||
}
|
||||
}; // --- End of Refactored Save Function ---
|
||||
|
||||
// Add save button using the refactored function
|
||||
const saveButtonWidget = this.addWidget("button", " 💾 Save Shader Parameters", null, saveParameters);
|
||||
|
||||
// Extend the widget with a tooltip property that ComfyUI's system recognizes
|
||||
saveButtonWidget.options = saveButtonWidget.options || {};
|
||||
saveButtonWidget.options.className = "save-shader-params";
|
||||
|
||||
// Add tooltip following ComfyUI's pattern
|
||||
saveButtonWidget.tooltip = "Save parameters (Alt+S) -- file must be named shader_params.json -- always overwrite old file -- [Save location: `custom_nodes/ComfyUI-ShaderNoiseKsampler/data/shader_params.json`][WIP]";
|
||||
|
||||
// Move both the indicator and save button widgets to the end of all widgets
|
||||
// This ensures they appear at the bottom of all settings
|
||||
setTimeout(() => {
|
||||
if (this.widgets && this.widgets.length > 0) {
|
||||
// Get the indicators's current index
|
||||
const indicatorIndex = this.widgets.indexOf(indicatorWidget);
|
||||
if (indicatorIndex !== -1) {
|
||||
// Remove it from its current position
|
||||
this.widgets.splice(indicatorIndex, 1);
|
||||
}
|
||||
|
||||
// Get the save button's current index
|
||||
const buttonIndex = this.widgets.indexOf(saveButtonWidget);
|
||||
if (buttonIndex !== -1) {
|
||||
// Remove it from its current position
|
||||
this.widgets.splice(buttonIndex, 1);
|
||||
}
|
||||
|
||||
// Add them back at the end, indicator first then save button
|
||||
this.widgets.push(indicatorWidget);
|
||||
this.widgets.push(saveButtonWidget);
|
||||
|
||||
// Ensure the node is redrawn to show the updated widget positions
|
||||
this.setDirtyCanvas(true, true);
|
||||
}
|
||||
}, 100);
|
||||
|
||||
// --- Keybinding Logic ---
|
||||
const handleKeyDown = (event) => {
|
||||
// Check for Alt+S and if this node is currently selected
|
||||
if (event.altKey && event.key === 's') {
|
||||
// Check if the graph canvas and selected nodes exist
|
||||
if (app.canvas && app.canvas.current_node) {
|
||||
// Check if the currently selected node is this node
|
||||
if (app.canvas.current_node === node) {
|
||||
console.log("Alt+S detected for selected ShaderNoiseKSampler node.");
|
||||
event.preventDefault(); // Prevent browser's default Alt+S action
|
||||
event.stopPropagation(); // Stop event from bubbling up
|
||||
saveParameters(); // Trigger the save function
|
||||
}
|
||||
} else if (app.canvas && app.canvas.selected_nodes && Object.keys(app.canvas.selected_nodes).length === 1 && app.canvas.selected_nodes[node.id]) {
|
||||
// Fallback check for selected_nodes if current_node isn't reliable
|
||||
console.log("Alt+S detected for selected ShaderNoiseKSampler node (using selected_nodes).");
|
||||
event.preventDefault();
|
||||
event.stopPropagation();
|
||||
saveParameters();
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
// Attach the event listener to the document
|
||||
document.addEventListener('keydown', handleKeyDown);
|
||||
|
||||
// Store handler reference for removal
|
||||
this.handleKeyDown = handleKeyDown;
|
||||
|
||||
// Original onRemoved method if it exists
|
||||
const origOnRemoved = this.onRemoved;
|
||||
|
||||
// Add logic to remove the event listener when the node is removed
|
||||
this.onRemoved = function() {
|
||||
console.log("Removing keydown listener for node:", this.id);
|
||||
document.removeEventListener('keydown', this.handleKeyDown);
|
||||
|
||||
// Call original onRemoved if it existed
|
||||
if (origOnRemoved) {
|
||||
origOnRemoved.apply(this, arguments);
|
||||
}
|
||||
};
|
||||
// --- End Keybinding Logic ---
|
||||
};
|
||||
}
|
||||
},
|
||||
// Add CSS styling for the button
|
||||
async setup(app) {
|
||||
// Add a small CSS rule for spacing and tooltip styling
|
||||
const style = document.createElement("style");
|
||||
style.textContent = `
|
||||
.save-shader-params {
|
||||
padding: 6px;
|
||||
background-color: #5c5c5c;
|
||||
color: white;
|
||||
border-radius: 4px;
|
||||
cursor: pointer;
|
||||
}
|
||||
|
||||
.shader-params-indicator {
|
||||
padding: 4px;
|
||||
font-weight: bold;
|
||||
color: #ff7700;
|
||||
background-color: rgba(255, 119, 0, 0.1);
|
||||
border-left: 3px solid #ff7700;
|
||||
border-radius: 2px;
|
||||
text-align: center;
|
||||
pointer-events: none;
|
||||
user-select: none;
|
||||
}
|
||||
|
||||
/* The tooltip container needs positioning */
|
||||
.tooltip-container {
|
||||
position: relative;
|
||||
display: inline-block;
|
||||
}
|
||||
|
||||
/* Style for ComfyUI-compatible tooltips */
|
||||
.comfy-tooltip {
|
||||
visibility: hidden;
|
||||
background-color: rgba(40, 40, 40, 0.95);
|
||||
color: #fff;
|
||||
text-align: center;
|
||||
padding: 8px;
|
||||
border-radius: 6px;
|
||||
position: absolute;
|
||||
z-index: 1000;
|
||||
bottom: 125%;
|
||||
left: 50%;
|
||||
transform: translateX(-50%);
|
||||
white-space: nowrap;
|
||||
font-size: 12px;
|
||||
box-shadow: 0 4px 6px rgba(0, 0, 0, 0.3);
|
||||
}
|
||||
|
||||
/* Show tooltip on hover */
|
||||
.tooltip-container:hover .comfy-tooltip {
|
||||
visibility: visible;
|
||||
}
|
||||
`;
|
||||
document.head.appendChild(style);
|
||||
|
||||
// Hook into the app's widget drawing system to add tooltip support
|
||||
// This is done after initial setup to ensure proper integration
|
||||
const originalDrawNodeWidgets = LGraphCanvas.prototype.drawNodeWidgets;
|
||||
if (originalDrawNodeWidgets) {
|
||||
LGraphCanvas.prototype.drawNodeWidgets = function(node, pos, ctx, active_widget) {
|
||||
// Call the original method first
|
||||
const result = originalDrawNodeWidgets.call(this, node, pos, ctx, active_widget);
|
||||
|
||||
// After rendering widgets, check for our tooltip property
|
||||
if (node && node.widgets) {
|
||||
for (const widget of node.widgets) {
|
||||
// If widget has our tooltip property and mouse is over it
|
||||
if (widget === active_widget && widget.tooltip) {
|
||||
// Get canvas position
|
||||
const rect = this.canvas.getBoundingClientRect();
|
||||
|
||||
// Create tooltip if it doesn't exist yet
|
||||
if (!widget._tooltip_elem) {
|
||||
const tooltipContainer = document.createElement('div');
|
||||
tooltipContainer.className = 'tooltip-container';
|
||||
|
||||
const tooltip = document.createElement('span');
|
||||
tooltip.className = 'comfy-tooltip';
|
||||
tooltip.textContent = widget.tooltip;
|
||||
|
||||
tooltipContainer.appendChild(tooltip);
|
||||
document.body.appendChild(tooltipContainer);
|
||||
|
||||
widget._tooltip_elem = tooltipContainer;
|
||||
}
|
||||
|
||||
// Position tooltip
|
||||
if (widget._tooltip_elem) {
|
||||
const x = pos[0] + rect.left;
|
||||
const y = pos[1] + rect.top;
|
||||
|
||||
widget._tooltip_elem.style.left = x + 'px';
|
||||
widget._tooltip_elem.style.top = y - 20 + 'px';
|
||||
widget._tooltip_elem.style.display = 'block';
|
||||
}
|
||||
} else if (widget._tooltip_elem) {
|
||||
// Hide tooltip when not hovering
|
||||
widget._tooltip_elem.style.display = 'none';
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return result;
|
||||
};
|
||||
}
|
||||
}
|
||||
});
|
||||
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user