136 lines
5.4 KiB
Markdown
136 lines
5.4 KiB
Markdown
<a id="readme-top"></a>
|
|
|
|
<div align="center">
|
|
<h1 align="center">ComfyUI-Chroma-RoPE</h1>
|
|
|
|
<p align="center">
|
|
Advanced Rotary Position Embedding (RoPE) modifications for Chroma/FLUX models
|
|
<br />
|
|
. Enable ultra-high-resolution image generation with YaRN-like modifications.
|
|
<br />
|
|
</p>
|
|
</div>
|
|
|
|
## About The Project
|
|
|
|
**ComfyUI-Chroma-RoPE** is a ComfyUI custom node that implements advanced RoPE (Rotary Position Embedding) modifications for Chroma and FLUX-based diffusion models. It enables generating images at resolutions far beyond the model's native training scale without additional training.
|
|
|
|
### Key Features
|
|
|
|
- **YaRN (Yet another RoPE extensioN)**: Extrapolate position encodings to handle longer sequences/resolutions
|
|
- **p-RoPE (Proportional RoPE)**: Selectively truncate low-frequency RoPE dimensions for semantic coherence (Kinda not done right)
|
|
- **DyPE**: Timestep-dependent frequency modulation that adapts to the diffusion process
|
|
- **Multiple Ramp Functions**: Linear, sigmoid, power-2, and square root blending options
|
|
- **Timestep Modulation**: Dynamic theta scaling based on diffusion noise levels
|
|
- **Zero Training Required**: Works with pre-trained models out of the box
|
|
|
|
This implementation is based on concepts from the original [ComfyUI-DyPE](https://github.com/wildminder/ComfyUI-DyPE) repository and extends it with additional RoPE modification methods.
|
|
|
|
## Installation
|
|
|
|
### Via ComfyUI Manager (Recommended)
|
|
|
|
1. Open ComfyUI Manager in your ComfyUI interface
|
|
2. Click "Install Custom Nodes"
|
|
3. Search for `ComfyUI-Chroma-RoPE`
|
|
4. Click Install
|
|
|
|
### Manual Installation
|
|
|
|
1. Navigate to your `ComfyUI/custom_nodes/` directory:
|
|
```bash
|
|
cd ComfyUI/custom_nodes/
|
|
```
|
|
|
|
2. Clone this repository:
|
|
```bash
|
|
git clone https://github.com/Clybius/ComfyUI-Chroma-RoPE.git
|
|
```
|
|
|
|
3. Restart ComfyUI
|
|
|
|
## Usage
|
|
|
|
1. **Load your Chroma/FLUX model** using a standard model loader
|
|
2. **Add the Chroma RoPE Patch node** (found under `model_patches/unet`)
|
|
3. **Connect the model** output from your loader to the `model` input
|
|
4. **Configure parameters** based on your target resolution
|
|
5. **Connect to KSampler** and generate
|
|
|
|
### Node Parameters
|
|
|
|
| Parameter | Type | Default | Description |
|
|
|-----------|------|---------|-------------|
|
|
| `model` | Model | Required | The Chroma/FLUX model to patch |
|
|
| `method` | Combo | `yarn_freq_stretch` | Position encoding method: `yarn`, `yarn_freq_stretch`, `yarn+dynamic_ntk`, `dynamic_ntk`, `ntk`, `base` |
|
|
| `rope_percentage` | Float | 1.0 | p-RoPE proportion (0.0-1.0). 1.0=full RoPE, 0.0=NoPE (semantic only) |
|
|
| `dype` | Boolean | True | Enable DyPE (Dynamic Position Extrapolation) with timestep modulation |
|
|
| `max_pe_length` | Int | 64 | Original trained position encoding length |
|
|
| `yarn_ramp_type` | Combo | `sqrt` | Frequency blending function: `linear`, `sigmoid`, `pow2`, `sqrt` |
|
|
| `yarn_ratio` | Float | 1.0 | YaRN scaling ratio multiplier |
|
|
| `yarn_beta_fast` | Int | 32 | High-frequency rotation cutoff |
|
|
| `yarn_beta_slow` | Int | 2 | Low-frequency rotation cutoff |
|
|
| `timestep_modulation` | Boolean | False | Enable timestep-dependent theta scaling |
|
|
| `timestep_period_min` | Float | 1000.0 | Theta period at max noise (t=1.0) |
|
|
| `timestep_period_max` | Float | 10000.0 | Theta period at min noise (t=0.0) |
|
|
| `attn_ratio` | Float | 1.0 | Attention scaling factor for RoPE embeddings |
|
|
|
|
## Position Encoding Methods
|
|
|
|
### YaRN (yarn)
|
|
The default and recommended method. Combines interpolation and extrapolation with frequency-aware blending for smooth high-resolution generation.
|
|
|
|
### YaRN Frequency Stretch (yarn_freq_stretch)
|
|
Novel method that non-linearly stretches the frequency space, preserving high-frequency relationships while extrapolating low frequencies.
|
|
|
|
### YaRN + Dynamic NTK (yarn+dynamic_ntk)
|
|
Combines YaRN with Dynamic NTK scaling for aggressive extrapolation scenarios.
|
|
|
|
### Dynamic NTK (dynamic_ntk)
|
|
Dynamic scaling based on sequence length ratio. More stable for moderate extrapolation.
|
|
|
|
### NTK (ntk)
|
|
Standard NTK-aware scaling with fixed extrapolation factor.
|
|
|
|
### Base (base)
|
|
No extrapolation. Uses original model position encodings.
|
|
|
|
## Ramp Functions
|
|
|
|
The ramp function controls how interpolated and extrapolated frequencies blend:
|
|
|
|
- **Linear**: Smooth linear interpolation between regions
|
|
- **Sigmoid**: S-curve transition for sharper boundaries
|
|
- **Pow2**: Aggressive blending
|
|
- **Sqrt**: Conservative blending (gentler transitions)
|
|
|
|
## Compatibility
|
|
|
|
- **Models**: Chroma, FLUX-based architectures
|
|
- **ComfyUI**: Compatible with standard ComfyUI workflows
|
|
- **Other Nodes**: Works alongside quantization, attention optimization, and other model patches
|
|
|
|
**Not compatible with:** SD 1.5, SDXL, or non-FLUX architectures.
|
|
|
|
## Known Limitations
|
|
|
|
- FLUX/Chroma architectures only
|
|
- Parameter tuning required for optimal results at different resolutions
|
|
- Higher resolutions may require more sampling steps for best quality
|
|
|
|
## Credits
|
|
|
|
- Original [ComfyUI-DyPE](https://github.com/wildminder/ComfyUI-DyPE) repository by wildminder
|
|
- YaRN paper and implementation concepts
|
|
- The ComfyUI team for the extensible platform
|
|
|
|
## License
|
|
|
|
This project is released under the Apache 2.0 License. See LICENSE file for details.
|
|
|
|
## Acknowledgments
|
|
|
|
This project builds upon the work from the DyPE research and the original ComfyUI-DyPE implementation. Special thanks to the diffusion model research community for advancing position encoding techniques.
|
|
|
|
<p align="right">(<a href="#readme-top">back to top</a>)</p>
|