203 lines
7.6 KiB
Markdown
203 lines
7.6 KiB
Markdown
# ComfyUI-LCS
|
||
|
||
Training-free color control via the **Latent Color Subspace**.
|
||
|
||
> **Note:** This is an unofficial community implementation. For the official code, see [ExplainableML/LCS](https://github.com/ExplainableML/LCS).
|
||
|
||
Based on ["The Latent Color Subspace"](https://arxiv.org/abs/2603.12261v1) (ICML 2026): color in diffusion model latent patch spaces lives in a **3D subspace** (PCA captures 100% color variance), while the remaining 61 dimensions encode structure and detail orthogonally.
|
||
|
||
This plugin steers colors directly in the 3D LCS during diffusion sampling — no training, no LoRA, no post-processing.
|
||
|
||
> [中文版 README](README_zh.md)
|
||
|
||
## LCS vs Traditional Post-Processing
|
||
|
||
LCS operates **during** diffusion sampling, not after — this is the key difference from traditional color grading (Photoshop, filters, etc.).
|
||
|
||
| | Traditional Post-Processing | LCS |
|
||
|---|---|---|
|
||
| **When** | After VAE decode, in pixel space | During sampling, in latent space |
|
||
| **Mechanism** | Color filter on the final image | Modifies 3D color subspace mid-generation |
|
||
| **Model awareness** | None — structure already locked | Model adapts to color shifts in subsequent steps |
|
||
| **Result** | Colors can look "painted on" | Colors look naturally intended by the model |
|
||
|
||
For example: to get a warm orange sunset, post-processing tints everything orange (muddying shadows and skin tones), while LCS nudges the color subspace early in sampling so clouds, lighting, and reflections are *coherently* warm.
|
||
|
||
The core insight: color and structure are **orthogonal** in the latent patch space — you can steer one without disturbing the other.
|
||
|
||
## Tested Models
|
||
|
||
| Model | Status |
|
||
|-------|--------|
|
||
| FLUX | Tested |
|
||
| z-image | Tested |
|
||
| z-image-turbo | Tested |
|
||
|
||
LCS calibrates per-VAE, so it should work with any model using a compatible VAE. Feel free to report results with other models.
|
||
|
||
## Features
|
||
|
||
- **Color Steering** — Push colors toward any target color
|
||
- **Batch Multi-Color** — Different colors per batch item
|
||
- **Tone Adjustment** — Contrast, brightness, saturation, temperature with one-click presets
|
||
- **Localized Control** — Optional mask for region-specific changes
|
||
- **Latent Color Preview** — Visualize color structure without VAE decoding
|
||
- **Step Observer** — Per-step color previews for debugging
|
||
|
||
## Installation
|
||
|
||
```bash
|
||
cd ComfyUI/custom_nodes
|
||
git clone https://github.com/facok/ComfyUI-LCS.git
|
||
```
|
||
|
||
Dependencies (usually already present in ComfyUI):
|
||
|
||
```bash
|
||
pip install einops safetensors
|
||
```
|
||
|
||
## Quick Start
|
||
|
||
### Basic Color Control
|
||
|
||
```
|
||
LCS Load Data → LCS Color Intervene → KSampler
|
||
↑
|
||
(pick a color)
|
||
```
|
||
|
||
1. **LCS Load Data** — connect your VAE (auto-calibrates on first run)
|
||
2. **LCS Color Intervene** — connect MODEL and LCS_DATA, pick a target color
|
||
3. Connect the output MODEL to KSampler
|
||
|
||
### Tone Adjustment
|
||
|
||
```
|
||
LCS Load Data → LCS Tone Adjust → KSampler
|
||
```
|
||
|
||
1. **LCS Load Data** → **LCS Tone Adjust**
|
||
2. Select a preset (e.g., "Cinematic") or adjust sliders manually
|
||
|
||
|
||
|
||
### Multi-Color Batch
|
||
|
||
```
|
||
LCS Load Data → LCS Color Batch → KSampler
|
||
↓
|
||
batch_size → EmptyLatentImage
|
||
```
|
||
|
||
Enter comma-separated hex colors (e.g., `#FF0000,#00FF00,#0000FF`). Each color applies to one batch item.
|
||
|
||
## Nodes
|
||
|
||
### Calibration
|
||
|
||
| Node | Description |
|
||
|------|-------------|
|
||
| **LCS Load Data** | Auto-calibrate and cache LCS data per-VAE. Fingerprints VAE weights for automatic cache management — just connect your VAE. |
|
||
|
||
Calibration runs once per VAE and caches automatically. Subsequent runs load instantly.
|
||
|
||
### Intervention
|
||
|
||
| Node | Description |
|
||
|------|-------------|
|
||
| **LCS Color Intervene** | Steer colors toward a target. Supports Type I (LCS shift), Type II (HSL shift), or interpolated mode. |
|
||
| **LCS Color Batch** | Different target colors per batch item. Outputs `batch_size` for EmptyLatentImage. |
|
||
| **LCS Tone Adjust** | Contrast, brightness, saturation, temperature. Preset dropdown with real-time slider sync. |
|
||
|
||
### Observation
|
||
|
||
| Node | Description |
|
||
|------|-------------|
|
||
| **LCS Preview Colors** | Decode latent colors to RGB preview without VAE decoding. |
|
||
| **LCS Step Observer** | Save per-step color preview PNGs to ComfyUI temp directory. |
|
||
|
||
## Intervention Modes
|
||
|
||
| Mode | Description | Best For |
|
||
|------|-------------|----------|
|
||
| **interpolated** (default) | Blends Type I and Type II using sigma | General use |
|
||
| **type_i** | Direct translation in 3D LCS space | Strong global color shifts |
|
||
| **type_ii** | Per-patch HSL interpolation via bicone geometry | Precise local color control |
|
||
|
||
## Key Parameters
|
||
|
||
- **strength** (0.0–2.0): Intervention intensity. 1.0 = full, 0.0 = none.
|
||
- **start_step / end_step**: Step range for intervention. Paper optimal: steps 8–10 of 50.
|
||
- **mask**: Optional. Downsampled to patch grid for localized control.
|
||
|
||
## Tone Presets
|
||
|
||
Select a preset — sliders update in real-time. Tweak after selecting for fine-tuning. Select **Custom** to set values manually.
|
||
|
||
| Preset | Contrast | Brightness | Saturation | Temperature |
|
||
|--------|----------|------------|------------|-------------|
|
||
| Base | 1.0 | 0.0 | 1.0 | 0.0 |
|
||
| Cinematic | 1.20 | -0.05 | 0.90 | 0.05 |
|
||
| HDR | 1.40 | 0.0 | 1.20 | 0.0 |
|
||
| Vivid | 1.10 | 0.0 | 1.50 | 0.0 |
|
||
| Dramatic | 1.50 | -0.10 | 0.85 | 0.0 |
|
||
| Low Key | 1.30 | -0.20 | 0.80 | 0.0 |
|
||
| High Key | 0.80 | 0.20 | 0.90 | 0.0 |
|
||
| Warm | 1.05 | 0.03 | 1.10 | 0.30 |
|
||
| Cool | 1.05 | 0.0 | 1.05 | -0.30 |
|
||
| Desaturated | 1.0 | 0.0 | 0.40 | 0.0 |
|
||
|
||
## How It Works
|
||
|
||
1. **Project** — Convert denoised prediction to 64D patch space, project onto 3D LCS basis
|
||
2. **Decompose** — Separate 3D color coordinates from the 61D structural residual
|
||
3. **Normalize** — Transform to reference timestep (t=50) using learned alpha/beta statistics
|
||
4. **Manipulate** — Shift colors, adjust tone, or apply other transformations in 3D LCS
|
||
5. **Reconstruct** — Denormalize, add back the preserved 61D residual, convert to latent space
|
||
|
||
The 61D residual (structure, texture, detail) is never modified — only the 3D color subspace is touched.
|
||
|
||
## File Structure
|
||
|
||
```
|
||
ComfyUI-LCS/
|
||
├── __init__.py # Entry point (V3 + V2 compat)
|
||
├── requirements.txt
|
||
├── core/
|
||
│ ├── calibration.py # PCA calibration pipeline
|
||
│ ├── color_space.py # Bicone LCS ↔ HSL mapping
|
||
│ ├── defaults.py # Alpha/beta tables from paper
|
||
│ ├── lcs_data.py # LCSData dataclass
|
||
│ ├── patchify.py # Patch ↔ latent conversion
|
||
│ └── timestep.py # Sigma/timestep utilities
|
||
├── nodes/
|
||
│ ├── calibrate.py # LCSLoadData (auto-calibrate + cache)
|
||
│ ├── intervene.py # LCSColorIntervene, LCSColorBatch, LCSToneAdjust
|
||
│ └── observe.py # LCSPreviewColors, LCSStepObserver
|
||
├── data/ # Cached calibration files
|
||
└── web/js/
|
||
└── tone_preset.js # Frontend preset sync
|
||
```
|
||
|
||
## Citation
|
||
|
||
Official repository: [ExplainableML/LCS](https://github.com/ExplainableML/LCS)
|
||
|
||
```bibtex
|
||
@article{pach2026latentcolorsubspace,
|
||
title={The Latent Color Subspace: Emergent Order in High-Dimensional Chaos},
|
||
author={Mateusz Pach and Jessica Bader and Quentin Bouniot and Serge Belongie and Zeynep Akata},
|
||
journal={arxiv},
|
||
year={2026}
|
||
}
|
||
```
|
||
|
||
## Acknowledgments
|
||
|
||
Thanks to Mateusz Pach, Jessica Bader, Quentin Bouniot, Serge Belongie, and Zeynep Akata for their research making training-free color control possible.
|
||
|
||
## License
|
||
|
||
MIT
|