Add LICENSE, README, Examples.
This commit is contained in:
ZHOU He
@@ -0,0 +1,21 @@
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2025 ZHOU He
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
@@ -0,0 +1,194 @@
|
||||
# ComfyUI-CharaConsistent
|
||||
|
||||
Training-free CharaConsist algorithm implementation for ComfyUI - Generate consistent subjects across multiple generations with enhanced mask generation and multi-model support.
|
||||
|
||||
## License
|
||||
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2025 ZHOU He
|
||||
Email: he.zhou@unity.cn
|
||||
|
||||
Repository: https://github.com/thatname/Comfyui_CharaConsist
|
||||
## Credits & References
|
||||
|
||||
This is a ComfyUI implementation of the CharaConsist algorithm originally developed by Murray-Wang et al.
|
||||
|
||||
- **Original Research**: [CharaConsist: Fine-Grained Consistent Character Generation](https://arxiv.org/abs/2507.11533) (ICCV 2025)
|
||||
- **Original Implementation**: [Murray-Wang/CharaConsist](https://github.com/Murray-Wang/CharaConsist)
|
||||
- **Project Page**: [https://murray-wang.github.io/CharaConsist/](https://murray-wang.github.io/CharaConsist/)
|
||||
|
||||
This ComfyUI implementation includes significant enhancements and improvements over the original algorithm.
|
||||
|
||||
## Overview
|
||||
|
||||
ComfyUI-CharaConsistent implements the training-free CharaConsist algorithm within ComfyUI, enabling generation of consistent subjects across multiple images. Unlike the original implementation which primarily supported FLUX models, this version extends compatibility to multiple state-of-the-art text-to-image models while introducing improvements to mask generation and workflow integration.
|
||||
|
||||
## Key Improvements Over Original
|
||||
|
||||
### Enhanced Mask Generation
|
||||
- **Original Issue**: The original mask generation was not perfect, especially for Chroma models
|
||||
- **Solution**: Support for external GroundingDinoSAM for more accurate mask extraction
|
||||
- **Flexibility**: Users can choose between original or improved mask generation methods
|
||||
|
||||
### Expanded Model Support
|
||||
- **Original**: Primarily FLUX models only
|
||||
- **This Implementation**:
|
||||
- FLUX models
|
||||
- Chroma models (optimal performance)
|
||||
- Qwen-Image models
|
||||
- Chroma1-Radiance T2I models
|
||||
|
||||
### Multi-Subject Support
|
||||
- Supports simultaneous processing of multiple subjects in a single workflow
|
||||
- Each subject gets independent attention caching and mask processing
|
||||
|
||||
### ComfyUI Integration
|
||||
- Native ComfyUI node implementation
|
||||
- Seamless workflow integration with other ComfyUI custom nodes
|
||||
- Visual workflow examples provided
|
||||
|
||||
## Features
|
||||
|
||||
- ✅ **Training-free consistency** - No fine-tuning required
|
||||
- ✅ **Multi-subject support** - Generate consistent multiple characters/objects
|
||||
- ✅ **Enhanced mask generation** - GroundingDinoSAM integration for better results
|
||||
- ✅ **Broad model compatibility** - Works with FLUX, Chroma, Qwen-Image, Chroma1-Radiance
|
||||
- ✅ **Optimal for Chroma** - Best performance with Chroma models
|
||||
- ✅ **Attention caching** - Efficient memory management for subject consistency
|
||||
- ✅ **Cross-similarity matching** - Advanced subject matching algorithms
|
||||
- ✅ **Visual workflow examples** - Ready-to-use JSON workflows
|
||||
|
||||
## Installation
|
||||
|
||||
1. Navigate to your ComfyUI `custom_nodes` directory
|
||||
2. Clone or copy this repository:
|
||||
```bash
|
||||
git clone https://github.com/thatname/Comfyui_CharaConsist.git
|
||||
```
|
||||
3. Restart ComfyUI
|
||||
4. The nodes will appear in the `chara_consist` category
|
||||
|
||||
## Supported Models
|
||||
|
||||
| Model | Compatibility | Performance Notes |
|
||||
|--------|---------------|------------------|
|
||||
| **Chroma Models** | ✅ Excellent | **Best performance** - Recommended |
|
||||
| **FLUX Models** | ✅ Good | Compatible, original algorithm focus |
|
||||
| **Qwen-Image** | ✅ Good | Full support with example workflow |
|
||||
| **Chroma1-Radiance** | ✅ Good | Compatible with T2I workflows |
|
||||
|
||||
### Model-Specific Notes
|
||||
- **Chroma Models**: Optimal performance, enhanced mask generation recommended
|
||||
- **FLUX Models**: Good compatibility, original mask generation works adequately
|
||||
- **Qwen-Image**: Full support, see example workflow for best practices
|
||||
|
||||
## Mask Generation Options
|
||||
|
||||
### Option 1: Enhanced Mask Generation (Recommended)
|
||||
- **Method**: External GroundingDinoSAM
|
||||
- **Benefits**:
|
||||
- Superior mask accuracy
|
||||
- Better edge detection
|
||||
- Improved results for Chroma models
|
||||
- **Requirements**: GroundingDinoSAM custom node
|
||||
- **Example Workflow**: See JSON examples for implementation
|
||||
|
||||
### Option 2: Original Mask Generation
|
||||
- **Method**: Built-in algorithm
|
||||
- **Benefits**:
|
||||
- No additional dependencies
|
||||
- Faster processing
|
||||
- Works well with FLUX models
|
||||
- **Limitations**:
|
||||
- Less accurate for Chroma models
|
||||
- May have edge detection issues
|
||||
|
||||
## Example Workflows
|
||||
|
||||
### Chroma Workflow
|
||||
- **File**: `chroma_chara_consist.json`
|
||||
- **Model**: Chroma1-HD
|
||||
- **Features**:
|
||||
- Enhanced mask generation with GroundingDinoSAM
|
||||
- Multi-step generation pipeline
|
||||
- Optimal settings for Chroma models
|
||||
|
||||
### Qwen-Image Workflow
|
||||
- **File**: `qwen_chara_consist.json`
|
||||
- **Model**: Qwen-Image
|
||||
- **Features**:
|
||||
- LoRA integration
|
||||
- CFG normalization
|
||||
- Optimized for Qwen architecture
|
||||
|
||||
## Node Documentation
|
||||
|
||||
This implementation includes 9 custom nodes:
|
||||
|
||||
### Core Nodes
|
||||
1. **ExtractAttn** - Extracts attention data from model layers for subject analysis
|
||||
2. **GenConsistent** - Generates consistent images using cached attention data
|
||||
3. **GetCrossSim** - Computes cross-similarity between subject and target attention
|
||||
4. **BatchedMaskedReferenceGen** - Applies masked attention for consistent generation
|
||||
|
||||
### Mask Processing Nodes
|
||||
5. **MasksToPatches** - Converts pixel masks to patch-level masks for attention
|
||||
6. **MaskToPatchMask** - Converts masks to patch format with configurable parameters
|
||||
7. **PreviewSubjectMask** - Preview extracted subject masks
|
||||
|
||||
### Conditioning Nodes
|
||||
8. **ReferenceConditionCombine** - Combines reference and target conditioning
|
||||
9. **ConditioningMatchMask** - Matches reference conditions to target prompts
|
||||
|
||||
## Memory Requirements
|
||||
|
||||
**⚠️ Important RAM Considerations**
|
||||
|
||||
- **Per Subject**: 10s of gigabytes of System RAM required
|
||||
- **Attention Caching**: Each subject needs separate attention cache
|
||||
- **Multi-Subject**: Memory usage scales linearly with number of subjects
|
||||
- **Recommendations**:
|
||||
- Minimum 32GB RAM for single subject
|
||||
- 64GB+ RAM recommended for multiple subjects
|
||||
- Monitor RAM usage during generation
|
||||
|
||||
### Optimization Tips
|
||||
- Clear attention cache between sessions
|
||||
- Use appropriate batch sizes
|
||||
- Consider system RAM when planning multi-subject workflows
|
||||
|
||||
## Performance Tips
|
||||
|
||||
### Best Practices
|
||||
1. **Model Selection**: Use Chroma models for best results
|
||||
2. **Mask Generation**: Prefer GroundingDinoSAM for Chroma models
|
||||
3. **Memory Management**: Monitor RAM usage closely
|
||||
4. **Workflow Optimization**: Use provided example workflows as templates
|
||||
|
||||
### Troubleshooting
|
||||
- **High RAM Usage**: Reduce batch size or number of subjects
|
||||
- **Poor Mask Quality**: Switch to GroundingDinoSAM mask generation
|
||||
- **Model Compatibility**: Ensure you're using supported model versions
|
||||
- **Performance Issues**: Check system RAM availability
|
||||
|
||||
## Academic Citation
|
||||
|
||||
If you use CharaConsist in your research, please cite the original paper:
|
||||
|
||||
```bibtex
|
||||
@inproceedings{CharaConsist,
|
||||
title={{CharaConsist}: Fine-Grained Consistent Character Generation},
|
||||
author={Wang, Mengyu and Ding, Henghui and Peng, Jianing and Zhao, Yao and Chen, Yunpeng and Wei, Yunchao},
|
||||
booktitle={ICCV},
|
||||
year={2025}
|
||||
}
|
||||
```
|
||||
|
||||
## Contributing
|
||||
|
||||
Contributions are welcome! Please feel free to submit issues and enhancement requests.
|
||||
|
||||
## License
|
||||
|
||||
This project is licensed under the MIT License - see the LICENSE file for details.
|
||||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
Reference in New Issue
Block a user