41 lines
1.4 KiB
Markdown
41 lines
1.4 KiB
Markdown
## ComfyUI-BrushNet
|
|
|
|
Custom nodes for ComfyUI allow to inpaint using Brushnet: ["BrushNet: A Plug-and-Play Image Inpainting Model with Decomposed Dual-Branch Diffusion"](https://arxiv.org/abs/2403.06976).
|
|
|
|
My contribution is limited to the ComfyUI adaptation, and all credit goes to the authors of the paper.
|
|
|
|
|
|
## Installation
|
|
|
|
Clone the repo into the custom_nodes directory and install the requirements:
|
|
|
|
```
|
|
git clone https://github.com/nullquant/ComfyUI-BrushNet.git
|
|
pip install -r requirements.txt
|
|
```
|
|
|
|
Checkpoints of BrushNet can be downloaded from [here](https://drive.google.com/drive/folders/1fqmS1CEOvXCxNWFrsSYd_jHYXxrydh1n?usp=drive_link).
|
|
Both `segmentation_mask_brushnet_ckpt` and `random_mask_brushnet_ckpt` should be placed inside `models/inpaint/brushnet` folder.
|
|
Base model should be SD1.5 type (realisticVisionV60B1_v51VAE, for example).
|
|
|
|
## Usage
|
|
|
|
Below is an example for the intended workflow. The [json file](example/BrushNet_example.json) for the example can be found inside the 'workflow' directory.
|
|
|
|
|
|
|
|
|
|
## Bugs
|
|
|
|
1. Nodes use custom `diffusers` library. It can potentially conflicts with other custom nodes.
|
|
2. Sometimes `BrushNetPipeline` freezes on loading. Looking into the problem.
|
|
|
|
|
|
## Credits
|
|
|
|
The code is based on
|
|
|
|
- [BrushNet](https://github.com/TencentARC/BrushNet)
|
|
- [PowerPaint](https://github.com/zhuang2002/PowerPaint)
|
|
- [diffusers](https://github.com/huggingface/diffusers)
|