ComfyUI-BrushNet
Custom nodes for ComfyUI allow to inpaint using Brushnet: "BrushNet: A Plug-and-Play Image Inpainting Model with Decomposed Dual-Branch Diffusion".
My contribution is limited to the ComfyUI adaptation, and all credit goes to the authors of the paper.
Installation
Clone the repo into the custom_nodes directory and install the requirements:
git clone https://github.com/nullquant/ComfyUI-BrushNet.git
pip install -r requirements.txt
Checkpoints of BrushNet can be downloaded from here.
The checkpoint in segmentation_mask_brushnet_ckpt provides checkpoints trained on BrushData, which has segmentation prior (mask are with the same shape of objects). The random_mask_brushnet_ckpt provides a more general ckpt for random mask shape.
Both segmentation_mask_brushnet_ckpt and random_mask_brushnet_ckpt or only one of them should be placed inside models/inpaint/brushnet folder.
Base model should be SD1.5 type (realisticVisionV60B1_v51VAE, for example).
This is complete rework of previous version. There is no more custom diffusers library.
There is no loadings from github. I also changed the workflow so now it is possible to use LoRa models.
To users who already have previoous version: please, uninstall diffusers library before installation:
pip uninstall diffusers
Then install as usual.
Usage
Below is an example for the intended workflow. The json file for the example can be found inside the 'example' directory.
This is example with LoRA: json file.
Credits
The code is based on