diff --git a/README.md b/README.md index 0c39206..b065ca1 100644 --- a/README.md +++ b/README.md @@ -1,27 +1,34 @@ -## Info ComfyUI nodes for outpainting images with diffusers, based on [diffusers-image-outpaint](https://huggingface.co/spaces/fffiloni/diffusers-image-outpaint/tree/main) by fffiloni. ![DiffusersImageOutpaint-Nodes-Screen](https://github.com/user-attachments/assets/df6ee871-08ab-4e34-b47e-410673a026ed) -The extension gives 3 nodes: -- #### Load Diffusion Outpaint Models - - a simple node to load diffusion models; you can download them from Huggingface (the extension doesn't download them automatically), then you have to follow this order (the model links are an example): - | ComfyUI/**Unet folder** | ComfyUI/**Vae folder** | ComfyUI/**Controlnet folder** | +## Installation +- Download this extension or `git clone` it in comfyui/custom_nodes, then (if comfyui-manager didn't already install the requirements or you have missing modules), from comfyui virtual env write `cd your/path/to/this/extension` and `pip install -r requirements.txt`. +- Download models in comfyui/models/`unet`, `vae` and `controlnet` folder, following the grid below (the links are the suggested models, and you can change the model in models/unet, but you need the specified vae and controlnet since the extension is hardcoded to use them. You can always change the code to use different models): + | ComfyUI/models/**Unet folder** | ComfyUI/models/**Vae folder** | ComfyUI/models/**Controlnet folder** | | :-----: | :-----: | :-----: | - | **[Diffuser Model folder](https://huggingface.co/SG161222/RealVisXL_V5.0_Lightning/tree/main)** | **[Diffuser Vae folder](https://huggingface.co/madebyollin/sdxl-vae-fp16-fix/tree/main)** | **[Diffuser Controlnet folder](https://huggingface.co/xinsir/controlnet-union-sdxl-1.0/tree/main)** | - | **Unet folder** | config.json, sdxl_vae.safetensors | config_promax.json, diffusion_pytorch_model_promax.safetensors | + | **[Diffuser Model folder](https://huggingface.co/SG161222/RealVisXL_V5.0_Lightning/tree/main)** (you can change this model) | **[Diffuser Vae folder](https://huggingface.co/madebyollin/sdxl-vae-fp16-fix/tree/main)** (you need this model) | **[Diffuser Controlnet folder](https://huggingface.co/xinsir/controlnet-union-sdxl-1.0/tree/main)** (you need this model) | + | model_index-json | config.json, sdxl_vae.safetensors | config_promax.json, diffusion_pytorch_model_promax.safetensors | + | **Unet folder** | | | | config.json, diffusion_pytorch_model.fp16.safetensors | | | - | **Scheduler** | | | - | File 1, File 2, File 3 | | | + | **Scheduler folder** | | | + | scheduler_config.json | | | | **Text encoder folder** | | | - | File 1, File 2, File 3 | | | + | config.json, model.fp16.safetensors | | | | **Text encoder 2 folder** | | | - | File 1, File 2, File 3 | | | + | config.json, model.fp16.safetensors | | | | **Tokenizer folder** | | | - | File 1, File 2, File 3 | | | + | merges.txt, special_tokens_map.json, tokenizer_config.json, vocab.json | | | | **Tokenizer 2 folder** | | | - | File 1, File 2, File 3 | | | + | merges.txt, special_tokens_map.json, tokenizer_config.json, vocab.json | | | + +## Overview +The extension gives 3 nodes: +- **Load Diffusion Outpaint Models**: a simple node to load diffusion `models`. You can download them from Huggingface (the extension doesn't download them automatically); +- **Paid Image for Diffusers Outpaint**: this node creates an empty image of the `desired size`, fits the original image in the new one based on the chosen `alignment`, then mask the rest; +- **Diffusers Image Outpaint**: This is the main node, that outpaints the image. Currently the generation process is based on fffiloni's one, so you can't reproduce a specific a specific outpaint, and the `seed` option you see is only used to change the UI and generate a new image. Anyway, you can specify the amount of `steps` to generate the image and the `prompt` to specify what to add to the image. -- ****Paid Image for Diffusers Outpaint****: this node creates an empty image of the desired size, fits the original image in the new one based on the chosen alignment, then mask the rest; -- ****Diffusers Image Outpaint****: This is the main node, that outpaints the image. Currently the generation process is based on fffiloni's one, so you can't reproduce a specific a specific outpaint, and the seed option you see is only used to change the UI and generate a new image. +- You can also pass image and mask to `vae encode (for inpainting)` node, then pass the latent to a `sampler`, but controlnets and ip-adapters are harder to use compared to diffusers outpaint. + +## Credits +diffusers-image-outpaint by [fffiloni](https://huggingface.co/spaces/fffiloni/diffusers-image-outpaint/tree/main)