nodes/cubiq-ComfyUI_IPAdapter_plus
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

add global models directory, fix #82

Browse Source
This commit is contained in:
matt3o
2023-11-23 15:04:17 +01:00
parent 2255ae7660
commit d6cfa11bef
2 changed files with 19 additions and 11 deletions
Download Patch File Download Diff File Expand all files Collapse all files
IPAdapterPlus.py
+9 -6
View File
@@ -15,15 +15,18 @@ import torchvision.transforms as TT
from .resampler import Resampler
MODELS_DIR = os.path.join(os.path.dirname(os.path.realpath(__file__)), "models")
# set the models directory backward compatible
GLOBAL_MODELS_DIR = os.path.join(folder_paths.models_dir, "ipadapter")
MODELS_DIR = GLOBAL_MODELS_DIR if os.path.isdir(GLOBAL_MODELS_DIR) else os.path.join(os.path.dirname(os.path.realpath(__file__)), "models")
if "ipadapter" not in folder_paths.folder_names_and_paths:
folder_paths.folder_names_and_paths["ipadapter"] = ([MODELS_DIR], folder_paths.supported_pt_extensions)
else:
folder_paths.folder_names_and_paths["ipadapter"][1].update(folder_paths.supported_pt_extensions)
# attention_channels
SD_V12_CHANNELS = [320] * 4 + [640] * 4 + [1280] * 4 + [1280] * 6 + [640] * 6 + [320] * 6 + [1280] * 2
SD_XL_CHANNELS = [640] * 8 + [1280] * 40 + [1280] * 60 + [640] * 12 + [1280] * 20
def get_filename_list(path):
return [f for f in os.listdir(path) if f.endswith('.bin') or f.endswith('.safetensors')]
class MLPProjModel(torch.nn.Module):
"""SD model with image prompt"""
def __init__(self, cross_attention_dim=1024, clip_embeddings_dim=1024):
@@ -296,7 +299,7 @@ class CrossAttentionPatch:
class IPAdapterModelLoader:
@classmethod
def INPUT_TYPES(s):
return {"required": { "ipadapter_file": (get_filename_list(MODELS_DIR), )}}
return {"required": { "ipadapter_file": (folder_paths.get_filename_list("ipadapter"), )}}
RETURN_TYPES = ("IPADAPTER",)
FUNCTION = "load_ipadapter_model"
@@ -304,7 +307,7 @@ class IPAdapterModelLoader:
CATEGORY = "ipadapter"
def load_ipadapter_model(self, ipadapter_file):
ckpt_path = os.path.join(MODELS_DIR, ipadapter_file)
ckpt_path = folder_paths.get_full_path("ipadapter", ipadapter_file)
model = comfy.utils.load_torch_file(ckpt_path, safe_load=True)
README.md
+10 -5
View File
@@ -5,7 +5,9 @@ IPAdapter implementation that follows the ComfyUI way of doing things. The code
## Important updates
**2023/11/08**: Added [attention masking](#attention-masking).
**2023/11/23**: Small but important update: the new default location for the IPAdapter models is `ComfyUI/models/ipadapter`. **No panic**: the legacy `ComfyUI/custom_nodes/ComfyUI_IPAdapter_plus/models` location still works and nothing will break.
**2023/11/08**: Added [attention masking](#attention-masking).
**2023/11/07**: Added three ways to apply the weight. [See below](#weight-types) for more info. **This might break things!** Please let me know if you are having issues. When loading an old workflow try to reload the page a couple of times or delete the `IPAdapter Apply` node and insert a new one.
@@ -41,15 +43,18 @@ The IPAdapter are very powerful models for image-to-image conditioning. Given a
Download or git clone this repository inside `ComfyUI/custom_nodes/` directory.
The pre-trained models are available on [huggingface](https://huggingface.co/h94/IP-Adapter), download and place them in the `ComfyUI/custom_nodes/ComfyUI_IPAdapter_plus/models` directory.
The pre-trained models are available on [huggingface](https://huggingface.co/h94/IP-Adapter), download and place them in the `ComfyUI/models/ipadapter` directory (create it if not present). You can also use any custom location setting an `ipadapter` entry in the `extra_model_paths.yaml` file.
Note: the legacy `ComfyUI/custom_nodes/ComfyUI_IPAdapter_plus/models` is still supported and it will be ignored only if the global directory is present.
For SD1.5 you need:
- [ip-adapter_sd15.bin](https://huggingface.co/h94/IP-Adapter/resolve/main/models/ip-adapter_sd15.bin)
- [ip-adapter_sd15_light.bin](https://huggingface.co/h94/IP-Adapter/blob/main/models/ip-adapter_sd15_light.bin), use this when text prompt is more important than reference images
- [ip-adapter_sd15_light.bin](https://huggingface.co/h94/IP-Adapter/blob/main/models/ip-adapter_sd15_light.safetensors), use this when text prompt is more important than reference images
- [ip-adapter-plus_sd15.bin](https://huggingface.co/h94/IP-Adapter/resolve/main/models/ip-adapter-plus_sd15.bin)
- [ip-adapter-plus-face_sd15.bin](https://huggingface.co/h94/IP-Adapter/resolve/main/models/ip-adapter-plus-face_sd15.bin)
- [ip-adapter-full-face_sd15.bin](https://huggingface.co/h94/IP-Adapter/resolve/main/models/ip-adapter-full-face_sd15.bin)
- [ip-adapter_sd15_vit-G.bin](https://huggingface.co/h94/IP-Adapter/resolve/main/models/ip-adapter_sd15_vit-G.bin), this model requires the vit-bigG image encoder (the SDXL one below)
For SDXL you need:
- [ip-adapter_sdxl.bin](https://huggingface.co/h94/IP-Adapter/resolve/main/sdxl_models/ip-adapter_sdxl.bin)
@@ -166,9 +171,9 @@ In the examples directory you'll find a couple of masking workflows: [simple](ex
## Troubleshooting
**Error: 'CLIPVisionModelOutput' object has no attribute 'penultimate_hidden_states'**
**Error: 'CLIPVisionModelOutput' object has no attribute 'penultimate_hidden_states'** and **AttributeError: 'NoneType' object has no attribute 'encode_image'**
You are using an old version of ComfyUI. Update and you'll be fine. **Please note** that on Windows for a full update you might need to re-download the latest standalone version.
You are using an old version of ComfyUI or of this extension. Update and you'll be fine. **Please note** that on Windows for a full update you might need to re-download the latest standalone version. Also sometimes the Manager fails to update even if everything seems fine. In that case you need to update manually or remove the extension and re-install it.
**size mismatch for proj_in.weight: copying a param with shape torch.Size([..., ...]) from checkpoint, the shape in current model is torch.Size([..., ...])**