07/24 --新增社区单体模型支持，单体controlnet模型支持
-- 通过菜单model_category来选择使用不同模型；
--VAE可以填写你喜欢的vae地址 --Added support for community monolithic models and monolithic ControlNet models
--Select different models through the menu “model_category”;
--VAE can fill in your favorite VAE address

-- --增加adapter_style支持，SDXL需求的显存较大，虽然能跑CPU，但是不推荐，会爆显存，SD1.5测试没问题。
--Adding adapter_style support, SDXL requires a large amount of graphics memory. Although it can run on CPU, it is not recommended as it may cause graphics memory "explosion". SD1.5 testing is not a problem.

My ComfyUI node list：

1、ParlerTTS node:ComfyUI_ParlerTTS
2、Llama3_8B node:ComfyUI_Llama3_8B
3、HiDiffusion node：ComfyUI_HiDiffusion_Pro
4、ID_Animator node： ComfyUI_ID_Animator
5、StoryDiffusion node：ComfyUI_StoryDiffusion
6、Pops node：ComfyUI_Pops
7、stable-audio-open-1.0 node ：ComfyUI_StableAudio_Open
8、GLM4 node：ComfyUI_ChatGLM_API
9、CustomNet node：ComfyUI_CustomNet
10、Pipeline_Tool node :ComfyUI_Pipeline_Tool
11、Pic2Story node :ComfyUI_Pic2Story
12、PBR_Maker node:ComfyUI_PBR_Maker
13、ComfyUI_Streamv2v_Plus node:ComfyUI_Streamv2v_Plus
14、ComfyUI_MS_Diffusion node:ComfyUI_MS_Diffusion
15、ComfyUI_AnyDoor node: ComfyUI_AnyDoor
16、ComfyUI_Stable_Makeup node: ComfyUI_Stable_Makeup
17、ComfyUI_EchoMimic node: ComfyUI_EchoMimic
18、ComfyUI_FollowYourEmoji node: ComfyUI_FollowYourEmoji

Notice（节点的特殊功能说明 Special Function Description of Nodes）

-- 增加 manne加速Lora
-- 加入controlnet-tile-sdxl的支持，内置图片预处理，默认512尺寸，新增apply_window_attn 条件控制。
--修复节点连接逻辑，现在文生图模式，无需接入image，无controlnet也无需接入control_image
--支持SDXL-lighting\Hyper\LCM\DMD2\的加速Unet，建议适当提高步数；
--基于官方的更新，加入lora支持，需要填关键词；
--加入skip，去掉意义不大的其他参数；

-- add manne lighting lora
--Added support for control net file sdxl, built-in image preprocessing, default size of 512, and added condition control for apply_window_attn.
--Fix node connection logic, now in text-based graphics mode, there is no need to connect to image, no controllnet, and no need to connect to controll_image
--Support acceleration Unet for SDXL lighting, Hyper, LCM, and DMD2. It is recommended to increase the number of steps appropriately;
--Based on official updates, adding support for Lora requires filling in keywords;
--Add skip and remove other parameters that are not significant;

1.Installation

1.1 In the .\ComfyUI \ custom_node directory, run the following:

git clone https://github.com/smthemex/ComfyUI_HiDiffusion_Pro.git

1.2 using it

2.requirements

diffusers >=0.28.0 is best yaml

3 About models

3.1 使用社区单体模型 Using the Community Monomer Model

3.2 填写repo_id 和controlnet_repo_id
可以是stabilityai/stable-diffusion-xl-base-1.0
也可是“x:/xx/xx/stabilityai/stable-diffusion-xl-base-1.0”

controlnet_repo_id 如果使用controlnet_repo_id，本地模型存放示例：

├── your path/   
|     ├──xinsir/controlnet-openpose-sdxl-1.0    
|         ├── config.json   
|         ├── diffusion_pytorch_model.fp16.safetensors   
|     ├──xinsir/controlnet-scribble-sdxl-1.0   
|         ├── config.json   
|         ├── diffusion_pytorch_model.fp16.safetensors
|     ├──xinsir/controlnet-tile-sdxl-1.0  
|         ├── config.json   
|         ├── diffusion_pytorch_model.fp16.safetensors  
|     ├──diffusers/controlnet-canny-sdxl-1.0   
|         ├── config.json   
|         ├── diffusion_pytorch_model.fp16.safetensors   
|     ├──diffusers/controlnet-depth-sdxl-1.0   
|         ├── config.json   
|         ├── diffusion_pytorch_model.fp16.safetensors
|     ├──TheMistoAI/MistoLine 
|         ├── config.json   
|         ├── diffusion_pytorch_model.fp16.safetensors

ip_adapter model 模型存放示例：

如果不存放对应，运行时会自动下载。 If the corresponding file is not stored, it will be automatically downloaded at runtime.

├── ComfyUI/models/photomaker
|     ├──models/
|         ├── ip-adapter_sd15.bin
|         ├── image_encoder/
|             ├── config.json
|             ├── model.safetensors
|     ├──sdxl_models/
|         ├── ip-adapter_sdxl.bin
|         ├── image_encoder/
|             ├── config.json
|             ├── model.safetensors

4 other

部分支持的模型请查看model.yaml 文件
For partially supported models, please refer to the model.yaml file

5 example

sd1.5 using ip_adapter_style 使用ip_adapter_style

img2img use lora 图生图和lora

img2img + controlnet 图生图加controlnet

img2img use Hyper unet 图生图加加速unet

6 Citation

@article{zhang2023hidiffusion,
  title={HiDiffusion: Unlocking Higher-Resolution Creativity and Efficiency in Pretrained Diffusion Models},
  author={Zhang, Shen and Chen, Zhaowei and Zhao, Zhenyu and Chen, Yuhao and Tang, Yao and Liang, Jiajun},
  journal={arXiv preprint arXiv:2311.17528},
  year={2023}
}

README.md Unescape Escape

ComfyUI_HiDiffusion_Pro

HiDiffusion From: link

Update