nodes/smthemex-ComfyUI_Pops
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Update README.md

Browse Source
This commit is contained in:
smthemex
2024-08-12 17:11:49 +08:00
committed by GitHub
parent 165c415555
commit 6764bdfcdd
1 changed files with 52 additions and 71 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+52 -71
View File
@@ -1,34 +1,16 @@
# ComfyUI_Pops
using pOpsPaper method in ComfyUI
pOps: Photo-Inspired Diffusion Operators
pOpsPaper method From: [pOpsPaper](https://github.com/pOpsPaper/pOps)
----
My ComfyUI node list：
-----
1、ParlerTTS node:[ComfyUI_ParlerTTS](https://github.com/smthemex/ComfyUI_ParlerTTS)
2、Llama3_8B node:[ComfyUI_Llama3_8B](https://github.com/smthemex/ComfyUI_Llama3_8B)
3、HiDiffusion node：[ComfyUI_HiDiffusion_Pro](https://github.com/smthemex/ComfyUI_HiDiffusion_Pro)
4、ID_Animator node： [ComfyUI_ID_Animator](https://github.com/smthemex/ComfyUI_ID_Animator)
5、StoryDiffusion node：[ComfyUI_StoryDiffusion](https://github.com/smthemex/ComfyUI_StoryDiffusion)
6、Pops node：[ComfyUI_Pops](https://github.com/smthemex/ComfyUI_Pops)
7、stable-audio-open-1.0 node ：[ComfyUI_StableAudio_Open](https://github.com/smthemex/ComfyUI_StableAudio_Open)
8、GLM4 node：[ComfyUI_ChatGLM_API](https://github.com/smthemex/ComfyUI_ChatGLM_API)
9、CustomNet node：[ComfyUI_CustomNet](https://github.com/smthemex/ComfyUI_CustomNet)
10、Pipeline_Tool node :[ComfyUI_Pipeline_Tool](https://github.com/smthemex/ComfyUI_Pipeline_Tool)
11、Pic2Story node :[ComfyUI_Pic2Story](https://github.com/smthemex/ComfyUI_Pic2Story)
12、PBR_Maker node:[ComfyUI_PBR_Maker](https://github.com/smthemex/ComfyUI_PBR_Maker)
13、ComfyUI_Streamv2v_Plus node:[ComfyUI_Streamv2v_Plus](https://github.com/smthemex/ComfyUI_Streamv2v_Plus)
14、ComfyUI_MS_Diffusion node:[ComfyUI_MS_Diffusion](https://github.com/smthemex/ComfyUI_MS_Diffusion)
Tips
Update
---
--本节点主要引用pOpsPaper的方法，可以达成材质迁移，背景迁移，融合，语义增强功能，以及这四种功能的组合，每次调用embedding节点，都会在output目录生成该次工作流创建的embedding文件，你可以在下次使用中，加载该embedding文件，从而达到树状组合的目的。
--采样方式分为unet，ipadapter，controlnet+ipadapter，mean 四种，其中ipadapter，controlnet+ipadapter调用SDXL社区模型（比较耗时），unet和mean 调用kandinsky模型。
-- 暂时去掉SDXL板块/Temporarily remove the SDXL sector
-- 改了代码结构，运行很快，但是抽卡的几率太大了。
--后期直接连入其他工作流，待完成。
---This node mainly references the methods of pOpsPaper, which can achieve material migration, background migration, fusion, semantic enhancement functions, and a combination of these four functions. Each time the embedding node is called, the embedding file created in the output directory will be generated. You can load the embedding file in the next use to achieve the goal of tree structure combination.
---The sampling methods are divided into four types: unet, ipadapter, controllnet+ipadapter, and mean. Among them, ipadapter, controllnet+ipadapter call the SDXL community model (which is relatively time-consuming), while unet and mean call the Kandinsky model.
1.Installation
-----
@@ -46,13 +28,7 @@ pip install -r requirements.txt
3 Need models
----
3.1
默认的repo节点会自动联网下载所有必需的模型，如果你要使用离线模式，可以把模型先下载到comfyUI的diffuers目录下，这些就可以使用repo节点的菜单。
当然，你的C盘足够大，或者你的huggingface的缓存已经切换到其他盘，建议用默认的下载就好。
可能会要求联外网，包括会下载一些clip文件，每个人的系统不同，不能一一例举。。
3.1 online
The default repo node will automatically connect to the network to download all necessary models ！！
--
@@ -60,16 +36,8 @@ kandinsky-community/kandinsky-2-2-prior: [link](https://huggingface.co/kandinsky
kandinsky-community/kandinsky-2-2-decoder: [link](https://huggingface.co/kandinsky-community/kandinsky-2-2-decoder)
h94/IP-Adapter ip-adapter_sdxl.bin [link](https://huggingface.co/h94/IP-Adapter)
pOpsPaper/operators （four models） [link](https://huggingface.co/pOpsPaper/operators)
----and---
Menu 'checkpoint' choice any comfyUI or Web UI SDXL model
Example：Juggernaut-XL_v9_RunDiffusionPhoto_v2.safetensors
Menu 'vae' choice any comfyUI or Web UI SDXL vae
Example：madebyollin/sdxl-vae-fp16-fix or sdxl.vae.safetensors
3.2 离线模式 offline
在插件的weights目录下，如图放置ip-adapter_sdxl.bin，然后diffusers目录，放置其余3组模型。
@@ -87,7 +55,9 @@ Example：madebyollin/sdxl-vae-fp16-fix or sdxl.vae.safetensors
| ├──movq/
| ├── config.json
| ├── diffusion_pytorch_model.safetensors
| ├── kandinsky-2-2-prior/
├── ComfyUI/models/diffusers/
| ├──kandinsky-community/
| ├── kandinsky-2-2-prior/
| ├── model_index.json
| ├──tokenizer/
| ├── merges.txt
@@ -108,52 +78,63 @@ Example：madebyollin/sdxl-vae-fp16-fix or sdxl.vae.safetensors
| ├── config.json
| ├── model.safetensors
│
│ ├──pOpsPaper/operators/models/
| ├── instruct/
| ├── learned_prior.pth
| ├── scene/
| ├── learned_prior.pth
| ├── texturing/
| ├── learned_prior.pth
| ├── union/
| ├── learned_prior.pth
|
├── ComfyUI/custom_nodes/ComfyUI_Pops/
| ├──weights/
| ├── ip-adapter_sdxl.bin
├──ComfyUI/models/checkpoints
| ├── instruct_learned_prior.pth (rename from instruct/learned_prior.pth )
| ├── scene_learned_prior.pth (rename from scene/learned_prior.pth )
| ├── texturing_learned_prior.pth (rename from texturing/learned_prior.pth )
| ├── union_learned_prior.pth (rename from union/learned_prior.pth )
```
如果使用离线模式，pOpsPaper/operators 这一栏，填写方式如下(后面的类似models/instructlearned_prior.pth路径切勿填写)：
If using offline mode, fill in the pOpsPaper/operators column as follows (do not fill in the path similar to models/instructlearned_primary.pth):
--Example--
Your_local_model_path/pOpsPaper/operators
---
4 Example
----
object2texture using controlnet 物体赋予材质，controlnet加ip采样。
![](https://github.com/smthemex/ComfyUI_Pops/blob/main/example/controlnet.png)
object2texture 物体材质模型
![](https://github.com/smthemex/ComfyUI_Pops/blob/main/example/texture.png))
object scene 物体场景模型，
![](https://github.com/smthemex/ComfyUI_Pops/blob/main/example/scene.png)
drop cond 删掉某个条件 物体场景模型，
![](https://github.com/smthemex/ComfyUI_Pops/blob/main/example/drop_cond.png)
object2texture using IPadapter 物体赋予材质，
![](https://github.com/smthemex/ComfyUI_Pops/blob/main/example/ipsampler.png)
5 My ComfyUI node list：
-----
use pt 使用插件生成的pt
![](https://github.com/smthemex/ComfyUI_Pops/blob/main/example/use_pt.png)
1、ParlerTTS node:[ComfyUI_ParlerTTS](https://github.com/smthemex/ComfyUI_ParlerTTS)
2、Llama3_8B node:[ComfyUI_Llama3_8B](https://github.com/smthemex/ComfyUI_Llama3_8B)
3、HiDiffusion node：[ComfyUI_HiDiffusion_Pro](https://github.com/smthemex/ComfyUI_HiDiffusion_Pro)
4、ID_Animator node： [ComfyUI_ID_Animator](https://github.com/smthemex/ComfyUI_ID_Animator)
5、StoryDiffusion node：[ComfyUI_StoryDiffusion](https://github.com/smthemex/ComfyUI_StoryDiffusion)
6、Pops node：[ComfyUI_Pops](https://github.com/smthemex/ComfyUI_Pops)
7、stable-audio-open-1.0 node ：[ComfyUI_StableAudio_Open](https://github.com/smthemex/ComfyUI_StableAudio_Open)
8、GLM4 node：[ComfyUI_ChatGLM_API](https://github.com/smthemex/ComfyUI_ChatGLM_API)
9、CustomNet node：[ComfyUI_CustomNet](https://github.com/smthemex/ComfyUI_CustomNet)
10、Pipeline_Tool node :[ComfyUI_Pipeline_Tool](https://github.com/smthemex/ComfyUI_Pipeline_Tool)
11、Pic2Story node :[ComfyUI_Pic2Story](https://github.com/smthemex/ComfyUI_Pic2Story)
12、PBR_Maker node:[ComfyUI_PBR_Maker](https://github.com/smthemex/ComfyUI_PBR_Maker)
13、ComfyUI_Streamv2v_Plus node:[ComfyUI_Streamv2v_Plus](https://github.com/smthemex/ComfyUI_Streamv2v_Plus)
14、ComfyUI_MS_Diffusion node:[ComfyUI_MS_Diffusion](https://github.com/smthemex/ComfyUI_MS_Diffusion)
15、ComfyUI_AnyDoor node: [ComfyUI_AnyDoor](https://github.com/smthemex/ComfyUI_AnyDoor)
16、ComfyUI_Stable_Makeup node: [ComfyUI_Stable_Makeup](https://github.com/smthemex/ComfyUI_Stable_Makeup)
17、ComfyUI_EchoMimic node: [ComfyUI_EchoMimic](https://github.com/smthemex/ComfyUI_EchoMimic)
18、ComfyUI_FollowYourEmoji node: [ComfyUI_FollowYourEmoji](https://github.com/smthemex/ComfyUI_FollowYourEmoji)
19、ComfyUI_Diffree node: [ComfyUI_Diffree](https://github.com/smthemex/ComfyUI_Diffree)
20、ComfyUI_FoleyCrafter node: [ComfyUI_FoleyCrafter](https://github.com/smthemex/ComfyUI_FoleyCrafter)
mix pt 多PT混合
![](https://github.com/smthemex/ComfyUI_Pops/blob/main/example/mix.png)
Citation
6 Citation
------
pOps
``` python
https://popspaper.github.io/pOps
@article{richardson2024pops,
title={pOps: Photo-Inspired Diffusion Operators},
author={Richardson, Elad and Alaluf, Yuval and Mahdavi-Amiri, Ali and Cohen-Or, Daniel},
journal={arXiv preprint arXiv:2406.01300},
year={2024}
}
```
IP-Adapter