nodes/Danand-ComfyUI-ComfyCouple
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Replace README.md

Browse Source
This commit is contained in:
Rei Koshka
2023-11-23 00:32:04 +03:00
parent d9e8704cee
commit ab22addfd3
6 changed files with 645 additions and 18 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+50 -18
View File
@@ -1,25 +1,57 @@
# attention-couple-ComfyUI
プロンプトの領域指定を行うカスタムノードです。ComfyUIにはlatent coupleベースの領域指定法が存在しますが、こちらはUNet内のAttention層で領域分けします。
# Comfy Couple
# 使い方
1. custom_nodesにくろーん
2. latent coupleと同様のノードを作る。([参考](https://comfyui.creamlab.net/nodes/ConditioningSetMask))
3. loaderからattention coupleのノードをロードしてMODELとCONDITIONINGを繋げる
4. 画像を作る
## What is
使用例：[attention_couple.json](https://github.com/laksjdjf/attention-couple-ComfyUI/blob/main/attention_couple.json)
This is simple custom node for [**ComfyUI**](https://github.com/comfyanonymous/ComfyUI) which helps to generate images of actual _couples_, easier.
比較用として、modeをLatentにすると、latent coupleになります。といってもこのノードで何かするわけではなく入力をそのまま出力するモードになるだけです。
If you want to draw two different characters together without blending their features, so you could try to check out this custom node.
# 注意点
実験段階でたぶん色々問題があります。
| ⭕ with Comfy Couple | ❌ without Comfy Couple |
| --- | --- |
| ![Ayaka x Lumine](docs/images/ayalumi-comfy-couple.png) | ![Ayaka x OC](docs/images/ayalumi-plain.png) |
| _Lumine with her own hair style_ | _Lumine with hair style of Ayaka_ |
1. どのプロンプトにも指定されていない領域があるとエラーが起こります。
2. LoRAの領域指定は実装していないし見当もつきません。
3. SDXLにおけるpooled outputは分割せずに1番目のプロンプトがそのまま入力されます。
It's fork of [**laksjdjf/attention-couple-ComfyUI**](https://github.com/laksjdjf/attention-couple-ComfyUI), but implementing shortcut for the most of required nodes.
# Latent couple(通常実装)との違い
Latent coupleはプロンプトの数だけUNetの計算が必要になります。Attention coupleは計算量が比較的小さいCross Attention層のみ複数回計算が必要になり、計算時間は単純な生成とほとんど変わりません。またUNet内部で領域分けすることでより自然な分割が期待できますが、UNet内は画像を縮小してしまう層があるため、細かい分割がしづらいという欠点もあるようです。
## Installation
# 参考
https://note.com/gcem156/n/nb3d516e376d7
1. Change directory to custom nodes of **ComfyUI**:
```bash
cd ~/ComfyUI/custom_nodes
```
2. Clone this repo here:
```bash
git clone https://github.com/Danand/ComfyUI-ComfyCouple.git
```
3. Restart **ComfyUI**.
## Usage
1. Right click in workflow.
2. Choose node: **loaders → Comfy Couple**
3. Connect inputs, connect outputs, notice **two** positive prompts for left side and right side of image respectively.
Example workflow is [here](workflows/workflow-comfy-couple.json).
## Known issues
It **is not** quite actual regional prompting.
## Comparison with [**laksjdjf/attention-couple-ComfyUI**](https://github.com/laksjdjf/attention-couple-ComfyUI)
Mask magic was replaced with comfy shortcut.
| Comfy Couple | attention-couple-ComfyUI |
| --- | --- |
| ![Comfy Couple workflow](docs/images/workflow-comfy-couple.svg) | ![attention-couple-ComfyUI workflow](docs/images/workflow-attention-couple.svg) |
## Credits
- [**@laksjdjf**](https://github.com/laksjdjf) – [original repo](https://github.com/laksjdjf/attention-couple-ComfyUI).
- [**@pythongosssss**](https://github.com/pythongosssss) – [ComfyUI-Custom-Scripts](https://github.com/pythongosssss/ComfyUI-Custom-Scripts) used for capturing SVG for `README.md`
- [**@Meina**](https://civitai.com/user/Meina) – [MeinaMix V11](https://civitai.com/models/7240/meinamix) used in example.
- [**@Numeratic**](https://civitai.com/user/Numeratic) – [Genshin Impact All In One](https://civitai.com/models/108649?modelVersionId=116970) used in example.
docs/images/ayalumi-comfy-couple.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 442 KiB

docs/images/ayalumi-plain.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 412 KiB

docs/images/workflow-attention-couple.svg
+1
View File
File diff suppressed because one or more lines are too long
Side by Side

After

Width:  |  Height:  |  Size: 1.4 MiB

docs/images/workflow-comfy-couple.svg
+1
View File
File diff suppressed because one or more lines are too long
Side by Side

After

Width:  |  Height:  |  Size: 391 KiB

workflows/workflow-comfy-couple.json
+593
View File
@@ -0,0 +1,593 @@
{
"last_node_id": 30,
"last_link_id": 71,
"nodes": [
{
"id": 10,
"type": "CLIPTextEncode",
"pos": [
-20,
-550
],
"size": {
"0": 425,
"1": 165
},
"flags": {},
"order": 4,
"mode": 0,
"inputs": [
{
"name": "clip",
"type": "CLIP",
"link": 70
}
],
"outputs": [
{
"name": "CONDITIONING",
"type": "CONDITIONING",
"links": [
58
],
"slot_index": 0
}
],
"properties": {
"Node name for S&R": "CLIPTextEncode"
},
"widgets_values": [
"holding hands, sitting on ground, picnic, starry sky, fireworks, (lumine:1.1), tender, cute, heartwarming, blushing, forest lake, scenery, 2girl"
]
},
{
"id": 4,
"type": "CheckpointLoaderSimple",
"pos": [
-965,
-890
],
"size": {
"0": 315,
"1": 100
},
"flags": {},
"order": 0,
"mode": 0,
"outputs": [
{
"name": "MODEL",
"type": "MODEL",
"links": [
66
],
"slot_index": 0
},
{
"name": "CLIP",
"type": "CLIP",
"links": [
67
],
"slot_index": 1
},
{
"name": "VAE",
"type": "VAE",
"links": [
8
],
"slot_index": 2
}
],
"properties": {
"Node name for S&R": "CheckpointLoaderSimple"
},
"widgets_values": [
"meinamix_meinaV11.safetensors"
]
},
{
"id": 30,
"type": "LoraLoader",
"pos": [
-605,
-890
],
"size": {
"0": 315,
"1": 126
},
"flags": {},
"order": 2,
"mode": 0,
"inputs": [
{
"name": "model",
"type": "MODEL",
"link": 66
},
{
"name": "clip",
"type": "CLIP",
"link": 67
}
],
"outputs": [
{
"name": "MODEL",
"type": "MODEL",
"links": [
68
],
"shape": 3,
"slot_index": 0
},
{
"name": "CLIP",
"type": "CLIP",
"links": [
69,
70,
71
],
"shape": 3,
"slot_index": 1
}
],
"properties": {
"Node name for S&R": "LoraLoader"
},
"widgets_values": [
"genshinfull1-000006.safetensors",
0.84,
0.84
]
},
{
"id": 6,
"type": "CLIPTextEncode",
"pos": [
-20,
-770
],
"size": {
"0": 425,
"1": 165
},
"flags": {},
"order": 3,
"mode": 0,
"inputs": [
{
"name": "clip",
"type": "CLIP",
"link": 69
}
],
"outputs": [
{
"name": "CONDITIONING",
"type": "CONDITIONING",
"links": [
57
],
"slot_index": 0
}
],
"properties": {
"Node name for S&R": "CLIPTextEncode"
},
"widgets_values": [
"holding hands, sitting on ground, picnic, starry sky, fireworks, (kamisatoayakadef:1.1), tender, cute, heartwarming, blushing, forest lake, scenery, 2girl"
]
},
{
"id": 7,
"type": "CLIPTextEncode",
"pos": [
-20,
-320
],
"size": {
"0": 425,
"1": 165
},
"flags": {},
"order": 5,
"mode": 0,
"inputs": [
{
"name": "clip",
"type": "CLIP",
"link": 71
}
],
"outputs": [
{
"name": "CONDITIONING",
"type": "CONDITIONING",
"links": [
59
],
"slot_index": 0
}
],
"properties": {
"Node name for S&R": "CLIPTextEncode"
},
"widgets_values": [
"(worst quality, low quality:1.64), (jpeg artifacts:1.4), (1boy, abs, muscular, rib:1.0), greyscale, monochrome, dusty sunbeams, trembling, motion lines, motion blur, emphasis lines, text, title, logo, signature, extra body parts, duplicated body parts, extra limbs, extra body parts, duplication, barcode, overexposed, low detailed"
]
},
{
"id": 5,
"type": "EmptyLatentImage",
"pos": [
585,
-535
],
"size": {
"0": 315,
"1": 106
},
"flags": {},
"order": 1,
"mode": 0,
"outputs": [
{
"name": "LATENT",
"type": "LATENT",
"links": [
2
],
"slot_index": 0
}
],
"properties": {
"Node name for S&R": "EmptyLatentImage"
},
"widgets_values": [
512,
512,
1
]
},
{
"id": 3,
"type": "KSampler",
"pos": [
1000,
-820
],
"size": {
"0": 315,
"1": 475
},
"flags": {},
"order": 7,
"mode": 0,
"inputs": [
{
"name": "model",
"type": "MODEL",
"link": 62
},
{
"name": "positive",
"type": "CONDITIONING",
"link": 60
},
{
"name": "negative",
"type": "CONDITIONING",
"link": 63
},
{
"name": "latent_image",
"type": "LATENT",
"link": 2
}
],
"outputs": [
{
"name": "LATENT",
"type": "LATENT",
"links": [
7
],
"slot_index": 0
}
],
"properties": {
"Node name for S&R": "KSampler"
},
"widgets_values": [
41,
"fixed",
30,
6.5,
"dpm_2",
"ddim_uniform",
0.98
]
},
{
"id": 8,
"type": "VAEDecode",
"pos": [
1390,
-320
],
"size": {
"0": 210,
"1": 46
},
"flags": {},
"order": 8,
"mode": 0,
"inputs": [
{
"name": "samples",
"type": "LATENT",
"link": 7
},
{
"name": "vae",
"type": "VAE",
"link": 8
}
],
"outputs": [
{
"name": "IMAGE",
"type": "IMAGE",
"links": [
65
],
"slot_index": 0
}
],
"properties": {
"Node name for S&R": "VAEDecode"
}
},
{
"id": 29,
"type": "SaveImage",
"pos": [
1660,
-820
],
"size": {
"0": 340,
"1": 320
},
"flags": {},
"order": 9,
"mode": 0,
"inputs": [
{
"name": "images",
"type": "IMAGE",
"link": 65
}
],
"properties": {},
"widgets_values": [
"ComfyUI"
]
},
{
"id": 28,
"type": "Comfy Couple",
"pos": [
585,
-820
],
"size": {
"0": 315,
"1": 215
},
"flags": {},
"order": 6,
"mode": 0,
"inputs": [
{
"name": "model",
"type": "MODEL",
"link": 68
},
{
"name": "positive_1",
"type": "CONDITIONING",
"link": 57
},
{
"name": "positive_2",
"type": "CONDITIONING",
"link": 58
},
{
"name": "negative",
"type": "CONDITIONING",
"link": 59
}
],
"outputs": [
{
"name": "MODEL",
"type": "MODEL",
"links": [
62
],
"shape": 3,
"slot_index": 0
},
{
"name": "CONDITIONING",
"type": "CONDITIONING",
"links": [
60
],
"shape": 3,
"slot_index": 1
},
{
"name": "CONDITIONING",
"type": "CONDITIONING",
"links": [
63
],
"shape": 3,
"slot_index": 2
}
],
"properties": {
"Node name for S&R": "Comfy Couple"
},
"widgets_values": [
"horizontal",
0.5,
512,
512
],
"color": "#322",
"bgcolor": "#533"
}
],
"links": [
[
2,
5,
0,
3,
3,
"LATENT"
],
[
7,
3,
0,
8,
0,
"LATENT"
],
[
8,
4,
2,
8,
1,
"VAE"
],
[
57,
6,
0,
28,
1,
"CONDITIONING"
],
[
58,
10,
0,
28,
2,
"CONDITIONING"
],
[
59,
7,
0,
28,
3,
"CONDITIONING"
],
[
60,
28,
1,
3,
1,
"CONDITIONING"
],
[
62,
28,
0,
3,
0,
"MODEL"
],
[
63,
28,
2,
3,
2,
"CONDITIONING"
],
[
65,
8,
0,
29,
0,
"IMAGE"
],
[
66,
4,
0,
30,
0,
"MODEL"
],
[
67,
4,
1,
30,
1,
"CLIP"
],
[
68,
30,
0,
28,
0,
"MODEL"
],
[
69,
30,
1,
6,
0,
"CLIP"
],
[
70,
30,
1,
10,
0,
"CLIP"
],
[
71,
30,
1,
7,
0,
"CLIP"
]
],
"groups": [],
"config": {},
"extra": {},
"version": 0.4
}