Upgrade stable version v1.1.9 to comfyregistry

Fix: ipadapterApplyEncoder & ipadapterApplyEmbeds Clip_Vision missing…

.gitignore

@@ -11,4 +11,5 @@ docs/**
 .vscode/
 .idea/
 mmb-preset.custom.txt
-config.yaml
+config.yaml
+node.tar.gz

README.en.md

@@ -9,7 +9,7 @@
 
 **ComfyUI-Easy-Use** is a simplified node integration package, which is extended on the basis of [tinyterraNodes](https://github.com/TinyTerra/ComfyUI_tinyterraNodes), and has been integrated and optimized for many mainstream node packages to achieve the purpose of faster and more convenient use of ComfyUI. While ensuring the degree of freedom, it restores the ultimate smooth image production experience that belongs to Stable Diffusion.
 
-<img src="https://raw.githubusercontent.com/yolain/yolain-comfyui-workflow/main/Docs/workflow_node_compare.png">
+[![ComfyUI-Yolain-Workflows](https://github.com/yolain/ComfyUI-Easy-Use/assets/73304135/9a3f54bc-a677-4bf1-a196-8845dd57c942)](https://github.com/yolain/ComfyUI-Yolain-Workflows)
 
 ## Introduce
 
@@ -30,9 +30,33 @@
 - Background removal nodes for the RMBG-1.4 model supporting BriaAI, [BriaAI Guide](https://huggingface.co/briaai/RMBG-1.4)
 - Forcibly cleared the memory usage of the comfy UI model are supported
 - Stable Diffusion 3 multi-account API nodes are supported
-- 
+- Support Stable Diffusion 3 model
+
+## Installation
+Clone the repo into the **custom_nodes** directory and install the requirements:
+```shell
+#1. Clone the repo
+git clone https://github.com/yolain/ComfyUI-Easy-Use
+#2. Install the requirements
+Double-click install.bat to install the required dependencies
+```
+
 ## Changelog
 
+**v1.1.9**
+
+- Added **gitsScheduler**
+- Added `easy imageBatchToImageList` and `easy imageListToImageBatch` 
+- Recursive subcategories nested for models
+- Support for Stable Diffusion 3 model
+- Added `easy applyInpaint` - All inpainting mode in this node
+
+**v1.1.8**
+
+- Added `easy controlnetStack`
+- Added `easy applyBrushNet` - [Workflow Example](https://github.com/yolain/ComfyUI-Yolain-Workflows/blob/main/workflows/2_advanced/2-4inpainting/2-4brushnet_1.1.8.json)
+- Added `easy applyPowerPaint` - [Workflow Example](https://github.com/yolain/ComfyUI-Yolain-Workflows/blob/main/workflows/2_advanced/2-4inpainting/2-4powerpaint_outpaint_1.1.8.json)
+
 **v1.1.7**
 
 - Added `easy prompt` - Subject and light presets, maybe adjusted later
@@ -334,33 +358,6 @@ Disclaimer: Opened source was not easy. I have a lot of respect for the contribu
 | easy imageChooser              | [cg-image-picker](https://github.com/chrisgoringe/cg-image-picker) | Preview Chooser         | 
 | easy styleAlignedBatchAlign              | [style_aligned_comfy](https://github.com/chrisgoringe/cg-image-picker) | styleAlignedBatchAlign       | 
 
-## Workflow Examples
-
-### Text to image
-
-<img src="https://raw.githubusercontent.com/yolain/yolain-comfyui-workflow/main/Workflows/Simple/text_to_image.png">
-
-### Image to image + controlnet
-
-<img src="https://raw.githubusercontent.com/yolain/yolain-comfyui-workflow/main/Workflows/Simple/image_to_image_controlnet.png">
-
-### SDTurbo + HiresFix + SVD
-
-<img src="https://raw.githubusercontent.com/yolain/yolain-comfyui-workflow/main/Workflows/Simple/sdturbo_hiresfix_svd.png">
-
-### LayerDiffusion
-#### SD15
-<img src="https://raw.githubusercontent.com/yolain/yolain-comfyui-workflow/main/Workflows/Simple/layer_diffusion_sd15.png">
-
-#### SDXL
-<img src="https://raw.githubusercontent.com/yolain/yolain-comfyui-workflow/main/Workflows/Simple/layer_diffusion_example.png">
-
-### StableCascade
-#### Text to image
-<img src="https://raw.githubusercontent.com/yolain/yolain-comfyui-workflow/main/Workflows/StableCascade/text_to_image.png">
-
-#### Image to image
-<img src="https://raw.githubusercontent.com/yolain/yolain-comfyui-workflow/main/Workflows/StableCascade/image_to_image.png">
 
 ## Credits
 

README.md

@@ -6,7 +6,7 @@
 
 # ComfyUI Easy Use
 
-[![Bilibili Badge](https://img.shields.io/badge/1.0版本-00A1D6?style=for-the-badge&logo=bilibili&logoColor=white&link=https://www.bilibili.com/video/BV1Wi4y1h76G)](https://www.bilibili.com/video/BV1Wi4y1h76G)
+[![Bilibili Badge](https://img.shields.io/badge/1.1版本-00A1D6?style=for-the-badge&logo=bilibili&logoColor=white&link=https://www.bilibili.com/video/BV1w6421F7Uv)](https://www.bilibili.com/video/BV1w6421F7Uv)
 [![Bilibili Badge](https://img.shields.io/badge/基本介绍-00A1D6?style=for-the-badge&logo=bilibili&logoColor=white&link=https://www.bilibili.com/video/BV1vQ4y1G7z7)](https://www.bilibili.com/video/BV1vQ4y1G7z7/)
 </div>
 
@@ -36,9 +36,35 @@
 - 支持 强制清理comfyUI模型显存占用
 - 支持Stable Diffusion 3 多账号API节点
 - 支持IC-Light的应用 [示例参考](https://github.com/yolain/ComfyUI-Yolain-Workflows?tab=readme-ov-file#2-5-ic-light) | [代码整合来源](https://github.com/huchenlei/ComfyUI-IC-Light) | [技术参考](https://github.com/lllyasviel/IC-Light)
+- 中文提示词自动识别，使用[opus-mt-zh-en模型](https://huggingface.co/Helsinki-NLP/opus-mt-zh-en)
+- 支持 sd3 模型 
+
+## 安装
+将存储库克隆到 **custom_nodes** 目录并安装依赖
+```shell
+#1. git下载
+git clone https://github.com/yolain/ComfyUI-Easy-Use
+#2. 安装依赖
+双击install.bat安装依赖
+```
 
 ## 更新日志
 
+**v1.1.9**
+
+- 增加 新的调度器 **gitsScheduler**
+- 增加 `easy imageBatchToImageList` 和 `easy imageListToImageBatch` (修复Impact版的一点小问题)
+- 递归模型子目录嵌套
+- 支持 sd3 模型 
+- 增加 `easy applyInpaint` - 局部重绘全模式节点 (相比与之前的kSamplerInpating节点逻辑会更合理些)
+
+**v1.1.8**
+
+- 增加中文提示词自动翻译，使用[opus-mt-zh-en模型](https://huggingface.co/Helsinki-NLP/opus-mt-zh-en), 默认已对wildcard、lora正则处理, 其他需要保留的中文，可使用`@你的提示词@`包裹 (若依赖安装完成后报错, 请重启)，测算大约会占0.3GB显存
+- 增加 `easy controlnetStack` - controlnet堆
+- 增加 `easy applyBrushNet` - [示例参考](https://github.com/yolain/ComfyUI-Yolain-Workflows/blob/main/workflows/2_advanced/2-4inpainting/2-4brushnet_1.1.8.json)
+- 增加 `easy applyPowerPaint` - [示例参考](https://github.com/yolain/ComfyUI-Yolain-Workflows/blob/main/workflows/2_advanced/2-4inpainting/2-4powerpaint_outpaint_1.1.8.json)
+
 **v1.1.7**
 
 - 修复 一些模型(如controlnet模型等)未成功写入缓存，导致修改前置节点束参数（如提示词）需要二次载入模型的问题

__init__.py

@@ -1,7 +1,6 @@
-__version__ = "1.1.7"
+__version__ = "1.1.9"
 
 import os
-import glob
 import folder_paths
 import importlib
 from pathlib import Path
@@ -26,7 +25,7 @@ cwd_path = os.path.dirname(os.path.realpath(__file__))
 comfy_path = folder_paths.base_path
 
 #Wildcards读取
-from .py.wildcards import read_wildcard_dict
+from .py.libs.wildcards import read_wildcard_dict
 wildcards_path = os.path.join(os.path.dirname(__file__), "wildcards")
 if os.path.exists(wildcards_path):
     read_wildcard_dict(wildcards_path)
@@ -43,36 +42,7 @@ else:
     os.mkdir(styles_path)
     os.mkdir(samples_path)
 
-#合并autocomplete覆盖到pyssss包
-pyssss_path = os.path.join(comfy_path, "custom_nodes", "ComfyUI-Custom-Scripts", "user")
-combine_folder = os.path.join(cwd_path, "autocomplete")
-if os.path.exists(combine_folder):
-    pass
-else:
-    os.mkdir(combine_folder)
-if os.path.exists(pyssss_path):
-    output_file = os.path.join(pyssss_path, "autocomplete.txt")
-    # 遍历 combine 目录下的所有 txt 文件，读取内容并合并
-    merged_content = ''
-    for file_path in glob.glob(os.path.join(combine_folder, '*.txt')):
-        with open(file_path, 'r', encoding='utf-8', errors='ignore') as file:
-            try:
-                file_content = file.read()
-                merged_content += file_content + '\n'
-            except UnicodeDecodeError:
-                pass
-    # 备份之前的autocomplete
-    # bak_file = os.path.join(pyssss_path, "autocomplete.txt.bak")
-    # if os.path.exists(bak_file):
-    #     pass
-    # elif os.path.exists(output_file):
-    #     shutil.copy(output_file, bak_file)
-    if merged_content != '':
-        # 将合并的内容写入目标文件 autocomplete.txt，并指定编码为 utf-8
-        with open(output_file, 'w', encoding='utf-8') as target_file:
-            target_file.write(merged_content)
-
-# ComfyUI-Easy-PS相关 (需要把模型预览图暴露给PS读取，此处借鉴了 AIGODLIKE-ComfyUI-Studio 的部分代码)
+# 需要把模型预览图暴露给PS读取，此处借鉴了 AIGODLIKE-ComfyUI-Studio 的部分代码
 from .py.libs.add_resources import add_static_resource
 from .py.libs.model import easyModelManager
 model_config = easyModelManager().models_config

install.bat

@@ -0,0 +1,16 @@
+@echo off
+
+set "requirements_txt=%~dp0\requirements.txt"
+set "python_exec=..\..\..\python_embeded\python.exe"
+
+echo Installing EasyUse Requirements...
+
+if exist "%python_exec%" (
+    echo Installing with ComfyUI Portable
+    "%python_exec%" -s -m pip install -r "%requirements_txt%"
+) else (
+    echo Installing with system Python
+    pip install -r "%requirements_txt%"
+)
+
+pause

prestartup_script.py

@@ -28,6 +28,7 @@ add_folder_path_and_extensions("ipadapter", [os.path.join(model_path, "ipadapter
 add_folder_path_and_extensions("dynamicrafter_models", [os.path.join(model_path, "dynamicrafter_models")], folder_paths.supported_pt_extensions)
 add_folder_path_and_extensions("mediapipe", [os.path.join(model_path, "mediapipe")], set(['.tflite','.pth']))
 add_folder_path_and_extensions("inpaint", [os.path.join(model_path, "inpaint")], folder_paths.supported_pt_extensions)
+add_folder_path_and_extensions("prompt_generator", [os.path.join(model_path, "prompt_generator")], folder_paths.supported_pt_extensions)
 
 add_folder_path_and_extensions("checkpoints_thumb", [os.path.join(model_path, "checkpoints")], image_suffixs)
 add_folder_path_and_extensions("loras_thumb", [os.path.join(model_path, "loras")], image_suffixs)

py/api.py

@@ -10,6 +10,8 @@ from .config import RESOURCES_DIR, FOOOCUS_STYLES_DIR, FOOOCUS_STYLES_SAMPLES
 from .logic import ConvertAnything
 from .libs.model import easyModelManager
 from .libs.utils import getMetadata, cleanGPUUsedForce, get_local_filepath
+from .libs.cache import remove_cache
+from .libs.translate import has_chinese, zh_to_en
 
 try:
     import aiohttp
@@ -23,11 +25,21 @@ except ImportError:
 def cleanGPU(request):
     try:
         cleanGPUUsedForce()
+        remove_cache('*')
         return web.Response(status=200)
     except Exception as e:
         return web.Response(status=500)
         pass
 
+@PromptServer.instance.routes.post("/easyuse/translate")
+async def translate(request):
+    post = await request.post()
+    text = post.get("text")
+    if has_chinese(text):
+        return web.json_response({"text": zh_to_en([text])[0]})
+    else:
+        return web.json_response({"text": text})
+
 @PromptServer.instance.routes.get("/easyuse/reboot")
 def reboot(request):
     try:

py/brushnet/__init__.py

@@ -0,0 +1,806 @@
+#credit to nullquant for this module
+#from https://github.com/nullquant/ComfyUI-BrushNet
+
+import os
+import types
+
+import torch
+from accelerate import init_empty_weights, load_checkpoint_and_dispatch
+
+import comfy
+
+from .model import BrushNetModel, PowerPaintModel
+from .model_patch import add_model_patch_option, patch_model_function_wrapper
+from .powerpaint_utils import TokenizerWrapper, add_tokens
+
+cwd_path = os.path.dirname(os.path.realpath(__file__))
+brushnet_config_file = os.path.join(cwd_path, 'config', 'brushnet.json')
+brushnet_xl_config_file = os.path.join(cwd_path, 'config', 'brushnet_xl.json')
+powerpaint_config_file = os.path.join(cwd_path, 'config', 'powerpaint.json')
+
+sd15_scaling_factor = 0.18215
+sdxl_scaling_factor = 0.13025
+
+ModelsToUnload = [comfy.sd1_clip.SD1ClipModel, comfy.ldm.models.autoencoder.AutoencoderKL]
+
+class BrushNet:
+
+    # Check models compatibility
+    def check_compatibilty(self, model, brushnet):
+        is_SDXL = False
+        is_PP = False
+        if isinstance(model.model.model_config, comfy.supported_models.SD15):
+            print('Base model type: SD1.5')
+            is_SDXL = False
+            if brushnet["SDXL"]:
+                raise Exception("Base model is SD15, but BrushNet is SDXL type")
+            if brushnet["PP"]:
+                is_PP = True
+        elif isinstance(model.model.model_config, comfy.supported_models.SDXL):
+            print('Base model type: SDXL')
+            is_SDXL = True
+            if not brushnet["SDXL"]:
+                raise Exception("Base model is SDXL, but BrushNet is SD15 type")
+        else:
+            print('Base model type: ', type(model.model.model_config))
+            raise Exception("Unsupported model type: " + str(type(model.model.model_config)))
+
+        return (is_SDXL, is_PP)
+
+    def check_image_mask(self, image, mask, name):
+        if len(image.shape) < 4:
+            # image tensor shape should be [B, H, W, C], but batch somehow is missing
+            image = image[None, :, :, :]
+
+        if len(mask.shape) > 3:
+            # mask tensor shape should be [B, H, W] but we get [B, H, W, C], image may be?
+            # take first mask, red channel
+            mask = (mask[:, :, :, 0])[:, :, :]
+        elif len(mask.shape) < 3:
+            # mask tensor shape should be [B, H, W] but batch somehow is missing
+            mask = mask[None, :, :]
+
+        if image.shape[0] > mask.shape[0]:
+            print(name, "gets batch of images (%d) but only %d masks" % (image.shape[0], mask.shape[0]))
+            if mask.shape[0] == 1:
+                print(name, "will copy the mask to fill batch")
+                mask = torch.cat([mask] * image.shape[0], dim=0)
+            else:
+                print(name, "will add empty masks to fill batch")
+                empty_mask = torch.zeros([image.shape[0] - mask.shape[0], mask.shape[1], mask.shape[2]])
+                mask = torch.cat([mask, empty_mask], dim=0)
+        elif image.shape[0] < mask.shape[0]:
+            print(name, "gets batch of images (%d) but too many (%d) masks" % (image.shape[0], mask.shape[0]))
+            mask = mask[:image.shape[0], :, :]
+
+        return (image, mask)
+
+    # Prepare image and mask
+    def prepare_image(self, image, mask):
+
+        image, mask = self.check_image_mask(image, mask, 'BrushNet')
+
+        print("BrushNet image.shape =", image.shape, "mask.shape =", mask.shape)
+
+        if mask.shape[2] != image.shape[2] or mask.shape[1] != image.shape[1]:
+            raise Exception("Image and mask should be the same size")
+
+        # As a suggestion of inferno46n2 (https://github.com/nullquant/ComfyUI-BrushNet/issues/64)
+        mask = mask.round()
+
+        masked_image = image * (1.0 - mask[:, :, :, None])
+
+        return (masked_image, mask)
+
+    # Get origin of the mask
+    def cut_with_mask(self, mask, width, height):
+        iy, ix = (mask == 1).nonzero(as_tuple=True)
+
+        h0, w0 = mask.shape
+
+        if iy.numel() == 0:
+            x_c = w0 / 2.0
+            y_c = h0 / 2.0
+        else:
+            x_min = ix.min().item()
+            x_max = ix.max().item()
+            y_min = iy.min().item()
+            y_max = iy.max().item()
+
+            if x_max - x_min > width or y_max - y_min > height:
+                raise Exception("Mask is bigger than provided dimensions")
+
+            x_c = (x_min + x_max) / 2.0
+            y_c = (y_min + y_max) / 2.0
+
+        width2 = width / 2.0
+        height2 = height / 2.0
+
+        if w0 <= width:
+            x0 = 0
+            w = w0
+        else:
+            x0 = max(0, x_c - width2)
+            w = width
+            if x0 + width > w0:
+                x0 = w0 - width
+
+        if h0 <= height:
+            y0 = 0
+            h = h0
+        else:
+            y0 = max(0, y_c - height2)
+            h = height
+            if y0 + height > h0:
+                y0 = h0 - height
+
+        return (int(x0), int(y0), int(w), int(h))
+
+    # Prepare conditioning_latents
+    @torch.inference_mode()
+    def get_image_latents(self, masked_image, mask, vae, scaling_factor):
+        processed_image = masked_image.to(vae.device)
+        image_latents = vae.encode(processed_image[:, :, :, :3]) * scaling_factor
+        processed_mask = 1. - mask[:, None, :, :]
+        interpolated_mask = torch.nn.functional.interpolate(
+            processed_mask,
+            size=(
+                image_latents.shape[-2],
+                image_latents.shape[-1]
+            )
+        )
+        interpolated_mask = interpolated_mask.to(image_latents.device)
+
+        conditioning_latents = [image_latents, interpolated_mask]
+
+        print('BrushNet CL: image_latents shape =', image_latents.shape, 'interpolated_mask shape =',
+              interpolated_mask.shape)
+
+        return conditioning_latents
+
+    def brushnet_blocks(self, sd):
+        brushnet_down_block = 0
+        brushnet_mid_block = 0
+        brushnet_up_block = 0
+        for key in sd:
+            if 'brushnet_down_block' in key:
+                brushnet_down_block += 1
+            if 'brushnet_mid_block' in key:
+                brushnet_mid_block += 1
+            if 'brushnet_up_block' in key:
+                brushnet_up_block += 1
+        return (brushnet_down_block, brushnet_mid_block, brushnet_up_block, len(sd))
+
+    def get_model_type(self, brushnet_file):
+        sd = comfy.utils.load_torch_file(brushnet_file)
+        brushnet_down_block, brushnet_mid_block, brushnet_up_block, keys = self.brushnet_blocks(sd)
+        del sd
+        if brushnet_down_block == 24 and brushnet_mid_block == 2 and brushnet_up_block == 30:
+            is_SDXL = False
+            if keys == 322:
+                is_PP = False
+                print('BrushNet model type: SD1.5')
+            else:
+                is_PP = True
+                print('PowerPaint model type: SD1.5')
+        elif brushnet_down_block == 18 and brushnet_mid_block == 2 and brushnet_up_block == 22:
+            print('BrushNet model type: Loading SDXL')
+            is_SDXL = True
+            is_PP = False
+        else:
+            raise Exception("Unknown BrushNet model")
+        return is_SDXL, is_PP
+
+    def load_brushnet_model(self, brushnet_file, dtype='float16'):
+        is_SDXL, is_PP = self.get_model_type(brushnet_file)
+        with init_empty_weights():
+            if is_SDXL:
+                brushnet_config = BrushNetModel.load_config(brushnet_xl_config_file)
+                brushnet_model = BrushNetModel.from_config(brushnet_config)
+            elif is_PP:
+                brushnet_config = PowerPaintModel.load_config(powerpaint_config_file)
+                brushnet_model = PowerPaintModel.from_config(brushnet_config)
+            else:
+                brushnet_config = BrushNetModel.load_config(brushnet_config_file)
+                brushnet_model = BrushNetModel.from_config(brushnet_config)
+        if is_PP:
+            print("PowerPaint model file:", brushnet_file)
+        else:
+            print("BrushNet model file:", brushnet_file)
+
+        if dtype == 'float16':
+            torch_dtype = torch.float16
+        elif dtype == 'bfloat16':
+            torch_dtype = torch.bfloat16
+        elif dtype == 'float32':
+            torch_dtype = torch.float32
+        else:
+            torch_dtype = torch.float64
+
+        brushnet_model = load_checkpoint_and_dispatch(
+            brushnet_model,
+            brushnet_file,
+            device_map="sequential",
+            max_memory=None,
+            offload_folder=None,
+            offload_state_dict=False,
+            dtype=torch_dtype,
+            force_hooks=False,
+        )
+
+        if is_PP:
+            print("PowerPaint model is loaded")
+        elif is_SDXL:
+            print("BrushNet SDXL model is loaded")
+        else:
+            print("BrushNet SD1.5 model is loaded")
+
+        return ({"brushnet": brushnet_model, "SDXL": is_SDXL, "PP": is_PP, "dtype": torch_dtype},)
+
+    def brushnet_model_update(self, model, vae, image, mask, brushnet, positive, negative, scale, start_at, end_at):
+
+        is_SDXL, is_PP = self.check_compatibilty(model, brushnet)
+
+        if is_PP:
+            raise Exception("PowerPaint model was loaded, please use PowerPaint node")
+
+            # Make a copy of the model so that we're not patching it everywhere in the workflow.
+        model = model.clone()
+
+        # prepare image and mask
+        # no batches for original image and mask
+        masked_image, mask = self.prepare_image(image, mask)
+
+        batch = masked_image.shape[0]
+        width = masked_image.shape[2]
+        height = masked_image.shape[1]
+
+        if hasattr(model.model.model_config, 'latent_format') and hasattr(model.model.model_config.latent_format,
+                                                                          'scale_factor'):
+            scaling_factor = model.model.model_config.latent_format.scale_factor
+        elif is_SDXL:
+            scaling_factor = sdxl_scaling_factor
+        else:
+            scaling_factor = sd15_scaling_factor
+
+        torch_dtype = brushnet['dtype']
+
+        # prepare conditioning latents
+        conditioning_latents = self.get_image_latents(masked_image, mask, vae, scaling_factor)
+        conditioning_latents[0] = conditioning_latents[0].to(dtype=torch_dtype).to(brushnet['brushnet'].device)
+        conditioning_latents[1] = conditioning_latents[1].to(dtype=torch_dtype).to(brushnet['brushnet'].device)
+
+        # unload vae
+        del vae
+        for loaded_model in comfy.model_management.current_loaded_models:
+            if type(loaded_model.model.model) in ModelsToUnload:
+                comfy.model_management.current_loaded_models.remove(loaded_model)
+                loaded_model.model_unload()
+                del loaded_model
+
+        # prepare embeddings
+        prompt_embeds = positive[0][0].to(dtype=torch_dtype).to(brushnet['brushnet'].device)
+        negative_prompt_embeds = negative[0][0].to(dtype=torch_dtype).to(brushnet['brushnet'].device)
+
+        max_tokens = max(prompt_embeds.shape[1], negative_prompt_embeds.shape[1])
+        if prompt_embeds.shape[1] < max_tokens:
+            multiplier = max_tokens // 77 - prompt_embeds.shape[1] // 77
+            prompt_embeds = torch.concat([prompt_embeds] + [prompt_embeds[:, -77:, :]] * multiplier, dim=1)
+            print('BrushNet: negative prompt more than 75 tokens:', negative_prompt_embeds.shape,
+                  'multiplying prompt_embeds')
+        if negative_prompt_embeds.shape[1] < max_tokens:
+            multiplier = max_tokens // 77 - negative_prompt_embeds.shape[1] // 77
+            negative_prompt_embeds = torch.concat(
+                [negative_prompt_embeds] + [negative_prompt_embeds[:, -77:, :]] * multiplier, dim=1)
+            print('BrushNet: positive prompt more than 75 tokens:', prompt_embeds.shape,
+                  'multiplying negative_prompt_embeds')
+
+        if len(positive[0]) > 1 and 'pooled_output' in positive[0][1] and positive[0][1]['pooled_output'] is not None:
+            pooled_prompt_embeds = positive[0][1]['pooled_output'].to(dtype=torch_dtype).to(brushnet['brushnet'].device)
+        else:
+            print('BrushNet: positive conditioning has not pooled_output')
+            if is_SDXL:
+                print('BrushNet will not produce correct results')
+            pooled_prompt_embeds = torch.empty([2, 1280], device=brushnet['brushnet'].device).to(dtype=torch_dtype)
+
+        if len(negative[0]) > 1 and 'pooled_output' in negative[0][1] and negative[0][1]['pooled_output'] is not None:
+            negative_pooled_prompt_embeds = negative[0][1]['pooled_output'].to(dtype=torch_dtype).to(
+                brushnet['brushnet'].device)
+        else:
+            print('BrushNet: negative conditioning has not pooled_output')
+            if is_SDXL:
+                print('BrushNet will not produce correct results')
+            negative_pooled_prompt_embeds = torch.empty([1, pooled_prompt_embeds.shape[1]],
+                                                        device=brushnet['brushnet'].device).to(dtype=torch_dtype)
+
+        time_ids = torch.FloatTensor([[height, width, 0., 0., height, width]]).to(dtype=torch_dtype).to(
+            brushnet['brushnet'].device)
+
+        if not is_SDXL:
+            pooled_prompt_embeds = None
+            negative_pooled_prompt_embeds = None
+            time_ids = None
+
+        # apply patch to model
+        brushnet_conditioning_scale = scale
+        control_guidance_start = start_at
+        control_guidance_end = end_at
+
+        add_brushnet_patch(model,
+                           brushnet['brushnet'],
+                           torch_dtype,
+                           conditioning_latents,
+                           (brushnet_conditioning_scale, control_guidance_start, control_guidance_end),
+                           prompt_embeds, negative_prompt_embeds,
+                           pooled_prompt_embeds, negative_pooled_prompt_embeds, time_ids,
+                           False)
+
+        latent = torch.zeros([batch, 4, conditioning_latents[0].shape[2], conditioning_latents[0].shape[3]],
+                             device=brushnet['brushnet'].device)
+
+        return (model, positive, negative, {"samples": latent},)
+
+    #powperpaint
+    def load_powerpaint_clip(self, base_clip_file, pp_clip_file):
+        pp_clip = comfy.sd.load_clip(ckpt_paths=[base_clip_file])
+
+        print('PowerPaint base CLIP file: ', base_clip_file)
+
+        pp_tokenizer = TokenizerWrapper(pp_clip.tokenizer.clip_l.tokenizer)
+        pp_text_encoder = pp_clip.patcher.model.clip_l.transformer
+
+        add_tokens(
+            tokenizer=pp_tokenizer,
+            text_encoder=pp_text_encoder,
+            placeholder_tokens=["P_ctxt", "P_shape", "P_obj"],
+            initialize_tokens=["a", "a", "a"],
+            num_vectors_per_token=10,
+        )
+
+        pp_text_encoder.load_state_dict(comfy.utils.load_torch_file(pp_clip_file), strict=False)
+
+        print('PowerPaint CLIP file: ', pp_clip_file)
+
+        pp_clip.tokenizer.clip_l.tokenizer = pp_tokenizer
+        pp_clip.patcher.model.clip_l.transformer = pp_text_encoder
+
+        return (pp_clip,)
+
+    def powerpaint_model_update(self, model, vae, image, mask, powerpaint, clip, positive, negative, fitting, function, scale, start_at, end_at, save_memory):
+        is_SDXL, is_PP = self.check_compatibilty(model, powerpaint)
+        if not is_PP:
+            raise Exception("BrushNet model was loaded, please use BrushNet node")
+
+            # Make a copy of the model so that we're not patching it everywhere in the workflow.
+        model = model.clone()
+
+        # prepare image and mask
+        # no batches for original image and mask
+        masked_image, mask = self.prepare_image(image, mask)
+
+        batch = masked_image.shape[0]
+        # width = masked_image.shape[2]
+        # height = masked_image.shape[1]
+
+        if hasattr(model.model.model_config, 'latent_format') and hasattr(model.model.model_config.latent_format,
+                                                                          'scale_factor'):
+            scaling_factor = model.model.model_config.latent_format.scale_factor
+        else:
+            scaling_factor = sd15_scaling_factor
+
+        torch_dtype = powerpaint['dtype']
+
+        # prepare conditioning latents
+        conditioning_latents = self.get_image_latents(masked_image, mask, vae, scaling_factor)
+        conditioning_latents[0] = conditioning_latents[0].to(dtype=torch_dtype).to(powerpaint['brushnet'].device)
+        conditioning_latents[1] = conditioning_latents[1].to(dtype=torch_dtype).to(powerpaint['brushnet'].device)
+
+        # prepare embeddings
+
+        if function == "object removal":
+            promptA = "P_ctxt"
+            promptB = "P_ctxt"
+            negative_promptA = "P_obj"
+            negative_promptB = "P_obj"
+            print('You should add to positive prompt: "empty scene blur"')
+            # positive = positive + " empty scene blur"
+        elif function == "context aware":
+            promptA = "P_ctxt"
+            promptB = "P_ctxt"
+            negative_promptA = ""
+            negative_promptB = ""
+            # positive = positive + " empty scene"
+            print('You should add to positive prompt: "empty scene"')
+        elif function == "shape guided":
+            promptA = "P_shape"
+            promptB = "P_ctxt"
+            negative_promptA = "P_shape"
+            negative_promptB = "P_ctxt"
+        elif function == "image outpainting":
+            promptA = "P_ctxt"
+            promptB = "P_ctxt"
+            negative_promptA = "P_obj"
+            negative_promptB = "P_obj"
+            # positive = positive + " empty scene"
+            print('You should add to positive prompt: "empty scene"')
+        else:
+            promptA = "P_obj"
+            promptB = "P_obj"
+            negative_promptA = "P_obj"
+            negative_promptB = "P_obj"
+
+        tokens = clip.tokenize(promptA)
+        prompt_embedsA = clip.encode_from_tokens(tokens, return_pooled=False)
+
+        tokens = clip.tokenize(negative_promptA)
+        negative_prompt_embedsA = clip.encode_from_tokens(tokens, return_pooled=False)
+
+        tokens = clip.tokenize(promptB)
+        prompt_embedsB = clip.encode_from_tokens(tokens, return_pooled=False)
+
+        tokens = clip.tokenize(negative_promptB)
+        negative_prompt_embedsB = clip.encode_from_tokens(tokens, return_pooled=False)
+
+        prompt_embeds_pp = (prompt_embedsA * fitting + (1.0 - fitting) * prompt_embedsB).to(dtype=torch_dtype).to(
+            powerpaint['brushnet'].device)
+        negative_prompt_embeds_pp = (negative_prompt_embedsA * fitting + (1.0 - fitting) * negative_prompt_embedsB).to(
+            dtype=torch_dtype).to(powerpaint['brushnet'].device)
+
+        # unload vae and CLIPs
+        del vae
+        del clip
+        for loaded_model in comfy.model_management.current_loaded_models:
+            if type(loaded_model.model.model) in ModelsToUnload:
+                comfy.model_management.current_loaded_models.remove(loaded_model)
+                loaded_model.model_unload()
+                del loaded_model
+
+        # apply patch to model
+
+        brushnet_conditioning_scale = scale
+        control_guidance_start = start_at
+        control_guidance_end = end_at
+
+        if save_memory != 'none':
+            powerpaint['brushnet'].set_attention_slice(save_memory)
+
+        add_brushnet_patch(model,
+                           powerpaint['brushnet'],
+                           torch_dtype,
+                           conditioning_latents,
+                           (brushnet_conditioning_scale, control_guidance_start, control_guidance_end),
+                           negative_prompt_embeds_pp, prompt_embeds_pp,
+                           None, None, None,
+                           False)
+
+        latent = torch.zeros([batch, 4, conditioning_latents[0].shape[2], conditioning_latents[0].shape[3]],
+                             device=powerpaint['brushnet'].device)
+
+        return (model, positive, negative, {"samples": latent},)
+@torch.inference_mode()
+def brushnet_inference(x, timesteps, transformer_options, debug):
+    if 'model_patch' not in transformer_options:
+        print('BrushNet inference: there is no model_patch key in transformer_options')
+        return ([], 0, [])
+    mp = transformer_options['model_patch']
+    if 'brushnet' not in mp:
+        print('BrushNet inference: there is no brushnet key in mdel_patch')
+        return ([], 0, [])
+    bo = mp['brushnet']
+    if 'model' not in bo:
+        print('BrushNet inference: there is no model key in brushnet')
+        return ([], 0, [])
+    brushnet = bo['model']
+    if not (isinstance(brushnet, BrushNetModel) or isinstance(brushnet, PowerPaintModel)):
+        print('BrushNet model is not a BrushNetModel class')
+        return ([], 0, [])
+
+    torch_dtype = bo['dtype']
+    cl_list = bo['latents']
+    brushnet_conditioning_scale, control_guidance_start, control_guidance_end = bo['controls']
+    pe = bo['prompt_embeds']
+    npe = bo['negative_prompt_embeds']
+    ppe, nppe, time_ids = bo['add_embeds']
+
+    #do_classifier_free_guidance = mp['free_guidance']
+    do_classifier_free_guidance = len(transformer_options['cond_or_uncond']) > 1
+
+    x = x.detach().clone()
+    x = x.to(torch_dtype).to(brushnet.device)
+
+    timesteps = timesteps.detach().clone()
+    timesteps = timesteps.to(torch_dtype).to(brushnet.device)
+
+    total_steps = mp['total_steps']
+    step = mp['step']
+
+    added_cond_kwargs = {}
+
+    if do_classifier_free_guidance and step == 0:
+        print('BrushNet inference: do_classifier_free_guidance is True')
+
+    sub_idx = None
+    if 'ad_params' in transformer_options and 'sub_idxs' in transformer_options['ad_params']:
+        sub_idx = transformer_options['ad_params']['sub_idxs']
+
+    # we have batch input images
+    batch = cl_list[0].shape[0]
+    # we have incoming latents
+    latents_incoming = x.shape[0]
+    # and we already got some
+    latents_got = bo['latent_id']
+    if step == 0 or batch > 1:
+        print('BrushNet inference, step = %d: image batch = %d, got %d latents, starting from %d' \
+                % (step, batch, latents_incoming, latents_got))
+
+    image_latents = []
+    masks = []
+    prompt_embeds = []
+    negative_prompt_embeds = []
+    pooled_prompt_embeds = []
+    negative_pooled_prompt_embeds = []
+    if sub_idx:
+        # AnimateDiff indexes detected
+        if step == 0:
+            print('BrushNet inference: AnimateDiff indexes detected and applied')
+
+        batch = len(sub_idx)
+
+        if do_classifier_free_guidance:
+            for i in sub_idx:
+                image_latents.append(cl_list[0][i][None,:,:,:])
+                masks.append(cl_list[1][i][None,:,:,:])
+                prompt_embeds.append(pe)
+                negative_prompt_embeds.append(npe)
+                pooled_prompt_embeds.append(ppe)
+                negative_pooled_prompt_embeds.append(nppe)
+            for i in sub_idx:
+                image_latents.append(cl_list[0][i][None,:,:,:])
+                masks.append(cl_list[1][i][None,:,:,:])
+        else:
+            for i in sub_idx:
+                image_latents.append(cl_list[0][i][None,:,:,:])
+                masks.append(cl_list[1][i][None,:,:,:])
+                prompt_embeds.append(pe)
+                pooled_prompt_embeds.append(ppe)
+    else:
+        # do_classifier_free_guidance = 2 passes, 1st pass is cond, 2nd is uncond
+        continue_batch = True
+        for i in range(latents_incoming):
+            number = latents_got + i
+            if number < batch:
+                # 1st pass, cond
+                image_latents.append(cl_list[0][number][None,:,:,:])
+                masks.append(cl_list[1][number][None,:,:,:])
+                prompt_embeds.append(pe)
+                pooled_prompt_embeds.append(ppe)
+            elif do_classifier_free_guidance and number < batch * 2:
+                # 2nd pass, uncond
+                image_latents.append(cl_list[0][number-batch][None,:,:,:])
+                masks.append(cl_list[1][number-batch][None,:,:,:])
+                negative_prompt_embeds.append(npe)
+                negative_pooled_prompt_embeds.append(nppe)
+            else:
+                # latent batch
+                image_latents.append(cl_list[0][0][None,:,:,:])
+                masks.append(cl_list[1][0][None,:,:,:])
+                prompt_embeds.append(pe)
+                pooled_prompt_embeds.append(ppe)
+                latents_got = -i
+                continue_batch = False
+
+        if continue_batch:
+            # we don't have full batch yet
+            if do_classifier_free_guidance:
+                if number < batch * 2 - 1:
+                    bo['latent_id'] = number + 1
+                else:
+                    bo['latent_id'] = 0
+            else:
+                if number < batch - 1:
+                    bo['latent_id'] = number + 1
+                else:
+                    bo['latent_id'] = 0
+        else:
+            bo['latent_id'] = 0
+
+    cl = []
+    for il, m in zip(image_latents, masks):
+        cl.append(torch.concat([il, m], dim=1))
+    cl2apply = torch.concat(cl, dim=0)
+
+    conditioning_latents = cl2apply.to(torch_dtype).to(brushnet.device)
+
+    prompt_embeds.extend(negative_prompt_embeds)
+    prompt_embeds = torch.concat(prompt_embeds, dim=0).to(torch_dtype).to(brushnet.device)
+
+    if ppe is not None:
+        added_cond_kwargs = {}
+        added_cond_kwargs['time_ids'] = torch.concat([time_ids] * latents_incoming, dim = 0).to(torch_dtype).to(brushnet.device)
+
+        pooled_prompt_embeds.extend(negative_pooled_prompt_embeds)
+        pooled_prompt_embeds = torch.concat(pooled_prompt_embeds, dim=0).to(torch_dtype).to(brushnet.device)
+        added_cond_kwargs['text_embeds'] = pooled_prompt_embeds
+    else:
+        added_cond_kwargs = None
+
+    if x.shape[2] != conditioning_latents.shape[2] or x.shape[3] != conditioning_latents.shape[3]:
+        if step == 0:
+            print('BrushNet inference: image', conditioning_latents.shape, 'and latent', x.shape, 'have different size, resizing image')
+        conditioning_latents = torch.nn.functional.interpolate(
+            conditioning_latents, size=(
+                x.shape[2],
+                x.shape[3],
+            ), mode='bicubic',
+        ).to(torch_dtype).to(brushnet.device)
+
+    if step == 0:
+        print('BrushNet inference: sample', x.shape, ', CL', conditioning_latents.shape, 'dtype', torch_dtype)
+
+    if debug: print('BrushNet: step =', step)
+
+    if step < control_guidance_start or step > control_guidance_end:
+        cond_scale = 0.0
+    else:
+        cond_scale = brushnet_conditioning_scale
+
+    return brushnet(x,
+        encoder_hidden_states=prompt_embeds,
+        brushnet_cond=conditioning_latents,
+        timestep = timesteps,
+        conditioning_scale=cond_scale,
+        guess_mode=False,
+        added_cond_kwargs=added_cond_kwargs,
+        return_dict=False,
+        debug=debug,
+    )
+
+def add_brushnet_patch(model, brushnet, torch_dtype, conditioning_latents,
+                       controls,
+                       prompt_embeds, negative_prompt_embeds,
+                       pooled_prompt_embeds, negative_pooled_prompt_embeds, time_ids,
+                       debug):
+
+    is_SDXL = isinstance(model.model.model_config, comfy.supported_models.SDXL)
+
+    if is_SDXL:
+        input_blocks = [[0, comfy.ops.disable_weight_init.Conv2d],
+                        [1, comfy.ldm.modules.diffusionmodules.openaimodel.ResBlock],
+                        [2, comfy.ldm.modules.diffusionmodules.openaimodel.ResBlock],
+                        [3, comfy.ldm.modules.diffusionmodules.openaimodel.Downsample],
+                        [4, comfy.ldm.modules.attention.SpatialTransformer],
+                        [5, comfy.ldm.modules.attention.SpatialTransformer],
+                        [6, comfy.ldm.modules.diffusionmodules.openaimodel.Downsample],
+                        [7, comfy.ldm.modules.attention.SpatialTransformer],
+                        [8, comfy.ldm.modules.attention.SpatialTransformer]]
+        middle_block = [0, comfy.ldm.modules.diffusionmodules.openaimodel.ResBlock]
+        output_blocks = [[0, comfy.ldm.modules.attention.SpatialTransformer],
+                         [1, comfy.ldm.modules.attention.SpatialTransformer],
+                         [2, comfy.ldm.modules.attention.SpatialTransformer],
+                         [2, comfy.ldm.modules.diffusionmodules.openaimodel.Upsample],
+                         [3, comfy.ldm.modules.attention.SpatialTransformer],
+                         [4, comfy.ldm.modules.attention.SpatialTransformer],
+                         [5, comfy.ldm.modules.attention.SpatialTransformer],
+                         [5, comfy.ldm.modules.diffusionmodules.openaimodel.Upsample],
+                         [6, comfy.ldm.modules.diffusionmodules.openaimodel.ResBlock],
+                         [7, comfy.ldm.modules.diffusionmodules.openaimodel.ResBlock],
+                         [8, comfy.ldm.modules.diffusionmodules.openaimodel.ResBlock]]
+    else:
+        input_blocks = [[0, comfy.ops.disable_weight_init.Conv2d],
+                        [1, comfy.ldm.modules.attention.SpatialTransformer],
+                        [2, comfy.ldm.modules.attention.SpatialTransformer],
+                        [3, comfy.ldm.modules.diffusionmodules.openaimodel.Downsample],
+                        [4, comfy.ldm.modules.attention.SpatialTransformer],
+                        [5, comfy.ldm.modules.attention.SpatialTransformer],
+                        [6, comfy.ldm.modules.diffusionmodules.openaimodel.Downsample],
+                        [7, comfy.ldm.modules.attention.SpatialTransformer],
+                        [8, comfy.ldm.modules.attention.SpatialTransformer],
+                        [9, comfy.ldm.modules.diffusionmodules.openaimodel.Downsample],
+                        [10, comfy.ldm.modules.diffusionmodules.openaimodel.ResBlock],
+                        [11, comfy.ldm.modules.diffusionmodules.openaimodel.ResBlock]]
+        middle_block = [0, comfy.ldm.modules.diffusionmodules.openaimodel.ResBlock]
+        output_blocks = [[0, comfy.ldm.modules.diffusionmodules.openaimodel.ResBlock],
+                         [1, comfy.ldm.modules.diffusionmodules.openaimodel.ResBlock],
+                         [2, comfy.ldm.modules.diffusionmodules.openaimodel.ResBlock],
+                         [2, comfy.ldm.modules.diffusionmodules.openaimodel.Upsample],
+                         [3, comfy.ldm.modules.attention.SpatialTransformer],
+                         [4, comfy.ldm.modules.attention.SpatialTransformer],
+                         [5, comfy.ldm.modules.attention.SpatialTransformer],
+                         [5, comfy.ldm.modules.diffusionmodules.openaimodel.Upsample],
+                         [6, comfy.ldm.modules.attention.SpatialTransformer],
+                         [7, comfy.ldm.modules.attention.SpatialTransformer],
+                         [8, comfy.ldm.modules.attention.SpatialTransformer],
+                         [8, comfy.ldm.modules.diffusionmodules.openaimodel.Upsample],
+                         [9, comfy.ldm.modules.attention.SpatialTransformer],
+                         [10, comfy.ldm.modules.attention.SpatialTransformer],
+                         [11, comfy.ldm.modules.attention.SpatialTransformer]]
+
+    def last_layer_index(block, tp):
+        layer_list = []
+        for layer in block:
+            layer_list.append(type(layer))
+        layer_list.reverse()
+        if tp not in layer_list:
+            return -1, layer_list.reverse()
+        return len(layer_list) - 1 - layer_list.index(tp), layer_list
+
+    def brushnet_forward(model, x, timesteps, transformer_options, control):
+        if 'brushnet' not in transformer_options['model_patch']:
+            input_samples = []
+            mid_sample = 0
+            output_samples = []
+        else:
+            # brushnet inference
+            input_samples, mid_sample, output_samples = brushnet_inference(x, timesteps, transformer_options, debug)
+
+        # give additional samples to blocks
+        for i, tp in input_blocks:
+            idx, layer_list = last_layer_index(model.input_blocks[i], tp)
+            if idx < 0:
+                print("BrushNet can't find", tp, "layer in", i, "input block:", layer_list)
+                continue
+            model.input_blocks[i][idx].add_sample_after = input_samples.pop(0) if input_samples else 0
+
+        idx, layer_list = last_layer_index(model.middle_block, middle_block[1])
+        if idx < 0:
+            print("BrushNet can't find", middle_block[1], "layer in middle block", layer_list)
+        model.middle_block[idx].add_sample_after = mid_sample
+
+        for i, tp in output_blocks:
+            idx, layer_list = last_layer_index(model.output_blocks[i], tp)
+            if idx < 0:
+                print("BrushNet can't find", tp, "layer in", i, "outnput block:", layer_list)
+                continue
+            model.output_blocks[i][idx].add_sample_after = output_samples.pop(0) if output_samples else 0
+
+    patch_model_function_wrapper(model, brushnet_forward)
+
+    to = add_model_patch_option(model)
+    mp = to['model_patch']
+    if 'brushnet' not in mp:
+        mp['brushnet'] = {}
+    bo = mp['brushnet']
+
+    bo['model'] = brushnet
+    bo['dtype'] = torch_dtype
+    bo['latents'] = conditioning_latents
+    bo['controls'] = controls
+    bo['prompt_embeds'] = prompt_embeds
+    bo['negative_prompt_embeds'] = negative_prompt_embeds
+    bo['add_embeds'] = (pooled_prompt_embeds, negative_pooled_prompt_embeds, time_ids)
+    bo['latent_id'] = 0
+
+    # patch layers `forward` so we can apply brushnet
+    def forward_patched_by_brushnet(self, x, *args, **kwargs):
+        h = self.original_forward(x, *args, **kwargs)
+        if hasattr(self, 'add_sample_after') and type(self):
+            to_add = self.add_sample_after
+            if torch.is_tensor(to_add):
+                # interpolate due to RAUNet
+                if h.shape[2] != to_add.shape[2] or h.shape[3] != to_add.shape[3]:
+                    to_add = torch.nn.functional.interpolate(to_add, size=(h.shape[2], h.shape[3]), mode='bicubic')
+                h += to_add.to(h.dtype).to(h.device)
+            else:
+                h += self.add_sample_after
+            self.add_sample_after = 0
+        return h
+
+    for i, block in enumerate(model.model.diffusion_model.input_blocks):
+        for j, layer in enumerate(block):
+            if not hasattr(layer, 'original_forward'):
+                layer.original_forward = layer.forward
+            layer.forward = types.MethodType(forward_patched_by_brushnet, layer)
+            layer.add_sample_after = 0
+
+    for j, layer in enumerate(model.model.diffusion_model.middle_block):
+        if not hasattr(layer, 'original_forward'):
+            layer.original_forward = layer.forward
+        layer.forward = types.MethodType(forward_patched_by_brushnet, layer)
+        layer.add_sample_after = 0
+
+    for i, block in enumerate(model.model.diffusion_model.output_blocks):
+        for j, layer in enumerate(block):
+            if not hasattr(layer, 'original_forward'):
+                layer.original_forward = layer.forward
+            layer.forward = types.MethodType(forward_patched_by_brushnet, layer)
+            layer.add_sample_after = 0

py/brushnet/config/brushnet.json

@@ -0,0 +1,58 @@
+{
+  "_class_name": "BrushNetModel",
+  "_diffusers_version": "0.27.0.dev0",
+  "_name_or_path": "runs/logs/brushnet_randommask/checkpoint-100000",
+  "act_fn": "silu",
+  "addition_embed_type": null,
+  "addition_embed_type_num_heads": 64,
+  "addition_time_embed_dim": null,
+  "attention_head_dim": 8,
+  "block_out_channels": [
+    320,
+    640,
+    1280,
+    1280
+  ],
+  "brushnet_conditioning_channel_order": "rgb",
+  "class_embed_type": null,
+  "conditioning_channels": 5,
+  "conditioning_embedding_out_channels": [
+    16,
+    32,
+    96,
+    256
+  ],
+  "cross_attention_dim": 768,
+  "down_block_types": [
+    "DownBlock2D",
+    "DownBlock2D",
+    "DownBlock2D",
+    "DownBlock2D"
+  ],
+  "downsample_padding": 1,
+  "encoder_hid_dim": null,
+  "encoder_hid_dim_type": null,
+  "flip_sin_to_cos": true,
+  "freq_shift": 0,
+  "global_pool_conditions": false,
+  "in_channels": 4,
+  "layers_per_block": 2,
+  "mid_block_scale_factor": 1,
+  "mid_block_type": "MidBlock2D",
+  "norm_eps": 1e-05,
+  "norm_num_groups": 32,
+  "num_attention_heads": null,
+  "num_class_embeds": null,
+  "only_cross_attention": false,
+  "projection_class_embeddings_input_dim": null,
+  "resnet_time_scale_shift": "default",
+  "transformer_layers_per_block": 1,
+  "up_block_types": [
+    "UpBlock2D",
+    "UpBlock2D",
+    "UpBlock2D",
+    "UpBlock2D"
+  ],
+  "upcast_attention": false,
+  "use_linear_projection": false
+}

py/brushnet/config/brushnet_xl.json

@@ -0,0 +1,63 @@
+{
+  "_class_name": "BrushNetModel",
+  "_diffusers_version": "0.27.0.dev0",
+  "_name_or_path": "runs/logs/brushnetsdxl_randommask/checkpoint-80000",
+  "act_fn": "silu",
+  "addition_embed_type": "text_time",
+  "addition_embed_type_num_heads": 64,
+  "addition_time_embed_dim": 256,
+  "attention_head_dim": [
+    5,
+    10,
+    20
+  ],
+  "block_out_channels": [
+    320,
+    640,
+    1280
+  ],
+  "brushnet_conditioning_channel_order": "rgb",
+  "class_embed_type": null,
+  "conditioning_channels": 5,
+  "conditioning_embedding_out_channels": [
+    16,
+    32,
+    96,
+    256
+  ],
+  "cross_attention_dim": 2048,
+  "down_block_types": [
+    "DownBlock2D",
+    "DownBlock2D",
+    "DownBlock2D"
+  ],
+  "downsample_padding": 1,
+  "encoder_hid_dim": null,
+  "encoder_hid_dim_type": null,
+  "flip_sin_to_cos": true,
+  "freq_shift": 0,
+  "global_pool_conditions": false,
+  "in_channels": 4,
+  "layers_per_block": 2,
+  "mid_block_scale_factor": 1,
+  "mid_block_type": "MidBlock2D",
+  "norm_eps": 1e-05,
+  "norm_num_groups": 32,
+  "num_attention_heads": null,
+  "num_class_embeds": null,
+  "only_cross_attention": false,
+  "projection_class_embeddings_input_dim": 2816,
+  "resnet_time_scale_shift": "default",
+  "transformer_layers_per_block": [
+    1,
+    2,
+    10
+  ],
+  "up_block_types": [
+    "UpBlock2D",
+    "UpBlock2D",
+    "UpBlock2D"
+  ],
+  "upcast_attention": null,
+  "use_linear_projection": true
+}

py/brushnet/config/powerpaint.json

@@ -0,0 +1,57 @@
+{
+  "_class_name": "BrushNetModel",
+  "_diffusers_version": "0.27.2",
+  "act_fn": "silu",
+  "addition_embed_type": null,
+  "addition_embed_type_num_heads": 64,
+  "addition_time_embed_dim": null,
+  "attention_head_dim": 8,
+  "block_out_channels": [
+    320,
+    640,
+    1280,
+    1280
+  ],
+  "brushnet_conditioning_channel_order": "rgb",
+  "class_embed_type": null,
+  "conditioning_channels": 5,
+  "conditioning_embedding_out_channels": [
+    16,
+    32,
+    96,
+    256
+  ],
+  "cross_attention_dim": 768,
+  "down_block_types": [
+    "CrossAttnDownBlock2D",
+    "CrossAttnDownBlock2D",
+    "CrossAttnDownBlock2D",
+    "DownBlock2D"
+  ],
+  "downsample_padding": 1,
+  "encoder_hid_dim": null,
+  "encoder_hid_dim_type": null,
+  "flip_sin_to_cos": true,
+  "freq_shift": 0,
+  "global_pool_conditions": false,
+  "in_channels": 4,
+  "layers_per_block": 2,
+  "mid_block_scale_factor": 1,
+  "mid_block_type": "UNetMidBlock2DCrossAttn",
+  "norm_eps": 1e-05,
+  "norm_num_groups": 32,
+  "num_attention_heads": null,
+  "num_class_embeds": null,
+  "only_cross_attention": false,
+  "projection_class_embeddings_input_dim": null,
+  "resnet_time_scale_shift": "default",
+  "transformer_layers_per_block": 1,
+  "up_block_types": [
+    "UpBlock2D",
+    "CrossAttnUpBlock2D",
+    "CrossAttnUpBlock2D",
+    "CrossAttnUpBlock2D"
+  ],
+  "upcast_attention": false,
+  "use_linear_projection": false
+}

py/brushnet/model_patch.py

@@ -0,0 +1,137 @@
+import torch
+import comfy
+
+# Check and add 'model_patch' to model.model_options['transformer_options']
+def add_model_patch_option(model):
+    if 'transformer_options' not in model.model_options:
+        model.model_options['transformer_options'] = {}
+    to = model.model_options['transformer_options']
+    if "model_patch" not in to:
+        to["model_patch"] = {}
+    return to
+
+
+# Patch model with model_function_wrapper
+def patch_model_function_wrapper(model, forward_patch, remove=False):
+    def brushnet_model_function_wrapper(apply_model_method, options_dict):
+        to = options_dict['c']['transformer_options']
+
+        control = None
+        if 'control' in options_dict['c']:
+            control = options_dict['c']['control']
+
+        x = options_dict['input']
+        timestep = options_dict['timestep']
+
+        # check if there are patches to execute
+        if 'model_patch' not in to or 'forward' not in to['model_patch']:
+            return apply_model_method(x, timestep, **options_dict['c'])
+
+        mp = to['model_patch']
+        unet = mp['unet']
+
+        all_sigmas = mp['all_sigmas']
+        sigma = to['sigmas'][0].item()
+        total_steps = all_sigmas.shape[0] - 1
+        step = torch.argmin((all_sigmas - sigma).abs()).item()
+
+        mp['step'] = step
+        mp['total_steps'] = total_steps
+
+        # comfy.model_base.apply_model
+        xc = model.model.model_sampling.calculate_input(timestep, x)
+        if 'c_concat' in options_dict['c'] and options_dict['c']['c_concat'] is not None:
+            xc = torch.cat([xc] + [options_dict['c']['c_concat']], dim=1)
+        t = model.model.model_sampling.timestep(timestep).float()
+        # execute all patches
+        for method in mp['forward']:
+            method(unet, xc, t, to, control)
+
+        return apply_model_method(x, timestep, **options_dict['c'])
+
+    if "model_function_wrapper" in model.model_options and model.model_options["model_function_wrapper"]:
+        print('BrushNet is going to replace existing model_function_wrapper:',
+              model.model_options["model_function_wrapper"])
+    model.set_model_unet_function_wrapper(brushnet_model_function_wrapper)
+
+    to = add_model_patch_option(model)
+    mp = to['model_patch']
+
+    if isinstance(model.model.model_config, comfy.supported_models.SD15):
+        mp['SDXL'] = False
+    elif isinstance(model.model.model_config, comfy.supported_models.SDXL):
+        mp['SDXL'] = True
+    else:
+        print('Base model type: ', type(model.model.model_config))
+        raise Exception("Unsupported model type: ", type(model.model.model_config))
+
+    if 'forward' not in mp:
+        mp['forward'] = []
+
+    if remove:
+        if forward_patch in mp['forward']:
+            mp['forward'].remove(forward_patch)
+    else:
+        mp['forward'].append(forward_patch)
+
+    mp['unet'] = model.model.diffusion_model
+    mp['step'] = 0
+    mp['total_steps'] = 1
+
+    # apply patches to code
+    if comfy.samplers.sample.__doc__ is None or 'BrushNet' not in comfy.samplers.sample.__doc__:
+        comfy.samplers.original_sample = comfy.samplers.sample
+        comfy.samplers.sample = modified_sample
+
+    if comfy.ldm.modules.diffusionmodules.openaimodel.apply_control.__doc__ is None or \
+            'BrushNet' not in comfy.ldm.modules.diffusionmodules.openaimodel.apply_control.__doc__:
+        comfy.ldm.modules.diffusionmodules.openaimodel.original_apply_control = comfy.ldm.modules.diffusionmodules.openaimodel.apply_control
+        comfy.ldm.modules.diffusionmodules.openaimodel.apply_control = modified_apply_control
+
+
+# Model needs current step number and cfg at inference step. It is possible to write a custom KSampler but I'd like to use ComfyUI's one.
+# The first versions had modified_common_ksampler, but it broke custom KSampler nodes
+def modified_sample(model, noise, positive, negative, cfg, device, sampler, sigmas, model_options={},
+                    latent_image=None, denoise_mask=None, callback=None, disable_pbar=False, seed=None):
+    ''' Modified by BrushNet nodes'''
+    cfg_guider = comfy.samplers.CFGGuider(model)
+    cfg_guider.set_conds(positive, negative)
+    cfg_guider.set_cfg(cfg)
+
+    ### Modified part ######################################################################
+    to = add_model_patch_option(model)
+    to['model_patch']['all_sigmas'] = sigmas
+    #######################################################################################
+
+    return cfg_guider.sample(noise, latent_image, sampler, sigmas, denoise_mask, callback, disable_pbar, seed)
+
+# To use Controlnet with RAUNet it is much easier to modify apply_control a little
+def modified_apply_control(h, control, name):
+    '''Modified by BrushNet nodes'''
+    if control is not None and name in control and len(control[name]) > 0:
+        ctrl = control[name].pop()
+        if ctrl is not None:
+            if h.shape[2] != ctrl.shape[2] or h.shape[3] != ctrl.shape[3]:
+                ctrl = torch.nn.functional.interpolate(ctrl, size=(h.shape[2], h.shape[3]), mode='bicubic').to(
+                    h.dtype).to(h.device)
+            try:
+                h += ctrl
+            except:
+                print.warning("warning control could not be applied {} {}".format(h.shape, ctrl.shape))
+    return h
+
+def add_model_patch(model):
+    to = add_model_patch_option(model)
+    mp = to['model_patch']
+    if "brushnet" in mp:
+        if isinstance(model.model.model_config, comfy.supported_models.SD15):
+            mp['SDXL'] = False
+        elif isinstance(model.model.model_config, comfy.supported_models.SDXL):
+            mp['SDXL'] = True
+        else:
+            print('Base model type: ', type(model.model.model_config))
+            raise Exception("Unsupported model type: ", type(model.model.model_config))
+
+        mp['unet'] = model.model.diffusion_model
+        mp['step'] = 0
+        mp['total_steps'] = 1

py/brushnet/powerpaint_utils.py

@@ -0,0 +1,467 @@
+import copy
+import random
+
+import torch
+import torch.nn as nn
+from transformers import CLIPTokenizer
+from typing import Any, List, Optional, Union
+
+
+class TokenizerWrapper:
+    """Tokenizer wrapper for CLIPTokenizer. Only support CLIPTokenizer
+    currently. This wrapper is modified from https://github.com/huggingface/dif
+    fusers/blob/e51f19aee82c8dd874b715a09dbc521d88835d68/src/diffusers/loaders.
+    py#L358  # noqa.
+
+    Args:
+        from_pretrained (Union[str, os.PathLike], optional): The *model id*
+            of a pretrained model or a path to a *directory* containing
+            model weights and config. Defaults to None.
+        from_config (Union[str, os.PathLike], optional): The *model id*
+            of a pretrained model or a path to a *directory* containing
+            model weights and config. Defaults to None.
+
+        *args, **kwargs: If `from_pretrained` is passed, *args and **kwargs
+            will be passed to `from_pretrained` function. Otherwise, *args
+            and **kwargs will be used to initialize the model by
+            `self._module_cls(*args, **kwargs)`.
+    """
+
+    def __init__(self, tokenizer: CLIPTokenizer):
+        self.wrapped = tokenizer
+        self.token_map = {}
+
+    def __getattr__(self, name: str) -> Any:
+        if name in self.__dict__:
+            return getattr(self, name)
+        # if name == "wrapped":
+        #    return getattr(self, 'wrapped')#super().__getattr__("wrapped")
+
+        try:
+            return getattr(self.wrapped, name)
+        except AttributeError:
+            raise AttributeError(
+                "'name' cannot be found in both "
+                f"'{self.__class__.__name__}' and "
+                f"'{self.__class__.__name__}.tokenizer'."
+            )
+
+    def try_adding_tokens(self, tokens: Union[str, List[str]], *args, **kwargs):
+        """Attempt to add tokens to the tokenizer.
+
+        Args:
+            tokens (Union[str, List[str]]): The tokens to be added.
+        """
+        num_added_tokens = self.wrapped.add_tokens(tokens, *args, **kwargs)
+        assert num_added_tokens != 0, (
+            f"The tokenizer already contains the token {tokens}. Please pass "
+            "a different `placeholder_token` that is not already in the "
+            "tokenizer."
+        )
+
+    def get_token_info(self, token: str) -> dict:
+        """Get the information of a token, including its start and end index in
+        the current tokenizer.
+
+        Args:
+            token (str): The token to be queried.
+
+        Returns:
+            dict: The information of the token, including its start and end
+                index in current tokenizer.
+        """
+        token_ids = self.__call__(token).input_ids
+        start, end = token_ids[1], token_ids[-2] + 1
+        return {"name": token, "start": start, "end": end}
+
+    def add_placeholder_token(self, placeholder_token: str, *args, num_vec_per_token: int = 1, **kwargs):
+        """Add placeholder tokens to the tokenizer.
+
+        Args:
+            placeholder_token (str): The placeholder token to be added.
+            num_vec_per_token (int, optional): The number of vectors of
+                the added placeholder token.
+            *args, **kwargs: The arguments for `self.wrapped.add_tokens`.
+        """
+        output = []
+        if num_vec_per_token == 1:
+            self.try_adding_tokens(placeholder_token, *args, **kwargs)
+            output.append(placeholder_token)
+        else:
+            output = []
+            for i in range(num_vec_per_token):
+                ith_token = placeholder_token + f"_{i}"
+                self.try_adding_tokens(ith_token, *args, **kwargs)
+                output.append(ith_token)
+
+        for token in self.token_map:
+            if token in placeholder_token:
+                raise ValueError(
+                    f"The tokenizer already has placeholder token {token} "
+                    f"that can get confused with {placeholder_token} "
+                    "keep placeholder tokens independent"
+                )
+        self.token_map[placeholder_token] = output
+
+    def replace_placeholder_tokens_in_text(
+            self, text: Union[str, List[str]], vector_shuffle: bool = False, prop_tokens_to_load: float = 1.0
+    ) -> Union[str, List[str]]:
+        """Replace the keywords in text with placeholder tokens. This function
+        will be called in `self.__call__` and `self.encode`.
+
+        Args:
+            text (Union[str, List[str]]): The text to be processed.
+            vector_shuffle (bool, optional): Whether to shuffle the vectors.
+                Defaults to False.
+            prop_tokens_to_load (float, optional): The proportion of tokens to
+                be loaded. If 1.0, all tokens will be loaded. Defaults to 1.0.
+
+        Returns:
+            Union[str, List[str]]: The processed text.
+        """
+        if isinstance(text, list):
+            output = []
+            for i in range(len(text)):
+                output.append(self.replace_placeholder_tokens_in_text(text[i], vector_shuffle=vector_shuffle))
+            return output
+
+        for placeholder_token in self.token_map:
+            if placeholder_token in text:
+                tokens = self.token_map[placeholder_token]
+                tokens = tokens[: 1 + int(len(tokens) * prop_tokens_to_load)]
+                if vector_shuffle:
+                    tokens = copy.copy(tokens)
+                    random.shuffle(tokens)
+                text = text.replace(placeholder_token, " ".join(tokens))
+        return text
+
+    def replace_text_with_placeholder_tokens(self, text: Union[str, List[str]]) -> Union[str, List[str]]:
+        """Replace the placeholder tokens in text with the original keywords.
+        This function will be called in `self.decode`.
+
+        Args:
+            text (Union[str, List[str]]): The text to be processed.
+
+        Returns:
+            Union[str, List[str]]: The processed text.
+        """
+        if isinstance(text, list):
+            output = []
+            for i in range(len(text)):
+                output.append(self.replace_text_with_placeholder_tokens(text[i]))
+            return output
+
+        for placeholder_token, tokens in self.token_map.items():
+            merged_tokens = " ".join(tokens)
+            if merged_tokens in text:
+                text = text.replace(merged_tokens, placeholder_token)
+        return text
+
+    def __call__(
+            self,
+            text: Union[str, List[str]],
+            *args,
+            vector_shuffle: bool = False,
+            prop_tokens_to_load: float = 1.0,
+            **kwargs,
+    ):
+        """The call function of the wrapper.
+
+        Args:
+            text (Union[str, List[str]]): The text to be tokenized.
+            vector_shuffle (bool, optional): Whether to shuffle the vectors.
+                Defaults to False.
+            prop_tokens_to_load (float, optional): The proportion of tokens to
+                be loaded. If 1.0, all tokens will be loaded. Defaults to 1.0
+            *args, **kwargs: The arguments for `self.wrapped.__call__`.
+        """
+        replaced_text = self.replace_placeholder_tokens_in_text(
+            text, vector_shuffle=vector_shuffle, prop_tokens_to_load=prop_tokens_to_load
+        )
+
+        return self.wrapped.__call__(replaced_text, *args, **kwargs)
+
+    def encode(self, text: Union[str, List[str]], *args, **kwargs):
+        """Encode the passed text to token index.
+
+        Args:
+            text (Union[str, List[str]]): The text to be encode.
+            *args, **kwargs: The arguments for `self.wrapped.__call__`.
+        """
+        replaced_text = self.replace_placeholder_tokens_in_text(text)
+        return self.wrapped(replaced_text, *args, **kwargs)
+
+    def decode(self, token_ids, return_raw: bool = False, *args, **kwargs) -> Union[str, List[str]]:
+        """Decode the token index to text.
+
+        Args:
+            token_ids: The token index to be decoded.
+            return_raw: Whether keep the placeholder token in the text.
+                Defaults to False.
+            *args, **kwargs: The arguments for `self.wrapped.decode`.
+
+        Returns:
+            Union[str, List[str]]: The decoded text.
+        """
+        text = self.wrapped.decode(token_ids, *args, **kwargs)
+        if return_raw:
+            return text
+        replaced_text = self.replace_text_with_placeholder_tokens(text)
+        return replaced_text
+
+    def __repr__(self):
+        """The representation of the wrapper."""
+        s = super().__repr__()
+        prefix = f"Wrapped Module Class: {self._module_cls}\n"
+        prefix += f"Wrapped Module Name: {self._module_name}\n"
+        if self._from_pretrained:
+            prefix += f"From Pretrained: {self._from_pretrained}\n"
+        s = prefix + s
+        return s
+
+
+class EmbeddingLayerWithFixes(nn.Module):
+    """The revised embedding layer to support external embeddings. This design
+    of this class is inspired by https://github.com/AUTOMATIC1111/stable-
+    diffusion-webui/blob/22bcc7be428c94e9408f589966c2040187245d81/modules/sd_hi
+    jack.py#L224  # noqa.
+
+    Args:
+        wrapped (nn.Emebdding): The embedding layer to be wrapped.
+        external_embeddings (Union[dict, List[dict]], optional): The external
+            embeddings added to this layer. Defaults to None.
+    """
+
+    def __init__(self, wrapped: nn.Embedding, external_embeddings: Optional[Union[dict, List[dict]]] = None):
+        super().__init__()
+        self.wrapped = wrapped
+        self.num_embeddings = wrapped.weight.shape[0]
+
+        self.external_embeddings = []
+        if external_embeddings:
+            self.add_embeddings(external_embeddings)
+
+        self.trainable_embeddings = nn.ParameterDict()
+
+    @property
+    def weight(self):
+        """Get the weight of wrapped embedding layer."""
+        return self.wrapped.weight
+
+    def check_duplicate_names(self, embeddings: List[dict]):
+        """Check whether duplicate names exist in list of 'external
+        embeddings'.
+
+        Args:
+            embeddings (List[dict]): A list of embedding to be check.
+        """
+        names = [emb["name"] for emb in embeddings]
+        assert len(names) == len(set(names)), (
+            "Found duplicated names in 'external_embeddings'. Name list: " f"'{names}'"
+        )
+
+    def check_ids_overlap(self, embeddings):
+        """Check whether overlap exist in token ids of 'external_embeddings'.
+
+        Args:
+            embeddings (List[dict]): A list of embedding to be check.
+        """
+        ids_range = [[emb["start"], emb["end"], emb["name"]] for emb in embeddings]
+        ids_range.sort()  # sort by 'start'
+        # check if 'end' has overlapping
+        for idx in range(len(ids_range) - 1):
+            name1, name2 = ids_range[idx][-1], ids_range[idx + 1][-1]
+            assert ids_range[idx][1] <= ids_range[idx + 1][0], (
+                f"Found ids overlapping between embeddings '{name1}' " f"and '{name2}'."
+            )
+
+    def add_embeddings(self, embeddings: Optional[Union[dict, List[dict]]]):
+        """Add external embeddings to this layer.
+        Use case:
+        Args:
+            embeddings (Union[dict, list[dict]]): The external embeddings to
+                be added. Each dict must contain the following 4 fields: 'name'
+                (the name of this embedding), 'embedding' (the embedding
+                tensor), 'start' (the start token id of this embedding), 'end'
+                (the end token id of this embedding). For example:
+                `{name: NAME, start: START, end: END, embedding: torch.Tensor}`
+        """
+        if isinstance(embeddings, dict):
+            embeddings = [embeddings]
+
+        self.external_embeddings += embeddings
+        self.check_duplicate_names(self.external_embeddings)
+        self.check_ids_overlap(self.external_embeddings)
+
+        # set for trainable
+        added_trainable_emb_info = []
+        for embedding in embeddings:
+            trainable = embedding.get("trainable", False)
+            if trainable:
+                name = embedding["name"]
+                embedding["embedding"] = torch.nn.Parameter(embedding["embedding"])
+                self.trainable_embeddings[name] = embedding["embedding"]
+                added_trainable_emb_info.append(name)
+
+        added_emb_info = [emb["name"] for emb in embeddings]
+        added_emb_info = ", ".join(added_emb_info)
+        print(f"Successfully add external embeddings: {added_emb_info}.", "current")
+
+        if added_trainable_emb_info:
+            added_trainable_emb_info = ", ".join(added_trainable_emb_info)
+            print("Successfully add trainable external embeddings: " f"{added_trainable_emb_info}", "current")
+
+    def replace_input_ids(self, input_ids: torch.Tensor) -> torch.Tensor:
+        """Replace external input ids to 0.
+
+        Args:
+            input_ids (torch.Tensor): The input ids to be replaced.
+
+        Returns:
+            torch.Tensor: The replaced input ids.
+        """
+        input_ids_fwd = input_ids.clone()
+        input_ids_fwd[input_ids_fwd >= self.num_embeddings] = 0
+        return input_ids_fwd
+
+    def replace_embeddings(
+            self, input_ids: torch.Tensor, embedding: torch.Tensor, external_embedding: dict
+    ) -> torch.Tensor:
+        """Replace external embedding to the embedding layer. Noted that, in
+        this function we use `torch.cat` to avoid inplace modification.
+
+        Args:
+            input_ids (torch.Tensor): The original token ids. Shape like
+                [LENGTH, ].
+            embedding (torch.Tensor): The embedding of token ids after
+                `replace_input_ids` function.
+            external_embedding (dict): The external embedding to be replaced.
+
+        Returns:
+            torch.Tensor: The replaced embedding.
+        """
+        new_embedding = []
+
+        name = external_embedding["name"]
+        start = external_embedding["start"]
+        end = external_embedding["end"]
+        target_ids_to_replace = [i for i in range(start, end)]
+        ext_emb = external_embedding["embedding"]
+
+        # do not need to replace
+        if not (input_ids == start).any():
+            return embedding
+
+        # start replace
+        s_idx, e_idx = 0, 0
+        while e_idx < len(input_ids):
+            if input_ids[e_idx] == start:
+                if e_idx != 0:
+                    # add embedding do not need to replace
+                    new_embedding.append(embedding[s_idx:e_idx])
+
+                # check if the next embedding need to replace is valid
+                actually_ids_to_replace = [int(i) for i in input_ids[e_idx: e_idx + end - start]]
+                assert actually_ids_to_replace == target_ids_to_replace, (
+                    f"Invalid 'input_ids' in position: {s_idx} to {e_idx}. "
+                    f"Expect '{target_ids_to_replace}' for embedding "
+                    f"'{name}' but found '{actually_ids_to_replace}'."
+                )
+
+                new_embedding.append(ext_emb)
+
+                s_idx = e_idx + end - start
+                e_idx = s_idx + 1
+            else:
+                e_idx += 1
+
+        if e_idx == len(input_ids):
+            new_embedding.append(embedding[s_idx:e_idx])
+
+        return torch.cat(new_embedding, dim=0)
+
+    def forward(self, input_ids: torch.Tensor, external_embeddings: Optional[List[dict]] = None):
+        """The forward function.
+
+        Args:
+            input_ids (torch.Tensor): The token ids shape like [bz, LENGTH] or
+                [LENGTH, ].
+            external_embeddings (Optional[List[dict]]): The external
+                embeddings. If not passed, only `self.external_embeddings`
+                will be used.  Defaults to None.
+
+        input_ids: shape like [bz, LENGTH] or [LENGTH].
+        """
+        assert input_ids.ndim in [1, 2]
+        if input_ids.ndim == 1:
+            input_ids = input_ids.unsqueeze(0)
+
+        if external_embeddings is None and not self.external_embeddings:
+            return self.wrapped(input_ids)
+
+        input_ids_fwd = self.replace_input_ids(input_ids)
+        inputs_embeds = self.wrapped(input_ids_fwd)
+
+        vecs = []
+
+        if external_embeddings is None:
+            external_embeddings = []
+        elif isinstance(external_embeddings, dict):
+            external_embeddings = [external_embeddings]
+        embeddings = self.external_embeddings + external_embeddings
+
+        for input_id, embedding in zip(input_ids, inputs_embeds):
+            new_embedding = embedding
+            for external_embedding in embeddings:
+                new_embedding = self.replace_embeddings(input_id, new_embedding, external_embedding)
+            vecs.append(new_embedding)
+
+        return torch.stack(vecs)
+
+
+def add_tokens(
+        tokenizer, text_encoder, placeholder_tokens: list, initialize_tokens: list = None,
+        num_vectors_per_token: int = 1
+):
+    """Add token for training.
+
+    # TODO: support add tokens as dict, then we can load pretrained tokens.
+    """
+    if initialize_tokens is not None:
+        assert len(initialize_tokens) == len(
+            placeholder_tokens
+        ), "placeholder_token should be the same length as initialize_token"
+    for ii in range(len(placeholder_tokens)):
+        tokenizer.add_placeholder_token(placeholder_tokens[ii], num_vec_per_token=num_vectors_per_token)
+
+    # text_encoder.set_embedding_layer()
+    embedding_layer = text_encoder.text_model.embeddings.token_embedding
+    text_encoder.text_model.embeddings.token_embedding = EmbeddingLayerWithFixes(embedding_layer)
+    embedding_layer = text_encoder.text_model.embeddings.token_embedding
+
+    assert embedding_layer is not None, (
+        "Do not support get embedding layer for current text encoder. " "Please check your configuration."
+    )
+    initialize_embedding = []
+    if initialize_tokens is not None:
+        for ii in range(len(placeholder_tokens)):
+            init_id = tokenizer(initialize_tokens[ii]).input_ids[1]
+            temp_embedding = embedding_layer.weight[init_id]
+            initialize_embedding.append(temp_embedding[None, ...].repeat(num_vectors_per_token, 1))
+    else:
+        for ii in range(len(placeholder_tokens)):
+            init_id = tokenizer("a").input_ids[1]
+            temp_embedding = embedding_layer.weight[init_id]
+            len_emb = temp_embedding.shape[0]
+            init_weight = (torch.rand(num_vectors_per_token, len_emb) - 0.5) / 2.0
+            initialize_embedding.append(init_weight)
+
+    # initialize_embedding  = torch.cat(initialize_embedding,dim=0)
+
+    token_info_all = []
+    for ii in range(len(placeholder_tokens)):
+        token_info = tokenizer.get_token_info(placeholder_tokens[ii])
+        token_info["embedding"] = initialize_embedding[ii]
+        token_info["trainable"] = True
+        token_info_all.append(token_info)
+    embedding_layer.add_embeddings(token_info_all)

py/config.py

@@ -76,6 +76,15 @@ BRUSHNET_MODELS = {
         }
     }
 }
+POWERPAINT_MODELS = {
+    "base_fp16": {
+        "model_url": "https://huggingface.co/runwayml/stable-diffusion-v1-5/resolve/main/text_encoder/model.fp16.safetensors"
+    },
+    "v2.1": {
+        "model_url": "https://huggingface.co/JunhaoZhuang/PowerPaint-v2-1/resolve/main/PowerPaint_Brushnet/diffusion_pytorch_model.safetensors",
+        "clip_url": "https://huggingface.co/JunhaoZhuang/PowerPaint-v2-1/resolve/main/PowerPaint_Brushnet/pytorch_model.bin",
+    }
+}
 
 # layerDiffuse
 LAYER_DIFFUSION_DIR = os.path.join(folder_paths.models_dir, "layer_model")

py/dynamiCrafter/__init__.py

@@ -1,3 +1,5 @@
+#credit to ExponentialML for this module
+#from https://github.com/ExponentialML/ComfyUI_Native_DynamiCrafter
 import os
 import torch
 import comfy

py/ic_light/__init__.py

@@ -1,3 +1,5 @@
+#credit to huchenlei for this module
+#from https://github.com/huchenlei/ComfyUI-IC-Light-Native
 import torch
 import numpy as np
 from typing import Tuple, TypedDict, Callable

py/image.py

@@ -1,17 +1,19 @@
-from PIL import Image, ImageDraw, ImageFilter
 import os
 import hashlib
 import folder_paths
 import torch
 import numpy as np
+import comfy.utils
 import comfy.model_management
 from comfy_extras.nodes_compositing import JoinImageWithAlpha
 from server import PromptServer
 from nodes import MAX_RESOLUTION
-from torchvision.transforms import Resize, CenterCrop, InterpolationMode
+from PIL import Image, ImageDraw, ImageFilter
+from torchvision.transforms import Resize, CenterCrop, GaussianBlur
 from torchvision.transforms.functional import to_pil_image
-from .log import log_node_info
-from .libs.image import pil2tensor, tensor2pil, ResizeMode, get_new_bounds, RGB2RGBA, image2mask, mask2image, blendImage
+from .libs.log import log_node_info
+from .libs.utils import AlwaysEqualProxy
+from .libs.image import pil2tensor, tensor2pil, ResizeMode, get_new_bounds, RGB2RGBA, image2mask
 from .libs.colorfix import adain_color_fix, wavelet_color_fix
 from .libs.chooser import ChooserMessage, ChooserCancelled
 from .config import REMBG_DIR, REMBG_MODELS, HUMANPARSING_MODELS, MEDIAPIPE_MODELS, MEDIAPIPE_DIR
@@ -505,6 +507,48 @@ class JoinImageBatch:
       image = torch.transpose(torch.transpose(images, 1, 2).reshape(1, n * w, h, c), 1, 2)
     return (image,)
 
+class imageListToImageBatch:
+  @classmethod
+  def INPUT_TYPES(s):
+    return {"required": {
+      "images": ("IMAGE",),
+    }}
+
+  INPUT_IS_LIST = True
+
+  RETURN_TYPES = ("IMAGE",)
+  FUNCTION = "doit"
+
+  CATEGORY = "EasyUse/Image"
+
+  def doit(self, images):
+    if len(images) <= 1:
+      return (images[0],)
+    else:
+      image1 = images[0]
+      for image2 in images[1:]:
+        if image1.shape[1:] != image2.shape[1:]:
+          image2 = comfy.utils.common_upscale(image2.movedim(-1, 1), image1.shape[2], image1.shape[1], "lanczos",
+                                              "center").movedim(1, -1)
+        image1 = torch.cat((image1, image2), dim=0)
+      return (image1,)
+
+
+class imageBatchToImageList:
+  @classmethod
+  def INPUT_TYPES(s):
+    return {"required": {"image": ("IMAGE",), }}
+
+  RETURN_TYPES = ("IMAGE",)
+  OUTPUT_IS_LIST = (True,)
+  FUNCTION = "doit"
+
+  CATEGORY = "EasyUse/Image"
+
+  def doit(self, image):
+    images = [image[i:i + 1, ...] for i in range(image.shape[0])]
+    return (images,)
+
 # 图像拆分
 class imageSplitList:
   @classmethod
@@ -751,7 +795,11 @@ class imageChooser(PreviewImage):
     mode = kwargs.pop('mode', 'Always Pause')
     last_choosen = None
     if mode == 'Keep Last Selection':
-      if id and extra_pnginfo[0] and "workflow" in extra_pnginfo[0]:
+      if not extra_pnginfo:
+        print("Error: extra_pnginfo is empty")
+      elif (not isinstance(extra_pnginfo[0], dict) or "workflow" not in extra_pnginfo[0]):
+        print("Error: extra_pnginfo[0] is not a dict or missing 'workflow' key")
+      else:
         workflow = extra_pnginfo[0]["workflow"]
         node = next((x for x in workflow["nodes"] if str(x["id"]) == id), None)
         if node:
@@ -831,6 +879,94 @@ class imageColorMatch(PreviewImage):
     return {"ui": {"images": results},
             "result": (new_images,)}
 
+class imageDetailTransfer:
+  @classmethod
+  def INPUT_TYPES(s):
+    return {
+      "required": {
+        "target": ("IMAGE",),
+        "source": ("IMAGE",),
+        "mode": (["add", "multiply", "screen", "overlay", "soft_light", "hard_light", "color_dodge", "color_burn", "difference", "exclusion", "divide",],{"default": "add"}),
+        "blur_sigma": ("FLOAT", {"default": 1.0, "min": 0.1, "max": 100.0, "step": 0.01}),
+        "blend_factor": ("FLOAT", {"default": 1.0, "min": -10.0, "max": 10.0, "step": 0.001, "round": 0.001}),
+        "image_output": (["Hide", "Preview", "Save", "Hide/Save"], {"default": "Preview"}),
+        "save_prefix": ("STRING", {"default": "ComfyUI"}),
+      },
+      "optional": {
+        "mask": ("MASK",),
+      },
+      "hidden": {"prompt": "PROMPT", "extra_pnginfo": "EXTRA_PNGINFO"},
+    }
+
+  RETURN_TYPES = ("IMAGE",)
+  RETURN_NAMES = ("image",)
+  OUTPUT_NODE = True
+  FUNCTION = "transfer"
+  CATEGORY = "EasyUse/Image"
+
+
+
+  def transfer(self, target, source, mode, blur_sigma, blend_factor, image_output, save_prefix, mask=None, prompt=None, extra_pnginfo=None):
+    batch_size, height, width, _ = target.shape
+    device = comfy.model_management.get_torch_device()
+    target_tensor = target.permute(0, 3, 1, 2).clone().to(device)
+    source_tensor = source.permute(0, 3, 1, 2).clone().to(device)
+
+    if target.shape[1:] != source.shape[1:]:
+      source_tensor = comfy.utils.common_upscale(source_tensor, width, height, "bilinear", "disabled")
+
+    if source.shape[0] < batch_size:
+      source = source[0].unsqueeze(0).repeat(batch_size, 1, 1, 1)
+
+    kernel_size = int(6 * int(blur_sigma) + 1)
+
+    gaussian_blur = GaussianBlur(kernel_size=(kernel_size, kernel_size), sigma=(blur_sigma, blur_sigma))
+
+    blurred_target = gaussian_blur(target_tensor)
+    blurred_source = gaussian_blur(source_tensor)
+
+    if mode == "add":
+      new_image = (source_tensor - blurred_source) + blurred_target
+    elif mode == "multiply":
+      new_image = source_tensor * blurred_target
+    elif mode == "screen":
+      new_image = 1 - (1 - source_tensor) * (1 - blurred_target)
+    elif mode == "overlay":
+      new_image = torch.where(blurred_target < 0.5, 2 * source_tensor * blurred_target,
+                               1 - 2 * (1 - source_tensor) * (1 - blurred_target))
+    elif mode == "soft_light":
+      new_image = (1 - 2 * blurred_target) * source_tensor ** 2 + 2 * blurred_target * source_tensor
+    elif mode == "hard_light":
+      new_image = torch.where(source_tensor < 0.5, 2 * source_tensor * blurred_target,
+                               1 - 2 * (1 - source_tensor) * (1 - blurred_target))
+    elif mode == "difference":
+      new_image = torch.abs(blurred_target - source_tensor)
+    elif mode == "exclusion":
+      new_image = 0.5 - 2 * (blurred_target - 0.5) * (source_tensor - 0.5)
+    elif mode == "color_dodge":
+      new_image = blurred_target / (1 - source_tensor)
+    elif mode == "color_burn":
+      new_image = 1 - (1 - blurred_target) / source_tensor
+    elif mode == "divide":
+      new_image = (source_tensor / blurred_source) * blurred_target
+    else:
+      new_image = source_tensor
+
+    new_image = torch.lerp(target_tensor, new_image, blend_factor)
+    if mask is not None:
+      mask = mask.to(device)
+      new_image = torch.lerp(target_tensor, new_image, mask)
+    new_image = torch.clamp(new_image, 0, 1)
+    new_image = new_image.permute(0, 2, 3, 1).cpu().float()
+
+    results = easySave(new_image, save_prefix, image_output, prompt, extra_pnginfo)
+
+    if image_output in ("Hide", "Hide/Save"):
+      return {"ui": {},
+              "result": (new_image,)}
+
+    return {"ui": {"images": results},
+            "result": (new_image,)}
 
 # 图像反推
 from .libs.image import ci
@@ -1322,10 +1458,15 @@ class imageToBase64:
       return {"result": (base64_str,)}
 
 class removeLocalImage:
+
+  def __init__(self):
+    self.hasFile = False
+
   @classmethod
   def INPUT_TYPES(s):
       return {
         "required": {
+          "any": (AlwaysEqualProxy("*"),),
           "file_name": ("STRING",{"default":""}),
         },
       }
@@ -1335,15 +1476,27 @@ class removeLocalImage:
   FUNCTION = "remove"
   CATEGORY = "EasyUse/Image"
 
-  def remove(self, file_name):
-    hasFile = False
-    for file in os.listdir(folder_paths.input_directory):
-      name_without_extension, file_extension = os.path.splitext(file)
-      if name_without_extension == file_name or file == file_name:
-        os.remove(os.path.join(folder_paths.input_directory, file))
-        hasFile = True
-        break
-    if hasFile:
+
+
+  def remove(self, any, file_name):
+    self.hasFile = False
+    def listdir(path, dir_name=''):
+      for file in os.listdir(path):
+        file_path = os.path.join(path, file)
+        if os.path.isdir(file_path):
+          dir_name = os.path.basename(file_path)
+          listdir(file_path, dir_name)
+        else:
+          file = os.path.join(dir_name, file)
+          name_without_extension, file_extension = os.path.splitext(file)
+          if name_without_extension == file_name or file == file_name:
+            os.remove(os.path.join(folder_paths.input_directory, file))
+            self.hasFile = True
+            break
+
+    listdir(folder_paths.input_directory, '')
+
+    if self.hasFile:
       PromptServer.instance.send_sync("easyuse-toast", {"content": "Removed SuccessFully", "type":'success'})
     else:
       PromptServer.instance.send_sync("easyuse-toast", {"content": "Removed Failed", "type": 'error'})
@@ -1405,6 +1558,8 @@ NODE_CLASS_MAPPINGS = {
   "easy imageRatio": imageRatio,
   "easy imageToMask": imageToMask,
   "easy imageConcat": imageConcat,
+  "easy imageListToImageBatch": imageListToImageBatch,
+  "easy imageBatchToImageList": imageBatchToImageList,
   "easy imageSplitList": imageSplitList,
   "easy imageSplitGrid": imageSplitGrid,
   "easy imagesSplitImage": imagesSplitImage,
@@ -1414,6 +1569,7 @@ NODE_CLASS_MAPPINGS = {
   "easy imageRemBg": imageRemBg,
   "easy imageChooser": imageChooser,
   "easy imageColorMatch": imageColorMatch,
+  "easy imageDetailTransfer": imageDetailTransfer,
   "easy imageInterrogator": imageInterrogator,
   "easy loadImageBase64": loadImageBase64,
   "easy imageToBase64": imageToBase64,
@@ -1437,6 +1593,8 @@ NODE_DISPLAY_NAME_MAPPINGS = {
   "easy imageToMask": "ImageToMask",
   "easy imageHSVMask": "ImageHSVMask",
   "easy imageConcat": "imageConcat",
+  "easy imageListToImageBatch": "Image List To Image Batch",
+  "easy imageBatchToImageList": "Image Batch To Image List",
   "easy imageSplitList": "imageSplitList",
   "easy imageSplitGrid": "imageSplitGrid",
   "easy imagesSplitImage": "imagesSplitImage",
@@ -1446,6 +1604,7 @@ NODE_DISPLAY_NAME_MAPPINGS = {
   "easy imageRemBg": "Image Remove Bg",
   "easy imageChooser": "Image Chooser",
   "easy imageColorMatch": "Image Color Match",
+  "easy imageDetailTransfer": "Image Detail Transfer",
   "easy imageInterrogator": "Image To Prompt",
   "easy joinImageBatch": "JoinImageBatch",
   "easy loadImageBase64": "Load Image (Base64)",

py/layer_diffuse/__init__.py

@@ -0,0 +1,209 @@
+#credit to huchenlei for this module
+#from https://github.com/huchenlei/ComfyUI-layerdiffuse
+import torch
+import comfy.model_management
+import copy
+from typing import Optional
+from enum import Enum
+from comfy.utils import load_torch_file
+from comfy.conds import CONDRegular
+from comfy_extras.nodes_compositing import JoinImageWithAlpha
+from .model import ModelPatcher, TransparentVAEDecoder, calculate_weight_adjust_channel
+from .attension_sharing import AttentionSharingPatcher
+from ..config import LAYER_DIFFUSION, LAYER_DIFFUSION_DIR, LAYER_DIFFUSION_VAE
+from ..libs.utils import to_lora_patch_dict, get_local_filepath, get_sd_version
+
+load_layer_model_state_dict = load_torch_file
+class LayerMethod(Enum):
+    FG_ONLY_ATTN = "Attention Injection"
+    FG_ONLY_CONV = "Conv Injection"
+    FG_TO_BLEND = "Foreground"
+    FG_BLEND_TO_BG = "Foreground to Background"
+    BG_TO_BLEND = "Background"
+    BG_BLEND_TO_FG = "Background to Foreground"
+    EVERYTHING = "Everything"
+
+class LayerDiffuse:
+
+    def __init__(self) -> None:
+        self.vae_transparent_decoder = None
+        self.frames = 1
+
+    def get_layer_diffusion_method(self, method, has_blend_latent):
+        method = LayerMethod(method)
+        if method == LayerMethod.BG_TO_BLEND and has_blend_latent:
+            method = LayerMethod.BG_BLEND_TO_FG
+        elif method == LayerMethod.FG_TO_BLEND and has_blend_latent:
+            method = LayerMethod.FG_BLEND_TO_BG
+        return method
+
+    def apply_layer_c_concat(self, cond, uncond, c_concat):
+        def write_c_concat(cond):
+            new_cond = []
+            for t in cond:
+                n = [t[0], t[1].copy()]
+                if "model_conds" not in n[1]:
+                    n[1]["model_conds"] = {}
+                n[1]["model_conds"]["c_concat"] = CONDRegular(c_concat)
+                new_cond.append(n)
+            return new_cond
+
+        return (write_c_concat(cond), write_c_concat(uncond))
+
+    def apply_layer_diffusion(self, model: ModelPatcher, method, weight, samples, blend_samples, positive, negative, image=None, additional_cond=(None, None, None)):
+        control_img: Optional[torch.TensorType] = None
+        sd_version = get_sd_version(model)
+        model_url = LAYER_DIFFUSION[method.value][sd_version]["model_url"]
+
+        if image is not None:
+            image = image.movedim(-1, 1)
+
+        try:
+            ModelPatcher.calculate_weight = calculate_weight_adjust_channel(ModelPatcher.calculate_weight)
+        except:
+            pass
+
+        if method in [LayerMethod.FG_ONLY_CONV, LayerMethod.FG_ONLY_ATTN] and sd_version == 'sd1':
+            self.frames = 1
+        elif method in [LayerMethod.BG_TO_BLEND, LayerMethod.FG_TO_BLEND, LayerMethod.BG_BLEND_TO_FG, LayerMethod.FG_BLEND_TO_BG] and sd_version == 'sd1':
+            self.frames = 2
+            batch_size, _, height, width = samples['samples'].shape
+            if batch_size % 2 != 0:
+                raise Exception(f"The batch size should be a multiple of 2. 批次大小需为2的倍数")
+            control_img = image
+        elif method == LayerMethod.EVERYTHING and sd_version == 'sd1':
+            batch_size, _, height, width = samples['samples'].shape
+            self.frames = 3
+            if batch_size % 3 != 0:
+                raise Exception(f"The batch size should be a multiple of 3. 批次大小需为3的倍数")
+        if model_url is None:
+            raise Exception(f"{method.value} is not supported for {sd_version} model")
+
+        model_path = get_local_filepath(model_url, LAYER_DIFFUSION_DIR)
+        layer_lora_state_dict = load_layer_model_state_dict(model_path)
+        work_model = model.clone()
+        if sd_version == 'sd1':
+            patcher = AttentionSharingPatcher(
+                work_model, self.frames, use_control=control_img is not None
+            )
+            patcher.load_state_dict(layer_lora_state_dict, strict=True)
+            if control_img is not None:
+                patcher.set_control(control_img)
+        else:
+            layer_lora_patch_dict = to_lora_patch_dict(layer_lora_state_dict)
+            work_model.add_patches(layer_lora_patch_dict, weight)
+
+        # cond_contact
+        if method in [LayerMethod.FG_ONLY_ATTN, LayerMethod.FG_ONLY_CONV]:
+            samp_model = work_model
+        elif sd_version == 'sdxl':
+            if method in [LayerMethod.BG_TO_BLEND, LayerMethod.FG_TO_BLEND]:
+                c_concat = model.model.latent_format.process_in(samples["samples"])
+            else:
+                c_concat = model.model.latent_format.process_in(torch.cat([samples["samples"], blend_samples["samples"]], dim=1))
+            samp_model, positive, negative = (work_model,) + self.apply_layer_c_concat(positive, negative, c_concat)
+        elif sd_version == 'sd1':
+            if method in [LayerMethod.BG_TO_BLEND, LayerMethod.BG_BLEND_TO_FG]:
+                additional_cond = (additional_cond[0], None)
+            elif method in [LayerMethod.FG_TO_BLEND, LayerMethod.FG_BLEND_TO_BG]:
+                additional_cond = (additional_cond[1], None)
+
+            work_model.model_options.setdefault("transformer_options", {})
+            work_model.model_options["transformer_options"]["cond_overwrite"] = [
+                cond[0][0] if cond is not None else None
+                for cond in additional_cond
+            ]
+            samp_model = work_model
+
+        return samp_model, positive, negative
+
+    def join_image_with_alpha(self, image, alpha):
+        out = image.movedim(-1, 1)
+        if out.shape[1] == 3:  # RGB
+            out = torch.cat([out, torch.ones_like(out[:, :1, :, :])], dim=1)
+        for i in range(out.shape[0]):
+            out[i, 3, :, :] = alpha
+        return out.movedim(1, -1)
+
+    def image_to_alpha(self, image, latent):
+        pixel = image.movedim(-1, 1)  # [B, H, W, C] => [B, C, H, W]
+        decoded = []
+        sub_batch_size = 16
+        for start_idx in range(0, latent.shape[0], sub_batch_size):
+            decoded.append(
+                self.vae_transparent_decoder.decode_pixel(
+                    pixel[start_idx: start_idx + sub_batch_size],
+                    latent[start_idx: start_idx + sub_batch_size],
+                )
+            )
+        pixel_with_alpha = torch.cat(decoded, dim=0)
+        # [B, C, H, W] => [B, H, W, C]
+        pixel_with_alpha = pixel_with_alpha.movedim(1, -1)
+        image = pixel_with_alpha[..., 1:]
+        alpha = pixel_with_alpha[..., 0]
+
+        alpha = 1.0 - alpha
+        new_images, = JoinImageWithAlpha().join_image_with_alpha(image, alpha)
+        return new_images, alpha
+
+    def make_3d_mask(self, mask):
+        if len(mask.shape) == 4:
+            return mask.squeeze(0)
+
+        elif len(mask.shape) == 2:
+            return mask.unsqueeze(0)
+
+        return mask
+
+    def masks_to_list(self, masks):
+        if masks is None:
+            empty_mask = torch.zeros((64, 64), dtype=torch.float32, device="cpu")
+            return ([empty_mask],)
+
+        res = []
+
+        for mask in masks:
+            res.append(mask)
+
+        return [self.make_3d_mask(x) for x in res]
+
+    def layer_diffusion_decode(self, layer_diffusion_method, latent, blend_samples, samp_images, model):
+        alpha = []
+        if layer_diffusion_method is not None:
+            sd_version = get_sd_version(model)
+            if sd_version not in ['sdxl', 'sd1']:
+                raise Exception(f"Only SDXL and SD1.5 model supported for Layer Diffusion")
+            method = self.get_layer_diffusion_method(layer_diffusion_method, blend_samples is not None)
+            sd15_allow = True if sd_version == 'sd1' and method in [LayerMethod.FG_ONLY_ATTN, LayerMethod.EVERYTHING, LayerMethod.BG_TO_BLEND, LayerMethod.BG_BLEND_TO_FG] else False
+            sdxl_allow = True if sd_version == 'sdxl' and method in [LayerMethod.FG_ONLY_CONV, LayerMethod.FG_ONLY_ATTN, LayerMethod.BG_BLEND_TO_FG] else False
+            if sdxl_allow or sd15_allow:
+                if self.vae_transparent_decoder is None:
+                    model_url = LAYER_DIFFUSION_VAE['decode'][sd_version]["model_url"]
+                    if model_url is None:
+                        raise Exception(f"{method.value} is not supported for {sd_version} model")
+                    decoder_file = get_local_filepath(model_url, LAYER_DIFFUSION_DIR)
+                    self.vae_transparent_decoder = TransparentVAEDecoder(
+                        load_torch_file(decoder_file),
+                        device=comfy.model_management.get_torch_device(),
+                        dtype=(torch.float16 if comfy.model_management.should_use_fp16() else torch.float32),
+                    )
+                if method in [LayerMethod.EVERYTHING, LayerMethod.BG_BLEND_TO_FG, LayerMethod.BG_TO_BLEND]:
+                    new_images = []
+                    sliced_samples = copy.copy({"samples": latent})
+                    for index in range(len(samp_images)):
+                        if index % self.frames == 0:
+                            img = samp_images[index::self.frames]
+                            alpha_images, _alpha = self.image_to_alpha(img, sliced_samples["samples"][index::self.frames])
+                            alpha.append(self.make_3d_mask(_alpha[0]))
+                            new_images.append(alpha_images[0])
+                        else:
+                            new_images.append(samp_images[index])
+                else:
+                    new_images, alpha = self.image_to_alpha(samp_images, latent)
+            else:
+                new_images = samp_images
+        else:
+            new_images = samp_images
+
+
+        return (new_images, samp_images, alpha)

py/layer_diffuse/func.py

@@ -1,204 +0,0 @@
-import torch
-import comfy.model_management
-import copy
-from typing import Optional
-from enum import Enum
-from comfy.utils import load_torch_file
-from comfy.conds import CONDRegular
-from comfy_extras.nodes_compositing import JoinImageWithAlpha
-from .model import ModelPatcher, TransparentVAEDecoder, calculate_weight_adjust_channel
-from .attension_sharing import AttentionSharingPatcher
-from ..config import LAYER_DIFFUSION, LAYER_DIFFUSION_DIR, LAYER_DIFFUSION_VAE
-from ..libs.utils import to_lora_patch_dict, get_local_filepath, get_sd_version
-
-load_layer_model_state_dict = load_torch_file
-class LayerMethod(Enum):
-    FG_ONLY_ATTN = "Attention Injection"
-    FG_ONLY_CONV = "Conv Injection"
-    FG_TO_BLEND = "Foreground"
-    FG_BLEND_TO_BG = "Foreground to Background"
-    BG_TO_BLEND = "Background"
-    BG_BLEND_TO_FG = "Background to Foreground"
-    EVERYTHING = "Everything"
-
-class LayerDiffuse:
-
-    def __init__(self) -> None:
-        self.vae_transparent_decoder = None
-        self.frames = 1
-
-    def get_layer_diffusion_method(self, method, has_blend_latent):
-        method = LayerMethod(method)
-        if method == LayerMethod.BG_TO_BLEND and has_blend_latent:
-            method = LayerMethod.BG_BLEND_TO_FG
-        elif method == LayerMethod.FG_TO_BLEND and has_blend_latent:
-            method = LayerMethod.FG_BLEND_TO_BG
-        return method
-
-    def apply_layer_c_concat(self, cond, uncond, c_concat):
-        def write_c_concat(cond):
-            new_cond = []
-            for t in cond:
-                n = [t[0], t[1].copy()]
-                if "model_conds" not in n[1]:
-                    n[1]["model_conds"] = {}
-                n[1]["model_conds"]["c_concat"] = CONDRegular(c_concat)
-                new_cond.append(n)
-            return new_cond
-
-        return (write_c_concat(cond), write_c_concat(uncond))
-
-    def apply_layer_diffusion(self, model: ModelPatcher, method, weight, samples, blend_samples, positive, negative, image=None, additional_cond=(None, None, None)):
-        control_img: Optional[torch.TensorType] = None
-        sd_version = get_sd_version(model)
-        model_url = LAYER_DIFFUSION[method.value][sd_version]["model_url"]
-
-        try:
-            ModelPatcher.calculate_weight = calculate_weight_adjust_channel(ModelPatcher.calculate_weight)
-        except:
-            pass
-
-        if method in [LayerMethod.FG_ONLY_CONV, LayerMethod.FG_ONLY_ATTN] and sd_version == 'sd1':
-            self.frames = 1
-        elif method in [LayerMethod.BG_TO_BLEND, LayerMethod.FG_TO_BLEND, LayerMethod.BG_BLEND_TO_FG, LayerMethod.FG_BLEND_TO_BG] and sd_version == 'sd1':
-            self.frames = 2
-            batch_size, _, height, width = samples['samples'].shape
-            if batch_size % 2 != 0:
-                raise Exception(f"The batch size should be a multiple of 2. 批次大小需为2的倍数")
-            control_img = image
-        elif method == LayerMethod.EVERYTHING and sd_version == 'sd1':
-            batch_size, _, height, width = samples['samples'].shape
-            self.frames = 3
-            if batch_size % 3 != 0:
-                raise Exception(f"The batch size should be a multiple of 3. 批次大小需为3的倍数")
-        if model_url is None:
-            raise Exception(f"{method.value} is not supported for {sd_version} model")
-
-        model_path = get_local_filepath(model_url, LAYER_DIFFUSION_DIR)
-        layer_lora_state_dict = load_layer_model_state_dict(model_path)
-        work_model = model.clone()
-        if sd_version == 'sd1':
-            patcher = AttentionSharingPatcher(
-                work_model, self.frames, use_control=control_img is not None
-            )
-            patcher.load_state_dict(layer_lora_state_dict, strict=True)
-            if control_img is not None:
-                patcher.set_control(control_img)
-        else:
-            layer_lora_patch_dict = to_lora_patch_dict(layer_lora_state_dict)
-            work_model.add_patches(layer_lora_patch_dict, weight)
-
-        # cond_contact
-        if method in [LayerMethod.FG_ONLY_ATTN, LayerMethod.FG_ONLY_CONV]:
-            samp_model = work_model
-        elif sd_version == 'sdxl':
-            if method in [LayerMethod.BG_TO_BLEND, LayerMethod.FG_TO_BLEND]:
-                c_concat = model.model.latent_format.process_in(samples["samples"])
-            else:
-                c_concat = model.model.latent_format.process_in(torch.cat([samples["samples"], blend_samples["samples"]], dim=1))
-            samp_model, positive, negative = (work_model,) + self.apply_layer_c_concat(positive, negative, c_concat)
-        elif sd_version == 'sd1':
-            if method in [LayerMethod.BG_TO_BLEND, LayerMethod.BG_BLEND_TO_FG]:
-                additional_cond = (additional_cond[0], None)
-            elif method in [LayerMethod.FG_TO_BLEND, LayerMethod.FG_BLEND_TO_BG]:
-                additional_cond = (additional_cond[1], None)
-
-            work_model.model_options.setdefault("transformer_options", {})
-            work_model.model_options["transformer_options"]["cond_overwrite"] = [
-                cond[0][0] if cond is not None else None
-                for cond in additional_cond
-            ]
-            samp_model = work_model
-
-        return samp_model, positive, negative
-
-    def join_image_with_alpha(self, image, alpha):
-        out = image.movedim(-1, 1)
-        if out.shape[1] == 3:  # RGB
-            out = torch.cat([out, torch.ones_like(out[:, :1, :, :])], dim=1)
-        for i in range(out.shape[0]):
-            out[i, 3, :, :] = alpha
-        return out.movedim(1, -1)
-
-    def image_to_alpha(self, image, latent):
-        pixel = image.movedim(-1, 1)  # [B, H, W, C] => [B, C, H, W]
-        decoded = []
-        sub_batch_size = 16
-        for start_idx in range(0, latent.shape[0], sub_batch_size):
-            decoded.append(
-                self.vae_transparent_decoder.decode_pixel(
-                    pixel[start_idx: start_idx + sub_batch_size],
-                    latent[start_idx: start_idx + sub_batch_size],
-                )
-            )
-        pixel_with_alpha = torch.cat(decoded, dim=0)
-        # [B, C, H, W] => [B, H, W, C]
-        pixel_with_alpha = pixel_with_alpha.movedim(1, -1)
-        image = pixel_with_alpha[..., 1:]
-        alpha = pixel_with_alpha[..., 0]
-
-        alpha = 1.0 - alpha
-        new_images, = JoinImageWithAlpha().join_image_with_alpha(image, alpha)
-        return new_images, alpha
-
-    def make_3d_mask(self, mask):
-        if len(mask.shape) == 4:
-            return mask.squeeze(0)
-
-        elif len(mask.shape) == 2:
-            return mask.unsqueeze(0)
-
-        return mask
-
-    def masks_to_list(self, masks):
-        if masks is None:
-            empty_mask = torch.zeros((64, 64), dtype=torch.float32, device="cpu")
-            return ([empty_mask],)
-
-        res = []
-
-        for mask in masks:
-            res.append(mask)
-
-        return [self.make_3d_mask(x) for x in res]
-
-    def layer_diffusion_decode(self, layer_diffusion_method, latent, blend_samples, samp_images, model):
-        alpha = []
-        if layer_diffusion_method is not None:
-            sd_version = get_sd_version(model)
-            if sd_version not in ['sdxl', 'sd1']:
-                raise Exception(f"Only SDXL and SD1.5 model supported for Layer Diffusion")
-            method = self.get_layer_diffusion_method(layer_diffusion_method, blend_samples is not None)
-            sd15_allow = True if sd_version == 'sd1' and method in [LayerMethod.FG_ONLY_ATTN, LayerMethod.EVERYTHING, LayerMethod.BG_TO_BLEND, LayerMethod.BG_BLEND_TO_FG] else False
-            sdxl_allow = True if sd_version == 'sdxl' and method in [LayerMethod.FG_ONLY_CONV, LayerMethod.FG_ONLY_ATTN, LayerMethod.BG_BLEND_TO_FG] else False
-            if sdxl_allow or sd15_allow:
-                if self.vae_transparent_decoder is None:
-                    model_url = LAYER_DIFFUSION_VAE['decode'][sd_version]["model_url"]
-                    if model_url is None:
-                        raise Exception(f"{method.value} is not supported for {sd_version} model")
-                    decoder_file = get_local_filepath(model_url, LAYER_DIFFUSION_DIR)
-                    self.vae_transparent_decoder = TransparentVAEDecoder(
-                        load_torch_file(decoder_file),
-                        device=comfy.model_management.get_torch_device(),
-                        dtype=(torch.float16 if comfy.model_management.should_use_fp16() else torch.float32),
-                    )
-                if method in [LayerMethod.EVERYTHING, LayerMethod.BG_BLEND_TO_FG, LayerMethod.BG_TO_BLEND]:
-                    new_images = []
-                    sliced_samples = copy.copy({"samples": latent})
-                    for index in range(len(samp_images)):
-                        if index % self.frames == 0:
-                            img = samp_images[index::self.frames]
-                            alpha_images, _alpha = self.image_to_alpha(img, sliced_samples["samples"][index::self.frames])
-                            alpha.append(self.make_3d_mask(_alpha[0]))
-                            new_images.append(alpha_images[0])
-                        else:
-                            new_images.append(samp_images[index])
-                else:
-                    new_images, alpha = self.image_to_alpha(samp_images, latent)
-            else:
-                new_images = samp_images
-        else:
-            new_images = samp_images
-
-
-        return (new_images, samp_images, alpha)

py/libs/adv_encode.py

@@ -1,12 +1,16 @@
 import torch
 import numpy as np
+import re
 import itertools
 
 from comfy import model_management
 from comfy.sdxl_clip import SDXLClipModel, SDXLRefinerClipModel, SDXLClipG
-from nodes import NODE_CLASS_MAPPINGS, ConditioningConcat, CLIPTextEncode
-
-from .libs.utils import compare_revision
+try:
+    from comfy.sd3_clip import SD3ClipModel, T5XXLModel
+except:
+    SD3ClipModel, T5XXLModel = None, None
+    pass
+from nodes import NODE_CLASS_MAPPINGS, ConditioningConcat, ConditioningZeroOut, ConditioningSetTimestepRange, ConditioningCombine
 
 def _grouper(n, iterable):
     it = iter(iterable)
@@ -240,6 +244,9 @@ def encode_token_weights_l(model, token_weight_pairs):
     l_out, pooled = model.clip_l.encode_token_weights(token_weight_pairs)
     return l_out, pooled
 
+def encode_token_weights_t5(model, token_weight_pairs):
+    return model.t5xxl.encode_token_weights(token_weight_pairs)
+
 
 def encode_token_weights(model, token_weight_pairs, encode_func):
     if model.layer_idx is not None:
@@ -260,6 +267,14 @@ def prepareXL(embs_l, embs_g, pooled, clip_balance):
     else:
         return embs_g, pooled
 
+def prepareSD3(out, pooled, clip_balance):
+    lg_w = 1 - max(0, clip_balance - .5) * 2
+    t5_w = 1 - max(0, .5 - clip_balance) * 2
+    if out.shape[0] > 1:
+        return torch.cat([out[0] * lg_w, out[1] * t5_w], dim=-1), pooled
+    else:
+        return out, pooled
+
 def advanced_encode(clip, text, token_normalization, weight_interpretation, w_max=1.0, clip_balance=.5,
                     apply_to_pooled=True, width=1024, height=1024, crop_w=0, crop_h=0, target_width=1024, target_height=1024, a1111_prompt_style=False, steps=1):
 
@@ -272,6 +287,25 @@ def advanced_encode(clip, text, token_normalization, weight_interpretation, w_ma
         else:
             raise Exception(f"[smzNodes Not Found] you need to install 'ComfyUI-smzNodes'")
 
+    time_start = 0
+    time_end = 1
+    match = re.search(r'TIMESTEP.*$', text)
+    if match:
+        timestep = match.group()
+        timestep = timestep.split(' ')
+        timestep = timestep[0]
+        text = text.replace(timestep, '')
+        value = timestep.split(':')
+        if len(value) >= 3:
+            time_start = float(value[1])
+            time_end = float(value[2])
+        elif len(value) == 2:
+            time_start = float(value[1])
+            time_end = 1
+        elif len(value) == 1:
+            time_start = 0.1
+            time_end = 1
+
     pass3 = [x.strip() for x in text.split("BREAK")]
     pass3 = [x for x in pass3 if x != '']
 
@@ -285,7 +319,62 @@ def advanced_encode(clip, text, token_normalization, weight_interpretation, w_ma
 
     for text in pass3:
         tokenized = clip.tokenize(text, return_word_ids=True)
-        if isinstance(clip.cond_stage_model, (SDXLClipModel, SDXLRefinerClipModel, SDXLClipG)):
+        if SD3ClipModel and isinstance(clip.cond_stage_model, SD3ClipModel):
+            lg_out = None
+            pooled = None
+            out = None
+
+            if len(tokenized['l']) > 0 or len(tokenized['g']) > 0:
+                if 'l' in tokenized:
+                    lg_out, l_pooled = advanced_encode_from_tokens(tokenized['l'],
+                                                                           token_normalization,
+                                                                           weight_interpretation,
+                                                                           lambda x: encode_token_weights(clip, x, encode_token_weights_l),
+                                                                           w_max=w_max, return_pooled=True,)
+                else:
+                    l_pooled = torch.zeros((1, 768), device=model_management.intermediate_device())
+
+                if 'g' in tokenized:
+                    g_out, g_pooled = advanced_encode_from_tokens(tokenized['g'],
+                               token_normalization,
+                               weight_interpretation,
+                               lambda x: encode_token_weights(clip, x, encode_token_weights_g),
+                               w_max=w_max, return_pooled=True)
+                    if lg_out is not None:
+                        lg_out = torch.cat([lg_out, g_out], dim=-1)
+                    else:
+                        lg_out = torch.nn.functional.pad(g_out, (768, 0))
+                else:
+                    g_out = None
+                    g_pooled = torch.zeros((1, 1280), device=model_management.intermediate_device())
+
+                if lg_out is not None:
+                    lg_out = torch.nn.functional.pad(lg_out, (0, 4096 - lg_out.shape[-1]))
+                    out = lg_out
+                pooled = torch.cat((l_pooled, g_pooled), dim=-1)
+
+            # t5xxl
+            if 't5xxl' in tokenized and clip.cond_stage_model.t5xxl is not None:
+                t5_out, t5_pooled = advanced_encode_from_tokens(tokenized['t5xxl'],
+                               token_normalization,
+                               weight_interpretation,
+                               lambda x: encode_token_weights(clip, x, encode_token_weights_t5),
+                               w_max=w_max, return_pooled=True)
+                if lg_out is not None:
+                    out = torch.cat([lg_out, t5_out], dim=-2)
+                else:
+                    out = t5_out
+
+            if out is None:
+                out = torch.zeros((1, 77, 4096), device=model_management.intermediate_device())
+
+            if pooled is None:
+                pooled = torch.zeros((1, 768 + 1280), device=model_management.intermediate_device())
+
+            embeddings_final, pooled = prepareSD3(out, pooled, clip_balance)
+            cond = [[embeddings_final, {"pooled_output": pooled}]]
+
+        elif isinstance(clip.cond_stage_model, (SDXLClipModel, SDXLRefinerClipModel, SDXLClipG)):
             embs_l = None
             embs_g = None
             pooled = None
@@ -325,6 +414,13 @@ def advanced_encode(clip, text, token_normalization, weight_interpretation, w_ma
         else:
             conditioning = cond
 
+    # setTimeStepRange
+    if time_start > 0 or time_end < 1:
+        conditioning_2, = ConditioningSetTimestepRange().set_range(conditioning, 0, time_start)
+        conditioning_1, = ConditioningZeroOut().zero_out(conditioning)
+        conditioning_1, = ConditioningSetTimestepRange().set_range(conditioning_1, time_start, time_end)
+        conditioning, = ConditioningCombine().combine(conditioning_1, conditioning_2)
+
     return conditioning
 
 

py/libs/conditioning.py

@@ -1,14 +1,20 @@
 from .utils import find_wildcards_seed, find_nearest_steps, is_linked_styles_selector
-from ..log import log_node_warn
-from ..adv_encode import advanced_encode
-from ..wildcards import process_with_loras
+from .log import log_node_warn
+from .translate import zh_to_en, has_chinese
+from .wildcards import process_with_loras
+from .adv_encode import advanced_encode
 
 from nodes import ConditioningConcat, ConditioningCombine, ConditioningAverage, ConditioningSetTimestepRange
 
 def prompt_to_cond(type, model, clip, clip_skip, lora_stack, text, prompt_token_normalization, prompt_weight_interpretation, a1111_prompt_style ,my_unique_id, prompt, easyCache, can_load_lora=True, steps=None):
     styles_selector = is_linked_styles_selector(prompt, my_unique_id, type)
     title = "正面提示词" if type == 'positive' else "负面提示词"
-    log_node_warn("正在处理" + title + "...")
+    log_node_warn("正在进行" + title + "...")
+
+    # Translate cn to en
+    if has_chinese(text):
+        text = zh_to_en([text])[0]
+
     positive_seed = find_wildcards_seed(my_unique_id, text, prompt)
     model, clip, text, cond_decode, show_prompt, pipe_lora_stack = process_with_loras(
         text, model, clip, type, positive_seed, can_load_lora, lora_stack, easyCache)
@@ -18,7 +24,7 @@ def prompt_to_cond(type, model, clip, clip_skip, lora_stack, text, prompt_token_
     if clip_skip != 0:
         clipped.clip_layer(clip_skip)
 
-    log_node_warn("正在处理" + title + "编码...")
+    log_node_warn("正在进行" + title + "编码...")
     steps = steps if steps is not None else find_nearest_steps(my_unique_id, prompt)
     return (advanced_encode(clipped, text, prompt_token_normalization,
                             prompt_weight_interpretation, w_max=1.0,

py/libs/controlnet.py

@@ -7,12 +7,12 @@ class easyControlnet:
     def __init__(self):
         pass
 
-    def apply(self, control_net_name, image, positive, negative, strength, start_percent=0, end_percent=1, control_net=None, scale_soft_weights=1, mask=None, easyCache=None):
+    def apply(self, control_net_name, image, positive, negative, strength, start_percent=0, end_percent=1, control_net=None, scale_soft_weights=1, mask=None, easyCache=None, use_cache=True):
         if strength == 0:
             return (positive, negative)
 
         if control_net is None:
-            control_net = easyCache.load_controlnet(control_net_name, scale_soft_weights)
+            control_net = easyCache.load_controlnet(control_net_name, scale_soft_weights, use_cache)
 
         if mask is not None:
             mask = mask.to(self.device)

py/libs/fooocus.py

@@ -1,3 +1,5 @@
+#credit to Acly for this module
+#from https://github.com/Acly/comfyui-inpaint-nodes
 import torch
 import comfy
 from comfy.model_patcher import ModelPatcher
@@ -32,16 +34,18 @@ class InpaintWorker:
                 loaded_keys.add(key)
 
         not_loaded = sum(1 for x in lora if x not in loaded_keys)
-        log_node_info(self.node_name,
-            f"{len(loaded_keys)} Lora keys loaded, {not_loaded} remaining keys not found in model."
-        )
+        if not_loaded > 0:
+            log_node_info(self.node_name,
+                f"{len(loaded_keys)} Lora keys loaded, {not_loaded} remaining keys not found in model."
+            )
         return patch_dict
 
     def calculate_weight_patched(self: ModelPatcher, patches, weight, key):
         remaining = []
 
         for p in patches:
-            alpha, v, strength_model = p
+            alpha = p[0]
+            v = p[1]
 
             is_fooocus_patch = isinstance(v, tuple) and len(v) == 2 and v[0] == "fooocus"
             if not is_fooocus_patch:

py/libs/loader.py

@@ -1,11 +1,13 @@
 import time, os, psutil
+import folder_paths
 import comfy.utils
 import comfy.sd
 import comfy.controlnet
-import folder_paths
+
+from comfy.model_patcher import ModelPatcher
 from nodes import NODE_CLASS_MAPPINGS
 from collections import defaultdict
-from ..log import log_node_info, log_node_error
+from .log import log_node_info, log_node_error
 
 stable_diffusion_loaders = ["easy fullLoader", "easy a1111Loader", "easy comfyLoader", "easy zero123Loader", "easy svdLoader"]
 stable_cascade_loaders = ["easy cascadeLoader"]
@@ -27,7 +29,7 @@ class easyLoader:
             "lora": defaultdict(dict),  # {lora_name: {UID: (model_lora, clip_lora)}}
             "controlnet": defaultdict(dict),
         }
-        self.memory_threshold = self.determine_memory_threshold(0.9)
+        self.memory_threshold = self.determine_memory_threshold(0.7)
         self.lora_name_cache = []
 
     def clean_values(self, values: str):
@@ -249,9 +251,9 @@ class easyLoader:
 
         return model
 
-    def load_controlnet(self, control_net_name, scale_soft_weights=1):
+    def load_controlnet(self, control_net_name, scale_soft_weights=1, use_cache=True):
         unique_id = f'{control_net_name};{str(scale_soft_weights)}'
-        if unique_id in self.loaded_objects["controlnet"]:
+        if use_cache and unique_id in self.loaded_objects["controlnet"]:
             return self.loaded_objects["controlnet"][unique_id][0]
         if scale_soft_weights < 1:
             if "ScaledSoftControlNetWeights" in NODE_CLASS_MAPPINGS:
@@ -265,10 +267,11 @@ class easyLoader:
         else:
             controlnet_path = folder_paths.get_full_path("controlnet", control_net_name)
             control_net = comfy.controlnet.load_controlnet(controlnet_path)
-        self.add_to_cache("controlnet", unique_id, control_net)
-        self.eviction_based_on_memory()
+        if use_cache:
+            self.add_to_cache("controlnet", unique_id, control_net)
+            self.eviction_based_on_memory()
         return control_net
-    def load_clip(self, clip_name, type='stable_diffusion'):
+    def load_clip(self, clip_name, type='stable_diffusion', load_clip=None):
         if type == 'stable_diffusion':
             clip_type = comfy.sd.CLIPType.STABLE_DIFFUSION
         else:
@@ -373,4 +376,62 @@ class easyLoader:
                     self.lora_name_cache.append(x)
                     return x
 
-            return None
+            return None
+
+    def load_main(self, ckpt_name, config_name, vae_name, lora_name, lora_model_strength, lora_clip_strength, optional_lora_stack, model_override, clip_override, vae_override, prompt):
+        model: ModelPatcher | None = None
+        clip: comfy.sd.CLIP | None = None
+        vae: comfy.sd.VAE | None = None
+        clip_vision = None
+        lora_stack = []
+
+        can_load_lora = True
+        # 判断是否存在 模型或Lora叠加xyplot, 若存在优先缓存第一个模型
+        xy_model_id = next((x for x in prompt if str(prompt[x]["class_type"]) in ["easy XYInputs: ModelMergeBlocks",
+                                                                                  "easy XYInputs: Checkpoint"]), None)
+        xy_lora_id = next((x for x in prompt if str(prompt[x]["class_type"]) == "easy XYInputs: Lora"), None)
+        if xy_lora_id is not None:
+            can_load_lora = False
+        if xy_model_id is not None:
+            node = prompt[xy_model_id]
+            if "ckpt_name_1" in node["inputs"]:
+                ckpt_name_1 = node["inputs"]["ckpt_name_1"]
+                model, clip, vae, clip_vision = self.load_checkpoint(ckpt_name_1)
+                can_load_lora = False
+        # Load models
+        elif model_override is not None and clip_override is not None and vae_override is not None:
+            model = model_override
+            clip = clip_override
+            vae = vae_override
+        elif model_override is not None:
+            raise Exception(f"[ERROR] clip or vae is missing")
+        elif vae_override is not None:
+            raise Exception(f"[ERROR] model or clip is missing")
+        elif clip_override is not None:
+            raise Exception(f"[ERROR] model or vae is missing")
+        else:
+            model, clip, vae, clip_vision = self.load_checkpoint(ckpt_name, config_name)
+
+        if optional_lora_stack is not None and can_load_lora:
+            for lora in optional_lora_stack:
+                lora = {"lora_name": lora[0], "model": model, "clip": clip, "model_strength": lora[1],
+                        "clip_strength": lora[2]}
+                model, clip = self.load_lora(lora)
+                lora['model'] = model
+                lora['clip'] = clip
+                lora_stack.append(lora)
+
+        if lora_name != "None" and can_load_lora:
+            lora = {"lora_name": lora_name, "model": model, "clip": clip, "model_strength": lora_model_strength,
+                    "clip_strength": lora_clip_strength}
+            model, clip = self.load_lora(lora)
+            lora_stack.append(lora)
+
+        # Check for custom VAE
+        if vae_name not in ["Baked VAE", "Baked-VAE"]:
+            vae = self.load_vae(vae_name)
+        # CLIP skip
+        if not clip:
+            raise Exception("No CLIP found")
+
+        return model, clip, vae, clip_vision, lora_stack

py/libs/sampler.py

@@ -1,17 +1,21 @@
 import comfy
+import comfy.model_management
+import comfy.samplers
 import torch
 import numpy as np
 import latent_preview
 from nodes import MAX_RESOLUTION
 from PIL import Image
 from typing import Dict, List, Optional, Tuple, Union, Any
-from .utils import get_sd_version
+from ..brushnet.model_patch import add_model_patch
+
 class easySampler:
     def __init__(self):
         self.last_helds: dict[str, list] = {
             "results": [],
             "pipe_line": [],
         }
+        self.device = comfy.model_management.intermediate_device()
 
     @staticmethod
     def tensor2pil(image: torch.Tensor) -> Image.Image:
@@ -47,17 +51,32 @@ class easySampler:
             parts.append('None')
         return parts
 
-    def add_model_patch_option(self, model):
-        if 'transformer_options' not in model.model_options:
-            model.model_options['transformer_options'] = {}
-        to = model.model_options['transformer_options']
-        if "model_patch" not in to:
-            to["model_patch"] = {}
-        return to
+    def emptyLatent(self, resolution, empty_latent_width, empty_latent_height, batch_size=1, compression=0, sd3=False):
+        if resolution != "自定义 x 自定义":
+            try:
+                width, height = map(int, resolution.split(' x '))
+                empty_latent_width = width
+                empty_latent_height = height
+            except ValueError:
+                raise ValueError("Invalid base_resolution format.")
+        if sd3:
+            latent = torch.ones([batch_size, 16, empty_latent_height // 8, empty_latent_width // 8], device=self.device) * 0.0609
+            samples = {"samples": latent}
+        elif compression == 0:
+            latent = torch.zeros([batch_size, 4, empty_latent_height // 8, empty_latent_width // 8], device=self.device)
+            samples = {"samples": latent}
+        else:
+            latent_c = torch.zeros(
+                [batch_size, 16, empty_latent_height // compression, empty_latent_width // compression])
+            latent_b = torch.zeros([batch_size, 4, empty_latent_height // 4, empty_latent_width // 4])
+
+            samples = ({"samples": latent_c}, {"samples": latent_b})
+        return samples
+
 
     def common_ksampler(self, model, seed, steps, cfg, sampler_name, scheduler, positive, negative, latent, denoise=1.0,
                         disable_noise=False, start_step=None, last_step=None, force_full_denoise=False,
-                        preview_latent=True, disable_pbar=False, custom=None):
+                        preview_latent=True, disable_pbar=False):
         device = comfy.model_management.get_torch_device()
         latent_image = latent["samples"]
 
@@ -82,50 +101,25 @@ class easySampler:
                 preview_bytes = previewer.decode_latent_to_preview_image(preview_format, x0)
             pbar.update_absolute(step + 1, total_steps, preview_bytes)
 
-        if custom is not None:
-            guider = custom['guider'] if 'guider' in custom else None
-            sampler = custom['sampler'] if 'sampler' in custom else None
-            sigmas = custom['sigmas'] if 'sigmas' in custom else None
-            noise = custom['noise'] if 'noise' in custom else None
-            samples = guider.sample(noise.generate_noise(latent), latent_image, sampler, sigmas,
-                                    denoise_mask=noise_mask, callback=callback, disable_pbar=disable_pbar,
-                                    seed=noise.seed)
-            samples = samples.to(comfy.model_management.intermediate_device())
+        if disable_noise:
+            noise = torch.zeros(latent_image.size(), dtype=latent_image.dtype, layout=latent_image.layout,
+                                device="cpu")
         else:
-            if disable_noise:
-                noise = torch.zeros(latent_image.size(), dtype=latent_image.dtype, layout=latent_image.layout,
-                                    device="cpu")
-            else:
-                batch_inds = latent["batch_index"] if "batch_index" in latent else None
-                noise = comfy.sample.prepare_noise(latent_image, seed, batch_inds)
-
-            #######################################################################################
-            # brushnet
-            transformer_options = model.model_options['transformer_options'] if "transformer_options" in model.model_options else {}
-            if 'model_patch' in transformer_options and 'brushnet' in transformer_options['model_patch']:
-                to = self.add_model_patch_option(model)
-                mp = to['model_patch']
-                if isinstance(model.model.model_config, comfy.supported_models.SD15):
-                    mp['SDXL'] = False
-                elif isinstance(model.model.model_config, comfy.supported_models.SDXL):
-                    mp['SDXL'] = True
-                else:
-                    print('Base model type: ', type(model.model.model_config))
-                    raise Exception("Unsupported model type: ", type(model.model.model_config))
-
-                mp['unet'] = model.model.diffusion_model
-                mp['step'] = 0
-                mp['total_steps'] = 1
-
-            #
-            #######################################################################################
-            samples = comfy.sample.sample(model, noise, steps, cfg, sampler_name, scheduler, positive, negative,
-                                          latent_image,
-                                          denoise=denoise, disable_noise=disable_noise, start_step=start_step,
-                                          last_step=last_step,
-                                          force_full_denoise=force_full_denoise, noise_mask=noise_mask, callback=callback,
-                                          disable_pbar=disable_pbar, seed=seed)
+            batch_inds = latent["batch_index"] if "batch_index" in latent else None
+            noise = comfy.sample.prepare_noise(latent_image, seed, batch_inds)
 
+        #######################################################################################
+        # brushnet
+        add_model_patch(model)
+        #
+        #######################################################################################
+        samples = comfy.sample.sample(model, noise, steps, cfg, sampler_name, scheduler, positive, negative,
+                                      latent_image,
+                                      denoise=denoise, disable_noise=disable_noise, start_step=start_step,
+                                      last_step=last_step,
+                                      force_full_denoise=force_full_denoise, noise_mask=noise_mask,
+                                      callback=callback,
+                                      disable_pbar=disable_pbar, seed=seed)
         out = latent.copy()
         out["samples"] = samples
         return out
@@ -157,43 +151,49 @@ class easySampler:
 
         pbar = comfy.utils.ProgressBar(steps)
 
-        #######################################################################################
-        # brushnet
-        to = None
-        transformer_options = model.model_options['transformer_options'] if "transformer_options" in model.model_options else {}
-        if 'model_patch' in transformer_options and 'brushnet_model' in transformer_options['model_patch']:
-            to = self.add_model_patch_option(model)
-            mp = to['model_patch']
-            if isinstance(model.model.model_config, comfy.supported_models.SD15):
-                mp['SDXL'] = False
-            elif isinstance(model.model.model_config, comfy.supported_models.SDXL):
-                mp['SDXL'] = True
-            else:
-                print('Base model type: ', type(model.model.model_config))
-                raise Exception("Unsupported model type: ", type(model.model.model_config))
-
-            mp['unet'] = model.model.diffusion_model
-            mp['step'] = 0
-            mp['total_steps'] = 1
-        #
-        #######################################################################################
-
         def callback(step, x0, x, total_steps):
             preview_bytes = None
-            if to is not None and "model_patch" in to:
-                to['model_patch']['step'] = step + 1
             if previewer:
                 preview_bytes = previewer.decode_latent_to_preview_image(preview_format, x0)
             pbar.update_absolute(step + 1, total_steps, preview_bytes)
 
-        samples = comfy.sample.sample_custom(model, noise, cfg, _sampler, sigmas, positive, negative, latent_image,
-                                             noise_mask=noise_mask, callback=callback, disable_pbar=disable_pbar,
-                                             seed=seed)
+        # samples = comfy.sample.sample_custom(model, noise, cfg, _sampler, sigmas, positive, negative, latent_image,
+        #                                      noise_mask=noise_mask, callback=callback, disable_pbar=disable_pbar,
+        #                                      seed=seed)
+
+        samples = comfy.samplers.sample(model, noise, positive, negative, cfg, device, _sampler, sigmas, latent_image=latent_image, model_options=model.model_options,
+               denoise_mask=noise_mask, callback=callback, disable_pbar=disable_pbar, seed=seed)
 
         out = latent.copy()
         out["samples"] = samples
         return out
 
+    def custom_advanced_ksampler(self, noise, guider, sampler, sigmas, latent_image):
+        latent = latent_image
+        latent_image = latent["samples"]
+
+        noise_mask = None
+        if "noise_mask" in latent:
+            noise_mask = latent["noise_mask"]
+
+        x0_output = {}
+        callback = latent_preview.prepare_callback(guider.model_patcher, sigmas.shape[-1] - 1, x0_output)
+
+        disable_pbar = not comfy.utils.PROGRESS_BAR_ENABLED
+        samples = guider.sample(noise.generate_noise(latent), latent_image, sampler, sigmas, denoise_mask=noise_mask,
+                                callback=callback, disable_pbar=disable_pbar, seed=noise.seed)
+        samples = samples.to(comfy.model_management.intermediate_device())
+
+        out = latent.copy()
+        out["samples"] = samples
+        if "x0" in x0_output:
+            out_denoised = latent.copy()
+            out_denoised["samples"] = guider.model_patcher.model.process_latent_out(x0_output["x0"].cpu())
+        else:
+            out_denoised = out
+
+        return (out, out_denoised)
+
     def get_value_by_id(self, key: str, my_unique_id: Any) -> Optional[Any]:
         """Retrieve value by its associated ID."""
         try:
@@ -273,6 +273,19 @@ class easySampler:
             sdxl_pipe.get("seed")
         )
 
+def loglinear_interp(t_steps, num_steps):
+    """
+    Performs log-linear interpolation of a given array of decreasing numbers.
+    """
+    xs = np.linspace(0, 1, len(t_steps))
+    ys = np.log(t_steps[::-1])
+
+    new_xs = np.linspace(0, 1, num_steps)
+    new_ys = np.interp(new_xs, xs, ys)
+
+    interped_ys = np.exp(new_ys)[::-1].copy()
+    return interped_ys
+
 class alignYourStepsScheduler:
 
     NOISE_LEVELS = {
@@ -282,20 +295,6 @@ class alignYourStepsScheduler:
                  0.3798540708, 0.2332364134, 0.1114188177, 0.0291671582],
         "SVD": [700.00, 54.5, 15.886, 7.977, 4.248, 1.789, 0.981, 0.403, 0.173, 0.034, 0.002]}
 
-
-    def loglinear_interp(self, t_steps, num_steps):
-        """
-        Performs log-linear interpolation of a given array of decreasing numbers.
-        """
-        xs = np.linspace(0, 1, len(t_steps))
-        ys = np.log(t_steps[::-1])
-
-        new_xs = np.linspace(0, 1, num_steps)
-        new_ys = np.interp(new_xs, xs, ys)
-
-        interped_ys = np.exp(new_ys)[::-1].copy()
-        return interped_ys
-
     def get_sigmas(self, model_type, steps, denoise):
 
         total_steps = steps
@@ -306,8 +305,607 @@ class alignYourStepsScheduler:
 
         sigmas = self.NOISE_LEVELS[model_type][:]
         if (steps + 1) != len(sigmas):
-            sigmas = self.loglinear_interp(sigmas, steps + 1)
+            sigmas = loglinear_interp(sigmas, steps + 1)
 
         sigmas = sigmas[-(total_steps + 1):]
         sigmas[-1] = 0
-        return (torch.FloatTensor(sigmas),)
+        return (torch.FloatTensor(sigmas),)
+
+
+class gitsScheduler:
+
+    NOISE_LEVELS = {
+        0.80: [
+            [14.61464119, 7.49001646, 0.02916753],
+            [14.61464119, 11.54541874, 6.77309084, 0.02916753],
+            [14.61464119, 11.54541874, 7.49001646, 3.07277966, 0.02916753],
+            [14.61464119, 11.54541874, 7.49001646, 5.85520077, 2.05039096, 0.02916753],
+            [14.61464119, 12.2308979, 8.75849152, 7.49001646, 5.85520077, 2.05039096, 0.02916753],
+            [14.61464119, 12.2308979, 8.75849152, 7.49001646, 5.85520077, 3.07277966, 1.56271636, 0.02916753],
+            [14.61464119, 12.96784878, 11.54541874, 8.75849152, 7.49001646, 5.85520077, 3.07277966, 1.56271636,
+             0.02916753],
+            [14.61464119, 13.76078796, 12.2308979, 10.90732002, 8.75849152, 7.49001646, 5.85520077, 3.07277966,
+             1.56271636, 0.02916753],
+            [14.61464119, 13.76078796, 12.96784878, 12.2308979, 10.90732002, 8.75849152, 7.49001646, 5.85520077,
+             3.07277966, 1.56271636, 0.02916753],
+            [14.61464119, 13.76078796, 12.96784878, 12.2308979, 10.90732002, 9.24142551, 8.30717278, 7.49001646,
+             5.85520077, 3.07277966, 1.56271636, 0.02916753],
+            [14.61464119, 13.76078796, 12.96784878, 12.2308979, 10.90732002, 9.24142551, 8.30717278, 7.49001646,
+             6.14220476, 4.86714602, 3.07277966, 1.56271636, 0.02916753],
+            [14.61464119, 13.76078796, 12.96784878, 12.2308979, 11.54541874, 10.31284904, 9.24142551, 8.30717278,
+             7.49001646, 6.14220476, 4.86714602, 3.07277966, 1.56271636, 0.02916753],
+            [14.61464119, 13.76078796, 12.96784878, 12.2308979, 11.54541874, 10.90732002, 10.31284904, 9.24142551,
+             8.30717278, 7.49001646, 6.14220476, 4.86714602, 3.07277966, 1.56271636, 0.02916753],
+            [14.61464119, 13.76078796, 12.96784878, 12.2308979, 11.54541874, 10.90732002, 10.31284904, 9.24142551,
+             8.75849152, 8.30717278, 7.49001646, 6.14220476, 4.86714602, 3.07277966, 1.56271636, 0.02916753],
+            [14.61464119, 13.76078796, 12.96784878, 12.2308979, 11.54541874, 10.90732002, 10.31284904, 9.24142551,
+             8.75849152, 8.30717278, 7.49001646, 6.14220476, 4.86714602, 3.1956799, 1.98035145, 0.86115354, 0.02916753],
+            [14.61464119, 13.76078796, 12.96784878, 12.2308979, 11.54541874, 10.90732002, 10.31284904, 9.75859547,
+             9.24142551, 8.75849152, 8.30717278, 7.49001646, 6.14220476, 4.86714602, 3.1956799, 1.98035145, 0.86115354,
+             0.02916753],
+            [14.61464119, 13.76078796, 12.96784878, 12.2308979, 11.54541874, 10.90732002, 10.31284904, 9.75859547,
+             9.24142551, 8.75849152, 8.30717278, 7.49001646, 6.77309084, 5.85520077, 4.65472794, 3.07277966, 1.84880662,
+             0.83188516, 0.02916753],
+            [14.61464119, 13.76078796, 12.96784878, 12.2308979, 11.54541874, 10.90732002, 10.31284904, 9.75859547,
+             9.24142551, 8.75849152, 8.30717278, 7.88507891, 7.49001646, 6.77309084, 5.85520077, 4.65472794, 3.07277966,
+             1.84880662, 0.83188516, 0.02916753],
+            [14.61464119, 13.76078796, 12.96784878, 12.2308979, 11.54541874, 10.90732002, 10.31284904, 9.75859547,
+             9.24142551, 8.75849152, 8.30717278, 7.88507891, 7.49001646, 6.77309084, 5.85520077, 4.86714602, 3.75677586,
+             2.84484982, 1.78698075, 0.803307, 0.02916753],
+        ],
+        0.85: [
+            [14.61464119, 7.49001646, 0.02916753],
+            [14.61464119, 7.49001646, 1.84880662, 0.02916753],
+            [14.61464119, 11.54541874, 6.77309084, 1.56271636, 0.02916753],
+            [14.61464119, 11.54541874, 7.11996698, 3.07277966, 1.24153244, 0.02916753],
+            [14.61464119, 11.54541874, 7.49001646, 5.09240818, 2.84484982, 0.95350921, 0.02916753],
+            [14.61464119, 12.2308979, 8.75849152, 7.49001646, 5.09240818, 2.84484982, 0.95350921, 0.02916753],
+            [14.61464119, 12.2308979, 8.75849152, 7.49001646, 5.58536053, 3.1956799, 1.84880662, 0.803307, 0.02916753],
+            [14.61464119, 12.96784878, 11.54541874, 8.75849152, 7.49001646, 5.58536053, 3.1956799, 1.84880662, 0.803307,
+             0.02916753],
+            [14.61464119, 12.96784878, 11.54541874, 8.75849152, 7.49001646, 6.14220476, 4.65472794, 3.07277966,
+             1.84880662, 0.803307, 0.02916753],
+            [14.61464119, 13.76078796, 12.2308979, 10.90732002, 8.75849152, 7.49001646, 6.14220476, 4.65472794,
+             3.07277966, 1.84880662, 0.803307, 0.02916753],
+            [14.61464119, 13.76078796, 12.2308979, 10.90732002, 9.24142551, 8.30717278, 7.49001646, 6.14220476,
+             4.65472794, 3.07277966, 1.84880662, 0.803307, 0.02916753],
+            [14.61464119, 13.76078796, 12.96784878, 12.2308979, 10.90732002, 9.24142551, 8.30717278, 7.49001646,
+             6.14220476, 4.65472794, 3.07277966, 1.84880662, 0.803307, 0.02916753],
+            [14.61464119, 13.76078796, 12.96784878, 12.2308979, 11.54541874, 10.31284904, 9.24142551, 8.30717278,
+             7.49001646, 6.14220476, 4.65472794, 3.07277966, 1.84880662, 0.803307, 0.02916753],
+            [14.61464119, 13.76078796, 12.96784878, 12.2308979, 11.54541874, 10.31284904, 9.24142551, 8.30717278,
+             7.49001646, 6.14220476, 4.86714602, 3.60512662, 2.6383388, 1.56271636, 0.72133851, 0.02916753],
+            [14.61464119, 13.76078796, 12.96784878, 12.2308979, 11.54541874, 10.31284904, 9.24142551, 8.30717278,
+             7.49001646, 6.77309084, 5.85520077, 4.65472794, 3.46139455, 2.45070267, 1.56271636, 0.72133851,
+             0.02916753],
+            [14.61464119, 13.76078796, 12.96784878, 12.2308979, 11.54541874, 10.31284904, 9.24142551, 8.75849152,
+             8.30717278, 7.49001646, 6.77309084, 5.85520077, 4.65472794, 3.46139455, 2.45070267, 1.56271636, 0.72133851,
+             0.02916753],
+            [14.61464119, 13.76078796, 12.96784878, 12.2308979, 11.54541874, 10.90732002, 10.31284904, 9.24142551,
+             8.75849152, 8.30717278, 7.49001646, 6.77309084, 5.85520077, 4.65472794, 3.46139455, 2.45070267, 1.56271636,
+             0.72133851, 0.02916753],
+            [14.61464119, 13.76078796, 12.96784878, 12.2308979, 11.54541874, 10.90732002, 10.31284904, 9.75859547,
+             9.24142551, 8.75849152, 8.30717278, 7.49001646, 6.77309084, 5.85520077, 4.65472794, 3.46139455, 2.45070267,
+             1.56271636, 0.72133851, 0.02916753],
+            [14.61464119, 13.76078796, 12.96784878, 12.2308979, 11.54541874, 10.90732002, 10.31284904, 9.75859547,
+             9.24142551, 8.75849152, 8.30717278, 7.88507891, 7.49001646, 6.77309084, 5.85520077, 4.65472794, 3.46139455,
+             2.45070267, 1.56271636, 0.72133851, 0.02916753],
+        ],
+        0.90: [
+            [14.61464119, 6.77309084, 0.02916753],
+            [14.61464119, 7.49001646, 1.56271636, 0.02916753],
+            [14.61464119, 7.49001646, 3.07277966, 0.95350921, 0.02916753],
+            [14.61464119, 7.49001646, 4.86714602, 2.54230714, 0.89115214, 0.02916753],
+            [14.61464119, 11.54541874, 7.49001646, 4.86714602, 2.54230714, 0.89115214, 0.02916753],
+            [14.61464119, 11.54541874, 7.49001646, 5.09240818, 3.07277966, 1.61558151, 0.69515091, 0.02916753],
+            [14.61464119, 12.2308979, 8.75849152, 7.11996698, 4.86714602, 3.07277966, 1.61558151, 0.69515091,
+             0.02916753],
+            [14.61464119, 12.2308979, 8.75849152, 7.49001646, 5.85520077, 4.45427561, 2.95596409, 1.61558151,
+             0.69515091, 0.02916753],
+            [14.61464119, 12.2308979, 8.75849152, 7.49001646, 5.85520077, 4.45427561, 3.1956799, 2.19988537, 1.24153244,
+             0.57119018, 0.02916753],
+            [14.61464119, 12.96784878, 10.90732002, 8.75849152, 7.49001646, 5.85520077, 4.45427561, 3.1956799,
+             2.19988537, 1.24153244, 0.57119018, 0.02916753],
+            [14.61464119, 12.96784878, 11.54541874, 9.24142551, 8.30717278, 7.49001646, 5.85520077, 4.45427561,
+             3.1956799, 2.19988537, 1.24153244, 0.57119018, 0.02916753],
+            [14.61464119, 12.96784878, 11.54541874, 9.24142551, 8.30717278, 7.49001646, 6.14220476, 4.86714602,
+             3.75677586, 2.84484982, 1.84880662, 1.08895338, 0.52423614, 0.02916753],
+            [14.61464119, 13.76078796, 12.2308979, 10.90732002, 9.24142551, 8.30717278, 7.49001646, 6.14220476,
+             4.86714602, 3.75677586, 2.84484982, 1.84880662, 1.08895338, 0.52423614, 0.02916753],
+            [14.61464119, 13.76078796, 12.2308979, 10.90732002, 9.24142551, 8.30717278, 7.49001646, 6.44769001,
+             5.58536053, 4.45427561, 3.32507086, 2.45070267, 1.61558151, 0.95350921, 0.45573691, 0.02916753],
+            [14.61464119, 13.76078796, 12.96784878, 12.2308979, 10.90732002, 9.24142551, 8.30717278, 7.49001646,
+             6.44769001, 5.58536053, 4.45427561, 3.32507086, 2.45070267, 1.61558151, 0.95350921, 0.45573691,
+             0.02916753],
+            [14.61464119, 13.76078796, 12.96784878, 12.2308979, 10.90732002, 9.24142551, 8.30717278, 7.49001646,
+             6.77309084, 5.85520077, 4.86714602, 3.91689563, 3.07277966, 2.27973175, 1.56271636, 0.95350921, 0.45573691,
+             0.02916753],
+            [14.61464119, 13.76078796, 12.96784878, 12.2308979, 11.54541874, 10.31284904, 9.24142551, 8.30717278,
+             7.49001646, 6.77309084, 5.85520077, 4.86714602, 3.91689563, 3.07277966, 2.27973175, 1.56271636, 0.95350921,
+             0.45573691, 0.02916753],
+            [14.61464119, 13.76078796, 12.96784878, 12.2308979, 11.54541874, 10.31284904, 9.24142551, 8.75849152,
+             8.30717278, 7.49001646, 6.77309084, 5.85520077, 4.86714602, 3.91689563, 3.07277966, 2.27973175, 1.56271636,
+             0.95350921, 0.45573691, 0.02916753],
+            [14.61464119, 13.76078796, 12.96784878, 12.2308979, 11.54541874, 10.31284904, 9.24142551, 8.75849152,
+             8.30717278, 7.49001646, 6.77309084, 5.85520077, 5.09240818, 4.45427561, 3.60512662, 2.95596409, 2.19988537,
+             1.51179266, 0.89115214, 0.43325692, 0.02916753],
+        ],
+        0.95: [
+            [14.61464119, 6.77309084, 0.02916753],
+            [14.61464119, 6.77309084, 1.56271636, 0.02916753],
+            [14.61464119, 7.49001646, 2.84484982, 0.89115214, 0.02916753],
+            [14.61464119, 7.49001646, 4.86714602, 2.36326075, 0.803307, 0.02916753],
+            [14.61464119, 7.49001646, 4.86714602, 2.95596409, 1.56271636, 0.64427125, 0.02916753],
+            [14.61464119, 11.54541874, 7.49001646, 4.86714602, 2.95596409, 1.56271636, 0.64427125, 0.02916753],
+            [14.61464119, 11.54541874, 7.49001646, 4.86714602, 3.07277966, 1.91321158, 1.08895338, 0.50118381,
+             0.02916753],
+            [14.61464119, 11.54541874, 7.49001646, 5.85520077, 4.45427561, 3.07277966, 1.91321158, 1.08895338,
+             0.50118381, 0.02916753],
+            [14.61464119, 12.2308979, 8.75849152, 7.49001646, 5.85520077, 4.45427561, 3.07277966, 1.91321158,
+             1.08895338, 0.50118381, 0.02916753],
+            [14.61464119, 12.2308979, 8.75849152, 7.49001646, 5.85520077, 4.45427561, 3.1956799, 2.19988537, 1.41535246,
+             0.803307, 0.38853383, 0.02916753],
+            [14.61464119, 12.2308979, 8.75849152, 7.49001646, 5.85520077, 4.65472794, 3.46139455, 2.6383388, 1.84880662,
+             1.24153244, 0.72133851, 0.34370604, 0.02916753],
+            [14.61464119, 12.96784878, 10.90732002, 8.75849152, 7.49001646, 5.85520077, 4.65472794, 3.46139455,
+             2.6383388, 1.84880662, 1.24153244, 0.72133851, 0.34370604, 0.02916753],
+            [14.61464119, 12.96784878, 10.90732002, 8.75849152, 7.49001646, 6.14220476, 4.86714602, 3.75677586,
+             2.95596409, 2.19988537, 1.56271636, 1.05362725, 0.64427125, 0.32104823, 0.02916753],
+            [14.61464119, 12.96784878, 10.90732002, 8.75849152, 7.49001646, 6.44769001, 5.58536053, 4.65472794,
+             3.60512662, 2.95596409, 2.19988537, 1.56271636, 1.05362725, 0.64427125, 0.32104823, 0.02916753],
+            [14.61464119, 12.96784878, 11.54541874, 9.24142551, 8.30717278, 7.49001646, 6.44769001, 5.58536053,
+             4.65472794, 3.60512662, 2.95596409, 2.19988537, 1.56271636, 1.05362725, 0.64427125, 0.32104823,
+             0.02916753],
+            [14.61464119, 12.96784878, 11.54541874, 9.24142551, 8.30717278, 7.49001646, 6.44769001, 5.58536053,
+             4.65472794, 3.75677586, 3.07277966, 2.45070267, 1.78698075, 1.24153244, 0.83188516, 0.50118381, 0.22545385,
+             0.02916753],
+            [14.61464119, 12.96784878, 11.54541874, 9.24142551, 8.30717278, 7.49001646, 6.77309084, 5.85520077,
+             5.09240818, 4.45427561, 3.60512662, 2.95596409, 2.36326075, 1.72759056, 1.24153244, 0.83188516, 0.50118381,
+             0.22545385, 0.02916753],
+            [14.61464119, 13.76078796, 12.2308979, 10.90732002, 9.24142551, 8.30717278, 7.49001646, 6.77309084,
+             5.85520077, 5.09240818, 4.45427561, 3.60512662, 2.95596409, 2.36326075, 1.72759056, 1.24153244, 0.83188516,
+             0.50118381, 0.22545385, 0.02916753],
+            [14.61464119, 13.76078796, 12.2308979, 10.90732002, 9.24142551, 8.30717278, 7.49001646, 6.77309084,
+             5.85520077, 5.09240818, 4.45427561, 3.75677586, 3.07277966, 2.45070267, 1.91321158, 1.46270394, 1.05362725,
+             0.72133851, 0.43325692, 0.19894916, 0.02916753],
+        ],
+        1.00: [
+            [14.61464119, 1.56271636, 0.02916753],
+            [14.61464119, 6.77309084, 0.95350921, 0.02916753],
+            [14.61464119, 6.77309084, 2.36326075, 0.803307, 0.02916753],
+            [14.61464119, 7.11996698, 3.07277966, 1.56271636, 0.59516323, 0.02916753],
+            [14.61464119, 7.49001646, 4.86714602, 2.84484982, 1.41535246, 0.57119018, 0.02916753],
+            [14.61464119, 7.49001646, 4.86714602, 2.84484982, 1.61558151, 0.86115354, 0.38853383, 0.02916753],
+            [14.61464119, 11.54541874, 7.49001646, 4.86714602, 2.84484982, 1.61558151, 0.86115354, 0.38853383,
+             0.02916753],
+            [14.61464119, 11.54541874, 7.49001646, 4.86714602, 3.07277966, 1.98035145, 1.24153244, 0.72133851,
+             0.34370604, 0.02916753],
+            [14.61464119, 11.54541874, 7.49001646, 5.85520077, 4.45427561, 3.07277966, 1.98035145, 1.24153244,
+             0.72133851, 0.34370604, 0.02916753],
+            [14.61464119, 11.54541874, 7.49001646, 5.85520077, 4.45427561, 3.1956799, 2.27973175, 1.51179266,
+             0.95350921, 0.54755926, 0.25053367, 0.02916753],
+            [14.61464119, 11.54541874, 7.49001646, 5.85520077, 4.45427561, 3.1956799, 2.36326075, 1.61558151,
+             1.08895338, 0.72133851, 0.41087446, 0.17026083, 0.02916753],
+            [14.61464119, 11.54541874, 8.75849152, 7.49001646, 5.85520077, 4.45427561, 3.1956799, 2.36326075,
+             1.61558151, 1.08895338, 0.72133851, 0.41087446, 0.17026083, 0.02916753],
+            [14.61464119, 11.54541874, 8.75849152, 7.49001646, 5.85520077, 4.65472794, 3.60512662, 2.84484982,
+             2.12350607, 1.56271636, 1.08895338, 0.72133851, 0.41087446, 0.17026083, 0.02916753],
+            [14.61464119, 11.54541874, 8.75849152, 7.49001646, 5.85520077, 4.65472794, 3.60512662, 2.84484982,
+             2.19988537, 1.61558151, 1.162866, 0.803307, 0.50118381, 0.27464288, 0.09824532, 0.02916753],
+            [14.61464119, 11.54541874, 8.75849152, 7.49001646, 5.85520077, 4.65472794, 3.75677586, 3.07277966,
+             2.45070267, 1.84880662, 1.36964464, 1.01931262, 0.72133851, 0.45573691, 0.25053367, 0.09824532,
+             0.02916753],
+            [14.61464119, 11.54541874, 8.75849152, 7.49001646, 6.14220476, 5.09240818, 4.26497746, 3.46139455,
+             2.84484982, 2.19988537, 1.67050016, 1.24153244, 0.92192322, 0.64427125, 0.43325692, 0.25053367, 0.09824532,
+             0.02916753],
+            [14.61464119, 11.54541874, 8.75849152, 7.49001646, 6.14220476, 5.09240818, 4.26497746, 3.60512662,
+             2.95596409, 2.45070267, 1.91321158, 1.51179266, 1.12534678, 0.83188516, 0.59516323, 0.38853383, 0.22545385,
+             0.09824532, 0.02916753],
+            [14.61464119, 12.2308979, 9.24142551, 8.30717278, 7.49001646, 6.14220476, 5.09240818, 4.26497746,
+             3.60512662, 2.95596409, 2.45070267, 1.91321158, 1.51179266, 1.12534678, 0.83188516, 0.59516323, 0.38853383,
+             0.22545385, 0.09824532, 0.02916753],
+            [14.61464119, 12.2308979, 9.24142551, 8.30717278, 7.49001646, 6.77309084, 5.85520077, 5.09240818,
+             4.26497746, 3.60512662, 2.95596409, 2.45070267, 1.91321158, 1.51179266, 1.12534678, 0.83188516, 0.59516323,
+             0.38853383, 0.22545385, 0.09824532, 0.02916753],
+        ],
+        1.05: [
+            [14.61464119, 0.95350921, 0.02916753],
+            [14.61464119, 6.77309084, 0.89115214, 0.02916753],
+            [14.61464119, 6.77309084, 2.05039096, 0.72133851, 0.02916753],
+            [14.61464119, 6.77309084, 2.84484982, 1.28281462, 0.52423614, 0.02916753],
+            [14.61464119, 6.77309084, 3.07277966, 1.61558151, 0.803307, 0.34370604, 0.02916753],
+            [14.61464119, 7.49001646, 4.86714602, 2.84484982, 1.56271636, 0.803307, 0.34370604, 0.02916753],
+            [14.61464119, 7.49001646, 4.86714602, 2.84484982, 1.61558151, 0.95350921, 0.52423614, 0.22545385,
+             0.02916753],
+            [14.61464119, 7.49001646, 4.86714602, 3.07277966, 1.98035145, 1.24153244, 0.74807048, 0.41087446,
+             0.17026083, 0.02916753],
+            [14.61464119, 7.49001646, 4.86714602, 3.1956799, 2.27973175, 1.51179266, 0.95350921, 0.59516323, 0.34370604,
+             0.13792117, 0.02916753],
+            [14.61464119, 7.49001646, 5.09240818, 3.46139455, 2.45070267, 1.61558151, 1.08895338, 0.72133851,
+             0.45573691, 0.25053367, 0.09824532, 0.02916753],
+            [14.61464119, 11.54541874, 7.49001646, 5.09240818, 3.46139455, 2.45070267, 1.61558151, 1.08895338,
+             0.72133851, 0.45573691, 0.25053367, 0.09824532, 0.02916753],
+            [14.61464119, 11.54541874, 7.49001646, 5.85520077, 4.45427561, 3.1956799, 2.36326075, 1.61558151,
+             1.08895338, 0.72133851, 0.45573691, 0.25053367, 0.09824532, 0.02916753],
+            [14.61464119, 11.54541874, 7.49001646, 5.85520077, 4.45427561, 3.1956799, 2.45070267, 1.72759056,
+             1.24153244, 0.86115354, 0.59516323, 0.38853383, 0.22545385, 0.09824532, 0.02916753],
+            [14.61464119, 11.54541874, 7.49001646, 5.85520077, 4.65472794, 3.60512662, 2.84484982, 2.19988537,
+             1.61558151, 1.162866, 0.83188516, 0.59516323, 0.38853383, 0.22545385, 0.09824532, 0.02916753],
+            [14.61464119, 11.54541874, 7.49001646, 5.85520077, 4.65472794, 3.60512662, 2.84484982, 2.19988537,
+             1.67050016, 1.28281462, 0.95350921, 0.72133851, 0.52423614, 0.34370604, 0.19894916, 0.09824532,
+             0.02916753],
+            [14.61464119, 11.54541874, 7.49001646, 5.85520077, 4.65472794, 3.60512662, 2.95596409, 2.36326075,
+             1.84880662, 1.41535246, 1.08895338, 0.83188516, 0.61951244, 0.45573691, 0.32104823, 0.19894916, 0.09824532,
+             0.02916753],
+            [14.61464119, 11.54541874, 7.49001646, 5.85520077, 4.65472794, 3.60512662, 2.95596409, 2.45070267,
+             1.91321158, 1.51179266, 1.20157266, 0.95350921, 0.74807048, 0.57119018, 0.43325692, 0.29807833, 0.19894916,
+             0.09824532, 0.02916753],
+            [14.61464119, 11.54541874, 8.30717278, 7.11996698, 5.85520077, 4.65472794, 3.60512662, 2.95596409,
+             2.45070267, 1.91321158, 1.51179266, 1.20157266, 0.95350921, 0.74807048, 0.57119018, 0.43325692, 0.29807833,
+             0.19894916, 0.09824532, 0.02916753],
+            [14.61464119, 11.54541874, 8.30717278, 7.11996698, 5.85520077, 4.65472794, 3.60512662, 2.95596409,
+             2.45070267, 1.98035145, 1.61558151, 1.32549286, 1.08895338, 0.86115354, 0.69515091, 0.54755926, 0.41087446,
+             0.29807833, 0.19894916, 0.09824532, 0.02916753],
+        ],
+        1.10: [
+            [14.61464119, 0.89115214, 0.02916753],
+            [14.61464119, 2.36326075, 0.72133851, 0.02916753],
+            [14.61464119, 5.85520077, 1.61558151, 0.57119018, 0.02916753],
+            [14.61464119, 6.77309084, 2.45070267, 1.08895338, 0.45573691, 0.02916753],
+            [14.61464119, 6.77309084, 2.95596409, 1.56271636, 0.803307, 0.34370604, 0.02916753],
+            [14.61464119, 6.77309084, 3.07277966, 1.61558151, 0.89115214, 0.4783645, 0.19894916, 0.02916753],
+            [14.61464119, 6.77309084, 3.07277966, 1.84880662, 1.08895338, 0.64427125, 0.34370604, 0.13792117,
+             0.02916753],
+            [14.61464119, 7.49001646, 4.86714602, 2.84484982, 1.61558151, 0.95350921, 0.54755926, 0.27464288,
+             0.09824532, 0.02916753],
+            [14.61464119, 7.49001646, 4.86714602, 2.95596409, 1.91321158, 1.24153244, 0.803307, 0.4783645, 0.25053367,
+             0.09824532, 0.02916753],
+            [14.61464119, 7.49001646, 4.86714602, 3.07277966, 2.05039096, 1.41535246, 0.95350921, 0.64427125,
+             0.41087446, 0.22545385, 0.09824532, 0.02916753],
+            [14.61464119, 7.49001646, 4.86714602, 3.1956799, 2.27973175, 1.61558151, 1.12534678, 0.803307, 0.54755926,
+             0.36617002, 0.22545385, 0.09824532, 0.02916753],
+            [14.61464119, 7.49001646, 4.86714602, 3.32507086, 2.45070267, 1.72759056, 1.24153244, 0.89115214,
+             0.64427125, 0.45573691, 0.32104823, 0.19894916, 0.09824532, 0.02916753],
+            [14.61464119, 7.49001646, 5.09240818, 3.60512662, 2.84484982, 2.05039096, 1.51179266, 1.08895338, 0.803307,
+             0.59516323, 0.43325692, 0.29807833, 0.19894916, 0.09824532, 0.02916753],
+            [14.61464119, 7.49001646, 5.09240818, 3.60512662, 2.84484982, 2.12350607, 1.61558151, 1.24153244,
+             0.95350921, 0.72133851, 0.54755926, 0.41087446, 0.29807833, 0.19894916, 0.09824532, 0.02916753],
+            [14.61464119, 7.49001646, 5.85520077, 4.45427561, 3.1956799, 2.45070267, 1.84880662, 1.41535246, 1.08895338,
+             0.83188516, 0.64427125, 0.50118381, 0.36617002, 0.25053367, 0.17026083, 0.09824532, 0.02916753],
+            [14.61464119, 7.49001646, 5.85520077, 4.45427561, 3.1956799, 2.45070267, 1.91321158, 1.51179266, 1.20157266,
+             0.95350921, 0.74807048, 0.59516323, 0.45573691, 0.34370604, 0.25053367, 0.17026083, 0.09824532,
+             0.02916753],
+            [14.61464119, 7.49001646, 5.85520077, 4.45427561, 3.46139455, 2.84484982, 2.19988537, 1.72759056,
+             1.36964464, 1.08895338, 0.86115354, 0.69515091, 0.54755926, 0.43325692, 0.34370604, 0.25053367, 0.17026083,
+             0.09824532, 0.02916753],
+            [14.61464119, 11.54541874, 7.49001646, 5.85520077, 4.45427561, 3.46139455, 2.84484982, 2.19988537,
+             1.72759056, 1.36964464, 1.08895338, 0.86115354, 0.69515091, 0.54755926, 0.43325692, 0.34370604, 0.25053367,
+             0.17026083, 0.09824532, 0.02916753],
+            [14.61464119, 11.54541874, 7.49001646, 5.85520077, 4.45427561, 3.46139455, 2.84484982, 2.19988537,
+             1.72759056, 1.36964464, 1.08895338, 0.89115214, 0.72133851, 0.59516323, 0.4783645, 0.38853383, 0.29807833,
+             0.22545385, 0.17026083, 0.09824532, 0.02916753],
+        ],
+        1.15: [
+            [14.61464119, 0.83188516, 0.02916753],
+            [14.61464119, 1.84880662, 0.59516323, 0.02916753],
+            [14.61464119, 5.85520077, 1.56271636, 0.52423614, 0.02916753],
+            [14.61464119, 5.85520077, 1.91321158, 0.83188516, 0.34370604, 0.02916753],
+            [14.61464119, 5.85520077, 2.45070267, 1.24153244, 0.59516323, 0.25053367, 0.02916753],
+            [14.61464119, 5.85520077, 2.84484982, 1.51179266, 0.803307, 0.41087446, 0.17026083, 0.02916753],
+            [14.61464119, 5.85520077, 2.84484982, 1.56271636, 0.89115214, 0.50118381, 0.25053367, 0.09824532,
+             0.02916753],
+            [14.61464119, 6.77309084, 3.07277966, 1.84880662, 1.12534678, 0.72133851, 0.43325692, 0.22545385,
+             0.09824532, 0.02916753],
+            [14.61464119, 6.77309084, 3.07277966, 1.91321158, 1.24153244, 0.803307, 0.52423614, 0.34370604, 0.19894916,
+             0.09824532, 0.02916753],
+            [14.61464119, 7.49001646, 4.86714602, 2.95596409, 1.91321158, 1.24153244, 0.803307, 0.52423614, 0.34370604,
+             0.19894916, 0.09824532, 0.02916753],
+            [14.61464119, 7.49001646, 4.86714602, 3.07277966, 2.05039096, 1.36964464, 0.95350921, 0.69515091, 0.4783645,
+             0.32104823, 0.19894916, 0.09824532, 0.02916753],
+            [14.61464119, 7.49001646, 4.86714602, 3.07277966, 2.12350607, 1.51179266, 1.08895338, 0.803307, 0.59516323,
+             0.43325692, 0.29807833, 0.19894916, 0.09824532, 0.02916753],
+            [14.61464119, 7.49001646, 4.86714602, 3.07277966, 2.12350607, 1.51179266, 1.08895338, 0.803307, 0.59516323,
+             0.45573691, 0.34370604, 0.25053367, 0.17026083, 0.09824532, 0.02916753],
+            [14.61464119, 7.49001646, 4.86714602, 3.07277966, 2.19988537, 1.61558151, 1.24153244, 0.95350921,
+             0.74807048, 0.59516323, 0.45573691, 0.34370604, 0.25053367, 0.17026083, 0.09824532, 0.02916753],
+            [14.61464119, 7.49001646, 4.86714602, 3.1956799, 2.45070267, 1.78698075, 1.32549286, 1.01931262, 0.803307,
+             0.64427125, 0.50118381, 0.38853383, 0.29807833, 0.22545385, 0.17026083, 0.09824532, 0.02916753],
+            [14.61464119, 7.49001646, 4.86714602, 3.1956799, 2.45070267, 1.78698075, 1.32549286, 1.01931262, 0.803307,
+             0.64427125, 0.52423614, 0.41087446, 0.32104823, 0.25053367, 0.19894916, 0.13792117, 0.09824532,
+             0.02916753],
+            [14.61464119, 7.49001646, 4.86714602, 3.1956799, 2.45070267, 1.84880662, 1.41535246, 1.12534678, 0.89115214,
+             0.72133851, 0.59516323, 0.4783645, 0.38853383, 0.32104823, 0.25053367, 0.19894916, 0.13792117, 0.09824532,
+             0.02916753],
+            [14.61464119, 7.49001646, 4.86714602, 3.1956799, 2.45070267, 1.84880662, 1.41535246, 1.12534678, 0.89115214,
+             0.72133851, 0.59516323, 0.50118381, 0.41087446, 0.34370604, 0.27464288, 0.22545385, 0.17026083, 0.13792117,
+             0.09824532, 0.02916753],
+            [14.61464119, 7.49001646, 4.86714602, 3.1956799, 2.45070267, 1.84880662, 1.41535246, 1.12534678, 0.89115214,
+             0.72133851, 0.59516323, 0.50118381, 0.41087446, 0.34370604, 0.29807833, 0.25053367, 0.19894916, 0.17026083,
+             0.13792117, 0.09824532, 0.02916753],
+        ],
+        1.20: [
+            [14.61464119, 0.803307, 0.02916753],
+            [14.61464119, 1.56271636, 0.52423614, 0.02916753],
+            [14.61464119, 2.36326075, 0.92192322, 0.36617002, 0.02916753],
+            [14.61464119, 2.84484982, 1.24153244, 0.59516323, 0.25053367, 0.02916753],
+            [14.61464119, 5.85520077, 2.05039096, 0.95350921, 0.45573691, 0.17026083, 0.02916753],
+            [14.61464119, 5.85520077, 2.45070267, 1.24153244, 0.64427125, 0.29807833, 0.09824532, 0.02916753],
+            [14.61464119, 5.85520077, 2.45070267, 1.36964464, 0.803307, 0.45573691, 0.25053367, 0.09824532, 0.02916753],
+            [14.61464119, 5.85520077, 2.84484982, 1.61558151, 0.95350921, 0.59516323, 0.36617002, 0.19894916,
+             0.09824532, 0.02916753],
+            [14.61464119, 5.85520077, 2.84484982, 1.67050016, 1.08895338, 0.74807048, 0.50118381, 0.32104823,
+             0.19894916, 0.09824532, 0.02916753],
+            [14.61464119, 5.85520077, 2.95596409, 1.84880662, 1.24153244, 0.83188516, 0.59516323, 0.41087446,
+             0.27464288, 0.17026083, 0.09824532, 0.02916753],
+            [14.61464119, 5.85520077, 3.07277966, 1.98035145, 1.36964464, 0.95350921, 0.69515091, 0.50118381,
+             0.36617002, 0.25053367, 0.17026083, 0.09824532, 0.02916753],
+            [14.61464119, 6.77309084, 3.46139455, 2.36326075, 1.56271636, 1.08895338, 0.803307, 0.59516323, 0.45573691,
+             0.34370604, 0.25053367, 0.17026083, 0.09824532, 0.02916753],
+            [14.61464119, 6.77309084, 3.46139455, 2.45070267, 1.61558151, 1.162866, 0.86115354, 0.64427125, 0.50118381,
+             0.38853383, 0.29807833, 0.22545385, 0.17026083, 0.09824532, 0.02916753],
+            [14.61464119, 7.49001646, 4.65472794, 3.07277966, 2.12350607, 1.51179266, 1.08895338, 0.83188516,
+             0.64427125, 0.50118381, 0.38853383, 0.29807833, 0.22545385, 0.17026083, 0.09824532, 0.02916753],
+            [14.61464119, 7.49001646, 4.65472794, 3.07277966, 2.12350607, 1.51179266, 1.08895338, 0.83188516,
+             0.64427125, 0.50118381, 0.41087446, 0.32104823, 0.25053367, 0.19894916, 0.13792117, 0.09824532,
+             0.02916753],
+            [14.61464119, 7.49001646, 4.65472794, 3.07277966, 2.12350607, 1.51179266, 1.08895338, 0.83188516,
+             0.64427125, 0.50118381, 0.41087446, 0.34370604, 0.27464288, 0.22545385, 0.17026083, 0.13792117, 0.09824532,
+             0.02916753],
+            [14.61464119, 7.49001646, 4.65472794, 3.07277966, 2.19988537, 1.61558151, 1.20157266, 0.92192322,
+             0.72133851, 0.57119018, 0.45573691, 0.36617002, 0.29807833, 0.25053367, 0.19894916, 0.17026083, 0.13792117,
+             0.09824532, 0.02916753],
+            [14.61464119, 7.49001646, 4.65472794, 3.07277966, 2.19988537, 1.61558151, 1.24153244, 0.95350921,
+             0.74807048, 0.59516323, 0.4783645, 0.38853383, 0.32104823, 0.27464288, 0.22545385, 0.19894916, 0.17026083,
+             0.13792117, 0.09824532, 0.02916753],
+            [14.61464119, 7.49001646, 4.65472794, 3.07277966, 2.19988537, 1.61558151, 1.24153244, 0.95350921,
+             0.74807048, 0.59516323, 0.50118381, 0.41087446, 0.34370604, 0.29807833, 0.25053367, 0.22545385, 0.19894916,
+             0.17026083, 0.13792117, 0.09824532, 0.02916753],
+        ],
+        1.25: [
+            [14.61464119, 0.72133851, 0.02916753],
+            [14.61464119, 1.56271636, 0.50118381, 0.02916753],
+            [14.61464119, 2.05039096, 0.803307, 0.32104823, 0.02916753],
+            [14.61464119, 2.36326075, 0.95350921, 0.43325692, 0.17026083, 0.02916753],
+            [14.61464119, 2.84484982, 1.24153244, 0.59516323, 0.27464288, 0.09824532, 0.02916753],
+            [14.61464119, 3.07277966, 1.51179266, 0.803307, 0.43325692, 0.22545385, 0.09824532, 0.02916753],
+            [14.61464119, 5.85520077, 2.36326075, 1.24153244, 0.72133851, 0.41087446, 0.22545385, 0.09824532,
+             0.02916753],
+            [14.61464119, 5.85520077, 2.45070267, 1.36964464, 0.83188516, 0.52423614, 0.34370604, 0.19894916,
+             0.09824532, 0.02916753],
+            [14.61464119, 5.85520077, 2.84484982, 1.61558151, 0.98595673, 0.64427125, 0.43325692, 0.27464288,
+             0.17026083, 0.09824532, 0.02916753],
+            [14.61464119, 5.85520077, 2.84484982, 1.67050016, 1.08895338, 0.74807048, 0.52423614, 0.36617002,
+             0.25053367, 0.17026083, 0.09824532, 0.02916753],
+            [14.61464119, 5.85520077, 2.84484982, 1.72759056, 1.162866, 0.803307, 0.59516323, 0.45573691, 0.34370604,
+             0.25053367, 0.17026083, 0.09824532, 0.02916753],
+            [14.61464119, 5.85520077, 2.95596409, 1.84880662, 1.24153244, 0.86115354, 0.64427125, 0.4783645, 0.36617002,
+             0.27464288, 0.19894916, 0.13792117, 0.09824532, 0.02916753],
+            [14.61464119, 5.85520077, 2.95596409, 1.84880662, 1.28281462, 0.92192322, 0.69515091, 0.52423614,
+             0.41087446, 0.32104823, 0.25053367, 0.19894916, 0.13792117, 0.09824532, 0.02916753],
+            [14.61464119, 5.85520077, 2.95596409, 1.91321158, 1.32549286, 0.95350921, 0.72133851, 0.54755926,
+             0.43325692, 0.34370604, 0.27464288, 0.22545385, 0.17026083, 0.13792117, 0.09824532, 0.02916753],
+            [14.61464119, 5.85520077, 2.95596409, 1.91321158, 1.32549286, 0.95350921, 0.72133851, 0.57119018,
+             0.45573691, 0.36617002, 0.29807833, 0.25053367, 0.19894916, 0.17026083, 0.13792117, 0.09824532,
+             0.02916753],
+            [14.61464119, 5.85520077, 2.95596409, 1.91321158, 1.32549286, 0.95350921, 0.74807048, 0.59516323, 0.4783645,
+             0.38853383, 0.32104823, 0.27464288, 0.22545385, 0.19894916, 0.17026083, 0.13792117, 0.09824532,
+             0.02916753],
+            [14.61464119, 5.85520077, 3.07277966, 2.05039096, 1.41535246, 1.05362725, 0.803307, 0.61951244, 0.50118381,
+             0.41087446, 0.34370604, 0.29807833, 0.25053367, 0.22545385, 0.19894916, 0.17026083, 0.13792117, 0.09824532,
+             0.02916753],
+            [14.61464119, 5.85520077, 3.07277966, 2.05039096, 1.41535246, 1.05362725, 0.803307, 0.64427125, 0.52423614,
+             0.43325692, 0.36617002, 0.32104823, 0.27464288, 0.25053367, 0.22545385, 0.19894916, 0.17026083, 0.13792117,
+             0.09824532, 0.02916753],
+            [14.61464119, 5.85520077, 3.07277966, 2.05039096, 1.46270394, 1.08895338, 0.83188516, 0.66947293,
+             0.54755926, 0.45573691, 0.38853383, 0.34370604, 0.29807833, 0.27464288, 0.25053367, 0.22545385, 0.19894916,
+             0.17026083, 0.13792117, 0.09824532, 0.02916753],
+        ],
+        1.30: [
+            [14.61464119, 0.72133851, 0.02916753],
+            [14.61464119, 1.24153244, 0.43325692, 0.02916753],
+            [14.61464119, 1.56271636, 0.59516323, 0.22545385, 0.02916753],
+            [14.61464119, 1.84880662, 0.803307, 0.36617002, 0.13792117, 0.02916753],
+            [14.61464119, 2.36326075, 1.01931262, 0.52423614, 0.25053367, 0.09824532, 0.02916753],
+            [14.61464119, 2.84484982, 1.36964464, 0.74807048, 0.41087446, 0.22545385, 0.09824532, 0.02916753],
+            [14.61464119, 3.07277966, 1.56271636, 0.89115214, 0.54755926, 0.34370604, 0.19894916, 0.09824532,
+             0.02916753],
+            [14.61464119, 3.07277966, 1.61558151, 0.95350921, 0.61951244, 0.41087446, 0.27464288, 0.17026083,
+             0.09824532, 0.02916753],
+            [14.61464119, 5.85520077, 2.45070267, 1.36964464, 0.83188516, 0.54755926, 0.36617002, 0.25053367,
+             0.17026083, 0.09824532, 0.02916753],
+            [14.61464119, 5.85520077, 2.45070267, 1.41535246, 0.92192322, 0.64427125, 0.45573691, 0.34370604,
+             0.25053367, 0.17026083, 0.09824532, 0.02916753],
+            [14.61464119, 5.85520077, 2.6383388, 1.56271636, 1.01931262, 0.72133851, 0.50118381, 0.36617002, 0.27464288,
+             0.19894916, 0.13792117, 0.09824532, 0.02916753],
+            [14.61464119, 5.85520077, 2.84484982, 1.61558151, 1.05362725, 0.74807048, 0.54755926, 0.41087446,
+             0.32104823, 0.25053367, 0.19894916, 0.13792117, 0.09824532, 0.02916753],
+            [14.61464119, 5.85520077, 2.84484982, 1.61558151, 1.08895338, 0.77538133, 0.57119018, 0.43325692,
+             0.34370604, 0.27464288, 0.22545385, 0.17026083, 0.13792117, 0.09824532, 0.02916753],
+            [14.61464119, 5.85520077, 2.84484982, 1.61558151, 1.08895338, 0.803307, 0.59516323, 0.45573691, 0.36617002,
+             0.29807833, 0.25053367, 0.19894916, 0.17026083, 0.13792117, 0.09824532, 0.02916753],
+            [14.61464119, 5.85520077, 2.84484982, 1.61558151, 1.08895338, 0.803307, 0.59516323, 0.4783645, 0.38853383,
+             0.32104823, 0.27464288, 0.22545385, 0.19894916, 0.17026083, 0.13792117, 0.09824532, 0.02916753],
+            [14.61464119, 5.85520077, 2.84484982, 1.72759056, 1.162866, 0.83188516, 0.64427125, 0.50118381, 0.41087446,
+             0.34370604, 0.29807833, 0.25053367, 0.22545385, 0.19894916, 0.17026083, 0.13792117, 0.09824532,
+             0.02916753],
+            [14.61464119, 5.85520077, 2.84484982, 1.72759056, 1.162866, 0.83188516, 0.64427125, 0.52423614, 0.43325692,
+             0.36617002, 0.32104823, 0.27464288, 0.25053367, 0.22545385, 0.19894916, 0.17026083, 0.13792117, 0.09824532,
+             0.02916753],
+            [14.61464119, 5.85520077, 2.84484982, 1.78698075, 1.24153244, 0.92192322, 0.72133851, 0.57119018,
+             0.45573691, 0.38853383, 0.34370604, 0.29807833, 0.27464288, 0.25053367, 0.22545385, 0.19894916, 0.17026083,
+             0.13792117, 0.09824532, 0.02916753],
+            [14.61464119, 5.85520077, 2.84484982, 1.78698075, 1.24153244, 0.92192322, 0.72133851, 0.57119018, 0.4783645,
+             0.41087446, 0.36617002, 0.32104823, 0.29807833, 0.27464288, 0.25053367, 0.22545385, 0.19894916, 0.17026083,
+             0.13792117, 0.09824532, 0.02916753],
+        ],
+        1.35: [
+            [14.61464119, 0.69515091, 0.02916753],
+            [14.61464119, 0.95350921, 0.34370604, 0.02916753],
+            [14.61464119, 1.56271636, 0.57119018, 0.19894916, 0.02916753],
+            [14.61464119, 1.61558151, 0.69515091, 0.29807833, 0.09824532, 0.02916753],
+            [14.61464119, 1.84880662, 0.83188516, 0.43325692, 0.22545385, 0.09824532, 0.02916753],
+            [14.61464119, 2.45070267, 1.162866, 0.64427125, 0.36617002, 0.19894916, 0.09824532, 0.02916753],
+            [14.61464119, 2.84484982, 1.36964464, 0.803307, 0.50118381, 0.32104823, 0.19894916, 0.09824532, 0.02916753],
+            [14.61464119, 2.84484982, 1.41535246, 0.83188516, 0.54755926, 0.36617002, 0.25053367, 0.17026083,
+             0.09824532, 0.02916753],
+            [14.61464119, 2.84484982, 1.56271636, 0.95350921, 0.64427125, 0.45573691, 0.32104823, 0.22545385,
+             0.17026083, 0.09824532, 0.02916753],
+            [14.61464119, 2.84484982, 1.56271636, 0.95350921, 0.64427125, 0.45573691, 0.34370604, 0.25053367,
+             0.19894916, 0.13792117, 0.09824532, 0.02916753],
+            [14.61464119, 3.07277966, 1.61558151, 1.01931262, 0.72133851, 0.52423614, 0.38853383, 0.29807833,
+             0.22545385, 0.17026083, 0.13792117, 0.09824532, 0.02916753],
+            [14.61464119, 3.07277966, 1.61558151, 1.01931262, 0.72133851, 0.52423614, 0.41087446, 0.32104823,
+             0.25053367, 0.19894916, 0.17026083, 0.13792117, 0.09824532, 0.02916753],
+            [14.61464119, 3.07277966, 1.61558151, 1.05362725, 0.74807048, 0.54755926, 0.43325692, 0.34370604,
+             0.27464288, 0.22545385, 0.19894916, 0.17026083, 0.13792117, 0.09824532, 0.02916753],
+            [14.61464119, 3.07277966, 1.72759056, 1.12534678, 0.803307, 0.59516323, 0.45573691, 0.36617002, 0.29807833,
+             0.25053367, 0.22545385, 0.19894916, 0.17026083, 0.13792117, 0.09824532, 0.02916753],
+            [14.61464119, 3.07277966, 1.72759056, 1.12534678, 0.803307, 0.59516323, 0.4783645, 0.38853383, 0.32104823,
+             0.27464288, 0.25053367, 0.22545385, 0.19894916, 0.17026083, 0.13792117, 0.09824532, 0.02916753],
+            [14.61464119, 5.85520077, 2.45070267, 1.51179266, 1.01931262, 0.74807048, 0.57119018, 0.45573691,
+             0.36617002, 0.32104823, 0.27464288, 0.25053367, 0.22545385, 0.19894916, 0.17026083, 0.13792117, 0.09824532,
+             0.02916753],
+            [14.61464119, 5.85520077, 2.6383388, 1.61558151, 1.08895338, 0.803307, 0.61951244, 0.50118381, 0.41087446,
+             0.34370604, 0.29807833, 0.27464288, 0.25053367, 0.22545385, 0.19894916, 0.17026083, 0.13792117, 0.09824532,
+             0.02916753],
+            [14.61464119, 5.85520077, 2.6383388, 1.61558151, 1.08895338, 0.803307, 0.64427125, 0.52423614, 0.43325692,
+             0.36617002, 0.32104823, 0.29807833, 0.27464288, 0.25053367, 0.22545385, 0.19894916, 0.17026083, 0.13792117,
+             0.09824532, 0.02916753],
+            [14.61464119, 5.85520077, 2.6383388, 1.61558151, 1.08895338, 0.803307, 0.64427125, 0.52423614, 0.45573691,
+             0.38853383, 0.34370604, 0.32104823, 0.29807833, 0.27464288, 0.25053367, 0.22545385, 0.19894916, 0.17026083,
+             0.13792117, 0.09824532, 0.02916753],
+        ],
+        1.40: [
+            [14.61464119, 0.59516323, 0.02916753],
+            [14.61464119, 0.95350921, 0.34370604, 0.02916753],
+            [14.61464119, 1.08895338, 0.43325692, 0.13792117, 0.02916753],
+            [14.61464119, 1.56271636, 0.64427125, 0.27464288, 0.09824532, 0.02916753],
+            [14.61464119, 1.61558151, 0.803307, 0.43325692, 0.22545385, 0.09824532, 0.02916753],
+            [14.61464119, 2.05039096, 0.95350921, 0.54755926, 0.34370604, 0.19894916, 0.09824532, 0.02916753],
+            [14.61464119, 2.45070267, 1.24153244, 0.72133851, 0.43325692, 0.27464288, 0.17026083, 0.09824532,
+             0.02916753],
+            [14.61464119, 2.45070267, 1.24153244, 0.74807048, 0.50118381, 0.34370604, 0.25053367, 0.17026083,
+             0.09824532, 0.02916753],
+            [14.61464119, 2.45070267, 1.28281462, 0.803307, 0.52423614, 0.36617002, 0.27464288, 0.19894916, 0.13792117,
+             0.09824532, 0.02916753],
+            [14.61464119, 2.45070267, 1.28281462, 0.803307, 0.54755926, 0.38853383, 0.29807833, 0.22545385, 0.17026083,
+             0.13792117, 0.09824532, 0.02916753],
+            [14.61464119, 2.84484982, 1.41535246, 0.86115354, 0.59516323, 0.43325692, 0.32104823, 0.25053367,
+             0.19894916, 0.17026083, 0.13792117, 0.09824532, 0.02916753],
+            [14.61464119, 2.84484982, 1.51179266, 0.95350921, 0.64427125, 0.45573691, 0.34370604, 0.27464288,
+             0.22545385, 0.19894916, 0.17026083, 0.13792117, 0.09824532, 0.02916753],
+            [14.61464119, 2.84484982, 1.51179266, 0.95350921, 0.64427125, 0.4783645, 0.36617002, 0.29807833, 0.25053367,
+             0.22545385, 0.19894916, 0.17026083, 0.13792117, 0.09824532, 0.02916753],
+            [14.61464119, 2.84484982, 1.56271636, 0.98595673, 0.69515091, 0.52423614, 0.41087446, 0.34370604,
+             0.29807833, 0.25053367, 0.22545385, 0.19894916, 0.17026083, 0.13792117, 0.09824532, 0.02916753],
+            [14.61464119, 2.84484982, 1.56271636, 1.01931262, 0.72133851, 0.54755926, 0.43325692, 0.36617002,
+             0.32104823, 0.27464288, 0.25053367, 0.22545385, 0.19894916, 0.17026083, 0.13792117, 0.09824532,
+             0.02916753],
+            [14.61464119, 2.84484982, 1.61558151, 1.05362725, 0.74807048, 0.57119018, 0.45573691, 0.38853383,
+             0.34370604, 0.29807833, 0.27464288, 0.25053367, 0.22545385, 0.19894916, 0.17026083, 0.13792117, 0.09824532,
+             0.02916753],
+            [14.61464119, 2.84484982, 1.61558151, 1.08895338, 0.803307, 0.61951244, 0.50118381, 0.41087446, 0.36617002,
+             0.32104823, 0.29807833, 0.27464288, 0.25053367, 0.22545385, 0.19894916, 0.17026083, 0.13792117, 0.09824532,
+             0.02916753],
+            [14.61464119, 2.84484982, 1.61558151, 1.08895338, 0.803307, 0.61951244, 0.50118381, 0.43325692, 0.38853383,
+             0.34370604, 0.32104823, 0.29807833, 0.27464288, 0.25053367, 0.22545385, 0.19894916, 0.17026083, 0.13792117,
+             0.09824532, 0.02916753],
+            [14.61464119, 2.84484982, 1.61558151, 1.08895338, 0.803307, 0.64427125, 0.52423614, 0.45573691, 0.41087446,
+             0.36617002, 0.34370604, 0.32104823, 0.29807833, 0.27464288, 0.25053367, 0.22545385, 0.19894916, 0.17026083,
+             0.13792117, 0.09824532, 0.02916753],
+        ],
+        1.45: [
+            [14.61464119, 0.59516323, 0.02916753],
+            [14.61464119, 0.803307, 0.25053367, 0.02916753],
+            [14.61464119, 0.95350921, 0.34370604, 0.09824532, 0.02916753],
+            [14.61464119, 1.24153244, 0.54755926, 0.25053367, 0.09824532, 0.02916753],
+            [14.61464119, 1.56271636, 0.72133851, 0.36617002, 0.19894916, 0.09824532, 0.02916753],
+            [14.61464119, 1.61558151, 0.803307, 0.45573691, 0.27464288, 0.17026083, 0.09824532, 0.02916753],
+            [14.61464119, 1.91321158, 0.95350921, 0.57119018, 0.36617002, 0.25053367, 0.17026083, 0.09824532,
+             0.02916753],
+            [14.61464119, 2.19988537, 1.08895338, 0.64427125, 0.41087446, 0.27464288, 0.19894916, 0.13792117,
+             0.09824532, 0.02916753],
+            [14.61464119, 2.45070267, 1.24153244, 0.74807048, 0.50118381, 0.34370604, 0.25053367, 0.19894916,
+             0.13792117, 0.09824532, 0.02916753],
+            [14.61464119, 2.45070267, 1.24153244, 0.74807048, 0.50118381, 0.36617002, 0.27464288, 0.22545385,
+             0.17026083, 0.13792117, 0.09824532, 0.02916753],
+            [14.61464119, 2.45070267, 1.28281462, 0.803307, 0.54755926, 0.41087446, 0.32104823, 0.25053367, 0.19894916,
+             0.17026083, 0.13792117, 0.09824532, 0.02916753],
+            [14.61464119, 2.45070267, 1.28281462, 0.803307, 0.57119018, 0.43325692, 0.34370604, 0.27464288, 0.22545385,
+             0.19894916, 0.17026083, 0.13792117, 0.09824532, 0.02916753],
+            [14.61464119, 2.45070267, 1.28281462, 0.83188516, 0.59516323, 0.45573691, 0.36617002, 0.29807833,
+             0.25053367, 0.22545385, 0.19894916, 0.17026083, 0.13792117, 0.09824532, 0.02916753],
+            [14.61464119, 2.45070267, 1.28281462, 0.83188516, 0.59516323, 0.45573691, 0.36617002, 0.32104823,
+             0.27464288, 0.25053367, 0.22545385, 0.19894916, 0.17026083, 0.13792117, 0.09824532, 0.02916753],
+            [14.61464119, 2.84484982, 1.51179266, 0.95350921, 0.69515091, 0.52423614, 0.41087446, 0.34370604,
+             0.29807833, 0.27464288, 0.25053367, 0.22545385, 0.19894916, 0.17026083, 0.13792117, 0.09824532,
+             0.02916753],
+            [14.61464119, 2.84484982, 1.51179266, 0.95350921, 0.69515091, 0.52423614, 0.43325692, 0.36617002,
+             0.32104823, 0.29807833, 0.27464288, 0.25053367, 0.22545385, 0.19894916, 0.17026083, 0.13792117, 0.09824532,
+             0.02916753],
+            [14.61464119, 2.84484982, 1.56271636, 0.98595673, 0.72133851, 0.54755926, 0.45573691, 0.38853383,
+             0.34370604, 0.32104823, 0.29807833, 0.27464288, 0.25053367, 0.22545385, 0.19894916, 0.17026083, 0.13792117,
+             0.09824532, 0.02916753],
+            [14.61464119, 2.84484982, 1.56271636, 1.01931262, 0.74807048, 0.57119018, 0.4783645, 0.41087446, 0.36617002,
+             0.34370604, 0.32104823, 0.29807833, 0.27464288, 0.25053367, 0.22545385, 0.19894916, 0.17026083, 0.13792117,
+             0.09824532, 0.02916753],
+            [14.61464119, 2.84484982, 1.56271636, 1.01931262, 0.74807048, 0.59516323, 0.50118381, 0.43325692,
+             0.38853383, 0.36617002, 0.34370604, 0.32104823, 0.29807833, 0.27464288, 0.25053367, 0.22545385, 0.19894916,
+             0.17026083, 0.13792117, 0.09824532, 0.02916753],
+        ],
+        1.50: [
+            [14.61464119, 0.54755926, 0.02916753],
+            [14.61464119, 0.803307, 0.25053367, 0.02916753],
+            [14.61464119, 0.86115354, 0.32104823, 0.09824532, 0.02916753],
+            [14.61464119, 1.24153244, 0.54755926, 0.25053367, 0.09824532, 0.02916753],
+            [14.61464119, 1.56271636, 0.72133851, 0.36617002, 0.19894916, 0.09824532, 0.02916753],
+            [14.61464119, 1.61558151, 0.803307, 0.45573691, 0.27464288, 0.17026083, 0.09824532, 0.02916753],
+            [14.61464119, 1.61558151, 0.83188516, 0.52423614, 0.34370604, 0.25053367, 0.17026083, 0.09824532,
+             0.02916753],
+            [14.61464119, 1.84880662, 0.95350921, 0.59516323, 0.38853383, 0.27464288, 0.19894916, 0.13792117,
+             0.09824532, 0.02916753],
+            [14.61464119, 1.84880662, 0.95350921, 0.59516323, 0.41087446, 0.29807833, 0.22545385, 0.17026083,
+             0.13792117, 0.09824532, 0.02916753],
+            [14.61464119, 1.84880662, 0.95350921, 0.61951244, 0.43325692, 0.32104823, 0.25053367, 0.19894916,
+             0.17026083, 0.13792117, 0.09824532, 0.02916753],
+            [14.61464119, 2.19988537, 1.12534678, 0.72133851, 0.50118381, 0.36617002, 0.27464288, 0.22545385,
+             0.19894916, 0.17026083, 0.13792117, 0.09824532, 0.02916753],
+            [14.61464119, 2.19988537, 1.12534678, 0.72133851, 0.50118381, 0.36617002, 0.29807833, 0.25053367,
+             0.22545385, 0.19894916, 0.17026083, 0.13792117, 0.09824532, 0.02916753],
+            [14.61464119, 2.36326075, 1.24153244, 0.803307, 0.57119018, 0.43325692, 0.34370604, 0.29807833, 0.25053367,
+             0.22545385, 0.19894916, 0.17026083, 0.13792117, 0.09824532, 0.02916753],
+            [14.61464119, 2.36326075, 1.24153244, 0.803307, 0.57119018, 0.43325692, 0.34370604, 0.29807833, 0.27464288,
+             0.25053367, 0.22545385, 0.19894916, 0.17026083, 0.13792117, 0.09824532, 0.02916753],
+            [14.61464119, 2.36326075, 1.24153244, 0.803307, 0.59516323, 0.45573691, 0.36617002, 0.32104823, 0.29807833,
+             0.27464288, 0.25053367, 0.22545385, 0.19894916, 0.17026083, 0.13792117, 0.09824532, 0.02916753],
+            [14.61464119, 2.36326075, 1.24153244, 0.803307, 0.59516323, 0.45573691, 0.38853383, 0.34370604, 0.32104823,
+             0.29807833, 0.27464288, 0.25053367, 0.22545385, 0.19894916, 0.17026083, 0.13792117, 0.09824532,
+             0.02916753],
+            [14.61464119, 2.45070267, 1.32549286, 0.86115354, 0.64427125, 0.50118381, 0.41087446, 0.36617002,
+             0.34370604, 0.32104823, 0.29807833, 0.27464288, 0.25053367, 0.22545385, 0.19894916, 0.17026083, 0.13792117,
+             0.09824532, 0.02916753],
+            [14.61464119, 2.45070267, 1.36964464, 0.92192322, 0.69515091, 0.54755926, 0.45573691, 0.41087446,
+             0.36617002, 0.34370604, 0.32104823, 0.29807833, 0.27464288, 0.25053367, 0.22545385, 0.19894916, 0.17026083,
+             0.13792117, 0.09824532, 0.02916753],
+            [14.61464119, 2.45070267, 1.41535246, 0.95350921, 0.72133851, 0.57119018, 0.4783645, 0.43325692, 0.38853383,
+             0.36617002, 0.34370604, 0.32104823, 0.29807833, 0.27464288, 0.25053367, 0.22545385, 0.19894916, 0.17026083,
+             0.13792117, 0.09824532, 0.02916753],
+        ],
+    }
+
+    def get_sigmas(self, coeff, steps, denoise):
+        total_steps = steps
+        if denoise < 1.0:
+            if denoise <= 0.0:
+                return (torch.FloatTensor([]),)
+            total_steps = round(steps * denoise)
+
+        if steps <= 20:
+            sigmas = self.NOISE_LEVELS[round(coeff, 2)][steps-2][:]
+        else:
+            sigmas = self.NOISE_LEVELS[round(coeff, 2)][-1][:]
+            sigmas = loglinear_interp(sigmas, steps + 1)
+
+        sigmas = sigmas[-(total_steps + 1):]
+        sigmas[-1] = 0
+        return (torch.FloatTensor(sigmas), )

py/libs/stability.py

@@ -6,7 +6,7 @@ import pathlib
 from aiohttp import web
 from server import PromptServer
 from .image import tensor2pil, pil2tensor, image2base64, pil2byte
-from ..log import log_node_error
+from .log import log_node_error
 
 
 root_path = pathlib.Path(__file__).parent.parent.parent

py/libs/translate.py

@@ -0,0 +1,247 @@
+#credit to shadowcz007 for this module
+#from https://github.com/shadowcz007/comfyui-mixlab-nodes/blob/main/nodes/TextGenerateNode.py
+import re
+import os
+import folder_paths
+
+import comfy.utils
+import torch
+from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
+
+from .utils import install_package
+try:
+    from lark import Lark, Transformer, v_args
+except:
+    print('install lark-parser...')
+    install_package('lark-parser')
+    from lark import Lark, Transformer, v_args
+
+model_path = os.path.join(folder_paths.models_dir, 'prompt_generator')
+zh_en_model_path = os.path.join(model_path, 'opus-mt-zh-en')
+zh_en_model, zh_en_tokenizer = None, None
+
+def correct_prompt_syntax(prompt=""):
+    # print("input prompt",prompt)
+    corrected_elements = []
+    # 处理成统一的英文标点
+    prompt = prompt.replace('（', '(').replace('）', ')').replace('，', ',').replace(';', ',').replace('。', '.').replace('：',':')
+    # 删除多余的空格
+    prompt = re.sub(r'\s+', ' ', prompt).strip()
+    prompt = prompt.replace("< ","<").replace(" >",">").replace("( ","(").replace(" )",")").replace("[ ","[").replace(' ]',']')
+
+    # 分词
+    prompt_elements = prompt.split(',')
+
+    def balance_brackets(element, open_bracket, close_bracket):
+        open_brackets_count = element.count(open_bracket)
+        close_brackets_count = element.count(close_bracket)
+        return element + close_bracket * (open_brackets_count - close_brackets_count)
+
+    for element in prompt_elements:
+        element = element.strip()
+
+        # 处理空元素
+        if not element:
+            continue
+
+        # 检查并处理圆括号、方括号、尖括号
+        if element[0] in '([':
+            corrected_element = balance_brackets(element, '(', ')') if element[0] == '(' else balance_brackets(element, '[', ']')
+        elif element[0] == '<':
+            corrected_element = balance_brackets(element, '<', '>')
+        else:
+            # 删除开头的右括号或右方括号
+            corrected_element = element.lstrip(')]')
+
+        corrected_elements.append(corrected_element)
+
+    # 重组修正后的prompt
+    return  ','.join(corrected_elements)
+
+def detect_language(input_str):
+    # 统计中文和英文字符的数量
+    count_cn = count_en = 0
+    for char in input_str:
+        if '\u4e00' <= char <= '\u9fff':
+            count_cn += 1
+        elif char.isalpha():
+            count_en += 1
+
+    # 根据统计的字符数量判断主要语言
+    if count_cn > count_en:
+        return "cn"
+    elif count_en > count_cn:
+        return "en"
+    else:
+        return "unknow"
+
+def has_chinese(text):
+    has_cn = False
+    _text = text
+    _text = re.sub(r'<.*?>', '', _text)
+    _text = re.sub(r'__.*?__', '', _text)
+    _text = re.sub(r'embedding:.*?(\d+)?', '', _text)
+    for char in _text:
+        if '\u4e00' <= char <= '\u9fff':
+            has_cn = True
+            break
+        elif char.isalpha():
+            continue
+    return has_cn
+
+def translate(text):
+    global zh_en_model_path, zh_en_model, zh_en_tokenizer
+
+    if not os.path.exists(zh_en_model_path):
+        zh_en_model_path = 'Helsinki-NLP/opus-mt-zh-en'
+
+    if zh_en_model is None:
+
+        zh_en_model = AutoModelForSeq2SeqLM.from_pretrained(zh_en_model_path).eval()
+        zh_en_tokenizer = AutoTokenizer.from_pretrained(zh_en_model_path, padding=True, truncation=True)
+
+    zh_en_model.to("cuda" if torch.cuda.is_available() else "cpu")
+    with torch.no_grad():
+        encoded = zh_en_tokenizer([text], return_tensors="pt")
+        encoded.to(zh_en_model.device)
+        sequences = zh_en_model.generate(**encoded)
+        return zh_en_tokenizer.batch_decode(sequences, skip_special_tokens=True)[0]
+
+@v_args(inline=True)  # Decorator to flatten the tree directly into the function arguments
+class ChinesePromptTranslate(Transformer):
+
+    def sentence(self, *args):
+        return ", ".join(args)
+
+    def phrase(self, *args):
+        return "".join(args)
+
+    def emphasis(self, *args):
+        # Reconstruct the emphasis with translated content
+        return "(" + "".join(args) + ")"
+
+    def weak_emphasis(self, *args):
+        print('weak_emphasis:', args)
+        return "[" + "".join(args) + "]"
+
+    def embedding(self, *args):
+        print('prompt embedding', args[0])
+        if len(args) == 1:
+            embedding_name = str(args[0])
+            return f"embedding:{embedding_name}"
+        elif len(args) > 1:
+            embedding_name, *numbers = args
+
+            if len(numbers) == 2:
+                return f"embedding:{embedding_name}:{numbers[0]}:{numbers[1]}"
+            elif len(numbers) == 1:
+                return f"embedding:{embedding_name}:{numbers[0]}"
+            else:
+                return f"embedding:{embedding_name}"
+
+    def lora(self, *args):
+        if len(args) == 1:
+            return f"<lora:{args[0]}>"
+        elif len(args) > 1:
+            # print('lora', args)
+            _, loar_name, *numbers = args
+            loar_name = str(loar_name).strip()
+            if len(numbers) == 2:
+                return f"<lora:{loar_name}:{numbers[0]}:{numbers[1]}>"
+            elif len(numbers) == 1:
+                return f"<lora:{loar_name}:{numbers[0]}>"
+            else:
+                return f"<lora:{loar_name}>"
+
+    def weight(self, word, number):
+        translated_word = translate(str(word)).rstrip('.')
+        return f"({translated_word}:{str(number).strip()})"
+
+    def schedule(self, *args):
+        print('prompt schedule', args)
+        data = [str(arg).strip() for arg in args]
+
+        return f"[{':'.join(data)}]"
+
+    def word(self, word):
+        # Translate each word using the dictionary
+        word = str(word)
+        match_cn = re.search(r'@.*?@', word)
+        if re.search(r'__.*?__', word):
+            return word.rstrip('.')
+        elif match_cn:
+            chinese = match_cn.group()
+            before = word.split('@', 1)
+            before = before[0] if len(before) > 0 else ''
+            before = translate(str(before)).rstrip('.') if before else ''
+            after = word.rsplit('@', 1)
+            after = after[len(after)-1] if len(after) > 1 else ''
+            after = translate(after).rstrip('.') if after else ''
+            return before + chinese.replace('@', '').rstrip('.') + after
+        elif detect_language(word) == "cn":
+            return translate(word).rstrip('.')
+        else:
+            return word.rstrip('.')
+
+
+#定义Prompt文法
+grammar = """
+start: sentence
+sentence: phrase ("," phrase)*
+phrase: emphasis | weight | word | lora | embedding | schedule 
+emphasis: "(" sentence ")" -> emphasis
+        | "[" sentence "]" -> weak_emphasis
+weight: "(" word ":" NUMBER ")"
+schedule: "[" word ":" word ":" NUMBER "]"
+lora: "<" WORD ":" WORD (":" NUMBER)? (":" NUMBER)? ">"
+embedding: "embedding" ":" WORD (":" NUMBER)? (":" NUMBER)?
+word: WORD
+
+NUMBER: /\s*-?\d+(\.\d+)?\s*/
+WORD: /[^,:\(\)\[\]<>]+/
+"""
+def zh_to_en(text):
+    global zh_en_model_path, zh_en_model, zh_en_tokenizer
+    # 进度条
+    pbar = comfy.utils.ProgressBar(len(text) + 1)
+    texts = [correct_prompt_syntax(t) for t in text]
+
+    install_package('sentencepiece', '0.2.0')
+
+    if not os.path.exists(zh_en_model_path):
+        zh_en_model_path = 'Helsinki-NLP/opus-mt-zh-en'
+
+    if zh_en_model is None:
+        zh_en_model = AutoModelForSeq2SeqLM.from_pretrained(zh_en_model_path).eval()
+        zh_en_tokenizer = AutoTokenizer.from_pretrained(zh_en_model_path, padding=True, truncation=True)
+
+    zh_en_model.to("cuda" if torch.cuda.is_available() else "cpu")
+
+    prompt_result = []
+
+    en_texts = []
+
+    for t in texts:
+        if t:
+            # translated_text =  translated_word = translate(zh_en_tokenizer,zh_en_model,str(t))
+            parser = Lark(grammar, start="start", parser="lalr", transformer=ChinesePromptTranslate())
+            # print('t',t)
+            result = parser.parse(t).children
+            # print('en_result',result)
+            # en_text=translate(zh_en_tokenizer,zh_en_model,text_without_syntax)
+            en_texts.append(result[0])
+
+    zh_en_model.to('cpu')
+    # print("test en_text", en_texts)
+    # en_text.to("cuda" if torch.cuda.is_available() else "cpu")
+
+    pbar.update(1)
+    for t in en_texts:
+        prompt_result.append(t)
+        pbar.update(1)
+
+    # print('prompt_result', prompt_result, )
+    if len(prompt_result) == 0:
+        prompt_result = [""]
+
+    return prompt_result

py/libs/utils.py

@@ -100,6 +100,8 @@ def get_sd_version(model):
             model_config, (comfy.supported_models.SVD_img2vid)
     ):
         return 'svd'
+    elif isinstance(model_config, comfy.supported_models.SD3):
+        return 'sd3'
     else:
         return 'unknown'
 

py/libs/xyplot.py

@@ -2,11 +2,12 @@ import os, torch
 from pathlib import Path
 from PIL import Image, ImageDraw, ImageFont
 from .utils import easySave
-from ..config import RESOURCES_DIR
-from ..log import log_node_warn
-from ..adv_encode import advanced_encode
+from .adv_encode import advanced_encode
 from .controlnet import easyControlnet
-from ..layer_diffuse.func import LayerDiffuse
+from .log import log_node_warn
+from ..layer_diffuse import LayerDiffuse
+from ..config import RESOURCES_DIR
+
 class easyXYPlot():
 
     def __init__(self, xyPlotData, save_prefix, image_output, prompt, extra_pnginfo, my_unique_id, sampler, easyCache):

py/logic.py

@@ -2,6 +2,7 @@ from typing import Iterator, List, Tuple, Dict, Any, Union, Optional
 from _decimal import Context, getcontext
 from decimal import Decimal
 from .libs.utils import AlwaysEqualProxy, cleanGPUUsedForce
+from .libs.cache import remove_cache
 import numpy as np
 import json
 
@@ -74,7 +75,7 @@ class Int:
     @classmethod
     def INPUT_TYPES(s):
         return {
-            "required": {"value": ("INT", {"default": 0})},
+            "required": {"value": ("INT", {"default": 0, "min": -999999, "max": 999999,})},
         }
 
     RETURN_TYPES = ("INT",)
@@ -143,7 +144,7 @@ class Float:
     @classmethod
     def INPUT_TYPES(s):
         return {
-            "required": {"value": ("FLOAT", {"default": 0, "step": 0.01})},
+            "required": {"value": ("FLOAT", {"default": 0, "step": 0.01, "min": -999999, "max": 999999,})},
         }
 
     RETURN_TYPES = ("FLOAT",)
@@ -477,7 +478,11 @@ class showAnything:
                     values.append(str(val))
                     pass
 
-        if unique_id and extra_pnginfo and "workflow" in extra_pnginfo[0]:
+        if not extra_pnginfo:
+            print("Error: extra_pnginfo is empty")
+        elif (not isinstance(extra_pnginfo[0], dict) or "workflow" not in extra_pnginfo[0]):
+            print("Error: extra_pnginfo[0] is not a dict or missing 'workflow' key")
+        else:
             workflow = extra_pnginfo[0]["workflow"]
             node = next((x for x in workflow["nodes"] if str(x["id"]) == unique_id[0]), None)
             if node:
@@ -531,9 +536,9 @@ class cleanGPUUsed:
 
     def empty_cache(self, anything, unique_id=None, extra_pnginfo=None):
         cleanGPUUsedForce()
+        remove_cache('*')
         return ()
 
-from .libs.cache import remove_cache
 class clearCacheKey:
     @classmethod
     def INPUT_TYPES(s):

py/server.py

@@ -2,9 +2,6 @@ import random
 import server
 from enum import Enum
 
-
-seed_nodes = ["easy wildcards","easy preSampling","easy preSamplingAdvanced","easy preSamplingSdTurbo","easy preSamplingDynamicCFG","easy preSamplingLayerDiffusion","easy preSamplingCascade","easy fullCascadeKSampler","easy fullkSampler","easy seed","easy latentNoisy", "easy preSamplingNoiseIn"]
-
 class SGmode(Enum):
     FIX = 1
     INCR = 2
@@ -123,41 +120,6 @@ def prompt_seed_update(json_data):
         # control after generated
         if mode is not None and not mode:
             control_seed(node[1], action, seed_is_global)
-    # else:
-    #     prompts = json_data['prompt'].items()
-    #     for k, v in prompts:
-    #         if 'class_type' not in v:
-    #             continue
-    #         cls = v['class_type']
-    #         if cls in seed_nodes:
-    #             extra_data = next((x for x in workflow["nodes"] if str(x["id"]) == k), None)
-    #             if extra_data is not None:
-    #                 inputs = extra_data.get('inputs')
-    #                 widgets_value = extra_data.get('widgets_values')
-    #                 widgets_length = len(widgets_value)
-    #                 if "disable" in widgets_value:
-    #                     break
-    #                 if inputs is not None and inputs != []:
-    #                     seed_num_input = next((x for x in inputs if x['name'] == 'seed_num' and x['type'] == 'INT'), None)
-    #                     if seed_num_input is not None:
-    #                         action = 'fixed'
-    #                     else:
-    #                         action = widgets_value[widgets_length - 1]
-    #                 else:
-    #                     control_index = widgets_length - 2 if cls == 'easy seed' else widgets_length - 1
-    #                     action = widgets_value[control_index]
-    #
-    #                 # print(action)
-    #                 node = k, v
-    #                 value = control_seed(node[1], action, False)
-    #
-    #                 if k not in seed_widget_map:
-    #                     continue
-    #
-    #                 if 'seed_num' in v['inputs']:
-    #                     if isinstance(v['inputs']['seed_num'], int):
-    #                         v['inputs']['seed_num'] = value
-
 
     return value is not None
 

py/xyplot.py

@@ -0,0 +1,603 @@
+import os
+import comfy
+import folder_paths
+from .config import RESOURCES_DIR
+def load_preset(filename):
+    path = os.path.join(RESOURCES_DIR, filename)
+    path = os.path.abspath(path)
+    preset_list = []
+
+    if os.path.exists(path):
+        with open(path, 'r') as file:
+            for line in file:
+                preset_list.append(line.strip())
+
+        return preset_list
+    else:
+        return []
+def generate_floats(batch_count, first_float, last_float):
+    if batch_count > 1:
+        interval = (last_float - first_float) / (batch_count - 1)
+        values = [str(round(first_float + i * interval, 3)) for i in range(batch_count)]
+    else:
+        values = [str(first_float)] if batch_count == 1 else []
+    return "; ".join(values)
+
+def generate_ints(batch_count, first_int, last_int):
+    if batch_count > 1:
+        interval = (last_int - first_int) / (batch_count - 1)
+        values = [str(int(first_int + i * interval)) for i in range(batch_count)]
+    else:
+        values = [str(first_int)] if batch_count == 1 else []
+    # values = list(set(values))  # Remove duplicates
+    # values.sort()  # Sort in ascending order
+    return "; ".join(values)
+
+# Seed++ Batch
+class XYplot_SeedsBatch:
+
+    @classmethod
+    def INPUT_TYPES(cls):
+        return {"required": {
+            "batch_count": ("INT", {"default": 3, "min": 1, "max": 50}), },
+        }
+
+    RETURN_TYPES = ("X_Y",)
+    RETURN_NAMES = ("X or Y",)
+    FUNCTION = "xy_value"
+    CATEGORY = "EasyUse/XY Inputs"
+
+    def xy_value(self, batch_count):
+
+        axis = "advanced: Seeds++ Batch"
+        xy_values = {"axis": axis, "values": batch_count}
+        return (xy_values,)
+
+# Step Values
+class XYplot_Steps:
+    parameters = ["steps", "start_at_step", "end_at_step",]
+
+    @classmethod
+    def INPUT_TYPES(cls):
+        return {
+            "required": {
+                "target_parameter": (cls.parameters,),
+                "batch_count": ("INT", {"default": 3, "min": 0, "max": 50}),
+                "first_step": ("INT", {"default": 10, "min": 1, "max": 10000}),
+                "last_step": ("INT", {"default": 20, "min": 1, "max": 10000}),
+                "first_start_step": ("INT", {"default": 0, "min": 0, "max": 10000}),
+                "last_start_step": ("INT", {"default": 10, "min": 0, "max": 10000}),
+                "first_end_step": ("INT", {"default": 10, "min": 0, "max": 10000}),
+                "last_end_step": ("INT", {"default": 20, "min": 0, "max": 10000}),
+            }
+        }
+
+    RETURN_TYPES = ("X_Y",)
+    RETURN_NAMES = ("X or Y",)
+    FUNCTION = "xy_value"
+    CATEGORY = "EasyUse/XY Inputs"
+
+    def xy_value(self, target_parameter, batch_count, first_step, last_step, first_start_step, last_start_step,
+                 first_end_step, last_end_step,):
+
+        axis, xy_first, xy_last = None, None, None
+
+        if target_parameter == "steps":
+            axis = "advanced: Steps"
+            xy_first = first_step
+            xy_last = last_step
+        elif target_parameter == "start_at_step":
+            axis = "advanced: StartStep"
+            xy_first = first_start_step
+            xy_last = last_start_step
+        elif target_parameter == "end_at_step":
+            axis = "advanced: EndStep"
+            xy_first = first_end_step
+            xy_last = last_end_step
+
+        values = generate_ints(batch_count, xy_first, xy_last)
+        return ({"axis": axis, "values": values},) if values is not None else (None,)
+
+class XYplot_CFG:
+
+    @classmethod
+    def INPUT_TYPES(cls):
+        return {
+            "required": {
+                "batch_count": ("INT", {"default": 3, "min": 0, "max": 50}),
+                "first_cfg": ("FLOAT", {"default": 7.0, "min": 0.0, "max": 100.0}),
+                "last_cfg": ("FLOAT", {"default": 9.0, "min": 0.0, "max": 100.0}),
+            }
+        }
+
+    RETURN_TYPES = ("X_Y",)
+    RETURN_NAMES = ("X or Y",)
+    FUNCTION = "xy_value"
+    CATEGORY = "EasyUse/XY Inputs"
+
+    def xy_value(self, batch_count, first_cfg, last_cfg):
+        axis = "advanced: CFG Scale"
+        values = generate_floats(batch_count, first_cfg, last_cfg)
+        return ({"axis": axis, "values": values},) if values else (None,)
+
+# Step Values
+class XYplot_Sampler_Scheduler:
+    parameters = ["sampler", "scheduler", "sampler & scheduler"]
+
+    @classmethod
+    def INPUT_TYPES(cls):
+        samplers = ["None"] + comfy.samplers.KSampler.SAMPLERS
+        schedulers = ["None"] + comfy.samplers.KSampler.SCHEDULERS
+        inputs = {
+            "required": {
+                "target_parameter": (cls.parameters,),
+                "input_count": ("INT", {"default": 1, "min": 1, "max": 30, "step": 1})
+            }
+        }
+        for i in range(1, 30 + 1):
+            inputs["required"][f"sampler_{i}"] = (samplers,)
+            inputs["required"][f"scheduler_{i}"] = (schedulers,)
+
+        return inputs
+
+    RETURN_TYPES = ("X_Y",)
+    RETURN_NAMES = ("X or Y",)
+    FUNCTION = "xy_value"
+    CATEGORY = "EasyUse/XY Inputs"
+
+    def xy_value(self, target_parameter, input_count, **kwargs):
+        axis, values, = None, None,
+        if target_parameter == "scheduler":
+            axis = "advanced: Scheduler"
+            schedulers = [kwargs.get(f"scheduler_{i}") for i in range(1, input_count + 1)]
+            values = [scheduler for scheduler in schedulers if scheduler != "None"]
+        elif target_parameter == "sampler":
+            axis = "advanced: Sampler"
+            samplers = [kwargs.get(f"sampler_{i}") for i in range(1, input_count + 1)]
+            values = [sampler for sampler in samplers if sampler != "None"]
+        else:
+            axis = "advanced: Sampler&Scheduler"
+            samplers = [kwargs.get(f"sampler_{i}") for i in range(1, input_count + 1)]
+            schedulers = [kwargs.get(f"scheduler_{i}") for i in range(1, input_count + 1)]
+            values = []
+            for sampler, scheduler in zip(samplers, schedulers):
+                sampler = sampler if sampler else 'None'
+                scheduler = scheduler if scheduler else 'None'
+                values.append(sampler +', '+ scheduler)
+        values = "; ".join(values)
+        return ({"axis": axis, "values": values},) if values else (None,)
+
+class XYplot_Denoise:
+
+    @classmethod
+    def INPUT_TYPES(cls):
+        return {
+            "required": {
+                "batch_count": ("INT", {"default": 3, "min": 0, "max": 50}),
+                "first_denoise": ("FLOAT", {"default": 0.5, "min": 0.0, "max": 1.0, "step": 0.1}),
+                "last_denoise": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.1}),
+            }
+        }
+
+    RETURN_TYPES = ("X_Y",)
+    RETURN_NAMES = ("X or Y",)
+    FUNCTION = "xy_value"
+    CATEGORY = "EasyUse/XY Inputs"
+
+    def xy_value(self, batch_count, first_denoise, last_denoise):
+        axis = "advanced: Denoise"
+        values = generate_floats(batch_count, first_denoise, last_denoise)
+        return ({"axis": axis, "values": values},) if values else (None,)
+
+# PromptSR
+class XYplot_PromptSR:
+
+    @classmethod
+    def INPUT_TYPES(cls):
+        inputs = {
+            "required": {
+                "target_prompt": (["positive", "negative"],),
+                "search_txt": ("STRING", {"default": "", "multiline": False}),
+                "replace_all_text": ("BOOLEAN", {"default": False}),
+                "replace_count": ("INT", {"default": 3, "min": 1, "max": 30 - 1}),
+            }
+        }
+
+        # Dynamically add replace_X inputs
+        for i in range(1, 30):
+            replace_key = f"replace_{i}"
+            inputs["required"][replace_key] = ("STRING", {"default": "", "multiline": False, "placeholder": replace_key})
+
+        return inputs
+
+    RETURN_TYPES = ("X_Y",)
+    RETURN_NAMES = ("X or Y",)
+    FUNCTION = "xy_value"
+    CATEGORY = "EasyUse/XY Inputs"
+
+    def xy_value(self, target_prompt, search_txt, replace_all_text, replace_count, **kwargs):
+        axis = None
+
+        if target_prompt == "positive":
+            axis = "advanced: Positive Prompt S/R"
+        elif target_prompt == "negative":
+            axis = "advanced: Negative Prompt S/R"
+
+        # Create base entry
+        values = [(search_txt, None, replace_all_text)]
+
+        if replace_count > 0:
+            # Append additional entries based on replace_count
+            values.extend([(search_txt, kwargs.get(f"replace_{i+1}"), replace_all_text) for i in range(replace_count)])
+        return ({"axis": axis, "values": values},) if values is not None else (None,)
+
+# XYPlot Pos Condition
+class XYplot_Positive_Cond:
+
+    @classmethod
+    def INPUT_TYPES(cls):
+        inputs = {
+            "optional": {
+                "positive_1": ("CONDITIONING",),
+                "positive_2": ("CONDITIONING",),
+                "positive_3": ("CONDITIONING",),
+                "positive_4": ("CONDITIONING",),
+            }
+        }
+
+        return inputs
+
+    RETURN_TYPES = ("X_Y",)
+    RETURN_NAMES = ("X or Y",)
+    FUNCTION = "xy_value"
+    CATEGORY = "EasyUse/XY Inputs"
+
+    def xy_value(self, positive_1=None, positive_2=None, positive_3=None, positive_4=None):
+        axis = "advanced: Pos Condition"
+        values = []
+        cond = []
+        # Create base entry
+        if positive_1 is not None:
+            values.append("0")
+            cond.append(positive_1)
+        if positive_2 is not None:
+            values.append("1")
+            cond.append(positive_2)
+        if positive_3 is not None:
+            values.append("2")
+            cond.append(positive_3)
+        if positive_4 is not None:
+            values.append("3")
+            cond.append(positive_4)
+
+        return ({"axis": axis, "values": values, "cond": cond},) if values is not None else (None,)
+
+# XYPlot Neg Condition
+class XYplot_Negative_Cond:
+
+    @classmethod
+    def INPUT_TYPES(cls):
+        inputs = {
+            "optional": {
+                "negative_1": ("CONDITIONING"),
+                "negative_2": ("CONDITIONING"),
+                "negative_3": ("CONDITIONING"),
+                "negative_4": ("CONDITIONING"),
+            }
+        }
+
+        return inputs
+
+    RETURN_TYPES = ("X_Y",)
+    RETURN_NAMES = ("X or Y",)
+    FUNCTION = "xy_value"
+    CATEGORY = "EasyUse/XY Inputs"
+
+    def xy_value(self, negative_1=None, negative_2=None, negative_3=None, negative_4=None):
+        axis = "advanced: Neg Condition"
+        values = []
+        cond = []
+        # Create base entry
+        if negative_1 is not None:
+            values.append(0)
+            cond.append(negative_1)
+        if negative_2 is not None:
+            values.append(1)
+            cond.append(negative_2)
+        if negative_3 is not None:
+            values.append(2)
+            cond.append(negative_3)
+        if negative_4 is not None:
+            values.append(3)
+            cond.append(negative_4)
+
+        return ({"axis": axis, "values": values, "cond": cond},) if values is not None else (None,)
+
+# XYPlot Pos Condition List
+class XYplot_Positive_Cond_List:
+    @classmethod
+    def INPUT_TYPES(cls):
+        return {
+            "required": {
+                "positive": ("CONDITIONING",),
+            }
+        }
+
+    INPUT_IS_LIST = True
+    RETURN_TYPES = ("X_Y",)
+    RETURN_NAMES = ("X or Y",)
+    FUNCTION = "xy_value"
+    CATEGORY = "EasyUse/XY Inputs"
+
+    def xy_value(self, positive):
+        axis = "advanced: Pos Condition"
+        values = []
+        cond = []
+        for index, c in enumerate(positive):
+            values.append(str(index))
+            cond.append(c)
+
+        return ({"axis": axis, "values": values, "cond": cond},) if values is not None else (None,)
+
+# XYPlot Neg Condition List
+class XYplot_Negative_Cond_List:
+    @classmethod
+    def INPUT_TYPES(cls):
+        return {
+            "required": {
+                "negative": ("CONDITIONING",),
+            }
+        }
+
+    INPUT_IS_LIST = True
+    RETURN_TYPES = ("X_Y",)
+    RETURN_NAMES = ("X or Y",)
+    FUNCTION = "xy_value"
+    CATEGORY = "EasyUse/XY Inputs"
+
+    def xy_value(self, negative):
+        axis = "advanced: Neg Condition"
+        values = []
+        cond = []
+        for index, c in enumerate(negative):
+            values.append(index)
+            cond.append(c)
+
+        return ({"axis": axis, "values": values, "cond": cond},) if values is not None else (None,)
+
+# XY Plot: ControlNet
+class XYplot_Control_Net:
+    parameters = ["strength", "start_percent", "end_percent"]
+    @classmethod
+    def INPUT_TYPES(cls):
+        def get_file_list(filenames):
+            return [file for file in filenames if file != "put_models_here.txt" and "lllite" not in file]
+
+        return {
+            "required": {
+                "control_net_name": (get_file_list(folder_paths.get_filename_list("controlnet")),),
+                "image": ("IMAGE",),
+                "target_parameter": (cls.parameters,),
+                "batch_count": ("INT", {"default": 3, "min": 1, "max": 30}),
+                "first_strength": ("FLOAT", {"default": 0.0, "min": 0.00, "max": 10.0, "step": 0.01}),
+                "last_strength": ("FLOAT", {"default": 1.0, "min": 0.00, "max": 10.0, "step": 0.01}),
+                "first_start_percent": ("FLOAT", {"default": 0.0, "min": 0.00, "max": 1.0, "step": 0.01}),
+                "last_start_percent": ("FLOAT", {"default": 1.0, "min": 0.00, "max": 1.0, "step": 0.01}),
+                "first_end_percent": ("FLOAT", {"default": 0.0, "min": 0.00, "max": 1.0, "step": 0.01}),
+                "last_end_percent": ("FLOAT", {"default": 1.0, "min": 0.00, "max": 1.0, "step": 0.01}),
+                "strength": ("FLOAT", {"default": 1.0, "min": 0.00, "max": 10.0, "step": 0.01}),
+                "start_percent": ("FLOAT", {"default": 0.0, "min": 0.00, "max": 1.0, "step": 0.01}),
+                "end_percent": ("FLOAT", {"default": 1.0, "min": 0.00, "max": 1.0, "step": 0.01}),
+            },
+        }
+
+    RETURN_TYPES = ("X_Y",)
+    RETURN_NAMES = ("X or Y",)
+    FUNCTION = "xy_value"
+    CATEGORY = "EasyUse/XY Inputs"
+
+    def xy_value(self, control_net_name, image, target_parameter, batch_count, first_strength, last_strength, first_start_percent,
+                 last_start_percent, first_end_percent, last_end_percent, strength, start_percent, end_percent):
+
+        axis, = None,
+
+        values = []
+
+        if target_parameter == "strength":
+            axis = "advanced: ControlNetStrength"
+
+            values.append([(control_net_name, image, first_strength, start_percent, end_percent)])
+            strength_increment = (last_strength - first_strength) / (batch_count - 1) if batch_count > 1 else 0
+            for i in range(1, batch_count - 1):
+                values.append([(control_net_name, image, first_strength + i * strength_increment, start_percent,
+                                end_percent)])
+            if batch_count > 1:
+                values.append([(control_net_name, image, last_strength, start_percent, end_percent)])
+
+        elif target_parameter == "start_percent":
+            axis = "advanced: ControlNetStart%"
+
+            percent_increment = (last_start_percent - first_start_percent) / (batch_count - 1) if batch_count > 1 else 0
+            values.append([(control_net_name, image, strength, first_start_percent, end_percent)])
+            for i in range(1, batch_count - 1):
+                values.append([(control_net_name, image, strength, first_start_percent + i * percent_increment,
+                                  end_percent)])
+
+            # Always add the last start_percent if batch_count is more than 1.
+            if batch_count > 1:
+                values.append((control_net_name, image, strength, last_start_percent, end_percent))
+
+        elif target_parameter == "end_percent":
+            axis = "advanced: ControlNetEnd%"
+
+            percent_increment = (last_end_percent - first_end_percent) / (batch_count - 1) if batch_count > 1 else 0
+            values.append([(control_net_name, image, image, strength, start_percent, first_end_percent)])
+            for i in range(1, batch_count - 1):
+                values.append([(control_net_name, image, strength, start_percent,
+                                  first_end_percent + i * percent_increment)])
+
+            if batch_count > 1:
+                values.append([(control_net_name, image, strength, start_percent, last_end_percent)])
+
+
+        return ({"axis": axis, "values": values},)
+
+
+#Checkpoints
+class XYplot_Checkpoint:
+
+    modes = ["Ckpt Names", "Ckpt Names+ClipSkip", "Ckpt Names+ClipSkip+VAE"]
+
+    @classmethod
+    def INPUT_TYPES(cls):
+
+        checkpoints = ["None"] + folder_paths.get_filename_list("checkpoints")
+        vaes = ["Baked VAE"] + folder_paths.get_filename_list("vae")
+
+        inputs = {
+            "required": {
+                "input_mode": (cls.modes,),
+                "ckpt_count": ("INT", {"default": 3, "min": 0, "max": 10, "step": 1}),
+            }
+        }
+
+        for i in range(1, 10 + 1):
+            inputs["required"][f"ckpt_name_{i}"] = (checkpoints,)
+            inputs["required"][f"clip_skip_{i}"] = ("INT", {"default": -1, "min": -24, "max": -1, "step": 1})
+            inputs["required"][f"vae_name_{i}"] = (vaes,)
+
+        inputs["optional"] = {
+            "optional_lora_stack": ("LORA_STACK",)
+        }
+        return inputs
+
+    RETURN_TYPES = ("X_Y",)
+    RETURN_NAMES = ("X or Y",)
+    FUNCTION = "xy_value"
+
+    CATEGORY = "EasyUse/XY Inputs"
+
+    def xy_value(self, input_mode, ckpt_count, **kwargs):
+
+        axis = "advanced: Checkpoint"
+
+        checkpoints = [kwargs.get(f"ckpt_name_{i}") for i in range(1, ckpt_count + 1)]
+        clip_skips = [kwargs.get(f"clip_skip_{i}") for i in range(1, ckpt_count + 1)]
+        vaes = [kwargs.get(f"vae_name_{i}") for i in range(1, ckpt_count + 1)]
+
+        # Set None for Clip Skip and/or VAE if not correct modes
+        for i in range(ckpt_count):
+            if "ClipSkip" not in input_mode:
+                clip_skips[i] = 'None'
+            if "VAE" not in input_mode:
+                vaes[i] = 'None'
+
+        # Extend each sub-array with lora_stack if it's not None
+        values = [checkpoint.replace(',', '*')+','+str(clip_skip)+','+vae.replace(',', '*') for checkpoint, clip_skip, vae in zip(checkpoints, clip_skips, vaes) if
+                        checkpoint != "None"]
+
+        optional_lora_stack = kwargs.get("optional_lora_stack") if "optional_lora_stack" in kwargs else []
+
+        xy_values = {"axis": axis, "values": values, "lora_stack": optional_lora_stack}
+        return (xy_values,)
+
+#Loras
+class XYplot_Lora:
+
+    modes = ["Lora Names", "Lora Names+Weights"]
+
+    @classmethod
+    def INPUT_TYPES(cls):
+        loras = ["None"] + folder_paths.get_filename_list("loras")
+
+        inputs = {
+            "required": {
+                "input_mode": (cls.modes,),
+                "lora_count": ("INT", {"default": 3, "min": 0, "max": 10, "step": 1}),
+                "model_strength": ("FLOAT", {"default": 1.0, "min": -10.0, "max": 10.0, "step": 0.01}),
+                "clip_strength": ("FLOAT", {"default": 1.0, "min": -10.0, "max": 10.0, "step": 0.01}),
+            }
+        }
+
+        for i in range(1, 10 + 1):
+            inputs["required"][f"lora_name_{i}"] = (loras,)
+            inputs["required"][f"model_str_{i}"] = ("FLOAT", {"default": 1.0, "min": -10.0, "max": 10.0, "step": 0.01})
+            inputs["required"][f"clip_str_{i}"] = ("FLOAT", {"default": 1.0, "min": -10.0, "max": 10.0, "step": 0.01})
+
+        inputs["optional"] = {
+            "optional_lora_stack": ("LORA_STACK",)
+        }
+        return inputs
+
+    RETURN_TYPES = ("X_Y",)
+    RETURN_NAMES = ("X or Y",)
+    FUNCTION = "xy_value"
+
+    CATEGORY = "EasyUse/XY Inputs"
+
+    def xy_value(self, input_mode, lora_count, model_strength, clip_strength, **kwargs):
+
+        axis = "advanced: Lora"
+        # Extract values from kwargs
+        loras = [kwargs.get(f"lora_name_{i}") for i in range(1, lora_count + 1)]
+        model_strs = [kwargs.get(f"model_str_{i}", model_strength) for i in range(1, lora_count + 1)]
+        clip_strs = [kwargs.get(f"clip_str_{i}", clip_strength) for i in range(1, lora_count + 1)]
+
+        # Use model_strength and clip_strength for the loras where values are not provided
+        if "Weights" not in input_mode:
+            for i in range(lora_count):
+                model_strs[i] = model_strength
+                clip_strs[i] = clip_strength
+
+        # Extend each sub-array with lora_stack if it's not None
+        values = [lora.replace(',', '*')+','+str(model_str)+','+str(clip_str) for lora, model_str, clip_str
+                    in zip(loras, model_strs, clip_strs) if lora != "None"]
+
+        optional_lora_stack = kwargs.get("optional_lora_stack") if "optional_lora_stack" in kwargs else []
+
+        print(values)
+        xy_values = {"axis": axis, "values": values, "lora_stack": optional_lora_stack}
+        return (xy_values,)
+
+# 模型叠加
+class XYplot_ModelMergeBlocks:
+
+    @classmethod
+    def INPUT_TYPES(s):
+        checkpoints = folder_paths.get_filename_list("checkpoints")
+        vae = ["Use Model 1", "Use Model 2"] + folder_paths.get_filename_list("vae")
+
+        preset = ["Preset"]  # 20
+        preset += load_preset("mmb-preset.txt")
+        preset += load_preset("mmb-preset.custom.txt")
+
+        default_vectors = "1,0,0; \n0,1,0; \n0,0,1; \n1,1,0; \n1,0,1; \n0,1,1; "
+        return {
+            "required": {
+                "ckpt_name_1": (checkpoints,),
+                "ckpt_name_2": (checkpoints,),
+                "vae_use": (vae, {"default": "Use Model 1"}),
+                "preset": (preset, {"default": "preset"}),
+                "values": ("STRING", {"default": default_vectors, "multiline": True, "placeholder": 'Support 2 methods:\n\n1.input, middle, out in same line and insert values seperated by "; "\n\n2.model merge block number seperated by ", " in same line and insert values seperated by "; "'}),
+            },
+            "hidden": {"my_unique_id": "UNIQUE_ID"}
+        }
+
+    RETURN_TYPES = ("X_Y",)
+    RETURN_NAMES = ("X or Y",)
+    FUNCTION = "xy_value"
+
+    CATEGORY = "EasyUse/XY Inputs"
+
+    def xy_value(self, ckpt_name_1, ckpt_name_2, vae_use, preset, values, my_unique_id=None):
+
+        axis = "advanced: ModelMergeBlocks"
+        if ckpt_name_1 is None:
+            raise Exception("ckpt_name_1 is not found")
+        if ckpt_name_2 is None:
+            raise Exception("ckpt_name_2 is not found")
+
+        models = (ckpt_name_1, ckpt_name_2)
+
+        xy_values = {"axis":axis, "values":values, "models":models, "vae_use": vae_use}
+        return (xy_values,)

pyproject.toml

@@ -1,9 +1,9 @@
 [project]
 name = "comfyui-easy-use"
 description = "To enhance the usability of ComfyUI, optimizations and integrations have been implemented for several commonly used nodes."
-version = "1.1.7"
+version = "1.1.9"
 license = "LICENSE"
-dependencies = ["diffusers>=0.25.0", "clip_interrogator>=0.6.0", "onnxruntime", "aiohttp"]
+dependencies = ["diffusers>=0.25.0", "accelerate>=0.25.0", "clip_interrogator>=0.6.0", "sentencepiece==0.2.0", "lark-parser", "onnxruntime", "spandrel"]
 
 [project.urls]
 Repository = "https://github.com/yolain/ComfyUI-Easy-Use"

requirements.txt

@@ -1,4 +1,7 @@
 diffusers>=0.25.0
+accelerate>=0.25.0
 clip_interrogator>=0.6.0
+sentencepiece==0.2.0
+lark-parser
 onnxruntime
-aiohttp
+spandrel

web/css/easy.css

@@ -112,4 +112,11 @@ hr{
 }
 ::-webkit-scrollbar-thumb:hover {
     background-color: transparent;
+}
+
+[data-theme="dark"] .workspace_manager .chakra-card{
+    background-color:var(--comfy-menu-bg)!important;
+}
+.workspace_manager .chakra-card{
+    width: 400px;
 }

web/css/theme.css

@@ -5,4 +5,4 @@
    --error-color: #ff4d4f;
    --warning-color: #faad14;
    --font-family: Inter, -apple-system, BlinkMacSystemFont, Helvetica Neue, sans-serif;
-}
+}

web/css/toolbar.css

@@ -35,6 +35,10 @@
     color:white;
     transition: all 0.3s ease-in-out;
 }
+.easyuse-toolbar-icon svg{
+    width: 14px;
+    height: 14px;
+}
 .easyuse-toolbar-tips{
     visibility: hidden;
     opacity: 0;

web/js/easy/easy.js

@@ -516,7 +516,7 @@ app.registerExtension({
                     note = null
                 }
             }
-            return loadGraphDataEvent.apply(this, [...arguments])
+            return await loadGraphDataEvent.apply(this, [...arguments])
         }
 
         addToolBar(app)

web/js/easy/easyContextMenu.js

@@ -109,19 +109,60 @@ app.registerExtension({
                     }
                 }
                 const newValues = [];
+                const add_sub_folder = (folder, folderName) => {
+                    let subs = []
+                    let less = []
+                    const b = folder.map(name=> {
+                        const _folders = {};
+                        const splitBy = name.indexOf('/') > -1 ? '/' : '\\';
+                        const valueSplit = name.split(splitBy);
+                        if(valueSplit.length > 1){
+                            const key = valueSplit.shift();
+                            _folders[key] = _folders[key] || [];
+                            _folders[key].push(valueSplit.join(splitBy));
+                        }
+                        const foldersCount = Object.values(folders).length;
+                        if(foldersCount > 0){
+                            let key = Object.keys(_folders)[0]
+                            if(key && _folders[key]) subs.push({key, value:_folders[key][0]})
+                            else{
+                                less.push(addContent(name,key))
+                            }
+                        }
+                        return addContent(name,folderName)
+                    })
+                    if(subs.length>0){
+                      let subs_obj = {}
+                      subs.forEach(item => {
+                        subs_obj[item.key] = subs_obj[item.key] || []
+                        subs_obj[item.key].push(item.value)
+                      })
+                      return [...Object.entries(subs_obj).map(f => {
+                          return {
+                              content: f[0],
+                              has_submenu: true,
+                              callback: () => {},
+                              submenu: {
+                                  options: add_sub_folder(f[1], f[0]),
+                              }
+                          }
+                      }),...less]
+                    }
+                    else return b
+                }
+
                 for(const [folderName,folder] of Object.entries(folders)){
                     newValues.push({
                         content:folderName,
                         has_submenu:true,
                         callback:() => {},
                         submenu:{
-                            options:folder.map(f => addContent(f,folderName)),
+                            options:add_sub_folder(folder,folderName),
                         }
                     });
                 }
                 newValues.push(...folderless.map(f => addContent(f, '')));
-                if(specialOps.length > 0)
-                    newValues.push(...specialOps.map(f => addContent(f, '')));
+                if(specialOps.length > 0) newValues.push(...specialOps.map(f => addContent(f, '')));
                 return existingContextMenu.call(this,newValues,options);
             }
             return existingContextMenu.apply(this,[...arguments]);

web/js/easy/easyDynamicWidgets.js

@@ -110,11 +110,34 @@ function widgetLogic(node, widget) {
 		}
 		updateNodeHeight(node)
 	}
+	if (widget.name === 'num_controlnet') {
+		let number_to_show = widget.value + 1
+		for (let i = 0; i < number_to_show; i++) {
+			toggleWidget(node, findWidgetByName(node, 'controlnet_'+i), true)
+			toggleWidget(node, findWidgetByName(node, 'controlnet_'+i+'_strength'), true)
+			toggleWidget(node, findWidgetByName(node, 'scale_soft_weight_'+i),true)
+			if (findWidgetByName(node, 'mode').value === "simple") {
+				toggleWidget(node, findWidgetByName(node, 'start_percent_'+i))
+				toggleWidget(node, findWidgetByName(node, 'end_percent_'+i))
+			} else {
+				toggleWidget(node, findWidgetByName(node, 'start_percent_'+i),true)
+				toggleWidget(node, findWidgetByName(node, 'end_percent_'+i), true)
+			}
+		}
+		for (let i = number_to_show; i < 10; i++) {
+			toggleWidget(node, findWidgetByName(node, 'controlnet_'+i))
+			toggleWidget(node, findWidgetByName(node, 'controlnet_'+i+'_strength'))
+			toggleWidget(node, findWidgetByName(node, 'start_percent_'+i))
+			toggleWidget(node, findWidgetByName(node, 'end_percent_'+i))
+			toggleWidget(node, findWidgetByName(node, 'scale_soft_weight_'+i))
+		}
+		updateNodeHeight(node)
+	}
+
 	if (widget.name === 'mode') {
 		switch (node.comfyClass) {
 			case 'easy loraStack':
-				let number_to_show = findWidgetByName(node, 'num_loras').value + 1
-				for (let i = 0; i < number_to_show; i++) {
+				for (let i = 0; i < (findWidgetByName(node, 'num_loras').value + 1); i++) {
 					if (widget.value === "simple") {
 						toggleWidget(node, findWidgetByName(node, 'lora_'+i+'_strength'), true)
 						toggleWidget(node, findWidgetByName(node, 'lora_'+i+'_model_strength'))
@@ -124,6 +147,19 @@ function widgetLogic(node, widget) {
 						toggleWidget(node, findWidgetByName(node, 'lora_'+i+'_model_strength'), true)
 						toggleWidget(node, findWidgetByName(node, 'lora_'+i+'_clip_strength'), true)}
 				}
+				updateNodeHeight(node)
+				break
+			case 'easy controlnetStack':
+				for (let i = 0; i < (findWidgetByName(node, 'num_controlnet').value + 1); i++) {
+					if (widget.value === "simple") {
+						toggleWidget(node, findWidgetByName(node, 'start_percent_'+i))
+						toggleWidget(node, findWidgetByName(node, 'end_percent_'+i))
+					} else {
+						toggleWidget(node, findWidgetByName(node, 'start_percent_' + i), true)
+						toggleWidget(node, findWidgetByName(node, 'end_percent_' + i), true)
+					}
+				}
+				updateNodeHeight(node)
 				break
 			case 'easy icLightApply':
 				if (widget.value === "Foreground") {
@@ -135,9 +171,9 @@ function widgetLogic(node, widget) {
 					toggleWidget(node, findWidgetByName(node, 'source'), true)
 					toggleWidget(node, findWidgetByName(node, 'remove_bg'))
 				}
+				updateNodeHeight(node)
 				break
 		}
-		updateNodeHeight(node)
 	}
 
 	if (widget.name === 'resolution') {
@@ -148,7 +184,6 @@ function widgetLogic(node, widget) {
 			toggleWidget(node, findWidgetByName(node, 'empty_latent_width'), false)
 			toggleWidget(node, findWidgetByName(node, 'empty_latent_height'), false)
 		}
-		updateNodeHeight(node)
 	}
 	if (widget.name === 'downscale_mode') {
 		const widget_names = ['block_number', 'downscale_factor', 'start_percent', 'end_percent', 'downscale_after_skip', 'downscale_method', 'upscale_method']
@@ -298,6 +333,7 @@ function widgetLogic(node, widget) {
 			toggleWidget(node, findWidgetByName(node, 'beta_d'))
 			toggleWidget(node, findWidgetByName(node, 'beta_min'))
 			toggleWidget(node, findWidgetByName(node, 'eps_s'))
+			toggleWidget(node, findWidgetByName(node, 'coeff'))
 			if(widget.value != 'exponentialADV'){
 				toggleWidget(node, findWidgetByName(node, 'rho'), true)
 			}else{
@@ -311,7 +347,9 @@ function widgetLogic(node, widget) {
 			toggleWidget(node, findWidgetByName(node, 'beta_d'),true)
 			toggleWidget(node, findWidgetByName(node, 'beta_min'),true)
 			toggleWidget(node, findWidgetByName(node, 'eps_s'),true)
-		}else{
+			toggleWidget(node, findWidgetByName(node, 'coeff'))
+		}
+		else{
 			toggleWidget(node, findWidgetByName(node, 'denoise'),true)
 			toggleWidget(node, findWidgetByName(node, 'sigma_max'))
 			toggleWidget(node, findWidgetByName(node, 'sigma_min'))
@@ -319,6 +357,40 @@ function widgetLogic(node, widget) {
 			toggleWidget(node, findWidgetByName(node, 'beta_min'))
 			toggleWidget(node, findWidgetByName(node, 'eps_s'))
 			toggleWidget(node, findWidgetByName(node, 'rho'))
+			if(widget.value == 'gits') 	toggleWidget(node, findWidgetByName(node, 'coeff'), true)
+			else toggleWidget(node, findWidgetByName(node, 'coeff'))
+		}
+		updateNodeHeight(node)
+	}
+
+	if(widget.name === 'inpaint_mode'){
+		switch (widget.value){
+			case 'normal':
+			case 'fooocus_inpaint':
+				toggleWidget(node, findWidgetByName(node, 'dtype'))
+				toggleWidget(node, findWidgetByName(node, 'fitting'))
+				toggleWidget(node, findWidgetByName(node, 'function'))
+				toggleWidget(node, findWidgetByName(node, 'scale'))
+				toggleWidget(node, findWidgetByName(node, 'start_at'))
+				toggleWidget(node, findWidgetByName(node, 'end_at'))
+				break
+			case 'brushnet_random':
+			case 'brushnet_segmentation':
+				toggleWidget(node, findWidgetByName(node, 'dtype'), true)
+				toggleWidget(node, findWidgetByName(node, 'fitting'))
+				toggleWidget(node, findWidgetByName(node, 'function'))
+				toggleWidget(node, findWidgetByName(node, 'scale'), true)
+				toggleWidget(node, findWidgetByName(node, 'start_at'), true)
+				toggleWidget(node, findWidgetByName(node, 'end_at'), true)
+				break
+			case 'powerpaint':
+				toggleWidget(node, findWidgetByName(node, 'dtype'), true)
+				toggleWidget(node, findWidgetByName(node, 'fitting'),true)
+				toggleWidget(node, findWidgetByName(node, 'function'),true)
+				toggleWidget(node, findWidgetByName(node, 'scale'), true)
+				toggleWidget(node, findWidgetByName(node, 'start_at'), true)
+				toggleWidget(node, findWidgetByName(node, 'end_at'), true)
+				break
 		}
 		updateNodeHeight(node)
 	}
@@ -573,6 +645,7 @@ app.registerExtension({
 			case "easy svdLoader":
 			case "easy dynamiCrafterLoader":
 			case "easy loraStack":
+			case "easy controlnetStack":
 			case "easy latentNoisy":
 			case "easy preSampling":
 			case "easy preSamplingAdvanced":
@@ -594,6 +667,7 @@ app.registerExtension({
 			case "easy detailerFix":
 			case "easy imageRemBg":
 			case "easy imageColorMatch":
+			case "easy imageDetailTransfer":
 			case "easy loadImageBase64":
 			case "easy XYInputs: Steps":
 			case "easy XYInputs: Sampler/Scheduler":
@@ -609,6 +683,7 @@ app.registerExtension({
 			case 'easy ipadapterApply':
 			case 'easy ipadapterApplyADV':
 			case 'easy ipadapterApplyEncoder':
+			case 'easy applyInpaint':
 				getSetters(node)
 				break
 			case "easy wildcards":
@@ -947,24 +1022,15 @@ app.registerExtension({
 			const onNodeCreated = nodeType.prototype.onNodeCreated;
 			nodeType.prototype.onNodeCreated = async function () {
 				onNodeCreated ? onNodeCreated.apply(this, []) : undefined;
-				// const values = ["randomize", "fixed", "increment", "decrement"]
-				// const seed_widget = this.widgets.find(w => w.name == 'seed_num')
-				// const seed_control = this.addWidget("combo", "control_before_generate", values[0], () => {
-				// }, {
-				// 	values,
-				// 	serialize: false
-				// })
-				// seed_widget.linkedWidgets = [seed_control]
 				const seed_widget = this.widgets.find(w => ['seed_num','seed'].includes(w.name))
 				const seed_control = this.widgets.find(w=> ['control_before_generate','control_after_generate'].includes(w.name))
 				if(nodeData.name == 'easy seed'){
-					this.addWidget("button", "🎲 Manual Random Seed", null, _=>{
-						if(seed_control.value != 'fixed'){
-							seed_control.value = 'fixed'
-						}
+					const randomSeedButton = this.addWidget("button", "🎲 Manual Random Seed", null, _=>{
+						if(seed_control.value != 'fixed') seed_control.value = 'fixed'
 						seed_widget.value = Math.floor(Math.random() * 1125899906842624)
 						app.queuePrompt(0, 1)
-					})
+					},{ serialize:false})
+					seed_widget.linkedWidgets = [randomSeedButton, seed_control];
 				}
 			}
 			const onAdded = nodeType.prototype.onAdded;
@@ -1119,10 +1185,10 @@ const getSetWidgets = ['rescale_after_model', 'rescale',
 						'refiner_lora1_name', 'refiner_lora2_name', 'upscale_method', 
 						'image_output', 'add_noise', 'info', 'sampler_name',
 						'ckpt_B_name', 'ckpt_C_name', 'save_model', 'refiner_ckpt_name',
-						'num_loras', 'mode', 'toggle', 'resolution', 'target_parameter',
+						'num_loras', 'num_controlnet', 'mode', 'toggle', 'resolution', 'target_parameter',
 	'input_count', 'replace_count', 'downscale_mode', 'range_mode','text_combine_mode', 'input_mode',
 	'lora_count','ckpt_count', 'conditioning_mode', 'preset', 'use_tiled', 'use_batch', 'num_embeds',
-	"easing_mode", "guider", "scheduler"
+	"easing_mode", "guider", "scheduler", "inpaint_mode",
 ]
 
 function getSetters(node) {

web/js/easy/easyExtraMenu.js

@@ -9,6 +9,7 @@ const controlnet = ['easy controlnetLoader', 'easy controlnetLoaderADV', 'easy i
 const ipadapter = ['easy ipadapterApply', 'easy ipadapterApplyADV', 'easy ipadapterStyleComposition', 'easy ipadapterApplyFromParams']
 const positive_prompt = ['easy positive', 'easy wildcards']
 const imageNode = ['easy loadImageBase64', 'LoadImage', 'LoadImageMask']
+const inpaint = ['easy applyBrushNet', 'easy applyPowerPaint', 'easy applyInpaint']
 const widgetMapping = {
     "positive_prompt":{
         "text": "positive",
@@ -64,6 +65,14 @@ const widgetMapping = {
         "image":"image",
         "base64_data":"base64_data",
         "channel": "channel"
+    },
+    "inpaint":{
+        "dtype": "dtype",
+        "fitting": "fitting",
+        "function": "function",
+        "scale": "scale",
+        "start_at": "start_at",
+        "end_at": "end_at"
     }
 }
 const inputMapping = {
@@ -99,6 +108,11 @@ const inputMapping = {
         "image_style": "image",
         "attn_mask":"attn_mask",
         "optional_ipadapter":"optional_ipadapter"
+    },
+    "inpaint":{
+        "pipe": "pipe",
+        "image": "image",
+        "mask": "mask"
     }
 };
 
@@ -138,6 +152,9 @@ const outputMapping = {
         "masks":"masks",
         "ipadapter":"ipadapter"
     },
+    "inpaint":{
+        "pipe": "pipe",
+    }
 };
 
 // 替换节点
@@ -421,12 +438,14 @@ const reloadNode = function (node) {
 
     function handleLinks() {
         // re-convert inputs
-        for (let w of oldNode.widgets) {
-            if (w.type === 'converted-widget') {
-                const WidgetToConvert = newNode.widgets.find((nw) => nw.name === w.name);
-                for (let i of oldNode.inputs) {
-                    if (i.name === w.name) {
-                        convertToInput(newNode, WidgetToConvert, i.widget);
+        if(oldNode.widgets) {
+            for (let w of oldNode.widgets) {
+                if (w.type === 'converted-widget') {
+                    const WidgetToConvert = newNode.widgets.find((nw) => nw.name === w.name);
+                    for (let i of oldNode.inputs) {
+                        if (i.name === w.name) {
+                            convertToInput(newNode, WidgetToConvert, i.widget);
+                        }
                     }
                 }
             }
@@ -444,7 +463,7 @@ const reloadNode = function (node) {
 
     // fix widget values
     let values = oldNode.widgets_values;
-    if (!values) {
+    if (!values && newNode.widgets?.length>0) {
         newNode.widgets.forEach((newWidget, index) => {
             const oldWidget = oldNode.widgets[index];
             if (newWidget.name === oldWidget.name && newWidget.type === oldWidget.type) {
@@ -455,7 +474,7 @@ const reloadNode = function (node) {
         return;
     }
     let pass = false
-    const isIterateForwards = values.length <= newNode.widgets.length;
+    const isIterateForwards = values?.length <= newNode.widgets?.length;
     let vi = isIterateForwards ? 0 : values.length - 1;
     function evalWidgetValues(testValue, newWidg) {
         if (testValue === true || testValue === false) {
@@ -487,15 +506,15 @@ const reloadNode = function (node) {
             }
             vi++
             if (!isIterateForwards) {
-                vi = values.length - (newNode.widgets.length - 1 - wi);
+                vi = values.length - (newNode.widgets?.length - 1 - wi);
             }
         }
     };
-    if (isIterateForwards) {
+    if (isIterateForwards && newNode.widgets?.length>0) {
         for (let wi = 0; wi < newNode.widgets.length; wi++) {
             updateValue(wi);
         }
-    } else {
+    } else if(newNode.widgets?.length>0){
         for (let wi = newNode.widgets.length - 1; wi >= 0; wi--) {
             updateValue(wi);
         }
@@ -563,6 +582,10 @@ app.registerExtension({
         if (imageNode.includes(nodeData.name)) {
             addMenu("↪️ Swap LoadImage", 'load_image', imageNode, nodeType)
         }
+        // Swap inpaint
+        if (inpaint.includes(nodeData.name)) {
+            addMenu("↪️ Swap InpaintNode", 'inpaint', inpaint, nodeType)
+        }
     }
 });
 

web/js/easy/easySelector.js

@@ -180,6 +180,7 @@ app.registerExtension({
                                 selector.element.children[1].innerHTML=''
                                 if(styles_list_cache[styles_values]){
                                     let tags = styles_list_cache[styles_values]
+                                    this.properties["values"] = []
                                     // 重新排序
                                     if(selector.value) tags = tags.sort((a,b)=> selector.value.includes(b.name) - selector.value.includes(a.name))
                                     let list = getTagList(tags, value, language);

web/js/image_chooser/chooser.js

@@ -3,7 +3,7 @@ import { api } from "../../../../scripts/api.js";
 import { ComfyDialog, $el } from "../../../../scripts/ui.js";
 
 import { restart_from_here } from "./prompt.js";
-import { FlowState } from "./state.js";
+import { hud, FlowState } from "./state.js";
 import { send_cancel, send_message, send_onstart, skip_next_restart_message } from "./messaging.js";
 import { display_preview_images, additionalDrawBackground, click_is_in_image } from "./preview.js";
 import {$t} from "../common/i18n.js";
@@ -83,25 +83,24 @@ function progressButtonPressed() {
             skip_next_restart_message();
             restart_from_here(node.id).then(() => { send_message(node.id, [...node.selected, -1, ...node.anti_selected]); });
         }
-        const maxlength = node.imgs.length;
-        if (FlowState.paused_here(node.id) && selected>0) {
-            node.send_button_widget.name = (selected>1) ? "Progress selected (" + selected + '/' + maxlength  +")" : "Progress selected image";
-        } else if (FlowState.idle() && selected>0) {
-            node.send_button_widget.name = (selected>1) ? "Progress selected (" + selected + '/' + maxlength  +")" : "Progress selected image as restart";
-        }
-        else {
-            node.send_button_widget.name = "";
-        }
     }
 }
 
 function cancelButtonPressed() {
+
     if (FlowState.running()) { send_cancel();}
-    const node = app.graph._nodes_by_id[this.node_id];
-    if (node) {
-        node.send_button_widget.name = "";
-        node.cancel_button_widget.name = "";
-    }
+}
+
+function enable_disabling(button) {
+    Object.defineProperty(button, 'clicked', {
+        get : function() { return this._clicked; },
+        set : function(v) { this._clicked = (v && this.name!=''); }
+    })
+}
+
+function disable_serialize(widget) {
+    if (!widget.options) widget.options = {  };
+    widget.options.serialize = false;
 }
 
 app.registerExtension({
@@ -110,6 +109,16 @@ app.registerExtension({
         window.addEventListener("beforeunload", send_cancel, true);
     },
     setup(app) {
+
+        const draw = LGraphCanvas.prototype.draw;
+        LGraphCanvas.prototype.draw = function() {
+            if (hud.update()) {
+                app.graph._nodes.forEach((node)=> { if (node.update) { node.update(); } })
+            }
+            draw.apply(this,arguments);
+        }
+
+
         function easyuseImageChooser(event) {
             const {node,image,isKSampler} = display_preview_images(event);
             if(isKSampler) {
@@ -148,13 +157,8 @@ app.registerExtension({
     async nodeCreated(node, app) {
 
         if(node.comfyClass == 'easy imageChooser'){
-            node.send_button_widget = node.addWidget("button", "", "", progressButtonPressed, {serialize: false});
-            node.cancel_button_widget = node.addWidget("button", "", "", cancelButtonPressed, {serialize: false});
             node.setProperty('values',[])
 
-            /* Capture clicks */
-            const org_onMouseDown = node.onMouseDown;
-
             /* A property defining the top of the image when there is just one */
             if(node?.imageIndex === undefined){
               Object.defineProperty(node, 'imageIndex', {
@@ -169,6 +173,8 @@ app.registerExtension({
                 })
             }
 
+            /* Capture clicks */
+            const org_onMouseDown = node.onMouseDown;
             node.onMouseDown = function( e, pos, canvas ) {
                 if (e.isPrimary) {
                     const i = click_is_in_image(node, pos);
@@ -177,6 +183,13 @@ app.registerExtension({
                 return (org_onMouseDown && org_onMouseDown.apply(this, arguments));
             }
 
+            node.send_button_widget = node.addWidget("button", "", "", progressButtonPressed);
+            node.cancel_button_widget = node.addWidget("button", "", "", cancelButtonPressed);
+            enable_disabling(node.cancel_button_widget);
+            enable_disabling(node.send_button_widget);
+            disable_serialize(node.cancel_button_widget);
+            disable_serialize(node.send_button_widget);
+
         }
     },
 
@@ -190,9 +203,11 @@ app.registerExtension({
             }
 
             nodeType.prototype.imageClicked = function (imageIndex) {
-                if (this.selected.has(imageIndex)) this.selected.delete(imageIndex);
-                else this.selected.add(imageIndex);
-                this.update();
+                if (nodeType?.comfyClass==="easy imageChooser") {
+                    if (this.selected.has(imageIndex)) this.selected.delete(imageIndex);
+                    else this.selected.add(imageIndex);
+                    this.update();
+                }
             }
 
             const update = nodeType.prototype.update;
@@ -204,7 +219,7 @@ app.registerExtension({
                     const maxlength = this.imgs?.length || 0;
                     if (FlowState.paused_here(this.id) && selection>0) {
                         this.send_button_widget.name = (selection>1) ? "Progress selected (" + selection + '/' + maxlength  +")" : "Progress selected image";
-                    } else if (FlowState.idle() && selection>0) {
+                    } else if (selection>0) {
                         this.send_button_widget.name = (selection>1) ? "Progress selected (" + selection + '/' + maxlength  +")" : "Progress selected image as restart";
                     }
                     else {

web/js/image_chooser/messaging.js

@@ -15,9 +15,9 @@ function send_message(id, message) {
 
 function send_cancel() {
     send_message(-1,'__cancel__');
-    //FlowState.cancelling = true;
-    //api.interrupt();
-    //FlowState.cancelling = false;
+    FlowState.cancelling = true;
+    api.interrupt();
+    FlowState.cancelling = false;
 }
 
 var skip_next = 0;

web/js/image_chooser/state.js

@@ -1,6 +1,33 @@
 import { app } from "../../../../scripts/app.js";
 
-export class FlowState {
+
+class HUD {
+    constructor() {
+        this.current_node_id = undefined;
+        this.class_of_current_node = null;
+        this.current_node_is_chooser = false;
+    }
+
+    update() {
+        if (app.runningNodeId==this.current_node_id) return false;
+
+        this.current_node_id = app.runningNodeId;
+
+        if (this.current_node_id) {
+            this.class_of_current_node = app.graph?._nodes_by_id[app.runningNodeId.toString()]?.comfyClass;
+            this.current_node_is_chooser = this.class_of_current_node === "easy imageChooser"
+        } else {
+            this.class_of_current_node = undefined;
+            this.current_node_is_chooser = false;
+        }
+        return true;
+    }
+}
+
+const hud = new HUD();
+
+
+class FlowState {
     constructor(){}
     static idle() {
         return (!app.runningNodeId);
@@ -23,4 +50,6 @@ export class FlowState {
         return "Idle";
     }
     static cancelling = false;
-}
+}
+
+export { hud, FlowState}