Upgrade v1.2.3 to ComfyRegistry

Merge pull request #385 from yolain/execution_blocker

.github/FUNDING.yml

@@ -1,3 +1,3 @@
 # These are supported funding model platforms
 
-custom: ["https://afdian.net/a/yolain"]
+custom: ["https://space.bilibili.com/1840885116"]

.gitignore

@@ -7,6 +7,9 @@ wildcards/**
 styles/**
 workflow/**
 autocomplete/**
+web_beta/**
+web_version/dev/**
+ComfyUI-Easy-Use-Frontend/
 docs/**
 .vscode/
 .idea/

README.ZH_CN.md

@@ -0,0 +1,472 @@
+![comfyui-easy-use](https://github.com/user-attachments/assets/9b7a5e44-f5e2-4c27-aed2-d0e6b50c46bb)
+
+<div align="center">
+<a href="https://space.bilibili.com/1840885116">视频介绍</a> |
+文档 (康明孙) | 
+<a href="https://github.com/yolain/ComfyUI-Yolain-Workflows">工作流合集</a> |
+<a href="#%EF%B8%8F-donation">捐助</a> 
+<br><br>
+<a href="./README.md"><img src="https://img.shields.io/badge/🇬🇧English-e9e9e9"></a>
+<a href="./README.ZH_CN.md"><img src="https://img.shields.io/badge/🇨🇳中文简体-0b8cf5"></a>
+</div>
+
+**ComfyUI-Easy-Use** 是一个化繁为简的节点整合包, 在 [tinyterraNodes](https://github.com/TinyTerra/ComfyUI_tinyterraNodes) 的基础上进行延展，并针对了诸多主流的节点包做了整合与优化，以达到更快更方便使用ComfyUI的目的，在保证自由度的同时还原了本属于Stable Diffusion的极致畅快出图体验。
+
+## 👨🏻‍🎨 特色介绍
+
+- 沿用了 [tinyterraNodes](https://github.com/TinyTerra/ComfyUI_tinyterraNodes) 的思路，大大减少了折腾工作流的时间成本。
+- UI界面美化，首次安装的用户，如需使用UI主题，请在 Settings -> Color Palette 中自行切换主题并**刷新页面**即可
+- 增加了预采样参数配置的节点，可与采样节点分离，更方便预览。
+- 支持通配符与Lora的提示词节点，如需使用Lora Block Weight用法，需先保证自定义节点包中安装了 [ComfyUI-Inspire-Pack](https://github.com/ltdrdata/ComfyUI-Inspire-Pack)
+- 可多选的风格化提示词选择器，默认是Fooocus的样式json，可自定义json放在styles底下，samples文件夹里可放预览图(名称和name一致,图片文件名如有空格需转为下划线'_')
+- 加载器可开启A1111提示词风格模式，可重现与webui生成近乎相同的图像，需先安装 [ComfyUI_smZNodes](https://github.com/shiimizu/ComfyUI_smZNodes)
+- 可使用`easy latentNoisy`或`easy preSamplingNoiseIn`节点实现对潜空间的噪声注入
+- 简化 SD1.x、SD2.x、SDXL、SVD、Zero123等流程 
+- 简化 Stable Cascade [示例参考](https://github.com/yolain/ComfyUI-Yolain-Workflows?tab=readme-ov-file#1-13-stable-cascade)
+- 简化 Layer Diffuse [示例参考](https://github.com/yolain/ComfyUI-Yolain-Workflows?tab=readme-ov-file#2-3-layerdiffusion)
+- 简化 InstantID [示例参考](https://github.com/yolain/ComfyUI-Yolain-Workflows?tab=readme-ov-file#2-2-instantid), 需先保证自定义节点包中安装了 [ComfyUI_InstantID](https://github.com/cubiq/ComfyUI_InstantID)
+- 简化 IPAdapter, 需先保证自定义节点包中安装最新版v2的 [ComfyUI_IPAdapter_plus](https://github.com/cubiq/ComfyUI_IPAdapter_plus)
+- 扩展 XYplot 的可用性
+- 整合了Fooocus Inpaint功能
+- 整合了常用的逻辑计算、转换类型、展示所有类型等
+- 支持节点上checkpoint、lora模型子目录分类及预览图 (请在设置中开启上下文菜单嵌套子目录)
+- 支持BriaAI的RMBG-1.4模型的背景去除节点，[技术参考](https://huggingface.co/briaai/RMBG-1.4)
+- 支持 强制清理comfyUI模型显存占用
+- 支持Stable Diffusion 3 多账号API节点
+- 支持IC-Light的应用 [示例参考](https://github.com/yolain/ComfyUI-Yolain-Workflows?tab=readme-ov-file#2-5-ic-light) | [代码整合来源](https://github.com/huchenlei/ComfyUI-IC-Light) | [技术参考](https://github.com/lllyasviel/IC-Light)
+- 中文提示词自动识别，使用[opus-mt-zh-en模型](https://huggingface.co/Helsinki-NLP/opus-mt-zh-en)
+- 支持 sd3 模型
+- 支持 kolors 模型
+- 支持 flux 模型
+- 支持 惰性条件判断（ifElse）和 for循环
+
+## 👨🏻‍🔧 安装
+
+1. 将存储库克隆到 **custom_nodes** 目录并安装依赖
+```shell
+#1. git下载
+git clone https://github.com/yolain/ComfyUI-Easy-Use
+#2. 安装依赖
+双击install.bat安装依赖
+```
+
+## 👨🏻‍🚀 计划
+
+- [x] 更新便于维护的新前端代码
+  - [x] 使用sass维护css样式
+  - [x] 对原有扩展进行优化
+  - [x] 增加新的组件（如节点时间统计等） 
+- [ ] 在[ComfyUI-Yolain-Workflows](https://github.com/yolain/ComfyUI-Yolain-Workflows)中上传更多的工作流（如kolors,sd3等），并更新english版本的readme  
+- [ ] 更详细功能介绍的 gitbook   
+
+## 📜 更新日志
+
+**v1.2.3**
+
+- `easy showAnything` 和 `easy cleanGPUUsed` 增加输出插槽
+- 添加新的人体分割在 `easy humanSegmentation` 节点上 - 代码从 (ComfyUI_Human_Parts)[https://github.com/metal3d/ComfyUI_Human_Parts] 整合
+- 当你在 `easy preSamplingCustom` 节点上选择basicGuider，CFG>0 且当前模型为Flux时，将使用FluxGuidance
+- 增加 `easy loraStackApply` and `easy controlnetStackApply`
+
+**v1.2.2**
+
+- 增加 `easy batchAny`
+- 增加 `easy anythingIndexSwitch`
+- 增加 `easy forLoopStart` 和 `easy forLoopEnd`  
+- 增加 `easy ifElse`
+- 增加 v2 版本新前端代码
+- 增加 `easy fluxLoader`
+- 增加 `controlnetApply` 相关节点对sd3和hunyuanDiT的支持
+- 修复 当使用fooocus inpaint后，再使用Lora模型无法生效的问题
+
+**v1.2.1**
+
+- 增加 `easy ipadapterApplyFaceIDKolors`
+- `easy ipadapterApply` 和 `easy ipadapterApplyADV` 增加 **PLUS (kolors genernal)** 和 **FACEID PLUS KOLORS** 预置项
+- `easy imageRemBg` 增加 **inspyrenet** 选项
+- 增加 `easy controlnetLoader++`
+- 去除 `easy positive` `easy negative` 等prompt节点的自动将中文翻译功能，自动翻译仅在 `easy a1111Loader` 等不支持中文TE的加载器中生效
+- 增加 `easy kolorsLoader` - 可灵加载器，参考了 [MinusZoneAI](https://github.com/MinusZoneAI/ComfyUI-Kolors-MZ) 和 [kijai](https://github.com/kijai/ComfyUI-KwaiKolorsWrapper) 的代码。
+
+**v1.2.0**
+
+- 增加 `easy pulIDApply` 和 `easy pulIDApplyADV`
+- 增加 `easy hunyuanDiTLoader` 和 `easy pixArtLoader`
+- 当新菜单的位置在上或者下时增加上 crystools 的显示，推荐开两个就好（如果后续crystools有更新UI适配我可能会删除掉）
+- 增加 **easy sliderControl** - 滑块控制节点，当前可用于控制ipadapterMS的参数 (双击滑块可重置为默认值)
+- 增加 **layer_weights** 属性在 `easy ipadapterApplyADV` 节点
+
+**v1.1.9**
+
+- 增加 新的调度器 **gitsScheduler**
+- 增加 `easy imageBatchToImageList` 和 `easy imageListToImageBatch` (修复Impact版的一点小问题)
+- 递归模型子目录嵌套
+- 支持 sd3 模型 
+- 增加 `easy applyInpaint` - 局部重绘全模式节点 (相比与之前的kSamplerInpating节点逻辑会更合理些)
+
+**v1.1.8**
+
+- 增加中文提示词自动翻译，使用[opus-mt-zh-en模型](https://huggingface.co/Helsinki-NLP/opus-mt-zh-en), 默认已对wildcard、lora正则处理, 其他需要保留的中文，可使用`@你的提示词@`包裹 (若依赖安装完成后报错, 请重启)，测算大约会占0.3GB显存
+- 增加 `easy controlnetStack` - controlnet堆
+- 增加 `easy applyBrushNet` - [示例参考](https://github.com/yolain/ComfyUI-Yolain-Workflows/blob/main/workflows/2_advanced/2-4inpainting/2-4brushnet_1.1.8.json)
+- 增加 `easy applyPowerPaint` - [示例参考](https://github.com/yolain/ComfyUI-Yolain-Workflows/blob/main/workflows/2_advanced/2-4inpainting/2-4powerpaint_outpaint_1.1.8.json)
+
+**v1.1.7**
+
+- 修复 一些模型(如controlnet模型等)未成功写入缓存，导致修改前置节点束参数（如提示词）需要二次载入模型的问题
+- 增加 `easy prompt` - 主体和光影预置项，后期可能会调整
+- 增加 `easy icLightApply` - 重绘光影, 从[ComfyUI-IC-Light](https://github.com/huchenlei/ComfyUI-IC-Light)优化
+- 增加 `easy imageSplitGrid` - 图像网格拆分
+- `easy kSamplerInpainting` 的 **additional** 属性增加差异扩散和brushnet等相关选项 
+- 增加 brushnet模型加载的支持 - [ComfyUI-BrushNet](https://github.com/nullquant/ComfyUI-BrushNet)
+- 增加 `easy applyFooocusInpaint` - Fooocus内补节点 替代原有的 FooocusInpaintLoader
+- 移除 `easy fooocusInpaintLoader` - 容易bug，不再使用
+- 修改 easy kSampler等采样器中并联的model 不再替换输出中pipe里的model
+
+**v1.1.6**
+
+- 增加步调齐整适配 - 在所有的预采样和全采样器节点中的 调度器(schedulder) 增加了 **alignYourSteps** 选项
+- `easy kSampler` 和 `easy fullkSampler` 的 **image_output** 增加 **Preview&Choose**选项
+- 增加 `easy styleAlignedBatchAlign` - 风格对齐 [style_aligned_comfy](https://github.com/brianfitzgerald/style_aligned_comfy)
+- 增加 `easy ckptNames`
+- 增加 `easy controlnetNames`
+- 增加 `easy imagesSplitimage` - 批次图像拆分单张
+- 增加 `easy imageCount` - 图像数量
+- 增加 `easy textSwitch` - 文字切换
+
+**v1.1.5**
+
+- 重写 `easy cleanGPUUsed` - 可强制清理comfyUI的模型显存占用
+- 增加 `easy humanSegmentation` - 多类分割、人像分割
+- 增加 `easy imageColorMatch`
+- 增加 `easy ipadapterApplyRegional`
+- 增加 `easy ipadapterApplyFromParams`
+- 增加 `easy imageInterrogator` - 图像反推
+- 增加 `easy stableDiffusion3API` - 简易的Stable Diffusion 3 多账号API节点
+
+**v1.1.4**
+
+- 增加 `easy imageChooser` - 从[cg-image-picker](https://github.com/chrisgoringe/cg-image-picker)简化的图片选择器
+- 增加 `easy preSamplingCustom` - 自定义预采样，可支持cosXL-edit
+- 增加 `easy ipadapterStyleComposition`
+- 增加 在Loaders上右键菜单可查看 checkpoints、lora 信息
+- 修复 `easy preSamplingNoiseIn`、`easy latentNoisy`、`east Unsampler` 以兼容ComfyUI Revision>=2098 [0542088e] 以上版本
+- 修复 FooocusInpaint修改ModelPatcher计算权重引发的问题，理应在生成model后重置ModelPatcher为默认值
+
+**v1.1.3**
+
+- `easy ipadapterApply` 增加 **COMPOSITION** 预置项
+- 增加 对[ResAdapter](https://huggingface.co/jiaxiangc/res-adapter) lora模型 的加载支持
+- 增加 `easy promptLine`
+- 增加 `easy promptReplace`
+- 增加 `easy promptConcat`
+- `easy wildcards` 增加 **multiline_mode**属性 
+- 增加 当节点需要下载模型时，若huggingface连接超时，会切换至镜像地址下载模型
+
+<details>
+<summary><b>v1.1.2</b></summary>
+
+- 改写 EasyUse 相关节点的部分插槽推荐节点
+- 增加 **启用上下文菜单自动嵌套子目录** 设置项，默认为启用状态，可分类子目录及checkpoints、loras预览图
+- 增加 `easy sv3dLoader` 
+- 增加 `easy dynamiCrafterLoader` 
+- 增加 `easy ipadapterApply`
+- 增加 `easy ipadapterApplyADV`
+- 增加 `easy ipadapterApplyEncoder`
+- 增加 `easy ipadapterApplyEmbeds`
+- 增加 `easy preMaskDetailerFix`
+- `easy kSamplerInpainting` 增加 **additional** 属性，可设置成 Differential Diffusion 或 Only InpaintModelConditioning
+- 修复 `easy stylesSelector` 当未选择样式时，原有提示词发生了变化
+- 修复 `easy pipeEdit` 提示词输入lora时报错
+- 修复 layerDiffuse xyplot相关bug
+</details>
+
+<details>
+<summary><b>v1.1.1/b></summary>
+
+- 修复首次添加含seed的节点且当前模式为control_before_generate时，seed为0的问题
+- `easy preSamplingAdvanced` 增加 **return_with_leftover_noise**
+- 修复 `easy stylesSelector` 当选择自定义样式文件时运行队列报错
+- `easy preSamplingLayerDiffusion` 增加 mask 可选传入参数
+- 将所有 **seed_num** 调整回 **seed**
+- 修补官方BUG: 当control_mode为before 在首次加载页面时未修改节点中widget名称为 control_before_generate
+- 去除强制**control_before_generate**设定
+- 增加 `easy imageRemBg` - 默认为BriaAI的RMBG-1.4模型, 移除背景效果更加，速度更快
+</details>
+
+<details>
+<summary><b>v1.1.0</b></summary>
+
+- 增加 `easy imageSplitList` - 拆分每 N 张图像
+- 增加 `easy preSamplingDiffusionADDTL` - 可配置前景、背景、blended的additional_prompt等   
+- 增加 `easy preSamplingNoiseIn` 可替代需要前置的`easy latentNoisy`节点 实现效果更好的噪声注入
+- `easy pipeEdit` 增加 条件拼接模式选择，可选择替换、合并、联结、平均、设置条件时间
+- 增加 `easy pipeEdit` - 可编辑Pipe的节点（包含可重新输入提示词）
+- 增加 `easy preSamplingLayerDiffusion` 与 `easy kSamplerLayerDiffusion` （连接 `easy kSampler` 也能通）
+- 增加 在 加载器、预采样、采样器、Controlnet等节点上右键可快速替换同类型节点的便捷菜单
+- 增加 `easy instantIDApplyADV` 可连入 positive 与 negative
+- 修复 `easy wildcards` 读取lora未填写完整路径时未自动检索导致加载lora失败的问题
+- 修复 `easy instantIDApply` mask 未传入正确值
+- 修复 在 非a1111提示词风格下 BREAK 不生效的问题
+</details>
+
+<details>
+<summary><b>v1.0.9</b></summary>
+
+- 修复未安装 ComfyUI-Impack-Pack 和 ComfyUI_InstantID 时报错
+- 修复 `easy pipeIn` - pipe设为可不必选
+- 增加 `easy instantIDApply` - 需要先安装 [ComfyUI_InstantID](https://github.com/cubiq/ComfyUI_InstantID), 工作流参考[示例](https://github.com/yolain/ComfyUI-Yolain-Workflows?tab=readme-ov-file#2-2-instantid)
+- 修复 `easy detailerFix` 未添加到保存图片格式化扩展名可用节点列表
+- 修复 `easy XYInputs: PromptSR` 在替换负面提示词时报错
+</details>
+
+<details>
+<summary><b>v1.0.8</b></summary>
+
+- `easy cascadeLoader` stage_c 与 stage_b 支持checkpoint模型 (需要下载[checkpoints](https://huggingface.co/stabilityai/stable-cascade/tree/main/comfyui_checkpoints)) 
+- `easy styleSelector` 搜索框修改为不区分大小写匹配
+- `easy fullLoader` 增加 **positive**、**negative**、**latent** 输出项
+- 修复 SDXLClipModel 在 ComfyUI 修订版本号 2016[c2cb8e88] 及以上的报错（判断了版本号可兼容老版本）
+- 修复 `easy detailerFix` 批次大小大于1时生成出错
+- 修复`easy preSampling`等 latent传入后无法根据批次索引生成的问题
+- 修复 `easy svdLoader` 报错
+- 优化代码，减少了诸多冗余，提升运行速度
+- 去除中文翻译对照文本
+
+（翻译对照已由 [AIGODLIKE-COMFYUI-TRANSLATION](https://github.com/AIGODLIKE/AIGODLIKE-ComfyUI-Translation) 统一维护啦！
+首次下载或者版本较早的朋友请更新 AIGODLIKE-COMFYUI-TRANSLATION 和本节点包至最新版本。）
+</details>
+
+<details>
+<summary><b>v1.0.7</b></summary>
+
+- 增加 `easy cascadeLoader` - stable cascade 加载器
+- 增加 `easy preSamplingCascade` - stabled cascade stage_c 预采样参数
+- 增加 `easy fullCascadeKSampler` - stable cascade stage_c 完整版采样器
+- 增加 `easy cascadeKSampler` - stable cascade stage-c ksampler simple
+</details>
+
+<details>
+<summary><b>v1.0.6</b></summary>
+
+- 增加 `easy XYInputs: Checkpoint`
+- 增加 `easy XYInputs: Lora`
+- `easy seed` 增加固定种子值时可手动切换随机种
+- 修复 `easy fullLoader`等加载器切换lora时自动调整节点大小的问题
+- 去除原有ttn的图片保存逻辑并适配ComfyUI默认的图片保存格式化扩展
+</details>
+
+<details>
+<summary><b>v1.0.5</b></summary>
+
+- 增加 `easy isSDXL` 
+- `easy svdLoader` 增加提示词控制, 可配合open_clip模型进行使用
+- `easy wildcards` 增加 **populated_text** 可输出通配填充后文本
+</details>
+
+<details>
+<summary><b>v1.0.4</b></summary>
+
+- 增加 `easy showLoaderSettingsNames` 可显示与输出加载器部件中的 模型与VAE名称
+- 增加 `easy promptList` - 提示词列表
+- 增加 `easy fooocusInpaintLoader` - Fooocus内补节点（仅支持XL模型的流程）
+- 增加 **Logic** 逻辑类节点 - 包含类型、计算、判断和转换类型等
+- 增加 `easy imageSave` - 带日期转换和宽高格式化的图像保存节点
+- 增加 `easy joinImageBatch` - 合并图像批次
+- `easy showAnything` 增加支持转换其他类型（如：tensor类型的条件、图像等）
+- `easy kSamplerInpainting` 增加 **patch** 传入值，配合Fooocus内补节点使用
+- `easy imageSave` 增加 **only_preivew**
+
+- 修复 xyplot在pillow>9.5中报错
+- 修复 `easy wildcards` 在使用PS扩展插件运行时报错
+- 修复 `easy latentCompositeMaskedWithCond`
+- 修复 `easy XYInputs: ControlNet` 报错
+- 修复 `easy loraStack` **toggle** 为 disabled 时报错
+
+- 修改首次安装节点包不再自动替换主题，需手动调整并刷新页面
+</details>
+
+<details>
+<summary><b>v1.0.3</b></summary>
+
+- 增加 `easy stylesSelector` 风格化提示词选择器
+- 增加队列进度条设置项，默认为未启用状态
+- `easy controlnetLoader` 和 `easy controlnetLoaderADV` 增加参数 **scale_soft_weights**
+
+
+- 修复 `easy XYInputs: Sampler/Scheduler` 报错
+- 修复 右侧菜单 点击按钮时老是跑位的问题
+- 修复 styles 路径在其他环境报错
+- 修复 `easy comfyLoader` 读取错误
+- 修复 xyPlot 在连接 zero123 时报错
+- 修复加载器中提示词为组件时报错
+- 修复 `easy getNode` 和 `easy setNode` 加载时标题未更改
+- 修复所有采样器中存储图片使用子目录前缀不生效的问题
+
+
+- 调整UI主题
+</details>
+
+<details>
+<summary><b>v1.0.2</b></summary>
+
+- 增加 **autocomplete** 文件夹，如果您安装了 [ComfyUI-Custom-Scripts](https://github.com/pythongosssss/ComfyUI-Custom-Scripts), 将在启动时合并该文件夹下的所有txt文件并覆盖到pyssss包里的autocomplete.txt文件。
+- 增加 `easy XYPlotAdvanced` 和 `easy XYInputs` 等相关节点
+- 增加 **Alt+1到9** 快捷键，可快速粘贴 Node templates 的节点预设 （对应 1到9 顺序）
+
+- 修复 `easy imageInsetCrop` 测量值为百分比时步进为1
+- 修复 开启 `a1111_prompt_style` 时XY图表无法使用的问题
+- 右键菜单中增加了一个 `📜Groups Map(EasyUse)` 
+
+- 修复在Comfy新版本中UI加载失败
+- 修复 `easy pipeToBasicPipe` 报错
+- 修改 `easy fullLoader` 和 `easy a1111Loader` 中的 **a1111_prompt_style** 默认值为 False
+- `easy XYInputs ModelMergeBlocks` 支持csv文件导入数值
+
+- 替换了XY图生成时的字体文件
+
+- 移除 `easy imageRemBg`
+- 移除包中的介绍图和工作流文件，减少包体积
+
+</details>
+
+<details>
+<summary><b>v1.0.1</b></summary>
+
+- 新增 `easy seed` - 简易随机种
+- `easy preDetailerFix` 新增了 `optional_image` 传入图像可选，如未传默认取值为pipe里的图像
+- 新增 `easy kSamplerInpainting` 用于内补潜空间的采样器
+- 新增 `easy pipeToBasicPipe` 用于转换到Impact的某些节点上
+
+- 修复 `easy comfyLoader` 报错
+- 修复所有包含输出图片尺寸的节点取值方式无法批处理的问题
+- 修复 `width` 和 `height` 无法在 `easy svdLoader` 自定义的报错问题
+- 修复所有采样器预览图片的地址链接 (解决在 MACOS 系统中图片无法在采样器中预览的问题）
+- 修复 `vae_name` 在 `easy fullLoader` 和 `easy a1111Loader` 和 `easy comfyLoader` 中选择但未替换原始vae问题
+- 修复 `easy fullkSampler` 除pipe外其他输出值的报错
+- 修复 `easy hiresFix` 输入连接pipe和image、vae同时存在时报错
+- 修复 `easy fullLoader` 中 `model_override` 连接后未执行 
+- 修复 因新增`easy seed` 导致action错误
+- 修复 `easy xyplot` 的字体文件路径读取错误
+- 修复 convert 到 `easy seed` 随机种无法固定的问题
+- 修复 `easy pipeIn` 值传入的报错问题
+- 修复 `easy zero123Loader` 和 `easy svdLoader` 读取模型时将模型加入到缓存中
+- 修复 `easy kSampler` `easy kSamplerTiled` `easy detailerFix` 的 `image_output` 默认值为 Preview
+- `easy fullLoader` 和 `easy a1111Loader` 新增了 `a1111_prompt_style` 参数可以重现和webui生成相同的图像，当前您需要安装 [ComfyUI_smZNodes](https://github.com/shiimizu/ComfyUI_smZNodes) 才能使用此功能
+</details>
+
+<details>
+<summary><b>v1.0.0</b></summary>
+
+- 新增`easy positive` - 简易正面提示词文本
+- 新增`easy negative`  - 简易负面提示词文本
+- 新增`easy wildcards` - 支持通配符和Lora选择的提示词文本
+- 新增`easy portraitMaster` - 肖像大师v2.2
+- 新增`easy loraStack` - Lora堆
+- 新增`easy fullLoader` - 完整版的加载器
+- 新增`easy zero123Loader` - 简易zero123加载器
+- 新增`easy svdLoader` - 简易svd加载器
+- 新增`easy fullkSampler` - 完整版的采样器（无分离）
+- 新增`easy hiresFix` - 支持Pipe的高清修复
+- 新增`easy predetailerFix` `easy DetailerFix` - 支持Pipe的细节修复
+- 新增`easy ultralyticsDetectorPipe` `easy samLoaderPipe` - 检测加载器（细节修复的输入项）
+- 新增`easy pipein` `easy pipeout` - Pipe的输入与输出
+- 新增`easy xyPlot` - 简易的xyplot (后续会更新更多可控参数)
+- 新增`easy imageRemoveBG` - 图像去除背景
+- 新增`easy imagePixelPerfect` - 图像完美像素
+- 新增`easy poseEditor` - 姿势编辑器
+- 新增UI主题（黑曜石）- 默认自动加载UI, 也可在设置中自行更替 
+
+- 修复 `easy globalSeed` 不生效问题
+- 修复所有的`seed_num` 因 [cg-use-everywhere](https://github.com/chrisgoringe/cg-use-everywhere) 实时更新图表导致值错乱的问题
+- 修复`easy imageSize` `easy imageSizeBySide` `easy imageSizeByLongerSide` 可作为终节点
+- 修复 `seed_num` (随机种子值) 在历史记录中读取无法一致的Bug
+</details>
+
+
+<details>
+<summary><b>v0.5</b></summary>
+
+- 新增 `easy controlnetLoaderADV` 节点
+-  新增 `easy imageSizeBySide` 节点，可选输出为长边或短边
+-  新增 `easy LLLiteLoader` 节点，如果您预先安装过 kohya-ss/ControlNet-LLLite-ComfyUI 包，请将 models 里的模型文件移动至 ComfyUI\models\controlnet\ (即comfy默认的controlnet路径里，请勿修改模型的文件名，不然会读取不到)。
+-  新增 `easy imageSize` 和 `easy imageSizeByLongerSize` 输出的尺寸显示。
+-  新增 `easy showSpentTime` 节点用于展示图片推理花费时间与VAE解码花费时间。
+- `easy controlnetLoaderADV` 和 `easy controlnetLoader` 新增 `control_net` 可选传入参数
+- `easy preSampling` 和 `easy preSamplingAdvanced` 新增 `image_to_latent` 可选传入参数
+- `easy a1111Loader` 和 `easy comfyLoader` 新增 `batch_size` 传入参数
+
+-  修改 `easy controlnetLoader` 到 loader 分类底下。
+</details>
+
+## 整合参考到的相关节点包
+
+声明: 非常尊重这些原作者们的付出，开源不易，我仅仅只是做了一些整合与优化。
+
+| 节点名 (搜索名)                      | 相关的库                                                                        | 库相关的节点                  |
+|:-------------------------------|:----------------------------------------------------------------------------|:------------------------|
+| easy setNode                   | [ComfyUI-extensions](https://github.com/diffus3/ComfyUI-extensions) | diffus3.SetNode         |
+| easy getNode                   | [ComfyUI-extensions](https://github.com/diffus3/ComfyUI-extensions) | diffus3.GetNode         |
+| easy bookmark                  | [rgthree-comfy](https://github.com/rgthree/rgthree-comfy) | Bookmark 🔖             |
+| easy portraitMarker            | [comfyui-portrait-master](https://github.com/florestefano1975/comfyui-portrait-master) | Portrait Master         |
+| easy LLLiteLoader              | [ControlNet-LLLite-ComfyUI](https://github.com/kohya-ss/ControlNet-LLLite-ComfyUI) | LLLiteLoader            |
+| easy globalSeed                | [ComfyUI-Inspire-Pack](https://github.com/ltdrdata/ComfyUI-Inspire-Pack) | Global Seed (Inspire)   | 
+| easy preSamplingDynamicCFG     | [sd-dynamic-thresholding](https://github.com/mcmonkeyprojects/sd-dynamic-thresholding) | DynamicThresholdingFull | 
+| dynamicThresholdingFull        | [sd-dynamic-thresholding](https://github.com/mcmonkeyprojects/sd-dynamic-thresholding) | DynamicThresholdingFull | 
+| easy imageInsetCrop            | [rgthree-comfy](https://github.com/rgthree/rgthree-comfy) | ImageInsetCrop          | 
+| easy poseEditor                | [ComfyUI_Custom_Nodes_AlekPet](https://github.com/AlekPet/ComfyUI_Custom_Nodes_AlekPet) | poseNode                | 
+| easy if                        | [ComfyUI-Logic](https://github.com/theUpsider/ComfyUI-Logic) | IfExecute               | 
+| easy preSamplingLayerDiffusion | [ComfyUI-layerdiffusion](https://github.com/huchenlei/ComfyUI-layerdiffusion) | LayeredDiffusionApply等  | 
+| easy dynamiCrafterLoader       | [ComfyUI-layerdiffusion](https://github.com/ExponentialML/ComfyUI_Native_DynamiCrafter) | Apply Dynamicrafter     | 
+| easy imageChooser              | [cg-image-picker](https://github.com/chrisgoringe/cg-image-picker) | Preview Chooser         | 
+| easy styleAlignedBatchAlign    | [style_aligned_comfy](https://github.com/chrisgoringe/cg-image-picker) | styleAlignedBatchAlign  | 
+| easy icLightApply              | [ComfyUI-IC-Light](https://github.com/huchenlei/ComfyUI-IC-Light) | ICLightApply等           |
+| easy kolorsLoader              | [ComfyUI-Kolors-MZ](https://github.com/MinusZoneAI/ComfyUI-Kolors-MZ) | kolorsLoader            |
+
+## Credits
+
+[ComfyUI](https://github.com/comfyanonymous/ComfyUI) - 功能强大且模块化的Stable Diffusion GUI
+
+[ComfyUI-ComfyUI-Manager](https://github.com/ltdrdata/ComfyUI-Manager) - ComfyUI管理器
+
+[tinyterraNodes](https://github.com/TinyTerra/ComfyUI_tinyterraNodes) - 管道节点（节点束）让用户减少了不必要的连接
+
+[ComfyUI-extensions](https://github.com/diffus3/ComfyUI-extensions) - diffus3的获取与设置点让用户可以分离工作流构成
+
+[ComfyUI-Impact-Pack](https://github.com/ltdrdata/ComfyUI-Impact-Pack) - 常规整合包1
+
+[ComfyUI-Inspire-Pack](https://github.com/ltdrdata/ComfyUI-Inspire-Pack) - 常规整合包2
+
+[ComfyUI-Logic](https://github.com/theUpsider/ComfyUI-Logic) -  ComfyUI逻辑运算
+
+[ComfyUI-ResAdapter](https://github.com/jiaxiangc/ComfyUI-ResAdapter) - 让模型生成不受训练分辨率限制
+
+[ComfyUI_IPAdapter_plus](https://github.com/cubiq/ComfyUI_IPAdapter_plus) - 风格迁移
+
+[ComfyUI_InstantID](https://github.com/cubiq/ComfyUI_InstantID) - 人脸迁移
+
+[ComfyUI_PuLID](https://github.com/cubiq/PuLID_ComfyUI) - 人脸迁移
+
+[ComfyUI-Custom-Scripts](https://github.com/pythongosssss/ComfyUI-Custom-Scripts) - pyssss 小蛇🐍脚本
+
+[cg-image-picker](https://github.com/chrisgoringe/cg-image-picker) - 图片选择器
+
+[ComfyUI-BrushNet](https://github.com/nullquant/ComfyUI-BrushNet) - BrushNet 内补节点
+
+[ComfyUI_ExtraModels](https://github.com/city96/ComfyUI_ExtraModels) - DiT架构相关节点（Pixart、混元DiT等）
+
+## ☕️ Donation
+
+**Comfyui-Easy-Use** 是一个 GPL 许可的开源项目。为了项目取得更好、可持续的发展，我希望能够获得更多的支持。 如果我的自定义节点为您的一天增添了价值，请考虑喝杯咖啡来进一步补充能量！ 💖感谢您的支持，每一杯咖啡都是我创作的动力！
+
+- [BiliBili充电](https://space.bilibili.com/1840885116)
+- [爱发电](https://afdian.com/a/yolain)
+- [Wechat/Alipay](https://github.com/user-attachments/assets/803469bd-ed6a-4fab-932d-50e5088a2d03)
+
+感谢您的捐助，我将用这些费用来租用 GPU 或购买其他 GPT 服务，以便更好地调试和完善 ComfyUI-Easy-Use 功能
+
+##  🌟Stargazers
+
+My gratitude extends to the generous souls who bestow a star. Your support is much appreciated!
+
+[![Stargazers repo roster for @yolain/ComfyUI-Easy-Use](https://reporoster.com/stars/yolain/ComfyUI-Easy-Use)](https://github.com/yolain/ComfyUI-Easy-Use/stargazers)

README.en.md

@@ -1,402 +0,0 @@
-<p align="right">
-   <a href="./README.md">中文</a> | <strong>English</strong>
-</p>
-
-<div align="center">
-
-# ComfyUI Easy Use
-</div>
-
-**ComfyUI-Easy-Use** is a simplified node integration package, which is extended on the basis of [tinyterraNodes](https://github.com/TinyTerra/ComfyUI_tinyterraNodes), and has been integrated and optimized for many mainstream node packages to achieve the purpose of faster and more convenient use of ComfyUI. While ensuring the degree of freedom, it restores the ultimate smooth image production experience that belongs to Stable Diffusion.
-
-[![ComfyUI-Yolain-Workflows](https://github.com/yolain/ComfyUI-Easy-Use/assets/73304135/9a3f54bc-a677-4bf1-a196-8845dd57c942)](https://github.com/yolain/ComfyUI-Yolain-Workflows)
-
-## Introduce
-
-- Inspire by [tinyterraNodes](https://github.com/TinyTerra/ComfyUI_tinyterraNodes), which greatly reduces the time cost of tossing workflows。
-- UI interface beautification, the first time you install the user, if you need to use the UI theme, please switch the theme in Settings -> Color Palette and refresh page.
-- Added a node for pre-sampling parameter configuration, which can be separated from the sampling node for easier previewing
-- Wildcards and lora's are supported, for Lora Block Weight usage, ensure that the custom node package has the [ComfyUI-Inspire-Pack](https://github.com/ltdrdata/ComfyUI-Inspire-Pack)
-- Multi-selectable styled cue word selector, default is Fooocus style json, custom json can be placed under styles, samples folder can be placed in the preview image (name and name consistent, image file name such as spaces need to be converted to underscores '_')
-- The loader enables the A1111 prompt mode, which reproduces nearly identical images to those generated by webui, and needs to be installed [ComfyUI_smZNodes](https://github.com/shiimizu/ComfyUI_smZNodes) first. 
-- Noise injection into the latent space can be achieved using the `easy latentNoisy` or `easy preSamplingNoiseIn` node
-- Simplified processes for SD1.x, SD2.x, SDXL, SVD, Zero123, etc. [Example](https://github.com/yolain/ComfyUI-Easy-Use?tab=readme-ov-file#StableDiffusion)
-- Simplified Stable Cascade [Example](https://github.com/yolain/ComfyUI-Easy-Use?tab=readme-ov-file#StableCascade)
-- Simplified Layer Diffuse [Example](https://github.com/yolain/ComfyUI-Easy-Use?tab=readme-ov-file#LayerDiffusion)，The first time you use it you may need to run `pip install -r requirements.txt` to install the required dependencies.
-- Simplified InstantID [Example](https://github.com/yolain/ComfyUI-Easy-Use?tab=readme-ov-file#InstantID), You need to make sure that the custom node package has the [ComfyUI_InstantID](https://github.com/cubiq/ComfyUI_InstantID)
-- Extending the usability of XYplot
-- Fooocus Inpaint integration
-- Integration of common logical calculations, conversion of types, display of all types, etc.
-- Background removal nodes for the RMBG-1.4 model supporting BriaAI, [BriaAI Guide](https://huggingface.co/briaai/RMBG-1.4)
-- Forcibly cleared the memory usage of the comfy UI model are supported
-- Stable Diffusion 3 multi-account API nodes are supported
-- Support Stable Diffusion 3 model
-
-## Installation
-Clone the repo into the **custom_nodes** directory and install the requirements:
-```shell
-#1. Clone the repo
-git clone https://github.com/yolain/ComfyUI-Easy-Use
-#2. Install the requirements
-Double-click install.bat to install the required dependencies
-```
-
-## Changelog
-
-**v1.2.0**
-
-- Added `easy pulIDApply` and `easy pulIDApplyADV`
-- Added `easy huanyuanDiTLoader` and `easy pixArtLoader`
-- Added **easy sliderControl** - Slider control node, which can currently be used to control the parameters of ipadapterMS (double-click the slider to reset to default)
-- Added **layer_weights** in `easy ipadapterApplyADV`
-
-**v1.1.9**
-
-- Added **gitsScheduler**
-- Added `easy imageBatchToImageList` and `easy imageListToImageBatch` 
-- Recursive subcategories nested for models
-- Support for Stable Diffusion 3 model
-- Added `easy applyInpaint` - All inpainting mode in this node
-
-**v1.1.8**
-
-- Added `easy controlnetStack`
-- Added `easy applyBrushNet` - [Workflow Example](https://github.com/yolain/ComfyUI-Yolain-Workflows/blob/main/workflows/2_advanced/2-4inpainting/2-4brushnet_1.1.8.json)
-- Added `easy applyPowerPaint` - [Workflow Example](https://github.com/yolain/ComfyUI-Yolain-Workflows/blob/main/workflows/2_advanced/2-4inpainting/2-4powerpaint_outpaint_1.1.8.json)
-
-**v1.1.7**
-
-- Added `easy prompt` - Subject and light presets, maybe adjusted later
-- Added `easy icLightApply` - Light and shadow migration, Code based on [ComfyUI-IC-Light](https://github.com/huchenlei/ComfyUI-IC-Light)
-- Added `easy imageSplitGrid`
-- `easy kSamplerInpainting` added options such as different diffusion and brushnet in **additional** widget
-- Support for brushnet model loading - [ComfyUI-BrushNet](https://github.com/nullquant/ComfyUI-BrushNet)
-- Added `easy applyFooocusInpaint` - Replace FooocusInpaintLoader
-- Removed `easy fooocusInpaintLoader`
-
-**v1.1.6**
-
-- Added **alignYourSteps** to **schedulder** widget in all `easy preSampling` and `easy fullkSampler`
-- Added **Preview&Choose** to **image_output** widget in `easy kSampler` & `easy fullkSampler`
-- Added `easy styleAlignedBatchAlign` - Credit of [style_aligned_comfy](https://github.com/brianfitzgerald/style_aligned_comfy)
-- Added `easy ckptNames`
-- Added `easy controlnetNames`
-- Added `easy imagesSplitimage` - Batch images split into single images
-- Added `easy imageCount` - Get Image Count
-- Added `easy textSwitch` - Text Switch
-
-**v1.1.5**
-
-- Rewrite `easy cleanGPUUsed` - the memory usage of the comfyUI can to be cleared
-- Added `easy humanSegmentation` - Human Part Segmentation
-- Added `easy imageColorMatch`
-- Added `easy ipadapterApplyRegional`
-- Added `easy ipadapterApplyFromParams`
-- Added `easy imageInterrogator` - Image To Prompt
-- Added `easy stableDiffusion3API` - Easy Stable Diffusion 3 Multiple accounts API Node
-
-**v1.1.4**
-
-- Added `easy preSamplingCustom` - Custom-PreSampling, can be supported cosXL-edit
-- Added `easy ipadapterStyleComposition`
-- Added the right-click menu to view checkpoints and lora information in all Loaders
-- Fixed `easy preSamplingNoiseIn`、`easy latentNoisy`、`east Unsampler` compatible with ComfyUI Revision>=2098 [0542088e] or later
- 
-
-**v1.1.3**
-
-- `easy ipadapterApply` Added **COMPOSITION** preset
-- Supported [ResAdapter](https://huggingface.co/jiaxiangc/res-adapter) when load ResAdapter lora
-- Added `easy promptLine`
-- Added `easy promptReplace`
-- Added `easy promptConcat`
-- `easy wildcards` Added **multiline_mode**  
-
-**v1.1.2**
-
-- Optimized some of the recommended nodes for slots related to EasyUse
-- Added **Enable ContextMenu Auto Nest Subdirectories** The setting item is enabled by default, and it can be classified into subdirectories, checkpoints and loras previews
-- Added `easy sv3dLoader` 
-- Added `easy dynamiCrafterLoader` 
-- Added `easy ipadapterApply`
-- Added `easy ipadapterApplyADV`
-- Added `easy ipadapterApplyEncoder`
-- Added `easy ipadapterApplyEmbeds`
-- Added `easy preMaskDetailerFix`
-- Fixed `easy stylesSelector` is change the prompt when not select the style
-- Fixed `easy pipeEdit` error when add lora to prompt
-- Fixed layerDiffuse xyplot bug
-- `easy kSamplerInpainting` add *additional* widget，you can choose 'Differential Diffusion' or 'Only InpaintModelConditioning'
-
-**v1.1.1**
-
-- The issue that the seed is 0 when a node with a seed control is added and **control before generate** is fixed for the first time run queue prompt.
-- `easy preSamplingAdvanced` Added **return_with_leftover_noise**
-- Fixed `easy stylesSelector` error when choose the custom file
-- `easy preSamplingLayerDiffusion` Added optional input parameter for mask
-- Renamed all nodes widget name named seed_num to seed
-- Remove forced **control_before_generate** settings。 If you want to use control_before_generate, change widget_value_control_mode to before in system settings
-- Added `easy imageRemBg` - The default is BriaAI's RMBG-1.4 model, which removes the background effect more and faster
-
-**v1.1.0**
-
-- Added `easy imageSplitList` - to split every N images
-- Added `easy preSamplingDiffusionADDTL` - It can modify foreground、background or blended additional prompt
-- Added `easy preSamplingNoiseIn` It can replace the `easy latentNoisy` node that needs to be fronted to achieve better noise injection
-- `easy pipeEdit` Added conditioning splicing mode selection, you can choose to replace, concat, combine, average, and set timestep range
-- Added `easy pipeEdit` - nodes that can edit pipes (including re-enterable prompts)
-- Added `easy preSamplingLayerDiffusion` and `easy kSamplerLayerDiffusion`
-- Added a convenient menu to right-click on nodes such as Loader, Presampler, Sampler, Controlnet, etc. to quickly replace nodes of the same type
-- Added `easy instantIDApplyADV` can link positive and negative
-- Fixed layerDiffusion error when batch size greater than 1
-- Fixed `easy wildcards` When LoRa is not filled in completely, LoRa is not automatically retrieved, resulting in failure to load LoRa
-- Fixed the issue that 'BREAK' non-initiation when didn't use a1111 prompt style
-- Fixed `easy instantIDApply` mask not input right
-
-
-<details>
-<summary><b>v1.0.9</b></summary>
-
-- Fixed the error when ComfyUI-Impack-Pack and ComfyUI_InstantID were not installed
-- Fixed `easy pipeIn`
-- Added `easy instantIDApply` - you need installed [ComfyUI_InstantID](https://github.com/cubiq/ComfyUI_InstantID) fisrt, Workflow[Example](https://github.com/yolain/ComfyUI-Easy-Use/blob/main/README.en.md#InstantID)
-- Fixed `easy detailerFix` not added to the list of nodes available for saving images formatting extensions
-- Fixed `easy XYInputs: PromptSR` errors are reported when replacing negative prompts
-</details>
-
-<details>
-<summary><b>v1.0.8</b></summary>
-
-- `easy cascadeLoader` stage_c and stage_b support the checkpoint model (Download [checkpoints](https://huggingface.co/stabilityai/stable-cascade/tree/main/comfyui_checkpoints) models) 
-- `easy styleSelector` The search box is modified to be case-insensitive
-- `easy fullLoader` **positive**、**negative**、**latent** added to the output items
-- Fixed the issue that 'easy preSampling' and other similar node, latent could not be generated based on the batch index after passing in
-- Fixed `easy svdLoader` error when the positive or negative is empty
-- Fixed the error of SDXLClipModel in ComfyUI revision 2016[c2cb8e88] and above (the revision number was judged to be compatible with the old revision)
-- Fixed `easy detailerFix` generation error when batch size is greater than 1
-- Optimize the code, reduce a lot of redundant code and improve the running speed
-</details>
-
-<details>
-<summary><b>v1.0.7</b></summary>
-
-- Added `easy cascadeLoader` - stable cascade Loader
-- Added `easy preSamplingCascade` - stable cascade preSampling Settings
-- Added `easy fullCascadeKSampler` - stable cascade stage-c ksampler full
-- Added `easy cascadeKSampler` - stable cascade stage-c ksampler simple
-- 
-- Optimize the image to image[Example](https://github.com/yolain/ComfyUI-Easy-Use/blob/main/README.en.md#image-to-image)
-</details>
-
-<details>
-<summary><b>v1.0.6</b></summary>
-
-- Added `easy XYInputs: Checkpoint`
-- Added `easy XYInputs: Lora`
-- `easy seed` can manually switch the random seed when increasing the fixed seed value
-- Fixed `easy fullLoader` and all loaders to automatically adjust the node size when switching LoRa
-- Removed the original ttn image saving logic and adapted to the default image saving format extension of ComfyUI
-</details>
-
-<details>
-<summary><b>v1.0.5</b></summary>
-
-- Added `easy isSDXL` 
-- Added prompt word control on `easy svdLoader`, which can be used with open_clip model
-- Added **populated_text** on `easy wildcards`, wildcard populated text can be output
-</details>
-
-<details>
-<summary><b>v1.0.4</b></summary>
-
-- `easy showAnything` added support for converting other types (e.g., tensor conditions, images, etc.)
-- Added `easy showLoaderSettingsNames` can display the model and VAE name in the output loader assembly
-- Added `easy promptList`
-- Added `easy fooocusInpaintLoader` （only the process of SDXLModel is supported）
-- Added **Logic** nodes
-- Added `easy imageSave` - Image saving node with date conversion and aspect and height formatting
-- Added `easy joinImageBatch`
-- `easy kSamplerInpainting` Added the **patch** input value to be used with the FooocusInpaintLoader node
-
-- Fixed xyplot error when with Pillow>9.5
-- Fixed `easy wildcards` An error is reported when running with the PS extension
-- Fixed `easy XYInputs: ControlNet` Error
-- Fixed `easy loraStack` error when **toggle** is disabled
-
-
-- Changing the first-time install node package no longer automatically replaces the theme, you need to manually adjust and refresh the page
-- `easy imageSave` added **only_preivew**
-- Adjust the `easy latentCompositeMaskedWithCond` node
-</details>
-
-<details>
-<summary><b>v1.0.3</b></summary>
-
-- Added `easy stylesSelector`
-- Added **scale_soft_weights** in `easy controlnetLoader` and `easy controlnetLoaderADV` 
-- Added the queue progress bar setting item, which is not enabled by default
-
-
-- Fixed `easy XYInputs: Sampler/Scheduler` Error
-- Fixed the right menu has a problem when clicking the button
-- Fixed `easy comfyLoader` error
-- Fixed xyPlot error when connecting to zero123
-- Fixed the error message in the loader when the prompt word was component
-- Fixed `easy getNode` and `easy setNode` the title does not change when loading
-- Fixed all samplers using subdirectories to store images
-
-
-- Adjust the UI theme, divided into two sets of styles: the official default background and the dark black background, which can be switched in the color palette in the settings
-- Modify the styles path to be compatible with other environments
-</details>
-
-<details>
-<summary><b>v1.0.2</b></summary>
-
-- Added `easy XYPlotAdvanced` and some nodes about `easy XYInputs`
-- Added **Alt+1-Alt+9** Shortcut keys to quickly paste node presets for Node templates (corresponding to 1~9 sequences)
-- Added a `📜Groups Map(EasyUse)` to the context menu.
-- An `autocomplete` folder has been added, If you have [ComfyUI-Custom-Scripts](https://github.com/pythongosssss/ComfyUI-Custom-Scripts) installed, the txt files in that folder will be merged and overwritten to the autocomplete .txt file of the pyssss package at startup.
-
-
-- Fixed XYPlot is not working when `a1111_prompt_style` is True
-- Fixed UI loading failure in the new version of ComfyUI
-- `easy XYInputs ModelMergeBlocks` Values can be imported from CSV files
-- Fixed `easy pipeToBasicPipe` Bug
-
-
-- Removed `easy imageRemBg`
-- Remove the introductory diagram and workflow files from the package to reduce the package size
-- Replaced the font file used in the generation of XY diagrams
-</details>
-
-<details>
-<summary><b>v1.0.1</b></summary>
-
-- Fixed `easy comfyLoader` error
-- Fixed All nodes that contain the value of the image size
-- Added `easy kSamplerInpainting`
-- Added `easy pipeToBasicPipe`
-- Fixed `width` and `height` can not customize in `easy svdLoader`
-- Fixed all preview image path (Previously, it was not possible to preview the image on the Mac system)
-- Fixed `vae_name` is not working in `easy fullLoader` and `easy a1111Loader` and `easy comfyLoader`
-- Fixed `easy fullkSampler` outputs error
-- Fixed `model_override` is not working in `easy fullLoader`
-- Fixed `easy hiresFix` error
-- Fixed `easy xyplot` font file path error
-- Fixed seed that cannot be fixed when you convert `seed_num` to `easy seed` 
-- Fixed `easy pipeIn` inputs bug
-- `easy preDetailerFix` have added a new parameter `optional_image`
-- Fixed `easy zero123Loader` and `easy svdLoader` model into cache.
-- Added `easy seed`
-- Fixed `image_output` default value is "Preview"
-- `easy fullLoader` and `easy a1111Loader` have added a new parameter `a1111_prompt_style`,that can reproduce the same image generated from stable-diffusion-webui on comfyui, but you need to install [ComfyUI_smZNodes](https://github.com/shiimizu/ComfyUI_smZNodes) to use this feature in the current version
-</details>
-
-<details>
-<summary><b>v1.0.0</b></summary>
-
-- Added `easy positive` - simple positive prompt text
-- Added `easy negative` - simple negative prompt text
-- Added `easy wildcards` - support for wildcards and hint text selected by Lora
-- Added `easy portraitMaster` - PortraitMaster v2.2
-- Added `easy loraStack` - Lora stack
-- Added `easy fullLoader` - full version of the loader
-- Added `easy zero123Loader` - simple zero123 loader
-- Added `easy svdLoader` - easy svd loader
-- Added `easy fullkSampler` - full version of the sampler (no separation)
-- Added `easy hiresFix` - support for HD repair of Pipe
-- Added `easy predetailerFix` and `easy DetailerFix` - support for Pipe detail fixing
-- Added `easy ultralyticsDetectorPipe` and `easy samLoaderPipe` - Detect loader (detail fixed input)
-- Added `easy pipein` `easy pipeout` - Pipe input and output
-- Added `easy xyPlot` - simple xyplot (more controllable parameters will be updated in the future)
-- Added `easy imageRemoveBG` - image to remove background
-- Added `easy imagePixelPerfect` - image pixel perfect
-- Added `easy poseEditor` - Pose editor
-- New UI Theme (Obsidian) - Auto-load UI by default, which can also be changed in the settings 
-
-- Fixed `easy globalSeed` is not working
-- Fixed an issue where all `seed_num` values were out of order due to [cg-use-everywhere](https://github.com/chrisgoringe/cg-use-everywhere) updating the chart in real time
-- Fixed `easy imageSize`, `easy imageSizeBySide`, `easy imageSizeByLongerSide` as end nodes
-- Fixed the bug that `seed_num` (random seed value) could not be read consistently in history
-</details>
-
-<details>
-<summary><b>Updated at 12/14/2023</b></summary>
-
-- `easy a1111Loader` and `easy comfyLoader` added `batch_size` of required input parameters 
-- Added the `easy controlnetLoaderADV` node
-- `easy controlnetLoaderADV` and `easy controlnetLoader` added `control_net ` of optional input parameters
-- `easy preSampling` and `easy preSamplingAdvanced` added `image_to_latent` optional input parameters
-- Added the `easy imageSizeBySide` node, which can be output as a long side or a short side
-</details>
-
-<details>
-<summary><b>Updated at 12/13/2023</b></summary>
-
--  Added the `easy LLLiteLoader` node, if you have pre-installed the kohya-ss/ControlNet-LLLite-ComfyUI package, please move the model files in the models to `ComfyUI\models\controlnet\` (i.e. in the default controlnet path of comfy, please do not change the file name of the model, otherwise it will not be read).
--  Modify `easy controlnetLoader` to the bottom of the loader category.
--  Added size display for `easy imageSize` and `easy imageSizeByLongerSize` outputs.
-</details>
-
-<details>
-<summary><b>Updated at 12/11/2023</b></summary>
--  Added the `showSpentTime` node to display the time spent on image diffusion and the time spent on VAE decoding images
-</details>
-
-## The relevant node package involved
-
-Disclaimer: Opened source was not easy. I have a lot of respect for the contributions of these original authors. I just did some integration and optimization.
-
-| Nodes Name(Search Name)        | Related libraries                                                                        | Library-related node     |
-|:-------------------------------|:----------------------------------------------------------------------------|:-------------------------|
-| easy setNode                   | [ComfyUI-extensions](https://github.com/diffus3/ComfyUI-extensions) | diffus3.SetNode          |
-| easy getNode                   | [ComfyUI-extensions](https://github.com/diffus3/ComfyUI-extensions) | diffus3.GetNode          |
-| easy bookmark                  | [rgthree-comfy](https://github.com/rgthree/rgthree-comfy) | Bookmark 🔖              |
-| easy portraitMarker            | [comfyui-portrait-master](https://github.com/florestefano1975/comfyui-portrait-master) | Portrait Master          |
-| easy LLLiteLoader              | [ControlNet-LLLite-ComfyUI](https://github.com/kohya-ss/ControlNet-LLLite-ComfyUI) | LLLiteLoader             |
-| easy globalSeed                | [ComfyUI-Inspire-Pack](https://github.com/ltdrdata/ComfyUI-Inspire-Pack) | Global Seed (Inspire)    | 
-| easy preSamplingDynamicCFG     | [sd-dynamic-thresholding](https://github.com/mcmonkeyprojects/sd-dynamic-thresholding) | DynamicThresholdingFull  | 
-| dynamicThresholdingFull        | [sd-dynamic-thresholding](https://github.com/mcmonkeyprojects/sd-dynamic-thresholding) | DynamicThresholdingFull  | 
-| easy imageInsetCrop            | [rgthree-comfy](https://github.com/rgthree/rgthree-comfy) | ImageInsetCrop           | 
-| easy poseEditor                | [ComfyUI_Custom_Nodes_AlekPet](https://github.com/AlekPet/ComfyUI_Custom_Nodes_AlekPet) | poseNode                 | 
-| easy preSamplingLayerDiffusion | [ComfyUI-layerdiffusion](https://github.com/huchenlei/ComfyUI-layerdiffusion) | LayeredDiffusionApply... | 
-| easy dynamiCrafterLoader       | [ComfyUI-layerdiffusion](https://github.com/ExponentialML/ComfyUI_Native_DynamiCrafter) | Apply Dynamicrafter    | 
-| easy imageChooser              | [cg-image-picker](https://github.com/chrisgoringe/cg-image-picker) | Preview Chooser         | 
-| easy styleAlignedBatchAlign              | [style_aligned_comfy](https://github.com/chrisgoringe/cg-image-picker) | styleAlignedBatchAlign       | 
-
-
-## Credits
-
-[ComfyUI](https://github.com/comfyanonymous/ComfyUI) - Powerful and modular Stable Diffusion GUI
-
-[ComfyUI-ComfyUI-Manager](https://github.com/ltdrdata/ComfyUI-Manager) - ComfyUI Manager
-
-[tinyterraNodes](https://github.com/TinyTerra/ComfyUI_tinyterraNodes) - Pipe nodes (node bundles) allow users to reduce unnecessary connections
-
-[ComfyUI-extensions](https://github.com/diffus3/ComfyUI-extensions) - Diffus3 gets and sets points that allow the user to detach the composition of the workflow 
-
-[ComfyUI-Impact-Pack](https://github.com/ltdrdata/ComfyUI-Impact-Pack) - General modpack 1
-
-[ComfyUI-Inspire-Pack](https://github.com/ltdrdata/ComfyUI-Inspire-Pack) - General Modpack 2
-
-[ComfyUI-ResAdapter](https://github.com/jiaxiangc/ComfyUI-ResAdapter) - Make model generation independent of training resolution
-
-[ComfyUI_IPAdapter_plus](https://github.com/cubiq/ComfyUI_IPAdapter_plus) - Style migration
-
-[ComfyUI_InstantID](https://github.com/cubiq/ComfyUI_InstantID) - Face migration
-
-[ComfyUI_PuLID](https://github.com/cubiq/PuLID_ComfyUI) - Face migration
-
-[ComfyUI-Custom-Scripts](https://github.com/pythongosssss/ComfyUI-Custom-Scripts) - pyssss🐍
-
-[cg-image-picker](https://github.com/chrisgoringe/cg-image-picker) - Image Preview Chooser
-
-[ComfyUI_ExtraModels](https://github.com/city96/ComfyUI_ExtraModels) - DiT custom nodes
-
-
-##  🌟Stargazers
-
-My gratitude extends to the generous souls who bestow a star. Your support is much appreciated!
-
-[![Stargazers repo roster for @yolain/ComfyUI-Easy-Use](https://reporoster.com/stars/yolain/ComfyUI-Easy-Use)](https://github.com/yolain/ComfyUI-Easy-Use/stargazers)

README.md

@@ -1,414 +1,458 @@
-<p align="right">
-   <strong>中文</strong> | <a href="./README.en.md">English</a>
-</p>
+![comfyui-easy-use](https://github.com/user-attachments/assets/9b7a5e44-f5e2-4c27-aed2-d0e6b50c46bb)
 
 <div align="center">
-
-# ComfyUI Easy Use
-
-[![Bilibili Badge](https://img.shields.io/badge/1.1版本-00A1D6?style=for-the-badge&logo=bilibili&logoColor=white&link=https://www.bilibili.com/video/BV1w6421F7Uv)](https://www.bilibili.com/video/BV1w6421F7Uv)
-[![Bilibili Badge](https://img.shields.io/badge/基本介绍-00A1D6?style=for-the-badge&logo=bilibili&logoColor=white&link=https://www.bilibili.com/video/BV1vQ4y1G7z7)](https://www.bilibili.com/video/BV1vQ4y1G7z7/)
+<a href="https://space.bilibili.com/1840885116">Video Tutorial</a> |
+Docs (Cooming Soon) | 
+<a href="https://github.com/yolain/ComfyUI-Yolain-Workflows">Workflow Collection</a> |
+<a href="#%EF%B8%8F-donation">Donation</a> 
+<br><br>
+<a href="./README.md"><img src="https://img.shields.io/badge/🇬🇧English-0b8cf5"></a>
+<a href="./README.ZH_CN.md"><img src="https://img.shields.io/badge/🇨🇳中文简体-e9e9e9"></a>
 </div>
 
-**ComfyUI-Easy-Use** 是一个化繁为简的节点整合包, 在 [tinyterraNodes](https://github.com/TinyTerra/ComfyUI_tinyterraNodes) 的基础上进行延展，并针对了诸多主流的节点包做了整合与优化，以达到更快更方便使用ComfyUI的目的，在保证自由度的同时还原了本属于Stable Diffusion的极致畅快出图体验。
+**ComfyUI-Easy-Use** is an efficiency custom nodes integration package, which is extended on the basis of [TinyTerraNodes](https://github.com/TinyTerra/ComfyUI_tinyterraNodes). It has been integrated and optimized for many popular awesome custom nodes to achieve the purpose of faster and more convenient use of ComfyUI. While ensuring the degree of freedom, it restores the ultimate smooth image production experience that belongs to Stable Diffusion.
 
-[![ComfyUI-Yolain-Workflows](https://github.com/yolain/ComfyUI-Easy-Use/assets/73304135/9a3f54bc-a677-4bf1-a196-8845dd57c942)](https://github.com/yolain/ComfyUI-Yolain-Workflows)
+## 👨🏻‍🎨 Introduce
 
-## 特色介绍
+- Inspire by [tinyterraNodes](https://github.com/TinyTerra/ComfyUI_tinyterraNodes), which greatly reduces the time cost of tossing workflows。
+- UI interface beautification, the first time you install the user, if you need to use the UI theme, please switch the theme in Settings -> Color Palette and refresh page.
+- Added a node for pre-sampling parameter configuration, which can be separated from the sampling node for easier previewing
+- Wildcards and lora's are supported, for Lora Block Weight usage, ensure that the custom node package has the [ComfyUI-Inspire-Pack](https://github.com/ltdrdata/ComfyUI-Inspire-Pack)
+- Multi-selectable styled cue word selector, default is Fooocus style json, custom json can be placed under styles, samples folder can be placed in the preview image (name and name consistent, image file name such as spaces need to be converted to underscores '_')
+- The loader enables the A1111 prompt mode, which reproduces nearly identical images to those generated by webui, and needs to be installed [ComfyUI_smZNodes](https://github.com/shiimizu/ComfyUI_smZNodes) first. 
+- Noise injection into the latent space can be achieved using the `easy latentNoisy` or `easy preSamplingNoiseIn` node
+- Simplified processes for SD1.x, SD2.x, SDXL, SVD, Zero123, etc. [Example](https://github.com/yolain/ComfyUI-Easy-Use?tab=readme-ov-file#StableDiffusion)
+- Simplified Stable Cascade [Example](https://github.com/yolain/ComfyUI-Easy-Use?tab=readme-ov-file#StableCascade)
+- Simplified Layer Diffuse [Example](https://github.com/yolain/ComfyUI-Easy-Use?tab=readme-ov-file#LayerDiffusion)，The first time you use it you may need to run `pip install -r requirements.txt` to install the required dependencies.
+- Simplified InstantID [Example](https://github.com/yolain/ComfyUI-Easy-Use?tab=readme-ov-file#InstantID), You need to make sure that the custom node package has the [ComfyUI_InstantID](https://github.com/cubiq/ComfyUI_InstantID)
+- Extending the usability of XYplot
+- Fooocus Inpaint integration
+- Integration of common logical calculations, conversion of types, display of all types, etc.
+- Background removal nodes for the RMBG-1.4 model supporting BriaAI, [BriaAI Guide](https://huggingface.co/briaai/RMBG-1.4)
+- Forcibly cleared the memory usage of the comfy UI model are supported
+- Stable Diffusion 3 multi-account API nodes are supported
+- Support SD3's model
+- Support Kolors‘s model
+- Support Flux's model
+- Support lazy if else and for loops
 
-- 沿用了 [tinyterraNodes](https://github.com/TinyTerra/ComfyUI_tinyterraNodes) 的思路，大大减少了折腾工作流的时间成本。
-- UI界面美化，首次安装的用户，如需使用UI主题，请在 Settings -> Color Palette 中自行切换主题并**刷新页面**即可
-- 增加了预采样参数配置的节点，可与采样节点分离，更方便预览。
-- 支持通配符与Lora的提示词节点，如需使用Lora Block Weight用法，需先保证自定义节点包中安装了 [ComfyUI-Inspire-Pack](https://github.com/ltdrdata/ComfyUI-Inspire-Pack)
-- 可多选的风格化提示词选择器，默认是Fooocus的样式json，可自定义json放在styles底下，samples文件夹里可放预览图(名称和name一致,图片文件名如有空格需转为下划线'_')
-- 加载器可开启A1111提示词风格模式，可重现与webui生成近乎相同的图像，需先安装 [ComfyUI_smZNodes](https://github.com/shiimizu/ComfyUI_smZNodes)
-- 可使用`easy latentNoisy`或`easy preSamplingNoiseIn`节点实现对潜空间的噪声注入
-- 简化 SD1.x、SD2.x、SDXL、SVD、Zero123等流程 
-- 简化 Stable Cascade [示例参考](https://github.com/yolain/ComfyUI-Yolain-Workflows?tab=readme-ov-file#1-13-stable-cascade)
-- 简化 Layer Diffuse [示例参考](https://github.com/yolain/ComfyUI-Yolain-Workflows?tab=readme-ov-file#2-3-layerdiffusion)
-- 简化 InstantID [示例参考](https://github.com/yolain/ComfyUI-Yolain-Workflows?tab=readme-ov-file#2-2-instantid), 需先保证自定义节点包中安装了 [ComfyUI_InstantID](https://github.com/cubiq/ComfyUI_InstantID)
-- 简化 IPAdapter, 需先保证自定义节点包中安装最新版v2的 [ComfyUI_IPAdapter_plus](https://github.com/cubiq/ComfyUI_IPAdapter_plus)
-- 扩展 XYplot 的可用性
-- 整合了Fooocus Inpaint功能
-- 整合了常用的逻辑计算、转换类型、展示所有类型等
-- 支持节点上checkpoint、lora模型子目录分类及预览图 (请在设置中开启上下文菜单嵌套子目录)
-- 支持BriaAI的RMBG-1.4模型的背景去除节点，[技术参考](https://huggingface.co/briaai/RMBG-1.4)
-- 支持 强制清理comfyUI模型显存占用
-- 支持Stable Diffusion 3 多账号API节点
-- 支持IC-Light的应用 [示例参考](https://github.com/yolain/ComfyUI-Yolain-Workflows?tab=readme-ov-file#2-5-ic-light) | [代码整合来源](https://github.com/huchenlei/ComfyUI-IC-Light) | [技术参考](https://github.com/lllyasviel/IC-Light)
-- 中文提示词自动识别，使用[opus-mt-zh-en模型](https://huggingface.co/Helsinki-NLP/opus-mt-zh-en)
-- 支持 sd3 模型 
-
-## 安装
-将存储库克隆到 **custom_nodes** 目录并安装依赖
+## 👨🏻‍🔧 Installation
+Clone the repo into the **custom_nodes** directory and install the requirements:
 ```shell
-#1. git下载
+#1. Clone the repo
 git clone https://github.com/yolain/ComfyUI-Easy-Use
-#2. 安装依赖
-双击install.bat安装依赖
+#2. Install the requirements
+Double-click install.bat to install the required dependencies
 ```
 
-## 更新日志
+## 👨🏻‍🚀 Plan
+
+- [x] Updated new front-end code for easier maintenance
+  - [x] Maintain css styles using sass
+  - [x] Optimize existing extensions
+  - [x] Add new components
+- [ ] Upload new workflows to [ComfyUI-Yolain-Workflows](https://github.com/yolain/ComfyUI-Yolain-Workflows) and translate readme to english version.
+- [ ] Write gitbook with more detailed function introdution
+
+## 📜 Changelog
+
+**v1.2.3**
+
+- `easy showAnything` and `easy cleanGPUUsed` added slot of output
+- Added human parts segmentation to `easy humanSegmentation` - Code based on (ComfyUI_Human_Parts)[https://github.com/metal3d/ComfyUI_Human_Parts]
+- Using FluxGuidance when you are using a flux model and choose basicGuider and set the cfg>0 on `easy preSamplingCustom`
+- Added `easy loraStackApply` and `easy controlnetStackApply` - Apply loraStack and controlnetStack
+
+**v1.2.2**
+
+- Added `easy batchAny`
+- Added `easy anythingIndexSwitch`
+- Added `easy forLoopStart` and `easy forLoopEnd`  
+- Added `easy ifElse`  
+- Added v2 web frond-end code
+- Added `easy fluxLoader`
+- Added support for `controlnetApply` Related nodes with SD3 and hunyuanDiT
+- Fixed after using `easy applyFooocusInpaint`, all lora models become unusable
+
+**v1.2.1**
+
+- Added `easy ipadapterApplyFaceIDKolors`
+- Added **inspyrenet** to `easy imageRemBg` 
+- Added `easy controlnetLoader++`
+- Added **PLUS (kolors genernal)** and **FACEID PLUS KOLORS** preset to `easy ipadapterApply` and `easy ipadapterApplyADV` (Supported kolors ipadapter）
+- Added `easy kolorsLoader` - Code based on [MinusZoneAI](https://github.com/MinusZoneAI/ComfyUI-Kolors-MZ)'s and [kijai](https://github.com/kijai/ComfyUI-KwaiKolorsWrapper)'s repo, thanks for their contribution.
 
 **v1.2.0**
 
-- 增加 `easy pulIDApply` 和 `easy pulIDApplyADV`
-- 增加 `easy huanyuanDiTLoader` 和 `easy pixArtLoader`
-- 当新菜单的位置在上或者下时增加上 crystools 的显示，推荐开两个就好（如果后续crystools有更新UI适配我可能会删除掉）
-- 增加 **easy sliderControl** - 滑块控制节点，当前可用于控制ipadapterMS的参数 (双击滑块可重置为默认值)
-- 增加 **layer_weights** 属性在 `easy ipadapterApplyADV` 节点
+- Added `easy pulIDApply` and `easy pulIDApplyADV`
+- Added `easy huanyuanDiTLoader` and `easy pixArtLoader`
+- Added **easy sliderControl** - Slider control node, which can currently be used to control the parameters of ipadapterMS (double-click the slider to reset to default)
+- Added **layer_weights** in `easy ipadapterApplyADV`
 
 **v1.1.9**
 
-- 增加 新的调度器 **gitsScheduler**
-- 增加 `easy imageBatchToImageList` 和 `easy imageListToImageBatch` (修复Impact版的一点小问题)
-- 递归模型子目录嵌套
-- 支持 sd3 模型 
-- 增加 `easy applyInpaint` - 局部重绘全模式节点 (相比与之前的kSamplerInpating节点逻辑会更合理些)
+- Added **gitsScheduler**
+- Added `easy imageBatchToImageList` and `easy imageListToImageBatch` 
+- Recursive subcategories nested for models
+- Support for Stable Diffusion 3 model
+- Added `easy applyInpaint` - All inpainting mode in this node
 
 **v1.1.8**
 
-- 增加中文提示词自动翻译，使用[opus-mt-zh-en模型](https://huggingface.co/Helsinki-NLP/opus-mt-zh-en), 默认已对wildcard、lora正则处理, 其他需要保留的中文，可使用`@你的提示词@`包裹 (若依赖安装完成后报错, 请重启)，测算大约会占0.3GB显存
-- 增加 `easy controlnetStack` - controlnet堆
-- 增加 `easy applyBrushNet` - [示例参考](https://github.com/yolain/ComfyUI-Yolain-Workflows/blob/main/workflows/2_advanced/2-4inpainting/2-4brushnet_1.1.8.json)
-- 增加 `easy applyPowerPaint` - [示例参考](https://github.com/yolain/ComfyUI-Yolain-Workflows/blob/main/workflows/2_advanced/2-4inpainting/2-4powerpaint_outpaint_1.1.8.json)
+- Added `easy controlnetStack`
+- Added `easy applyBrushNet` - [Workflow Example](https://github.com/yolain/ComfyUI-Yolain-Workflows/blob/main/workflows/2_advanced/2-4inpainting/2-4brushnet_1.1.8.json)
+- Added `easy applyPowerPaint` - [Workflow Example](https://github.com/yolain/ComfyUI-Yolain-Workflows/blob/main/workflows/2_advanced/2-4inpainting/2-4powerpaint_outpaint_1.1.8.json)
 
 **v1.1.7**
 
-- 修复 一些模型(如controlnet模型等)未成功写入缓存，导致修改前置节点束参数（如提示词）需要二次载入模型的问题
-- 增加 `easy prompt` - 主体和光影预置项，后期可能会调整
-- 增加 `easy icLightApply` - 重绘光影, 从[ComfyUI-IC-Light](https://github.com/huchenlei/ComfyUI-IC-Light)优化
-- 增加 `easy imageSplitGrid` - 图像网格拆分
-- `easy kSamplerInpainting` 的 **additional** 属性增加差异扩散和brushnet等相关选项 
-- 增加 brushnet模型加载的支持 - [ComfyUI-BrushNet](https://github.com/nullquant/ComfyUI-BrushNet)
-- 增加 `easy applyFooocusInpaint` - Fooocus内补节点 替代原有的 FooocusInpaintLoader
-- 移除 `easy fooocusInpaintLoader` - 容易bug，不再使用
-- 修改 easy kSampler等采样器中并联的model 不再替换输出中pipe里的model
+- Added `easy prompt` - Subject and light presets, maybe adjusted later
+- Added `easy icLightApply` - Light and shadow migration, Code based on [ComfyUI-IC-Light](https://github.com/huchenlei/ComfyUI-IC-Light)
+- Added `easy imageSplitGrid`
+- `easy kSamplerInpainting` added options such as different diffusion and brushnet in **additional** widget
+- Support for brushnet model loading - [ComfyUI-BrushNet](https://github.com/nullquant/ComfyUI-BrushNet)
+- Added `easy applyFooocusInpaint` - Replace FooocusInpaintLoader
+- Removed `easy fooocusInpaintLoader`
 
 **v1.1.6**
 
-- 增加步调齐整适配 - 在所有的预采样和全采样器节点中的 调度器(schedulder) 增加了 **alignYourSteps** 选项
-- `easy kSampler` 和 `easy fullkSampler` 的 **image_output** 增加 **Preview&Choose**选项
-- 增加 `easy styleAlignedBatchAlign` - 风格对齐 [style_aligned_comfy](https://github.com/brianfitzgerald/style_aligned_comfy)
-- 增加 `easy ckptNames`
-- 增加 `easy controlnetNames`
-- 增加 `easy imagesSplitimage` - 批次图像拆分单张
-- 增加 `easy imageCount` - 图像数量
-- 增加 `easy textSwitch` - 文字切换
+- Added **alignYourSteps** to **schedulder** widget in all `easy preSampling` and `easy fullkSampler`
+- Added **Preview&Choose** to **image_output** widget in `easy kSampler` & `easy fullkSampler`
+- Added `easy styleAlignedBatchAlign` - Credit of [style_aligned_comfy](https://github.com/brianfitzgerald/style_aligned_comfy)
+- Added `easy ckptNames`
+- Added `easy controlnetNames`
+- Added `easy imagesSplitimage` - Batch images split into single images
+- Added `easy imageCount` - Get Image Count
+- Added `easy textSwitch` - Text Switch
 
 **v1.1.5**
 
-- 重写 `easy cleanGPUUsed` - 可强制清理comfyUI的模型显存占用
-- 增加 `easy humanSegmentation` - 多类分割、人像分割
-- 增加 `easy imageColorMatch`
-- 增加 `easy ipadapterApplyRegional`
-- 增加 `easy ipadapterApplyFromParams`
-- 增加 `easy imageInterrogator` - 图像反推
-- 增加 `easy stableDiffusion3API` - 简易的Stable Diffusion 3 多账号API节点
+- Rewrite `easy cleanGPUUsed` - the memory usage of the comfyUI can to be cleared
+- Added `easy humanSegmentation` - Human Part Segmentation
+- Added `easy imageColorMatch`
+- Added `easy ipadapterApplyRegional`
+- Added `easy ipadapterApplyFromParams`
+- Added `easy imageInterrogator` - Image To Prompt
+- Added `easy stableDiffusion3API` - Easy Stable Diffusion 3 Multiple accounts API Node
 
 **v1.1.4**
 
-- 增加 `easy imageChooser` - 从[cg-image-picker](https://github.com/chrisgoringe/cg-image-picker)简化的图片选择器
-- 增加 `easy preSamplingCustom` - 自定义预采样，可支持cosXL-edit
-- 增加 `easy ipadapterStyleComposition`
-- 增加 在Loaders上右键菜单可查看 checkpoints、lora 信息
-- 修复 `easy preSamplingNoiseIn`、`easy latentNoisy`、`east Unsampler` 以兼容ComfyUI Revision>=2098 [0542088e] 以上版本
-- 修复 FooocusInpaint修改ModelPatcher计算权重引发的问题，理应在生成model后重置ModelPatcher为默认值
+- Added `easy preSamplingCustom` - Custom-PreSampling, can be supported cosXL-edit
+- Added `easy ipadapterStyleComposition`
+- Added the right-click menu to view checkpoints and lora information in all Loaders
+- Fixed `easy preSamplingNoiseIn`、`easy latentNoisy`、`east Unsampler` compatible with ComfyUI Revision>=2098 [0542088e] or later
+ 
 
 **v1.1.3**
 
-- `easy ipadapterApply` 增加 **COMPOSITION** 预置项
-- 增加 对[ResAdapter](https://huggingface.co/jiaxiangc/res-adapter) lora模型 的加载支持
-- 增加 `easy promptLine`
-- 增加 `easy promptReplace`
-- 增加 `easy promptConcat`
-- `easy wildcards` 增加 **multiline_mode**属性 
-- 增加 当节点需要下载模型时，若huggingface连接超时，会切换至镜像地址下载模型
+- `easy ipadapterApply` Added **COMPOSITION** preset
+- Supported [ResAdapter](https://huggingface.co/jiaxiangc/res-adapter) when load ResAdapter lora
+- Added `easy promptLine`
+- Added `easy promptReplace`
+- Added `easy promptConcat`
+- `easy wildcards` Added **multiline_mode**  
 
-**v1.1.2**
+<details>
+<summary><b>v1.1.2</b></summary>
 
-- 改写 EasyUse 相关节点的部分插槽推荐节点
-- 增加 **启用上下文菜单自动嵌套子目录** 设置项，默认为启用状态，可分类子目录及checkpoints、loras预览图
-- 增加 `easy sv3dLoader` 
-- 增加 `easy dynamiCrafterLoader` 
-- 增加 `easy ipadapterApply`
-- 增加 `easy ipadapterApplyADV`
-- 增加 `easy ipadapterApplyEncoder`
-- 增加 `easy ipadapterApplyEmbeds`
-- 增加 `easy preMaskDetailerFix`
-- `easy kSamplerInpainting` 增加 **additional** 属性，可设置成 Differential Diffusion 或 Only InpaintModelConditioning
-- 修复 `easy stylesSelector` 当未选择样式时，原有提示词发生了变化
-- 修复 `easy pipeEdit` 提示词输入lora时报错
-- 修复 layerDiffuse xyplot相关bug
+- Optimized some of the recommended nodes for slots related to EasyUse
+- Added **Enable ContextMenu Auto Nest Subdirectories** The setting item is enabled by default, and it can be classified into subdirectories, checkpoints and loras previews
+- Added `easy sv3dLoader` 
+- Added `easy dynamiCrafterLoader` 
+- Added `easy ipadapterApply`
+- Added `easy ipadapterApplyADV`
+- Added `easy ipadapterApplyEncoder`
+- Added `easy ipadapterApplyEmbeds`
+- Added `easy preMaskDetailerFix`
+- Fixed `easy stylesSelector` is change the prompt when not select the style
+- Fixed `easy pipeEdit` error when add lora to prompt
+- Fixed layerDiffuse xyplot bug
+- `easy kSamplerInpainting` add *additional* widget，you can choose 'Differential Diffusion' or 'Only InpaintModelConditioning'
+</details>
 
-**v1.1.1**
+<details>
+<summary><b>v1.1.1</b></summary>
 
-- 修复首次添加含seed的节点且当前模式为control_before_generate时，seed为0的问题
-- `easy preSamplingAdvanced` 增加 **return_with_leftover_noise**
-- 修复 `easy stylesSelector` 当选择自定义样式文件时运行队列报错
-- `easy preSamplingLayerDiffusion` 增加 mask 可选传入参数
-- 将所有 **seed_num** 调整回 **seed**
-- 修补官方BUG: 当control_mode为before 在首次加载页面时未修改节点中widget名称为 control_before_generate
-- 去除强制**control_before_generate**设定
-- 增加 `easy imageRemBg` - 默认为BriaAI的RMBG-1.4模型, 移除背景效果更加，速度更快
+- The issue that the seed is 0 when a node with a seed control is added and **control before generate** is fixed for the first time run queue prompt.
+- `easy preSamplingAdvanced` Added **return_with_leftover_noise**
+- Fixed `easy stylesSelector` error when choose the custom file
+- `easy preSamplingLayerDiffusion` Added optional input parameter for mask
+- Renamed all nodes widget name named seed_num to seed
+- Remove forced **control_before_generate** settings。 If you want to use control_before_generate, change widget_value_control_mode to before in system settings
+- Added `easy imageRemBg` - The default is BriaAI's RMBG-1.4 model, which removes the background effect more and faster
+</details>
 
-**v1.1.0**
+<details>
+<summary><b>v1.1.0</b></summary>
 
-- 增加 `easy imageSplitList` - 拆分每 N 张图像
-- 增加 `easy preSamplingDiffusionADDTL` - 可配置前景、背景、blended的additional_prompt等   
-- 增加 `easy preSamplingNoiseIn` 可替代需要前置的`easy latentNoisy`节点 实现效果更好的噪声注入
-- `easy pipeEdit` 增加 条件拼接模式选择，可选择替换、合并、联结、平均、设置条件时间
-- 增加 `easy pipeEdit` - 可编辑Pipe的节点（包含可重新输入提示词）
-- 增加 `easy preSamplingLayerDiffusion` 与 `easy kSamplerLayerDiffusion` （连接 `easy kSampler` 也能通）
-- 增加 在 加载器、预采样、采样器、Controlnet等节点上右键可快速替换同类型节点的便捷菜单
-- 增加 `easy instantIDApplyADV` 可连入 positive 与 negative
-- 修复 `easy wildcards` 读取lora未填写完整路径时未自动检索导致加载lora失败的问题
-- 修复 `easy instantIDApply` mask 未传入正确值
-- 修复 在 非a1111提示词风格下 BREAK 不生效的问题
+- Added `easy imageSplitList` - to split every N images
+- Added `easy preSamplingDiffusionADDTL` - It can modify foreground、background or blended additional prompt
+- Added `easy preSamplingNoiseIn` It can replace the `easy latentNoisy` node that needs to be fronted to achieve better noise injection
+- `easy pipeEdit` Added conditioning splicing mode selection, you can choose to replace, concat, combine, average, and set timestep range
+- Added `easy pipeEdit` - nodes that can edit pipes (including re-enterable prompts)
+- Added `easy preSamplingLayerDiffusion` and `easy kSamplerLayerDiffusion`
+- Added a convenient menu to right-click on nodes such as Loader, Presampler, Sampler, Controlnet, etc. to quickly replace nodes of the same type
+- Added `easy instantIDApplyADV` can link positive and negative
+- Fixed layerDiffusion error when batch size greater than 1
+- Fixed `easy wildcards` When LoRa is not filled in completely, LoRa is not automatically retrieved, resulting in failure to load LoRa
+- Fixed the issue that 'BREAK' non-initiation when didn't use a1111 prompt style
+- Fixed `easy instantIDApply` mask not input right
+</details>
 
 <details>
 <summary><b>v1.0.9</b></summary>
 
-- 修复未安装 ComfyUI-Impack-Pack 和 ComfyUI_InstantID 时报错
-- 修复 `easy pipeIn` - pipe设为可不必选
-- 增加 `easy instantIDApply` - 需要先安装 [ComfyUI_InstantID](https://github.com/cubiq/ComfyUI_InstantID), 工作流参考[示例](https://github.com/yolain/ComfyUI-Yolain-Workflows?tab=readme-ov-file#2-2-instantid)
-- 修复 `easy detailerFix` 未添加到保存图片格式化扩展名可用节点列表
-- 修复 `easy XYInputs: PromptSR` 在替换负面提示词时报错
+- Fixed the error when ComfyUI-Impack-Pack and ComfyUI_InstantID were not installed
+- Fixed `easy pipeIn`
+- Added `easy instantIDApply` - you need installed [ComfyUI_InstantID](https://github.com/cubiq/ComfyUI_InstantID) fisrt, Workflow[Example](https://github.com/yolain/ComfyUI-Easy-Use/blob/main/README.en.md#InstantID)
+- Fixed `easy detailerFix` not added to the list of nodes available for saving images formatting extensions
+- Fixed `easy XYInputs: PromptSR` errors are reported when replacing negative prompts
 </details>
 
 <details>
 <summary><b>v1.0.8</b></summary>
 
-- `easy cascadeLoader` stage_c 与 stage_b 支持checkpoint模型 (需要下载[checkpoints](https://huggingface.co/stabilityai/stable-cascade/tree/main/comfyui_checkpoints)) 
-- `easy styleSelector` 搜索框修改为不区分大小写匹配
-- `easy fullLoader` 增加 **positive**、**negative**、**latent** 输出项
-- 修复 SDXLClipModel 在 ComfyUI 修订版本号 2016[c2cb8e88] 及以上的报错（判断了版本号可兼容老版本）
-- 修复 `easy detailerFix` 批次大小大于1时生成出错
-- 修复`easy preSampling`等 latent传入后无法根据批次索引生成的问题
-- 修复 `easy svdLoader` 报错
-- 优化代码，减少了诸多冗余，提升运行速度
-- 去除中文翻译对照文本
-
-（翻译对照已由 [AIGODLIKE-COMFYUI-TRANSLATION](https://github.com/AIGODLIKE/AIGODLIKE-ComfyUI-Translation) 统一维护啦！
-首次下载或者版本较早的朋友请更新 AIGODLIKE-COMFYUI-TRANSLATION 和本节点包至最新版本。）
+- `easy cascadeLoader` stage_c and stage_b support the checkpoint model (Download [checkpoints](https://huggingface.co/stabilityai/stable-cascade/tree/main/comfyui_checkpoints) models) 
+- `easy styleSelector` The search box is modified to be case-insensitive
+- `easy fullLoader` **positive**、**negative**、**latent** added to the output items
+- Fixed the issue that 'easy preSampling' and other similar node, latent could not be generated based on the batch index after passing in
+- Fixed `easy svdLoader` error when the positive or negative is empty
+- Fixed the error of SDXLClipModel in ComfyUI revision 2016[c2cb8e88] and above (the revision number was judged to be compatible with the old revision)
+- Fixed `easy detailerFix` generation error when batch size is greater than 1
+- Optimize the code, reduce a lot of redundant code and improve the running speed
 </details>
 
 <details>
 <summary><b>v1.0.7</b></summary>
 
-- 增加 `easy cascadeLoader` - stable cascade 加载器
-- 增加 `easy preSamplingCascade` - stabled cascade stage_c 预采样参数
-- 增加 `easy fullCascadeKSampler` - stable cascade stage_c 完整版采样器
-- 增加 `easy cascadeKSampler` - stable cascade stage-c ksampler simple
+- Added `easy cascadeLoader` - stable cascade Loader
+- Added `easy preSamplingCascade` - stable cascade preSampling Settings
+- Added `easy fullCascadeKSampler` - stable cascade stage-c ksampler full
+- Added `easy cascadeKSampler` - stable cascade stage-c ksampler simple
+- 
+- Optimize the image to image[Example](https://github.com/yolain/ComfyUI-Easy-Use/blob/main/README.en.md#image-to-image)
 </details>
 
 <details>
 <summary><b>v1.0.6</b></summary>
 
-- 增加 `easy XYInputs: Checkpoint`
-- 增加 `easy XYInputs: Lora`
-- `easy seed` 增加固定种子值时可手动切换随机种
-- 修复 `easy fullLoader`等加载器切换lora时自动调整节点大小的问题
-- 去除原有ttn的图片保存逻辑并适配ComfyUI默认的图片保存格式化扩展
+- Added `easy XYInputs: Checkpoint`
+- Added `easy XYInputs: Lora`
+- `easy seed` can manually switch the random seed when increasing the fixed seed value
+- Fixed `easy fullLoader` and all loaders to automatically adjust the node size when switching LoRa
+- Removed the original ttn image saving logic and adapted to the default image saving format extension of ComfyUI
 </details>
 
 <details>
 <summary><b>v1.0.5</b></summary>
 
-- 增加 `easy isSDXL` 
-- `easy svdLoader` 增加提示词控制, 可配合open_clip模型进行使用
-- `easy wildcards` 增加 **populated_text** 可输出通配填充后文本
+- Added `easy isSDXL` 
+- Added prompt word control on `easy svdLoader`, which can be used with open_clip model
+- Added **populated_text** on `easy wildcards`, wildcard populated text can be output
 </details>
 
 <details>
 <summary><b>v1.0.4</b></summary>
 
-- 增加 `easy showLoaderSettingsNames` 可显示与输出加载器部件中的 模型与VAE名称
-- 增加 `easy promptList` - 提示词列表
-- 增加 `easy fooocusInpaintLoader` - Fooocus内补节点（仅支持XL模型的流程）
-- 增加 **Logic** 逻辑类节点 - 包含类型、计算、判断和转换类型等
-- 增加 `easy imageSave` - 带日期转换和宽高格式化的图像保存节点
-- 增加 `easy joinImageBatch` - 合并图像批次
-- `easy showAnything` 增加支持转换其他类型（如：tensor类型的条件、图像等）
-- `easy kSamplerInpainting` 增加 **patch** 传入值，配合Fooocus内补节点使用
-- `easy imageSave` 增加 **only_preivew**
+- `easy showAnything` added support for converting other types (e.g., tensor conditions, images, etc.)
+- Added `easy showLoaderSettingsNames` can display the model and VAE name in the output loader assembly
+- Added `easy promptList`
+- Added `easy fooocusInpaintLoader` （only the process of SDXLModel is supported）
+- Added **Logic** nodes
+- Added `easy imageSave` - Image saving node with date conversion and aspect and height formatting
+- Added `easy joinImageBatch`
+- `easy kSamplerInpainting` Added the **patch** input value to be used with the FooocusInpaintLoader node
 
-- 修复 xyplot在pillow>9.5中报错
-- 修复 `easy wildcards` 在使用PS扩展插件运行时报错
-- 修复 `easy latentCompositeMaskedWithCond`
-- 修复 `easy XYInputs: ControlNet` 报错
-- 修复 `easy loraStack` **toggle** 为 disabled 时报错
+- Fixed xyplot error when with Pillow>9.5
+- Fixed `easy wildcards` An error is reported when running with the PS extension
+- Fixed `easy XYInputs: ControlNet` Error
+- Fixed `easy loraStack` error when **toggle** is disabled
 
-- 修改首次安装节点包不再自动替换主题，需手动调整并刷新页面
+
+- Changing the first-time install node package no longer automatically replaces the theme, you need to manually adjust and refresh the page
+- `easy imageSave` added **only_preivew**
+- Adjust the `easy latentCompositeMaskedWithCond` node
 </details>
 
 <details>
 <summary><b>v1.0.3</b></summary>
 
-- 增加 `easy stylesSelector` 风格化提示词选择器
-- 增加队列进度条设置项，默认为未启用状态
-- `easy controlnetLoader` 和 `easy controlnetLoaderADV` 增加参数 **scale_soft_weights**
+- Added `easy stylesSelector`
+- Added **scale_soft_weights** in `easy controlnetLoader` and `easy controlnetLoaderADV` 
+- Added the queue progress bar setting item, which is not enabled by default
 
 
-- 修复 `easy XYInputs: Sampler/Scheduler` 报错
-- 修复 右侧菜单 点击按钮时老是跑位的问题
-- 修复 styles 路径在其他环境报错
-- 修复 `easy comfyLoader` 读取错误
-- 修复 xyPlot 在连接 zero123 时报错
-- 修复加载器中提示词为组件时报错
-- 修复 `easy getNode` 和 `easy setNode` 加载时标题未更改
-- 修复所有采样器中存储图片使用子目录前缀不生效的问题
+- Fixed `easy XYInputs: Sampler/Scheduler` Error
+- Fixed the right menu has a problem when clicking the button
+- Fixed `easy comfyLoader` error
+- Fixed xyPlot error when connecting to zero123
+- Fixed the error message in the loader when the prompt word was component
+- Fixed `easy getNode` and `easy setNode` the title does not change when loading
+- Fixed all samplers using subdirectories to store images
 
 
-- 调整UI主题
+- Adjust the UI theme, divided into two sets of styles: the official default background and the dark black background, which can be switched in the color palette in the settings
+- Modify the styles path to be compatible with other environments
 </details>
 
 <details>
 <summary><b>v1.0.2</b></summary>
 
-- 增加 **autocomplete** 文件夹，如果您安装了 [ComfyUI-Custom-Scripts](https://github.com/pythongosssss/ComfyUI-Custom-Scripts), 将在启动时合并该文件夹下的所有txt文件并覆盖到pyssss包里的autocomplete.txt文件。
-- 增加 `easy XYPlotAdvanced` 和 `easy XYInputs` 等相关节点
-- 增加 **Alt+1到9** 快捷键，可快速粘贴 Node templates 的节点预设 （对应 1到9 顺序）
+- Added `easy XYPlotAdvanced` and some nodes about `easy XYInputs`
+- Added **Alt+1-Alt+9** Shortcut keys to quickly paste node presets for Node templates (corresponding to 1~9 sequences)
+- Added a `📜Groups Map(EasyUse)` to the context menu.
+- An `autocomplete` folder has been added, If you have [ComfyUI-Custom-Scripts](https://github.com/pythongosssss/ComfyUI-Custom-Scripts) installed, the txt files in that folder will be merged and overwritten to the autocomplete .txt file of the pyssss package at startup.
 
-- 修复 `easy imageInsetCrop` 测量值为百分比时步进为1
-- 修复 开启 `a1111_prompt_style` 时XY图表无法使用的问题
-- 右键菜单中增加了一个 `📜Groups Map(EasyUse)` 
 
-- 修复在Comfy新版本中UI加载失败
-- 修复 `easy pipeToBasicPipe` 报错
-- 修改 `easy fullLoader` 和 `easy a1111Loader` 中的 **a1111_prompt_style** 默认值为 False
-- `easy XYInputs ModelMergeBlocks` 支持csv文件导入数值
+- Fixed XYPlot is not working when `a1111_prompt_style` is True
+- Fixed UI loading failure in the new version of ComfyUI
+- `easy XYInputs ModelMergeBlocks` Values can be imported from CSV files
+- Fixed `easy pipeToBasicPipe` Bug
 
-- 替换了XY图生成时的字体文件
-
-- 移除 `easy imageRemBg`
-- 移除包中的介绍图和工作流文件，减少包体积
 
+- Removed `easy imageRemBg`
+- Remove the introductory diagram and workflow files from the package to reduce the package size
+- Replaced the font file used in the generation of XY diagrams
 </details>
 
 <details>
 <summary><b>v1.0.1</b></summary>
 
-- 新增 `easy seed` - 简易随机种
-- `easy preDetailerFix` 新增了 `optional_image` 传入图像可选，如未传默认取值为pipe里的图像
-- 新增 `easy kSamplerInpainting` 用于内补潜空间的采样器
-- 新增 `easy pipeToBasicPipe` 用于转换到Impact的某些节点上
-
-- 修复 `easy comfyLoader` 报错
-- 修复所有包含输出图片尺寸的节点取值方式无法批处理的问题
-- 修复 `width` 和 `height` 无法在 `easy svdLoader` 自定义的报错问题
-- 修复所有采样器预览图片的地址链接 (解决在 MACOS 系统中图片无法在采样器中预览的问题）
-- 修复 `vae_name` 在 `easy fullLoader` 和 `easy a1111Loader` 和 `easy comfyLoader` 中选择但未替换原始vae问题
-- 修复 `easy fullkSampler` 除pipe外其他输出值的报错
-- 修复 `easy hiresFix` 输入连接pipe和image、vae同时存在时报错
-- 修复 `easy fullLoader` 中 `model_override` 连接后未执行 
-- 修复 因新增`easy seed` 导致action错误
-- 修复 `easy xyplot` 的字体文件路径读取错误
-- 修复 convert 到 `easy seed` 随机种无法固定的问题
-- 修复 `easy pipeIn` 值传入的报错问题
-- 修复 `easy zero123Loader` 和 `easy svdLoader` 读取模型时将模型加入到缓存中
-- 修复 `easy kSampler` `easy kSamplerTiled` `easy detailerFix` 的 `image_output` 默认值为 Preview
-- `easy fullLoader` 和 `easy a1111Loader` 新增了 `a1111_prompt_style` 参数可以重现和webui生成相同的图像，当前您需要安装 [ComfyUI_smZNodes](https://github.com/shiimizu/ComfyUI_smZNodes) 才能使用此功能
+- Fixed `easy comfyLoader` error
+- Fixed All nodes that contain the value of the image size
+- Added `easy kSamplerInpainting`
+- Added `easy pipeToBasicPipe`
+- Fixed `width` and `height` can not customize in `easy svdLoader`
+- Fixed all preview image path (Previously, it was not possible to preview the image on the Mac system)
+- Fixed `vae_name` is not working in `easy fullLoader` and `easy a1111Loader` and `easy comfyLoader`
+- Fixed `easy fullkSampler` outputs error
+- Fixed `model_override` is not working in `easy fullLoader`
+- Fixed `easy hiresFix` error
+- Fixed `easy xyplot` font file path error
+- Fixed seed that cannot be fixed when you convert `seed_num` to `easy seed` 
+- Fixed `easy pipeIn` inputs bug
+- `easy preDetailerFix` have added a new parameter `optional_image`
+- Fixed `easy zero123Loader` and `easy svdLoader` model into cache.
+- Added `easy seed`
+- Fixed `image_output` default value is "Preview"
+- `easy fullLoader` and `easy a1111Loader` have added a new parameter `a1111_prompt_style`,that can reproduce the same image generated from stable-diffusion-webui on comfyui, but you need to install [ComfyUI_smZNodes](https://github.com/shiimizu/ComfyUI_smZNodes) to use this feature in the current version
 </details>
 
 <details>
 <summary><b>v1.0.0</b></summary>
 
-- 新增`easy positive` - 简易正面提示词文本
-- 新增`easy negative`  - 简易负面提示词文本
-- 新增`easy wildcards` - 支持通配符和Lora选择的提示词文本
-- 新增`easy portraitMaster` - 肖像大师v2.2
-- 新增`easy loraStack` - Lora堆
-- 新增`easy fullLoader` - 完整版的加载器
-- 新增`easy zero123Loader` - 简易zero123加载器
-- 新增`easy svdLoader` - 简易svd加载器
-- 新增`easy fullkSampler` - 完整版的采样器（无分离）
-- 新增`easy hiresFix` - 支持Pipe的高清修复
-- 新增`easy predetailerFix` `easy DetailerFix` - 支持Pipe的细节修复
-- 新增`easy ultralyticsDetectorPipe` `easy samLoaderPipe` - 检测加载器（细节修复的输入项）
-- 新增`easy pipein` `easy pipeout` - Pipe的输入与输出
-- 新增`easy xyPlot` - 简易的xyplot (后续会更新更多可控参数)
-- 新增`easy imageRemoveBG` - 图像去除背景
-- 新增`easy imagePixelPerfect` - 图像完美像素
-- 新增`easy poseEditor` - 姿势编辑器
-- 新增UI主题（黑曜石）- 默认自动加载UI, 也可在设置中自行更替 
+- Added `easy positive` - simple positive prompt text
+- Added `easy negative` - simple negative prompt text
+- Added `easy wildcards` - support for wildcards and hint text selected by Lora
+- Added `easy portraitMaster` - PortraitMaster v2.2
+- Added `easy loraStack` - Lora stack
+- Added `easy fullLoader` - full version of the loader
+- Added `easy zero123Loader` - simple zero123 loader
+- Added `easy svdLoader` - easy svd loader
+- Added `easy fullkSampler` - full version of the sampler (no separation)
+- Added `easy hiresFix` - support for HD repair of Pipe
+- Added `easy predetailerFix` and `easy DetailerFix` - support for Pipe detail fixing
+- Added `easy ultralyticsDetectorPipe` and `easy samLoaderPipe` - Detect loader (detail fixed input)
+- Added `easy pipein` `easy pipeout` - Pipe input and output
+- Added `easy xyPlot` - simple xyplot (more controllable parameters will be updated in the future)
+- Added `easy imageRemoveBG` - image to remove background
+- Added `easy imagePixelPerfect` - image pixel perfect
+- Added `easy poseEditor` - Pose editor
+- New UI Theme (Obsidian) - Auto-load UI by default, which can also be changed in the settings 
 
-- 修复 `easy globalSeed` 不生效问题
-- 修复所有的`seed_num` 因 [cg-use-everywhere](https://github.com/chrisgoringe/cg-use-everywhere) 实时更新图表导致值错乱的问题
-- 修复`easy imageSize` `easy imageSizeBySide` `easy imageSizeByLongerSide` 可作为终节点
-- 修复 `seed_num` (随机种子值) 在历史记录中读取无法一致的Bug
+- Fixed `easy globalSeed` is not working
+- Fixed an issue where all `seed_num` values were out of order due to [cg-use-everywhere](https://github.com/chrisgoringe/cg-use-everywhere) updating the chart in real time
+- Fixed `easy imageSize`, `easy imageSizeBySide`, `easy imageSizeByLongerSide` as end nodes
+- Fixed the bug that `seed_num` (random seed value) could not be read consistently in history
 </details>
 
-
 <details>
-<summary><b>v0.5</b></summary>
+<summary><b>Updated at 12/14/2023</b></summary>
 
-- 新增 `easy controlnetLoaderADV` 节点
--  新增 `easy imageSizeBySide` 节点，可选输出为长边或短边
--  新增 `easy LLLiteLoader` 节点，如果您预先安装过 kohya-ss/ControlNet-LLLite-ComfyUI 包，请将 models 里的模型文件移动至 ComfyUI\models\controlnet\ (即comfy默认的controlnet路径里，请勿修改模型的文件名，不然会读取不到)。
--  新增 `easy imageSize` 和 `easy imageSizeByLongerSize` 输出的尺寸显示。
--  新增 `easy showSpentTime` 节点用于展示图片推理花费时间与VAE解码花费时间。
-- `easy controlnetLoaderADV` 和 `easy controlnetLoader` 新增 `control_net` 可选传入参数
-- `easy preSampling` 和 `easy preSamplingAdvanced` 新增 `image_to_latent` 可选传入参数
-- `easy a1111Loader` 和 `easy comfyLoader` 新增 `batch_size` 传入参数
-
--  修改 `easy controlnetLoader` 到 loader 分类底下。
+- `easy a1111Loader` and `easy comfyLoader` added `batch_size` of required input parameters 
+- Added the `easy controlnetLoaderADV` node
+- `easy controlnetLoaderADV` and `easy controlnetLoader` added `control_net ` of optional input parameters
+- `easy preSampling` and `easy preSamplingAdvanced` added `image_to_latent` optional input parameters
+- Added the `easy imageSizeBySide` node, which can be output as a long side or a short side
 </details>
 
-## 整合参考到的相关节点包
+<details>
+<summary><b>Updated at 12/13/2023</b></summary>
 
-声明: 非常尊重这些原作者们的付出，开源不易，我仅仅只是做了一些整合与优化。
+-  Added the `easy LLLiteLoader` node, if you have pre-installed the kohya-ss/ControlNet-LLLite-ComfyUI package, please move the model files in the models to `ComfyUI\models\controlnet\` (i.e. in the default controlnet path of comfy, please do not change the file name of the model, otherwise it will not be read).
+-  Modify `easy controlnetLoader` to the bottom of the loader category.
+-  Added size display for `easy imageSize` and `easy imageSizeByLongerSize` outputs.
+</details>
 
-| 节点名 (搜索名)                      | 相关的库                                                                        | 库相关的节点                  |
-|:-------------------------------|:----------------------------------------------------------------------------|:------------------------|
-| easy setNode                   | [ComfyUI-extensions](https://github.com/diffus3/ComfyUI-extensions) | diffus3.SetNode         |
-| easy getNode                   | [ComfyUI-extensions](https://github.com/diffus3/ComfyUI-extensions) | diffus3.GetNode         |
-| easy bookmark                  | [rgthree-comfy](https://github.com/rgthree/rgthree-comfy) | Bookmark 🔖             |
-| easy portraitMarker            | [comfyui-portrait-master](https://github.com/florestefano1975/comfyui-portrait-master) | Portrait Master         |
-| easy LLLiteLoader              | [ControlNet-LLLite-ComfyUI](https://github.com/kohya-ss/ControlNet-LLLite-ComfyUI) | LLLiteLoader            |
-| easy globalSeed                | [ComfyUI-Inspire-Pack](https://github.com/ltdrdata/ComfyUI-Inspire-Pack) | Global Seed (Inspire)   | 
-| easy preSamplingDynamicCFG     | [sd-dynamic-thresholding](https://github.com/mcmonkeyprojects/sd-dynamic-thresholding) | DynamicThresholdingFull | 
-| dynamicThresholdingFull        | [sd-dynamic-thresholding](https://github.com/mcmonkeyprojects/sd-dynamic-thresholding) | DynamicThresholdingFull | 
-| easy imageInsetCrop            | [rgthree-comfy](https://github.com/rgthree/rgthree-comfy) | ImageInsetCrop          | 
-| easy poseEditor                | [ComfyUI_Custom_Nodes_AlekPet](https://github.com/AlekPet/ComfyUI_Custom_Nodes_AlekPet) | poseNode                | 
-| easy if                        | [ComfyUI-Logic](https://github.com/theUpsider/ComfyUI-Logic) | IfExecute               | 
-| easy preSamplingLayerDiffusion | [ComfyUI-layerdiffusion](https://github.com/huchenlei/ComfyUI-layerdiffusion) | LayeredDiffusionApply等  | 
-| easy dynamiCrafterLoader       | [ComfyUI-layerdiffusion](https://github.com/ExponentialML/ComfyUI_Native_DynamiCrafter) | Apply Dynamicrafter     | 
+<details>
+<summary><b>Updated at 12/11/2023</b></summary>
+-  Added the `showSpentTime` node to display the time spent on image diffusion and the time spent on VAE decoding images
+</details>
+
+## The relevant node package involved
+
+Disclaimer: Opened source was not easy. I have a lot of respect for the contributions of these original authors. I just did some integration and optimization.
+
+| Nodes Name(Search Name)        | Related libraries                                                                        | Library-related node     |
+|:-------------------------------|:----------------------------------------------------------------------------|:-------------------------|
+| easy setNode                   | [ComfyUI-extensions](https://github.com/diffus3/ComfyUI-extensions) | diffus3.SetNode          |
+| easy getNode                   | [ComfyUI-extensions](https://github.com/diffus3/ComfyUI-extensions) | diffus3.GetNode          |
+| easy bookmark                  | [rgthree-comfy](https://github.com/rgthree/rgthree-comfy) | Bookmark 🔖              |
+| easy portraitMarker            | [comfyui-portrait-master](https://github.com/florestefano1975/comfyui-portrait-master) | Portrait Master          |
+| easy LLLiteLoader              | [ControlNet-LLLite-ComfyUI](https://github.com/kohya-ss/ControlNet-LLLite-ComfyUI) | LLLiteLoader             |
+| easy globalSeed                | [ComfyUI-Inspire-Pack](https://github.com/ltdrdata/ComfyUI-Inspire-Pack) | Global Seed (Inspire)    | 
+| easy preSamplingDynamicCFG     | [sd-dynamic-thresholding](https://github.com/mcmonkeyprojects/sd-dynamic-thresholding) | DynamicThresholdingFull  | 
+| dynamicThresholdingFull        | [sd-dynamic-thresholding](https://github.com/mcmonkeyprojects/sd-dynamic-thresholding) | DynamicThresholdingFull  | 
+| easy imageInsetCrop            | [rgthree-comfy](https://github.com/rgthree/rgthree-comfy) | ImageInsetCrop           | 
+| easy poseEditor                | [ComfyUI_Custom_Nodes_AlekPet](https://github.com/AlekPet/ComfyUI_Custom_Nodes_AlekPet) | poseNode                 | 
+| easy preSamplingLayerDiffusion | [ComfyUI-layerdiffusion](https://github.com/huchenlei/ComfyUI-layerdiffusion) | LayeredDiffusionApply... | 
+| easy dynamiCrafterLoader       | [ComfyUI-layerdiffusion](https://github.com/ExponentialML/ComfyUI_Native_DynamiCrafter) | Apply Dynamicrafter    | 
 | easy imageChooser              | [cg-image-picker](https://github.com/chrisgoringe/cg-image-picker) | Preview Chooser         | 
-| easy styleAlignedBatchAlign    | [style_aligned_comfy](https://github.com/chrisgoringe/cg-image-picker) | styleAlignedBatchAlign  | 
-| easy icLightApply              | [ComfyUI-IC-Light](https://github.com/huchenlei/ComfyUI-IC-Light) | ICLightApply等           |
+| easy styleAlignedBatchAlign              | [style_aligned_comfy](https://github.com/chrisgoringe/cg-image-picker) | styleAlignedBatchAlign       | 
+| easy kolorsLoader              | [ComfyUI-Kolors-MZ](https://github.com/MinusZoneAI/ComfyUI-Kolors-MZ) | kolorsLoader            |
+
 
 ## Credits
 
-[ComfyUI](https://github.com/comfyanonymous/ComfyUI) - 功能强大且模块化的Stable Diffusion GUI
+[ComfyUI](https://github.com/comfyanonymous/ComfyUI) - Powerful and modular Stable Diffusion GUI
 
-[ComfyUI-ComfyUI-Manager](https://github.com/ltdrdata/ComfyUI-Manager) - ComfyUI管理器
+[ComfyUI-ComfyUI-Manager](https://github.com/ltdrdata/ComfyUI-Manager) - ComfyUI Manager
 
-[tinyterraNodes](https://github.com/TinyTerra/ComfyUI_tinyterraNodes) - 管道节点（节点束）让用户减少了不必要的连接
+[tinyterraNodes](https://github.com/TinyTerra/ComfyUI_tinyterraNodes) - Pipe nodes (node bundles) allow users to reduce unnecessary connections
 
-[ComfyUI-extensions](https://github.com/diffus3/ComfyUI-extensions) - diffus3的获取与设置点让用户可以分离工作流构成
+[ComfyUI-extensions](https://github.com/diffus3/ComfyUI-extensions) - Diffus3 gets and sets points that allow the user to detach the composition of the workflow 
 
-[ComfyUI-Impact-Pack](https://github.com/ltdrdata/ComfyUI-Impact-Pack) - 常规整合包1
+[ComfyUI-Impact-Pack](https://github.com/ltdrdata/ComfyUI-Impact-Pack) - General modpack 1
 
-[ComfyUI-Inspire-Pack](https://github.com/ltdrdata/ComfyUI-Inspire-Pack) - 常规整合包2
+[ComfyUI-Inspire-Pack](https://github.com/ltdrdata/ComfyUI-Inspire-Pack) - General Modpack 2
 
-[ComfyUI-Logic](https://github.com/theUpsider/ComfyUI-Logic) -  ComfyUI逻辑运算
+[ComfyUI-ResAdapter](https://github.com/jiaxiangc/ComfyUI-ResAdapter) - Make model generation independent of training resolution
 
-[ComfyUI-ResAdapter](https://github.com/jiaxiangc/ComfyUI-ResAdapter) - 让模型生成不受训练分辨率限制
+[ComfyUI_IPAdapter_plus](https://github.com/cubiq/ComfyUI_IPAdapter_plus) - Style migration
 
-[ComfyUI_IPAdapter_plus](https://github.com/cubiq/ComfyUI_IPAdapter_plus) - 风格迁移
+[ComfyUI_InstantID](https://github.com/cubiq/ComfyUI_InstantID) - Face migration
 
-[ComfyUI_InstantID](https://github.com/cubiq/ComfyUI_InstantID) - 人脸迁移
+[ComfyUI_PuLID](https://github.com/cubiq/PuLID_ComfyUI) - Face migration
 
-[ComfyUI_PuLID](https://github.com/cubiq/PuLID_ComfyUI) - 人脸迁移
+[ComfyUI-Custom-Scripts](https://github.com/pythongosssss/ComfyUI-Custom-Scripts) - pyssss🐍
 
-[ComfyUI-Custom-Scripts](https://github.com/pythongosssss/ComfyUI-Custom-Scripts) - pyssss 小蛇🐍脚本
+[cg-image-picker](https://github.com/chrisgoringe/cg-image-picker) - Image Preview Chooser
 
-[cg-image-picker](https://github.com/chrisgoringe/cg-image-picker) - 图片选择器
+[ComfyUI_ExtraModels](https://github.com/city96/ComfyUI_ExtraModels) - DiT custom nodes
 
-[ComfyUI-BrushNet](https://github.com/nullquant/ComfyUI-BrushNet) - BrushNet 内补节点
+## ☕️ Donation
 
-[ComfyUI_ExtraModels](https://github.com/city96/ComfyUI_ExtraModels) - DiT架构相关节点（Pixart、混元DiT等）
+**Comfyui-Easy-Use** is an GPL-licensed open source project. In order to achieve better and sustainable development of the project, i expect to gain more backers. <br>
+If my custom nodes has added value to your day, consider indulging in a coffee to fuel it further! <br>
+💖You can support me in any of the following ways:
+
+- [BiliBili](https://space.bilibili.com/1840885116)
+- [Afdian](https://afdian.com/a/yolain)
+- [Wechat / Alipay](https://github.com/user-attachments/assets/803469bd-ed6a-4fab-932d-50e5088a2d03)
+- 🪙 Wallet Address:
+  - ETH: 0x01f7CEd3245CaB3891A0ec8f528178db352EaC74
+  - USDT(tron): TP3AnJXkAzfebL2GKmFAvQvXgsxzivweV6
+
+（This is a newly created wallet, and if it receives sponsorship, I'll use it to rent GPUs or other GPT services for better debugging and refinement of ComfyUI-Easy-Use features.）
 
 ##  🌟Stargazers
 

__init__.py

@@ -1,5 +1,6 @@
-__version__ = "1.2.0"
+__version__ = "1.2.3"
 
+import yaml
 import os
 import folder_paths
 import importlib
@@ -24,7 +25,7 @@ for module_name in node_list:
 cwd_path = os.path.dirname(os.path.realpath(__file__))
 comfy_path = folder_paths.base_path
 
-#Wildcards读取
+#Wildcards
 from .py.libs.wildcards import read_wildcard_dict
 wildcards_path = os.path.join(os.path.dirname(__file__), "wildcards")
 if os.path.exists(wildcards_path):
@@ -53,8 +54,39 @@ for model in model_config:
             continue
         add_static_resource(path, path, limit=True)
 
-WEB_DIRECTORY = "./web"
+# get comfyui revision
+from .py.libs.utils import compare_revision
+
+new_frontend_revision = 2546
+web_default_version = 'v2' if compare_revision(new_frontend_revision) else 'v1'
+# web directory
+config_path = os.path.join(cwd_path, "config.yaml")
+if os.path.isfile(config_path):
+    with open(config_path, 'r') as f:
+        data = yaml.load(f, Loader=yaml.FullLoader)
+        if data and "WEB_VERSION" in data:
+            directory = f"web_version/{data['WEB_VERSION']}"
+            with open(config_path, 'w') as f:
+                yaml.dump(data, f)
+        elif web_default_version != 'v1':
+            if not data:
+                data = {'WEB_VERSION': web_default_version}
+            elif 'WEB_VERSION' not in data:
+                data = {**data, 'WEB_VERSION': web_default_version}
+            with open(config_path, 'w') as f:
+                yaml.dump(data, f)
+            directory = f"web_version/{web_default_version}"
+        else:
+            directory = f"web_version/v1"
+    if not os.path.exists(os.path.join(cwd_path, directory)):
+        print(f"web root {data['WEB_VERSION']} not found, using default")
+        directory = f"web_version/{web_default_version}"
+    WEB_DIRECTORY = directory
+else:
+    directory = f"web_version/{web_default_version}"
+    WEB_DIRECTORY =  directory
+
 __all__ = ['NODE_CLASS_MAPPINGS', 'NODE_DISPLAY_NAME_MAPPINGS', "WEB_DIRECTORY"]
 
-
-print(f'\033[34mComfy-Easy-Use v{__version__}: \033[92mLoaded\033[0m')
+print(f'\033[34m[ComfyUI-Easy-Use] server: \033[0mv{__version__} \033[92mLoaded\033[0m')
+print(f'\033[34m[ComfyUI-Easy-Use] web root: \033[0m{os.path.join(cwd_path, directory)} \033[92mLoaded\033[0m')

install.bat

@@ -1,6 +1,7 @@
 @echo off
 
 set "requirements_txt=%~dp0\requirements.txt"
+set "requirements_repair_txt=%~dp0\repair_dependency_list.txt"
 set "python_exec=..\..\..\python_embeded\python.exe"
 set "aki_python_exec=..\..\python\python.exe"
 
@@ -12,7 +13,10 @@ if exist "%python_exec%" (
 )^
 else if exist "%aki_python_exec%" (
     echo Installing with ComfyUI Aki
-    "%python_exec%" -s -m pip install -r "%requirements_txt%" 
+    "%aki_python_exec%" -s -m pip install -r "%requirements_txt%"
+    for /f "delims=" %%i in (%requirements_repair_txt%) do (
+        %aki_python_exec% -s -m pip install -i https://pypi.tuna.tsinghua.edu.cn/simple "%%i"
+        )
 )^
 else (
     echo Installing with system Python

install.sh

@@ -0,0 +1,24 @@
+#!/bin/bash
+
+requirements_txt="$(dirname "$0")/requirements.txt"
+requirements_repair_txt="$(dirname "$0")/repair_dependency_list.txt"
+python_exec="../../../python_embeded/python.exe"
+aki_python_exec="../../python/python.exe"
+
+echo "Installing EasyUse Requirements..."
+
+if [ -f "$python_exec" ]; then
+    echo "Installing with ComfyUI Portable"
+    "$python_exec" -s -m pip install -r "$requirements_txt"
+elif [ -f "$aki_python_exec" ]; then
+    echo "Installing with ComfyUI Aki"
+    "$aki_python_exec" -s -m pip install -r "$requirements_txt"
+    while IFS= read -r line; do
+        "$aki_python_exec" -s -m pip install -i https://pypi.tuna.tsinghua.edu.cn/simple "$line"
+    done < "$requirements_repair_txt"
+else
+    echo "Installing with system Python"
+    pip install -r "$requirements_txt"
+fi
+
+read -p "Press any key to continue..."

prestartup_script.py

@@ -31,6 +31,7 @@ add_folder_path_and_extensions("mediapipe", [os.path.join(model_path, "mediapipe
 add_folder_path_and_extensions("inpaint", [os.path.join(model_path, "inpaint")], folder_paths.supported_pt_extensions)
 add_folder_path_and_extensions("prompt_generator", [os.path.join(model_path, "prompt_generator")], folder_paths.supported_pt_extensions)
 add_folder_path_and_extensions("t5", [os.path.join(model_path, "t5")], folder_paths.supported_pt_extensions)
+add_folder_path_and_extensions("llm", [os.path.join(model_path, "LLM")], folder_paths.supported_pt_extensions)
 
 add_folder_path_and_extensions("checkpoints_thumb", [os.path.join(model_path, "checkpoints")], image_suffixs)
 add_folder_path_and_extensions("loras_thumb", [os.path.join(model_path, "loras")], image_suffixs)

py/api.py

@@ -7,7 +7,6 @@ import folder_paths
 from folder_paths import get_directory_by_type
 from server import PromptServer
 from .config import RESOURCES_DIR, FOOOCUS_STYLES_DIR, FOOOCUS_STYLES_SAMPLES
-from .logic import ConvertAnything
 from .libs.model import easyModelManager
 from .libs.utils import getMetadata, cleanGPUUsedForce, get_local_filepath
 from .libs.cache import remove_cache
@@ -31,6 +30,17 @@ def cleanGPU(request):
         return web.Response(status=500)
         pass
 
+@PromptServer.instance.routes.post("/easyuse/removecache")
+async def removecache(request):
+    post = await request.post()
+    key = post.get("key")
+    try:
+        remove_cache(key)
+        return web.Response(status=200)
+    except Exception as e:
+        return web.Response(status=500)
+        pass
+
 @PromptServer.instance.routes.post("/easyuse/translate")
 async def translate(request):
     post = await request.post()
@@ -99,6 +109,10 @@ async def getStylesList(request):
                         nd['name_cn'] = cn_data[key] if key in cn_data else key
                     nd["name"] = d['name']
                     nd['imgName'] = img_name
+                    if "prompt" in d:
+                        nd['prompt'] = d['prompt']
+                    if "negative_prompt" in d:
+                        nd['negative_prompt'] = d['negative_prompt']
                     ndata.append(nd)
                 return web.json_response(ndata)
     return web.Response(status=400)
@@ -119,18 +133,6 @@ async def getStylesImage(request):
             return web.Response(text=FOOOCUS_STYLES_SAMPLES + name + '.jpg')
     return web.Response(status=400)
 
-# convert type
-@PromptServer.instance.routes.post("/easyuse/convert")
-async def convertType(request):
-    post = await request.post()
-    type = post.get('type')
-    if type:
-        ConvertAnything.RETURN_TYPES = (type.upper(),)
-        ConvertAnything.RETURN_NAMES = (type,)
-        return web.Response(status=200)
-    else:
-        return web.Response(status=400)
-
 # get models lists
 @PromptServer.instance.routes.get("/easyuse/models/list")
 async def getModelsList(request):
@@ -146,11 +148,15 @@ async def getModelsList(request):
 # get models thumbnails
 @PromptServer.instance.routes.get("/easyuse/models/thumbnail")
 async def getModelsThumbnail(request):
+    limit = 500
+    if "limit" in request.rel_url.query:
+        limit = request.rel_url.query.get("limit")
+        limit = int(limit)
     checkpoints = folder_paths.get_filename_list("checkpoints_thumb")
     loras = folder_paths.get_filename_list("loras_thumb")
     checkpoints_full = []
     loras_full = []
-    if len(checkpoints) + len(loras) >= 500:
+    if len(checkpoints) + len(loras) >= limit:
         return web.Response(status=400)
     for index, i in enumerate(checkpoints):
         full_path = folder_paths.get_full_path('checkpoints_thumb', str(i))

py/bitsandbytes_NF4/__init__.py

@@ -0,0 +1,167 @@
+#credit to comfyanonymous for this module
+#from https://github.com/comfyanonymous/ComfyUI_bitsandbytes_NF4
+import comfy.ops
+import torch
+import folder_paths
+from ..libs.utils import install_package
+
+try:
+    from bitsandbytes.nn.modules import Params4bit, QuantState
+except ImportError:
+    Params4bit = torch.nn.Parameter
+    raise ImportError("Please install bitsandbytes>=0.43.3")
+
+def functional_linear_4bits(x, weight, bias):
+    try:
+        install_package("bitsandbytes", "0.43.3", True, "0.43.3")
+        import bitsandbytes as bnb
+    except ImportError:
+        raise ImportError("Please install bitsandbytes>=0.43.3")
+
+    out = bnb.matmul_4bit(x, weight.t(), bias=bias, quant_state=weight.quant_state)
+    out = out.to(x)
+    return out
+
+
+def copy_quant_state(state, device: torch.device = None):
+    if state is None:
+        return None
+
+    device = device or state.absmax.device
+
+    state2 = (
+        QuantState(
+            absmax=state.state2.absmax.to(device),
+            shape=state.state2.shape,
+            code=state.state2.code.to(device),
+            blocksize=state.state2.blocksize,
+            quant_type=state.state2.quant_type,
+            dtype=state.state2.dtype,
+        )
+        if state.nested
+        else None
+    )
+
+    return QuantState(
+        absmax=state.absmax.to(device),
+        shape=state.shape,
+        code=state.code.to(device),
+        blocksize=state.blocksize,
+        quant_type=state.quant_type,
+        dtype=state.dtype,
+        offset=state.offset.to(device) if state.nested else None,
+        state2=state2,
+    )
+
+
+class ForgeParams4bit(Params4bit):
+
+    def to(self, *args, **kwargs):
+        device, dtype, non_blocking, convert_to_format = torch._C._nn._parse_to(*args, **kwargs)
+        if device is not None and device.type == "cuda" and not self.bnb_quantized:
+            return self._quantize(device)
+        else:
+            n = ForgeParams4bit(
+                torch.nn.Parameter.to(self, device=device, dtype=dtype, non_blocking=non_blocking),
+                requires_grad=self.requires_grad,
+                quant_state=copy_quant_state(self.quant_state, device),
+                blocksize=self.blocksize,
+                compress_statistics=self.compress_statistics,
+                quant_type=self.quant_type,
+                quant_storage=self.quant_storage,
+                bnb_quantized=self.bnb_quantized,
+                module=self.module
+            )
+            self.module.quant_state = n.quant_state
+            self.data = n.data
+            self.quant_state = n.quant_state
+            return n
+
+class ForgeLoader4Bit(torch.nn.Module):
+    def __init__(self, *, device, dtype, quant_type, **kwargs):
+        super().__init__()
+        self.dummy = torch.nn.Parameter(torch.empty(1, device=device, dtype=dtype))
+        self.weight = None
+        self.quant_state = None
+        self.bias = None
+        self.quant_type = quant_type
+
+    def _save_to_state_dict(self, destination, prefix, keep_vars):
+        super()._save_to_state_dict(destination, prefix, keep_vars)
+        quant_state = getattr(self.weight, "quant_state", None)
+        if quant_state is not None:
+            for k, v in quant_state.as_dict(packed=True).items():
+                destination[prefix + "weight." + k] = v if keep_vars else v.detach()
+        return
+
+    def _load_from_state_dict(self, state_dict, prefix, local_metadata, strict, missing_keys, unexpected_keys, error_msgs):
+        quant_state_keys = {k[len(prefix + "weight."):] for k in state_dict.keys() if k.startswith(prefix + "weight.")}
+
+        if any('bitsandbytes' in k for k in quant_state_keys):
+            quant_state_dict = {k: state_dict[prefix + "weight." + k] for k in quant_state_keys}
+
+            self.weight = ForgeParams4bit().from_prequantized(
+                data=state_dict[prefix + 'weight'],
+                quantized_stats=quant_state_dict,
+                requires_grad=False,
+                device=self.dummy.device,
+                module=self
+            )
+            self.quant_state = self.weight.quant_state
+
+            if prefix + 'bias' in state_dict:
+                self.bias = torch.nn.Parameter(state_dict[prefix + 'bias'].to(self.dummy))
+
+            del self.dummy
+        elif hasattr(self, 'dummy'):
+            if prefix + 'weight' in state_dict:
+                self.weight = ForgeParams4bit(
+                    state_dict[prefix + 'weight'].to(self.dummy),
+                    requires_grad=False,
+                    compress_statistics=True,
+                    quant_type=self.quant_type,
+                    quant_storage=torch.uint8,
+                    module=self,
+                )
+                self.quant_state = self.weight.quant_state
+
+            if prefix + 'bias' in state_dict:
+                self.bias = torch.nn.Parameter(state_dict[prefix + 'bias'].to(self.dummy))
+
+            del self.dummy
+        else:
+            super()._load_from_state_dict(state_dict, prefix, local_metadata, strict, missing_keys, unexpected_keys, error_msgs)
+
+current_device = None
+current_dtype = None
+current_manual_cast_enabled = False
+current_bnb_dtype = None
+
+class OPS(comfy.ops.manual_cast):
+    class Linear(ForgeLoader4Bit):
+        def __init__(self, *args, device=None, dtype=None, **kwargs):
+            super().__init__(device=device, dtype=dtype, quant_type=current_bnb_dtype)
+            self.parameters_manual_cast = current_manual_cast_enabled
+
+        def forward(self, x):
+            self.weight.quant_state = self.quant_state
+
+            if self.bias is not None and self.bias.dtype != x.dtype:
+                # Maybe this can also be set to all non-bnb ops since the cost is very low.
+                # And it only invokes one time, and most linear does not have bias
+                self.bias.data = self.bias.data.to(x.dtype)
+
+            if not self.parameters_manual_cast:
+                return functional_linear_4bits(x, self.weight, self.bias)
+            elif not self.weight.bnb_quantized:
+                assert x.device.type == 'cuda', 'BNB Must Use CUDA as Computation Device!'
+                layer_original_device = self.weight.device
+                self.weight = self.weight._quantize(x.device)
+                bias = self.bias.to(x.device) if self.bias is not None else None
+                out = functional_linear_4bits(x, self.weight, bias)
+                self.weight = self.weight.to(layer_original_device)
+                return out
+            else:
+                weight, bias, signal = weights_manual_cast(self, x, skip_weight_dtype=True, skip_bias_dtype=True)
+                with main_stream_worker(weight, bias, signal):
+                    return functional_linear_4bits(x, weight, bias)

py/brushnet/powerpaint_utils.py

@@ -380,7 +380,7 @@ class EmbeddingLayerWithFixes(nn.Module):
 
         return torch.cat(new_embedding, dim=0)
 
-    def forward(self, input_ids: torch.Tensor, external_embeddings: Optional[List[dict]] = None):
+    def forward(self, input_ids: torch.Tensor, external_embeddings: Optional[List[dict]] = None, out_dtype = None):
         """The forward function.
 
         Args:
@@ -397,7 +397,7 @@ class EmbeddingLayerWithFixes(nn.Module):
             input_ids = input_ids.unsqueeze(0)
 
         if external_embeddings is None and not self.external_embeddings:
-            return self.wrapped(input_ids)
+            return self.wrapped(input_ids, out_dtype=out_dtype)
 
         input_ids_fwd = self.replace_input_ids(input_ids)
         inputs_embeds = self.wrapped(input_ids_fwd)
@@ -416,7 +416,7 @@ class EmbeddingLayerWithFixes(nn.Module):
                 new_embedding = self.replace_embeddings(input_id, new_embedding, external_embedding)
             vecs.append(new_embedding)
 
-        return torch.stack(vecs)
+        return torch.stack(vecs).to(out_dtype)
 
 
 def add_tokens(

py/config.py

@@ -228,6 +228,14 @@ IPADAPTER_MODELS = {
             "model_url": "https://huggingface.co/h94/IP-Adapter/resolve/main/sdxl_models/ip-adapter-plus_sdxl_vit-h.safetensors"
         }
     },
+    "PLUS (kolors genernal)":{
+        "sd15":{
+            "model_url":""
+        },
+        "sdxl":{
+            "model_url":"https://huggingface.co/Kwai-Kolors/Kolors-IP-Adapter-Plus/resolve/main/ip_adapter_plus_general.bin"
+        }
+    },
     "PLUS FACE (portraits)": {
         "sd15": {
             "model_url": "https://huggingface.co/h94/IP-Adapter/resolve/main/models/ip-adapter-plus-face_sd15.safetensors"
@@ -274,6 +282,14 @@ IPADAPTER_MODELS = {
             "lora_url": "https://huggingface.co/h94/IP-Adapter-FaceID/resolve/main/ip-adapter-faceid-plusv2_sdxl_lora.safetensors"
         }
     },
+    "FACEID PLUS KOLORS":{
+        "sd15":{
+
+        },
+        "sdxl":{
+            "model_url":"https://huggingface.co/Kwai-Kolors/Kolors-IP-Adapter-FaceID-Plus/resolve/main/ipa-faceid-plus.bin"
+        }
+    },
     "FACEID PORTRAIT (style transfer)": {
         "sd15": {
             "model_url": "https://huggingface.co/h94/IP-Adapter-FaceID/resolve/main/ip-adapter-faceid-portrait-v11_sd15.bin",
@@ -282,11 +298,11 @@ IPADAPTER_MODELS = {
             "model_url": "https://huggingface.co/h94/IP-Adapter-FaceID/resolve/main/ip-adapter-faceid-portrait_sdxl.bin",
         }
     },
-    "FACEID PORTRAIT UNNORM - SDXL only (strong)":{
-        "SD15":{
+    "FACEID PORTRAIT UNNORM - SDXL only (strong)": {
+        "sd15": {
             "model_url":""
         },
-        "SDXL":{
+        "sdxl": {
             "model_url": "https://huggingface.co/h94/IP-Adapter-FaceID/resolve/main/ip-adapter-faceid-portrait_sdxl_unnorm.bin",
         }
     },
@@ -299,6 +315,14 @@ IPADAPTER_MODELS = {
         }
     }
 }
+IPADAPTER_CLIPVISION_MODELS = {
+    "clip-vit-large-patch14-336":{
+        "model_url": "https://huggingface.co/openai/clip-vit-large-patch14-336/resolve/main/pytorch_model.bin"
+    },
+    "clip-vit-h-14-laion2B-s32B-b79K":{
+        "model_url": "https://huggingface.co/laion/CLIP-ViT-H-14-laion2B-s32B-b79K/resolve/main/open_clip_pytorch_model.safetensors"
+    }
+}
 
 # dynamiCrafter
 DYNAMICRAFTER_DIR = os.path.join(folder_paths.models_dir, "dynamicrafter_models")
@@ -325,6 +349,9 @@ HUMANPARSING_MODELS = {
     "parsing_lip": {
         "model_url": "https://huggingface.co/levihsu/OOTDiffusion/resolve/main/checkpoints/humanparsing/parsing_lip.onnx",
     },
+    "human-parts":{
+        "model_url":"https://huggingface.co/Metal3d/deeplabv3p-resnet50-human/resolve/main/deeplabv3p-resnet50-human.onnx",
+    }
 }
 
 #mediapipe

py/dit/hunyuanDiT/LICENSE-HYDIT

@@ -1,74 +0,0 @@
-TENCENT HUNYUAN COMMUNITY LICENSE AGREEMENT
-Tencent Hunyuan Release Date: 2024/5/14
-By clicking to agree or by using, reproducing, modifying, distributing, performing or displaying any portion or element of the Tencent Hunyuan Works, including via any Hosted Service, You will be deemed to have recognized and accepted the content of this Agreement, which is effective immediately.
-1.	DEFINITIONS.
-a.	“Acceptable Use Policy” shall mean the policy made available by Tencent as set forth in the Exhibit A.
-b.	“Agreement” shall mean the terms and conditions for use, reproduction, distribution, modification, performance and displaying of the Hunyuan Works or any portion or element thereof set forth herein.
-c.	“Documentation” shall mean the specifications, manuals and documentation for Tencent Hunyuan made publicly available by Tencent.
-d.	“Hosted Service” shall mean a hosted service offered via an application programming interface (API), web access, or any other electronic or remote means.
-e.	“Licensee,” “You” or “Your” shall mean a natural person or legal entity exercising the rights granted by this Agreement and/or using the Tencent Hunyuan Works for any purpose and in any field of use.
-f.	“Materials” shall mean, collectively, Tencent’s proprietary Tencent Hunyuan and Documentation (and any portion thereof) as made available by Tencent under this Agreement.
-g.	“Model Derivatives” shall mean all: (i) modifications to Tencent Hunyuan or any Model Derivative of Tencent Hunyuan; (ii) works based on Tencent Hunyuan or any Model Derivative of Tencent Hunyuan; or (iii) any other machine learning model which is created by transfer of patterns of the weights, parameters, operations, or Output of Tencent Hunyuan or any Model Derivative of Tencent Hunyuan, to that model in order to cause that model to perform similarly to Tencent Hunyuan or a Model Derivative of Tencent Hunyuan, including distillation methods, methods that use intermediate data representations, or methods based on the generation of synthetic data Outputs by Tencent Hunyuan or a Model Derivative of Tencent Hunyuan for training that model. For clarity, Outputs by themselves are not deemed Model Derivatives.
-h.	“Output” shall mean the information and/or content output of Tencent Hunyuan or a Model Derivative that results from operating or otherwise using Tencent Hunyuan or a Model Derivative, including via a Hosted Service.
-i.	“Tencent,” “We” or “Us” shall mean THL A29 Limited.
-j.	“Tencent Hunyuan” shall mean the large language models, image/video/audio/3D generation models, and multimodal large language models and their software and algorithms, including trained model weights, parameters (including optimizer states), machine-learning model code, inference-enabling code, training-enabling code, fine-tuning enabling code and other elements of the foregoing made publicly available by Us at  https://huggingface.co/Tencent-Hunyuan/HunyuanDiT and https://github.com/Tencent/HunyuanDiT .
-k.	“Tencent Hunyuan Works” shall mean: (i) the Materials; (ii) Model Derivatives; and (iii) all derivative works thereof.
-l.	“Third Party” or “Third Parties” shall mean individuals or legal entities that are not under common control with Us or You.
-m.	“including” shall mean including but not limited to.
-2.	GRANT OF RIGHTS.
-We grant You a non-exclusive, worldwide, non-transferable and royalty-free limited license under Tencent’s intellectual property or other rights owned by Us embodied in or utilized by the Materials to use, reproduce, distribute, create derivative works of (including Model Derivatives), and make modifications to the Materials, only in accordance with the terms of this Agreement and the Acceptable Use Policy, and You must not violate (or encourage or permit anyone else to violate) any term of this Agreement or the Acceptable Use Policy.
-3.	DISTRIBUTION.
-You may, subject to Your compliance with this Agreement, distribute or make available to Third Parties the Tencent Hunyuan Works, provided that You meet all of the following conditions:
-a.	You must provide all such Third Party recipients of the Tencent Hunyuan Works or products or services using them a copy of this Agreement;
-b.	You must cause any modified files to carry prominent notices stating that You changed the files;
-c.	You are encouraged to: (i) publish at least one technology introduction blogpost or one public statement expressing Your experience of using the Tencent Hunyuan Works; and (ii) mark the products or services developed by using the Tencent Hunyuan Works to indicate that the product/service is “Powered by Tencent Hunyuan”; and
-d.	All distributions to Third Parties (other than through a Hosted Service) must be accompanied by a “Notice” text file that contains the following notice: “Tencent Hunyuan is licensed under the Tencent Hunyuan Community License Agreement, Copyright © 2024 Tencent. All Rights Reserved. The trademark rights of “Tencent Hunyuan” are owned by Tencent or its affiliate.”
-You may add Your own copyright statement to Your modifications and, except as set forth in this Section and in Section 5, may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Model Derivatives as a whole, provided Your use, reproduction, modification, distribution, performance and display of the work otherwise complies with the terms and conditions of this Agreement. If You receive Tencent Hunyuan Works from a Licensee as part of an integrated end user product, then this Section 3 of this Agreement will not apply to You.
-4.	ADDITIONAL COMMERCIAL TERMS.
-If, on the Tencent Hunyuan version release date, the monthly active users of all products or services made available by or for Licensee is greater than 100 million monthly active users in the preceding calendar month, You must request a license from Tencent, which Tencent may grant to You in its sole discretion, and You are not authorized to exercise any of the rights under this Agreement unless or until Tencent otherwise expressly grants You such rights.
-5.	RULES OF USE.
-a.	Your use of the Tencent Hunyuan Works must comply with applicable laws and regulations (including trade compliance laws and regulations) and adhere to the Acceptable Use Policy for the Tencent Hunyuan Works, which is hereby incorporated by reference into this Agreement. You must include the use restrictions referenced in these Sections 5(a) and 5(b) as an enforceable provision in any agreement (e.g., license agreement, terms of use, etc.) governing the use and/or distribution of Tencent Hunyuan Works and You must provide notice to subsequent users to whom You distribute that Tencent Hunyuan Works are subject to the use restrictions in these Sections 5(a) and 5(b).
-b.	You must not use the Tencent Hunyuan Works or any Output or results of the Tencent Hunyuan Works to improve any other large language model (other than Tencent Hunyuan or Model Derivatives thereof).
-6.	INTELLECTUAL PROPERTY.
-a.	Subject to Tencent’s ownership of Tencent Hunyuan Works made by or for Tencent and intellectual property rights therein, conditioned upon Your compliance with the terms and conditions of this Agreement, as between You and Tencent, You will be the owner of any derivative works and modifications of the Materials and any Model Derivatives that are made by or for You.
-b.	No trademark licenses are granted under this Agreement, and in connection with the Tencent Hunyuan Works, Licensee may not use any name or mark owned by or associated with Tencent or any of its affiliates, except as required for reasonable and customary use in describing and distributing the Tencent Hunyuan Works. Tencent hereby grants You a license to use “Tencent Hunyuan” (the “Mark”) solely as required to comply with the provisions of Section 3(c), provided that You comply with any applicable laws related to trademark protection. All goodwill arising out of Your use of the Mark will inure to the benefit of Tencent.
-c.	If You commence a lawsuit or other proceedings (including a cross-claim or counterclaim in a lawsuit) against Us or any person or entity alleging that the Materials or any Output, or any portion of any of the foregoing, infringe any intellectual property or other right owned or licensable by You, then all licenses granted to You under this Agreement shall terminate as of the date such lawsuit or other proceeding is filed. You will defend, indemnify and hold harmless Us from and against any claim by any Third Party arising out of or related to Your or the Third Party’s use or distribution of the Tencent Hunyuan Works.
-d.	Tencent claims no rights in Outputs You generate. You and Your users are solely responsible for Outputs and their subsequent uses.
-7.	DISCLAIMERS OF WARRANTY AND LIMITATIONS OF LIABILITY.
-a.	We are not obligated to support, update, provide training for, or develop any further version of the Tencent Hunyuan Works or to grant any license thereto.
-b.	UNLESS AND ONLY TO THE EXTENT REQUIRED BY APPLICABLE LAW, THE TENCENT HUNYUAN WORKS AND ANY OUTPUT AND RESULTS THEREFROM ARE PROVIDED “AS IS” WITHOUT ANY EXPRESS OR IMPLIED WARRANTIES OF ANY KIND INCLUDING ANY WARRANTIES OF TITLE, MERCHANTABILITY, NONINFRINGEMENT, COURSE OF DEALING, USAGE OF TRADE, OR FITNESS FOR A PARTICULAR PURPOSE. YOU ARE SOLELY RESPONSIBLE FOR DETERMINING THE APPROPRIATENESS OF USING, REPRODUCING, MODIFYING, PERFORMING, DISPLAYING OR DISTRIBUTING ANY OF THE TENCENT HUNYUAN WORKS OR OUTPUTS AND ASSUME ANY AND ALL RISKS ASSOCIATED WITH YOUR OR A THIRD PARTY’S USE OR DISTRIBUTION OF ANY OF THE TENCENT HUNYUAN WORKS OR OUTPUTS AND YOUR EXERCISE OF RIGHTS AND PERMISSIONS UNDER THIS AGREEMENT.
-c.	TO THE FULLEST EXTENT PERMITTED BY APPLICABLE LAW, IN NO EVENT SHALL TENCENT OR ITS AFFILIATES BE LIABLE UNDER ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, TORT, NEGLIGENCE, PRODUCTS LIABILITY, OR OTHERWISE, FOR ANY DAMAGES, INCLUDING ANY DIRECT, INDIRECT, SPECIAL, INCIDENTAL, EXEMPLARY, CONSEQUENTIAL OR PUNITIVE DAMAGES, OR LOST PROFITS OF ANY KIND ARISING FROM THIS AGREEMENT OR RELATED TO ANY OF THE TENCENT HUNYUAN WORKS OR OUTPUTS, EVEN IF TENCENT OR ITS AFFILIATES HAVE BEEN ADVISED OF THE POSSIBILITY OF ANY OF THE FOREGOING.
-8.	SURVIVAL AND TERMINATION.
-a.	The term of this Agreement shall commence upon Your acceptance of this Agreement or access to the Materials and will continue in full force and effect until terminated in accordance with the terms and conditions herein.
-b.	We may terminate this Agreement if You breach any of the terms or conditions of this Agreement. Upon termination of this Agreement, You must promptly delete and cease use of the Tencent Hunyuan Works. Sections 6(a), 6(c), 7 and 9 shall survive the termination of this Agreement.
-9.	GOVERNING LAW AND JURISDICTION.
-a.	This Agreement and any dispute arising out of or relating to it will be governed by the laws of the Hong Kong Special Administrative Region of the People’s Republic of China, without regard to conflict of law principles, and the UN Convention on Contracts for the International Sale of Goods does not apply to this Agreement.
-b.	Exclusive jurisdiction and venue for any dispute arising out of or relating to this Agreement will be a court of competent jurisdiction in the Hong Kong Special Administrative Region of the People’s Republic of China, and Tencent and Licensee consent to the exclusive jurisdiction of such court with respect to any such dispute.
- 
-
-EXHIBIT A
-ACCEPTABLE USE POLICY
-
-Tencent reserves the right to update this Acceptable Use Policy from time to time.
-Last modified: 2024/5/14
-
-Tencent endeavors to promote safe and fair use of its tools and features, including Tencent Hunyuan. You agree not to use Tencent Hunyuan or Model Derivatives:
-1.	In any way that violates any applicable national, federal, state, local, international or any other law or regulation;
-2.	To harm Yourself or others;
-3.	To repurpose or distribute output from Tencent Hunyuan or any Model Derivatives to harm Yourself or others;
-4.	To override or circumvent the safety guardrails and safeguards We have put in place;
-5.	For the purpose of exploiting, harming or attempting to exploit or harm minors in any way;
-6.	To generate or disseminate verifiably false information and/or content with the purpose of harming others or influencing elections;
-7.	To generate or facilitate false online engagement, including fake reviews and other means of fake online engagement;
-8.	To intentionally defame, disparage or otherwise harass others;
-9.	To generate and/or disseminate malware (including ransomware) or any other content to be used for the purpose of harming electronic systems;
-10.	To generate or disseminate personal identifiable information with the purpose of harming others;
-11.	To generate or disseminate information (including images, code, posts, articles), and place the information in any public context (including –through the use of bot generated tweets), without expressly and conspicuously identifying that the information and/or content is machine generated;
-12.	To impersonate another individual without consent, authorization, or legal right;
-13.	To make high-stakes automated decisions in domains that affect an individual’s safety, rights or wellbeing (e.g., law enforcement, migration, medicine/health, management of critical infrastructure, safety components of products, essential services, credit, employment, housing, education, social scoring, or insurance);
-14.	In a manner that violates or disrespects the social ethics and moral standards of other countries or regions;
-15.	To perform, facilitate, threaten, incite, plan, promote or encourage violent extremism or terrorism;
-16.	For any use intended to discriminate against or harm individuals or groups based on protected characteristics or categories, online or offline social behavior or known or predicted personal or personality characteristics;
-17.	To intentionally exploit any of the vulnerabilities of a specific group of persons based on their age, social, physical or mental characteristics, in order to materially distort the behavior of a person pertaining to that group in a manner that causes or is likely to cause that person or another person physical or psychological harm;
-18.	For military purposes;
-19.	To engage in the unauthorized or unlicensed practice of any profession including, but not limited to, financial, legal, medical/health, or other professional practices.

py/dit/hunyuanDiT/config.py

@@ -1,46 +0,0 @@
-"""List of all HYDiT model types / settings"""
-sampling_settings = {
-	"beta_schedule" : "linear",
-	"linear_start"  : 0.00085,
-	"linear_end"    : 0.03,
-	"timesteps"     : 1000,
-}
-
-from argparse import Namespace
-hydit_args = Namespace(**{ # normally from argparse
-	"infer_mode": "torch",
-	"norm": "layer",
-	"learn_sigma": True,
-	"text_states_dim": 1024,
-	"text_states_dim_t5": 2048,
-	"text_len": 77,
-	"text_len_t5": 256,
-})
-
-hydit_conf = {
-	"G/2": { # Seems to be the main one
-		"unet_config": {
-			"depth"       :   40,
-			"num_heads"   :   16,
-			"patch_size"  :    2,
-			"hidden_size" : 1408,
-			"mlp_ratio" : 4.3637,
-			"input_size": (1024//8, 1024//8),
-			"args": hydit_args,
-		},
-		"sampling_settings" : sampling_settings,
-	},
-}
-
-dtypes = ["default", "auto (comfy)", "FP32", "FP16", "BF16"]
-devices = ["auto", "cpu", "gpu"]
-
-
-# these are the same as regular DiT, I think
-from ..config import dit_conf
-for name in ["XL/2", "L/2", "B/2"]:
-	hydit_conf[name] = {
-		"unet_config": dit_conf[name]["unet_config"].copy(),
-		"sampling_settings": sampling_settings,
-	}
-	hydit_conf[name]["unet_config"]["args"] = hydit_args

py/dit/hunyuanDiT/config_clip.json

@@ -1,34 +0,0 @@
-{
-  "_name_or_path": "hfl/chinese-roberta-wwm-ext-large",
-  "architectures": [
-    "BertModel"
-  ],
-  "attention_probs_dropout_prob": 0.1,
-  "bos_token_id": 0,
-  "classifier_dropout": null,
-  "directionality": "bidi",
-  "eos_token_id": 2,
-  "hidden_act": "gelu",
-  "hidden_dropout_prob": 0.1,
-  "hidden_size": 1024,
-  "initializer_range": 0.02,
-  "intermediate_size": 4096,
-  "layer_norm_eps": 1e-12,
-  "max_position_embeddings": 512,
-  "model_type": "bert",
-  "num_attention_heads": 16,
-  "num_hidden_layers": 24,
-  "output_past": true,
-  "pad_token_id": 0,
-  "pooler_fc_size": 768,
-  "pooler_num_attention_heads": 12,
-  "pooler_num_fc_layers": 3,
-  "pooler_size_per_head": 128,
-  "pooler_type": "first_token_transform",
-  "position_embedding_type": "absolute",
-  "torch_dtype": "float32",
-  "transformers_version": "4.22.1",
-  "type_vocab_size": 2,
-  "use_cache": true,
-  "vocab_size": 47020
-}

py/dit/hunyuanDiT/config_mt5.json

@@ -1,33 +0,0 @@
-{
-  "_name_or_path": "mt5",
-  "architectures": [
-    "MT5EncoderModel"
-  ],
-  "classifier_dropout": 0.0,
-  "d_ff": 5120,
-  "d_kv": 64,
-  "d_model": 2048,
-  "decoder_start_token_id": 0,
-  "dense_act_fn": "gelu_new",
-  "dropout_rate": 0.1,
-  "eos_token_id": 1,
-  "feed_forward_proj": "gated-gelu",
-  "initializer_factor": 1.0,
-  "is_encoder_decoder": true,
-  "is_gated_act": true,
-  "layer_norm_epsilon": 1e-06,
-  "model_type": "mt5",
-  "num_decoder_layers": 24,
-  "num_heads": 32,
-  "num_layers": 24,
-  "output_past": true,
-  "pad_token_id": 0,
-  "relative_attention_max_distance": 128,
-  "relative_attention_num_buckets": 32,
-  "tie_word_embeddings": false,
-  "tokenizer_class": "T5Tokenizer",
-  "torch_dtype": "float16",
-  "transformers_version": "4.40.2",
-  "use_cache": true,
-  "vocab_size": 250112
-}

py/dit/hunyuanDiT/loader.py

@@ -1,240 +0,0 @@
-
-
-import os
-import torch
-import comfy.supported_models_base
-import comfy.latent_formats
-import comfy.model_patcher
-import comfy.model_base
-import comfy.utils
-import comfy.conds
-
-from comfy import model_management
-from tqdm import tqdm
-from transformers import AutoTokenizer, modeling_utils
-from transformers import T5Config, T5EncoderModel, BertConfig, BertModel
-
-class EXM_HYDiT(comfy.supported_models_base.BASE):
-    unet_config = {}
-    unet_extra_config = {}
-    latent_format = comfy.latent_formats.SDXL
-
-    def __init__(self, model_conf):
-        self.unet_config = model_conf.get("unet_config", {})
-        self.sampling_settings = model_conf.get("sampling_settings", {})
-        self.latent_format = self.latent_format()
-        # UNET is handled by extension
-        self.unet_config["disable_unet_model_creation"] = True
-
-    def model_type(self, state_dict, prefix=""):
-        return comfy.model_base.ModelType.V_PREDICTION
-
-
-class EXM_HYDiT_Model(comfy.model_base.BaseModel):
-    def __init__(self, *args, **kwargs):
-        super().__init__(*args, **kwargs)
-
-    def extra_conds(self, **kwargs):
-        out = super().extra_conds(**kwargs)
-
-        for name in ["context_t5", "context_mask", "context_t5_mask"]:
-            out[name] = comfy.conds.CONDRegular(kwargs[name])
-
-        src_size_cond = kwargs.get("src_size_cond", None)
-        if src_size_cond is not None:
-            out["src_size_cond"] = comfy.conds.CONDRegular(torch.tensor(src_size_cond))
-
-        return out
-
-
-def load_hydit(model_path, model_conf):
-    state_dict = comfy.utils.load_torch_file(model_path)
-    state_dict = state_dict.get("model", state_dict)
-
-    parameters = comfy.utils.calculate_parameters(state_dict)
-    unet_dtype = model_management.unet_dtype(model_params=parameters)
-    load_device = comfy.model_management.get_torch_device()
-    offload_device = comfy.model_management.unet_offload_device()
-
-    # ignore fp8/etc and use directly for now
-    manual_cast_dtype = model_management.unet_manual_cast(unet_dtype, load_device)
-    if manual_cast_dtype:
-        print(f"HunYuanDiT: falling back to {manual_cast_dtype}")
-        unet_dtype = manual_cast_dtype
-
-    model_conf = EXM_HYDiT(model_conf)
-    model = EXM_HYDiT_Model(
-        model_conf,
-        model_type=comfy.model_base.ModelType.V_PREDICTION,
-        device=model_management.get_torch_device()
-    )
-
-    from .models.models import HunYuanDiT
-    model.diffusion_model = HunYuanDiT(
-        **model_conf.unet_config,
-        log_fn=tqdm.write,
-    )
-
-    model.diffusion_model.load_state_dict(state_dict)
-    model.diffusion_model.dtype = unet_dtype
-    model.diffusion_model.eval()
-    model.diffusion_model.to(unet_dtype)
-
-    model_patcher = comfy.model_patcher.ModelPatcher(
-        model,
-        load_device=load_device,
-        offload_device=offload_device,
-        current_device="cpu",
-    )
-    return model_patcher
-
-
-# CLIP Model
-class hyCLIPModel(torch.nn.Module):
-	def __init__(self, textmodel_json_config=None, device="cpu", max_length=77, freeze=True, dtype=None):
-		super().__init__()
-		self.device = device
-		self.dtype = dtype
-		self.max_length = max_length
-		if textmodel_json_config is None:
-			textmodel_json_config = os.path.join(
-				os.path.dirname(os.path.realpath(__file__)),
-				f"config_clip.json"
-			)
-		config = BertConfig.from_json_file(textmodel_json_config)
-		with modeling_utils.no_init_weights():
-			self.transformer = BertModel(config)
-		self.to(dtype)
-		if freeze:
-			self.freeze()
-
-	def freeze(self):
-		self.transformer = self.transformer.eval()
-		for param in self.parameters():
-			param.requires_grad = False
-
-	def load_sd(self, sd):
-		return self.transformer.load_state_dict(sd, strict=False)
-
-	def to(self, *args, **kwargs):
-		return self.transformer.to(*args, **kwargs)
-
-class EXM_HyDiT_Tenc_Temp:
-	def __init__(self, no_init=False, device="cpu", dtype=None, model_class="mT5", *kwargs):
-		if no_init:
-			return
-
-		size = 8 if model_class == "mT5" else 2
-		if dtype == torch.float32:
-			size *= 2
-		size *= (1024**3)
-
-		if device == "auto":
-			self.load_device = model_management.text_encoder_device()
-			self.offload_device = model_management.text_encoder_offload_device()
-			self.init_device = "cpu"
-		elif device == "cpu":
-			size = 0 # doesn't matter
-			self.load_device = "cpu"
-			self.offload_device = "cpu"
-			self.init_device="cpu"
-		elif device.startswith("cuda"):
-			print("Direct CUDA device override!\nVRAM will not be freed by default.")
-			size = 0 # not used
-			self.load_device = device
-			self.offload_device = device
-			self.init_device = device
-		else:
-			self.load_device = model_management.get_torch_device()
-			self.offload_device = "cpu"
-			self.init_device="cpu"
-
-		self.dtype = dtype
-		self.device = self.load_device
-		if model_class == "mT5":
-			self.cond_stage_model = mT5Model(
-				device         = self.load_device,
-				dtype          = self.dtype,
-			)
-			tokenizer_args = {"subfolder": "t2i/mt5"} # web
-			tokenizer_path = os.path.join( # local
-				os.path.dirname(os.path.realpath(__file__)),
-				"mt5_tokenizer",
-			)
-		else:
-			self.cond_stage_model = hyCLIPModel(
-				device         = self.load_device,
-				dtype          = self.dtype,
-			)
-			tokenizer_args = {"subfolder": "t2i/tokenizer",} # web
-			tokenizer_path = os.path.join( # local
-				os.path.dirname(os.path.realpath(__file__)),
-				"tokenizer",
-			)
-		# self.tokenizer = AutoTokenizer.from_pretrained(
-			# "Tencent-Hunyuan/HunyuanDiT",
-			# **tokenizer_args
-		# )
-		self.tokenizer = AutoTokenizer.from_pretrained(tokenizer_path)
-		self.patcher = comfy.model_patcher.ModelPatcher(
-			self.cond_stage_model,
-			load_device    = self.load_device,
-			offload_device = self.offload_device,
-			current_device = self.load_device,
-			size           = size,
-		)
-
-	def clone(self):
-		n = EXM_HyDiT_Tenc_Temp(no_init=True)
-		n.patcher = self.patcher.clone()
-		n.cond_stage_model = self.cond_stage_model
-		n.tokenizer = self.tokenizer
-		return n
-
-	def load_sd(self, sd):
-		return self.cond_stage_model.load_sd(sd)
-
-	def get_sd(self):
-		return self.cond_stage_model.state_dict()
-
-	def load_model(self):
-		if self.load_device != "cpu":
-			model_management.load_model_gpu(self.patcher)
-		return self.patcher
-
-	def add_patches(self, patches, strength_patch=1.0, strength_model=1.0):
-		return self.patcher.add_patches(patches, strength_patch, strength_model)
-
-	def get_key_patches(self):
-		return self.patcher.get_key_patches()
-
-# MT5 model
-class mT5Model(torch.nn.Module):
-	def __init__(self, textmodel_json_config=None, device="cpu", max_length=256, freeze=True, dtype=None):
-		super().__init__()
-		self.device = device
-		self.dtype = dtype
-		self.max_length = max_length
-		if textmodel_json_config is None:
-			textmodel_json_config = os.path.join(
-				os.path.dirname(os.path.realpath(__file__)),
-				f"config_mt5.json"
-			)
-		config = T5Config.from_json_file(textmodel_json_config)
-		with modeling_utils.no_init_weights():
-			self.transformer = T5EncoderModel(config)
-		self.to(dtype)
-		if freeze:
-			self.freeze()
-
-	def freeze(self):
-		self.transformer = self.transformer.eval()
-		for param in self.parameters():
-			param.requires_grad = False
-
-	def load_sd(self, sd):
-		return self.transformer.load_state_dict(sd, strict=False)
-
-	def to(self, *args, **kwargs):
-		return self.transformer.to(*args, **kwargs)
-

py/dit/hunyuanDiT/models/attn_layers.py

@@ -1,377 +0,0 @@
-import torch
-import torch.nn as nn
-from typing import Tuple, Union, Optional
-
-try:
-    import flash_attn
-    if hasattr(flash_attn, '__version__') and int(flash_attn.__version__[0]) == 2:
-        from flash_attn.flash_attn_interface import flash_attn_kvpacked_func
-        from flash_attn.modules.mha import FlashSelfAttention, FlashCrossAttention
-    else:
-        from flash_attn.flash_attn_interface import flash_attn_unpadded_kvpacked_func
-        from flash_attn.modules.mha import FlashSelfAttention, FlashCrossAttention
-except Exception as e:
-    print(f'flash_attn import failed: {e}')
-
-
-def reshape_for_broadcast(freqs_cis: Union[torch.Tensor, Tuple[torch.Tensor]], x: torch.Tensor, head_first=False):
-    """
-    Reshape frequency tensor for broadcasting it with another tensor.
-
-    This function reshapes the frequency tensor to have the same shape as the target tensor 'x'
-    for the purpose of broadcasting the frequency tensor during element-wise operations.
-
-    Args:
-        freqs_cis (Union[torch.Tensor, Tuple[torch.Tensor]]): Frequency tensor to be reshaped.
-        x (torch.Tensor): Target tensor for broadcasting compatibility.
-        head_first (bool): head dimension first (except batch dim) or not.
-
-    Returns:
-        torch.Tensor: Reshaped frequency tensor.
-
-    Raises:
-        AssertionError: If the frequency tensor doesn't match the expected shape.
-        AssertionError: If the target tensor 'x' doesn't have the expected number of dimensions.
-    """
-    ndim = x.ndim
-    assert 0 <= 1 < ndim
-
-    if isinstance(freqs_cis, tuple):
-        # freqs_cis: (cos, sin) in real space
-        if head_first:
-            assert freqs_cis[0].shape == (x.shape[-2], x.shape[-1]), f'freqs_cis shape {freqs_cis[0].shape} does not match x shape {x.shape}'
-            shape = [d if i == ndim - 2 or i == ndim - 1 else 1 for i, d in enumerate(x.shape)]
-        else:
-            assert freqs_cis[0].shape == (x.shape[1], x.shape[-1]), f'freqs_cis shape {freqs_cis[0].shape} does not match x shape {x.shape}'
-            shape = [d if i == 1 or i == ndim - 1 else 1 for i, d in enumerate(x.shape)]
-        return freqs_cis[0].view(*shape), freqs_cis[1].view(*shape)
-    else:
-        # freqs_cis: values in complex space
-        if head_first:
-            assert freqs_cis.shape == (x.shape[-2], x.shape[-1]), f'freqs_cis shape {freqs_cis.shape} does not match x shape {x.shape}'
-            shape = [d if i == ndim - 2 or i == ndim - 1 else 1 for i, d in enumerate(x.shape)]
-        else:
-            assert freqs_cis.shape == (x.shape[1], x.shape[-1]), f'freqs_cis shape {freqs_cis.shape} does not match x shape {x.shape}'
-            shape = [d if i == 1 or i == ndim - 1 else 1 for i, d in enumerate(x.shape)]
-        return freqs_cis.view(*shape)
-
-
-def rotate_half(x):
-    x_real, x_imag = x.float().reshape(*x.shape[:-1], -1, 2).unbind(-1)  # [B, S, H, D//2]
-    return torch.stack([-x_imag, x_real], dim=-1).flatten(3)
-
-
-def apply_rotary_emb(
-        xq: torch.Tensor,
-        xk: Optional[torch.Tensor],
-        freqs_cis: Union[torch.Tensor, Tuple[torch.Tensor]],
-        head_first: bool = False,
-) -> Tuple[torch.Tensor, torch.Tensor]:
-    """
-    Apply rotary embeddings to input tensors using the given frequency tensor.
-
-    This function applies rotary embeddings to the given query 'xq' and key 'xk' tensors using the provided
-    frequency tensor 'freqs_cis'. The input tensors are reshaped as complex numbers, and the frequency tensor
-    is reshaped for broadcasting compatibility. The resulting tensors contain rotary embeddings and are
-    returned as real tensors.
-
-    Args:
-        xq (torch.Tensor): Query tensor to apply rotary embeddings. [B, S, H, D]
-        xk (torch.Tensor): Key tensor to apply rotary embeddings.   [B, S, H, D]
-        freqs_cis (Union[torch.Tensor, Tuple[torch.Tensor]]): Precomputed frequency tensor for complex exponentials.
-        head_first (bool): head dimension first (except batch dim) or not.
-
-    Returns:
-        Tuple[torch.Tensor, torch.Tensor]: Tuple of modified query tensor and key tensor with rotary embeddings.
-
-    """
-    xk_out = None
-    if isinstance(freqs_cis, tuple):
-        cos, sin = reshape_for_broadcast(freqs_cis, xq, head_first)    # [S, D]
-        cos, sin = cos.to(xq.device), sin.to(xq.device)
-        xq_out = (xq.float() * cos + rotate_half(xq.float()) * sin).type_as(xq)
-        if xk is not None:
-            xk_out = (xk.float() * cos + rotate_half(xk.float()) * sin).type_as(xk)
-    else:
-        xq_ = torch.view_as_complex(xq.float().reshape(*xq.shape[:-1], -1, 2))  # [B, S, H, D//2]
-        freqs_cis = reshape_for_broadcast(freqs_cis, xq_, head_first).to(xq.device)   # [S, D//2] --> [1, S, 1, D//2]
-        xq_out = torch.view_as_real(xq_ * freqs_cis).flatten(3).type_as(xq)
-        if xk is not None:
-            xk_ = torch.view_as_complex(xk.float().reshape(*xk.shape[:-1], -1, 2))  # [B, S, H, D//2]
-            xk_out = torch.view_as_real(xk_ * freqs_cis).flatten(3).type_as(xk)
-
-    return xq_out, xk_out
-
-
-class FlashSelfMHAModified(nn.Module):
-    """
-    Use QK Normalization.
-    """
-    def __init__(self,
-                 dim,
-                 num_heads,
-                 qkv_bias=True,
-                 qk_norm=False,
-                 attn_drop=0.0,
-                 proj_drop=0.0,
-                 device=None,
-                 dtype=None,
-                 norm_layer=nn.LayerNorm,
-                 ):
-        factory_kwargs = {'device': device, 'dtype': dtype}
-        super().__init__()
-        self.dim = dim
-        self.num_heads = num_heads
-        assert self.dim % num_heads == 0, "self.kdim must be divisible by num_heads"
-        self.head_dim = self.dim // num_heads
-        assert self.head_dim % 8 == 0 and self.head_dim <= 128, "Only support head_dim <= 128 and divisible by 8"
-
-        self.Wqkv = nn.Linear(dim, 3 * dim, bias=qkv_bias, **factory_kwargs)
-        # TODO: eps should be 1 / 65530 if using fp16
-        self.q_norm = norm_layer(self.head_dim, elementwise_affine=True, eps=1e-6) if qk_norm else nn.Identity()
-        self.k_norm = norm_layer(self.head_dim, elementwise_affine=True, eps=1e-6) if qk_norm else nn.Identity()
-        self.inner_attn = FlashSelfAttention(attention_dropout=attn_drop)
-        self.out_proj = nn.Linear(dim, dim, bias=qkv_bias, **factory_kwargs)
-        self.proj_drop = nn.Dropout(proj_drop)
-
-    def forward(self, x, freqs_cis_img=None):
-        """
-        Parameters
-        ----------
-        x: torch.Tensor
-            (batch, seqlen, hidden_dim) (where hidden_dim = num heads * head dim)
-        freqs_cis_img: torch.Tensor
-            (batch, hidden_dim // 2), RoPE for image
-        """
-        b, s, d = x.shape
-
-        qkv = self.Wqkv(x)
-        qkv = qkv.view(b, s, 3, self.num_heads, self.head_dim)  # [b, s, 3, h, d]
-        q, k, v = qkv.unbind(dim=2) # [b, s, h, d]
-        q = self.q_norm(q).half()   # [b, s, h, d]
-        k = self.k_norm(k).half()
-
-        # Apply RoPE if needed
-        if freqs_cis_img is not None:
-            qq, kk = apply_rotary_emb(q, k, freqs_cis_img)
-            assert qq.shape == q.shape and kk.shape == k.shape, f'qq: {qq.shape}, q: {q.shape}, kk: {kk.shape}, k: {k.shape}'
-            q, k = qq, kk
-
-        qkv = torch.stack([q, k, v], dim=2)     # [b, s, 3, h, d]
-        context = self.inner_attn(qkv)
-        out = self.out_proj(context.view(b, s, d))
-        out = self.proj_drop(out)
-
-        out_tuple = (out,)
-
-        return out_tuple
-
-
-class FlashCrossMHAModified(nn.Module):
-    """
-    Use QK Normalization.
-    """
-    def __init__(self,
-                 qdim,
-                 kdim,
-                 num_heads,
-                 qkv_bias=True,
-                 qk_norm=False,
-                 attn_drop=0.0,
-                 proj_drop=0.0,
-                 device=None,
-                 dtype=None,
-                 norm_layer=nn.LayerNorm,
-                 ):
-        factory_kwargs = {'device': device, 'dtype': dtype}
-        super().__init__()
-        self.qdim = qdim
-        self.kdim = kdim
-        self.num_heads = num_heads
-        assert self.qdim % num_heads == 0, "self.qdim must be divisible by num_heads"
-        self.head_dim = self.qdim // num_heads
-        assert self.head_dim % 8 == 0 and self.head_dim <= 128, "Only support head_dim <= 128 and divisible by 8"
-
-        self.scale = self.head_dim ** -0.5
-
-        self.q_proj = nn.Linear(qdim, qdim, bias=qkv_bias, **factory_kwargs)
-        self.kv_proj = nn.Linear(kdim, 2 * qdim, bias=qkv_bias, **factory_kwargs)
-
-        # TODO: eps should be 1 / 65530 if using fp16
-        self.q_norm = norm_layer(self.head_dim, elementwise_affine=True, eps=1e-6) if qk_norm else nn.Identity()
-        self.k_norm = norm_layer(self.head_dim, elementwise_affine=True, eps=1e-6) if qk_norm else nn.Identity()
-
-        self.inner_attn = FlashCrossAttention(attention_dropout=attn_drop)
-        self.out_proj = nn.Linear(qdim, qdim, bias=qkv_bias, **factory_kwargs)
-        self.proj_drop = nn.Dropout(proj_drop)
-
-    def forward(self, x, y, freqs_cis_img=None):
-        """
-        Parameters
-        ----------
-        x: torch.Tensor
-            (batch, seqlen1, hidden_dim) (where hidden_dim = num_heads * head_dim)
-        y: torch.Tensor
-            (batch, seqlen2, hidden_dim2)
-        freqs_cis_img: torch.Tensor
-            (batch, hidden_dim // num_heads), RoPE for image
-        """
-        b, s1, _ = x.shape     # [b, s1, D]
-        _, s2, _ = y.shape     # [b, s2, 1024]
-
-        q = self.q_proj(x).view(b, s1, self.num_heads, self.head_dim)       # [b, s1, h, d]
-        kv = self.kv_proj(y).view(b, s2, 2, self.num_heads, self.head_dim)  # [b, s2, 2, h, d]
-        k, v = kv.unbind(dim=2)                 # [b, s2, h, d]
-        q = self.q_norm(q).half()               # [b, s1, h, d]
-        k = self.k_norm(k).half()               # [b, s2, h, d]
-
-        # Apply RoPE if needed
-        if freqs_cis_img is not None:
-            qq, _ = apply_rotary_emb(q, None, freqs_cis_img)
-            assert qq.shape == q.shape, f'qq: {qq.shape}, q: {q.shape}'
-            q = qq                              # [b, s1, h, d]
-        kv = torch.stack([k, v], dim=2)         # [b, s1, 2, h, d]
-        context = self.inner_attn(q, kv)        # [b, s1, h, d]
-        context = context.view(b, s1, -1)       # [b, s1, D]
-
-        out = self.out_proj(context)
-        out = self.proj_drop(out)
-
-        out_tuple = (out,)
-
-        return out_tuple
-
-
-class CrossAttention(nn.Module):
-    """
-    Use QK Normalization.
-    """
-    def __init__(self,
-                 qdim,
-                 kdim,
-                 num_heads,
-                 qkv_bias=True,
-                 qk_norm=False,
-                 attn_drop=0.0,
-                 proj_drop=0.0,
-                 device=None,
-                 dtype=None,
-                 norm_layer=nn.LayerNorm,
-                 ):
-        factory_kwargs = {'device': device, 'dtype': dtype}
-        super().__init__()
-        self.qdim = qdim
-        self.kdim = kdim
-        self.num_heads = num_heads
-        assert self.qdim % num_heads == 0, "self.qdim must be divisible by num_heads"
-        self.head_dim = self.qdim // num_heads
-        assert self.head_dim % 8 == 0 and self.head_dim <= 128, "Only support head_dim <= 128 and divisible by 8"
-        self.scale = self.head_dim ** -0.5
-
-        self.q_proj = nn.Linear(qdim, qdim, bias=qkv_bias, **factory_kwargs)
-        self.kv_proj = nn.Linear(kdim, 2 * qdim, bias=qkv_bias, **factory_kwargs)
-
-        # TODO: eps should be 1 / 65530 if using fp16
-        self.q_norm = norm_layer(self.head_dim, elementwise_affine=True, eps=1e-6) if qk_norm else nn.Identity()
-        self.k_norm = norm_layer(self.head_dim, elementwise_affine=True, eps=1e-6) if qk_norm else nn.Identity()
-        self.attn_drop = nn.Dropout(attn_drop)
-        self.out_proj = nn.Linear(qdim, qdim, bias=qkv_bias, **factory_kwargs)
-        self.proj_drop = nn.Dropout(proj_drop)
-
-    def forward(self, x, y, freqs_cis_img=None):
-        """
-        Parameters
-        ----------
-        x: torch.Tensor
-            (batch, seqlen1, hidden_dim) (where hidden_dim = num heads * head dim)
-        y: torch.Tensor
-            (batch, seqlen2, hidden_dim2)
-        freqs_cis_img: torch.Tensor
-            (batch, hidden_dim // 2), RoPE for image
-        """
-        b, s1, c = x.shape     # [b, s1, D]
-        _, s2, c = y.shape     # [b, s2, 1024]
-
-        q = self.q_proj(x).view(b, s1, self.num_heads, self.head_dim)   # [b, s1, h, d]
-        kv = self.kv_proj(y).view(b, s2, 2, self.num_heads, self.head_dim)    # [b, s2, 2, h, d]
-        k, v = kv.unbind(dim=2) # [b, s, h, d]
-        q = self.q_norm(q)
-        k = self.k_norm(k)
-
-        # Apply RoPE if needed
-        if freqs_cis_img is not None:
-            qq, _ = apply_rotary_emb(q, None, freqs_cis_img)
-            assert qq.shape == q.shape, f'qq: {qq.shape}, q: {q.shape}'
-            q = qq
-
-        q = q * self.scale
-        q = q.transpose(-2, -3).contiguous()        # q ->  B, L1, H, C - B, H, L1, C
-        k = k.permute(0, 2, 3, 1).contiguous()      # k ->  B, L2, H, C - B, H, C, L2
-        attn = q @ k                                # attn -> B, H, L1, L2
-        attn = attn.softmax(dim=-1)                 # attn -> B, H, L1, L2
-        attn = self.attn_drop(attn)
-        x = attn @ v.transpose(-2, -3)              # v -> B, L2, H, C - B, H, L2, C    x-> B, H, L1, C
-        context = x.transpose(1, 2)                 # context -> B, H, L1, C - B, L1, H, C
-
-        context = context.contiguous().view(b, s1, -1)
-
-        out = self.out_proj(context)  # context.reshape - B, L1, -1
-        out = self.proj_drop(out)
-
-        out_tuple = (out,)
-
-        return out_tuple
-
-
-class Attention(nn.Module):
-    """
-    We rename some layer names to align with flash attention
-    """
-    def __init__(self, dim, num_heads, qkv_bias=True, qk_norm=False, attn_drop=0., proj_drop=0.,
-                 norm_layer=nn.LayerNorm,
-                 ):
-        super().__init__()
-        self.dim = dim
-        self.num_heads = num_heads
-        assert self.dim % num_heads == 0, 'dim should be divisible by num_heads'
-        self.head_dim = self.dim // num_heads
-        # This assertion is aligned with flash attention
-        assert self.head_dim % 8 == 0 and self.head_dim <= 128, "Only support head_dim <= 128 and divisible by 8"
-        self.scale = self.head_dim ** -0.5
-
-        # qkv --> Wqkv
-        self.Wqkv = nn.Linear(dim, dim * 3, bias=qkv_bias)
-        # TODO: eps should be 1 / 65530 if using fp16
-        self.q_norm = norm_layer(self.head_dim, elementwise_affine=True, eps=1e-6) if qk_norm else nn.Identity()
-        self.k_norm = norm_layer(self.head_dim, elementwise_affine=True, eps=1e-6) if qk_norm else nn.Identity()
-        self.attn_drop = nn.Dropout(attn_drop)
-        self.out_proj = nn.Linear(dim, dim)
-        self.proj_drop = nn.Dropout(proj_drop)
-
-    def forward(self, x, freqs_cis_img=None):
-        B, N, C = x.shape
-        qkv = self.Wqkv(x).reshape(B, N, 3, self.num_heads, self.head_dim).permute(2, 0, 3, 1, 4)   # [3, b, h, s, d]
-        q, k, v = qkv.unbind(0)     # [b, h, s, d]
-        q = self.q_norm(q)          # [b, h, s, d]
-        k = self.k_norm(k)          # [b, h, s, d]
-
-        # Apply RoPE if needed
-        if freqs_cis_img is not None:
-            qq, kk = apply_rotary_emb(q, k, freqs_cis_img, head_first=True)
-            assert qq.shape == q.shape and kk.shape == k.shape, \
-                f'qq: {qq.shape}, q: {q.shape}, kk: {kk.shape}, k: {k.shape}'
-            q, k = qq, kk
-
-        q = q * self.scale
-        attn = q @ k.transpose(-2, -1)              # [b, h, s, d] @ [b, h, d, s]
-        attn = attn.softmax(dim=-1)                 # [b, h, s, s]
-        attn = self.attn_drop(attn)
-        x = attn @ v                                # [b, h, s, d]
-
-        x = x.transpose(1, 2).reshape(B, N, C)      # [b, s, h, d]
-        x = self.out_proj(x)
-        x = self.proj_drop(x)
-
-        out_tuple = (x,)
-
-        return out_tuple

py/dit/hunyuanDiT/models/embedders.py

@@ -1,111 +0,0 @@
-import math
-import torch
-import torch.nn as nn
-from einops import repeat
-
-from timm.models.layers import to_2tuple
-
-
-class PatchEmbed(nn.Module):
-    """ 2D Image to Patch Embedding
-
-    Image to Patch Embedding using Conv2d
-
-    A convolution based approach to patchifying a 2D image w/ embedding projection.
-
-    Based on the impl in https://github.com/google-research/vision_transformer
-
-    Hacked together by / Copyright 2020 Ross Wightman
-
-    Remove the _assert function in forward function to be compatible with multi-resolution images.
-    """
-    def __init__(
-            self,
-            img_size=224,
-            patch_size=16,
-            in_chans=3,
-            embed_dim=768,
-            norm_layer=None,
-            flatten=True,
-            bias=True,
-    ):
-        super().__init__()
-        if isinstance(img_size, int):
-            img_size = to_2tuple(img_size)
-        elif isinstance(img_size, (tuple, list)) and len(img_size) == 2:
-            img_size = tuple(img_size)
-        else:
-            raise ValueError(f"img_size must be int or tuple/list of length 2. Got {img_size}")
-        patch_size = to_2tuple(patch_size)
-        self.img_size = img_size
-        self.patch_size = patch_size
-        self.grid_size = (img_size[0] // patch_size[0], img_size[1] // patch_size[1])
-        self.num_patches = self.grid_size[0] * self.grid_size[1]
-        self.flatten = flatten
-
-        self.proj = nn.Conv2d(in_chans, embed_dim, kernel_size=patch_size, stride=patch_size, bias=bias)
-        self.norm = norm_layer(embed_dim) if norm_layer else nn.Identity()
-
-    def update_image_size(self, img_size):
-        self.img_size = img_size
-        self.grid_size = (img_size[0] // self.patch_size[0], img_size[1] // self.patch_size[1])
-        self.num_patches = self.grid_size[0] * self.grid_size[1]
-
-    def forward(self, x):
-        # B, C, H, W = x.shape
-        # _assert(H == self.img_size[0], f"Input image height ({H}) doesn't match model ({self.img_size[0]}).")
-        # _assert(W == self.img_size[1], f"Input image width ({W}) doesn't match model ({self.img_size[1]}).")
-        x = self.proj(x)
-        if self.flatten:
-            x = x.flatten(2).transpose(1, 2)  # BCHW -> BNC
-        x = self.norm(x)
-        return x
-
-
-def timestep_embedding(t, dim, max_period=10000, repeat_only=False):
-    """
-    Create sinusoidal timestep embeddings.
-    :param t: a 1-D Tensor of N indices, one per batch element.
-                      These may be fractional.
-    :param dim: the dimension of the output.
-    :param max_period: controls the minimum frequency of the embeddings.
-    :return: an (N, D) Tensor of positional embeddings.
-    """
-    # https://github.com/openai/glide-text2im/blob/main/glide_text2im/nn.py
-    if not repeat_only:
-        half = dim // 2
-        freqs = torch.exp(
-            -math.log(max_period)
-            * torch.arange(start=0, end=half, dtype=torch.float32)
-            / half
-        ).to(device=t.device)   # size: [dim/2], 一个指数衰减的曲线
-        args = t[:, None].float() * freqs[None]
-        embedding = torch.cat([torch.cos(args), torch.sin(args)], dim=-1)
-        if dim % 2:
-            embedding = torch.cat(
-                [embedding, torch.zeros_like(embedding[:, :1])], dim=-1
-            )
-    else:
-        embedding = repeat(t, "b -> b d", d=dim)
-    return embedding
-
-
-class TimestepEmbedder(nn.Module):
-    """
-    Embeds scalar timesteps into vector representations.
-    """
-    def __init__(self, hidden_size, frequency_embedding_size=256, out_size=None):
-        super().__init__()
-        if out_size is None:
-            out_size = hidden_size
-        self.mlp = nn.Sequential(
-            nn.Linear(frequency_embedding_size, hidden_size, bias=True),
-            nn.SiLU(),
-            nn.Linear(hidden_size, out_size, bias=True),
-        )
-        self.frequency_embedding_size = frequency_embedding_size
-
-    def forward(self, t):
-        t_freq = timestep_embedding(t, self.frequency_embedding_size).type(self.mlp[0].weight.dtype)
-        t_emb = self.mlp(t_freq)
-        return t_emb

py/dit/hunyuanDiT/models/models.py

@@ -1,428 +0,0 @@
-import torch
-import torch.nn as nn
-import torch.nn.functional as F
-from timm.models.vision_transformer import Mlp
-
-from .attn_layers import Attention, FlashCrossMHAModified, FlashSelfMHAModified, CrossAttention
-from .embedders import TimestepEmbedder, PatchEmbed, timestep_embedding
-from .norm_layers import RMSNorm
-from .poolers import AttentionPool
-from .posemb_layers import get_2d_rotary_pos_embed, get_fill_resize_and_crop
-
-def modulate(x, shift, scale):
-    return x * (1 + scale.unsqueeze(1)) + shift.unsqueeze(1)
-
-
-class FP32_Layernorm(nn.LayerNorm):
-    def forward(self, inputs: torch.Tensor) -> torch.Tensor:
-        origin_dtype = inputs.dtype
-        return F.layer_norm(inputs.float(), self.normalized_shape, self.weight.float(), self.bias.float(),
-                            self.eps).to(origin_dtype)
-
-
-class FP32_SiLU(nn.SiLU):
-    def forward(self, inputs: torch.Tensor) -> torch.Tensor:
-        return torch.nn.functional.silu(inputs.float(), inplace=False).to(inputs.dtype)
-
-
-class HunYuanDiTBlock(nn.Module):
-    """
-    A HunYuanDiT block with `add` conditioning.
-    """
-    def __init__(self,
-                 hidden_size,
-                 c_emb_size,
-                 num_heads,
-                 mlp_ratio=4.0,
-                 text_states_dim=1024,
-                 use_flash_attn=False,
-                 qk_norm=False,
-                 norm_type="layer",
-                 skip=False,
-                 ):
-        super().__init__()
-        self.use_flash_attn = use_flash_attn
-        use_ele_affine = True
-
-        if norm_type == "layer":
-            norm_layer = FP32_Layernorm
-        elif norm_type == "rms":
-            norm_layer = RMSNorm
-        else:
-            raise ValueError(f"Unknown norm_type: {norm_type}")
-
-        # ========================= Self-Attention =========================
-        self.norm1 = norm_layer(hidden_size, elementwise_affine=use_ele_affine, eps=1e-6)
-        if use_flash_attn:
-            self.attn1 = FlashSelfMHAModified(hidden_size, num_heads=num_heads, qkv_bias=True, qk_norm=qk_norm)
-        else:
-            self.attn1 = Attention(hidden_size, num_heads=num_heads, qkv_bias=True, qk_norm=qk_norm)
-
-        # ========================= FFN =========================
-        self.norm2 = norm_layer(hidden_size, elementwise_affine=use_ele_affine, eps=1e-6)
-        mlp_hidden_dim = int(hidden_size * mlp_ratio)
-        approx_gelu = lambda: nn.GELU(approximate="tanh")
-        self.mlp = Mlp(in_features=hidden_size, hidden_features=mlp_hidden_dim, act_layer=approx_gelu, drop=0)
-
-        # ========================= Add =========================
-        # Simply use add like SDXL.
-        self.default_modulation = nn.Sequential(
-            FP32_SiLU(),
-            nn.Linear(c_emb_size, hidden_size, bias=True)
-        )
-
-        # ========================= Cross-Attention =========================
-        if use_flash_attn:
-            self.attn2 = FlashCrossMHAModified(hidden_size, text_states_dim, num_heads=num_heads, qkv_bias=True,
-                                               qk_norm=qk_norm)
-        else:
-            self.attn2 = CrossAttention(hidden_size, text_states_dim, num_heads=num_heads, qkv_bias=True,
-                                        qk_norm=qk_norm)
-        self.norm3 = norm_layer(hidden_size, elementwise_affine=True, eps=1e-6)
-
-        # ========================= Skip Connection =========================
-        if skip:
-            self.skip_norm = norm_layer(2 * hidden_size, elementwise_affine=True, eps=1e-6)
-            self.skip_linear = nn.Linear(2 * hidden_size, hidden_size)
-        else:
-            self.skip_linear = None
-
-    def forward(self, x, c=None, text_states=None, freq_cis_img=None, skip=None):
-        # Long Skip Connection
-        if self.skip_linear is not None:
-            cat = torch.cat([x, skip], dim=-1)
-            cat = self.skip_norm(cat)
-            x = self.skip_linear(cat)
-
-        # Self-Attention
-        shift_msa = self.default_modulation(c).unsqueeze(dim=1)
-        attn_inputs = (
-            self.norm1(x) + shift_msa, freq_cis_img,
-        )
-        x = x + self.attn1(*attn_inputs)[0]
-
-        # Cross-Attention
-        cross_inputs = (
-            self.norm3(x), text_states, freq_cis_img
-        )
-        x = x + self.attn2(*cross_inputs)[0]
-
-        # FFN Layer
-        mlp_inputs = self.norm2(x)
-        x = x + self.mlp(mlp_inputs)
-
-        return x
-
-
-class FinalLayer(nn.Module):
-    """
-    The final layer of HunYuanDiT.
-    """
-    def __init__(self, final_hidden_size, c_emb_size, patch_size, out_channels):
-        super().__init__()
-        self.norm_final = nn.LayerNorm(final_hidden_size, elementwise_affine=False, eps=1e-6)
-        self.linear = nn.Linear(final_hidden_size, patch_size * patch_size * out_channels, bias=True)
-        self.adaLN_modulation = nn.Sequential(
-            FP32_SiLU(),
-            nn.Linear(c_emb_size, 2 * final_hidden_size, bias=True)
-        )
-
-    def forward(self, x, c):
-        shift, scale = self.adaLN_modulation(c).chunk(2, dim=1)
-        x = modulate(self.norm_final(x), shift, scale)
-        x = self.linear(x)
-        return x
-
-
-class HunYuanDiT(nn.Module):
-    """
-    HunYuanDiT: Diffusion model with a Transformer backbone.
-
-    Parameters
-    ----------
-    args: argparse.Namespace
-        The arguments parsed by argparse.
-    input_size: tuple
-        The size of the input image.
-    patch_size: int
-        The size of the patch.
-    in_channels: int
-        The number of input channels.
-    hidden_size: int
-        The hidden size of the transformer backbone.
-    depth: int
-        The number of transformer blocks.
-    num_heads: int
-        The number of attention heads.
-    mlp_ratio: float
-        The ratio of the hidden size of the MLP in the transformer block.
-    log_fn: callable
-        The logging function.
-    """
-    def __init__(
-            self, args,
-            input_size=(32, 32),
-            patch_size=2,
-            in_channels=4,
-            hidden_size=1152,
-            depth=28,
-            num_heads=16,
-            mlp_ratio=4.0,
-            log_fn=print,
-            **kwargs,
-    ):
-        super().__init__()
-        self.args = args
-        self.log_fn = log_fn
-        self.depth = depth
-        self.learn_sigma = args.learn_sigma
-        self.in_channels = in_channels
-        self.out_channels = in_channels * 2 if args.learn_sigma else in_channels
-        self.patch_size = patch_size
-        self.num_heads = num_heads
-        self.hidden_size = hidden_size
-        self.head_size = hidden_size // num_heads
-        self.text_states_dim = args.text_states_dim
-        self.text_states_dim_t5 = args.text_states_dim_t5
-        self.text_len = args.text_len
-        self.text_len_t5 = args.text_len_t5
-        self.norm = args.norm
-
-        use_flash_attn = args.infer_mode == 'fa'
-        if use_flash_attn:
-            log_fn(f"    Enable Flash Attention.")
-        qk_norm = True  # See http://arxiv.org/abs/2302.05442 for details.
-
-        self.mlp_t5 = nn.Sequential(
-            nn.Linear(self.text_states_dim_t5, self.text_states_dim_t5 * 4, bias=True),
-            FP32_SiLU(),
-            nn.Linear(self.text_states_dim_t5 * 4, self.text_states_dim, bias=True),
-        )
-        # learnable replace
-        self.text_embedding_padding = nn.Parameter(
-            torch.randn(self.text_len + self.text_len_t5, self.text_states_dim, dtype=torch.float32))
-
-        # Attention pooling
-        self.pooler = AttentionPool(self.text_len_t5, self.text_states_dim_t5, num_heads=8, output_dim=1024)
-
-        # Here we use a default learned embedder layer for future extension.
-        self.style_embedder = nn.Embedding(1, hidden_size)
-
-        # Image size and crop size conditions
-        self.extra_in_dim = 256 * 6 + hidden_size
-
-        # Text embedding for `add`
-        self.last_size = input_size
-        self.x_embedder = PatchEmbed(input_size, patch_size, in_channels, hidden_size)
-        self.t_embedder = TimestepEmbedder(hidden_size)
-        self.extra_in_dim += 1024
-        self.extra_embedder = nn.Sequential(
-            nn.Linear(self.extra_in_dim, hidden_size * 4),
-            FP32_SiLU(),
-            nn.Linear(hidden_size * 4, hidden_size, bias=True),
-        )
-
-        # Image embedding
-        num_patches = self.x_embedder.num_patches
-        log_fn(f"    Number of tokens: {num_patches}")
-
-        # HUnYuanDiT Blocks
-        self.blocks = nn.ModuleList([
-            HunYuanDiTBlock(hidden_size=hidden_size,
-                            c_emb_size=hidden_size,
-                            num_heads=num_heads,
-                            mlp_ratio=mlp_ratio,
-                            text_states_dim=self.text_states_dim,
-                            use_flash_attn=use_flash_attn,
-                            qk_norm=qk_norm,
-                            norm_type=self.norm,
-                            skip=layer > depth // 2,
-                            )
-            for layer in range(depth)
-        ])
-
-        self.final_layer = FinalLayer(hidden_size, hidden_size, patch_size, self.out_channels)
-        self.unpatchify_channels = self.out_channels
-
-    def forward_raw(self,
-                x,
-                t,
-                encoder_hidden_states=None,
-                text_embedding_mask=None,
-                encoder_hidden_states_t5=None,
-                text_embedding_mask_t5=None,
-                image_meta_size=None,
-                style=None,
-                cos_cis_img=None,
-                sin_cis_img=None,
-                return_dict=False,
-                ):
-        """
-        Forward pass of the encoder.
-
-        Parameters
-        ----------
-        x: torch.Tensor
-            (B, D, H, W)
-        t: torch.Tensor
-            (B)
-        encoder_hidden_states: torch.Tensor
-            CLIP text embedding, (B, L_clip, D)
-        text_embedding_mask: torch.Tensor
-            CLIP text embedding mask, (B, L_clip)
-        encoder_hidden_states_t5: torch.Tensor
-            T5 text embedding, (B, L_t5, D)
-        text_embedding_mask_t5: torch.Tensor
-            T5 text embedding mask, (B, L_t5)
-        image_meta_size: torch.Tensor
-            (B, 6)
-        style: torch.Tensor
-            (B)
-        cos_cis_img: torch.Tensor
-        sin_cis_img: torch.Tensor
-        return_dict: bool
-            Whether to return a dictionary.
-        """
-
-        text_states = encoder_hidden_states                     # 2,77,1024
-        text_states_t5 = encoder_hidden_states_t5               # 2,256,2048
-        text_states_mask = text_embedding_mask.bool()           # 2,77
-        text_states_t5_mask = text_embedding_mask_t5.bool()     # 2,256
-        b_t5, l_t5, c_t5 = text_states_t5.shape
-        text_states_t5 = self.mlp_t5(text_states_t5.view(-1, c_t5))
-        text_states = torch.cat([text_states, text_states_t5.view(b_t5, l_t5, -1)], dim=1)  # 2,205，1024
-        clip_t5_mask = torch.cat([text_states_mask, text_states_t5_mask], dim=-1)
-
-        clip_t5_mask = clip_t5_mask
-        text_states = torch.where(clip_t5_mask.unsqueeze(2), text_states, self.text_embedding_padding.to(text_states))
-
-        _, _, oh, ow = x.shape
-        th, tw = oh // self.patch_size, ow // self.patch_size
-
-        # ========================= Build time and image embedding =========================
-        t = self.t_embedder(t)
-        x = self.x_embedder(x)
-
-        # Get image RoPE embedding according to `reso`lution.
-        freqs_cis_img = (cos_cis_img, sin_cis_img)
-
-        # ========================= Concatenate all extra vectors =========================
-        # Build text tokens with pooling
-        extra_vec = self.pooler(encoder_hidden_states_t5)
-
-        # Build image meta size tokens
-        image_meta_size = timestep_embedding(image_meta_size.view(-1), 256)   # [B * 6, 256]
-        # if self.args.use_fp16:
-            # image_meta_size = image_meta_size.half()
-        image_meta_size = image_meta_size.view(-1, 6 * 256)
-        extra_vec = torch.cat([extra_vec, image_meta_size], dim=1)  # [B, D + 6 * 256]
-
-        # Build style tokens
-        style_embedding = self.style_embedder(style)
-        extra_vec = torch.cat([extra_vec, style_embedding], dim=1)
-
-        # Concatenate all extra vectors
-        c = t + self.extra_embedder(extra_vec.to(self.dtype))  # [B, D]
-
-        # ========================= Forward pass through HunYuanDiT blocks =========================
-        skips = []
-        for layer, block in enumerate(self.blocks):
-            if layer > self.depth // 2:
-                skip = skips.pop()
-                x = block(x, c, text_states, freqs_cis_img, skip)   # (N, L, D)
-            else:
-                x = block(x, c, text_states, freqs_cis_img)         # (N, L, D)
-
-            if layer < (self.depth // 2 - 1):
-                skips.append(x)
-
-        # ========================= Final layer =========================
-        x = self.final_layer(x, c)                              # (N, L, patch_size ** 2 * out_channels)
-        x = self.unpatchify(x, th, tw)                          # (N, out_channels, H, W)
-
-        if return_dict:
-            return {'x': x}
-        return x
-   
-    def calc_rope(self, height, width):
-        """
-        Probably not the best in terms of perf to have this here
-        """
-        th = height // 8 // self.patch_size
-        tw = width // 8 // self.patch_size
-        base_size = 512 // 8 // self.patch_size
-        start, stop = get_fill_resize_and_crop((th, tw), base_size)
-        sub_args = [start, stop, (th, tw)]
-        rope = get_2d_rotary_pos_embed(self.head_size, *sub_args)
-        return rope
-
-    def forward(self, x, timesteps, context, context_mask=None, context_t5=None, context_t5_mask=None, src_size_cond=(1024,1024), **kwargs):
-        """
-        Forward pass that adapts comfy input to original forward function
-        x: (N, C, H, W) tensor of spatial inputs (images or latent representations of images)
-        timesteps: (N,) tensor of diffusion timesteps
-        context: (N, 1, 77, C) CLIP conditioning
-        context_t5: (N, 1, 256, C) MT5 conditioning
-        """
-        # context_mask = torch.zeros(x.shape[0], 77, device=x.device)
-        # context_t5_mask = torch.zeros(x.shape[0], 256, device=x.device)
-
-        # style
-        style = torch.as_tensor([0] * (x.shape[0]), device=x.device)
-
-        # image size - todo separate for cond/uncond when batched
-        if torch.is_tensor(src_size_cond):
-                src_size_cond = (int(src_size_cond[0][0]), int(src_size_cond[0][1]))
-        
-        image_size = (x.shape[2]//2*16, x.shape[3]//2*16)
-        size_cond = list(src_size_cond) + [image_size[1], image_size[0], 0, 0]
-        image_meta_size = torch.as_tensor([size_cond] * x.shape[0], device=x.device)
-
-        # RoPE
-        rope = self.calc_rope(*image_size)
-
-        # Update x_embedder if image size changed
-        if self.last_size != image_size:
-                from tqdm import tqdm
-                tqdm.write(f"HyDiT: New image size {image_size}")
-                self.x_embedder.update_image_size(
-                        (image_size[0]//8, image_size[1]//8), 
-                )
-                self.last_size = image_size
-
-        # Run original forward pass
-        out = self.forward_raw(
-                x = x.to(self.dtype),
-                t = timesteps.to(self.dtype),
-                encoder_hidden_states = context.to(self.dtype),
-                text_embedding_mask   = context_mask.to(self.dtype),
-                encoder_hidden_states_t5 = context_t5.to(self.dtype),
-                text_embedding_mask_t5   = context_t5_mask.to(self.dtype),
-                image_meta_size = image_meta_size.to(self.dtype),
-                style = style,
-                cos_cis_img = rope[0],
-                sin_cis_img = rope[1],
-        )
-        
-        # return
-        out = out.to(torch.float)
-        if self.learn_sigma:
-                eps, rest = out[:, :self.in_channels], out[:, self.in_channels:]
-                return eps
-        else:
-                return out
-
-    def unpatchify(self, x, h, w):
-        """
-        x: (N, T, patch_size**2 * C)
-        imgs: (N, H, W, C)
-        """
-        c = self.unpatchify_channels
-        p = self.x_embedder.patch_size[0]
-        # h = w = int(x.shape[1] ** 0.5)
-        assert h * w == x.shape[1]
-
-        x = x.reshape(shape=(x.shape[0], h, w, p, p, c))
-        x = torch.einsum('nhwpqc->nchpwq', x)
-        imgs = x.reshape(shape=(x.shape[0], c, h * p, w * p))
-        return imgs

py/dit/hunyuanDiT/models/norm_layers.py

@@ -1,68 +0,0 @@
-import torch
-import torch.nn as nn
-
-
-class RMSNorm(nn.Module):
-    def __init__(self, dim: int, elementwise_affine=True, eps: float = 1e-6):
-        """
-        Initialize the RMSNorm normalization layer.
-
-        Args:
-            dim (int): The dimension of the input tensor.
-            eps (float, optional): A small value added to the denominator for numerical stability. Default is 1e-6.
-
-        Attributes:
-            eps (float): A small value added to the denominator for numerical stability.
-            weight (nn.Parameter): Learnable scaling parameter.
-
-        """
-        super().__init__()
-        self.eps = eps
-        if elementwise_affine:
-            self.weight = nn.Parameter(torch.ones(dim))
-
-    def _norm(self, x):
-        """
-        Apply the RMSNorm normalization to the input tensor.
-
-        Args:
-            x (torch.Tensor): The input tensor.
-
-        Returns:
-            torch.Tensor: The normalized tensor.
-
-        """
-        return x * torch.rsqrt(x.pow(2).mean(-1, keepdim=True) + self.eps)
-
-    def forward(self, x):
-        """
-        Forward pass through the RMSNorm layer.
-
-        Args:
-            x (torch.Tensor): The input tensor.
-
-        Returns:
-            torch.Tensor: The output tensor after applying RMSNorm.
-
-        """
-        output = self._norm(x.float()).type_as(x)
-        if hasattr(self, "weight"):
-            output = output * self.weight
-        return output
-
-
-class GroupNorm32(nn.GroupNorm):
-    def __init__(self, num_groups, num_channels, eps=1e-5, dtype=None):
-        super().__init__(num_groups=num_groups, num_channels=num_channels, eps=eps, dtype=dtype)
-
-    def forward(self, x):
-        y = super().forward(x).to(x.dtype)
-        return y
-
-def normalization(channels, dtype=None):
-    """
-    Make a standard normalization layer.
-    :param channels: number of input channels.
-    :return: an nn.Module for normalization.
-    """
-    return GroupNorm32(num_channels=channels, num_groups=32, dtype=dtype)

py/dit/hunyuanDiT/models/poolers.py

@@ -1,39 +0,0 @@
-import torch
-import torch.nn as nn
-import torch.nn.functional as F
-
-
-class AttentionPool(nn.Module):
-    def __init__(self, spacial_dim: int, embed_dim: int, num_heads: int, output_dim: int = None):
-        super().__init__()
-        self.positional_embedding = nn.Parameter(torch.randn(spacial_dim + 1, embed_dim) / embed_dim ** 0.5)
-        self.k_proj = nn.Linear(embed_dim, embed_dim)
-        self.q_proj = nn.Linear(embed_dim, embed_dim)
-        self.v_proj = nn.Linear(embed_dim, embed_dim)
-        self.c_proj = nn.Linear(embed_dim, output_dim or embed_dim)
-        self.num_heads = num_heads
-
-    def forward(self, x):
-        x = x.permute(1, 0, 2)  # NLC -> LNC
-        x = torch.cat([x.mean(dim=0, keepdim=True), x], dim=0)  # (L+1)NC
-        x = x + self.positional_embedding[:, None, :].to(x.dtype)  # (L+1)NC
-        x, _ = F.multi_head_attention_forward(
-            query=x[:1], key=x, value=x,
-            embed_dim_to_check=x.shape[-1],
-            num_heads=self.num_heads,
-            q_proj_weight=self.q_proj.weight,
-            k_proj_weight=self.k_proj.weight,
-            v_proj_weight=self.v_proj.weight,
-            in_proj_weight=None,
-            in_proj_bias=torch.cat([self.q_proj.bias, self.k_proj.bias, self.v_proj.bias]),
-            bias_k=None,
-            bias_v=None,
-            add_zero_attn=False,
-            dropout_p=0,
-            out_proj_weight=self.c_proj.weight,
-            out_proj_bias=self.c_proj.bias,
-            use_separate_proj_weight=True,
-            training=self.training,
-            need_weights=False
-        )
-        return x.squeeze(0)

py/dit/hunyuanDiT/models/posemb_layers.py

@@ -1,225 +0,0 @@
-import torch
-import numpy as np
-from typing import Union
-
-
-def _to_tuple(x):
-    if isinstance(x, int):
-        return x, x
-    else:
-        return x
-
-
-def get_fill_resize_and_crop(src, tgt):    # src 来源的分辨率   tgt base 分辨率
-    th, tw = _to_tuple(tgt)
-    h, w = _to_tuple(src)
-
-    tr = th / tw        # base 分辨率
-    r = h / w           # 目标分辨率
-
-    # resize
-    if r > tr:
-        resize_height = th
-        resize_width = int(round(th / h * w))
-    else:
-        resize_width = tw
-        resize_height = int(round(tw / w * h))    # 根据base分辨率，将目标分辨率resize下来
-
-    crop_top = int(round((th - resize_height) / 2.0))
-    crop_left = int(round((tw - resize_width) / 2.0))
-
-    return (crop_top, crop_left), (crop_top + resize_height, crop_left + resize_width)
-
-
-def get_meshgrid(start, *args):
-    if len(args) == 0:
-        # start is grid_size
-        num = _to_tuple(start)
-        start = (0, 0)
-        stop = num
-    elif len(args) == 1:
-        # start is start, args[0] is stop, step is 1
-        start = _to_tuple(start)
-        stop = _to_tuple(args[0])
-        num = (stop[0] - start[0], stop[1] - start[1])
-    elif len(args) == 2:
-        # start is start, args[0] is stop, args[1] is num
-        start = _to_tuple(start)       # 左上角   eg: 12,0
-        stop = _to_tuple(args[0])      # 右下角   eg: 20,32
-        num = _to_tuple(args[1])       # 目标大小  eg: 32,124
-    else:
-        raise ValueError(f"len(args) should be 0, 1 or 2, but got {len(args)}")
-
-    grid_h = np.linspace(start[0], stop[0], num[0], endpoint=False, dtype=np.float32) # 12-20 中间差值32份   0-32 中间差值124份
-    grid_w = np.linspace(start[1], stop[1], num[1], endpoint=False, dtype=np.float32)
-    grid = np.meshgrid(grid_w, grid_h)  # here w goes first
-    grid = np.stack(grid, axis=0)   # [2, W, H]
-    return grid
-
-#################################################################################
-#                   Sine/Cosine Positional Embedding Functions                  #
-#################################################################################
-# https://github.com/facebookresearch/mae/blob/main/util/pos_embed.py
-
-def get_2d_sincos_pos_embed(embed_dim, start, *args, cls_token=False, extra_tokens=0):
-    """
-    grid_size: int of the grid height and width
-    return:
-    pos_embed: [grid_size*grid_size, embed_dim] or [1+grid_size*grid_size, embed_dim] (w/ or w/o cls_token)
-    """
-    grid = get_meshgrid(start, *args)   # [2, H, w]
-    # grid_h = np.arange(grid_size, dtype=np.float32)
-    # grid_w = np.arange(grid_size, dtype=np.float32)
-    # grid = np.meshgrid(grid_w, grid_h)  # here w goes first
-    # grid = np.stack(grid, axis=0)   # [2, W, H]
-
-    grid = grid.reshape([2, 1, *grid.shape[1:]])
-    pos_embed = get_2d_sincos_pos_embed_from_grid(embed_dim, grid)
-    if cls_token and extra_tokens > 0:
-        pos_embed = np.concatenate([np.zeros([extra_tokens, embed_dim]), pos_embed], axis=0)
-    return pos_embed
-
-
-def get_2d_sincos_pos_embed_from_grid(embed_dim, grid):
-    assert embed_dim % 2 == 0
-
-    # use half of dimensions to encode grid_h
-    emb_h = get_1d_sincos_pos_embed_from_grid(embed_dim // 2, grid[0])  # (H*W, D/2)
-    emb_w = get_1d_sincos_pos_embed_from_grid(embed_dim // 2, grid[1])  # (H*W, D/2)
-
-    emb = np.concatenate([emb_h, emb_w], axis=1)    # (H*W, D)
-    return emb
-
-
-def get_1d_sincos_pos_embed_from_grid(embed_dim, pos):
-    """
-    embed_dim: output dimension for each position
-    pos: a list of positions to be encoded: size (W,H)
-    out: (M, D)
-    """
-    assert embed_dim % 2 == 0
-    omega = np.arange(embed_dim // 2, dtype=np.float64)
-    omega /= embed_dim / 2.
-    omega = 1. / 10000**omega  # (D/2,)
-
-    pos = pos.reshape(-1)  # (M,)
-    out = np.einsum('m,d->md', pos, omega)  # (M, D/2), outer product
-
-    emb_sin = np.sin(out)   # (M, D/2)
-    emb_cos = np.cos(out)   # (M, D/2)
-
-    emb = np.concatenate([emb_sin, emb_cos], axis=1)  # (M, D)
-    return emb
-
-
-#################################################################################
-#                   Rotary Positional Embedding Functions                       #
-#################################################################################
-# https://github.com/facebookresearch/llama/blob/main/llama/model.py#L443
-
-def get_2d_rotary_pos_embed(embed_dim, start, *args, use_real=True):
-    """
-    This is a 2d version of precompute_freqs_cis, which is a RoPE for image tokens with 2d structure.
-
-    Parameters
-    ----------
-    embed_dim: int
-        embedding dimension size
-    start: int or tuple of int
-        If len(args) == 0, start is num; If len(args) == 1, start is start, args[0] is stop, step is 1;
-        If len(args) == 2, start is start, args[0] is stop, args[1] is num.
-    use_real: bool
-        If True, return real part and imaginary part separately. Otherwise, return complex numbers.
-
-    Returns
-    -------
-    pos_embed: torch.Tensor
-        [HW, D/2]
-    """
-    grid = get_meshgrid(start, *args)   # [2, H, w]
-    grid = grid.reshape([2, 1, *grid.shape[1:]])   # 返回一个采样矩阵  分辨率与目标分辨率一致
-    pos_embed = get_2d_rotary_pos_embed_from_grid(embed_dim, grid, use_real=use_real)
-    return pos_embed
-
-
-def get_2d_rotary_pos_embed_from_grid(embed_dim, grid, use_real=False):
-    assert embed_dim % 4 == 0
-
-    # use half of dimensions to encode grid_h
-    emb_h = get_1d_rotary_pos_embed(embed_dim // 2, grid[0].reshape(-1), use_real=use_real)  # (H*W, D/4)
-    emb_w = get_1d_rotary_pos_embed(embed_dim // 2, grid[1].reshape(-1), use_real=use_real)  # (H*W, D/4)
-
-    if use_real:
-        cos = torch.cat([emb_h[0], emb_w[0]], dim=1)    # (H*W, D/2)
-        sin = torch.cat([emb_h[1], emb_w[1]], dim=1)    # (H*W, D/2)
-        return cos, sin
-    else:
-        emb = torch.cat([emb_h, emb_w], dim=1)    # (H*W, D/2)
-        return emb
-
-
-def get_1d_rotary_pos_embed(dim: int, pos: Union[np.ndarray, int], theta: float = 10000.0, use_real=False):
-    """
-    Precompute the frequency tensor for complex exponentials (cis) with given dimensions.
-
-    This function calculates a frequency tensor with complex exponentials using the given dimension 'dim'
-    and the end index 'end'. The 'theta' parameter scales the frequencies.
-    The returned tensor contains complex values in complex64 data type.
-
-    Args:
-        dim (int): Dimension of the frequency tensor.
-        pos (np.ndarray, int): Position indices for the frequency tensor. [S] or scalar
-        theta (float, optional): Scaling factor for frequency computation. Defaults to 10000.0.
-        use_real (bool, optional): If True, return real part and imaginary part separately.
-                                   Otherwise, return complex numbers.
-
-    Returns:
-        torch.Tensor: Precomputed frequency tensor with complex exponentials. [S, D/2]
-
-    """
-    if isinstance(pos, int):
-        pos = np.arange(pos)
-    freqs = 1.0 / (theta ** (torch.arange(0, dim, 2)[: (dim // 2)].float() / dim))  # [D/2]
-    t = torch.from_numpy(pos).to(freqs.device)  # type: ignore  # [S]
-    freqs = torch.outer(t, freqs).float()  # type: ignore   # [S, D/2]
-    if use_real:
-        freqs_cos = freqs.cos().repeat_interleave(2, dim=1)  # [S, D]
-        freqs_sin = freqs.sin().repeat_interleave(2, dim=1)  # [S, D]
-        return freqs_cos, freqs_sin
-    else:
-        freqs_cis = torch.polar(torch.ones_like(freqs), freqs)  # complex64     # [S, D/2]
-        return freqs_cis
-
-
-
-def calc_sizes(rope_img, patch_size, th, tw):
-    """ 计算 RoPE 的尺寸. """
-    if rope_img == 'extend':
-        # 拓展模式
-        sub_args = [(th, tw)]
-    elif rope_img.startswith('base'):
-        # 基于一个尺寸, 其他尺寸插值获得.
-        base_size = int(rope_img[4:]) // 8 // patch_size            # 基于512作为base，其他根据512差值得到
-        start, stop = get_fill_resize_and_crop((th, tw), base_size)   # 需要在32x32里面 crop的左上角和右下角
-        sub_args = [start, stop, (th, tw)]
-    else:
-        raise ValueError(f"Unknown rope_img: {rope_img}")
-    return sub_args
-
-
-def init_image_posemb(rope_img,
-                      resolutions,
-                      patch_size,
-                      hidden_size,
-                      num_heads,
-                      log_fn,
-                      rope_real=True,
-                      ):
-    freqs_cis_img = {}
-    for reso in resolutions:
-        th, tw = reso.height // 8 // patch_size, reso.width // 8 // patch_size
-        sub_args = calc_sizes(rope_img, patch_size, th, tw)      #  [左上角, 右下角, 目标高宽]   需要在32x32里面 crop的左上角和右下角
-        freqs_cis_img[str(reso)] = get_2d_rotary_pos_embed(hidden_size // num_heads, *sub_args, use_real=rope_real)
-        log_fn(f"    Using image RoPE ({rope_img}) ({'real' if rope_real else 'complex'}): {sub_args} | ({reso}) "
-               f"{freqs_cis_img[str(reso)][0].shape if rope_real else freqs_cis_img[str(reso)].shape}")
-    return freqs_cis_img

py/dit/hunyuanDiT/mt5_tokenizer/config.json

@@ -1,33 +0,0 @@
-{
-  "_name_or_path": "mt5",
-  "architectures": [
-    "MT5ForConditionalGeneration"
-  ],
-  "classifier_dropout": 0.0,
-  "d_ff": 5120,
-  "d_kv": 64,
-  "d_model": 2048,
-  "decoder_start_token_id": 0,
-  "dense_act_fn": "gelu_new",
-  "dropout_rate": 0.1,
-  "eos_token_id": 1,
-  "feed_forward_proj": "gated-gelu",
-  "initializer_factor": 1.0,
-  "is_encoder_decoder": true,
-  "is_gated_act": true,
-  "layer_norm_epsilon": 1e-06,
-  "model_type": "mt5",
-  "num_decoder_layers": 24,
-  "num_heads": 32,
-  "num_layers": 24,
-  "output_past": true,
-  "pad_token_id": 0,
-  "relative_attention_max_distance": 128,
-  "relative_attention_num_buckets": 32,
-  "tie_word_embeddings": false,
-  "tokenizer_class": "T5Tokenizer",
-  "torch_dtype": "float16",
-  "transformers_version": "4.40.2",
-  "use_cache": true,
-  "vocab_size": 250112
-}

py/dit/hunyuanDiT/mt5_tokenizer/special_tokens_map.json

@@ -1 +0,0 @@
-{"eos_token": "</s>", "unk_token": "<unk>", "pad_token": "<pad>"}

py/dit/hunyuanDiT/mt5_tokenizer/tokenizer_config.json

@@ -1 +0,0 @@
-{"eos_token": "</s>", "unk_token": "<unk>", "pad_token": "<pad>", "extra_ids": 0, "additional_special_tokens": null, "special_tokens_map_file": "/home/patrick/.cache/torch/transformers/685ac0ca8568ec593a48b61b0a3c272beee9bc194a3c7241d15dcadb5f875e53.f76030f3ec1b96a8199b2593390c610e76ca8028ef3d24680000619ffb646276", "tokenizer_file": null, "name_or_path": "google/mt5-small"}

py/dit/hunyuanDiT/tokenizer/config.json

@@ -1,34 +0,0 @@
-{
-  "_name_or_path": "hfl/chinese-roberta-wwm-ext-large",
-  "architectures": [
-    "BertModel"
-  ],
-  "attention_probs_dropout_prob": 0.1,
-  "bos_token_id": 0,
-  "classifier_dropout": null,
-  "directionality": "bidi",
-  "eos_token_id": 2,
-  "hidden_act": "gelu",
-  "hidden_dropout_prob": 0.1,
-  "hidden_size": 1024,
-  "initializer_range": 0.02,
-  "intermediate_size": 4096,
-  "layer_norm_eps": 1e-12,
-  "max_position_embeddings": 512,
-  "model_type": "bert",
-  "num_attention_heads": 16,
-  "num_hidden_layers": 24,
-  "output_past": true,
-  "pad_token_id": 0,
-  "pooler_fc_size": 768,
-  "pooler_num_attention_heads": 12,
-  "pooler_num_fc_layers": 3,
-  "pooler_size_per_head": 128,
-  "pooler_type": "first_token_transform",
-  "position_embedding_type": "absolute",
-  "torch_dtype": "float32",
-  "transformers_version": "4.22.1",
-  "type_vocab_size": 2,
-  "use_cache": true,
-  "vocab_size": 47020
-}

py/dit/hunyuanDiT/tokenizer/special_tokens_map.json

@@ -1,7 +0,0 @@
-{
-  "cls_token": "[CLS]",
-  "mask_token": "[MASK]",
-  "pad_token": "[PAD]",
-  "sep_token": "[SEP]",
-  "unk_token": "[UNK]"
-}

py/dit/hunyuanDiT/tokenizer/tokenizer_config.json

@@ -1,16 +0,0 @@
-{
-  "cls_token": "[CLS]",
-  "do_basic_tokenize": true,
-  "do_lower_case": true,
-  "mask_token": "[MASK]",
-  "name_or_path": "hfl/chinese-roberta-wwm-ext",
-  "never_split": null,
-  "pad_token": "[PAD]",
-  "sep_token": "[SEP]",
-  "special_tokens_map_file": "/home/chenweifeng/.cache/huggingface/hub/models--hfl--chinese-roberta-wwm-ext/snapshots/5c58d0b8ec1d9014354d691c538661bf00bfdb44/special_tokens_map.json",
-  "strip_accents": null,
-  "tokenize_chinese_chars": true,
-  "tokenizer_class": "BertTokenizer",
-  "unk_token": "[UNK]",
-  "model_max_length": 77
-}

py/dit/pixArt/loader.py

@@ -118,7 +118,6 @@ def load_pixart(model_path, model_conf=None):
         model,
         load_device=load_device,
         offload_device=offload_device,
-        current_device="cpu",
     )
     return model_patcher
 

py/dynamiCrafter/utils/model_utils.py

@@ -7,7 +7,10 @@ from comfy import model_base
 from comfy import utils
 from comfy import diffusers_convert
 
-from comfy import sd2_clip
+try:
+    import comfy.text_encoders.sd2_clip
+except ImportError:
+    from comfy import sd2_clip
 
 from comfy import supported_models_base
 from comfy import latent_formats

py/fooocus/__init__.py

@@ -0,0 +1,139 @@
+#credit to Acly for this module
+#from https://github.com/Acly/comfyui-inpaint-nodes
+import torch
+import torch.nn.functional as F
+import comfy
+from comfy.model_base import BaseModel
+from comfy.model_patcher import ModelPatcher
+from comfy.model_management import cast_to_device
+
+from ..libs.log import log_node_warn, log_node_error, log_node_info
+
+class InpaintHead(torch.nn.Module):
+    def __init__(self, *args, **kwargs):
+        super().__init__(*args, **kwargs)
+        self.head = torch.nn.Parameter(torch.empty(size=(320, 5, 3, 3), device="cpu"))
+
+    def __call__(self, x):
+        x = F.pad(x, (1, 1, 1, 1), "replicate")
+        return F.conv2d(x, weight=self.head)
+
+# injected_model_patcher_calculate_weight = False
+# original_calculate_weight = None
+
+class applyFooocusInpaint:
+    def calculate_weight_patched(self, patches, weight, key, intermediate_dtype=torch.float32):
+        remaining = []
+
+        for p in patches:
+            alpha = p[0]
+            v = p[1]
+
+            is_fooocus_patch = isinstance(v, tuple) and len(v) == 2 and v[0] == "fooocus"
+            if not is_fooocus_patch:
+                remaining.append(p)
+                continue
+
+            if alpha != 0.0:
+                v = v[1]
+                w1 = cast_to_device(v[0], weight.device, torch.float32)
+                if w1.shape == weight.shape:
+                    w_min = cast_to_device(v[1], weight.device, torch.float32)
+                    w_max = cast_to_device(v[2], weight.device, torch.float32)
+                    w1 = (w1 / 255.0) * (w_max - w_min) + w_min
+                    weight += alpha * cast_to_device(w1, weight.device, weight.dtype)
+                else:
+                    print(
+                        f"[ApplyFooocusInpaint] Shape mismatch {key}, weight not merged ({w1.shape} != {weight.shape})"
+                    )
+
+        if len(remaining) > 0:
+            return original_calculate_weight(remaining, weight, key, intermediate_dtype)
+        return weight
+
+    def __enter__(self):
+        try:
+            print("[comfyui-easy-use] Injecting patched comfy.lora.calculate_weight.calculate_weight")
+            self.original_calculate_weight = comfy.lora.calculate_weight
+            comfy.lora.calculate_weight = self.calculate_weight_patched
+        except AttributeError:
+            print("[comfyui-easy-use] Injecting patched comfy.model_patcher.ModelPatcher.calculate_weight")
+            self.original_calculate_weight = ModelPatcher.calculate_weight
+            ModelPatcher.calculate_weight = self.calculate_weight_patched
+
+    def __exit__(self, exc_type, exc_value, traceback):
+        try:
+            comfy.lora.calculate_weight = self.original_calculate_weight
+        except:
+            ModelPatcher.calculate_weight = self.original_calculate_weight
+
+# def inject_patched_calculate_weight():
+#     global injected_model_patcher_calculate_weight
+#     if not injected_model_patcher_calculate_weight:
+#         try:
+#             print("[comfyui-easy-use] Injecting patched comfy.lora.calculate_weight.calculate_weight")
+#             original_calculate_weight = comfy.lora.calculate_weight
+#             comfy.lora.original_calculate_weight = original_calculate_weight
+#             comfy.lora.calculate_weight = calculate_weight_patched
+#         except AttributeError:
+#             print("[comfyui-easy-use] Injecting patched comfy.model_patcher.ModelPatcher.calculate_weight")
+#             original_calculate_weight = ModelPatcher.calculate_weight
+#             ModelPatcher.original_calculate_weight = original_calculate_weight
+#             ModelPatcher.calculate_weight = calculate_weight_patched
+#         injected_model_patcher_calculate_weight = True
+
+
+class InpaintWorker:
+    def __init__(self, node_name):
+        self.node_name = node_name if node_name is not None else ""
+
+    def load_fooocus_patch(self, lora: dict, to_load: dict):
+        patch_dict = {}
+        loaded_keys = set()
+        for key in to_load.values():
+            if value := lora.get(key, None):
+                patch_dict[key] = ("fooocus", value)
+                loaded_keys.add(key)
+
+        not_loaded = sum(1 for x in lora if x not in loaded_keys)
+        if not_loaded > 0:
+            log_node_info(self.node_name,
+                f"{len(loaded_keys)} Lora keys loaded, {not_loaded} remaining keys not found in model."
+            )
+        return patch_dict
+
+    def _input_block_patch(self, h: torch.Tensor, transformer_options: dict):
+        if transformer_options["block"][1] == 0:
+            if self._inpaint_block is None or self._inpaint_block.shape != h.shape:
+                assert self._inpaint_head_feature is not None
+                batch = h.shape[0] // self._inpaint_head_feature.shape[0]
+                self._inpaint_block = self._inpaint_head_feature.to(h).repeat(batch, 1, 1, 1)
+            h = h + self._inpaint_block
+        return h
+
+    def patch(self, model, latent, patch):
+        base_model: BaseModel = model.model
+        latent_pixels = base_model.process_latent_in(latent["samples"])
+        noise_mask = latent["noise_mask"].round()
+        latent_mask = F.max_pool2d(noise_mask, (8, 8)).round().to(latent_pixels)
+
+        inpaint_head_model, inpaint_lora = patch
+        feed = torch.cat([latent_mask, latent_pixels], dim=1)
+        inpaint_head_model.to(device=feed.device, dtype=feed.dtype)
+        self._inpaint_head_feature = inpaint_head_model(feed)
+        self._inpaint_block = None
+
+        lora_keys = comfy.lora.model_lora_keys_unet(model.model, {})
+        lora_keys.update({x: x for x in base_model.state_dict().keys()})
+        loaded_lora = self.load_fooocus_patch(inpaint_lora, lora_keys)
+
+        m = model.clone()
+        m.set_model_input_block_patch(self._input_block_patch)
+        patched = m.add_patches(loaded_lora, 1.0)
+        m.model_options['transformer_options']['fooocus'] = True
+        not_patched_count = sum(1 for x in loaded_lora if x not in patched)
+        if not_patched_count > 0:
+            log_node_error(self.node_name, f"Failed to patch {not_patched_count} keys")
+
+        # inject_patched_calculate_weight()
+        return (m,)

py/human_parsing/run_parsing.py

@@ -1,3 +1,6 @@
+import numpy as np
+import torch
+from PIL import Image
 from .parsing_api import onnx_inference
 from ..libs.utils import install_package
 
@@ -21,3 +24,86 @@ class HumanParsing:
         parsed_image, mask = onnx_inference(self.session, input_image, mask_components)
         return parsed_image, mask
 
+
+class HumanParts:
+
+    def __init__(self, model_path):
+        self.model_path = model_path
+        self.session = None
+        # self.classes_dict = {
+        #     "background": 0,
+        #     "hair": 2,
+        #     "glasses": 4,
+        #     "top-clothes": 5,
+        #     "bottom-clothes": 9,
+        #     "torso-skin": 10,
+        #     "face": 13,
+        #     "left-arm": 14,
+        #     "right-arm": 15,
+        #     "left-leg": 16,
+        #     "right-leg": 17,
+        #     "left-foot": 18,
+        #     "right-foot": 19,
+        # },
+        self.classes = [0, 13, 2, 4, 5, 9, 10, 14, 15, 16, 17, 18, 19]
+
+
+    def __call__(self, input_image, mask_components):
+        if self.session is None:
+            install_package('onnxruntime')
+            import onnxruntime as ort
+
+            self.session = ort.InferenceSession(self.model_path, providers=['TensorrtExecutionProvider', 'CUDAExecutionProvider', 'CPUExecutionProvider'])
+
+        mask, = self.get_mask(self.session, input_image, 0, mask_components)
+        return mask
+
+    def get_mask(self, model, image, rotation, mask_components):
+        image = image.squeeze(0)
+        image_np = image.numpy() * 255
+
+        pil_image = Image.fromarray(image_np.astype(np.uint8))
+        original_size = pil_image.size  # to resize the mask later
+        # resize to 512x512 as the model expects
+        pil_image = pil_image.resize((512, 512))
+        center = (256, 256)
+
+        if rotation != 0:
+            pil_image = pil_image.rotate(rotation, center=center)
+
+        # normalize the image
+        image_np = np.array(pil_image).astype(np.float32) / 127.5 - 1
+        image_np = np.expand_dims(image_np, axis=0)
+
+        # use the onnx model to get the mask
+        input_name = model.get_inputs()[0].name
+        output_name = model.get_outputs()[0].name
+        result = model.run([output_name], {input_name: image_np})
+        result = np.array(result[0]).argmax(axis=3).squeeze(0)
+
+        score: int = 0
+
+        mask = np.zeros_like(result)
+        for class_index in mask_components:
+            detected = result == self.classes[class_index]
+            mask[detected] = 255
+            score += mask.sum()
+
+        # back to the original size
+        mask_image = Image.fromarray(mask.astype(np.uint8), mode="L")
+        if rotation != 0:
+            mask_image = mask_image.rotate(-rotation, center=center)
+
+        mask_image = mask_image.resize(original_size)
+
+        # and back to numpy...
+        mask = np.array(mask_image).astype(np.float32) / 255
+
+        # add 2 dimensions to match the expected output
+        mask = np.expand_dims(mask, axis=0)
+        mask = np.expand_dims(mask, axis=0)
+        # ensure to return a "binary mask_image"
+
+        del image_np, result  # free up memory, maybe not necessary
+
+        return (torch.from_numpy(mask.astype(np.uint8)),)

py/ic_light/__init__.py

@@ -8,10 +8,9 @@ import comfy.model_management
 from comfy.sd import load_unet
 from comfy.ldm.models.autoencoder import AutoencoderKL
 from comfy.model_base import BaseModel
+from comfy.model_patcher import ModelPatcher
 from PIL import Image
 from nodes import VAEEncode
-
-from ..layer_diffuse.model import ModelPatcher, calculate_weight_adjust_channel
 from ..libs.image import np2tensor, pil2tensor
 
 class UnetParams(TypedDict):
@@ -20,7 +19,6 @@ class UnetParams(TypedDict):
     c: dict
     cond_or_uncond: torch.Tensor
 
-
 class VAEEncodeArgMax(VAEEncode):
     def encode(self, vae, pixels):
         assert isinstance(
@@ -141,12 +139,7 @@ class ICLight:
                 return image
 
 
-    def apply(self, ic_model_path, model: ModelPatcher, c_concat: dict, ic_model=None) -> Tuple[ModelPatcher]:
-        try:
-            ModelPatcher.calculate_weight = calculate_weight_adjust_channel(ModelPatcher.calculate_weight)
-        except:
-            pass
-
+    def apply(self, ic_model_path, model, c_concat: dict, ic_model=None) -> Tuple[ModelPatcher]:
         device = comfy.model_management.get_torch_device()
         dtype = comfy.model_management.unet_dtype()
         work_model = model.clone()
@@ -179,7 +172,13 @@ class ICLight:
 
         work_model.add_patches(
             patches={
-                ("diffusion_model." + key): (value.to(dtype=dtype, device=device),)
+                ("diffusion_model." + key): (
+                    'diff',
+                    [
+                        value.to(dtype=dtype, device=device),
+                        {"pad_weight": key == 'input_blocks.0.0.weight'}
+                    ]
+                )
                 for key, value in ic_model_state_dict.items()
             }
         )

py/image.py

@@ -7,7 +7,7 @@ import comfy.utils
 import comfy.model_management
 from comfy_extras.nodes_compositing import JoinImageWithAlpha
 from server import PromptServer
-from nodes import MAX_RESOLUTION
+from nodes import MAX_RESOLUTION, NODE_CLASS_MAPPINGS as ALL_NODE_CLASS_MAPPINGS
 from PIL import Image, ImageDraw, ImageFilter
 from torchvision.transforms import Resize, CenterCrop, GaussianBlur
 from torchvision.transforms.functional import to_pil_image
@@ -18,7 +18,7 @@ from .libs.colorfix import adain_color_fix, wavelet_color_fix
 from .libs.chooser import ChooserMessage, ChooserCancelled
 from .config import REMBG_DIR, REMBG_MODELS, HUMANPARSING_MODELS, MEDIAPIPE_MODELS, MEDIAPIPE_DIR
 
-
+any_type = AlwaysEqualProxy("*")
 # 图像数量
 class imageCount:
   @classmethod
@@ -318,6 +318,30 @@ class imageScaleDownToSize(imageScaleDownBy):
     scale_by = min(scale_by, 1.0)
     return self.image_scale_down_by(images, scale_by)
 
+class imageScaleToNormPixels:
+  upscale_methods = ["nearest-exact", "bilinear", "area", "bicubic", "lanczos"]
+  @classmethod
+  def INPUT_TYPES(s):
+    return {
+      "required": {
+        "image": ("IMAGE",),
+        "upscale_method": (s.upscale_methods,),
+        "scale_by": ("FLOAT", {"default": 1.0, "min": 0.01, "max": 8.0, "step": 0.01}),
+      }
+    }
+
+  RETURN_TYPES = ("IMAGE",)
+  RETURN_NAMES = ("image",)
+  FUNCTION = "scale"
+  CATEGORY = "EasyUse/Image"
+
+  def scale(self, image, upscale_method, scale_by):
+    height, width = image.shape[1:3]
+    width = int(width * scale_by - width * scale_by % 8)
+    height = int(height * scale_by - height * scale_by % 8)
+    upscale_image_cls = ALL_NODE_CLASS_MAPPINGS['ImageScale']
+    image, = upscale_image_cls().upscale(image, upscale_method, width, height, "disabled")
+    return (image,)
 
 # 图像比率
 class imageRatio:
@@ -404,46 +428,6 @@ class imagePixelPerfect:
 
     return {"ui": {"text": text}, "result": (result,)}
 
-# 图片到遮罩
-class imageToMask:
-  @classmethod
-  def INPUT_TYPES(s):
-    return {"required": {
-        "image": ("IMAGE",),
-        "channel": (['red', 'green', 'blue'],),
-       }
-    }
-
-  RETURN_TYPES = ("MASK",)
-  FUNCTION = "convert"
-  CATEGORY = "EasyUse/Image"
-
-  def convert_to_single_channel(self, image, channel='red'):
-    # Convert to RGB mode to access individual channels
-    image = image.convert('RGB')
-
-    # Extract the desired channel and convert to greyscale
-    if channel == 'red':
-      channel_img = image.split()[0].convert('L')
-    elif channel == 'green':
-      channel_img = image.split()[1].convert('L')
-    elif channel == 'blue':
-      channel_img = image.split()[2].convert('L')
-    else:
-      raise ValueError(
-        "Invalid channel option. Please choose 'red', 'green', or 'blue'.")
-
-    # Convert the greyscale channel back to RGB mode
-    channel_img = Image.merge(
-      'RGB', (channel_img, channel_img, channel_img))
-
-    return channel_img
-
-  def convert(self, image, channel='red'):
-    image = self.convert_to_single_channel(tensor2pil(image), channel)
-    image = pil2tensor(image)
-    return (image.squeeze().mean(2),)
-
 # 图像保存 (简易)
 from nodes import PreviewImage, SaveImage
 class imageSaveSimple:
@@ -477,6 +461,34 @@ class imageSaveSimple:
     else:
       return SaveImage().save_images(images, filename_prefix, prompt, extra_pnginfo)
 
+class imageSaveWithText(SaveImage):
+
+  @classmethod
+  def INPUT_TYPES(s):
+    input_types = SaveImage.INPUT_TYPES()
+    input_types['optional'] = {
+      "text": ("STRING", {"default": "", "forceInput": True})
+    }
+    return input_types
+
+  RETURN_TYPES = ("IMAGE", "STRING")
+  RETURN_NAMES = ('image', "text")
+
+  FUNCTION = "save"
+  OUTPUT_NODE = True
+  CATEGORY = "EasyUse/Image"
+
+  def save(self, images, filename_prefix="ComfyUI", text=None, prompt=None, extra_pnginfo=None):
+    result = self.save_images(images, filename_prefix, prompt, extra_pnginfo)
+    if text is not None:
+      for image in result['ui']['images']:
+        path = os.path.join(folder_paths.output_directory, image['subfolder'], image['filename'])
+        text_path = os.path.splitext(path)[0] + '.txt'
+        with open(text_path, 'w') as f:
+            f.write(text)
+      result['result']['text'] = text
+    result['result']['images'] = images
+    return result
 
 # 图像批次合并
 class JoinImageBatch:
@@ -619,6 +631,65 @@ class imageSplitGrid:
 
     return (torch.cat(new_images, dim=0),)
 
+class imageSplitTiles:
+
+  @classmethod
+  def INPUT_TYPES(s):
+    return {
+      "required": {
+        "image": ("IMAGE",),
+        "overlap_ratio": ("FLOAT", {"default": 0, "min": 0, "max": 0.5, "step": 0.01, }),
+        "overlap_offset": ("INT", {"default": 0, "min": - MAX_RESOLUTION // 2, "max": MAX_RESOLUTION // 2, "step": 1, }),
+        "tiles_num": ("INT", {"default": 2, "min": 2, "max": 50, "step": 1}),
+      },
+      "optional": {
+        "norm": ("BOOLEAN", {"default": True}),
+      }
+    }
+
+  RETURN_TYPES = ("IMAGE", "MASK", "INT", "INT")
+  RETURN_NAMES = ("tiles", "masks", "overlap_x", "overlap_y")
+  FUNCTION = "doit"
+  CATEGORY = "EasyUse/Image"
+
+  def doit(self, image, overlap_ratio, overlap_offset, tiles_num, norm=True):
+    height, width = image.shape[1:3]
+
+    is_landscape = width >= height
+
+    tite_w = width // tiles_num
+    tile_h = height // tiles_num
+    overlap = int(tite_w * overlap_ratio) + overlap_offset  if is_landscape else int(tile_h * overlap_ratio) + overlap_offset
+    overlap_w = tite_w + overlap if is_landscape else width
+    overlap_h = height if is_landscape else tile_h + overlap
+    if norm:
+      overlap_w = int(overlap_w - overlap_w % 8)
+      overlap_h = int(overlap_h - overlap_h % 8)
+    else:
+      overlap_w = int(overlap_w)
+      overlap_h = int(overlap_h)
+    cls = ALL_NODE_CLASS_MAPPINGS['ImageCrop']
+    solid_mask_cls = ALL_NODE_CLASS_MAPPINGS['SolidMask']
+    feather_mask_cls = ALL_NODE_CLASS_MAPPINGS['FeatherMask']
+
+    overlap_x = int((width - overlap_w) / (tiles_num - 1)) if is_landscape else 0
+    overlap_y = 0 if is_landscape else int((height - overlap_h) / (tiles_num - 1))
+
+    tiles, masks = [], []
+    for i in range(tiles_num):
+      tile, = cls().crop(image, overlap_w, overlap_h, int(overlap_x * i), int(overlap_y * i))
+      tiles.append(tile)
+      fearing_left = int(overlap) if overlap_x * i > 0 else 0
+      fearing_top = int(overlap) if overlap_y * i > 0 else 0
+      mask, = solid_mask_cls().solid(1, overlap_w, overlap_h)
+      mask, = feather_mask_cls().feather(mask, fearing_left, fearing_top, 0, 0)
+      masks.append(mask)
+
+    tiles = torch.cat(tiles, dim=0)
+    masks = torch.cat(masks, dim=0)
+
+    return (tiles, masks, overlap_x, overlap_y)
+
 class imagesSplitImage:
     @classmethod
     def INPUT_TYPES(s):
@@ -637,7 +708,6 @@ class imagesSplitImage:
       new_images = torch.chunk(images, len(images), dim=0)
       return new_images
 
-
 class imageConcat:
   @classmethod
   def INPUT_TYPES(s):
@@ -674,9 +744,13 @@ class imageRemBg:
     return {
       "required": {
         "images": ("IMAGE",),
-        "rem_mode": (("RMBG-1.4",),),
+        "rem_mode": (("RMBG-1.4","Inspyrenet"),),
         "image_output": (["Hide", "Preview", "Save", "Hide/Save"], {"default": "Preview"}),
         "save_prefix": ("STRING", {"default": "ComfyUI"}),
+
+      },
+      "optional":{
+        "torchscript_jit": ("BOOLEAN", {"default": False}),
       },
       "hidden": {"prompt": "PROMPT", "extra_pnginfo": "EXTRA_PNGINFO"},
     }
@@ -688,7 +762,9 @@ class imageRemBg:
 
   CATEGORY = "EasyUse/Image"
 
-  def remove(self, rem_mode, images, image_output, save_prefix, prompt=None, extra_pnginfo=None):
+  def remove(self, rem_mode, images, image_output, save_prefix, torchscript_jit=False, prompt=None, extra_pnginfo=None):
+    new_images = list()
+    masks = list()
     if rem_mode == "RMBG-1.4":
       # load model
       model_url = REMBG_MODELS[rem_mode]['model_url']
@@ -702,8 +778,6 @@ class imageRemBg:
       net.eval()
       # prepare input
       model_input_size = [1024, 1024]
-      new_images = list()
-      masks = list()
       for image in images:
         orig_im = tensor2pil(image)
         w, h = orig_im.size
@@ -721,18 +795,33 @@ class imageRemBg:
       new_images = torch.cat(new_images, dim=0)
       masks = torch.cat(masks, dim=0)
 
+    elif rem_mode == "Inspyrenet":
+      from tqdm import tqdm
+      try:
+        from transparent_background import Remover
+      except:
+          install_package("transparent_background")
+          from transparent_background import Remover
 
-      results = easySave(new_images, save_prefix, image_output, prompt, extra_pnginfo)
+      remover = Remover(jit=torchscript_jit)
 
-      if image_output in ("Hide", "Hide/Save"):
-        return {"ui": {},
-                "result": (new_images, masks)}
+      for img in tqdm(images, "Inspyrenet Rembg"):
+        mid = remover.process(tensor2pil(img), type='rgba')
+        out = pil2tensor(mid)
+        new_images.append(out)
+        mask = out[:, :, :, 3]
+        masks.append(mask)
+      new_images = torch.cat(new_images, dim=0)
+      masks = torch.cat(masks, dim=0)
 
-      return {"ui": {"images": results},
+    results = easySave(new_images, save_prefix, image_output, prompt, extra_pnginfo)
+
+    if image_output in ("Hide", "Hide/Save"):
+      return {"ui": {},
               "result": (new_images, masks)}
 
-    else:
-      return (None, None)
+    return {"ui": {"images": results},
+            "result": (new_images, masks)}
 
 # 图像选择器
 class imageChooser(PreviewImage):
@@ -985,12 +1074,12 @@ class imageInterrogator:
     RETURN_NAMES = ("prompt",)
     FUNCTION = "interrogate"
     CATEGORY = "EasyUse/Image"
-    OUTPUT_NODE = True
+    OUTPUT_NODE = False
     OUTPUT_IS_LIST = (True,)
 
     def interrogate(self, image, mode, use_lowvram=False):
       prompt = ci.image_to_prompt(image, mode, low_vram=use_lowvram)
-      return {"ui":{"text":prompt},"result":(prompt,)}
+      return (prompt,)
 
 # 人类分割器
 class humanSegmentation:
@@ -1001,7 +1090,7 @@ class humanSegmentation:
         return {
           "required":{
             "image": ("IMAGE",),
-            "method": (["selfie_multiclass_256x256", "human_parsing_lip"],),
+            "method": (["selfie_multiclass_256x256", "human_parsing_lip", "human_parts (deeplabv3p)"],),
             "confidence": ("FLOAT", {"default": 0.4, "min": 0.05, "max": 0.95, "step": 0.01},),
             "crop_multi": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 10.0, "step": 0.001},),
           },
@@ -1129,6 +1218,20 @@ class humanSegmentation:
 
         output_image, = JoinImageWithAlpha().join_image_with_alpha(image, alpha)
 
+      elif method == "human_parts (deeplabv3p)":
+        from .human_parsing.run_parsing import HumanParts
+        onnx_path = os.path.join(folder_paths.models_dir, 'onnx')
+        human_parts_path = os.path.join(onnx_path, 'human-parts')
+        model_path = get_local_filepath(HUMANPARSING_MODELS['human-parts']['model_url'], human_parts_path)
+        parsing = HumanParts(model_path=model_path)
+
+        mask, = parsing(image, mask_components)
+
+        alpha = 1.0 - mask
+
+        output_image, = JoinImageWithAlpha().join_image_with_alpha(image, alpha)
+
+
       # use crop
       bbox = [[0, 0, 0, 0]]
       if crop_multi > 0.0:
@@ -1136,7 +1239,6 @@ class humanSegmentation:
 
       return (output_image, mask, bbox)
 
-
 class imageCropFromMask:
     @classmethod
     def INPUT_TYPES(s):
@@ -1466,7 +1568,7 @@ class removeLocalImage:
   def INPUT_TYPES(s):
       return {
         "required": {
-          "any": (AlwaysEqualProxy("*"),),
+          "any": (any_type,),
           "file_name": ("STRING",{"default":""}),
         },
       }
@@ -1504,46 +1606,46 @@ class removeLocalImage:
 
 
 # 姿势编辑器
-class poseEditor:
-  @classmethod
-  def INPUT_TYPES(self):
-    temp_dir = folder_paths.get_temp_directory()
-
-    if not os.path.isdir(temp_dir):
-      os.makedirs(temp_dir)
-
-    temp_dir = folder_paths.get_temp_directory()
-
-    return {"required":
-              {"image": (sorted(os.listdir(temp_dir)),)},
-            }
-
-  RETURN_TYPES = ("IMAGE",)
-  FUNCTION = "output_pose"
-
-  CATEGORY = "EasyUse/Image"
-
-  def output_pose(self, image):
-    image_path = os.path.join(folder_paths.get_temp_directory(), image)
-    # print(f"Create: {image_path}")
-
-    i = Image.open(image_path)
-    image = i.convert("RGB")
-    image = np.array(image).astype(np.float32) / 255.0
-    image = torch.from_numpy(image)[None,]
-
-    return (image,)
-
-  @classmethod
-  def IS_CHANGED(self, image):
-    image_path = os.path.join(
-      folder_paths.get_temp_directory(), image)
-    # print(f'Change: {image_path}')
-
-    m = hashlib.sha256()
-    with open(image_path, 'rb') as f:
-      m.update(f.read())
-    return m.digest().hex()
+# class poseEditor:
+#   @classmethod
+#   def INPUT_TYPES(self):
+#     temp_dir = folder_paths.get_temp_directory()
+#
+#     if not os.path.isdir(temp_dir):
+#       os.makedirs(temp_dir)
+#
+#     temp_dir = folder_paths.get_temp_directory()
+#
+#     return {"required":
+#               {"image": (sorted(os.listdir(temp_dir)),)},
+#             }
+#
+#   RETURN_TYPES = ("IMAGE",)
+#   FUNCTION = "output_pose"
+#
+#   CATEGORY = "EasyUse/🚫 Deprecated"
+#
+#   def output_pose(self, image):
+#     image_path = os.path.join(folder_paths.get_temp_directory(), image)
+#     # print(f"Create: {image_path}")
+#
+#     i = Image.open(image_path)
+#     image = i.convert("RGB")
+#     image = np.array(image).astype(np.float32) / 255.0
+#     image = torch.from_numpy(image)[None,]
+#
+#     return (image,)
+#
+#   @classmethod
+#   def IS_CHANGED(self, image):
+#     image_path = os.path.join(
+#       folder_paths.get_temp_directory(), image)
+#     # print(f'Change: {image_path}')
+#
+#     m = hashlib.sha256()
+#     with open(image_path, 'rb') as f:
+#       m.update(f.read())
+#     return m.digest().hex()
 
 NODE_CLASS_MAPPINGS = {
   "easy imageInsetCrop": imageInsetCrop,
@@ -1555,17 +1657,19 @@ NODE_CLASS_MAPPINGS = {
   "easy imageScaleDown": imageScaleDown,
   "easy imageScaleDownBy": imageScaleDownBy,
   "easy imageScaleDownToSize": imageScaleDownToSize,
+  "easy imageScaleToNormPixels": imageScaleToNormPixels,
   "easy imageRatio": imageRatio,
-  "easy imageToMask": imageToMask,
   "easy imageConcat": imageConcat,
   "easy imageListToImageBatch": imageListToImageBatch,
   "easy imageBatchToImageList": imageBatchToImageList,
   "easy imageSplitList": imageSplitList,
   "easy imageSplitGrid": imageSplitGrid,
   "easy imagesSplitImage": imagesSplitImage,
+  "easy imageSplitTiles": imageSplitTiles,
   "easy imageCropFromMask": imageCropFromMask,
   "easy imageUncropFromBBOX": imageUncropFromBBOX,
   "easy imageSave": imageSaveSimple,
+  # "easy imageSaveWithText": imageSaveWithText,
   "easy imageRemBg": imageRemBg,
   "easy imageChooser": imageChooser,
   "easy imageColorMatch": imageColorMatch,
@@ -1576,7 +1680,6 @@ NODE_CLASS_MAPPINGS = {
   "easy joinImageBatch": JoinImageBatch,
   "easy humanSegmentation": humanSegmentation,
   "easy removeLocalImage": removeLocalImage,
-  "easy poseEditor": poseEditor
 }
 
 NODE_DISPLAY_NAME_MAPPINGS = {
@@ -1589,18 +1692,20 @@ NODE_DISPLAY_NAME_MAPPINGS = {
   "easy imageScaleDown": "Image Scale Down",
   "easy imageScaleDownBy": "Image Scale Down By",
   "easy imageScaleDownToSize": "Image Scale Down To Size",
+  "easy imageScaleToNormPixels": "ImageScaleToNormPixels",
   "easy imageRatio": "ImageRatio",
-  "easy imageToMask": "ImageToMask",
   "easy imageHSVMask": "ImageHSVMask",
   "easy imageConcat": "imageConcat",
   "easy imageListToImageBatch": "Image List To Image Batch",
   "easy imageBatchToImageList": "Image Batch To Image List",
   "easy imageSplitList": "imageSplitList",
   "easy imageSplitGrid": "imageSplitGrid",
+  "easy imageSplitTiles": "imageSplitTiles",
   "easy imagesSplitImage": "imagesSplitImage",
   "easy imageCropFromMask": "imageCropFromMask",
   "easy imageUncropFromBBOX": "imageUncropFromBBOX",
-  "easy imageSave": "SaveImage (Simple)",
+  "easy imageSave": "Save Image (Simple)",
+  # "easy imageSaveWithText": "Save Image With Text",
   "easy imageRemBg": "Image Remove Bg",
   "easy imageChooser": "Image Chooser",
   "easy imageColorMatch": "Image Color Match",
@@ -1611,5 +1716,4 @@ NODE_DISPLAY_NAME_MAPPINGS = {
   "easy imageToBase64": "Image To Base64",
   "easy humanSegmentation": "Human Segmentation",
   "easy removeLocalImage": "Remove Local Image",
-  "easy poseEditor": "PoseEditor",
 }

py/kolors/chatglm/config_chatglm.json

@@ -0,0 +1,42 @@
+{
+  "_name_or_path": "THUDM/chatglm3-6b-base",
+  "model_type": "chatglm",
+  "architectures": [
+    "ChatGLMModel"
+  ],
+  "auto_map": {
+    "AutoConfig": "configuration_chatglm.ChatGLMConfig",
+    "AutoModel": "modeling_chatglm.ChatGLMForConditionalGeneration",
+    "AutoModelForCausalLM": "modeling_chatglm.ChatGLMForConditionalGeneration",
+    "AutoModelForSeq2SeqLM": "modeling_chatglm.ChatGLMForConditionalGeneration",
+    "AutoModelForSequenceClassification": "modeling_chatglm.ChatGLMForSequenceClassification"
+  },
+  "add_bias_linear": false,
+  "add_qkv_bias": true,
+  "apply_query_key_layer_scaling": true,
+  "apply_residual_connection_post_layernorm": false,
+  "attention_dropout": 0.0,
+  "attention_softmax_in_fp32": true,
+  "bias_dropout_fusion": true,
+  "ffn_hidden_size": 13696,
+  "fp32_residual_connection": false,
+  "hidden_dropout": 0.0,
+  "hidden_size": 4096,
+  "kv_channels": 128,
+  "layernorm_epsilon": 1e-05,
+  "multi_query_attention": true,
+  "multi_query_group_num": 2,
+  "num_attention_heads": 32,
+  "num_layers": 28,
+  "original_rope": true,
+  "padded_vocab_size": 65024,
+  "post_layer_norm": true,
+  "rmsnorm": true,
+  "seq_length": 32768,
+  "use_cache": true,
+  "torch_dtype": "float16",
+  "transformers_version": "4.30.2",
+  "tie_word_embeddings": false,
+  "eos_token_id": 2,
+  "pad_token_id": 0
+}

py/kolors/chatglm/configuration_chatglm.py

@@ -0,0 +1,60 @@
+from transformers import PretrainedConfig
+
+class ChatGLMConfig(PretrainedConfig):
+    model_type = "chatglm"
+    def __init__(
+        self,
+        num_layers=28,
+        padded_vocab_size=65024,
+        hidden_size=4096,
+        ffn_hidden_size=13696,
+        kv_channels=128,
+        num_attention_heads=32,
+        seq_length=2048,
+        hidden_dropout=0.0,
+        classifier_dropout=None,
+        attention_dropout=0.0,
+        layernorm_epsilon=1e-5,
+        rmsnorm=True,
+        apply_residual_connection_post_layernorm=False,
+        post_layer_norm=True,
+        add_bias_linear=False,
+        add_qkv_bias=False,
+        bias_dropout_fusion=True,
+        multi_query_attention=False,
+        multi_query_group_num=1,
+        apply_query_key_layer_scaling=True,
+        attention_softmax_in_fp32=True,
+        fp32_residual_connection=False,
+        quantization_bit=0,
+        pre_seq_len=None,
+        prefix_projection=False,
+        **kwargs
+    ):
+        self.num_layers = num_layers
+        self.vocab_size = padded_vocab_size
+        self.padded_vocab_size = padded_vocab_size
+        self.hidden_size = hidden_size
+        self.ffn_hidden_size = ffn_hidden_size
+        self.kv_channels = kv_channels
+        self.num_attention_heads = num_attention_heads
+        self.seq_length = seq_length
+        self.hidden_dropout = hidden_dropout
+        self.classifier_dropout = classifier_dropout
+        self.attention_dropout = attention_dropout
+        self.layernorm_epsilon = layernorm_epsilon
+        self.rmsnorm = rmsnorm
+        self.apply_residual_connection_post_layernorm = apply_residual_connection_post_layernorm
+        self.post_layer_norm = post_layer_norm
+        self.add_bias_linear = add_bias_linear
+        self.add_qkv_bias = add_qkv_bias
+        self.bias_dropout_fusion = bias_dropout_fusion
+        self.multi_query_attention = multi_query_attention
+        self.multi_query_group_num = multi_query_group_num
+        self.apply_query_key_layer_scaling = apply_query_key_layer_scaling
+        self.attention_softmax_in_fp32 = attention_softmax_in_fp32
+        self.fp32_residual_connection = fp32_residual_connection
+        self.quantization_bit = quantization_bit
+        self.pre_seq_len = pre_seq_len
+        self.prefix_projection = prefix_projection
+        super().__init__(**kwargs)

py/kolors/chatglm/tokenization_chatglm.py

@@ -0,0 +1,299 @@
+import json
+import os
+import re
+from typing import List, Optional, Union, Dict
+from sentencepiece import SentencePieceProcessor
+from transformers import PreTrainedTokenizer
+from transformers.utils import logging, PaddingStrategy
+from transformers.tokenization_utils_base import EncodedInput, BatchEncoding
+
+class SPTokenizer:
+    def __init__(self, model_path: str):
+        # reload tokenizer
+        assert os.path.isfile(model_path), model_path
+        self.sp_model = SentencePieceProcessor(model_file=model_path)
+
+        # BOS / EOS token IDs
+        self.n_words: int = self.sp_model.vocab_size()
+        self.bos_id: int = self.sp_model.bos_id()
+        self.eos_id: int = self.sp_model.eos_id()
+        self.pad_id: int = self.sp_model.unk_id()
+        assert self.sp_model.vocab_size() == self.sp_model.get_piece_size()
+
+        role_special_tokens = ["<|system|>", "<|user|>", "<|assistant|>", "<|observation|>"]
+        special_tokens = ["[MASK]", "[gMASK]", "[sMASK]", "sop", "eop"] + role_special_tokens
+        self.special_tokens = {}
+        self.index_special_tokens = {}
+        for token in special_tokens:
+            self.special_tokens[token] = self.n_words
+            self.index_special_tokens[self.n_words] = token
+            self.n_words += 1
+        self.role_special_token_expression = "|".join([re.escape(token) for token in role_special_tokens])
+
+    def tokenize(self, s: str, encode_special_tokens=False):
+        if encode_special_tokens:
+            last_index = 0
+            t = []
+            for match in re.finditer(self.role_special_token_expression, s):
+                if last_index < match.start():
+                    t.extend(self.sp_model.EncodeAsPieces(s[last_index:match.start()]))
+                t.append(s[match.start():match.end()])
+                last_index = match.end()
+            if last_index < len(s):
+                t.extend(self.sp_model.EncodeAsPieces(s[last_index:]))
+            return t
+        else:
+            return self.sp_model.EncodeAsPieces(s)
+
+    def encode(self, s: str, bos: bool = False, eos: bool = False) -> List[int]:
+        assert type(s) is str
+        t = self.sp_model.encode(s)
+        if bos:
+            t = [self.bos_id] + t
+        if eos:
+            t = t + [self.eos_id]
+        return t
+
+    def decode(self, t: List[int]) -> str:
+        text, buffer = "", []
+        for token in t:
+            if token in self.index_special_tokens:
+                if buffer:
+                    text += self.sp_model.decode(buffer)
+                    buffer = []
+                text += self.index_special_tokens[token]
+            else:
+                buffer.append(token)
+        if buffer:
+            text += self.sp_model.decode(buffer)
+        return text
+
+    def decode_tokens(self, tokens: List[str]) -> str:
+        text = self.sp_model.DecodePieces(tokens)
+        return text
+
+    def convert_token_to_id(self, token):
+        """ Converts a token (str) in an id using the vocab. """
+        if token in self.special_tokens:
+            return self.special_tokens[token]
+        return self.sp_model.PieceToId(token)
+
+    def convert_id_to_token(self, index):
+        """Converts an index (integer) in a token (str) using the vocab."""
+        if index in self.index_special_tokens:
+            return self.index_special_tokens[index]
+        if index in [self.eos_id, self.bos_id, self.pad_id] or index < 0:
+            return ""
+        return self.sp_model.IdToPiece(index)
+
+
+class ChatGLMTokenizer(PreTrainedTokenizer):
+    vocab_files_names = {"vocab_file": "tokenizer.model"}
+
+    model_input_names = ["input_ids", "attention_mask", "position_ids"]
+
+    def __init__(self, vocab_file, padding_side="left", clean_up_tokenization_spaces=False, encode_special_tokens=False,
+                 **kwargs):
+        self.name = "GLMTokenizer"
+
+        self.vocab_file = vocab_file
+        self.tokenizer = SPTokenizer(vocab_file)
+        self.special_tokens = {
+            "<bos>": self.tokenizer.bos_id,
+            "<eos>": self.tokenizer.eos_id,
+            "<pad>": self.tokenizer.pad_id
+        }
+        self.encode_special_tokens = encode_special_tokens
+        super().__init__(padding_side=padding_side, clean_up_tokenization_spaces=clean_up_tokenization_spaces,
+                         encode_special_tokens=encode_special_tokens,
+                         **kwargs)
+
+    def get_command(self, token):
+        if token in self.special_tokens:
+            return self.special_tokens[token]
+        assert token in self.tokenizer.special_tokens, f"{token} is not a special token for {self.name}"
+        return self.tokenizer.special_tokens[token]
+
+    @property
+    def unk_token(self) -> str:
+        return "<unk>"
+
+    @property
+    def pad_token(self) -> str:
+        return "<unk>"
+
+    @property
+    def pad_token_id(self):
+        return self.get_command("<pad>")
+
+    @property
+    def eos_token(self) -> str:
+        return "</s>"
+
+    @property
+    def eos_token_id(self):
+        return self.get_command("<eos>")
+
+    @property
+    def vocab_size(self):
+        return self.tokenizer.n_words
+
+    def get_vocab(self):
+        """ Returns vocab as a dict """
+        vocab = {self._convert_id_to_token(i): i for i in range(self.vocab_size)}
+        vocab.update(self.added_tokens_encoder)
+        return vocab
+
+    def _tokenize(self, text, **kwargs):
+        return self.tokenizer.tokenize(text, encode_special_tokens=self.encode_special_tokens)
+
+    def _convert_token_to_id(self, token):
+        """ Converts a token (str) in an id using the vocab. """
+        return self.tokenizer.convert_token_to_id(token)
+
+    def _convert_id_to_token(self, index):
+        """Converts an index (integer) in a token (str) using the vocab."""
+        return self.tokenizer.convert_id_to_token(index)
+
+    def convert_tokens_to_string(self, tokens: List[str]) -> str:
+        return self.tokenizer.decode_tokens(tokens)
+
+    def save_vocabulary(self, save_directory, filename_prefix=None):
+        """
+        Save the vocabulary and special tokens file to a directory.
+
+        Args:
+            save_directory (`str`):
+                The directory in which to save the vocabulary.
+            filename_prefix (`str`, *optional*):
+                An optional prefix to add to the named of the saved files.
+
+        Returns:
+            `Tuple(str)`: Paths to the files saved.
+        """
+        if os.path.isdir(save_directory):
+            vocab_file = os.path.join(
+                save_directory, self.vocab_files_names["vocab_file"]
+            )
+        else:
+            vocab_file = save_directory
+
+        with open(self.vocab_file, 'rb') as fin:
+            proto_str = fin.read()
+
+        with open(vocab_file, "wb") as writer:
+            writer.write(proto_str)
+
+        return (vocab_file,)
+
+    def get_prefix_tokens(self):
+        prefix_tokens = [self.get_command("[gMASK]"), self.get_command("sop")]
+        return prefix_tokens
+
+    def build_single_message(self, role, metadata, message):
+        assert role in ["system", "user", "assistant", "observation"], role
+        role_tokens = [self.get_command(f"<|{role}|>")] + self.tokenizer.encode(f"{metadata}\n")
+        message_tokens = self.tokenizer.encode(message)
+        tokens = role_tokens + message_tokens
+        return tokens
+
+    def build_chat_input(self, query, history=None, role="user"):
+        if history is None:
+            history = []
+        input_ids = []
+        for item in history:
+            content = item["content"]
+            if item["role"] == "system" and "tools" in item:
+                content = content + "\n" + json.dumps(item["tools"], indent=4, ensure_ascii=False)
+            input_ids.extend(self.build_single_message(item["role"], item.get("metadata", ""), content))
+        input_ids.extend(self.build_single_message(role, "", query))
+        input_ids.extend([self.get_command("<|assistant|>")])
+        return self.batch_encode_plus([input_ids], return_tensors="pt", is_split_into_words=True)
+
+    def build_inputs_with_special_tokens(
+            self, token_ids_0: List[int], token_ids_1: Optional[List[int]] = None
+    ) -> List[int]:
+        """
+        Build model inputs from a sequence or a pair of sequence for sequence classification tasks by concatenating and
+        adding special tokens. A BERT sequence has the following format:
+
+        - single sequence: `[CLS] X [SEP]`
+        - pair of sequences: `[CLS] A [SEP] B [SEP]`
+
+        Args:
+            token_ids_0 (`List[int]`):
+                List of IDs to which the special tokens will be added.
+            token_ids_1 (`List[int]`, *optional*):
+                Optional second list of IDs for sequence pairs.
+
+        Returns:
+            `List[int]`: List of [input IDs](../glossary#input-ids) with the appropriate special tokens.
+        """
+        prefix_tokens = self.get_prefix_tokens()
+        token_ids_0 = prefix_tokens + token_ids_0
+        if token_ids_1 is not None:
+            token_ids_0 = token_ids_0 + token_ids_1 + [self.get_command("<eos>")]
+        return token_ids_0
+
+    def _pad(
+            self,
+            encoded_inputs: Union[Dict[str, EncodedInput], BatchEncoding],
+            max_length: Optional[int] = None,
+            padding_strategy: PaddingStrategy = PaddingStrategy.DO_NOT_PAD,
+            pad_to_multiple_of: Optional[int] = None,
+            return_attention_mask: Optional[bool] = None,
+    ) -> dict:
+        """
+        Pad encoded inputs (on left/right and up to predefined length or max length in the batch)
+
+        Args:
+            encoded_inputs:
+                Dictionary of tokenized inputs (`List[int]`) or batch of tokenized inputs (`List[List[int]]`).
+            max_length: maximum length of the returned list and optionally padding length (see below).
+                Will truncate by taking into account the special tokens.
+            padding_strategy: PaddingStrategy to use for padding.
+
+                - PaddingStrategy.LONGEST Pad to the longest sequence in the batch
+                - PaddingStrategy.MAX_LENGTH: Pad to the max length (default)
+                - PaddingStrategy.DO_NOT_PAD: Do not pad
+                The tokenizer padding sides are defined in self.padding_side:
+
+                    - 'left': pads on the left of the sequences
+                    - 'right': pads on the right of the sequences
+            pad_to_multiple_of: (optional) Integer if set will pad the sequence to a multiple of the provided value.
+                This is especially useful to enable the use of Tensor Core on NVIDIA hardware with compute capability
+                `>= 7.5` (Volta).
+            return_attention_mask:
+                (optional) Set to False to avoid returning attention mask (default: set to model specifics)
+        """
+        # Load from model defaults
+        assert self.padding_side == "left"
+
+        required_input = encoded_inputs[self.model_input_names[0]]
+        seq_length = len(required_input)
+
+        if padding_strategy == PaddingStrategy.LONGEST:
+            max_length = len(required_input)
+
+        if max_length is not None and pad_to_multiple_of is not None and (max_length % pad_to_multiple_of != 0):
+            max_length = ((max_length // pad_to_multiple_of) + 1) * pad_to_multiple_of
+
+        needs_to_be_padded = padding_strategy != PaddingStrategy.DO_NOT_PAD and len(required_input) != max_length
+
+        # Initialize attention mask if not present.
+        if "attention_mask" not in encoded_inputs:
+            encoded_inputs["attention_mask"] = [1] * seq_length
+
+        if "position_ids" not in encoded_inputs:
+            encoded_inputs["position_ids"] = list(range(seq_length))
+
+        if needs_to_be_padded:
+            difference = max_length - len(required_input)
+
+            if "attention_mask" in encoded_inputs:
+                encoded_inputs["attention_mask"] = [0] * difference + encoded_inputs["attention_mask"]
+            if "position_ids" in encoded_inputs:
+                encoded_inputs["position_ids"] = [0] * difference + encoded_inputs["position_ids"]
+            encoded_inputs[self.model_input_names[0]] = [self.pad_token_id] * difference + required_input
+
+        return encoded_inputs

py/kolors/chatglm/tokenizer/tokenizer_config.json

@@ -0,0 +1,12 @@
+{
+  "name_or_path": "THUDM/chatglm3-6b-base",
+  "remove_space": false,
+  "do_lower_case": false,
+  "tokenizer_class": "ChatGLMTokenizer",
+  "auto_map": {
+    "AutoTokenizer": [
+      "tokenization_chatglm.ChatGLMTokenizer",
+      null
+      ]
+  }
+}

py/kolors/clip_vision_config_vitl_336.json

@@ -0,0 +1,18 @@
+{
+  "attention_dropout": 0.0,
+  "dropout": 0.0,
+  "hidden_act": "quick_gelu",
+  "hidden_size": 1024,
+  "image_size": 336,
+  "initializer_factor": 1.0,
+  "initializer_range": 0.02,
+  "intermediate_size": 4096,
+  "layer_norm_eps": 1e-05,
+  "model_type": "clip_vision_model",
+  "num_attention_heads": 16,
+  "num_channels": 3,
+  "num_hidden_layers": 24,
+  "patch_size": 14,
+  "projection_dim": 768,
+  "torch_dtype": "float32"
+}

py/kolors/loader.py

@@ -0,0 +1,303 @@
+import json
+import os
+import torch
+import subprocess
+import sys
+import comfy.supported_models
+import comfy.model_patcher
+import comfy.model_management
+import comfy.model_detection as model_detection
+import comfy.model_base as model_base
+from comfy.model_base import sdxl_pooled, CLIPEmbeddingNoiseAugmentation, Timestep, ModelType
+from comfy.ldm.modules.diffusionmodules.openaimodel import UNetModel
+from comfy.clip_vision import ClipVisionModel, Output
+from comfy.utils import load_torch_file
+from .chatglm.modeling_chatglm import ChatGLMModel, ChatGLMConfig
+from .chatglm.tokenization_chatglm import ChatGLMTokenizer
+
+class KolorsUNetModel(UNetModel):
+    def __init__(self, *args, **kwargs):
+        super().__init__(*args, **kwargs)
+        self.encoder_hid_proj = torch.nn.Linear(4096, 2048, bias=True)
+
+    def forward(self, *args, **kwargs):
+        with torch.cuda.amp.autocast(enabled=True):
+            if "context" in kwargs:
+                kwargs["context"] = self.encoder_hid_proj(kwargs["context"])
+            result = super().forward(*args, **kwargs)
+            return result
+
+class KolorsSDXL(model_base.SDXL):
+    def __init__(self, model_config, model_type=ModelType.EPS, device=None):
+        model_base.BaseModel.__init__(self, model_config, model_type, device=device, unet_model=KolorsUNetModel)
+        self.embedder = Timestep(256)
+        self.noise_augmentor = CLIPEmbeddingNoiseAugmentation(**{"noise_schedule_config": {"timesteps": 1000, "beta_schedule": "squaredcos_cap_v2"}, "timestep_dim": 1280})
+
+    def encode_adm(self, **kwargs):
+        clip_pooled = sdxl_pooled(kwargs, self.noise_augmentor)
+        width = kwargs.get("width", 768)
+        height = kwargs.get("height", 768)
+        crop_w = kwargs.get("crop_w", 0)
+        crop_h = kwargs.get("crop_h", 0)
+        target_width = kwargs.get("target_width", width)
+        target_height = kwargs.get("target_height", height)
+
+        out = []
+        out.append(self.embedder(torch.Tensor([height])))
+        out.append(self.embedder(torch.Tensor([width])))
+        out.append(self.embedder(torch.Tensor([crop_h])))
+        out.append(self.embedder(torch.Tensor([crop_w])))
+        out.append(self.embedder(torch.Tensor([target_height])))
+        out.append(self.embedder(torch.Tensor([target_width])))
+        flat = torch.flatten(torch.cat(out)).unsqueeze(
+            dim=0).repeat(clip_pooled.shape[0], 1)
+        return torch.cat((clip_pooled.to(flat.device), flat), dim=1)
+
+class Kolors(comfy.supported_models.SDXL):
+    unet_config = {
+        "model_channels": 320,
+        "use_linear_in_transformer": True,
+        "transformer_depth": [0, 0, 2, 2, 10, 10],
+        "context_dim": 2048,
+        "adm_in_channels": 5632,
+        "use_temporal_attention": False,
+    }
+
+    def get_model(self, state_dict, prefix="", device=None):
+        out = KolorsSDXL(self, model_type=self.model_type(state_dict, prefix), device=device, )
+        out.__class__ = model_base.SDXL
+        if self.inpaint_model():
+            out.set_inpaint()
+        return out
+
+def kolors_unet_config_from_diffusers_unet(state_dict, dtype=None):
+    match = {}
+    transformer_depth = []
+
+    attn_res = 1
+    count_blocks = model_detection.count_blocks
+    down_blocks = count_blocks(state_dict, "down_blocks.{}")
+    for i in range(down_blocks):
+        attn_blocks = count_blocks(
+            state_dict, "down_blocks.{}.attentions.".format(i) + '{}')
+        res_blocks = count_blocks(
+            state_dict, "down_blocks.{}.resnets.".format(i) + '{}')
+        for ab in range(attn_blocks):
+            transformer_count = count_blocks(
+                state_dict, "down_blocks.{}.attentions.{}.transformer_blocks.".format(i, ab) + '{}')
+            transformer_depth.append(transformer_count)
+            if transformer_count > 0:
+                match["context_dim"] = state_dict["down_blocks.{}.attentions.{}.transformer_blocks.0.attn2.to_k.weight".format(
+                    i, ab)].shape[1]
+
+        attn_res *= 2
+        if attn_blocks == 0:
+            for i in range(res_blocks):
+                transformer_depth.append(0)
+
+    match["transformer_depth"] = transformer_depth
+
+    match["model_channels"] = state_dict["conv_in.weight"].shape[0]
+    match["in_channels"] = state_dict["conv_in.weight"].shape[1]
+    match["adm_in_channels"] = None
+    if "class_embedding.linear_1.weight" in state_dict:
+        match["adm_in_channels"] = state_dict["class_embedding.linear_1.weight"].shape[1]
+    elif "add_embedding.linear_1.weight" in state_dict:
+        match["adm_in_channels"] = state_dict["add_embedding.linear_1.weight"].shape[1]
+
+    Kolors = {'use_checkpoint': False, 'image_size': 32, 'out_channels': 4, 'use_spatial_transformer': True, 'legacy': False,
+              'num_classes': 'sequential', 'adm_in_channels': 5632, 'dtype': dtype, 'in_channels': 4, 'model_channels': 320,
+              'num_res_blocks': [2, 2, 2], 'transformer_depth': [0, 0, 2, 2, 10, 10], 'channel_mult': [1, 2, 4], 'transformer_depth_middle': 10,
+              'use_linear_in_transformer': True, 'context_dim': 2048, 'num_head_channels': 64, 'transformer_depth_output': [0, 0, 0, 2, 2, 2, 10, 10, 10],
+              'use_temporal_attention': False, 'use_temporal_resblock': False}
+
+    Kolors_inpaint = {'use_checkpoint': False, 'image_size': 32, 'out_channels': 4, 'use_spatial_transformer': True,
+                      'legacy': False,
+                      'num_classes': 'sequential', 'adm_in_channels': 5632, 'dtype': dtype, 'in_channels': 9,
+                      'model_channels': 320,
+                      'num_res_blocks': [2, 2, 2], 'transformer_depth': [0, 0, 2, 2, 10, 10], 'channel_mult': [1, 2, 4],
+                      'transformer_depth_middle': 10,
+                      'use_linear_in_transformer': True, 'context_dim': 2048, 'num_head_channels': 64,
+                      'transformer_depth_output': [0, 0, 0, 2, 2, 2, 10, 10, 10],
+                      'use_temporal_attention': False, 'use_temporal_resblock': False}
+
+    Kolors_ip2p = {'use_checkpoint': False, 'image_size': 32, 'out_channels': 4, 'use_spatial_transformer': True,
+                   'legacy': False,
+                   'num_classes': 'sequential', 'adm_in_channels': 5632, 'dtype': dtype, 'in_channels': 8,
+                   'model_channels': 320,
+                   'num_res_blocks': [2, 2, 2], 'transformer_depth': [0, 0, 2, 2, 10, 10], 'channel_mult': [1, 2, 4],
+                   'transformer_depth_middle': 10,
+                   'use_linear_in_transformer': True, 'context_dim': 2048, 'num_head_channels': 64,
+                   'transformer_depth_output': [0, 0, 0, 2, 2, 2, 10, 10, 10],
+                   'use_temporal_attention': False, 'use_temporal_resblock': False}
+
+    SDXL = {'use_checkpoint': False, 'image_size': 32, 'out_channels': 4, 'use_spatial_transformer': True,
+            'legacy': False,
+            'num_classes': 'sequential', 'adm_in_channels': 2816, 'dtype': dtype, 'in_channels': 4,
+            'model_channels': 320,
+            'num_res_blocks': [2, 2, 2], 'transformer_depth': [0, 0, 2, 2, 10, 10], 'channel_mult': [1, 2, 4],
+            'transformer_depth_middle': 10,
+            'use_linear_in_transformer': True, 'context_dim': 2048, 'num_head_channels': 64,
+            'transformer_depth_output': [0, 0, 0, 2, 2, 2, 10, 10, 10],
+            'use_temporal_attention': False, 'use_temporal_resblock': False}
+
+    SDXL_mid_cnet = {'use_checkpoint': False, 'image_size': 32, 'out_channels': 4, 'use_spatial_transformer': True,
+                     'legacy': False,
+                     'num_classes': 'sequential', 'adm_in_channels': 2816, 'dtype': dtype, 'in_channels': 4,
+                     'model_channels': 320,
+                     'num_res_blocks': [2, 2, 2], 'transformer_depth': [0, 0, 0, 0, 1, 1], 'channel_mult': [1, 2, 4],
+                     'transformer_depth_middle': 1,
+                     'use_linear_in_transformer': True, 'context_dim': 2048, 'num_head_channels': 64,
+                     'transformer_depth_output': [0, 0, 0, 0, 0, 0, 1, 1, 1],
+                     'use_temporal_attention': False, 'use_temporal_resblock': False}
+
+    SDXL_small_cnet = {'use_checkpoint': False, 'image_size': 32, 'out_channels': 4, 'use_spatial_transformer': True,
+                       'legacy': False,
+                       'num_classes': 'sequential', 'adm_in_channels': 2816, 'dtype': dtype, 'in_channels': 4,
+                       'model_channels': 320,
+                       'num_res_blocks': [2, 2, 2], 'transformer_depth': [0, 0, 0, 0, 0, 0], 'channel_mult': [1, 2, 4],
+                       'transformer_depth_middle': 0,
+                       'use_linear_in_transformer': True, 'num_head_channels': 64, 'context_dim': 1,
+                       'transformer_depth_output': [0, 0, 0, 0, 0, 0, 0, 0, 0],
+                       'use_temporal_attention': False, 'use_temporal_resblock': False}
+
+    supported_models = [Kolors, Kolors_inpaint,
+                        Kolors_ip2p, SDXL, SDXL_mid_cnet, SDXL_small_cnet]
+
+
+    for unet_config in supported_models:
+        matches = True
+        for k in match:
+            if match[k] != unet_config[k]:
+                # print("key {} does not match".format(k), match[k], "||", unet_config[k])
+                matches = False
+                break
+        if matches:
+            return model_detection.convert_config(unet_config)
+    return None
+
+# chatglm3 model
+class chatGLM3Model(torch.nn.Module):
+    def __init__(self, textmodel_json_config=None, device='cpu', offload_device='cpu', model_path=None):
+        super().__init__()
+        if model_path is None:
+            raise ValueError("model_path is required")
+        self.device = device
+        if textmodel_json_config is None:
+            textmodel_json_config = os.path.join(
+                os.path.dirname(os.path.realpath(__file__)),
+                "chatglm",
+                "config_chatglm.json"
+            )
+        with open(textmodel_json_config, 'r') as file:
+            config = json.load(file)
+        textmodel_json_config = ChatGLMConfig(**config)
+        is_accelerate_available = False
+        try:
+            from accelerate import init_empty_weights
+            from accelerate.utils import set_module_tensor_to_device
+            is_accelerate_available = True
+        except:
+            pass
+
+        from contextlib import nullcontext
+        with (init_empty_weights() if is_accelerate_available else nullcontext()):
+            with torch.no_grad():
+                print('torch version:', torch.__version__)
+                self.text_encoder = ChatGLMModel(textmodel_json_config).eval()
+                if '4bit' in model_path:
+                    try:
+                        import cpm_kernels
+                    except ImportError:
+                        print("Installing cpm_kernels...")
+                        subprocess.run([sys.executable, "-m", "pip", "install", "cpm_kernels"], check=True)
+                        pass
+                    self.text_encoder.quantize(4)
+                elif '8bit' in model_path:
+                    self.text_encoder.quantize(8)
+
+        sd = load_torch_file(model_path)
+        if is_accelerate_available:
+            for key in sd:
+                set_module_tensor_to_device(self.text_encoder, key, device=offload_device, value=sd[key])
+        else:
+            print("WARNING: Accelerate not available, use load_state_dict load model")
+            self.text_encoder.load_state_dict()
+
+def load_chatglm3(model_path=None):
+    if model_path is None:
+        return
+
+    load_device = comfy.model_management.text_encoder_device()
+    offload_device = comfy.model_management.text_encoder_offload_device()
+
+    glm3model = chatGLM3Model(
+        device=load_device,
+        offload_device=offload_device,
+        model_path=model_path
+    )
+    tokenizer_path = os.path.join(os.path.dirname(os.path.realpath(__file__)), 'chatglm', "tokenizer")
+    tokenizer = ChatGLMTokenizer.from_pretrained(tokenizer_path)
+    text_encoder = glm3model.text_encoder
+    return {"text_encoder":text_encoder, "tokenizer":tokenizer}
+
+
+# clipvision model
+def load_clipvision_vitl_336(path):
+    sd = load_torch_file(path)
+    if "vision_model.encoder.layers.22.layer_norm1.weight" in sd:
+        json_config = os.path.join(os.path.dirname(os.path.realpath(__file__)), "clip_vision_config_vitl_336.json")
+    else:
+        raise Exception("Unsupported clip vision model")
+    clip = ClipVisionModel(json_config)
+    m, u = clip.load_sd(sd)
+    if len(m) > 0:
+        print("missing clip vision: {}".format(m))
+    u = set(u)
+    keys = list(sd.keys())
+    for k in keys:
+        if k not in u:
+            t = sd.pop(k)
+            del t
+    return clip
+
+class applyKolorsUnet:
+    def __enter__(self):
+        import comfy.ldm.modules.diffusionmodules.openaimodel
+        import comfy.utils
+        import comfy.clip_vision
+
+        self.original_UNET_MAP_BASIC = comfy.utils.UNET_MAP_BASIC.copy()
+        comfy.utils.UNET_MAP_BASIC.add(("encoder_hid_proj.weight", "encoder_hid_proj.weight"),)
+        comfy.utils.UNET_MAP_BASIC.add(("encoder_hid_proj.bias", "encoder_hid_proj.bias"),)
+
+        self.original_unet_config_from_diffusers_unet = model_detection.unet_config_from_diffusers_unet
+        model_detection.unet_config_from_diffusers_unet = kolors_unet_config_from_diffusers_unet
+
+        import comfy.supported_models
+        self.original_supported_models = comfy.supported_models.models
+        comfy.supported_models.models = [Kolors]
+
+        self.original_load_clipvision_from_sd = comfy.clip_vision.load_clipvision_from_sd
+        comfy.clip_vision.load_clipvision_from_sd = load_clipvision_vitl_336
+
+    def __exit__(self, type, value, traceback):
+        import comfy.ldm.modules.diffusionmodules.openaimodel
+        import comfy.utils
+        import comfy.supported_models
+        import comfy.clip_vision
+
+        comfy.utils.UNET_MAP_BASIC = self.original_UNET_MAP_BASIC
+
+        model_detection.unet_config_from_diffusers_unet = self.original_unet_config_from_diffusers_unet
+        comfy.supported_models.models = self.original_supported_models
+
+        comfy.clip_vision.load_clipvision_from_sd = self.original_load_clipvision_from_sd
+
+
+def is_kolors_model(model):
+    unet_config = model.model.model_config.unet_config
+    if unet_config and "adm_in_channels" in unet_config and unet_config["adm_in_channels"] == 5632:
+        return True
+    else:
+        return False

py/kolors/model_patch.py

@@ -0,0 +1,66 @@
+import torch
+from torch.nn import Linear
+from types import MethodType
+import comfy.model_management
+import comfy.samplers
+from comfy.cldm.cldm import ControlNet
+from comfy.controlnet import ControlLora
+
+def patch_controlnet(model, control_net):
+    import comfy.controlnet
+    if isinstance(control_net, ControlLora):
+        del_keys = []
+        for k in control_net.control_weights:
+            if k.startswith("label_emb.0.0."):
+                del_keys.append(k)
+
+        for k in del_keys:
+            control_net.control_weights.pop(k)
+
+        super_pre_run = ControlLora.pre_run
+        super_copy = ControlLora.copy
+
+        super_forward = ControlNet.forward
+
+        def KolorsControlNet_forward(self, x, hint, timesteps, context, **kwargs):
+            with torch.cuda.amp.autocast(enabled=True):
+                context = model.model.diffusion_model.encoder_hid_proj(context)
+                return super_forward(self, x, hint, timesteps, context, **kwargs)
+
+        def KolorsControlLora_pre_run(self, *args, **kwargs):
+            result = super_pre_run(self, *args, **kwargs)
+
+            if hasattr(self, "control_model"):
+                self.control_model.forward = MethodType(
+                    KolorsControlNet_forward, self.control_model)
+            return result
+
+        control_net.pre_run = MethodType(
+            KolorsControlLora_pre_run, control_net)
+
+        def KolorsControlLora_copy(self, *args, **kwargs):
+            c = super_copy(self, *args, **kwargs)
+            c.pre_run = MethodType(
+                KolorsControlLora_pre_run, c)
+            return c
+
+        control_net.copy = MethodType(KolorsControlLora_copy, control_net)
+
+    elif isinstance(control_net, comfy.controlnet.ControlNet):
+        model_label_emb = model.model.diffusion_model.label_emb
+        control_net.control_model.label_emb = model_label_emb
+        control_net.control_model_wrapped.model.label_emb = model_label_emb
+        super_forward = ControlNet.forward
+
+        def KolorsControlNet_forward(self, x, hint, timesteps, context, **kwargs):
+            with torch.cuda.amp.autocast(enabled=True):
+                context = model.model.diffusion_model.encoder_hid_proj(context)
+                return super_forward(self, x, hint, timesteps, context, **kwargs)
+
+        control_net.control_model.forward = MethodType(
+            KolorsControlNet_forward, control_net.control_model)
+
+    else:
+        raise NotImplementedError(f"Type {control_net} not supported for KolorsControlNetPatch")
+
+    return control_net

py/kolors/text_encode.py

@@ -0,0 +1,105 @@
+import re
+import random
+import gc
+import comfy.model_management as mm
+from nodes import ConditioningConcat, ConditioningZeroOut, ConditioningSetTimestepRange, ConditioningCombine
+
+def chatglm3_text_encode(chatglm3_model, prompt, clean_gpu=False):
+    device = mm.get_torch_device()
+    offload_device = mm.unet_offload_device()
+    if clean_gpu:
+        mm.unload_all_models()
+        mm.soft_empty_cache()
+    # Function to randomly select an option from the brackets
+
+    def choose_random_option(match):
+        options = match.group(1).split('|')
+        return random.choice(options)
+
+    prompt = re.sub(r'\{([^{}]*)\}', choose_random_option, prompt)
+
+    if "|" in prompt:
+        prompt = prompt.split("|")
+
+    if prompt is not None and isinstance(prompt, str):
+        batch_size = 1
+    elif prompt is not None and isinstance(prompt, list):
+        batch_size = len(prompt)
+
+    # Define tokenizers and text encoders
+    tokenizer = chatglm3_model['tokenizer']
+    text_encoder = chatglm3_model['text_encoder']
+    text_encoder.to(device)
+    text_inputs = tokenizer(
+        prompt,
+        padding="max_length",
+        max_length=256,
+        truncation=True,
+        return_tensors="pt",
+    ).to(device)
+
+    output = text_encoder(
+        input_ids=text_inputs['input_ids'],
+        attention_mask=text_inputs['attention_mask'],
+        position_ids=text_inputs['position_ids'],
+        output_hidden_states=True)
+
+    # [batch_size, 77, 4096]
+    prompt_embeds = output.hidden_states[-2].permute(1, 0, 2).clone()
+    text_proj = output.hidden_states[-1][-1, :, :].clone()  # [batch_size, 4096]
+    bs_embed, seq_len, _ = prompt_embeds.shape
+    prompt_embeds = prompt_embeds.repeat(1, 1, 1)
+    prompt_embeds = prompt_embeds.view(bs_embed, seq_len, -1)
+
+    bs_embed = text_proj.shape[0]
+    text_proj = text_proj.repeat(1, 1).view(bs_embed, -1)
+    text_encoder.to(offload_device)
+    if clean_gpu:
+        mm.soft_empty_cache()
+        gc.collect()
+    return [[prompt_embeds, {"pooled_output": text_proj},]]
+
+def chatglm3_adv_text_encode(chatglm3_model, text, clean_gpu=False):
+    time_start = 0
+    time_end = 1
+    match = re.search(r'TIMESTEP.*$', text)
+    if match:
+        timestep = match.group()
+        timestep = timestep.split(' ')
+        timestep = timestep[0]
+        text = text.replace(timestep, '')
+        value = timestep.split(':')
+        if len(value) >= 3:
+            time_start = float(value[1])
+            time_end = float(value[2])
+        elif len(value) == 2:
+            time_start = float(value[1])
+            time_end = 1
+        elif len(value) == 1:
+            time_start = 0.1
+            time_end = 1
+
+
+    pass3 = [x.strip() for x in text.split("BREAK")]
+    pass3 = [x for x in pass3 if x != '']
+
+    if len(pass3) == 0:
+        pass3 = ['']
+
+    conditioning = None
+
+    for text in pass3:
+        cond = chatglm3_text_encode(chatglm3_model, text, clean_gpu)
+        if conditioning is not None:
+            conditioning = ConditioningConcat().concat(conditioning, cond)[0]
+        else:
+            conditioning = cond
+
+    # setTimeStepRange
+    if time_start > 0 or time_end < 1:
+        conditioning_2, = ConditioningSetTimestepRange().set_range(conditioning, 0, time_start)
+        conditioning_1, = ConditioningZeroOut().zero_out(conditioning)
+        conditioning_1, = ConditioningSetTimestepRange().set_range(conditioning_1, time_start, time_end)
+        conditioning, = ConditioningCombine().combine(conditioning_1, conditioning_2)
+
+    return conditioning

py/layer_diffuse/__init__.py

@@ -2,6 +2,7 @@
 #from https://github.com/huchenlei/ComfyUI-layerdiffuse
 import torch
 import comfy.model_management
+import comfy.lora
 import copy
 from typing import Optional
 from enum import Enum
@@ -59,7 +60,10 @@ class LayerDiffuse:
             image = image.movedim(-1, 1)
 
         try:
-            ModelPatcher.calculate_weight = calculate_weight_adjust_channel(ModelPatcher.calculate_weight)
+            if hasattr(comfy.lora, "calculate_weight"):
+                comfy.lora.calculate_weight = calculate_weight_adjust_channel(comfy.lora.calculate_weight)
+            else:
+                ModelPatcher.calculate_weight = calculate_weight_adjust_channel(ModelPatcher.calculate_weight)
         except:
             pass
 

py/layer_diffuse/attension_sharing.py

@@ -7,7 +7,6 @@ import einops
 from comfy import model_management, utils
 from comfy.ldm.modules.attention import optimized_attention
 
-
 module_mapping_sd15 = {
     0: "input_blocks.1.1.transformer_blocks.0.attn1",
     1: "input_blocks.1.1.transformer_blocks.0.attn2",

py/layer_diffuse/model.py

@@ -324,9 +324,9 @@ try:
         """Patches ComfyUI's LoRA weight application to accept multi-channel inputs."""
         @functools.wraps(func)
         def calculate_weight(
-                self: ModelPatcher, patches, weight: torch.Tensor, key: str
+            patches, weight: torch.Tensor, key: str, intermediate_type=torch.float32
         ) -> torch.Tensor:
-            weight = func(self, patches, weight, key)
+            weight = func(patches, weight, key, intermediate_type)
 
             for p in patches:
                 alpha = p[0]

py/libs/adv_encode.py

@@ -6,10 +6,10 @@ import itertools
 from comfy import model_management
 from comfy.sdxl_clip import SDXLClipModel, SDXLRefinerClipModel, SDXLClipG
 try:
+    from comfy.text_encoders.sd3_clip import SD3ClipModel, T5XXLModel
+except ImportError:
     from comfy.sd3_clip import SD3ClipModel, T5XXLModel
-except:
-    SD3ClipModel, T5XXLModel = None, None
-    pass
+
 from nodes import NODE_CLASS_MAPPINGS, ConditioningConcat, ConditioningZeroOut, ConditioningSetTimestepRange, ConditioningCombine
 
 def _grouper(n, iterable):

py/libs/conditioning.py

@@ -4,17 +4,24 @@ from .translate import zh_to_en, has_chinese
 from .wildcards import process_with_loras
 from .adv_encode import advanced_encode
 
-from nodes import ConditioningConcat, ConditioningCombine, ConditioningAverage, ConditioningSetTimestepRange
+from nodes import ConditioningConcat, ConditioningCombine, ConditioningAverage, ConditioningSetTimestepRange, CLIPTextEncode
 
-def prompt_to_cond(type, model, clip, clip_skip, lora_stack, text, prompt_token_normalization, prompt_weight_interpretation, a1111_prompt_style ,my_unique_id, prompt, easyCache, can_load_lora=True, steps=None):
+def prompt_to_cond(type, model, clip, clip_skip, lora_stack, text, prompt_token_normalization, prompt_weight_interpretation, a1111_prompt_style ,my_unique_id, prompt, easyCache, can_load_lora=True, steps=None, model_type=None):
     styles_selector = is_linked_styles_selector(prompt, my_unique_id, type)
-    title = "正面提示词" if type == 'positive' else "负面提示词"
-    log_node_warn("正在进行" + title + "...")
+    title = "Positive encoding" if type == 'positive' else "Negative encoding"
 
     # Translate cn to en
-    if has_chinese(text):
+    if model_type not in ['hydit'] and text is not None and has_chinese(text):
         text = zh_to_en([text])[0]
 
+    if model_type in ['hydit', 'flux']:
+        log_node_warn(title + "...")
+        embeddings_final, = CLIPTextEncode().encode(clip, text) if text is not None else (None,)
+
+        return (embeddings_final, "", model, clip)
+
+    log_node_warn(title + "...")
+
     positive_seed = find_wildcards_seed(my_unique_id, text, prompt)
     model, clip, text, cond_decode, show_prompt, pipe_lora_stack = process_with_loras(
         text, model, clip, type, positive_seed, can_load_lora, lora_stack, easyCache)
@@ -24,12 +31,11 @@ def prompt_to_cond(type, model, clip, clip_skip, lora_stack, text, prompt_token_
     if clip_skip != 0:
         clipped.clip_layer(clip_skip)
 
-    log_node_warn("正在进行" + title + "编码...")
     steps = steps if steps is not None else find_nearest_steps(my_unique_id, prompt)
     return (advanced_encode(clipped, text, prompt_token_normalization,
                             prompt_weight_interpretation, w_max=1.0,
                             apply_to_pooled='enable',
-                            a1111_prompt_style=a1111_prompt_style, steps=steps), wildcard_prompt, model, clipped)
+                            a1111_prompt_style=a1111_prompt_style, steps=steps) if text is not None else None, wildcard_prompt, model, clipped)
 
 def set_cond(old_cond, new_cond, mode, average_strength, old_cond_start, old_cond_end, new_cond_start, new_cond_end):
     if not old_cond:

py/libs/controlnet.py

@@ -3,16 +3,36 @@ import comfy.controlnet
 import comfy.model_management
 from nodes import NODE_CLASS_MAPPINGS
 
+union_controlnet_types = {"auto": -1, "openpose": 0, "depth": 1, "hed/pidi/scribble/ted": 2, "canny/lineart/anime_lineart/mlsd": 3, "normal": 4, "segment": 5, "tile": 6, "repaint": 7}
+
 class easyControlnet:
     def __init__(self):
         pass
 
-    def apply(self, control_net_name, image, positive, negative, strength, start_percent=0, end_percent=1, control_net=None, scale_soft_weights=1, mask=None, easyCache=None, use_cache=True):
+    def apply(self, control_net_name, image, positive, negative, strength, start_percent=0, end_percent=1, control_net=None, scale_soft_weights=1, mask=None, union_type=None, easyCache=None, use_cache=True, model=None, vae=None):
         if strength == 0:
             return (positive, negative)
 
-        if control_net is None:
-            control_net = easyCache.load_controlnet(control_net_name, scale_soft_weights, use_cache)
+        # kolors controlnet patch
+        from ..kolors.loader import is_kolors_model, applyKolorsUnet
+        if is_kolors_model(model):
+            from ..kolors.model_patch import patch_controlnet
+            if control_net is None:
+                with applyKolorsUnet():
+                    control_net = easyCache.load_controlnet(control_net_name, scale_soft_weights, use_cache)
+                control_net = patch_controlnet(model, control_net)
+        else:
+            if control_net is None:
+                control_net = easyCache.load_controlnet(control_net_name, scale_soft_weights, use_cache)
+
+        # union controlnet
+        if union_type is not None:
+            control_net = control_net.copy()
+            type_number = union_controlnet_types[union_type]
+            if type_number >= 0:
+                control_net.set_extra_arg("control_type", [type_number])
+            else:
+                control_net.set_extra_arg("control_type", [])
 
         if mask is not None:
             mask = mask.to(self.device)
@@ -49,7 +69,7 @@ class easyControlnet:
                     if prev_cnet in cnets:
                         c_net = cnets[prev_cnet]
                     else:
-                        c_net = control_net.copy().set_cond_hint(control_hint, strength, (start_percent, end_percent))
+                        c_net = control_net.copy().set_cond_hint(control_hint, strength, (start_percent, end_percent), vae)
                         c_net.set_previous_controlnet(prev_cnet)
                         cnets[prev_cnet] = c_net
 

py/libs/fooocus.py

@@ -1,113 +0,0 @@
-#credit to Acly for this module
-#from https://github.com/Acly/comfyui-inpaint-nodes
-import torch
-import torch.nn.functional as F
-import comfy
-from comfy.model_base import BaseModel
-from comfy.model_patcher import ModelPatcher
-from comfy.model_management import cast_to_device
-
-from .log import log_node_warn, log_node_error, log_node_info
-
-# Inpaint
-original_calculate_weight = ModelPatcher.calculate_weight
-injected_model_patcher_calculate_weight = False
-
-class InpaintHead(torch.nn.Module):
-    def __init__(self, *args, **kwargs):
-        super().__init__(*args, **kwargs)
-        self.head = torch.nn.Parameter(torch.empty(size=(320, 5, 3, 3), device="cpu"))
-
-    def __call__(self, x):
-        x = F.pad(x, (1, 1, 1, 1), "replicate")
-        return F.conv2d(x, weight=self.head)
-
-def calculate_weight_patched(self: ModelPatcher, patches, weight, key):
-    remaining = []
-
-    for p in patches:
-        alpha = p[0]
-        v = p[1]
-
-        is_fooocus_patch = isinstance(v, tuple) and len(v) == 2 and v[0] == "fooocus"
-        if not is_fooocus_patch:
-            remaining.append(p)
-            continue
-
-        if alpha != 0.0:
-            v = v[1]
-            w1 = cast_to_device(v[0], weight.device, torch.float32)
-            if w1.shape == weight.shape:
-                w_min = cast_to_device(v[1], weight.device, torch.float32)
-                w_max = cast_to_device(v[2], weight.device, torch.float32)
-                w1 = (w1 / 255.0) * (w_max - w_min) + w_min
-                weight += alpha * cast_to_device(w1, weight.device, weight.dtype)
-            else:
-                pass
-                # log_node_warn(self.node_name,
-                #     f"Shape mismatch {key}, weight not merged ({w1.shape} != {weight.shape})"
-                # )
-
-        if len(remaining) > 0:
-            return original_calculate_weight(self, remaining, weight, key)
-    return weight
-
-def inject_patched_calculate_weight():
-    global injected_model_patcher_calculate_weight
-    if not injected_model_patcher_calculate_weight:
-        print(
-            "[comfyui-inpaint-nodes] Injecting patched comfy.model_patcher.ModelPatcher.calculate_weight"
-        )
-        ModelPatcher.calculate_weight = calculate_weight_patched
-        injected_model_patcher_calculate_weight = True
-
-class InpaintWorker:
-    def __init__(self, node_name):
-        self.node_name = node_name if node_name is not None else ""
-
-    def load_fooocus_patch(self, lora: dict, to_load: dict):
-        patch_dict = {}
-        loaded_keys = set()
-        for key in to_load.values():
-            if value := lora.get(key, None):
-                patch_dict[key] = ("fooocus", value)
-                loaded_keys.add(key)
-
-        not_loaded = sum(1 for x in lora if x not in loaded_keys)
-        if not_loaded > 0:
-            log_node_info(self.node_name,
-                f"{len(loaded_keys)} Lora keys loaded, {not_loaded} remaining keys not found in model."
-            )
-        return patch_dict
-
-
-    def patch(self, model, latent, patch):
-        base_model: BaseModel = model.model
-        latent_pixels = base_model.process_latent_in(latent["samples"])
-        noise_mask = latent["noise_mask"].round()
-        latent_mask = F.max_pool2d(noise_mask, (8, 8)).round().to(latent_pixels)
-
-        inpaint_head_model, inpaint_lora = patch
-        feed = torch.cat([latent_mask, latent_pixels], dim=1)
-        inpaint_head_model.to(device=feed.device, dtype=feed.dtype)
-        inpaint_head_feature = inpaint_head_model(feed)
-
-        def input_block_patch(h, transformer_options):
-            if transformer_options["block"][1] == 0:
-                h = h + inpaint_head_feature.to(h)
-            return h
-
-        lora_keys = comfy.lora.model_lora_keys_unet(model.model, {})
-        lora_keys.update({x: x for x in base_model.state_dict().keys()})
-        loaded_lora = self.load_fooocus_patch(inpaint_lora, lora_keys)
-
-        m = model.clone()
-        m.set_model_input_block_patch(input_block_patch)
-        patched = m.add_patches(loaded_lora, 1.0)
-
-        not_patched_count = sum(1 for x in loaded_lora if x not in patched)
-        if not_patched_count > 0:
-            log_node_error(self.node_name, f"Failed to patch {not_patched_count} keys")
-
-        inject_patched_calculate_weight()
-        return (m,)

py/libs/loader.py

@@ -1,4 +1,4 @@
-import time, os, psutil
+import re, time, os, psutil
 import folder_paths
 import comfy.utils
 import comfy.sd
@@ -8,12 +8,11 @@ from comfy.model_patcher import ModelPatcher
 from nodes import NODE_CLASS_MAPPINGS
 from collections import defaultdict
 from .log import log_node_info, log_node_error
-from ..dit.hunyuanDiT.loader import EXM_HyDiT_Tenc_Temp, load_hydit
 from ..dit.pixArt.loader import load_pixart
 
-stable_diffusion_loaders = ["easy fullLoader", "easy a1111Loader", "easy comfyLoader", "easy zero123Loader", "easy svdLoader"]
+stable_diffusion_loaders = ["easy fullLoader", "easy a1111Loader", "easy comfyLoader", "easy hunyuanDiTLoader","easy zero123Loader", "easy svdLoader"]
 stable_cascade_loaders = ["easy cascadeLoader"]
-dit_loaders = ['easy hunyuanDiTLoader', 'easy pixArtLoader']
+dit_loaders = ['easy pixArtLoader']
 controlnet_loaders = ["easy controlnetLoader", "easy controlnetLoaderADV"]
 instant_loaders = ["easy instantIDApply", "easy instantIDApplyADV"]
 cascade_vae_node = ["easy preSamplingCascade", "easy fullCascadeKSampler"]
@@ -32,6 +31,7 @@ class easyLoader:
             "lora": defaultdict(dict),  # {lora_name: {UID: (model_lora, clip_lora)}}
             "controlnet": defaultdict(dict),
             "t5": defaultdict(tuple),
+            "chatglm3": defaultdict(tuple),
         }
         self.memory_threshold = self.determine_memory_threshold(0.7)
         self.lora_name_cache = []
@@ -91,6 +91,7 @@ class easyLoader:
         desired_lora_settings = set()
         desired_controlnet_names = set()
         desired_t5_names = set()
+        desired_glm3_names = set()
 
         for entry in prompt.values():
             class_type = entry["class_type"]
@@ -104,6 +105,11 @@ class easyLoader:
                 desired_ckpt_names.add(self.get_input_value(entry, "ckpt_name", prompt))
                 desired_vae_names.add(self.get_input_value(entry, "vae_name"))
 
+            elif class_type in ['easy kolorsLoader']:
+                desired_unet_names.add(self.get_input_value(entry, "unet_name"))
+                desired_vae_names.add(self.get_input_value(entry, "vae_name"))
+                desired_glm3_names.add(self.get_input_value(entry, "chatglm3_name"))
+
             elif class_type in dit_loaders:
                 t5_name = self.get_input_value(entry, "mt5_name") if "mt5_name" in entry["inputs"] else  None
                 clip_name = self.get_input_value(entry, "clip_name") if "clip_name" in entry["inputs"] else None
@@ -161,6 +167,8 @@ class easyLoader:
                 desired_names = desired_controlnet_names
             elif object_type == "t5":
                 desired_names = desired_t5_names
+            elif object_type == "chatglm3":
+                desired_names = desired_glm3_names
             else:
                 desired_names = desired_lora_names
             self.clear_unused_objects(desired_names, object_type)
@@ -199,7 +207,7 @@ class easyLoader:
         current_memory = self.get_memory_usage()
         if current_memory < self.memory_threshold:
             return
-        eviction_order = ["vae", "lora", "bvae", "clip", "ckpt", "controlnet"]
+        eviction_order = ["vae", "lora", "bvae", "clip", "ckpt", "controlnet", "unet", "t5", "chatglm3"]
         for obj_type in eviction_order:
             if current_memory < self.memory_threshold:
                 break
@@ -230,7 +238,11 @@ class easyLoader:
             config_path = folder_paths.get_full_path("configs", config_name)
             loaded_ckpt = comfy.sd.load_checkpoint(config_path, ckpt_path, output_vae=True, output_clip=output_clip, embedding_directory=folder_paths.get_folder_paths("embeddings"))
         else:
-            loaded_ckpt = comfy.sd.load_checkpoint_guess_config(ckpt_path, output_vae=True, output_clip=output_clip, output_clipvision=output_clipvision, embedding_directory=folder_paths.get_folder_paths("embeddings"))
+            model_options = {}
+            if re.search("nf4", ckpt_name):
+                from ..bitsandbytes_NF4 import OPS
+                model_options = {"custom_operations": OPS}
+            loaded_ckpt = comfy.sd.load_checkpoint_guess_config(ckpt_path, output_vae=True, output_clip=output_clip, output_clipvision=output_clipvision, embedding_directory=folder_paths.get_folder_paths("embeddings"), model_options=model_options)
 
         self.add_to_cache("ckpt", cache_name, loaded_ckpt[0])
         self.add_to_cache("bvae", cache_name, loaded_ckpt[2])
@@ -260,6 +272,7 @@ class easyLoader:
 
     def load_unet(self, unet_name):
         if unet_name in self.loaded_objects["unet"]:
+            log_node_info("Load UNet", f"{unet_name} cached")
             return self.loaded_objects["unet"][unet_name][0]
 
         unet_path = folder_paths.get_full_path("unet", unet_name)
@@ -280,14 +293,14 @@ class easyLoader:
                 cn_adv_cls = NODE_CLASS_MAPPINGS['ControlNetLoaderAdvanced']
                 control_net, = cn_adv_cls().load_controlnet(control_net_name, timestep_keyframe)
             else:
-                raise Exception(
-                    f"[Advanced-ControlNet Not Found] you need to install 'COMFYUI-Advanced-ControlNet'")
+                raise Exception(f"[Advanced-ControlNet Not Found] you need to install 'COMFYUI-Advanced-ControlNet'")
         else:
             controlnet_path = folder_paths.get_full_path("controlnet", control_net_name)
             control_net = comfy.controlnet.load_controlnet(controlnet_path)
         if use_cache:
             self.add_to_cache("controlnet", unique_id, control_net)
             self.eviction_based_on_memory()
+
         return control_net
     def load_clip(self, clip_name, type='stable_diffusion', load_clip=None):
         if clip_name in self.loaded_objects["clip"]:
@@ -299,6 +312,8 @@ class easyLoader:
             clip_type = comfy.sd.CLIPType.STABLE_CASCADE
         elif type == 'sd3':
             clip_type = comfy.sd.CLIPType.SD3
+        elif type == 'flux':
+            clip_type = comfy.sd.CLIPType.FLUX
         elif type == 'stable_audio':
             clip_type = comfy.sd.CLIPType.STABLE_AUDIO
         clip_path = folder_paths.get_full_path("clip", clip_name)
@@ -308,7 +323,7 @@ class easyLoader:
 
         return load_clip
 
-    def load_lora(self, lora, model=None, clip=None, type=None):
+    def load_lora(self, lora, model=None, clip=None, type=None , use_cache=True):
         lora_name = lora["lora_name"]
         model = model if model is not None else lora["model"]
         clip = clip if clip is not None else lora["clip"]
@@ -319,11 +334,12 @@ class easyLoader:
         lbw_b = lora["lbw_b"] if "lbw_b" in lora else None
 
         model_hash = str(model)[44:-1]
-        clip_hash = str(clip)[25:-1]
+        clip_hash = str(clip)[25:-1] if clip else ''
 
         unique_id = f'{model_hash};{clip_hash};{lora_name};{model_strength};{clip_strength}'
 
-        if unique_id in self.loaded_objects["lora"] and unique_id in self.loaded_objects["lora"][lora_name]:
+        if use_cache and unique_id in self.loaded_objects["lora"]:
+            log_node_info("Load LORA",f"{lora_name} cached")
             return self.loaded_objects["lora"][unique_id][0]
 
         orig_lora_name = lora_name
@@ -380,8 +396,9 @@ class easyLoader:
                 else:
                     model, clip = comfy.sd.load_lora_for_models(model, clip, _lora, model_strength, clip_strength)
 
-            self.add_to_cache("lora", unique_id, (model, clip))
-            self.eviction_based_on_memory()
+            if use_cache:
+                self.add_to_cache("lora", unique_id, (model, clip))
+                self.eviction_based_on_memory()
         else:
             log_node_error(f"LORA NOT FOUND", orig_lora_name)
 
@@ -408,7 +425,7 @@ class easyLoader:
 
             return None
 
-    def load_main(self, ckpt_name, config_name, vae_name, lora_name, lora_model_strength, lora_clip_strength, optional_lora_stack, model_override, clip_override, vae_override, prompt):
+    def load_main(self, ckpt_name, config_name, vae_name, lora_name, lora_model_strength, lora_clip_strength, optional_lora_stack, model_override, clip_override, vae_override, prompt, nf4=False):
         model: ModelPatcher | None = None
         clip: comfy.sd.CLIP | None = None
         vae: comfy.sd.VAE | None = None
@@ -466,18 +483,46 @@ class easyLoader:
 
         return model, clip, vae, clip_vision, lora_stack
 
+    # Kolors
+    def load_kolors_unet(self, unet_name):
+        if unet_name in self.loaded_objects["unet"]:
+            log_node_info("Load Kolors UNet", f"{unet_name} cached")
+            return self.loaded_objects["unet"][unet_name][0]
+        else:
+            from ..kolors.loader import applyKolorsUnet
+            with applyKolorsUnet():
+                unet_path = folder_paths.get_full_path("unet", unet_name)
+                sd = comfy.utils.load_torch_file(unet_path)
+                model = comfy.sd.load_unet_state_dict(sd)
+                if model is None:
+                    raise RuntimeError("ERROR: Could not detect model type of: {}".format(unet_path))
+
+                self.add_to_cache("unet", unet_name, model)
+                self.eviction_based_on_memory()
+
+                return model
+
+    def load_chatglm3(self, chatglm3_name):
+        from ..kolors.loader import load_chatglm3
+        if chatglm3_name in self.loaded_objects["chatglm3"]:
+            log_node_info("Load ChatGLM3", f"{chatglm3_name} cached")
+            return self.loaded_objects["chatglm3"][chatglm3_name][0]
+
+        chatglm_model = load_chatglm3(model_path=folder_paths.get_full_path("llm", chatglm3_name))
+        self.add_to_cache("chatglm3", chatglm3_name, chatglm_model)
+        self.eviction_based_on_memory()
+
+        return chatglm_model
+
+
     # DiT
     def load_dit_ckpt(self, ckpt_name, model_name, **kwargs):
         if (ckpt_name+'_'+model_name) in self.loaded_objects["ckpt"]:
             return self.loaded_objects["ckpt"][ckpt_name+'_'+model_name][0]
         model = None
         ckpt_path = folder_paths.get_full_path("checkpoints", ckpt_name)
-        model_type = kwargs['model_type'] if "model_type" in kwargs else 'HyDiT'
-        if model_type == 'HyDiT':
-            hydit_conf = kwargs['hydit_conf']
-            model_conf = hydit_conf[model_name]
-            model = load_hydit(ckpt_path, model_conf)
-        elif model_type == 'PixArt':
+        model_type = kwargs['model_type'] if "model_type" in kwargs else 'PixArt'
+        if model_type == 'PixArt':
             pixart_conf = kwargs['pixart_conf']
             model_conf = pixart_conf[model_name]
             model = load_pixart(ckpt_path, model_conf)
@@ -491,10 +536,6 @@ class easyLoader:
         if clip_name in self.loaded_objects["clip"]:
             return self.loaded_objects["clip"][clip_name][0]
 
-        model_type = kwargs['model_type'] if "model_type" in kwargs else 'HyDiT'
-        if model_type == 'HyDiT':
-            del kwargs['model_type']
-            model = EXM_HyDiT_Tenc_Temp(model_class="clip", **kwargs)
         clip_path = folder_paths.get_full_path("clip", clip_name)
         sd = comfy.utils.load_torch_file(clip_path)
 
@@ -535,7 +576,10 @@ class easyLoader:
         return model
 
     def load_t5_from_sd3_clip(self, sd3_clip, padding):
-        from comfy.sd3_clip import SD3Tokenizer, SD3ClipModel
+        try:
+            from comfy.text_encoders.sd3_clip import SD3Tokenizer, SD3ClipModel
+        except:
+            from comfy.sd3_clip import SD3Tokenizer, SD3ClipModel
         import copy
 
         clip = sd3_clip.clone()

py/libs/sampler.py

@@ -108,9 +108,9 @@ class easySampler:
             noise = comfy.sample.prepare_noise(latent_image, seed, batch_inds)
 
         #######################################################################################
+        # add model patch
         # brushnet
         add_model_patch(model)
-        #
         #######################################################################################
         samples = comfy.sample.sample(model, noise, steps, cfg, sampler_name, scheduler, positive, negative,
                                       latent_image,
@@ -156,10 +156,6 @@ class easySampler:
                 preview_bytes = previewer.decode_latent_to_preview_image(preview_format, x0)
             pbar.update_absolute(step + 1, total_steps, preview_bytes)
 
-        # samples = comfy.sample.sample_custom(model, noise, cfg, _sampler, sigmas, positive, negative, latent_image,
-        #                                      noise_mask=noise_mask, callback=callback, disable_pbar=disable_pbar,
-        #                                      seed=seed)
-
         samples = comfy.samplers.sample(model, noise, positive, negative, cfg, device, _sampler, sigmas, latent_image=latent_image, model_options=model.model_options,
                denoise_mask=noise_mask, callback=callback, disable_pbar=disable_pbar, seed=seed)
 
@@ -167,16 +163,38 @@ class easySampler:
         out["samples"] = samples
         return out
 
-    def custom_advanced_ksampler(self, noise, guider, sampler, sigmas, latent_image):
+    def custom_advanced_ksampler(self, noise, guider, sampler, sigmas, latent_image, preview_latent=False):
         latent = latent_image
         latent_image = latent["samples"]
+        latent = latent.copy()
+        latent_image = comfy.sample.fix_empty_latent_channels(guider.model_patcher, latent_image)
+        latent["samples"] = latent_image
 
         noise_mask = None
         if "noise_mask" in latent:
             noise_mask = latent["noise_mask"]
 
         x0_output = {}
-        callback = latent_preview.prepare_callback(guider.model_patcher, sigmas.shape[-1] - 1, x0_output)
+        previewer = False
+
+        model = guider.model_patcher
+        steps = sigmas.shape[-1] - 1
+        if preview_latent:
+            previewer = latent_preview.get_previewer(model.load_device, model.model.latent_format)
+
+        pbar = comfy.utils.ProgressBar(steps)
+
+        preview_format = "JPEG"
+        if preview_format not in ["JPEG", "PNG"]:
+            preview_format = "JPEG"
+        def callback(step, x0, x, total_steps):
+            if x0_output is not None:
+                x0_output["x0"] = x0
+
+            preview_bytes = None
+            if previewer:
+                preview_bytes = previewer.decode_latent_to_preview_image(preview_format, x0)
+            pbar.update_absolute(step + 1, total_steps, preview_bytes)
 
         disable_pbar = not comfy.utils.PROGRESS_BAR_ENABLED
         samples = guider.sample(noise.generate_noise(latent), latent_image, sampler, sigmas, denoise_mask=noise_mask,
@@ -190,7 +208,6 @@ class easySampler:
             out_denoised["samples"] = guider.model_patcher.model.process_latent_out(x0_output["x0"].cpu())
         else:
             out_denoised = out
-
         return (out, out_denoised)
 
     def get_value_by_id(self, key: str, my_unique_id: Any) -> Optional[Any]:

py/libs/translate.py

@@ -80,7 +80,7 @@ def has_chinese(text):
     _text = text
     _text = re.sub(r'<.*?>', '', _text)
     _text = re.sub(r'__.*?__', '', _text)
-    _text = re.sub(r'embedding:.*?(\d+)?', '', _text)
+    _text = re.sub(r'embedding:.*?$', '', _text)
     for char in _text:
         if '\u4e00' <= char <= '\u9fff':
             has_cn = True
@@ -95,7 +95,6 @@ def translate(text):
     if not os.path.exists(zh_en_model_path):
         zh_en_model_path = 'Helsinki-NLP/opus-mt-zh-en'
 
-    print(zh_en_model_path)
     if zh_en_model is None:
 
         zh_en_model = AutoModelForSeq2SeqLM.from_pretrained(zh_en_model_path).eval()

py/libs/utils.py

@@ -5,6 +5,19 @@ class AlwaysEqualProxy(str):
     def __ne__(self, _):
         return False
 
+class TautologyStr(str):
+    def __ne__(self, other):
+        return False
+
+class ByPassTypeTuple(tuple):
+    def __getitem__(self, index):
+        if index>0:
+            index=0
+        item = super().__getitem__(index)
+        if isinstance(item, str):
+            return TautologyStr(item)
+        return item
+
 comfy_ui_revision = None
 def get_comfyui_revision():
     try:
@@ -92,6 +105,8 @@ def get_sd_version(model):
     model_config: comfy.supported_models.supported_models_base.BASE = base.model_config
     if isinstance(model_config, comfy.supported_models.SDXL):
         return 'sdxl'
+    elif isinstance(model_config, comfy.supported_models.SDXLRefiner):
+        return 'sdxl_refiner'
     elif isinstance(
             model_config, (comfy.supported_models.SD15, comfy.supported_models.SD20)
     ):
@@ -102,6 +117,10 @@ def get_sd_version(model):
         return 'svd'
     elif isinstance(model_config, comfy.supported_models.SD3):
         return 'sd3'
+    elif isinstance(model_config, comfy.supported_models.HunyuanDiT):
+        return 'hydit'
+    elif isinstance(model_config, comfy.supported_models.Flux):
+        return 'flux'
     else:
         return 'unknown'
 
@@ -227,9 +246,9 @@ def to_lora_patch_dict(state_dict: dict) -> dict:
 def easySave(images, filename_prefix, output_type, prompt=None, extra_pnginfo=None):
     """Save or Preview Image"""
     from nodes import PreviewImage, SaveImage
-    if output_type == "Hide":
+    if output_type in ["Hide", "None"]:
         return list()
-    if output_type in ["Preview", "Preview&Choose"]:
+    elif output_type in ["Preview", "Preview&Choose"]:
         filename_prefix = 'easyPreview'
         results = PreviewImage().save_images(images, filename_prefix, prompt, extra_pnginfo)
         return results['ui']['images']

py/libs/xyplot.py

@@ -394,7 +394,7 @@ class easyXYPlot():
             model, clip, vae, clip_vision = self.easyCache.load_checkpoint(plot_image_vars['ckpt_name'])
 
             if plot_image_vars['lora_name'] != "None":
-                lora = {"lora_name": plot_image_vars['lora_name'], "model": model, "clip": clip, "model_strength": plot_image_vars['model_strength'], "clip_strength": plot_image_vars['lora_clip_strength']}
+                lora = {"lora_name": plot_image_vars['lora_name'], "model": model, "clip": clip, "model_strength": plot_image_vars['lora_model_strength'], "clip_strength": plot_image_vars['lora_clip_strength']}
                 model, clip = self.easyCache.load_lora(lora)
 
             # Check for custom VAE

py/xyplot.py

@@ -279,10 +279,10 @@ class XYplot_Negative_Cond:
     def INPUT_TYPES(cls):
         inputs = {
             "optional": {
-                "negative_1": ("CONDITIONING"),
-                "negative_2": ("CONDITIONING"),
-                "negative_3": ("CONDITIONING"),
-                "negative_4": ("CONDITIONING"),
+                "negative_1": ("CONDITIONING",),
+                "negative_2": ("CONDITIONING",),
+                "negative_3": ("CONDITIONING",),
+                "negative_4": ("CONDITIONING",),
             }
         }
 

pyproject.toml

@@ -1,9 +1,9 @@
 [project]
 name = "comfyui-easy-use"
 description = "To enhance the usability of ComfyUI, optimizations and integrations have been implemented for several commonly used nodes."
-version = "1.2.0"
-license = "LICENSE"
-dependencies = ["diffusers>=0.25.0", "accelerate>=0.25.0", "clip_interrogator>=0.6.0", "sentencepiece", "lark-parser", "onnxruntime", "spandrel"]
+version = "1.2.3"
+license = { file = "LICENSE" }
+dependencies = ["diffusers", "accelerate", "clip_interrogator>=0.6.0", "sentencepiece", "lark-parser", "onnxruntime", "spandrel", "opencv-python"]
 
 [project.urls]
 Repository = "https://github.com/yolain/ComfyUI-Easy-Use"

repair_dependency_list.txt

@@ -0,0 +1,4 @@
+numpy>=1.19.0
+huggingface_hub>=0.23.3
+transformers>=4.38.2
+protobuf>=4.25.3

requirements.txt

@@ -1,7 +1,8 @@
-diffusers>=0.25.0
-accelerate>=0.25.0
+diffusers
+accelerate
 clip_interrogator>=0.6.0
-sentencepiece
 lark-parser
 onnxruntime
-spandrel
+opencv-python
+sentencepiece
+spandrel

web_version/v1/css/easy.css

@@ -50,6 +50,9 @@ textarea{
 .comfy-modal button {
     border-width:1px;
 }
+.comfy-modal-content{
+    width: 100%;
+}
 
 
 dialog{

web_version/v1/js/easy/easyDynamicWidgets.js

@@ -7,8 +7,7 @@ import { $t } from '../common/i18n.js';
 import { findWidgetByName, toggleWidget, updateNodeHeight} from "../common/utils.js";
 
 const seedNodes = ["easy seed", "easy latentNoisy", "easy wildcards", "easy preSampling", "easy preSamplingAdvanced", "easy preSamplingNoiseIn", "easy preSamplingSdTurbo", "easy preSamplingCascade", "easy preSamplingDynamicCFG", "easy preSamplingLayerDiffusion", "easy fullkSampler", "easy fullCascadeKSampler"]
-const loaderNodes = ["easy fullLoader", "easy a1111Loader", "easy comfyLoader", "easy hunyuanDiTLoader", "easy pixArtLoader"]
-
+const loaderNodes = ["easy fullLoader", "easy a1111Loader", "easy comfyLoader", "easy fluxLoader", "easy hunyuanDiTLoader", "easy pixArtLoader"]
 
 function widgetLogic(node, widget) {
 	if (widget.name === 'lora_name') {
@@ -279,6 +278,7 @@ function widgetLogic(node, widget) {
 		const faceid_presets = [
             'FACEID',
             'FACEID PLUS - SD1.5 only',
+			'FACEID PLUS KOLORS',
             'FACEID PLUS V2',
 			'FACEID PORTRAIT (style transfer)',
 			'FACEID PORTRAIT UNNORM - SDXL only (strong)'
@@ -287,6 +287,7 @@ function widgetLogic(node, widget) {
 			toggleWidget(node, findWidgetByName(node, 'lora_strength'))
 			toggleWidget(node, findWidgetByName(node, 'provider'))
 			toggleWidget(node, findWidgetByName(node, 'weight_faceidv2'))
+			toggleWidget(node, findWidgetByName(node, 'weight_kolors'))
 			toggleWidget(node, findWidgetByName(node, 'use_tiled'), true)
 			let use_tiled = findWidgetByName(node, 'use_tiled')
 			if(use_tiled && use_tiled.value){
@@ -297,12 +298,9 @@ function widgetLogic(node, widget) {
 
 		}
 		else if(faceid_presets.includes(widget.value)){
-			if(widget.value == 'FACEID PLUS V2'){
-				toggleWidget(node, findWidgetByName(node, 'weight_faceidv2'), true)
-			}else{
-				toggleWidget(node, findWidgetByName(node, 'weight_faceidv2'))
-			}
-			if(['FACEID PORTRAIT (style transfer)','FACEID PORTRAIT UNNORM - SDXL only (strong)'].includes(widget.value)){
+			toggleWidget(node, findWidgetByName(node, 'weight_faceidv2'), ['FACEID PLUS V2','FACEID PLUS KOLORS'].includes(widget.value) ? true : false);
+			toggleWidget(node, findWidgetByName(node, 'weight_kolors'), ['FACEID PLUS KOLORS'].includes(widget.value) ? true : false);
+			if(['FACEID PLUS KOLORS','FACEID PORTRAIT (style transfer)','FACEID PORTRAIT UNNORM - SDXL only (strong)'].includes(widget.value)){
 				toggleWidget(node, findWidgetByName(node, 'lora_strength'), false)
 			}
 			else{
@@ -442,6 +440,17 @@ function widgetLogic(node, widget) {
 		}
 		updateNodeHeight(node)
 	}
+
+	if(widget.name == 'rem_mode'){
+		switch (widget.value){
+			case 'Inspyrenet':
+				toggleWidget(node, findWidgetByName(node, 'torchscript_jit'), true)
+				break
+			default:
+				toggleWidget(node, findWidgetByName(node, 'torchscript_jit'), false)
+				break
+		}
+	}
 }
 
 function widgetLogic2(node, widget) {
@@ -688,12 +697,14 @@ app.registerExtension({
 		switch (node.comfyClass){
 			case "easy fullLoader":
 			case "easy a1111Loader":
+			case "easy fluxLoader":
 			case "easy comfyLoader":
 			case "easy cascadeLoader":
 			case "easy svdLoader":
 			case "easy dynamiCrafterLoader":
 			case "easy hunyuanDiTLoader":
 			case "easy pixArtLoader":
+			case "easy kolorsLoader":
 			case "easy loraStack":
 			case "easy controlnetStack":
 			case "easy latentNoisy":
@@ -732,6 +743,7 @@ app.registerExtension({
 			case 'easy icLightApply':
 			case 'easy ipadapterApply':
 			case 'easy ipadapterApplyADV':
+			case 'easy ipadapterApplyFaceIDKolors':
 			case 'easy ipadapterApplyEncoder':
 			case 'easy applyInpaint':
 				getSetters(node)
@@ -1175,22 +1187,15 @@ app.registerExtension({
 
 		if(nodeData.name == 'easy convertAnything'){
 			const onNodeCreated = nodeType.prototype.onNodeCreated;
-			const changeType = async function (type) {
-				const body = new FormData();
-				body.append("type", type);
-				const response = await api.fetchApi("/easyuse/convert", { method:'POST',body});
-			}
 			nodeType.prototype.onNodeCreated = async function () {
 				onNodeCreated ? onNodeCreated.apply(this, []) : undefined;
 				setTimeout(_=>{
 					const type_control = this.widgets[this.widgets.findIndex((w) => w.name === "output_type")]
 					let _this = this
-					changeType(type_control.value)
 					type_control.callback = async() => {
 						_this.outputs[0].type = (type_control.value).toUpperCase()
 						_this.outputs[0].name = type_control.value
 						_this.outputs[0].label = type_control.value
-						changeType(type_control.value)
 					}
 				},300)
 
@@ -1238,7 +1243,7 @@ const getSetWidgets = ['rescale_after_model', 'rescale',
 						'num_loras', 'num_controlnet', 'mode', 'toggle', 'resolution', 'ratio', 'target_parameter',
 	'input_count', 'replace_count', 'downscale_mode', 'range_mode','text_combine_mode', 'input_mode',
 	'lora_count','ckpt_count', 'conditioning_mode', 'preset', 'use_tiled', 'use_batch', 'num_embeds',
-	"easing_mode", "guider", "scheduler", "inpaint_mode", 't5_type'
+	"easing_mode", "guider", "scheduler", "inpaint_mode", 't5_type', 'rem_mode'
 ]
 
 function getSetters(node) {

web_version/v1/js/easy/easyExtraMenu.js

@@ -2,11 +2,11 @@ import {app} from "../../../../scripts/app.js";
 import {$t} from '../common/i18n.js'
 import {CheckpointInfoDialog, LoraInfoDialog} from "../common/model.js";
 
-const loaders = ['easy fullLoader', 'easy a1111Loader', 'easy comfyLoader', 'easy hunyuanDiTLoader', 'easy pixArtLoader']
+const loaders = ['easy fullLoader', 'easy a1111Loader', 'easy comfyLoader', 'easy kolorsLoader', 'easy hunyuanDiTLoader', 'easy pixArtLoader']
 const preSampling = ['easy preSampling', 'easy preSamplingAdvanced', 'easy preSamplingDynamicCFG', 'easy preSamplingNoiseIn', 'easy preSamplingCustom', 'easy preSamplingLayerDiffusion', 'easy fullkSampler']
 const kSampler = ['easy kSampler', 'easy kSamplerTiled', 'easy kSamplerInpainting', 'easy kSamplerDownscaleUnet', 'easy kSamplerLayerDiffusion']
-const controlnet = ['easy controlnetLoader', 'easy controlnetLoaderADV', 'easy instantIDApply', 'easy instantIDApplyADV']
-const ipadapter = ['easy ipadapterApply', 'easy ipadapterApplyADV', 'easy ipadapterStyleComposition', 'easy ipadapterApplyFromParams', 'easy pulIDApply', 'easy pulIDApplyADV']
+const controlnet = ['easy controlnetLoader', 'easy controlnetLoaderADV', 'easy controlnetLoader++', 'easy instantIDApply', 'easy instantIDApplyADV']
+const ipadapter = ['easy ipadapterApply', 'easy ipadapterApplyADV', 'easy ipadapterApplyFaceIDKolors', 'easy ipadapterStyleComposition', 'easy ipadapterApplyFromParams', 'easy pulIDApply', 'easy pulIDApplyADV']
 const positive_prompt = ['easy positive', 'easy wildcards']
 const imageNode = ['easy loadImageBase64', 'LoadImage', 'LoadImageMask']
 const inpaint = ['easy applyBrushNet', 'easy applyPowerPaint', 'easy applyInpaint']

web_version/v1/js/easy/easyInterface.js

@@ -113,13 +113,13 @@ try{
     }
     let theme_name = localStorage.getItem('Comfy.Settings.Comfy.ColorPalette')
     control_mode = localStorage.getItem('Comfy.Settings.Comfy.WidgetControlMode')
-    if(control_mode) {
-        control_mode = JSON.parse(control_mode)
-        if(control_mode == 'before'){
-            localStorage['Comfy.Settings.AE.mouseover'] = false
-            localStorage['Comfy.Settings.AE.highlight'] = false
-        }
-    }
+    // if(control_mode) {
+    //     control_mode = JSON.parse(control_mode)
+    //     if(control_mode == 'before'){
+    //         localStorage['Comfy.Settings.AE.mouseover'] = false
+    //         localStorage['Comfy.Settings.AE.highlight'] = false
+    //     }
+    // }
     // 兼容 ComfyUI Revision: 1887 [235727fe] 以上版本
     if(api.storeSettings){
         const _settings = await api.getSettings()