v0.22.0

- 优化ImageColorTransfer
- 支持动态提示
- 添加更多节点支持，AppInfo自动填充id
- LoadImagesToBatch加载
- zhipuai 按需安装

.gitignore

@@ -3,4 +3,6 @@ https/
 nodes/config.json
 workflow/my_workflow.json
 workflow/my_workflow_app.json
-app/*
+workflow/prompt_result.json
+app/*
+workflow/prompt_result.json

README.md

@@ -1,8 +1,10 @@
 > 适配了最新版comfyui的py3.11 ，torch 2.1.2+cu121
 
-> [discord](https://discord.gg/cXs9vZSqeK)
+> [Mixlab nodes discord](https://discord.gg/cXs9vZSqeK)
 
 #### 
+[comfyui-Image-reward](https://github.com/shadowcz007/comfyui-Image-reward)
+
 [comfyui-ultralytics-yolo](https://github.com/shadowcz007/comfyui-ultralytics-yolo)
 
 [comfyui-moondream](https://github.com/shadowcz007/comfyui-moondream)
@@ -15,7 +17,9 @@
 - 支持多个web app 切换
 - 发布为app的workflow，可以在右键里再次编辑了
 - web app可以设置分类，在comfyui右键菜单可以编辑更新web app
+- 支持动态提示
 
+![](./assets/微信图片_20240421205440.png)
 
 - Support multiple web app switching.
 - Add the AppInfo node, which allows you to transform the workflow into a web app by simple configuration.
@@ -39,12 +43,19 @@ APP-JSON:
 - [image-to-image](./example/Image-to-Image_2.json)
 - text-to-text
 
-> 暂时支持8种节点作为界面上的输入节点：Load Image、CLIPTextEncode、PromptSlide、TextInput_、Color、FloatSlider、IntNumber、CheckpointLoaderSimple、LoraLoader
+> 暂时支持 9 种节点作为界面上的输入节点：Load Image、VHS_LoadVideo、CLIPTextEncode、PromptSlide、TextInput_、Color、FloatSlider、IntNumber、CheckpointLoaderSimple、LoraLoader
 
 > 输出节点：PreviewImage 、SaveImage、ShowTextForGPT、VHS_VideoCombine、PromptImage
 
 > seed统一输入控件，支持：SamplerCustom、KSampler
 
+> 配套[ps插件](https://github.com/shadowcz007/comfyui-ps-plugin)
+
+> 如果遇到上传图片不成功，请检查下：局域网或者是云服务，请使用https，端口8189这个服务（ 感谢 @Damien 反馈问题）
+
+> If you encounter difficulties in uploading images, please check the following: for local network or cloud services, please use HTTPS and the service on port 8189. (Thanks to @Damien for reporting the issue.)
+
+
 
 ## 🏃🚗🚚🚀  Real-time Design
 > ScreenShareNode & FloatingVideoNode. Now comfyui supports capturing screen pixel streams from any software and can be used for LCM-Lora integration. Let's get started with implementation and design! 💻🌐
@@ -66,7 +77,7 @@ https://github.com/shadowcz007/comfyui-mixlab-nodes/assets/12645064/e7e77f90-e43
 [Voice + Real-time Face Swap Workflow](./workflow/语音+实时换脸workflow.json)
 
 ### GPT
-> Support for calling multiple GPTs.ChatGPT、ChatGLM3 , Some code provided by rui. If you are using OpenAI's service, fill in https://api.openai.com/v1 . If you are using a local LLM service, fill in http://127.0.0.1:xxxx/v1 .  Azure OpenAI:https://xxxx.openai.azure.com 
+> Support for calling multiple GPTs.ChatGPT、ChatGLM3 、ChatGLM4 , Some code provided by rui. If you are using OpenAI's service, fill in https://api.openai.com/v1 . If you are using a local LLM service, fill in http://127.0.0.1:xxxx/v1 .  Azure OpenAI:https://xxxx.openai.azure.com 
 
 ![gpt-workflow.svg](./assets/gpt-workflow.svg)
 
@@ -107,18 +118,31 @@ https://github.com/shadowcz007/comfyui-mixlab-nodes/assets/12645064/e7e77f90-e43
 [workflow](./workflow/3D-workflow.json)
 
 
-### LoadImagesFromLocal
+### Image
+
+#### LoadImagesToBatch
+> Upload multiple images for batch input into the IP adapter. 
+
+#### LoadImagesFromLocal
 > Monitor changes to images in a local folder, and trigger real-time execution of workflows, supporting common image formats, especially PSD format, in conjunction with Photoshop. 
 
 ![watch](./assets/4-loadfromlocal-watcher-workflow.svg)
 
 [workflow-4](./workflow/4-loadfromlocal-watcher-workflow.json)
 
-### LoadImagesFromURL
+#### LoadImagesFromURL
 > Conveniently load images from a fixed address on the internet to ensure that default images in the workflow can be executed.
 
 
-## Utils
+### Style
+> Apply VisualStyle Prompting , Modified from [ComfyUI_VisualStylePrompting](https://github.com/ExponentialML/ComfyUI_VisualStylePrompting) 
+
+![](./assets/VisualStylePrompting.png)
+
+> StyleAligned , Modified from [style_aligned_comfy](https://github.com/brianfitzgerald/style_aligned_comfy) 
+
+
+### Utils
 > The Color node provides a color picker for easy color selection, the Font node offers built-in font selection for use with TextImage to generate text images, and the DynamicDelayByText node allows delayed execution based on the length of the input text.
 
 - [添加了DynamicDelayByText功能，可以根据输入文本的长度进行延迟执行。](./workflow/audio-chatgpt-workflow.json)
@@ -131,7 +155,7 @@ https://github.com/shadowcz007/comfyui-mixlab-nodes/assets/12645064/e7e77f90-e43
 
 
 
-## Other Nodes
+### Other Nodes
 
 ![main](./assets/all-workflow.svg)
 ![main2](./assets/detect-face-all.png)
@@ -161,6 +185,7 @@ from [simple-lama-inpainting](https://github.com/enesmsahin/simple-lama-inpainti
 
 "briarmbg","u2net","u2netp","u2net_human_seg","u2net_cloth_seg","silueta","isnet-general-use","isnet-anime"
 
+*** briarmbg ***  model was developed by BRlA Al and can be used as an open-source model for non-commercial purposes
 
 
 ### Improvement 
@@ -224,6 +249,14 @@ pip3 install -r requirements.txt
 访问 [www.mixcomfy.com](https://www.mixcomfy.com)，获得更多内测功能，关注微信公众号：Mixlab无界社区
 
 
+
+
+#### 
+File / LoadImagesFromPath SaveImageToLocal LoadImagesFromURL
+
+
+
+
 #### discussions:
 [discussions](https://github.com/shadowcz007/comfyui-mixlab-nodes/discussions)
 

__init__.py

@@ -6,7 +6,7 @@ import sys,json
 import urllib
 import hashlib
 import datetime
-
+import folder_paths
 
 python = sys.executable
 
@@ -252,7 +252,8 @@ def get_my_workflow_for_app(filename="my_workflow_app.json",category="",is_all=F
                                 "name":x['app']['name'],
                                 "version":x['app']['version'],
                                 "input":input,
-                                "output":output
+                                "output":output,
+                                "id":x['app']['id']
                             }
                         },
                         "date":item["date"]
@@ -300,7 +301,8 @@ def get_my_workflow_for_app(filename="my_workflow_app.json",category="",is_all=F
                                 "name":x['app']['name'],
                                 "version":x['app']['version'],
                                 "input":input,
-                                "output":output
+                                "output":output,
+                                "id":x['app']['id']
                             }
                         },
                         "date":item["date"]
@@ -308,6 +310,31 @@ def get_my_workflow_for_app(filename="my_workflow_app.json",category="",is_all=F
 
     return apps
 
+# 历史记录
+def save_prompt_result(id,data):
+    prompt_result_path=os.path.join(current_path, "workflow/prompt_result.json")
+    prompt_result={}
+    if os.path.exists(prompt_result_path):
+        with open(prompt_result_path) as json_file:
+            prompt_result = json.load(json_file)
+    
+    prompt_result[id]=data
+    
+    with open(prompt_result_path, 'w') as file:
+        json.dump(prompt_result, file)
+    return prompt_result_path
+
+def get_prompt_result():
+    prompt_result_path=os.path.join(current_path, "workflow/prompt_result.json")
+    prompt_result={}
+    if os.path.exists(prompt_result_path):
+        with open(prompt_result_path) as json_file:
+            prompt_result = json.load(json_file)
+    res=list(prompt_result.values())
+    # print(res)
+    return res
+
+
 def save_workflow_json(data):
     workflow_path=os.path.join(current_path, "workflow/my_workflow.json")
     with open(workflow_path, 'w') as file:
@@ -518,6 +545,37 @@ async def nodes_map_hander(request):
     return web.json_response(result)
 
 
+@routes.post("/mixlab/folder_paths")
+async def get_checkpoints(request):
+    data = await request.json()
+    t="checkpoints"
+    try:
+        t=data['type']
+    except Exception as e:
+        print('/mixlab/folder_paths',False,e)
+    
+    names = folder_paths.get_filename_list(t)
+
+    return web.json_response({"names":names,"types":list(folder_paths.folder_names_and_paths.keys())})
+
+
+@routes.post("/mixlab/prompt_result")
+async def post_prompt_result(request):
+    data = await request.json()
+    res=None
+    # print(data)
+    try:
+        action=data['action']
+        if action=='save':
+            result=data['data']
+            res=save_prompt_result(result['prompt_id'],result)
+        elif action=='all':
+            res=get_prompt_result()
+    except Exception as e:
+        print('/mixlab/prompt_result',False,e)
+    
+    return web.json_response({"result":res})
+
 # 扩展api接口
 # from server import PromptServer
 # from aiohttp import web
@@ -530,15 +588,19 @@ async def nodes_map_hander(request):
 
 
 # 导入节点
-from .nodes.PromptNode import EmbeddingPrompt,RandomPrompt,PromptSlide,PromptSimplification,PromptImage,JoinWithDelimiter
-from .nodes.ImageNode import SaveImageToLocal,SplitImage,GridOutput,GetImageSize_,MirroredImage,ImageColorTransfer,NoiseImage,TransparentImage,GradientImage,LoadImagesFromPath,LoadImagesFromURL,ResizeImage,TextImage,SvgImage,Image3D,ShowLayer,NewLayer,MergeLayers,CenterImage,AreaToMask,SmoothMask,SplitLongMask,ImageCropByAlpha,EnhanceImage,FaceToMask
+from .nodes.PromptNode import GLIGENTextBoxApply_Advanced,EmbeddingPrompt,RandomPrompt,PromptSlide,PromptSimplification,PromptImage,JoinWithDelimiter
+from .nodes.ImageNode import LoadImages_,CompositeImages,GridDisplayAndSave,GridInput,ImagesPrompt,SaveImageAndMetadata,SaveImageToLocal,SplitImage,GridOutput,GetImageSize_,MirroredImage,ImageColorTransfer,NoiseImage,TransparentImage,GradientImage,LoadImagesFromPath,LoadImagesFromURL,ResizeImage,TextImage,SvgImage,Image3D,ShowLayer,NewLayer,MergeLayers,CenterImage,AreaToMask,SmoothMask,SplitLongMask,ImageCropByAlpha,EnhanceImage,FaceToMask
 # from .nodes.Vae import VAELoader,VAEDecode
 from .nodes.ScreenShareNode import ScreenShareNode,FloatingVideo
 
-from .nodes.ChatGPT import ChatGPTNode,ShowTextForGPT,CharacterInText
+from .nodes.ChatGPT import ChatGPTNode,ShowTextForGPT,CharacterInText,TextSplitByDelimiter
 from .nodes.Audio import GamePal,SpeechRecognition,SpeechSynthesis
-from .nodes.Utils import CreateLoraNames,CreateSampler_names,CreateCkptNames,CreateSeedNode,TESTNODE_,AppInfo,IntNumber,FloatSlider,TextInput,ColorInput,FontInput,TextToNumber,DynamicDelayProcessor,LimitNumber,SwitchByIndex,MultiplicationNode
-from .nodes.Mask import OutlineMask,FeatheredMask
+from .nodes.Utils import IncrementingListNode,ListSplit,CreateLoraNames,CreateSampler_names,CreateCkptNames,CreateSeedNode,TESTNODE_,TESTNODE_TOKEN,AppInfo,IntNumber,FloatSlider,TextInput,ColorInput,FontInput,TextToNumber,DynamicDelayProcessor,LimitNumber,SwitchByIndex,MultiplicationNode
+from .nodes.Mask import MaskListReplace,MaskListMerge,OutlineMask,FeatheredMask
+
+from .nodes.Style import ApplyVisualStylePrompting,StyleAlignedReferenceSampler,StyleAlignedBatchAlign,StyleAlignedSampleReferenceLatents
+
+from .nodes.Video import VideoCombine_Adv,LoadVideoAndSegment,ImageListReplace,VAEEncodeForInpaint_Frames
 
 
 # 要导出的所有节点及其名称的字典
@@ -546,9 +608,12 @@ from .nodes.Mask import OutlineMask,FeatheredMask
 NODE_CLASS_MAPPINGS = {
     "AppInfo":AppInfo,
     "TESTNODE_":TESTNODE_,
+    "TESTNODE_TOKEN":TESTNODE_TOKEN,
     "RandomPrompt":RandomPrompt,
+    # "LoraPrompt":LoraPrompt,
     "EmbeddingPrompt":EmbeddingPrompt,
     "PromptSlide":PromptSlide,
+    "GLIGENTextBoxApply_Advanced":GLIGENTextBoxApply_Advanced,
     "PromptSimplification":PromptSimplification,
     "PromptImage":PromptImage,
     "MirroredImage":MirroredImage,
@@ -558,6 +623,7 @@ NODE_CLASS_MAPPINGS = {
     "ResizeImageMixlab":ResizeImage,
     "LoadImagesFromPath":LoadImagesFromPath,
     "LoadImagesFromURL":LoadImagesFromURL,
+    "LoadImagesToBatch":LoadImages_,
     "TextImage":TextImage,
     "EnhanceImage":EnhanceImage,
     "SvgImage":SvgImage,
@@ -565,9 +631,12 @@ NODE_CLASS_MAPPINGS = {
     "ImageColorTransfer":ImageColorTransfer,
     "ShowLayer":ShowLayer,
     "NewLayer":NewLayer,
+    "CompositeImages_":CompositeImages,
     "SplitImage":SplitImage,
     "CenterImage":CenterImage,
     "GridOutput":GridOutput,
+    "GridDisplayAndSave":GridDisplayAndSave,
+    "GridInput":GridInput,
     "MergeLayers":MergeLayers,
     "SplitLongMask":SplitLongMask,
     "FeatheredMask":FeatheredMask,
@@ -575,14 +644,17 @@ NODE_CLASS_MAPPINGS = {
     "FaceToMask":FaceToMask,
     "AreaToMask":AreaToMask,
     "ImageCropByAlpha":ImageCropByAlpha,
+    "ImagesPrompt_":ImagesPrompt,
     # "VAELoaderConsistencyDecoder":VAELoader,
     "SaveImageToLocal":SaveImageToLocal,
+    "SaveImageAndMetadata_":SaveImageAndMetadata,
     # "VAEDecodeConsistencyDecoder":VAEDecode,
     "ScreenShare":ScreenShareNode,
     "FloatingVideo":FloatingVideo,
     "ChatGPTOpenAI":ChatGPTNode,
     "ShowTextForGPT":ShowTextForGPT,
     "CharacterInText":CharacterInText,
+    "TextSplitByDelimiter":TextSplitByDelimiter,
     "SpeechRecognition":SpeechRecognition,
     "SpeechSynthesis":SpeechSynthesis,
     "Color":ColorInput,
@@ -597,39 +669,77 @@ NODE_CLASS_MAPPINGS = {
     "SwitchByIndex":SwitchByIndex,
     "LimitNumber":LimitNumber, 
     "OutlineMask":OutlineMask,
+    "MaskListMerge_":MaskListMerge,
     "JoinWithDelimiter":JoinWithDelimiter,
     "Seed_":CreateSeedNode,
     "CkptNames_":CreateCkptNames,
     "SamplerNames_":CreateSampler_names,
-    "LoraNames_":CreateLoraNames
-    # "LaMaInpainting":LaMaInpainting
+    "LoraNames_":CreateLoraNames,
+    "ApplyVisualStylePrompting_":ApplyVisualStylePrompting,
+    "StyleAlignedReferenceSampler_": StyleAlignedReferenceSampler,
+    "StyleAlignedSampleReferenceLatents_": StyleAlignedSampleReferenceLatents,
+    "StyleAlignedBatchAlign_": StyleAlignedBatchAlign,
+    "LoadVideoAndSegment_":LoadVideoAndSegment,
+    "VideoCombine_Adv":VideoCombine_Adv,
+    "ListSplit_":ListSplit,
+    "MaskListReplace_":MaskListReplace,
+    "ImageListReplace_":ImageListReplace,
+    "VAEEncodeForInpaint_Frames":VAEEncodeForInpaint_Frames,
+    "IncrementingListNode_":IncrementingListNode
     # "GamePal":GamePal
 }
 
 # 一个包含节点友好/可读的标题的字典
 NODE_DISPLAY_NAME_MAPPINGS = {
-    "AppInfo":"AppInfo ♾️Mixlab",
-    "ResizeImageMixlab":"ResizeImage ♾️Mixlab",
+    "AppInfo":"App Info ♾️MixlabApp",
+    "Color":"Color Input ♾️MixlabApp",
+    "TextInput_":"Text Input ♾️MixlabApp",
+    "FloatSlider":"Float Slider Input ♾️MixlabApp",
+    "IntNumber":"Int Input ♾️MixlabApp",
+    "ImagesPrompt_":"Images Input ♾️MixlabApp",
+    "SaveImageAndMetadata_":"Save Image Output ♾️MixlabApp",
+    "ResizeImageMixlab":"Resize Image ♾️Mixlab",
     "RandomPrompt": "Random Prompt ♾️Mixlab",
+    "PromptImage":"Output Prompt and Image",
     "SplitLongMask":"Splitting a long image into sections",
     "VAELoaderConsistencyDecoder":"Consistency Decoder Loader",
     "VAEDecodeConsistencyDecoder":"Consistency Decoder Decode",
-    "ScreenShare":"ScreenShare ♾️Mixlab",
+    "ScreenShare":"Screen Share ♾️Mixlab",
     "FloatingVideo":"FloatingVideo ♾️Mixlab",
     "ChatGPTOpenAI":"ChatGPT ♾️Mixlab",
-    "ShowTextForGPT":"ShowTextForGPT ♾️Mixlab",
-    "MergeLayers":"MergeLayers ♾️Mixlab",
+    "ShowTextForGPT":"Show Text ♾️MixlabApp",
+    "MergeLayers":"Merge Layers ♾️Mixlab",
     "SpeechSynthesis":"SpeechSynthesis ♾️Mixlab",
     "SpeechRecognition":"SpeechRecognition ♾️Mixlab",
     "3DImage":"3DImage ♾️Mixlab",
+    "CompositeImages_":"Composite Images",
     "DynamicDelayProcessor":"DynamicDelayByText ♾️Mixlab",
     "LaMaInpainting":"LaMaInpainting ♾️Mixlab",
-    "PromptSlide":"PromptSlide ♾️Mixlab",
-    "PromptGenerate_Mix":"PromptGenerate ♾️Mixlab",
-    "ChinesePrompt_Mix":"ChinesePrompt ♾️Mixlab",
+    "PromptSlide":"Prompt Slide ♾️Mixlab",
+    "PromptGenerate_Mix":"Prompt Generate ♾️Mixlab",
+    "ChinesePrompt_Mix":"Chinese Prompt ♾️Mixlab",
     "GamePal":"GamePal ♾️Mixlab",
-    "RembgNode_Mix":"Removebg",
-    "LoraNames_":"LoraName_TriggerWords.safetensors"
+    "RembgNode_Mix":"Remove Background ♾️Mixlab",
+    "LoraNames_":"LoraName",
+    "ApplyVisualStylePrompting_":"Apply VisualStyle Prompting",
+    "StyleAlignedReferenceSampler_": "StyleAligned Reference Sampler",
+    "StyleAlignedSampleReferenceLatents_": "StyleAligned Sample Reference Latents",
+    "StyleAlignedBatchAlign_": "StyleAligned Batch Align",
+    "LoadVideoAndSegment_":"Load Video And Segment",
+    "VideoCombine_Adv":"Video Combine",
+    "MaskListMerge_":"MaskList to Mask",
+    "ListSplit_":"Split List",
+    "MaskListReplace_":"MaskList Replace",
+    "ImageListReplace_":"ImageList Replace",
+    "SwitchByIndex":"List Switch By Index",
+    "GLIGENTextBoxApply_Advanced":"GLIGEN TextBox Apply ♾️Mixlab",
+    "GridDisplayAndSave":"Grid Display And Save",
+    "GridInput":"Grid Input",
+    "GridOutput":"Grid Output",
+    "GetImageSize_":"Get Image Size",
+    "VAEEncodeForInpaint_Frames":"VAE Encode For Inpaint Frames",
+    "IncrementingListNode_":"Create Incrementing Number List",
+    "LoadImagesToBatch":"Load Images(base64) ♾️Mixlab"
 }
 
 # web ui的节点功能

data/image_prompt_example.json

@@ -0,0 +1,10 @@
+[
+    {
+        "keyword":"Dog",
+        "imgurl":"http://127.0.0.1:8188/view?filename=1709966910233.png&type=input&subfolder=&rand=0.2734446552394221"
+    },
+    {
+        "keyword":"x",
+        "imgurl":"http://127.0.0.1:8188/view?filename=image%20(33).png&type=input&subfolder=pasted&rand=0.6984318219852814"
+    }
+]

data/loras_tags.json

@@ -0,0 +1 @@
+{}

nodes/ChatGPT.py

@@ -4,6 +4,17 @@ import urllib.error
 import re,json,os,string,random
 import folder_paths
 import hashlib
+import codecs,sys
+import importlib.util
+
+
+def is_installed(package):
+    try:
+        spec = importlib.util.find_spec(package)
+    except ModuleNotFoundError:
+        return False
+    return spec is not None
+
 
 def get_unique_hash(string):
     hash_object = hashlib.sha1(string.encode())
@@ -47,6 +58,33 @@ def openai_client(key,url):
     )
     return client
 
+def ZhipuAI_client(key):
+
+    try:
+        if is_installed('zhipuai')==False:
+            import subprocess
+
+            # 安装
+            print('#pip install zhipuai')
+
+            result = subprocess.run([sys.executable, '-s', '-m', 'pip', 'install', 'zhipuai'], capture_output=True, text=True)
+
+            #检查命令执行结果
+            if result.returncode == 0:
+                print("#install success")
+                from zhipuai import ZhipuAI
+            else:
+                print("#install error")
+            
+        else:
+            from zhipuai import ZhipuAI
+    except:
+        print("#install zhipuai error")
+
+    client = ZhipuAI(
+        api_key=key, # 填写您的 APIKey
+    ) 
+    return client
 
 
 def chat(client, model_name,messages ):
@@ -100,7 +138,15 @@ class ChatGPTNode:
                                        "default": "You are ChatGPT, a large language model trained by OpenAI. Answer as concisely as possible.", 
                                        "multiline": True,"dynamicPrompts": False
                                        }),
-                "model": (["gpt-3.5-turbo","gpt-35-turbo","gpt-3.5-turbo-16k", "gpt-3.5-turbo-16k-0613", "gpt-4-0613","gpt-4-1106-preview"], 
+                "model": ([
+                    "gpt-3.5-turbo",
+                    "gpt-3.5-turbo-0125",
+                    "gpt-35-turbo",
+                    "gpt-3.5-turbo-16k", 
+                    "gpt-3.5-turbo-16k-0613", 
+                    "gpt-4-0613",
+                    "gpt-4-1106-preview",
+                    "glm-4"], 
                           {"default": "gpt-3.5-turbo"}),
                 "seed": ("INT", {"default": 0, "min": 0, "max": 0xffffffffffffffff, "step": 1}),
                 "context_size":("INT", {"default": 1, "min": 0, "max":30, "step": 1}),
@@ -143,8 +189,13 @@ class ChatGPTNode:
         if is_azure_url(api_url):
             client=azure_client(api_key,api_url)
         else:
-            client=openai_client(api_key,api_url)
-            print('openai url')
+            # 根据用户选择的模型，设置相应的接口和模型名称
+            if model == "glm-4" :
+                client = ZhipuAI_client(api_key)  # 使用 Zhipuai 的接口
+                print('using Zhipuai interface')
+            else :
+                client = openai_client(api_key,api_url)  # 使用 ChatGPT  的接口
+                print('using ChatGPT interface')
 
         # 把用户的提示添加到会话历史中
         # 调用API时传递整个会话历史
@@ -199,7 +250,7 @@ class ShowTextForGPT:
     OUTPUT_NODE = True
     OUTPUT_IS_LIST = (True,)
 
-    CATEGORY = "♾️Mixlab/GPT"
+    CATEGORY = "♾️Mixlab/Text"
 
     def run(self, text,output_dir=[""]):
         
@@ -283,7 +334,7 @@ class CharacterInText:
     # OUTPUT_NODE = True
     OUTPUT_IS_LIST = (False,)
 
-    CATEGORY = "♾️Mixlab/GPT"
+    CATEGORY = "♾️Mixlab/Text"
 
     def run(self, text,character,start_index):
         # print(text,character,start_index)
@@ -291,3 +342,53 @@ class CharacterInText:
         
         return (b+start_index,)
 
+class TextSplitByDelimiter:
+    @classmethod
+    def INPUT_TYPES(s):
+        return {
+            "required": {
+                "text": ("STRING", {"multiline": True,"dynamicPrompts": False}),
+                "delimiter":("STRING", {"multiline": False,"default":",","dynamicPrompts": False}),
+                "start_index": ("INT", {
+                    "default": 0,
+                    "min": 0, #Minimum value
+                    "max": 1000, #Maximum value
+                    "step": 1, #Slider's step
+                    "display": "number" # Cosmetic only: display as "number" or "slider"
+                }),
+                 "skip_every": ("INT", {
+                    "default": 0,
+                    "min": 0, #Minimum value
+                    "max": 10, #Maximum value
+                    "step": 1, #Slider's step
+                    "display": "number" # Cosmetic only: display as "number" or "slider"
+                }),
+                "max_count": ("INT", {
+                    "default": 10,
+                    "min": 1, #Minimum value
+                    "max": 1000, #Maximum value
+                    "step": 1, #Slider's step
+                    "display": "number" # Cosmetic only: display as "number" or "slider"
+                }),
+            }
+        }
+
+    INPUT_IS_LIST = False
+    RETURN_TYPES = ("STRING",)
+    FUNCTION = "run"
+    # OUTPUT_NODE = True
+    OUTPUT_IS_LIST = (True,)
+
+    CATEGORY = "♾️Mixlab/Text"
+
+    def run(self, text,delimiter,start_index,skip_every,max_count):
+         
+        if delimiter=="":
+            arr=[text.strip()]
+        else:
+            delimiter=codecs.decode(delimiter, 'unicode_escape')
+            arr= [line for line in text.split(delimiter) if line.strip()]
+
+        arr= arr[start_index:start_index + max_count * (skip_every+1):(skip_every+1)]
+
+        return (arr,)

nodes/ImageNode.py

@@ -8,16 +8,77 @@ import base64,os,random
 from io import BytesIO
 import folder_paths
 import json,io
+import comfy.utils
 from comfy.cli_args import args
 import cv2
 import string
 import math,glob
 from .Watcher import FolderWatcher
+import hashlib
+
+def composite_images(foreground, background, mask):
+    width,height=foreground.size
+ 
+    bg_image=background
+    # 按z-index排序
+    layer = {
+            "x":0,
+            "y":0,
+            "width":width,
+            "height":height,
+            "z_index":88,
+            "scale_option":'overall',
+            "image":foreground,
+            "mask":mask
+    }
+                
+    width, height = bg_image.size
+
+    layer_image=layer['image']
+    layer_mask=layer['mask']
+    
+    bg_image=merge_images(bg_image,
+                        layer_image,
+                        layer_mask,
+                                        layer['x'],
+                                        layer['y'],
+                                        layer['width'],
+                                        layer['height'],
+                                        layer['scale_option']
+                                        )
+                   
+    bg_image=bg_image.convert('RGB')
+    
+    return bg_image
+
+
+def count_files_in_directory(directory):
+    file_count = 0
+    for _, _, files in os.walk(directory):
+        file_count += len(files)
+    return file_count
+
+def save_json_to_file(data, file_path):
+    with open(file_path, 'w') as file:
+        json.dump(data, file)
+
+def draw_rectangle(image, grid, color,width):
+    x, y, w, h = grid
+    draw = ImageDraw.Draw(image)
+    draw.rectangle([(x, y), (x+w, y+h)], outline=color,width=width)
 
 def generate_random_string(length):
     letters = string.ascii_letters + string.digits
     return ''.join(random.choice(letters) for _ in range(length))
 
+def padding_rectangle(grid, padding):
+    x, y, w, h = grid
+    x -= padding
+    y -= padding
+    w += 2 * padding
+    h += 2 * padding
+    return (x, y, w, h)
+
 class AnyType(str):
   """A special class that is always equal in not equal comparisons. Credit to pythongosssss"""
 
@@ -628,6 +689,17 @@ def merge_images(bg_image, layer_image, mask, x, y, width, height, scale_option)
     return bg_image
 
 
+def resize_2(img):
+    # 检查图像的高度是否是2的倍数，如果不是，则调整高度
+    if img.height % 2 != 0:
+        img = img.resize((img.width, img.height + 1))
+
+    # 检查图像的宽度是否是2的倍数，如果不是，则调整宽度
+    if img.width % 2 != 0:
+        img = img.resize((img.width + 1, img.height))
+
+    return img
+
 # TODO 几个像素点的底
 def resize_image(layer_image, scale_option, width, height,color="white"):
     layer_image = layer_image.convert("RGB")
@@ -657,8 +729,10 @@ def resize_image(layer_image, scale_option, width, height,color="white"):
         resized_image = Image.new("RGB", (width, height), color=color)
         resized_image.paste(layer_image.resize((new_width, new_height)), ((width - new_width) // 2, (height - new_height) // 2))
         resized_image = resized_image.convert("RGB")
+        resized_image=resize_2(resized_image)
         return resized_image
     
+    layer_image=resize_2(layer_image)
     return layer_image
 
 
@@ -982,6 +1056,35 @@ class TransparentImage:
 
 
 
+class ImagesPrompt:
+    @classmethod
+    def INPUT_TYPES(s):
+        # input_dir = folder_paths.get_input_directory()
+        # files = [f for f in os.listdir(input_dir) if os.path.isfile(os.path.join(input_dir, f))]
+        return {
+            "required": { 
+               "image_base64": ("STRING",{"multiline": False,"default": "","dynamicPrompts": False}),
+               "text": ("STRING",{"multiline": True,"default": "","dynamicPrompts": True}),
+            }
+            }
+    
+    RETURN_TYPES = ("IMAGE","STRING",)
+    RETURN_NAMES = ("image","text",)
+
+    FUNCTION = "run"
+
+    CATEGORY = "♾️Mixlab/Input"
+
+    INPUT_IS_LIST = False
+    OUTPUT_IS_LIST = (False,False,)
+    OUTPUT_NODE = False
+
+    # 运行的函数
+    def run(self,image_base64,text):
+        image = base64_to_image(image_base64)
+        image=image.convert('RGB')
+        image=pil2tensor(image)
+        return (image,text,)
 
 
 class EnhanceImage:
@@ -1029,6 +1132,42 @@ class EnhanceImage:
 
 
 
+class LoadImages_:
+    @classmethod
+    def INPUT_TYPES(s):
+        
+        return {"required":
+                    {"images": ("IMAGEBASE64",), 
+                     },
+                }
+
+    CATEGORY = "♾️Mixlab/Image"
+
+    INPUT_IS_LIST = False
+    OUTPUT_IS_LIST = (False,)
+
+    RETURN_TYPES = ("IMAGE",)
+    FUNCTION = "load_image"
+    def load_image(self, images):
+
+        # print(images)
+        ims=[]
+        for im in images['base64']:
+            image = base64_to_image(im)
+            image=image.convert('RGB')
+            image=pil2tensor(image)
+            ims.append(image)
+
+        image1 = ims[0]
+        for image2 in ims[1:]:
+            if image1.shape[1:] != image2.shape[1:]:
+                image2 = comfy.utils.common_upscale(image2.movedim(-1, 1), image1.shape[2], image1.shape[1], "bilinear", "center").movedim(1, -1)
+            image1 = torch.cat((image1, image2), dim=0)
+        return (image1,)
+        
+ 
+
+
 ''' 
 ("STRING",{"multiline": False,"default": "Hello World!"})
 对应 widgets.js 里：
@@ -1378,7 +1517,7 @@ class Image3D:
 
     FUNCTION = "run"
 
-    CATEGORY = "♾️Mixlab/Image"
+    CATEGORY = "♾️Mixlab/3D"
 
     INPUT_IS_LIST = False
     OUTPUT_IS_LIST = (False,False,False,False,)
@@ -1488,6 +1627,37 @@ class FaceToMask:
         return (mask,)
 
 
+class CompositeImages:
+    @classmethod
+    def INPUT_TYPES(s):
+        return {
+            "required": { 
+            "foreground": ("IMAGE",),
+            "mask":("MASK",),
+            "background": ("IMAGE",),
+            },
+                }
+    
+    RETURN_TYPES = ("IMAGE",)
+    RETURN_NAMES = ("IMAGE",)
+
+    FUNCTION = "run"
+
+    CATEGORY = "♾️Mixlab/Layer"
+
+    # OUTPUT_IS_LIST = (True,)
+
+    def run(self, foreground,mask,background):
+        foreground= tensor2pil(foreground)
+        mask= tensor2pil(mask)
+        background= tensor2pil(background)
+        res=composite_images(foreground,background,mask)
+        
+        return (pil2tensor(res),)
+
+
+
+
 class EmptyLayer:
     @classmethod
     def INPUT_TYPES(s):
@@ -1720,17 +1890,24 @@ class SplitImage:
     # OUTPUT_IS_LIST = (True,)
 
     def run(self,image,num,seed):
+        
+        if type(seed) == list and len(seed)==1:
+            seed=seed[0]
+
         image=tensor2pil(image)
         
         grids=splitImage(image,num)
 
         if seed>num:
-            num=int(seed / 500 * num)-1
+            num=seed % (num + 1)
         else:
             num=seed-1
+        
+        print('#SplitImage',seed)
 
         num=max(0,num) 
-
+        num=min(num,len(grids)-1)
+        
         g=grids[num]
 
         x,y,w,h=g
@@ -1797,8 +1974,182 @@ class CenterImage:
 
         return (grid,pil2tensor(mask),)
 
+class GridDisplayAndSave:
+    @classmethod
+    def INPUT_TYPES(s):
+        return { 
+            "required": {
+                "labels": ("STRING", 
+                                        {
+                                            "multiline": True, 
+                                            "default": "",
+                                            "forceInput": True,
+                                            "dynamicPrompts": False
+                                        }),
+                "grids": ("_GRID",),
+                
+                "image": ("IMAGE",),
+                "filename_prefix": ("STRING", {"default": "mixlab/grids"})
+            }
+                }
+    
+    RETURN_TYPES = ( )
+    RETURN_NAMES = ( )
+
+    FUNCTION = "run"
+
+    CATEGORY = "♾️Mixlab/Layer"
+
+    INPUT_IS_LIST = True
+    OUTPUT_NODE = True
+    # OUTPUT_IS_LIST = (True,)
+
+    def run(self,labels,grids,image,filename_prefix):
+
+        # print(image.shape)
+
+        img= tensor2pil(image[0])
+
+        for grid in grids:
+            draw_rectangle(img, grid, 'red',8)
+
+        #获取临时目录：temp
+        output_dir = folder_paths.get_temp_directory()
+
+        (
+            full_output_folder,
+            filename,
+            counter,
+            subfolder,
+            _,
+        ) = folder_paths.get_save_image_path('tmp_', output_dir)
+        
+        image_file = f"{filename}_{counter:05}.png"
+
+        image_path=os.path.join(full_output_folder, image_file)
+        # 保存图片
+        img.save(image_path,compress_level=6)
+        width, height = img.size
+
+        (
+            full_output_folder,
+            filename,
+            counter,
+            _,
+            _,
+        ) = folder_paths.get_save_image_path(filename_prefix[0], output_dir)
 
 
+        data_converted = [{
+            "label":labels[i],
+            "grid":[float(grids[i][0]),
+                           float(grids[i][1]),
+                           float(grids[i][2]),
+                           float(grids[i][3])
+                           ]
+        } for i in range(len(grids))]
+
+        data={
+            "width":int(width),
+            "height":int(height),
+            "grids":data_converted
+        }
+
+        save_json_to_file(data,os.path.join(full_output_folder,f"${filename}_{counter:05}.json"))
+
+        return {"ui":{"image": [{
+                "filename": image_file,
+                "subfolder": subfolder,
+                "type":"temp"
+            }],
+            "json":[data["width"],data['height'],data["grids"]]
+            },"result": ()}
+        # return {"ui":{"image": [ ],
+             
+        #     },"result": ()}
+
+class GridInput:
+    @classmethod
+    def INPUT_TYPES(s):
+        return { 
+            "required": {
+                "grids": ("STRING", 
+                                        {
+                                            "multiline": True, 
+                                            "default": "",
+                                            "dynamicPrompts": False
+                                        }),
+                "padding":("INT",{
+                    "default": 24, 
+                    "min": -500, #Minimum value
+                    "max": 5000, #Maximum value
+                    "step": 1, #Slider's step
+                    "display": "number" # Cosmetic only: display as "number" or "slider"
+                }),
+                
+            },
+            "optional":{
+                    "width":("INT",{
+                        "forceInput": True,
+                    }),
+                     "height":("INT",{
+                        "forceInput": True,
+                    }),
+                }
+
+                }
+    
+    RETURN_TYPES = ("_GRID","STRING","IMAGE",)
+    RETURN_NAMES = ("grids","labels","image",)
+
+    FUNCTION = "run"
+
+    CATEGORY = "♾️Mixlab/Input"
+
+    INPUT_IS_LIST = True
+    OUTPUT_IS_LIST = (True,True,False,)
+    OUTPUT_NODE = True
+
+    def run(self,grids,padding,width=[-1],height=[-1]):
+        # print(padding[0],grids[0])
+        width=width[0]
+        height=height[0]
+
+        grids=grids[0]
+        data=json.loads(grids)
+        grids=data['grids']
+
+        if width>-1:
+            data['width']=width
+        if height>-1:
+            data['height']=height
+    
+        new_grids=[]
+        labels=[]
+        
+        for g in grids:
+            labels.append(g['label'])
+            new_grids.append(padding_rectangle(g['grid'],padding[0]))
+
+        image = Image.new("RGB", (int(data['width']),int(data["height"])), "white")
+        im=pil2tensor(image)
+        # image=create_temp_file(im)
+
+        data_converted = [{
+            "label":labels[i],
+            "grid":[float(new_grids[i][0]),
+                           float(new_grids[i][1]),
+                           float(new_grids[i][2]),
+                           float(new_grids[i][3])
+                           ]
+        } for i in range(len(new_grids))]
+
+        # 传递到前端节点的数据 报错,需要处理成 key:[x,x,x,x]
+        return {"ui":{
+            "json":[data["width"],data["height"],data_converted]
+            },"result": (new_grids,labels,im,)}
+    
+        # return (new_grids,labels,pil2tensor(image),)
 
 class GridOutput:
     @classmethod
@@ -1823,6 +2174,9 @@ class GridOutput:
         x,y,w,h=grid
         return (x,y,w,h,)
 
+
+
+
 class ShowLayer:
     @classmethod
     def INPUT_TYPES(s):
@@ -2218,11 +2572,15 @@ class ResizeImage:
             for ims in image:
                 for im in ims:
                     im=tensor2pil(im)
-                    im=resize_image(im,scale_option,w,h,fill_color)
-                    im=im.convert('RGB')
 
+                    im=im.convert('RGB')
                     a_im,hex=get_average_color_image(im)
 
+                    if average_color=='on':
+                        fill_color=hex
+                        
+                    im=resize_image(im,scale_option,w,h,fill_color)
+
                     im=pil2tensor(im)
                     imgs.append(im)
 
@@ -2322,7 +2680,57 @@ class GetImageSize_:
 
         return (width, height,min_width,min_height,)
 
+class SaveImageAndMetadata:
+    def __init__(self):
+        self.output_dir = folder_paths.get_output_directory()
+        self.type = "output"
+        self.prefix_append = ""
+        self.compress_level = 4
 
+    @classmethod
+    def INPUT_TYPES(s):
+        return {"required": 
+                    {"images": ("IMAGE", ),
+                     "filename_prefix": ("STRING", {"default": "Mixlab"}),
+                     "metadata": (["disable","enable"],),
+                     },
+                "hidden": {"prompt": "PROMPT", "extra_pnginfo": "EXTRA_PNGINFO"},
+                }
+
+    RETURN_TYPES = ()
+    FUNCTION = "save_images"
+
+    OUTPUT_NODE = True
+
+    CATEGORY = "♾️Mixlab/Output"
+
+    def save_images(self, images, filename_prefix="Mixlab",metadata="disable", prompt=None, extra_pnginfo=None):
+        filename_prefix += self.prefix_append
+        full_output_folder, filename, counter, subfolder, filename_prefix = folder_paths.get_save_image_path(filename_prefix, self.output_dir, images[0].shape[1], images[0].shape[0])
+        results = list()
+        for image in images:
+            i = 255. * image.cpu().numpy()
+            img = Image.fromarray(np.clip(i, 0, 255).astype(np.uint8))
+            metadata = None
+            if (not args.disable_metadata) and (metadata=="enable"):
+                print('##enable_metadata')
+                metadata = PngInfo()
+                if prompt is not None:
+                    metadata.add_text("prompt", json.dumps(prompt))
+                if extra_pnginfo is not None:
+                    for x in extra_pnginfo:
+                        metadata.add_text(x, json.dumps(extra_pnginfo[x]))
+
+            file = f"{filename}_{counter:05}_.png"
+            img.save(os.path.join(full_output_folder, file), pnginfo=metadata, compress_level=self.compress_level)
+            results.append({
+                "filename": file,
+                "subfolder": subfolder,
+                "type": self.type
+            })
+            counter += 1
+
+        return { "ui": { "images": results } }
 
 class ImageColorTransfer:
     @classmethod
@@ -2330,6 +2738,7 @@ class ImageColorTransfer:
         return {"required": {
                 "source": ("IMAGE",),
                 "target": ("IMAGE",),
+                "weight": ("FLOAT", {"default": 0.5, "min": 0.0, "max": 1.0, "step": 0.01}),
                 },
                 }
     
@@ -2340,28 +2749,48 @@ class ImageColorTransfer:
     FUNCTION = "run"
 
     # 右键菜单目录
-    CATEGORY = "♾️Mixlab/Image"
+    CATEGORY = "♾️Mixlab/Color"
 
     # 输入是否为列表
-    INPUT_IS_LIST = True
+    # INPUT_IS_LIST = True
 
     # 输出是否为列表
-    OUTPUT_IS_LIST = (True,)
+    # OUTPUT_IS_LIST = (True,)
 
-    def run(self,source,target):
+    def run(self,source,target,weight):
 
         res=[]
 
-        target=target[0][0]
-        print(target.shape)
-        target=tensor2pil(target)
+        #batch-list
+        source_list = [source[i:i + 1, ...] for i in range(source.shape[0])]
+        target_list = [target[i:i + 1, ...] for i in range(target.shape[0])]
 
-        for ims in source:
-            for im in ims:
-                image=tensor2pil(im)
-                image=color_transfer(image,target)
-                image=pil2tensor(image)
-                res.append(image)
+        # 长度纠正为相等
+        if len(target_list) != len(source_list):
+            target_list = target_list * (len(source_list) // len(target_list)) + target_list[:len(source_list) % len(target_list)]
+        
+        for i in range(len(source_list)):
+            target=target_list[i]
+            source=source_list[i]
+            target=tensor2pil(target)
+
+            image=tensor2pil(source)
+
+            image_res=color_transfer(image,target)
+
+            # weight Blend image # contributors:@ning
+            blend_mask = Image.new(mode="L", size=image.size,
+                                    color=(round(weight * 255)))
+            blend_mask = ImageOps.invert(blend_mask)
+            img_result = Image.composite(image, image_res, blend_mask)
+            del image, image_res, blend_mask
+            
+            img_result=pil2tensor(img_result)
+
+            res.append(img_result)
+
+        # list - batch
+        res=torch.cat(res, dim=0)
 
         return (res,)
 
@@ -2388,7 +2817,7 @@ class SaveImageToLocal:
 
     OUTPUT_NODE = True
 
-    CATEGORY = "♾️Mixlab/Image"
+    CATEGORY = "♾️Mixlab/Output"
 
     def save_images(self, images,file_path , prompt=None, extra_pnginfo=None):
         filename_prefix = os.path.basename(file_path)
@@ -2408,7 +2837,7 @@ class SaveImageToLocal:
         full_output_folder, filename, counter, subfolder, filename_prefix = folder_paths.get_save_image_path(filename_prefix, self.output_dir, images[0].shape[1], images[0].shape[0])
         
     
-        if not os.path.exists(file_path) and not extension:
+        if not os.path.exists(file_path):
             # 使用os.makedirs函数创建新目录
             os.makedirs(file_path)
             print("目录已创建")

nodes/Mask.py

@@ -22,6 +22,19 @@ def pil2tensor(image):
     return torch.from_numpy(np.array(image).astype(np.float32) / 255.0).unsqueeze(0)
 
 
+def add_masks(mask1, mask2):
+    mask1 = mask1.cpu()
+    mask2 = mask2.cpu()
+    cv2_mask1 = np.array(mask1) * 255
+    cv2_mask2 = np.array(mask2) * 255
+
+    if cv2_mask1.shape == cv2_mask2.shape:
+        cv2_mask = cv2.add(cv2_mask1, cv2_mask2)
+        return torch.clamp(torch.from_numpy(cv2_mask) / 255.0, min=0, max=1)
+    else:
+        return mask1
+    
+
 def grow(mask, expand, tapered_corners):
     c = 0 if tapered_corners else 1
     kernel = np.array([[c, 1, c],
@@ -87,6 +100,69 @@ class OutlineMask:
         return (m3,)
     
 
+class MaskListReplace:
+    @classmethod
+    def INPUT_TYPES(s):
+        return {"required": {
+            "masks": ("MASK",),
+            "mask_replace": ("MASK",),
+            "start_index":("INT", {"default": 0, "min": 0, "step": 1}),
+            "end_index":("INT", {"default": 0, "min": 0, "step": 1}),
+            "invert": ("BOOLEAN", {"default": False}),
+        }
+        }
+
+    RETURN_TYPES = ("MASK",)
+    FUNCTION = "run"
+    CATEGORY = "♾️Mixlab/Video"
+
+    INPUT_IS_LIST = True
+    OUTPUT_IS_LIST = (True,)
+
+    def run(self, masks,mask_replace,start_index,end_index,invert):
+        mask_replace=mask_replace[0]
+        start_index=start_index[0]
+        end_index=end_index[0]
+        invert=invert[0]
+        
+        new_masks=[]
+        for i in range(len(masks)):
+            if i>=start_index and i<=end_index:
+                if invert:
+                    new_masks.append(masks[i])
+                else:
+                    new_masks.append(mask_replace)
+            else:
+                if invert:
+                    new_masks.append(mask_replace)
+                else:
+                    new_masks.append(masks[i])
+
+        return (new_masks,)
+
+
+class MaskListMerge:
+    @classmethod
+    def INPUT_TYPES(s):
+        return {"required": {
+            "masks": ("MASK",),
+        }
+        }
+
+    RETURN_TYPES = ("MASK",)
+    FUNCTION = "run"
+    CATEGORY = "♾️Mixlab/Mask"
+
+    INPUT_IS_LIST = True
+    OUTPUT_IS_LIST = (False,)
+
+    def run(self, masks):
+        mask=masks[0]
+        if isinstance(masks, list):
+            for m in masks:
+                # print(m.shape)
+                mask = add_masks(mask, m)
+        return (mask,)
 
 
 class FeatheredMask:

nodes/PromptNode.py

@@ -6,6 +6,12 @@ from urllib import request, parse
 import folder_paths
 from PIL import Image, ImageOps,ImageFilter,ImageEnhance,ImageDraw,ImageSequence, ImageFont
 from PIL.PngImagePlugin import PngInfo
+
+import hashlib
+import requests
+import json
+
+
 # def queue_prompt(prompt_workflow):
 #     p = {"prompt": prompt_workflow}
 #     data = json.dumps(p).encode('utf-8')
@@ -15,6 +21,7 @@ from PIL.PngImagePlugin import PngInfo
 embeddings_path=os.path.join(folder_paths.models_dir, "embeddings")
 
 def get_files_with_extension(directory, extension):
+    
     file_list = []
     for root, dirs, files in os.walk(directory):
         for file in files:
@@ -28,6 +35,50 @@ def join_with_(text_list,delimiter):
     return joined_text
 
 
+
+def load_json(file_path):
+    try:
+        with open(file_path, 'r') as json_file:
+            data = json.load(json_file)
+            return data
+    except FileNotFoundError:
+        print(f"File not found: {file_path}")
+        return None
+    except json.JSONDecodeError:
+        print(f"Error decoding JSON in file: {file_path}")
+        return None
+
+def save_json(data_dict, file_path):
+    try:
+        with open(file_path, 'w') as json_file:
+            json.dump(data_dict, json_file, indent=4)
+            print(f"Data saved to {file_path}")
+    except Exception as e:
+        print(f"Error saving JSON to file: {e}")
+
+# pysss的lora加载器
+# def get_model_version_info(hash_value):
+#     # http://127.0.0.1:1082
+#     proxies = {'http': 'http://127.0.0.1:1082', 'https': 'https://127.0.0.1:1082'}
+#     api_url = f"https://civitai.com/api/v1/model-versions/by-hash/{hash_value}"
+#     print(api_url)
+#     response = requests.get(api_url,proxies=proxies, verify=False)
+    
+#     if response.status_code == 200:
+#         return response.json()
+#     else:
+#         return None
+    
+# def calculate_sha256(file_path):
+#     sha256_hash = hashlib.sha256()
+#     with open(file_path, "rb") as f:
+#         for chunk in iter(lambda: f.read(4096), b""):
+#             sha256_hash.update(chunk)
+#     return sha256_hash.hexdigest()
+
+
+
+
 class AnyType(str):
   """A special class that is always equal in not equal comparisons. Credit to pythongosssss"""
 
@@ -138,7 +189,7 @@ class PromptImage:
 
     FUNCTION = "run"
 
-    CATEGORY = "♾️Mixlab/Prompt"
+    CATEGORY = "♾️Mixlab/Output"
 
     # 运行的函数
     def run(self,prompts,images,save_to_image):
@@ -385,16 +436,92 @@ class RandomPrompt:
         return {"ui": {"prompts": prompts}, "result": (prompts,)}
 
 
+# class LoraPrompt:
+#     @classmethod
+#     def INPUT_TYPES(s):
+#         return {
+#             "required": {
+#                 "lora_name":(sorted(folder_paths.get_filename_list("loras"), key=str.lower),),
+#                 "weight": ("FLOAT", {"default": 1, "min": -2, "max": 2,"step":0.01 ,"display": "slider"}),
+#                 "force_update": ("BOOLEAN", {"default": False}),
+#                 },
+                
+#             }
+    
+#     RETURN_TYPES = ("STRING","STRING",any_type)
+#     RETURN_NAMES = ("lora_name","prompt","tags",)
 
+#     FUNCTION = "run"
+
+#     CATEGORY = "♾️Mixlab/Prompt"
+
+#     OUTPUT_IS_LIST = (False,False,True,)
+#     # OUTPUT_NODE = True
+
+#     # 运行的函数
+#     def run(self,lora_name,weight,force_update=False):
+       
+#         # print('##LoraPrompt',__file__)
+#         # 从本地数据库读取
+#         json_tags_path = os.path.join(os.path.dirname(os.path.dirname(__file__)),r'data/loras_tags.json')
+
+#         if not os.path.exists(json_tags_path):
+#             save_json({},json_tags_path)
+
+#         lora_tags = load_json(json_tags_path)
+#         output_tags = lora_tags.get(lora_name, None) if lora_tags is not None else None
+#         if output_tags is not None:
+#             output_tags = ",".join(output_tags)
+#             print("trainedWords:",output_tags)
+#         else:
+#             output_tags = ""
+
+
+#         lora_path = folder_paths.get_full_path("loras", lora_name)
+#         if output_tags == "" or force_update:
+#             print("calculating lora hash")
+#             LORAsha256 = calculate_sha256(lora_path)
+#             print("requesting infos")
+#             model_info = get_model_version_info(LORAsha256)
+#             if model_info is not None:
+#                 if "trainedWords" in model_info:
+#                     print("tags found!")
+#                     if lora_tags is None:
+#                         lora_tags = {}
+#                     lora_tags[lora_name] = model_info["trainedWords"]
+#                     save_json(lora_tags,json_tags_path)
+#                     output_tags = ",".join(model_info["trainedWords"])
+#                     print("trainedWords:",output_tags)
+#             else:
+#                 print("No informations found.")
+#                 if lora_tags is None:
+#                     lora_tags = {}
+#                 lora_tags[lora_name] = []
+#                 save_json(lora_tags,json_tags_path)
+
+
+#         weight = round(weight, 3)
+#         prompt=[]
+#         for p in output_tags.split(','):
+            
+#             if weight!=1:
+#                 prompt.append('('+p+':'+str(weight)+')')
+#             else:
+#                 prompt.append(p)
+
+#         prompt=",".join(prompt)
+
+#         return (lora_name,prompt,output_tags.split(','),)
 
 
 
 class EmbeddingPrompt:
     @classmethod
     def INPUT_TYPES(s):
+
         return {
             "required": {
-                "embedding":(get_files_with_extension(embeddings_path,'.pt'),),
+                "embedding":(folder_paths.get_filename_list("embeddings"),),
                 "weight": ("FLOAT", {"default": 1, "min": -2, "max": 2,"step":0.01 ,"display": "slider"}),
                 },
                 
@@ -419,14 +546,119 @@ class EmbeddingPrompt:
         # return (new_prompt)
         return (prompt,)
 
-RETURN_TYPES = (any_type,)
+# RETURN_TYPES = (any_type,)
+    
+# conditioning ：提示，正向or负向
+# clip：clip模型
+# gligen_textbox_model：gligen模型
+# grids：矩形框的集合
+# labels：每个矩形框对应的标签的集合
+# index：选取第几个矩形框作为gligen的box
+
+class GLIGENTextBoxApply_Advanced:
+    @classmethod
+    def INPUT_TYPES(s):
+        return {"required": {"conditioning": ("CONDITIONING", ),
+                              "clip": ("CLIP", ),
+                              "gligen_textbox_model": ("GLIGEN", ),
+                              "grids": ("_GRID",),
+                              "labels": ("STRING", 
+                                        {
+                                            "multiline": True, 
+                                            "default": "",
+                                            "forceInput": True
+                                        }),
+                              "index": ("INT", {"default": -1, "min": -1, "max": 300, "step": 1}),
+                              "max_size": ("INT", {"default": 8, "min": 1, "max": 300, "step": 1}),
+                              "random_shuffle":(["on","off"],),
+                             },
+                 "optional":{
+                    "seed": (any_type,  {"default": 0, "min": 0, "max": 0xffffffffffffffff,"step": 1}),
+                } 
+                            }
+    RETURN_TYPES = ("CONDITIONING","STRING",)
+    RETURN_NAMES = ("CONDITIONING","label",)
+
+    FUNCTION = "run"
+    # INPUT_IS_LIST = True
+    CATEGORY = "♾️Mixlab/Prompt"
+
+    def run(self, conditioning, clip, gligen_textbox_model, grids, labels, index,max_size,random_shuffle,seed=0):
+        # print('grids',grids)
+        # conditioning=conditioning[0]
+        # clip=clip[0]
+        # gligen_textbox_model=gligen_textbox_model[0]
+        # index=index[0]
+        # max_size=max_size[0]
+        # random_shuffle=random_shuffle[0]
+
+        texts=labels
+        
+        if index>-1:
+            texts=[labels[index]]
+            grids=[grids[index]]
+
+        if random_shuffle=='on':
+            sss=[[texts[i],grids[i]] for i in range(len(texts))]
+            random.shuffle(sss)
+            texts=[s[0] for s in sss]
+            grids=[s[1] for s in sss]
+
+        if len(texts) > max_size:
+            texts = texts[:max_size]
+
+        c = []
+        
+        for t in conditioning:
+            n = [t[0], t[1].copy()]
+
+
+            # 多个
+            position_params=[]
+            for i in range(len(texts)):
+                text=texts[i]
+                grid=grids[i]
+                x,y,width,height=grid
+                # print(text)
+                cond, cond_pooled = clip.encode_from_tokens(clip.tokenize(text), return_pooled=True)
+                position_params =position_params+ [(cond_pooled, height // 8, width // 8, y // 8, x // 8)]
+
+            # 前一个
+            prev = []
+            if "gligen" in n[1]:
+                prev = n[1]['gligen'][2]
+
+            n[1]['gligen'] = ("position", gligen_textbox_model, prev + position_params)
+            # print('gligen',n)
+            c.append(n)
+        
+        #  下面这个写法有bug
+        # for i in range(len(texts)):
+        #     text=texts[i]
+        #     grid=grids[i]
+        #     x,y,width,height=grid
+
+        #     cond, cond_pooled = clip.encode_from_tokens(clip.tokenize(text), return_pooled=True)
+        #     for t in conditioning:
+        #         n = [t[0], t[1].copy()]
+        #         position_params = [(cond_pooled, height // 8, width // 8, y // 8, x // 8)]
+        #         prev = []
+        #         if "gligen" in n[1]:
+        #             prev = n[1]['gligen'][2]
+
+        #         n[1]['gligen'] = ("position", gligen_textbox_model, prev + position_params)
+        #         c.append(n)
+
+
+        return (c,texts, )
+    
 
 class JoinWithDelimiter:
     @classmethod
     def INPUT_TYPES(s):
         return {"required": {
                     "text_list": (any_type,),
-                    "delimiter":(["newline","comma"],),
+                    "delimiter":(["newline","comma","backslash","space"],),
                              },
                 }
     
@@ -434,7 +666,7 @@ class JoinWithDelimiter:
 
     FUNCTION = "run"
 
-    CATEGORY = "♾️Mixlab/Prompt"
+    CATEGORY = "♾️Mixlab/Text"
 
     INPUT_IS_LIST = True # 当true的时候，输入时list，当false的时候，如果输入是list，则会自动包一层for循环调用
     OUTPUT_IS_LIST = (False,)
@@ -445,6 +677,10 @@ class JoinWithDelimiter:
             delimiter='\n'
         elif delimiter=='comma':
             delimiter=','
+        elif delimiter=='backslash':
+            delimiter='\\'
+        elif delimiter=='space':
+            delimiter=' '
         t=''
         if isinstance(text_list, list):
             t=join_with_(text_list,delimiter)

nodes/ScreenShareNode.py

@@ -93,7 +93,7 @@ class ScreenShareNode:
     RETURN_NAMES = ("IMAGE","PROMPT","FLOAT","INT")
     FUNCTION = "run"
 
-    CATEGORY = "♾️Mixlab/Image"
+    CATEGORY = "♾️Mixlab/Screen"
 
     # INPUT_IS_LIST = True
     OUTPUT_IS_LIST = (False,False,False,False)
@@ -118,7 +118,7 @@ class FloatingVideo:
     OUTPUT_NODE = True
     FUNCTION = "run"
 
-    CATEGORY = "♾️Mixlab/Image"
+    CATEGORY = "♾️Mixlab/Screen"
 
     # INPUT_IS_LIST = True
     # OUTPUT_IS_LIST = (False,False,)

nodes/Style.py

@@ -0,0 +1,503 @@
+import comfy
+import torch
+
+from dataclasses import dataclass
+import torch.nn as nn
+from comfy.model_patcher import ModelPatcher
+import comfy.ops
+from typing import Union
+import comfy.sample
+import latent_preview
+import comfy.utils
+
+T = torch.Tensor
+
+
+from .VisualStylePrompting.attention_functions import VisualStyleProcessor
+
+class ApplyVisualStylePrompting:
+
+    @classmethod
+    def INPUT_TYPES(s):
+        return {
+            "required": {
+                "reference_image": ("IMAGE",),
+                "reference_image_text": ("STRING", {"multiline": True}),
+                "model": ("MODEL",),
+                "clip": ("CLIP", ),
+                "vae": ("VAE", ),
+                "positive": ("CONDITIONING",),
+                "negative": ("CONDITIONING", ),
+                "enabled": ("BOOLEAN", {"default": True}),
+                "denoise": ("FLOAT", {"default": 1., "min": 0., "max": 1., "step": 1e-2}),
+                "batch_size": ("INT", {"default": 1, "min": 1, "max": 4096,"step":2})
+            }
+        }
+
+    RETURN_TYPES = ("MODEL", "CONDITIONING","CONDITIONING", "LATENT")
+    RETURN_NAMES = ("model", "positive", "negative", "latents")
+    
+    CATEGORY = "♾️Mixlab/Style"
+
+    FUNCTION = "run"
+
+    def run(
+        self, 
+        reference_image, 
+        reference_image_text,
+        model: comfy.model_patcher.ModelPatcher, 
+        clip,
+        vae,
+        positive,
+        negative, 
+        enabled,
+        denoise,
+        batch_size=1
+    ):
+        
+        tokens = clip.tokenize(reference_image_text)
+        cond, pooled = clip.encode_from_tokens(tokens, return_pooled=True)
+        reference_image_prompt=[[cond, {"pooled_output": pooled}]]
+   
+        reference_image = reference_image.repeat(((batch_size+1)//2, 1,1,1))
+
+        self.model = model
+        reference_latent = vae.encode(reference_image[:,:,:,:3])
+        
+        for n, m in model.model.diffusion_model.named_modules():
+            if m.__class__.__name__  == "CrossAttention":
+                processor = VisualStyleProcessor(m, enabled=enabled)
+                setattr(m, 'forward', processor.visual_style_forward)
+
+        conditioning_prompt = reference_image_prompt + positive
+        negative_prompt = negative * 2 
+
+        latents = torch.zeros_like(reference_latent) 
+        latents = torch.cat([latents] * 2)
+
+        if denoise < 1.0:
+            latents[::1] = reference_latent[:1]
+        else:
+            latents[::2] = reference_latent
+
+        denoise_mask = torch.ones_like(latents)[:, :1, ...] * denoise
+
+        denoise_mask[0] = 0.
+
+        return (model, conditioning_prompt, negative_prompt, {"samples": latents, "noise_mask": denoise_mask})    
+
+
+
+
+
+def exists(val):
+    return val is not None
+
+def default(val, d):
+    if exists(val):
+        return val
+    return d
+
+
+class StyleAlignedArgs:
+    def __init__(self, share_attn: str) -> None:
+        self.adain_keys = "k" in share_attn
+        self.adain_values = "v" in share_attn
+        self.adain_queries = "q" in share_attn
+
+    share_attention: bool = True
+    adain_queries: bool = True
+    adain_keys: bool = True
+    adain_values: bool = True
+
+
+def expand_first(
+    feat: T,
+    scale=1.0,
+) -> T:
+    """
+    Expand the first element so it has the same shape as the rest of the batch.
+    """
+    b = feat.shape[0]
+    feat_style = torch.stack((feat[0], feat[b // 2])).unsqueeze(1)
+    if scale == 1:
+        feat_style = feat_style.expand(2, b // 2, *feat.shape[1:])
+    else:
+        feat_style = feat_style.repeat(1, b // 2, 1, 1, 1)
+        feat_style = torch.cat([feat_style[:, :1], scale * feat_style[:, 1:]], dim=1)
+    return feat_style.reshape(*feat.shape)
+
+
+def concat_first(feat: T, dim=2, scale=1.0) -> T:
+    """
+    concat the the feature and the style feature expanded above
+    """
+    feat_style = expand_first(feat, scale=scale)
+    return torch.cat((feat, feat_style), dim=dim)
+
+
+def calc_mean_std(feat, eps: float = 1e-5) -> "tuple[T, T]":
+    feat_std = (feat.var(dim=-2, keepdims=True) + eps).sqrt()
+    feat_mean = feat.mean(dim=-2, keepdims=True)
+    return feat_mean, feat_std
+
+def adain(feat: T) -> T:
+    feat_mean, feat_std = calc_mean_std(feat)
+    feat_style_mean = expand_first(feat_mean)
+    feat_style_std = expand_first(feat_std)
+    feat = (feat - feat_mean) / feat_std
+    feat = feat * feat_style_std + feat_style_mean
+    return feat
+
+class SharedAttentionProcessor:
+    def __init__(self, args: StyleAlignedArgs, scale: float):
+        self.args = args
+        self.scale = scale
+
+    def __call__(self, q, k, v, extra_options):
+        if self.args.adain_queries:
+            q = adain(q)
+        if self.args.adain_keys:
+            k = adain(k)
+        if self.args.adain_values:
+            v = adain(v)
+        if self.args.share_attention:
+            k = concat_first(k, -2, scale=self.scale)
+            v = concat_first(v, -2)
+
+        return q, k, v
+
+
+def get_norm_layers(
+    layer: nn.Module,
+    norm_layers_: "dict[str, list[Union[nn.GroupNorm, nn.LayerNorm]]]",
+    share_layer_norm: bool,
+    share_group_norm: bool,
+):
+    if isinstance(layer, nn.LayerNorm) and share_layer_norm:
+        norm_layers_["layer"].append(layer)
+    if isinstance(layer, nn.GroupNorm) and share_group_norm:
+        norm_layers_["group"].append(layer)
+    else:
+        for child_layer in layer.children():
+            get_norm_layers(
+                child_layer, norm_layers_, share_layer_norm, share_group_norm
+            )
+
+
+def register_norm_forward(
+    norm_layer: Union[nn.GroupNorm, nn.LayerNorm],
+) -> Union[nn.GroupNorm, nn.LayerNorm]:
+    if not hasattr(norm_layer, "orig_forward"):
+        setattr(norm_layer, "orig_forward", norm_layer.forward)
+    orig_forward = norm_layer.orig_forward
+
+    def forward_(hidden_states: T) -> T:
+        n = hidden_states.shape[-2]
+        hidden_states = concat_first(hidden_states, dim=-2)
+        hidden_states = orig_forward(hidden_states)  # type: ignore
+        return hidden_states[..., :n, :]
+
+    norm_layer.forward = forward_  # type: ignore
+    return norm_layer
+
+
+def register_shared_norm(
+    model: ModelPatcher,
+    share_group_norm: bool = True,
+    share_layer_norm: bool = True,
+):
+    norm_layers = {"group": [], "layer": []}
+    get_norm_layers(model.model, norm_layers, share_layer_norm, share_group_norm)
+    print(
+        f"Patching {len(norm_layers['group'])} group norms, {len(norm_layers['layer'])} layer norms."
+    )
+    return [register_norm_forward(layer) for layer in norm_layers["group"]] + [
+        register_norm_forward(layer) for layer in norm_layers["layer"]
+    ]
+
+
+SHARE_NORM_OPTIONS = ["both", "group", "layer", "disabled"]
+SHARE_ATTN_OPTIONS = ["q+k", "q+k+v", "disabled"]
+
+class StyleAlignedSampleReferenceLatents:
+    @classmethod
+    def INPUT_TYPES(s):
+        return {"required":
+                    { 
+                    "reference_image": ("IMAGE",),
+                    "positive": ("CONDITIONING",),
+                    "negative": ("CONDITIONING", ),
+                    "model": ("MODEL",),
+                    "vae": ("VAE", ),
+                    "seed": ("INT", {"default": 0, "min": 0, "max": 0xffffffffffffffff}),
+                    "steps": ("INT", {"default": 20, "min": 1, "max": 10000}),
+                    "cfg": ("FLOAT", {"default": 8.0, "min": 0.0, "max": 100.0, "step":0.1, "round": 0.01}),
+
+                    "scheduler": (comfy.samplers.KSampler.SCHEDULERS.reverse(), ), 
+                    "denoise": ("FLOAT", {"default": 1, "min": 0.0, "max": 1.0, "step": 0.01}),
+                    
+                     }
+                }
+
+    RETURN_TYPES = ("STEP_LATENTS","LATENT")
+    RETURN_NAMES = ("ref_latents", "noised_output")
+
+    FUNCTION = "run"
+
+    # CATEGORY = "style_aligned"
+    CATEGORY = "♾️Mixlab/Style"
+
+    def run(self, reference_image, positive, negative, model, vae, seed, steps, cfg,scheduler,denoise):
+
+        # TODO noise_mask?
+        def vae_encode_crop_pixels(pixels):
+            x = (pixels.shape[1] // 8) * 8
+            y = (pixels.shape[2] // 8) * 8
+            if pixels.shape[1] != x or pixels.shape[2] != y:
+                x_offset = (pixels.shape[1] % 8) // 2
+                y_offset = (pixels.shape[2] % 8) // 2
+                pixels = pixels[:, x_offset:x + x_offset, y_offset:y + y_offset, :]
+            return pixels
+
+        pixels=vae_encode_crop_pixels(reference_image)
+        t = vae.encode(pixels[:,:,:,:3])
+        latent_image =  {"samples":t}
+
+        noise_seed=seed
+
+        sampler_name="ddim"
+        
+        sampler = comfy.samplers.sampler_object(sampler_name)
+
+        total_steps = steps
+        if denoise < 1.0:
+            total_steps = int(steps/denoise)
+
+        comfy.model_management.load_models_gpu([model])
+        sigmas = comfy.samplers.calculate_sigmas_scheduler(model.model, scheduler, total_steps).cpu()
+        sigmas = sigmas[-(steps + 1):]
+
+        sigmas = sigmas.flip(0)
+        if sigmas[0] == 0:
+            sigmas[0] = 0.0001
+
+        latent = latent_image
+        latent_image = latent["samples"]
+        noise = torch.zeros(latent_image.size(), dtype=latent_image.dtype, layout=latent_image.layout, device="cpu")
+
+    
+        noise_mask = None
+        if "noise_mask" in latent:
+            noise_mask = latent["noise_mask"]
+
+        ref_latents = []
+        def callback(step: int, x0: T, x: T, steps: int):
+            ref_latents.insert(0, x[0])
+        
+        disable_pbar = not comfy.utils.PROGRESS_BAR_ENABLED
+        samples = comfy.sample.sample_custom(model, noise, cfg, sampler, sigmas, positive, negative, latent_image, noise_mask=noise_mask, callback=callback, disable_pbar=disable_pbar, seed=noise_seed)
+
+        out = latent.copy()
+        out["samples"] = samples
+        out_noised = out
+
+        ref_latents = torch.stack(ref_latents)
+
+        return (ref_latents, out_noised)
+
+class StyleAlignedReferenceSampler:
+    @classmethod
+    def INPUT_TYPES(cls):
+        return {
+            "required": {
+
+                "ref_latents": ("STEP_LATENTS",),
+                "reference_image_text": ("STRING", {"multiline": True}),
+                "model": ("MODEL",),
+                "clip": ("CLIP", ),
+
+                "positive": ("CONDITIONING",),
+                "negative": ("CONDITIONING",),
+
+                "share_norm": (SHARE_NORM_OPTIONS,),
+                "share_attn": (SHARE_ATTN_OPTIONS,),
+                "scale": ("FLOAT", {"default": 1, "min": 0, "max": 2.0, "step": 0.01}),
+                "batch_size": ("INT", {"default": 2, "min": 1, "max": 8, "step": 1}),
+                "seed": ("INT", {"default": 0, "min": 0, "max": 0xffffffffffffffff}),
+                "steps": ("INT", {"default": 20, "min": 1, "max": 10000}),
+                "cfg": ("FLOAT", {"default": 8.0, "min": 0.0, "max": 100.0, "step":0.1, "round": 0.01}),
+                "scheduler": (comfy.samplers.KSampler.SCHEDULERS, ), 
+                "denoise": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.01}),
+            },
+        }
+
+    RETURN_TYPES = ("LATENT", "LATENT")
+    RETURN_NAMES = ("output", "denoised_output")
+    FUNCTION = "patch"
+    # CATEGORY = "style_aligned"
+    CATEGORY = "♾️Mixlab/Style"
+    def patch(
+        self,
+        ref_latents,
+        reference_image_text,
+        model,
+        clip,
+        positive,
+        negative,
+        share_norm,
+        share_attn,
+        scale,
+        batch_size,
+        seed,steps,cfg,scheduler,denoise
+
+    ) -> "tuple[dict, dict]":
+        
+        m = model.clone()
+
+        # ref_latents = vae.encode(reference_image[:,:,:,:3])
+
+        tokens = clip.tokenize(reference_image_text)
+        cond, pooled = clip.encode_from_tokens(tokens, return_pooled=True)
+        ref_positive=[[cond, {"pooled_output": pooled}]]
+
+        noise_seed=seed
+
+
+        total_steps = steps
+        if denoise < 1.0:
+            total_steps = int(steps/denoise)
+
+        # comfy.model_management.load_models_gpu([model])
+        sigmas = comfy.samplers.calculate_sigmas_scheduler(model.model, scheduler, total_steps).cpu()
+        sigmas = sigmas[-(steps + 1):]
+
+        sampler_name="ddim"
+        
+        sampler = comfy.samplers.sampler_object(sampler_name)
+
+        args = StyleAlignedArgs(share_attn)
+
+        # Concat batch with style latent
+        style_latent_tensor = ref_latents[0].unsqueeze(0)
+        height, width = style_latent_tensor.shape[-2:]
+        latent_t = torch.zeros(
+            [batch_size, 4, height, width], device=ref_latents.device
+        )
+        latent = {"samples": latent_t}
+        noise = comfy.sample.prepare_noise(latent_t, noise_seed)
+
+        latent_t = torch.cat((style_latent_tensor, latent_t), dim=0)
+        ref_noise = torch.zeros_like(noise[0]).unsqueeze(0)
+        noise = torch.cat((ref_noise, noise), dim=0)
+
+        x0_output = {}
+        preview_callback = latent_preview.prepare_callback(m, sigmas.shape[-1] - 1, x0_output)
+
+        # Replace first latent with the corresponding reference latent after each step
+        def callback(step: int, x0: T, x: T, steps: int):
+            preview_callback(step, x0, x, steps)
+            if (step + 1 < steps):
+                # 当ref_latents的step不够时
+                if step+1>len(ref_latents)-1:
+                    step=len(ref_latents)-2
+                
+                x[0] = ref_latents[step+1]
+                x0[0] = ref_latents[step+1]
+
+        # Register shared norms
+        share_group_norm = share_norm in ["group", "both"]
+        share_layer_norm = share_norm in ["layer", "both"]
+        register_shared_norm(m, share_group_norm, share_layer_norm)
+
+        # Patch cross attn
+        m.set_model_attn1_patch(SharedAttentionProcessor(args, scale))
+
+        # Add reference conditioning to batch 
+        batched_condition = []
+        for i,condition in enumerate(positive):
+            additional = condition[1].copy()
+            batch_with_reference = torch.cat([ref_positive[i][0], condition[0].repeat([batch_size] + [1] * len(condition[0].shape[1:]))], dim=0)
+            if 'pooled_output' in additional and 'pooled_output' in ref_positive[i][1]:
+                # combine pooled output
+                pooled_output = torch.cat([ref_positive[i][1]['pooled_output'], additional['pooled_output'].repeat([batch_size] 
+                    + [1] * len(additional['pooled_output'].shape[1:]))], dim=0)
+                additional['pooled_output'] = pooled_output
+            if 'control' in additional:
+                if 'control' in ref_positive[i][1]:
+                    # combine control conditioning
+                    control_hint = torch.cat([ref_positive[i][1]['control'].cond_hint_original, additional['control'].cond_hint_original.repeat([batch_size] 
+                        + [1] * len(additional['control'].cond_hint_original.shape[1:]))], dim=0)
+                    cloned_controlnet = additional['control'].copy()
+                    cloned_controlnet.set_cond_hint(control_hint, strength=additional['control'].strength, timestep_percent_range=additional['control'].timestep_percent_range)
+                    additional['control'] = cloned_controlnet
+                else:
+                    # add zeros for first in batch
+                    control_hint = torch.cat([torch.zeros_like(additional['control'].cond_hint_original), additional['control'].cond_hint_original.repeat([batch_size] 
+                        + [1] * len(additional['control'].cond_hint_original.shape[1:]))], dim=0)
+                    cloned_controlnet = additional['control'].copy()
+                    cloned_controlnet.set_cond_hint(control_hint, strength=additional['control'].strength, timestep_percent_range=additional['control'].timestep_percent_range)
+                    additional['control'] = cloned_controlnet
+            batched_condition.append([batch_with_reference, additional])
+
+        disable_pbar = not comfy.utils.PROGRESS_BAR_ENABLED
+        samples = comfy.sample.sample_custom(
+            m,
+            noise,
+            cfg,
+            sampler,
+            sigmas,
+            batched_condition,
+            negative,
+            latent_t,
+            callback=callback,
+            disable_pbar=disable_pbar,
+            seed=noise_seed,
+        )
+
+        # remove reference image
+        samples = samples[1:]
+
+        out = latent.copy()
+        out["samples"] = samples
+        if "x0" in x0_output:
+            out_denoised = latent.copy()
+            x0 = x0_output["x0"][1:]
+            out_denoised["samples"] = m.model.process_latent_out(x0.cpu())
+        else:
+            out_denoised = out
+        return (out, out_denoised)
+
+
+class StyleAlignedBatchAlign:
+    @classmethod
+    def INPUT_TYPES(cls):
+        return {
+            "required": {
+                "model": ("MODEL",),
+                "share_norm": (SHARE_NORM_OPTIONS,),
+                "share_attn": (SHARE_ATTN_OPTIONS,),
+                "scale": ("FLOAT", {"default": 1, "min": 0, "max": 1.0, "step": 0.1}),
+            }
+        }
+
+    RETURN_TYPES = ("MODEL",)
+    FUNCTION = "patch"
+    # CATEGORY = "style_aligned"
+    CATEGORY = "♾️Mixlab/Style"
+    def patch(
+        self,
+        model: ModelPatcher,
+        share_norm: str,
+        share_attn: str,
+        scale: float,
+    ):
+        m = model.clone()
+        share_group_norm = share_norm in ["group", "both"]
+        share_layer_norm = share_norm in ["layer", "both"]
+        register_shared_norm(model, share_group_norm, share_layer_norm)
+        args = StyleAlignedArgs(share_attn)
+        m.set_model_attn1_patch(SharedAttentionProcessor(args, scale))
+        return (m,)
+
+

nodes/TextGenerateNode.py

@@ -12,6 +12,8 @@ import comfy.utils
 # import numpy as np
 import torch
 import random
+from lark import Lark, Transformer, v_args
+
 
 global _available
 _available=True
@@ -62,7 +64,14 @@ except:
 
 
 
-def translate(zh_en_tokenizer,zh_en_model,text):
+def translate(text):
+    global text_pipe,zh_en_model,zh_en_tokenizer
+    
+    if zh_en_model==None:
+        zh_en_model = AutoModelForSeq2SeqLM.from_pretrained(zh_en_model_path).eval()
+        zh_en_tokenizer = AutoTokenizer.from_pretrained(zh_en_model_path,padding=True, truncation=True)
+    
+    zh_en_model.to("cuda" if torch.cuda.is_available() else "cpu")
     with torch.no_grad():
         encoded = zh_en_tokenizer([text], return_tensors="pt")
         encoded.to(zh_en_model.device)
@@ -102,18 +111,24 @@ def text_generate(text_pipe,input,seed=None):
     
 import re
 
-def correct_prompt_syntax(prompt):
+def correct_prompt_syntax(prompt=""):
 
-    print("input prompt",prompt)
+    # print("input prompt",prompt)
     corrected_elements = []
     # 处理成统一的英文标点
     prompt = prompt.replace('（', '(').replace('）', ')').replace('，', ',').replace(';', ',').replace('。', '.').replace('：',':')
     # 删除多余的空格
     prompt = re.sub(r'\s+', ' ', prompt).strip()
+    prompt = prompt.replace("< ","<").replace(" >",">").replace("( ","(").replace(" )",")").replace("[ ","[").replace(' ]',']')
 
     # 分词
     prompt_elements = prompt.split(',')
 
+    def balance_brackets(element, open_bracket, close_bracket):
+        open_brackets_count = element.count(open_bracket)
+        close_brackets_count = element.count(close_bracket)
+        return element + close_bracket * (open_brackets_count - close_brackets_count)
+
     for element in prompt_elements:
         element = element.strip()
 
@@ -133,21 +148,118 @@ def correct_prompt_syntax(prompt):
         corrected_elements.append(corrected_element)
 
     # 重组修正后的prompt
-    corrected_prompt = ', '.join(corrected_elements)
-    print("output prompt",corrected_prompt)
-    return corrected_prompt
+    return  ','.join(corrected_elements)
 
-def balance_brackets(element, open_bracket, close_bracket):
-    open_brackets_count = element.count(open_bracket)
-    close_brackets_count = element.count(close_bracket)
-    return element + close_bracket * (open_brackets_count - close_brackets_count)
 
 # # 示例使用
 # test_prompt = "((middle-century castles)), [forsaken: 0.8], (mystery dragons: 1.3, mist forests, sunsets, quiet; (((dummy)), [fisting city: 0.5] background, radiant, soft and flavoured,] promising mountains, ((starry: 1.6), [[crowds], [middle-century castle: urban landscapes of the future: 0.5], [yellow: bright sun: 0.7], overlooking"
 # corrected_prompt = correct_prompt_syntax(test_prompt)
 # print(corrected_prompt)
 
+def detect_language(input_str):
+    # 统计中文和英文字符的数量
+    count_cn = count_en = 0
+    for char in input_str:
+        if '\u4e00' <= char <= '\u9fff':
+            count_cn += 1
+        elif char.isalpha():
+            count_en += 1
+
+    # 根据统计的字符数量判断主要语言
+    if count_cn > count_en:
+        return "cn"
+    elif count_en > count_cn:
+        return "en"
+    else:
+        return "unknow"
+
+
+
+
+#定义Prompt文法
+grammar = """
+start: sentence
+sentence: phrase ("," phrase)*
+phrase: emphasis | weight | word | lora | embedding | schedule
+emphasis: "(" sentence ")" -> emphasis
+        | "[" sentence "]" -> weak_emphasis
+weight: "(" word ":" NUMBER ")"
+schedule: "[" word ":" word ":" NUMBER "]"
+lora: "<" WORD ":" WORD (":" NUMBER)? (":" NUMBER)? ">"
+embedding: "embedding" ":" WORD (":" NUMBER)? (":" NUMBER)?
+word: WORD
+
+NUMBER: /\s*-?\d+(\.\d+)?\s*/
+WORD: /[^,:\(\)\[\]<>]+/
+"""
+
+
+
+@v_args(inline=True)  # Decorator to flatten the tree directly into the function arguments
+class ChinesePromptTranslate(Transformer):
+ 
+    def sentence(self, *args):
+        return ", ".join(args)
+
+    def phrase(self, *args):
+        return "".join(args)
+
+    def emphasis(self, *args):
+        # Reconstruct the emphasis with translated content
+        return "(" + "".join(args) + ")"
+
+    def weak_emphasis(self, *args):
+        print('weak_emphasis:',args)
+        return "[" + "".join(args) + "]"
+    
+    def embedding(self,*args):
+        print('prompt embedding',args[0])
+        if len(args) == 1:
+            # print('prompt embedding',str(args[0]))
+            # 只传递了一个参数，意味着只有embedding名称没有数字
+            embedding_name = str(args[0])
+            return f"embedding:{embedding_name}"
+        elif len(args) > 1:
+            embedding_name,*numbers = args
+           
+            if len(numbers)==2:
+                return f"embedding:{embedding_name}:{numbers[0]}:{numbers[1]}"
+            elif len(numbers)==1:
+                return f"embedding:{embedding_name}:{numbers[0]}"
+            else:
+                return f"embedding:{embedding_name}"
+
+    def lora(self,*args):
+        print('lora prompt',*args)
+        if len(args) == 1:
+            return f"<lora:{loar_name}>"
+        elif len(args) > 1: 
+            # print('lora', args)
+            _,loar_name,*numbers = args
+            loar_name = str(loar_name).strip()
+            if len(numbers)==2:
+                return f"<lora:{loar_name}:{numbers[0]}:{numbers[1]}>"
+            elif len(numbers)==1:
+                return f"<lora:{loar_name}:{numbers[0]}>"
+            else:
+                return f"<lora:{loar_name}>"
+    
+    def weight(self, word,number):
+        translated_word = translate(str(word)).rstrip('.')
+        return f"({translated_word}:{str(number).strip()})"
+    
+    def schedule(self,*args):
+        print('prompt schedule',args)
+        data = [str(arg).strip() for arg in args] 
         
+        return f"[{':'.join(data)}]"
+
+    def word(self, word):
+        # Translate each word using the dictionary
+        if detect_language(str(word)) == "cn":
+            return  translate(str(word)).rstrip('.')
+        else:
+            return str(word).rstrip('.')
         
 class ChinesePrompt:
 
@@ -185,16 +297,16 @@ class ChinesePrompt:
     zh_en_tokenizer=None 
 
     def run(self,text,seed,generation):
-        global text_pipe,zh_en_model,zh_en_tokenizer
+        
 
         seed=seed[0]
         generation=generation[0]
 
         # 进度条
         pbar = comfy.utils.ProgressBar(len(text)+1)
-        
         texts = [correct_prompt_syntax(t) for t in text]
-        print('correct_prompt_syntax::',texts)
+
+        global text_pipe,zh_en_model,zh_en_tokenizer
         if zh_en_model==None:
             zh_en_model = AutoModelForSeq2SeqLM.from_pretrained(zh_en_model_path).eval()
             zh_en_tokenizer = AutoTokenizer.from_pretrained(zh_en_model_path,padding=True, truncation=True)
@@ -210,9 +322,16 @@ class ChinesePrompt:
 
         # print('zh_en_model device',zh_en_model.device,text_pipe.model.device,torch.cuda.current_device() )
         en_texts=[]
+
         for t in texts:
-            en_text=translate(zh_en_tokenizer,zh_en_model,t)
-            en_texts.append(en_text)
+            if t:
+                # translated_text =  translated_word = translate(zh_en_tokenizer,zh_en_model,str(t))
+                parser = Lark(grammar, start="start", parser="lalr", transformer=ChinesePromptTranslate())
+                # print('t',t)
+                result = parser.parse(t).children
+                # print('en_result',result)        
+                # en_text=translate(zh_en_tokenizer,zh_en_model,text_without_syntax)
+                en_texts.append(result[0])
 
         zh_en_model.to('cpu')
         print("test en_text",en_texts)
@@ -232,8 +351,11 @@ class ChinesePrompt:
             pbar.update(1)
 
         text_pipe.model.to('cpu')
-        prompt_result = [correct_prompt_syntax(p) for p in prompt_result]
-        
+
+        print('prompt_result',prompt_result,)
+        # prompt_result = [','.join(correct_prompt_syntax(p)) for p in prompt_result]
+        if len(prompt_result)==0:
+            prompt_result=[""]
         return {
             "ui":{
                     "prompt": prompt_result
@@ -241,6 +363,8 @@ class ChinesePrompt:
             "result": (prompt_result,)}
 
 
+
+
 class PromptGenerate:
 
     global _available

nodes/Utils.py

@@ -6,8 +6,20 @@ import numpy as np
 import folder_paths
 import matplotlib.font_manager as fm
 import torch
+import importlib.util
 
+def create_incrementing_list(min_value, max_value, step, count):
+    l1 = [int(min_value + i * step) for i in range(count) if min_value + i * step <= max_value]
+    l2 = [float(min_value + i * step) for i in range(count) if min_value + i * step <= max_value]
+    return (l1,l2)
 
+def split_list(lst, chunk_size, transition_size):
+    result = []
+    for i in range(0, len(lst), chunk_size):
+        start = i - transition_size
+        end = i + chunk_size + transition_size
+        result.append(lst[max(start, 0):end])
+    return result
 
 def recursive_search(directory, excluded_dir_names=None):
     if not os.path.isdir(directory):
@@ -146,7 +158,6 @@ class ColorInput:
     @classmethod
     def INPUT_TYPES(s):
         return {"required": { 
-             
                     "color":("TCOLOR",), 
                              },
                 }
@@ -156,7 +167,7 @@ class ColorInput:
 
     FUNCTION = "run"
 
-    CATEGORY = "♾️Mixlab/Utils"
+    CATEGORY = "♾️Mixlab/Color"
 
     INPUT_IS_LIST = False
     OUTPUT_IS_LIST = (False,False,False,False,False,)
@@ -185,7 +196,7 @@ class FontInput:
 
     FUNCTION = "run"
 
-    CATEGORY = "♾️Mixlab/Utils"
+    CATEGORY = "♾️Mixlab/Input"
 
     INPUT_IS_LIST = False
     OUTPUT_IS_LIST = (False,)
@@ -218,7 +229,7 @@ class TextToNumber:
 
     FUNCTION = "run"
 
-    CATEGORY = "♾️Mixlab/Utils"
+    CATEGORY = "♾️Mixlab/Text"
 
     INPUT_IS_LIST = False
     OUTPUT_IS_LIST = (False,)
@@ -273,10 +284,10 @@ class FloatSlider:
                 }
     
     RETURN_TYPES = ("FLOAT",) 
-
+    RETURN_NAMES = ('weight(0-1)',)
     FUNCTION = "run"
 
-    CATEGORY = "♾️Mixlab/Utils"
+    CATEGORY = "♾️Mixlab/Input"
 
     INPUT_IS_LIST = False
     OUTPUT_IS_LIST = (False,)
@@ -329,7 +340,7 @@ class IntNumber:
 
     FUNCTION = "run"
 
-    CATEGORY = "♾️Mixlab/Utils"
+    CATEGORY = "♾️Mixlab/Input"
 
     INPUT_IS_LIST = False
     OUTPUT_IS_LIST = (False,)
@@ -347,7 +358,7 @@ class MultiplicationNode:
         return {"required": {
                     "numberA":(any_type,),
                     "multiply_by":("FLOAT", {
-                        "default": 0, 
+                        "default": 1, 
                         "min": -2, #Minimum value
                         "max": 0xffffffffffffffff,
                         "step": 0.01, #Slider's step
@@ -381,7 +392,7 @@ class TextInput:
     @classmethod
     def INPUT_TYPES(s):
         return {"required": {
-                    "text": ("STRING",{"multiline": True,"default": ""}),
+                    "text": ("STRING",{"multiline": True,"default": ""})
                              },
                 }
     
@@ -389,7 +400,7 @@ class TextInput:
 
     FUNCTION = "run"
 
-    CATEGORY = "♾️Mixlab/Utils"
+    CATEGORY = "♾️Mixlab/Input"
 
     INPUT_IS_LIST = False
     OUTPUT_IS_LIST = (False,)
@@ -398,6 +409,61 @@ class TextInput:
        
         return (text,)
     
+
+class IncrementingListNode:
+    @classmethod
+    def INPUT_TYPES(s):
+        return {"required": {
+                    "min_value": ("FLOAT", {
+                        "default": 0, 
+                        "min": -2000, #Minimum value
+                        "max": 0xffffffffffffffff,
+                        "step": 0.01, #Slider's step
+                        "display": "number" # Cosmetic only: display as "number" or "slider"
+                    }),
+                    "max_value": ("FLOAT", {
+                        "default": 10, 
+                        "min": -2000, #Minimum value
+                        "max": 0xffffffffffffffff,
+                        "step": 0.01, #Slider's step
+                        "display": "number" # Cosmetic only: display as "number" or "slider"
+                    }),
+                    "step": ("FLOAT", {
+                        "default": 0, 
+                        "min": -2000, #Minimum value
+                        "max": 0xffffffffffffffff,
+                        "step": 0.01, #Slider's step
+                        "display": "number" # Cosmetic only: display as "number" or "slider"
+                    }),
+                    "count": ("INT", {
+                        "default": 1, 
+                        "min": 1, #Minimum value
+                        "max": 0xffffffffffffffff,
+                        "step":1, #Slider's step
+                        "display": "number" # Cosmetic only: display as "number" or "slider"
+                    })
+                             },
+                             "optional":{
+                    "seed":("INT", {"default": -1, "min": -1, "max": 1000000}), 
+                    
+                },
+                }
+    
+    RETURN_TYPES = ("INT","FLOAT",) 
+    RETURN_NAMES = ('int_list','float_list',)
+
+    FUNCTION = "run"
+
+    CATEGORY = "♾️Mixlab/Video"
+
+    INPUT_IS_LIST = False
+    OUTPUT_IS_LIST = (True,True,)
+
+    def run(self,min_value,max_value,step,count,seed):
+        print('create_incrementing_list',seed)
+        l1,l2=create_incrementing_list(min_value,max_value,step,count)
+        return (l1,l2,)
+    
 # 接收一个值，然后根据字符串或数值长度计算延迟时间，用户可以自定义延迟"字/s"，延迟之后将转化
 
 import comfy.samplers
@@ -560,9 +626,11 @@ class SwitchByIndex:
     @classmethod
     def INPUT_TYPES(cls):
         return {
-            "required": {
-                "A":(any_type,),
-                "B":(any_type,),
+             "optional":{
+                  "A":(any_type,),
+                  "B":(any_type,),
+            },
+            "required": { 
                 "index":("INT", {
                         "default": -1, 
                         "min": -1, 
@@ -574,17 +642,17 @@ class SwitchByIndex:
             }
         }
 
-    RETURN_TYPES = (any_type,)
-    RETURN_NAMES = ("C",)
+    RETURN_TYPES = (any_type,"INT",)
+    RETURN_NAMES = ("list", "count",)
 
     FUNCTION = "run"
 
     CATEGORY = "♾️Mixlab/Utils"
 
     INPUT_IS_LIST = True
-    OUTPUT_IS_LIST = (True,)
+    OUTPUT_IS_LIST = (True, False,)
 
-    def run(self, A,B,index,flat):
+    def run(self, A=[],B=[],index=-1,flat='on'):
 
         flat=flat[0]
 
@@ -603,10 +671,43 @@ class SwitchByIndex:
             try:
                 C=[C[index]]
             except Exception as e:
-                C=[]
-        
-        return (C,)
+                C=[C[-1]] #最后一个
 
+        return (C, len(C),)
+
+class ListSplit:
+    @classmethod
+    def INPUT_TYPES(cls):
+        return {
+             "optional":{
+                  "A":(any_type,), 
+            },
+            "required": { 
+                "chunk_size": ("INT", {"default": 10, "min": 1, "step": 1}),
+                "transition_size": ("INT", {"default": 0, "min": 0, "step": 1}),
+                "index": ("INT", {"default": -1, "min": -1, "step": 1}),
+            }
+        }
+
+    RETURN_TYPES = (any_type,)
+    RETURN_NAMES = ("B",)
+
+    FUNCTION = "run"
+
+    CATEGORY = "♾️Mixlab/Utils"
+
+    INPUT_IS_LIST = True
+    OUTPUT_IS_LIST = (True,)
+
+    def run(self, A=[],chunk_size=[10],transition_size=[0],index=[-1]):
+        # print(len(A))
+        B=split_list(A,chunk_size[0],transition_size[0])
+
+        if index[0]>-1:
+            B=B[index[0]]
+
+        return (B,)
+    
 
 
 class LimitNumber:
@@ -637,7 +738,7 @@ class LimitNumber:
 
     FUNCTION = "run"
 
-    CATEGORY = "♾️Mixlab/Utils"
+    CATEGORY = "♾️Mixlab/Input"
 
     INPUT_IS_LIST = False
     OUTPUT_IS_LIST = (False,)
@@ -695,30 +796,85 @@ class ListStatistics:
 class TESTNODE_:
     @classmethod
     def INPUT_TYPES(s):
-        return {"required": { "ANY":(any_type,), },
+        return {"required": { 
+            "ANY":(any_type,),                
+                              },
                 }
     
     RETURN_TYPES = (any_type,)
 
     FUNCTION = "run"
 
-    CATEGORY = "♾️Mixlab/__TEST"
+    CATEGORY = "♾️Mixlab/Test"
 
     OUTPUT_NODE = True
     INPUT_IS_LIST = True
     OUTPUT_IS_LIST = (True,)
 
     def run(self,ANY):
-        print(ANY)
+        print(type(ANY))
+        try:
+            print(ANY[0].shape)
+            img= tensor2pil(ANY[0])
+            print(img.size)
+        except:
+            print('')
+        
         # data=ANY
         list_stats = ListStatistics()
 
         # 调用count_types方法进行统计
         result = list_stats.count_types(ANY)
-            
+ 
+
+        # 假设我们有一个模块文件名为 my_module.py，它位于 'importables' 目录下
+        module_path = os.path.join(os.path.dirname(__file__),'test.py')
+
+        # 使用 spec_from_file_location 获取模块的元数据（名称、定义等）
+        spec = importlib.util.spec_from_file_location('test', module_path)
+
+        module = importlib.util.module_from_spec(spec)
+        spec.loader.exec_module(module)
+
+        functions = getattr(module, 'run')  # 获取函数
+        
+        functions(ANY)
+
+
         return {"ui": {"data": result,"type":[str(type(ANY[0]))]}, "result": (ANY,)}
 
 
+class TESTNODE_TOKEN:
+    @classmethod
+    def INPUT_TYPES(s):
+        return {"required": { 
+            "text":("STRING", {"forceInput": True,}),
+            "clip": ("CLIP", )                 
+                              },
+                }
+    
+    RETURN_TYPES = ("STRING",)
+
+    FUNCTION = "run"
+
+    CATEGORY = "♾️Mixlab/Test"
+
+    OUTPUT_NODE = True
+    INPUT_IS_LIST = False
+    OUTPUT_IS_LIST = (False,)
+
+    def run(self,text,clip=None):
+        # print(text)
+        
+        tokens = clip.tokenize(text)
+        
+        tokens=[v for v in tokens.values()][0][0]
+
+        tokens=json.dumps(tokens)
+        
+        return (tokens,)
+
+
 
 class CreateSeedNode:
     def __init__(self):
@@ -738,7 +894,7 @@ class CreateSeedNode:
     OUTPUT_NODE = True
     FUNCTION = "run"
 
-    CATEGORY = "♾️Mixlab/Utils"
+    CATEGORY = "♾️Mixlab/Experiment"
 
     def run(self, seed):
         return (seed,)
@@ -765,7 +921,7 @@ class CreateCkptNames:
     # OUTPUT_NODE = True
     FUNCTION = "run"
 
-    CATEGORY = "♾️Mixlab/Utils"
+    CATEGORY = "♾️Mixlab/Experiment"
 
     def run(self, ckpt_names):
         ckpt_names=ckpt_names.split('\n')
@@ -794,7 +950,7 @@ class CreateLoraNames:
     # OUTPUT_NODE = True
     FUNCTION = "run"
 
-    CATEGORY = "♾️Mixlab/Utils"
+    CATEGORY = "♾️Mixlab/Experiment"
 
     def run(self, lora_names):
         lora_names=lora_names.split('\n')
@@ -825,7 +981,7 @@ class CreateSampler_names:
     # OUTPUT_NODE = True
     FUNCTION = "run"
 
-    CATEGORY = "♾️Mixlab/Utils"
+    CATEGORY = "♾️Mixlab/Experiment"
 
     def run(self, sampler_names):
         sampler_names=sampler_names.split('\n')

nodes/Video.py

@@ -0,0 +1,692 @@
+import os
+import hashlib
+import json
+import subprocess
+import shutil
+import re
+import time,math
+import numpy as np
+from typing import List
+import torch
+from PIL import Image, ImageOps
+from PIL.PngImagePlugin import PngInfo
+import cv2,random,string
+from pathlib import Path
+
+import folder_paths
+from comfy.k_diffusion.utils import FolderOfImages
+from comfy.utils import common_upscale
+
+
+
+
+def generate_folder_name(directory,video_path):
+    # Get the directory and filename from the video path
+    _, filename = os.path.split(video_path)
+    # Generate a random string of lowercase letters and digits
+    random_string = ''.join(random.choices(string.ascii_lowercase + string.digits, k=8))
+    # Create the folder name by combining the random string and the filename
+    folder_name = random_string + '_' + filename
+    # Create the full folder path by joining the directory and the folder name
+    folder_path = os.path.join(directory, folder_name)
+    return folder_path
+
+def create_folder(directory,video_path):
+    folder_path = generate_folder_name(directory,video_path)
+    os.makedirs(folder_path)
+    return folder_path
+
+
+def split_video(video_path, video_segment_frames, transition_frames, output_dir):
+    # 读取视频文件
+    video_capture = cv2.VideoCapture(video_path)
+    
+    # 获取视频的总帧数和帧率
+    total_frames = int(video_capture.get(cv2.CAP_PROP_FRAME_COUNT))
+    fps = video_capture.get(cv2.CAP_PROP_FPS)
+    
+    # 计算每个视频片段的总帧数，包括过渡帧
+    segment_total_frames = video_segment_frames + transition_frames
+    
+    # 计算可以分割的片段数量，向上取整
+    num_segments = (total_frames + transition_frames - 1) // segment_total_frames
+    
+    vs=[]
+    # 计算每个片段的起始帧和结束帧
+    start_frame = 0
+    for i in range(num_segments):
+        # 计算当前片段的结束帧，注意最后一个片段可能没有过渡帧
+        end_frame = min(start_frame + segment_total_frames, total_frames)
+        
+        # 打印当前片段的起始帧和结束帧
+        print(f"Segment {i+1}: Start Frame {start_frame}, End Frame {end_frame}")
+        
+        # 保存当前片段为一个视频文件
+        segment_video_path = f"{output_dir}/segment_{i+1}.avi"
+
+        fourcc = cv2.VideoWriter_fourcc(*'XVID')
+        segment_video = cv2.VideoWriter(segment_video_path, fourcc, fps, (int(video_capture.get(cv2.CAP_PROP_FRAME_WIDTH)),
+                                                              int(video_capture.get(cv2.CAP_PROP_FRAME_HEIGHT))))
+        
+        for frame_num in range(start_frame, end_frame):
+            ret, frame = video_capture.read()
+            if ret:
+                segment_video.write(frame)
+            else:
+                break  # 如果读取失败，则退出循环
+        
+        # 更新起始帧为下一个片段的起始位置
+        start_frame = end_frame + transition_frames
+        vs.append(segment_video_path)
+    
+    # 释放视频捕获对象
+    video_capture.release()
+    # print(vs)
+    return (vs,total_frames,fps)
+
+
+folder_paths.folder_names_and_paths["video_formats"] = (
+    [
+        os.path.join(os.path.dirname(os.path.abspath(__file__)), "..", "video_formats"),
+    ],
+    [".json"]
+)
+
+ffmpeg_path = shutil.which("ffmpeg")
+if ffmpeg_path is None:
+    print("ffmpeg could not be found. Using ffmpeg from imageio-ffmpeg.")
+    from imageio_ffmpeg import get_ffmpeg_exe
+    try:
+        ffmpeg_path = get_ffmpeg_exe()
+    except:
+        print("ffmpeg could not be found. Outputs that require it have been disabled")
+
+# Tensor to PIL
+def tensor2pil(image):
+    return Image.fromarray(np.clip(255. * image.cpu().numpy().squeeze(), 0, 255).astype(np.uint8))
+
+# Convert PIL to Tensor
+def pil2tensor(image):
+    return torch.from_numpy(np.array(image).astype(np.float32) / 255.0).unsqueeze(0)
+
+def count_files(directory):
+    count = 0
+    for root, dirs, files in os.walk(directory):
+        count += len(files)
+    return count
+
+def create_temp_file(image):
+    output_dir = folder_paths.get_temp_directory()
+
+    c=count_files(output_dir)
+
+    (
+            full_output_folder,
+            filename,
+            counter,
+            subfolder,
+            _,
+        ) = folder_paths.get_save_image_path('temp_', output_dir)
+
+    
+    image=tensor2pil(image)
+ 
+    image_file = f"{filename}_{c}_{counter:05}.png"
+     
+    image_path=os.path.join(full_output_folder, image_file)
+
+    image.save(image_path,compress_level=4)
+
+    return [{
+                "filename": image_file,
+                "subfolder": subfolder,
+                "type": "temp"
+                }]
+
+
+def split_list(lst, chunk_size, transition_size):
+    result = []
+    for i in range(0, len(lst), chunk_size):
+        start = i - transition_size
+        end = i + chunk_size + transition_size
+        result.append(lst[max(start, 0):end])
+    return result
+
+# images = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
+# chunk_size = 3
+# transition_size = 1
+
+# result = split_list(images, chunk_size, transition_size)
+# print(result)
+
+class ImageListReplace:
+    @classmethod
+    def INPUT_TYPES(s):
+        return {"required": {
+            "images": ("IMAGE",),
+            "start_index":("INT", {"default": 0, "min": 0, "step": 1}),
+            "end_index":("INT", {"default": 0, "min": 0, "step": 1}),
+            "invert": ("BOOLEAN", {"default": False}),
+        },
+         "optional":{
+                    "image_replace": ("IMAGE",),
+                    "images_replace": ("IMAGE",),
+                    
+                    }
+        }
+
+    RETURN_TYPES = ("IMAGE","IMAGE",)
+    RETURN_NAMES = ("images","select_images",)
+    FUNCTION = "run"
+    CATEGORY = "♾️Mixlab/Video"
+
+    OUTPUT_NODE = True
+    INPUT_IS_LIST = True
+    OUTPUT_IS_LIST = (True,True,)
+
+    def run(self, images,start_index=[0],end_index=[0],invert=[False],image_replace=None,images_replace=None):
+        start_index=start_index[0]
+        end_index=end_index[0]
+        invert=invert[0]
+
+        image_rs=[]
+
+        if image_replace!=None:
+            for i in range(end_index-start_index+1):
+                image_rs.append(image_replace[0])
+
+        if images_replace!=None:
+            image_rs=images_replace
+
+        # 如果image replace 为空
+        if image_replace==None and images_replace==None:
+            # print('如果image replace 为空',images[0])
+            # [[tensor(
+            # tensor([[[[0.
+            first_image=tensor2pil(images[0][0])
+            width, height = first_image.size
+            image_replace=Image.new("RGB", (width, height), (0, 0, 0))
+            image_replace=pil2tensor(image_replace)
+            for i in range(end_index-start_index+1):
+                image_rs.append(image_replace)
+
+        
+        new_images=[]
+        select_images=[]
+        k=0
+        for i in range(len(images)):
+            if i>=start_index and i<=end_index:
+                if invert:
+                    new_images.append(images[i])
+                else:
+                    new_images.append(image_rs[k])
+                    select_images.append(images[i])
+                    k+=1
+            else:
+                if invert:
+                    new_images.append(image_rs[k])
+                    select_images.append(images[i])
+                    k+=1
+                else:
+                    new_images.append(images[i])
+
+        imss=[]
+        # print(len(images))
+        for i in range(len(images)):
+            t=images[i][0]
+            t=tensor2pil(t)
+            t = t.convert("RGB")
+            original_width, original_height = t.size
+            scale = 300 / original_width
+            new_height = int(original_height * scale)
+            t = t.resize((300, new_height))
+
+            ims=create_temp_file(pil2tensor(t))
+            imss.append(ims[0])
+
+        # image_replace=create_temp_file(image_replace)
+
+        return {"ui":{"_images": imss},"result": (new_images,select_images,)}
+
+# The code is based on ComfyUI-VideoHelperSuite modification.
+class LoadVideoAndSegment:
+    @classmethod
+    def INPUT_TYPES(s):
+        video_extensions = ['webm', 'mp4', 'mkv', 'gif']
+        input_dir = folder_paths.get_input_directory()
+        files = []
+        for f in os.listdir(input_dir):
+            if os.path.isfile(os.path.join(input_dir, f)):
+                file_parts = f.split('.')
+                if len(file_parts) > 1 and (file_parts[-1] in video_extensions):
+                    files.append(f)
+        return {"required": {
+                    "video": (sorted(files), {"video_upload": True}),
+                     "video_segment_frames": ("INT", {"default": 10, "min": 1, "step": 1}),
+                     "transition_frames": ("INT", {"default": 0, "min": 0, "step": 1}), 
+                     },}
+
+    CATEGORY = "♾️Mixlab/Video"
+
+    RETURN_TYPES = ("SCENE_VIDEO","INT", "INT","INT",)
+    RETURN_NAMES = ("scenes_video","scenes_count","frame_count","fps",)
+    FUNCTION = "load_video"
+    OUTPUT_NODE = True
+    OUTPUT_IS_LIST = (True,False,False,False,)
+
+
+    def is_gif(self, filename):
+        file_parts = filename.split('.')
+        return len(file_parts) > 1 and file_parts[-1] == "gif"
+
+    def load_video_cv_fallback(self, video, frame_load_cap, skip_first_frames):
+        try:
+            video_cap = cv2.VideoCapture(folder_paths.get_annotated_filepath(video))
+            if not video_cap.isOpened():
+                raise ValueError(f"{video} could not be loaded with cv fallback.")
+            # set video_cap to look at start_index frame
+            images = []
+            total_frame_count = 0
+            frames_added = 0
+            base_frame_time = 1/video_cap.get(cv2.CAP_PROP_FPS)
+            
+            target_frame_time = base_frame_time
+
+            time_offset=0.0
+            while video_cap.isOpened():
+                if time_offset < target_frame_time:
+                    is_returned, frame = video_cap.read()
+                    # if didn't return frame, video has ended
+                    if not is_returned:
+                        break
+                    time_offset += base_frame_time
+                if time_offset < target_frame_time:
+                    continue
+                time_offset -= target_frame_time
+                # if not at start_index, skip doing anything with frame
+                total_frame_count += 1
+                if total_frame_count <= skip_first_frames:
+                    continue
+                # TODO: do whatever operations need to happen, like force_size, etc
+
+                # opencv loads images in BGR format (yuck), so need to convert to RGB for ComfyUI use
+                # follow up: can videos ever have an alpha channel?
+                frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
+                # convert frame to comfyui's expected format (taken from comfy's load image code)
+                image = Image.fromarray(frame)
+                image = ImageOps.exif_transpose(image)
+                image = np.array(image, dtype=np.float32) / 255.0
+                image = torch.from_numpy(image)[None,]
+                images.append(image)
+                frames_added += 1
+                # if cap exists and we've reached it, stop processing frames
+                if frame_load_cap > 0 and frames_added >= frame_load_cap:
+                    break
+        finally:
+            video_cap.release()
+        images = torch.cat(images, dim=0)
+       
+        return (images, frames_added)
+
+    def load_video(self, video,video_segment_frames,transition_frames ):
+        
+        video_path = folder_paths.get_annotated_filepath(video)
+        
+        # check if video is a gif - will need to use cv fallback to read frames
+        # use cv fallback if ffmpeg not installed or gif
+        # if ffmpeg_path is None:
+        #     return self.load_video_cv_fallback(video, frame_load_cap, skip_first_frames)
+        # otherwise, continue with ffmpeg
+        
+        # args_dummy = [ffmpeg_path, "-i", video_path, "-f", "null", "-"]
+        # try:
+        #     with subprocess.Popen(args_dummy, stdout=subprocess.DEVNULL, stderr=subprocess.PIPE) as proc:
+        #         for line in proc.stderr.readlines():
+        #             match = re.search(", ([1-9]|\\d{2,})x(\\d+)",line.decode('utf-8'))
+        #             if match is not None:
+        #                 size = [int(match.group(1)), int(match.group(2))]
+        #                 break
+        # except Exception as e:
+        #     print(f"Retrying with opencv due to ffmpeg error: {e}")
+        #     return self.load_video_cv_fallback(video, frame_load_cap, skip_first_frames)
+        # args_all_frames = [ffmpeg_path, "-i", video_path, "-v", "error",
+        #                      "-pix_fmt", "rgb24"]
+
+        # vfilters = []
+       
+        # if skip_first_frames > 0:
+        #     vfilters.append(f"select=gt(n\\,{skip_first_frames-1})")
+        # if frame_load_cap > 0:
+        #     vfilters.append(f"select=gt({frame_load_cap}\\,n)")
+        # #manually calculate aspect ratio to ensure reads remain aligned
+        
+        # if len(vfilters) > 0:
+        #     args_all_frames += ["-vf", ",".join(vfilters)]
+
+        # args_all_frames += ["-f", "rawvideo", "-"]
+        # images = []
+        # try:
+        #     with subprocess.Popen(args_all_frames, stdout=subprocess.PIPE) as proc:
+        #         #Manually buffer enough bytes for an image
+        #         bpi = size[0]*size[1]*3
+        #         current_bytes = bytearray(bpi)
+        #         current_offset=0
+        #         while True:
+        #             bytes_read = proc.stdout.read(bpi - current_offset)
+        #             if bytes_read is None:#sleep to wait for more data
+        #                 time.sleep(.2)
+        #                 continue
+        #             if len(bytes_read) == 0:#EOF
+        #                 break
+        #             current_bytes[current_offset:len(bytes_read)] = bytes_read
+        #             current_offset+=len(bytes_read)
+        #             if current_offset == bpi:
+        #                 images.append(np.array(current_bytes, dtype=np.float32).reshape(size[1], size[0], 3) / 255.0)
+        #                 current_offset = 0
+        # except Exception as e:
+        #     print(f"Retrying with opencv due to ffmpeg error: {e}")
+        #     return self.load_video_cv_fallback(video, frame_load_cap, skip_first_frames)
+
+        # imgs=split_list(images,video_segment_frames,transition_frames)
+
+        # temp path 
+        tp=folder_paths.get_temp_directory()
+        basename = os.path.basename(video_path)  # 获取文件名
+        name_without_extension = os.path.splitext(basename)[0]  # 去掉文件后缀
+
+        folder_path = create_folder(tp,name_without_extension)
+        
+ 
+        # 导出的数据
+        scenes_video,total_frames,fps=split_video(video_path,video_segment_frames,
+                                 transition_frames,folder_path)
+      
+
+        # imgs=[torch.from_numpy(np.stack(im)) for im in imgs]
+
+        # images = torch.from_numpy(np.stack(images))
+
+        return (scenes_video,len(scenes_video), total_frames,fps,)
+
+    @classmethod
+    def IS_CHANGED(s, video, **kwargs):
+        image_path = folder_paths.get_annotated_filepath(video)
+        m = hashlib.sha256()
+        with open(image_path, 'rb') as f:
+            m.update(f.read())
+        return m.digest().hex()
+
+    @classmethod
+    def VALIDATE_INPUTS(s, video, **kwargs):
+        if not folder_paths.exists_annotated_filepath(video):
+            return "Invalid image file: {}".format(video)
+
+        return True
+    
+
+# The code is based on ComfyUI-VideoHelperSuite modification.
+class VideoCombine_Adv:
+    @classmethod
+    def INPUT_TYPES(s):
+        #Hide ffmpeg formats if ffmpeg isn't available
+        if ffmpeg_path is not None:
+            ffmpeg_formats = ["video/"+x[:-5] for x in folder_paths.get_filename_list("video_formats")]
+        else:
+            ffmpeg_formats = []
+        return {
+            "required": {
+                "image_batch": ("IMAGE",),
+                "frame_rate": (
+                    "INT",
+                    {"default": 8, "min": 1, "step": 1},
+                ),
+                "loop_count": ("INT", {"default": 0, "min": 0, "max": 100, "step": 1}),
+                "filename_prefix": ("STRING", {"default": "Comfyui"}),
+                "format": (["image/gif", "image/webp"] + ffmpeg_formats,),
+                "pingpong": ("BOOLEAN", {"default": False}),
+                "save_image": ("BOOLEAN", {"default": True}),
+                "metadata": ("BOOLEAN", {"default": False}),
+            },
+            "hidden": {
+                "prompt": "PROMPT",
+                "extra_pnginfo": "EXTRA_PNGINFO",
+            },
+        }
+
+    RETURN_TYPES = ()
+    OUTPUT_NODE = True
+    CATEGORY = "♾️Mixlab/Video"
+    FUNCTION = "run"
+
+    def save_with_tempfile(self, args, metadata, file_path, frames, env):
+        #Ensure temp directory exists
+        os.makedirs(folder_paths.get_temp_directory(), exist_ok=True)
+
+        metadata_path = os.path.join(folder_paths.get_temp_directory(), "metadata.txt")
+        #metadata from file should  escape = ; # \ and newline
+        #From my testing, though, only backslashes need escapes and = in particular causes problems
+        #It is likely better to prioritize future compatibility with containers that don't support
+        #or shouldn't use the comment tag for embedding metadata
+        metadata = metadata.replace("\\","\\\\")
+        metadata = metadata.replace(";","\\;")
+        metadata = metadata.replace("#","\\#")
+        #metadata = metadata.replace("=","\\=")
+        metadata = metadata.replace("\n","\\\n")
+        with open(metadata_path, "w") as f:
+            f.write(";FFMETADATA1\n")
+            f.write(metadata)
+        args = args[:1] + ["-i", metadata_path] + args[1:] + [file_path]
+        with subprocess.Popen(args, stdin=subprocess.PIPE, env=env) as proc:
+            for frame in frames:
+                proc.stdin.write(frame.tobytes())
+
+    def run(
+        self,
+        image_batch,
+        frame_rate: int,
+        loop_count: int,
+        filename_prefix="AnimateDiff",
+        format="image/gif",
+        pingpong=False,
+        save_image=True,
+        metadata=False,
+        prompt=None,
+        extra_pnginfo=None,
+    ):
+        images=image_batch
+
+        frames: List[Image.Image] = []
+        for image in images:
+            img = 255.0 * image.cpu().numpy()
+            img = Image.fromarray(np.clip(img, 0, 255).astype(np.uint8))
+            # resize 保证
+            # 检查图像的高度是否是2的倍数，如果不是，则调整高度
+            if img.height % 2 != 0:
+                img = img.resize((img.width, img.height + 1))
+
+            # 检查图像的宽度是否是2的倍数，如果不是，则调整宽度
+            if img.width % 2 != 0:
+                img = img.resize((img.width + 1, img.height))
+                
+            frames.append(img)
+
+        # get output information
+        output_dir = (
+            folder_paths.get_output_directory()
+            if save_image
+            else folder_paths.get_temp_directory()
+        )
+        (
+            full_output_folder,
+            filename,
+            counter,
+            subfolder,
+            _,
+        ) = folder_paths.get_save_image_path(filename_prefix, output_dir)
+
+        metadata = PngInfo()
+        video_metadata = {}
+        if prompt is not None:
+            metadata.add_text("prompt", json.dumps(prompt))
+            video_metadata["prompt"] = prompt
+        if extra_pnginfo is not None:
+            for x in extra_pnginfo:
+                metadata.add_text(x, json.dumps(extra_pnginfo[x]))
+                video_metadata[x] = extra_pnginfo[x]
+
+        # 取消保存metadata
+        if metadata==False:
+            metadata = PngInfo()
+
+        # save first frame as png to keep metadata
+        file = f"{filename}_{counter:05}_.png"
+        file_path = os.path.join(full_output_folder, file)
+        frames[0].save(
+            file_path,
+            pnginfo=metadata,
+            compress_level=4,
+        )
+        if pingpong:
+            frames = frames + frames[-2:0:-1]
+
+        format_type, format_ext = format.split("/")
+        file = f"{filename}_{counter:05}_.{format_ext}"
+        file_path = os.path.join(full_output_folder, file)
+        if format_type == "image":
+            # Use pillow directly to save an animated image
+            frames[0].save(
+                file_path,
+                format=format_ext.upper(),
+                save_all=True,
+                append_images=frames[1:],
+                duration=round(1000 / frame_rate),
+                loop=loop_count,
+                compress_level=4,
+            )
+        else:
+            # Use ffmpeg to save a video
+            if ffmpeg_path is None:
+                #Should never be reachable
+                raise ProcessLookupError("Could not find ffmpeg")
+
+            video_format_path = folder_paths.get_full_path("video_formats", format_ext + ".json")
+            with open(video_format_path, 'r') as stream:
+                video_format = json.load(stream)
+            file = f"{filename}_{counter:05}_.{video_format['extension']}"
+            file_path = os.path.join(full_output_folder, file)
+            dimensions = f"{frames[0].width}x{frames[0].height}"
+            metadata_args = ["-metadata", "comment=" + json.dumps(video_metadata)]
+            args = [ffmpeg_path, "-v", "error", "-f", "rawvideo", "-pix_fmt", "rgb24",
+                    "-s", dimensions, "-r", str(frame_rate), "-i", "-"] \
+                    + video_format['main_pass']
+            # On linux, max arg length is Pagesize * 32 -> 131072
+            # On windows, this around 32767 but seems to vary wildly by > 500
+            # in a manor not solely related to other arguments
+            if os.name == 'posix':
+                max_arg_length = 4096*32
+            else:
+                max_arg_length = 32767 - len(" ".join(args + [metadata_args[0]] + [file_path])) - 1
+            #test max limit
+            #metadata_args[1] = metadata_args[1] + "a"*(max_arg_length - len(metadata_args[1])-1)
+
+            env=os.environ.copy()
+            if  "environment" in video_format:
+                env.update(video_format["environment"])
+            if len(metadata_args[1]) >= max_arg_length:
+                print(f"Using fallback file for extremely long metadata: {len(metadata_args[1])}/{max_arg_length}")
+                self.save_with_tempfile(args, metadata_args[1], file_path, frames, env)
+            else:
+                try:
+                    with subprocess.Popen(args + metadata_args + [file_path],
+                                          stdin=subprocess.PIPE, env=env) as proc:
+                        for frame in frames:
+                            proc.stdin.write(frame.tobytes())
+                except FileNotFoundError as e:
+                    if "winerror" in dir(e) and e.winerror == 206:
+                        print("Metadata was too long. Retrying with fallback file")
+                        self.save_with_tempfile(args, metadata_args[1], file_path, frames, env)
+                    else:
+                        raise
+                except OSError as e:
+                    if "errno" in dir(e) and e.errno == 7:
+                        print("Metadata was too long. Retrying with fallback file")
+                        self.save_with_tempfile(args, metadata_args[1], file_path, frames, env)
+                    else:
+                        raise
+
+        previews = [
+            {
+                "filename": file,
+                "subfolder": subfolder,
+                "type": "output" if save_image else "temp",
+                "format": format,
+            }
+        ]
+        return {"ui": {"gifs": previews}}
+
+
+class VAEEncodeForInpaint_Frames:
+    @classmethod
+    def INPUT_TYPES(s):
+        return {"required": { 
+            "vae": ("VAE", ), 
+            "images": ("IMAGE", ), 
+            "masks": ("MASK", ), 
+            "grow_mask_by": ("INT", {"default": 6, "min": 0, "max": 64, "step": 1}),
+            }}
+    
+    FUNCTION = "encode"
+
+
+    RETURN_TYPES = ("LATENT",)
+    RETURN_NAMES = ("LATENT",)
+ 
+    CATEGORY = "♾️Mixlab/Video"
+
+    OUTPUT_NODE = True
+    INPUT_IS_LIST = True
+    OUTPUT_IS_LIST = (True,)
+
+
+    def encode(self, vae, images, masks, grow_mask_by=[6]):
+        vae=vae[0]
+        grow_mask_by=grow_mask_by[0]
+
+        result=[]
+
+        for i in range(len(images)):
+            pixels=images[i]
+            mask=masks[i]
+
+
+            x = (pixels.shape[1] // 8) * 8
+            y = (pixels.shape[2] // 8) * 8
+            mask = torch.nn.functional.interpolate(mask.reshape((-1, 1, mask.shape[-2], mask.shape[-1])), size=(pixels.shape[1], pixels.shape[2]), mode="bilinear")
+
+            pixels = pixels.clone()
+            if pixels.shape[1] != x or pixels.shape[2] != y:
+                x_offset = (pixels.shape[1] % 8) // 2
+                y_offset = (pixels.shape[2] % 8) // 2
+                pixels = pixels[:,x_offset:x + x_offset, y_offset:y + y_offset,:]
+                mask = mask[:,:,x_offset:x + x_offset, y_offset:y + y_offset]
+
+            #grow mask by a few pixels to keep things seamless in latent space
+            if grow_mask_by == 0:
+                mask_erosion = mask
+            else:
+                kernel_tensor = torch.ones((1, 1, grow_mask_by, grow_mask_by))
+                padding = math.ceil((grow_mask_by - 1) / 2)
+
+                mask_erosion = torch.clamp(torch.nn.functional.conv2d(mask.round(), kernel_tensor, padding=padding), 0, 1)
+
+            m = (1.0 - mask.round()).squeeze(1)
+            for i in range(3):
+                pixels[:,:,:,i] -= 0.5
+                pixels[:,:,:,i] *= m
+                pixels[:,:,:,i] += 0.5
+            t = vae.encode(pixels)
+
+            result.append({"samples":t, "noise_mask": (mask_erosion[:,:,:x,:y].round())})
+
+
+        return (result, )

nodes/VisualStylePrompting/attention_functions.py

@@ -0,0 +1,45 @@
+from comfy.ldm.modules.attention import default, optimized_attention, optimized_attention_masked
+from .style_functions import adain, concat_first
+
+class VisualStyleProcessor(object):
+    def __init__(self, 
+        module_self, 
+        keys_scale: float = 1.0,
+        enabled: bool = True, 
+        adain_queries: bool = True,
+        adain_keys: bool = True,
+        adain_values: bool = False 
+    ):
+        self.module_self = module_self
+        self.keys_scale = keys_scale
+        self.enabled = enabled
+        self.adain_queries = adain_queries
+        self.adain_keys = adain_keys
+        self.adain_values = adain_values
+
+    def visual_style_forward(self, x, context, value, mask=None):
+        q = self.module_self.to_q(x)
+        context = default(context, x)
+        k = self.module_self.to_k(context)
+        if value is not None:
+            v = self.module_self.to_v(value)
+            del value
+        else:
+            v = self.module_self.to_v(context)
+
+        if self.enabled:
+            if self.adain_queries:
+                q = adain(q)
+            if self.adain_keys:
+                k = adain(k)
+            if self.adain_values:
+                v = adain(v)
+            
+            k = concat_first(k, -2, self.keys_scale)
+            v = concat_first(v, -2)
+
+        if mask is None:
+            out = optimized_attention(q, k, v, self.module_self.heads)
+        else:
+            out = optimized_attention_masked(q, k, v, self.module_self.heads, mask)
+        return self.module_self.to_out(out)

nodes/VisualStylePrompting/style_functions.py

@@ -0,0 +1,60 @@
+import torch
+
+from einops import rearrange
+from dataclasses import dataclass
+
+T = torch.Tensor
+
+@dataclass(frozen=True)
+class StyleAlignedArgs:
+    share_group_norm: bool = True
+    share_layer_norm: bool = True,
+    share_attention: bool = True
+    adain_queries: bool = True
+    adain_keys: bool = True
+    adain_values: bool = False
+    full_attention_share: bool = False
+    keys_scale: float = 1.
+    only_self_level: float = 0.
+
+def expand_first(feat: T, scale=1., ) -> T:
+    b = feat.shape[0]
+    feat_style = torch.stack((feat[0], feat[b // 2])).unsqueeze(1)
+    if scale == 1:
+        feat_style = feat_style.expand(2, b // 2, *feat.shape[1:])
+    else:
+        feat_style = feat_style.repeat(1, b // 2, 1, 1, 1)
+        feat_style = torch.cat([feat_style[:, :1], scale * feat_style[:, 1:]], dim=1)
+    return feat_style.reshape(*feat.shape)
+
+
+def concat_first(feat: T, dim=2, scale=1.) -> T:
+    feat_style = expand_first(feat, scale=scale)
+    return torch.cat((feat, feat_style), dim=dim)
+
+
+def calc_mean_std(feat, eps: float = 1e-5) -> tuple[T, T]:
+    feat_std = (feat.var(dim=-2, keepdims=True) + eps).sqrt()
+    feat_mean = feat.mean(dim=-2, keepdims=True)
+    return feat_mean, feat_std
+
+
+def adain(feat: T) -> T:
+    feat_mean, feat_std = calc_mean_std(feat)
+    feat_style_mean = expand_first(feat_mean)
+    feat_style_std = expand_first(feat_std)
+    feat = (feat - feat_mean) / feat_std
+    feat = feat * feat_style_std + feat_style_mean
+    return feat
+
+def swapping_attention(key, value, chunk_size=2):
+    chunk_length = key.size()[0] // chunk_size  # [text-condition, null-condition]
+    reference_image_index = [0] * chunk_length  # [0 0 0 0 0]
+    key = rearrange(key, "(b f) d c -> b f d c", f=chunk_length)
+    key = key[:, reference_image_index]  # ref to all
+    key = rearrange(key, "b f d c -> (b f) d c")
+    value = rearrange(value, "(b f) d c -> b f d c", f=chunk_length)
+    value = value[:, reference_image_index]  # ref to all
+    value = rearrange(value, "b f d c -> (b f) d c")
+
+    return key, value

nodes/test.py

@@ -0,0 +1,8 @@
+import folder_paths
+
+# 外挂一个文件，用来编写新的节点
+def run(v):
+
+    output_dir = folder_paths.get_temp_directory()
+
+    print('1323',v,output_dir)

requirements.txt

@@ -6,4 +6,7 @@ matplotlib
 openai
 simple-lama-inpainting
 clip-interrogator==0.6.0
-transformers>=4.36.0
+transformers>=4.36.0
+lark-parser
+imageio-ffmpeg
+rembg[gpu]

web/javascript/app_mixlab.js

@@ -2,6 +2,46 @@ import { app } from '../../../scripts/app.js'
 import { $el } from '../../../scripts/ui.js'
 import { api } from '../../../scripts/api.js'
 
+//本机安装的插件节点全集
+window._nodesAll = null
+
+//获取当前系统的插件，节点清单
+function getObjectInfo () {
+  return new Promise(async (resolve, reject) => {
+    let url = getUrl()
+
+    try {
+      const response = await fetch(`${url}/object_info`)
+      const data = await response.json()
+      resolve(data)
+    } catch (error) {
+      reject(error)
+    }
+  })
+}
+
+const base64Df =
+  'data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAwAAAAMCAYAAABWdVznAAAAAXNSR0IArs4c6QAAALZJREFUKFOFkLERwjAQBPdbgBkInECGaMLUQDsE0AkRVRAYWqAByxldPPOWHwnw4OBGye1p50UDSoA+W2ABLPN7i+C5dyC6R/uiAUXRQCs0bXoNIu4QPQzAxDKxHoALOrZcqtiyR/T6CXw7+3IGHhkYcy6BOR2izwT8LptG8rbMiCRAUb+CQ6WzQVb0SNOi5Z2/nX35DRyb/ENazhpWKoGwrpD6nICp5c2qogc4of+c7QcrhgF4Aa/aoAFHiL+RAAAAAElFTkSuQmCC'
+
+const parseImageToBase64 = url => {
+  return new Promise((res, rej) => {
+    fetch(url)
+      .then(response => response.blob())
+      .then(blob => {
+        const reader = new FileReader()
+        reader.onloadend = () => {
+          const base64data = reader.result
+          res(base64data)
+          // 在这里可以将base64数据用于进一步处理或显示图片
+        }
+        reader.readAsDataURL(blob)
+      })
+      .catch(error => {
+        console.log('发生错误:', error)
+      })
+  })
+}
+
 function get_position_style (ctx, widget_width, y, node_height) {
   const MARGIN = 12 // the margin around the html element
 
@@ -28,20 +68,20 @@ function get_position_style (ctx, widget_width, y, node_height) {
     // height: `${node_height * 0.3 - MARGIN * 2}px`,
     // background: '#EEEEEE',
     display: 'flex',
-    flexDirection: 'row',
+    flexDirection: 'column',
     // alignItems: 'center',
     justifyContent: 'flex-start'
   }
 }
 
-async function drawImageToCanvas (imageUrl) {
+async function drawImageToCanvas (imageUrl, sFactor = 320) {
   var canvas = document.createElement('canvas')
   var ctx = canvas.getContext('2d')
   var img = new Image()
 
   await new Promise((resolve, reject) => {
     img.onload = function () {
-      var scaleFactor = 320 / img.width
+      var scaleFactor = sFactor / img.width
       var canvasWidth = img.width * scaleFactor
       var canvasHeight = img.height * scaleFactor
 
@@ -66,7 +106,11 @@ async function drawImageToCanvas (imageUrl) {
   // 可以在这里执行其他操作，比如将Base64数据保存到服务器或显示在页面上
 }
 
-function extractInputAndOutputData (jsonData, inputIds = [], outputIds = []) {
+async function extractInputAndOutputData (
+  jsonData,
+  inputIds = [],
+  outputIds = []
+) {
   // workflow
   // const workflow=jsonData.workflow;
   // const nodes=workflow.nodes;
@@ -75,6 +119,7 @@ function extractInputAndOutputData (jsonData, inputIds = [], outputIds = []) {
   let input = []
   let output = []
   const seed = {}
+  const seedTitle = {}
 
   for (const id in data) {
     if (data.hasOwnProperty(id)) {
@@ -114,16 +159,43 @@ function extractInputAndOutputData (jsonData, inputIds = [], outputIds = []) {
           }
         }
 
+        if (node.type == 'ImagesPrompt_') {
+          //图库
+          // console.log('ImagesPrompt_', data[id])
+          let image_base64 = data[id].inputs.image_base64
+          let img_index = 0
+          let imgsData = JSON.parse(data[id].inputs.upload)
+          for (let index = 0; index < imgsData.length; index++) {
+            const imgd = imgsData[index].imgurl
+            imgsData[index].index = index
+            //TODO缩放大小
+            imgsData[index].imgurl = await parseImageToBase64(imgd)
+            if (image_base64 == imgsData[index].imgurl) {
+              img_index = index
+            }
+          }
+          options.images = imgsData
+          delete data[id].inputs.upload
+          delete data[id].inputs.image_base64
+
+          data[id].inputs.imageIndex = img_index
+        }
+
         if (node.type == 'Color') {
         }
 
-        // loadImage的mask支持
         if (node.type === 'LoadImage') {
+          // loadImage的mask支持
           let output = node.outputs.filter(ot => ot.type == 'MASK')[0]
           if (output.links) {
             // 有输出
             options.hasMask = true
           }
+          // loadImage的默认图，转为base64
+          let imgurl = app.graph.getNodeById(id).imgs[0].src
+
+          options.defaultImage = await drawImageToCanvas(imgurl, 512)
+          console.log('#loadImage的默认图', options)
         }
 
         input[inputIds.indexOf(id)] = {
@@ -140,12 +212,18 @@ function extractInputAndOutputData (jsonData, inputIds = [], outputIds = []) {
         output[outputIds.indexOf(id)] = { ...data[id], title: node.title, id }
       }
 
-      if (node.type === 'KSampler' || node.type == 'SamplerCustom') {
+      if (
+        node.type === 'KSampler' ||
+        node.type == 'SamplerCustom' ||
+        node.type === 'ChinesePrompt_Mix' ||
+        node.type === 'Seed_'
+      ) {
         // seed 的类型收集
         try {
           seed[id] = node.widgets.filter(
             w => w.name === 'seed' || w.name == 'noise_seed'
           )[0].linkedWidgets[0].value
+          seedTitle[id] = node.title
         } catch (error) {}
       }
     }
@@ -155,7 +233,7 @@ function extractInputAndOutputData (jsonData, inputIds = [], outputIds = []) {
   input = input.filter(i => i)
   output = output.filter(i => i)
 
-  return { input, output, seed }
+  return { input, output, seed, seedTitle }
 }
 
 function getUrl () {
@@ -207,7 +285,10 @@ function downloadJsonFile (jsonData, fileName = 'mix_app.json') {
 }
 
 async function save (json, download = false, showInfo = true) {
-  console.log('####SAVE', json[0])
+  let nodesAll = window._nodesAll || (await getObjectInfo())
+
+  console.log('####SAVE', nodesAll, json[0])
+
   const name = json[0],
     version = json[5],
     share_prefix = json[6], //用于分享的功能扩展
@@ -228,12 +309,26 @@ async function save (json, download = false, showInfo = true) {
   try {
     let data = await app.graphToPrompt()
 
-    let { input, output, seed } = extractInputAndOutputData(
+    //从output数据里把工作流的节点，插件数据统计出来
+    data.nodesMap = {}
+    for (const id in data.output) {
+      data.nodesMap[data.output[id].class_type] =
+        nodesAll[data.output[id].class_type]
+    }
+
+    let { input, output, seed, seedTitle } = await extractInputAndOutputData(
       data,
       inputIds,
       outputIds
     )
 
+    let authorAvatar =
+        localStorage.getItem('_mixlab_author_avatar') || base64Df,
+      authorName =
+        localStorage.getItem('_mixlab_author_name') ||
+        localStorage.getItem('Comfy.userName'),
+      authorLink = localStorage.getItem('_mixlab_author_link') || ''
+
     data.app = {
       name,
       description,
@@ -241,10 +336,16 @@ async function save (json, download = false, showInfo = true) {
       input,
       output,
       seed, //控制是fixed 还是random
+      seedTitle,
       share_prefix,
       link,
       category,
-      filename: `${name}_${version}.json`
+      filename: `${name}_${version}.json`,
+      author: {
+        avatar: authorAvatar,
+        name: authorName,
+        link: authorLink
+      }
     }
 
     try {
@@ -277,12 +378,13 @@ async function save (json, download = false, showInfo = true) {
 
 function getInputsAndOutputs () {
   const inputs =
-      `LoadImage CLIPTextEncode PromptSlide TextInput_ Color FloatSlider IntNumber CheckpointLoaderSimple LoraLoader`.split(
+      `LoadImage LoadImagesToBatch ImagesPrompt_ VHS_LoadVideo CLIPTextEncode PromptSlide TextInput_ Color FloatSlider IntNumber CheckpointLoaderSimple LoraLoader`.split(
         ' '
       ),
-    outputs = `PreviewImage SaveImage ShowTextForGPT VHS_VideoCombine`.split(
-      ' '
-    )
+    outputs =
+      `SaveTripoSRMesh,PreviewImage,SaveImage,TransparentImage,ShowTextForGPT,VHS_VideoCombine,VideoCombine_Adv,Image Save,SaveImageAndMetadata_,ClipInterrogator`.split(
+        ','
+      )
 
   let inputsId = [],
     outputsId = []
@@ -305,6 +407,11 @@ function getInputsAndOutputs () {
 
 app.registerExtension({
   name: 'Mixlab.utils.AppInfo',
+  init () {
+    if (!window._nodesAll) {
+      getObjectInfo().then(r => (window._nodesAll = r))
+    }
+  },
   async beforeRegisterNodeDef (nodeType, nodeData, app) {
     if (nodeType.comfyClass == 'AppInfo') {
       const orig_nodeCreated = nodeType.prototype.onNodeCreated
@@ -377,10 +484,164 @@ app.registerExtension({
           }
         })
 
-        document.body.appendChild(widget.div)
-        widget.div.appendChild(btn)
-        widget.div.appendChild(download)
+        // author
+        let author = document.createElement('div')
+        // author.style=`display: flex`
 
+        let authorAvatar = document.createElement('img')
+        authorAvatar.className = `${'comfy-multiline-input'}`
+        authorAvatar.style = `outline: none;
+          border: none;
+          padding: 4px;
+          width: 32px;
+          cursor: pointer;
+          height: 32px;`
+
+        if (localStorage.getItem('_mixlab_author_avatar')) {
+          authorAvatar.src =
+            localStorage.getItem('_mixlab_author_avatar') || base64Df
+        }
+
+        let authorAvatarUpload = document.createElement('input')
+        authorAvatarUpload.type = 'file'
+        authorAvatarUpload.style = `display:none`
+
+        let authorAvatarInput = document.createElement('div')
+        authorAvatarInput.style = `display: flex;justify-content: flex-start;
+        align-items: center;`
+        let authorAvatarInputLabel = document.createElement('p')
+        authorAvatarInputLabel.innerText = 'Author Avatar'
+        authorAvatarInputLabel.className = `${'comfy-multiline-input'}`
+        authorAvatarInputLabel.style = `font-size:12px`
+
+        authorAvatar.addEventListener('click', e => {
+          authorAvatarUpload.click()
+        })
+
+        authorAvatarInputLabel.addEventListener('click', e => {
+          authorAvatarUpload.click()
+        })
+
+        authorAvatarUpload.addEventListener('change', event => {
+          const file = event.target.files[0]
+          const reader = new FileReader()
+
+          reader.onload = async e => {
+            let im = new Image()
+            im.src = e.target.result
+            authorAvatar.src = e.target.result
+            im.onload = () => {
+              let c = document.createElement('canvas')
+              let ctx = c.getContext('2d')
+              c.width = 72
+              c.height = 72
+              ctx.drawImage(
+                im,
+                0,
+                0,
+                im.naturalWidth,
+                im.naturalHeight,
+                0,
+                0,
+                c.width,
+                c.height
+              )
+              window._mixlab_author_avatar = c.toDataURL()
+              localStorage.setItem(
+                '_mixlab_author_avatar',
+                window._mixlab_author_avatar
+              )
+            }
+          }
+
+          // 以文本形式读取文件
+          reader.readAsDataURL(file)
+        })
+
+        author.appendChild(authorAvatarInput)
+        authorAvatarInput.appendChild(authorAvatarInputLabel)
+        authorAvatarInput.appendChild(authorAvatar)
+        authorAvatarInput.appendChild(authorAvatarUpload)
+
+        let authorName = document.createElement('input')
+        authorName.type = 'text'
+        authorName.value =
+          localStorage.getItem('_mixlab_author_name') ||
+          localStorage.getItem('Comfy.userName')
+        authorName.placeholder = 'author name'
+        authorName.className = `${'comfy-multiline-input'}`
+        authorName.style = `
+          outline: none;
+          border: none;
+          padding: 4px;
+          width: 100%;
+          cursor: pointer;
+          height: 32px;`
+
+        let authorNameInput = document.createElement('div')
+        authorNameInput.style = `display: flex;justify-content: flex-start;
+          align-items: center;`
+        let authorNameInputLabel = document.createElement('p')
+        authorNameInputLabel.innerText = 'Author Name'
+        authorNameInputLabel.className = `${'comfy-multiline-input'}`
+        authorNameInputLabel.style = `font-size:12px;width: 110px`
+
+        authorName.addEventListener('change', e => {
+          window._mixlab_author_name = authorName.value.trim()
+          localStorage.setItem(
+            '_mixlab_author_name',
+            window._mixlab_author_name
+          )
+        })
+
+        author.appendChild(authorNameInput)
+        authorNameInput.appendChild(authorNameInputLabel)
+        authorNameInput.appendChild(authorName)
+
+        // 社交链接
+        let authorLink = document.createElement('input')
+        authorLink.type = 'text'
+        authorLink.value = localStorage.getItem('_mixlab_author_link') || ''
+        authorLink.placeholder = 'author link'
+        authorLink.className = `${'comfy-multiline-input'}`
+        authorLink.style = `
+          outline: none;
+          border: none;
+          padding: 4px;
+          width: 100%;
+          cursor: pointer;
+          height: 32px;`
+
+        let authorLinkInput = document.createElement('div')
+        authorLinkInput.style = `display: flex;justify-content: flex-start;
+          align-items: center;`
+        let authorLinkInputLabel = document.createElement('p')
+        authorLinkInputLabel.innerText = 'Author Link'
+        authorLinkInputLabel.className = `${'comfy-multiline-input'}`
+        authorLinkInputLabel.style = `font-size:12px;width: 110px`
+
+        authorLink.addEventListener('change', e => {
+          window._mixlab_author_link = authorLink.value.trim()
+          localStorage.setItem(
+            '_mixlab_author_link',
+            window._mixlab_author_link
+          )
+        })
+
+        author.appendChild(authorLinkInput)
+        authorLinkInput.appendChild(authorLinkInputLabel)
+        authorLinkInput.appendChild(authorLink)
+
+        widget.div.appendChild(author)
+
+        let btns = document.createElement('div')
+
+        widget.div.appendChild(btns)
+
+        btns.appendChild(btn)
+        btns.appendChild(download)
+
+        document.body.appendChild(widget.div)
         this.addCustomWidget(widget)
 
         const onRemoved = this.onRemoved

web/javascript/checkVersion_mixlab.js

@@ -3,7 +3,7 @@ import { app } from '../../../scripts/app.js'
 const repoOwner = 'shadowcz007' // 替换为仓库的所有者
 const repoName = 'comfyui-mixlab-nodes' // 替换为仓库的名称
 
-const version = 'v0.16.0'
+const version = 'v0.22.0'
 
 fetch(`https://api.github.com/repos/${repoOwner}/${repoName}/releases/latest`)
   .then(response => response.json())

web/javascript/gpt_mixlab.js

@@ -61,7 +61,7 @@ app.registerExtension({
   async getCustomWidgets (app) {
     return {
       KEY (node, inputName, inputData, app) {
-        console.log('##inputData', inputData)
+        // console.log('##inputData', inputData)
         const widget = {
           type: inputData[0], // the type, CHEESE
           name: inputName, // the name, slice

web/javascript/image_mixlab.js

@@ -1,7 +1,39 @@
 import { app } from '../../../scripts/app.js'
 import { api } from '../../../scripts/api.js'
-import { ComfyWidgets } from '../../../scripts/widgets.js'
+// import { ComfyWidgets } from '../../../scripts/widgets.js'
 import { $el } from '../../../scripts/ui.js'
+import { applyTextReplacements } from '../../../scripts/utils.js'
+
+function loadImageToCanvas (base64Image) {
+  var img = new Image()
+  var canvas = document.createElement('canvas')
+  var ctx = canvas.getContext('2d')
+  return new Promise((res, rej) => {
+    img.onload = function () {
+      // 等比例缩放图片
+      var width = img.width
+      var height = img.height
+      var max_width = 1024
+      if (width > max_width) {
+        height *= max_width / width
+        width = max_width
+      }
+
+      // 设置canvas尺寸
+      canvas.width = width
+      canvas.height = height
+
+      // 在canvas上绘制图片
+      ctx.drawImage(img, 0, 0, width, height)
+
+      // 将canvas转换为base64图片数据
+      var canvasData = canvas.toDataURL()
+      res(canvasData) //  canvas转换后的base64图片数据
+    }
+
+    img.src = base64Image
+  })
+}
 
 async function uploadImage (blob, fileType = '.svg', filename) {
   // const blob = await (await fetch(src)).blob();
@@ -28,6 +60,9 @@ async function uploadImage (blob, fileType = '.svg', filename) {
   return src
 }
 
+const base64Df =
+  'data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAwAAAAMCAYAAABWdVznAAAAAXNSR0IArs4c6QAAALZJREFUKFOFkLERwjAQBPdbgBkInECGaMLUQDsE0AkRVRAYWqAByxldPPOWHwnw4OBGye1p50UDSoA+W2ABLPN7i+C5dyC6R/uiAUXRQCs0bXoNIu4QPQzAxDKxHoALOrZcqtiyR/T6CXw7+3IGHhkYcy6BOR2izwT8LptG8rbMiCRAUb+CQ6WzQVb0SNOi5Z2/nX35DRyb/ENazhpWKoGwrpD6nICp5c2qogc4of+c7QcrhgF4Aa/aoAFHiL+RAAAAAElFTkSuQmCC'
+
 function base64ToBlobFromURL (base64URL, contentType) {
   return fetch(base64URL).then(response => response.blob())
 }
@@ -108,7 +143,7 @@ function createImage (url) {
   })
 }
 
-const parseImage = url => {
+const parseImageToBase64 = url => {
   return new Promise((res, rej) => {
     fetch(url)
       .then(response => response.blob())
@@ -406,9 +441,7 @@ app.registerExtension({
 
         this.serialize_widgets = true //需要保存参数
       }
-    };
-
- 
+    }
   },
   async loadedGraphNode (node, app) {
     // Fires every time a node is constructed
@@ -442,3 +475,369 @@ app.registerExtension({
     }
   }
 })
+
+const createSelect = (imgDiv, select, opts, targetWidget, textWidget) => {
+  select.style.display = 'block'
+  let html = ''
+  let isMatch = false
+  for (const opt of opts) {
+    html += `<option value='${opt.keyword}' ${opt.selected ? 'selected' : ''}>${
+      opt.keyword
+    }</option>`
+    if (opt.selected) {
+      isMatch = true
+      imgDiv.src = opt.imgurl
+      // targetWidget.value = opt.keyword
+    }
+  }
+  select.innerHTML = html
+  if (!isMatch) {
+    // targetWidget.value = opts[0].keyword
+    imgDiv.src = opts[0].imgurl
+  }
+
+  // 添加change事件监听器
+  select.addEventListener('change', async function () {
+    // 获取选中的选项的值
+    var selectedOption = select.options[select.selectedIndex].value
+    let t = opts.filter(opt => opt.keyword === selectedOption)[0]
+
+    targetWidget.value = await parseImageToBase64(t.imgurl)
+    imgDiv.src = targetWidget.value
+    textWidget.value = t.keyword
+  })
+  // console.log(select)
+}
+
+app.registerExtension({
+  name: 'Mixlab.prompt.ImagesPrompt_',
+  async beforeRegisterNodeDef (nodeType, nodeData, app) {
+    if (nodeType.comfyClass == 'ImagesPrompt_') {
+      const orig_nodeCreated = nodeType.prototype.onNodeCreated
+      nodeType.prototype.onNodeCreated = async function () {
+        orig_nodeCreated?.apply(this, arguments)
+
+        const image_prompt = this.widgets.filter(
+          w => w.name == 'image_base64'
+        )[0]
+        const image_text = this.widgets.filter(w => w.name == 'text')[0]
+
+        const node = this
+
+        const widget = {
+          type: 'div',
+          name: 'upload',
+          draw (ctx, node, widget_width, y, widget_height) {
+            Object.assign(
+              this.div.style,
+              get_position_style(ctx, widget_width, y, node.size[1])
+            )
+          }
+        }
+
+        widget.div = $el('div', {})
+
+        // console.log('image_prompt',image_prompt)
+        const img = new Image()
+        img.src = image_prompt?.value || base64Df
+        widget.div.appendChild(img)
+
+        const btn = document.createElement('button')
+        btn.innerText = 'Upload Images JSON'
+
+        btn.style = `cursor: pointer;
+        font-weight: 300;
+        margin: 2px; 
+        color: var(--descrip-text);
+        background-color: var(--comfy-input-bg);
+        border-radius: 8px;
+        border-color: var(--border-color);
+        border-style: solid;height: 30px;min-width: 122px;
+       `
+
+        const select = document.createElement('select')
+        select.style = `display:none;cursor: pointer;
+        font-weight: 300;
+        margin: 2px; 
+        color: var(--descrip-text);
+        background-color: var(--comfy-input-bg);
+        border-radius: 8px;
+        border-color: var(--border-color);
+        border-style: solid;height: 30px;min-width: 100px;
+       `
+        widget.select = select
+
+        // const btn=document.createElement('button');
+        // btn.innerText='Upload'
+        btn.addEventListener('click', () => {
+          let inp = document.createElement('input')
+          inp.type = 'file'
+          inp.accept = '.json'
+          inp.click()
+          inp.addEventListener('change', event => {
+            // 获取选择的文件
+            // [{title,imageUrl}]
+            const file = event.target.files[0]
+            this.title = file.name.split('.')[0]
+
+            // console.log(file.name.split('.')[0])
+            // 创建文件读取器
+            const reader = new FileReader()
+
+            // 定义读取完成事件的回调函数
+            reader.onload = async event => {
+              // 读取完成后的文本内容
+              const json = JSON.parse(event.target.result)
+              console.log(node, json)
+
+              widget.value = JSON.stringify(json)
+
+              let img = widget.div.querySelector('img')
+
+              createSelect(img, select, json, image_prompt, image_text)
+
+              image_prompt.value = await parseImageToBase64(json[0].imgurl)
+              image_text.value = json[0].keyword
+
+              if (img) {
+                img.src = image_prompt.value
+              }
+
+              inp.remove()
+            }
+
+            // 以文本方式读取文件
+            reader.readAsText(file)
+          })
+        })
+
+        widget.div.appendChild(btn)
+        widget.div.appendChild(select)
+        document.body.appendChild(widget.div)
+        this.addCustomWidget(widget)
+
+        const onRemoved = this.onRemoved
+        this.onRemoved = () => {
+          widget.div.remove()
+          return onRemoved?.()
+        }
+
+        if (this.onResize) {
+          this.onResize(this.size)
+        }
+
+        this.serialize_widgets = true //需要保存参数
+      }
+    }
+  },
+  async loadedGraphNode (node, app) {
+    if (node.type === 'ImagesPrompt_') {
+      try {
+        let prompt = node.widgets.filter(w => w.name === 'image_base64')[0]
+        let text = node.widgets.filter(w => w.name === 'text')[0]
+        let uploadWidget = node.widgets.filter(w => w.name == 'upload')[0]
+        // console.log('##prompt',prompt.value)
+        let img = uploadWidget.div.querySelector('img')
+        let json = JSON.parse(uploadWidget.value)
+
+        for (let index = 0; index < json.length; index++) {
+          const j = json[index]
+          let base64 = await parseImageToBase64(j.imgurl)
+          if (base64 === prompt.value) {
+            json[index].selected = true
+          }
+        }
+
+        if (json && json[0]) {
+          uploadWidget.select.style.display = 'block'
+          createSelect(img, uploadWidget.select, json, prompt, text)
+        }
+      } catch (error) {}
+    }
+  }
+})
+
+const createInputImageForBatch = (base64, widget) => {
+  let im = new Image()
+  im.src = base64
+  im.style = `width: 88px;`
+
+  im.addEventListener('click', e => {
+    let newValue = []
+    let items = widget.value?.base64 || []
+    for (const v of items) {
+      if (v != base64) newValue.push(v)
+    }
+    widget.value.base64 = newValue
+    im.remove()
+  })
+
+  return im
+}
+
+app.registerExtension({
+  name: 'Mixlab.Comfy.LoadImagesToBatch',
+  async getCustomWidgets (app) {
+    return {
+      IMAGEBASE64 (node, inputName, inputData, app) {
+        // console.log('##node', node)
+        const widget = {
+          value: {
+            base64: []
+          }, // 不能[x,x,x]
+          type: inputData[0], // the type
+          name: inputName, // the name, slice
+          size: [128, 32], // a default size
+          draw (ctx, node, width, y) {},
+          computeSize (...args) {
+            return [128, 32] // a method to compute the current size of the widget
+          }
+          // serializeValue (nodeId, widgetIndex) {
+          //   return widget.value
+          // },
+        }
+        //  widget.something = something;          // maybe adds stuff to it
+        node.addCustomWidget(widget) // adds it to the node
+        return widget // and returns it.
+      }
+    }
+  },
+
+  async beforeRegisterNodeDef (nodeType, nodeData, app) {
+    if (nodeType.comfyClass == 'LoadImagesToBatch') {
+      const orig_nodeCreated = nodeType.prototype.onNodeCreated
+      nodeType.prototype.onNodeCreated = function () {
+        orig_nodeCreated?.apply(this, arguments)
+
+        let imagesWidget = this.widgets.filter(w => w.name == 'images')[0]
+
+        const widget = {
+          type: 'div',
+          name: 'image_base64',
+          draw (ctx, node, widget_width, y, widget_height) {
+            Object.assign(
+              this.div.style,
+              get_position_style(ctx, widget_width, 44, node.size[1])
+            )
+          },
+          serialize: false
+        }
+
+        widget.div = $el('div', {})
+
+        document.body.appendChild(widget.div)
+
+        let imagePreview = document.createElement('div')
+        let imagesDiv = document.createElement('div') //显示图片
+        imagesDiv.className = 'images_preview'
+        imagesDiv.style = `width: calc(100% - 14px);
+        display: flex;
+        flex-wrap: wrap;
+        padding: 7px;    justify-content: space-between;
+        align-items: center;`
+
+        let inputImage = document.createElement('input')
+        inputImage.type = 'file'
+        inputImage.style.display = 'none'
+        inputImage.addEventListener('change', e => {
+          e.preventDefault()
+          const file = e.target.files[0]
+          const reader = new FileReader()
+          reader.onload = async event => {
+            let base64 = event.target.result
+            //压缩图片，控制1024以内
+            base64 = await loadImageToCanvas(base64)
+            // console.log(base64)
+            if (!imagesWidget.value) imagesWidget.value = { base64: [] }
+            imagesWidget.value.base64.push(base64)
+            let im = createInputImageForBatch(base64, imagesWidget)
+            imagesDiv.appendChild(im)
+          }
+          reader.readAsDataURL(file)
+        })
+
+        const btn = document.createElement('button')
+        btn.innerText = 'Upload Image'
+
+        btn.style = `cursor: pointer;
+        font-weight: 300;
+        margin: 2px; 
+        color: var(--descrip-text);
+        background-color: var(--comfy-input-bg);
+        border-radius: 8px;
+        border-color: var(--border-color);
+        border-style: solid;height: 30px;min-width: 122px;
+       `
+
+        btn.addEventListener('click', e => {
+          e.preventDefault()
+          inputImage.click()
+        })
+
+        widget.div.appendChild(imagePreview)
+        imagePreview.appendChild(imagesDiv)
+        imagePreview.appendChild(btn)
+        imagePreview.appendChild(inputImage)
+
+        this.addCustomWidget(widget)
+
+        // document.addEventListener('wheel', handleMouseWheel)
+
+        const onRemoved = this.onRemoved
+        this.onRemoved = () => {
+          inputImage.remove()
+          widget.div.remove()
+          try {
+            // document.removeEventListener('wheel', handleMouseWheel)
+          } catch (error) {
+            console.log(error)
+          }
+
+          return onRemoved?.()
+        }
+
+        this.serialize_widgets = true //需要保存参数
+      }
+    }
+
+    if (nodeData.name === 'SaveImageAndMetadata_') {
+      const onNodeCreated = nodeType.prototype.onNodeCreated
+      // /web/extensions/core/saveImageExtraOutput.js
+      nodeType.prototype.onNodeCreated = function () {
+        const r = onNodeCreated
+          ? onNodeCreated.apply(this, arguments)
+          : undefined
+        const widget = this.widgets.find(w => w.name === 'filename_prefix')
+        widget.serializeValue = () => {
+          return applyTextReplacements(app, widget.value)
+        }
+
+        return r
+      }
+
+      const onExecuted = nodeType.prototype.onExecuted
+      nodeType.prototype.onExecuted = function (message) {
+        onExecuted?.apply(this, arguments)
+        console.log('##onExecuted', this, message)
+        //TODO 是否 保存base64
+        if (message.base64) {
+          if (Array.isArray(message.base64)) {
+          }
+        }
+      }
+    }
+  },
+  async loadedGraphNode (node, app) {
+    if (node.type === 'LoadImagesToBatch') {
+      // await sleep(0)
+      let imagesWidget = node.widgets.filter(w => w.name === 'images')[0]
+      let imagePreview = node.widgets.filter(w => w.name == 'image_base64')[0]
+
+      let pre = imagePreview.div.querySelector('.images_preview')
+      for (const d of imagesWidget.value?.base64 || []) {
+        let im = createInputImageForBatch(d, imagesWidget)
+        pre.appendChild(im)
+      }
+    }
+  }
+})

web/javascript/layer_mixlab.js

@@ -1,8 +1,70 @@
 import { app } from '../../../scripts/app.js'
-// import { api } from '../../../scripts/api.js'
+import { api } from '../../../scripts/api.js'
 import { ComfyWidgets } from '../../../scripts/widgets.js'
 import { $el } from '../../../scripts/ui.js'
 
+function downloadJsonFile (jsonData, fileName = 'grid.json') {
+  const dataString = JSON.stringify(jsonData)
+  const blob = new Blob([dataString], { type: 'application/json' })
+  const url = URL.createObjectURL(blob)
+
+  const link = document.createElement('a')
+  link.href = url
+  link.download = fileName
+  link.click()
+
+  // 释放URL对象
+  setTimeout(() => {
+    URL.revokeObjectURL(url)
+  }, 0)
+}
+
+function createSelectWithOptions (options) {
+  const select = document.createElement('select')
+
+  options.forEach(option => {
+    const optionElement = document.createElement('option')
+    optionElement.text = option
+    optionElement.value = option
+    select.appendChild(optionElement)
+  })
+
+  select.style = `cursor: pointer;
+  font-weight: 300; 
+  height: 30px;
+  min-width: 122px;
+  position: absolute;
+  top: 24px;
+  left: 88px;
+  z-index: 999999999999999;
+       `
+
+  return select
+}
+
+function drawCanvasWithText (w, h, tag, color = 'rgba(255,255,255,0.4)') {
+  const canvas = document.createElement('canvas')
+  const ctx = canvas.getContext('2d')
+
+  // 设置画布大小
+  canvas.width = w
+  canvas.height = h
+
+  // 绘制白色背景
+  ctx.fillStyle = color
+  ctx.fillRect(0, 0, canvas.width, canvas.height)
+
+  // 绘制文字
+  ctx.fillStyle = '#000000'
+  ctx.font = '20px Arial'
+  ctx.fillText(tag, 50, 50)
+
+  // 导出为Base64
+  const base64 = canvas.toDataURL()
+
+  return base64
+}
+
 function get_position_style (ctx, widget_width, y, node_height) {
   const MARGIN = 4 // the margin around the html element
 
@@ -156,32 +218,29 @@ const parseSvg = async svgContent => {
   return { data, image: base64, svgElement }
 }
 
-
-function findImages(nodeId) {
+function findImages (nodeId) {
   // 检查当前节点是否有 imgs 字段
   const n = app.graph.getNodeById(nodeId)
   if (n.imgs) {
-    return n.imgs;
+    return n.imgs
   }
 
   // 检查当前节点的 inputs 是否有 image 字段
   if (n.inputs) {
     for (let i = 0; i < n.inputs.length; i++) {
-      if (n.inputs[i].name==='image'||n.inputs[i].name==='images') {
+      if (n.inputs[i].name === 'image' || n.inputs[i].name === 'images') {
         // 获取新的 nodeId，并递归调用 findImages 函数
-        var linkId = n.inputs[i]?.link;
+        var linkId = n.inputs[i]?.link
         var origin_id = app.graph.links[linkId].origin_id
-        return findImages(origin_id);
+        return findImages(origin_id)
       }
     }
   }
 
   // 如果没有找到 imgs 字段或者 image 字段，则返回 null
-  return null;
+  return null
 }
 
- 
-
 async function setArea (cw, ch, topBase64, base64, data, fn) {
   let displayHeight = Math.round(window.screen.availHeight * 0.8)
   let div = document.createElement('div')
@@ -353,6 +412,196 @@ async function setArea (cw, ch, topBase64, base64, data, fn) {
   }
 }
 
+async function setAreaTags (cw, ch, grids, fn) {
+  let base64 = drawCanvasWithText(cw, ch, '', 'white')
+  let displayHeight = Math.round(window.screen.availHeight * 0.8)
+  let div = document.createElement('div')
+  div.innerHTML = `
+    <div id='ml_overlay' style='position: absolute;top:0;background: #251f1fc4;
+    height: 100vh;
+    z-index:999999;
+    width: 100%;'>
+      <img id='ml_video' style='position: absolute; 
+      height: ${displayHeight}px;user-select: none; 
+      -webkit-user-drag: none;
+      outline: 2px solid #eaeaea;
+      box-shadow: 8px 9px 17px #575757;' />
+      ${Array.from(grids, g => {
+        const { label: tag, grid } = g
+        const [dx, dy, dw, dh] = grid
+        const base64Data = drawCanvasWithText(dw, dh, tag)
+
+        let x = 0,
+          y = 0,
+          width = (cw * displayHeight) / ch,
+          height = displayHeight
+
+        let imgWidth = cw
+        let imgHeight = ch
+
+        if (dw > 0 && dh > 0) {
+          // 相同尺寸窗口，恢复选区
+          x = (width * dx) / imgWidth
+          y = (height * dy) / imgHeight
+          width = (width * dw) / imgWidth
+          height = (height * dh) / imgHeight
+        }
+
+        return `<div class='ml_selection' 
+          data-tag="${tag}"
+          style='position:absolute; 
+          border: 2px dashed red;
+          pointer-events: none;
+          background-image: url("${base64Data}");
+                background-repeat: no-repeat;
+                background-size: cover;
+                left:${x}px;
+                top:${y}px;
+                width:${width}px;
+                height:${height}px;
+                '></div>`
+      })}
+      <div class="mx_close"> X </div>
+    </div>`
+  // document.body.querySelector('#ml_overlay')
+  document.body.appendChild(div)
+
+  const tags = Array.from(grids, g => g.label)
+  let select = createSelectWithOptions(tags)
+  document.body.appendChild(select)
+
+  let img = div.querySelector('#ml_video')
+  // let overlay = div.querySelector('#ml_overlay')
+  let selections = [...div.querySelectorAll('.ml_selection')]
+
+  let selection = selections.filter(
+    s => s.getAttribute('data-tag') === select.value
+  )[0]
+
+  select.addEventListener('change', e => {
+    selection = selections.filter(
+      s => s.getAttribute('data-tag') === select.value
+    )[0]
+  })
+
+  // console.log(select.value,selection)
+  let close = div.querySelector('.mx_close')
+  let startX, startY, endX, endY
+  let start = false
+  let setDone = false
+  // Set video source
+  img.src = base64
+  // canvas.toDataURL();
+  close.style = `cursor: pointer;
+  position: fixed;
+  left: 12px;
+  top: 12px;
+  z-index: 99999999;
+  background: black;
+  width: 44px;
+  height: 44px;
+  text-align: center;
+  line-height: 44px;`
+
+  // Add mouse events
+  img.addEventListener('mousedown', startSelection)
+  img.addEventListener('mousemove', updateSelection)
+  img.addEventListener('mouseup', endSelection)
+
+  const removeDiv = () => {
+    div.remove()
+    select?.remove()
+    close.removeEventListener('click', removeDiv)
+    img.removeEventListener('mousedown', startSelection)
+    img.removeEventListener('mousemove', updateSelection)
+    img.removeEventListener('mouseup', endSelection)
+    img.removeEventListener('mousedown', setDoneCheck)
+  }
+  close.addEventListener('click', removeDiv)
+
+  const setDoneCheck = event => {
+    console.log(setDone)
+    if (setDone) {
+      img.addEventListener('mousedown', startSelection)
+      img.addEventListener('mousemove', updateSelection)
+      img.addEventListener('mouseup', endSelection)
+      setDone = false
+      start = false
+      startX = event.clientX
+      startY = event.clientY
+    }
+  }
+  img.addEventListener('mousedown', setDoneCheck)
+
+  function remove () {
+    img.removeEventListener('mousedown', startSelection)
+    img.removeEventListener('mousemove', updateSelection)
+    img.removeEventListener('mouseup', endSelection)
+    setDone = true
+    // select?.remove()
+  }
+
+  function startSelection (event) {
+    if (start == false) {
+      startX = event.clientX
+      startY = event.clientY
+      updateSelection(event)
+      start = true
+    } else {
+    }
+  }
+
+  function updateSelection (event) {
+    endX = event.clientX
+    endY = event.clientY
+
+    // Calculate width, height, and coordinates
+    let width = Math.abs(endX - startX)
+    let height = Math.abs(endY - startY)
+    let left = Math.min(startX, endX)
+    let top = Math.min(startY, endY)
+
+    // Set selection style
+    selection.style.left = left + 'px'
+    selection.style.top = top + 'px'
+    selection.style.width = width + 'px'
+    selection.style.height = height + 'px'
+  }
+
+  function endSelection (event) {
+    endX = event.clientX
+    endY = event.clientY
+
+    // 获取img元素的真实宽度和高度
+    let imgWidth = img.naturalWidth
+    let imgHeight = img.naturalHeight
+
+    // 换算起始坐标
+    let realStartX = (startX / img.offsetWidth) * imgWidth
+    let realStartY = (startY / img.offsetHeight) * imgHeight
+
+    // 换算起始坐标
+    let realEndX = (endX / img.offsetWidth) * imgWidth
+    let realEndY = (endY / img.offsetHeight) * imgHeight
+
+    startX = realStartX
+    startY = realStartY
+    endX = realEndX
+    endY = realEndY
+    // Calculate width, height, and coordinates
+    let width = Math.round(Math.abs(endX - startX))
+    let height = Math.round(Math.abs(endY - startY))
+    let left = Math.round(Math.min(startX, endX))
+    let top = Math.round(Math.min(startY, endY))
+
+    if (width <= 0 && height <= 0) return remove()
+
+    if (!!fn) fn(select.value, left, top, width, height)
+
+    remove()
+  }
+}
+
 app.registerExtension({
   name: 'Mixlab.layer.ShowLayer',
   async getCustomWidgets (app) {
@@ -598,8 +847,8 @@ app.registerExtension({
             }
             try {
               console.log('this.inputs', this.id)
-              let imgs=findImages(this.id)
-              
+              let imgs = findImages(this.id)
+
               // let topLinkId = this.inputs[0].link
               // let topNodeId = app.graph.links[topLinkId].origin_id
               let topIm = imgs[0]
@@ -607,9 +856,9 @@ app.registerExtension({
               let linkId = this.inputs[3].link
               let nodeId = app.graph.links[linkId].origin_id
               // console.log(linkId,this.inputs)
-              let imgs2=findImages(nodeId)
+              let imgs2 = findImages(nodeId)
               let im = imgs2[0]
-              console.log(topIm,im)
+              console.log(topIm, im)
               // let src = im.src
               setArea(
                 im.naturalWidth,
@@ -641,3 +890,306 @@ app.registerExtension({
     }
   }
 })
+
+app.registerExtension({
+  name: 'Mixlab.layer.GridInput',
+  async beforeRegisterNodeDef (nodeType, nodeData, app) {
+    if (nodeType.comfyClass == 'GridInput') {
+      const orig_nodeCreated = nodeType.prototype.onNodeCreated
+      nodeType.prototype.onNodeCreated = async function () {
+        orig_nodeCreated?.apply(this, arguments)
+
+        const grids_widget = this.widgets.filter(w => w.name == 'grids')[0]
+
+        const widget = {
+          type: 'div',
+          name: 'upload',
+          draw (ctx, node, widget_width, y, widget_height) {
+            Object.assign(
+              this.div.style,
+              get_position_style(ctx, widget_width, y, node.size[1]),
+              {
+                justifyContent: 'flex-start'
+              }
+            )
+          }
+        }
+
+        widget.div = $el('div', {})
+
+        const addBtn = document.createElement('button')
+        addBtn.innerText = 'Add Box'
+        addBtn.style = `cursor: pointer;
+        font-weight: 300;
+        margin: 2px; 
+        color: var(--descrip-text);
+        background-color: var(--comfy-input-bg);
+        border-radius: 8px;
+        border-color: var(--border-color);
+        border-style: solid;height: 30px;min-width: 122px;
+       `
+
+        const vbtn = document.createElement('button')
+        vbtn.innerText = 'Set Box'
+        vbtn.style = `cursor: pointer;
+        font-weight: 300;
+        margin: 2px; 
+        color: var(--descrip-text);
+        background-color: var(--comfy-input-bg);
+        border-radius: 8px;
+        border-color: var(--border-color);
+        border-style: solid;height: 30px;min-width: 122px;
+       `
+
+        const btn = document.createElement('button')
+        btn.innerText = 'Upload JSON'
+
+        btn.style = `cursor: pointer;
+        font-weight: 300;
+        margin: 2px; 
+        color: var(--descrip-text);
+        background-color: var(--comfy-input-bg);
+        border-radius: 8px;
+        border-color: var(--border-color);
+        border-style: solid;height: 30px;min-width: 122px;
+       `
+
+        addBtn.addEventListener('click', () => {
+          const { width, height, grids } = JSON.parse(grids_widget.value)
+          grids.push({
+            label: 'background',
+            grid: [12, 12, width - 24, height - 24]
+          })
+          grids_widget.value = JSON.stringify(
+            {
+              width,
+              height,
+              grids
+            },
+            null,
+            2
+          )
+        })
+
+        vbtn.addEventListener('click', () => {
+          const { width, height, grids } = JSON.parse(grids_widget.value)
+
+          setAreaTags(width, height, grids, (tag, x, y, w, h) => {
+            grids_widget.value = JSON.stringify(
+              {
+                width,
+                height,
+                grids: Array.from(grids, g => {
+                  if (g.label === tag) {
+                    g.grid = [x, y, w, h]
+                  }
+                  return g
+                })
+              },
+              null,
+              2
+            )
+          })
+        })
+
+        btn.addEventListener('click', () => {
+          let inp = document.createElement('input')
+          inp.type = 'file'
+          inp.accept = '.json'
+          inp.click()
+          inp.addEventListener('change', event => {
+            // 获取选择的文件
+            const file = event.target.files[0]
+            this.title = file.name.split('.')[0]
+
+            // console.log(file.name.split('.')[0])
+            // 创建文件读取器
+            const reader = new FileReader()
+
+            // 定义读取完成事件的回调函数
+            reader.onload = event => {
+              // 读取完成后的文本内容
+              const fileContent = JSON.parse(event.target.result)
+              const grids = fileContent
+              grids_widget.value = JSON.stringify(grids, null, 2)
+              // widget.value = grids
+
+              inp.remove()
+            }
+
+            // 以文本方式读取文件
+            reader.readAsText(file)
+          })
+        })
+
+        widget.div.appendChild(addBtn)
+        widget.div.appendChild(vbtn)
+        widget.div.appendChild(btn)
+        document.body.appendChild(widget.div)
+        this.addCustomWidget(widget)
+
+        const onExecuted = nodeType.prototype.onExecuted
+        nodeType.prototype.onExecuted = function (message) {
+          const r = onExecuted?.apply?.(this, arguments)
+
+          let json = message.json
+          if (json) {
+            json = {
+              width: json[0],
+              height: json[1],
+              grids: json[2]
+            }
+            grids_widget.value = JSON.stringify(json, null, 2)
+            // widget.value = json
+          }
+
+          return r
+        }
+
+        const onRemoved = this.onRemoved
+        this.onRemoved = () => {
+          widget.div.remove()
+          return onRemoved?.()
+        }
+
+        if (this.onResize) {
+          this.onResize(this.size)
+        }
+
+        this.serialize_widgets = true //需要保存参数
+      }
+    }
+  },
+  async loadedGraphNode (node, app) {
+    if (node.type === 'GridInput') {
+      try {
+        const grids_widget = node.widgets.filter(w => w.name == 'grids')[0]
+        const { width, height, grids } = JSON.parse(grids_widget.value)
+        console.log('#GridInput', node, grids)
+
+        const div = node.widgets.filter(w => w.name == 'upload')[0]
+        div.div.querySelector('select').innerHTML = Array.from(
+          grids,
+          g => `<option value="${g.label}">${g.label}</option>`
+        ).join('')
+      } catch (error) {}
+    }
+  }
+})
+
+app.registerExtension({
+  name: 'Mixlab.layer.GridDisplayAndSave',
+  async beforeRegisterNodeDef (nodeType, nodeData, app) {
+    if (nodeType.comfyClass == 'GridDisplayAndSave') {
+      const orig_nodeCreated = nodeType.prototype.onNodeCreated
+      nodeType.prototype.onNodeCreated = async function () {
+        orig_nodeCreated?.apply(this, arguments)
+
+        const grids_widget = this.widgets.filter(w => w.name == 'grids')[0]
+        console.log('GridDisplayAndSave', grids_widget)
+        const widget = {
+          type: 'div',
+          name: 'save_json',
+          draw (ctx, node, widget_width, y, widget_height) {
+            Object.assign(
+              this.div.style,
+              get_position_style(ctx, widget_width, y, node.size[1]),
+              {
+                justifyContent: 'flex-start',
+                flexDirection: 'column'
+              }
+            )
+          }
+        }
+
+        widget.div = $el('div', {})
+
+        const btn = document.createElement('button')
+        btn.innerText = 'Save JSON'
+
+        btn.style = `cursor: pointer;
+        font-weight: 300;
+        margin: 2px; 
+        color: var(--descrip-text);
+        background-color: var(--comfy-input-bg);
+        border-radius: 8px;
+        border-color: var(--border-color);
+        border-style: solid;height: 30px;min-width: 122px;
+        max-width: 122px;
+       `
+
+        btn.addEventListener('click', () => {
+          if (window._mixlab_grid)
+            downloadJsonFile(
+              window._mixlab_grid,
+              this.widgets.filter(w => w.name == 'filename_prefix')[0]?.value +
+                '_grid.json'
+            )
+        })
+
+        widget.div.appendChild(btn)
+        document.body.appendChild(widget.div)
+        this.addCustomWidget(widget)
+
+        const onExecuted = nodeType.prototype.onExecuted
+        nodeType.prototype.onExecuted = function (message) {
+          const r = onExecuted?.apply?.(this, arguments)
+          let save_json = this.widgets.filter(d => d.name == 'save_json')[0]
+          let div = save_json?.div
+          // console.log('Test',message)
+
+          let image = message.image[0]
+          let json = message.json
+          if (image) {
+            const { filename, subfolder, type } = image
+
+            if (!div.querySelector('img')) {
+              let im = new Image()
+              div.appendChild(im)
+              im.style.width = '100%'
+            }
+            div.querySelector('img').src = api.apiURL(
+              `/view?filename=${encodeURIComponent(
+                filename
+              )}&type=${type}&subfolder=${subfolder}${app.getPreviewFormatParam()}${app.getRandParam()}`
+            )
+
+            window._mixlab_grid = {
+              width: json[0],
+              height: json[1],
+              grids: json[2]
+            }
+            // console.log(src)
+          }
+
+          this.onResize?.(this.size)
+
+          return r
+        }
+
+        const onRemoved = this.onRemoved
+        this.onRemoved = () => {
+          widget.div.remove()
+          return onRemoved?.()
+        }
+
+        if (this.onResize) {
+          this.onResize(this.size)
+        }
+
+        this.serialize_widgets = true //需要保存参数
+      }
+    }
+  },
+  async loadedGraphNode (node, app) {
+    if (node.type === 'GridDisplayAndSave') {
+      try {
+        let grids_widget = node.widgets.filter(w => w.name === 'grids')[0]
+        // let ks = getLocalData(`_mixlab_PromptSlide`)
+        let uploadWidget = node.widgets.filter(w => w.name == 'upload')[0]
+        // console.log('##widget', uploadWidget.value)
+        let grids = JSON.parse(uploadWidget.value)
+      } catch (error) {}
+    }
+  }
+})

web/javascript/prompt_mixlab.js

@@ -408,7 +408,7 @@ const _createResult = async (node, widget, message) => {
   const width = node.size[0] * 0.5 - 12
 
   let height_add = 0
-
+  
   for (let index = 0; index < message._images.length; index++) {
     const imgs = message._images[index]
 
@@ -559,8 +559,7 @@ app.registerExtension({
 
       let cards = widget.div.querySelectorAll('.card')
       if (cards.length == 0) node.size = [280, 120]
-
-      _createResult(node, widget, widget.value)
+      if(widget.value) _createResult(node, widget, widget.value)
     }
   }
 })

web/javascript/smart_connect.js

@@ -46,6 +46,18 @@ const smart_connect_config_input = [
     node_widget_name: 'image',
     inputNodeName: 'LoadImage',
     inputNode_output_name: 'IMAGE'
+  },
+  {
+    node_type: 'TripoSRSampler_',
+    node_widget_name: 'image',
+    inputNodeName: 'LoadImagesToBatch',
+    inputNode_output_name: 'IMAGE'
+  },
+  {
+    node_type: 'TripoSRSampler_',
+    node_widget_name: 'mask',
+    inputNodeName: 'RembgNode_Mix',
+    inputNode_output_name: 'masks'
   }
 ]
 
@@ -74,6 +86,18 @@ const smart_connect_config_output = [
     outputNodeName: 'SaveImage',
     outputNode_input_name: 'images'
   },
+  {
+    node_type: 'VAEDecode',
+    node_output_name: 'IMAGE',
+    outputNodeName: 'AppInfo',
+    outputNode_input_name: 'IMAGE'
+  },
+  {
+    node_type: 'VAEDecode',
+    node_output_name: 'IMAGE',
+    outputNodeName: 'SaveImageAndMetadata_',
+    outputNode_input_name: 'images'
+  },
   {
     node_type: 'Moondream',
     node_output_name: 'STRING',
@@ -181,7 +205,10 @@ export function smart_init () {
       ]
       let node_slotType = config[0]
       // 如果input没有，则创建
-      if (!node.inputs?.filter(inp => inp.name === widget.name)[0]||!node.inputs)
+      if (
+        !node.inputs?.filter(inp => inp.name === widget.name)[0] ||
+        !node.inputs
+      )
         convertToInput(node, widget, config)
       input_node.connectByType(inputNode_slot, node, node_slotType)
     }

web/javascript/ui_mixlab.js

@@ -9,33 +9,31 @@ import {
 
 import { smart_init, addSmartMenu } from './smart_connect.js'
 
-let isScriptLoaded = {};
+let isScriptLoaded = {}
 
-function loadExternalScript(url) {
+function loadExternalScript (url) {
   return new Promise((resolve, reject) => {
     if (isScriptLoaded[url]) {
-      resolve();
-      return;
+      resolve()
+      return
     }
 
-    const script = document.createElement('script');
-    script.src = url;
+    const script = document.createElement('script')
+    script.src = url
     script.onload = () => {
-      isScriptLoaded[url]= true;
-      resolve();
-    };
-    script.onerror = reject;
-    document.head.appendChild(script);
-  });
+      isScriptLoaded[url] = true
+      resolve()
+    }
+    script.onerror = reject
+    document.head.appendChild(script)
+  })
 }
 
-
 //
 
-function createChart(chartDom,nodes){
-
-  var myChart = echarts.init(chartDom);
-  var option;
+function createChart (chartDom, nodes) {
+  var myChart = echarts.init(chartDom)
+  var option
 
   console.log(nodes)
   option = {
@@ -46,24 +44,25 @@ function createChart(chartDom,nodes){
           {
             name: 'nodeA',
             value: 10,
-            children: Array.from(nodes,n=>{
+            children: Array.from(nodes, n => {
               return {
-                name:n.type,
-                value:n.count
+                name: n.type,
+                value: n.count
               }
             })
-          },
-          
+          }
         ]
       }
     ]
-  };
-  
-  option && myChart.setOption(option);
+  }
+
+  option && myChart.setOption(option)
 }
 
 async function createNodesCharts () {
-  await loadExternalScript('/extensions/comfyui-mixlab-nodes/lib/echarts.min.js')
+  await loadExternalScript(
+    '/extensions/comfyui-mixlab-nodes/lib/echarts.min.js'
+  )
   const templates = await loadTemplate()
   var nodes = {}
   Array.from(templates, t => {
@@ -74,7 +73,6 @@ async function createNodesCharts () {
     }
   })
   nodes = Object.values(nodes).sort((a, b) => b.count - a.count)
- 
 
   const menu = document.querySelector('.comfy-menu')
   const separator = document.createElement('div')
@@ -86,11 +84,9 @@ async function createNodesCharts () {
   menu.append(separator)
 
   const appsButton = document.createElement('button')
-  appsButton.textContent = 'Nodes';
+  appsButton.textContent = 'Nodes'
 
-  
-
-  appsButton.onclick = () => { 
+  appsButton.onclick = () => {
     let div = document.querySelector('#mixlab_apps')
     if (!div) {
       div = document.createElement('div')
@@ -150,10 +146,9 @@ async function createNodesCharts () {
       div.appendChild(btn)
 
       let chartDom = document.createElement('div')
-      chartDom.style=`height:80vh;width:450px`
-      chartDom.className='chart'
+      chartDom.style = `height:80vh;width:450px`
+      chartDom.className = 'chart'
       div.appendChild(chartDom)
-      
     }
     if (div.style.display == 'flex') {
       div.style.display = 'none'
@@ -173,14 +168,11 @@ async function createNodesCharts () {
       background-color: var(--comfy-menu-bg);
       padding: 10px; 
       border: 1px solid black;z-index: 999999999;padding-top: 0;`
-    };
-
-   
-    createChart(div.querySelector('.chart'),nodes)
+    }
 
+    createChart(div.querySelector('.chart'), nodes)
   }
   menu.append(appsButton)
-
 }
 
 function copyNodeValues (src, dest) {
@@ -290,28 +282,33 @@ function get_url () {
 
 async function get_my_app (filename = null, category = '') {
   let url = get_url()
-  const res = await fetch(`${url}/mixlab/workflow`, {
-    method: 'POST',
-    body: JSON.stringify({
-      task: 'my_app',
-      filename,
-      category,
-      admin: true
-    })
-  })
-  let result = await res.json()
-  let data = []
+  let data = null
+
   try {
+    const res = await fetch(`${url}/mixlab/workflow`, {
+      method: 'POST',
+      body: JSON.stringify({
+        task: 'my_app',
+        filename,
+        category,
+        admin: true
+      })
+    })
+    let result = await res.json()
+
+    data = []
+
     for (const res of result.data) {
-      let { app, workflow } = res.data
-      if (app.filename)
-        data.push({
+      let { app, workflow } = res.data;
+      if (app?.filename) data.push({
           ...app,
           data: workflow,
           date: res.date
         })
     }
-  } catch (error) {}
+  } catch (error) {
+    console.log(error)
+  }
   return data
 }
 
@@ -366,7 +363,14 @@ injectCSS(`::-webkit-scrollbar {
   animation-name: loading_mixlab;
   animation-duration: 2s;
   animation-iteration-count: infinite;
-}`)
+}
+
+.dynamic_prompt{
+  border-left: 2px solid var(--input-text); 
+}
+ 
+
+`)
 
 async function getCustomnodeMappings (mode = 'url') {
   // mode = "local";
@@ -824,6 +828,73 @@ const loadTemplate = async () => {
   return templates ?? []
 }
 
+function drawBadge (node, orig, restArgs) {
+  let ctx = restArgs[0]
+  const r = orig?.apply?.(node, restArgs)
+
+  if (
+    !node.flags.collapsed &&
+    node.constructor.title_mode != LiteGraph.NO_TITLE
+  ) {
+    let text = `#${node.id} `
+
+    let nick = node.getNickname()
+    if (nick) {
+      if (nick == 'ComfyUI') {
+        nick = '🦊'
+      }
+
+      if (nick.length > 25) {
+        text += nick.substring(0, 23) + '..'
+      } else {
+        text += nick
+      }
+    }
+
+    if (text != '') {
+      let fgColor = 'white'
+      let bgColor = '#0F1F0F'
+      let visible = true
+
+      ctx.save()
+      ctx.font = '12px sans-serif'
+      const sz = ctx.measureText(text)
+      ctx.fillStyle = bgColor
+      ctx.beginPath()
+      ctx.roundRect(
+        node.size[0] - sz.width - 12,
+        -LiteGraph.NODE_TITLE_HEIGHT - 20,
+        sz.width + 12,
+        20,
+        5
+      )
+      ctx.fill()
+
+      ctx.fillStyle = fgColor
+      ctx.fillText(
+        text,
+        node.size[0] - sz.width - 6,
+        -LiteGraph.NODE_TITLE_HEIGHT - 6
+      )
+      ctx.restore()
+
+      if (node.has_errors) {
+        ctx.save()
+        ctx.font = 'bold 14px sans-serif'
+        const sz2 = ctx.measureText(node.type)
+        ctx.fillStyle = 'white'
+        ctx.fillText(
+          node.type,
+          node.size[0] / 2 - sz2.width / 2,
+          node.size[1] / 2
+        )
+        ctx.restore()
+      }
+    }
+  }
+  return r
+}
+
 app.registerExtension({
   name: 'Comfy.Mixlab.ui',
   init () {
@@ -873,6 +944,20 @@ app.registerExtension({
         }
       ]
 
+      if (node.widgets) {
+        // let text_widget = node.widgets.filter(
+        //   w => w.name === 'text' && typeof w.value == 'string'
+        // )
+        // if (text_widget && text_widget.length == 1) {
+        //   opts.push({
+        //     content: 'Text-to-Text ♾️Mixlab', // with a name
+        //     callback: () => {
+        //       LGraphCanvas.prototype.text2text(node)
+        //     } // and the callback
+        //   })
+        // }
+      }
+
       opts = addSmartMenu(opts, node)
 
       // if (node.type == 'CLIPTextEncode') {
@@ -1053,6 +1138,57 @@ app.registerExtension({
         (this.canvas.height * 0.5) / (this.ds.scale * dpr) // 考虑设备像素比
       this.setDirty(true, true)
     }
+
+    // 支持app模式的json
+    const loadAppJson = async data => {
+      let workflow
+      try {
+        let w = JSON.parse(data)
+        if (w.app && w.output) workflow = w.workflow
+      } catch (err) {}
+
+      if (workflow && workflow.version && workflow.nodes && workflow.extra) {
+        await app.loadGraphData(workflow)
+      }
+    }
+
+    if (!window._mixlab_app_paste_listener) {
+      window._mixlab_app_paste_listener = true
+      //粘贴json的事件
+      document.addEventListener('paste', async e => {
+        // ctrl+shift+v is used to paste nodes with connections
+        // this is handled by litegraph
+        if (this.shiftDown) return
+
+        let data = e.clipboardData || window.clipboardData
+
+        // No image found. Look for node data
+        data = data.getData('text/plain')
+
+        loadAppJson(data)
+      })
+
+      // 把json往里 拖
+      document.addEventListener('drop', async event => {
+        event.preventDefault()
+        event.stopPropagation()
+
+       
+
+        // Dragging from Chrome->Firefox there is a file but its a bmp, so ignore that
+        if (
+          event.dataTransfer.files.length &&
+          event.dataTransfer.files[0].type == 'application/json'
+        ) {
+          const reader = new FileReader()
+          reader.onload = async () => {
+       
+            loadAppJson(reader.result)
+          }
+          reader.readAsText(event.dataTransfer.files[0])
+        }
+      })
+    }
   },
   setup () {
     setTimeout(async () => {
@@ -1060,6 +1196,9 @@ app.registerExtension({
       const orig = LGraphCanvas.prototype.getCanvasMenuOptions
 
       const apps = await get_my_app()
+      if (!apps) return
+
+      console.log('apps',apps)
 
       let apps_map = { 0: [] }
 
@@ -1074,7 +1213,7 @@ app.registerExtension({
 
       let apps_opts = []
       for (const category in apps_map) {
-        console.log('category', typeof category)
+        // console.log('category', typeof category)
         if (category === '0') {
           apps_opts.push(
             ...Array.from(apps_map[category], a => {
@@ -1084,8 +1223,30 @@ app.registerExtension({
                 has_submenu: false,
                 callback: async () => {
                   try {
-                    let item = (await get_my_app(a.filename))[0]
+                    let ddd = await get_my_app(a.filename)
+                    if (!ddd) return
+                    let item = ddd[0]
                     if (item) {
+                      if (item.author) {
+                        // 有作者信息
+                        if (item.author.avatar)
+                          localStorage.setItem(
+                            '_mixlab_author_avatar',
+                            item.author.avatar
+                          )
+                        if (item.author.name)
+                          localStorage.setItem(
+                            '_mixlab_author_name',
+                            item.author.name
+                          )
+
+                        if (item.author.link)
+                          localStorage.setItem(
+                            '_mixlab_author_link',
+                            item.author.link
+                          )
+                      }
+
                       // console.log(item.data)
                       app.loadGraphData(item.data)
                       setTimeout(() => {
@@ -1113,8 +1274,31 @@ app.registerExtension({
                   content: `${a.name}_${a.version}`,
                   callback: async () => {
                     try {
-                      let item = (await get_my_app(a.filename, a.category))[0]
+                      let ddd = await get_my_app(a.filename, a.category)
+
+                      if (!ddd) return
+                      let item = ddd[0]
                       if (item) {
+                        console.log(item)
+                        if (item.author) {
+                          // 有作者信息
+                          if (item.author.avatar)
+                            localStorage.setItem(
+                              '_mixlab_author_avatar',
+                              item.author.avatar
+                            )
+                          if (item.author.name)
+                            localStorage.setItem(
+                              '_mixlab_author_name',
+                              item.author.name
+                            )
+                          if (item.author.link)
+                            localStorage.setItem(
+                              '_mixlab_author_link',
+                              item.author.link
+                            )
+                        }
+
                         // console.log(item.data)
                         app.loadGraphData(item.data)
                         setTimeout(() => {
@@ -1381,14 +1565,16 @@ app.registerExtension({
                 document.body.appendChild(div)
             }
           },
-          {
-            content: 'Workflow App ♾️Mixlab',
-            has_submenu: true,
-            disabled: false,
-            submenu: {
-              options: apps_opts
-            }
-          }
+          apps_opts.length > 0
+            ? {
+                content: 'Workflow App ♾️Mixlab',
+                has_submenu: true,
+                disabled: false,
+                submenu: {
+                  options: apps_opts
+                }
+              }
+            : null
         )
 
         return options
@@ -1397,6 +1583,51 @@ app.registerExtension({
 
     // createNodesCharts()
   },
+  nodeCreated (node) {
+    if (node.widgets) {
+      // Locate dynamic prompt text widgets
+      // Include any widgets with dynamicPrompts set to true, and customtext
+
+      for (let index = 0; index < node.widgets.length; index++) {
+        const widget = node.widgets[index]
+        if (
+          (widget.type === 'customtext' && widget.dynamicPrompts !== false) ||
+          widget.dynamicPrompts
+        ) {
+          widget.element.classList.add('dynamic_prompt')
+
+          widget.element.addEventListener('mouseover', e => {
+            // console.log(node.widgets_values[index])
+            if (node.widgets_values && node.widgets_values[index])
+              widget.element.setAttribute('title', node.widgets_values[index])
+          })
+        }
+      }
+    }
+
+    fetch('manager/badge_mode').then(r => {
+      if (r.status === 404) {
+        // 右上角的badge是否已经绘制
+        if (!node.badge_enabled) {
+          if (!node.getNickname) {
+            node.getNickname = function () {
+              if (node.nickname) {
+                return node.nickname
+              }
+              return
+              // return getNickname(node, node.comfyClass.trim())
+            }
+          }
+
+          const orig = node.__proto__.onDrawForeground
+          node.onDrawForeground = function (ctx) {
+            drawBadge(node, orig, arguments)
+          }
+          node.badge_enabled = true
+        }
+      }
+    })
+  },
   async loadedGraphNode (node, app) {
     // console.log(
     //   '#ui init',
@@ -1412,3 +1643,41 @@ app.registerExtension({
     } catch (error) {}
   }
 })
+
+//获取当前显存
+function fetchSystemStats () {
+  return new Promise(async (resolve, reject) => {
+    try {
+      const response = await fetch('/system_stats')
+      const data = await response.json()
+      resolve(data)
+    } catch (error) {
+      reject(error)
+    }
+  })
+}
+//清理显存
+function postFreeData () {
+  return new Promise(async (resolve, reject) => {
+    try {
+      const postData = {
+        unload_models: true,
+        free_memory: true
+      }
+      const response = await fetch('/free', {
+        method: 'POST',
+        headers: {
+          'Content-Type': 'application/json'
+        },
+        body: JSON.stringify(postData)
+      })
+      if (response.ok) {
+        resolve()
+      } else {
+        reject(new Error('Request failed'))
+      }
+    } catch (error) {
+      reject(error)
+    }
+  })
+}

web/javascript/video_mixlab.js

@@ -0,0 +1,515 @@
+import { app } from '../../../scripts/app.js'
+import { api } from '../../../scripts/api.js'
+import { ComfyWidgets } from '../../../scripts/widgets.js'
+
+import { $el } from '../../../scripts/ui.js'
+
+// The code is based on ComfyUI-VideoHelperSuite modification.
+
+
+
+function injectCSS (css) {
+  // 检查页面中是否已经存在具有相同内容的style标签
+  const existingStyle = document.querySelector('style')
+  if (existingStyle && existingStyle.textContent === css) {
+    return // 如果已经存在相同的样式，则不进行注入
+  }
+
+  // 创建一个新的style标签，并将CSS内容注入其中
+  const style = document.createElement('style')
+  style.textContent = css
+
+  // 将style标签插入到页面的head元素中
+  const head = document.querySelector('head')
+  head.appendChild(style)
+}
+
+injectCSS(`
+.hidden{
+  display:none !important
+}`)
+
+function get_position_style (ctx, widget_width, y, node_height) {
+  const MARGIN = 4 // the margin around the html element
+
+  /* Create a transform that deals with all the scrolling and zooming */
+  const elRect = ctx.canvas.getBoundingClientRect()
+  const transform = new DOMMatrix()
+    .scaleSelf(
+      elRect.width / ctx.canvas.width,
+      elRect.height / ctx.canvas.height
+    )
+    .multiplySelf(ctx.getTransform())
+    .translateSelf(MARGIN, MARGIN + y)
+
+  return {
+    transformOrigin: '0 0',
+    transform: transform,
+    left: `0`,
+    top: `0`,
+    cursor: 'pointer',
+    position: 'absolute',
+    maxWidth: `${widget_width - MARGIN * 2}px`,
+    // maxHeight: `${node_height - MARGIN * 2}px`, // we're assuming we have the whole height of the node
+    width: `${widget_width - MARGIN * 2}px`,
+    // height: `${node_height * 0.3 - MARGIN * 2}px`,
+    // background: '#EEEEEE',
+    display: 'flex',
+    flexDirection: 'column',
+    // alignItems: 'center',
+    justifyContent: 'space-around'
+  }
+}
+
+function videoUpload (node, inputName, inputData, app) {
+  const imageWidget = node.widgets.find(w => w.name === 'video')
+  let uploadWidget
+
+  const widget = {
+    type: 'div',
+    name: 'upload-preview',
+    draw (ctx, node, widget_width, y, widget_height) {
+      Object.assign(
+        this.div.style,
+        get_position_style(ctx, widget_width, 220, node.size[1]),
+        {
+          outline: '1px solid'
+        }
+      )
+    }
+  }
+
+  widget.div = $el('div', {})
+  widget.div.style.width = `120px`
+  document.body.appendChild(widget.div)
+  node.addCustomWidget(widget)
+  // console.log('#imageWidget', imageWidget)
+  const displayDiv = document.createElement('video')
+  displayDiv.controls = true
+  // displayDiv.style=`width:200px;height:200px`
+  imageWidget.callback = () => {
+    displayDiv.src = `/view?filename=${
+      imageWidget.value
+    }&type=input&subfolder=${''}&rand=${Math.random()}`
+
+    // displayDiv.onloadedmetadata = function () {
+    //   var frameCount = displayDiv.duration * displayDiv.webkitDecodedFrameCount
+    //   console.log('视频帧数：' + frameCount)
+    //   node.widgets.filter(w => w.name == 'video_segment_frames')[0].value =
+    //     frameCount
+    // }
+  }
+
+  if (imageWidget.value) {
+    // console.log(imageWidget.value)
+    displayDiv.src = `/view?filename=${
+      imageWidget.value
+    }&type=input&subfolder=${''}&rand=${Math.random()}`
+  }
+
+  widget.div.appendChild(displayDiv)
+
+  const onRemoved = node.onRemoved
+  node.onRemoved = () => {
+    widget.div.remove()
+    return onRemoved?.()
+  }
+
+  var default_value = imageWidget.value
+  Object.defineProperty(imageWidget, 'value', {
+    set: function (value) {
+      this._real_value = value
+    },
+
+    get: function () {
+      let value = ''
+      if (this._real_value) {
+        value = this._real_value
+      } else {
+        return default_value
+      }
+
+      if (value.filename) {
+        let real_value = value
+        value = ''
+        if (real_value.subfolder) {
+          value = real_value.subfolder + '/'
+        }
+
+        value += real_value.filename
+
+        if (real_value.type && real_value.type !== 'input')
+          value += ` [${real_value.type}]`
+      }
+      return value
+    }
+  })
+  async function uploadFile (file, updateNode, pasted = false) {
+    try {
+      // Wrap file in formdata so it includes filename
+      const body = new FormData()
+      body.append('image', file)
+      if (pasted) body.append('subfolder', 'pasted')
+      const resp = await api.fetchApi('/upload/image', {
+        method: 'POST',
+        body
+      })
+
+      if (resp.status === 200) {
+        const data = await resp.json()
+        // Add the file to the dropdown list and update the widget value
+        let path = data.name
+        if (data.subfolder) path = data.subfolder + '/' + path
+
+        if (!imageWidget.options.values.includes(path)) {
+          imageWidget.options.values.push(path)
+        }
+
+        if (updateNode) {
+          imageWidget.value = path
+        }
+
+        return `/view?filename=${path}&type=input&subfolder=${
+          pasted ? 'pasted' : ''
+        }&rand=${Math.random()}`
+      } else {
+        alert(resp.status + ' - ' + resp.statusText)
+      }
+    } catch (error) {
+      alert(error)
+    }
+  }
+
+  const fileInput = document.createElement('input')
+  Object.assign(fileInput, {
+    type: 'file',
+    accept: 'video/*,.mkv,video/webm,video/mp4,video/x-matroska,image/gif',
+    style: 'display: none',
+    onchange: async () => {
+      if (fileInput.files.length) {
+        let file = fileInput.files[0]
+
+        const url = await uploadFile(file, true)
+
+        // console.log('fileInput', file)
+        var reader = new FileReader()
+        reader.onload = function () {
+          displayDiv.src = url
+          displayDiv.onloadedmetadata = function () {
+            // var frameCount =
+            //   displayDiv.duration * displayDiv.webkitDecodedFrameCount
+            // console.log('视频帧数：' + frameCount)
+            // node.widgets.filter(
+            //   w => w.name == 'video_segment_frames'
+            // )[0].value = frameCount
+          }
+        }
+        reader.readAsDataURL(file)
+      }
+    }
+  })
+  document.body.append(fileInput)
+
+  // Create the button widget for selecting the files
+  uploadWidget = node.addWidget('button', 'upload file', 'video', () => {
+    fileInput.click()
+  })
+  uploadWidget.serialize = false
+  return { widget: uploadWidget }
+}
+ComfyWidgets.VIDEOUPLOAD_ = videoUpload
+
+app.registerExtension({
+  name: 'Mixlab.Video.LoadVideoAndSegment_',
+  async beforeRegisterNodeDef (nodeType, nodeData, app) {
+    if (nodeData?.name == 'LoadVideoAndSegment_') {
+      nodeData.input.required.upload = ['VIDEOUPLOAD_']
+    }
+  },
+  async loadedGraphNode (node, app) {
+    if (node.type === 'LoadVideoAndSegment_') {
+      const imageWidget = node.widgets.find(w => w.name === 'video')
+      const uploadPreview = node.widgets.find(w => w.name === 'upload-preview')
+      if (imageWidget.value) {
+        // console.log(imageWidget.value)
+        uploadPreview.div.querySelector('video').src = `/view?filename=${
+          imageWidget.value
+        }&type=input&subfolder=${''}&rand=${Math.random()}`
+      }
+    }
+  }
+})
+
+
+function offsetDOMWidget(
+  widget,
+  ctx,
+  node,
+  widgetWidth,
+  widgetY,
+  height
+) {
+  const margin = 10
+  const elRect = ctx.canvas.getBoundingClientRect()
+  const transform = new DOMMatrix()
+    .scaleSelf(
+      elRect.width / ctx.canvas.width,
+      elRect.height / ctx.canvas.height
+    )
+    .multiplySelf(ctx.getTransform())
+    .translateSelf(0, widgetY + margin)
+
+  const scale = new DOMMatrix().scaleSelf(transform.a, transform.d)
+  Object.assign(widget.inputEl.style, {
+    transformOrigin: '0 0',
+    transform: scale,
+    left: `${transform.e}px`,
+    top: `${transform.d + transform.f}px`,
+    width: `${widgetWidth}px`,
+    height: `${(height || widget.parent?.inputHeight || 32) - margin}px`,
+    position: 'absolute',
+    background: !node.color ? '' : node.color,
+    color: !node.color ? '' : 'white',
+    zIndex: 5, //app.graph._nodes.indexOf(node),
+  })
+}
+
+export const hasWidgets = (node) => {
+  if (!node.widgets || !node.widgets?.[Symbol.iterator]) {
+    return false
+  }
+  return true
+}
+
+export const cleanupNode = (node) => {
+  if (!hasWidgets(node)) {
+    return
+  }
+
+  for (const w of node.widgets) {
+    if (w.canvas) {
+      w.canvas.remove()
+    }
+    if (w.inputEl) {
+      w.inputEl.remove()
+    }
+    // calls the widget remove callback
+    w.onRemoved?.()
+  }
+}
+
+const CreatePreviewElement = (name, val, format) => {
+  const [type] = format.split('/')  
+  const w = {
+      name,
+      type,
+      value: val,
+      draw: function (ctx, node, widgetWidth, widgetY, height) {
+        const [cw, ch] = this.computeSize(widgetWidth)
+        offsetDOMWidget(this, ctx, node, widgetWidth, widgetY, ch)
+      },
+      computeSize: function (_) {
+        const ratio = this.inputRatio || 1
+        const width = Math.max(220, this.parent.size[0])
+        return [width, (width / ratio + 10)]
+      },
+      onRemoved: function () {
+        if (this.inputEl) {
+          this.inputEl.remove()
+        }
+      },
+    }
+
+    w.inputEl = document.createElement(type === 'video' ? 'video' : 'img')
+    w.inputEl.src = w.value
+    if (type === 'video') {
+      w.inputEl.setAttribute('type', 'video/webm');
+      w.inputEl.autoplay = true
+      w.inputEl.loop = true
+      w.inputEl.controls = false;
+    }
+    w.inputEl.onload = function () {
+      w.inputRatio = w.inputEl.naturalWidth / w.inputEl.naturalHeight
+    }
+    document.body.appendChild(w.inputEl)
+    return w
+  }
+
+
+app.registerExtension({
+  name: 'Mixlab.Video.ImageListReplace',
+  async beforeRegisterNodeDef (nodeType, nodeData, app) {
+    if (nodeData?.name == 'ImageListReplace_') {
+      const orig_nodeCreated = nodeType.prototype.onNodeCreated
+      nodeType.prototype.onNodeCreated = function () {
+        orig_nodeCreated?.apply(this, arguments)
+        const widget = {
+          type: 'div',
+          name: 'preview',
+          draw (ctx, node, widget_width, y, widget_height) {
+            Object.assign(
+              this.div.style,
+              get_position_style(ctx, widget_width, 188, node.size[1]),
+              {
+                outline: '1px solid',
+                display: 'flex',
+                flexWrap: 'wrap',
+                flexDirection: 'row',
+                justifyContent: 'flex-start'
+              }
+            )
+          }
+        }
+
+        widget.div = $el('div', {})
+        widget.div.style.width = `120px`
+        widget.div.className = 'hidden'
+        document.body.appendChild(widget.div)
+        this.addCustomWidget(widget)
+        // console.log('#ImageListReplace', widget)
+
+        const onRemoved = this.onRemoved
+        this.onRemoved = () => {
+          widget.div.remove()
+          return onRemoved?.()
+        }
+      }
+
+      const onExecuted = nodeType.prototype.onExecuted
+      nodeType.prototype.onExecuted = function (message) {
+        onExecuted?.apply(this, arguments)
+
+        // let _image_replace = message._image_replace[0]
+        // _image_replace = `/view?filename=${_image_replace.filename}&type=${
+        //   _image_replace.type
+        // }&subfolder=${_image_replace.subfolder}&rand=${Math.random()}`
+
+        let preview = this.widgets.filter(w => w.name == 'preview')[0]
+
+        if (message._images.length > 0) {
+          preview.div.className = ''
+          // console.log('#ImageListReplace', preview.div)
+        }
+
+        preview.div.innerHTML = ''
+        for (const img_ of message._images) {
+          let img = new Image()
+          img.style = `width: 100px;
+          margin: 4px;`
+          img.src = `/view?filename=${img_.filename}&type=${
+            img_.type
+          }&subfolder=${img_.subfolder}&rand=${Math.random()}`
+          preview.div.appendChild(img)
+        }
+
+        let start_index = this.widgets.filter(w => w.name == 'start_index')[0]
+        let end_index = this.widgets.filter(w => w.name == 'end_index')[0]
+        let invert = this.widgets.filter(w => w.name == 'invert')[0]
+        let _sc = start_index.callback.bind(start_index)
+        let _ec = end_index.callback.bind(end_index)
+
+        const selectImages = () => {
+          // console.log(v)
+          let s = start_index.value,
+            e = end_index.value
+          let imgs = preview.div.querySelectorAll('img')
+          for (let index = 0; index < imgs.length; index++) {
+            if (invert.value) {
+              imgs[index].style.outline =
+                index >= s && index <= e ? 'none' : '4px solid #cbd3fe'
+            } else {
+              imgs[index].style.outline =
+                index >= s && index <= e ? '4px solid #cbd3fe' : 'none'
+            }
+          }
+        }
+
+        selectImages()
+
+        start_index.callback = v => {
+          let s = v,
+            e = end_index.value
+          let imgs = preview.div.querySelectorAll('img')
+          for (let index = 0; index < imgs.length; index++) {
+            if (invert.value) {
+              imgs[index].style.outline =
+                index >= s && index <= e ? 'none' : '4px solid #cbd3fe'
+            } else {
+              imgs[index].style.outline =
+                index >= s && index <= e ? '4px solid #cbd3fe' : 'none'
+            }
+          }
+
+          _sc(v)
+        }
+
+        end_index.callback = v => {
+          let s = start_index.value,
+            e = v
+          let imgs = preview.div.querySelectorAll('img')
+          for (let index = 0; index < imgs.length; index++) {
+            if (invert.value) {
+              imgs[index].style.outline =
+                index >= s && index <= e ? 'none' : '4px solid #cbd3fe'
+            } else {
+              imgs[index].style.outline =
+                index >= s && index <= e ? '4px solid #cbd3fe' : 'none'
+            }
+          }
+
+          _ec(v)
+        }
+
+        invert.callback = v => {
+          selectImages()
+        }
+
+        try {
+        } catch (error) {}
+      }
+    }
+
+    if (nodeData?.name == 'VideoCombine_Adv') {
+      const onExecuted = nodeType.prototype.onExecuted
+      nodeType.prototype.onExecuted = function (message) {
+        const prefix = 'vhs_gif_preview_'
+        const r = onExecuted ? onExecuted.apply(this, message) : undefined
+
+        if (this.widgets) {
+          const pos = this.widgets.findIndex(w => w.name === `${prefix}_0`)
+          if (pos !== -1) {
+            for (let i = pos; i < this.widgets.length; i++) {
+              this.widgets[i].onRemoved?.()
+            }
+            this.widgets.length = pos
+          }
+          if (message?.gifs) {
+            message.gifs.forEach((params, i) => {
+              const previewUrl = api.apiURL(
+                '/view?' + new URLSearchParams(params).toString()
+              )
+              const w = this.addCustomWidget(
+                CreatePreviewElement(
+                  `${prefix}_${i}`,
+                  previewUrl,
+                  params.format || 'image/gif'
+                )
+              )
+              w.parent = this
+            })
+          }
+          const onRemoved = this.onRemoved
+          this.onRemoved = () => {
+            cleanupNode(this)
+            return onRemoved?.()
+          }
+        }
+        this.setSize([
+          this.size[0],
+          this.computeSize([this.size[0], this.size[1]])[1]
+        ])
+        return r
+      }
+    }
+  }
+})

web/main.css

@@ -1,24 +0,0 @@
-::-webkit-scrollbar {
-  width: 2px;
-}
-
-@keyframes loading_mixlab {
-  0% {
-    background-color: green;
-  }
-
-  50% {
-    background-color: lightgreen;
-  }
-
-  100% {
-    background-color: green;
-  }
-}
-
-.loading_mixlab {
-  background-color: green;
-  animation-name: loading_mixlab;
-  animation-duration: 2s;
-  animation-iteration-count: infinite;
-}

workflow/Prompt-weight-workflow.json

@@ -1,6 +1,6 @@
 {
-  "last_node_id": 23,
-  "last_link_id": 25,
+  "last_node_id": 24,
+  "last_link_id": 26,
   "nodes": [
     {
       "id": 9,
@@ -203,6 +203,7 @@
         "Node name for S&R": "ShowTextForGPT"
       },
       "widgets_values": [
+        "a girl face,super,(Pop Art:1.26),(Black and White:1.26)",
         "a girl face,super,(Pop Art:1.26),(Black and White:1.26)"
       ]
     },
@@ -537,7 +538,7 @@
           "type": "IMAGE",
           "links": [
             24,
-            25
+            26
           ],
           "slot_index": 0
         }
@@ -547,7 +548,7 @@
       }
     },
     {
-      "id": 23,
+      "id": 24,
       "type": "AppInfo",
       "pos": [
         3363.0014990624995,
@@ -562,9 +563,9 @@
       "mode": 0,
       "inputs": [
         {
-          "name": "LOGO",
+          "name": "IMAGE",
           "type": "IMAGE",
-          "link": 25
+          "link": 26
         }
       ],
       "properties": {
@@ -580,7 +581,7 @@
         "https://",
         "",
         "enable",
-        1
+        2
       ]
     }
   ],
@@ -706,10 +707,10 @@
       "IMAGE"
     ],
     [
-      25,
+      26,
       10,
       0,
-      23,
+      24,
       0,
       "IMAGE"
     ]