0.3.0

v0.3.0 🚀🚗🚚🏃‍

- Added support for setting proxies: HTTP_PROXY, HTTPS_PROXY, http_proxy, https_proxy ✅

- Added a new Speech feature node, enabling the use of a voice assistant: SpeechRecognition & SpeechSynthesis 🎙️

- Added TextImage node, allowing conversion of text into image format 📷

- Added SvgImage node, enabling layout parsing and poster generation in conjunction with the Layer class node 🖼️

- Added an experimental 3DImage node for loading 3D models 🌟

README.md

@@ -1,11 +1,30 @@
 ## 
+v0.3.0 🚀🚗🚚🏃‍
+
+- Added support for setting proxies: HTTP_PROXY, HTTPS_PROXY, http_proxy, https_proxy ✅
+
+- Added a new Speech feature node, enabling the use of a voice assistant: SpeechRecognition & SpeechSynthesis 🎙️
+
+- Added TextImage node, allowing conversion of text into image format 📷
+
+- Added SvgImage node, enabling layout parsing and poster generation in conjunction with the Layer class node 🖼️
+
+- Added an experimental 3DImage node for loading 3D models 🌟
+
 
 ![screenshare](./assets/screenshare.png)
 
 
+### SpeechRecognition & SpeechSynthesis
+![f](./assets/audio-workflow.svg)
+
+
+
 ### ScreenShareNode & FloatingVideoNode
 > Now comfyui supports capturing screen pixel streams from any software and can be used for LCM-Lora integration. Let's get started with implementation and design! 💻🌐
 
+> 
+
 https://github.com/shadowcz007/comfyui-mixlab-nodes/assets/12645064/e7e77f90-e43e-410a-ab3a-1952b7b4e7da
 
 
@@ -16,7 +35,7 @@ https://github.com/shadowcz007/comfyui-mixlab-nodes/assets/12645064/e7e77f90-e43
 
 
 ### LoadImagesFromLocal
-> Monitor changes to images in a local folder, and trigger real-time execution of workflows, supporting common image formats, especially PSD format, in conjunction with Photoshop. Q: Translate into English
+> Monitor changes to images in a local folder, and trigger real-time execution of workflows, supporting common image formats, especially PSD format, in conjunction with Photoshop. 
 
 ![watch](./assets/4-loadfromlocal-watcher-workflow.svg)
 
@@ -24,7 +43,7 @@ https://github.com/shadowcz007/comfyui-mixlab-nodes/assets/12645064/e7e77f90-e43
 
 
 ### GPT
-> ChatGPT、ChatGLM3 , Some code provided by rui.
+> Support for calling multiple GPTs.ChatGPT、ChatGLM3 , Some code provided by rui. If you are using OpenAI's service, fill in https://api.openai.com/v1 . If you are using a local LLM service, fill in http://127.0.0.1:xxxx/v1 .  Azure OpenAI:https://xxxx.openai.azure.com 
 
 
 ![gpt-workflow.svg](./assets/gpt-workflow.svg)
@@ -32,38 +51,14 @@ https://github.com/shadowcz007/comfyui-mixlab-nodes/assets/12645064/e7e77f90-e43
 [workflow-5](./workflow/5-gpt-workflow.json)
 
 
-## Installation
+### Layers
+> A new layer class node has been added, allowing you to separate the image into layers. After merging the images, you can input the controlnet for further processing.
 
-manually install, simply clone the repo into the custom_nodes directory with this command:
+![layers](./assets/layers-workflow.svg)
 
-```
-cd ComfyUI/custom_nodes
+![poster](./assets/poster-workflow.svg)
 
-git clone https://github.com/shadowcz007/comfyui-mixlab-nodes.git
-
-```
-
-Install the requirements:
-
-run directly:
-```
-cd ComfyUI_Mixlab
-install.bat
-```
-
-or install the requirements using:
-```
-../../../python_embeded/python.exe -s -m pip install -r requirements.txt
-```
-
-If you are using a venv, make sure you have it activated before installation and use:
-```
-pip3 install -r requirements.txt
-```
-
-
-
-## Nodes
+## Other Nodes
 
 ![main](./assets/all-workflow.svg)
 ![main2](./assets/detect-face-all.png)
@@ -79,9 +74,6 @@ pip3 install -r requirements.txt
 ![TransparentImage](./assets/TransparentImage.png)
 
 
-
-
-
 > Consistency Decoder
 
 [openai Consistency Decoder]( https://github.com/openai/consistencydecoder)
@@ -111,6 +103,42 @@ An improvement has been made to directly redirect to GitHub to search for missin
 <!-- ### Workflow
 [Workflow](./workflow.md) -->
 
+
+
+## Installation
+
+manually install, simply clone the repo into the custom_nodes directory with this command:
+
+```
+cd ComfyUI/custom_nodes
+
+git clone https://github.com/shadowcz007/comfyui-mixlab-nodes.git
+
+```
+
+Install the requirements:
+
+run directly:
+```
+cd ComfyUI/custom_nodes/comfyui-mixlab-nodes
+install.bat
+```
+
+or install the requirements using:
+```
+../../../python_embeded/python.exe -s -m pip install -r requirements.txt
+```
+
+If you are using a venv, make sure you have it activated before installation and use:
+```
+pip3 install -r requirements.txt
+```
+
+#### Chinese community
+访问 [www.mixcomfy.com](https://www.mixcomfy.com)，获得更多内测功能，关注微信公众号：Mixlab无界社区
+
+
+
 #### Thanks:
 [ComfyUI-CLIPSeg](https://github.com/biegert/ComfyUI-CLIPSeg/tree/main)
 
@@ -118,5 +146,6 @@ An improvement has been made to directly redirect to GitHub to search for missin
 [discussions](https://github.com/shadowcz007/comfyui-mixlab-nodes/discussions)
 
 ### TODO:
+- 音频播放节点：带可视化、支持多音轨、可配置音轨音量
 - vector https://github.com/GeorgLegato/stable-diffusion-webui-vectorstudio
 

__init__.py

@@ -64,6 +64,18 @@ except ImportError:
     sys.exit()
 
 
+
+def install_openai():
+    # Helper function to install the OpenAI module if not already installed
+    try:
+        importlib.import_module('openai')
+    except ImportError:
+        import pip
+        pip.main(['install', 'openai'])
+
+install_openai()
+
+
 current_path = os.path.abspath(os.path.dirname(__file__))
 
 
@@ -156,8 +168,28 @@ def save_workflow_json(data):
         json.dump(data, file)
     return workflow_path
 
+
+# 保存原始的 get 方法
+_original_request = aiohttp.ClientSession._request
+
+# 定义新的 get 方法
+async def new_request(self, method, url, *args, **kwargs):
+   # 检查环境变量以确定是否使用代理
+    proxy = os.environ.get('HTTP_PROXY') or os.environ.get('HTTPS_PROXY') or os.environ.get('http_proxy') or os.environ.get('https_proxy') 
+    # print('Proxy Config:',proxy)
+    if proxy and 'proxy' not in kwargs:
+        kwargs['proxy'] = proxy
+        print('Use Proxy:',proxy)
+    # 调用原始的 _request 方法
+    return await _original_request(self, method, url, *args, **kwargs)
+
+# 应用 Monkey Patch
+aiohttp.ClientSession._request = new_request
+
 # https
 async def new_start(self, address, port, verbose=True, call_on_start=None):
+        
+
         runner = web.AppRunner(self.app, access_log=None)
         await runner.setup()
         site = web.TCPSite(runner, address, port)
@@ -256,11 +288,14 @@ PromptServer.add_routes=new_add_routes
 
 # 导入节点
 from .nodes.PromptNode import RandomPrompt
-from .nodes.ImageNode import TransparentImage,LoadImagesFromPath,AreaToMask,SmoothMask,FeatheredMask,SplitLongMask,ImageCropByAlpha,EnhanceImage,FaceToMask
+from .nodes.ImageNode import TransparentImage,LoadImagesFromPath,TextImage,SvgImage,Image3D,EmptyLayer,ShowLayer,NewLayer,MergeLayers,AreaToMask,SmoothMask,FeatheredMask,SplitLongMask,ImageCropByAlpha,EnhanceImage,FaceToMask
 from .nodes.Vae import VAELoader,VAEDecode
 from .nodes.ScreenShareNode import ScreenShareNode,FloatingVideo
 from .nodes.Clipseg import CLIPSeg,CombineMasks
-from .nodes.ChatGPT import ChatGPTNode,SessionHistory
+from .nodes.ChatGPT import ChatGPTNode,ShowTextForGPT,CharacterInText
+from .nodes.Audio import SpeechRecognition,SpeechSynthesis
+from .nodes.Utils import ColorInput,FontInput
+
 
 # 要导出的所有节点及其名称的字典
 # 注意：名称应全局唯一
@@ -268,7 +303,14 @@ NODE_CLASS_MAPPINGS = {
     "RandomPrompt":RandomPrompt,
     "TransparentImage":TransparentImage,
     "LoadImagesFromPath":LoadImagesFromPath,
+    "TextImage":TextImage,
     "EnhanceImage":EnhanceImage,
+    "SvgImage":SvgImage,
+    "3DImage":Image3D,
+    "EmptyLayer":EmptyLayer,
+    "ShowLayer":ShowLayer,
+    "NewLayer":NewLayer,
+    "MergeLayers":MergeLayers,
     "SplitLongMask":SplitLongMask,
     "FeatheredMask":FeatheredMask,
     "SmoothMask":SmoothMask,
@@ -281,20 +323,28 @@ NODE_CLASS_MAPPINGS = {
     "FloatingVideo":FloatingVideo,
     "CLIPSeg_":CLIPSeg,
     "CombineMasks_":CombineMasks,
-    "ChatGPT":ChatGPTNode,
-    "SessionHistory":SessionHistory
+    "ChatGPTOpenAI":ChatGPTNode,
+    "ShowTextForGPT":ShowTextForGPT,
+    "CharacterInText":CharacterInText,
+    "SpeechRecognition":SpeechRecognition,
+    "SpeechSynthesis":SpeechSynthesis,
+    "Color":ColorInput,
+    "Font":FontInput
 }
 
 # 一个包含节点友好/可读的标题的字典
 NODE_DISPLAY_NAME_MAPPINGS = {
-    "RandomPrompt": "Random Prompt #Mixlab",
+    "RandomPrompt": "Random Prompt ♾️Mixlab",
     "SplitLongMask":"Splitting a long image into sections",
     "VAELoaderConsistencyDecoder":"Consistency Decoder Loader",
     "VAEDecodeConsistencyDecoder":"Consistency Decoder Decode",
-    "ScreenShare":"ScreenShare #Mixlab",
-    "FloatingVideo":"FloatingVideo #Mixlab",
-    "ChatGPT":"ChatGPT #Mixlab",
-    "SessionHistory":"SessionHistory #Mixlab"
+    "ScreenShare":"ScreenShare ♾️Mixlab",
+    "FloatingVideo":"FloatingVideo ♾️Mixlab",
+    "ChatGPTOpenAI":"ChatGPT ♾️Mixlab",
+    "ShowTextForGPT":"ShowTextForGPT ♾️Mixlab",
+    "MergeLayers":"MergeLayers ♾️Mixlab",
+    "SpeechSynthesis":"SpeechSynthesis ♾️Mixlab",
+    "SpeechRecognition":"SpeechRecognition ♾️Mixlab"
 }
 
 # web ui的节点功能

nodes/Audio.py

@@ -0,0 +1,46 @@
+
+
+
+
+class SpeechRecognition:
+    @classmethod
+    def INPUT_TYPES(s):
+        return {"required": { 
+                    "upload":("AUDIOINPUTMIX",),   },
+                }
+    
+    RETURN_TYPES = ("STRING",)
+    RETURN_NAMES = ("prompt",)
+
+    FUNCTION = "run"
+
+    CATEGORY = "♾️Mixlab/audio"
+
+    INPUT_IS_LIST = False
+    OUTPUT_IS_LIST = (False,)
+
+    def run(self,upload):
+        return (upload,)
+
+
+class SpeechSynthesis:
+    @classmethod
+    def INPUT_TYPES(s):
+        return {
+            "required": {
+                "text": ("STRING", {"forceInput": True}),
+            }
+        }
+
+    INPUT_IS_LIST = True
+    RETURN_TYPES = ("STRING",)
+    FUNCTION = "run"
+    OUTPUT_NODE = True
+    OUTPUT_IS_LIST = (True,)
+
+    CATEGORY = "♾️Mixlab/audio"
+
+    def run(self, text):
+        # print(session_history)
+        return {"ui": {"text": text}, "result": (text,)}
+    

nodes/ChatGPT.py

@@ -46,7 +46,7 @@ def chat(client, model_name,messages ):
             except (urllib.error.HTTPError, openai.OpenAIError) as ex:
                 if try_count >= 3:
                     raise ex
-                time.sleep(5)
+                time.sleep(3)
                 continue
 
         finish_reason = response.choices[0].finish_reason
@@ -66,7 +66,7 @@ class ChatGPTNode:
     def __init__(self):
         # self.__client = OpenAI()
         self.session_history = []  # 用于存储会话历史的列表
-        self.seed=0
+        # self.seed=0
         self.system_content="You are ChatGPT, a large language model trained by OpenAI. Answer as concisely as possible."
 
     @classmethod
@@ -75,24 +75,29 @@ class ChatGPTNode:
             "required": {
                 "api_key":("KEY", {"default": "", "multiline": True}),
                 "api_url":("URL", {"default": "", "multiline": True}),
-                "prompt": ("STRING", {"default": "", "multiline": True}),
+                "prompt": ("STRING", {"multiline": True}),
                 "system_content": ("STRING", 
                                    {
                                        "default": "You are ChatGPT, a large language model trained by OpenAI. Answer as concisely as possible.", 
                                        "multiline": True
                                        }),
-                "model": (["gpt-3.5-turbo", "gpt-3.5-turbo-16k-0613", "gpt-4-0613","gpt-4-1106-preview"], 
+                "model": (["gpt-3.5-turbo","gpt-35-turbo","gpt-3.5-turbo-16k", "gpt-3.5-turbo-16k-0613", "gpt-4-0613","gpt-4-1106-preview"], 
                           {"default": "gpt-3.5-turbo"}),
                 "seed": ("INT", {"default": 0, "min": 0, "max": 10000, "step": 1}),
+                "context_size":("INT", {"default": 1, "min": 0, "max":30, "step": 1}),
+            },
+             "hidden": {
+                "unique_id": "UNIQUE_ID",
+                "extra_pnginfo": "EXTRA_PNGINFO",
             },
         }
 
-    RETURN_TYPES = ("STRING","STRING",)
-    RETURN_NAMES = ("text","session_history",)
+    RETURN_TYPES = ("STRING","STRING","STRING",)
+    RETURN_NAMES = ("text","messages","session_history",)
     FUNCTION = "generate_contextual_text"
-    CATEGORY = "Mixlab/GPT"
+    CATEGORY = "♾️Mixlab/GPT"
     INPUT_IS_LIST = False
-    OUTPUT_IS_LIST = (False,False,)
+    OUTPUT_IS_LIST = (False,False,False,)
 
     
     def generate_contextual_text(self,
@@ -101,18 +106,13 @@ class ChatGPTNode:
                                  prompt, 
                                  system_content,
                                    model, 
-                                   seed):
-        print(api_key,
-                                 api_url, 
-                                 prompt, 
-                                 system_content,
-                                   model, 
-                                   seed)
+                                   seed,context_size,unique_id = None, extra_pnginfo=None):
+        # print(api_key!='',api_url,prompt,system_content,model,seed)
         # 可以选择保留会话历史以维持上下文记忆
         # 或者在此处清除会话历史 self.session_history.clear()
-        if seed!=self.seed:
-            self.seed=seed
-            self.session_history=[]
+        # if seed!=self.seed:
+        #     self.seed=seed
+        #     self.session_history=[]
         
         # 把系统信息和初始信息添加到会话历史中
         if system_content:
@@ -129,21 +129,45 @@ class ChatGPTNode:
 
         # 把用户的提示添加到会话历史中
         # 调用API时传递整个会话历史
-        messages=[{"role": "system", "content": self.system_content}]+self.session_history+[{"role": "user", "content": prompt}]
+
+        def crop_list_tail(lst, size):
+            if size >= len(lst):
+                return lst
+            elif size==0:
+                return []
+            else:
+                return lst[-size:]
+            
+        session_history=crop_list_tail(self.session_history,context_size)
+
+        messages=[{"role": "system", "content": self.system_content}]+session_history+[{"role": "user", "content": prompt}]
         response_content = chat(client,model,messages)
         
         self.session_history=self.session_history+[{"role": "user", "content": prompt}]+[{'role':'assistant',"content":response_content}]
+
+
+        # if unique_id and extra_pnginfo and "workflow" in extra_pnginfo[0]:
+        #     workflow = extra_pnginfo[0]["workflow"]
+        #     node = next((x for x in workflow["nodes"] if str(x["id"]) == unique_id[0]), None)
+        #     if node:
+        #         node["widgets_values"] = ["",
+        #                          api_url, 
+        #                          prompt, 
+        #                          system_content,
+        #                            model,
+        #                            seed,
+        #                            context_size]
         
-        return (response_content,json.dumps(self.session_history, indent=4),)
+        return (response_content,json.dumps(messages, indent=4),json.dumps(self.session_history, indent=4),)
 
 
 
-class SessionHistory:
+class ShowTextForGPT:
     @classmethod
     def INPUT_TYPES(s):
         return {
             "required": {
-                "session_history": ("STRING", {"forceInput": True}),
+                "text": ("STRING", {"forceInput": True}),
             }
         }
 
@@ -153,8 +177,41 @@ class SessionHistory:
     OUTPUT_NODE = True
     OUTPUT_IS_LIST = (True,)
 
-    CATEGORY = "Mixlab/GPT"
+    CATEGORY = "♾️Mixlab/GPT"
 
-    def run(self, session_history):
+    def run(self, text):
         # print(session_history)
-        return {"ui": {"text": session_history}, "result": (session_history,)}
+        return {"ui": {"text": text}, "result": (text,)}
+
+
+class CharacterInText:
+    @classmethod
+    def INPUT_TYPES(s):
+        return {
+            "required": {
+                 "text": ("STRING", {"multiline": True}),
+                 "character": ("STRING", {"multiline": True}),
+                 "start_index": ("INT", {
+                    "default": 1,
+                    "min": 0, #Minimum value
+                    "max": 1024, #Maximum value
+                    "step": 1, #Slider's step
+                    "display": "number" # Cosmetic only: display as "number" or "slider"
+                }),
+            }
+        }
+
+    INPUT_IS_LIST = False
+    RETURN_TYPES = ("INT",)
+    FUNCTION = "run"
+    # OUTPUT_NODE = True
+    OUTPUT_IS_LIST = (False,)
+
+    CATEGORY = "♾️Mixlab/GPT"
+
+    def run(self, text,character,start_index):
+        # print(text,character,start_index)
+        b=1 if character in text else 0
+        
+        return (b+start_index,)
+

nodes/Clipseg.py

@@ -1,3 +1,6 @@
+#### Thanks:
+# [ComfyUI-CLIPSeg](https://github.com/biegert/ComfyUI-CLIPSeg/tree/main)
+
 from transformers import CLIPSegProcessor, CLIPSegForImageSegmentation
 
 from PIL import Image
@@ -99,7 +102,7 @@ class CLIPSeg:
                     }
                 }
 
-    CATEGORY = "Mixlab/mask"
+    CATEGORY = "♾️Mixlab/mask"
     RETURN_TYPES = ("MASK", "IMAGE", "IMAGE",)
     RETURN_NAMES = ("Mask","Heatmap Mask", "BW Mask")
 
@@ -204,7 +207,7 @@ class CombineMasks:
                     },
                 }
         
-    CATEGORY = "Mixlab/mask"
+    CATEGORY = "♾️Mixlab/mask"
     RETURN_TYPES = ("MASK", "IMAGE", "IMAGE",)
     RETURN_NAMES = ("Combined Mask","Heatmap Mask", "BW Mask")
 
@@ -252,7 +255,7 @@ class CombineMasks:
 
 # A dictionary that contains all nodes you want to export with their names
 # NOTE: names should be globally unique
-NODE_CLASS_MAPPINGS = {
-    "CLIPSeg": CLIPSeg,
-    "CombineSegMasks": CombineMasks,
-}
+# NODE_CLASS_MAPPINGS = {
+#     "CLIPSeg": CLIPSeg,
+#     "CombineSegMasks": CombineMasks,
+# }

nodes/ImageNode.py

@@ -1,17 +1,18 @@
 import numpy as np
 import torch
-from PIL import Image, ImageOps,ImageFilter,ImageEnhance,ImageDraw,ImageSequence
+from PIL import Image, ImageOps,ImageFilter,ImageEnhance,ImageDraw,ImageSequence, ImageFont
 from PIL.PngImagePlugin import PngInfo
 import base64,os
 from io import BytesIO
 import folder_paths
-import json
+import json,io
 from comfy.cli_args import args
 import cv2 
 
 from .Watcher import FolderWatcher
 
 
+FONT_PATH= os.path.abspath(os.path.join(os.path.dirname(__file__),'../assets/王汉宗颜楷体繁.ttf'))
 
 MAX_RESOLUTION=8192
 
@@ -322,6 +323,117 @@ def areaToMask(x,y,w,h,image):
     return mask
 
 
+# def merge_images(bg_image, layer_image,mask, x, y, width, height):
+#     # 打开底图
+#     # bg_image = Image.open(background)
+#     bg_image=bg_image.convert("RGBA")
+
+#     # 打开图层
+#     layer_image=layer_image.convert("RGBA")
+#     layer_image = layer_image.resize((width, height))
+#     # mask = Image.new("L", layer_image.size, 255)
+#     mask = mask.resize((width, height))
+#     # 在底图上粘贴图层
+#     bg_image.paste(layer_image, (x, y), mask=mask)
+
+#     # 输出合成后的图片
+#     # bg_image.save("output.jpg")
+#     return bg_image
+
+def merge_images(bg_image, layer_image, mask, x, y, width, height, scale_option):
+    # 打开底图
+    bg_image = bg_image.convert("RGBA")
+
+    # 打开图层
+    layer_image = layer_image.convert("RGBA")
+    # layer_image = layer_image.resize((width, height))
+
+    # 根据缩放选项调整图像大小
+    if scale_option == "height":
+        # 按照高度比例缩放
+        original_width, original_height = layer_image.size
+        scale = height / original_height
+        new_width = int(original_width * scale)
+        layer_image = layer_image.resize((new_width, height))
+    elif scale_option == "width":
+        # 按照宽度比例缩放
+        original_width, original_height = layer_image.size
+        scale = width / original_width
+        new_height = int(original_height * scale)
+        layer_image = layer_image.resize((width, new_height))
+    elif scale_option == "overall":
+        # 整体缩放
+        layer_image = layer_image.resize((width, height))
+
+    # 调整mask的大小
+    nw, nh = layer_image.size
+    mask = mask.resize((nw, nh))
+
+    # 在底图上粘贴图层
+    bg_image.paste(layer_image, (x, y), mask=mask)
+
+    # 输出合成后的图片
+    return bg_image
+
+
+
+def generate_text_image(text_list, font_path, font_size, text_color, vertical=True, spacing=0):
+    # Load Chinese font
+    font = ImageFont.truetype(font_path, font_size)
+
+    # Calculate image size based on the number of characters and orientation
+    if vertical:
+        width = font_size + 100
+        height = font_size * len(text_list) + (len(text_list) - 1) * spacing + 100
+    else:
+        width = font_size * len(text_list) + (len(text_list) - 1) * spacing + 100
+        height = font_size + 100
+
+    # Create a blank image
+    image = Image.new('RGBA', (width, height), (255, 255, 255,0))
+    draw = ImageDraw.Draw(image)
+
+    # Draw text
+    if vertical:
+        for i, char in enumerate(text_list):
+            char_position = (50, 50 + i * font_size)
+            draw.text(char_position, char, font=font, fill=text_color)
+    else:
+        for i, char in enumerate(text_list):
+            char_position = (50 + i * (font_size + spacing), 50)
+            draw.text(char_position, char, font=font, fill=text_color)
+
+    # Save the image
+    # image.save(output_image_path)
+
+    # 分离alpha通道
+    alpha_channel = image.split()[3]
+
+    # 创建一个只有alpha通道的新图像
+    alpha_image = Image.new('L', image.size)
+    alpha_image.putdata(alpha_channel.getdata())
+
+    image=image.convert('RGB')
+
+    return (image,alpha_image)
+
+
+def base64_to_image(base64_string):
+    # 去除前缀
+    prefix, base64_data = base64_string.split(",", 1)
+    
+    # 从base64字符串中解码图像数据
+    image_data = base64.b64decode(base64_data)
+    
+    # 创建一个内存流对象
+    image_stream = io.BytesIO(image_data)
+    
+    # 使用PIL的Image模块打开图像数据
+    image = Image.open(image_stream)
+    
+    return image
+
+
 
 class SmoothMask:
     @classmethod
@@ -341,7 +453,7 @@ class SmoothMask:
 
     FUNCTION = "run"
 
-    CATEGORY = "Mixlab/mask"
+    CATEGORY = "♾️Mixlab/mask"
 
     INPUT_IS_LIST = False
 
@@ -389,7 +501,7 @@ class FeatheredMask:
 
     FUNCTION = "run"
 
-    CATEGORY = "Mixlab/mask"
+    CATEGORY = "♾️Mixlab/mask"
 
     OUTPUT_IS_LIST = (False,)
   
@@ -456,7 +568,7 @@ class SplitLongMask:
 
     FUNCTION = "run"
 
-    CATEGORY = "Mixlab/mask"
+    CATEGORY = "♾️Mixlab/mask"
 
     OUTPUT_IS_LIST = (True,)
   
@@ -494,12 +606,13 @@ class TransparentImage:
             }
     
     RETURN_TYPES = ('STRING','IMAGE','RGBA')
+    RETURN_NAMES = ("file_path","IMAGE","RGBA",)
 
     OUTPUT_NODE = True
 
     FUNCTION = "run"
 
-    CATEGORY = "Mixlab/image"
+    CATEGORY = "♾️Mixlab/image"
 
     # INPUT_IS_LIST = True， 一个batch传进来
     OUTPUT_IS_LIST = (True,True,True,)
@@ -568,7 +681,7 @@ class EnhanceImage:
 
     FUNCTION = "run"
 
-    CATEGORY = "Mixlab/image"
+    CATEGORY = "♾️Mixlab/image"
 
     INPUT_IS_LIST = False
 
@@ -627,7 +740,7 @@ class LoadImagesFromPath:
 
     FUNCTION = "run"
 
-    CATEGORY = "Mixlab/image"
+    CATEGORY = "♾️Mixlab/image"
 
     # INPUT_IS_LIST = True
     OUTPUT_IS_LIST = (True,True,False,)
@@ -671,7 +784,7 @@ class LoadImagesFromPath:
             imgs=[imgs[index_variable]] if index_variable < len(imgs) else None
             masks=[masks[index_variable]] if index_variable < len(masks) else None
 
-        print('#prompt::::',prompt)
+        # print('#prompt::::',prompt)
         return (imgs,masks,prompt,)
 
 
@@ -688,7 +801,7 @@ class ImageCropByAlpha:
 
     FUNCTION = "run"
 
-    CATEGORY = "Mixlab/image"
+    CATEGORY = "♾️Mixlab/image"
 
     INPUT_IS_LIST = False
     OUTPUT_IS_LIST = (False,)
@@ -708,6 +821,117 @@ class ImageCropByAlpha:
         return (img,)
 
 
+
+class TextImage:
+    @classmethod
+    def INPUT_TYPES(s):
+        return {"required": { 
+            
+                    "text": ("STRING",{"multiline": False,"default": "龍馬精神迎新歲"}),
+                    "font_path": ("STRING",{"multiline": False,"default": FONT_PATH}),
+                    "font_size": ("INT",{
+                                "default":100, 
+                                "min": 100, #Minimum value
+                                "max": 1000, #Maximum value
+                                "step": 1, #Slider's step
+                                "display": "number" # Cosmetic only: display as "number" or "slider"
+                                }), 
+                    "spacing": ("INT",{
+                                "default":12, 
+                                "min": 1, #Minimum value
+                                "max": 200, #Maximum value
+                                "step": 1, #Slider's step
+                                "display": "number" # Cosmetic only: display as "number" or "slider"
+                                }), 
+                    "text_color":("STRING",{"multiline": False,"default": "#000000"}),
+                    "vertical":("BOOLEAN", {"default": True},),
+                             },
+                }
+    
+    RETURN_TYPES = ("IMAGE","MASK")
+    # RETURN_NAMES = ("WIDTH","HEIGHT","X","Y",)
+
+    FUNCTION = "run"
+
+    CATEGORY = "♾️Mixlab/image"
+
+    INPUT_IS_LIST = False
+    OUTPUT_IS_LIST = (False,False,)
+
+    def run(self,text,font_path,font_size,spacing,text_color,vertical):
+        
+        text_list=list(text)
+
+        img,mask=generate_text_image(text_list,font_path,font_size,text_color,vertical,spacing)
+        
+        img=pil2tensor(img)
+        mask=pil2tensor(mask)
+
+        return (img,mask,)
+
+
+
+class SvgImage:
+    @classmethod
+    def INPUT_TYPES(s):
+        return {"required": { 
+                    "upload":("SVG",),},
+                }
+    
+    RETURN_TYPES = ("IMAGE","LAYER")
+    RETURN_NAMES = ("IMAGE","layers",)
+
+    FUNCTION = "run"
+
+    CATEGORY = "♾️Mixlab/image"
+
+    INPUT_IS_LIST = False
+    OUTPUT_IS_LIST = (False,True,)
+
+    def run(self,upload):
+        layers=[]
+
+        image = base64_to_image(upload['image'])
+        image=image.convert('RGB')
+        image=pil2tensor(image)
+
+        for layer in upload['data']:
+            layers.append(layer)
+    
+        return (image,layers,)
+
+
+
+class Image3D:
+    @classmethod
+    def INPUT_TYPES(s):
+        return {"required": { 
+                    "upload":("THREED",),   },
+                }
+    
+    RETURN_TYPES = ("IMAGE","MASK",)
+    # RETURN_NAMES = ("IMAGE",)
+
+    FUNCTION = "run"
+
+    CATEGORY = "♾️Mixlab/image"
+
+    INPUT_IS_LIST = False
+    OUTPUT_IS_LIST = (False,False,)
+
+    def run(self,upload):
+        # print(upload['image'])
+        image = base64_to_image(upload['image'])
+        image=image.convert('RGB')
+        mask=image.convert('L')
+        
+        mask=pil2tensor(mask)
+        image=pil2tensor(image)
+
+        return (image,mask,)
+
+
+
 class AreaToMask:
     @classmethod
     def INPUT_TYPES(s):
@@ -719,7 +943,7 @@ class AreaToMask:
 
     FUNCTION = "run"
 
-    CATEGORY = "Mixlab/mask"
+    CATEGORY = "♾️Mixlab/mask"
 
     INPUT_IS_LIST = False
     OUTPUT_IS_LIST = (False,)
@@ -754,7 +978,7 @@ class FaceToMask:
 
     FUNCTION = "run"
 
-    CATEGORY = "Mixlab/mask"
+    CATEGORY = "♾️Mixlab/mask"
 
     INPUT_IS_LIST = False
     OUTPUT_IS_LIST = (False,)
@@ -770,3 +994,292 @@ class FaceToMask:
 
         return (mask,)
 
+
+class EmptyLayer:
+    @classmethod
+    def INPUT_TYPES(s):
+        return {
+            "required": { 
+                "width": ("INT",{
+                    "default":512, 
+                    "min": 1, #Minimum value
+                    "max": 8192, #Maximum value
+                    "step": 1, #Slider's step
+                    "display": "number" # Cosmetic only: display as "number" or "slider"
+                }),
+               "height": ("INT",{
+                    "default": 512, 
+                    "min": 1, #Minimum value
+                    "max": 8192, #Maximum value
+                    "step": 1, #Slider's step
+                    "display": "number" # Cosmetic only: display as "number" or "slider"
+                }),
+            },
+        
+                }
+    
+    RETURN_TYPES = ("LAYER",)
+    RETURN_NAMES = ("layers",)
+
+    FUNCTION = "run"
+
+    CATEGORY = "♾️Mixlab/layer"
+
+    OUTPUT_IS_LIST = (True,)
+
+    def run(self, width,height):
+        blank_image = Image.new("RGB", (width, height))
+        
+        mask=blank_image.convert('L')
+
+        blank_image=pil2tensor(blank_image)
+        mask=pil2tensor(mask)
+
+        layer_n=[{
+            "x":0,
+            "y":0,
+            "width":width,
+            "height":height,
+            "z_index":0,
+            "scale_option":'width',
+            "image":blank_image,
+            "mask":mask
+        }]
+        return (layer_n,)
+
+
+class NewLayer:
+    @classmethod
+    def INPUT_TYPES(s):
+        return {
+            
+            "required": { 
+                "x": ("INT",{
+                    "default": 0, 
+                    "min": -100, #Minimum value
+                    "max": 8192, #Maximum value
+                    "step": 1, #Slider's step
+                    "display": "number" # Cosmetic only: display as "number" or "slider"
+                }),
+                "y": ("INT",{
+                    "default": 0, 
+                    "min": 0, #Minimum value
+                    "max": 8192, #Maximum value
+                    "step": 1, #Slider's step
+                    "display": "number" # Cosmetic only: display as "number" or "slider"
+                }),
+                "width": ("INT",{
+                    "default": 512, 
+                    "min": 1, #Minimum value
+                    "max": 8192, #Maximum value
+                    "step": 1, #Slider's step
+                    "display": "number" # Cosmetic only: display as "number" or "slider"
+                }),
+                "height": ("INT",{
+                    "default": 512, 
+                    "min": 1, #Minimum value
+                    "max": 8192, #Maximum value
+                    "step": 1, #Slider's step
+                    "display": "number" # Cosmetic only: display as "number" or "slider"
+                }),
+                "z_index": ("INT",{
+                    "default": 0, 
+                    "min":0, #Minimum value
+                    "max": 100, #Maximum value
+                    "step": 1, #Slider's step
+                    "display": "number" # Cosmetic only: display as "number" or "slider"
+                }),
+                "scale_option": (["width","height",'overall'],),
+                "image": ("IMAGE",),
+            },
+             "optional":{
+                    "mask": ("MASK",{"default": None}),
+                    "layers": ("LAYER",{"default": None}), 
+                }
+                }
+    
+    RETURN_TYPES = ("LAYER",)
+    RETURN_NAMES = ("layers",)
+
+    FUNCTION = "run"
+
+    CATEGORY = "♾️Mixlab/layer"
+
+    INPUT_IS_LIST = True
+    OUTPUT_IS_LIST = (True,)
+
+    def run(self,x,y,width,height,z_index,scale_option,image,mask,layers):
+        # print(x,y,width,height,z_index,image,mask)
+        
+        if mask==None:
+            im=tensor2pil(image)
+            mask=im.convert('L')
+            mask=pil2tensor(mask)
+        else:
+            mask=mask[0]
+
+        layer_n=[{
+            "x":x[0],
+            "y":y[0],
+            "width":width[0],
+            "height":height[0],
+            "z_index":z_index[0],
+            "scale_option":scale_option[0],
+            "image":image[0],
+            "mask":mask
+        }]
+
+        if layers!=None:
+            layer_n=layer_n+layers
+
+        return (layer_n,)
+
+
+class ShowLayer:
+    @classmethod
+    def INPUT_TYPES(s):
+        return {
+            
+            "required": { 
+                "edit": ("EDIT",),
+               
+                "x": ("INT",{
+                    "default": 0, 
+                    "min": -100, #Minimum value
+                    "max": 8192, #Maximum value
+                    "step": 1, #Slider's step
+                    "display": "number" # Cosmetic only: display as "number" or "slider"
+                }),
+                "y": ("INT",{
+                    "default": 0, 
+                    "min": 0, #Minimum value
+                    "max": 8192, #Maximum value
+                    "step": 1, #Slider's step
+                    "display": "number" # Cosmetic only: display as "number" or "slider"
+                }),
+                "width": ("INT",{
+                    "default": 512, 
+                    "min": 1, #Minimum value
+                    "max": 8192, #Maximum value
+                    "step": 1, #Slider's step
+                    "display": "number" # Cosmetic only: display as "number" or "slider"
+                }),
+                "height": ("INT",{
+                    "default": 512, 
+                    "min": 1, #Minimum value
+                    "max": 8192, #Maximum value
+                    "step": 1, #Slider's step
+                    "display": "number" # Cosmetic only: display as "number" or "slider"
+                }),
+                "z_index": ("INT",{
+                    "default": 0, 
+                    "min":0, #Minimum value
+                    "max": 100, #Maximum value
+                    "step": 1, #Slider's step
+                    "display": "number" # Cosmetic only: display as "number" or "slider"
+                }),
+                "scale_option": (["width","height",'overall'],),
+                # "image": ("IMAGE",),
+            },
+             "optional":{
+                    # "mask": ("MASK",{"default": None}),
+                    "layers": ("LAYER",{"default": None}), 
+                }
+                }
+    
+    RETURN_TYPES = ( )
+    RETURN_NAMES = ( )
+
+    FUNCTION = "run"
+
+    CATEGORY = "♾️Mixlab/layer"
+
+    INPUT_IS_LIST = True
+    # OUTPUT_IS_LIST = (True,)
+
+    def run(self,edit,x,y,width,height,z_index,scale_option,layers):
+        # print(x,y,width,height,z_index,image,mask)
+        
+        # if mask==None:
+        #     im=tensor2pil(image)
+        #     mask=im.convert('L')
+        #     mask=pil2tensor(mask)
+        # else:
+        #     mask=mask[0]
+
+        # layers[edit[0]]={
+        #     "x":x[0],
+        #     "y":y[0],
+        #     "width":width[0],
+        #     "height":height[0],
+        #     "z_index":z_index[0],
+        #     "scale_option":scale_option[0],
+        #     "image":image[0],
+        #     "mask":mask
+        # }
+
+        return ( )
+
+
+class MergeLayers:
+    @classmethod
+    def INPUT_TYPES(s):
+        return {"required": { 
+            "layers": ("LAYER",),
+            "image": ("IMAGE",),
+                             },
+                }
+    
+    RETURN_TYPES = ("IMAGE",)
+    # RETURN_NAMES = ("WIDTH","HEIGHT","X","Y",)
+
+    FUNCTION = "run"
+
+    CATEGORY = "♾️Mixlab/layer"
+
+    INPUT_IS_LIST = True
+    # OUTPUT_IS_LIST = (False,)
+
+    def run(self,layers,image):
+        # print(len(layers),len(image))
+        bg_image=image[0]
+        bg_image=tensor2pil(bg_image)
+        # 按z-index排序
+        layers_new = sorted(layers, key=lambda x: x["z_index"])
+     
+        for layer in layers_new:
+            image=layer['image']
+            mask=layer['mask']
+            if 'type' in layer and layer['type']=='base64' and type(image) == str:
+                im=base64_to_image(image)
+                im=im.convert('RGB')
+                image=pil2tensor(im)
+
+                mask=base64_to_image(mask)
+                mask=mask.convert('L')
+                mask=pil2tensor(mask)
+            
+            
+            layer_image=tensor2pil(image)
+            layer_mask=tensor2pil(mask)
+            bg_image=merge_images(bg_image,
+                                  layer_image,
+                                  layer_mask,
+                                  layer['x'],
+                                  layer['y'],
+                                  layer['width'],
+                                  layer['height'],
+                                  layer['scale_option']
+                                  )
+            
+        mask=bg_image.convert('RGBA')
+        mask=pil2tensor(mask)
+        
+        bg_image=bg_image.convert('RGB')
+        bg_image=pil2tensor(bg_image)
+
+        channels = ["red", "green", "blue", "alpha"]
+        # print(mask,mask.shape)
+        mask = mask[:, :, :, channels.index("green")]
+        
+        return (bg_image,mask,)

nodes/PromptNode.py

@@ -79,7 +79,7 @@ class RandomPrompt:
 
     FUNCTION = "run"
 
-    CATEGORY = "Mixlab/prompt"
+    CATEGORY = "♾️Mixlab/prompt"
 
     OUTPUT_IS_LIST = (True,)
     OUTPUT_NODE = True
@@ -158,7 +158,7 @@ class RunWorkflow:
 
     FUNCTION = "run"
 
-    CATEGORY = "Mixlab/workflow"
+    CATEGORY = "♾️Mixlab/workflow"
 
     OUTPUT_IS_LIST = (True,)
     OUTPUT_NODE = True

nodes/ScreenShareNode.py

@@ -82,23 +82,25 @@ class ScreenShareNode:
         },
           "optional":{
               "prompt": ("PROMPT",),
+              "slide": ("SLIDE",),
+              "seed": ("SEED",),
             #   "seed": ("INT", {"default": 1, "min": 0, "max": 0xffffffffffffffff}),
                 } }
     
-    RETURN_TYPES = ('IMAGE','STRING')
-
+    RETURN_TYPES = ('IMAGE','STRING','FLOAT',"INT")
+    RETURN_NAMES = ("IMAGE","PROMPT","FLOAT","INT")
     FUNCTION = "run"
 
-    CATEGORY = "Mixlab/image"
+    CATEGORY = "♾️Mixlab/image"
 
     # INPUT_IS_LIST = True
-    OUTPUT_IS_LIST = (False,False,False)
+    OUTPUT_IS_LIST = (False,False,False,False)
   
     # 运行的函数
-    def run(self,image_base64,prompt):
+    def run(self,image_base64,prompt,slide,seed):
         im,mask=base64_save(image_base64)
         # print('##########prompt',prompt)
-        return (im,prompt)
+        return (im,prompt,slide,seed)
     
 
 class FloatingVideo:
@@ -114,7 +116,7 @@ class FloatingVideo:
     OUTPUT_NODE = True
     FUNCTION = "run"
 
-    CATEGORY = "Mixlab/image"
+    CATEGORY = "♾️Mixlab/image"
 
     # INPUT_IS_LIST = True
     # OUTPUT_IS_LIST = (False,False,)
@@ -137,3 +139,15 @@ class FloatingVideo:
 
         return { "ui": { "images_": results } }
 
+
+
+# class SildeNode:
+#     CATEGORY = "quicknodes"
+#     @classmethod    
+#     def INPUT_TYPES(s):
+#         return { "required":{} }
+#     RETURN_TYPES = ()
+#     RETURN_NAMES = ()
+#     FUNCTION = "func"
+#     def func(self):
+#         return ()

nodes/Utils.py

@@ -0,0 +1,81 @@
+import os
+
+# FONT_PATH= os.path.abspath(os.path.join(os.path.dirname(__file__),'../assets/王汉宗颜楷体繁.ttf'))
+import matplotlib.font_manager as fm
+
+
+
+def get_font_files(directory):
+ 
+    font_files = {}
+
+    for file in os.listdir(directory):
+        if file.endswith('.ttf') or file.endswith('.otf'):
+            font_name = os.path.splitext(file)[0]
+            font_path = os.path.join(directory, file)
+            font_files[font_name] = os.path.abspath(font_path)
+
+    try:
+        font_paths = fm.findSystemFonts()
+        for path in font_paths:
+            font_prop = fm.FontProperties(fname=path)
+            font_name = font_prop.get_name()
+            font_files[font_name] = path
+    except ValueError:
+        print("findSystemFonts error")
+
+    
+    return font_files
+
+r_directory = os.path.join(os.path.dirname(__file__), '../assets/')
+
+font_files = get_font_files(r_directory)
+# print(font_files)
+
+
+class ColorInput:
+    @classmethod
+    def INPUT_TYPES(s):
+        return {"required": { 
+             
+                    "color":("TCOLOR",), 
+                             },
+                }
+    
+    RETURN_TYPES = ("STRING",)
+    # RETURN_NAMES = ("WIDTH","HEIGHT","X","Y",)
+
+    FUNCTION = "run"
+
+    CATEGORY = "♾️Mixlab/utils"
+
+    INPUT_IS_LIST = False
+    OUTPUT_IS_LIST = (False,False,)
+
+    def run(self,color):
+        return (color,)
+
+
+
+class FontInput:
+    @classmethod
+    def INPUT_TYPES(s):
+        return {"required": { 
+             
+                    "font": (list(font_files.keys()),),
+                             },
+                }
+    
+    RETURN_TYPES = ("STRING",)
+    # RETURN_NAMES = ("WIDTH","HEIGHT","X","Y",)
+
+    FUNCTION = "run"
+
+    CATEGORY = "♾️Mixlab/utils"
+
+    INPUT_IS_LIST = False
+    OUTPUT_IS_LIST = (False,False,)
+
+    def run(self,font):
+
+        return (font_files[font],)

nodes/Vae.py

@@ -145,7 +145,7 @@ class VAELoader:
     RETURN_TYPES = ("VAE",)
     FUNCTION = "load_vae"
 
-    CATEGORY = "Mixlab/ConsistencyDecoder"
+    CATEGORY = "♾️Mixlab/ConsistencyDecoder"
 
     #TODO: scale factor?
     def load_vae(self, vae_name):
@@ -165,7 +165,7 @@ class VAEDecode:
     RETURN_TYPES = ("IMAGE",)
     FUNCTION = "decode"
 
-    CATEGORY = "Mixlab/ConsistencyDecoder"
+    CATEGORY = "♾️Mixlab/ConsistencyDecoder"
 
     def decode(self, vae, samples):
         image = vae.decode(samples["samples"].to("cuda:0"))

web/javascript/audio_mixlab.js

@@ -0,0 +1,311 @@
+import { app } from '../../../scripts/app.js'
+// import { api } from '../../../scripts/api.js'
+import { ComfyWidgets } from '../../../scripts/widgets.js'
+import { $el } from '../../../scripts/ui.js'
+
+function get_position_style (ctx, widget_width, y, node_height) {
+  const MARGIN = 4 // the margin around the html element
+
+  /* Create a transform that deals with all the scrolling and zooming */
+  const elRect = ctx.canvas.getBoundingClientRect()
+  const transform = new DOMMatrix()
+    .scaleSelf(
+      elRect.width / ctx.canvas.width,
+      elRect.height / ctx.canvas.height
+    )
+    .multiplySelf(ctx.getTransform())
+    .translateSelf(MARGIN, MARGIN + y)
+
+  return {
+    transformOrigin: '0 0',
+    transform: transform,
+    left: `0`,
+    top: `0`,
+    cursor: 'pointer',
+    position: 'absolute',
+    maxWidth: `${widget_width - MARGIN * 2}px`,
+    // maxHeight: `${node_height - MARGIN * 2}px`, // we're assuming we have the whole height of the node
+    width: `${widget_width - MARGIN * 2}px`,
+    // height: `${node_height * 0.3 - MARGIN * 2}px`,
+    // background: '#EEEEEE',
+    display: 'flex',
+    flexDirection: 'column',
+    // alignItems: 'center',
+    justifyContent: 'space-around'
+  }
+}
+
+const getLocalData = key => {
+  let data = {}
+  try {
+    data = JSON.parse(localStorage.getItem(key)) || {}
+  } catch (error) {
+    return {}
+  }
+  return data
+}
+
+function speakText (text) {
+  const speechMsg = new SpeechSynthesisUtterance()
+  speechMsg.text = text
+
+  // 语音合成结束时触发的事件
+  speechMsg.onend = function (event) {
+    console.log('语音播放结束')
+    window._mixlab_speech_synthesis_onend = true
+  }
+
+  // 语音合成错误时触发的事件
+  speechMsg.onerror = function (event) {
+    console.error('语音播放错误:', event.error)
+  }
+
+  // 使用浏览器默认语音合成器进行语音播放
+  speechSynthesis.speak(speechMsg)
+}
+
+// 调用方法，将文字转换为语音播放
+// speakText('Hello, how are you?');
+//                      #MixCopilot
+
+const start = (element, id, startBtn) => {
+
+  startBtn.className='loading_mixlab'
+
+  window.recognition = new webkitSpeechRecognition()
+
+  window.recognition.continuous = true
+  window.recognition.interimResults = true
+  window.recognition.lang = navigator.language
+
+  let timeoutId, intervalId
+
+  window.recognition.onstart = () => {
+    console.log('开始语音输入', window._mixlab_speech_synthesis_onend)
+    window._mixlab_speech_synthesis_onend = false
+  }
+
+  window.recognition.onresult = function (event) {
+    const result = event.results[event.results.length - 1][0].transcript
+    console.log('识别结果：', result)
+    element.value = result
+
+    let data = getLocalData('_mixlab_speech_recognition')
+    data[id] = result.trim()
+    localStorage.setItem('_mixlab_speech_recognition', JSON.stringify(data))
+
+    if (timeoutId) clearTimeout(timeoutId)
+   
+    if (!window.recognition) return
+
+    timeoutId = setTimeout(function () {
+      console.log('结果传递：：', result)
+      app.queuePrompt(0, 1)
+      window.recognition?.stop()
+      window.recognition = null;
+      startBtn.className=''
+      startBtn.innerText = 'START'
+
+      timeoutId = null
+
+      intervalId = setInterval(() => {
+        if (
+          app.ui.lastQueueSize === 0 &&
+          !window.recognition &&
+          window._mixlab_speech_synthesis_onend
+        ) {
+          start(element, id, startBtn)
+          startBtn.innerText = 'STOP'
+          if (intervalId) {
+            clearInterval(intervalId)
+          }
+        }
+      }, 2200)
+    }, 2000)
+  }
+
+  window.recognition.onend = function () {
+    console.log('语音输入结束')
+  }
+
+  window.recognition.onspeechend = function () {
+    console.log('onspeechend')
+  }
+
+  window.recognition.onerror = function (event) {
+    console.log('Error occurred in recognition: ' + event.error)
+  }
+
+  window.recognition.start()
+}
+
+app.registerExtension({
+  name: 'Mixlab.audio.SpeechRecognition',
+  async getCustomWidgets (app) {
+    return {
+      AUDIOINPUTMIX (node, inputName, inputData, app) {
+        // console.log('##node', node)
+        const widget = {
+          type: inputData[0], // the type, CHEESE
+          name: inputName, // the name, slice
+          size: [128, 32], // a default size
+          draw (ctx, node, width, y) {},
+          computeSize (...args) {
+            return [128, 32] // a method to compute the current size of the widget
+          },
+          async serializeValue (nodeId, widgetIndex) {
+            let data = getLocalData('_mixlab_speech_recognition')
+            return data[node.id] || 'Hello Mixlab'
+          }
+        }
+        //  widget.something = something;          // maybe adds stuff to it
+        node.addCustomWidget(widget) // adds it to the node
+        return widget // and returns it.
+      }
+    }
+  },
+
+  async beforeRegisterNodeDef (nodeType, nodeData, app) {
+    if (nodeType.comfyClass == 'SpeechRecognition') {
+      const orig_nodeCreated = nodeType.prototype.onNodeCreated
+      nodeType.prototype.onNodeCreated = function () {
+        orig_nodeCreated?.apply(this, arguments)
+
+        const widget = {
+          type: 'div',
+          name: 'chatgptdiv',
+          draw (ctx, node, widget_width, y, widget_height) {
+            Object.assign(
+              this.div.style,
+              get_position_style(ctx, widget_width, 44, node.size[1])
+            )
+          }
+        }
+
+        widget.div = $el('div', {})
+
+        document.body.appendChild(widget.div)
+
+        const inputDiv = (key, placeholder) => {
+          let div = document.createElement('div')
+          const startBtn = document.createElement('button')
+          const textArea = document.createElement('textarea')
+
+          textArea.className = `${'comfy-multiline-input'} ${placeholder}`
+
+          textArea.style = `margin-top: 14px;
+          height: 44px;`
+
+          div.style = `flex-direction: column;
+          display: flex;
+          margin: 0px 8px 6px;`
+
+          startBtn.style = `
+            outline: none;
+            border: none;
+            padding: 4px; `
+
+          startBtn.innerText = 'START'
+
+          div.appendChild(startBtn)
+          div.appendChild(textArea)
+
+          startBtn.addEventListener('click', () => {
+            if (window.recognition) {
+              window.recognition.stop()
+              window.recognition = null
+              startBtn.innerText = 'START'
+              startBtn.className=''
+            } else {
+              start(textArea, this.id, startBtn)
+              startBtn.innerText = 'STOP'
+            }
+          })
+
+          return div
+        }
+
+        let inputAudio = inputDiv('_mixlab_speech_recognition', 'audio')
+        widget.div.appendChild(inputAudio)
+
+        this.addCustomWidget(widget)
+
+        const onRemoved = this.onRemoved
+        this.onRemoved = () => {
+          inputAudio.remove()
+          widget.div.remove()
+          return onRemoved?.()
+        }
+
+        this.serialize_widgets = true //需要保存参数
+      }
+    }
+  },
+  async loadedGraphNode (node, app) {
+    if (node.type === 'SpeechRecognition') {
+      let data = getLocalData('_mixlab_speech_recognition')
+      // console.log('_mixlab_speech_recognition', node.widgets)
+      let div = node.widgets.filter(f => f.type === 'div')[0]
+      if (div && data[node.id]) {
+        div.div.querySelector('textarea').value = data[node.id]
+      }
+    }
+  }
+})
+
+app.registerExtension({
+  name: 'Mixlab.audio.SpeechSynthesis',
+  async beforeRegisterNodeDef (nodeType, nodeData, app) {
+    if (nodeData.name === 'SpeechSynthesis') {
+      function populate (text) {
+        // console.log('SpeechSynthesis',this.widgets)
+
+        if (this.widgets) {
+          const pos = this.widgets.findIndex(w => w.name === 'text')
+          if (pos !== -1) {
+            for (let i = pos; i < this.widgets.length; i++) {
+              this.widgets[i].onRemove?.()
+            }
+            this.widgets.length = pos
+          }
+        }
+
+        for (let list of text) {
+          const w = ComfyWidgets['STRING'](
+            this,
+            'text',
+            ['STRING', { multiline: true }],
+            app
+          ).widget
+          w.inputEl.readOnly = true
+          w.inputEl.style.opacity = 0.6
+          w.value = list
+        }
+
+        speakText(text.join('\n'))
+
+        // console.log('ShowTextForGPT',this.widgets.length)
+        requestAnimationFrame(() => {
+          const sz = this.computeSize()
+          if (sz[0] < this.size[0]) {
+            sz[0] = this.size[0]
+          }
+          if (sz[1] < this.size[1]) {
+            sz[1] = this.size[1]
+          }
+          this.onResize?.(sz)
+          app.graph.setDirtyCanvas(true, false)
+        })
+      }
+
+      // When the node is executed we will be sent the input text, display this in the widget
+      const onExecuted = nodeType.prototype.onExecuted
+      nodeType.prototype.onExecuted = function (message) {
+        onExecuted?.apply(this, arguments)
+        populate.call(this, message.text)
+      }
+
+      this.serialize_widgets = true //需要保存参数
+    }
+  }
+})

web/javascript/checkVersion_mixlab.js

@@ -3,20 +3,24 @@ import { app } from '../../../scripts/app.js'
 const repoOwner = 'shadowcz007' // 替换为仓库的所有者
 const repoName = 'comfyui-mixlab-nodes' // 替换为仓库的名称
 
-const version = 'v0.2.5.1'
+const version = 'v0.3.0'
 
 fetch(`https://api.github.com/repos/${repoOwner}/${repoName}/releases/latest`)
   .then(response => response.json())
   .then(data => {
     const latestVersion = data.tag_name
     console.log('Latest release version:', latestVersion)
-    // if (latestVersion === localStorage.getItem('_mixlab_nodes_vesion')) return
-    if (latestVersion != version) {
-      // localStorage.setItem('_mixlab_nodes_vesion', latestVersion)
+    if (
+      latestVersion &&
+      latestVersion === localStorage.getItem('_mixlab_nodes_vesion')
+    )
+      return
+    if (latestVersion && latestVersion != version) {
+      localStorage.setItem('_mixlab_nodes_vesion', latestVersion)
       app.ui.dialog.show(`<h4 style="font-size: 18px;">${repoName} <br>
       Latest release version: ${latestVersion}</h4>
       <p>Please proceed to the official repository to download the latest version.</p>
-      <a style="    color: #2196F3;
+      <a style="color: #2196F3;
       font-size: 18px;
       font-weight: 800;
       letter-spacing: 2px;

web/javascript/gpt_mixlab.js

@@ -46,22 +46,33 @@ function get_position_style (ctx, widget_width, y, node_height) {
   }
 }
 
+const getLocalData = key => {
+  let data = {}
+  try {
+    data = JSON.parse(localStorage.getItem(key)) || {}
+  } catch (error) {
+    return {}
+  }
+  return data
+}
+
 app.registerExtension({
-  name: 'Mixlab.GPT.ChatGPT',
+  name: 'Mixlab.GPT.ChatGPTOpenAI',
   async getCustomWidgets (app) {
     return {
       KEY (node, inputName, inputData, app) {
-        // console.log('node', inputName, inputData[0])
+        // console.log('##node', node)
         const widget = {
           type: inputData[0], // the type, CHEESE
           name: inputName, // the name, slice
-          size: [128, 24], // a default size
+          size: [128, 32], // a default size
           draw (ctx, node, width, y) {},
           computeSize (...args) {
-            return [128, 24] // a method to compute the current size of the widget
+            return [128,32] // a method to compute the current size of the widget
           },
           async serializeValue (nodeId, widgetIndex) {
-            return localStorage.getItem('_mixlab_api_key') || ''
+            let data = getLocalData('_mixlab_api_key')
+            return data[node.id] || 'by Mixlab'
           }
         }
         //  widget.something = something;          // maybe adds stuff to it
@@ -73,15 +84,16 @@ app.registerExtension({
         const widget = {
           type: inputData[0], // the type, CHEESE
           name: inputName, // the name, slice
-          size: [128, 24], // a default size
+          size: [128, 32], // a default size
           draw (ctx, node, width, y) {
             // a method to draw the widget (ctx is a CanvasRenderingContext2D)
           },
           computeSize (...args) {
-            return [128, 24] // a method to compute the current size of the widget
+            return [128, 32] // a method to compute the current size of the widget
           },
           async serializeValue (nodeId, widgetIndex) {
-            return localStorage.getItem('_mixlab_api_url') || ''
+            let data = getLocalData('_mixlab_api_url')
+            return data[node.id] || 'https://api.openai.com/v1'
           }
         }
         //  widget.something = something;          // maybe adds stuff to it
@@ -90,22 +102,21 @@ app.registerExtension({
       }
     }
   },
+
   async beforeRegisterNodeDef (nodeType, nodeData, app) {
-    // console.log(nodeType.comfyClass)
-    if (nodeType.comfyClass == 'ChatGPT') {
+    if (nodeType.comfyClass == 'ChatGPTOpenAI') {
       const orig_nodeCreated = nodeType.prototype.onNodeCreated
       nodeType.prototype.onNodeCreated = function () {
         orig_nodeCreated?.apply(this, arguments)
 
-        console.log('ChatGPT widtget', this.widgets)
-
         const api_key = this.widgets.filter(w => w.name == 'api_key')[0]
         const api_url = this.widgets.filter(w => w.name == 'api_url')[0]
-        console.log('api_key', api_key, api_url)
+
+        console.log('ChatGPTOpenAI nodeData', this.widgets)
 
         const widget = {
           type: 'div',
-          name: 'chatgpt div',
+          name: 'chatgptdiv',
           draw (ctx, node, widget_width, y, widget_height) {
             Object.assign(
               this.div.style,
@@ -118,36 +129,43 @@ app.registerExtension({
 
         document.body.appendChild(widget.div)
 
-        const inputKey = document.createElement('input'),
-          inputUrl = document.createElement('input')
-        inputKey.type = 'text'
-        inputUrl.type = 'text'
+        const inputDiv = (key, placeholder) => {
+          let div = document.createElement('div')
+          const ip = document.createElement('input')
+          ip.type = placeholder === 'Key' ? 'password' : 'text'
+          ip.className = `${'comfy-multiline-input'} ${placeholder}`
+          div.style = `display: flex;
+          align-items: center; 
+          margin: 6px 8px;
+          margin-top: 0;`
+          ip.placeholder = placeholder
+          ip.value = placeholder
 
-        inputKey.placeholder = 'Key'
-        inputUrl.placeholder = 'URL'
+          ip.style = `margin-left: 24px;
+          outline: none;
+          border: none;
+          padding: 4px;width: 100%;`
+          const label = document.createElement('label')
+          label.style = 'font-size: 10px;min-width:32px'
+          label.innerText = placeholder
+          div.appendChild(label)
+          div.appendChild(ip)
 
-        inputKey.style = `margin:4px 48px;`
-        inputUrl.style = `margin:4px 48px`
+          ip.addEventListener('change', () => {
+            let data = getLocalData(key)
+            data[this.id] = ip.value.trim()
+            localStorage.setItem(key, JSON.stringify(data))
+            console.log(this.id, key)
+          })
+          return div
+        }
 
-        inputUrl.value = localStorage.getItem('_mixlab_api_url') || ''
-        inputKey.value = localStorage.getItem('_mixlab_api_key') || ''
+        let inputKey = inputDiv('_mixlab_api_key', 'Key')
+        let inputUrl = inputDiv('_mixlab_api_url', 'URL')
 
         widget.div.appendChild(inputKey)
         widget.div.appendChild(inputUrl)
 
-        inputKey.addEventListener('change', () => {
-          api_key.serializeValue = () => inputKey.value || ''
-          localStorage.setItem('_mixlab_api_key', inputKey.value)
-        })
-
-        inputUrl.addEventListener('change', () => {
-          api_url.serializeValue = () => inputUrl.value || ''
-          localStorage.setItem('_mixlab_api_url', inputUrl.value)
-        })
-
-        /*
-                Add the widget, make sure we clean up nicely, and we do not want to be serialized!
-                */
         this.addCustomWidget(widget)
 
         const onRemoved = this.onRemoved
@@ -158,82 +176,102 @@ app.registerExtension({
           return onRemoved?.()
         }
 
-        this.serialize_widgets = false
+        this.serialize_widgets = true //需要保存参数
       }
     }
+  },
+  async loadedGraphNode (node, app) {
+    // Fires every time a node is constructed
+    // You can modify widgets/add handlers/etc here
+
+    if (node.type === 'ChatGPTOpenAI') {
+      let widget = node.widgets.filter(w => w.div)[0]
+
+      let apiKey = getLocalData('_mixlab_api_key'),
+        url = getLocalData('_mixlab_api_url')
+
+      let id = node.id
+
+      // console.log('ChatGPTOpenAI serialize_widgets', this)
+
+      widget.div.querySelector('.Key').value = apiKey[id] || 'by Mixlab'
+      widget.div.querySelector('.URL').value =
+        url[id] || 'https://api.openai.com/v1'
+    }
   }
 })
 
 app.registerExtension({
-  name: 'Mixlab.GPT.SessionHistory',
-  async beforeRegisterNodeDef (nodeType, nodeData, app) {
-    if (nodeData.name === 'SessionHistory') {
-      function populate (text) {
-        if (this.widgets) {
-          // const pos = this.widgets.findIndex((w) => w.name === "text");
-          // if (pos !== -1) {
-          // 	for (let i = pos; i < this.widgets.length; i++) {
-          // 		this.widgets[i].onRemove?.();
-          // 	}
-          // 	this.widgets.length = pos;
-          // }
-
-          for (let i = 0; i < this.widgets.length; i++) {
-            this.widgets[i].onRemove?.()
-          }
-          this.widgets.length = 0
-        }
-
-        console.log('SessionHistory', this.widgets, text)
-
-        for (const list of text) {
-          const w = ComfyWidgets['STRING'](
-            this,
-            'text',
-            ['STRING', { multiline: true }],
-            app
-          ).widget
-          w.inputEl.readOnly = true
-          w.inputEl.style.opacity = 0.6
-
-          let res = list
+  name: 'Mixlab.GPT.ShowTextForGPT',
+  async beforeRegisterNodeDef(nodeType, nodeData, app) {
+		if (nodeData.name === "ShowTextForGPT") {
+			function populate(text) {
+				if (this.widgets) {
+          
+					const pos = this.widgets.findIndex((w) => w.name === "text");
+					if (pos !== -1) {
+						for (let i = pos; i < this.widgets.length; i++) {
+							this.widgets[i].onRemove?.();
+						}
+						this.widgets.length = pos;
+					}
+				}
+        // console.log('ShowTextForGPT',text)
+				for (let list of text) {
+					const w = ComfyWidgets["STRING"](this, "text", ["STRING", { multiline: true }], app).widget;
+					w.inputEl.readOnly = true;
+					w.inputEl.style.opacity = 0.6;
 
           try {
-            res = JSON.stringify(JSON.parse(list), null, 2)
+            let data=JSON.parse(list);
+            data=Array.from(data,d=>{
+              return {
+                ...d,
+                content:decodeURIComponent(d.content)
+              }
+            })
+            list=JSON.stringify(data,null,2)
           } catch (error) {
-            // console.log(list)
+            // console.log(error)
           }
 
-          w.value = res
-        }
+					w.value =list;
+        
+				}
+        // console.log('ShowTextForGPT',this.widgets.length)
+				requestAnimationFrame(() => {
+					const sz = this.computeSize();
+					if (sz[0] < this.size[0]) {
+						sz[0] = this.size[0];
+					}
+					if (sz[1] < this.size[1]) {
+						sz[1] = this.size[1];
+					}
+					this.onResize?.(sz);
+					app.graph.setDirtyCanvas(true, false);
+				});
+			}
 
-        requestAnimationFrame(() => {
-          const sz = this.computeSize()
-          if (sz[0] < this.size[0]) {
-            sz[0] = this.size[0]
-          }
-          if (sz[1] < this.size[1]) {
-            sz[1] = this.size[1]
-          }
-          this.onResize?.(sz)
-          app.graph.setDirtyCanvas(true, false)
-        })
-      }
+			// When the node is executed we will be sent the input text, display this in the widget
+			const onExecuted = nodeType.prototype.onExecuted;
+			nodeType.prototype.onExecuted = function (message) {
+				onExecuted?.apply(this, arguments);
+				populate.call(this, message.text);
+			};
 
-      // When the node is executed we will be sent the input text, display this in the widget
-      const onExecuted = nodeType.prototype.onExecuted
-      nodeType.prototype.onExecuted = function (message) {
-        onExecuted?.apply(this, arguments)
-        populate.call(this, message.text)
-      }
+			const onConfigure = nodeType.prototype.onConfigure;
+			nodeType.prototype.onConfigure = function () {
+				onConfigure?.apply(this, arguments);
+				if (this.widgets_values?.length) {
+          
+					populate.call(this, this.widgets_values);
+				}
+			};
 
-      const onConfigure = nodeType.prototype.onConfigure
-      nodeType.prototype.onConfigure = function () {
-        onConfigure?.apply(this, arguments)
-        if (this.widgets_values?.length) {
-          populate.call(this, this.widgets_values)
-        }
-      }
-    }
-  }
+      this.serialize_widgets = true //需要保存参数
+
+		}
+
+
+	},
 })

web/javascript/image_mixlab.js

@@ -0,0 +1,627 @@
+import { app } from '../../../scripts/app.js'
+import { api } from '../../../scripts/api.js'
+import { ComfyWidgets } from '../../../scripts/widgets.js'
+import { $el } from '../../../scripts/ui.js'
+
+async function uploadImage (blob,fileType='.svg') {
+  // const blob = await (await fetch(src)).blob();
+  const body = new FormData()
+  body.append('image', new File([blob], new Date().getTime() + fileType))
+
+  const resp = await api.fetchApi('/upload/image', {
+    method: 'POST',
+    body
+  })
+
+  // console.log(resp)
+  let data = await resp.json()
+  let { name, subfolder } = data
+  let src = api.apiURL(
+    `/view?filename=${encodeURIComponent(
+      name
+    )}&type=input&subfolder=${subfolder}${app.getPreviewFormatParam()}${app.getRandParam()}`
+  )
+
+  return src
+}
+
+function base64ToBlobFromURL(base64URL, contentType) {
+  return fetch(base64URL)
+    .then(response => response.blob());
+}
+
+function getContentTypeFromBase64 (base64Data) {
+  const regex = /^data:(.+);base64,/
+  const matches = base64Data.match(regex)
+  if (matches && matches.length >= 2) {
+    return matches[1]
+  }
+  return null
+}
+
+// 示例用法
+// const base64Data = 'data:image/jpeg;base64,/9j/4AAQSkZJRgABAQEAAAAAAAD/...'; // 替换为实际的base64图片数据
+// const contentType = getContentTypeFromBase64(base64Data);
+// console.log(contentType);
+
+// // 示例用法
+// const base64Data = '...'; // 替换为实际的base64图片数据
+// const contentType = 'image/jpeg'; // 替换为实际的图片类型
+
+// const blob = base64ToBlob(base64Data, contentType);
+// console.log(blob);
+
+function get_position_style (ctx, widget_width, y, node_height) {
+  const MARGIN = 4 // the margin around the html element
+
+  /* Create a transform that deals with all the scrolling and zooming */
+  const elRect = ctx.canvas.getBoundingClientRect()
+  const transform = new DOMMatrix()
+    .scaleSelf(
+      elRect.width / ctx.canvas.width,
+      elRect.height / ctx.canvas.height
+    )
+    .multiplySelf(ctx.getTransform())
+    .translateSelf(MARGIN, MARGIN + y)
+
+  return {
+    transformOrigin: '0 0',
+    transform: transform,
+    left: `0`,
+    top: `0`,
+    cursor: 'pointer',
+    position: 'absolute',
+    maxWidth: `${widget_width - MARGIN * 2}px`,
+    // maxHeight: `${node_height - MARGIN * 2}px`, // we're assuming we have the whole height of the node
+    width: `${widget_width - MARGIN * 2}px`,
+    // height: `${node_height * 0.3 - MARGIN * 2}px`,
+    // background: '#EEEEEE',
+    display: 'flex',
+    flexDirection: 'column',
+    // alignItems: 'center',
+    justifyContent: 'space-around'
+  }
+}
+
+const getLocalData = key => {
+  let data = {}
+  try {
+    data = JSON.parse(localStorage.getItem(key)) || {}
+  } catch (error) {
+    return {}
+  }
+  return data
+}
+
+const setLocalDataOfWin = (key, value) => {
+  localStorage.setItem(key, JSON.stringify(value))
+  // window[key] = value
+}
+
+function createImage (url) {
+  let im = new Image()
+  return new Promise((res, rej) => {
+    im.onload = () => res(im)
+    im.src = url
+  })
+}
+
+const parseImage = url => {
+  return new Promise((res, rej) => {
+    fetch(url)
+      .then(response => response.blob())
+      .then(blob => {
+        const reader = new FileReader()
+        reader.onloadend = () => {
+          const base64data = reader.result
+          res(base64data)
+          // 在这里可以将base64数据用于进一步处理或显示图片
+        }
+        reader.readAsDataURL(blob)
+      })
+      .catch(error => {
+        console.log('发生错误:', error)
+      })
+  })
+}
+
+const parseSvg = async svgContent => {
+  // 创建一个临时的DOM元素来解析SVG
+  const tempContainer = document.createElement('div')
+  tempContainer.innerHTML = svgContent
+
+  // 提取SVG元素
+  const svgElement = tempContainer.querySelector('svg')
+  if (!svgElement) return
+  // 获取SVG中 rect元素
+  var rectElements = svgElement?.querySelectorAll('rect') || []
+
+  // 定义一个数组来存储处理后的数据
+  var data = []
+
+  Array.from(rectElements, (rectElement, i) => {
+    // 获取rect元素的属性值
+    var x = rectElement.getAttribute('x')
+    var y = rectElement.getAttribute('y')
+    var width = rectElement.getAttribute('width')
+    var height = rectElement.getAttribute('height')
+    if (x != undefined && y != undefined) {
+      // 创建一个新的canvas元素
+      var canvas = document.createElement('canvas')
+      canvas.width = width
+      canvas.height = height
+      var context = canvas.getContext('2d')
+
+      // 填充颜色到canvas
+      var fill = rectElement.getAttribute('fill')
+      context.fillStyle = fill
+      context.fillRect(0, 0, width, height)
+
+      // 将canvas转换为base64格式
+      var base64 = canvas.toDataURL()
+
+      // 将数据转化为指定的JSON格式
+
+      var rectData = {
+        x: parseInt(x),
+        y: parseInt(y),
+        width: parseInt(width),
+        height: parseInt(height),
+        z_index: i + 1,
+        scale_option: 'width',
+        image: base64,
+        mask: base64,
+        type: 'base64'
+      }
+
+      // 将处理后的数据添加到数组中
+      data.push(rectData)
+    }
+  })
+
+  var svgWidth = svgElement.getAttribute('width')
+  var svgHeight = svgElement.getAttribute('height')
+  // 创建一个新的canvas元素
+  var canvas = document.createElement('canvas')
+  canvas.width = svgWidth
+  canvas.height = svgHeight
+  var context = canvas.getContext('2d')
+  // 绘制SVG到canvas
+  var svgString = new XMLSerializer().serializeToString(svgElement)
+  var DOMURL = window.URL || window.webkitURL || window
+
+  var svgBlob = new Blob([svgString], { type: 'image/svg+xml;charset=utf-8' })
+  var url = DOMURL.createObjectURL(svgBlob)
+
+  let img = await createImage(url)
+  context.drawImage(img, 0, 0)
+
+  let base64 = canvas.toDataURL()
+
+  var rectData = {
+    x: 0,
+    y: 0,
+    width: parseInt(svgWidth),
+    height: parseInt(svgHeight),
+    z_index: 0,
+    scale_option: 'width',
+    image: base64,
+    mask: base64,
+    type: 'base64'
+  }
+  data.push(rectData)
+
+  // 打印处理后的数据
+  // console.log({ data, image: base64, svgElement })
+  return { data, image: base64, svgElement }
+}
+
+app.registerExtension({
+  name: 'Mixlab.image.SvgImage',
+  async getCustomWidgets (app) {
+    return {
+      SVG (node, inputName, inputData, app) {
+        // console.log('##node', node, inputName, inputData)
+        const widget = {
+          type: inputData[0], // the type, CHEESE
+          name: inputName, // the name, slice
+          size: [128, 88], // a default size
+          draw (ctx, node, width, y) {},
+          computeSize (...args) {
+            return [128, 88] // a method to compute the current size of the widget
+          },
+          async serializeValue (nodeId, widgetIndex) {
+            let d = getLocalData('_mixlab_svg_image')
+            // console.log('serializeValue',d)
+            if (d) {
+              let url = d[node.id]
+              let dt = await fetch(url)
+              let svgStr = await dt.text()
+              const { data, image } = (await parseSvg(svgStr)) || {}
+              // console.log(data, image)
+              return JSON.parse(JSON.stringify({ data, image }))
+            } else {
+              return
+            }
+          }
+        }
+
+        // console.log('##node',node.serialize)
+        //  widget.something = something;          // maybe adds stuff to it
+        node.addCustomWidget(widget) // adds it to the node
+
+        return widget // and returns it.
+      }
+    }
+  },
+
+  async beforeRegisterNodeDef (nodeType, nodeData, app) {
+    if (nodeType.comfyClass == 'SvgImage') {
+      const orig_nodeCreated = nodeType.prototype.onNodeCreated
+      nodeType.prototype.onNodeCreated = async function () {
+        orig_nodeCreated?.apply(this, arguments)
+
+        const uploadWidget = this.widgets.filter(w => w.name == 'upload')[0]
+        // console.log('SvgImage nodeData',await uploadWidget.serializeValue())
+
+        const widget = {
+          type: 'div',
+          name: 'upload-preview',
+          draw (ctx, node, widget_width, y, widget_height) {
+            Object.assign(
+              this.div.style,
+              get_position_style(ctx, widget_width, 44, node.size[1])
+            )
+          }
+        }
+
+        widget.div = $el('div', {})
+
+        document.body.appendChild(widget.div)
+
+        const inputDiv = (key, placeholder, svgContainer) => {
+          let div = document.createElement('div')
+          const ip = document.createElement('input')
+          ip.type = 'file'
+          ip.className = `${'comfy-multiline-input'} ${placeholder}`
+          div.style = `display: flex;
+          align-items: center; 
+          margin: 6px 8px;
+          margin-top: 0;`
+          ip.placeholder = placeholder
+          // ip.value = value
+
+          ip.style = `outline: none;
+          border: none;
+          padding: 4px;
+          width: 60%;cursor: pointer;
+          height: 32px;`
+          const label = document.createElement('label')
+          label.style = 'font-size: 10px;min-width:32px'
+          label.innerText = placeholder
+          div.appendChild(label)
+          div.appendChild(ip)
+
+          let that = this
+
+          ip.addEventListener('change', event => {
+            const file = event.target.files[0]
+            const reader = new FileReader()
+
+            // 读取文件内容
+            reader.onload = async e => {
+              const svgContent = e.target.result
+
+              var blob = new Blob([svgContent], { type: 'image/svg+xml' })
+              let url = await uploadImage(blob)
+              // console.log(url)
+              const { svgElement, data, image } = await parseSvg(svgContent)
+              // 将提取的SVG元素显示在页面上
+              let dd = getLocalData(key)
+              dd[that.id] = url
+              setLocalDataOfWin(key, dd)
+              // console.log(this.id, ip.value.trim())
+
+              svgElement.style = `width: 90%;padding: 5%;`
+              // 将提取的SVG元素显示在页面上
+
+              svgContainer.innerHTML = ''
+              svgContainer.appendChild(svgElement)
+
+              uploadWidget.value = await uploadWidget.serializeValue()
+            }
+
+            // 以文本形式读取文件
+            reader.readAsText(file)
+          })
+          return div
+        }
+
+        let svg = document.createElement('div')
+        svg.className = 'preview'
+        svg.style = `background:#eee;margin-top: 12px;`
+
+        let upload = inputDiv('_mixlab_svg_image', 'Svg', svg)
+
+        widget.div.appendChild(upload)
+        widget.div.appendChild(svg)
+        this.addCustomWidget(widget)
+
+        const onRemoved = this.onRemoved
+        this.onRemoved = () => {
+          upload.remove()
+          svg.remove()
+          widget.div.remove()
+          return onRemoved?.()
+        }
+
+        if (this.onResize) {
+          this.onResize(this.size)
+        }
+
+        this.serialize_widgets = true //需要保存参数
+      }
+    }
+  },
+  async loadedGraphNode (node, app) {
+    // Fires every time a node is constructed
+    // You can modify widgets/add handlers/etc here
+    const sleep = (t = 1000) => {
+      return new Promise((res, rej) => {
+        setTimeout(() => res(1), t)
+      })
+    }
+    if (node.type === 'SvgImage') {
+      // await sleep(0)
+      let widget = node.widgets.filter(w => w.name === 'upload-preview')[0]
+
+      let dd = getLocalData('_mixlab_svg_image')
+
+      let id = node.id
+      console.log('SvgImage load', node.widgets[0], node.widgets)
+      if (!dd[id]) return
+      let dt = await fetch(dd[id])
+      let svgStr = await dt.text()
+
+      const { svgElement, data, image } = await parseSvg(svgStr)
+      svgElement.style = `width: 90%;padding: 5%;`
+      // 将提取的SVG元素显示在页面上
+
+      widget.div.querySelector('.preview').innerHTML = ''
+      widget.div.querySelector('.preview').appendChild(svgElement)
+
+      const uploadWidget = node.widgets.filter(w => w.name == 'upload')[0]
+      uploadWidget.value = await uploadWidget.serializeValue()
+
+      // let h=~~getComputedStyle(widget.div).height.replace('px','');
+      // let w=~~getComputedStyle(widget.div).width.replace('px','');
+      // // console.log('svg', w,h,node.size)
+      // node.setSize([
+      //   w,h
+      // ])
+      // app.graph.setDirtyCanvas(true)
+
+      // console.log(node.widgets_values)
+    }
+  }
+})
+
+app.registerExtension({
+  name: 'Mixlab.image.3DImage',
+  async getCustomWidgets (app) {
+    return {
+      THREED (node, inputName, inputData, app) {
+        // console.log('##node', node, inputName, inputData)
+        const widget = {
+          type: inputData[0], // the type, CHEESE
+          name: inputName, // the name, slice
+          size: [128, 88], // a default size
+          draw (ctx, node, width, y) {},
+          computeSize (...args) {
+            return [128, 88] // a method to compute the current size of the widget
+          },
+          async serializeValue (nodeId, widgetIndex) {
+            let d = getLocalData('_mixlab_3d_image')
+            // console.log('serializeValue',d)
+            if (d) {
+              let url = d[node.id]
+              let base64 = await parseImage(url)
+
+              return JSON.parse(JSON.stringify({ image: base64 }))
+            } else {
+              return {}
+            }
+          }
+        }
+        node.addCustomWidget(widget)
+        return widget
+      }
+    }
+  },
+
+  async beforeRegisterNodeDef (nodeType, nodeData, app) {
+    if (nodeType.comfyClass == '3DImage') {
+      const orig_nodeCreated = nodeType.prototype.onNodeCreated
+      nodeType.prototype.onNodeCreated = async function () {
+        orig_nodeCreated?.apply(this, arguments)
+
+        const uploadWidget = this.widgets.filter(w => w.name == 'upload')[0]
+        console.log('3d nodeData', this.inputs)
+
+        const widget = {
+          type: 'div',
+          name: 'upload-preview',
+          draw (ctx, node, widget_width, y, widget_height) {
+            Object.assign(
+              this.div.style,
+              get_position_style(ctx, widget_width, 44, node.size[1])
+            )
+          }
+        }
+
+        widget.div = $el('div', {})
+
+        document.body.appendChild(widget.div)
+
+        const inputDiv = (key, placeholder, preview) => {
+          let div = document.createElement('div')
+          const ip = document.createElement('input')
+          ip.type = 'file'
+          ip.className = `${'comfy-multiline-input'} ${placeholder}`
+          div.style = `display: flex;
+          align-items: center; 
+          margin: 6px 8px;
+          margin-top: 0;`
+          ip.placeholder = placeholder
+          // ip.value = value
+
+          ip.style = `outline: none;
+          border: none;
+          padding: 4px;
+          width: 60%;cursor: pointer;
+          height: 32px;`
+          const label = document.createElement('label')
+          label.style = 'font-size: 10px;min-width:32px'
+          label.innerText = placeholder
+          div.appendChild(label)
+          div.appendChild(ip)
+
+          let that = this
+
+          ip.addEventListener('change', event => {
+            const file = event.target.files[0]
+            const reader = new FileReader()
+
+            // 读取文件内容
+            reader.onload = async e => {
+              const fileURL = URL.createObjectURL(file)
+              // console.log('文件URL: ', fileURL)
+              let html = `<model-viewer 
+              alt="Neil Armstrong's Spacesuit from the Smithsonian Digitization Programs Office and National Air and Space Museum"
+               src="${fileURL}" 
+               ar 
+               shadow-intensity="1" 
+               camera-controls 
+               touch-action="pan-y">
+               
+               <div class="controls">
+                <div>Variant: <select class="variant"></select></div>
+                <div><button class="capture">Capture</button></div>
+              </div></model-viewer>`
+
+              preview.innerHTML = html
+
+              const modelViewerVariants = preview.querySelector('model-viewer')
+              const select = preview.querySelector('.variant')
+              const capture = preview.querySelector('.capture')
+
+              modelViewerVariants.addEventListener('load', () => {
+                const names = modelViewerVariants.availableVariants
+                for (const name of names) {
+                  const option = document.createElement('option')
+                  option.value = name
+                  option.textContent = name
+                  select.appendChild(option)
+                }
+                // Adds a default option.
+                const option = document.createElement('option')
+                option.value = 'default'
+                option.textContent = 'Default'
+                select.appendChild(option)
+              })
+
+              select.addEventListener('input', event => {
+                modelViewerVariants.variantName =
+                  event.target.value === 'default' ? null : event.target.value
+              })
+
+              capture.addEventListener('click', async () => {
+                let base64Data = modelViewerVariants.toDataURL()
+
+                const contentType = getContentTypeFromBase64(base64Data)
+
+                const blob =await base64ToBlobFromURL(base64Data, contentType)
+
+                //  const fileBlob = new Blob([e.target.result], { type: file.type });
+                let url = await uploadImage(blob,'.png')
+                console.log(url)
+
+                let dd = getLocalData(key)
+                dd[that.id] = url
+
+                setLocalDataOfWin(key, dd)
+              })
+
+              uploadWidget.value = await uploadWidget.serializeValue()
+            }
+
+            // 以文本形式读取文件
+            reader.readAsDataURL(file)
+          })
+          return div
+        }
+
+        let preview = document.createElement('div')
+        preview.className = 'preview'
+        preview.style = `background:#eee;margin-top: 12px;`
+
+        let upload = inputDiv('_mixlab_3d_image', '3D Model', preview)
+
+        widget.div.appendChild(upload)
+        widget.div.appendChild(preview)
+        this.addCustomWidget(widget)
+
+        const onRemoved = this.onRemoved
+        this.onRemoved = () => {
+          upload.remove()
+          preview.remove()
+          widget.div.remove()
+          return onRemoved?.()
+        }
+
+        if (this.onResize) {
+          this.onResize(this.size)
+        }
+
+        this.serialize_widgets = true //需要保存参数
+      }
+    }
+  },
+  async loadedGraphNode (node, app) {
+    // Fires every time a node is constructed
+    // You can modify widgets/add handlers/etc here
+    const sleep = (t = 1000) => {
+      return new Promise((res, rej) => {
+        setTimeout(() => res(1), t)
+      })
+    }
+    if (node.type === '3DImage') {
+      // await sleep(0)
+      let widget = node.widgets.filter(w => w.name === 'upload-preview')[0]
+
+      let dd = getLocalData('_mixlab_3d_image')
+
+      let id = node.id
+      console.log('3dImage load', node.widgets[0], node.widgets)
+      if (!dd[id]) return
+
+      let url = dd[id]
+      // let base64 = await parseImage(url)
+
+      widget.div.querySelector('.preview').innerHTML = `<img src="${url}"/>` 
+
+      const uploadWidget = node.widgets.filter(w => w.name == 'upload')[0]
+      uploadWidget.value = await uploadWidget.serializeValue()
+
+      // let h=~~getComputedStyle(widget.div).height.replace('px','');
+      // let w=~~getComputedStyle(widget.div).width.replace('px','');
+      // // console.log('svg', w,h,node.size)
+      // node.setSize([
+      //   w,h
+      // ])
+      // app.graph.setDirtyCanvas(true)
+
+      // console.log(node.widgets_values)
+    }
+  }
+})

web/javascript/layer_mixlab.js

@@ -0,0 +1,333 @@
+import { app } from '../../../scripts/app.js'
+// import { api } from '../../../scripts/api.js'
+import { ComfyWidgets } from '../../../scripts/widgets.js'
+import { $el } from '../../../scripts/ui.js'
+
+function get_position_style (ctx, widget_width, y, node_height) {
+  const MARGIN = 4 // the margin around the html element
+
+  /* Create a transform that deals with all the scrolling and zooming */
+  const elRect = ctx.canvas.getBoundingClientRect()
+  const transform = new DOMMatrix()
+    .scaleSelf(
+      elRect.width / ctx.canvas.width,
+      elRect.height / ctx.canvas.height
+    )
+    .multiplySelf(ctx.getTransform())
+    .translateSelf(MARGIN, MARGIN + y)
+
+  return {
+    transformOrigin: '0 0',
+    transform: transform,
+    left: `0`,
+    top: `0`,
+    cursor: 'pointer',
+    position: 'absolute',
+    maxWidth: `${widget_width - MARGIN * 2}px`,
+    // maxHeight: `${node_height - MARGIN * 2}px`, // we're assuming we have the whole height of the node
+    width: `${widget_width - MARGIN * 2}px`,
+    // height: `${node_height * 0.3 - MARGIN * 2}px`,
+    // background: '#EEEEEE',
+    display: 'flex',
+    // flexDirection: 'column',
+    // alignItems: 'center',
+    justifyContent: 'space-around'
+  }
+}
+
+const getLocalData = key => {
+  let data = {}
+  try {
+    data = JSON.parse(localStorage.getItem(key)) || {}
+  } catch (error) {
+    return {}
+  }
+  return data
+}
+
+function createImage (url) {
+  let im = new Image()
+  return new Promise((res, rej) => {
+    im.onload = () => res(im)
+    im.src = url
+  })
+}
+
+const parseSvg = async svgContent => {
+  // 创建一个临时的DOM元素来解析SVG
+  const tempContainer = document.createElement('div')
+  tempContainer.innerHTML = svgContent
+
+  // 提取SVG元素
+  const svgElement = tempContainer.querySelector('svg')
+  if (!svgElement) return
+  // 获取SVG中 rect元素
+  var rectElements = svgElement?.querySelectorAll('rect') || []
+
+  // 定义一个数组来存储处理后的数据
+  var data = []
+
+  Array.from(rectElements, (rectElement, i) => {
+    // 获取rect元素的属性值
+    var x = rectElement.getAttribute('x')
+    var y = rectElement.getAttribute('y')
+    var width = rectElement.getAttribute('width')
+    var height = rectElement.getAttribute('height')
+    if (x != undefined && y != undefined) {
+      // 创建一个新的canvas元素
+      var canvas = document.createElement('canvas')
+      canvas.width = width
+      canvas.height = height
+      var context = canvas.getContext('2d')
+
+      // 填充颜色到canvas
+      var fill = rectElement.getAttribute('fill')
+      context.fillStyle = fill
+      context.fillRect(0, 0, width, height)
+
+      // 将canvas转换为base64格式
+      var base64 = canvas.toDataURL()
+
+      // 将数据转化为指定的JSON格式
+
+      var rectData = {
+        x: parseInt(x),
+        y: parseInt(y),
+        width: parseInt(width),
+        height: parseInt(height),
+        z_index: i + 1,
+        scale_option: 'width',
+        image: base64,
+        mask: base64,
+        type: 'base64'
+      }
+
+      // 将处理后的数据添加到数组中
+      data.push(rectData)
+    }
+  })
+
+  var svgWidth = svgElement.getAttribute('width')
+  var svgHeight = svgElement.getAttribute('height')
+  // 创建一个新的canvas元素
+  var canvas = document.createElement('canvas')
+  canvas.width = svgWidth
+  canvas.height = svgHeight
+  var context = canvas.getContext('2d')
+  // 绘制SVG到canvas
+  var svgString = new XMLSerializer().serializeToString(svgElement)
+  var DOMURL = window.URL || window.webkitURL || window
+
+  var svgBlob = new Blob([svgString], { type: 'image/svg+xml;charset=utf-8' })
+  var url = DOMURL.createObjectURL(svgBlob)
+
+  let img = await createImage(url)
+  context.drawImage(img, 0, 0)
+
+  let base64 = canvas.toDataURL()
+
+  var rectData = {
+    x: 0,
+    y: 0,
+    width: parseInt(svgWidth),
+    height: parseInt(svgHeight),
+    z_index: 0,
+    scale_option: 'width',
+    image: base64,
+    mask: base64,
+    type: 'base64'
+  }
+  data.push(rectData)
+
+  // 打印处理后的数据
+  console.log({ data, image: base64, svgElement })
+  return { data, image: base64, svgElement }
+}
+
+app.registerExtension({
+  name: 'Mixlab.layer.ShowLayer',
+  async getCustomWidgets (app) {
+    return {
+      EDIT (node, inputName, inputData, app) {
+        //   console.log('EditLayer##node', node,inputName, inputData)
+        const widget = {
+          type: inputData[0], // the type, CHEESE
+          name: inputName, // the name, slice
+          size: [128, 44], // a default size
+          draw (ctx, node, widget_width, y, widget_height) {
+            // console.log('EditLayer', this)
+            if (this.input)
+              Object.assign(
+                this.input.style,
+                get_position_style(ctx, widget_width, 32, node.size[1])
+              )
+          },
+          computeSize (...args) {
+            return [128, 44] // a method to compute the current size of the widget
+          },
+          async serializeValue (nodeId, widgetIndex) {
+            let d = getLocalData('_mixlab_edit_layer')
+            // console.log('EditLayer',d[node.id])
+            return d[node.id]
+          }
+        }
+        //  widget.something = something;          // maybe adds stuff to it
+        node.addCustomWidget(widget) // adds it to the node
+        return widget // and returns it.
+      }
+    }
+  },
+
+  async beforeRegisterNodeDef (nodeType, nodeData, app) {
+    if (nodeType.comfyClass == 'ShowLayer') {
+      const orig_nodeCreated = nodeType.prototype.onNodeCreated
+      nodeType.prototype.onNodeCreated = async function () {
+        orig_nodeCreated?.apply(this, arguments)
+
+        const findNode = nodeId => {
+          let node = app.graph._nodes_by_id[nodeId]
+          if (node?.type == 'Reroute') {
+            
+            let linkId =node.inputs.filter(i=>i.type=='*')[0].link
+            nodeId = app.graph.links.filter(link => link.id == linkId)[0]
+              ?.origin_id
+            return findNode(nodeId)
+          } else {
+            return nodeId
+          }
+        }
+
+        // 获取layers数据
+        const getLayers = async () => {
+          let linkId = this.inputs.filter(ip => ip.name === 'layers')[0].link
+          let nodeId = app.graph.links.filter(link => link.id == linkId)[0]
+            ?.origin_id
+
+          nodeId = findNode(nodeId)
+
+          //   let node = app.graph._nodes_by_id[nodeId]
+          //   if (node?.type == 'Reroute') {
+          //     linkId = node.inputs[0].link
+          //     nodeId = app.graph.links.filter(link => link.id == linkId)[0]
+          //       ?.origin_id
+          //   }
+         
+          let d = getLocalData('_mixlab_svg_image')
+          console.log('test',d[nodeId])
+
+          if (d[nodeId]) {
+            let url = d[nodeId]
+            let dt = await fetch(url)
+            let svgStr = await dt.text()
+            const { data } = (await parseSvg(svgStr)) || {}
+            return data
+          } else {
+            return []
+          }
+        }
+
+        // 修改layers数据
+        const setLayer = async (editIndex, layers = null) => {
+          //   let editIndex = 0
+          let lys = layers || (await getLayers())
+          let layer = lys[editIndex]
+          //   console.log(layer)
+
+          const updateValue = name => {
+            const x = this.widgets.filter(w => w.name == name)[0]
+            x.value = layer[name]
+          }
+          if (layer) {
+            Array.from(['x', 'y', 'width', 'height', 'z_index'], n =>
+              updateValue(n)
+            )
+          }
+        }
+
+        let that = this
+        const save_edit_layer_index = i => {
+          let data = getLocalData('_mixlab_edit_layer')
+          data[that.id] = i
+          localStorage.setItem('_mixlab_edit_layer', JSON.stringify(data))
+        }
+
+        await setLayer(0)
+        save_edit_layer_index(0)
+
+        const edit = this.widgets.filter(w => w.name == 'edit')[0]
+
+        edit.input = $el('div', {})
+        edit.input.style = `
+        display: flex;
+        flex-direction:row;
+        align-items: center;
+        margin-top: 0;`
+
+        const ip = $el('input', {})
+        ip.className = 'comfy-multiline-input'
+        ip.type = 'number'
+        ip.min = 0
+        ip.step = 1
+        ip.max = Math.max(0, (await getLayers()).length - 1)
+        // ip.className = `${'comfy-multiline-input'} `
+
+        ip.value = 0
+
+        ip.style = `
+        background-color: var(--comfy-input-bg);
+        color: var(--input-text);
+        outline: none;
+          border: none;
+          padding: 4px;
+          width: 60%;
+          cursor: pointer;
+          height: 24px;`
+        const label = document.createElement('label')
+        label.style = 'font-size: 10px;min-width:32px'
+        label.innerText = 'Layer Index'
+        edit.input.appendChild(label)
+        edit.input.appendChild(ip)
+
+        document.body.appendChild(edit.input)
+
+        ip.addEventListener('click', async event => {
+          console.log(await getLayers())
+          ip.max = Math.max(0, (await getLayers()).length - 1)
+        })
+
+        ip.addEventListener('change', async event => {
+          let index = ~~ip.value
+          let lys = await getLayers()
+          await setLayer(index, lys)
+          app.graph.setDirtyCanvas(true, true)
+          save_edit_layer_index(index)
+        })
+
+        // console.log('EditLayer nodeData', edit)
+
+        const onRemoved = this.onRemoved
+        this.onRemoved = () => {
+          edit.input.remove()
+          return onRemoved?.()
+        }
+
+        if (this.onResize) {
+          this.onResize(this.size)
+        }
+
+        this.serialize_widgets = false //需要保存参数
+      }
+    }
+  },
+  async loadedGraphNode (node, app) {
+    // Fires every time a node is constructed
+    // You can modify widgets/add handlers/etc here
+    if (node.type === 'SvgImage') {
+      let widget = node.widgets.filter(w => w.div)[0]
+      let data = getLocalData('_mixlab_svg_image')
+      let id = node.id
+
+      // widget.div.querySelector('.Svg').value = data[id] || '#000000'
+    }
+  }
+})

web/javascript/main_mixlab.js

@@ -106,24 +106,65 @@ async function uploadFile (file) {
 
 // alert(navigator.mediaDevices)
 
-async function shareScreen (webcamVideo, shareBtn, liveBtn, previewArea) {
+async function shareScreen (
+  isCamera = false,
+  webcamVideo,
+  shareBtn,
+  liveBtn,
+  previewArea
+) {
   try {
-    // let webcamVideo = document.createElement('video')
-    const mediaStream = await navigator.mediaDevices.getDisplayMedia({
-      video: true
-    })
+    let mediaStream
+
+    if (!isCamera) {
+      mediaStream = await navigator.mediaDevices.getDisplayMedia({
+        video: true
+      })
+    } else {
+      if (!localStorage.getItem('_mixlab_webcamera_select')) return
+      let constraints =
+        JSON.parse(localStorage.getItem('_mixlab_webcamera_select')) || {}
+      mediaStream = await navigator.mediaDevices.getUserMedia(constraints)
+    }
 
     webcamVideo.removeEventListener('timeupdate', videoTimeUpdateHandler)
     webcamVideo.srcObject = mediaStream
     webcamVideo.onloadedmetadata = () => {
+      let x = 0,
+        y = 0,
+        width = webcamVideo.videoWidth,
+        height = webcamVideo.videoHeight,
+        imgWidth = webcamVideo.videoWidth,
+        imgHeight = webcamVideo.videoHeight
+
+      let d = getSetAreaData()
+      if (
+        d &&
+        d.x >= 0 &&
+        d.imgWidth === imgWidth &&
+        d.imgHeight === imgHeight
+      ) {
+        x = d.x
+        y = d.y
+        width = d.width
+        height = d.height
+        imgWidth = d.imgWidth
+        imgHeight = d.imgHeight
+        console.log('#screen_share::使用上一次选区')
+      }
+      updateSetAreaData(x, y, width, height, imgWidth, imgHeight)
+
       webcamVideo.play()
+
+      createBlobFromVideo(webcamVideo, true)
+
       webcamVideo.addEventListener('timeupdate', videoTimeUpdateHandler)
 
-      window._mixlab_screen_x = 0
-      window._mixlab_screen_y = 0
-      // console.log(webcamVideo)
-      window._mixlab_screen_width = webcamVideo.videoWidth
-      window._mixlab_screen_height = webcamVideo.videoHeight
+      // window._mixlab_screen_x = 0
+      // window._mixlab_screen_y = 0
+      // // console.log(webcamVideo)
+      // window._mixlab_screen_width = webcamVideo.videoWidth
+      // window._mixlab_screen_height = webcamVideo.videoHeight
     }
 
     mediaStream.addEventListener('inactive', handleStopSharing)
@@ -278,7 +319,7 @@ async function startLive (btn) {
         previousImage,
         currentImage
       )
-      // console.log('#图片是否有变化:', imageChanged)
+      console.log('#图片是否有变化:', imageChanged)
 
       if (imageChanged) {
         window._mixlab_screen_imagePath = currentImage
@@ -328,26 +369,17 @@ async function createBlobFromVideoForArea (webcamVideo) {
   return blob
 }
 
-async function createBlobFromVideo (webcamVideo) {
+async function createBlobFromVideo (webcamVideo, updateImageBase64 = false) {
   const videoW = webcamVideo.videoWidth
   const videoH = webcamVideo.videoHeight
   const aspectRatio = videoW / videoH
-  const WIDTH = window._mixlab_screen_width,
-    HEIGHT = window._mixlab_screen_height
-  const canvas = new OffscreenCanvas(WIDTH, HEIGHT)
+
+  const { x, y, width, height } = window._mixlab_share_screen
+
+  const canvas = new OffscreenCanvas(width, height)
   const ctx = canvas.getContext('2d')
-  // console.log('#createBlobFromVideo', WIDTH, HEIGHT)
-  ctx.drawImage(
-    webcamVideo,
-    window._mixlab_screen_x,
-    window._mixlab_screen_y,
-    WIDTH,
-    HEIGHT,
-    0,
-    0,
-    WIDTH,
-    HEIGHT
-  )
+
+  ctx.drawImage(webcamVideo, x, y, width, height, 0, 0, width, height)
 
   const blob = await canvas.convertToBlob({
     type: 'image/jpeg',
@@ -356,12 +388,17 @@ async function createBlobFromVideo (webcamVideo) {
   // imgElement.src = await blobToBase64(blob)
   window._mixlab_screen_blob = blob
 
-  console.log('########')
-  // let currentImage = await blobToBase64(blob)
-  // // console.log(window._mixlab_screen_imagePath)
-  // if (!window._mixlab_screen_imagePath) {
-  //   window._mixlab_screen_imagePath = currentImage
-  // }
+  console.log(
+    '########updateImageBase64 ',
+    updateImageBase64,
+    x,
+    y,
+    width,
+    height
+  )
+  if (updateImageBase64) {
+    window._mixlab_screen_imagePath = await blobToBase64(blob)
+  }
 }
 
 async function blobToBase64 (blob) {
@@ -391,6 +428,36 @@ function base64ToBlob (base64) {
 
   return blob
 }
+
+async function requestCamera () {
+  // 请求授权
+  try {
+    let stream = await navigator.mediaDevices.getUserMedia({ video: true })
+    console.log('摄像头授权成功')
+    // 获取视频轨道
+    var videoTrack = stream.getVideoTracks()[0]
+
+    // 停止视频轨道
+    videoTrack.stop()
+
+    return true
+  } catch (error) {
+    // 用户拒绝授权或发生其他错误
+    console.error('摄像头授权失败：', error)
+
+    // 提示用户授权摄像头访问权限
+    if (error.name === 'NotAllowedError') {
+      alert('请授权摄像头访问权限 chrome://settings/content/camera')
+    } else {
+      alert('摄像头访问权限请求失败，请重试 chrome://settings/content/camera')
+    }
+
+    // // 跳转到浏览器的授权设置页面
+    // window.location.href = 'chrome://settings/content/camera'
+  }
+  return false
+}
+
 /* 
 A method that returns the required style for the html 
 */
@@ -432,7 +499,7 @@ const base64Df =
 app.registerExtension({
   name: 'Mixlab.image.ScreenShareNode',
   async getCustomWidgets (app) {
-    console.log('#Mixlab.image.ScreenShareNode', app)
+    // console.log('#Mixlab.image.ScreenShareNode', app)
     return {
       CHEESE (node, inputName, inputData, app) {
         // We return an object containing a field CHEESE which has a function (taking node, name, data, app)
@@ -473,6 +540,46 @@ app.registerExtension({
         // console.log('###widget', widget)
         node.addCustomWidget(widget) // adds it to the node
         return widget // and returns it.
+      },
+      SLIDE (node, inputName, inputData, app) {
+        // We return an object containing a field CHEESE which has a function (taking node, name, data, app)
+        const widget = {
+          type: inputData[0], // the type, CHEESE
+          name: inputName, // the name, slice
+          size: [128, 12], // a default size
+          draw (ctx, node, width, y) {
+            // a method to draw the widget (ctx is a CanvasRenderingContext2D)
+          },
+          computeSize (...args) {
+            return [128, 12] // a method to compute the current size of the widget
+          },
+          async serializeValue (nodeId, widgetIndex) {
+            return window._mixlab_screen_slide_input || 0.5
+          }
+        }
+        // console.log('###widget', widget)
+        node.addCustomWidget(widget) // adds it to the node
+        return widget // and returns it.
+      },
+      SEED (node, inputName, inputData, app) {
+        // We return an object containing a field CHEESE which has a function (taking node, name, data, app)
+        const widget = {
+          type: inputData[0], // the type, CHEESE
+          name: inputName, // the name, slice
+          size: [128, 12], // a default size
+          draw (ctx, node, width, y) {
+            // a method to draw the widget (ctx is a CanvasRenderingContext2D)
+          },
+          computeSize (...args) {
+            return [128, 12] // a method to compute the current size of the widget
+          },
+          async serializeValue (nodeId, widgetIndex) {
+            return window._mixlab_screen_seed_input || 0
+          }
+        }
+        // console.log('###widget', widget)
+        node.addCustomWidget(widget) // adds it to the node
+        return widget // and returns it.
       }
     }
   },
@@ -489,9 +596,15 @@ app.registerExtension({
           type: 'HTML', // whatever
           name: 'sreen_share', // whatever
           draw (ctx, node, widget_width, y, widget_height) {
+            console.log('ScreenSHare', y, widget_height)
             Object.assign(
               this.card.style,
-              get_position_style(ctx, widget_width, y, node.size[1])
+              get_position_style(
+                ctx,
+                widget_width,
+                widget_height * 5,
+                node.size[1]
+              )
             )
           }
         }
@@ -510,8 +623,15 @@ app.registerExtension({
         })
 
         widget.previewArea = $el('div', {
+          style: {}
+        })
+
+        widget.shareDiv = $el('div', {
+          // innerText: 'Share Screen',
           style: {
-            // position:'ab'
+            cursor: 'pointer',
+            fontWeight: '300',
+            display: 'flex'
           }
         })
 
@@ -521,7 +641,19 @@ app.registerExtension({
             cursor: 'pointer',
             padding: '8px 0',
             fontWeight: '300',
-            margin: '2px'
+            margin: '2px',
+            width: '100%'
+          }
+        })
+
+        widget.shareOfWebCamBtn = $el('button', {
+          innerText: 'Camera',
+          style: {
+            cursor: 'pointer',
+            padding: '8px 0',
+            fontWeight: '300',
+            margin: '2px',
+            width: '100%'
           }
         })
 
@@ -563,20 +695,30 @@ app.registerExtension({
         widget.previewCard.appendChild(widget.preview)
         widget.previewCard.appendChild(widget.previewArea)
 
-        widget.card.appendChild(widget.shareBtn)
+        widget.card.appendChild(widget.shareDiv)
+        widget.shareDiv.appendChild(widget.shareBtn)
+        widget.shareDiv.appendChild(widget.shareOfWebCamBtn)
         widget.card.appendChild(widget.openFloatingWinBtn)
         widget.card.appendChild(widget.refreshInput)
         widget.card.appendChild(widget.liveBtn)
 
-        widget.shareBtn.addEventListener('click', async () => {
+        const toggleShare = async (isCamera = false) => {
           if (widget.preview.paused) {
             window._mixlab_stopVideo = await shareScreen(
+              isCamera,
               widget.preview,
               widget.shareBtn,
               widget.liveBtn,
               widget.previewArea
             )
-            widget.shareBtn.innerText = 'Stop Share'
+
+            if (isCamera) {
+              widget.shareOfWebCamBtn.innerText = 'Stop Share'
+              widget.shareBtn.innerText = 'Stop'
+            } else {
+              widget.shareOfWebCamBtn.innerText = 'Stop'
+              widget.shareBtn.innerText = 'Stop Share'
+            }
 
             console.log('视频已暂停')
             if (window._mixlab_stopLive) {
@@ -584,12 +726,15 @@ app.registerExtension({
               window._mixlab_stopLive = null
               widget.liveBtn.innerText = 'Live Run'
             }
+
+            setTimeout(() => updateSetAreaDisplay(), 2000)
           } else {
             console.log('视频正在播放')
             if (window._mixlab_stopVideo) {
               window._mixlab_stopVideo()
               window._mixlab_stopVideo = null
               widget.shareBtn.innerText = 'Share Screen'
+              widget.shareOfWebCamBtn.innerText = 'Camera'
             }
             if (window._mixlab_stopLive) {
               window._mixlab_stopLive()
@@ -597,6 +742,121 @@ app.registerExtension({
               widget.liveBtn.innerText = 'Live Run'
             }
           }
+        }
+
+        // updateSetAreaDisplay(widget.previewArea, 200, 200)
+        widget.shareOfWebCamBtn.addEventListener('click', async () => {
+          if (!widget.preview.paused) {
+            if (window._mixlab_stopVideo) {
+              window._mixlab_stopVideo()
+              window._mixlab_stopVideo = null
+              widget.shareBtn.innerText = 'Share Screen'
+              widget.shareOfWebCamBtn.innerText = 'Camera'
+            }
+            if (window._mixlab_stopLive) {
+              window._mixlab_stopLive()
+              window._mixlab_stopLive = null
+              widget.liveBtn.innerText = 'Live Run'
+            }
+            return
+          }
+
+          let r = await requestCamera()
+          if (r === false) return
+          const devices = await navigator.mediaDevices.enumerateDevices()
+
+          // 查找摄像头设备
+          var cameras = devices.filter(function (device) {
+            // console.log(device)
+            return device.kind === 'videoinput'
+          })
+
+          // 创建 <select> 元素
+          var select = document.createElement('select')
+
+          // 创建默认选项
+          // var defaultOption = document.createElement('option')
+          // defaultOption.text = '请选择摄像头'
+          // defaultOption.disabled = true
+          // defaultOption.selected = true
+          // select.appendChild(defaultOption)
+
+          // 创建每个摄像头设备的选项
+          Array.from(cameras, (camera, i) => {
+            var option = document.createElement('option')
+            option.value = camera.deviceId
+            option.text = camera.label || 'Camera ' + (select.length - 1)
+            if (i === 0) option.selected = true
+            select.appendChild(option)
+          })
+
+          let modal = document.createElement('div')
+          modal.className = 'comfy-modal'
+          modal.style.display = 'flex'
+
+          let modalContent = document.createElement('div')
+          modalContent.className = 'comfy-modal-content'
+
+          let title = document.createElement('p')
+          title.innerText = 'Please select a camera'
+
+          modalContent.appendChild(title)
+          modalContent.appendChild(select)
+
+          let btns = document.createElement('div')
+          btns.style = `display: flex;
+          justify-content: space-between;
+          margin: 24px 0;`
+
+          let btn = document.createElement('button')
+          btn.innerText = 'OK'
+          btn.style = `width: 112px;`
+
+          let closeBtn = document.createElement('button')
+          closeBtn.innerText = 'Cancel'
+          closeBtn.style = `width: 112px;`
+
+          modalContent.appendChild(btns)
+          btns.appendChild(btn)
+          btns.appendChild(closeBtn)
+
+          modal.appendChild(modalContent)
+          document.body.appendChild(modal)
+
+          btn.addEventListener('click', () => {
+            // 获取所选择的选项的索引
+            var selectedIndex = select.selectedIndex
+
+            // 获取所选择的选项的值
+            var selectedValue = select.options[selectedIndex].value
+            if (selectedValue) {
+              const constraints = {
+                audio: false,
+                video: {
+                  width: { ideal: 1920, max: 1920 },
+                  height: { ideal: 1080, max: 1080 },
+                  deviceId: selectedValue
+                }
+              }
+
+              localStorage.setItem(
+                '_mixlab_webcamera_select',
+                JSON.stringify(constraints)
+              )
+
+              toggleShare(true)
+            }
+
+            modal.remove()
+          })
+
+          closeBtn.addEventListener('click', () => {
+            modal.remove()
+          })
+        })
+
+        widget.shareBtn.addEventListener('click', async () => {
+          toggleShare()
         })
 
         widget.refreshInput.addEventListener('change', async () => {
@@ -636,6 +896,8 @@ app.registerExtension({
         this.addCustomWidget(widget)
         this.onRemoved = function () {
           widget.preview.remove()
+          widget.shareDiv.remove()
+          widget.shareOfWebCamBtn.remove()
           widget.shareBtn.remove()
           widget.liveBtn.remove()
           widget.card.remove()
@@ -643,22 +905,114 @@ app.registerExtension({
           widget.previewArea.remove()
           widget.previewCard.remove()
         }
-        this.serialize_widgets = false
+        this.serialize_widgets = true
       }
     }
   }
 })
 
-function setArea (src) {
+function updateSetAreaDisplay () {
+  try {
+    let canvas = document.createElement('canvas')
+    canvas.width = window._mixlab_screen_webcamVideo.videoWidth
+    canvas.height = window._mixlab_screen_webcamVideo.videoHeight
+    let ctx = canvas.getContext('2d')
+    const lineWidth = 2 // Width of the stroke line
+    const strokeColor = 'red' // Color of the stroke
+
+    // Draw the rectangle
+    ctx.strokeStyle = strokeColor // Set the stroke color
+    ctx.lineWidth = lineWidth // Set the stroke line width
+
+    ctx.fillStyle = 'rgba(255,0,0,0.35)'
+
+    let x = 0,
+      y = 0,
+      width = canvas.width,
+      height = canvas.height
+
+    if (!window._mixlab_share_screen) {
+      let d = getSetAreaData()
+      if (d) {
+        window._mixlab_share_screen = d
+      }
+    }
+
+    if (window._mixlab_share_screen) {
+      x = window._mixlab_share_screen.x
+      y = window._mixlab_share_screen.y
+      width = window._mixlab_share_screen.width
+      height = window._mixlab_share_screen.height
+    }
+
+    ctx.strokeRect(x, y, width, height) // Draw the stroked rectangle
+    ctx.fillRect(x, y, width, height)
+
+    canvas.style.width = '100%'
+
+    let area = graph._nodes
+      .filter(n => n.type === 'ScreenShare')[0]
+      .widgets.filter(w => w.name == 'sreen_share')[0].previewArea
+
+    area.innerHTML = ''
+    area.appendChild(canvas)
+    area.style = `
+    position: absolute;
+    width:100%%;
+    left:0;
+    top:0;
+    `
+  } catch (error) {
+    console.log(error)
+  }
+}
+
+function updateSetAreaData (left, top, width, height, imgWidth, imgHeight) {
+  window._mixlab_share_screen = {
+    x: left,
+    y: top,
+    width,
+    height,
+    imgWidth,
+    imgHeight
+  }
+  localStorage.setItem(
+    '_mixlab_share_screen',
+    JSON.stringify(window._mixlab_share_screen)
+  )
+}
+
+function getSetAreaData () {
+  try {
+    let data = JSON.parse(localStorage.getItem('_mixlab_share_screen')) || {}
+    if (data.width === 0 || data.height === 0 || data.width === undefined)
+      return
+    return data
+  } catch (error) {}
+  return
+}
+
+async function setArea (src) {
+  let displayHeight = Math.round(window.screen.availHeight * 0.6)
   let div = document.createElement('div')
   div.innerHTML = `
-    <div id='ml_overlay' style='position: absolute;top:0'>
-      <img id='ml_video' style='position: absolute; width: 500px;   user-select: none; -webkit-user-drag: none;' />
-      <div id='ml_selection' style='position: absolute; border: 2px dashed red; pointer-events: none;'></div>
+    <div id='ml_overlay' style='position: absolute;top:0;background: #251f1fc4;
+    height: 100vh;
+    width: 100%;'>
+      <img id='ml_video' style='position: absolute; 
+      height: ${displayHeight}px;user-select: none; 
+      -webkit-user-drag: none;
+      outline: 2px solid #eaeaea;
+      box-shadow: 8px 9px 17px #575757;' />
+      <div id='ml_selection' style='position: absolute; 
+      border: 2px dashed red; 
+      pointer-events: none;'></div>
     </div>`
   // document.body.querySelector('#ml_overlay')
   document.body.appendChild(div)
 
+  let im = await createImage(src)
+
   let img = div.querySelector('#ml_video')
   let overlay = div.querySelector('#ml_overlay')
   let selection = div.querySelector('#ml_selection')
@@ -667,10 +1021,56 @@ function setArea (src) {
   // Set video source
   img.src = src
 
+  // init area
+  const data = getSetAreaData()
+  let x = 0,
+    y = 0,
+    width = (im.naturalWidth * displayHeight) / im.naturalHeight,
+    height = displayHeight
+  let imgWidth = im.naturalWidth
+  let imgHeight = im.naturalHeight
+  // console.log(
+  //   '#screen_share::使用上一次选区 selection',
+  //   data,
+  //   imgWidth,
+  //   img.width
+  // )
+  if (
+    data &&
+    data.width > 0 &&
+    data.height > 0 &&
+    data.imgWidth === imgWidth &&
+    data.imgHeight === imgHeight &&
+    data.imgHeight > 0
+  ) {
+    // 相同尺寸窗口，恢复选区
+    x = (img.width * data.x) / data.imgWidth
+    y = (img.height * data.y) / data.imgHeight
+    width = (img.width * data.width) / data.imgWidth
+    height = (img.height * data.height) / data.imgHeight
+    // imgWidth = data.imgWidth
+    // imgHeight = data.imgHeight;
+    // console.log('#screen_share::使用上一次选区 selection', x, y, width, height)
+  }
+
+  selection.style.left = x + 'px'
+  selection.style.top = y + 'px'
+  selection.style.width = width + 'px'
+  selection.style.height = height + 'px'
+
   // Add mouse events
   img.addEventListener('mousedown', startSelection)
   img.addEventListener('mousemove', updateSelection)
   img.addEventListener('mouseup', endSelection)
+  overlay.addEventListener('click', remove)
+
+  function remove () {
+    overlay.removeEventListener('click', remove)
+    img.removeEventListener('mousedown', startSelection)
+    img.removeEventListener('mousemove', updateSelection)
+    img.removeEventListener('mouseup', endSelection)
+    div.remove()
+  }
 
   function startSelection (event) {
     if (start == false) {
@@ -732,49 +1132,27 @@ function setArea (src) {
     // console.log('宽度: ' + width)
     // console.log('高度: ' + height)
 
-    img.removeEventListener('mousedown', startSelection)
-    img.removeEventListener('mousemove', updateSelection)
-    img.removeEventListener('mouseup', endSelection)
+    // img.removeEventListener('mousedown', startSelection)
+    // img.removeEventListener('mousemove', updateSelection)
+    // img.removeEventListener('mouseup', endSelection)
 
-    window._mixlab_screen_x = left
-    window._mixlab_screen_y = top
-    window._mixlab_screen_width = width
-    window._mixlab_screen_height = height
+    // window._mixlab_screen_x = left
+    // window._mixlab_screen_y = top
+    // window._mixlab_screen_width = width
+    // window._mixlab_screen_height = height
 
-    try {
-      let canvas = document.createElement('canvas')
-      canvas.width = _mixlab_screen_webcamVideo.videoWidth
-      canvas.height = _mixlab_screen_webcamVideo.videoHeight
-      let ctx = canvas.getContext('2d')
-      const lineWidth = 1 // Width of the stroke line
-      const strokeColor = 'red' // Color of the stroke
+    if (width <= 0 && height <= 0) return remove()
 
-      // Draw the rectangle
-      ctx.strokeStyle = strokeColor // Set the stroke color
-      ctx.lineWidth = lineWidth // Set the stroke line width
-      ctx.strokeRect(
-        _mixlab_screen_x,
-        _mixlab_screen_y,
-        _mixlab_screen_width,
-        _mixlab_screen_height
-      ) // Draw the stroked rectangle
+    updateSetAreaData(left, top, width, height, imgWidth, imgHeight)
 
-      canvas.style.width = '100%'
+    updateSetAreaDisplay()
 
-      let area = graph._nodes
-        .filter(n => n.type === 'ScreenShare')[0]
-        .widgets.filter(w => w.name == 'sreen_share')[0].previewArea
-      area.innerHTML = ''
-      area.appendChild(canvas)
-      area.style = `
-      position: absolute;
-      width:100%%;
-  left:0;
-  top:0;
-      `
-    } catch (error) {}
+    createBlobFromVideo(
+      window._mixlab_screen_webcamVideo,
+      !window._mixlab_screen_live
+    )
 
-    div.remove()
+    remove()
   }
 }
 
@@ -941,7 +1319,7 @@ app.registerExtension({
           outline: none;background: black;`
 
           let div = document.createElement('div')
-          div.style = `display:flex;position: fixed;
+          div.style = `display:flex;position: fixed;flex-direction: column;
           bottom: 0px;
           z-index: 9999;
           left: 0px;
@@ -949,15 +1327,42 @@ app.registerExtension({
           margin: 12px;`
 
           let inputDiv = document.createElement('div')
+          inputDiv.style = `width: 100%;`
+
           // inputDiv.style = ``
           let infoDiv = document.createElement('div')
-          infoDiv.style = `    width: 100%;
+          infoDiv.style = `width: 100%;
+          display: flex;
+          justify-content: space-between;
           height: 16px;
           color: white;
           margin-bottom: 4px;
           font-size: 12px;
-          text-shadow: 1px 1px gray;`
-          infoDiv.id = 'info'
+          text-shadow: gray 1px 1px;
+          align-items: center;`
+
+          let infoText = document.createElement('div')
+          infoText.id = 'info'
+
+          let hideBtn = document.createElement('button')
+          hideBtn.innerText = '🤖'
+          hideBtn.style = `
+          border: none;
+          background: none;
+          cursor: pointer; height: 24px; margin: 4px; color: red;`
+
+          hideBtn.addEventListener('click', () => {
+            if (fnDiv.style.display == 'none') {
+              fnDiv.style.display = 'flex'
+            } else {
+              fnDiv.style.display = 'none'
+            }
+            try {
+              pipWindow.document.querySelector('#info').innerText = ''
+            } catch (error) {
+              console.log(error)
+            }
+          })
 
           // Move the player to the Picture-in-Picture window.
           let input = document.createElement('textarea')
@@ -1002,21 +1407,25 @@ app.registerExtension({
           font-size: 16px;
           margin-right: 6px;user-select: none;`
 
-          let btn = document.createElement('butotn')
-          btn.innerText = '❤'
-          btn.style = `cursor: pointer;height: 24px;margin:4px;
+          let seedBtn = document.createElement('butotn')
+          seedBtn.innerText = '🎲'
+          seedBtn.style = `cursor: pointer;height: 24px;margin:4px;
           color: red;`
-          btn.addEventListener('click', () => {
-            if (input.style.display == 'none') {
-              input.style.display = 'block'
-            } else {
-              input.style.display = 'none'
-            }
+
+          seedBtn.addEventListener('click', () => {
+            window._mixlab_screen_seed_input = Math.round(
+              Math.floor(Math.random() * 0xffffffffffffffff)
+            )
+
             try {
-              pipWindow.document.querySelector('#info').innerText = ''
+              pipWindow.document.querySelector('#info').innerText =window._mixlab_screen_seed_input
             } catch (error) {
               console.log(error)
             }
+
+            // console.log(window._mixlab_screen_seed_input)
+            if (window._mixlab_screen_imagePath)
+              document.querySelector('#queue-button').click()
           })
 
           // TODO 需要判断是否有screenshare节点，没有的话，不需要添加
@@ -1074,7 +1483,9 @@ app.registerExtension({
             console.log('##更新Prompt')
             window._mixlab_screen_prompt =
               window._mixlab_screen_prompt_input || window._mixlab_screen_prompt
-            document.querySelector('#queue-button').click()
+
+            if (window._mixlab_screen_imagePath)
+              document.querySelector('#queue-button').click()
 
             try {
               pipWindow.document.querySelector('#info').innerText =
@@ -1119,16 +1530,52 @@ app.registerExtension({
 
           pipWindow.document.body.append(widget.preview)
           pipWindow.document.body.append(div)
+
+          // 滑动条
+          const createSlide = () => {
+            let d = document.createElement('div')
+            d.style = `width: 100%;margin-bottom: 12px;`
+            let range = document.createElement('input')
+            range.type = 'range'
+            d.appendChild(range)
+            return range
+          }
+
+          let slideInp = createSlide()
+          slideInp.addEventListener('change', () => {
+            console.log(~~slideInp.value / 100)
+            window._mixlab_screen_slide_input = ~~slideInp.value / 100
+            try {
+              pipWindow.document.querySelector('#info').innerText =
+                window._mixlab_screen_slide_input
+              if (window._mixlab_screen_imagePath)
+                document.querySelector('#queue-button').click()
+            } catch (error) {
+              console.log(error)
+            }
+          })
+
           // console.log(pipWindow)
 
-          div.appendChild(btnDiv)
-          btnDiv.appendChild(btn)
+          let fnDiv = document.createElement('div')
+          fnDiv.style = `display: flex;`
+
+          div.appendChild(infoDiv)
+          div.appendChild(fnDiv)
+
+          infoDiv.appendChild(infoText)
+          infoDiv.appendChild(hideBtn)
+
+          fnDiv.appendChild(btnDiv)
+          // 按钮区域
+          btnDiv.appendChild(seedBtn)
           btnDiv.appendChild(pauseBtn)
           btnDiv.appendChild(promptFinishBtn)
 
           // 输入框
-          div.appendChild(inputDiv)
-          inputDiv.appendChild(infoDiv)
+          fnDiv.appendChild(inputDiv)
+          inputDiv.appendChild(slideInp)
+
           inputDiv.appendChild(input)
 
           input.addEventListener('input', () => {
@@ -1152,7 +1599,9 @@ app.registerExtension({
                 window._mixlab_screen_prompt =
                   window._mixlab_screen_prompt_input ||
                   window._mixlab_screen_prompt
-                document.querySelector('#queue-button').click()
+
+                if (window._mixlab_screen_imagePath)
+                  document.querySelector('#queue-button').click()
 
                 try {
                   pipWindow.document.querySelector('#info').innerText =
@@ -1179,7 +1628,7 @@ app.registerExtension({
           widget.card.remove()
           widget.PictureInPicture.remove()
         }
-        this.serialize_widgets = false
+        this.serialize_widgets = true
       }
 
       const onExecuted = nodeType.prototype.onExecuted
@@ -1628,7 +2077,7 @@ const node = {
   },
   async setup (a) {
     for (const node of app.graph._nodes) {
-      console.log('#setup', node)
+      // console.log('#setup', node)
       if (node.type === 'RandomPrompt') {
         updateUI(node)
       }

web/javascript/ui_mixlab.js

@@ -1,19 +1,61 @@
 import { app } from '../../../scripts/app.js'
 
+function injectCSS(css) {
+  // 检查页面中是否已经存在具有相同内容的style标签
+  const existingStyle = document.querySelector('style');
+  if (existingStyle && existingStyle.textContent === css) {
+    return; // 如果已经存在相同的样式，则不进行注入
+  }
+
+  // 创建一个新的style标签，并将CSS内容注入其中
+  const style = document.createElement('style');
+  style.textContent = css;
+
+  // 将style标签插入到页面的head元素中
+  const head = document.querySelector('head');
+  head.appendChild(style);
+}
+
+injectCSS(`::-webkit-scrollbar {
+  width: 2px;
+}
+
+@keyframes loading_mixlab {
+  0% {
+    background-color: green;
+  }
+
+  50% {
+    background-color: lightgreen;
+  }
+
+  100% {
+    background-color: green;
+  }
+}
+
+.loading_mixlab {
+  background-color: green;
+  animation-name: loading_mixlab;
+  animation-duration: 2s;
+  animation-iteration-count: infinite;
+}`);
+
+
 async function getCustomnodeMappings (mode = 'url') {
   // mode = "local";
   let api_host = `${window.location.hostname}:${window.location.port}`
   let api_base = ''
   let url = `${window.location.protocol}//${api_host}${api_base}`
 
-  const response = await fetch(`${url}/customnode/getmappings?mode=${mode}`)
-  const data = await response.json()
-
   let nodes = {}
   try {
+    const response = await fetch(`${url}/customnode/getmappings?mode=${mode}`)
+    const data = await response.json()
     for (let url in data) {
       let n = data[url]
       for (let node of n[0]) {
+        // if(node=='CLIPSeg')console.log('#CLIPSeg',n)
         nodes[node] = { url, title: n[1].title_aux }
       }
     }
@@ -59,9 +101,10 @@ app.showMissingNodesError = async function (
   missingNodeTypes,
   hasAddedNodes = true
 ) {
+
   const nodesMap = await getCustomnodeMappings()
   console.log('#nodesMap', nodesMap)
-  //   console.log('###MIXLAB', missingNodeTypes, hasAddedNodes)
+    console.log('###MIXLAB', missingNodeTypes, hasAddedNodes)
   this.ui.dialog.show(
     `When loading the graph, the following node types were not found: <ul>${missingNodeGithub(
       missingNodeTypes,

web/javascript/utils_mixlab.js

@@ -0,0 +1,166 @@
+
+import { app } from '../../../scripts/app.js'
+import { api } from '../../../scripts/api.js'
+import { ComfyWidgets } from '../../../scripts/widgets.js'
+import { $el } from '../../../scripts/ui.js'
+
+
+const getLocalData = key => {
+  let data = {}
+  try {
+    data = JSON.parse(localStorage.getItem(key)) || {}
+  } catch (error) {
+    return {}
+  }
+  return data
+}
+function get_position_style (ctx, widget_width, y, node_height) {
+  const MARGIN = 4 // the margin around the html element
+
+  /* Create a transform that deals with all the scrolling and zooming */
+  const elRect = ctx.canvas.getBoundingClientRect()
+  const transform = new DOMMatrix()
+    .scaleSelf(
+      elRect.width / ctx.canvas.width,
+      elRect.height / ctx.canvas.height
+    )
+    .multiplySelf(ctx.getTransform())
+    .translateSelf(MARGIN, MARGIN + y)
+
+  return {
+    transformOrigin: '0 0',
+    transform: transform,
+    left: `0`,
+    top: `0`,
+    cursor: 'pointer',
+    position: 'absolute',
+    maxWidth: `${widget_width - MARGIN * 2}px`,
+    // maxHeight: `${node_height - MARGIN * 2}px`, // we're assuming we have the whole height of the node
+    width: `${widget_width - MARGIN * 2}px`,
+    // height: `${node_height * 0.3 - MARGIN * 2}px`,
+    // background: '#EEEEEE',
+    display: 'flex',
+    flexDirection: 'column',
+    // alignItems: 'center',
+    justifyContent: 'space-around'
+  }
+}
+
+app.registerExtension({
+    name: 'Mixlab.utils.Color',
+    async getCustomWidgets (app) {
+      return {
+        TCOLOR (node, inputName, inputData, app) {
+          // console.log('##node', node)
+          const widget = {
+            type: inputData[0], // the type, CHEESE
+            name: inputName, // the name, slice
+            size: [128, 32], // a default size
+            draw (ctx, node, width, y) {},
+            computeSize (...args) {
+              return [128, 32] // a method to compute the current size of the widget
+            },
+            async serializeValue (nodeId, widgetIndex) {
+              let data = getLocalData('_mixlab_utils_color')
+              return data[node.id] || '#000000'
+            }
+          }
+          //  widget.something = something;          // maybe adds stuff to it
+          node.addCustomWidget(widget) // adds it to the node
+          return widget // and returns it.
+        }
+      }
+    },
+  
+    async beforeRegisterNodeDef (nodeType, nodeData, app) {
+      if (nodeType.comfyClass == 'Color') {
+        const orig_nodeCreated = nodeType.prototype.onNodeCreated
+        nodeType.prototype.onNodeCreated = function () {
+          orig_nodeCreated?.apply(this, arguments)
+  
+          console.log('Color nodeData', this.widgets)
+   
+          const widget = {
+            type: 'div',
+            name: 'input_color',
+            draw (ctx, node, widget_width, y, widget_height) {
+              Object.assign(
+                this.div.style,
+                get_position_style(
+                  ctx,
+                  widget_width,
+                  44,
+                  node.size[1]
+                )
+              )
+            }
+          }
+  
+          widget.div = $el('div', {})
+  
+          document.body.appendChild(widget.div)
+  
+          const inputDiv = (key, placeholder, value) => {
+            let div = document.createElement('div')
+            const ip = document.createElement('input')
+            ip.type = 'color'
+            ip.className = `${'comfy-multiline-input'} ${placeholder}`
+            div.style = `display: flex;
+            align-items: center; 
+            margin: 6px 8px;
+            margin-top: 0;`
+            ip.placeholder = placeholder
+            ip.value = value
+  
+            ip.style = `outline: none;
+            border: none;
+            padding: 4px;
+            width: 100%;cursor: pointer;
+            height: 32px;`
+            const label = document.createElement('label')
+            label.style = 'font-size: 10px;min-width:32px'
+            label.innerText = placeholder
+            div.appendChild(label)
+            div.appendChild(ip)
+  
+            ip.addEventListener('change', () => {
+              let data = getLocalData(key)
+              data[this.id] = ip.value.trim()
+              localStorage.setItem(key, JSON.stringify(data))
+              // console.log(this.id, ip.value.trim())
+            })
+            return div
+          }
+  
+          let inputColor = inputDiv('_mixlab_utils_color', 'Color', '#000000')
+  
+          widget.div.appendChild(inputColor)
+  
+          this.addCustomWidget(widget)
+  
+          const onRemoved = this.onRemoved
+          this.onRemoved = () => {
+            inputColor.remove()
+            widget.div.remove()
+            return onRemoved?.()
+          }
+  
+          this.serialize_widgets = true //需要保存参数
+        }
+      }
+    },
+    async loadedGraphNode (node, app) {
+      // Fires every time a node is constructed
+      // You can modify widgets/add handlers/etc here
+  
+      if (node.type === 'Color') {
+        let widget = node.widgets.filter(w => w.div)[0]
+  
+        let data = getLocalData('_mixlab_utils_color')
+  
+        let id = node.id
+  
+        widget.div.querySelector('.Color').value = data[id] || '#000000'
+      }
+    }
+  })

web/javascript/watcher_mixlab.js

@@ -169,7 +169,7 @@ app.registerExtension({
         this.onRemoved = function () {
           // widget.card.remove()
         }
-        this.serialize_widgets = false
+        this.serialize_widgets = true
       }
     }
   }

web/main.css

@@ -1,3 +1,24 @@
 ::-webkit-scrollbar {
-    width: 2px;
-  }
+  width: 2px;
+}
+
+@keyframes loading_mixlab {
+  0% {
+    background-color: green;
+  }
+
+  50% {
+    background-color: lightgreen;
+  }
+
+  100% {
+    background-color: green;
+  }
+}
+
+.loading_mixlab {
+  background-color: green;
+  animation-name: loading_mixlab;
+  animation-duration: 2s;
+  animation-iteration-count: infinite;
+}

workflow/3-FloatVideo-workflow.json

@@ -1,6 +1,6 @@
 {
-  "last_node_id": 22,
-  "last_link_id": 48,
+  "last_node_id": 23,
+  "last_link_id": 51,
   "nodes": [
     {
       "id": 7,
@@ -105,7 +105,7 @@
         {
           "name": "text",
           "type": "STRING",
-          "link": 48,
+          "link": 51,
           "widget": {
             "name": "text"
           }
@@ -316,13 +316,13 @@
         "1": 246
       },
       "flags": {},
-      "order": 6,
+      "order": 7,
       "mode": 0,
       "inputs": [
         {
           "name": "images",
           "type": "IMAGE",
-          "link": 46
+          "link": 50
         }
       ],
       "properties": {
@@ -445,13 +445,13 @@
         "1": 58
       },
       "flags": {},
-      "order": 7,
+      "order": 6,
       "mode": 0,
       "inputs": [
         {
           "name": "image",
           "type": "IMAGE",
-          "link": 47
+          "link": 49
         }
       ],
       "outputs": [
@@ -472,45 +472,6 @@
         512
       ]
     },
-    {
-      "id": 22,
-      "type": "ScreenShare",
-      "pos": [
-        -63,
-        483
-      ],
-      "size": {
-        "0": 315,
-        "1": 98
-      },
-      "flags": {},
-      "order": 3,
-      "mode": 0,
-      "outputs": [
-        {
-          "name": "IMAGE",
-          "type": "IMAGE",
-          "links": [
-            46,
-            47
-          ],
-          "shape": 3,
-          "slot_index": 0
-        },
-        {
-          "name": "STRING",
-          "type": "STRING",
-          "links": [
-            48
-          ],
-          "shape": 3,
-          "slot_index": 1
-        }
-      ],
-      "properties": {
-        "Node name for S&R": "ScreenShare"
-      }
-    },
     {
       "id": 20,
       "type": "FloatingVideo",
@@ -518,10 +479,10 @@
         1928,
         295
       ],
-      "size": [
-        315,
-        58
-      ],
+      "size": {
+        "0": 315,
+        "1": 58
+      },
       "flags": {},
       "order": 12,
       "mode": 0,
@@ -534,7 +495,68 @@
       ],
       "properties": {
         "Node name for S&R": "FloatingVideo"
-      }
+      },
+      "widgets_values": [
+        null
+      ]
+    },
+    {
+      "id": 23,
+      "type": "ScreenShare",
+      "pos": [
+        -65,
+        446
+      ],
+      "size": {
+        "0": 315,
+        "1": 170
+      },
+      "flags": {},
+      "order": 3,
+      "mode": 0,
+      "outputs": [
+        {
+          "name": "IMAGE",
+          "type": "IMAGE",
+          "links": [
+            49,
+            50
+          ],
+          "shape": 3,
+          "slot_index": 0
+        },
+        {
+          "name": "PROMPT",
+          "type": "STRING",
+          "links": [
+            51
+          ],
+          "shape": 3,
+          "slot_index": 1
+        },
+        {
+          "name": "FLOAT",
+          "type": "FLOAT",
+          "links": null,
+          "shape": 3
+        },
+        {
+          "name": "INT",
+          "type": "INT",
+          "links": null,
+          "shape": 3
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "ScreenShare"
+      },
+      "widgets_values": [
+        null,
+        null,
+        null,
+        null,
+        null
+      ]
     }
   ],
   "links": [
@@ -787,24 +809,24 @@
       "CLIP"
     ],
     [
-      46,
-      22,
-      0,
-      2,
-      0,
-      "IMAGE"
-    ],
-    [
-      47,
-      22,
+      49,
+      23,
       0,
       18,
       0,
       "IMAGE"
     ],
     [
-      48,
-      22,
+      50,
+      23,
+      0,
+      2,
+      0,
+      "IMAGE"
+    ],
+    [
+      51,
+      23,
       1,
       8,
       1,

workflow/5-gpt-workflow.json

@@ -1,6 +1,6 @@
 {
-  "last_node_id": 34,
-  "last_link_id": 35,
+  "last_node_id": 47,
+  "last_link_id": 46,
   "nodes": [
     {
       "id": 27,
@@ -14,7 +14,7 @@
         "1": 164.31304931640625
       },
       "flags": {},
-      "order": 7,
+      "order": 5,
       "mode": 0,
       "inputs": [
         {
@@ -25,7 +25,7 @@
         {
           "name": "text",
           "type": "STRING",
-          "link": 35,
+          "link": 46,
           "widget": {
             "name": "text"
           }
@@ -136,7 +136,7 @@
         "Node name for S&R": "KSampler"
       },
       "widgets_values": [
-        264070254513477,
+        971428736321335,
         "randomize",
         15,
         8,
@@ -294,26 +294,26 @@
       }
     },
     {
-      "id": 33,
-      "type": "SessionHistory",
+      "id": 44,
+      "type": "ShowTextForGPT",
       "pos": [
-        1300,
-        430
-      ],
-      "size": [
-        580.1640810546876,
-        223.1105875244142
+        1171,
+        425
       ],
+      "size": {
+        "0": 432.46002197265625,
+        "1": 264.40771484375
+      },
       "flags": {},
       "order": 6,
       "mode": 0,
       "inputs": [
         {
-          "name": "session_history",
+          "name": "text",
           "type": "STRING",
-          "link": 33,
+          "link": 44,
           "widget": {
-            "name": "session_history"
+            "name": "text"
           }
         }
       ],
@@ -322,38 +322,37 @@
           "name": "STRING",
           "type": "STRING",
           "links": null,
-          "shape": 6,
-          "slot_index": 0
+          "shape": 6
         }
       ],
       "properties": {
-        "Node name for S&R": "SessionHistory"
+        "Node name for S&R": "ShowTextForGPT"
       },
       "widgets_values": [
-        "[\n  {\n    \"role\": \"user\",\n    \"content\": \"黑客与画家\"\n  },\n  {\n    \"role\": \"assistant\",\n    \"content\": \"\\\"Hacker and Painter\\\" is a popular Chinese comic series written and illustrated by Cheng先辈， which has gained a large following worldwide.\\n\\nThe series revolves around the lives of two main characters, a young hacker and an artist, who embark on a journey of self-discovery and personal growth. Set in a futuristic, dystopian world, the story combines elements of science fiction, technology, and action with a touch of humor and satire.\\n\\nThe art style in \\\"Hacker and Painter\\\" is a mix of sci-fi and traditional Chinese ink painting, with bold lines, vibrant colors, and intricate details. Cheng先辈 employs a unique painting technique, using watercolor washes and ink to create a dynamic, expressive, and visually striking visual style.\\n\\nThe artist's use of photography and various camera angles adds an extra layer of depth to the storytelling, allowing the viewer to experience the world from different perspectives and gain a deeper understanding of the characters' emotions and motivations.\\n\\nLighting plays a significant role in the series, with the use of dramatic, contrasting light sources adding to the intensity and emotion of various scenes. Cheng先辈 expertly utilizes lighting to enhance the mood and setting, drawing the audience into the world of \\\"Hacker and Painter\\\" and making them feel like they are part of the action.\\n\\nOverall, \\\"Hacker and Painter\\\" is a visually stunning and thought-provoking series that combines the best of science fiction, action, and art, making it a must-read for fans of all ages.\"\n  }\n]"
+        "[\n    {\n        \"role\": \"system\",\n        \"content\": \"You are ChatGPT, a large language model trained by OpenAI. Answer as concisely as possible.\"\n    },\n    {\n        \"role\": \"user\",\n        \"content\": \"\"\n    }\n]"
       ]
     },
     {
-      "id": 34,
-      "type": "SessionHistory",
+      "id": 43,
+      "type": "ShowTextForGPT",
       "pos": [
-        1300,
-        230
-      ],
-      "size": [
-        562.8610810546875,
-        125.32558752441412
+        1166,
+        740
       ],
+      "size": {
+        "0": 424.0079650878906,
+        "1": 430.6391296386719
+      },
       "flags": {},
-      "order": 5,
+      "order": 7,
       "mode": 0,
       "inputs": [
         {
-          "name": "session_history",
+          "name": "text",
           "type": "STRING",
-          "link": 34,
+          "link": 45,
           "widget": {
-            "name": "session_history"
+            "name": "text"
           }
         }
       ],
@@ -361,100 +360,114 @@
         {
           "name": "STRING",
           "type": "STRING",
-          "links": [
-            35
-          ],
-          "shape": 6,
-          "slot_index": 0
+          "links": null,
+          "shape": 6
         }
       ],
       "properties": {
-        "Node name for S&R": "SessionHistory"
+        "Node name for S&R": "ShowTextForGPT"
       },
       "widgets_values": [
-        "\"Hacker and Painter\" is a popular Chinese comic series written and illustrated by Cheng先辈， which has gained a large following worldwide.\n\nThe series revolves around the lives of two main characters, a young hacker and an artist, who embark on a journey of self-discovery and personal growth. Set in a futuristic, dystopian world, the story combines elements of science fiction, technology, and action with a touch of humor and satire.\n\nThe art style in \"Hacker and Painter\" is a mix of sci-fi and traditional Chinese ink painting, with bold lines, vibrant colors, and intricate details. Cheng先辈 employs a unique painting technique, using watercolor washes and ink to create a dynamic, expressive, and visually striking visual style.\n\nThe artist's use of photography and various camera angles adds an extra layer of depth to the storytelling, allowing the viewer to experience the world from different perspectives and gain a deeper understanding of the characters' emotions and motivations.\n\nLighting plays a significant role in the series, with the use of dramatic, contrasting light sources adding to the intensity and emotion of various scenes. Cheng先辈 expertly utilizes lighting to enhance the mood and setting, drawing the audience into the world of \"Hacker and Painter\" and making them feel like they are part of the action.\n\nOverall, \"Hacker and Painter\" is a visually stunning and thought-provoking series that combines the best of science fiction, action, and art, making it a must-read for fans of all ages."
+        "[\n    {\n        \"role\": \"user\",\n        \"content\": \"\"\n    },\n    {\n        \"role\": \"assistant\",\n        \"content\": \"I'm sorry, I'm ChatGLM3-6B, not ChatGPT. I am a language model jointly trained by Tsinghua University KEG Lab and Zhipu AI Company.\"\n    }\n]"
       ]
     },
     {
-      "id": 32,
-      "type": "PrimitiveNode",
+      "id": 45,
+      "type": "ShowTextForGPT",
       "pos": [
-        1314,
-        1075
+        1092,
+        262
       ],
       "size": {
-        "0": 571.9803466796875,
-        "1": 100.6700210571289
-      },
-      "flags": {},
-      "order": 2,
-      "mode": 0,
-      "outputs": [
-        {
-          "name": "STRING",
-          "type": "STRING",
-          "links": [
-            32
-          ],
-          "widget": {
-            "name": "prompt"
-          }
-        }
-      ],
-      "title": "prompt",
-      "properties": {},
-      "widgets_values": [
-        "黑客与画家"
-      ]
-    },
-    {
-      "id": 10,
-      "type": "ChatGPT",
-      "pos": [
-        1300,
-        720
-      ],
-      "size": {
-        "0": 586.7649536132812,
-        "1": 301.5857849121094
+        "0": 635.8358154296875,
+        "1": 101.46092224121094
       },
       "flags": {},
       "order": 4,
       "mode": 0,
       "inputs": [
         {
-          "name": "prompt",
+          "name": "text",
           "type": "STRING",
-          "link": 32,
+          "link": 43,
           "widget": {
-            "name": "prompt"
-          },
-          "slot_index": 0
+            "name": "text"
+          }
         }
       ],
       "outputs": [
         {
           "name": "STRING",
           "type": "STRING",
+          "links": null,
+          "shape": 6
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "ShowTextForGPT"
+      },
+      "widgets_values": [
+        "I'm sorry, I'm ChatGLM3-6B, not ChatGPT. I am a language model jointly trained by Tsinghua University KEG Lab and Zhipu AI Company."
+      ]
+    },
+    {
+      "id": 47,
+      "type": "ChatGPTOpenAI",
+      "pos": [
+        585,
+        306
+      ],
+      "size": {
+        "0": 400,
+        "1": 342
+      },
+      "flags": {},
+      "order": 2,
+      "mode": 0,
+      "outputs": [
+        {
+          "name": "text",
+          "type": "STRING",
           "links": [
-            34
+            43,
+            46
           ],
           "shape": 3,
           "slot_index": 0
         },
         {
-          "name": "JSON",
+          "name": "messages",
           "type": "STRING",
           "links": [
-            33
+            44
           ],
           "shape": 3,
           "slot_index": 1
+        },
+        {
+          "name": "session_history",
+          "type": "STRING",
+          "links": [
+            45
+          ],
+          "shape": 3,
+          "slot_index": 2
         }
       ],
       "properties": {
-        "Node name for S&R": "ChatGPT"
-      }
+        "Node name for S&R": "ChatGPTOpenAI"
+      },
+      "widgets_values": [
+        null,
+        null,
+        "",
+        "You are ChatGPT, a large language model trained by OpenAI. Answer as concisely as possible.",
+        "gpt-3.5-turbo-16k",
+        2220,
+        "randomize",
+        1,
+        null
+      ]
     }
   ],
   "links": [
@@ -579,32 +592,32 @@
       "IMAGE"
     ],
     [
-      32,
-      32,
+      43,
+      47,
       0,
-      10,
+      45,
       0,
       "STRING"
     ],
     [
-      33,
-      10,
+      44,
+      47,
       1,
-      33,
+      44,
       0,
       "STRING"
     ],
     [
-      34,
-      10,
-      0,
-      34,
+      45,
+      47,
+      2,
+      43,
       0,
       "STRING"
     ],
     [
-      35,
-      34,
+      46,
+      47,
       0,
       27,
       1,

workflow/6-gpt-all-workflow.json

@@ -0,0 +1,251 @@
+{
+  "last_node_id": 5,
+  "last_link_id": 4,
+  "nodes": [
+    {
+      "id": 4,
+      "type": "ChatGPTOpenAI",
+      "pos": [
+        -512,
+        -236
+      ],
+      "size": {
+        "0": 400,
+        "1": 342
+      },
+      "flags": {},
+      "order": 0,
+      "mode": 0,
+      "outputs": [
+        {
+          "name": "text",
+          "type": "STRING",
+          "links": [
+            2
+          ],
+          "shape": 3,
+          "slot_index": 0
+        },
+        {
+          "name": "messages",
+          "type": "STRING",
+          "links": null,
+          "shape": 3
+        },
+        {
+          "name": "session_history",
+          "type": "STRING",
+          "links": null,
+          "shape": 3
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "ChatGPTOpenAI"
+      },
+      "widgets_values": [
+        null,
+        null,
+        "描述一个科幻的场景",
+        "You are ChatGPT, a large language model trained by OpenAI. Answer as concisely as possible.",
+        "gpt-35-turbo",
+        6933,
+        "randomize",
+        1,
+        null
+      ]
+    },
+    {
+      "id": 2,
+      "type": "ChatGPTOpenAI",
+      "pos": [
+        -36,
+        -234
+      ],
+      "size": {
+        "0": 400,
+        "1": 342
+      },
+      "flags": {},
+      "order": 1,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "prompt",
+          "type": "STRING",
+          "link": 2,
+          "widget": {
+            "name": "prompt"
+          }
+        }
+      ],
+      "outputs": [
+        {
+          "name": "text",
+          "type": "STRING",
+          "links": [
+            3
+          ],
+          "shape": 3,
+          "slot_index": 0
+        },
+        {
+          "name": "messages",
+          "type": "STRING",
+          "links": null,
+          "shape": 3
+        },
+        {
+          "name": "session_history",
+          "type": "STRING",
+          "links": null,
+          "shape": 3
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "ChatGPTOpenAI"
+      },
+      "widgets_values": [
+        null,
+        null,
+        "",
+        "增加丰富的细节和光影，摄影技巧，镜头语言，材质肌理",
+        "gpt-3.5-turbo",
+        2836,
+        "randomize",
+        1,
+        null
+      ]
+    },
+    {
+      "id": 5,
+      "type": "ChatGPTOpenAI",
+      "pos": [
+        -30,
+        183
+      ],
+      "size": {
+        "0": 400,
+        "1": 342
+      },
+      "flags": {},
+      "order": 2,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "prompt",
+          "type": "STRING",
+          "link": 3,
+          "widget": {
+            "name": "prompt"
+          }
+        }
+      ],
+      "outputs": [
+        {
+          "name": "text",
+          "type": "STRING",
+          "links": [
+            4
+          ],
+          "shape": 3,
+          "slot_index": 0
+        },
+        {
+          "name": "messages",
+          "type": "STRING",
+          "links": null,
+          "shape": 3
+        },
+        {
+          "name": "session_history",
+          "type": "STRING",
+          "links": null,
+          "shape": 3
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "ChatGPTOpenAI"
+      },
+      "widgets_values": [
+        null,
+        null,
+        "",
+        "翻译成英文，并按照格式输出: 画面、主题、细节、灯光、氛围、艺术家、其他",
+        "gpt-3.5-turbo",
+        1085,
+        "randomize",
+        1,
+        null
+      ]
+    },
+    {
+      "id": 3,
+      "type": "ShowTextForGPT",
+      "pos": [
+        447,
+        -229
+      ],
+      "size": [
+        503.79851499517997,
+        356.0560985581077
+      ],
+      "flags": {},
+      "order": 3,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "text",
+          "type": "STRING",
+          "link": 4,
+          "widget": {
+            "name": "text"
+          }
+        }
+      ],
+      "outputs": [
+        {
+          "name": "STRING",
+          "type": "STRING",
+          "links": null,
+          "shape": 6
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "ShowTextForGPT"
+      },
+      "widgets_values": [
+        "Picture: The picture is composed of carefully chosen elements, capturing the subject matter in a visually striking way.\n\nTheme: The theme of the photograph could vary, from capturing nature's beauty to showcasing urban landscapes, human emotions, or abstract concepts.\n\nDetails: The photograph captures intricate details, bringing attention to the subject's textures, colors, shapes, and patterns.\n\nLighting: The photographer manipulates lighting, using techniques like natural light, dramatic shadows, or artificial lighting to enhance the mood and atmosphere of the photograph.\n\nAmbiance: The photograph evokes a specific ambiance or mood, whether it's serene, mysterious, joyful, melancholic, or any other emotional response.\n\nArtist: The photographer skillfully crafts the image, demonstrating their artistic vision, technical skills, and creative expression through the composition, framing, and post-processing choices.\n\nOthers: Apart from the elements mentioned above, the photograph may also incorporate other creative techniques like long exposure, multiple exposures, color grading, or unconventional perspectives to create a unique and captivating image.\n\nIn the future, as technology and imagination continue to advance, photography will likely continue to evolve and innovate, offering even more realistic and awe-inspiring visual experiences for humans."
+      ]
+    }
+  ],
+  "links": [
+    [
+      2,
+      4,
+      0,
+      2,
+      0,
+      "STRING"
+    ],
+    [
+      3,
+      2,
+      0,
+      5,
+      0,
+      "STRING"
+    ],
+    [
+      4,
+      5,
+      0,
+      3,
+      0,
+      "STRING"
+    ]
+  ],
+  "groups": [],
+  "config": {},
+  "extra": {},
+  "version": 0.4
+}

workflow/7-camera-workflow.json

@@ -0,0 +1,731 @@
+{
+  "last_node_id": 24,
+  "last_link_id": 59,
+  "nodes": [
+    {
+      "id": 7,
+      "type": "CLIPTextEncode",
+      "pos": [
+        515,
+        130
+      ],
+      "size": {
+        "0": 400,
+        "1": 200
+      },
+      "flags": {},
+      "order": 2,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "clip",
+          "type": "CLIP",
+          "link": 33
+        }
+      ],
+      "outputs": [
+        {
+          "name": "CONDITIONING",
+          "type": "CONDITIONING",
+          "links": [
+            29
+          ],
+          "shape": 3,
+          "slot_index": 0
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "CLIPTextEncode"
+      },
+      "widgets_values": [
+        ""
+      ]
+    },
+    {
+      "id": 8,
+      "type": "CLIPTextEncode",
+      "pos": [
+        515,
+        1268
+      ],
+      "size": {
+        "0": 400,
+        "1": 200
+      },
+      "flags": {},
+      "order": 6,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "clip",
+          "type": "CLIP",
+          "link": 32
+        },
+        {
+          "name": "text",
+          "type": "STRING",
+          "link": 59,
+          "widget": {
+            "name": "text"
+          }
+        }
+      ],
+      "outputs": [
+        {
+          "name": "CONDITIONING",
+          "type": "CONDITIONING",
+          "links": [
+            51
+          ],
+          "shape": 3,
+          "slot_index": 0
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "CLIPTextEncode"
+      },
+      "widgets_values": [
+        ""
+      ]
+    },
+    {
+      "id": 6,
+      "type": "LoraLoader",
+      "pos": [
+        515,
+        460
+      ],
+      "size": {
+        "0": 315,
+        "1": 126
+      },
+      "flags": {},
+      "order": 3,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "model",
+          "type": "MODEL",
+          "link": 31
+        },
+        {
+          "name": "clip",
+          "type": "CLIP",
+          "link": 45
+        }
+      ],
+      "outputs": [
+        {
+          "name": "MODEL",
+          "type": "MODEL",
+          "links": [
+            44
+          ],
+          "shape": 3,
+          "slot_index": 0
+        },
+        {
+          "name": "CLIP",
+          "type": "CLIP",
+          "links": null,
+          "shape": 3
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "LoraLoader"
+      },
+      "widgets_values": [
+        "lcm-lora-sdv1-5.safetensors",
+        1,
+        1
+      ]
+    },
+    {
+      "id": 5,
+      "type": "KSampler",
+      "pos": [
+        1015,
+        130
+      ],
+      "size": {
+        "0": 315,
+        "1": 262
+      },
+      "flags": {},
+      "order": 7,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "model",
+          "type": "MODEL",
+          "link": 44
+        },
+        {
+          "name": "positive",
+          "type": "CONDITIONING",
+          "link": 51
+        },
+        {
+          "name": "negative",
+          "type": "CONDITIONING",
+          "link": 29
+        },
+        {
+          "name": "latent_image",
+          "type": "LATENT",
+          "link": 54
+        },
+        {
+          "name": "denoise",
+          "type": "FLOAT",
+          "link": 56,
+          "widget": {
+            "name": "denoise"
+          }
+        }
+      ],
+      "outputs": [
+        {
+          "name": "LATENT",
+          "type": "LATENT",
+          "links": [
+            35
+          ],
+          "shape": 3,
+          "slot_index": 0
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "KSampler"
+      },
+      "widgets_values": [
+        758428944049342,
+        "fixed",
+        4,
+        1.6,
+        "lcm",
+        "karras",
+        1
+      ]
+    },
+    {
+      "id": 11,
+      "type": "VAEDecode",
+      "pos": [
+        1430,
+        130
+      ],
+      "size": {
+        "0": 210,
+        "1": 46
+      },
+      "flags": {},
+      "order": 8,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "samples",
+          "type": "LATENT",
+          "link": 35
+        },
+        {
+          "name": "vae",
+          "type": "VAE",
+          "link": 34
+        }
+      ],
+      "outputs": [
+        {
+          "name": "IMAGE",
+          "type": "IMAGE",
+          "links": [
+            41
+          ],
+          "shape": 3,
+          "slot_index": 0
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "VAEDecode"
+      }
+    },
+    {
+      "id": 20,
+      "type": "FloatingVideo",
+      "pos": [
+        1740,
+        130
+      ],
+      "size": {
+        "0": 315,
+        "1": 58
+      },
+      "flags": {},
+      "order": 9,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "images",
+          "type": "IMAGE",
+          "link": 41
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "FloatingVideo"
+      },
+      "widgets_values": [
+        null
+      ]
+    },
+    {
+      "id": 2,
+      "type": "PreviewImage",
+      "pos": [
+        515,
+        716
+      ],
+      "size": {
+        "0": 210,
+        "1": 246
+      },
+      "flags": {},
+      "order": 4,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "images",
+          "type": "IMAGE",
+          "link": 55
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "PreviewImage"
+      }
+    },
+    {
+      "id": 23,
+      "type": "VAEEncode",
+      "pos": [
+        515,
+        1092
+      ],
+      "size": {
+        "0": 210,
+        "1": 46
+      },
+      "flags": {},
+      "order": 5,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "pixels",
+          "type": "IMAGE",
+          "link": 57
+        },
+        {
+          "name": "vae",
+          "type": "VAE",
+          "link": 58
+        }
+      ],
+      "outputs": [
+        {
+          "name": "LATENT",
+          "type": "LATENT",
+          "links": [
+            54
+          ],
+          "shape": 3,
+          "slot_index": 0
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "VAEEncode"
+      }
+    },
+    {
+      "id": 10,
+      "type": "CheckpointLoaderSimple",
+      "pos": [
+        100,
+        130
+      ],
+      "size": {
+        "0": 315,
+        "1": 98
+      },
+      "flags": {},
+      "order": 0,
+      "mode": 0,
+      "outputs": [
+        {
+          "name": "MODEL",
+          "type": "MODEL",
+          "links": [
+            31
+          ],
+          "shape": 3,
+          "slot_index": 0
+        },
+        {
+          "name": "CLIP",
+          "type": "CLIP",
+          "links": [
+            32,
+            33,
+            45
+          ],
+          "shape": 3,
+          "slot_index": 1
+        },
+        {
+          "name": "VAE",
+          "type": "VAE",
+          "links": [
+            34,
+            58
+          ],
+          "shape": 3,
+          "slot_index": 2
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "CheckpointLoaderSimple"
+      },
+      "widgets_values": [
+        "deliberate_v2.safetensors"
+      ]
+    },
+    {
+      "id": 24,
+      "type": "ScreenShare",
+      "pos": [
+        100,
+        358
+      ],
+      "size": {
+        "0": 315,
+        "1": 170
+      },
+      "flags": {},
+      "order": 1,
+      "mode": 0,
+      "outputs": [
+        {
+          "name": "IMAGE",
+          "type": "IMAGE",
+          "links": [
+            55,
+            57
+          ],
+          "shape": 3,
+          "slot_index": 0
+        },
+        {
+          "name": "PROMPT",
+          "type": "STRING",
+          "links": [
+            59
+          ],
+          "shape": 3,
+          "slot_index": 1
+        },
+        {
+          "name": "FLOAT",
+          "type": "FLOAT",
+          "links": [
+            56
+          ],
+          "shape": 3,
+          "slot_index": 2
+        },
+        {
+          "name": "INT",
+          "type": "INT",
+          "links": null,
+          "shape": 3
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "ScreenShare"
+      },
+      "widgets_values": [
+        null,
+        null,
+        null,
+        null,
+        null
+      ]
+    }
+  ],
+  "links": [
+    [
+      1,
+      1,
+      0,
+      2,
+      0,
+      "IMAGE"
+    ],
+    [
+      3,
+      6,
+      0,
+      5,
+      0,
+      "MODEL"
+    ],
+    [
+      5,
+      8,
+      0,
+      5,
+      2,
+      "CONDITIONING"
+    ],
+    [
+      6,
+      9,
+      0,
+      5,
+      3,
+      "LATENT"
+    ],
+    [
+      7,
+      10,
+      0,
+      6,
+      0,
+      "MODEL"
+    ],
+    [
+      8,
+      10,
+      1,
+      6,
+      1,
+      "CLIP"
+    ],
+    [
+      9,
+      5,
+      0,
+      11,
+      0,
+      "LATENT"
+    ],
+    [
+      10,
+      10,
+      2,
+      11,
+      1,
+      "VAE"
+    ],
+    [
+      11,
+      6,
+      1,
+      7,
+      0,
+      "CLIP"
+    ],
+    [
+      12,
+      6,
+      1,
+      8,
+      0,
+      "CLIP"
+    ],
+    [
+      14,
+      11,
+      0,
+      3,
+      0,
+      "IMAGE"
+    ],
+    [
+      17,
+      1,
+      2,
+      7,
+      1,
+      "STRING"
+    ],
+    [
+      18,
+      7,
+      0,
+      15,
+      0,
+      "CONDITIONING"
+    ],
+    [
+      19,
+      15,
+      0,
+      5,
+      1,
+      "CONDITIONING"
+    ],
+    [
+      20,
+      16,
+      0,
+      15,
+      1,
+      "CONTROL_NET"
+    ],
+    [
+      24,
+      1,
+      0,
+      18,
+      0,
+      "IMAGE"
+    ],
+    [
+      25,
+      18,
+      0,
+      15,
+      2,
+      "IMAGE"
+    ],
+    [
+      29,
+      7,
+      0,
+      5,
+      2,
+      "CONDITIONING"
+    ],
+    [
+      31,
+      10,
+      0,
+      6,
+      0,
+      "MODEL"
+    ],
+    [
+      32,
+      10,
+      1,
+      8,
+      0,
+      "CLIP"
+    ],
+    [
+      33,
+      10,
+      1,
+      7,
+      0,
+      "CLIP"
+    ],
+    [
+      34,
+      10,
+      2,
+      11,
+      1,
+      "VAE"
+    ],
+    [
+      35,
+      5,
+      0,
+      11,
+      0,
+      "LATENT"
+    ],
+    [
+      41,
+      11,
+      0,
+      20,
+      0,
+      "IMAGE"
+    ],
+    [
+      44,
+      6,
+      0,
+      5,
+      0,
+      "MODEL"
+    ],
+    [
+      45,
+      10,
+      1,
+      6,
+      1,
+      "CLIP"
+    ],
+    [
+      51,
+      8,
+      0,
+      5,
+      1,
+      "CONDITIONING"
+    ],
+    [
+      54,
+      23,
+      0,
+      5,
+      3,
+      "LATENT"
+    ],
+    [
+      55,
+      24,
+      0,
+      2,
+      0,
+      "IMAGE"
+    ],
+    [
+      56,
+      24,
+      2,
+      5,
+      4,
+      "FLOAT"
+    ],
+    [
+      57,
+      24,
+      0,
+      23,
+      0,
+      "IMAGE"
+    ],
+    [
+      58,
+      10,
+      2,
+      23,
+      1,
+      "VAE"
+    ],
+    [
+      59,
+      24,
+      1,
+      8,
+      1,
+      "STRING"
+    ]
+  ],
+  "groups": [],
+  "config": {},
+  "extra": {},
+  "version": 0.4
+}