支持换行的textimage

Delete layers-test-workflow.json

README.md

@@ -1,19 +1,19 @@
 ## 
+v0.4.0 🚀🚗🚚🏃‍
+- Add "help" option to the context menu for each node.
+- Add "find the node" option to the global context menu.
+- Optimize the 3D Image node and add workflow.
 
-v0.2.6 🚀🚗🚚🏃‍
-
-- [Add getting camera video stream](./workflow/7-camera-workflow.json)
-- Add a slider to the floating window, which can be used as input for denoise
-- OSupport for calling multiple GPTs
-
-
-![screenshare](./assets/screenshare.png)
+### 3D
+![](./assets/3dimage.png)
+[workflow](./workflow/3D-workflow.json)
 
 
 ### ScreenShareNode & FloatingVideoNode
 > Now comfyui supports capturing screen pixel streams from any software and can be used for LCM-Lora integration. Let's get started with implementation and design! 💻🌐
 
 > 
+![screenshare](./assets/screenshare.png)
 
 https://github.com/shadowcz007/comfyui-mixlab-nodes/assets/12645064/e7e77f90-e43e-410a-ab3a-1952b7b4e7da
 
@@ -23,6 +23,18 @@ https://github.com/shadowcz007/comfyui-mixlab-nodes/assets/12645064/e7e77f90-e43
 
 !! Please use the address with HTTPS (https://127.0.0.1).
 
+### SpeechRecognition & SpeechSynthesis
+![f](./assets/audio-workflow.svg)
+
+[Voice + Real-time Face Swap Workflow](./workflow/语音+实时换脸workflow.json)
+
+### GPT
+> Support for calling multiple GPTs.ChatGPT、ChatGLM3 , Some code provided by rui. If you are using OpenAI's service, fill in https://api.openai.com/v1 . If you are using a local LLM service, fill in http://127.0.0.1:xxxx/v1 .  Azure OpenAI:https://xxxx.openai.azure.com 
+
+
+![gpt-workflow.svg](./assets/gpt-workflow.svg)
+
+[workflow-5](./workflow/5-gpt-workflow.json)
 
 ### LoadImagesFromLocal
 > Monitor changes to images in a local folder, and trigger real-time execution of workflows, supporting common image formats, especially PSD format, in conjunction with Photoshop. 
@@ -32,47 +44,14 @@ https://github.com/shadowcz007/comfyui-mixlab-nodes/assets/12645064/e7e77f90-e43
 [workflow-4](./workflow/4-loadfromlocal-watcher-workflow.json)
 
 
-### GPT
->Support for calling multiple GPTs.ChatGPT、ChatGLM3 , Some code provided by rui. If you are using OpenAI's service, fill in https://api.openai.com/v1 . If you are using a local LLM service, fill in http://127.0.0.1:xxxx/v1 .  Azure OpenAI:https://xxxx.openai.azure.com 
+### Layers
+> A new layer class node has been added, allowing you to separate the image into layers. After merging the images, you can input the controlnet for further processing.
 
+![layers](./assets/layers-workflow.svg)
 
-![gpt-workflow.svg](./assets/gpt-workflow.svg)
+![poster](./assets/poster-workflow.svg)
 
-[workflow-5](./workflow/5-gpt-workflow.json)
-
-
-## Installation
-
-manually install, simply clone the repo into the custom_nodes directory with this command:
-
-```
-cd ComfyUI/custom_nodes
-
-git clone https://github.com/shadowcz007/comfyui-mixlab-nodes.git
-
-```
-
-Install the requirements:
-
-run directly:
-```
-cd ComfyUI_Mixlab
-install.bat
-```
-
-or install the requirements using:
-```
-../../../python_embeded/python.exe -s -m pip install -r requirements.txt
-```
-
-If you are using a venv, make sure you have it activated before installation and use:
-```
-pip3 install -r requirements.txt
-```
-
-
-
-## Nodes
+## Other Nodes
 
 ![main](./assets/all-workflow.svg)
 ![main2](./assets/detect-face-all.png)
@@ -88,9 +67,6 @@ pip3 install -r requirements.txt
 ![TransparentImage](./assets/TransparentImage.png)
 
 
-
-
-
 > Consistency Decoder
 
 [openai Consistency Decoder]( https://github.com/openai/consistencydecoder)
@@ -109,8 +85,14 @@ Add edges to an image.
 
 
 ### Improvement 
+
+- Add "help" option to the context menu for each node.
+- Add "find the node" option to the global context menu.
+
 An improvement has been made to directly redirect to GitHub to search for missing nodes when loading the graph.
 
+![help](./assets/help.png)
+
 ![node-not-found](./assets/node-not-found.png)
 
 
@@ -120,6 +102,42 @@ An improvement has been made to directly redirect to GitHub to search for missin
 <!-- ### Workflow
 [Workflow](./workflow.md) -->
 
+
+
+## Installation
+
+manually install, simply clone the repo into the custom_nodes directory with this command:
+
+```
+cd ComfyUI/custom_nodes
+
+git clone https://github.com/shadowcz007/comfyui-mixlab-nodes.git
+
+```
+
+Install the requirements:
+
+run directly:
+```
+cd ComfyUI/custom_nodes/comfyui-mixlab-nodes
+install.bat
+```
+
+or install the requirements using:
+```
+../../../python_embeded/python.exe -s -m pip install -r requirements.txt
+```
+
+If you are using a venv, make sure you have it activated before installation and use:
+```
+pip3 install -r requirements.txt
+```
+
+#### Chinese community
+访问 [www.mixcomfy.com](https://www.mixcomfy.com)，获得更多内测功能，关注微信公众号：Mixlab无界社区
+
+
+
 #### Thanks:
 [ComfyUI-CLIPSeg](https://github.com/biegert/ComfyUI-CLIPSeg/tree/main)
 
@@ -127,5 +145,26 @@ An improvement has been made to directly redirect to GitHub to search for missin
 [discussions](https://github.com/shadowcz007/comfyui-mixlab-nodes/discussions)
 
 ### TODO:
+- 音频播放节点：带可视化、支持多音轨、可配置音轨音量
 - vector https://github.com/GeorgLegato/stable-diffusion-webui-vectorstudio
 
+
+<picture>
+  <source
+    media="(prefers-color-scheme: dark)"
+    srcset="
+      https://api.star-history.com/svg?repos=shadowcz007/comfyui-mixlab-nodes&type=Date&theme=dark
+    "
+  />
+  <source
+    media="(prefers-color-scheme: light)"
+    srcset="
+      https://api.star-history.com/svg?repos=shadowcz007/comfyui-mixlab-nodes&type=Date
+    "
+  />
+  <img
+    alt="Star History Chart"
+    src="https://api.star-history.com/svg?repos=shadowcz007/comfyui-mixlab-nodes&type=Date"
+  />
+</picture>
+

__init__.py

@@ -168,8 +168,44 @@ def save_workflow_json(data):
         json.dump(data, file)
     return workflow_path
 
+
+def get_nodes_map():
+    # print("#####path::", current_path)
+    data_path=os.path.join(current_path, "data")
+    print('data_path: ',data_path)
+    # if not os.path.exists(data_path):
+    #     # 使用mkdir()方法创建新目录
+    #     os.mkdir(data_path)
+    json_data={}
+    nodes_map=os.path.join(current_path, "data/extension-node-map.json")
+    if os.path.exists(nodes_map):
+        with open(nodes_map) as json_file:
+            json_data = json.load(json_file)
+
+    return json_data
+
+
+# 保存原始的 get 方法
+_original_request = aiohttp.ClientSession._request
+
+# 定义新的 get 方法
+async def new_request(self, method, url, *args, **kwargs):
+   # 检查环境变量以确定是否使用代理
+    proxy = os.environ.get('HTTP_PROXY') or os.environ.get('HTTPS_PROXY') or os.environ.get('http_proxy') or os.environ.get('https_proxy') 
+    # print('Proxy Config:',proxy)
+    if proxy and 'proxy' not in kwargs:
+        kwargs['proxy'] = proxy
+        print('Use Proxy:',proxy)
+    # 调用原始的 _request 方法
+    return await _original_request(self, method, url, *args, **kwargs)
+
+# 应用 Monkey Patch
+aiohttp.ClientSession._request = new_request
+
 # https
 async def new_start(self, address, port, verbose=True, call_on_start=None):
+        
+
         runner = web.AppRunner(self.app, access_log=None)
         await runner.setup()
         site = web.TCPSite(runner, address, port)
@@ -239,6 +275,20 @@ async def mixlab_workflow_hander(request):
 
     return web.json_response(result)
 
+@routes.post('/mixlab/nodes_map')
+async def nodes_map_hander(request):
+    data = await request.json()
+    result={}
+    try: 
+        result={
+            'data':get_nodes_map(),
+            'status':'success',
+                }
+    except Exception as e:
+            print(e)
+
+    return web.json_response(result)
+
 def new_add_routes(self):
         import nodes
         self.app.add_routes(routes)
@@ -268,19 +318,29 @@ PromptServer.add_routes=new_add_routes
 
 # 导入节点
 from .nodes.PromptNode import RandomPrompt
-from .nodes.ImageNode import TransparentImage,LoadImagesFromPath,AreaToMask,SmoothMask,FeatheredMask,SplitLongMask,ImageCropByAlpha,EnhanceImage,FaceToMask
+from .nodes.ImageNode import TransparentImage,LoadImagesFromPath,ResizeImage,TextImage,SvgImage,Image3D,EmptyLayer,ShowLayer,NewLayer,MergeLayers,AreaToMask,SmoothMask,FeatheredMask,SplitLongMask,ImageCropByAlpha,EnhanceImage,FaceToMask
 from .nodes.Vae import VAELoader,VAEDecode
 from .nodes.ScreenShareNode import ScreenShareNode,FloatingVideo
 from .nodes.Clipseg import CLIPSeg,CombineMasks
 from .nodes.ChatGPT import ChatGPTNode,ShowTextForGPT,CharacterInText
+from .nodes.Audio import GamePal,SpeechRecognition,SpeechSynthesis
+from .nodes.Utils import ColorInput,FontInput,TextToNumber
+
 
 # 要导出的所有节点及其名称的字典
 # 注意：名称应全局唯一
 NODE_CLASS_MAPPINGS = {
     "RandomPrompt":RandomPrompt,
     "TransparentImage":TransparentImage,
+    "ResizeImageMixlab":ResizeImage,
     "LoadImagesFromPath":LoadImagesFromPath,
+    "TextImage":TextImage,
     "EnhanceImage":EnhanceImage,
+    "SvgImage":SvgImage,
+    "3DImage":Image3D, 
+    "ShowLayer":ShowLayer,
+    "NewLayer":NewLayer,
+    "MergeLayers":MergeLayers,
     "SplitLongMask":SplitLongMask,
     "FeatheredMask":FeatheredMask,
     "SmoothMask":SmoothMask,
@@ -295,11 +355,18 @@ NODE_CLASS_MAPPINGS = {
     "CombineMasks_":CombineMasks,
     "ChatGPTOpenAI":ChatGPTNode,
     "ShowTextForGPT":ShowTextForGPT,
-    "CharacterInText":CharacterInText
+    "CharacterInText":CharacterInText,
+    "SpeechRecognition":SpeechRecognition,
+    "SpeechSynthesis":SpeechSynthesis,
+    "Color":ColorInput,
+    "Font":FontInput,
+    "TextToNumber":TextToNumber
+    # "GamePal":GamePal
 }
 
 # 一个包含节点友好/可读的标题的字典
 NODE_DISPLAY_NAME_MAPPINGS = {
+    "ResizeImageMixlab":"ResizeImage",
     "RandomPrompt": "Random Prompt ♾️Mixlab",
     "SplitLongMask":"Splitting a long image into sections",
     "VAELoaderConsistencyDecoder":"Consistency Decoder Loader",
@@ -307,7 +374,12 @@ NODE_DISPLAY_NAME_MAPPINGS = {
     "ScreenShare":"ScreenShare ♾️Mixlab",
     "FloatingVideo":"FloatingVideo ♾️Mixlab",
     "ChatGPTOpenAI":"ChatGPT ♾️Mixlab",
-    "ShowTextForGPT":"ShowTextForGPT ♾️Mixlab"
+    "ShowTextForGPT":"ShowTextForGPT ♾️Mixlab",
+    "MergeLayers":"MergeLayers ♾️Mixlab",
+    "SpeechSynthesis":"SpeechSynthesis ♾️Mixlab",
+    "SpeechRecognition":"SpeechRecognition ♾️Mixlab",
+    "3DImage":"3DImage ♾️Mixlab",
+    # "GamePal":"GamePal ♾️Mixlab"
 }
 
 # web ui的节点功能

nodes/Audio.py

@@ -0,0 +1,100 @@
+
+
+
+
+class SpeechRecognition:
+    @classmethod
+    def INPUT_TYPES(s):
+        return {"required": { 
+                    "upload":("AUDIOINPUTMIX",),   },
+                "optional":{
+                    "start_by":("INT", {
+                        "default": 0, 
+                        "min": 0, #Minimum value
+                        "max": 2048, #Maximum value
+                        "step": 1, #Slider's step
+                        "display": "number" # Cosmetic only: display as "number" or "slider"
+                    }),
+                   
+                }
+                }
+    
+    RETURN_TYPES = ("STRING",)
+    RETURN_NAMES = ("prompt",)
+
+    FUNCTION = "run"
+
+    CATEGORY = "♾️Mixlab/audio"
+
+    INPUT_IS_LIST = False
+    OUTPUT_IS_LIST = (False,)
+
+    def run(self,upload,start_by):
+        return {"ui": {"start_by": [start_by]}, "result": (upload,)}
+
+
+class SpeechSynthesis:
+    @classmethod
+    def INPUT_TYPES(s):
+        return {
+            "required": {
+                "text": ("STRING", {"forceInput": True}),
+            }
+        }
+
+    INPUT_IS_LIST = True
+    RETURN_TYPES = ("STRING",)
+    FUNCTION = "run"
+    OUTPUT_NODE = True
+    OUTPUT_IS_LIST = (True,)
+
+    CATEGORY = "♾️Mixlab/audio"
+
+    def run(self, text):
+        # print(session_history)
+        return {"ui": {"text": text}, "result": (text,)}
+    
+
+#
+class GamePal:
+    @classmethod
+    def INPUT_TYPES(s):
+        return {
+            "required": {
+                "input_text": ("STRING",{"multiline": True,"default": ""}),
+            },
+            "optional": {
+                 
+                "input_num": ("INT",{
+                                "default":100, 
+                                "min": -1, #Minimum value
+                                "max": 0xffffffffffffffff, #Maximum value
+                                "step": 1, #Slider's step
+                                "display": "slider" # Cosmetic only: display as "number" or "slider"
+                                }), 
+                "python_code": ("STRING",{"multiline": True,"default": "result= 1 if 'Mixlab' in input_text else 0"}),
+            }
+        }
+
+    INPUT_IS_LIST = False
+    RETURN_TYPES = ("INT",)
+    FUNCTION = "run"
+    OUTPUT_NODE = True
+    OUTPUT_IS_LIST = (False,)
+
+    CATEGORY = "♾️Mixlab/audio"
+
+    def run(self, input_text,input_num,python_code):
+        exec(python_code)
+        res=None
+        try:
+            # 可能会引发异常的代码
+            res=result
+        except:
+            # 处理异常的代码
+            print('')
+
+        print(res)
+
+        # print(session_history)
+        return {"ui": {"text": [input_text],"num":[input_num]}, "result": (res,)}

nodes/ChatGPT.py

@@ -46,7 +46,7 @@ def chat(client, model_name,messages ):
             except (urllib.error.HTTPError, openai.OpenAIError) as ex:
                 if try_count >= 3:
                     raise ex
-                time.sleep(5)
+                time.sleep(3)
                 continue
 
         finish_reason = response.choices[0].finish_reason

nodes/Clipseg.py

@@ -1,3 +1,6 @@
+#### Thanks:
+# [ComfyUI-CLIPSeg](https://github.com/biegert/ComfyUI-CLIPSeg/tree/main)
+
 from transformers import CLIPSegProcessor, CLIPSegForImageSegmentation
 
 from PIL import Image
@@ -252,7 +255,7 @@ class CombineMasks:
 
 # A dictionary that contains all nodes you want to export with their names
 # NOTE: names should be globally unique
-NODE_CLASS_MAPPINGS = {
-    "CLIPSeg": CLIPSeg,
-    "CombineSegMasks": CombineMasks,
-}
+# NODE_CLASS_MAPPINGS = {
+#     "CLIPSeg": CLIPSeg,
+#     "CombineSegMasks": CombineMasks,
+# }

nodes/ImageNode.py

@@ -1,17 +1,18 @@
 import numpy as np
 import torch
-from PIL import Image, ImageOps,ImageFilter,ImageEnhance,ImageDraw,ImageSequence
+from PIL import Image, ImageOps,ImageFilter,ImageEnhance,ImageDraw,ImageSequence, ImageFont
 from PIL.PngImagePlugin import PngInfo
 import base64,os
 from io import BytesIO
 import folder_paths
-import json
+import json,io
 from comfy.cli_args import args
 import cv2 
 
 from .Watcher import FolderWatcher
 
 
+FONT_PATH= os.path.abspath(os.path.join(os.path.dirname(__file__),'../assets/王汉宗颜楷体繁.ttf'))
 
 MAX_RESOLUTION=8192
 
@@ -322,6 +323,227 @@ def areaToMask(x,y,w,h,image):
     return mask
 
 
+# def merge_images(bg_image, layer_image,mask, x, y, width, height):
+#     # 打开底图
+#     # bg_image = Image.open(background)
+#     bg_image=bg_image.convert("RGBA")
+
+#     # 打开图层
+#     layer_image=layer_image.convert("RGBA")
+#     layer_image = layer_image.resize((width, height))
+#     # mask = Image.new("L", layer_image.size, 255)
+#     mask = mask.resize((width, height))
+#     # 在底图上粘贴图层
+#     bg_image.paste(layer_image, (x, y), mask=mask)
+
+#     # 输出合成后的图片
+#     # bg_image.save("output.jpg")
+#     return bg_image
+
+def merge_images(bg_image, layer_image, mask, x, y, width, height, scale_option):
+    # 打开底图
+    bg_image = bg_image.convert("RGBA")
+
+    # 打开图层
+    layer_image = layer_image.convert("RGBA")
+    # layer_image = layer_image.resize((width, height))
+
+    # 根据缩放选项调整图像大小
+    if scale_option == "height":
+        # 按照高度比例缩放
+        original_width, original_height = layer_image.size
+        scale = height / original_height
+        new_width = int(original_width * scale)
+        layer_image = layer_image.resize((new_width, height))
+    elif scale_option == "width":
+        # 按照宽度比例缩放
+        original_width, original_height = layer_image.size
+        scale = width / original_width
+        new_height = int(original_height * scale)
+        layer_image = layer_image.resize((width, new_height))
+    elif scale_option == "overall":
+        # 整体缩放
+        layer_image = layer_image.resize((width, height))
+
+    # 调整mask的大小
+    nw, nh = layer_image.size
+    mask = mask.resize((nw, nh))
+
+    # 在底图上粘贴图层
+    bg_image.paste(layer_image, (x, y), mask=mask)
+
+    # 输出合成后的图片
+    return bg_image
+
+
+def resize_image(layer_image,scale_option,width,height):
+    layer_image = layer_image.convert("RGB")
+    if scale_option == "height":
+        # 按照高度比例缩放
+        original_width, original_height = layer_image.size
+        scale = height / original_height
+        new_width = int(original_width * scale)
+        layer_image = layer_image.resize((new_width, height))
+    elif scale_option == "width":
+        # 按照宽度比例缩放
+        original_width, original_height = layer_image.size
+        scale = width / original_width
+        new_height = int(original_height * scale)
+        layer_image = layer_image.resize((width, new_height))
+    elif scale_option == "overall":
+        # 整体缩放
+        layer_image = layer_image.resize((width, height))
+    return layer_image
+
+
+# def generate_text_image(text_list, font_path, font_size, text_color, vertical=True, spacing=0):
+#     # Load Chinese font
+#     font = ImageFont.truetype(font_path, font_size)
+
+#     # Calculate image size based on the number of characters and orientation
+#     if vertical:
+#         width = font_size + 100
+#         height = font_size * len(text_list) + (len(text_list) - 1) * spacing + 100
+#     else:
+#         width = font_size * len(text_list) + (len(text_list) - 1) * spacing + 100
+#         height = font_size + 100
+
+#     # Create a blank image
+#     image = Image.new('RGBA', (width, height), (255, 255, 255,0))
+#     draw = ImageDraw.Draw(image)
+
+#     # Draw text
+#     if vertical:
+#         for i, char in enumerate(text_list):
+#             char_position = (50, 50 + i * font_size)
+#             draw.text(char_position, char, font=font, fill=text_color)
+#     else:
+#         for i, char in enumerate(text_list):
+#             char_position = (50 + i * (font_size + spacing), 50)
+#             draw.text(char_position, char, font=font, fill=text_color)
+
+#     # Save the image
+#     # image.save(output_image_path)
+
+#     # 分离alpha通道
+#     alpha_channel = image.split()[3]
+
+#     # 创建一个只有alpha通道的新图像
+#     alpha_image = Image.new('L', image.size)
+#     alpha_image.putdata(alpha_channel.getdata())
+
+#     image=image.convert('RGB')
+
+#     return (image,alpha_image)
+
+def generate_text_image(text, font_path, font_size, text_color, vertical=True, spacing=0):
+    # Split text into lines based on line breaks
+    lines = text.split("\n")
+
+    # 1. Determine layout direction
+    if vertical:
+        layout = "vertical"
+    else:
+        layout = "horizontal"
+
+    # 2. Calculate absolute coordinates for each character
+    char_coordinates = []
+    if layout == "vertical":
+        x = 0
+        y = 0
+        for i in range(len(lines)):
+            line=lines[i]
+            for char in line:
+                char_coordinates.append((x, y))
+                y += font_size + spacing
+            x += font_size + spacing
+            y = 0
+        print(char_coordinates)
+    else:
+        x = 0
+        y = 0
+        for line in lines:
+            for char in line:
+                char_coordinates.append((x, y))
+                x += font_size + spacing
+            y += font_size + spacing
+            x = 0
+
+    # 3. Calculate image width and height
+    if layout == "vertical":
+        width = (len(lines) * (font_size + spacing)) - spacing
+        height = (len(max(lines, key=len)) * (font_size + spacing)) - spacing
+    else:
+        width = (len(max(lines, key=len)) * (font_size + spacing)) - spacing
+        height = (len(lines) * (font_size + spacing)) - spacing
+
+    # 4. Draw each character on the image
+    image = Image.new('RGBA', (width, height), (255, 255, 255,0))
+    draw = ImageDraw.Draw(image)
+    font = ImageFont.truetype(font_path, font_size)
+    
+    index=0
+    for i, line in enumerate(lines):
+        for j, char in enumerate(line):
+            x, y = char_coordinates[index]
+            draw.text((x, y), char, font=font, fill=text_color)
+            index+=1
+
+    # image.save(output_image_path)
+
+    # 分离alpha通道
+    alpha_channel = image.split()[3]
+
+    # 创建一个只有alpha通道的新图像
+    alpha_image = Image.new('L', image.size)
+    alpha_image.putdata(alpha_channel.getdata())
+
+    image=image.convert('RGB')
+
+    return (image,alpha_image)
+
+def base64_to_image(base64_string):
+    # 去除前缀
+    prefix, base64_data = base64_string.split(",", 1)
+    
+    # 从base64字符串中解码图像数据
+    image_data = base64.b64decode(base64_data)
+    
+    # 创建一个内存流对象
+    image_stream = io.BytesIO(image_data)
+    
+    # 使用PIL的Image模块打开图像数据
+    image = Image.open(image_stream)
+    
+    return image
+
+
+def create_temp_file(image):
+    output_dir = folder_paths.get_temp_directory()
+
+    (
+            full_output_folder,
+            filename,
+            counter,
+            subfolder,
+            _,
+        ) = folder_paths.get_save_image_path('material', output_dir)
+
+    
+    image=tensor2pil(image)
+ 
+    image_file = f"{filename}_{counter:05}.png"
+     
+    image_path=os.path.join(full_output_folder, image_file)
+
+    image.save(image_path,compress_level=4)
+
+    return [{
+                "filename": image_file,
+                "subfolder": subfolder,
+                "type": "temp"
+                }]
+
 
 class SmoothMask:
     @classmethod
@@ -494,6 +716,7 @@ class TransparentImage:
             }
     
     RETURN_TYPES = ('STRING','IMAGE','RGBA')
+    RETURN_NAMES = ("file_path","IMAGE","RGBA",)
 
     OUTPUT_NODE = True
 
@@ -546,7 +769,7 @@ class TransparentImage:
         
         # ui.images 节点里显示图片，和 传参，image_path自定义的数据，需要写节点的自定义ui
         # result 里输出给下个节点的数据 
-        print('TransparentImage',len(images_rgb))
+        # print('TransparentImage',len(images_rgb))
         return {"ui":{"images": ui_images,"image_paths":image_paths},"result": (image_paths,images_rgb,images_rgba)}
         
 
@@ -671,7 +894,7 @@ class LoadImagesFromPath:
             imgs=[imgs[index_variable]] if index_variable < len(imgs) else None
             masks=[masks[index_variable]] if index_variable < len(masks) else None
 
-        print('#prompt::::',prompt)
+        # print('#prompt::::',prompt)
         return (imgs,masks,prompt,)
 
 
@@ -708,6 +931,140 @@ class ImageCropByAlpha:
         return (img,)
 
 
+
+
+
+class TextImage:
+    @classmethod
+    def INPUT_TYPES(s):
+        return {"required": { 
+            
+                    "text": ("STRING",{"multiline": True,"default": "龍馬精神迎新歲"}),
+                    "font_path": ("STRING",{"multiline": False,"default": FONT_PATH}),
+                    "font_size": ("INT",{
+                                "default":100, 
+                                "min": 100, #Minimum value
+                                "max": 1000, #Maximum value
+                                "step": 1, #Slider's step
+                                "display": "number" # Cosmetic only: display as "number" or "slider"
+                                }), 
+                    "spacing": ("INT",{
+                                "default":12, 
+                                "min": 1, #Minimum value
+                                "max": 200, #Maximum value
+                                "step": 1, #Slider's step
+                                "display": "number" # Cosmetic only: display as "number" or "slider"
+                                }), 
+                    "text_color":("STRING",{"multiline": False,"default": "#000000"}),
+                    "vertical":("BOOLEAN", {"default": True},),
+                             },
+                }
+    
+    RETURN_TYPES = ("IMAGE","MASK")
+    # RETURN_NAMES = ("WIDTH","HEIGHT","X","Y",)
+
+    FUNCTION = "run"
+
+    CATEGORY = "♾️Mixlab/image"
+
+    INPUT_IS_LIST = False
+    OUTPUT_IS_LIST = (False,False,)
+
+    def run(self,text,font_path,font_size,spacing,text_color,vertical):
+        
+        text_list=list(text)
+
+        img,mask=generate_text_image(text,font_path,font_size,text_color,vertical,spacing)
+        
+        img=pil2tensor(img)
+        mask=pil2tensor(mask)
+
+        return (img,mask,)
+
+
+
+class SvgImage:
+    @classmethod
+    def INPUT_TYPES(s):
+        return {"required": { 
+                    "upload":("SVG",),},
+                }
+    
+    RETURN_TYPES = ("IMAGE","LAYER")
+    RETURN_NAMES = ("IMAGE","layers",)
+
+    FUNCTION = "run"
+
+    CATEGORY = "♾️Mixlab/image"
+
+    INPUT_IS_LIST = False
+    OUTPUT_IS_LIST = (False,True,)
+
+    def run(self,upload):
+        layers=[]
+
+        image = base64_to_image(upload['image'])
+        image=image.convert('RGB')
+        image=pil2tensor(image)
+
+        for layer in upload['data']:
+            layers.append(layer)
+    
+        return (image,layers,)
+
+
+
+class Image3D:
+    @classmethod
+    def INPUT_TYPES(s):
+        return {"required": { 
+                    "upload":("THREED",),}, 
+                "optional":{
+                    "material": ("IMAGE",),
+                    }
+                }
+    
+    RETURN_TYPES = ("IMAGE","MASK","IMAGE","IMAGE",)
+    RETURN_NAMES = ("IMAGE","MASK","BG_IMAGE","MATERIAL",)
+
+    FUNCTION = "run"
+
+    CATEGORY = "♾️Mixlab/image"
+
+    INPUT_IS_LIST = False
+    OUTPUT_IS_LIST = (False,False,False,False,)
+    OUTPUT_NODE = True
+
+    def run(self,upload,material=None):
+        # print('material',material)
+        # print(upload['image'])
+        image = base64_to_image(upload['image'])
+        mat=base64_to_image(upload['material'])
+        mask = image.split()[3]
+        image=image.convert('RGB')
+        mat=mat.convert('RGB')
+        mask=mask.convert('L')
+
+        bg_image=None
+        if upload['bg_image']:
+            bg_image = base64_to_image(upload['bg_image'])
+            bg_image=bg_image.convert('RGB')
+            bg_image=pil2tensor(bg_image)
+
+
+        mask=pil2tensor(mask)
+        image=pil2tensor(image)
+        mat=pil2tensor(mat)
+
+        m=[]
+        if not material is None:
+            m=create_temp_file(material[0])
+        
+        return {"ui":{"material": m},"result": (image,mask,bg_image,mat,)}
+
+
+
+
 class AreaToMask:
     @classmethod
     def INPUT_TYPES(s):
@@ -770,3 +1127,343 @@ class FaceToMask:
 
         return (mask,)
 
+
+class EmptyLayer:
+    @classmethod
+    def INPUT_TYPES(s):
+        return {
+            "required": { 
+                "width": ("INT",{
+                    "default":512, 
+                    "min": 1, #Minimum value
+                    "max": 8192, #Maximum value
+                    "step": 1, #Slider's step
+                    "display": "number" # Cosmetic only: display as "number" or "slider"
+                }),
+               "height": ("INT",{
+                    "default": 512, 
+                    "min": 1, #Minimum value
+                    "max": 8192, #Maximum value
+                    "step": 1, #Slider's step
+                    "display": "number" # Cosmetic only: display as "number" or "slider"
+                }),
+            },
+        
+                }
+    
+    RETURN_TYPES = ("LAYER",)
+    RETURN_NAMES = ("layers",)
+
+    FUNCTION = "run"
+
+    CATEGORY = "♾️Mixlab/layer"
+
+    OUTPUT_IS_LIST = (True,)
+
+    def run(self, width,height):
+        blank_image = Image.new("RGB", (width, height))
+        
+        mask=blank_image.convert('L')
+
+        blank_image=pil2tensor(blank_image)
+        mask=pil2tensor(mask)
+
+        layer_n=[{
+            "x":0,
+            "y":0,
+            "width":width,
+            "height":height,
+            "z_index":0,
+            "scale_option":'width',
+            "image":blank_image,
+            "mask":mask
+        }]
+        return (layer_n,)
+
+
+class NewLayer:
+    @classmethod
+    def INPUT_TYPES(s):
+        return {
+            
+            "required": { 
+                "x": ("INT",{
+                    "default": 0, 
+                    "min": -1024, #Minimum value
+                    "max": 8192, #Maximum value
+                    "step": 1, #Slider's step
+                    "display": "number" # Cosmetic only: display as "number" or "slider"
+                }),
+                "y": ("INT",{
+                    "default": 0, 
+                    "min": -1024, #Minimum value
+                    "max": 8192, #Maximum value
+                    "step": 1, #Slider's step
+                    "display": "number" # Cosmetic only: display as "number" or "slider"
+                }),
+                "width": ("INT",{
+                    "default": 512, 
+                    "min": 1, #Minimum value
+                    "max": 8192, #Maximum value
+                    "step": 1, #Slider's step
+                    "display": "number" # Cosmetic only: display as "number" or "slider"
+                }),
+                "height": ("INT",{
+                    "default": 512, 
+                    "min": 1, #Minimum value
+                    "max": 8192, #Maximum value
+                    "step": 1, #Slider's step
+                    "display": "number" # Cosmetic only: display as "number" or "slider"
+                }),
+                "z_index": ("INT",{
+                    "default": 0, 
+                    "min":0, #Minimum value
+                    "max": 100, #Maximum value
+                    "step": 1, #Slider's step
+                    "display": "number" # Cosmetic only: display as "number" or "slider"
+                }),
+                "scale_option": (["width","height",'overall'],),
+                "image": ("IMAGE",),
+            },
+             "optional":{
+                    "mask": ("MASK",{"default": None}),
+                    "layers": ("LAYER",{"default": None}), 
+                    "canvas": ("IMAGE",{"default": None}), 
+                }
+                }
+    
+    RETURN_TYPES = ("LAYER",)
+    RETURN_NAMES = ("layers",)
+
+    FUNCTION = "run"
+
+    CATEGORY = "♾️Mixlab/layer"
+
+    INPUT_IS_LIST = True
+    OUTPUT_IS_LIST = (True,)
+
+    def run(self,x,y,width,height,z_index,scale_option,image,mask=None,layers=None,canvas=None):
+        # print(x,y,width,height,z_index,image,mask)
+        
+        if mask==None:
+            im=tensor2pil(image[0])
+            mask=im.convert('L')
+            mask=pil2tensor(mask)
+        else:
+            mask=mask[0]
+        
+        layer_n=[{
+            "x":x[0],
+            "y":y[0],
+            "width":width[0],
+            "height":height[0],
+            "z_index":z_index[0],
+            "scale_option":scale_option[0],
+            "image":image[0],
+            "mask":mask
+        }]
+
+        if layers!=None:
+            layer_n=layer_n+layers
+
+        return (layer_n,)
+
+class ShowLayer:
+    @classmethod
+    def INPUT_TYPES(s):
+        return {
+            
+            "required": { 
+                "edit": ("EDIT",),
+               
+                "x": ("INT",{
+                    "default": 0, 
+                    "min": -100, #Minimum value
+                    "max": 8192, #Maximum value
+                    "step": 1, #Slider's step
+                    "display": "number" # Cosmetic only: display as "number" or "slider"
+                }),
+                "y": ("INT",{
+                    "default": 0, 
+                    "min": 0, #Minimum value
+                    "max": 8192, #Maximum value
+                    "step": 1, #Slider's step
+                    "display": "number" # Cosmetic only: display as "number" or "slider"
+                }),
+                "width": ("INT",{
+                    "default": 512, 
+                    "min": 1, #Minimum value
+                    "max": 8192, #Maximum value
+                    "step": 1, #Slider's step
+                    "display": "number" # Cosmetic only: display as "number" or "slider"
+                }),
+                "height": ("INT",{
+                    "default": 512, 
+                    "min": 1, #Minimum value
+                    "max": 8192, #Maximum value
+                    "step": 1, #Slider's step
+                    "display": "number" # Cosmetic only: display as "number" or "slider"
+                }),
+                "z_index": ("INT",{
+                    "default": 0, 
+                    "min":0, #Minimum value
+                    "max": 100, #Maximum value
+                    "step": 1, #Slider's step
+                    "display": "number" # Cosmetic only: display as "number" or "slider"
+                }),
+                "scale_option": (["width","height",'overall'],),
+                # "image": ("IMAGE",),
+            },
+             "optional":{
+                    # "mask": ("MASK",{"default": None}),
+                    "layers": ("LAYER",{"default": None}), 
+                }
+                }
+    
+    RETURN_TYPES = ( )
+    RETURN_NAMES = ( )
+
+    FUNCTION = "run"
+
+    CATEGORY = "♾️Mixlab/layer"
+
+    INPUT_IS_LIST = True
+    # OUTPUT_IS_LIST = (True,)
+
+    def run(self,edit,x,y,width,height,z_index,scale_option,layers):
+        # print(x,y,width,height,z_index,image,mask)
+        
+        # if mask==None:
+        #     im=tensor2pil(image)
+        #     mask=im.convert('L')
+        #     mask=pil2tensor(mask)
+        # else:
+        #     mask=mask[0]
+
+        # layers[edit[0]]={
+        #     "x":x[0],
+        #     "y":y[0],
+        #     "width":width[0],
+        #     "height":height[0],
+        #     "z_index":z_index[0],
+        #     "scale_option":scale_option[0],
+        #     "image":image[0],
+        #     "mask":mask
+        # }
+
+        return ( )
+
+
+class MergeLayers:
+    @classmethod
+    def INPUT_TYPES(s):
+        return {"required": { 
+            "layers": ("LAYER",),
+            "image": ("IMAGE",),
+                             },
+                }
+    
+    RETURN_TYPES = ("IMAGE",)
+    # RETURN_NAMES = ("WIDTH","HEIGHT","X","Y",)
+
+    FUNCTION = "run"
+
+    CATEGORY = "♾️Mixlab/layer"
+
+    INPUT_IS_LIST = True
+    # OUTPUT_IS_LIST = (False,)
+
+    def run(self,layers,image):
+        # print(len(layers),len(image))
+        bg_image=image[0]
+        bg_image=tensor2pil(bg_image)
+        # 按z-index排序
+        layers_new = sorted(layers, key=lambda x: x["z_index"])
+     
+        for layer in layers_new:
+            image=layer['image']
+            mask=layer['mask']
+            if 'type' in layer and layer['type']=='base64' and type(image) == str:
+                im=base64_to_image(image)
+                im=im.convert('RGB')
+                image=pil2tensor(im)
+
+                mask=base64_to_image(mask)
+                mask=mask.convert('L')
+                mask=pil2tensor(mask)
+            
+            
+            layer_image=tensor2pil(image)
+            layer_mask=tensor2pil(mask)
+            bg_image=merge_images(bg_image,
+                                  layer_image,
+                                  layer_mask,
+                                  layer['x'],
+                                  layer['y'],
+                                  layer['width'],
+                                  layer['height'],
+                                  layer['scale_option']
+                                  )
+            
+        mask=bg_image.convert('RGBA')
+        mask=pil2tensor(mask)
+        
+        bg_image=bg_image.convert('RGB')
+        bg_image=pil2tensor(bg_image)
+
+        channels = ["red", "green", "blue", "alpha"]
+        # print(mask,mask.shape)
+        mask = mask[:, :, :, channels.index("green")]
+        
+        return (bg_image,mask,)
+    
+
+
+
+class ResizeImage:
+    @classmethod
+    def INPUT_TYPES(s):
+        return {"required": {
+            "image": ("IMAGE",),
+
+            "width": ("INT",{
+                    "default": 512, 
+                    "min": 1, #Minimum value
+                    "max": 8192, #Maximum value
+                    "step": 1, #Slider's step
+                    "display": "number" # Cosmetic only: display as "number" or "slider"
+                }),
+                "height": ("INT",{
+                    "default": 512, 
+                    "min": 1, #Minimum value
+                    "max": 8192, #Maximum value
+                    "step": 1, #Slider's step
+                    "display": "number" # Cosmetic only: display as "number" or "slider"
+                }),
+                "scale_option": (["width","height",'overall'],),
+
+                             },
+                }
+    
+    RETURN_TYPES = ("IMAGE",)
+
+    FUNCTION = "run"
+
+    CATEGORY = "♾️Mixlab/image"
+
+    INPUT_IS_LIST = True
+    OUTPUT_IS_LIST = (False,)
+
+    def run(self,image,width,height,scale_option):
+        
+        w=width[0]
+        h=height[0]
+        scale_option=scale_option[0]
+        im=image[0]
+
+        im=tensor2pil(im)
+        im=resize_image(im,scale_option,w,h)
+        im=im.convert('RGB')
+        im=pil2tensor(im)
+        
+        return (im,)

nodes/ScreenShareNode.py

@@ -79,11 +79,13 @@ class ScreenShareNode:
     def INPUT_TYPES(s):
         return { "required":{
             "image_base64": ("CHEESE",),
+            "refresh_rate": ("INT", {"default": 500, "min": 0,"step": 50, "max": 0xffffffffffffffff}),
         },
           "optional":{
               "prompt": ("PROMPT",),
               "slide": ("SLIDE",),
               "seed": ("SEED",),
+              
             #   "seed": ("INT", {"default": 1, "min": 0, "max": 0xffffffffffffffff}),
                 } }
     
@@ -97,11 +99,11 @@ class ScreenShareNode:
     OUTPUT_IS_LIST = (False,False,False,False)
   
     # 运行的函数
-    def run(self,image_base64,prompt,slide,seed):
+    def run(self,image_base64,refresh_rate ,prompt,slide,seed):
         im,mask=base64_save(image_base64)
         # print('##########prompt',prompt)
-        return (im,prompt,slide,seed)
-    
+        return {"ui":{"refresh_rate": [refresh_rate]},"result": (im,prompt,slide,seed,)}
+
 
 class FloatingVideo:
     @classmethod

nodes/Utils.py

@@ -0,0 +1,121 @@
+import os
+import re,random
+# FONT_PATH= os.path.abspath(os.path.join(os.path.dirname(__file__),'../assets/王汉宗颜楷体繁.ttf'))
+
+import matplotlib.font_manager as fm
+
+def get_font_files(directory):
+    font_files = {}
+
+    # 从指定目录加载字体
+    for file in os.listdir(directory):
+        if file.endswith('.ttf') or file.endswith('.otf'):
+            font_name = os.path.splitext(file)[0]
+            font_path = os.path.join(directory, file)
+            font_files[font_name] = os.path.abspath(font_path)
+
+    # 尝试获取系统字体
+    try:
+        font_paths = fm.findSystemFonts()
+        for path in font_paths:
+            try:
+                font_prop = fm.FontProperties(fname=path)
+                font_name = font_prop.get_name()
+                font_files[font_name] = path
+            except Exception as e:
+                print(f"Error processing font {path}: {e}")
+    except Exception as e:
+        print(f"Error finding system fonts: {e}")
+
+    return font_files
+
+r_directory = os.path.join(os.path.dirname(__file__), '../assets/')
+
+font_files = get_font_files(r_directory)
+# print(font_files)
+
+
+class ColorInput:
+    @classmethod
+    def INPUT_TYPES(s):
+        return {"required": { 
+             
+                    "color":("TCOLOR",), 
+                             },
+                }
+    
+    RETURN_TYPES = ("STRING",)
+    # RETURN_NAMES = ("WIDTH","HEIGHT","X","Y",)
+
+    FUNCTION = "run"
+
+    CATEGORY = "♾️Mixlab/utils"
+
+    INPUT_IS_LIST = False
+    OUTPUT_IS_LIST = (False,False,)
+
+    def run(self,color):
+        return (color,)
+
+
+
+class FontInput:
+    @classmethod
+    def INPUT_TYPES(s):
+        return {"required": { 
+             
+                    "font": (list(font_files.keys()),),
+                             },
+                }
+    
+    RETURN_TYPES = ("STRING",)
+    # RETURN_NAMES = ("WIDTH","HEIGHT","X","Y",)
+
+    FUNCTION = "run"
+
+    CATEGORY = "♾️Mixlab/utils"
+
+    INPUT_IS_LIST = False
+    OUTPUT_IS_LIST = (False,False,)
+
+    def run(self,font):
+
+        return (font_files[font],)
+    
+class TextToNumber:
+    @classmethod
+    def INPUT_TYPES(s):
+        return {"required": { 
+                    "text": ("STRING",{"multiline": False,"default": "1"}),
+                    "random_number": (["enable", "disable"],),
+                    "number":("INT", {
+                        "default": 0, 
+                        "min": 0, #Minimum value
+                        "max": 10000000000, #Maximum value
+                        "step": 1, #Slider's step
+                        "display": "number" # Cosmetic only: display as "number" or "slider"
+                    }),
+                             },
+                }
+    
+    RETURN_TYPES = ("INT",)
+    # RETURN_NAMES = ("WIDTH","HEIGHT","X","Y",)
+
+    FUNCTION = "run"
+
+    CATEGORY = "♾️Mixlab/utils"
+
+    INPUT_IS_LIST = False
+    OUTPUT_IS_LIST = (False,)
+
+    def run(self,text,random_number,number):
+        
+        numbers = re.findall(r'\d+', text)
+        result=0
+        for n in numbers:
+            result = int(n)
+            print(result)
+        
+        if random_number=='enable' and result>0:
+            result= random.randint(1, 10000000000)
+        return {"ui": {"text": [text],"num":[result]}, "result": (result,)}

nodes/Vae.py

@@ -145,7 +145,7 @@ class VAELoader:
     RETURN_TYPES = ("VAE",)
     FUNCTION = "load_vae"
 
-    CATEGORY = "♾️Mixlab/ConsistencyDecoder"
+    CATEGORY = "♾️Mixlab/_test"
 
     #TODO: scale factor?
     def load_vae(self, vae_name):
@@ -165,7 +165,7 @@ class VAEDecode:
     RETURN_TYPES = ("IMAGE",)
     FUNCTION = "decode"
 
-    CATEGORY = "♾️Mixlab/ConsistencyDecoder"
+    CATEGORY = "♾️Mixlab/_test"
 
     def decode(self, vae, samples):
         image = vae.decode(samples["samples"].to("cuda:0"))

web/javascript/3d_mixlab.js

@@ -0,0 +1,608 @@
+import { app } from '../../../scripts/app.js'
+import { api } from '../../../scripts/api.js'
+import { $el } from '../../../scripts/ui.js'
+
+const getLocalData = key => {
+  let data = {}
+  try {
+    data = JSON.parse(localStorage.getItem(key)) || {}
+  } catch (error) {
+    return {}
+  }
+  return data
+}
+function getContentTypeFromBase64 (base64Data) {
+  const regex = /^data:(.+);base64,/
+  const matches = base64Data.match(regex)
+  if (matches && matches.length >= 2) {
+    return matches[1]
+  }
+  return null
+}
+function base64ToBlobFromURL (base64URL, contentType) {
+  return fetch(base64URL).then(response => response.blob())
+}
+const setLocalDataOfWin = (key, value) => {
+  localStorage.setItem(key, JSON.stringify(value))
+  // window[key] = value
+}
+async function uploadImage (blob, fileType = '.svg', filename) {
+  // const blob = await (await fetch(src)).blob();
+  const body = new FormData()
+  body.append(
+    'image',
+    new File([blob], (filename || new Date().getTime()) + fileType)
+  )
+
+  const resp = await api.fetchApi('/upload/image', {
+    method: 'POST',
+    body
+  })
+
+  // console.log(resp)
+  let data = await resp.json()
+  let { name, subfolder } = data
+  let src = api.apiURL(
+    `/view?filename=${encodeURIComponent(
+      name
+    )}&type=input&subfolder=${subfolder}${app.getPreviewFormatParam()}${app.getRandParam()}`
+  )
+
+  return src
+}
+
+function createImage (url) {
+  let im = new Image()
+  return new Promise((res, rej) => {
+    im.onload = () => res(im)
+    im.src = url
+  })
+}
+
+const parseImage = url => {
+  return new Promise((res, rej) => {
+    fetch(url)
+      .then(response => response.blob())
+      .then(blob => {
+        const reader = new FileReader()
+        reader.onloadend = () => {
+          const base64data = reader.result
+          res(base64data)
+          // 在这里可以将base64数据用于进一步处理或显示图片
+        }
+        reader.readAsDataURL(blob)
+      })
+      .catch(error => {
+        console.log('发生错误:', error)
+      })
+  })
+}
+
+function get_position_style (ctx, widget_width, y, node_height) {
+  const MARGIN = 4 // the margin around the html element
+
+  /* Create a transform that deals with all the scrolling and zooming */
+  const elRect = ctx.canvas.getBoundingClientRect()
+  const transform = new DOMMatrix()
+    .scaleSelf(
+      elRect.width / ctx.canvas.width,
+      elRect.height / ctx.canvas.height
+    )
+    .multiplySelf(ctx.getTransform())
+    .translateSelf(MARGIN, MARGIN + y)
+
+  return {
+    transformOrigin: '0 0',
+    transform: transform,
+    left: `0`,
+    top: `0`,
+    cursor: 'pointer',
+    position: 'absolute',
+    maxWidth: `${widget_width - MARGIN * 2}px`,
+    // maxHeight: `${node_height - MARGIN * 2}px`, // we're assuming we have the whole height of the node
+    width: `${widget_width - MARGIN * 2}px`,
+    // height: `${node_height * 0.3 - MARGIN * 2}px`,
+    // background: '#EEEEEE',
+    display: 'flex',
+    flexDirection: 'column',
+    // alignItems: 'center',
+    justifyContent: 'space-around'
+  }
+}
+
+async function extractMaterial (
+  modelViewerVariants,
+  selectMaterial,
+  material_img
+) {
+  // 材质
+  const materialsNames = []
+  for (
+    let index = 0;
+    index < modelViewerVariants.model.materials.length;
+    index++
+  ) {
+    let m = modelViewerVariants.model.materials[index]
+    let thumbUrl
+    try {
+      thumbUrl =
+        await m.pbrMetallicRoughness.baseColorTexture.texture.source.createThumbnail(
+          1024,
+          1024
+        )
+    } catch (error) {}
+    if (thumbUrl)
+      materialsNames.push({
+        value: m.name,
+        text: `#${index} ${m.name}`,
+        index,
+        thumbUrl
+      })
+  }
+
+  selectMaterial.innerHTML = ''
+  material_img.innerHTML = ''
+
+  for (let index = 0; index < materialsNames.length; index++) {
+    const name = materialsNames[index]
+    const option = document.createElement('option')
+    option.value = name.thumbUrl
+    option.textContent = name.text
+    option.setAttribute('data-index', index)
+    selectMaterial.appendChild(option)
+    let img = new Image()
+    img.src = name.thumbUrl
+    // img.setAttribute('data-index',name.index)
+    img.style.width = '40px'
+    material_img.appendChild(img)
+    if (index == 0) {
+      material_img.setAttribute('src', name.thumbUrl)
+    }
+  }
+}
+
+async function changeMaterial (
+  modelViewerVariants,
+  targetMaterial,
+  newImageUrl
+) {
+  const targetTexture = await modelViewerVariants.createTexture(newImageUrl)
+  // 用图片创建纹理
+  targetMaterial.pbrMetallicRoughness.baseColorTexture.setTexture(targetTexture)
+}
+
+app.registerExtension({
+  name: 'Mixlab.3D.3DImage',
+  async getCustomWidgets (app) {
+    return {
+      THREED (node, inputName, inputData, app) {
+        // console.log('##node', node, inputName, inputData)
+        const widget = {
+          type: inputData[0], // the type, CHEESE
+          name: inputName, // the name, slice
+          size: [128, 88], // a default size
+          draw (ctx, node, width, y) {},
+          computeSize (...args) {
+            return [128, 88] // a method to compute the current size of the widget
+          },
+          async serializeValue (nodeId, widgetIndex) {
+            let d = getLocalData('_mixlab_3d_image')
+            console.log('serializeValue', node)
+            if (d && d[node.id]) {
+              let { url, bg, material } = d[node.id]
+              let base64 = await parseImage(url)
+              let bg_base64 = await parseImage(bg)
+              let material_base64 = await parseImage(material)
+
+              return JSON.parse(
+                JSON.stringify({
+                  image: base64,
+                  bg_image: bg_base64,
+                  material: material_base64
+                })
+              )
+            } else {
+              return {}
+            }
+          }
+        }
+        node.addCustomWidget(widget)
+        return widget
+      }
+    }
+  },
+
+  async beforeRegisterNodeDef (nodeType, nodeData, app) {
+    if (nodeType.comfyClass == '3DImage') {
+      const orig_nodeCreated = nodeType.prototype.onNodeCreated
+      nodeType.prototype.onNodeCreated = async function () {
+        orig_nodeCreated?.apply(this, arguments)
+
+        const uploadWidget = this.widgets.filter(w => w.name == 'upload')[0]
+
+        const widget = {
+          type: 'div',
+          name: 'upload-preview',
+          draw (ctx, node, widget_width, y, widget_height) {
+            Object.assign(
+              this.div.style,
+              get_position_style(ctx, widget_width, 88, node.size[1])
+            )
+          }
+        }
+
+        widget.div = $el('div', {})
+        widget.div.style.width = `120px`
+
+        document.body.appendChild(widget.div)
+
+        const inputDiv = (key, placeholder, preview) => {
+          let div = document.createElement('div')
+          const ip = document.createElement('input')
+          ip.type = 'file'
+          ip.className = `${'comfy-multiline-input'} ${placeholder}`
+          div.style = `display: flex;
+            align-items: center; 
+            margin: 6px 8px;
+            margin-top: 0;`
+          ip.placeholder = placeholder
+          // ip.value = value
+
+          ip.style = `outline: none;
+            border: none;
+            padding: 4px;
+            width: 60%;cursor: pointer;
+            height: 32px;`
+          const label = document.createElement('label')
+          label.style = 'font-size: 10px;min-width:32px'
+          label.innerText = placeholder
+          div.appendChild(label)
+          div.appendChild(ip)
+
+          let that = this,
+            filename = new Date().getTime()
+
+          ip.addEventListener('change', async event => {
+            const file = event.target.files[0]
+            const reader = new FileReader()
+            filename = new Date().getTime()
+            // 读取文件内容
+            reader.onload = async e => {
+              const fileURL = URL.createObjectURL(file)
+              // console.log('文件URL: ', fileURL)
+              let html = `<model-viewer  src="${fileURL}" 
+                min-field-of-view="0deg" max-field-of-view="180deg"
+                 shadow-intensity="1" 
+                 camera-controls 
+                 touch-action="pan-y">
+                 
+                 <div class="controls">
+                  <div>Variant: <select class="variant"></select></div>
+                  <div>Material: <select class="material"></select></div>
+                  <div>Material: <div class="material_img"> </div></div>
+                  <div><button class="bg">BG</button></div>
+                </div></model-viewer>`
+
+              preview.innerHTML = html
+              if (that.size[1] < 400) {
+                that.setSize([that.size[0], that.size[1] + 300])
+                app.canvas.draw(true, true)
+              }
+
+              const modelViewerVariants = preview.querySelector('model-viewer')
+              const select = preview.querySelector('.variant')
+              const selectMaterial = preview.querySelector('.material')
+              const material_img = preview.querySelector('.material_img')
+              const bg = preview.querySelector('.bg')
+
+              if (modelViewerVariants) {
+                modelViewerVariants.style.width = `${that.size[0] - 24}px`
+                modelViewerVariants.style.height = `${that.size[1] - 48}px`
+              }
+
+              modelViewerVariants.addEventListener('load', async () => {
+                const names = modelViewerVariants.availableVariants
+
+                // 变量
+                for (const name of names) {
+                  const option = document.createElement('option')
+                  option.value = name
+                  option.textContent = name
+                  select.appendChild(option)
+                }
+                // Adds a default option.
+                if (names.length === 0) {
+                  const option = document.createElement('option')
+                  option.value = 'default'
+                  option.textContent = 'Default'
+                  select.appendChild(option)
+                }
+
+                // 材质
+                extractMaterial(
+                  modelViewerVariants,
+                  selectMaterial,
+                  material_img
+                )
+              })
+
+              let timer = null
+              const delay = 500 // 延迟时间，单位为毫秒
+
+              async function checkCameraChange () {
+                let dd = getLocalData(key)
+                let base64Data = modelViewerVariants.toDataURL()
+
+                const contentType = getContentTypeFromBase64(base64Data)
+
+                const blob = await base64ToBlobFromURL(base64Data, contentType)
+
+                //  const fileBlob = new Blob([e.target.result], { type: file.type });
+                let url = await uploadImage(blob, '.png')
+                // console.log(url)
+
+                //  材质贴图
+                let thumbUrl = material_img.getAttribute('src')
+                let tb = await base64ToBlobFromURL(thumbUrl)
+                let tUrl = await uploadImage(tb, '.png')
+
+                // console.log(tUrl)
+
+                let bg_blob = await base64ToBlobFromURL(
+                  'data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mN88uXrPQAFwwK/6xJ6CQAAAABJRU5ErkJggg=='
+                )
+                let url_bg = await uploadImage(bg_blob, '.png')
+
+                if (!dd[that.id])
+                  dd[that.id] = { url, bg: url_bg, material: tUrl }
+                dd[that.id] = { ...dd[that.id], url, material: tUrl }
+
+                setLocalDataOfWin(key, dd)
+              }
+
+              function startTimer () {
+                if (timer) clearTimeout(timer)
+                timer = setTimeout(checkCameraChange, delay)
+              }
+
+              modelViewerVariants.addEventListener('camera-change', startTimer)
+
+              select.addEventListener('input', event => {
+                modelViewerVariants.variantName =
+                  event.target.value === 'default' ? null : event.target.value
+                // 材质
+                extractMaterial(
+                  modelViewerVariants,
+                  selectMaterial,
+                  material_img
+                )
+                checkCameraChange()
+              })
+
+              selectMaterial.addEventListener('input', event => {
+                // console.log(selectMaterial.value)
+                material_img.setAttribute('src', selectMaterial.value)
+
+                if (selectMaterial.getAttribute('data-new-material')) {
+                  let index =
+                    ~~selectMaterial.selectedOptions[0].getAttribute(
+                      'data-index'
+                    )
+                  changeMaterial(
+                    modelViewerVariants,
+                    modelViewerVariants.model.materials[index],
+                    selectMaterial.getAttribute('data-new-material')
+                  )
+                }
+
+                checkCameraChange()
+              })
+
+              bg.addEventListener('click', () => {
+                // 创建一个input元素
+                var input = document.createElement('input')
+                input.type = 'file'
+
+                // 监听input的change事件
+                input.addEventListener('change', function () {
+                  // 获取上传的文件
+                  var file = input.files[0]
+
+                  // 创建一个FileReader对象来读取文件
+                  var reader = new FileReader()
+
+                  // 监听FileReader的load事件
+                  reader.addEventListener('load', async () => {
+                    let base64 = reader.result
+                    // 将读取的文件内容设置为div的背景
+                    preview.style.backgroundImage = 'url(' + base64 + ')'
+
+                    const contentType = getContentTypeFromBase64(base64)
+
+                    const blob = await base64ToBlobFromURL(base64, contentType)
+
+                    //  const fileBlob = new Blob([e.target.result], { type: file.type });
+                    let bg_url = await uploadImage(blob, '.png')
+                    let bg_img = await createImage(base64)
+
+                    let dd = getLocalData(key)
+                    // console.log(dd[that.id],bg_url)
+                    if (!dd[that.id]) dd[that.id] = { url: '', bg: bg_url }
+                    dd[that.id] = {
+                      ...dd[that.id],
+                      bg: bg_url,
+                      bg_w: bg_img.naturalWidth,
+                      bg_h: bg_img.naturalHeight
+                    }
+
+                    setLocalDataOfWin(key, dd)
+
+                    // 更新尺寸
+                    let w = that.size[0] - 24,
+                      h = (w * bg_img.naturalHeight) / bg_img.naturalWidth
+
+                    if (modelViewerVariants) {
+                      modelViewerVariants.style.width = `${w}px`
+                      modelViewerVariants.style.height = `${h}px`
+                    }
+                    preview.style.width = `${w}px`
+                  })
+
+                  // 读取文件
+                  reader.readAsDataURL(file)
+                })
+
+                // 触发input的点击事件
+                input.click()
+              })
+
+              uploadWidget.value = await uploadWidget.serializeValue()
+
+              // 更新尺寸
+              let dd = getLocalData(key)
+              // console.log(dd[that.id],bg_url)
+              if (dd[that.id]) {
+                const { bg_w, bg_h } = dd[that.id]
+                if (bg_h && bg_w) {
+                  let w = that.size[0] - 24,
+                    h = (w * bg_h) / bg_w
+
+                  if (modelViewerVariants) {
+                    modelViewerVariants.style.width = `${w}px`
+                    modelViewerVariants.style.height = `${h}px`
+                  }
+                  preview.style.width = `${w}px`
+                }
+              }
+            }
+
+            // 以文本形式读取文件
+            reader.readAsDataURL(file)
+          })
+          return div
+        }
+
+        let preview = document.createElement('div')
+        preview.className = 'preview'
+        preview.style = `margin-top: 12px;display: flex;
+          justify-content: center;
+          align-items: center;background-repeat: no-repeat;background-size: contain;`
+
+        let upload = inputDiv('_mixlab_3d_image', '3D Model', preview)
+
+        widget.div.appendChild(upload)
+        widget.div.appendChild(preview)
+        this.addCustomWidget(widget)
+
+        const onResize = this.onResize
+        let that = this
+        this.onResize = function () {
+          let modelViewerVariants = preview.querySelector('model-viewer')
+
+          // 更新尺寸
+          let dd = getLocalData('_mixlab_3d_image')
+          // console.log(dd[that.id],bg_url)
+          if (dd[that.id]) {
+            const { bg_w, bg_h } = dd[that.id]
+            if (bg_h && bg_w) {
+              let w = that.size[0] - 24,
+                h = (w * bg_h) / bg_w
+
+              if (modelViewerVariants) {
+                modelViewerVariants.style.width = `${w}px`
+                modelViewerVariants.style.height = `${h}px`
+              }
+              preview.style.width = `${w}px`
+            }
+          }
+
+          return onResize?.apply(this, arguments)
+        }
+
+        const onRemoved = this.onRemoved
+        this.onRemoved = () => {
+          upload.remove()
+          preview.remove()
+          widget.div.remove()
+          return onRemoved?.()
+        }
+
+        if (this.onResize) {
+          this.onResize(this.size)
+        }
+        // this.isVirtualNode = true
+        this.serialize_widgets = false //需要保存参数
+      }
+
+      const onExecuted = nodeType.prototype.onExecuted
+      nodeType.prototype.onExecuted = function (message) {
+        const r = onExecuted?.apply?.(this, arguments)
+
+        let div = this.widgets.filter(d => d.div)[0]?.div
+        console.log('Test', this.widgets)
+
+        let material = message.material[0]
+        if (material) {
+          const { filename, subfolder, type } = material
+          let src = api.apiURL(
+            `/view?filename=${encodeURIComponent(
+              filename
+            )}&type=${type}&subfolder=${subfolder}${app.getPreviewFormatParam()}${app.getRandParam()}`
+          )
+
+          const modelViewerVariants = div.querySelector('model-viewer')
+
+          const selectMaterial = div.querySelector('.material')
+
+          let index =
+            ~~selectMaterial.selectedOptions[0].getAttribute('data-index')
+
+          selectMaterial.setAttribute('data-new-material', src)
+
+          changeMaterial(
+            modelViewerVariants,
+            modelViewerVariants.model.materials[index],
+            src
+          )
+        }
+
+        this.onResize?.(this.size)
+
+        return r
+      }
+    }
+  },
+  async loadedGraphNode (node, app) {
+    // Fires every time a node is constructed
+    // You can modify widgets/add handlers/etc here
+    const sleep = (t = 1000) => {
+      return new Promise((res, rej) => {
+        setTimeout(() => res(1), t)
+      })
+    }
+    if (node.type === '3DImage') {
+      // await sleep(0)
+      let widget = node.widgets.filter(w => w.name === 'upload-preview')[0]
+
+      let dd = getLocalData('_mixlab_3d_image')
+
+      let id = node.id
+      // console.log('3dImage load', node.widgets[0], node.widgets)
+      if (!dd[id]) return
+
+      let { url, bg } = dd[id]
+      if (!url) return
+      // let base64 = await parseImage(url)
+
+      let pre = widget.div.querySelector('.preview')
+      pre.style.width = `${node.size[0]}px`
+      pre.innerHTML = `
+        ${url ? `<img src="${url}" style="width:100%"/>` : ''}
+        `
+      pre.style.backgroundImage = 'url(' + bg + ')'
+
+      const uploadWidget = node.widgets.filter(w => w.name == 'upload')[0]
+      uploadWidget.value = await uploadWidget.serializeValue()
+    }
+  }
+})

web/javascript/audio_mixlab.js

@@ -0,0 +1,368 @@
+import { app } from '../../../scripts/app.js'
+// import { api } from '../../../scripts/api.js'
+import { ComfyWidgets } from '../../../scripts/widgets.js'
+import { $el } from '../../../scripts/ui.js'
+
+function get_position_style (ctx, widget_width, y, node_height) {
+  const MARGIN = 4 // the margin around the html element
+
+  /* Create a transform that deals with all the scrolling and zooming */
+  const elRect = ctx.canvas.getBoundingClientRect()
+  const transform = new DOMMatrix()
+    .scaleSelf(
+      elRect.width / ctx.canvas.width,
+      elRect.height / ctx.canvas.height
+    )
+    .multiplySelf(ctx.getTransform())
+    .translateSelf(MARGIN, MARGIN + y)
+
+  return {
+    transformOrigin: '0 0',
+    transform: transform,
+    left: `0`,
+    top: `0`,
+    cursor: 'pointer',
+    position: 'absolute',
+    maxWidth: `${widget_width - MARGIN * 2}px`,
+    // maxHeight: `${node_height - MARGIN * 2}px`, // we're assuming we have the whole height of the node
+    width: `${widget_width - MARGIN * 2}px`,
+    // height: `${node_height * 0.3 - MARGIN * 2}px`,
+    // background: '#EEEEEE',
+    display: 'flex',
+    flexDirection: 'column',
+    // alignItems: 'center',
+    justifyContent: 'space-around'
+  }
+}
+
+const getLocalData = key => {
+  let data = {}
+  try {
+    data = JSON.parse(localStorage.getItem(key)) || {}
+  } catch (error) {
+    return {}
+  }
+  return data
+}
+
+function speakText (text) {
+  const speechMsg = new SpeechSynthesisUtterance()
+  speechMsg.text = text
+
+  // 语音合成结束时触发的事件
+  speechMsg.onend = function (event) {
+    console.log('语音播放结束')
+    window._mixlab_speech_synthesis_onend = true
+  }
+
+  // 语音合成错误时触发的事件
+  speechMsg.onerror = function (event) {
+    console.error('语音播放错误:', event.error)
+  }
+
+  // 使用浏览器默认语音合成器进行语音播放
+  speechSynthesis.speak(speechMsg)
+}
+
+// 调用方法，将文字转换为语音播放
+// speakText('Hello, how are you?');
+//                      #MixCopilot
+
+const start = (element, id, startBtn) => {
+  startBtn.className = 'loading_mixlab'
+
+  window.recognition = new webkitSpeechRecognition()
+
+  window.recognition.continuous = true
+  window.recognition.interimResults = true
+  window.recognition.lang = navigator.language
+
+  let timeoutId, intervalId
+
+  window.recognition.onstart = () => {
+    console.log('开始语音输入', window._mixlab_speech_synthesis_onend)
+    window._mixlab_speech_synthesis_onend = false
+  }
+
+  window.recognition.onresult = function (event) {
+    const result = event.results[event.results.length - 1][0].transcript
+    console.log('识别结果：', result)
+    element.value = result
+
+    let data = getLocalData('_mixlab_speech_recognition')
+    data[id] = result.trim()
+    localStorage.setItem('_mixlab_speech_recognition', JSON.stringify(data))
+
+    if (timeoutId) clearTimeout(timeoutId)
+
+    if (!window.recognition) return
+
+    timeoutId = setTimeout(function () {
+      console.log('结果传递：：', result)
+      app.queuePrompt(0, 1)
+      window.recognition?.stop()
+      window.recognition = null
+      startBtn.className = ''
+      startBtn.innerText = 'START'
+
+      timeoutId = null
+
+      intervalId = setInterval(() => {
+        if (
+          app.ui.lastQueueSize === 0 &&
+          !window.recognition &&
+          window._mixlab_speech_synthesis_onend
+        ) {
+          start(element, id, startBtn)
+          startBtn.innerText = 'STOP'
+          if (intervalId) {
+            clearInterval(intervalId)
+          }
+        }
+      }, 2200)
+    }, 2000)
+  }
+
+  window.recognition.onend = function () {
+    console.log('语音输入结束')
+  }
+
+  window.recognition.onspeechend = function () {
+    console.log('onspeechend')
+  }
+
+  window.recognition.onerror = function (event) {
+    console.log('Error occurred in recognition: ' + event.error)
+  }
+
+  window.recognition.start()
+}
+
+app.registerExtension({
+  name: 'Mixlab.audio.SpeechRecognition',
+  async getCustomWidgets (app) {
+    return {
+      AUDIOINPUTMIX (node, inputName, inputData, app) {
+        // console.log('##node', node)
+        const widget = {
+          type: inputData[0], // the type, CHEESE
+          name: inputName, // the name, slice
+          size: [128, 32], // a default size
+          draw (ctx, node, width, y) {},
+          computeSize (...args) {
+            return [128, 32] // a method to compute the current size of the widget
+          },
+          async serializeValue (nodeId, widgetIndex) {
+            let data = getLocalData('_mixlab_speech_recognition')
+            return data[node.id] || 'Hello Mixlab'
+          }
+        }
+        //  widget.something = something;          // maybe adds stuff to it
+        node.addCustomWidget(widget) // adds it to the node
+        return widget // and returns it.
+      }
+    }
+  },
+
+  async beforeRegisterNodeDef (nodeType, nodeData, app) {
+    if (nodeType.comfyClass == 'SpeechRecognition') {
+      const orig_nodeCreated = nodeType.prototype.onNodeCreated
+      nodeType.prototype.onNodeCreated = function () {
+        orig_nodeCreated?.apply(this, arguments)
+
+        const widget = {
+          type: 'div',
+          name: 'chatgptdiv',
+          draw (ctx, node, widget_width, y, widget_height) {
+            Object.assign(
+              this.div.style,
+              get_position_style(ctx, widget_width, 44, node.size[1])
+            )
+          }
+        }
+
+        widget.div = $el('div', {})
+
+        document.body.appendChild(widget.div)
+
+        const inputDiv = (key, placeholder) => {
+          let div = document.createElement('div')
+          const startBtn = document.createElement('button')
+          const textArea = document.createElement('textarea')
+
+          textArea.className = `${'comfy-multiline-input'} ${placeholder}`
+
+          textArea.style = `margin-top: 14px;
+          height: 44px;`
+
+          div.style = `flex-direction: column;
+          display: flex;
+          margin: 0px 8px 6px;`
+
+          startBtn.style = `
+          margin-top:48px;
+          background-color: var(--comfy-input-bg);
+          border-radius: 8px;
+          border-color: var(--border-color);
+          border-style: solid;
+          color: var(--descrip-text);
+          `
+
+          startBtn.innerText = 'START'
+
+          div.appendChild(startBtn)
+          div.appendChild(textArea)
+
+          startBtn.addEventListener('click', () => {
+            if (window.recognition) {
+              window.recognition.stop()
+              window.recognition = null
+              startBtn.innerText = 'START'
+              startBtn.className = ''
+            } else {
+              start(textArea, this.id, startBtn)
+              startBtn.innerText = 'STOP'
+            }
+          })
+
+          return div
+        }
+
+        let inputAudio = inputDiv('_mixlab_speech_recognition', 'audio')
+        widget.div.appendChild(inputAudio)
+
+        this.addCustomWidget(widget)
+
+        const onRemoved = this.onRemoved
+        this.onRemoved = () => {
+          inputAudio.remove()
+          widget.div.remove()
+          return onRemoved?.()
+        }
+
+        this.serialize_widgets = true //需要保存参数
+      }
+
+      
+
+      // const onGraphConfigured=nodeType.prototype.onGraphConfigured;
+      // nodeType.prototype.onGraphConfigured = function (message) {
+      //   onGraphConfigured?.apply(this, arguments)
+      //   console.log('###SpeechRecognition onGraphConfigured',this,message)
+      // }
+
+      const onExecuted = nodeType.prototype.onExecuted
+      nodeType.prototype.onExecuted = function (message) {
+        onExecuted?.apply(this, arguments)
+        
+        try {
+          let open = message.start_by[0] > 0
+          if (open) {
+            const div = this.widgets.filter(w => w.name == 'chatgptdiv')[0].div
+            const startBtn = div.querySelector('button')
+            let textArea= div.querySelector('textarea')
+            if (open && !window.recognition) {
+              start(textArea, this.id, startBtn)
+              startBtn.innerText = 'STOP'
+            } else if (!open && window.recognition) {
+              window.recognition.stop()
+              window.recognition = null
+              startBtn.innerText = 'START'
+              startBtn.className = ''
+            }
+          }
+        } catch (error) {
+          console.log('###SpeechRecognition', error)
+        }
+      }
+    }
+  },
+  async loadedGraphNode (node, app) {
+    if (node.type === 'SpeechRecognition') {
+      let data = getLocalData('_mixlab_speech_recognition')
+      // console.log('_mixlab_speech_recognition', node )
+      let div = node.widgets.filter(f => f.type === 'div')[0]
+      if (div && data[node.id]) {
+        div.div.querySelector('textarea').value = data[node.id]
+      };
+
+      try {
+        let open = node.widgets_values[1] > 0
+        if (open) {
+          const div = node.widgets.filter(w => w.name == 'chatgptdiv')[0].div
+          const startBtn = div.querySelector('button')
+          let textArea= div.querySelector('textarea')
+          if (open && !window.recognition) {
+            start(textArea, node.id, startBtn)
+            startBtn.innerText = 'STOP'
+          } else if (!open && window.recognition) {
+            window.recognition.stop()
+            window.recognition = null
+            startBtn.innerText = 'START'
+            startBtn.className = ''
+          }
+        }
+      } catch (error) {
+        console.log('###SpeechRecognition', error)
+      }
+ 
+    }
+  }
+})
+
+app.registerExtension({
+  name: 'Mixlab.audio.SpeechSynthesis',
+  async beforeRegisterNodeDef (nodeType, nodeData, app) {
+    if (nodeData.name === 'SpeechSynthesis') {
+      function populate (text) {
+        // console.log('SpeechSynthesis',this.widgets)
+
+        if (this.widgets) {
+          const pos = this.widgets.findIndex(w => w.name === 'text')
+          if (pos !== -1) {
+            for (let i = pos; i < this.widgets.length; i++) {
+              this.widgets[i].onRemove?.()
+            }
+            this.widgets.length = pos
+          }
+        }
+
+        for (let list of text) {
+          const w = ComfyWidgets['STRING'](
+            this,
+            'text',
+            ['STRING', { multiline: true }],
+            app
+          ).widget
+          w.inputEl.readOnly = true
+          w.inputEl.style.opacity = 0.6
+          w.value = list
+        }
+
+        speakText(text.join('\n'))
+
+        // console.log('ShowTextForGPT',this.widgets.length)
+        requestAnimationFrame(() => {
+          const sz = this.computeSize()
+          if (sz[0] < this.size[0]) {
+            sz[0] = this.size[0]
+          }
+          if (sz[1] < this.size[1]) {
+            sz[1] = this.size[1]
+          }
+          this.onResize?.(sz)
+          app.graph.setDirtyCanvas(true, false)
+        })
+      }
+
+      // When the node is executed we will be sent the input text, display this in the widget
+      const onExecuted = nodeType.prototype.onExecuted
+      nodeType.prototype.onExecuted = function (message) {
+        onExecuted?.apply(this, arguments)
+        populate.call(this, message.text)
+      }
+
+      this.serialize_widgets = true //需要保存参数
+    }
+  }
+})

web/javascript/checkVersion_mixlab.js

@@ -3,7 +3,7 @@ import { app } from '../../../scripts/app.js'
 const repoOwner = 'shadowcz007' // 替换为仓库的所有者
 const repoName = 'comfyui-mixlab-nodes' // 替换为仓库的名称
 
-const version = 'v0.2.7'
+const version = 'v0.4.2'
 
 fetch(`https://api.github.com/repos/${repoOwner}/${repoName}/releases/latest`)
   .then(response => response.json())

web/javascript/gpt_mixlab.js

@@ -192,7 +192,7 @@ app.registerExtension({
 
       let id = node.id
 
-      console.log('ChatGPTOpenAI serialize_widgets', this)
+      // console.log('ChatGPTOpenAI serialize_widgets', this)
 
       widget.div.querySelector('.Key').value = apiKey[id] || 'by Mixlab'
       widget.div.querySelector('.URL').value =
@@ -216,12 +216,27 @@ app.registerExtension({
 						this.widgets.length = pos;
 					}
 				}
-        // console.log('ShowTextForGPT',this.widgets.length)
-				for (const list of text) {
+        // console.log('ShowTextForGPT',text)
+				for (let list of text) {
 					const w = ComfyWidgets["STRING"](this, "text", ["STRING", { multiline: true }], app).widget;
 					w.inputEl.readOnly = true;
 					w.inputEl.style.opacity = 0.6;
-					w.value = list;
+
+          try {
+            let data=JSON.parse(list);
+            data=Array.from(data,d=>{
+              return {
+                ...d,
+                content:decodeURIComponent(d.content)
+              }
+            })
+            list=JSON.stringify(data,null,2)
+          } catch (error) {
+            // console.log(error)
+          }
+
+					w.value =list;
+        
 				}
         // console.log('ShowTextForGPT',this.widgets.length)
 				requestAnimationFrame(() => {

web/javascript/image_mixlab.js

@@ -0,0 +1,438 @@
+import { app } from '../../../scripts/app.js'
+import { api } from '../../../scripts/api.js'
+import { ComfyWidgets } from '../../../scripts/widgets.js'
+import { $el } from '../../../scripts/ui.js'
+
+async function uploadImage (blob, fileType = '.svg', filename) {
+  // const blob = await (await fetch(src)).blob();
+  const body = new FormData()
+  body.append(
+    'image',
+    new File([blob], (filename || new Date().getTime()) + fileType)
+  )
+
+  const resp = await api.fetchApi('/upload/image', {
+    method: 'POST',
+    body
+  })
+
+  // console.log(resp)
+  let data = await resp.json()
+  let { name, subfolder } = data
+  let src = api.apiURL(
+    `/view?filename=${encodeURIComponent(
+      name
+    )}&type=input&subfolder=${subfolder}${app.getPreviewFormatParam()}${app.getRandParam()}`
+  )
+
+  return src
+}
+
+function base64ToBlobFromURL (base64URL, contentType) {
+  return fetch(base64URL).then(response => response.blob())
+}
+
+function getContentTypeFromBase64 (base64Data) {
+  const regex = /^data:(.+);base64,/
+  const matches = base64Data.match(regex)
+  if (matches && matches.length >= 2) {
+    return matches[1]
+  }
+  return null
+}
+
+// 示例用法
+// const base64Data = 'data:image/jpeg;base64,/9j/4AAQSkZJRgABAQEAAAAAAAD/...'; // 替换为实际的base64图片数据
+// const contentType = getContentTypeFromBase64(base64Data);
+// console.log(contentType);
+
+// // 示例用法
+// const base64Data = '...'; // 替换为实际的base64图片数据
+// const contentType = 'image/jpeg'; // 替换为实际的图片类型
+
+// const blob = base64ToBlob(base64Data, contentType);
+// console.log(blob);
+
+function get_position_style (ctx, widget_width, y, node_height) {
+  const MARGIN = 4 // the margin around the html element
+
+  /* Create a transform that deals with all the scrolling and zooming */
+  const elRect = ctx.canvas.getBoundingClientRect()
+  const transform = new DOMMatrix()
+    .scaleSelf(
+      elRect.width / ctx.canvas.width,
+      elRect.height / ctx.canvas.height
+    )
+    .multiplySelf(ctx.getTransform())
+    .translateSelf(MARGIN, MARGIN + y)
+
+  return {
+    transformOrigin: '0 0',
+    transform: transform,
+    left: `0`,
+    top: `0`,
+    cursor: 'pointer',
+    position: 'absolute',
+    maxWidth: `${widget_width - MARGIN * 2}px`,
+    // maxHeight: `${node_height - MARGIN * 2}px`, // we're assuming we have the whole height of the node
+    width: `${widget_width - MARGIN * 2}px`,
+    // height: `${node_height * 0.3 - MARGIN * 2}px`,
+    // background: '#EEEEEE',
+    display: 'flex',
+    flexDirection: 'column',
+    // alignItems: 'center',
+    justifyContent: 'space-around'
+  }
+}
+
+const getLocalData = key => {
+  let data = {}
+  try {
+    data = JSON.parse(localStorage.getItem(key)) || {}
+  } catch (error) {
+    return {}
+  }
+  return data
+}
+
+const setLocalDataOfWin = (key, value) => {
+  localStorage.setItem(key, JSON.stringify(value))
+  // window[key] = value
+}
+
+function createImage (url) {
+  let im = new Image()
+  return new Promise((res, rej) => {
+    im.onload = () => res(im)
+    im.src = url
+  })
+}
+
+const parseImage = url => {
+  return new Promise((res, rej) => {
+    fetch(url)
+      .then(response => response.blob())
+      .then(blob => {
+        const reader = new FileReader()
+        reader.onloadend = () => {
+          const base64data = reader.result
+          res(base64data)
+          // 在这里可以将base64数据用于进一步处理或显示图片
+        }
+        reader.readAsDataURL(blob)
+      })
+      .catch(error => {
+        console.log('发生错误:', error)
+      })
+  })
+}
+
+const parseSvg = async svgContent => {
+  let scale = 2
+  // 创建一个临时的DOM元素来解析SVG
+  const tempContainer = document.createElement('div')
+  tempContainer.innerHTML = svgContent
+
+  // 提取SVG元素
+  const svgElement = tempContainer.querySelector('svg')
+  if (!svgElement) return
+  // 获取SVG中 rect元素
+  var rectElements = svgElement?.querySelectorAll('rect') || []
+  // console.log(rectElements,svgElement)
+  // 定义一个数组来存储处理后的数据
+  var data = []
+
+  Array.from(rectElements, (rectElement, i) => {
+    // 获取rect元素的属性值
+    var x = ~~(rectElement.getAttribute('x') || 0)
+    var y = ~~(rectElement.getAttribute('y') || 0)
+    var width = ~~rectElement.getAttribute('width')
+    var height = ~~rectElement.getAttribute('height')
+    // console.log('rectElements',rectElement,x,y,width,height)
+    if (x != undefined && y != undefined && width && height) {
+      // 创建一个新的canvas元素
+      var canvas = document.createElement('canvas')
+      canvas.width = width
+      canvas.height = height
+      var context = canvas.getContext('2d')
+
+      // 填充颜色到canvas
+      var fill = rectElement.getAttribute('fill')
+      context.fillStyle = fill
+      context.fillRect(0, 0, width, height)
+
+      // 将canvas转换为base64格式
+      var base64 = canvas.toDataURL()
+
+      // 将数据转化为指定的JSON格式
+
+      var rectData = {
+        x: parseInt(x),
+        y: parseInt(y),
+        width: parseInt(width),
+        height: parseInt(height),
+        z_index: i + 1,
+        scale_option: 'width',
+        image: base64,
+        mask: base64,
+        type: 'base64',
+        _t: 'rect'
+      }
+
+      // 将处理后的数据添加到数组中
+      data.push(rectData)
+    }
+  })
+
+  var svgWidth = svgElement.getAttribute('width')
+  var svgHeight = svgElement.getAttribute('height')
+
+  if (!(svgWidth && svgHeight)) {
+    // viewBox
+    let viewBox = svgElement.viewBox.baseVal
+
+    svgWidth = viewBox.width
+    svgHeight = viewBox.height
+  }
+
+  // 创建一个新的canvas元素
+  var canvas = document.createElement('canvas')
+  canvas.width = svgWidth
+  canvas.height = svgHeight
+  var context = canvas.getContext('2d')
+  // 绘制SVG到canvas
+  var svgString = new XMLSerializer().serializeToString(svgElement)
+  var DOMURL = window.URL || window.webkitURL || window
+
+  var svgBlob = new Blob([svgString], { type: 'image/svg+xml;charset=utf-8' })
+  var url = DOMURL.createObjectURL(svgBlob)
+
+  let img = await createImage(url)
+  context.drawImage(img, 0, 0)
+
+  let base64 = canvas.toDataURL()
+
+  var rectData = {
+    x: 0,
+    y: 0,
+    width: parseInt(svgWidth),
+    height: parseInt(svgHeight),
+    z_index: 0,
+    scale_option: 'width',
+    image: base64,
+    mask: base64,
+    type: 'base64',
+    _t: 'canvas'
+  }
+  data.push(rectData)
+
+  // 打印处理后的数据
+  console.log('layers', { data, image: base64, svgElement })
+  return { data, image: base64, svgElement }
+}
+
+function exportModelViewerImage (
+  modelViewer,
+  width,
+  height,
+  format = 'image/png',
+  quality = 1.0
+) {
+  const canvas = document.createElement('canvas')
+  canvas.width = width
+  canvas.height = height
+  const context = canvas.getContext('2d')
+
+  return new Promise((resolve, reject) => {
+    context.drawImage(modelViewer, 0, 0, width, height)
+
+    resolve(canvas.toDataURL(format, quality))
+  })
+}
+
+app.registerExtension({
+  name: 'Mixlab.image.SvgImage',
+  async getCustomWidgets (app) {
+    return {
+      SVG (node, inputName, inputData, app) {
+        // console.log('##node', node, inputName, inputData)
+        const widget = {
+          type: inputData[0], // the type, CHEESE
+          name: inputName, // the name, slice
+          size: [128, 88], // a default size
+          draw (ctx, node, width, y) {},
+          computeSize (...args) {
+            return [128, 88] // a method to compute the current size of the widget
+          },
+          async serializeValue (nodeId, widgetIndex) {
+            let d = getLocalData('_mixlab_svg_image')
+            // console.log('serializeValue',d)
+            if (d) {
+              let url = d[node.id]
+              let dt = await fetch(url)
+              let svgStr = await dt.text()
+              const { data, image } = (await parseSvg(svgStr)) || {}
+              // console.log(data, image)
+              return JSON.parse(JSON.stringify({ data, image }))
+            } else {
+              return
+            }
+          }
+        }
+
+        // console.log('##node',node.serialize)
+        //  widget.something = something;          // maybe adds stuff to it
+        node.addCustomWidget(widget) // adds it to the node
+
+        return widget // and returns it.
+      }
+    }
+  },
+
+  async beforeRegisterNodeDef (nodeType, nodeData, app) {
+    if (nodeType.comfyClass == 'SvgImage') {
+      const orig_nodeCreated = nodeType.prototype.onNodeCreated
+      nodeType.prototype.onNodeCreated = async function () {
+        orig_nodeCreated?.apply(this, arguments)
+
+        const uploadWidget = this.widgets.filter(w => w.name == 'upload')[0]
+        // console.log('SvgImage nodeData',await uploadWidget.serializeValue())
+
+        const widget = {
+          type: 'div',
+          name: 'upload-preview',
+          draw (ctx, node, widget_width, y, widget_height) {
+            Object.assign(
+              this.div.style,
+              get_position_style(ctx, widget_width, 44, node.size[1])
+            )
+          }
+        }
+
+        widget.div = $el('div', {})
+
+        document.body.appendChild(widget.div)
+
+        const inputDiv = (key, placeholder, svgContainer) => {
+          let div = document.createElement('div')
+          const ip = document.createElement('input')
+          ip.type = 'file'
+          ip.className = `${'comfy-multiline-input'} ${placeholder}`
+          div.style = `display: flex;
+          align-items: center; 
+          margin: 6px 8px;
+          margin-top: 0;`
+          ip.placeholder = placeholder
+          // ip.value = value
+
+          ip.style = `outline: none;
+          border: none;
+          padding: 4px;
+          width: 60%;cursor: pointer;
+          height: 32px;`
+          const label = document.createElement('label')
+          label.style = 'font-size: 10px;min-width:32px'
+          label.innerText = placeholder
+          div.appendChild(label)
+          div.appendChild(ip)
+
+          let that = this
+
+          ip.addEventListener('change', event => {
+            const file = event.target.files[0]
+            const reader = new FileReader()
+
+            // 读取文件内容
+            reader.onload = async e => {
+              const svgContent = e.target.result
+
+              var blob = new Blob([svgContent], { type: 'image/svg+xml' })
+              let url = await uploadImage(blob)
+              // console.log(url)
+              const { svgElement, data, image } = await parseSvg(svgContent)
+              // 将提取的SVG元素显示在页面上
+              let dd = getLocalData(key)
+              dd[that.id] = url
+              setLocalDataOfWin(key, dd)
+              // console.log(this.id, ip.value.trim())
+
+              svgElement.style = `width: 90%;padding: 5%;`
+              // 将提取的SVG元素显示在页面上
+
+              svgContainer.innerHTML = ''
+              svgContainer.appendChild(svgElement)
+              let h = ~~getComputedStyle(svgElement).height.replace('px', '')
+              if (that.size && that.size[1] < h) {
+                that.setSize([that.size[0], that.size[1] + h])
+                app.canvas.draw(true, true)
+              }
+              // console.log(that.size,~~getComputedStyle(svgElement).height.replace('px',''))
+
+              uploadWidget.value = await uploadWidget.serializeValue()
+            }
+
+            // 以文本形式读取文件
+            reader.readAsText(file)
+          })
+          return div
+        }
+
+        let svg = document.createElement('div')
+        svg.className = 'preview'
+        svg.style = `background:#eee;margin-top: 12px;`
+
+        let upload = inputDiv('_mixlab_svg_image', 'Svg', svg)
+
+        widget.div.appendChild(upload)
+        widget.div.appendChild(svg)
+        this.addCustomWidget(widget)
+
+        const onRemoved = this.onRemoved
+        this.onRemoved = () => {
+          upload.remove()
+          svg.remove()
+          widget.div.remove()
+          return onRemoved?.()
+        }
+
+        if (this.onResize) {
+          this.onResize(this.size)
+        }
+
+        this.serialize_widgets = true //需要保存参数
+      }
+    }
+  },
+  async loadedGraphNode (node, app) {
+    // Fires every time a node is constructed
+    // You can modify widgets/add handlers/etc here
+    const sleep = (t = 1000) => {
+      return new Promise((res, rej) => {
+        setTimeout(() => res(1), t)
+      })
+    }
+    if (node.type === 'SvgImage') {
+      // await sleep(0)
+      let widget = node.widgets.filter(w => w.name === 'upload-preview')[0]
+
+      let dd = getLocalData('_mixlab_svg_image')
+
+      let id = node.id
+      console.log('SvgImage load', node.widgets[0], node.widgets)
+      if (!dd[id]) return
+      let dt = await fetch(dd[id])
+      let svgStr = await dt.text()
+
+      const { svgElement, data, image } = await parseSvg(svgStr)
+      svgElement.style = `width: 90%;padding: 5%;`
+      // 将提取的SVG元素显示在页面上
+
+      widget.div.querySelector('.preview').innerHTML = ''
+      widget.div.querySelector('.preview').appendChild(svgElement)
+
+      const uploadWidget = node.widgets.filter(w => w.name == 'upload')[0]
+      uploadWidget.value = await uploadWidget.serializeValue()
+ 
+    }
+  }
+})

web/javascript/layer_mixlab.js

@@ -0,0 +1,559 @@
+import { app } from '../../../scripts/app.js'
+// import { api } from '../../../scripts/api.js'
+import { ComfyWidgets } from '../../../scripts/widgets.js'
+import { $el } from '../../../scripts/ui.js'
+
+function get_position_style (ctx, widget_width, y, node_height) {
+  const MARGIN = 4 // the margin around the html element
+
+  /* Create a transform that deals with all the scrolling and zooming */
+  const elRect = ctx.canvas.getBoundingClientRect()
+  const transform = new DOMMatrix()
+    .scaleSelf(
+      elRect.width / ctx.canvas.width,
+      elRect.height / ctx.canvas.height
+    )
+    .multiplySelf(ctx.getTransform())
+    .translateSelf(MARGIN, MARGIN + y)
+
+  return {
+    transformOrigin: '0 0',
+    transform: transform,
+    left: `0`,
+    top: `0`,
+    cursor: 'pointer',
+    position: 'absolute',
+    maxWidth: `${widget_width - MARGIN * 2}px`,
+    // maxHeight: `${node_height - MARGIN * 2}px`, // we're assuming we have the whole height of the node
+    width: `${widget_width - MARGIN * 2}px`,
+    // height: `${node_height * 0.3 - MARGIN * 2}px`,
+    // background: '#EEEEEE',
+    display: 'flex',
+    // flexDirection: 'column',
+    // alignItems: 'center',
+    justifyContent: 'space-around'
+  }
+}
+
+const getLocalData = key => {
+  let data = {}
+  try {
+    data = JSON.parse(localStorage.getItem(key)) || {}
+  } catch (error) {
+    return {}
+  }
+  return data
+}
+
+function createImage (url) {
+  let im = new Image()
+  return new Promise((res, rej) => {
+    im.onload = () => res(im)
+    im.src = url
+  })
+}
+
+const parseSvg = async svgContent => {
+  // 创建一个临时的DOM元素来解析SVG
+  const tempContainer = document.createElement('div')
+  tempContainer.innerHTML = svgContent
+
+  // 提取SVG元素
+  const svgElement = tempContainer.querySelector('svg')
+  if (!svgElement) return
+  // 获取SVG中 rect元素
+  var rectElements = svgElement?.querySelectorAll('rect') || []
+  // console.log(rectElements,svgElement)
+  // 定义一个数组来存储处理后的数据
+  var data = []
+
+  Array.from(rectElements, (rectElement, i) => {
+    // 获取rect元素的属性值
+    var x = ~~(rectElement.getAttribute('x') || 0)
+    var y = ~~(rectElement.getAttribute('y') || 0)
+    var width = ~~rectElement.getAttribute('width')
+    var height = ~~rectElement.getAttribute('height')
+    // console.log('rectElements',rectElement,x,y,width,height)
+    if (x != undefined && y != undefined && width && height) {
+      // 创建一个新的canvas元素
+      var canvas = document.createElement('canvas')
+      canvas.width = width
+      canvas.height = height
+      var context = canvas.getContext('2d')
+
+      // 填充颜色到canvas
+      var fill = rectElement.getAttribute('fill')
+      context.fillStyle = fill
+      context.fillRect(0, 0, width, height)
+
+      // 将canvas转换为base64格式
+      var base64 = canvas.toDataURL()
+
+      // 将数据转化为指定的JSON格式
+
+      var rectData = {
+        x: parseInt(x),
+        y: parseInt(y),
+        width: parseInt(width),
+        height: parseInt(height),
+        z_index: i + 1,
+        scale_option: 'width',
+        image: base64,
+        mask: base64,
+        type: 'base64',
+        _t: 'rect'
+      }
+
+      // 将处理后的数据添加到数组中
+      data.push(rectData)
+    }
+  })
+
+  var svgWidth = svgElement.getAttribute('width')
+  var svgHeight = svgElement.getAttribute('height')
+
+  if (!(svgWidth && svgHeight)) {
+    // viewBox
+    let viewBox = svgElement.viewBox.baseVal
+
+    svgWidth = viewBox.width
+    svgHeight = viewBox.height
+  }
+
+  // 创建一个新的canvas元素
+  var canvas = document.createElement('canvas')
+  canvas.width = svgWidth
+  canvas.height = svgHeight
+  var context = canvas.getContext('2d')
+  // 绘制SVG到canvas
+  var svgString = new XMLSerializer().serializeToString(svgElement)
+  var DOMURL = window.URL || window.webkitURL || window
+
+  var svgBlob = new Blob([svgString], { type: 'image/svg+xml;charset=utf-8' })
+  var url = DOMURL.createObjectURL(svgBlob)
+
+  let img = await createImage(url)
+  context.drawImage(img, 0, 0)
+
+  let base64 = canvas.toDataURL()
+
+  var rectData = {
+    x: 0,
+    y: 0,
+    width: parseInt(svgWidth),
+    height: parseInt(svgHeight),
+    z_index: 0,
+    scale_option: 'width',
+    image: base64,
+    mask: base64,
+    type: 'base64',
+    _t: 'canvas'
+  }
+  data.push(rectData)
+
+  // 打印处理后的数据
+  console.log('layers', { data, image: base64, svgElement })
+  return { data, image: base64, svgElement }
+}
+
+async function setArea (cw, ch, base64, data, fn) {
+  let displayHeight = Math.round(window.screen.availHeight * 0.6)
+  let div = document.createElement('div')
+  div.innerHTML = `
+    <div id='ml_overlay' style='position: absolute;top:0;background: #251f1fc4;
+    height: 100vh;
+    z-index:999999;
+    width: 100%;'>
+      <img id='ml_video' style='position: absolute; 
+      height: ${displayHeight}px;user-select: none; 
+      -webkit-user-drag: none;
+      outline: 2px solid #eaeaea;
+      box-shadow: 8px 9px 17px #575757;' />
+      <div id='ml_selection' style='position: absolute; 
+      border: 2px dashed red; 
+      pointer-events: none;'></div>
+    </div>`
+  // document.body.querySelector('#ml_overlay')
+  document.body.appendChild(div)
+
+  // let canvas = document.createElement('canvas')
+  // canvas.width = cw
+  // canvas.height = ch
+
+  let img = div.querySelector('#ml_video')
+  let overlay = div.querySelector('#ml_overlay')
+  let selection = div.querySelector('#ml_selection')
+  let startX, startY, endX, endY
+  let start = false
+  // Set video source
+  img.src = base64
+  // canvas.toDataURL();
+
+  // init area
+  // const data = getSetAreaData()
+  let x = 0,
+    y = 0,
+    width = (cw * displayHeight) / ch,
+    height = displayHeight
+
+  let imgWidth = cw
+  let imgHeight = ch
+
+  if (data && data.width > 0 && data.height > 0) {
+    // 相同尺寸窗口，恢复选区
+    x = (width * data.x) / imgWidth
+    y = (height * data.y) / imgHeight
+    width = (width * data.width) / imgWidth
+    height = (height * data.height) / imgHeight
+  }
+
+  selection.style.left = x + 'px'
+  selection.style.top = y + 'px'
+  selection.style.width = width + 'px'
+  selection.style.height = height + 'px'
+
+  // Add mouse events
+  img.addEventListener('mousedown', startSelection)
+  img.addEventListener('mousemove', updateSelection)
+  img.addEventListener('mouseup', endSelection)
+  overlay.addEventListener('click', remove)
+
+  function remove () {
+    overlay.removeEventListener('click', remove)
+    img.removeEventListener('mousedown', startSelection)
+    img.removeEventListener('mousemove', updateSelection)
+    img.removeEventListener('mouseup', endSelection)
+    div.remove()
+  }
+
+  function startSelection (event) {
+    if (start == false) {
+      startX = event.clientX
+      startY = event.clientY
+      updateSelection(event)
+      start = true
+    } else {
+    }
+  }
+
+  function updateSelection (event) {
+    endX = event.clientX
+    endY = event.clientY
+
+    // Calculate width, height, and coordinates
+    let width = Math.abs(endX - startX)
+    let height = Math.abs(endY - startY)
+    let left = Math.min(startX, endX)
+    let top = Math.min(startY, endY)
+
+    // Set selection style
+    selection.style.left = left + 'px'
+    selection.style.top = top + 'px'
+    selection.style.width = width + 'px'
+    selection.style.height = height + 'px'
+  }
+
+  function endSelection (event) {
+    endX = event.clientX
+    endY = event.clientY
+
+    // 获取img元素的真实宽度和高度
+    let imgWidth = img.naturalWidth
+    let imgHeight = img.naturalHeight
+
+    // 换算起始坐标
+    let realStartX = (startX / img.offsetWidth) * imgWidth
+    let realStartY = (startY / img.offsetHeight) * imgHeight
+
+    // 换算起始坐标
+    let realEndX = (endX / img.offsetWidth) * imgWidth
+    let realEndY = (endY / img.offsetHeight) * imgHeight
+
+    startX = realStartX
+    startY = realStartY
+    endX = realEndX
+    endY = realEndY
+    // Calculate width, height, and coordinates
+    let width = Math.round(Math.abs(endX - startX))
+    let height = Math.round(Math.abs(endY - startY))
+    let left = Math.round(Math.min(startX, endX))
+    let top = Math.round(Math.min(startY, endY))
+
+    if (width <= 0 && height <= 0) return remove()
+
+    if (fn) fn(left, top, width, height)
+
+    remove()
+  }
+}
+
+app.registerExtension({
+  name: 'Mixlab.layer.ShowLayer',
+  async getCustomWidgets (app) {
+    return {
+      EDIT (node, inputName, inputData, app) {
+        //   console.log('EditLayer##node', node,inputName, inputData)
+        const widget = {
+          type: inputData[0], // the type, CHEESE
+          name: inputName, // the name, slice
+          size: [128, 44], // a default size
+          draw (ctx, node, widget_width, y, widget_height) {
+            // console.log('EditLayer', this)
+            if (this.input)
+              Object.assign(
+                this.input.style,
+                get_position_style(ctx, widget_width, 32, node.size[1])
+              )
+          },
+          computeSize (...args) {
+            return [128, 44] // a method to compute the current size of the widget
+          },
+          async serializeValue (nodeId, widgetIndex) {
+            let d = getLocalData('_mixlab_edit_layer')
+            // console.log('EditLayer',d[node.id])
+            return d[node.id]
+          }
+        }
+        //  widget.something = something;          // maybe adds stuff to it
+        node.addCustomWidget(widget) // adds it to the node
+        return widget // and returns it.
+      }
+    }
+  },
+
+  async beforeRegisterNodeDef (nodeType, nodeData, app) {
+    if (nodeType.comfyClass == 'ShowLayer') {
+      const orig_nodeCreated = nodeType.prototype.onNodeCreated
+      nodeType.prototype.onNodeCreated = async function () {
+        orig_nodeCreated?.apply(this, arguments)
+
+        const findNode = nodeId => {
+          let node = app.graph._nodes_by_id[nodeId]
+          if (node?.type == 'Reroute') {
+            let linkId = node.inputs.filter(i => i.type == '*')[0].link
+            nodeId = app.graph.links.filter(link => link.id == linkId)[0]
+              ?.origin_id
+            return findNode(nodeId)
+          } else {
+            return nodeId
+          }
+        }
+
+        // 获取layers数据
+        const getLayers = async () => {
+          console.log(
+            'getLayers1',
+            this.inputs.filter(ip => ip.name === 'layers')
+          )
+          let linkId = this.inputs.filter(ip => ip.name === 'layers')[0].link
+          let nodeId = app.graph.links?.filter(link => link.id == linkId)[0]
+            ?.origin_id
+
+          if (nodeId) {
+            nodeId = findNode(nodeId)
+          }
+
+          //   let node = app.graph._nodes_by_id[nodeId]
+          //   if (node?.type == 'Reroute') {
+          //     linkId = node.inputs[0].link
+          //     nodeId = app.graph.links.filter(link => link.id == linkId)[0]
+          //       ?.origin_id
+          //   }
+
+          let d = getLocalData('_mixlab_svg_image')
+          console.log('test', d[nodeId])
+
+          if (d[nodeId]) {
+            let url = d[nodeId]
+            let dt = await fetch(url)
+
+            let svgStr = await dt.text()
+
+            const { data } = (await parseSvg(svgStr)) || {}
+            console.log('fetch', data)
+            return data
+          } else {
+            return []
+          }
+        }
+
+        // 修改layers数据
+        const setLayer = async (editIndex, layers = null) => {
+          //   let editIndex = 0
+          let lys = layers || (await getLayers())
+          let layer = lys[editIndex]
+          //   console.log(layer)
+
+          const updateValue = name => {
+            const x = this.widgets.filter(w => w.name == name)[0]
+            x.value = layer[name]
+          }
+          if (layer) {
+            Array.from(['x', 'y', 'width', 'height', 'z_index'], n =>
+              updateValue(n)
+            )
+          }
+        }
+
+        let that = this
+        const save_edit_layer_index = i => {
+          let data = getLocalData('_mixlab_edit_layer')
+          data[that.id] = i
+          localStorage.setItem('_mixlab_edit_layer', JSON.stringify(data))
+        }
+
+        await setLayer(0)
+        save_edit_layer_index(0)
+
+        const edit = this.widgets.filter(w => w.name == 'edit')[0]
+
+        edit.input = $el('div', {})
+        edit.input.style = `
+        display: flex;
+        flex-direction:row;
+        align-items: center;
+        margin-top: 0;`
+
+        const ip = $el('input', {})
+        ip.className = 'comfy-multiline-input'
+        ip.type = 'number'
+        ip.min = 0
+        ip.step = 1
+        ip.max = Math.max(0, (await getLayers()).length - 1)
+        // ip.className = `${'comfy-multiline-input'} `
+
+        ip.value = 0
+
+        ip.style = `
+        background-color: var(--comfy-input-bg);
+        color: var(--input-text);
+        outline: none;
+          border: none;
+          padding: 4px;
+          width: 60%;
+          cursor: pointer;
+          height: 24px;`
+        const label = document.createElement('label')
+        label.style = 'font-size: 10px;min-width:32px'
+        label.innerText = 'Layer Index'
+        edit.input.appendChild(label)
+        edit.input.appendChild(ip)
+
+        document.body.appendChild(edit.input)
+
+        ip.addEventListener('click', async event => {
+          console.log(await getLayers())
+          ip.max = Math.max(0, (await getLayers()).length - 1)
+        })
+
+        ip.addEventListener('change', async event => {
+          let index = ~~ip.value
+          let lys = await getLayers()
+          await setLayer(index, lys)
+          app.graph.setDirtyCanvas(true, true)
+          save_edit_layer_index(index)
+        })
+
+        // console.log('EditLayer nodeData', edit)
+
+        const onRemoved = this.onRemoved
+        this.onRemoved = () => {
+          edit.input.remove()
+          return onRemoved?.()
+        }
+
+        if (this.onResize) {
+          this.onResize(this.size)
+        }
+
+        this.serialize_widgets = false //需要保存参数
+      }
+    }
+  },
+  async loadedGraphNode (node, app) {
+    // Fires every time a node is constructed
+    // You can modify widgets/add handlers/etc here
+    if (node.type === 'SvgImage') {
+      let widget = node.widgets.filter(w => w.div)[0]
+      let data = getLocalData('_mixlab_svg_image')
+      let id = node.id
+
+      // widget.div.querySelector('.Svg').value = data[id] || '#000000'
+    }
+  }
+})
+
+app.registerExtension({
+  name: 'Mixlab.layer.NewLayer',
+  async beforeRegisterNodeDef (nodeType, nodeData, app) {
+    if (nodeData.name === 'NewLayer') {
+      const orig_nodeCreated = nodeType.prototype.onNodeCreated
+      nodeType.prototype.onNodeCreated = async function () {
+        orig_nodeCreated?.apply(this, arguments)
+
+        let b = this.widgets.filter(w => w.type === 'button')[0]
+        // const [w, h, base64] = canvas
+
+        if (!b) {
+          const updateValue = (x1, y1, w1, h1) => {
+            if (this.widgets) {
+              for (const widget of this.widgets) {
+                if (widget.name === 'x') {
+                  widget.value = x1
+                }
+                if (widget.name === 'y') {
+                  widget.value = y1
+                }
+                if (widget.name === 'width') {
+                  widget.value = w1
+                }
+                if (widget.name === 'height') {
+                  widget.value = h1
+                }
+              }
+            }
+          }
+
+          this.addWidget('button', 'Set Area', '', () => {
+            let data = {}
+            for (const widget of this.widgets) {
+              if (widget.name === 'x') {
+                data.x = widget.value
+              }
+              if (widget.name === 'y') {
+                data.y = widget.value
+              }
+              if (widget.name === 'width') {
+                data.width = widget.value
+              }
+              if (widget.name === 'height') {
+                data.height = widget.value
+              }
+            }
+            try {
+              let linkId = this.inputs[3].link
+              let nodeId = app.graph.links[linkId].origin_id
+              // console.log(linkId,this.inputs)
+              let im = app.graph.getNodeById(nodeId).imgs[0]
+              let src = im.src
+              setArea(im.naturalWidth, im.naturalHeight, src, data, updateValue)
+            } catch (error) {}
+          })
+        }
+      }
+
+      const onRemoved = this.onRemoved
+      this.onRemoved = () => {
+        // let b = this.widgets.filter(w => w.type === 'button')[0];
+
+        return onRemoved?.()
+      }
+
+      if (this.onResize) {
+        this.onResize(this.size)
+      }
+
+      this.serialize_widgets = true //需要保存参数
+    }
+  }
+})

web/javascript/main_mixlab.js

@@ -461,7 +461,7 @@ async function requestCamera () {
 /* 
 A method that returns the required style for the html 
 */
-function get_position_style (ctx, widget_width, y, node_height) {
+function get_position_style (ctx, widget_width, y, node_height, top) {
   const MARGIN = 4 // the margin around the html element
 
   /* Create a transform that deals with all the scrolling and zooming */
@@ -478,14 +478,14 @@ function get_position_style (ctx, widget_width, y, node_height) {
     transformOrigin: '0 0',
     transform: transform,
     left: `0`,
-    top: `0`,
+    top: `${top}px`,
     cursor: 'pointer',
     position: 'absolute',
     maxWidth: `${widget_width - MARGIN * 2}px`,
     // maxHeight: `${node_height - MARGIN * 2}px`, // we're assuming we have the whole height of the node
     width: `${widget_width - MARGIN * 2}px`,
     // height: `${node_height - MARGIN * 2}px`,
-    background: '#EEEEEE',
+    // background: '#EEEEEE',
     display: 'flex',
     flexDirection: 'column',
     // alignItems: 'center',
@@ -585,9 +585,6 @@ app.registerExtension({
   },
   async beforeRegisterNodeDef (nodeType, nodeData, app) {
     if (nodeType.comfyClass == 'ScreenShare') {
-      /* 
-          Hijack the onNodeCreated call to add our widget
-          */
       const orig_nodeCreated = nodeType.prototype.onNodeCreated
       nodeType.prototype.onNodeCreated = function () {
         orig_nodeCreated?.apply(this, arguments)
@@ -596,22 +593,29 @@ app.registerExtension({
           type: 'HTML', // whatever
           name: 'sreen_share', // whatever
           draw (ctx, node, widget_width, y, widget_height) {
-            console.log('ScreenSHare', y, widget_height)
+            // console.log('ScreenSHare', y, widget_height)
             Object.assign(
               this.card.style,
               get_position_style(
                 ctx,
                 widget_width,
                 widget_height * 5,
-                node.size[1]
+                node.size[1],
+                40
               )
             )
           }
         }
 
-        widget.card = $el('div', {})
+        widget.card = $el('div', {
+          color: 'var(--descrip-text)',
+          backgroundColor: 'var(--comfy-input-bg)'
+        })
 
-        widget.previewCard = $el('div', {})
+        widget.previewCard = $el('div', {
+          color: 'var(--descrip-text)',
+          backgroundColor: 'var(--comfy-input-bg)'
+        })
 
         widget.preview = $el('video', {
           style: {
@@ -623,7 +627,10 @@ app.registerExtension({
         })
 
         widget.previewArea = $el('div', {
-          style: {}
+          style: {
+            color: 'var(--descrip-text)',
+            backgroundColor: 'var(--comfy-input-bg)'
+          }
         })
 
         widget.shareDiv = $el('div', {
@@ -631,7 +638,9 @@ app.registerExtension({
           style: {
             cursor: 'pointer',
             fontWeight: '300',
-            display: 'flex'
+            display: 'flex',
+            color: 'var(--descrip-text)',
+            backgroundColor: 'var(--comfy-input-bg)'
           }
         })
 
@@ -642,7 +651,12 @@ app.registerExtension({
             padding: '8px 0',
             fontWeight: '300',
             margin: '2px',
-            width: '100%'
+            width: '100%',
+            color: 'var(--descrip-text)',
+            backgroundColor: 'var(--comfy-input-bg)',
+            borderRadius: '8px',
+            borderColor: 'var(--border-color)',
+            borderStyle: 'solid'
           }
         })
 
@@ -653,7 +667,12 @@ app.registerExtension({
             padding: '8px 0',
             fontWeight: '300',
             margin: '2px',
-            width: '100%'
+            width: '100%',
+            color: 'var(--descrip-text)',
+            backgroundColor: 'var(--comfy-input-bg)',
+            borderRadius: '8px',
+            borderColor: 'var(--border-color)',
+            borderStyle: 'solid'
           }
         })
 
@@ -663,29 +682,43 @@ app.registerExtension({
             cursor: 'pointer',
             padding: '8px 0',
             fontWeight: '300',
-            margin: '2px'
+            margin: '2px',
+            color: 'var(--descrip-text)',
+            backgroundColor: 'var(--comfy-input-bg)',
+            borderRadius: '8px',
+            borderColor: 'var(--border-color)',
+            borderStyle: 'solid'
           }
         })
 
-        widget.refreshInput = $el('input', {
-          placeholder: '  Refresh rate:200 ms',
-          type: 'number',
-          min: 100,
-          step: 100,
-          style: {
-            cursor: 'pointer',
-            padding: '8px 0',
-            fontWeight: '300',
-            margin: '2px'
-          }
-        })
+        // widget.refreshInput = $el('input', {
+        //   placeholder: '  Refresh rate:200 ms',
+        //   type: 'number',
+        //   min: 100,
+        //   step: 100,
+        //   style: {
+        //     cursor: 'pointer',
+        //     padding: '8px 24px',
+        //     fontWeight: '300',
+        //     margin: '2px',
+        //     color: 'var(--descrip-text)',
+        //     backgroundColor: 'var(--comfy-input-bg)'
+        //   }
+        // });
+        // widget.refreshInput.className='comfy-multiline-input'
+
         widget.liveBtn = $el('button', {
           innerText: 'Live Run',
           style: {
             cursor: 'pointer',
             padding: '8px 0',
             fontWeight: '300',
-            margin: '2px'
+            margin: '2px',
+            color: 'var(--descrip-text)',
+            backgroundColor: 'var(--comfy-input-bg)',
+            borderRadius: '8px',
+            borderColor: 'var(--border-color)',
+            borderStyle: 'solid'
           }
         })
 
@@ -699,7 +732,7 @@ app.registerExtension({
         widget.shareDiv.appendChild(widget.shareBtn)
         widget.shareDiv.appendChild(widget.shareOfWebCamBtn)
         widget.card.appendChild(widget.openFloatingWinBtn)
-        widget.card.appendChild(widget.refreshInput)
+        // widget.card.appendChild(widget.refreshInput)
         widget.card.appendChild(widget.liveBtn)
 
         const toggleShare = async (isCamera = false) => {
@@ -859,11 +892,11 @@ app.registerExtension({
           toggleShare()
         })
 
-        widget.refreshInput.addEventListener('change', async () => {
-          window._mixlab_screen_refresh_rate = Math.round(
-            widget.refreshInput.value
-          )
-        })
+        // widget.refreshInput.addEventListener('change', async () => {
+        //   window._mixlab_screen_refresh_rate = Math.round(
+        //     widget.refreshInput.value
+        //   )
+        // })
 
         widget.liveBtn.addEventListener('click', async () => {
           if (window._mixlab_stopLive) {
@@ -890,6 +923,9 @@ app.registerExtension({
         })
         // console.log('widget.inputEl',widget.inputEl)
 
+        this.setSize([this.size[0], this.size[1] + 450])
+        app.canvas.draw(true, true)
+
         /*
               Add the widget, make sure we clean up nicely, and we do not want to be serialized!
               */
@@ -901,12 +937,21 @@ app.registerExtension({
           widget.shareBtn.remove()
           widget.liveBtn.remove()
           widget.card.remove()
-          widget.refreshInput.remove()
+          // widget.refreshInput.remove()
           widget.previewArea.remove()
           widget.previewCard.remove()
         }
         this.serialize_widgets = true
       }
+
+      const onExecuted = nodeType.prototype.onExecuted
+      nodeType.prototype.onExecuted = function (message) {
+        onExecuted?.apply(this, arguments)
+        // console.log('###ScreenShare', this, message.refresh_rate)
+        window._mixlab_screen_refresh_rate = Math.round(
+          message.refresh_rate[0] || 500
+        )
+      }
     }
   }
 })
@@ -998,6 +1043,7 @@ async function setArea (src) {
   div.innerHTML = `
     <div id='ml_overlay' style='position: absolute;top:0;background: #251f1fc4;
     height: 100vh;
+    z-index:999999;
     width: 100%;'>
       <img id='ml_video' style='position: absolute; 
       height: ${displayHeight}px;user-select: none; 
@@ -1029,12 +1075,7 @@ async function setArea (src) {
     height = displayHeight
   let imgWidth = im.naturalWidth
   let imgHeight = im.naturalHeight
-  // console.log(
-  //   '#screen_share::使用上一次选区 selection',
-  //   data,
-  //   imgWidth,
-  //   img.width
-  // )
+
   if (
     data &&
     data.width > 0 &&
@@ -1048,9 +1089,6 @@ async function setArea (src) {
     y = (img.height * data.y) / data.imgHeight
     width = (img.width * data.width) / data.imgWidth
     height = (img.height * data.height) / data.imgHeight
-    // imgWidth = data.imgWidth
-    // imgHeight = data.imgHeight;
-    // console.log('#screen_share::使用上一次选区 selection', x, y, width, height)
   }
 
   selection.style.left = x + 'px'
@@ -1115,9 +1153,6 @@ async function setArea (src) {
     let realEndX = (endX / img.offsetWidth) * imgWidth
     let realEndY = (endY / img.offsetHeight) * imgHeight
 
-    // 输出结果到控制台
-    // console.log('真实宽度: ' + realWidth)
-    // console.log('真实高度: ' + realHeight)
     startX = realStartX
     startY = realStartY
     endX = realEndX
@@ -1127,19 +1162,6 @@ async function setArea (src) {
     let height = Math.abs(endY - startY)
     let left = Math.min(startX, endX)
     let top = Math.min(startY, endY)
-    // Output results to console
-    // console.log('坐标位置: (' + left + ', ' + top + ')')
-    // console.log('宽度: ' + width)
-    // console.log('高度: ' + height)
-
-    // img.removeEventListener('mousedown', startSelection)
-    // img.removeEventListener('mousemove', updateSelection)
-    // img.removeEventListener('mouseup', endSelection)
-
-    // window._mixlab_screen_x = left
-    // window._mixlab_screen_y = top
-    // window._mixlab_screen_width = width
-    // window._mixlab_screen_height = height
 
     if (width <= 0 && height <= 0) return remove()
 
@@ -1151,7 +1173,6 @@ async function setArea (src) {
       window._mixlab_screen_webcamVideo,
       !window._mixlab_screen_live
     )
-
     remove()
   }
 }
@@ -1200,7 +1221,7 @@ app.registerExtension({
           draw (ctx, node, widget_width, y, widget_height) {
             Object.assign(
               this.card.style,
-              get_position_style(ctx, widget_width, y, node.size[1])
+              get_position_style(ctx, widget_width, y, node.size[1], 0)
             )
           }
         }
@@ -1252,7 +1273,12 @@ app.registerExtension({
             cursor: 'pointer',
             padding: '8px 0',
             fontWeight: '300',
-            margin: '2px'
+            margin: '2px',
+            color: 'var(--descrip-text)',
+            backgroundColor: 'var(--comfy-input-bg)',
+            borderRadius: '8px',
+            borderColor: 'var(--border-color)',
+            borderStyle: 'solid'
           }
         })
 
@@ -1418,7 +1444,8 @@ app.registerExtension({
             )
 
             try {
-              pipWindow.document.querySelector('#info').innerText =window._mixlab_screen_seed_input
+              pipWindow.document.querySelector('#info').innerText =
+                window._mixlab_screen_seed_input
             } catch (error) {
               console.log(error)
             }

web/javascript/svg_icons.js

@@ -0,0 +1 @@
+export const closeIcon = '<svg xmlns="http://www.w3.org/2000/svg" height="24" viewBox="0 -960 960 960" width="24"><path d="m256-200-56-56 224-224-224-224 56-56 224 224 224-224 56 56-224 224 224 224-56 56-224-224-224 224Z"/></svg>'

web/javascript/ui_mixlab.js

@@ -1,4 +1,99 @@
 import { app } from '../../../scripts/app.js'
+import { api } from '../../../scripts/api.js'
+import { ComfyWidgets } from '../../../scripts/widgets.js'
+import { $el } from '../../../scripts/ui.js'
+import {closeIcon} from './svg_icons.js'
+
+function deepEqual (obj1, obj2) {
+  if (typeof obj1 !== typeof obj2) {
+    return false
+  }
+
+  if (typeof obj1 !== 'object' || obj1 === null || obj2 === null) {
+    return obj1 === obj2
+  }
+
+  const keys1 = Object.keys(obj1)
+  const keys2 = Object.keys(obj2)
+
+  if (keys1.length !== keys2.length) {
+    return false
+  }
+
+  for (let key of keys1) {
+    if (!deepEqual(obj1[key], obj2[key])) {
+      return false
+    }
+  }
+
+  return true
+}
+
+async function get_nodes_map () {
+  let api_host = `${window.location.hostname}:${window.location.port}`
+  let api_base = ''
+  let url = `${window.location.protocol}//${api_host}${api_base}`
+
+  const res = await fetch(`${url}/mixlab/nodes_map`, {
+    method: 'POST',
+    body: JSON.stringify({
+      data: 'json'
+    })
+  })
+  return await res.json()
+}
+
+function loadCSS(url) {
+  var link = document.createElement('link');
+  link.rel = 'stylesheet';
+  link.type = 'text/css';
+  link.href = url;
+  document.getElementsByTagName('head')[0].appendChild(link);
+}
+
+var cssURL = 'https://cdnjs.cloudflare.com/ajax/libs/github-markdown-css/5.5.0/github-markdown-light.min.css';
+loadCSS(cssURL);
+
+function injectCSS (css) {
+  // 检查页面中是否已经存在具有相同内容的style标签
+  const existingStyle = document.querySelector('style')
+  if (existingStyle && existingStyle.textContent === css) {
+    return // 如果已经存在相同的样式，则不进行注入
+  }
+
+  // 创建一个新的style标签，并将CSS内容注入其中
+  const style = document.createElement('style')
+  style.textContent = css
+
+  // 将style标签插入到页面的head元素中
+  const head = document.querySelector('head')
+  head.appendChild(style)
+}
+
+injectCSS(`::-webkit-scrollbar {
+  width: 2px;
+}
+
+@keyframes loading_mixlab {
+  0% {
+    background-color: green;
+  }
+
+  50% {
+    background-color: lightgreen;
+  }
+
+  100% {
+    background-color: green;
+  }
+}
+
+.loading_mixlab {
+  background-color: green;
+  animation-name: loading_mixlab;
+  animation-duration: 2s;
+  animation-iteration-count: infinite;
+}`)
 
 async function getCustomnodeMappings (mode = 'url') {
   // mode = "local";
@@ -6,18 +101,39 @@ async function getCustomnodeMappings (mode = 'url') {
   let api_base = ''
   let url = `${window.location.protocol}//${api_host}${api_base}`
 
-  const response = await fetch(`${url}/customnode/getmappings?mode=${mode}`)
-  const data = await response.json()
-
   let nodes = {}
-  try {
-    for (let url in data) {
-      let n = data[url]
-      for (let node of n[0]) {
-        nodes[node] = { url, title: n[1].title_aux }
-      }
+
+  const data = (await get_nodes_map()).data
+
+  for (let url in data) {
+    let n = data[url]
+    for (let node of n[0]) {
+      // if(node=='CLIPSeg')console.log('#CLIPSeg',n)
+      nodes[node] = { url, title: n[1].title_aux }
     }
-  } catch (error) {}
+  }
+  
+  // try {
+  //   const response = await fetch(`${url}/customnode/getmappings?mode=${mode}`)
+  //   const data = await response.json()
+  //   for (let url in data) {
+  //     let n = data[url]
+  //     for (let node of n[0]) {
+  //       // if(node=='CLIPSeg')console.log('#CLIPSeg',n)
+  //       nodes[node] = { url, title: n[1].title_aux }
+  //     }
+  //   }
+  // } catch (error) {
+  //   const data = (await get_nodes_map()).data
+
+  //   for (let url in data) {
+  //     let n = data[url]
+  //     for (let node of n[0]) {
+  //       // if(node=='CLIPSeg')console.log('#CLIPSeg',n)
+  //       nodes[node] = { url, title: n[1].title_aux }
+  //     }
+  //   }
+  // }
 
   return nodes
 }
@@ -55,13 +171,51 @@ const missingNodeGithub = (missingNodeTypes, nodesMap) => {
   })
 }
 
+let nodesMap
+
+function get_position_style (ctx, widget_width, y, node_height) {
+  const MARGIN = 4 // the margin around the html element
+
+  /* Create a transform that deals with all the scrolling and zooming */
+  const elRect = ctx.canvas.getBoundingClientRect()
+  const transform = new DOMMatrix()
+    .scaleSelf(
+      elRect.width / ctx.canvas.width,
+      elRect.height / ctx.canvas.height
+    )
+    .multiplySelf(ctx.getTransform())
+    .translateSelf(MARGIN, MARGIN + y)
+
+  return {
+    transformOrigin: '0 0',
+    transform: transform,
+    left: `0`,
+    top: `0`,
+    cursor: 'pointer',
+    position: 'absolute',
+    maxWidth: `${widget_width - MARGIN * 2}px`,
+    // maxHeight: `${node_height - MARGIN * 2}px`, // we're assuming we have the whole height of the node
+    width: `${widget_width - MARGIN * 2}px`,
+    // height: `${node_height * 0.3 - MARGIN * 2}px`,
+    // background: '#EEEEEE',
+    display: 'flex',
+    flexDirection: 'column',
+    // alignItems: 'center',
+    justifyContent: 'space-around'
+  }
+}
+
 app.showMissingNodesError = async function (
   missingNodeTypes,
   hasAddedNodes = true
 ) {
-  const nodesMap = await getCustomnodeMappings()
-  console.log('#nodesMap', nodesMap)
-  //   console.log('###MIXLAB', missingNodeTypes, hasAddedNodes)
+  nodesMap =
+    nodesMap && Object.keys(nodesMap).length > 0
+      ? nodesMap
+      : await getCustomnodeMappings('url')
+
+  // console.log('#nodesMap', nodesMap)
+  // console.log('###MIXLAB', missingNodeTypes, hasAddedNodes)
   this.ui.dialog.show(
     `When loading the graph, the following node types were not found: <ul>${missingNodeGithub(
       missingNodeTypes,
@@ -77,12 +231,507 @@ app.showMissingNodesError = async function (
   })
 }
 
-// app.ui.dialog.show = function (html) {
-//   console.log('###MIXLAB', html)
-//   if (typeof html === 'string') {
-//     this.textElement.innerHTML = html
-//   } else {
-//     this.textElement.replaceChildren(html)
-//   }
-//   this.element.style.display = 'flex'
-// }
+// app.registerExtension({
+//   name: 'Comfy.MDNote',
+//   registerCustomNodes () {
+//     class NoteNode {
+//       // color = LGraphCanvas.node_colors.yellow.color
+//       // bgcolor = LGraphCanvas.node_colors.yellow.bgcolor
+//       // groupcolor = LGraphCanvas.node_colors.yellow.groupcolor
+//       constructor () {
+//         if (!this.properties) {
+//           this.properties = {}
+//           this.properties.text = ''
+//         }
+//         console.log('NoteNode1', this)
+
+//         const widget = {
+//           type: 'div',
+//           name: 'input_color',
+//           draw (ctx, node, widget_width, y, widget_height) {
+//             Object.assign(
+//               this.div.style,
+//               get_position_style(
+//                 ctx,
+//                 widget_width,
+//                 44,
+//                 node.size[1]
+//               )
+//             )
+//           }
+//         }
+
+//         widget.div = $el('div', {});
+//         widget.div.innerText='1111'
+
+//         document.body.appendChild(widget.div)
+
+//         this.addCustomWidget(widget)
+
+//         this.serialize_widgets = true
+//         this.isVirtualNode = true
+//       }
+//     }
+
+//     // Load default visibility
+
+//     LiteGraph.registerNodeType(
+//       'MDNote',
+//       Object.assign(NoteNode, {
+//         title_mode: LiteGraph.NORMAL_TITLE,
+//         title: 'MDNote',
+//         collapsable: true
+//       })
+//     )
+
+//     NoteNode.category = '♾️Mixlab/utils'
+//   },
+
+// })
+
+async function fetchReadmeContent (url) {
+  try {
+    // var repo = 'owner/repo'; // 仓库的拥有者和名称
+    var match = url.match(/github.com\/([^/]+\/[^/]+)/)
+    var repo = match[1]
+    var url = `https://api.github.com/repos/${repo}/readme`
+    var response = await fetch(url)
+    var data = await response.json()
+    var readmeUrl = data.download_url
+
+    var readmeResponse = await fetch(readmeUrl)
+    var content = await readmeResponse.text()
+    // console.log(content) // 在控制台输出readme.md文件的内容
+
+    return content
+  } catch (error) {
+    console.log('获取readme.md文件信息失败:', error)
+  }
+}
+
+function createModal (url, markdown,title) {
+  
+  // Create modal element
+  var div =
+    document.querySelector('#mix-modal') || document.createElement('div')
+  div.id = 'mix-modal'
+  div.innerHTML = ''
+  div.style.cssText = `width: 100%;
+    z-index: 9990;
+    height: 100vh;
+    display: flex;
+    color: var(--descrip-text);
+    position: fixed;
+    top: 0;
+    left: 0;
+    `
+
+  var modal = document.createElement('div')
+
+  div.appendChild(modal)
+  modal.classList.add("modal-body")
+  // Set modal styles
+  modal.style.cssText = `
+    background: white;
+    height: 80vh;
+    position: fixed;
+    overflow:hidden;
+    top: 50%;
+    left: 50%;
+    transform: translate(-50%, -50%);
+    z-index: 9999;
+    border-radius: 4px;
+    box-shadow: 4px 4px 14px rgba(255,255,255,0.5);
+
+  `
+  // Create modal content area
+  var modalContent = document.createElement('div')
+  modalContent.classList.add('modal-content')
+  // Create modal header
+  const headerElement =  document.createElement('div')
+  headerElement.classList.add('modal-header')
+  headerElement.style.cssText = `
+    display: flex;
+    padding: 20px 24px 8px 24px;
+    justify-content: space-between;
+  `
+  
+  const headTitleElement = document.createElement('a')
+  headTitleElement.classList.add('header-title')
+  headTitleElement.style.cssText=`
+    color: var(--descrip-text);
+    font-size: 18px;
+    display: flex;
+    align-items: center;
+    flex: 1;
+    overflow: hidden;
+    text-decoration: none;
+    font-weight: bold;
+  `
+  headTitleElement.onmouseenter = function(){
+    headTitleElement.style.color = 'var(--comfy-menu-bg)'
+  }
+  headTitleElement.onmouseleave = function(){
+    headTitleElement.style.color = 'var(--descrip-text)'
+  }
+  headTitleElement.textContent= title ||'';
+  headTitleElement.href = url
+  headTitleElement.target='_blank'
+  const linkIcon = document.createElement('small')
+  linkIcon.textContent = '🔗'
+  headTitleElement.appendChild(linkIcon);
+  headerElement.appendChild(headTitleElement)
+
+  
+
+  // Create close button
+  const closeButton = document.createElement('span')
+  closeButton.classList.add('close')
+  closeButton.innerHTML = closeIcon
+  // Set close button styles
+  closeButton.style.cssText = `
+      padding: 4px;
+      cursor: pointer;
+      width: 32px;
+      height: 32px;
+      display: flex;
+      justify-content: center;
+      align-items: center;
+      user-select: none;
+      fill: var(--descrip-text);
+      `
+  closeButton.onmouseenter = function(){
+    closeButton.style.fill = 'var(--comfy-menu-bg)'; 
+  }
+  closeButton.onmouseleave = function(){
+    closeButton.style.fill = 'var(--descrip-text)';
+  }
+
+  headerElement.appendChild(closeButton)
+
+
+  // Click event to close the modal
+  function closeMixModal(){
+    div.style.display = 'none'
+    window.removeEventListener('keydown',MixModalEscKeyEvent)
+  }
+  closeButton.onclick = function () {
+    closeMixModal()
+  }
+
+  // Set modal content area styles
+  modalContent.style.cssText = `
+    position: relative;
+    padding: 0px;
+    overflow: hidden scroll;;
+    height: 100%;
+    min-width:300px
+    `
+
+  // Append close button to modal content area
+  modal.appendChild(headerElement)
+
+  // Create element for displaying Markdown content
+  var markdownContent = document.createElement('div')
+  markdownContent.classList.add('markdown-content','markdown-body')
+  markdownContent.style.cssText = `max-width: 50vw;padding: 0px 24px 100px 24px;`
+
+  showdown.setFlavor('github');
+  var converter = new showdown.Converter()
+
+  var html = converter.makeHtml(markdown)
+
+  // Hide images in the markdown when they fail to load
+  var regex = /<img[^>]+src="?([^"\s]+)"?[^>]*>/g
+  html = html.replace(regex, function (match, src) {
+    return `<img src="${src}" onerror="this.style.display='none'">`
+  })
+
+  // Open links in a new tab or window
+  html = html.replace(/<a/g, '<a target="_blank"')
+
+  // Fix href attribute to absolute path
+  html = html.replace(
+    /<a([^>]+href=["'])(?!https?:\/\/)([^"'>]+)/g,
+    function (match, prefix, path) {
+      var absolutePath = url + '/' + path
+      return '<a' + prefix + absolutePath
+    }
+  )
+
+  markdownContent.innerHTML = html
+
+  // Append Markdown content element to modal content area
+  modalContent.appendChild(markdownContent)
+
+  // Append modal content area to modal element
+  modal.appendChild(modalContent)
+
+  const footerElement = document.createElement('div')
+  footerElement.style.cssText=`
+      position: absolute;
+      left: 0;
+      right: 0;
+      text-align: right;
+      padding:10px;
+      font-size:12px
+  `
+
+  const footerText = document.createElement('a')
+  footerText.href ="https://github.com/shadowcz007/comfyui-mixlab-nodes"
+  footerText.innerText =  "Support by Mixlab"
+  footerText.style.cssText=`color:inherit`
+  footerText.target = "_blank"
+  footerText.onmouseenter=function(){
+    footerText.style.color = 'var(--input-text)'
+  }
+  footerText.onmouseleave=function(){
+    footerText.style.color = 'inherit'
+  }
+  
+  
+  footerText.onclick = function(e){
+    e.stopPropagation()
+  }
+  footerElement.appendChild(footerText)
+
+  div.appendChild(footerElement)
+
+
+  // Append modal element to the page
+  if (!document.querySelector('#mix-modal')) {
+    document.body.appendChild(div)
+  }
+  function MixModalEscKeyEvent(event){
+      if(event.key == "Escape"){
+        closeMixModal()
+      }
+  }
+  window.removeEventListener('keydown',MixModalEscKeyEvent)
+  window.addEventListener('keydown',MixModalEscKeyEvent)
+
+  const bgElement = document.createElement('div')
+  bgElement.classList.add('mix-modal-bg')
+  bgElement.style.cssText = `
+    width:100%;
+    height:100%;
+    background-color: rgba(0,0,0,0.8);
+  `
+  bgElement.onclick =  function( ){   
+    closeMixModal()
+  }
+  
+  div.appendChild(bgElement)
+
+
+}
+
+app.registerExtension({
+  name: 'Comfy.Mixlab.ui',
+  init () {
+    LGraphCanvas.prototype.helpAboutNode = async function (node) {
+      nodesMap =
+        nodesMap && Object.keys(nodesMap).length > 0
+          ? nodesMap
+          : await getCustomnodeMappings('url')
+
+      console.log('node & node map', node, nodesMap, nodesMap[node.type])
+      let repo = nodesMap[node.type]
+      if (repo) {
+        let markdown = await fetchReadmeContent(repo.url)
+        createModal(repo.url,markdown,repo.title)
+      }
+    }
+
+    const getNodeMenuOptions = LGraphCanvas.prototype.getNodeMenuOptions // store the existing method
+    LGraphCanvas.prototype.getNodeMenuOptions = function (node) {
+      // replace it
+      const options = getNodeMenuOptions.apply(this, arguments) // start by calling the stored one
+      node.setDirtyCanvas(true, true) // force a redraw of (foreground, background)
+      // options.splice(
+      //   options.length - 1,
+      //   0, // splice a new option in at the end
+      //   {
+      //     content: '♾️Mixlab', // with a name
+      //     callback: () => {
+      //       LGraphCanvas.prototype.helpAboutNode(node)
+      //     } // and the callback
+      //   },
+      //   null // a divider
+      // )
+      return [
+        {
+          content: 'Help ♾️Mixlab', // with a name
+          callback: () => {
+            LGraphCanvas.prototype.helpAboutNode(node)
+          } // and the callback
+        },
+        null,
+        ...options
+      ] // and return the options
+    }
+  },
+  async setup () {
+
+    // Add canvas menu options 
+			const orig = LGraphCanvas.prototype.getCanvasMenuOptions;
+			LGraphCanvas.prototype.getCanvasMenuOptions = function () {
+				const options = orig.apply(this, arguments);
+
+				options.push(null,  {
+          content: `Find ♾️Mixlab`,
+          disabled: false, // or a function determining whether to disable
+          callback: async () => {
+            nodesMap =
+              nodesMap && Object.keys(nodesMap).length > 0
+                ? nodesMap
+                : await getCustomnodeMappings('url')
+  
+            const nodesDiv = document.createDocumentFragment()
+            const nodes = (await app.graphToPrompt()).output
+  
+            // console.log('[Mixlab]', 'loaded graph node: ', app)
+            let div =
+              document.querySelector('#mixlab_find_the_node') ||
+              document.createElement('div')
+            div.id = 'mixlab_find_the_node'
+            div.style = `
+              flex-direction: column;
+              align-items: end;
+              display:flex;position: absolute; 
+              top: 50px; left: 50px; width: 200px; 
+              color: var(--descrip-text);
+              background-color: var(--comfy-menu-bg);
+              padding: 10px; 
+              border: 1px solid black;z-index: 999999999;padding-top: 0;`
+  
+            div.innerHTML = ''
+  
+            let btn = document.createElement('div')
+            btn.style=`display: flex;
+            width: calc(100% - 24px);
+            justify-content: space-between;
+            align-items: center;
+            padding: 0 12px;
+            height: 32px;`
+            let btnB = document.createElement('button')
+            let textB = document.createElement('p')
+            btn.appendChild(textB)
+            btn.appendChild(btnB)
+            textB.innerText = `Find The Node`
+  
+            btnB.style = `float: right; border: none; color: var(--input-text);
+            background-color: var(--comfy-input-bg); border-color: var(--border-color);cursor: pointer;`
+            btnB.addEventListener('click', () => {
+              div.style.display = 'none'
+            })
+            btnB.innerText = 'X'
+  
+            // 悬浮框拖动事件
+            div.addEventListener('mousedown', function (e) {
+              var startX = e.clientX
+              var startY = e.clientY
+              var offsetX = div.offsetLeft
+              var offsetY = div.offsetTop
+  
+              function moveBox (e) {
+                var newX = e.clientX
+                var newY = e.clientY
+                var deltaX = newX - startX
+                var deltaY = newY - startY
+                div.style.left = offsetX + deltaX + 'px'
+                div.style.top = offsetY + deltaY + 'px'
+              }
+  
+              function stopMoving () {
+                document.removeEventListener('mousemove', moveBox)
+                document.removeEventListener('mouseup', stopMoving)
+              }
+  
+              document.addEventListener('mousemove', moveBox)
+              document.addEventListener('mouseup', stopMoving)
+            })
+  
+            div.appendChild(btn)
+  
+            const updateNodes = (ns, nd) => {
+              for (let nodeId in ns) {
+                let n = ns[nodeId].class_type
+                const { url, title } = nodesMap[n]
+                let d = document.createElement('button')
+                d.style = `text-align: left;margin:6px;color: var(--input-text);
+                background-color: var(--comfy-input-bg); border-color: var(--border-color);cursor: pointer;`
+                d.addEventListener('click', () => {
+                  const node = app.graph.getNodeById(nodeId)
+                  if (!node) return
+                  app.canvas.centerOnNode(node)
+                  app.canvas.setZoom(1)
+                })
+                d.addEventListener('mouseover', async () => {
+                  // console.log('mouseover')
+                  let n = (await app.graphToPrompt()).output
+                  if (!deepEqual(n, ns)) {
+                    nd.innerHTML = ''
+                    updateNodes(n, nd)
+                  }
+                })
+  
+                d.innerHTML = `
+                <span>${'#' + nodeId} ${n}</span>
+                <a href="${url}" target="_blank" style="text-decoration: none;">🔗</a>
+                `
+                d.title = title
+  
+                nd.appendChild(d)
+              }
+            }
+  
+            let nodesDivv = document.createElement('div')
+  
+            for (let nodeId in nodes) {
+              let n = nodes[nodeId].class_type
+              const { url, title } = nodesMap[n]
+              let d = document.createElement('button')
+              d.style = `text-align: left;margin:6px;color: var(--input-text);
+              background-color: var(--comfy-input-bg); border-color: var(--border-color);cursor: pointer;`
+              d.addEventListener('click', () => {
+                const node = app.graph.getNodeById(nodeId)
+                if (!node) return
+                app.canvas.centerOnNode(node)
+                app.canvas.setZoom(1)
+              })
+              d.addEventListener('mouseover', async () => {
+                console.log('mouseover')
+                let n = (await app.graphToPrompt()).output
+                if (!deepEqual(n, nodes)) {
+                  nodesDivv.innerHTML = ''
+                  updateNodes(n, nodesDivv)
+                }
+              })
+  
+              d.innerHTML = `
+              <span>${'#' + nodeId} ${n}</span>
+              <a href="${url}" target="_blank" style="text-decoration: none;">🔗</a>
+              `
+              d.title = title
+  
+              nodesDiv.appendChild(d)
+            }
+  
+            nodesDivv.appendChild(nodesDiv)
+            nodesDivv.style=`overflow: scroll;
+            height: 70vh;width: 100%;`
+  
+            div.appendChild(nodesDivv)
+  
+            if (!document.querySelector('#mixlab_find_the_node'))
+              document.body.appendChild(div)
+          }
+        });
+				return options;
+			};
+
+
+    
+  }
+})

web/javascript/utils_mixlab.js

@@ -0,0 +1,181 @@
+import { app } from '../../../scripts/app.js'
+import { api } from '../../../scripts/api.js'
+import { ComfyWidgets } from '../../../scripts/widgets.js'
+import { $el } from '../../../scripts/ui.js'
+import { addValueControlWidget } from "../../../scripts/widgets.js";
+
+const getLocalData = key => {
+  let data = {}
+  try {
+    data = JSON.parse(localStorage.getItem(key)) || {}
+  } catch (error) {
+    return {}
+  }
+  return data
+}
+function get_position_style (ctx, widget_width, y, node_height) {
+  const MARGIN = 4 // the margin around the html element
+
+  /* Create a transform that deals with all the scrolling and zooming */
+  const elRect = ctx.canvas.getBoundingClientRect()
+  const transform = new DOMMatrix()
+    .scaleSelf(
+      elRect.width / ctx.canvas.width,
+      elRect.height / ctx.canvas.height
+    )
+    .multiplySelf(ctx.getTransform())
+    .translateSelf(MARGIN, MARGIN + y)
+
+  return {
+    transformOrigin: '0 0',
+    transform: transform,
+    left: `0`,
+    top: `0`,
+    cursor: 'pointer',
+    position: 'absolute',
+    maxWidth: `${widget_width - MARGIN * 2}px`,
+    // maxHeight: `${node_height - MARGIN * 2}px`, // we're assuming we have the whole height of the node
+    width: `${widget_width - MARGIN * 2}px`,
+    // height: `${node_height * 0.3 - MARGIN * 2}px`,
+    // background: '#EEEEEE',
+    display: 'flex',
+    flexDirection: 'column',
+    // alignItems: 'center',
+    justifyContent: 'space-around'
+  }
+}
+
+app.registerExtension({
+  name: 'Mixlab.utils.Color',
+  async getCustomWidgets (app) {
+    return {
+      TCOLOR (node, inputName, inputData, app) {
+        // console.log('##node', node)
+        const widget = {
+          type: inputData[0], // the type, CHEESE
+          name: inputName, // the name, slice
+          size: [128, 32], // a default size
+          draw (ctx, node, width, y) {},
+          computeSize (...args) {
+            return [128, 32] // a method to compute the current size of the widget
+          },
+          async serializeValue (nodeId, widgetIndex) {
+            let data = getLocalData('_mixlab_utils_color')
+            return data[node.id] || '#000000'
+          }
+        }
+        //  widget.something = something;          // maybe adds stuff to it
+        node.addCustomWidget(widget) // adds it to the node
+        return widget // and returns it.
+      }
+    }
+  },
+
+  async beforeRegisterNodeDef (nodeType, nodeData, app) {
+    if (nodeType.comfyClass == 'Color') {
+      const orig_nodeCreated = nodeType.prototype.onNodeCreated
+      nodeType.prototype.onNodeCreated = function () {
+        orig_nodeCreated?.apply(this, arguments)
+
+        console.log('Color nodeData', this.widgets)
+
+        const widget = {
+          type: 'div',
+          name: 'input_color',
+          draw (ctx, node, widget_width, y, widget_height) {
+            Object.assign(
+              this.div.style,
+              get_position_style(ctx, widget_width, 44, node.size[1])
+            )
+          }
+        }
+
+        widget.div = $el('div', {})
+
+        document.body.appendChild(widget.div)
+
+        const inputDiv = (key, placeholder, value) => {
+          let div = document.createElement('div')
+          const ip = document.createElement('input')
+          ip.type = 'color'
+          ip.className = `${'comfy-multiline-input'} ${placeholder}`
+          div.style = `display: flex;
+            align-items: center; 
+            margin: 6px 8px;
+            margin-top: 0;`
+          ip.placeholder = placeholder
+          ip.value = value
+
+          ip.style = `outline: none;
+            border: none;
+            padding: 4px;
+            width: 100%;cursor: pointer;
+            height: 32px;`
+          const label = document.createElement('label')
+          label.style = 'font-size: 10px;min-width:32px'
+          label.innerText = placeholder
+          div.appendChild(label)
+          div.appendChild(ip)
+
+          ip.addEventListener('change', () => {
+            let data = getLocalData(key)
+            data[this.id] = ip.value.trim()
+            localStorage.setItem(key, JSON.stringify(data))
+            // console.log(this.id, ip.value.trim())
+          })
+          return div
+        }
+
+        let inputColor = inputDiv('_mixlab_utils_color', 'Color', '#000000')
+
+        widget.div.appendChild(inputColor)
+
+        this.addCustomWidget(widget)
+
+        const onRemoved = this.onRemoved
+        this.onRemoved = () => {
+          inputColor.remove()
+          widget.div.remove()
+          return onRemoved?.()
+        }
+
+        this.serialize_widgets = true //需要保存参数
+      }
+    }
+  },
+  async loadedGraphNode (node, app) {
+    // Fires every time a node is constructed
+    // You can modify widgets/add handlers/etc here
+
+    if (node.type === 'Color') {
+      let widget = node.widgets.filter(w => w.div)[0]
+
+      let data = getLocalData('_mixlab_utils_color')
+
+      let id = node.id
+
+      widget.div.querySelector('.Color').value = data[id] || '#000000'
+    }
+  }
+})
+
+app.registerExtension({
+  name: 'Mixlab.utils.TextToNumber',
+
+  async beforeRegisterNodeDef (nodeType, nodeData, app) {
+    if (nodeType.comfyClass == 'TextToNumber') {
+
+      const onExecuted = nodeType.prototype.onExecuted
+      nodeType.prototype.onExecuted = function (message) {
+        onExecuted?.apply(this, arguments)
+        const random_number=this.widgets.filter(w=>w.name==='random_number')[0]
+        if(random_number.value==='enable'){
+          const n=this.widgets.filter(w=>w.name==='number')[0]
+          n.value=message.num[0]
+        }
+        
+        console.log('TextToNumber', random_number.value)
+      }
+    }
+  }
+})

web/main.css

@@ -1,3 +1,24 @@
 ::-webkit-scrollbar {
-    width: 2px;
-  }
+  width: 2px;
+}
+
+@keyframes loading_mixlab {
+  0% {
+    background-color: green;
+  }
+
+  50% {
+    background-color: lightgreen;
+  }
+
+  100% {
+    background-color: green;
+  }
+}
+
+.loading_mixlab {
+  background-color: green;
+  animation-name: loading_mixlab;
+  animation-duration: 2s;
+  animation-iteration-count: infinite;
+}

workflow/3-FloatVideo-workflow.json

@@ -1,6 +1,6 @@
 {
-  "last_node_id": 22,
-  "last_link_id": 48,
+  "last_node_id": 23,
+  "last_link_id": 51,
   "nodes": [
     {
       "id": 7,
@@ -105,7 +105,7 @@
         {
           "name": "text",
           "type": "STRING",
-          "link": 48,
+          "link": 51,
           "widget": {
             "name": "text"
           }
@@ -295,7 +295,7 @@
         "Node name for S&R": "KSampler"
       },
       "widgets_values": [
-        1115769645491668,
+        482859286431021,
         "randomize",
         4,
         1.6,
@@ -316,13 +316,13 @@
         "1": 246
       },
       "flags": {},
-      "order": 6,
+      "order": 7,
       "mode": 0,
       "inputs": [
         {
           "name": "images",
           "type": "IMAGE",
-          "link": 46
+          "link": 50
         }
       ],
       "properties": {
@@ -445,13 +445,13 @@
         "1": 58
       },
       "flags": {},
-      "order": 7,
+      "order": 6,
       "mode": 0,
       "inputs": [
         {
           "name": "image",
           "type": "IMAGE",
-          "link": 47
+          "link": 49
         }
       ],
       "outputs": [
@@ -472,45 +472,6 @@
         512
       ]
     },
-    {
-      "id": 22,
-      "type": "ScreenShare",
-      "pos": [
-        -63,
-        483
-      ],
-      "size": {
-        "0": 315,
-        "1": 98
-      },
-      "flags": {},
-      "order": 3,
-      "mode": 0,
-      "outputs": [
-        {
-          "name": "IMAGE",
-          "type": "IMAGE",
-          "links": [
-            46,
-            47
-          ],
-          "shape": 3,
-          "slot_index": 0
-        },
-        {
-          "name": "STRING",
-          "type": "STRING",
-          "links": [
-            48
-          ],
-          "shape": 3,
-          "slot_index": 1
-        }
-      ],
-      "properties": {
-        "Node name for S&R": "ScreenShare"
-      }
-    },
     {
       "id": 20,
       "type": "FloatingVideo",
@@ -534,7 +495,69 @@
       ],
       "properties": {
         "Node name for S&R": "FloatingVideo"
-      }
+      },
+      "widgets_values": [
+        null
+      ]
+    },
+    {
+      "id": 23,
+      "type": "ScreenShare",
+      "pos": [
+        -65,
+        446
+      ],
+      "size": [
+        312.78457519531213,
+        606.2132135620109
+      ],
+      "flags": {},
+      "order": 3,
+      "mode": 0,
+      "outputs": [
+        {
+          "name": "IMAGE",
+          "type": "IMAGE",
+          "links": [
+            49,
+            50
+          ],
+          "shape": 3,
+          "slot_index": 0
+        },
+        {
+          "name": "PROMPT",
+          "type": "STRING",
+          "links": [
+            51
+          ],
+          "shape": 3,
+          "slot_index": 1
+        },
+        {
+          "name": "FLOAT",
+          "type": "FLOAT",
+          "links": null,
+          "shape": 3
+        },
+        {
+          "name": "INT",
+          "type": "INT",
+          "links": null,
+          "shape": 3
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "ScreenShare"
+      },
+      "widgets_values": [
+        null,
+        2018,
+        null,
+        null,
+        null,
+        null
+      ]
     }
   ],
   "links": [
@@ -787,24 +810,24 @@
       "CLIP"
     ],
     [
-      46,
-      22,
-      0,
-      2,
-      0,
-      "IMAGE"
-    ],
-    [
-      47,
-      22,
+      49,
+      23,
       0,
       18,
       0,
       "IMAGE"
     ],
     [
-      48,
-      22,
+      50,
+      23,
+      0,
+      2,
+      0,
+      "IMAGE"
+    ],
+    [
+      51,
+      23,
       1,
       8,
       1,

workflow/7-camera-workflow.json

@@ -1,13 +1,13 @@
 {
-  "last_node_id": 23,
-  "last_link_id": 54,
+  "last_node_id": 24,
+  "last_link_id": 59,
   "nodes": [
     {
       "id": 7,
       "type": "CLIPTextEncode",
       "pos": [
-        495,
-        307
+        515,
+        130
       ],
       "size": {
         "0": 400,
@@ -45,8 +45,8 @@
       "id": 8,
       "type": "CLIPTextEncode",
       "pos": [
-        504,
-        33
+        515,
+        1268
       ],
       "size": {
         "0": 400,
@@ -64,7 +64,7 @@
         {
           "name": "text",
           "type": "STRING",
-          "link": 48,
+          "link": 59,
           "widget": {
             "name": "text"
           }
@@ -92,8 +92,8 @@
       "id": 6,
       "type": "LoraLoader",
       "pos": [
-        112,
-        5
+        515,
+        460
       ],
       "size": {
         "0": 315,
@@ -140,58 +140,17 @@
         1
       ]
     },
-    {
-      "id": 23,
-      "type": "VAEEncode",
-      "pos": [
-        563,
-        663
-      ],
-      "size": {
-        "0": 210,
-        "1": 46
-      },
-      "flags": {},
-      "order": 5,
-      "mode": 0,
-      "inputs": [
-        {
-          "name": "pixels",
-          "type": "IMAGE",
-          "link": 52
-        },
-        {
-          "name": "vae",
-          "type": "VAE",
-          "link": 53
-        }
-      ],
-      "outputs": [
-        {
-          "name": "LATENT",
-          "type": "LATENT",
-          "links": [
-            54
-          ],
-          "shape": 3,
-          "slot_index": 0
-        }
-      ],
-      "properties": {
-        "Node name for S&R": "VAEEncode"
-      }
-    },
     {
       "id": 5,
       "type": "KSampler",
       "pos": [
-        1085,
-        49
-      ],
-      "size": [
-        315,
-        262
+        1015,
+        130
       ],
+      "size": {
+        "0": 315,
+        "1": 262
+      },
       "flags": {},
       "order": 7,
       "mode": 0,
@@ -219,7 +178,7 @@
         {
           "name": "denoise",
           "type": "FLOAT",
-          "link": 49,
+          "link": 56,
           "widget": {
             "name": "denoise"
           }
@@ -253,8 +212,8 @@
       "id": 11,
       "type": "VAEDecode",
       "pos": [
-        1451,
-        59
+        1430,
+        130
       ],
       "size": {
         "0": 210,
@@ -294,13 +253,13 @@
       "id": 20,
       "type": "FloatingVideo",
       "pos": [
-        1464,
-        243
-      ],
-      "size": [
-        315,
-        58
+        1740,
+        130
       ],
+      "size": {
+        "0": 315,
+        "1": 58
+      },
       "flags": {},
       "order": 9,
       "mode": 0,
@@ -318,12 +277,78 @@
         null
       ]
     },
+    {
+      "id": 2,
+      "type": "PreviewImage",
+      "pos": [
+        515,
+        716
+      ],
+      "size": {
+        "0": 210,
+        "1": 246
+      },
+      "flags": {},
+      "order": 4,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "images",
+          "type": "IMAGE",
+          "link": 55
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "PreviewImage"
+      }
+    },
+    {
+      "id": 23,
+      "type": "VAEEncode",
+      "pos": [
+        515,
+        1092
+      ],
+      "size": {
+        "0": 210,
+        "1": 46
+      },
+      "flags": {},
+      "order": 5,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "pixels",
+          "type": "IMAGE",
+          "link": 57
+        },
+        {
+          "name": "vae",
+          "type": "VAE",
+          "link": 58
+        }
+      ],
+      "outputs": [
+        {
+          "name": "LATENT",
+          "type": "LATENT",
+          "links": [
+            54
+          ],
+          "shape": 3,
+          "slot_index": 0
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "VAEEncode"
+      }
+    },
     {
       "id": 10,
       "type": "CheckpointLoaderSimple",
       "pos": [
-        80,
-        238
+        100,
+        130
       ],
       "size": {
         "0": 315,
@@ -358,7 +383,7 @@
           "type": "VAE",
           "links": [
             34,
-            53
+            58
           ],
           "shape": 3,
           "slot_index": 2
@@ -372,15 +397,15 @@
       ]
     },
     {
-      "id": 22,
+      "id": 24,
       "type": "ScreenShare",
       "pos": [
-        103,
-        412
+        100,
+        358
       ],
       "size": {
         "0": 315,
-        "1": 146
+        "1": 170
       },
       "flags": {},
       "order": 1,
@@ -390,17 +415,17 @@
           "name": "IMAGE",
           "type": "IMAGE",
           "links": [
-            46,
-            52
+            55,
+            57
           ],
           "shape": 3,
           "slot_index": 0
         },
         {
-          "name": "STRING",
+          "name": "PROMPT",
           "type": "STRING",
           "links": [
-            48
+            59
           ],
           "shape": 3,
           "slot_index": 1
@@ -409,10 +434,16 @@
           "name": "FLOAT",
           "type": "FLOAT",
           "links": [
-            49
+            56
           ],
           "shape": 3,
           "slot_index": 2
+        },
+        {
+          "name": "INT",
+          "type": "INT",
+          "links": null,
+          "shape": 3
         }
       ],
       "properties": {
@@ -422,33 +453,9 @@
         null,
         null,
         null,
+        null,
         null
       ]
-    },
-    {
-      "id": 2,
-      "type": "PreviewImage",
-      "pos": [
-        -501,
-        918
-      ],
-      "size": {
-        "0": 210,
-        "1": 246
-      },
-      "flags": {},
-      "order": 4,
-      "mode": 0,
-      "inputs": [
-        {
-          "name": "images",
-          "type": "IMAGE",
-          "link": 46
-        }
-      ],
-      "properties": {
-        "Node name for S&R": "PreviewImage"
-      }
     }
   ],
   "links": [
@@ -660,30 +667,6 @@
       1,
       "CLIP"
     ],
-    [
-      46,
-      22,
-      0,
-      2,
-      0,
-      "IMAGE"
-    ],
-    [
-      48,
-      22,
-      1,
-      8,
-      1,
-      "STRING"
-    ],
-    [
-      49,
-      22,
-      2,
-      5,
-      4,
-      "FLOAT"
-    ],
     [
       51,
       8,
@@ -693,15 +676,39 @@
       "CONDITIONING"
     ],
     [
-      52,
-      22,
+      54,
+      23,
+      0,
+      5,
+      3,
+      "LATENT"
+    ],
+    [
+      55,
+      24,
+      0,
+      2,
+      0,
+      "IMAGE"
+    ],
+    [
+      56,
+      24,
+      2,
+      5,
+      4,
+      "FLOAT"
+    ],
+    [
+      57,
+      24,
       0,
       23,
       0,
       "IMAGE"
     ],
     [
-      53,
+      58,
       10,
       2,
       23,
@@ -709,12 +716,12 @@
       "VAE"
     ],
     [
-      54,
-      23,
-      0,
-      5,
-      3,
-      "LATENT"
+      59,
+      24,
+      1,
+      8,
+      1,
+      "STRING"
     ]
   ],
   "groups": [],

workflow/nodes_map.json

@@ -0,0 +1,828 @@
+{
+  "last_node_id": 23,
+  "last_link_id": 0,
+  "nodes": [
+    {
+      "id": 7,
+      "type": "ResizeImageMixlab",
+      "pos": [
+        1096.229866976564,
+        731.8207909156248
+      ],
+      "size": {
+        "0": 315,
+        "1": 106
+      },
+      "flags": {},
+      "order": 0,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "image",
+          "type": "IMAGE",
+          "link": null
+        }
+      ],
+      "outputs": [
+        {
+          "name": "IMAGE",
+          "type": "IMAGE",
+          "links": null,
+          "shape": 3
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "ResizeImageMixlab"
+      },
+      "widgets_values": [
+        512,
+        512,
+        "width"
+      ]
+    },
+    {
+      "id": 8,
+      "type": "LoadImagesFromPath",
+      "pos": [
+        1101.229866976564,
+        900.8207909156248
+      ],
+      "size": {
+        "0": 315,
+        "1": 238
+      },
+      "flags": {},
+      "order": 1,
+      "mode": 0,
+      "outputs": [
+        {
+          "name": "IMAGE",
+          "type": "IMAGE",
+          "links": null,
+          "shape": 6
+        },
+        {
+          "name": "MASK",
+          "type": "MASK",
+          "links": null,
+          "shape": 6
+        },
+        {
+          "name": "STRING",
+          "type": "STRING",
+          "links": null,
+          "shape": 3
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "LoadImagesFromPath"
+      },
+      "widgets_values": [
+        "C:\\Users\\38957\\Documents\\GitHub\\extract-anything\\outputs",
+        "disable",
+        "enable",
+        0,
+        "disable",
+        null,
+        null
+      ]
+    },
+    {
+      "id": 3,
+      "type": "ShowTextForGPT",
+      "pos": [
+        1786,
+        252
+      ],
+      "size": {
+        "0": 315,
+        "1": 58
+      },
+      "flags": {},
+      "order": 2,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "text",
+          "type": "STRING",
+          "link": null,
+          "widget": {
+            "name": "text"
+          }
+        }
+      ],
+      "outputs": [
+        {
+          "name": "STRING",
+          "type": "STRING",
+          "links": null,
+          "shape": 6
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "ShowTextForGPT"
+      },
+      "widgets_values": [
+        ""
+      ]
+    },
+    {
+      "id": 9,
+      "type": "TextImage",
+      "pos": [
+        1448.4608623625004,
+        735.0514200906249
+      ],
+      "size": {
+        "0": 315,
+        "1": 198
+      },
+      "flags": {},
+      "order": 3,
+      "mode": 0,
+      "outputs": [
+        {
+          "name": "IMAGE",
+          "type": "IMAGE",
+          "links": null,
+          "shape": 3
+        },
+        {
+          "name": "MASK",
+          "type": "MASK",
+          "links": null,
+          "shape": 3
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "TextImage"
+      },
+      "widgets_values": [
+        "龍馬精神迎新歲",
+        "C:\\Users\\38957\\Documents\\ai-lab\\ComfyUI_windows_portable\\ComfyUI\\custom_nodes\\comfyui-mixlab-nodes\\assets\\王汉宗颜楷体繁.ttf",
+        100,
+        12,
+        "#000000",
+        true
+      ]
+    },
+    {
+      "id": 2,
+      "type": "ChatGPTOpenAI",
+      "pos": [
+        1086,
+        235
+      ],
+      "size": {
+        "0": 400,
+        "1": 358
+      },
+      "flags": {},
+      "order": 4,
+      "mode": 0,
+      "outputs": [
+        {
+          "name": "text",
+          "type": "STRING",
+          "links": null,
+          "shape": 3
+        },
+        {
+          "name": "messages",
+          "type": "STRING",
+          "links": null,
+          "shape": 3
+        },
+        {
+          "name": "session_history",
+          "type": "STRING",
+          "links": null,
+          "shape": 3
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "ChatGPTOpenAI"
+      },
+      "widgets_values": [
+        null,
+        null,
+        "",
+        "You are ChatGPT, a large language model trained by OpenAI. Answer as concisely as possible.",
+        "gpt-3.5-turbo",
+        0,
+        "randomize",
+        1,
+        null
+      ]
+    },
+    {
+      "id": 16,
+      "type": "SpeechRecognition",
+      "pos": [
+        2892.502226325001,
+        724.176680771093
+      ],
+      "size": {
+        "0": 322.593505859375,
+        "1": 135.22850036621094
+      },
+      "flags": {},
+      "order": 5,
+      "mode": 0,
+      "outputs": [
+        {
+          "name": "prompt",
+          "type": "STRING",
+          "links": null,
+          "shape": 3
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "SpeechRecognition"
+      },
+      "widgets_values": [
+        null,
+        null
+      ]
+    },
+    {
+      "id": 17,
+      "type": "SpeechSynthesis",
+      "pos": [
+        2901.502226325001,
+        924.1766807710948
+      ],
+      "size": {
+        "0": 315,
+        "1": 58
+      },
+      "flags": {},
+      "order": 6,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "text",
+          "type": "STRING",
+          "link": null,
+          "widget": {
+            "name": "text"
+          }
+        }
+      ],
+      "outputs": [
+        {
+          "name": "STRING",
+          "type": "STRING",
+          "links": null,
+          "shape": 6
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "SpeechSynthesis"
+      },
+      "widgets_values": [
+        ""
+      ]
+    },
+    {
+      "id": 18,
+      "type": "ShowLayer",
+      "pos": [
+        2157.913220668749,
+        232.7887044208986
+      ],
+      "size": {
+        "0": 315,
+        "1": 226
+      },
+      "flags": {},
+      "order": 7,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "layers",
+          "type": "LAYER",
+          "link": null
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "ShowLayer"
+      }
+    },
+    {
+      "id": 19,
+      "type": "NewLayer",
+      "pos": [
+        2494.913220668749,
+        232.7887044208986
+      ],
+      "size": {
+        "0": 315,
+        "1": 218
+      },
+      "flags": {},
+      "order": 8,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "image",
+          "type": "IMAGE",
+          "link": null
+        },
+        {
+          "name": "mask",
+          "type": "MASK",
+          "link": null
+        },
+        {
+          "name": "layers",
+          "type": "LAYER",
+          "link": null
+        }
+      ],
+      "outputs": [
+        {
+          "name": "layers",
+          "type": "LAYER",
+          "links": null,
+          "shape": 6
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "NewLayer"
+      },
+      "widgets_values": [
+        0,
+        0,
+        512,
+        512,
+        0,
+        "width"
+      ]
+    },
+    {
+      "id": 20,
+      "type": "MergeLayers",
+      "pos": [
+        2163.913220668749,
+        515.7887044208991
+      ],
+      "size": {
+        "0": 315.47509765625,
+        "1": 69.53152465820312
+      },
+      "flags": {},
+      "order": 9,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "layers",
+          "type": "LAYER",
+          "link": null
+        },
+        {
+          "name": "image",
+          "type": "IMAGE",
+          "link": null
+        }
+      ],
+      "outputs": [
+        {
+          "name": "IMAGE",
+          "type": "IMAGE",
+          "links": null,
+          "shape": 3
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "MergeLayers"
+      }
+    },
+    {
+      "id": 4,
+      "type": "CharacterInText",
+      "pos": [
+        1778,
+        386
+      ],
+      "size": {
+        "0": 319.45068359375,
+        "1": 194.45606994628906
+      },
+      "flags": {},
+      "order": 10,
+      "mode": 0,
+      "outputs": [
+        {
+          "name": "INT",
+          "type": "INT",
+          "links": null,
+          "shape": 3
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "CharacterInText"
+      },
+      "widgets_values": [
+        "",
+        "",
+        1
+      ]
+    },
+    {
+      "id": 21,
+      "type": "Note",
+      "pos": [
+        2507.333181606249,
+        517.1286262958992
+      ],
+      "size": {
+        "0": 210,
+        "1": 58
+      },
+      "flags": {},
+      "order": 11,
+      "mode": 0,
+      "properties": {
+        "text": ""
+      },
+      "widgets_values": [
+        "TODO：可视化操作"
+      ],
+      "color": "#323",
+      "bgcolor": "#535"
+    },
+    {
+      "id": 22,
+      "type": "Note",
+      "pos": [
+        1533,
+        347
+      ],
+      "size": {
+        "0": 210,
+        "1": 58
+      },
+      "flags": {},
+      "order": 12,
+      "mode": 0,
+      "properties": {
+        "text": ""
+      },
+      "widgets_values": [
+        "funtion call"
+      ],
+      "color": "#323",
+      "bgcolor": "#535"
+    },
+    {
+      "id": 11,
+      "type": "3DImage",
+      "pos": [
+        1794.6918193937495,
+        724.2820719460937
+      ],
+      "size": {
+        "0": 309.2974548339844,
+        "1": 433.12554931640625
+      },
+      "flags": {},
+      "order": 13,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "material",
+          "type": "IMAGE",
+          "link": null
+        }
+      ],
+      "outputs": [
+        {
+          "name": "IMAGE",
+          "type": "IMAGE",
+          "links": null,
+          "shape": 3
+        },
+        {
+          "name": "MASK",
+          "type": "MASK",
+          "links": null,
+          "shape": 3
+        },
+        {
+          "name": "BG_IMAGE",
+          "type": "IMAGE",
+          "links": null,
+          "shape": 3
+        },
+        {
+          "name": "MATERIAL",
+          "type": "IMAGE",
+          "links": null,
+          "shape": 3
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "3DImage"
+      }
+    },
+    {
+      "id": 13,
+      "type": "ScreenShare",
+      "pos": [
+        2138.691819393749,
+        728.2820719460937
+      ],
+      "size": {
+        "0": 322.9458923339844,
+        "1": 594.0604858398438
+      },
+      "flags": {},
+      "order": 14,
+      "mode": 0,
+      "outputs": [
+        {
+          "name": "IMAGE",
+          "type": "IMAGE",
+          "links": null,
+          "shape": 3
+        },
+        {
+          "name": "PROMPT",
+          "type": "STRING",
+          "links": null,
+          "shape": 3
+        },
+        {
+          "name": "FLOAT",
+          "type": "FLOAT",
+          "links": null,
+          "shape": 3
+        },
+        {
+          "name": "INT",
+          "type": "INT",
+          "links": null,
+          "shape": 3
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "ScreenShare"
+      },
+      "widgets_values": [
+        null,
+        null,
+        null,
+        null,
+        null
+      ]
+    },
+    {
+      "id": 14,
+      "type": "FloatingVideo",
+      "pos": [
+        2486.691819393749,
+        726.2820719460937
+      ],
+      "size": {
+        "0": 302.3232421875,
+        "1": 303.61279296875
+      },
+      "flags": {},
+      "order": 15,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "images",
+          "type": "IMAGE",
+          "link": null
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "FloatingVideo"
+      },
+      "widgets_values": [
+        null
+      ]
+    },
+    {
+      "id": 6,
+      "type": "TransparentImage",
+      "pos": [
+        2496.691819393749,
+        1157.2820719460938
+      ],
+      "size": {
+        "0": 315,
+        "1": 146
+      },
+      "flags": {},
+      "order": 16,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "images",
+          "type": "IMAGE",
+          "link": null
+        },
+        {
+          "name": "masks",
+          "type": "MASK",
+          "link": null
+        }
+      ],
+      "outputs": [
+        {
+          "name": "file_path",
+          "type": "STRING",
+          "links": null,
+          "shape": 6
+        },
+        {
+          "name": "IMAGE",
+          "type": "IMAGE",
+          "links": null,
+          "shape": 6
+        },
+        {
+          "name": "RGBA",
+          "type": "RGBA",
+          "links": null,
+          "shape": 6
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "TransparentImage"
+      },
+      "widgets_values": [
+        "yes",
+        "yes",
+        "Mixlab_save"
+      ]
+    },
+    {
+      "id": 12,
+      "type": "ImageCropByAlpha",
+      "pos": [
+        1806.6918193937495,
+        1240.2820719460938
+      ],
+      "size": {
+        "0": 289.27215576171875,
+        "1": 46
+      },
+      "flags": {},
+      "order": 17,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "image",
+          "type": "IMAGE",
+          "link": null
+        },
+        {
+          "name": "RGBA",
+          "type": "RGBA",
+          "link": null
+        }
+      ],
+      "outputs": [
+        {
+          "name": "IMAGE",
+          "type": "IMAGE",
+          "links": null,
+          "shape": 3
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "ImageCropByAlpha"
+      }
+    },
+    {
+      "id": 5,
+      "type": "EnhanceImage",
+      "pos": [
+        1098.6918193937504,
+        1217.2820719460938
+      ],
+      "size": {
+        "0": 315,
+        "1": 58
+      },
+      "flags": {},
+      "order": 18,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "image",
+          "type": "IMAGE",
+          "link": null
+        }
+      ],
+      "outputs": [
+        {
+          "name": "IMAGE",
+          "type": "IMAGE",
+          "links": null,
+          "shape": 3
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "EnhanceImage"
+      },
+      "widgets_values": [
+        0.5
+      ]
+    },
+    {
+      "id": 10,
+      "type": "SvgImage",
+      "pos": [
+        1454.6918193937504,
+        1074.2820719460938
+      ],
+      "size": {
+        "0": 315,
+        "1": 170
+      },
+      "flags": {},
+      "order": 19,
+      "mode": 0,
+      "outputs": [
+        {
+          "name": "IMAGE",
+          "type": "IMAGE",
+          "links": null,
+          "shape": 3
+        },
+        {
+          "name": "layers",
+          "type": "LAYER",
+          "links": null,
+          "shape": 6
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "SvgImage"
+      },
+      "widgets_values": [
+        null,
+        null
+      ]
+    }
+  ],
+  "links": [],
+  "groups": [
+    {
+      "title": "GPT",
+      "bounding": [
+        1083,
+        157,
+        1039,
+        450
+      ],
+      "color": "#b06634",
+      "font_size": 24,
+      "locked": false
+    },
+    {
+      "title": "Image",
+      "bounding": [
+        1086,
+        650,
+        1735,
+        682
+      ],
+      "color": "#A88",
+      "font_size": 24,
+      "locked": false
+    },
+    {
+      "title": "Audio",
+      "bounding": [
+        2882,
+        650,
+        344,
+        342
+      ],
+      "color": "#8A8",
+      "font_size": 24,
+      "locked": false
+    },
+    {
+      "title": "Layer",
+      "bounding": [
+        2148,
+        158,
+        672,
+        437
+      ],
+      "color": "#a1309b",
+      "font_size": 24,
+      "locked": false
+    }
+  ],
+  "config": {},
+  "extra": {},
+  "version": 0.4
+}

workflow/语音+实时换脸workflow.json

@@ -0,0 +1,862 @@
+{
+  "last_node_id": 28,
+  "last_link_id": 64,
+  "nodes": [
+    {
+      "id": 7,
+      "type": "CLIPTextEncode",
+      "pos": [
+        515,
+        130
+      ],
+      "size": {
+        "0": 400,
+        "1": 200
+      },
+      "flags": {},
+      "order": 3,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "clip",
+          "type": "CLIP",
+          "link": 33
+        }
+      ],
+      "outputs": [
+        {
+          "name": "CONDITIONING",
+          "type": "CONDITIONING",
+          "links": [
+            29
+          ],
+          "shape": 3,
+          "slot_index": 0
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "CLIPTextEncode"
+      },
+      "widgets_values": [
+        ""
+      ]
+    },
+    {
+      "id": 6,
+      "type": "LoraLoader",
+      "pos": [
+        515,
+        460
+      ],
+      "size": {
+        "0": 315,
+        "1": 126
+      },
+      "flags": {},
+      "order": 4,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "model",
+          "type": "MODEL",
+          "link": 31
+        },
+        {
+          "name": "clip",
+          "type": "CLIP",
+          "link": 45
+        }
+      ],
+      "outputs": [
+        {
+          "name": "MODEL",
+          "type": "MODEL",
+          "links": [
+            44
+          ],
+          "shape": 3,
+          "slot_index": 0
+        },
+        {
+          "name": "CLIP",
+          "type": "CLIP",
+          "links": null,
+          "shape": 3
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "LoraLoader"
+      },
+      "widgets_values": [
+        "lcm-lora-sdv1-5.safetensors",
+        1,
+        1
+      ]
+    },
+    {
+      "id": 11,
+      "type": "VAEDecode",
+      "pos": [
+        1430,
+        130
+      ],
+      "size": {
+        "0": 210,
+        "1": 46
+      },
+      "flags": {},
+      "order": 11,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "samples",
+          "type": "LATENT",
+          "link": 35
+        },
+        {
+          "name": "vae",
+          "type": "VAE",
+          "link": 34
+        }
+      ],
+      "outputs": [
+        {
+          "name": "IMAGE",
+          "type": "IMAGE",
+          "links": [
+            41
+          ],
+          "shape": 3,
+          "slot_index": 0
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "VAEDecode"
+      }
+    },
+    {
+      "id": 23,
+      "type": "VAEEncode",
+      "pos": [
+        515,
+        1092
+      ],
+      "size": {
+        "0": 210,
+        "1": 46
+      },
+      "flags": {},
+      "order": 6,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "pixels",
+          "type": "IMAGE",
+          "link": 57
+        },
+        {
+          "name": "vae",
+          "type": "VAE",
+          "link": 58
+        }
+      ],
+      "outputs": [
+        {
+          "name": "LATENT",
+          "type": "LATENT",
+          "links": [
+            54
+          ],
+          "shape": 3,
+          "slot_index": 0
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "VAEEncode"
+      }
+    },
+    {
+      "id": 10,
+      "type": "CheckpointLoaderSimple",
+      "pos": [
+        100,
+        130
+      ],
+      "size": {
+        "0": 315,
+        "1": 98
+      },
+      "flags": {},
+      "order": 0,
+      "mode": 0,
+      "outputs": [
+        {
+          "name": "MODEL",
+          "type": "MODEL",
+          "links": [
+            31
+          ],
+          "shape": 3,
+          "slot_index": 0
+        },
+        {
+          "name": "CLIP",
+          "type": "CLIP",
+          "links": [
+            32,
+            33,
+            45
+          ],
+          "shape": 3,
+          "slot_index": 1
+        },
+        {
+          "name": "VAE",
+          "type": "VAE",
+          "links": [
+            34,
+            58
+          ],
+          "shape": 3,
+          "slot_index": 2
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "CheckpointLoaderSimple"
+      },
+      "widgets_values": [
+        "deliberate_v2.safetensors"
+      ]
+    },
+    {
+      "id": 8,
+      "type": "CLIPTextEncode",
+      "pos": [
+        515,
+        1268
+      ],
+      "size": {
+        "0": 400,
+        "1": 200
+      },
+      "flags": {},
+      "order": 8,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "clip",
+          "type": "CLIP",
+          "link": 32
+        },
+        {
+          "name": "text",
+          "type": "STRING",
+          "link": 62,
+          "widget": {
+            "name": "text"
+          }
+        }
+      ],
+      "outputs": [
+        {
+          "name": "CONDITIONING",
+          "type": "CONDITIONING",
+          "links": [
+            51
+          ],
+          "shape": 3,
+          "slot_index": 0
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "CLIPTextEncode"
+      },
+      "widgets_values": [
+        ""
+      ]
+    },
+    {
+      "id": 20,
+      "type": "FloatingVideo",
+      "pos": [
+        1274,
+        536
+      ],
+      "size": [
+        315,
+        58
+      ],
+      "flags": {},
+      "order": 12,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "images",
+          "type": "IMAGE",
+          "link": 41
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "FloatingVideo"
+      },
+      "widgets_values": [
+        null
+      ]
+    },
+    {
+      "id": 5,
+      "type": "KSampler",
+      "pos": [
+        1004,
+        130
+      ],
+      "size": [
+        315,
+        262
+      ],
+      "flags": {},
+      "order": 10,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "model",
+          "type": "MODEL",
+          "link": 44
+        },
+        {
+          "name": "positive",
+          "type": "CONDITIONING",
+          "link": 51
+        },
+        {
+          "name": "negative",
+          "type": "CONDITIONING",
+          "link": 29
+        },
+        {
+          "name": "latent_image",
+          "type": "LATENT",
+          "link": 54
+        }
+      ],
+      "outputs": [
+        {
+          "name": "LATENT",
+          "type": "LATENT",
+          "links": [
+            35
+          ],
+          "shape": 3,
+          "slot_index": 0
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "KSampler"
+      },
+      "widgets_values": [
+        758428944049342,
+        "fixed",
+        4,
+        1.6,
+        "lcm",
+        "karras",
+        0.61
+      ]
+    },
+    {
+      "id": 24,
+      "type": "ScreenShare",
+      "pos": [
+        62,
+        543
+      ],
+      "size": {
+        "0": 315,
+        "1": 194
+      },
+      "flags": {},
+      "order": 1,
+      "mode": 0,
+      "outputs": [
+        {
+          "name": "IMAGE",
+          "type": "IMAGE",
+          "links": [
+            55,
+            57
+          ],
+          "shape": 3,
+          "slot_index": 0
+        },
+        {
+          "name": "PROMPT",
+          "type": "STRING",
+          "links": [],
+          "shape": 3,
+          "slot_index": 1
+        },
+        {
+          "name": "FLOAT",
+          "type": "FLOAT",
+          "links": [],
+          "shape": 3,
+          "slot_index": 2
+        },
+        {
+          "name": "INT",
+          "type": "INT",
+          "links": null,
+          "shape": 3
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "ScreenShare"
+      },
+      "widgets_values": [
+        null,
+        null,
+        null,
+        null,
+        null
+      ]
+    },
+    {
+      "id": 2,
+      "type": "PreviewImage",
+      "pos": [
+        -416,
+        516
+      ],
+      "size": {
+        "0": 210,
+        "1": 246
+      },
+      "flags": {},
+      "order": 5,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "images",
+          "type": "IMAGE",
+          "link": 55
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "PreviewImage"
+      }
+    },
+    {
+      "id": 26,
+      "type": "ChatGPTOpenAI",
+      "pos": [
+        5,
+        1325
+      ],
+      "size": [
+        400,
+        357.99993896484375
+      ],
+      "flags": {},
+      "order": 7,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "prompt",
+          "type": "STRING",
+          "link": 61,
+          "widget": {
+            "name": "prompt"
+          }
+        }
+      ],
+      "outputs": [
+        {
+          "name": "text",
+          "type": "STRING",
+          "links": [
+            62,
+            63
+          ],
+          "shape": 3,
+          "slot_index": 0
+        },
+        {
+          "name": "messages",
+          "type": "STRING",
+          "links": null,
+          "shape": 3
+        },
+        {
+          "name": "session_history",
+          "type": "STRING",
+          "links": null,
+          "shape": 3
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "ChatGPTOpenAI"
+      },
+      "widgets_values": [
+        null,
+        null,
+        "",
+        "你需要根据用户的输入，转化为英文单词的短句子组合，输出英文",
+        "gpt-35-turbo",
+        1249,
+        "randomize",
+        1,
+        null
+      ]
+    },
+    {
+      "id": 28,
+      "type": "SpeechSynthesis",
+      "pos": [
+        542,
+        1570
+      ],
+      "size": [
+        315,
+        76
+      ],
+      "flags": {},
+      "order": 9,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "text",
+          "type": "STRING",
+          "link": 63,
+          "widget": {
+            "name": "text"
+          }
+        }
+      ],
+      "outputs": [
+        {
+          "name": "STRING",
+          "type": "STRING",
+          "links": null,
+          "shape": 6
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "SpeechSynthesis"
+      },
+      "widgets_values": [
+        "A very handsome elderly gentleman."
+      ]
+    },
+    {
+      "id": 25,
+      "type": "SpeechRecognition",
+      "pos": [
+        -5,
+        1101
+      ],
+      "size": [
+        392.5606180664065,
+        166.64923671875022
+      ],
+      "flags": {},
+      "order": 2,
+      "mode": 0,
+      "outputs": [
+        {
+          "name": "prompt",
+          "type": "STRING",
+          "links": [
+            61
+          ],
+          "shape": 3,
+          "slot_index": 0
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "SpeechRecognition"
+      },
+      "widgets_values": [
+        null,
+        null
+      ]
+    }
+  ],
+  "links": [
+    [
+      1,
+      1,
+      0,
+      2,
+      0,
+      "IMAGE"
+    ],
+    [
+      3,
+      6,
+      0,
+      5,
+      0,
+      "MODEL"
+    ],
+    [
+      5,
+      8,
+      0,
+      5,
+      2,
+      "CONDITIONING"
+    ],
+    [
+      6,
+      9,
+      0,
+      5,
+      3,
+      "LATENT"
+    ],
+    [
+      7,
+      10,
+      0,
+      6,
+      0,
+      "MODEL"
+    ],
+    [
+      8,
+      10,
+      1,
+      6,
+      1,
+      "CLIP"
+    ],
+    [
+      9,
+      5,
+      0,
+      11,
+      0,
+      "LATENT"
+    ],
+    [
+      10,
+      10,
+      2,
+      11,
+      1,
+      "VAE"
+    ],
+    [
+      11,
+      6,
+      1,
+      7,
+      0,
+      "CLIP"
+    ],
+    [
+      12,
+      6,
+      1,
+      8,
+      0,
+      "CLIP"
+    ],
+    [
+      14,
+      11,
+      0,
+      3,
+      0,
+      "IMAGE"
+    ],
+    [
+      17,
+      1,
+      2,
+      7,
+      1,
+      "STRING"
+    ],
+    [
+      18,
+      7,
+      0,
+      15,
+      0,
+      "CONDITIONING"
+    ],
+    [
+      19,
+      15,
+      0,
+      5,
+      1,
+      "CONDITIONING"
+    ],
+    [
+      20,
+      16,
+      0,
+      15,
+      1,
+      "CONTROL_NET"
+    ],
+    [
+      24,
+      1,
+      0,
+      18,
+      0,
+      "IMAGE"
+    ],
+    [
+      25,
+      18,
+      0,
+      15,
+      2,
+      "IMAGE"
+    ],
+    [
+      29,
+      7,
+      0,
+      5,
+      2,
+      "CONDITIONING"
+    ],
+    [
+      31,
+      10,
+      0,
+      6,
+      0,
+      "MODEL"
+    ],
+    [
+      32,
+      10,
+      1,
+      8,
+      0,
+      "CLIP"
+    ],
+    [
+      33,
+      10,
+      1,
+      7,
+      0,
+      "CLIP"
+    ],
+    [
+      34,
+      10,
+      2,
+      11,
+      1,
+      "VAE"
+    ],
+    [
+      35,
+      5,
+      0,
+      11,
+      0,
+      "LATENT"
+    ],
+    [
+      41,
+      11,
+      0,
+      20,
+      0,
+      "IMAGE"
+    ],
+    [
+      44,
+      6,
+      0,
+      5,
+      0,
+      "MODEL"
+    ],
+    [
+      45,
+      10,
+      1,
+      6,
+      1,
+      "CLIP"
+    ],
+    [
+      51,
+      8,
+      0,
+      5,
+      1,
+      "CONDITIONING"
+    ],
+    [
+      54,
+      23,
+      0,
+      5,
+      3,
+      "LATENT"
+    ],
+    [
+      55,
+      24,
+      0,
+      2,
+      0,
+      "IMAGE"
+    ],
+    [
+      57,
+      24,
+      0,
+      23,
+      0,
+      "IMAGE"
+    ],
+    [
+      58,
+      10,
+      2,
+      23,
+      1,
+      "VAE"
+    ],
+    [
+      61,
+      25,
+      0,
+      26,
+      0,
+      "STRING"
+    ],
+    [
+      62,
+      26,
+      0,
+      8,
+      1,
+      "STRING"
+    ],
+    [
+      63,
+      26,
+      0,
+      28,
+      0,
+      "STRING"
+    ]
+  ],
+  "groups": [],
+  "config": {},
+  "extra": {},
+  "version": 0.4
+}